2024-04-25T09:48:54.941 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:48:54.941 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_neesingh@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042 --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/orig.config.yaml 2024-04-25T09:48:54.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:48:55.082 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673042' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXci1nadOkhLG29+OCxwETXBf3XCyRn6NxbkFi3aGGHLHapLhQ2tdn3aqH3zsjo6xMMQ+QVCM7BJJ3NxxMlKVU= smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMi0xE96PiAJfnBQBLaXTbGCLxk5vEBvrEhfWUzWFZrTje+F6V6vaPoRu5+VG4n0csAE6hJQEQu8H1AOvtp1TRU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:48:55.082 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:48:55.083 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:48:55.083 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:48:55.085 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:48:55.113 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:48:55.113 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:48:55.113 INFO:teuthology.packaging:ref: None 2024-04-25T09:48:55.113 INFO:teuthology.packaging:tag: None 2024-04-25T09:48:55.113 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:48:55.113 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:48:55.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:48:55.256 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:48:55.258 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:48:55.264 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:48:55.264 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:48:55.304 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:48:55.320 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:48:55.337 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:48:55.366 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:43:52.775681', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXci1nadOkhLG29+OCxwETXBf3XCyRn6NxbkFi3aGGHLHapLhQ2tdn3aqH3zsjo6xMMQ+QVCM7BJJ3NxxMlKVU='} 2024-04-25T09:48:55.387 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:43:52.773719', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMi0xE96PiAJfnBQBLaXTbGCLxk5vEBvrEhfWUzWFZrTje+F6V6vaPoRu5+VG4n0csAE6hJQEQu8H1AOvtp1TRU='} 2024-04-25T09:48:55.387 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:48:55.394 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:48:55.394 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:48:55.394 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:48:55.487 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe4d353b160>, signals=[15]) 2024-04-25T09:48:55.488 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:48:55.494 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:48:55.495 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T09:48:55.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:48:55.614 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T09:48:55.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:48:55.689 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:48:55.697 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-04-25T09:48:55.717 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-04-25T09:48:55.718 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-04-25T09:48:55.777 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:48:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:48:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:48:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:48:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:48:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:48:55.778 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-04-25T09:48:55.804 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-04-25T09:48:55.821 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-04-25T09:48:55.821 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-04-25T09:48:55.878 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="CentOS Stream" 2024-04-25T09:48:55.878 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="9" 2024-04-25T09:48:55.878 INFO:teuthology.orchestra.run.smithi080.stdout:ID="centos" 2024-04-25T09:48:55.878 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:48:55.878 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="9" 2024-04-25T09:48:55.878 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:48:55.879 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:48:55.879 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-04-25T09:48:55.904 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:48:55.913 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:48:55.946 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:48:55.946 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:48:55.949 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:48:55.964 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:48:55.979 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:48:55.980 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:48:56.008 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:48:56.035 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:48:56.090 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:48:56.104 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:48:56.104 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T09:48:56.104 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T09:48:56.104 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:48:56.105 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:48:56.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:48:56.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T09:48:56.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:48:56.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T09:48:56.293 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:48:56.293 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:48:56.293 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-04-25T09:48:56.311 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:48:56.311 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:48:56.312 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:48:56.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum upgrade -y linux-firmware 2024-04-25T09:48:56.346 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:48:56.346 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:48:56.346 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-25T09:48:56.363 INFO:teuthology.orchestra.run.smithi080.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:48:56.363 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:48:56.363 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:48:56.364 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum upgrade -y linux-firmware 2024-04-25T09:48:58.489 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T09:48:58.602 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T09:49:00.440 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T09:49:00.561 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T09:49:04.707 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T09:49:05.412 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-25T09:49:06.174 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-25T09:49:08.045 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-25T09:49:09.034 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 7.7 MB/s | 21 MB 00:02 2024-04-25T09:49:10.421 INFO:teuthology.orchestra.run.smithi080.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 21 MB 00:02 2024-04-25T09:49:14.251 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T09:49:15.404 INFO:teuthology.orchestra.run.smithi080.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T09:49:16.290 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-25T09:49:16.292 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-04-25T09:49:16.292 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-25T09:49:16.441 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:49:16.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:16.604 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T09:49:16.604 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T09:49:16.607 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi005.front.sepia.ceph.com to complete... 2024-04-25T09:49:17.386 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-25T09:49:17.386 INFO:teuthology.orchestra.run.smithi080.stdout:Nothing to do. 2024-04-25T09:49:17.387 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-04-25T09:49:17.505 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:49:17.506 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:17.707 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T09:49:17.707 DEBUG:teuthology.orchestra.run.smithi080:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T09:49:17.710 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi080.front.sepia.ceph.com to complete... 2024-04-25T09:49:31.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25 09:49:31 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T09:49:31.872 INFO:teuthology.orchestra.run.smithi005.stderr:FINISHED --2024-04-25 09:49:31-- 2024-04-25T09:49:31.872 INFO:teuthology.orchestra.run.smithi005.stderr:Total wall clock time: 15s 2024-04-25T09:49:31.872 INFO:teuthology.orchestra.run.smithi005.stderr:Downloaded: 1 files, 341M in 15s (22.8 MB/s) 2024-04-25T09:49:31.875 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:49:31.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T09:49:34.102 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying... ######################################## 2024-04-25T09:49:34.242 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing... ######################################## 2024-04-25T09:49:34.347 INFO:teuthology.orchestra.run.smithi005.stdout:Updating / installing... 2024-04-25T09:49:42.620 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T09:49:46.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25 09:49:46 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T09:49:46.950 INFO:teuthology.orchestra.run.smithi080.stderr:FINISHED --2024-04-25 09:49:46-- 2024-04-25T09:49:46.950 INFO:teuthology.orchestra.run.smithi080.stderr:Total wall clock time: 29s 2024-04-25T09:49:46.950 INFO:teuthology.orchestra.run.smithi080.stderr:Downloaded: 1 files, 341M in 29s (11.9 MB/s) 2024-04-25T09:49:46.953 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:49:46.953 DEBUG:teuthology.orchestra.run.smithi080:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T09:49:49.095 INFO:teuthology.orchestra.run.smithi080.stdout:Verifying... ######################################## 2024-04-25T09:49:49.222 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing... ######################################## 2024-04-25T09:49:49.341 INFO:teuthology.orchestra.run.smithi080.stdout:Updating / installing... 2024-04-25T09:49:57.320 INFO:teuthology.orchestra.run.smithi080.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T09:50:22.481 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T09:50:22.482 DEBUG:teuthology.orchestra.run.smithi005:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:22.594 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.594 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.594 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.594 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T09:50:22.594 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T09:50:22.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T09:50:22.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T09:50:22.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T09:50:22.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T09:50:22.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:50:22.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:50:22.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:50:22.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:50:22.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:50:22.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:50:22.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:50:22.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T09:50:22.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:50:22.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:50:22.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:50:22.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:50:22.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T09:50:22.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:50:22.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:50:22.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:50:22.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:50:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:50:22.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:50:22.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T09:50:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:50:22.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T09:50:22.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T09:50:22.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:50:22.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T09:50:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T09:50:22.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:50:22.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T09:50:22.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T09:50:22.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:50:22.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:50:22.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:50:22.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:50:22.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:50:22.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:50:22.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:50:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:50:22.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:50:22.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:50:22.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:50:22.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:50:22.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:50:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:50:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:50:22.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:50:22.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:50:22.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:50:22.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:50:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:50:22.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:50:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:50:22.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:50:22.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:50:22.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:50:22.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:50:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:50:22.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:50:22.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T09:50:22.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T09:50:22.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:50:22.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T09:50:22.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:50:22.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T09:50:22.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:50:22.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T09:50:22.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T09:50:22.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:50:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:50:22.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T09:50:22.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:50:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:50:22.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T09:50:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:50:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T09:50:22.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:50:22.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:50:22.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:50:22.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T09:50:22.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:50:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:50:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T09:50:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:50:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:50:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:50:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:50:22.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T09:50:22.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T09:50:22.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:50:22.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T09:50:22.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T09:50:22.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:50:22.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T09:50:22.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:50:22.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:50:22.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:50:22.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T09:50:22.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:50:22.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:50:22.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:50:22.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:50:22.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:50:22.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:50:22.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:50:22.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:50:22.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:50:22.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:50:22.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T09:50:22.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:50:22.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:50:22.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:50:22.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:50:22.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:50:22.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:50:22.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:50:22.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:50:22.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:50:22.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:50:22.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:50:22.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:50:22.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T09:50:22.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:50:22.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:50:22.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T09:50:22.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T09:50:22.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:50:22.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:50:22.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:50:22.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:50:22.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T09:50:22.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:50:22.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:50:22.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T09:50:22.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T09:50:22.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:50:22.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:50:22.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:50:22.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:50:22.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:50:22.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:50:22.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:50:22.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:50:22.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:50:22.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:50:22.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:50:22.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:50:22.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:50:22.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:50:22.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:50:22.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:50:22.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:50:22.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:50:22.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:50:22.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:50:22.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:50:22.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:50:22.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T09:50:22.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:50:22.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:50:22.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:50:22.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:50:22.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:50:22.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T09:50:22.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:50:22.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:50:22.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T09:50:22.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:50:22.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:50:22.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:50:22.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T09:50:22.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:50:22.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:50:22.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T09:50:22.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:50:22.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T09:50:22.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:50:22.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T09:50:22.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:50:22.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T09:50:22.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:50:22.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:50:22.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T09:50:22.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:50:22.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T09:50:22.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T09:50:22.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:50:22.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T09:50:22.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:50:22.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:50:22.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:50:22.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T09:50:22.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T09:50:22.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T09:50:22.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T09:50:22.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T09:50:22.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:50:22.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T09:50:22.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:50:22.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:50:22.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T09:50:22.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:50:22.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:50:22.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T09:50:22.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:50:22.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:50:22.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:50:22.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:50:22.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:50:22.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:50:22.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:50:22.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T09:50:22.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:50:22.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:50:22.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:50:22.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T09:50:22.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:50:22.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:50:22.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:50:22.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T09:50:22.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:50:22.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T09:50:22.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:50:22.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T09:50:22.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:50:22.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:50:22.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:50:22.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:50:22.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T09:50:22.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:50:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T09:50:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:50:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:50:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T09:50:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:50:22.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:50:22.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:50:22.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T09:50:22.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:50:22.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:50:22.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T09:50:22.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:50:22.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:50:22.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:50:22.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:50:22.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:50:22.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T09:50:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:50:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:50:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:50:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:50:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:50:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:50:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:50:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:50:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:50:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T09:50:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:50:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:50:22.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:50:22.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:50:22.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:50:22.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:50:22.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:50:22.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T09:50:22.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T09:50:22.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:50:22.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:50:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:50:22.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T09:50:22.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T09:50:22.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T09:50:22.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T09:50:22.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:50:22.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:50:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:50:22.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:50:22.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:50:22.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:50:22.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:50:22.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T09:50:22.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T09:50:22.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:50:22.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:50:22.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T09:50:22.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:50:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:50:22.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:50:22.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:50:22.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:50:22.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:50:22.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:50:22.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:50:22.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:50:22.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:50:22.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:50:22.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:50:22.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:50:22.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:50:22.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:50:22.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:50:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:50:22.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:50:22.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:50:22.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:50:22.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:50:22.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:50:22.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:50:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:50:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:50:22.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:50:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T09:50:22.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:50:22.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:50:22.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:50:22.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:22.765 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.765 DEBUG:teuthology.orchestra.run.smithi005:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout:for file in vmlinuz System.map config; do 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout: fi 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-04-25T09:50:22.789 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.790 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-04-25T09:50:22.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-04-25T09:50:22.864 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T09:50:22.865 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T09:50:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T09:50:22.867 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T09:50:22.868 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:22.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T09:50:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-04-25T09:50:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T09:50:24.867 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-04-25T09:50:24.868 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-04-25T09:50:24.884 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.LRazCeN5Kb 2024-04-25T09:50:24.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LRazCeN5Kb 2024-04-25T09:50:24.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.LRazCeN5Kb 2024-04-25T09:50:25.088 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.LRazCeN5Kb is 6KB 2024-04-25T09:50:25.136 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.LRazCeN5Kb 2024-04-25T09:50:25.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T09:50:25.218 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T09:50:25.218 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T09:50:25.218 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T09:50:25.218 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T09:50:25.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:25.389 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-04-25T09:50:35.232 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi080.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T09:50:35.232 DEBUG:teuthology.orchestra.run.smithi080:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T09:50:35.313 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:50:35.314 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T09:50:35.315 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:50:35.316 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:50:35.317 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:50:35.318 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:50:35.319 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:50:35.320 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:50:35.321 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:50:35.322 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:50:35.323 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:50:35.324 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:50:35.325 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T09:50:35.326 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:50:35.327 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T09:50:35.328 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:50:35.329 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:50:35.330 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:50:35.331 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:50:35.332 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:50:35.333 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:50:35.334 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:50:35.335 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:50:35.336 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:50:35.337 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:50:35.338 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:50:35.339 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:50:35.340 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:50:35.341 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:50:35.342 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:50:35.343 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:50:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:50:35.345 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:50:35.346 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:50:35.347 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:50:35.348 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:50:35.349 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:50:35.350 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:50:35.351 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T09:50:35.352 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:50:35.353 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:50:35.354 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:50:35.355 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:50:35.356 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:50:35.357 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:50:35.358 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:50:35.359 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:50:35.360 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T09:50:35.361 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:50:35.362 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T09:50:35.363 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T09:50:35.364 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:50:35.365 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:50:35.366 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T09:50:35.367 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T09:50:35.368 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T09:50:35.369 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:50:35.370 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:50:35.371 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:50:35.372 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T09:50:35.373 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:50:35.374 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T09:50:35.375 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:50:35.376 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:50:35.377 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:50:35.378 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T09:50:35.379 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:50:35.380 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:50:35.381 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:50:35.382 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:50:35.383 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T09:50:35.384 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:50:35.385 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:50:35.386 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:50:35.387 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T09:50:35.388 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T09:50:35.389 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:50:35.390 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T09:50:35.391 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T09:50:35.392 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:50:35.393 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T09:50:35.394 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:50:35.395 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:50:35.396 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:50:35.397 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:50:35.398 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:50:35.399 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:50:35.400 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:50:35.401 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:50:35.402 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:50:35.403 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:50:35.404 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:50:35.405 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:50:35.406 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T09:50:35.407 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:50:35.408 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:50:35.409 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:50:35.410 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T09:50:35.411 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T09:50:35.412 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T09:50:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:50:35.414 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:50:35.415 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:50:35.416 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:50:35.417 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:50:35.418 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:50:35.419 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T09:50:35.420 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T09:50:35.421 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T09:50:35.422 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:50:35.423 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:50:35.424 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T09:50:35.425 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:50:35.426 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:50:35.427 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:50:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:50:35.429 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:50:35.430 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:50:35.431 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:50:35.432 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:50:35.433 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:50:35.434 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:50:35.435 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:50:35.436 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:50:35.437 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T09:50:35.438 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:50:35.439 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:50:35.440 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:50:35.441 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:50:35.442 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:50:35.443 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:50:35.444 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:50:35.445 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:50:35.446 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:50:35.447 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:50:35.448 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:50:35.449 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:50:35.450 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:50:35.451 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T09:50:35.452 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:50:35.453 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:50:35.454 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:50:35.455 INFO:teuthology.orchestra.run.smithi080.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:35.459 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.459 DEBUG:teuthology.orchestra.run.smithi080:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:35.481 INFO:teuthology.orchestra.run.smithi080.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout:fi 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout:for file in vmlinuz System.map config; do 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: fi 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout:done 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:35.482 INFO:teuthology.orchestra.run.smithi080.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:fi 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:fi 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.483 INFO:teuthology.orchestra.run.smithi080.stdout:fi 2024-04-25T09:50:35.484 DEBUG:teuthology.orchestra.run.smithi080:> sudo rpm -qi grub2-tools 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Name : grub2-tools 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Epoch : 1 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Version : 2.06 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Release : 68.el9 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: x86_64 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Group : Unspecified 2024-04-25T09:50:35.556 INFO:teuthology.orchestra.run.smithi080.stdout:Size : 8264254 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:License : GPLv3+ 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Packager : builder@centos.org 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Vendor : CentOS 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Summary : Support tools for GRUB. 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Description : 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T09:50:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:hardware devices. 2024-04-25T09:50:35.558 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T09:50:35.558 INFO:teuthology.orchestra.run.smithi080.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T09:50:35.558 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T09:50:35.559 INFO:teuthology.task.kernel:Updating grub on smithi080 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:35.559 DEBUG:teuthology.orchestra.run.smithi080:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T09:50:36.096 INFO:teuthology.orchestra.run.smithi080.stderr:Generating grub configuration file ... 2024-04-25T09:50:37.529 INFO:teuthology.orchestra.run.smithi080.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T09:50:37.574 INFO:teuthology.orchestra.run.smithi080.stderr:done 2024-04-25T09:50:37.576 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T09:50:37.591 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.rNkcucDdhI 2024-04-25T09:50:37.592 DEBUG:teuthology.orchestra.run.smithi080:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rNkcucDdhI 2024-04-25T09:50:37.670 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.rNkcucDdhI 2024-04-25T09:50:37.806 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.rNkcucDdhI is 6KB 2024-04-25T09:50:37.855 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.rNkcucDdhI 2024-04-25T09:50:37.870 DEBUG:teuthology.orchestra.run.smithi080:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T09:50:37.931 INFO:teuthology.orchestra.run.smithi080.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T09:50:37.931 INFO:teuthology.orchestra.run.smithi080.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T09:50:37.931 INFO:teuthology.orchestra.run.smithi080.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T09:50:37.931 INFO:teuthology.orchestra.run.smithi080.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T09:50:37.932 DEBUG:teuthology.orchestra.run.smithi080:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:38.091 DEBUG:teuthology.orchestra.run.smithi080:> sudo shutdown -r now 2024-04-25T09:50:55.406 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T09:50:55.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-25T09:50:55.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:08.106 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T09:51:08.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:51:08.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:42.623 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T09:51:51.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:51:51.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:55.414 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:52:04.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-25T09:52:04.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:07.807 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-04-25T09:52:08.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-25T09:52:08.500 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T09:52:08.500 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-04-25T09:52:08.550 INFO:teuthology.orchestra.run.smithi005.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:08.550 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:08.550 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:52:08.550 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T09:52:09.553 DEBUG:teuthology.parallel:result is None 2024-04-25T09:52:10.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T09:52:22.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:52:22.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:25.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T09:52:40.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:52:40.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:40.558 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-25T09:52:41.392 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:52:41.392 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T09:52:41.393 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-25T09:52:41.468 INFO:teuthology.orchestra.run.smithi080.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:41.468 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:41.468 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:52:41.469 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T09:52:42.470 DEBUG:teuthology.parallel:result is None 2024-04-25T09:52:42.471 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:52:42.478 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:52:42.479 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:52:42.489 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:52:42.544 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:52:42.550 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:52:42.578 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:52:42.578 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:52:42.583 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:52:42.657 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:52:42.664 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:52:42.664 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:52:42.672 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:52:42.782 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:52:42.821 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:52:42.851 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:52:42.877 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:52:42.882 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:52:42.889 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:52:42.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:52:42.897 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:52:42.982 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:52:42.992 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:52:42.993 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:52:43.000 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:52:43.051 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:52:43.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:52:43.239 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:52:43.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:52:43.411 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T09:52:43.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:52:43.497 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:52:43.579 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:52:43.660 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:52:43.730 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:52:43.798 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T09:52:43.798 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:52:43.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-04-25T09:52:43.863 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-04-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:52:44.474 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:52:44.484 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:52:44.484 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:52:44.515 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:52:44.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-04-25T09:52:44.668 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-04-25T09:52:44.673 DEBUG:teuthology.orchestra.run.smithi080:> sudo service auditd rotate 2024-04-25T09:52:44.849 INFO:teuthology.orchestra.run.smithi080.stdout:Rotating logs: 2024-04-25T09:52:44.854 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:52:44.854 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-04-25T09:52:44.916 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-04-25T09:52:44.917 DEBUG:teuthology.orchestra.run.smithi080:> /usr/sbin/getenforce 2024-04-25T09:52:44.963 INFO:teuthology.orchestra.run.smithi080.stdout:Permissive 2024-04-25T09:52:44.963 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi080.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:52:44.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:52:45.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:52:45.041 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:52:45.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:52:45.120 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:52:45.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:52:45.217 DEBUG:teuthology.orchestra.run.smithi080:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:52:45.294 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:52:45.304 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T09:52:45.310 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T09:52:45.452 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:52:45.467 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:52:45.469 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi080.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:58:41.342 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi080.front.sepia.ceph.com')] 2024-04-25T09:58:41.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-25T09:58:41.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:41.457 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-04-25T09:58:41.615 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-04-25T09:58:41.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:58:41.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:41.724 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-25T09:58:41.866 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T09:58:41.866 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:58:41.877 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:58:41.877 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:41.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:41.885 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:41.885 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:41.974 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:58:41.977 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:58:42.015 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:58:42.025 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:58:42.096 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: ntpd: command not found 2024-04-25T09:58:42.103 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-04-25T09:58:42.126 INFO:teuthology.orchestra.run.smithi080.stdout:506 Cannot talk to daemon 2024-04-25T09:58:42.138 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-04-25T09:58:42.164 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:58:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:58:42.202 INFO:teuthology.orchestra.run.smithi080.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:58:42.229 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:58:42.332 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:58:42.340 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:58:42.340 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-04-25T09:58:42.341 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.341 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.341 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.341 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:58:42.380 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:58:42.380 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-04-25T09:58:42.380 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.381 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.381 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.381 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:42.382 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:58:42.392 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:58:42.392 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:58:42.392 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:58:42.392 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:58:42.399 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:58:42.399 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:58:42.399 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:58:42.400 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:42.405 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:58:42.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:42.551 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:58:42.552 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:58:42.569 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:58:42.569 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:58:42.675 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:58:42.676 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T09:58:42.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:58:42.692 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:58:42.692 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T09:58:42.692 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:58:42.773 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:58:42.773 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:58:42.780 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:58:42.781 DEBUG:teuthology.orchestra.run.smithi080:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:58:42.905 DEBUG:teuthology.orchestra.run.smithi080:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:58:42.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:58:43.020 DEBUG:teuthology.orchestra.run.smithi080:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:58:43.027 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:58:43.097 INFO:teuthology.orchestra.run.smithi080.stdout:check_obsoletes = 1 2024-04-25T09:58:43.101 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum clean all 2024-04-25T09:58:43.110 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-04-25T09:58:43.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-04-25T09:58:43.418 INFO:teuthology.orchestra.run.smithi080.stdout:48 files removed 2024-04-25T09:58:43.442 INFO:teuthology.orchestra.run.smithi005.stdout:48 files removed 2024-04-25T09:58:43.457 DEBUG:teuthology.orchestra.run.smithi080:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:58:43.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:58:44.280 INFO:teuthology.orchestra.run.smithi080.stdout:ceph packages for x86_64 210 kB/s | 82 kB 00:00 2024-04-25T09:58:44.342 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 199 kB/s | 82 kB 00:00 2024-04-25T09:58:44.806 INFO:teuthology.orchestra.run.smithi080.stdout:ceph noarch packages 23 kB/s | 10 kB 00:00 2024-04-25T09:58:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 24 kB/s | 10 kB 00:00 2024-04-25T09:58:45.281 INFO:teuthology.orchestra.run.smithi080.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-25T09:58:45.343 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-25T09:58:45.938 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T09:58:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T09:58:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T09:58:48.480 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T09:58:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-04-25T09:58:53.439 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-25T09:58:54.903 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T09:58:56.356 INFO:teuthology.orchestra.run.smithi080.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-25T09:58:59.837 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 21 MB 00:04 2024-04-25T09:58:59.866 INFO:teuthology.orchestra.run.smithi080.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-04-25T09:59:05.732 INFO:teuthology.orchestra.run.smithi080.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-04-25T09:59:06.067 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-25T09:59:08.142 INFO:teuthology.orchestra.run.smithi080.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:08.143 INFO:teuthology.orchestra.run.smithi080.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:08.163 INFO:teuthology.orchestra.run.smithi080.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:59:08.165 INFO:teuthology.orchestra.run.smithi080.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:59:08.166 INFO:teuthology.orchestra.run.smithi080.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:59:08.167 INFO:teuthology.orchestra.run.smithi080.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:59:08.170 INFO:teuthology.orchestra.run.smithi080.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:59:08.173 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:59:08.174 INFO:teuthology.orchestra.run.smithi080.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:59:08.246 INFO:teuthology.orchestra.run.smithi080.stdout:Dependencies resolved. 2024-04-25T09:59:08.260 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: Package Arch Version Repository Size 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout:Installing: 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:59:08.261 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:59:08.262 INFO:teuthology.orchestra.run.smithi080.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:59:08.263 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrading: 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout:Installing dependencies: 2024-04-25T09:59:08.264 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:59:08.265 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:59:08.266 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:59:08.267 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:59:08.268 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:59:08.269 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:59:08.270 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:59:08.271 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:59:08.272 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout:Installing weak dependencies: 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction Summary 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout:======================================================================================= 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout:Install 148 Packages 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout:Upgrade 4 Packages 2024-04-25T09:59:08.273 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T09:59:08.274 INFO:teuthology.orchestra.run.smithi080.stdout:Total download size: 215 M 2024-04-25T09:59:08.274 INFO:teuthology.orchestra.run.smithi080.stdout:Downloading Packages: 2024-04-25T09:59:08.624 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:08.626 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:08.648 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:59:08.649 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:59:08.650 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:59:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:59:08.654 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:59:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:59:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:59:08.737 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-04-25T09:59:08.762 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-25T09:59:08.762 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-04-25T09:59:08.762 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-25T09:59:08.762 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:59:08.763 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:59:08.764 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:59:08.765 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:59:08.766 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:59:08.767 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:59:08.768 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:59:08.769 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:59:08.770 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:59:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:59:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:59:08.773 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T09:59:08.774 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-04-25T09:59:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-04-25T09:59:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-04-25T09:59:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 4 Packages 2024-04-25T09:59:08.775 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T09:59:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 215 M 2024-04-25T09:59:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-04-25T09:59:09.584 INFO:teuthology.orchestra.run.smithi080.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-25T09:59:10.080 INFO:teuthology.orchestra.run.smithi005.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T09:59:10.500 INFO:teuthology.orchestra.run.smithi080.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 968 kB/s | 887 kB 00:00 2024-04-25T09:59:10.751 INFO:teuthology.orchestra.run.smithi080.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 584 kB/s | 145 kB 00:00 2024-04-25T09:59:10.838 INFO:teuthology.orchestra.run.smithi005.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.1 MB/s | 887 kB 00:00 2024-04-25T09:59:10.939 INFO:teuthology.orchestra.run.smithi005.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.4 MB/s | 145 kB 00:00 2024-04-25T09:59:11.559 INFO:teuthology.orchestra.run.smithi080.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 2.3 MB/s | 5.3 MB 00:02 2024-04-25T09:59:11.664 INFO:teuthology.orchestra.run.smithi005.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 2.7 MB/s | 5.3 MB 00:01 2024-04-25T09:59:11.739 INFO:teuthology.orchestra.run.smithi005.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 2.7 MB/s | 2.1 MB 00:00 2024-04-25T09:59:11.868 INFO:teuthology.orchestra.run.smithi080.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-04-25T09:59:12.031 INFO:teuthology.orchestra.run.smithi005.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-04-25T09:59:12.210 INFO:teuthology.orchestra.run.smithi080.stdout:(6/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 1.5 MB/s | 2.1 MB 00:01 2024-04-25T09:59:12.893 INFO:teuthology.orchestra.run.smithi080.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 4.6 MB/s | 4.7 MB 00:01 2024-04-25T09:59:13.265 INFO:teuthology.orchestra.run.smithi005.stdout:(7/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 5.6 MB/s | 20 MB 00:03 2024-04-25T09:59:13.382 INFO:teuthology.orchestra.run.smithi005.stdout:(8/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-04-25T09:59:13.440 INFO:teuthology.orchestra.run.smithi005.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 440 kB/s | 25 kB 00:00 2024-04-25T09:59:13.628 INFO:teuthology.orchestra.run.smithi080.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 4.6 MB/s | 20 MB 00:04 2024-04-25T09:59:13.719 INFO:teuthology.orchestra.run.smithi080.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 282 kB/s | 25 kB 00:00 2024-04-25T09:59:14.502 INFO:teuthology.orchestra.run.smithi080.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 6.1 MB/s | 9.8 MB 00:01 2024-04-25T09:59:14.602 INFO:teuthology.orchestra.run.smithi080.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.2 MB/s | 220 kB 00:00 2024-04-25T09:59:14.678 INFO:teuthology.orchestra.run.smithi080.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 363 kB/s | 27 kB 00:00 2024-04-25T09:59:14.765 INFO:teuthology.orchestra.run.smithi005.stdout:(10/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 6.5 MB/s | 9.8 MB 00:01 2024-04-25T09:59:14.769 INFO:teuthology.orchestra.run.smithi080.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 344 kB/s | 31 kB 00:00 2024-04-25T09:59:14.887 INFO:teuthology.orchestra.run.smithi080.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 6.3 MB/s | 744 kB 00:00 2024-04-25T09:59:14.953 INFO:teuthology.orchestra.run.smithi080.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.4 MB/s | 163 kB 00:00 2024-04-25T09:59:15.016 INFO:teuthology.orchestra.run.smithi005.stdout:(11/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 5.6 MB/s | 17 MB 00:02 2024-04-25T09:59:15.020 INFO:teuthology.orchestra.run.smithi080.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 127 kB 00:00 2024-04-25T09:59:15.049 INFO:teuthology.orchestra.run.smithi005.stdout:(12/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 778 kB/s | 220 kB 00:00 2024-04-25T09:59:15.074 INFO:teuthology.orchestra.run.smithi005.stdout:(13/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 473 kB/s | 27 kB 00:00 2024-04-25T09:59:15.116 INFO:teuthology.orchestra.run.smithi005.stdout:(14/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 474 kB/s | 31 kB 00:00 2024-04-25T09:59:15.154 INFO:teuthology.orchestra.run.smithi080.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 3.7 MB/s | 501 kB 00:00 2024-04-25T09:59:15.183 INFO:teuthology.orchestra.run.smithi005.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.4 MB/s | 163 kB 00:00 2024-04-25T09:59:15.234 INFO:teuthology.orchestra.run.smithi005.stdout:(16/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 4.6 MB/s | 744 kB 00:00 2024-04-25T09:59:15.267 INFO:teuthology.orchestra.run.smithi005.stdout:(17/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.5 MB/s | 127 kB 00:00 2024-04-25T09:59:15.409 INFO:teuthology.orchestra.run.smithi005.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 2.8 MB/s | 501 kB 00:00 2024-04-25T09:59:15.501 INFO:teuthology.orchestra.run.smithi005.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 496 kB/s | 45 kB 00:00 2024-04-25T09:59:15.601 INFO:teuthology.orchestra.run.smithi005.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.3 MB/s | 131 kB 00:00 2024-04-25T09:59:15.693 INFO:teuthology.orchestra.run.smithi005.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.7 MB/s | 162 kB 00:00 2024-04-25T09:59:15.802 INFO:teuthology.orchestra.run.smithi005.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.9 MB/s | 321 kB 00:00 2024-04-25T09:59:15.910 INFO:teuthology.orchestra.run.smithi005.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 2.7 MB/s | 298 kB 00:00 2024-04-25T09:59:15.986 INFO:teuthology.orchestra.run.smithi005.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T09:59:16.021 INFO:teuthology.orchestra.run.smithi080.stdout:(18/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 5.9 MB/s | 5.1 MB 00:00 2024-04-25T09:59:16.088 INFO:teuthology.orchestra.run.smithi080.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 688 kB/s | 45 kB 00:00 2024-04-25T09:59:16.119 INFO:teuthology.orchestra.run.smithi005.stdout:(25/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 6.0 MB/s | 5.1 MB 00:00 2024-04-25T09:59:16.144 INFO:teuthology.orchestra.run.smithi005.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 541 kB/s | 85 kB 00:00 2024-04-25T09:59:16.163 INFO:teuthology.orchestra.run.smithi080.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.7 MB/s | 131 kB 00:00 2024-04-25T09:59:16.270 INFO:teuthology.orchestra.run.smithi005.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 1.3 MB/s | 170 kB 00:00 2024-04-25T09:59:16.280 INFO:teuthology.orchestra.run.smithi080.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.4 MB/s | 162 kB 00:00 2024-04-25T09:59:16.362 INFO:teuthology.orchestra.run.smithi005.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 347 kB/s | 31 kB 00:00 2024-04-25T09:59:16.389 INFO:teuthology.orchestra.run.smithi080.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.9 MB/s | 321 kB 00:00 2024-04-25T09:59:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T09:59:16.481 INFO:teuthology.orchestra.run.smithi080.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T09:59:16.779 INFO:teuthology.orchestra.run.smithi005.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 4.7 MB/s | 3.1 MB 00:00 2024-04-25T09:59:16.854 INFO:teuthology.orchestra.run.smithi005.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 4.4 MB/s | 1.8 MB 00:00 2024-04-25T09:59:16.955 INFO:teuthology.orchestra.run.smithi005.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.4 MB/s | 248 kB 00:00 2024-04-25T09:59:17.047 INFO:teuthology.orchestra.run.smithi005.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 541 kB/s | 49 kB 00:00 2024-04-25T09:59:17.106 INFO:teuthology.orchestra.run.smithi080.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 159 kB/s | 99 kB 00:00 2024-04-25T09:59:17.139 INFO:teuthology.orchestra.run.smithi005.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 169 kB/s | 15 kB 00:00 2024-04-25T09:59:17.198 INFO:teuthology.orchestra.run.smithi080.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 943 kB/s | 85 kB 00:00 2024-04-25T09:59:17.231 INFO:teuthology.orchestra.run.smithi005.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 2.8 MB/s | 264 kB 00:00 2024-04-25T09:59:17.498 INFO:teuthology.orchestra.run.smithi005.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 2.7 MB/s | 748 kB 00:00 2024-04-25T09:59:17.573 INFO:teuthology.orchestra.run.smithi005.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 247 kB/s | 18 kB 00:00 2024-04-25T09:59:17.865 INFO:teuthology.orchestra.run.smithi005.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 276 kB/s | 80 kB 00:00 2024-04-25T09:59:17.973 INFO:teuthology.orchestra.run.smithi080.stdout:(26/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 4.0 MB/s | 3.1 MB 00:00 2024-04-25T09:59:18.049 INFO:teuthology.orchestra.run.smithi080.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.3 MB/s | 170 kB 00:00 2024-04-25T09:59:18.141 INFO:teuthology.orchestra.run.smithi080.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 346 kB/s | 31 kB 00:00 2024-04-25T09:59:18.207 INFO:teuthology.orchestra.run.smithi005.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-04-25T09:59:18.224 INFO:teuthology.orchestra.run.smithi080.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T09:59:18.341 INFO:teuthology.orchestra.run.smithi005.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-04-25T09:59:18.374 INFO:teuthology.orchestra.run.smithi005.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-25T09:59:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 3.4 MB/s | 1.2 MB 00:00 2024-04-25T09:59:18.749 INFO:teuthology.orchestra.run.smithi080.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 3.4 MB/s | 1.8 MB 00:00 2024-04-25T09:59:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 597 kB/s | 253 kB 00:00 2024-04-25T09:59:18.825 INFO:teuthology.orchestra.run.smithi005.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T09:59:18.858 INFO:teuthology.orchestra.run.smithi005.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 135 kB 00:00 2024-04-25T09:59:18.925 INFO:teuthology.orchestra.run.smithi005.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-25T09:59:18.959 INFO:teuthology.orchestra.run.smithi005.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T09:59:19.059 INFO:teuthology.orchestra.run.smithi005.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.1 MB/s | 689 kB 00:00 2024-04-25T09:59:19.118 INFO:teuthology.orchestra.run.smithi005.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-04-25T09:59:19.151 INFO:teuthology.orchestra.run.smithi005.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-25T09:59:19.177 INFO:teuthology.orchestra.run.smithi005.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 374 kB/s | 21 kB 00:00 2024-04-25T09:59:19.227 INFO:teuthology.orchestra.run.smithi005.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 667 kB/s | 33 kB 00:00 2024-04-25T09:59:19.269 INFO:teuthology.orchestra.run.smithi005.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-25T09:59:19.319 INFO:teuthology.orchestra.run.smithi005.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 375 kB/s | 18 kB 00:00 2024-04-25T09:59:19.361 INFO:teuthology.orchestra.run.smithi005.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 275 kB/s | 11 kB 00:00 2024-04-25T09:59:19.462 INFO:teuthology.orchestra.run.smithi005.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-25T09:59:19.495 INFO:teuthology.orchestra.run.smithi005.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-04-25T09:59:19.554 INFO:teuthology.orchestra.run.smithi005.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 498 kB/s | 45 kB 00:00 2024-04-25T09:59:19.662 INFO:teuthology.orchestra.run.smithi005.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-25T09:59:19.683 INFO:teuthology.orchestra.run.smithi080.stdout:(31/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 2.2 MB/s | 17 MB 00:07 2024-04-25T09:59:19.712 INFO:teuthology.orchestra.run.smithi005.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 520 kB/s | 25 kB 00:00 2024-04-25T09:59:19.804 INFO:teuthology.orchestra.run.smithi005.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 525 kB/s | 47 kB 00:00 2024-04-25T09:59:19.825 INFO:teuthology.orchestra.run.smithi080.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.7 MB/s | 248 kB 00:00 2024-04-25T09:59:19.883 INFO:teuthology.orchestra.run.smithi080.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 860 kB/s | 49 kB 00:00 2024-04-25T09:59:19.922 INFO:teuthology.orchestra.run.smithi005.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-25T09:59:19.942 INFO:teuthology.orchestra.run.smithi080.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 265 kB/s | 15 kB 00:00 2024-04-25T09:59:19.955 INFO:teuthology.orchestra.run.smithi005.stdout:(63/152): librdkafka-1.6.1-102.el9.x86_64.rpm 1.4 MB/s | 662 kB 00:00 2024-04-25T09:59:20.005 INFO:teuthology.orchestra.run.smithi005.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 719 kB/s | 35 kB 00:00 2024-04-25T09:59:20.039 INFO:teuthology.orchestra.run.smithi005.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-04-25T09:59:20.067 INFO:teuthology.orchestra.run.smithi080.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 2.1 MB/s | 264 kB 00:00 2024-04-25T09:59:20.251 INFO:teuthology.orchestra.run.smithi080.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-04-25T09:59:20.302 INFO:teuthology.orchestra.run.smithi080.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 373 kB/s | 18 kB 00:00 2024-04-25T09:59:20.331 INFO:teuthology.orchestra.run.smithi005.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T09:59:20.351 INFO:teuthology.orchestra.run.smithi080.stdout:(38/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 2.6 MB/s | 748 kB 00:00 2024-04-25T09:59:20.439 INFO:teuthology.orchestra.run.smithi005.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T09:59:20.548 INFO:teuthology.orchestra.run.smithi005.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.4 MB/s | 1.6 MB 00:00 2024-04-25T09:59:20.623 INFO:teuthology.orchestra.run.smithi005.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 911 kB 00:00 2024-04-25T09:59:20.643 INFO:teuthology.orchestra.run.smithi080.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 236 kB/s | 80 kB 00:00 2024-04-25T09:59:20.682 INFO:teuthology.orchestra.run.smithi005.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-25T09:59:20.808 INFO:teuthology.orchestra.run.smithi005.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-25T09:59:20.883 INFO:teuthology.orchestra.run.smithi005.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-25T09:59:20.902 INFO:teuthology.orchestra.run.smithi080.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 757 kB/s | 195 kB 00:00 2024-04-25T09:59:21.028 INFO:teuthology.orchestra.run.smithi080.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T09:59:21.033 INFO:teuthology.orchestra.run.smithi005.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T09:59:21.070 INFO:teuthology.orchestra.run.smithi080.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-25T09:59:21.117 INFO:teuthology.orchestra.run.smithi005.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T09:59:21.145 INFO:teuthology.orchestra.run.smithi080.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T09:59:21.200 INFO:teuthology.orchestra.run.smithi005.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-25T09:59:21.204 INFO:teuthology.orchestra.run.smithi080.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-04-25T09:59:21.250 INFO:teuthology.orchestra.run.smithi005.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 525 kB/s | 26 kB 00:00 2024-04-25T09:59:21.262 INFO:teuthology.orchestra.run.smithi080.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-25T09:59:21.296 INFO:teuthology.orchestra.run.smithi080.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-25T09:59:21.301 INFO:teuthology.orchestra.run.smithi005.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 392 kB/s | 19 kB 00:00 2024-04-25T09:59:21.338 INFO:teuthology.orchestra.run.smithi080.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T09:59:21.443 INFO:teuthology.orchestra.run.smithi005.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T09:59:21.493 INFO:teuthology.orchestra.run.smithi005.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 902 kB/s | 172 kB 00:00 2024-04-25T09:59:21.519 INFO:teuthology.orchestra.run.smithi005.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 469 kB/s | 35 kB 00:00 2024-04-25T09:59:21.580 INFO:teuthology.orchestra.run.smithi080.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-25T09:59:21.610 INFO:teuthology.orchestra.run.smithi005.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T09:59:21.638 INFO:teuthology.orchestra.run.smithi080.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T09:59:21.685 INFO:teuthology.orchestra.run.smithi005.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-25T09:59:21.744 INFO:teuthology.orchestra.run.smithi005.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T09:59:22.078 INFO:teuthology.orchestra.run.smithi005.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-04-25T09:59:22.103 INFO:teuthology.orchestra.run.smithi005.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 118 kB/s | 42 kB 00:00 2024-04-25T09:59:22.155 INFO:teuthology.orchestra.run.smithi080.stdout:(50/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 5.6 MB/s | 47 MB 00:08 2024-04-25T09:59:22.172 INFO:teuthology.orchestra.run.smithi080.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 40 kB/s | 21 kB 00:00 2024-04-25T09:59:22.178 INFO:teuthology.orchestra.run.smithi005.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T09:59:22.214 INFO:teuthology.orchestra.run.smithi080.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-25T09:59:22.231 INFO:teuthology.orchestra.run.smithi080.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 564 kB/s | 33 kB 00:00 2024-04-25T09:59:22.245 INFO:teuthology.orchestra.run.smithi005.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-04-25T09:59:22.281 INFO:teuthology.orchestra.run.smithi080.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 368 kB/s | 18 kB 00:00 2024-04-25T09:59:22.304 INFO:teuthology.orchestra.run.smithi005.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 892 kB/s | 51 kB 00:00 2024-04-25T09:59:22.329 INFO:teuthology.orchestra.run.smithi005.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-04-25T09:59:22.340 INFO:teuthology.orchestra.run.smithi080.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-04-25T09:59:22.398 INFO:teuthology.orchestra.run.smithi080.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T09:59:22.432 INFO:teuthology.orchestra.run.smithi080.stdout:(57/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-25T09:59:22.457 INFO:teuthology.orchestra.run.smithi080.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-04-25T09:59:22.541 INFO:teuthology.orchestra.run.smithi080.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-25T09:59:22.591 INFO:teuthology.orchestra.run.smithi080.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-25T09:59:22.617 INFO:teuthology.orchestra.run.smithi080.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 344 kB/s | 25 kB 00:00 2024-04-25T09:59:22.692 INFO:teuthology.orchestra.run.smithi080.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.3 MB/s | 3.0 MB 00:00 2024-04-25T09:59:22.717 INFO:teuthology.orchestra.run.smithi080.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 380 kB/s | 47 kB 00:00 2024-04-25T09:59:22.751 INFO:teuthology.orchestra.run.smithi080.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-25T09:59:22.785 INFO:teuthology.orchestra.run.smithi080.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T09:59:22.801 INFO:teuthology.orchestra.run.smithi080.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 423 kB/s | 35 kB 00:00 2024-04-25T09:59:22.846 INFO:teuthology.orchestra.run.smithi005.stdout:(90/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 5.0 MB/s | 47 MB 00:09 2024-04-25T09:59:22.872 INFO:teuthology.orchestra.run.smithi005.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 95 kB/s | 54 kB 00:00 2024-04-25T09:59:22.922 INFO:teuthology.orchestra.run.smithi005.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 562 kB/s | 42 kB 00:00 2024-04-25T09:59:22.935 INFO:teuthology.orchestra.run.smithi080.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T09:59:22.947 INFO:teuthology.orchestra.run.smithi005.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-25T09:59:22.998 INFO:teuthology.orchestra.run.smithi005.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 726 kB/s | 36 kB 00:00 2024-04-25T09:59:23.031 INFO:teuthology.orchestra.run.smithi005.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-25T09:59:23.077 INFO:teuthology.orchestra.run.smithi080.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T09:59:23.081 INFO:teuthology.orchestra.run.smithi005.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 576 kB/s | 47 kB 00:00 2024-04-25T09:59:23.121 INFO:teuthology.orchestra.run.smithi080.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.8 MB/s | 911 kB 00:00 2024-04-25T09:59:23.140 INFO:teuthology.orchestra.run.smithi005.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-04-25T09:59:23.155 INFO:teuthology.orchestra.run.smithi080.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-25T09:59:23.165 INFO:teuthology.orchestra.run.smithi005.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 701 kB/s | 58 kB 00:00 2024-04-25T09:59:23.249 INFO:teuthology.orchestra.run.smithi005.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 872 kB/s | 94 kB 00:00 2024-04-25T09:59:23.274 INFO:teuthology.orchestra.run.smithi005.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 593 kB/s | 64 kB 00:00 2024-04-25T09:59:23.313 INFO:teuthology.orchestra.run.smithi080.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-04-25T09:59:23.324 INFO:teuthology.orchestra.run.smithi005.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-04-25T09:59:23.350 INFO:teuthology.orchestra.run.smithi005.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 299 kB/s | 22 kB 00:00 2024-04-25T09:59:23.355 INFO:teuthology.orchestra.run.smithi080.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-04-25T09:59:23.406 INFO:teuthology.orchestra.run.smithi080.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 961 kB/s | 87 kB 00:00 2024-04-25T09:59:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.8 MB/s | 863 kB 00:00 2024-04-25T09:59:23.539 INFO:teuthology.orchestra.run.smithi080.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T09:59:23.573 INFO:teuthology.orchestra.run.smithi080.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T09:59:23.656 INFO:teuthology.orchestra.run.smithi080.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-25T09:59:23.682 INFO:teuthology.orchestra.run.smithi080.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-04-25T09:59:23.715 INFO:teuthology.orchestra.run.smithi080.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-25T09:59:23.734 INFO:teuthology.orchestra.run.smithi005.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-04-25T09:59:23.750 INFO:teuthology.orchestra.run.smithi080.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-25T09:59:23.774 INFO:teuthology.orchestra.run.smithi080.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 601 kB/s | 35 kB 00:00 2024-04-25T09:59:23.883 INFO:teuthology.orchestra.run.smithi080.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T09:59:23.916 INFO:teuthology.orchestra.run.smithi080.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-04-25T09:59:23.958 INFO:teuthology.orchestra.run.smithi080.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-25T09:59:23.959 INFO:teuthology.orchestra.run.smithi005.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 129 kB/s | 19 kB 00:00 2024-04-25T09:59:23.983 INFO:teuthology.orchestra.run.smithi080.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T09:59:23.993 INFO:teuthology.orchestra.run.smithi005.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 854 kB/s | 548 kB 00:00 2024-04-25T09:59:24.059 INFO:teuthology.orchestra.run.smithi080.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 424 kB/s | 42 kB 00:00 2024-04-25T09:59:24.085 INFO:teuthology.orchestra.run.smithi005.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 539 kB/s | 309 kB 00:00 2024-04-25T09:59:24.101 INFO:teuthology.orchestra.run.smithi080.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-25T09:59:24.110 INFO:teuthology.orchestra.run.smithi005.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-04-25T09:59:24.134 INFO:teuthology.orchestra.run.smithi080.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T09:59:24.160 INFO:teuthology.orchestra.run.smithi005.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 710 kB/s | 49 kB 00:00 2024-04-25T09:59:24.176 INFO:teuthology.orchestra.run.smithi080.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-25T09:59:24.202 INFO:teuthology.orchestra.run.smithi080.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 765 kB/s | 51 kB 00:00 2024-04-25T09:59:24.227 INFO:teuthology.orchestra.run.smithi080.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T09:59:24.260 INFO:teuthology.orchestra.run.smithi005.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-25T09:59:24.285 INFO:teuthology.orchestra.run.smithi005.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 419 kB/s | 67 kB 00:00 2024-04-25T09:59:24.319 INFO:teuthology.orchestra.run.smithi005.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-25T09:59:24.336 INFO:teuthology.orchestra.run.smithi080.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-04-25T09:59:24.361 INFO:teuthology.orchestra.run.smithi080.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 313 kB/s | 42 kB 00:00 2024-04-25T09:59:24.369 INFO:teuthology.orchestra.run.smithi005.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 10 MB/s | 838 kB 00:00 2024-04-25T09:59:24.420 INFO:teuthology.orchestra.run.smithi080.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 533 kB/s | 44 kB 00:00 2024-04-25T09:59:24.424 INFO:teuthology.orchestra.run.smithi005.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.9 MB/s | 548 kB 00:00 2024-04-25T09:59:24.445 INFO:teuthology.orchestra.run.smithi005.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 396 kB/s | 43 kB 00:00 2024-04-25T09:59:24.453 INFO:teuthology.orchestra.run.smithi080.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi080.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 728 kB/s | 36 kB 00:00 2024-04-25T09:59:24.470 INFO:teuthology.orchestra.run.smithi005.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 735 kB/s | 32 kB 00:00 2024-04-25T09:59:24.495 INFO:teuthology.orchestra.run.smithi005.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 218 kB/s | 15 kB 00:00 2024-04-25T09:59:24.504 INFO:teuthology.orchestra.run.smithi080.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 955 kB/s | 47 kB 00:00 2024-04-25T09:59:24.529 INFO:teuthology.orchestra.run.smithi005.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-25T09:59:24.537 INFO:teuthology.orchestra.run.smithi080.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-25T09:59:24.554 INFO:teuthology.orchestra.run.smithi080.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T09:59:24.562 INFO:teuthology.orchestra.run.smithi005.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 5.5 MB/s | 358 kB 00:00 2024-04-25T09:59:24.596 INFO:teuthology.orchestra.run.smithi005.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T09:59:24.596 INFO:teuthology.orchestra.run.smithi080.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T09:59:24.621 INFO:teuthology.orchestra.run.smithi005.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 416 kB/s | 35 kB 00:00 2024-04-25T09:59:24.622 INFO:teuthology.orchestra.run.smithi080.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 961 kB/s | 64 kB 00:00 2024-04-25T09:59:24.663 INFO:teuthology.orchestra.run.smithi005.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.1 MB/s | 206 kB 00:00 2024-04-25T09:59:24.664 INFO:teuthology.orchestra.run.smithi080.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-04-25T09:59:24.689 INFO:teuthology.orchestra.run.smithi080.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 337 kB/s | 22 kB 00:00 2024-04-25T09:59:24.739 INFO:teuthology.orchestra.run.smithi005.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-25T09:59:24.764 INFO:teuthology.orchestra.run.smithi005.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-25T09:59:24.789 INFO:teuthology.orchestra.run.smithi005.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 94 kB/s | 11 kB 00:00 2024-04-25T09:59:24.815 INFO:teuthology.orchestra.run.smithi080.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-25T09:59:24.848 INFO:teuthology.orchestra.run.smithi005.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 215 kB/s | 18 kB 00:00 2024-04-25T09:59:24.873 INFO:teuthology.orchestra.run.smithi005.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 216 kB/s | 23 kB 00:00 2024-04-25T09:59:24.898 INFO:teuthology.orchestra.run.smithi005.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-04-25T09:59:24.923 INFO:teuthology.orchestra.run.smithi005.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 403 kB/s | 20 kB 00:00 2024-04-25T09:59:24.973 INFO:teuthology.orchestra.run.smithi005.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-25T09:59:24.999 INFO:teuthology.orchestra.run.smithi005.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-04-25T09:59:25.024 INFO:teuthology.orchestra.run.smithi005.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 896 kB/s | 79 kB 00:00 2024-04-25T09:59:25.058 INFO:teuthology.orchestra.run.smithi005.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-04-25T09:59:25.083 INFO:teuthology.orchestra.run.smithi005.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.3 MB/s | 272 kB 00:00 2024-04-25T09:59:25.100 INFO:teuthology.orchestra.run.smithi005.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 231 kB/s | 16 kB 00:00 2024-04-25T09:59:25.123 INFO:teuthology.orchestra.run.smithi080.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-25T09:59:25.133 INFO:teuthology.orchestra.run.smithi005.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T09:59:25.150 INFO:teuthology.orchestra.run.smithi005.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 430 kB/s | 28 kB 00:00 2024-04-25T09:59:25.167 INFO:teuthology.orchestra.run.smithi005.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 463 kB/s | 31 kB 00:00 2024-04-25T09:59:25.201 INFO:teuthology.orchestra.run.smithi005.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 3.8 MB/s | 188 kB 00:00 2024-04-25T09:59:25.217 INFO:teuthology.orchestra.run.smithi005.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 905 kB/s | 59 kB 00:00 2024-04-25T09:59:25.268 INFO:teuthology.orchestra.run.smithi005.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-25T09:59:25.284 INFO:teuthology.orchestra.run.smithi005.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 778 kB/s | 51 kB 00:00 2024-04-25T09:59:25.310 INFO:teuthology.orchestra.run.smithi005.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-04-25T09:59:25.335 INFO:teuthology.orchestra.run.smithi005.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.4 MB/s | 90 kB 00:00 2024-04-25T09:59:25.369 INFO:teuthology.orchestra.run.smithi005.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.4 MB/s | 427 kB 00:00 2024-04-25T09:59:25.394 INFO:teuthology.orchestra.run.smithi005.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 245 kB/s | 20 kB 00:00 2024-04-25T09:59:25.427 INFO:teuthology.orchestra.run.smithi005.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T09:59:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-25T09:59:25.862 INFO:teuthology.orchestra.run.smithi005.stdout:(149/152): librados2-19.0.0-1628.g97ae6cc9.el9. 7.3 MB/s | 3.4 MB 00:00 2024-04-25T09:59:25.911 INFO:teuthology.orchestra.run.smithi005.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 1.3 MB 00:00 2024-04-25T09:59:25.987 INFO:teuthology.orchestra.run.smithi005.stdout:(151/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-25T09:59:26.354 INFO:teuthology.orchestra.run.smithi005.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-04-25T09:59:26.364 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:59:26.365 INFO:teuthology.orchestra.run.smithi005.stdout:Total 12 MB/s | 215 MB 00:17 2024-04-25T09:59:26.774 INFO:teuthology.orchestra.run.smithi080.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 263 kB/s | 548 kB 00:02 2024-04-25T09:59:27.440 INFO:teuthology.orchestra.run.smithi080.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 8.4 kB/s | 19 kB 00:02 2024-04-25T09:59:27.690 INFO:teuthology.orchestra.run.smithi080.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 104 kB/s | 26 kB 00:00 2024-04-25T09:59:27.857 INFO:teuthology.orchestra.run.smithi080.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 102 kB/s | 309 kB 00:03 2024-04-25T09:59:27.966 INFO:teuthology.orchestra.run.smithi080.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-04-25T09:59:29.373 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-04-25T09:59:29.631 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-04-25T09:59:29.632 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-04-25T09:59:30.082 INFO:teuthology.orchestra.run.smithi080.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 71 kB/s | 151 kB 00:02 2024-04-25T09:59:30.108 INFO:teuthology.orchestra.run.smithi080.stdout:(111/152): liboath-2.6.7-2.el9.x86_64.rpm 20 kB/s | 49 kB 00:02 2024-04-25T09:59:30.125 INFO:teuthology.orchestra.run.smithi080.stdout:(112/152): libunwind-1.6.2-1.el9.x86_64.rpm 30 kB/s | 67 kB 00:02 2024-04-25T09:59:32.208 INFO:teuthology.orchestra.run.smithi080.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 21 kB/s | 43 kB 00:02 2024-04-25T09:59:32.349 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-04-25T09:59:32.349 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-04-25T09:59:32.466 INFO:teuthology.orchestra.run.smithi080.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 125 kB/s | 32 kB 00:00 2024-04-25T09:59:32.542 INFO:teuthology.orchestra.run.smithi080.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 225 kB/s | 548 kB 00:02 2024-04-25T09:59:32.584 INFO:teuthology.orchestra.run.smithi080.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 335 kB/s | 838 kB 00:02 2024-04-25T09:59:32.826 INFO:teuthology.orchestra.run.smithi080.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 607 kB/s | 172 kB 00:00 2024-04-25T09:59:32.868 INFO:teuthology.orchestra.run.smithi080.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-04-25T09:59:33.075 INFO:teuthology.orchestra.run.smithi080.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 171 kB/s | 35 kB 00:00 2024-04-25T09:59:33.202 INFO:teuthology.orchestra.run.smithi080.stdout:(120/152): python3-certifi-2021.10.8-2.el9.noar 20 kB/s | 15 kB 00:00 2024-04-25T09:59:33.235 INFO:teuthology.orchestra.run.smithi080.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 742 kB/s | 303 kB 00:00 2024-04-25T09:59:33.260 INFO:teuthology.orchestra.run.smithi080.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-04-25T09:59:33.327 INFO:teuthology.orchestra.run.smithi080.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T09:59:33.361 INFO:teuthology.orchestra.run.smithi080.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-25T09:59:33.386 INFO:teuthology.orchestra.run.smithi080.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-04-25T09:59:33.454 INFO:teuthology.orchestra.run.smithi080.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-04-25T09:59:33.487 INFO:teuthology.orchestra.run.smithi080.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-04-25T09:59:33.512 INFO:teuthology.orchestra.run.smithi080.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-04-25T09:59:33.554 INFO:teuthology.orchestra.run.smithi080.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-25T09:59:33.755 INFO:teuthology.orchestra.run.smithi080.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 395 kB/s | 79 kB 00:00 2024-04-25T09:59:33.947 INFO:teuthology.orchestra.run.smithi080.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 302 kB/s | 58 kB 00:00 2024-04-25T09:59:34.063 INFO:teuthology.orchestra.run.smithi080.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.8 MB/s | 1.0 MB 00:00 2024-04-25T09:59:34.114 INFO:teuthology.orchestra.run.smithi080.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 332 kB/s | 16 kB 00:00 2024-04-25T09:59:34.214 INFO:teuthology.orchestra.run.smithi080.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-04-25T09:59:34.306 INFO:teuthology.orchestra.run.smithi080.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 310 kB/s | 28 kB 00:00 2024-04-25T09:59:34.373 INFO:teuthology.orchestra.run.smithi080.stdout:(136/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.7 MB/s | 2.0 MB 00:01 2024-04-25T09:59:34.398 INFO:teuthology.orchestra.run.smithi080.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 317 kB/s | 90 kB 00:00 2024-04-25T09:59:37.415 INFO:teuthology.orchestra.run.smithi080.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 20 kB/s | 59 kB 00:03 2024-04-25T09:59:37.431 INFO:teuthology.orchestra.run.smithi080.stdout:(139/152): python3-repoze-lru-0.7-16.el9.noarch 9.8 kB/s | 31 kB 00:03 2024-04-25T09:59:37.465 INFO:teuthology.orchestra.run.smithi080.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 61 kB/s | 188 kB 00:03 2024-04-25T09:59:37.607 INFO:teuthology.orchestra.run.smithi080.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-25T09:59:37.666 INFO:teuthology.orchestra.run.smithi080.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 219 kB/s | 51 kB 00:00 2024-04-25T09:59:37.691 INFO:teuthology.orchestra.run.smithi080.stdout:(143/152): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-25T09:59:37.808 INFO:teuthology.orchestra.run.smithi080.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 448 kB/s | 90 kB 00:00 2024-04-25T09:59:37.833 INFO:teuthology.orchestra.run.smithi080.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-04-25T09:59:37.992 INFO:teuthology.orchestra.run.smithi080.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.3 MB/s | 427 kB 00:00 2024-04-25T09:59:38.475 INFO:teuthology.orchestra.run.smithi080.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 287 kB/s | 192 kB 00:00 2024-04-25T09:59:38.551 INFO:teuthology.orchestra.run.smithi080.stdout:(148/152): librados2-19.0.0-1628.g97ae6cc9.el9. 6.0 MB/s | 3.4 MB 00:00 2024-04-25T09:59:38.743 INFO:teuthology.orchestra.run.smithi080.stdout:(149/152): thrift-0.15.0-2.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-04-25T09:59:38.809 INFO:teuthology.orchestra.run.smithi080.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-25T09:59:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-04-25T09:59:38.951 INFO:teuthology.orchestra.run.smithi080.stdout:(151/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-04-25T09:59:38.985 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:59:39.072 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:59:39.101 INFO:teuthology.orchestra.run.smithi080.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-04-25T09:59:39.108 INFO:teuthology.orchestra.run.smithi080.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:59:39.109 INFO:teuthology.orchestra.run.smithi080.stdout:Total 7.0 MB/s | 215 MB 00:30 2024-04-25T09:59:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:59:39.401 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:59:39.447 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:59:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:59:40.038 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:59:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:40.379 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:40.411 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:40.547 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:40.621 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:59:40.995 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:59:41.059 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:59:41.083 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:41.251 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:41.276 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:41.439 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:41.468 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:41.532 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:41.912 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:41.963 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:41.976 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction check 2024-04-25T09:59:42.119 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:59:42.218 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction check succeeded. 2024-04-25T09:59:42.218 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction test 2024-04-25T09:59:42.240 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:59:42.287 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:59:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:59:42.438 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:59:42.490 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:59:42.555 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:59:42.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:59:42.669 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:59:42.792 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:59:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:59:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:59:42.944 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:59:43.083 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:59:43.124 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:59:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:59:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:59:43.327 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:59:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:59:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:59:43.689 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:59:43.754 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:59:44.483 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:59:44.693 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:59:44.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:59:44.791 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:59:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:59:44.862 INFO:teuthology.orchestra.run.smithi080.stdout:Transaction test succeeded. 2024-04-25T09:59:44.863 INFO:teuthology.orchestra.run.smithi080.stdout:Running transaction 2024-04-25T09:59:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:59:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:59:45.308 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:59:45.391 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:59:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:59:45.564 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:59:45.605 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:59:45.675 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:59:45.720 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:59:45.774 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:59:45.841 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:59:46.054 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:59:46.109 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:59:46.146 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:59:46.204 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:59:46.372 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:59:46.412 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:59:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:59:46.494 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:59:46.527 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:59:46.734 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:59:46.921 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:59:46.966 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:59:47.008 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:59:47.081 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:59:47.151 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:59:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:59:47.254 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:59:47.303 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:59:47.350 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:59:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:59:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:59:47.628 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:59:47.702 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:59:47.750 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:59:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:59:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:59:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:59:49.069 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:59:49.288 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:59:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:59:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:59:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:59:49.746 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:59:49.782 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:59:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:59:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:59:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:59:51.335 INFO:teuthology.orchestra.run.smithi080.stdout: Preparing : 1/1 2024-04-25T09:59:51.440 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:59:51.545 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:59:51.668 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:59:51.729 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:59:51.768 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:59:51.921 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:59:52.375 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:59:52.482 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:52.602 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:59:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:52.692 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:52.704 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:59:52.743 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:59:52.790 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:59:52.821 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:52.857 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:59:53.184 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:59:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:59:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:59:53.245 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:59:53.270 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:53.365 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:59:53.399 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:59:53.428 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:53.463 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:59:53.549 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:53.581 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:53.643 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:53.964 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:59:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:59:54.161 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:59:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:59:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:59:54.282 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:59:54.332 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:59:54.377 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:59:54.470 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:59:54.526 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:59:54.589 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:59:54.632 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:59:54.663 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:59:54.785 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:59:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:59:54.901 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:59:54.965 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:59:55.074 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:59:55.118 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:59:55.164 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:59:55.200 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:59:55.347 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:59:55.413 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:59:55.522 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:59:55.703 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:59:55.773 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:59:56.458 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:59:56.569 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:59:56.615 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:59:56.660 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:59:56.702 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:59:56.796 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:59:56.896 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:59:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:59:57.080 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:59:57.165 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:59:57.251 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:59:57.349 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:59:57.385 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:59:57.421 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:59:57.489 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:59:57.548 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:59:57.610 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:59:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:59:57.668 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:59:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:59:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:59:57.855 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:59:57.855 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:59:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T09:59:57.861 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:59:57.897 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:59:57.937 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:59:57.974 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:59:58.024 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:59:58.162 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:59:58.217 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:59:58.313 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:59:58.359 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:59:58.392 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:59:58.584 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:59:58.796 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:59:58.847 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:59:58.885 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:59:58.931 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:59:58.990 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:59:59.075 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:59:59.117 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:59:59.171 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:59:59.221 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:59:59.297 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:59:59.345 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:59:59.487 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:59:59.548 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:59:59.592 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:59:59.661 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:59:59.808 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:59:59.881 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:00:00.832 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:00:00.920 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:00:00.956 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:00:01.087 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:00:01.158 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:00:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:00:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:00:01.553 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:00:01.588 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:00:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:00:02.512 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:00:02.650 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:00:04.206 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:00:04.405 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:00:04.625 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:00:04.702 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:00:05.090 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi080.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:00:05.262 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:00:05.298 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T10:00:05.358 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:00:05.910 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:00:05.943 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:00:06.017 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:00:06.053 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T10:00:08.736 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:00:08.801 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:00:09.460 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:00:09.536 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:00:09.700 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:00:09.701 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:00:09.701 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:00:09.701 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:00:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:00:11.505 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-04-25T10:00:11.505 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-04-25T10:00:11.506 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-04-25T10:00:11.506 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-04-25T10:00:11.506 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-04-25T10:00:11.506 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-04-25T10:00:11.506 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-04-25T10:00:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:11.761 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:11.812 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:11.812 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:11.812 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:00:11.812 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:13.829 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:13.900 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:13.900 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:13.900 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:00:13.901 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:13.901 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:13.901 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:14.430 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:14.501 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:14.501 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:14.501 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:00:14.502 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:14.502 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:14.502 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:14.657 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:00:14.725 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:00:14.810 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:00:15.095 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:00:15.153 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:00:15.254 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:00:15.374 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:15.555 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:15.622 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:00:15.690 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:00:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:00:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:16.704 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:00:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:00:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:18.354 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:18.389 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:18.543 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:18.737 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:00:18.774 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:18.845 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:18.845 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:18.845 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:00:18.845 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:18.845 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:18.845 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:18.925 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:18.966 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:19.119 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:00:19.199 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:00:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:00:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:20.332 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:20.332 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:20.332 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:00:20.333 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:20.333 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:20.333 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:20.386 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:20.479 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:20.479 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:20.479 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:00:20.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:21.119 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:21.120 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:21.120 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:00:21.120 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:21.120 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:21.120 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:22.977 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:00:22.977 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /sys 2024-04-25T10:00:22.977 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /proc 2024-04-25T10:00:22.977 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /mnt 2024-04-25T10:00:22.977 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /var/tmp 2024-04-25T10:00:22.978 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /home 2024-04-25T10:00:22.978 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /root 2024-04-25T10:00:22.978 INFO:teuthology.orchestra.run.smithi080.stdout:skipping the directory /tmp 2024-04-25T10:00:22.978 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:23.203 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:00:23.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:24.889 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:24.971 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:24.972 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:24.972 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:00:24.972 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:24.972 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:24.972 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:25.206 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:25.284 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:25.284 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:25.284 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:00:25.284 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:25.285 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:25.285 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:25.806 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:25.886 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:25.886 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:25.886 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:00:25.886 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:25.886 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:25.886 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:26.031 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:00:26.098 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:00:26.185 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:00:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:00:26.254 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:00:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:00:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:00:26.420 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:00:26.435 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:00:26.521 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:00:26.618 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:00:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:00:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:00:26.843 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:00:26.908 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:26.966 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:00:27.002 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:00:27.034 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:00:27.644 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:27.690 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:27.720 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:27.750 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:27.762 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:27.813 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:00:27.915 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:00:27.980 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:28.312 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:28.383 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:29.182 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:00:29.288 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:00:29.385 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:00:29.537 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:00:29.566 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:00:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:29.711 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:29.898 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:00:29.926 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:29.958 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:30.077 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:30.094 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:30.118 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:30.248 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:00:30.327 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:00:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:30.400 INFO:teuthology.orchestra.run.smithi005.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T10:00:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:31.299 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:00:31.363 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:31.431 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:31.431 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:31.431 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:00:31.431 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:31.432 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:31.432 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:31.497 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:31.568 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:31.568 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:31.568 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:00:31.568 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:31.631 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:31.732 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:31.733 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:31.733 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:00:31.733 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:31.733 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:31.733 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:32.123 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi080.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:32.193 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:32.193 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:35.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:00:35.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:00:35.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:00:35.648 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:00:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:00:35.668 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:00:35.669 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:00:35.670 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:00:35.671 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:00:35.672 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:00:35.673 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:00:35.674 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:00:35.675 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:35.676 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:00:37.081 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:00:37.139 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:00:37.220 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:00:37.266 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:00:37.312 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:00:37.647 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:00:37.722 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:00:37.754 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:00:38.233 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:00:38.234 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.235 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:00:38.236 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.237 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:00:38.238 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:00:38.239 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:00:38.240 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:00:38.241 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:00:38.243 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:00:38.244 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:00:38.245 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:00:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:00:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:00:38.247 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:38.247 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-04-25T10:00:38.672 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:38.718 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:38.772 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:39.036 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:40.112 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:00:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:00:40.297 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:00:40.502 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:00:40.549 INFO:teuthology.orchestra.run.smithi080.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:00:40.549 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:40.675 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:40.752 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:40.752 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:40.896 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:41.002 INFO:teuthology.orchestra.run.smithi080.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:41.315 INFO:teuthology.orchestra.run.smithi080.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:41.315 INFO:teuthology.orchestra.run.smithi080.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T10:00:41.315 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:46.496 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:00:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:00:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:00:46.499 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:00:46.499 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:00:46.499 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:00:46.499 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:00:46.499 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:00:46.499 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:00:46.500 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:00:46.501 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:00:46.502 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:00:46.503 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:00:46.504 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:00:46.505 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:00:46.506 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:00:46.507 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:00:46.508 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:46.509 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:00:48.965 INFO:teuthology.orchestra.run.smithi080.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout:Upgraded: 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout:Installed: 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:00:48.966 INFO:teuthology.orchestra.run.smithi080.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.970 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.971 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.971 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.971 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.971 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.971 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.971 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.972 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.972 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.972 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.972 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.972 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.972 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.973 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.973 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.973 INFO:teuthology.orchestra.run.smithi080.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.973 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.973 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.973 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:48.974 INFO:teuthology.orchestra.run.smithi080.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:00:48.974 INFO:teuthology.orchestra.run.smithi080.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:00:48.974 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:00:48.974 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:00:48.974 INFO:teuthology.orchestra.run.smithi080.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:00:48.974 INFO:teuthology.orchestra.run.smithi080.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:00:48.975 INFO:teuthology.orchestra.run.smithi080.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:00:48.975 INFO:teuthology.orchestra.run.smithi080.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:00:48.975 INFO:teuthology.orchestra.run.smithi080.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:00:48.975 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:00:48.975 INFO:teuthology.orchestra.run.smithi080.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:00:48.975 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.976 INFO:teuthology.orchestra.run.smithi080.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.976 INFO:teuthology.orchestra.run.smithi080.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.976 INFO:teuthology.orchestra.run.smithi080.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:00:48.976 INFO:teuthology.orchestra.run.smithi080.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:00:48.976 INFO:teuthology.orchestra.run.smithi080.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:00:48.976 INFO:teuthology.orchestra.run.smithi080.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:00:48.977 INFO:teuthology.orchestra.run.smithi080.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:00:48.977 INFO:teuthology.orchestra.run.smithi080.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.977 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.977 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:00:48.977 INFO:teuthology.orchestra.run.smithi080.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.978 INFO:teuthology.orchestra.run.smithi080.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:00:48.978 INFO:teuthology.orchestra.run.smithi080.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:00:48.978 INFO:teuthology.orchestra.run.smithi080.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:00:48.978 INFO:teuthology.orchestra.run.smithi080.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:00:48.978 INFO:teuthology.orchestra.run.smithi080.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:00:48.978 INFO:teuthology.orchestra.run.smithi080.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:00:48.979 INFO:teuthology.orchestra.run.smithi080.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:00:48.980 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:00:48.980 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:00:48.980 INFO:teuthology.orchestra.run.smithi080.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:00:48.981 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:00:48.982 INFO:teuthology.orchestra.run.smithi080.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:00:48.982 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:00:48.982 INFO:teuthology.orchestra.run.smithi080.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:00:48.982 INFO:teuthology.orchestra.run.smithi080.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:00:48.982 INFO:teuthology.orchestra.run.smithi080.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:00:48.982 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:00:48.983 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.983 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.983 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.983 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:00:48.983 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:00:48.984 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:00:48.984 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:00:48.984 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:00:48.984 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:00:48.984 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:00:48.984 INFO:teuthology.orchestra.run.smithi080.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:00:48.985 INFO:teuthology.orchestra.run.smithi080.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:00:48.985 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:00:48.985 INFO:teuthology.orchestra.run.smithi080.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:00:48.985 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:00:48.985 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:00:48.985 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:00:48.986 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:00:48.986 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:00:48.986 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:00:48.986 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:00:48.986 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:00:48.987 INFO:teuthology.orchestra.run.smithi080.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:00:48.987 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:00:48.987 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:00:48.987 INFO:teuthology.orchestra.run.smithi080.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:00:48.987 INFO:teuthology.orchestra.run.smithi080.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:00:48.987 INFO:teuthology.orchestra.run.smithi080.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:00:48.988 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:48.988 INFO:teuthology.orchestra.run.smithi080.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:48.988 INFO:teuthology.orchestra.run.smithi080.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:00:48.988 INFO:teuthology.orchestra.run.smithi080.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:00:48.988 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:00:48.988 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:00:48.989 INFO:teuthology.orchestra.run.smithi080.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:00:48.989 INFO:teuthology.orchestra.run.smithi080.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:00:48.989 INFO:teuthology.orchestra.run.smithi080.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:00:48.989 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:00:48.989 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:00:48.990 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:00:48.990 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:00:48.990 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:00:48.990 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:00:48.991 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.991 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.991 INFO:teuthology.orchestra.run.smithi080.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:00:48.991 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:00:48.992 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:00:48.992 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.992 INFO:teuthology.orchestra.run.smithi080.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:00:48.992 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:00:48.992 INFO:teuthology.orchestra.run.smithi080.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:00:48.992 INFO:teuthology.orchestra.run.smithi080.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:00:48.993 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:00:48.993 INFO:teuthology.orchestra.run.smithi080.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:00:48.993 INFO:teuthology.orchestra.run.smithi080.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:00:48.993 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:00:48.993 INFO:teuthology.orchestra.run.smithi080.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:00:48.993 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:00:48.994 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:00:48.994 INFO:teuthology.orchestra.run.smithi080.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:00:48.994 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:00:48.994 INFO:teuthology.orchestra.run.smithi080.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:00:48.994 INFO:teuthology.orchestra.run.smithi080.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:00:48.995 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.995 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.995 INFO:teuthology.orchestra.run.smithi080.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:48.995 INFO:teuthology.orchestra.run.smithi080.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:00:48.995 INFO:teuthology.orchestra.run.smithi080.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:00:48.995 INFO:teuthology.orchestra.run.smithi080.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:00:48.996 INFO:teuthology.orchestra.run.smithi080.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:00:48.996 INFO:teuthology.orchestra.run.smithi080.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:00:48.996 INFO:teuthology.orchestra.run.smithi080.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:00:48.997 INFO:teuthology.orchestra.run.smithi080.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:00:48.997 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:48.998 INFO:teuthology.orchestra.run.smithi080.stdout:Complete! 2024-04-25T10:00:49.590 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:49.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:00:49.758 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:00:49.833 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:49.834 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:49.834 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:00:49.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:00:50.004 DEBUG:teuthology.orchestra.run.smithi080:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:00:50.060 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:50.060 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:50.060 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:00:50.062 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:00:50.063 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:50.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:00:50.144 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:50.144 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:00:50.212 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:00:50.212 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:50.213 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:00:50.293 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:00:50.366 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:50.366 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:00:50.449 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:00:50.555 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:00:50.555 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:50.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:00:50.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:00:50.758 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:50.758 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:00:50.830 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:00:50.935 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:00:50.935 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:50.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:00:51.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:00:51.121 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:51.121 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:00:51.193 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:00:51.297 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:00:51.387 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:00:51.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2024-04-25T10:00:51.395 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /var/log/ceph 2024-04-25T10:00:51.464 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:00:51.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:00:51.469 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:00:51.529 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:00:51.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:00:51.536 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:00:51.608 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:00:51.608 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:00:51.608 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:00:51.608 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:51.609 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:00:51.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:51.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:51.931 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:51.931 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:00:51.969 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:52.148 DEBUG:teuthology.orchestra.run.smithi080:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:52.215 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:00:52.215 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:00:52.215 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673042', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXci1nadOkhLG29+OCxwETXBf3XCyRn6NxbkFi3aGGHLHapLhQ2tdn3aqH3zsjo6xMMQ+QVCM7BJJ3NxxMlKVU=', 'smithi080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMi0xE96PiAJfnBQBLaXTbGCLxk5vEBvrEhfWUzWFZrTje+F6V6vaPoRu5+VG4n0csAE6hJQEQu8H1AOvtp1TRU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:00:52.216 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:00:52.223 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:00:52.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:00:52.283 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:00:52.367 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:52.367 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:00:52.422 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:00:52.422 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-25 10:00:30.917764040 +0000 2024-04-25T10:00:52.476 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-25 09:58:12.565677976 +0000 2024-04-25T10:00:52.477 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-25 09:58:12.565677976 +0000 2024-04-25T10:00:52.477 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-25 09:58:12.565677976 +0000 2024-04-25T10:00:52.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:00:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-25T10:00:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-25T10:00:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000270403 s, 1.9 MB/s 2024-04-25T10:00:52.564 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:00:52.624 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-04-25T10:00:52.677 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:00:52.677 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:52.677 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T10:00:52.678 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:52.678 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:52.678 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-25 10:00:30.917764040 +0000 2024-04-25T10:00:52.678 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-25 09:58:13.195664968 +0000 2024-04-25T10:00:52.678 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-25 09:58:13.195664968 +0000 2024-04-25T10:00:52.678 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-25 09:58:13.195664968 +0000 2024-04-25T10:00:52.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:00:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-25T10:00:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-25T10:00:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000297163 s, 1.7 MB/s 2024-04-25T10:00:52.764 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:00:52.825 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-25 10:00:30.919763998 +0000 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-25 09:58:13.786652765 +0000 2024-04-25T10:00:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-25 09:58:13.786652765 +0000 2024-04-25T10:00:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-25 09:58:13.786652765 +0000 2024-04-25T10:00:52.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:00:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-25T10:00:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-25T10:00:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000211858 s, 2.4 MB/s 2024-04-25T10:00:52.951 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:00:53.003 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-04-25T10:00:53.050 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:00:53.050 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-04-25 10:00:30.919763998 +0000 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-04-25 09:58:14.441639241 +0000 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-04-25 09:58:14.441639241 +0000 2024-04-25T10:00:53.051 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-04-25 09:58:14.441639241 +0000 2024-04-25T10:00:53.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:00:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-04-25T10:00:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-04-25T10:00:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000267467 s, 1.9 MB/s 2024-04-25T10:00:53.137 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:00:53.198 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:53.198 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:53.198 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:00:53.252 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:00:53.252 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-04-25T10:00:53.299 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:00:53.299 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 10:00:41.742242305 +0000 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 09:58:12.461227917 +0000 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 09:58:12.461227917 +0000 2024-04-25T10:00:53.300 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-25 09:58:12.461227917 +0000 2024-04-25T10:00:53.301 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:00:53.401 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T10:00:53.401 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T10:00:53.401 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.00036506 s, 1.4 MB/s 2024-04-25T10:00:53.404 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:00:53.444 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-04-25T10:00:53.520 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:00:53.520 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 10:00:41.743242283 +0000 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 09:58:13.016215569 +0000 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 09:58:13.016215569 +0000 2024-04-25T10:00:53.521 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-25 09:58:13.016215569 +0000 2024-04-25T10:00:53.521 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:00:53.621 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T10:00:53.622 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T10:00:53.622 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000307621 s, 1.7 MB/s 2024-04-25T10:00:53.624 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:00:53.666 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 10:00:41.744242261 +0000 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 09:58:13.613202289 +0000 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 09:58:13.613202289 +0000 2024-04-25T10:00:53.745 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-25 09:58:13.613202289 +0000 2024-04-25T10:00:53.746 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:00:53.841 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T10:00:53.842 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T10:00:53.842 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000298494 s, 1.7 MB/s 2024-04-25T10:00:53.845 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:00:53.884 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 10:00:41.744242261 +0000 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 09:58:14.233188496 +0000 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 09:58:14.233188496 +0000 2024-04-25T10:00:53.963 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: 2024-04-25 09:58:14.233188496 +0000 2024-04-25T10:00:53.963 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:00:54.060 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T10:00:54.060 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T10:00:54.060 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000284176 s, 1.8 MB/s 2024-04-25T10:00:54.063 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:00:54.102 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:54.102 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:00:54.102 INFO:tasks.ceph:Generating config... 2024-04-25T10:00:54.103 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:00:54.103 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:00:54.103 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:00:54.103 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:00:54.103 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:00:54.103 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:00:54.104 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:00:54.104 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:00:54.105 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:00:54.105 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:00:54.105 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:00:54.106 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:00:54.106 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:00:54.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:00:54.240 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:00:54.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:00:54.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:54.461 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.b', '172.21.15.80'), ('mon.c', '[v2:172.21.15.80:3301,v1:172.21.15.80:6790]')] 2024-04-25T10:00:54.462 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:00:54.463 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:54.464 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:00:54.519 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --add b 172.21.15.80 --addv c '[v2:172.21.15.80:3301,v1:172.21.15.80:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-04-25T10:00:54.677 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2024-04-25T10:00:54.673948+0000 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-04-25T10:00:54.673948+0000 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 17 (quincy) 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-04-25T10:00:54.678 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] mon.c 2024-04-25T10:00:54.679 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:00:54.681 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:00:54.737 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8b6dc2a7-3300-467c-ae7a-376f479db0dd... 2024-04-25T10:00:54.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:00:54.743 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:00:54.849 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.850 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:00:54.851 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.852 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on skipped message = False 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.853 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T10:00:54.854 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T10:00:54.855 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:54.856 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:00:54.857 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T10:00:54.858 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T10:00:54.859 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = True 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = True 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:00:54.860 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:00:54.878 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-25T10:00:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-04-25T10:00:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-04-25T10:00:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-04-25T10:00:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:00:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:00:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-04-25T10:00:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:00:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-04-25T10:00:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:00:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-04-25T10:00:54.888 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:00:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:00:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:00:54.913 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:00:54.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:00:55.053 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:00:55.053 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:55.053 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:00:55.103 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:55.103 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:00:55.152 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:00:55.152 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:55.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:00:55.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:55.282 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:00:55.283 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:55.381 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:00:55.381 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:55.382 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:00:55.382 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:55.481 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:00:55.481 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:55.563 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:00:55.563 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:00:55.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:00:55.742 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:00:55.753 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:00:55.909 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:00:55.917 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:00:55.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:00:56.096 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:00:56.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:00:56.197 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:00:56.368 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:00:56.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:00:56.465 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:00:56.596 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:00:56.603 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:00:56.670 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:00:56.799 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:00:56.807 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:00:56.884 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:00:56.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:00:57.011 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:00:57.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:00:57.166 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:00:57.218 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:00:57.339 INFO:teuthology.orchestra.run.smithi080.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:00:57.381 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:00:57.496 INFO:teuthology.orchestra.run.smithi080.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:00:57.534 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:00:57.534 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:00:57.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:57.629 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:57.630 INFO:tasks.ceph:role: osd.0 2024-04-25T10:00:57.630 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:00:57.630 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:00:57.715 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:57.715 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:57.715 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:00:57.715 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:57.715 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:57.750 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-04-25T10:00:57.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-04-25T10:00:57.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:57.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:00:57.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:00:58.006 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:58.006 INFO:tasks.ceph:role: osd.1 2024-04-25T10:00:58.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:00:58.006 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:00:58.090 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:58.091 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:58.124 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-04-25T10:00:58.129 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-04-25T10:00:58.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:00:58.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:00:58.319 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:00:58.399 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:58.399 INFO:tasks.ceph:role: osd.2 2024-04-25T10:00:58.399 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:00:58.399 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:58.481 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:00:58.485 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:58.485 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:58.515 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-04-25T10:00:58.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-04-25T10:00:58.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:00:58.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:00:58.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:00:58.803 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:58.803 INFO:tasks.ceph:role: osd.3 2024-04-25T10:00:58.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:00:58.803 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:00:58.887 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:00:58.888 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:00:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:00:58.888 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:00:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-04-25T10:00:58.928 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-04-25T10:00:58.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:00:59.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:00:59.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:59.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:00:59.250+0000 7fa9a252c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:00:59.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:00:59.251+0000 7fa9a252c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:00:59.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:00:59.251+0000 7fa9a252c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:00:59.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:00:59.251+0000 7fa9a252c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:00:59.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:00:59.251+0000 7fa9a252c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:00:59.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:00:59.254+0000 7fa9a252c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:01:02.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:02.327 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:02.457+0000 7f252ebad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:01:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:02.457+0000 7f252ebad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:01:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:02.458+0000 7f252ebad640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:02.458+0000 7f252ebad640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:02.458+0000 7f252ebad640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:02.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:02.460+0000 7f252ebad640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:01:05.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:05.554 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:05.687+0000 7fadde4ff640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:01:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:05.688+0000 7fadde4ff640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:01:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:05.688+0000 7fadde4ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:05.688+0000 7fadde4ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:05.688+0000 7fadde4ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:05.690+0000 7fadde4ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:01:08.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:08.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:08.872+0000 7f0f2c7a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:01:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:08.873+0000 7f0f2c7a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:01:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:08.873+0000 7f0f2c7a5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:08.873+0000 7f0f2c7a5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:08.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:08.873+0000 7f0f2c7a5640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:08.875+0000 7f0f2c7a5640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:01:11.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:11.943 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:12.033 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:12.034 INFO:tasks.ceph:role: osd.4 2024-04-25T10:01:12.034 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:01:12.034 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:12.123 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:12.534 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T10:01:12.541 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T10:01:12.541 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:12.703 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:12.800 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:12.873 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:12.873 INFO:tasks.ceph:role: osd.5 2024-04-25T10:01:12.873 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:01:12.873 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:01:12.951 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:12.951 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:12.952 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:12.952 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:12.952 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:12.952 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:12.952 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:12.953 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:12.953 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:12.953 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:13.306 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T10:01:13.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T10:01:13.311 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:13.413 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:13.486 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:13.548 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:13.548 INFO:tasks.ceph:role: osd.6 2024-04-25T10:01:13.548 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:01:13.548 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:13.625 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:14.002 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T10:01:14.007 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T10:01:14.007 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:14.114 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:14.192 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:14.264 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:14.264 INFO:tasks.ceph:role: osd.7 2024-04-25T10:01:14.264 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:01:14.265 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:01:14.347 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:14.347 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:14.347 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:14.347 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:14.348 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:14.348 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:14.348 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:14.348 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:14.348 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:14.348 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:14.764 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T10:01:14.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T10:01:14.772 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:14.923 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:15.008 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:15.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:15.140+0000 7ff028733640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:01:15.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:15.141+0000 7ff028733640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:01:15.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:15.141+0000 7ff028733640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:15.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:15.141+0000 7ff028733640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:15.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:15.141+0000 7ff028733640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:15.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:15.144+0000 7ff028733640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:01:18.369 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:18.453 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:18.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:18.568+0000 7f6980a1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:01:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:18.569+0000 7f6980a1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:01:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:18.569+0000 7f6980a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:18.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:18.569+0000 7f6980a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:18.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:18.569+0000 7f6980a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:18.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:18.571+0000 7f6980a1a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:01:21.529 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:21.614 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:21.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:21.732+0000 7fd2fe8b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:01:21.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:21.733+0000 7fd2fe8b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:01:21.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:21.734+0000 7fd2fe8b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:21.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:21.734+0000 7fd2fe8b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:21.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:21.734+0000 7fd2fe8b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:21.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:21.736+0000 7fd2fe8b5640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:01:22.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:22.225 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:24.705 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:24.793 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:24.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:24.919+0000 7f5ca7624640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:01:24.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:24.920+0000 7f5ca7624640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:01:24.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:24.920+0000 7f5ca7624640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:24.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:24.920+0000 7f5ca7624640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:24.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:24.920+0000 7f5ca7624640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:24.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:24.922+0000 7f5ca7624640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:01:27.890 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:27.975 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:01:27.975 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:27.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:01:28.072 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:28.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:01:28.161 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:28.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:01:28.247 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:28.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:01:28.337 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:28.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:01:28.429 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:28.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:01:28.520 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:28.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:01:28.612 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:28.613 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:01:28.701 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:28.701 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:01:28.790 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:28.790 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:01:28.874 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:28.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:01:28.959 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:28.959 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:01:29.039 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:29.039 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:01:29.122 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:29.123 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:01:29.208 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:29.208 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:01:29.268 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:29.268 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:01:29.328 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:29.328 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:01:29.386 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:29.386 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:01:29.435 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:01:29.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:29.442 DEBUG:teuthology.orchestra.run.smithi080:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:29.512 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.y] 2024-04-25T10:01:29.512 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg== 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ== 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg== 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.0] 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA== 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.1] 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g== 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.2] 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g== 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.3] 2024-04-25T10:01:29.513 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.x] 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.4] 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.5] 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.6] 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ== 2024-04-25T10:01:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.7] 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg== 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout:[client.2] 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw== 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout:[client.1] 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg== 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout:[client.3] 2024-04-25T10:01:29.515 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ== 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg== 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ== 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg== 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA== 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2024-04-25T10:01:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA== 2024-04-25T10:01:29.521 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ== 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg== 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout:[client.2] 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw== 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout:[client.1] 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg== 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout:[client.3] 2024-04-25T10:01:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ== 2024-04-25T10:01:29.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:29.572 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:29.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:29.713 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:29.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:29.855 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:29.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:29.999 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.142 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.281 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.416 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:30.554 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:30.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:30.689 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:30.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:30.832 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:30.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:30.973 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.113 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.253 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.396 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:31.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.537 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.678 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.814 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.820 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:31.964 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:32.103 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:01:32.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:32.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:32.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:32.436 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:32.528 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:32.669 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:32.748 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:32.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:32.983 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:33.055 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:33.064 DEBUG:teuthology.orchestra.run.smithi080:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:33.116 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:01:33.116 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:01:33.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:01:33.121 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:01:33.122 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:01:33.122 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:01:33.127 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:01:33.127 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:01:33.127 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:01:33.131 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:01:33.132 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:01:33.132 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:01:33.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:01:33.138 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:01:33.139 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:01:33.139 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:01:33.143 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:01:33.143 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:01:33.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:01:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.318+0000 7f4b3c800640 1 Processor -- start 2024-04-25T10:01:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.319+0000 7f4b3c800640 1 -- start start 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b3c800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 0x7f4b340b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b3c800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b340b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b3c800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b340d1a40 con 0x7f4b340b6f20 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b3c800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b340d1d20 con 0x7f4b3415deb0 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b3c800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340d2000 con 0x7f4b340ce030 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b39400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b340b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b39400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b340b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:37886/0 (socket says 172.21.15.5:37886) 2024-04-25T10:01:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b39400640 1 -- 172.21.15.5:0/1217885011 learned_addr learned my addr 172.21.15.5:0/1217885011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.320+0000 7f4b39e00640 1 --2- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 0x7f4b340b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39400640 1 -- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 msgr2=0x7f4b340b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39400640 1 -- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 msgr2=0x7f4b340b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39400640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b340b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39400640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b340b69e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39e00640 1 -- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 msgr2=0x7f4b340b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39e00640 1 -- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 msgr2=0x7f4b340b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39e00640 1 --2- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 0x7f4b340b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.321+0000 7f4b39e00640 1 --2- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 0x7f4b340b2c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.329+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674105435 0 0) 0x7f4b340d1a40 con 0x7f4b340b6f20 2024-04-25T10:01:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.329+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b1c003520 con 0x7f4b340b6f20 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3514616543 0 0) 0x7f4b1c003520 con 0x7f4b340b6f20 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 msgr2=0x7f4b340b69e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b340b69e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 msgr2=0x7f4b340b2c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 --2- 172.21.15.5:0/1217885011 >> v2:172.21.15.5:3300/0 conn(0x7f4b3415deb0 0x7f4b340b2c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b340d22e0 con 0x7f4b340b6f20 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.330+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4b300019f0 con 0x7f4b340b6f20 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 >> v1:172.21.15.5:6789/0 conn(0x7f4b340b6f20 legacy=0x7f4b340d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c0039b0 0x7f4b1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340d1d20 con 0x7f4b1c004d40 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340d2000 con 0x7f4b1c0039b0 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c00baf0 con 0x7f4b340ce030 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b3a800640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b39400640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b3a800640 1 -- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 msgr2=0x7f4b1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b3a800640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b1c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b3a800640 1 -- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c0039b0 msgr2=0x7f4b1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b3a800640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c0039b0 0x7f4b1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b3a800640 1 -- 172.21.15.5:0/1217885011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c003520 con 0x7f4b1c004d40 2024-04-25T10:01:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.331+0000 7f4b39400640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340ce030 0x7f4b1c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.332+0000 7f4b3a800640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b1c0051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000c3b0 tx=0x7f4b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453b4ed2dc7bcdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.333+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b30033070 con 0x7f4b1c004d40 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.333+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3002f680 con 0x7f4b1c004d40 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.333+0000 7f4b38a00640 1 -- 172.21.15.5:0/1217885011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3000c0e0 con 0x7f4b1c004d40 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.333+0000 7f4b3c800640 1 -- 172.21.15.5:0/1217885011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 msgr2=0x7f4b1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.333+0000 7f4b3c800640 1 --2- 172.21.15.5:0/1217885011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b1c0051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000c3b0 tx=0x7f4b1c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.334+0000 7f4b3a800640 1 -- 172.21.15.5:0/1217885011 reap_dead start 2024-04-25T10:01:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.334+0000 7f4b3c800640 1 -- 172.21.15.5:0/1217885011 shutdown_connections 2024-04-25T10:01:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.334+0000 7f4b3c800640 1 -- 172.21.15.5:0/1217885011 >> 172.21.15.5:0/1217885011 conn(0x7f4b340726b0 msgr2=0x7f4b34072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.334+0000 7f4b3c800640 1 -- 172.21.15.5:0/1217885011 shutdown_connections 2024-04-25T10:01:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.334+0000 7f4b3c800640 1 -- 172.21.15.5:0/1217885011 wait complete. 2024-04-25T10:01:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.336+0000 7f4b3c800640 1 Processor -- start 2024-04-25T10:01:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.336+0000 7f4b3c800640 1 -- start start 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b3c800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b340b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b3c800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340aa280 0x7f4b340b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b3c800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b341f36b0 0x7f4b341f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b3c800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340d3e70 con 0x7f4b1c004d40 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b3c800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340ce490 con 0x7f4b341f36b0 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b3c800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b340d3ff0 con 0x7f4b340aa280 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b39e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b340b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b39e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b340b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55532/0 (socket says 172.21.15.5:55532) 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b39e00640 1 -- 172.21.15.5:0/3959786621 learned_addr learned my addr 172.21.15.5:0/3959786621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.337+0000 7f4b39400640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340aa280 0x7f4b340b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.338+0000 7f4b3a800640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b341f36b0 0x7f4b341f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.338+0000 7f4b39e00640 1 -- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340aa280 msgr2=0x7f4b340b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.338+0000 7f4b39e00640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340aa280 0x7f4b340b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.338+0000 7f4b39e00640 1 -- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b341f36b0 msgr2=0x7f4b341f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.338+0000 7f4b39e00640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b341f36b0 0x7f4b341f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.338+0000 7f4b39e00640 1 -- 172.21.15.5:0/3959786621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b340b3c50 con 0x7f4b1c004d40 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b39e00640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b340b4cf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400d600 tx=0x7f4b2400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240125f0 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b24019070 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b340b3f30 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240129f0 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b340b44f0 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4b24012b90 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.339+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4b24014570 con 0x7f4b1c004d40 2024-04-25T10:01:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.340+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34068f60 con 0x7f4b1c004d40 2024-04-25T10:01:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.344+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4b240147c0 con 0x7f4b1c004d40 2024-04-25T10:01:33.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.419+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4b340b8170 con 0x7f4b1c004d40 2024-04-25T10:01:33.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.422+0000 7f4b2e000640 1 -- 172.21.15.5:0/3959786621 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4b240149a0 con 0x7f4b1c004d40 2024-04-25T10:01:33.426 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.423+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 msgr2=0x7f4b340b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.423+0000 7f4b3c800640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b340b4cf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4b2400d600 tx=0x7f4b2400dac0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 shutdown_connections 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b341f36b0 0x7f4b341f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b340aa280 0x7f4b340b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 --2- 172.21.15.5:0/3959786621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b1c004d40 0x7f4b340b4cf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 >> 172.21.15.5:0/3959786621 conn(0x7f4b340726b0 msgr2=0x7f4b340b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 shutdown_connections 2024-04-25T10:01:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:33.424+0000 7f4b3c800640 1 -- 172.21.15.5:0/3959786621 wait complete. 2024-04-25T10:01:33.452 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:01:33.453 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:01:33.453 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:33.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:01:33.505 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:33.501+0000 7f3074123040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:33.525 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:33.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:01:33.642 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:33.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:01:33.669 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:33.665+0000 7f1b86f23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:33.717 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:33.713+0000 7f3074123040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:33.760 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:01:33.761 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:01:33.787 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:33.782+0000 7f3074123040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:33.857 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:33.857 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:01:33.888 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:33.884+0000 7f1b86f23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:33.935 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:33.935 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:01:33.955 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:33.952+0000 7f1b86f23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:34.032 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:34.033 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:01:34.100 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:34.096+0000 7f3074123040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:34.153 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:01:34.153 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:01:34.157 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:34.154+0000 7f3074123040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:34.247 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:34.243+0000 7f1b86f23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:34.267 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 17e3e259-06e4-4bec-b707-73dd90091223 0 2024-04-25T10:01:34.303 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:34.299+0000 7f1b86f23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:34.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.488+0000 7f89d6600640 1 Processor -- start 2024-04-25T10:01:34.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.488+0000 7f89d6600640 1 -- start start 2024-04-25T10:01:34.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.489+0000 7f89d6600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89d00b45c0 0x7f89d0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.489+0000 7f89d6600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89d0161240 0x7f89d01616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.489+0000 7f89d6600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d015ffa0 con 0x7f89d015ec80 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.489+0000 7f89d6600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d00b4a30 con 0x7f89d0161240 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.489+0000 7f89d6600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d0160240 con 0x7f89d00b45c0 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.489+0000 7f89cfe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89d00b45c0 0x7f89d0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89cfe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89d00b45c0 0x7f89d0160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48182/0 (socket says 172.21.15.80:48182) 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89cfe00640 1 -- 172.21.15.80:0/2698323331 learned_addr learned my addr 172.21.15.80:0/2698323331 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89d5c00640 1 --2- 172.21.15.80:0/2698323331 >> v2:172.21.15.5:3300/0 conn(0x7f89d0161240 0x7f89d01616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89cfe00640 1 -- 172.21.15.80:0/2698323331 >> v2:172.21.15.5:3300/0 conn(0x7f89d0161240 msgr2=0x7f89d01616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89cfe00640 1 --2- 172.21.15.80:0/2698323331 >> v2:172.21.15.5:3300/0 conn(0x7f89d0161240 0x7f89d01616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89cfe00640 1 -- 172.21.15.80:0/2698323331 >> v1:172.21.15.5:6789/0 conn(0x7f89d015ec80 legacy=0x7f89d00d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.490+0000 7f89cfe00640 1 -- 172.21.15.80:0/2698323331 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0160520 con 0x7f89d00b45c0 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.491+0000 7f89cfe00640 1 --2- 172.21.15.80:0/2698323331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d0160d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f89d00d1500 tx=0x7f89bc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6688818d7e084e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.491+0000 7f89cf400640 1 -- 172.21.15.80:0/2698323331 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc004030 con 0x7f89d00b45c0 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.491+0000 7f89cf400640 1 -- 172.21.15.80:0/2698323331 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc0041d0 con 0x7f89d00b45c0 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.491+0000 7f89cf400640 1 -- 172.21.15.80:0/2698323331 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc003610 con 0x7f89d00b45c0 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 -- 172.21.15.80:0/2698323331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 msgr2=0x7f89d0160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 --2- 172.21.15.80:0/2698323331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d0160d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f89d00d1500 tx=0x7f89bc02a210 comp rx=0 tx=0).stop 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 -- 172.21.15.80:0/2698323331 shutdown_connections 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 --2- 172.21.15.80:0/2698323331 >> v2:172.21.15.5:3300/0 conn(0x7f89d0161240 0x7f89d01616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 --2- 172.21.15.80:0/2698323331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d0160d00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 -- 172.21.15.80:0/2698323331 >> 172.21.15.80:0/2698323331 conn(0x7f89d008af80 msgr2=0x7f89d00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 -- 172.21.15.80:0/2698323331 shutdown_connections 2024-04-25T10:01:34.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.492+0000 7f89d6600640 1 -- 172.21.15.80:0/2698323331 wait complete. 2024-04-25T10:01:34.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.494+0000 7f89d6600640 1 Processor -- start 2024-04-25T10:01:34.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.494+0000 7f89d6600640 1 -- start start 2024-04-25T10:01:34.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.495+0000 7f89d6600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d00b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.495+0000 7f89d6600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d015ec80 0x7f89d006d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.495+0000 7f89d6600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 0x7f89d014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.495+0000 7f89d6600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d00b4c50 con 0x7f89d015ec80 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.495+0000 7f89d6600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d015f060 con 0x7f89d00b45c0 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.495+0000 7f89d6600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d0161c30 con 0x7f89d0161240 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d00b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 0x7f89d014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 0x7f89d014b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47486/0 (socket says 172.21.15.80:47486) 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d00b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48194/0 (socket says 172.21.15.80:48194) 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5c00640 1 -- 172.21.15.80:0/3377026396 learned_addr learned my addr 172.21.15.80:0/3377026396 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89cfe00640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d015ec80 0x7f89d006d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 -- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 msgr2=0x7f89d014b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 0x7f89d014b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 -- 172.21.15.80:0/3377026396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d015ec80 msgr2=0x7f89d006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d015ec80 0x7f89d006d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5200640 1 -- 172.21.15.80:0/3377026396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d00cd330 con 0x7f89d00b45c0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89d5c00640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 0x7f89d014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.496+0000 7f89cfe00640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d015ec80 0x7f89d006d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89d5200640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d00b7180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c360 tx=0x7f89c400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4011790 con 0x7f89d00b45c0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4017070 con 0x7f89d00b45c0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d00b66a0 con 0x7f89d00b45c0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4011b70 con 0x7f89d00b45c0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d00b76c0 con 0x7f89d00b45c0 2024-04-25T10:01:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.497+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f89c4011d10 con 0x7f89d00b45c0 2024-04-25T10:01:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.498+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f89c4012710 con 0x7f89d00b45c0 2024-04-25T10:01:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.498+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d0068f60 con 0x7f89d00b45c0 2024-04-25T10:01:34.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.501+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f89c4012910 con 0x7f89d00b45c0 2024-04-25T10:01:34.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.579+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "17e3e259-06e4-4bec-b707-73dd90091223", "id": 0} v 0) v1 -- 0x7f89d00d0e50 con 0x7f89d00b45c0 2024-04-25T10:01:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.584+0000 7f89ccc00640 1 -- 172.21.15.80:0/3377026396 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "17e3e259-06e4-4bec-b707-73dd90091223", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f89c4012b50 con 0x7f89d00b45c0 2024-04-25T10:01:34.588 INFO:teuthology.orchestra.run.smithi080.stdout:0 2024-04-25T10:01:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 msgr2=0x7f89d00b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d00b7180 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c360 tx=0x7f89c400c820 comp rx=0 tx=0).stop 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 shutdown_connections 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89d0161240 0x7f89d014b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d015ec80 0x7f89d006d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 --2- 172.21.15.80:0/3377026396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89d00b45c0 0x7f89d00b7180 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 >> 172.21.15.80:0/3377026396 conn(0x7f89d008af80 msgr2=0x7f89d00d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 shutdown_connections 2024-04-25T10:01:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.586+0000 7f89d6600640 1 -- 172.21.15.80:0/3377026396 wait complete. 2024-04-25T10:01:34.606 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 4dbef475-8322-4087-9182-f3b2785032d6 1 2024-04-25T10:01:34.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.777+0000 7f0253600640 1 Processor -- start 2024-04-25T10:01:34.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.777+0000 7f0253600640 1 -- start start 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.778+0000 7f0253600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f024c15deb0 0x7f024c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.778+0000 7f0253600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.778+0000 7f0253600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0d1a40 con 0x7f024c0b6f20 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.778+0000 7f0253600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024c0d1d20 con 0x7f024c15deb0 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.778+0000 7f0253600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d2000 con 0x7f024c0ce030 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.778+0000 7f024be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f024be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47502/0 (socket says 172.21.15.80:47502) 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f024be00640 1 -- 172.21.15.80:0/2769549587 learned_addr learned my addr 172.21.15.80:0/2769549587 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f0250c00640 1 --2- 172.21.15.80:0/2769549587 >> v2:172.21.15.5:3300/0 conn(0x7f024c15deb0 0x7f024c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f024be00640 1 -- 172.21.15.80:0/2769549587 >> v2:172.21.15.5:3300/0 conn(0x7f024c15deb0 msgr2=0x7f024c0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f024be00640 1 --2- 172.21.15.80:0/2769549587 >> v2:172.21.15.5:3300/0 conn(0x7f024c15deb0 0x7f024c0b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f024be00640 1 -- 172.21.15.80:0/2769549587 >> v1:172.21.15.5:6789/0 conn(0x7f024c0b6f20 legacy=0x7f024c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.779+0000 7f024be00640 1 -- 172.21.15.80:0/2769549587 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0d22e0 con 0x7f024c0ce030 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.780+0000 7f024be00640 1 --2- 172.21.15.80:0/2769549587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c0b69e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0238009870 tx=0x7f023802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67931aa6372a5878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f024b400640 1 -- 172.21.15.80:0/2769549587 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238037070 con 0x7f024c0ce030 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f024b400640 1 -- 172.21.15.80:0/2769549587 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238032440 con 0x7f024c0ce030 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f024b400640 1 -- 172.21.15.80:0/2769549587 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238032810 con 0x7f024c0ce030 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f0253600640 1 -- 172.21.15.80:0/2769549587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 msgr2=0x7f024c0b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f0253600640 1 --2- 172.21.15.80:0/2769549587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c0b69e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0238009870 tx=0x7f023802a210 comp rx=0 tx=0).stop 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f0253600640 1 -- 172.21.15.80:0/2769549587 shutdown_connections 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f0253600640 1 --2- 172.21.15.80:0/2769549587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c0b69e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f0253600640 1 --2- 172.21.15.80:0/2769549587 >> v2:172.21.15.5:3300/0 conn(0x7f024c15deb0 0x7f024c0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.781+0000 7f0253600640 1 -- 172.21.15.80:0/2769549587 >> 172.21.15.80:0/2769549587 conn(0x7f024c0726b0 msgr2=0x7f024c072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.782+0000 7f0253600640 1 -- 172.21.15.80:0/2769549587 shutdown_connections 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.782+0000 7f0253600640 1 -- 172.21.15.80:0/2769549587 wait complete. 2024-04-25T10:01:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.783+0000 7f0253600640 1 Processor -- start 2024-04-25T10:01:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.783+0000 7f0253600640 1 -- start start 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0253600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 0x7f024c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0253600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0253600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024c15deb0 0x7f024c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0253600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d3e70 con 0x7f024c15deb0 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0253600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0ce490 con 0x7f024c0b6f20 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0253600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d3ff0 con 0x7f024c0ce030 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.784+0000 7f0250c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 0x7f024c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f024be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 0x7f024c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48200/0 (socket says 172.21.15.80:48200) 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 -- 172.21.15.80:0/3765484252 learned_addr learned my addr 172.21.15.80:0/3765484252 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0251600640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024c15deb0 0x7f024c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 -- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 msgr2=0x7f024c147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 -- 172.21.15.80:0/3765484252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024c15deb0 msgr2=0x7f024c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024c15deb0 0x7f024c1594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 -- 172.21.15.80:0/3765484252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0caec0 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f024be00640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.785+0000 7f0250c00640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 0x7f024c0ca980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f024000d5f0 tx=0x7f024000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400125f0 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240019070 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400129d0 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c0b5b80 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c0b50a0 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0240012b70 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.786+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f02400148c0 con 0x7f024c0b6f20 2024-04-25T10:01:34.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.787+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c068f60 con 0x7f024c0b6f20 2024-04-25T10:01:34.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.790+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f024c0caec0 con 0x7f024c0b6f20 2024-04-25T10:01:34.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.869+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4dbef475-8322-4087-9182-f3b2785032d6", "id": 1} v 0) v1 -- 0x7f024c0b6520 con 0x7f024c0b6f20 2024-04-25T10:01:34.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.877+0000 7f0248c00640 1 -- 172.21.15.80:0/3765484252 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4dbef475-8322-4087-9182-f3b2785032d6", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0240014b90 con 0x7f024c0b6f20 2024-04-25T10:01:34.881 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T10:01:34.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 msgr2=0x7f024c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 0x7f024c0ca980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f024000d5f0 tx=0x7f024000dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 shutdown_connections 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024c15deb0 0x7f024c1594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f024c0ce030 0x7f024c147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 --2- 172.21.15.80:0/3765484252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f024c0b6f20 0x7f024c0ca980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 >> 172.21.15.80:0/3765484252 conn(0x7f024c0726b0 msgr2=0x7f024c0d3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 shutdown_connections 2024-04-25T10:01:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:34.879+0000 7f0253600640 1 -- 172.21.15.80:0/3765484252 wait complete. 2024-04-25T10:01:34.899 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 9f6c5afa-3005-44b9-bd05-14f532e166ea 2 2024-04-25T10:01:35.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.068+0000 7fba7a000640 1 Processor -- start 2024-04-25T10:01:35.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.068+0000 7fba7a000640 1 -- start start 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.069+0000 7fba7a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba7415db70 0x7fba740ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.069+0000 7fba7a000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba740cede0 0x7fba740d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.069+0000 7fba7a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740d21b0 0x7fba7406bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.069+0000 7fba7a000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba740cf250 con 0x7fba7415db70 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.069+0000 7fba7a000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba740b6ce0 con 0x7fba740cede0 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.069+0000 7fba7a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740d2f80 con 0x7fba740d21b0 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba740cede0 0x7fba740d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba740cede0 0x7fba740d1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48214/0 (socket says 172.21.15.80:48214) 2024-04-25T10:01:35.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 -- 172.21.15.80:0/2932500044 learned_addr learned my addr 172.21.15.80:0/2932500044 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba73e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740d21b0 0x7fba7406bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba73400640 1 --2- 172.21.15.80:0/2932500044 >> v2:172.21.15.5:3300/0 conn(0x7fba7415db70 0x7fba740ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 -- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740d21b0 msgr2=0x7fba7406bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 --2- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740d21b0 0x7fba7406bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 -- 172.21.15.80:0/2932500044 >> v2:172.21.15.5:3300/0 conn(0x7fba7415db70 msgr2=0x7fba740ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 --2- 172.21.15.80:0/2932500044 >> v2:172.21.15.5:3300/0 conn(0x7fba7415db70 0x7fba740ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba72a00640 1 -- 172.21.15.80:0/2932500044 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740d29c0 con 0x7fba740cede0 2024-04-25T10:01:35.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.070+0000 7fba73400640 1 --2- 172.21.15.80:0/2932500044 >> v2:172.21.15.5:3300/0 conn(0x7fba7415db70 0x7fba740ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:35.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.071+0000 7fba72a00640 1 --2- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba740cede0 0x7fba740d1c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fba740d3380 tx=0x7fba6002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65603058068cb1ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.072+0000 7fba72000640 1 -- 172.21.15.80:0/2932500044 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60037070 con 0x7fba740cede0 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.072+0000 7fba72000640 1 -- 172.21.15.80:0/2932500044 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba60032440 con 0x7fba740cede0 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.072+0000 7fba72000640 1 -- 172.21.15.80:0/2932500044 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60032810 con 0x7fba740cede0 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.072+0000 7fba7a000640 1 -- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba740cede0 msgr2=0x7fba740d1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.072+0000 7fba7a000640 1 --2- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba740cede0 0x7fba740d1c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fba740d3380 tx=0x7fba6002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 -- 172.21.15.80:0/2932500044 shutdown_connections 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 --2- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740d21b0 0x7fba7406bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 --2- 172.21.15.80:0/2932500044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba740cede0 0x7fba740d1c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 --2- 172.21.15.80:0/2932500044 >> v2:172.21.15.5:3300/0 conn(0x7fba7415db70 0x7fba740ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 -- 172.21.15.80:0/2932500044 >> 172.21.15.80:0/2932500044 conn(0x7fba740b5370 msgr2=0x7fba740b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 -- 172.21.15.80:0/2932500044 shutdown_connections 2024-04-25T10:01:35.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.073+0000 7fba7a000640 1 -- 172.21.15.80:0/2932500044 wait complete. 2024-04-25T10:01:35.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.075+0000 7fba7a000640 1 Processor -- start 2024-04-25T10:01:35.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.075+0000 7fba7a000640 1 -- start start 2024-04-25T10:01:35.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba7a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 0x7fba740c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba7a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba740d21b0 0x7fba74150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba7a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba7415db70 0x7fba740cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba7a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740b6f10 con 0x7fba740d21b0 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba7a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740b7280 con 0x7fba7415db70 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba7a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740d2620 con 0x7fba740cede0 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba73400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 0x7fba740c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba73e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba7415db70 0x7fba740cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba73400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 0x7fba740c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47540/0 (socket says 172.21.15.80:47540) 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.076+0000 7fba73400640 1 -- 172.21.15.80:0/3352539435 learned_addr learned my addr 172.21.15.80:0/3352539435 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba72a00640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba740d21b0 0x7fba74150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba73400640 1 -- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba7415db70 msgr2=0x7fba740cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba73400640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba7415db70 0x7fba740cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba73400640 1 -- 172.21.15.80:0/3352539435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba740d21b0 msgr2=0x7fba74150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba73400640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba740d21b0 0x7fba74150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba73400640 1 -- 172.21.15.80:0/3352539435 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740c94c0 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.077+0000 7fba73e00640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba7415db70 0x7fba740cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba73400640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 0x7fba740c8dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c7f0 tx=0x7fba6400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64015070 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64007820 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba740c97a0 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64007bd0 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba740c9d60 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.078+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fba64018070 con 0x7fba740cede0 2024-04-25T10:01:35.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.079+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fba640119a0 con 0x7fba740cede0 2024-04-25T10:01:35.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.080+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba74068f60 con 0x7fba740cede0 2024-04-25T10:01:35.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.083+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fba64011bf0 con 0x7fba740cede0 2024-04-25T10:01:35.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.162+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9f6c5afa-3005-44b9-bd05-14f532e166ea", "id": 2} v 0) v1 -- 0x7fba740d0ea0 con 0x7fba740cede0 2024-04-25T10:01:35.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.169+0000 7fba6b400640 1 -- 172.21.15.80:0/3352539435 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f6c5afa-3005-44b9-bd05-14f532e166ea", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba64011580 con 0x7fba740cede0 2024-04-25T10:01:35.173 INFO:teuthology.orchestra.run.smithi080.stdout:2 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 msgr2=0x7fba740c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 0x7fba740c8dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c7f0 tx=0x7fba6400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 shutdown_connections 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba7415db70 0x7fba740cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba740d21b0 0x7fba74150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 --2- 172.21.15.80:0/3352539435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba740cede0 0x7fba740c8dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 >> 172.21.15.80:0/3352539435 conn(0x7fba740b5370 msgr2=0x7fba740cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 shutdown_connections 2024-04-25T10:01:35.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.171+0000 7fba7a000640 1 -- 172.21.15.80:0/3352539435 wait complete. 2024-04-25T10:01:35.197 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 3f1ce5b3-75f7-4950-a030-2dfac8747c3b 3 2024-04-25T10:01:35.198 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.194+0000 7f3074123040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:35.273 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.270+0000 7f1b86f23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:35.313 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.309+0000 7f3074123040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:35.370 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.366+0000 7f3074123040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:35.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.371+0000 7f538ba00640 1 Processor -- start 2024-04-25T10:01:35.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.371+0000 7f538ba00640 1 -- start start 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f538ba00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f53840cf760 0x7f53840ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f538ba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cdc60 0x7f53840ca980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f538ba00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5384066d70 con 0x7f538415b820 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f538ba00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538406ab60 con 0x7f53840cf760 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f538ba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538406adb0 con 0x7f53840cdc60 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389000640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f538415b820 0x7f538406be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35068/0 (socket says 172.21.15.80:35068) 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389000640 1 -- 172.21.15.80:0/471418327 learned_addr learned my addr 172.21.15.80:0/471418327 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cdc60 0x7f53840ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f537be00640 1 --2- 172.21.15.80:0/471418327 >> v2:172.21.15.5:3300/0 conn(0x7f53840cf760 0x7f53840ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389a00640 1 -- 172.21.15.80:0/471418327 >> v2:172.21.15.5:3300/0 conn(0x7f53840cf760 msgr2=0x7f53840ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389a00640 1 --2- 172.21.15.80:0/471418327 >> v2:172.21.15.5:3300/0 conn(0x7f53840cf760 0x7f53840ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389a00640 1 -- 172.21.15.80:0/471418327 >> v1:172.21.15.80:6789/0 conn(0x7f538415b820 legacy=0x7f538406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5389a00640 1 -- 172.21.15.80:0/471418327 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538406b090 con 0x7f53840cdc60 2024-04-25T10:01:35.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.373+0000 7f5383e00640 1 -- 172.21.15.80:0/471418327 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998031112 0 0) 0x7f5384066d70 con 0x7f538415b820 2024-04-25T10:01:35.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.374+0000 7f5389a00640 1 --2- 172.21.15.80:0/471418327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cdc60 0x7f53840ca980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f537c009d90 tx=0x7f537c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e09388b669a1b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f5383e00640 1 -- 172.21.15.80:0/471418327 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c007740 con 0x7f53840cdc60 2024-04-25T10:01:35.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f5383e00640 1 -- 172.21.15.80:0/471418327 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c0078e0 con 0x7f53840cdc60 2024-04-25T10:01:35.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f5383e00640 1 -- 172.21.15.80:0/471418327 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c007cb0 con 0x7f53840cdc60 2024-04-25T10:01:35.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 -- 172.21.15.80:0/471418327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cdc60 msgr2=0x7f53840ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 --2- 172.21.15.80:0/471418327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cdc60 0x7f53840ca980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f537c009d90 tx=0x7f537c02a660 comp rx=0 tx=0).stop 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 -- 172.21.15.80:0/471418327 shutdown_connections 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 --2- 172.21.15.80:0/471418327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cdc60 0x7f53840ca980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 --2- 172.21.15.80:0/471418327 >> v2:172.21.15.5:3300/0 conn(0x7f53840cf760 0x7f53840ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 -- 172.21.15.80:0/471418327 >> 172.21.15.80:0/471418327 conn(0x7f53840cc870 msgr2=0x7f53840ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.375+0000 7f538ba00640 1 -- 172.21.15.80:0/471418327 shutdown_connections 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.376+0000 7f538ba00640 1 -- 172.21.15.80:0/471418327 wait complete. 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.377+0000 7f538ba00640 1 Processor -- start 2024-04-25T10:01:35.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.377+0000 7f538ba00640 1 -- start start 2024-04-25T10:01:35.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f538ba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53840cdc60 0x7f538414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f538ba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 0x7f538415cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f538ba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f538415b820 0x7f538414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f538ba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538406b930 con 0x7f53840cdc60 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f538ba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538406b650 con 0x7f538415b820 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f538ba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538415bc00 con 0x7f53840cf760 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.378+0000 7f537be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 0x7f538415cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 0x7f538415cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47546/0 (socket says 172.21.15.80:47546) 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f5389a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f538415b820 0x7f538414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 -- 172.21.15.80:0/2899678221 learned_addr learned my addr 172.21.15.80:0/2899678221 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f5389000640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53840cdc60 0x7f538414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 -- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f538415b820 msgr2=0x7f538414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f538415b820 0x7f538414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 -- 172.21.15.80:0/2899678221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53840cdc60 msgr2=0x7f538414e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53840cdc60 0x7f538414e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 -- 172.21.15.80:0/2899678221 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538415d6b0 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f5389000640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53840cdc60 0x7f538414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.379+0000 7f537be00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 0x7f538415cf40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d800 tx=0x7f536c00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00b6c0 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c019070 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00ba80 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538415d990 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538415df50 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f536c00bc20 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.380+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f536c01e8f0 con 0x7f53840cf760 2024-04-25T10:01:35.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.381+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5384068f60 con 0x7f53840cf760 2024-04-25T10:01:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.383+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f536c01eaf0 con 0x7f53840cf760 2024-04-25T10:01:35.388 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.385+0000 7f1b86f23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:35.441 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.438+0000 7f1b86f23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:35.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.462+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3f1ce5b3-75f7-4950-a030-2dfac8747c3b", "id": 3} v 0) v1 -- 0x7f5384066f60 con 0x7f53840cf760 2024-04-25T10:01:35.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.469+0000 7f5381600640 1 -- 172.21.15.80:0/2899678221 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3f1ce5b3-75f7-4950-a030-2dfac8747c3b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f536c01e460 con 0x7f53840cf760 2024-04-25T10:01:35.473 INFO:teuthology.orchestra.run.smithi080.stdout:3 2024-04-25T10:01:35.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.470+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 msgr2=0x7f538415cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.470+0000 7f538ba00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 0x7f538415cf40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d800 tx=0x7f536c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.470+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 shutdown_connections 2024-04-25T10:01:35.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.471+0000 7f538ba00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f538415b820 0x7f538414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.471+0000 7f538ba00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53840cf760 0x7f538415cf40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.471+0000 7f538ba00640 1 --2- 172.21.15.80:0/2899678221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53840cdc60 0x7f538414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.471+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 >> 172.21.15.80:0/2899678221 conn(0x7f53840cc870 msgr2=0x7f53840b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.471+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 shutdown_connections 2024-04-25T10:01:35.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.471+0000 7f538ba00640 1 -- 172.21.15.80:0/2899678221 wait complete. 2024-04-25T10:01:35.489 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 4c58b47b-8353-4349-acc6-f105bffbb9d5 4 2024-04-25T10:01:35.577 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.573+0000 7f3074123040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:35.646 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.642+0000 7f3074123040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:35.650 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.646+0000 7f1b86f23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:35.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.662+0000 7faec2a00640 1 Processor -- start 2024-04-25T10:01:35.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.662+0000 7faec2a00640 1 -- start start 2024-04-25T10:01:35.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.663+0000 7faec2a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faebc06b270 0x7faebc0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faebc0c3a50 0x7faebc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec2a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc15c040 con 0x7faebc159340 2024-04-25T10:01:35.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec2a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc0cbf10 con 0x7faebc0c3a50 2024-04-25T10:01:35.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec2a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc15c320 con 0x7faebc06b270 2024-04-25T10:01:35.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec0c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faebc06b270 0x7faebc0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec0c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faebc06b270 0x7faebc0cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48240/0 (socket says 172.21.15.80:48240) 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec0c00640 1 -- 172.21.15.80:0/1481311314 learned_addr learned my addr 172.21.15.80:0/1481311314 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec2000640 1 --2- 172.21.15.80:0/1481311314 >> v2:172.21.15.5:3300/0 conn(0x7faebc0c3a50 0x7faebc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec0c00640 1 -- 172.21.15.80:0/1481311314 >> v2:172.21.15.5:3300/0 conn(0x7faebc0c3a50 msgr2=0x7faebc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.664+0000 7faec0c00640 1 --2- 172.21.15.80:0/1481311314 >> v2:172.21.15.5:3300/0 conn(0x7faebc0c3a50 0x7faebc0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.665+0000 7faec0c00640 1 -- 172.21.15.80:0/1481311314 >> v1:172.21.15.5:6789/0 conn(0x7faebc159340 legacy=0x7faebc06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.665+0000 7faec0c00640 1 -- 172.21.15.80:0/1481311314 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc15c600 con 0x7faebc06b270 2024-04-25T10:01:35.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.665+0000 7faec2000640 1 --2- 172.21.15.80:0/1481311314 >> v2:172.21.15.5:3300/0 conn(0x7faebc0c3a50 0x7faebc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:35.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.666+0000 7faec0c00640 1 --2- 172.21.15.80:0/1481311314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc0cc380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faebc0cde70 tx=0x7faea802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee0eed74741ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.666+0000 7faebbe00640 1 -- 172.21.15.80:0/1481311314 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8037070 con 0x7faebc06b270 2024-04-25T10:01:35.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.666+0000 7faebbe00640 1 -- 172.21.15.80:0/1481311314 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8032440 con 0x7faebc06b270 2024-04-25T10:01:35.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.666+0000 7faebbe00640 1 -- 172.21.15.80:0/1481311314 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8032810 con 0x7faebc06b270 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 -- 172.21.15.80:0/1481311314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 msgr2=0x7faebc0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 --2- 172.21.15.80:0/1481311314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc0cc380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faebc0cde70 tx=0x7faea802a270 comp rx=0 tx=0).stop 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 -- 172.21.15.80:0/1481311314 shutdown_connections 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 --2- 172.21.15.80:0/1481311314 >> v2:172.21.15.5:3300/0 conn(0x7faebc0c3a50 0x7faebc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 --2- 172.21.15.80:0/1481311314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc0cc380 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 -- 172.21.15.80:0/1481311314 >> 172.21.15.80:0/1481311314 conn(0x7faebc0b1f70 msgr2=0x7faebc0b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 -- 172.21.15.80:0/1481311314 shutdown_connections 2024-04-25T10:01:35.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.667+0000 7faec2a00640 1 -- 172.21.15.80:0/1481311314 wait complete. 2024-04-25T10:01:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.669+0000 7faec2a00640 1 Processor -- start 2024-04-25T10:01:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.669+0000 7faec2a00640 1 -- start start 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc1f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 0x7faebc15ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc159340 0x7faebc152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15bdb0 con 0x7faebc159340 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15bc30 con 0x7faebc06b270 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0b2a90 con 0x7faebc0c3a50 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc1f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec0c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 0x7faebc15ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc1f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48256/0 (socket says 172.21.15.80:48256) 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 -- 172.21.15.80:0/4049445783 learned_addr learned my addr 172.21.15.80:0/4049445783 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:35.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec0c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 0x7faebc15ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47556/0 (socket says 172.21.15.80:47556) 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec2000640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc159340 0x7faebc152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 -- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 msgr2=0x7faebc15ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 0x7faebc15ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 -- 172.21.15.80:0/4049445783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc159340 msgr2=0x7faebc152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc159340 0x7faebc152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec1600640 1 -- 172.21.15.80:0/4049445783 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc15d020 con 0x7faebc06b270 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.670+0000 7faec0c00640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 0x7faebc15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faec1600640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc1f12a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c400 tx=0x7faeb000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0017070 con 0x7faebc06b270 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb00052d0 con 0x7faebc06b270 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0003830 con 0x7faebc06b270 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faebc15d300 con 0x7faebc06b270 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc15d8c0 con 0x7faebc06b270 2024-04-25T10:01:35.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faeb001a070 con 0x7faebc06b270 2024-04-25T10:01:35.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.671+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7faeb0011590 con 0x7faebc06b270 2024-04-25T10:01:35.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.672+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc068f60 con 0x7faebc06b270 2024-04-25T10:01:35.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.675+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faeb0021070 con 0x7faebc06b270 2024-04-25T10:01:35.710 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.706+0000 7f3074123040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:35.710 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.707+0000 7f1b86f23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.754+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4c58b47b-8353-4349-acc6-f105bffbb9d5", "id": 4} v 0) v1 -- 0x7faebc0ccef0 con 0x7faebc06b270 2024-04-25T10:01:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.761+0000 7faeb9600640 1 -- 172.21.15.80:0/4049445783 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4c58b47b-8353-4349-acc6-f105bffbb9d5", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faeb0011bd0 con 0x7faebc06b270 2024-04-25T10:01:35.764 INFO:teuthology.orchestra.run.smithi080.stdout:4 2024-04-25T10:01:35.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.762+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 msgr2=0x7faebc1f12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.762+0000 7faec2a00640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc1f12a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faeb000c400 tx=0x7faeb000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 shutdown_connections 2024-04-25T10:01:35.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faebc159340 0x7faebc152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faebc0c3a50 0x7faebc15ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 --2- 172.21.15.80:0/4049445783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faebc06b270 0x7faebc1f12a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 >> 172.21.15.80:0/4049445783 conn(0x7faebc0b1f70 msgr2=0x7faebc0cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 shutdown_connections 2024-04-25T10:01:35.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.763+0000 7faec2a00640 1 -- 172.21.15.80:0/4049445783 wait complete. 2024-04-25T10:01:35.774 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.771+0000 7f1b86f23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:35.781 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new e723c2ea-a079-45bb-9823-a810b23ff321 5 2024-04-25T10:01:35.843 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:35.839+0000 7f3074123040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:35.909 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:35.906+0000 7f1b86f23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:35.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.958+0000 7f961f200640 1 Processor -- start 2024-04-25T10:01:35.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.959+0000 7f961f200640 1 -- start start 2024-04-25T10:01:35.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.960+0000 7f961f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96200ce9c0 0x7f962006b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.960+0000 7f961f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f962006bf00 0x7f962006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.960+0000 7f961f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96200cc2f0 con 0x7f962015da70 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.960+0000 7f961f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96200cdaa0 con 0x7f96200ce9c0 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.960+0000 7f961f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96200b3410 con 0x7f962006bf00 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961de00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f962015da70 0x7f96200b2d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35084/0 (socket says 172.21.15.80:35084) 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961de00640 1 -- 172.21.15.80:0/3435775232 learned_addr learned my addr 172.21.15.80:0/3435775232 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961e800640 1 --2- 172.21.15.80:0/3435775232 >> v2:172.21.15.80:3300/0 conn(0x7f962006bf00 0x7f962006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961d400640 1 --2- 172.21.15.80:0/3435775232 >> v2:172.21.15.5:3300/0 conn(0x7f96200ce9c0 0x7f962006b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961e800640 1 -- 172.21.15.80:0/3435775232 >> v2:172.21.15.5:3300/0 conn(0x7f96200ce9c0 msgr2=0x7f962006b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961e800640 1 --2- 172.21.15.80:0/3435775232 >> v2:172.21.15.5:3300/0 conn(0x7f96200ce9c0 0x7f962006b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961e800640 1 -- 172.21.15.80:0/3435775232 >> v1:172.21.15.80:6789/0 conn(0x7f962015da70 legacy=0x7f96200b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.961+0000 7f961e800640 1 -- 172.21.15.80:0/3435775232 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962006ac00 con 0x7f962006bf00 2024-04-25T10:01:35.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.962+0000 7f961e800640 1 --2- 172.21.15.80:0/3435775232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962006bf00 0x7f962006d580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f962009e030 tx=0x7f961802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6e743d496c933c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.962+0000 7f961ca00640 1 -- 172.21.15.80:0/3435775232 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618007740 con 0x7f962006bf00 2024-04-25T10:01:35.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.962+0000 7f961ca00640 1 -- 172.21.15.80:0/3435775232 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96180078e0 con 0x7f962006bf00 2024-04-25T10:01:35.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.962+0000 7f961ca00640 1 -- 172.21.15.80:0/3435775232 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618007cb0 con 0x7f962006bf00 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 -- 172.21.15.80:0/3435775232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962006bf00 msgr2=0x7f962006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 --2- 172.21.15.80:0/3435775232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962006bf00 0x7f962006d580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f962009e030 tx=0x7f961802a680 comp rx=0 tx=0).stop 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 -- 172.21.15.80:0/3435775232 shutdown_connections 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 --2- 172.21.15.80:0/3435775232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962006bf00 0x7f962006d580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 --2- 172.21.15.80:0/3435775232 >> v2:172.21.15.5:3300/0 conn(0x7f96200ce9c0 0x7f962006b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 -- 172.21.15.80:0/3435775232 >> 172.21.15.80:0/3435775232 conn(0x7f96200cc6a0 msgr2=0x7f96200cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:35.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 -- 172.21.15.80:0/3435775232 shutdown_connections 2024-04-25T10:01:35.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.963+0000 7f961f200640 1 -- 172.21.15.80:0/3435775232 wait complete. 2024-04-25T10:01:35.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.965+0000 7f961f200640 1 Processor -- start 2024-04-25T10:01:35.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.965+0000 7f961f200640 1 -- start start 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f962006bf00 0x7f9620141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96200ce9c0 0x7f962015de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 0x7f962015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962006b4a0 con 0x7f962006bf00 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962006b1c0 con 0x7f962015da70 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96200cdcf0 con 0x7f96200ce9c0 2024-04-25T10:01:35.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 0x7f962015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 0x7f962015e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48270/0 (socket says 172.21.15.80:48270) 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961e800640 1 -- 172.21.15.80:0/306346597 learned_addr learned my addr 172.21.15.80:0/306346597 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961d400640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96200ce9c0 0x7f962015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.966+0000 7f961de00640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f962006bf00 0x7f9620141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961e800640 1 -- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96200ce9c0 msgr2=0x7f962015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961e800640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96200ce9c0 0x7f962015de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961e800640 1 -- 172.21.15.80:0/306346597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f962006bf00 msgr2=0x7f9620141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961e800640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f962006bf00 0x7f9620141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961e800640 1 -- 172.21.15.80:0/306346597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96200ad1e0 con 0x7f962015da70 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961d400640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96200ce9c0 0x7f962015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961e800640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 0x7f962015e390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f961802ac70 tx=0x7f9618004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618040070 con 0x7f962015da70 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618031bc0 con 0x7f962015da70 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618045460 con 0x7f962015da70 2024-04-25T10:01:35.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96200ad4c0 con 0x7f962015da70 2024-04-25T10:01:35.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.967+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96200ada80 con 0x7f962015da70 2024-04-25T10:01:35.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.968+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9618039040 con 0x7f962015da70 2024-04-25T10:01:35.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.968+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9618045dd0 con 0x7f962015da70 2024-04-25T10:01:35.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.969+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96200adc80 con 0x7f962015da70 2024-04-25T10:01:35.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:35.971+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f96200adc80 con 0x7f962015da70 2024-04-25T10:01:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.050+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e723c2ea-a079-45bb-9823-a810b23ff321", "id": 5} v 0) v1 -- 0x7f9620068f60 con 0x7f962015da70 2024-04-25T10:01:36.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.057+0000 7f9616000640 1 -- 172.21.15.80:0/306346597 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e723c2ea-a079-45bb-9823-a810b23ff321", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9618007790 con 0x7f962015da70 2024-04-25T10:01:36.060 INFO:teuthology.orchestra.run.smithi080.stdout:5 2024-04-25T10:01:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 msgr2=0x7f962015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 0x7f962015e390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f961802ac70 tx=0x7f9618004a40 comp rx=0 tx=0).stop 2024-04-25T10:01:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 shutdown_connections 2024-04-25T10:01:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f962015da70 0x7f962015e390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96200ce9c0 0x7f962015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 --2- 172.21.15.80:0/306346597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f962006bf00 0x7f9620141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 >> 172.21.15.80:0/306346597 conn(0x7f96200cc6a0 msgr2=0x7f96200b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 shutdown_connections 2024-04-25T10:01:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.058+0000 7f961f200640 1 -- 172.21.15.80:0/306346597 wait complete. 2024-04-25T10:01:36.079 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 313e8231-12a4-4eda-97cd-920f3ae7fcf4 6 2024-04-25T10:01:36.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.250+0000 7fb97de00640 1 Processor -- start 2024-04-25T10:01:36.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.251+0000 7fb97de00640 1 -- start start 2024-04-25T10:01:36.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97de00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9780b0bf0 0x7fb97815f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97de00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb9780b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9780b2370 0x7fb9780999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97de00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9781601b0 con 0x7fb9780b2370 2024-04-25T10:01:36.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97de00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9780cc640 con 0x7fb9780b0bf0 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97de00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780b2800 con 0x7fb97815f890 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9780b0bf0 0x7fb97815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb977e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb9780b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9780b0bf0 0x7fb97815f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48282/0 (socket says 172.21.15.80:48282) 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97ca00640 1 -- 172.21.15.80:0/3814920338 learned_addr learned my addr 172.21.15.80:0/3814920338 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.252+0000 7fb97d400640 1 --2- 172.21.15.80:0/3814920338 >> v2:172.21.15.5:3300/0 conn(0x7fb9780b2370 0x7fb9780999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.253+0000 7fb97ca00640 1 -- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 msgr2=0x7fb9780b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.253+0000 7fb97ca00640 1 --2- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb9780b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.253+0000 7fb97ca00640 1 -- 172.21.15.80:0/3814920338 >> v2:172.21.15.5:3300/0 conn(0x7fb9780b2370 msgr2=0x7fb9780999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.253+0000 7fb97ca00640 1 --2- 172.21.15.80:0/3814920338 >> v2:172.21.15.5:3300/0 conn(0x7fb9780b2370 0x7fb9780999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.253+0000 7fb97ca00640 1 -- 172.21.15.80:0/3814920338 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9780b2d40 con 0x7fb9780b0bf0 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.254+0000 7fb97ca00640 1 --2- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb97815f350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb978071b60 tx=0x7fb96c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ccd720734be94ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.254+0000 7fb977400640 1 -- 172.21.15.80:0/3814920338 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c037070 con 0x7fb9780b0bf0 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.254+0000 7fb977400640 1 -- 172.21.15.80:0/3814920338 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c032440 con 0x7fb9780b0bf0 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.254+0000 7fb977400640 1 -- 172.21.15.80:0/3814920338 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c032810 con 0x7fb9780b0bf0 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 -- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 msgr2=0x7fb97815f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 --2- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb97815f350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb978071b60 tx=0x7fb96c02a210 comp rx=0 tx=0).stop 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 -- 172.21.15.80:0/3814920338 shutdown_connections 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 --2- 172.21.15.80:0/3814920338 >> v2:172.21.15.5:3300/0 conn(0x7fb9780b2370 0x7fb9780999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 --2- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb9780b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 --2- 172.21.15.80:0/3814920338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb97815f350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 -- 172.21.15.80:0/3814920338 >> 172.21.15.80:0/3814920338 conn(0x7fb9780d1470 msgr2=0x7fb9780d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 -- 172.21.15.80:0/3814920338 shutdown_connections 2024-04-25T10:01:36.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.255+0000 7fb97de00640 1 -- 172.21.15.80:0/3814920338 wait complete. 2024-04-25T10:01:36.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.256+0000 7fb97de00640 1 Processor -- start 2024-04-25T10:01:36.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 -- start start 2024-04-25T10:01:36.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb978149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9780b2370 0x7fb9780b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb97815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780cbff0 con 0x7fb9780b2370 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780b2a60 con 0x7fb9780b0bf0 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.257+0000 7fb97de00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9780cc2d0 con 0x7fb97815f890 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb978149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb978149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48298/0 (socket says 172.21.15.80:48298) 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 -- 172.21.15.80:0/2261708111 learned_addr learned my addr 172.21.15.80:0/2261708111 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97d400640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb97815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb977e00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9780b2370 0x7fb9780b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 -- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 msgr2=0x7fb97815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb97815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 -- 172.21.15.80:0/2261708111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9780b2370 msgr2=0x7fb9780b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9780b2370 0x7fb9780b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 -- 172.21.15.80:0/2261708111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb978149f60 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.258+0000 7fb97ca00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb978149a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0097f0 tx=0x7fb96c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c037040 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c042070 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c030880 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9780c9160 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9780c9720 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb96c030a20 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.259+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fb96c030d10 con 0x7fb9780b0bf0 2024-04-25T10:01:36.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.260+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb978068f60 con 0x7fb9780b0bf0 2024-04-25T10:01:36.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.264+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb96c047460 con 0x7fb9780b0bf0 2024-04-25T10:01:36.318 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:36.314+0000 7f3074123040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:01:36.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.344+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "313e8231-12a4-4eda-97cd-920f3ae7fcf4", "id": 6} v 0) v1 -- 0x7fb9780b1200 con 0x7fb9780b0bf0 2024-04-25T10:01:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.350+0000 7fb974c00640 1 -- 172.21.15.80:0/2261708111 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "313e8231-12a4-4eda-97cd-920f3ae7fcf4", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb96c047640 con 0x7fb9780b0bf0 2024-04-25T10:01:36.354 INFO:teuthology.orchestra.run.smithi080.stdout:6 2024-04-25T10:01:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.351+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 msgr2=0x7fb978149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.351+0000 7fb97de00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb978149a20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0097f0 tx=0x7fb96c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 shutdown_connections 2024-04-25T10:01:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb97815f890 0x7fb97815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9780b2370 0x7fb9780b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 --2- 172.21.15.80:0/2261708111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9780b0bf0 0x7fb978149a20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 >> 172.21.15.80:0/2261708111 conn(0x7fb9780d1470 msgr2=0x7fb97806f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 shutdown_connections 2024-04-25T10:01:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.352+0000 7fb97de00640 1 -- 172.21.15.80:0/2261708111 wait complete. 2024-04-25T10:01:36.372 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new cd9e067f-4217-427a-b6a6-2e6f8578c9d2 7 2024-04-25T10:01:36.391 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:36.388+0000 7f1b86f23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:01:36.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.548+0000 7f49daa00640 1 Processor -- start 2024-04-25T10:01:36.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.548+0000 7f49daa00640 1 -- start start 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.549+0000 7f49daa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.549+0000 7f49daa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f49d40d4000 0x7f49d4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.549+0000 7f49daa00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d415ffa0 con 0x7f49d4161240 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.549+0000 7f49daa00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d4160240 con 0x7f49d40d4000 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49daa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4160520 con 0x7f49d415ec80 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47578/0 (socket says 172.21.15.80:47578) 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 -- 172.21.15.80:0/2364918138 learned_addr learned my addr 172.21.15.80:0/2364918138 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:36.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d8c00640 1 --2- 172.21.15.80:0/2364918138 >> v2:172.21.15.5:3300/0 conn(0x7f49d40d4000 0x7f49d4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 -- 172.21.15.80:0/2364918138 >> v2:172.21.15.5:3300/0 conn(0x7f49d40d4000 msgr2=0x7f49d4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 --2- 172.21.15.80:0/2364918138 >> v2:172.21.15.5:3300/0 conn(0x7f49d40d4000 0x7f49d4160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 -- 172.21.15.80:0/2364918138 >> v1:172.21.15.5:6789/0 conn(0x7f49d4161240 legacy=0x7f49d41616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.550+0000 7f49d9600640 1 -- 172.21.15.80:0/2364918138 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4160800 con 0x7f49d415ec80 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.551+0000 7f49d9600640 1 --2- 172.21.15.80:0/2364918138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b45c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009850 tx=0x7f49c802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7355556e016378c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.551+0000 7f49d3e00640 1 -- 172.21.15.80:0/2364918138 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8004030 con 0x7f49d415ec80 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49d3e00640 1 -- 172.21.15.80:0/2364918138 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c80041d0 con 0x7f49d415ec80 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49d3e00640 1 -- 172.21.15.80:0/2364918138 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8003610 con 0x7f49d415ec80 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 -- 172.21.15.80:0/2364918138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 msgr2=0x7f49d40b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 --2- 172.21.15.80:0/2364918138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b45c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009850 tx=0x7f49c802a230 comp rx=0 tx=0).stop 2024-04-25T10:01:36.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 -- 172.21.15.80:0/2364918138 shutdown_connections 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 --2- 172.21.15.80:0/2364918138 >> v2:172.21.15.5:3300/0 conn(0x7f49d40d4000 0x7f49d4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 --2- 172.21.15.80:0/2364918138 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b45c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 -- 172.21.15.80:0/2364918138 >> 172.21.15.80:0/2364918138 conn(0x7f49d408af80 msgr2=0x7f49d40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 -- 172.21.15.80:0/2364918138 shutdown_connections 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.552+0000 7f49daa00640 1 -- 172.21.15.80:0/2364918138 wait complete. 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.554+0000 7f49daa00640 1 Processor -- start 2024-04-25T10:01:36.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.554+0000 7f49daa00640 1 -- start start 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49daa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d40d4000 0x7f49d4150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49daa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49daa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49d4161240 0x7f49d40b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49daa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40d4460 con 0x7f49d40d4000 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49daa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40b4c50 con 0x7f49d4161240 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49daa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4161e00 con 0x7f49d415ec80 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49d8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49d8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:47580/0 (socket says 172.21.15.80:47580) 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49d8c00640 1 -- 172.21.15.80:0/776427198 learned_addr learned my addr 172.21.15.80:0/776427198 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.555+0000 7f49da000640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49d4161240 0x7f49d40b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d8c00640 1 -- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49d4161240 msgr2=0x7f49d40b7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d8c00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49d4161240 0x7f49d40b7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d8c00640 1 -- 172.21.15.80:0/776427198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d40d4000 msgr2=0x7f49d4150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d8c00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d40d4000 0x7f49d4150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d8c00640 1 -- 172.21.15.80:0/776427198 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40b8e30 con 0x7f49d415ec80 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49da000640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49d4161240 0x7f49d40b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d8c00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b6450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d800 tx=0x7f49c000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0018020 con 0x7f49d415ec80 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c00096c0 con 0x7f49d415ec80 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0009ab0 con 0x7f49d415ec80 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d40b9110 con 0x7f49d415ec80 2024-04-25T10:01:36.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.556+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d41e7a90 con 0x7f49d415ec80 2024-04-25T10:01:36.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.557+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f49c001b020 con 0x7f49d415ec80 2024-04-25T10:01:36.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.557+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f49c001dbf0 con 0x7f49d415ec80 2024-04-25T10:01:36.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.557+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d4068f60 con 0x7f49d415ec80 2024-04-25T10:01:36.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.560+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f49c0022020 con 0x7f49d415ec80 2024-04-25T10:01:36.626 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:36.622+0000 7f3074123040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.639+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cd9e067f-4217-427a-b6a6-2e6f8578c9d2", "id": 7} v 0) v1 -- 0x7f49d40b92e0 con 0x7f49d415ec80 2024-04-25T10:01:36.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.647+0000 7f49d1600640 1 -- 172.21.15.80:0/776427198 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cd9e067f-4217-427a-b6a6-2e6f8578c9d2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f49c001d650 con 0x7f49d415ec80 2024-04-25T10:01:36.651 INFO:teuthology.orchestra.run.smithi080.stdout:7 2024-04-25T10:01:36.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.648+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 msgr2=0x7f49d40b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.648+0000 7f49daa00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b6450 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f49c000d800 tx=0x7f49c000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 shutdown_connections 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49d4161240 0x7f49d40b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49d415ec80 0x7f49d40b6450 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 --2- 172.21.15.80:0/776427198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d40d4000 0x7f49d4150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 >> 172.21.15.80:0/776427198 conn(0x7f49d408af80 msgr2=0x7f49d415cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 shutdown_connections 2024-04-25T10:01:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T10:01:36.649+0000 7f49daa00640 1 -- 172.21.15.80:0/776427198 wait complete. 2024-04-25T10:01:36.669 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:01:36.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:01:36.677 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:01:36.677 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:01:36.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:01:36.685 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:01:36.686 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:01:36.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:01:36.691 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:01:36.692 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:01:36.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:01:36.692 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:36.687+0000 7f3074123040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:36.698 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:01:36.699 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:01:36.699 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:01:36.701 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:36.697+0000 7f1b86f23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:36.706 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:01:36.706 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:01:36.706 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:01:36.711 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:01:36.711 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:01:36.712 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:01:36.717 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:01:36.717 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:01:36.717 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:01:36.722 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:01:36.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:36.777 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:36.773+0000 7f1b86f23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:36.890 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:36.886+0000 7f3074123040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:01:36.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.927+0000 7fcbc7c00640 1 Processor -- start 2024-04-25T10:01:36.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.928+0000 7fcbc7c00640 1 -- start start 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc7c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbc015dd00 0x7fcbc00b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc7c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc7c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc006cd00 con 0x7fcbc006bfb0 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc7c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc015e0e0 con 0x7fcbc015dd00 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc7c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc006cfe0 con 0x7fcbc00b6a40 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc5200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbc015dd00 0x7fcbc00b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc5200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbc015dd00 0x7fcbc00b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:60536/0 (socket says 172.21.15.5:60536) 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbc5200640 1 -- 172.21.15.5:0/1281424722 learned_addr learned my addr 172.21.15.5:0/1281424722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.929+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc00b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:34766/0 (socket says 172.21.15.5:34766) 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.930+0000 7fcbc5200640 1 -- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 msgr2=0x7fcbc00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.930+0000 7fcbc5200640 1 --2- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc00b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.930+0000 7fcbc5200640 1 -- 172.21.15.5:0/1281424722 >> v1:172.21.15.80:6789/0 conn(0x7fcbc006bfb0 legacy=0x7fcbc006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.930+0000 7fcbc5200640 1 -- 172.21.15.5:0/1281424722 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00d1ca0 con 0x7fcbc015dd00 2024-04-25T10:01:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.930+0000 7fcbbfe00640 1 --2- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.931+0000 7fcbc5200640 1 --2- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc015dd00 0x7fcbc00b5750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009870 tx=0x7fcbb402a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98cd2b72a88cd086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.931+0000 7fcbbf400640 1 -- 172.21.15.5:0/1281424722 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4037070 con 0x7fcbc015dd00 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.931+0000 7fcbbf400640 1 -- 172.21.15.5:0/1281424722 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4032640 con 0x7fcbc015dd00 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.931+0000 7fcbbf400640 1 -- 172.21.15.5:0/1281424722 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4032a10 con 0x7fcbc015dd00 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.931+0000 7fcbc7c00640 1 -- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc015dd00 msgr2=0x7fcbc00b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.931+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc015dd00 0x7fcbc00b5750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009870 tx=0x7fcbb402a2f0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.932+0000 7fcbc7c00640 1 -- 172.21.15.5:0/1281424722 shutdown_connections 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.932+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.932+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/1281424722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc015dd00 0x7fcbc00b5750 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.932+0000 7fcbc7c00640 1 -- 172.21.15.5:0/1281424722 >> 172.21.15.5:0/1281424722 conn(0x7fcbc00d07b0 msgr2=0x7fcbc00d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.932+0000 7fcbc7c00640 1 -- 172.21.15.5:0/1281424722 shutdown_connections 2024-04-25T10:01:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.932+0000 7fcbc7c00640 1 -- 172.21.15.5:0/1281424722 wait complete. 2024-04-25T10:01:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.933+0000 7fcbc7c00640 1 Processor -- start 2024-04-25T10:01:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.933+0000 7fcbc7c00640 1 -- start start 2024-04-25T10:01:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.934+0000 7fcbc7c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 0x7fcbc00cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.934+0000 7fcbc7c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc0152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.934+0000 7fcbc7c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc00c9500 0x7fcbc0152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:36.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.934+0000 7fcbc7c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00d2780 con 0x7fcbc00c9500 2024-04-25T10:01:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.934+0000 7fcbc7c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00d2260 con 0x7fcbc006bfb0 2024-04-25T10:01:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.934+0000 7fcbc7c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc00d2900 con 0x7fcbc00b6a40 2024-04-25T10:01:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc00c9500 0x7fcbc0152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 0x7fcbc00cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc0152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:36.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 0x7fcbc00cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:60538/0 (socket says 172.21.15.5:60538) 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5200640 1 -- 172.21.15.5:0/2784624455 learned_addr learned my addr 172.21.15.5:0/2784624455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbbfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc0152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:34768/0 (socket says 172.21.15.5:34768) 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 -- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 msgr2=0x7fcbc0152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc0152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 -- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 msgr2=0x7fcbc00cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 0x7fcbc00cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 -- 172.21.15.5:0/2784624455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc00cd900 con 0x7fcbc00c9500 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbbfe00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc0152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5200640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 0x7fcbc00cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.935+0000 7fcbc5c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc00c9500 0x7fcbc0152fc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c6c0 tx=0x7fcbb800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8003b70 con 0x7fcbc00c9500 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb8003d10 con 0x7fcbc00c9500 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc015a210 con 0x7fcbc00c9500 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80052f0 con 0x7fcbc00c9500 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc015a7d0 con 0x7fcbc00c9500 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcbb8005490 con 0x7fcbc00c9500 2024-04-25T10:01:36.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.936+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fcbb8012b40 con 0x7fcbc00c9500 2024-04-25T10:01:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.937+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc014d010 con 0x7fcbc00c9500 2024-04-25T10:01:36.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:36.940+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcbc014d010 con 0x7fcbc00c9500 2024-04-25T10:01:36.948 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:36.945+0000 7f1b86f23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:01:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.004+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbc0068fd0 con 0x7fcbc00c9500 2024-04-25T10:01:37.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.006+0000 7fcbbcc00640 1 -- 172.21.15.5:0/2784624455 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fcbb8012e20 con 0x7fcbc00c9500 2024-04-25T10:01:37.010 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:01:37.010 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:01:36.641057+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:37.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc00c9500 msgr2=0x7fcbc0152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:37.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc00c9500 0x7fcbc0152fc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c6c0 tx=0x7fcbb800cb80 comp rx=0 tx=0).stop 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 shutdown_connections 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc00c9500 0x7fcbc0152fc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc00b6a40 0x7fcbc0152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 --2- 172.21.15.5:0/2784624455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbc006bfb0 0x7fcbc00cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 >> 172.21.15.5:0/2784624455 conn(0x7fcbc00d07b0 msgr2=0x7fcbc006c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 shutdown_connections 2024-04-25T10:01:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:37.007+0000 7fcbc7c00640 1 -- 172.21.15.5:0/2784624455 wait complete. 2024-04-25T10:01:37.027 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:01:37.028 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:01:37.237 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.233+0000 7f3074123040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:37.291 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.287+0000 7f1b86f23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:37.295 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.291+0000 7f3074123040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:37.347 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.344+0000 7f1b86f23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:37.374 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-04-25T10:01:37.370+0000 7f1ec969a640 -1 Falling back to public interface 2024-04-25T10:01:37.383 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T10:01:37.379+0000 7fca52e9a640 -1 Falling back to public interface 2024-04-25T10:01:37.388 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T10:01:37.384+0000 7f84720ee640 -1 Falling back to public interface 2024-04-25T10:01:37.397 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T10:01:37.393+0000 7fada89c4640 -1 Falling back to public interface 2024-04-25T10:01:37.405 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-04-25T10:01:37.401+0000 7f0247086640 -1 Falling back to public interface 2024-04-25T10:01:37.405 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-04-25T10:01:37.401+0000 7f2497baf640 -1 Falling back to public interface 2024-04-25T10:01:37.412 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-04-25T10:01:37.408+0000 7f4a78e1d640 -1 Falling back to public interface 2024-04-25T10:01:37.435 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T10:01:37.431+0000 7f4f3b41f640 -1 Falling back to public interface 2024-04-25T10:01:37.496 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.492+0000 7f3074123040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:37.542 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.538+0000 7f1b86f23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:37.672 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.668+0000 7f3074123040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:37.708 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.704+0000 7f1b86f23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:37.734 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.730+0000 7f3074123040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:37.768 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.764+0000 7f1b86f23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:37.818 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.814+0000 7f3074123040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:37.832 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:37.848 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.844+0000 7f1b86f23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:37.876 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:37.871+0000 7f3074123040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:37.901 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:37.897+0000 7f1b86f23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:37.994 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.990+0000 7f3d69200640 1 Processor -- start 2024-04-25T10:01:37.995 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.990+0000 7f3d69200640 1 -- start start 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.991+0000 7f3d69200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 0x7f3d640b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.991+0000 7f3d69200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d640d23b0 0x7f3d640d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d69200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640d1bf0 con 0x7f3d640b53f0 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d69200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640d1f20 con 0x7f3d640d23b0 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d69200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b5d10 con 0x7f3d640ce240 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d61e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3d640b53f0 0x7f3d6406c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39648/0 (socket says 172.21.15.5:39648) 2024-04-25T10:01:37.996 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d61e00640 1 -- 172.21.15.5:0/1557466112 learned_addr learned my addr 172.21.15.5:0/1557466112 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:37.999 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d62800640 1 --2- 172.21.15.5:0/1557466112 >> v2:172.21.15.80:3300/0 conn(0x7f3d640d23b0 0x7f3d640d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.999 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d5be00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 0x7f3d640b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:37.999 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704177972 0 0) 0x7f3d640d1bf0 con 0x7f3d640b53f0 2024-04-25T10:01:37.999 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.992+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d48003520 con 0x7f3d640b53f0 2024-04-25T10:01:37.999 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242726012 0 0) 0x7f3d48003520 con 0x7f3d640b53f0 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 msgr2=0x7f3d640b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 0x7f3d640b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 >> v2:172.21.15.80:3300/0 conn(0x7f3d640d23b0 msgr2=0x7f3d640d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 --2- 172.21.15.5:0/1557466112 >> v2:172.21.15.80:3300/0 conn(0x7f3d640d23b0 0x7f3d640d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640ceb00 con 0x7f3d640b53f0 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f3d54001670 con 0x7f3d640b53f0 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 >> v1:172.21.15.5:6789/0 conn(0x7f3d640b53f0 legacy=0x7f3d6406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d480039b0 0x7f3d48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d48004800 0x7f3d48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 0x7f3d480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640d1f20 con 0x7f3d480039b0 2024-04-25T10:01:38.000 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640b5d10 con 0x7f3d48004800 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4800b9c0 con 0x7f3d640ce240 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d5be00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d48004800 0x7f3d48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.993+0000 7f3d62800640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 0x7f3d480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d61e00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d480039b0 0x7f3d48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d5be00640 1 -- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 msgr2=0x7f3d480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d5be00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640ce240 0x7f3d480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d5be00640 1 -- 172.21.15.5:0/1557466112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d480039b0 msgr2=0x7f3d48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d5be00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d480039b0 0x7f3d48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d5be00640 1 -- 172.21.15.5:0/1557466112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48003520 con 0x7f3d48004800 2024-04-25T10:01:38.001 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.994+0000 7f3d61e00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d480039b0 0x7f3d48003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:38.002 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.995+0000 7f3d5be00640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d48004800 0x7f3d48004c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009ac0 tx=0x7f3d48005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946cc1de0ea707f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.002 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.995+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c036070 con 0x7f3d48004800 2024-04-25T10:01:38.002 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.995+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c031400 con 0x7f3d48004800 2024-04-25T10:01:38.002 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.995+0000 7f3d61400640 1 -- 172.21.15.5:0/1557466112 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c009690 con 0x7f3d48004800 2024-04-25T10:01:38.005 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d69200640 1 -- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d48004800 msgr2=0x7f3d48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.005 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d69200640 1 --2- 172.21.15.5:0/1557466112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d48004800 0x7f3d48004c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009ac0 tx=0x7f3d48005830 comp rx=0 tx=0).stop 2024-04-25T10:01:38.005 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d62800640 1 -- 172.21.15.5:0/1557466112 reap_dead start 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d69200640 1 -- 172.21.15.5:0/1557466112 shutdown_connections 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d69200640 1 -- 172.21.15.5:0/1557466112 >> 172.21.15.5:0/1557466112 conn(0x7f3d640d03f0 msgr2=0x7f3d640d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d69200640 1 -- 172.21.15.5:0/1557466112 shutdown_connections 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.996+0000 7f3d69200640 1 -- 172.21.15.5:0/1557466112 wait complete. 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.998+0000 7f3d69200640 1 Processor -- start 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.998+0000 7f3d69200640 1 -- start start 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d69200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 0x7f3d641f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d69200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d641f4970 0x7f3d640b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d69200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640b8a50 0x7f3d641f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d69200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640cf240 con 0x7f3d48004800 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d69200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640cf0c0 con 0x7f3d641f4970 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d69200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6415de40 con 0x7f3d640b8a50 2024-04-25T10:01:38.006 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 0x7f3d641f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 0x7f3d641f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34752/0 (socket says 172.21.15.5:34752) 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 -- 172.21.15.5:0/1850709101 learned_addr learned my addr 172.21.15.5:0/1850709101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d62800640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640b8a50 0x7f3d641f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d5be00640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d641f4970 0x7f3d640b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 -- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640b8a50 msgr2=0x7f3d641f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640b8a50 0x7f3d641f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 -- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d641f4970 msgr2=0x7f3d640b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:37.999+0000 7f3d61e00640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d641f4970 0x7f3d640b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d61e00640 1 -- 172.21.15.5:0/1850709101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d641f6b00 con 0x7f3d48004800 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d62800640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640b8a50 0x7f3d641f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d61e00640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 0x7f3d641f65c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400e730 tx=0x7f3d5400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400ee40 con 0x7f3d48004800 2024-04-25T10:01:38.007 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d54018070 con 0x7f3d48004800 2024-04-25T10:01:38.008 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400cad0 con 0x7f3d48004800 2024-04-25T10:01:38.008 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d641f46c0 con 0x7f3d48004800 2024-04-25T10:01:38.008 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d640c85c0 con 0x7f3d48004800 2024-04-25T10:01:38.008 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.000+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3d5400cc70 con 0x7f3d48004800 2024-04-25T10:01:38.009 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.001+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f3d5401db30 con 0x7f3d48004800 2024-04-25T10:01:38.009 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.001+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6404ef20 con 0x7f3d48004800 2024-04-25T10:01:38.009 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.004+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3d5401d460 con 0x7f3d48004800 2024-04-25T10:01:38.126 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.122+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d640ce7b0 con 0x7f3d48004800 2024-04-25T10:01:38.127 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.123+0000 7f3d5a000640 1 -- 172.21.15.5:0/1850709101 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f3d5401d640 con 0x7f3d48004800 2024-04-25T10:01:38.128 INFO:teuthology.misc.health.smithi005.stdout: 2024-04-25T10:01:38.128 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:01:36.641057+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:38.129 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 msgr2=0x7f3d641f65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:38.129 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 0x7f3d641f65c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400e730 tx=0x7f3d5400ebf0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.129 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 shutdown_connections 2024-04-25T10:01:38.129 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d640b8a50 0x7f3d641f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.129 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d641f4970 0x7f3d640b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.129 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 --2- 172.21.15.5:0/1850709101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d48004800 0x7f3d641f65c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:38.130 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 >> 172.21.15.5:0/1850709101 conn(0x7f3d640d03f0 msgr2=0x7f3d640d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:38.130 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 shutdown_connections 2024-04-25T10:01:38.130 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:38.125+0000 7f3d69200640 1 -- 172.21.15.5:0/1850709101 wait complete. 2024-04-25T10:01:38.142 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:01:38.445 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:38.440+0000 7f1b86f23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:38.486 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:38.482+0000 7f3074123040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:38.578 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:38.573+0000 7f1b86f23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:38.627 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-04-25T10:01:38.623+0000 7f3074123040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:39.624 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:39.619+0000 7f1b27600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:01:39.624 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-04-25T10:01:39.620+0000 7f1b27600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:01:40.539 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T10:01:40.535+0000 7f84720ee640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:01:40.768 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T10:01:40.764+0000 7fada89c4640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:01:41.037 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-04-25T10:01:41.032+0000 7f1ec969a640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:01:41.063 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-04-25T10:01:41.059+0000 7f4a78e1d640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:01:41.076 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T10:01:41.071+0000 7f4f3b41f640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:01:41.220 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T10:01:41.216+0000 7fca52e9a640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:01:41.308 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-04-25T10:01:41.304+0000 7f0247086640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:01:41.310 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-04-25T10:01:41.306+0000 7f2497baf640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:01:42.660 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T10:01:42.656+0000 7f846da00640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:01:42.858 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-04-25T10:01:42.854+0000 7f8463000640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.380 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-04-25T10:01:44.376+0000 7f2492a00640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:01:44.384 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-04-25T10:01:44.379+0000 7f1ec4600640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:01:44.388 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-04-25T10:01:44.384+0000 7f2488a00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.392 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-04-25T10:01:44.388+0000 7f4a73800640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:01:44.395 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-04-25T10:01:44.391+0000 7f1eba600640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.398 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-04-25T10:01:44.394+0000 7f4a69800640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.424 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-04-25T10:01:44.420+0000 7f0242000640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:01:44.431 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-04-25T10:01:44.427+0000 7f0238000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.488 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T10:01:44.483+0000 7fada3800640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:01:44.496 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-04-25T10:01:44.492+0000 7fad99800640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.509 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T10:01:44.503+0000 7fca4e800640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:01:44.516 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-04-25T10:01:44.512+0000 7fca43e00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.556 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T10:01:44.552+0000 7f4f36c00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:01:44.566 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-04-25T10:01:44.562+0000 7f4f2c200640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:44.949 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:45.143 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.138+0000 7fcf46a00640 1 Processor -- start 2024-04-25T10:01:45.143 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.139+0000 7fcf46a00640 1 -- start start 2024-04-25T10:01:45.145 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.140+0000 7fcf46a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcf400ce6f0 0x7fcf400cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.145 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf46a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf400ccf80 con 0x7fcf4006ee50 2024-04-25T10:01:45.145 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf46a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf40071e90 con 0x7fcf4006d4e0 2024-04-25T10:01:45.145 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf46a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf40071c40 con 0x7fcf400ce6f0 2024-04-25T10:01:45.145 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf3f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcf4006ee50 0x7fcf400cead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39660/0 (socket says 172.21.15.5:39660) 2024-04-25T10:01:45.145 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf3f400640 1 -- 172.21.15.5:0/3471617736 learned_addr learned my addr 172.21.15.5:0/3471617736 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:45.146 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf3fe00640 1 --2- 172.21.15.5:0/3471617736 >> v2:172.21.15.5:3300/0 conn(0x7fcf400ce6f0 0x7fcf400cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.146 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.141+0000 7fcf3ea00640 1 -- 172.21.15.5:0/3471617736 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400412764 0 0) 0x7fcf400ccf80 con 0x7fcf4006ee50 2024-04-25T10:01:45.147 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.142+0000 7fcf3ea00640 1 -- 172.21.15.5:0/3471617736 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003520 con 0x7fcf4006ee50 2024-04-25T10:01:45.147 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.142+0000 7fcf3fe00640 1 -- 172.21.15.5:0/3471617736 >> v1:172.21.15.80:6789/0 conn(0x7fcf4006d4e0 legacy=0x7fcf4006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.147 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.142+0000 7fcf3fe00640 1 -- 172.21.15.5:0/3471617736 >> v1:172.21.15.5:6789/0 conn(0x7fcf4006ee50 legacy=0x7fcf400cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.147 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.142+0000 7fcf3fe00640 1 -- 172.21.15.5:0/3471617736 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40072170 con 0x7fcf400ce6f0 2024-04-25T10:01:45.147 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.143+0000 7fcf3fe00640 1 --2- 172.21.15.5:0/3471617736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf400ce6f0 0x7fcf400cc8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009d90 tx=0x7fcf3002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f5a730819175ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.148 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.143+0000 7fcf3ea00640 1 -- 172.21.15.5:0/3471617736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30007740 con 0x7fcf400ce6f0 2024-04-25T10:01:45.148 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.143+0000 7fcf3ea00640 1 -- 172.21.15.5:0/3471617736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf300078e0 con 0x7fcf400ce6f0 2024-04-25T10:01:45.148 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.143+0000 7fcf3ea00640 1 -- 172.21.15.5:0/3471617736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf30007cb0 con 0x7fcf400ce6f0 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 -- 172.21.15.5:0/3471617736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf400ce6f0 msgr2=0x7fcf400cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3471617736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf400ce6f0 0x7fcf400cc8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009d90 tx=0x7fcf3002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 -- 172.21.15.5:0/3471617736 shutdown_connections 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3471617736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf400ce6f0 0x7fcf400cc8f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 -- 172.21.15.5:0/3471617736 >> 172.21.15.5:0/3471617736 conn(0x7fcf400b12b0 msgr2=0x7fcf400b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 -- 172.21.15.5:0/3471617736 shutdown_connections 2024-04-25T10:01:45.149 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.144+0000 7fcf46a00640 1 -- 172.21.15.5:0/3471617736 wait complete. 2024-04-25T10:01:45.150 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.146+0000 7fcf46a00640 1 Processor -- start 2024-04-25T10:01:45.150 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.146+0000 7fcf46a00640 1 -- start start 2024-04-25T10:01:45.152 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf46a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf4006d4e0 0x7fcf401f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.152 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf46a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 0x7fcf400a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.152 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf46a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcf400ce6f0 0x7fcf401f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.152 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf46a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4015caa0 con 0x7fcf4006ee50 2024-04-25T10:01:45.152 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf46a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400aee40 con 0x7fcf4006d4e0 2024-04-25T10:01:45.152 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf46a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4015cc20 con 0x7fcf400ce6f0 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 0x7fcf400a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 0x7fcf400a9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34808/0 (socket says 172.21.15.5:34808) 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 -- 172.21.15.5:0/3037915989 learned_addr learned my addr 172.21.15.5:0/3037915989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3fe00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf4006d4e0 0x7fcf401f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf44a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcf400ce6f0 0x7fcf401f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 -- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcf400ce6f0 msgr2=0x7fcf401f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcf400ce6f0 0x7fcf401f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 -- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf4006d4e0 msgr2=0x7fcf401f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf4006d4e0 0x7fcf401f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf3f400640 1 -- 172.21.15.5:0/3037915989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf400aa3a0 con 0x7fcf4006ee50 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.147+0000 7fcf44a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcf400ce6f0 0x7fcf401f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:45.153 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.148+0000 7fcf3f400640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 0x7fcf400a9e60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800c460 tx=0x7fcf2800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.154 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.148+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28017070 con 0x7fcf4006ee50 2024-04-25T10:01:45.154 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.148+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28005ce0 con 0x7fcf4006ee50 2024-04-25T10:01:45.154 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.148+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28007290 con 0x7fcf4006ee50 2024-04-25T10:01:45.154 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.148+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400aaac0 con 0x7fcf4006ee50 2024-04-25T10:01:45.154 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.148+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf400ab080 con 0x7fcf4006ee50 2024-04-25T10:01:45.155 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.149+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4004ef20 con 0x7fcf4006ee50 2024-04-25T10:01:45.155 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.150+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fcf2801a070 con 0x7fcf4006ee50 2024-04-25T10:01:45.155 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.151+0000 7fcf37e00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcf1c06b470 0x7fcf1c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.155 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.151+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7fcf28088830 con 0x7fcf4006ee50 2024-04-25T10:01:45.156 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.152+0000 7fcf3fe00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcf1c06b470 0x7fcf1c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.156 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.153+0000 7fcf3fe00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcf1c06b470 0x7fcf1c06d930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf30034000 tx=0x7fcf3002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.160 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.156+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf400aa3a0 con 0x7fcf4006ee50 2024-04-25T10:01:45.354 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.350+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf400af910 con 0x7fcf4006ee50 2024-04-25T10:01:45.355 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.351+0000 7fcf37e00640 1 -- 172.21.15.5:0/3037915989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7555 (secure 0 0 0) 0x7fcf28058370 con 0x7fcf4006ee50 2024-04-25T10:01:45.355 INFO:teuthology.misc.health.smithi005.stdout: 2024-04-25T10:01:45.355 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":14,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:01:43.694046+0000","last_up_change":"2024-04-25T10:01:43.694046+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcf1c06b470 msgr2=0x7fcf1c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcf1c06b470 0x7fcf1c06d930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf30034000 tx=0x7fcf3002f000 comp rx=0 tx=0).stop 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 msgr2=0x7fcf400a9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 0x7fcf400a9e60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800c460 tx=0x7fcf2800c920 comp rx=0 tx=0).stop 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 shutdown_connections 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcf400ce6f0 0x7fcf401f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.358 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf4006ee50 0x7fcf400a9e60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.359 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcf1c06b470 0x7fcf1c06d930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.359 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 --2- 172.21.15.5:0/3037915989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf4006d4e0 0x7fcf401f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.359 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 >> 172.21.15.5:0/3037915989 conn(0x7fcf400b12b0 msgr2=0x7fcf4015e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.359 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 shutdown_connections 2024-04-25T10:01:45.359 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:45.353+0000 7fcf46a00640 1 -- 172.21.15.5:0/3037915989 wait complete. 2024-04-25T10:01:45.371 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-25T10:01:52.179 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:52.367 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.363+0000 7fb141400640 1 Processor -- start 2024-04-25T10:01:52.367 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.363+0000 7fb141400640 1 -- start start 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb141400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 0x7fb13c0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb141400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb13c06d4e0 0x7fb13c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb141400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13c0ccf80 con 0x7fb13c0ce6f0 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb141400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb13c071c40 con 0x7fb13c06d4e0 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb141400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c071e90 con 0x7fb13c06ee50 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb13a800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb13c0ce6f0 0x7fb13c0cead0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45380/0 (socket says 172.21.15.5:45380) 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb13a800640 1 -- 172.21.15.5:0/3869946916 learned_addr learned my addr 172.21.15.5:0/3869946916 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb139e00640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 0x7fb13c0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719600166 0 0) 0x7fb13c0ccf80 con 0x7fb13c0ce6f0 2024-04-25T10:01:52.371 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb120003520 con 0x7fb13c0ce6f0 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.365+0000 7fb13b200640 1 --2- 172.21.15.5:0/3869946916 >> v2:172.21.15.80:3300/0 conn(0x7fb13c06d4e0 0x7fb13c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2577430322 0 0) 0x7fb120003520 con 0x7fb13c0ce6f0 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 msgr2=0x7fb13c0cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 0x7fb13c0cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 >> v2:172.21.15.80:3300/0 conn(0x7fb13c06d4e0 msgr2=0x7fb13c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 --2- 172.21.15.5:0/3869946916 >> v2:172.21.15.80:3300/0 conn(0x7fb13c06d4e0 0x7fb13c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c072170 con 0x7fb13c0ce6f0 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb12c001670 con 0x7fb13c0ce6f0 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 >> v1:172.21.15.5:6789/0 conn(0x7fb13c0ce6f0 legacy=0x7fb13c0cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1200039b0 0x7fb120003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb120004800 0x7fb120004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 0x7fb1200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c071c40 con 0x7fb1200039b0 2024-04-25T10:01:52.372 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c071e90 con 0x7fb120004800 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12000b9c0 con 0x7fb13c06ee50 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb13b200640 1 --2- 172.21.15.5:0/3869946916 >> v2:172.21.15.80:3300/0 conn(0x7fb13c06d4e0 0x7fb13c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.366+0000 7fb13a800640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1200039b0 0x7fb120003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.367+0000 7fb13a800640 1 -- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 msgr2=0x7fb1200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.367+0000 7fb13a800640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb13c06ee50 0x7fb1200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.367+0000 7fb13a800640 1 -- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb120004800 msgr2=0x7fb120004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.367+0000 7fb139e00640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb120004800 0x7fb120004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.367+0000 7fb13a800640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb120004800 0x7fb120004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.373 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.367+0000 7fb13a800640 1 -- 172.21.15.5:0/3869946916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120003520 con 0x7fb1200039b0 2024-04-25T10:01:52.374 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.368+0000 7fb13a800640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1200039b0 0x7fb120003dd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c210 tx=0x7fb120005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86579ff62c317474 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.374 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.368+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c007d50 con 0x7fb1200039b0 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.368+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c005af0 con 0x7fb1200039b0 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.368+0000 7fb139400640 1 -- 172.21.15.5:0/3869946916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c007070 con 0x7fb1200039b0 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb141400640 1 -- 172.21.15.5:0/3869946916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1200039b0 msgr2=0x7fb120003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb141400640 1 --2- 172.21.15.5:0/3869946916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1200039b0 0x7fb120003dd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00c210 tx=0x7fb120005830 comp rx=0 tx=0).stop 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb13b200640 1 -- 172.21.15.5:0/3869946916 reap_dead start 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb141400640 1 -- 172.21.15.5:0/3869946916 shutdown_connections 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb141400640 1 -- 172.21.15.5:0/3869946916 >> 172.21.15.5:0/3869946916 conn(0x7fb13c0b12b0 msgr2=0x7fb13c0b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb141400640 1 -- 172.21.15.5:0/3869946916 shutdown_connections 2024-04-25T10:01:52.375 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.369+0000 7fb141400640 1 -- 172.21.15.5:0/3869946916 wait complete. 2024-04-25T10:01:52.376 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.371+0000 7fb141400640 1 Processor -- start 2024-04-25T10:01:52.376 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.371+0000 7fb141400640 1 -- start start 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.372+0000 7fb141400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1200039b0 0x7fb13c1f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.372+0000 7fb141400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 0x7fb13c158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.372+0000 7fb141400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb13c159050 0x7fb13c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.372+0000 7fb141400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c15caa0 con 0x7fb13c14a4e0 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.372+0000 7fb141400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c0aee40 con 0x7fb13c159050 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.372+0000 7fb141400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13c15cc20 con 0x7fb1200039b0 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 0x7fb13c158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 0x7fb13c158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35030/0 (socket says 172.21.15.5:35030) 2024-04-25T10:01:52.378 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 -- 172.21.15.5:0/637829726 learned_addr learned my addr 172.21.15.5:0/637829726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb13a800640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1200039b0 0x7fb13c1f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb13b200640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb13c159050 0x7fb13c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 -- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1200039b0 msgr2=0x7fb13c1f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1200039b0 0x7fb13c1f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 -- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb13c159050 msgr2=0x7fb13c1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb13c159050 0x7fb13c1594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 -- 172.21.15.5:0/637829726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c157290 con 0x7fb13c14a4e0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.373+0000 7fb139e00640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 0x7fb13c158b10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1280108c0 tx=0x7fb128010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.374+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800f040 con 0x7fb13c14a4e0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.374+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12800eae0 con 0x7fb13c14a4e0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.374+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12801c400 con 0x7fb13c14a4e0 2024-04-25T10:01:52.379 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.374+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13c157570 con 0x7fb13c14a4e0 2024-04-25T10:01:52.380 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.374+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13c154c90 con 0x7fb13c14a4e0 2024-04-25T10:01:52.380 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.375+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13c0aeff0 con 0x7fb13c14a4e0 2024-04-25T10:01:52.380 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.376+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb12801c5a0 con 0x7fb13c14a4e0 2024-04-25T10:01:52.381 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.377+0000 7fb132a00640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb11c06b4d0 0x7fb11c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.381 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.377+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb12808a570 con 0x7fb13c14a4e0 2024-04-25T10:01:52.381 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.377+0000 7fb13a800640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb11c06b4d0 0x7fb11c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.382 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.379+0000 7fb13a800640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb11c06b4d0 0x7fb11c06d990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb12c033000 tx=0x7fb12c009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.386 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.382+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb13c0aeff0 con 0x7fb13c14a4e0 2024-04-25T10:01:52.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:52.396 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:52.591 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.587+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb13c04ef20 con 0x7fb13c14a4e0 2024-04-25T10:01:52.593 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.588+0000 7fb132a00640 1 -- 172.21.15.5:0/637829726 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12776 (secure 0 0 0) 0x7fb128018030 con 0x7fb13c14a4e0 2024-04-25T10:01:52.593 INFO:teuthology.misc.health.smithi005.stdout: 2024-04-25T10:01:52.593 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":18,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:01:48.706862+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:52.595 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb11c06b4d0 msgr2=0x7fb11c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb11c06b4d0 0x7fb11c06d990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb12c033000 tx=0x7fb12c009030 comp rx=0 tx=0).stop 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 msgr2=0x7fb13c158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 0x7fb13c158b10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb1280108c0 tx=0x7fb128010d80 comp rx=0 tx=0).stop 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 shutdown_connections 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb11c06b4d0 0x7fb11c06d990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb13c159050 0x7fb13c1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb13c14a4e0 0x7fb13c158b10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 --2- 172.21.15.5:0/637829726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1200039b0 0x7fb13c1f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 >> 172.21.15.5:0/637829726 conn(0x7fb13c0b12b0 msgr2=0x7fb13c0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.591+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 shutdown_connections 2024-04-25T10:01:52.596 INFO:teuthology.misc.health.smithi005.stderr:2024-04-25T10:01:52.592+0000 7fb141400640 1 -- 172.21.15.5:0/637829726 wait complete. 2024-04-25T10:01:52.610 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:01:52.610 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:01:52.611 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:52.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:52.616 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:52.616 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:52.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:52.621 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:52.621 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:52.622 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:52.629 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:52.630 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:52.630 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:52.640 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:52.641 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:01:52.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:52.765 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:01:52.766 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:01:52.784 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:01:52.788 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:01:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.823+0000 7ff9cde00640 1 Processor -- start 2024-04-25T10:01:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.823+0000 7ff9cde00640 1 -- start start 2024-04-25T10:01:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9cde00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff9c80d0a50 0x7ff9c80d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9cde00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c80d1e80 con 0x7ff9c80b53b0 2024-04-25T10:01:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9cde00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c80b5930 con 0x7ff9c815db00 2024-04-25T10:01:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9cde00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c80b5c10 con 0x7ff9c80d0a50 2024-04-25T10:01:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9c7400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff9c80b53b0 0x7ff9c80d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45394/0 (socket says 172.21.15.5:45394) 2024-04-25T10:01:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9c7400640 1 -- 172.21.15.5:0/2392629347 learned_addr learned my addr 172.21.15.5:0/2392629347 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.824+0000 7ff9be000640 1 --2- 172.21.15.5:0/2392629347 >> v2:172.21.15.80:3300/0 conn(0x7ff9c80d0a50 0x7ff9c80d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.825+0000 7ff9c6000640 1 -- 172.21.15.5:0/2392629347 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380473319 0 0) 0x7ff9c80d1e80 con 0x7ff9c80b53b0 2024-04-25T10:01:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.825+0000 7ff9c6000640 1 -- 172.21.15.5:0/2392629347 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9ac003520 con 0x7ff9c80b53b0 2024-04-25T10:01:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.825+0000 7ff9c6000640 1 -- 172.21.15.5:0/2392629347 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672966029 0 0) 0x7ff9c80b5930 con 0x7ff9c815db00 2024-04-25T10:01:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.825+0000 7ff9be000640 1 -- 172.21.15.5:0/2392629347 >> v1:172.21.15.80:6789/0 conn(0x7ff9c815db00 legacy=0x7ff9c80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.825+0000 7ff9be000640 1 -- 172.21.15.5:0/2392629347 >> v1:172.21.15.5:6789/0 conn(0x7ff9c80b53b0 legacy=0x7ff9c80d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.825+0000 7ff9be000640 1 -- 172.21.15.5:0/2392629347 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c80b5ef0 con 0x7ff9c80d0a50 2024-04-25T10:01:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.826+0000 7ff9be000640 1 --2- 172.21.15.5:0/2392629347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80d2330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00097a0 tx=0x7ff9b002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b154203b4ee852f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9c6000640 1 -- 172.21.15.5:0/2392629347 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0037070 con 0x7ff9c80d0a50 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9c6000640 1 -- 172.21.15.5:0/2392629347 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0032440 con 0x7ff9c80d0a50 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9c6000640 1 -- 172.21.15.5:0/2392629347 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0032810 con 0x7ff9c80d0a50 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9cde00640 1 -- 172.21.15.5:0/2392629347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 msgr2=0x7ff9c80d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9cde00640 1 --2- 172.21.15.5:0/2392629347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80d2330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00097a0 tx=0x7ff9b002a230 comp rx=0 tx=0).stop 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9cde00640 1 -- 172.21.15.5:0/2392629347 shutdown_connections 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9cde00640 1 --2- 172.21.15.5:0/2392629347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80d2330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.827+0000 7ff9cde00640 1 -- 172.21.15.5:0/2392629347 >> 172.21.15.5:0/2392629347 conn(0x7ff9c8072290 msgr2=0x7ff9c80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.828+0000 7ff9cde00640 1 -- 172.21.15.5:0/2392629347 shutdown_connections 2024-04-25T10:01:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.828+0000 7ff9cde00640 1 -- 172.21.15.5:0/2392629347 wait complete. 2024-04-25T10:01:52.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.829+0000 7ff9cde00640 1 Processor -- start 2024-04-25T10:01:52.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.829+0000 7ff9cde00640 1 -- start start 2024-04-25T10:01:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.830+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 0x7ff9c814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.830+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.830+0000 7ff9cde00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9c815db00 0x7ff9c80b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.830+0000 7ff9cde00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c80d3660 con 0x7ff9c80b53b0 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.830+0000 7ff9cde00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c80b61d0 con 0x7ff9c80d0a50 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.830+0000 7ff9cde00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c80d2f80 con 0x7ff9c815db00 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 0x7ff9c814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 0x7ff9c814e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35058/0 (socket says 172.21.15.5:35058) 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 -- 172.21.15.5:0/1433364405 learned_addr learned my addr 172.21.15.5:0/1433364405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9be000640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c7400640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9c815db00 0x7ff9c80b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 -- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9c815db00 msgr2=0x7ff9c80b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9c815db00 0x7ff9c80b8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 -- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 msgr2=0x7ff9c80b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.831+0000 7ff9c6a00640 1 -- 172.21.15.5:0/1433364405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c80cd130 con 0x7ff9c80b53b0 2024-04-25T10:01:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.832+0000 7ff9c6a00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 0x7ff9c814e230 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800c7b0 tx=0x7ff9b800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.832+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8003da0 con 0x7ff9c80b53b0 2024-04-25T10:01:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.832+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b8005070 con 0x7ff9c80b53b0 2024-04-25T10:01:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.832+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c80cd410 con 0x7ff9c80b53b0 2024-04-25T10:01:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.832+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8005420 con 0x7ff9c80b53b0 2024-04-25T10:01:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.832+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c80c7460 con 0x7ff9c80b53b0 2024-04-25T10:01:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.833+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c8068fd0 con 0x7ff9c80b53b0 2024-04-25T10:01:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.834+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9b80055c0 con 0x7ff9c80b53b0 2024-04-25T10:01:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.835+0000 7ff9bfe00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff9a406b500 0x7ff9a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.835+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9b8089cc0 con 0x7ff9c80b53b0 2024-04-25T10:01:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.836+0000 7ff9be000640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff9a406b500 0x7ff9a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.836+0000 7ff9be000640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff9a406b500 0x7ff9a406d9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0034000 tx=0x7ff9b002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:52.840+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9b8090080 con 0x7ff9c80b53b0 2024-04-25T10:01:53.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.040+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9c80d16b0 con 0x7ff9c80b53b0 2024-04-25T10:01:53.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.041+0000 7ff9bfe00640 1 -- 172.21.15.5:0/1433364405 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12776 (secure 0 0 0) 0x7ff9b8058830 con 0x7ff9c80b53b0 2024-04-25T10:01:53.045 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:01:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:01:48.706862+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff9a406b500 msgr2=0x7ff9a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff9a406b500 0x7ff9a406d9c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0034000 tx=0x7ff9b002e000 comp rx=0 tx=0).stop 2024-04-25T10:01:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 msgr2=0x7ff9c814e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 0x7ff9c814e230 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800c7b0 tx=0x7ff9b800cc70 comp rx=0 tx=0).stop 2024-04-25T10:01:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 shutdown_connections 2024-04-25T10:01:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9c815db00 0x7ff9c80b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff9a406b500 0x7ff9a406d9c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c80d0a50 0x7ff9c80b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 --2- 172.21.15.5:0/1433364405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80b53b0 0x7ff9c814e230 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 >> 172.21.15.5:0/1433364405 conn(0x7ff9c8072290 msgr2=0x7ff9c815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 shutdown_connections 2024-04-25T10:01:53.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.044+0000 7ff9cde00640 1 -- 172.21.15.5:0/1433364405 wait complete. 2024-04-25T10:01:53.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:53.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:53.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.265+0000 7f8eb3e00640 1 Processor -- start 2024-04-25T10:01:53.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.266+0000 7f8eb3e00640 1 -- start start 2024-04-25T10:01:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8eac071670 0x7f8eac15c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eac1453c0 con 0x7f8eac144fe0 2024-04-25T10:01:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eac0a8560 con 0x7f8eac071670 2024-04-25T10:01:53.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac06d6a0 con 0x7f8eac06d130 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb0a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb1e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8eac071670 0x7f8eac15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb0a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:47930/0 (socket says 172.21.15.5:47930) 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.268+0000 7f8eb0a00640 1 -- 172.21.15.5:0/2744847813 learned_addr learned my addr 172.21.15.5:0/2744847813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.269+0000 7f8eb0a00640 1 -- 172.21.15.5:0/2744847813 >> v2:172.21.15.80:3300/0 conn(0x7f8eac071670 msgr2=0x7f8eac15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.269+0000 7f8eb0a00640 1 --2- 172.21.15.5:0/2744847813 >> v2:172.21.15.80:3300/0 conn(0x7f8eac071670 0x7f8eac15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.269+0000 7f8eb0a00640 1 -- 172.21.15.5:0/2744847813 >> v1:172.21.15.80:6789/0 conn(0x7f8eac144fe0 legacy=0x7f8eac06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.269+0000 7f8eb0a00640 1 -- 172.21.15.5:0/2744847813 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac06d980 con 0x7f8eac06d130 2024-04-25T10:01:53.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.270+0000 7f8eb0a00640 1 --2- 172.21.15.5:0/2744847813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac071130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009870 tx=0x7f8e9802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c81b9d71fb5516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.271+0000 7f8eabe00640 1 -- 172.21.15.5:0/2744847813 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98037070 con 0x7f8eac06d130 2024-04-25T10:01:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.271+0000 7f8eabe00640 1 -- 172.21.15.5:0/2744847813 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98032440 con 0x7f8eac06d130 2024-04-25T10:01:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.271+0000 7f8eabe00640 1 -- 172.21.15.5:0/2744847813 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98032810 con 0x7f8eac06d130 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.271+0000 7f8eb3e00640 1 -- 172.21.15.5:0/2744847813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 msgr2=0x7f8eac071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.271+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/2744847813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac071130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009870 tx=0x7f8e9802a270 comp rx=0 tx=0).stop 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.272+0000 7f8eb3e00640 1 -- 172.21.15.5:0/2744847813 shutdown_connections 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.272+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/2744847813 >> v2:172.21.15.80:3300/0 conn(0x7f8eac071670 0x7f8eac15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.272+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/2744847813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac071130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.272+0000 7f8eb3e00640 1 -- 172.21.15.5:0/2744847813 >> 172.21.15.5:0/2744847813 conn(0x7f8eac06ab60 msgr2=0x7f8eac06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.272+0000 7f8eb3e00640 1 -- 172.21.15.5:0/2744847813 shutdown_connections 2024-04-25T10:01:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.272+0000 7f8eb3e00640 1 -- 172.21.15.5:0/2744847813 wait complete. 2024-04-25T10:01:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.273+0000 7f8eb3e00640 1 Processor -- start 2024-04-25T10:01:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.274+0000 7f8eb3e00640 1 -- start start 2024-04-25T10:01:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.274+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac0c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 0x7f8eac0aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eac144fe0 0x7f8eac1369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac15be50 con 0x7f8eac071670 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac06df40 con 0x7f8eac144fe0 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eac15c8b0 con 0x7f8eac06d130 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 0x7f8eac0aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 0x7f8eac0aabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35066/0 (socket says 172.21.15.5:35066) 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 -- 172.21.15.5:0/1780178175 learned_addr learned my addr 172.21.15.5:0/1780178175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb1400640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac0c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb1e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eac144fe0 0x7f8eac1369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 -- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 msgr2=0x7f8eac0c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac0c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 -- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eac144fe0 msgr2=0x7f8eac1369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eac144fe0 0x7f8eac1369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.275+0000 7f8eb0a00640 1 -- 172.21.15.5:0/1780178175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eac137880 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.276+0000 7f8eb0a00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 0x7f8eac0aabd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.276+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98037040 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.276+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98042070 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.276+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98030880 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.276+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eac0c4bc0 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.276+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eac0aa3d0 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.278+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eac068f60 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.279+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e98030a20 con 0x7f8eac071670 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.279+0000 7f8ea9600640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.280+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e980aa710 con 0x7f8eac071670 2024-04-25T10:01:53.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.280+0000 7f8eb1400640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.281+0000 7f8eb1400640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0007fd0 tx=0x7f8ea0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.284+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8eac137880 con 0x7f8eac071670 2024-04-25T10:01:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.484+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8eac15c310 con 0x7f8eac071670 2024-04-25T10:01:53.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.485+0000 7f8ea9600640 1 -- 172.21.15.5:0/1780178175 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e980798d0 con 0x7f8eac071670 2024-04-25T10:01:53.489 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e8c06b4d0 msgr2=0x7f8e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0007fd0 tx=0x7f8ea0009040 comp rx=0 tx=0).stop 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 msgr2=0x7f8eac0aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 0x7f8eac0aabd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e980046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 shutdown_connections 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eac144fe0 0x7f8eac1369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eac071670 0x7f8eac0aabd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 --2- 172.21.15.5:0/1780178175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eac06d130 0x7f8eac0c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.487+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 >> 172.21.15.5:0/1780178175 conn(0x7f8eac06ab60 msgr2=0x7f8eac1400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.488+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 shutdown_connections 2024-04-25T10:01:53.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.488+0000 7f8eb3e00640 1 -- 172.21.15.5:0/1780178175 wait complete. 2024-04-25T10:01:53.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:53.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.716+0000 7f84bf200640 1 Processor -- start 2024-04-25T10:01:53.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.716+0000 7f84bf200640 1 -- start start 2024-04-25T10:01:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.718+0000 7f84bf200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f84c00cdca0 0x7f84c015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.718+0000 7f84bf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0070f90 0x7f84c0071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.718+0000 7f84bf200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c00cd200 con 0x7f84c0154a10 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.718+0000 7f84bf200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c0071940 con 0x7f84c00cdca0 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.718+0000 7f84bf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c006e6f0 con 0x7f84c0070f90 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.719+0000 7f84bd400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f84c00cdca0 0x7f84c015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.719+0000 7f84be800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0070f90 0x7f84c0071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.719+0000 7f84bd400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f84c00cdca0 0x7f84c015e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:51156/0 (socket says 172.21.15.5:51156) 2024-04-25T10:01:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.719+0000 7f84bd400640 1 -- 172.21.15.5:0/503043151 learned_addr learned my addr 172.21.15.5:0/503043151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.720+0000 7f84bd400640 1 -- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0070f90 msgr2=0x7f84c0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.720+0000 7f84bd400640 1 --2- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0070f90 0x7f84c0071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.720+0000 7f84bd400640 1 -- 172.21.15.5:0/503043151 >> v1:172.21.15.80:6789/0 conn(0x7f84c0154a10 legacy=0x7f84c00b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.720+0000 7f84bd400640 1 -- 172.21.15.5:0/503043151 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c006e9d0 con 0x7f84c00cdca0 2024-04-25T10:01:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.721+0000 7f84bd400640 1 --2- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c015e8c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009870 tx=0x7f84ac02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657d8bc914d3f373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.722+0000 7f84bca00640 1 -- 172.21.15.5:0/503043151 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac037070 con 0x7f84c00cdca0 2024-04-25T10:01:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.722+0000 7f84bca00640 1 -- 172.21.15.5:0/503043151 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ac032440 con 0x7f84c00cdca0 2024-04-25T10:01:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.722+0000 7f84bca00640 1 -- 172.21.15.5:0/503043151 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac032810 con 0x7f84c00cdca0 2024-04-25T10:01:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.722+0000 7f84bf200640 1 -- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 msgr2=0x7f84c015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.722+0000 7f84bf200640 1 --2- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c015e8c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009870 tx=0x7f84ac02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.723+0000 7f84bf200640 1 -- 172.21.15.5:0/503043151 shutdown_connections 2024-04-25T10:01:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.723+0000 7f84bf200640 1 --2- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0070f90 0x7f84c0071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.723+0000 7f84bf200640 1 --2- 172.21.15.5:0/503043151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c015e8c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.723+0000 7f84bf200640 1 -- 172.21.15.5:0/503043151 >> 172.21.15.5:0/503043151 conn(0x7f84c0066f20 msgr2=0x7f84c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.723+0000 7f84bf200640 1 -- 172.21.15.5:0/503043151 shutdown_connections 2024-04-25T10:01:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.723+0000 7f84bf200640 1 -- 172.21.15.5:0/503043151 wait complete. 2024-04-25T10:01:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.724+0000 7f84bf200640 1 Processor -- start 2024-04-25T10:01:53.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 -- start start 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 0x7f84c0155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c0147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0154a10 0x7f84c015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00cd780 con 0x7f84c0070f90 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c00cd4a0 con 0x7f84c00cdca0 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.725+0000 7f84bf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c006cff0 con 0x7f84c0154a10 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 0x7f84c0155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 0x7f84c0155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35072/0 (socket says 172.21.15.5:35072) 2024-04-25T10:01:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 -- 172.21.15.5:0/1456314123 learned_addr learned my addr 172.21.15.5:0/1456314123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84be800640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0154a10 0x7f84c015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bd400640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c0147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 -- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0154a10 msgr2=0x7f84c015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0154a10 0x7f84c015f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 -- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 msgr2=0x7f84c0147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c0147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bde00640 1 -- 172.21.15.5:0/1456314123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0148140 con 0x7f84c0070f90 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.726+0000 7f84bd400640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c0147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.727+0000 7f84bde00640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 0x7f84c0155970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c400 tx=0x7f84b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.727+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0017070 con 0x7f84c0070f90 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.727+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b00052d0 con 0x7f84c0070f90 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.727+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0003830 con 0x7f84c0070f90 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.727+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c015fe00 con 0x7f84c0070f90 2024-04-25T10:01:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.727+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0154e90 con 0x7f84c0070f90 2024-04-25T10:01:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.728+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c0068f60 con 0x7f84c0070f90 2024-04-25T10:01:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.729+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84b001a070 con 0x7f84c0070f90 2024-04-25T10:01:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.731+0000 7f84b6000640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84a006b500 0x7f84a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.731+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84b0089210 con 0x7f84c0070f90 2024-04-25T10:01:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.731+0000 7f84bd400640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84a006b500 0x7f84a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.733+0000 7f84bd400640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84a006b500 0x7f84a006d9c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84ac034000 tx=0x7f84ac02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.736+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84b00583d0 con 0x7f84c0070f90 2024-04-25T10:01:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.978+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c0162190 con 0x7f84c0070f90 2024-04-25T10:01:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.978+0000 7f84b6000640 1 -- 172.21.15.5:0/1456314123 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3685 (secure 0 0 0) 0x7f84b0057cd0 con 0x7f84c0070f90 2024-04-25T10:01:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:01:53.983 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:01:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84a006b500 msgr2=0x7f84a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84a006b500 0x7f84a006d9c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84ac034000 tx=0x7f84ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 msgr2=0x7f84c0155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 0x7f84c0155970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c400 tx=0x7f84b000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 shutdown_connections 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84c0154a10 0x7f84c015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84a006b500 0x7f84a006d9c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.981+0000 7f84bf200640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84c00cdca0 0x7f84c0147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.982+0000 7f84bf200640 1 --2- 172.21.15.5:0/1456314123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c0070f90 0x7f84c0155970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.982+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 >> 172.21.15.5:0/1456314123 conn(0x7f84c0066f20 msgr2=0x7f84c00aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.982+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 shutdown_connections 2024-04-25T10:01:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:53.982+0000 7f84bf200640 1 -- 172.21.15.5:0/1456314123 wait complete. 2024-04-25T10:01:54.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.211+0000 7f3f57e00640 1 Processor -- start 2024-04-25T10:01:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.211+0000 7f3f57e00640 1 -- start start 2024-04-25T10:01:54.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.212+0000 7f3f57e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f5815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.212+0000 7f3f57e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f58071d30 con 0x7f3f58141da0 2024-04-25T10:01:54.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f57e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5806d620 con 0x7f3f580cd300 2024-04-25T10:01:54.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f57e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58142180 con 0x7f3f5815ce00 2024-04-25T10:01:54.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f56a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3f58141da0 0x7f3f5806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45406/0 (socket says 172.21.15.5:45406) 2024-04-25T10:01:54.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f56a00640 1 -- 172.21.15.5:0/4012171809 learned_addr learned my addr 172.21.15.5:0/4012171809 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f5815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262868902 0 0) 0x7f3f58071d30 con 0x7f3f58141da0 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.213+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f3c003520 con 0x7f3f58141da0 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205663321 0 0) 0x7f3f3c003520 con 0x7f3f58141da0 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 msgr2=0x7f3f5815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f5815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 >> v1:172.21.15.80:6789/0 conn(0x7f3f580cd300 legacy=0x7f3f58071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5806d900 con 0x7f3f58141da0 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f3f48001670 con 0x7f3f58141da0 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 >> v1:172.21.15.5:6789/0 conn(0x7f3f58141da0 legacy=0x7f3f5806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f3c0039e0 0x7f3f3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f3c004800 0x7f3f3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58142180 con 0x7f3f3c004800 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c00baa0 con 0x7f3f3c0039e0 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c00bca0 con 0x7f3f5815ce00 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.214+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f3c004800 0x7f3f3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f56a00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f3c0039e0 0x7f3f3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f57400640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f4fe00640 1 -- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 msgr2=0x7f3f3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f4fe00640 1 -- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f3c0039e0 msgr2=0x7f3f3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f3c0039e0 0x7f3f3c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f4fe00640 1 -- 172.21.15.5:0/4012171809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c003520 con 0x7f3f3c004800 2024-04-25T10:01:54.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.215+0000 7f3f57400640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f5815ce00 0x7f3f3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.216+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f3c004800 0x7f3f3c004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009a90 tx=0x7f3f3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fb5bc6d0a0cab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.216+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40036070 con 0x7f3f3c004800 2024-04-25T10:01:54.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.216+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40031400 con 0x7f3f3c004800 2024-04-25T10:01:54.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.216+0000 7f3f56000640 1 -- 172.21.15.5:0/4012171809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40009690 con 0x7f3f3c004800 2024-04-25T10:01:54.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57e00640 1 -- 172.21.15.5:0/4012171809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f3c004800 msgr2=0x7f3f3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57e00640 1 --2- 172.21.15.5:0/4012171809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f3c004800 0x7f3f3c004c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009a90 tx=0x7f3f3c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57400640 1 -- 172.21.15.5:0/4012171809 reap_dead start 2024-04-25T10:01:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57e00640 1 -- 172.21.15.5:0/4012171809 shutdown_connections 2024-04-25T10:01:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57e00640 1 -- 172.21.15.5:0/4012171809 >> 172.21.15.5:0/4012171809 conn(0x7f3f58066f20 msgr2=0x7f3f5806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57e00640 1 -- 172.21.15.5:0/4012171809 shutdown_connections 2024-04-25T10:01:54.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.217+0000 7f3f57e00640 1 -- 172.21.15.5:0/4012171809 wait complete. 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.219+0000 7f3f57e00640 1 Processor -- start 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.219+0000 7f3f57e00640 1 -- start start 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f3c004800 0x7f3f5813aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 0x7f3f5814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f5814dce0 0x7f3f581436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58071710 con 0x7f3f580b11f0 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5806e810 con 0x7f3f5814dce0 2024-04-25T10:01:54.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58071890 con 0x7f3f3c004800 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 0x7f3f5814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 0x7f3f5814d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35092/0 (socket says 172.21.15.5:35092) 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 -- 172.21.15.5:0/2377682048 learned_addr learned my addr 172.21.15.5:0/2377682048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f56a00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f3c004800 0x7f3f5813aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f57400640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f5814dce0 0x7f3f581436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 -- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f3c004800 msgr2=0x7f3f5813aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f3c004800 0x7f3f5813aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 -- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f5814dce0 msgr2=0x7f3f581436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.220+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f5814dce0 0x7f3f581436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f4fe00640 1 -- 172.21.15.5:0/2377682048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58142330 con 0x7f3f580b11f0 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f57400640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f5814dce0 0x7f3f581436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f56a00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f3c004800 0x7f3f5813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f4fe00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 0x7f3f5814d7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f40034040 tx=0x7f3f40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40040020 con 0x7f3f580b11f0 2024-04-25T10:01:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40003bc0 con 0x7f3f580b11f0 2024-04-25T10:01:54.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f58142610 con 0x7f3f580b11f0 2024-04-25T10:01:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40036040 con 0x7f3f580b11f0 2024-04-25T10:01:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.221+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58142bd0 con 0x7f3f580b11f0 2024-04-25T10:01:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.223+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f58068f60 con 0x7f3f580b11f0 2024-04-25T10:01:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.223+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f40043020 con 0x7f3f580b11f0 2024-04-25T10:01:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.224+0000 7f3f4ea00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f3406b500 0x7f3f3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.224+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f400aa2f0 con 0x7f3f580b11f0 2024-04-25T10:01:54.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.225+0000 7f3f56a00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f3406b500 0x7f3f3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.226+0000 7f3f56a00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f3406b500 0x7f3f3406d9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f48001bf0 tx=0x7f3f48000f50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.229+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f4004a030 con 0x7f3f580b11f0 2024-04-25T10:01:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.430+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f580cc860 con 0x7f3f580b11f0 2024-04-25T10:01:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.433+0000 7f3f4ea00640 1 -- 172.21.15.5:0/2377682048 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12776 (secure 0 0 0) 0x7f3f4007de20 con 0x7f3f580b11f0 2024-04-25T10:01:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:01:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:01:48.706862+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.435+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f3406b500 msgr2=0x7f3f3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.435+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f3406b500 0x7f3f3406d9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f48001bf0 tx=0x7f3f48000f50 comp rx=0 tx=0).stop 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.435+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 msgr2=0x7f3f5814d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.435+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 0x7f3f5814d7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f40034040 tx=0x7f3f40004730 comp rx=0 tx=0).stop 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 shutdown_connections 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f3406b500 0x7f3f3406d9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f5814dce0 0x7f3f581436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f580b11f0 0x7f3f5814d7a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2377682048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3f3c004800 0x7f3f5813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 >> 172.21.15.5:0/2377682048 conn(0x7f3f58066f20 msgr2=0x7f3f580b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 shutdown_connections 2024-04-25T10:01:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.436+0000 7f3f57e00640 1 -- 172.21.15.5:0/2377682048 wait complete. 2024-04-25T10:01:54.459 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:54.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.665+0000 7f2c67200640 1 Processor -- start 2024-04-25T10:01:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.665+0000 7f2c67200640 1 -- start start 2024-04-25T10:01:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.667+0000 7f2c67200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 0x7f2c680cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.667+0000 7f2c67200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2c6815ce20 0x7f2c6815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.667+0000 7f2c67200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c68144cd0 con 0x7f2c6815bc30 2024-04-25T10:01:54.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.667+0000 7f2c67200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6815f460 con 0x7f2c6815ce20 2024-04-25T10:01:54.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.667+0000 7f2c67200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6815c440 con 0x7f2c681448f0 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.668+0000 7f2c5d400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2c6815bc30 0x7f2c6815f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45414/0 (socket says 172.21.15.5:45414) 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.668+0000 7f2c5d400640 1 -- 172.21.15.5:0/3338935849 learned_addr learned my addr 172.21.15.5:0/3338935849 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.668+0000 7f2c66800640 1 --2- 172.21.15.5:0/3338935849 >> v2:172.21.15.80:3300/0 conn(0x7f2c6815ce20 0x7f2c6815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.668+0000 7f2c65e00640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 0x7f2c680cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.668+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626071851 0 0) 0x7f2c68144cd0 con 0x7f2c6815bc30 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.668+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c4c003520 con 0x7f2c6815bc30 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2264116804 0 0) 0x7f2c4c003520 con 0x7f2c6815bc30 2024-04-25T10:01:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 msgr2=0x7f2c680cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 0x7f2c680cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 >> v2:172.21.15.80:3300/0 conn(0x7f2c6815ce20 msgr2=0x7f2c6815d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 --2- 172.21.15.5:0/3338935849 >> v2:172.21.15.80:3300/0 conn(0x7f2c6815ce20 0x7f2c6815d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6815c720 con 0x7f2c6815bc30 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2c500019f0 con 0x7f2c6815bc30 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 >> v1:172.21.15.5:6789/0 conn(0x7f2c6815bc30 legacy=0x7f2c6815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c4c0039b0 0x7f2c4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 0x7f2c4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6815f460 con 0x7f2c4c004800 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6815c440 con 0x7f2c4c0039b0 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c00fdd0 con 0x7f2c681448f0 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c5d400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c65e00640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c4c0039b0 0x7f2c4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c5d400640 1 -- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 msgr2=0x7f2c4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c66800640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 0x7f2c4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.669+0000 7f2c5d400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c681448f0 0x7f2c4c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.670+0000 7f2c5d400640 1 -- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c4c0039b0 msgr2=0x7f2c4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.670+0000 7f2c5d400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c4c0039b0 0x7f2c4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.670+0000 7f2c5d400640 1 -- 172.21.15.5:0/3338935849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c003520 con 0x7f2c4c004800 2024-04-25T10:01:54.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.670+0000 7f2c65e00640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c4c0039b0 0x7f2c4c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c5d400640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c4c004c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000a7b0 tx=0x7f2c4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd17d6dbdbef094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5000a3d0 con 0x7f2c4c004800 2024-04-25T10:01:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5002f070 con 0x7f2c4c004800 2024-04-25T10:01:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c65400640 1 -- 172.21.15.5:0/3338935849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5002bc10 con 0x7f2c4c004800 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c67200640 1 -- 172.21.15.5:0/3338935849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 msgr2=0x7f2c4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c67200640 1 --2- 172.21.15.5:0/3338935849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c4c004c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c5000a7b0 tx=0x7f2c4c005830 comp rx=0 tx=0).stop 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.671+0000 7f2c66800640 1 -- 172.21.15.5:0/3338935849 reap_dead start 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.672+0000 7f2c67200640 1 -- 172.21.15.5:0/3338935849 shutdown_connections 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.672+0000 7f2c67200640 1 -- 172.21.15.5:0/3338935849 >> 172.21.15.5:0/3338935849 conn(0x7f2c68066f20 msgr2=0x7f2c6806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.672+0000 7f2c67200640 1 -- 172.21.15.5:0/3338935849 shutdown_connections 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.672+0000 7f2c67200640 1 -- 172.21.15.5:0/3338935849 wait complete. 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.673+0000 7f2c67200640 1 Processor -- start 2024-04-25T10:01:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.674+0000 7f2c67200640 1 -- start start 2024-04-25T10:01:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.674+0000 7f2c67200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c6813a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c67200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c680b1500 0x7f2c681ef510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c67200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c680b1980 0x7f2c6815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c67200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6815c090 con 0x7f2c4c004800 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c67200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68160000 con 0x7f2c680b1980 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c67200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68139490 con 0x7f2c680b1500 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c6813a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c6813a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35120/0 (socket says 172.21.15.5:35120) 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 -- 172.21.15.5:0/1418484405 learned_addr learned my addr 172.21.15.5:0/1418484405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c5d400640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c680b1500 0x7f2c681ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c66800640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c680b1980 0x7f2c6815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 -- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c680b1500 msgr2=0x7f2c681ef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c680b1500 0x7f2c681ef510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 -- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c680b1980 msgr2=0x7f2c6815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c680b1980 0x7f2c6815abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c65e00640 1 -- 172.21.15.5:0/1418484405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68138880 con 0x7f2c4c004800 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.675+0000 7f2c66800640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c680b1980 0x7f2c6815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.676+0000 7f2c65e00640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c6813a960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c580108c0 tx=0x7f2c58010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.676+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800f040 con 0x7f2c4c004800 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.676+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5800eb00 con 0x7f2c4c004800 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.676+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c68138b60 con 0x7f2c4c004800 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.676+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5801c400 con 0x7f2c4c004800 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.676+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c68139120 con 0x7f2c4c004800 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.677+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c68068f60 con 0x7f2c4c004800 2024-04-25T10:01:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.678+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c5801c5a0 con 0x7f2c4c004800 2024-04-25T10:01:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.680+0000 7f2c5ea00640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c4406b4d0 0x7f2c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.680+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2c5808a540 con 0x7f2c4c004800 2024-04-25T10:01:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.680+0000 7f2c5d400640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c4406b4d0 0x7f2c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.682+0000 7f2c5d400640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c4406b4d0 0x7f2c4406d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c50033000 tx=0x7f2c50007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.684+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c58018030 con 0x7f2c4c004800 2024-04-25T10:01:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.881+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c68072180 con 0x7f2c4c004800 2024-04-25T10:01:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.882+0000 7f2c5ea00640 1 -- 172.21.15.5:0/1418484405 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c580595e0 con 0x7f2c4c004800 2024-04-25T10:01:54.886 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:01:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c4406b4d0 msgr2=0x7f2c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c4406b4d0 0x7f2c4406d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c50033000 tx=0x7f2c50007000 comp rx=0 tx=0).stop 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 msgr2=0x7f2c6813a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c6813a960 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2c580108c0 tx=0x7f2c58010d80 comp rx=0 tx=0).stop 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 shutdown_connections 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c4406b4d0 0x7f2c4406d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c680b1980 0x7f2c6815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c680b1500 0x7f2c681ef510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 --2- 172.21.15.5:0/1418484405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c4c004800 0x7f2c6813a960 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 >> 172.21.15.5:0/1418484405 conn(0x7f2c68066f20 msgr2=0x7f2c680aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 shutdown_connections 2024-04-25T10:01:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:54.885+0000 7f2c67200640 1 -- 172.21.15.5:0/1418484405 wait complete. 2024-04-25T10:01:54.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:01:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.116+0000 7fbdfb200640 1 Processor -- start 2024-04-25T10:01:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.117+0000 7fbdfb200640 1 -- start start 2024-04-25T10:01:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.118+0000 7fbdfb200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdfc0d0570 0x7fbdfc0d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.118+0000 7fbdfb200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdfc0ce6f0 con 0x7fbdfc0d1590 2024-04-25T10:01:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.118+0000 7fbdfb200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdfc15ea10 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.118+0000 7fbdfb200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdfc0cea20 con 0x7fbdfc0d0570 2024-04-25T10:01:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.118+0000 7fbdf1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdfc0d0570 0x7fbdfc0d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf9e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbdfc0d1590 0x7fbdfc0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45424/0 (socket says 172.21.15.5:45424) 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf9e00640 1 -- 172.21.15.5:0/13729189 learned_addr learned my addr 172.21.15.5:0/13729189 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdfc0d0570 0x7fbdfc0d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35136/0 (socket says 172.21.15.5:35136) 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf1400640 1 -- 172.21.15.5:0/13729189 >> v1:172.21.15.80:6789/0 conn(0x7fbdfc15e4c0 legacy=0x7fbdfc1613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf1400640 1 -- 172.21.15.5:0/13729189 >> v1:172.21.15.5:6789/0 conn(0x7fbdfc0d1590 legacy=0x7fbdfc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf9400640 1 -- 172.21.15.5:0/13729189 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787100079 0 0) 0x7fbdfc0ce6f0 con 0x7fbdfc0d1590 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.119+0000 7fbdf1400640 1 -- 172.21.15.5:0/13729189 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc15ecf0 con 0x7fbdfc0d0570 2024-04-25T10:01:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.120+0000 7fbdf1400640 1 --2- 172.21.15.5:0/13729189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc0d2080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbde40097a0 tx=0x7fbde402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bc90028e349ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.120+0000 7fbdf9400640 1 -- 172.21.15.5:0/13729189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4037070 con 0x7fbdfc0d0570 2024-04-25T10:01:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.120+0000 7fbdf9400640 1 -- 172.21.15.5:0/13729189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde4032440 con 0x7fbdfc0d0570 2024-04-25T10:01:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.120+0000 7fbdf9400640 1 -- 172.21.15.5:0/13729189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4032810 con 0x7fbdfc0d0570 2024-04-25T10:01:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 -- 172.21.15.5:0/13729189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc0d0570 msgr2=0x7fbdfc0d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 --2- 172.21.15.5:0/13729189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc0d2080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbde40097a0 tx=0x7fbde402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 -- 172.21.15.5:0/13729189 shutdown_connections 2024-04-25T10:01:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 --2- 172.21.15.5:0/13729189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc0d2080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 -- 172.21.15.5:0/13729189 >> 172.21.15.5:0/13729189 conn(0x7fbdfc0726e0 msgr2=0x7fbdfc072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 -- 172.21.15.5:0/13729189 shutdown_connections 2024-04-25T10:01:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.121+0000 7fbdfb200640 1 -- 172.21.15.5:0/13729189 wait complete. 2024-04-25T10:01:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.123+0000 7fbdfb200640 1 Processor -- start 2024-04-25T10:01:55.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.123+0000 7fbdfb200640 1 -- start start 2024-04-25T10:01:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdfc0d1590 0x7fbdfc1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 0x7fbdfc1f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0d31c0 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0d3040 con 0x7fbdfc0d0570 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc161ad0 con 0x7fbdfc0d1590 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.124+0000 7fbdfa800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 0x7fbdfc1f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 0x7fbdfc1f6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35138/0 (socket says 172.21.15.5:35138) 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 -- 172.21.15.5:0/3124486315 learned_addr learned my addr 172.21.15.5:0/3124486315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdf9e00640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdf1400640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdfc0d1590 0x7fbdfc1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 -- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdfc0d1590 msgr2=0x7fbdfc1f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdfc0d1590 0x7fbdfc1f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 -- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdfc0d0570 msgr2=0x7fbdfc149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 -- 172.21.15.5:0/3124486315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc15bda0 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdf9e00640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.125+0000 7fbdfa800640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 0x7fbdfc1f6fc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400b970 tx=0x7fbdf400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.126+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4019070 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.126+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400cb40 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.126+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4014440 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.126+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc1f75f0 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.126+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc0c8e60 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.127+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc068080 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.128+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdf401c070 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.129+0000 7fbdf2a00640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd806b4d0 0x7fbdd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.129+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbdf4089b60 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.130+0000 7fbdf9e00640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd806b4d0 0x7fbdd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.131+0000 7fbdf9e00640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd806b4d0 0x7fbdd806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f60 tx=0x7fbdec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.134+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdf4058d30 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.334+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbdfc0d1370 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.334+0000 7fbdf2a00640 1 -- 172.21.15.5:0/3124486315 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbdf4058630 con 0x7fbdfc15e4c0 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:01:55.339 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd806b4d0 msgr2=0x7fbdd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd806b4d0 0x7fbdd806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000f60 tx=0x7fbdec009040 comp rx=0 tx=0).stop 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 msgr2=0x7fbdfc1f6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 0x7fbdfc1f6fc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400b970 tx=0x7fbdf400be30 comp rx=0 tx=0).stop 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 shutdown_connections 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdfc15e4c0 0x7fbdfc1f6fc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdfc0d1590 0x7fbdfc1f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd806b4d0 0x7fbdd806d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 --2- 172.21.15.5:0/3124486315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdfc0d0570 0x7fbdfc149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.337+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 >> 172.21.15.5:0/3124486315 conn(0x7fbdfc0726e0 msgr2=0x7fbdfc0b8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.338+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 shutdown_connections 2024-04-25T10:01:55.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.338+0000 7fbdfb200640 1 -- 172.21.15.5:0/3124486315 wait complete. 2024-04-25T10:01:55.361 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:01:55.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:01:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.572+0000 7f30f7200640 1 Processor -- start 2024-04-25T10:01:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.572+0000 7f30f7200640 1 -- start start 2024-04-25T10:01:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 0x7f30f8161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f8162290 con 0x7f30f815ef00 2024-04-25T10:01:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f7200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f80adcf0 con 0x7f30f8147080 2024-04-25T10:01:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f80adac0 con 0x7f30f8071710 2024-04-25T10:01:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f6800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f30f815ef00 0x7f30f80ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45440/0 (socket says 172.21.15.5:45440) 2024-04-25T10:01:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f6800640 1 -- 172.21.15.5:0/1174985028 learned_addr learned my addr 172.21.15.5:0/1174985028 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.574+0000 7f30f5400640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 0x7f30f8161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420004333 0 0) 0x7f30f8162290 con 0x7f30f815ef00 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30dc003520 con 0x7f30f815ef00 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1454819848 0 0) 0x7f30dc003520 con 0x7f30f815ef00 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 msgr2=0x7f30f8161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 0x7f30f8161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 >> v1:172.21.15.80:6789/0 conn(0x7f30f8147080 legacy=0x7f30f8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f815f350 con 0x7f30f815ef00 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f30f00019f0 con 0x7f30f815ef00 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 >> v1:172.21.15.5:6789/0 conn(0x7f30f815ef00 legacy=0x7f30f80ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30dc0039e0 0x7f30dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.575+0000 7f30f4a00640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 0x7f30dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f4a00640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc00b9c0 con 0x7f30dc004d40 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc00bb70 con 0x7f30dc0039e0 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc00bd70 con 0x7f30f8071710 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f6800640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f5e00640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30dc0039e0 0x7f30dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f6800640 1 -- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 msgr2=0x7f30dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f6800640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f8071710 0x7f30dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f6800640 1 -- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30dc0039e0 msgr2=0x7f30dc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f6800640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30dc0039e0 0x7f30dc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.576+0000 7f30f6800640 1 -- 172.21.15.5:0/1174985028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc003520 con 0x7f30dc004d40 2024-04-25T10:01:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.577+0000 7f30f6800640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30f000a880 tx=0x7f30dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f425d611f0d1a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.577+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000a4a0 con 0x7f30dc004d40 2024-04-25T10:01:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.577+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f002f070 con 0x7f30dc004d40 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.577+0000 7f30f4a00640 1 -- 172.21.15.5:0/1174985028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f002bce0 con 0x7f30dc004d40 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f7200640 1 -- 172.21.15.5:0/1174985028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30dc004d40 msgr2=0x7f30dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f7200640 1 --2- 172.21.15.5:0/1174985028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30dc004d40 0x7f30dc0051c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30f000a880 tx=0x7f30dc005830 comp rx=0 tx=0).stop 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f6800640 1 -- 172.21.15.5:0/1174985028 reap_dead start 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f7200640 1 -- 172.21.15.5:0/1174985028 shutdown_connections 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f7200640 1 -- 172.21.15.5:0/1174985028 >> 172.21.15.5:0/1174985028 conn(0x7f30f8066f20 msgr2=0x7f30f806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f7200640 1 -- 172.21.15.5:0/1174985028 shutdown_connections 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.578+0000 7f30f7200640 1 -- 172.21.15.5:0/1174985028 wait complete. 2024-04-25T10:01:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.580+0000 7f30f7200640 1 Processor -- start 2024-04-25T10:01:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.580+0000 7f30f7200640 1 -- start start 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30f815e390 0x7f30f80af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 0x7f30f80cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f80ce220 0x7f30f813eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8161fb0 con 0x7f30f815e770 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8161cd0 con 0x7f30f815e390 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f8161330 con 0x7f30f80ce220 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 0x7f30f80cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 0x7f30f80cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35148/0 (socket says 172.21.15.5:35148) 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.581+0000 7f30f5400640 1 -- 172.21.15.5:0/2516028089 learned_addr learned my addr 172.21.15.5:0/2516028089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5400640 1 -- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f80ce220 msgr2=0x7f30f813eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5400640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f80ce220 0x7f30f813eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5e00640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30f815e390 0x7f30f80af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5400640 1 -- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30f815e390 msgr2=0x7f30f80af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5400640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30f815e390 0x7f30f80af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5400640 1 -- 172.21.15.5:0/2516028089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80ce6a0 con 0x7f30f815e770 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f5400640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 0x7f30f80cdce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f30e400e7f0 tx=0x7f30e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4017070 con 0x7f30f815e770 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e400c860 con 0x7f30f815e770 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e400eea0 con 0x7f30f815e770 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.582+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f80afed0 con 0x7f30f815e770 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.583+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f80ae720 con 0x7f30f815e770 2024-04-25T10:01:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.584+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f80722c0 con 0x7f30f815e770 2024-04-25T10:01:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.585+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30e401a070 con 0x7f30f815e770 2024-04-25T10:01:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.586+0000 7f30ee000640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30d806b500 0x7f30d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.586+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f30e4089c60 con 0x7f30f815e770 2024-04-25T10:01:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.587+0000 7f30f5e00640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30d806b500 0x7f30d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.588+0000 7f30f5e00640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30d806b500 0x7f30d806d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30e8007fd0 tx=0x7f30e8000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:55.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.591+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30f80722c0 con 0x7f30f815e770 2024-04-25T10:01:55.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:55.789+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f30f80acae0 con 0x7f30f815e770 2024-04-25T10:01:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.007+0000 7f30ee000640 1 -- 172.21.15.5:0/2516028089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f30f80acae0 con 0x7f30f815e770 2024-04-25T10:01:56.012 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.011+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30d806b500 msgr2=0x7f30d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.011+0000 7f30f7200640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30d806b500 0x7f30d806d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f30e8007fd0 tx=0x7f30e8000ed0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.011+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 msgr2=0x7f30f80cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.011+0000 7f30f7200640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 0x7f30f80cdce0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f30e400e7f0 tx=0x7f30e400ecb0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 shutdown_connections 2024-04-25T10:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f30f80ce220 0x7f30f813eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30f815e770 0x7f30f80cdce0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30d806b500 0x7f30d806d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 --2- 172.21.15.5:0/2516028089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f30f815e390 0x7f30f80af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 >> 172.21.15.5:0/2516028089 conn(0x7f30f8066f20 msgr2=0x7f30f8071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 shutdown_connections 2024-04-25T10:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.013+0000 7f30f7200640 1 -- 172.21.15.5:0/2516028089 wait complete. 2024-04-25T10:01:56.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T10:01:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.231+0000 7f7499000640 1 Processor -- start 2024-04-25T10:01:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.232+0000 7f7499000640 1 -- start start 2024-04-25T10:01:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.233+0000 7f7499000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f749415f800 0x7f7494162ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.233+0000 7f7499000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 0x7f74940b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.233+0000 7f7499000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74941630e0 con 0x7f7494071000 2024-04-25T10:01:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.233+0000 7f7499000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749406ded0 con 0x7f749415f800 2024-04-25T10:01:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.233+0000 7f7499000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749406d020 con 0x7f7494071460 2024-04-25T10:01:56.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f748be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7494071000 0x7f74940b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45452/0 (socket says 172.21.15.5:45452) 2024-04-25T10:01:56.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f748be00640 1 -- 172.21.15.5:0/2500694634 learned_addr learned my addr 172.21.15.5:0/2500694634 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:01:56.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f7493400640 1 --2- 172.21.15.5:0/2500694634 >> v2:172.21.15.80:3300/0 conn(0x7f749415f800 0x7f7494162ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f7493e00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 0x7f74940b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849999712 0 0) 0x7f747c001110 con 0x7f7494071000 2024-04-25T10:01:56.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7478003520 con 0x7f7494071000 2024-04-25T10:01:56.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.234+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386444435 0 0) 0x7f7478003520 con 0x7f7494071000 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 msgr2=0x7f74940b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 0x7f74940b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 >> v2:172.21.15.80:3300/0 conn(0x7f749415f800 msgr2=0x7f7494162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 --2- 172.21.15.5:0/2500694634 >> v2:172.21.15.80:3300/0 conn(0x7f749415f800 0x7f7494162ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940b4850 con 0x7f7494071000 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f747c001860 con 0x7f7494071000 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 >> v1:172.21.15.5:6789/0 conn(0x7f7494071000 legacy=0x7f74940b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780039b0 0x7f7478003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74780043a0 0x7f7478004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 0x7f7478004f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749406ded0 con 0x7f74780043a0 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749406d020 con 0x7f74780039b0 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747800b9c0 con 0x7f7494071460 2024-04-25T10:01:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f748be00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74780043a0 0x7f7478004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7493e00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 0x7f7478004f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f7493400640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780039b0 0x7f7478003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f748be00640 1 -- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 msgr2=0x7f7478004f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f748be00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7494071460 0x7f7478004f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f748be00640 1 -- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780039b0 msgr2=0x7f7478003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.235+0000 7f748be00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780039b0 0x7f7478003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.236+0000 7f748be00640 1 -- 172.21.15.5:0/2500694634 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940b4850 con 0x7f74780043a0 2024-04-25T10:01:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.237+0000 7f748be00640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74780043a0 0x7f7478004880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f749406f430 tx=0x7f7478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6634b15801fcfc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.237+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c031070 con 0x7f74780043a0 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.237+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c005ad0 con 0x7f74780043a0 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.237+0000 7f7492a00640 1 -- 172.21.15.5:0/2500694634 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00c120 con 0x7f74780043a0 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7499000640 1 -- 172.21.15.5:0/2500694634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74780043a0 msgr2=0x7f7478004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7499000640 1 --2- 172.21.15.5:0/2500694634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74780043a0 0x7f7478004880 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f749406f430 tx=0x7f7478005830 comp rx=0 tx=0).stop 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7493e00640 1 -- 172.21.15.5:0/2500694634 reap_dead start 2024-04-25T10:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7499000640 1 -- 172.21.15.5:0/2500694634 shutdown_connections 2024-04-25T10:01:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7499000640 1 -- 172.21.15.5:0/2500694634 >> 172.21.15.5:0/2500694634 conn(0x7f7494066f20 msgr2=0x7f749406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7499000640 1 -- 172.21.15.5:0/2500694634 shutdown_connections 2024-04-25T10:01:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.238+0000 7f7499000640 1 -- 172.21.15.5:0/2500694634 wait complete. 2024-04-25T10:01:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.240+0000 7f7499000640 1 Processor -- start 2024-04-25T10:01:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.240+0000 7f7499000640 1 -- start start 2024-04-25T10:01:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7499000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780043a0 0x7f749415aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7499000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f749415bb10 0x7f7494157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7499000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 0x7f7494164980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7499000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749406d880 con 0x7f7494164500 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7499000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749406d5a0 con 0x7f74780043a0 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7499000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749406db60 con 0x7f749415bb10 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7493e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 0x7f7494164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7493e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 0x7f7494164980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35178/0 (socket says 172.21.15.5:35178) 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7493e00640 1 -- 172.21.15.5:0/2316936153 learned_addr learned my addr 172.21.15.5:0/2316936153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f7493400640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780043a0 0x7f749415aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.241+0000 7f748be00640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f749415bb10 0x7f7494157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493e00640 1 -- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f749415bb10 msgr2=0x7f7494157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493e00640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f749415bb10 0x7f7494157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493e00640 1 -- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780043a0 msgr2=0x7f749415aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493e00640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780043a0 0x7f749415aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:56.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493e00640 1 -- 172.21.15.5:0/2316936153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494159fa0 con 0x7f7494164500 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493400640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780043a0 0x7f749415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7493e00640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 0x7f7494164980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f749406bab0 tx=0x7f748c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c01c070 con 0x7f7494164500 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c017400 con 0x7f7494164500 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74941ea770 con 0x7f7494164500 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.242+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0177f0 con 0x7f7494164500 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.243+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7494158800 con 0x7f7494164500 2024-04-25T10:01:56.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.244+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7494068f60 con 0x7f7494164500 2024-04-25T10:01:56.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.245+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f748c005d10 con 0x7f7494164500 2024-04-25T10:01:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.246+0000 7f748b400640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f747006b500 0x7f747006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.246+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f748c08a1d0 con 0x7f7494164500 2024-04-25T10:01:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.246+0000 7f7493400640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f747006b500 0x7f747006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.248+0000 7f7493400640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f747006b500 0x7f747006d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7484009b50 tx=0x7f7484008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.251+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f748c012070 con 0x7f7494164500 2024-04-25T10:01:56.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:56.452+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f74940b4280 con 0x7f7494164500 2024-04-25T10:01:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.013+0000 7f748b400640 1 -- 172.21.15.5:0/2316936153 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7f748c059190 con 0x7f7494164500 2024-04-25T10:01:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:01:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.017+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f747006b500 msgr2=0x7f747006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.017+0000 7f7499000640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f747006b500 0x7f747006d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7484009b50 tx=0x7f7484008040 comp rx=0 tx=0).stop 2024-04-25T10:01:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.017+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 msgr2=0x7f7494164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.017+0000 7f7499000640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 0x7f7494164980 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f749406bab0 tx=0x7f748c00f4a0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 shutdown_connections 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7494164500 0x7f7494164980 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f749415bb10 0x7f7494157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f747006b500 0x7f747006d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 --2- 172.21.15.5:0/2316936153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74780043a0 0x7f749415aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 >> 172.21.15.5:0/2316936153 conn(0x7f7494066f20 msgr2=0x7f74941633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 shutdown_connections 2024-04-25T10:01:57.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.018+0000 7f7499000640 1 -- 172.21.15.5:0/2316936153 wait complete. 2024-04-25T10:01:57.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:01:57.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.240+0000 7f3b82400640 1 Processor -- start 2024-04-25T10:01:57.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.240+0000 7f3b82400640 1 -- start start 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.241+0000 7f3b82400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b7c0cc840 0x7f3b7c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.241+0000 7f3b82400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3b7c15b500 0x7f3b7c071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.241+0000 7f3b82400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0ce570 con 0x7f3b7c06e670 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.242+0000 7f3b82400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c15b910 con 0x7f3b7c15b500 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.242+0000 7f3b82400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c0ce850 con 0x7f3b7c0cc840 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.242+0000 7f3b81a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3b7c15b500 0x7f3b7c071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.242+0000 7f3b81a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3b7c15b500 0x7f3b7c071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42736/0 (socket says 172.21.15.5:42736) 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.242+0000 7f3b81a00640 1 -- 172.21.15.5:0/3332510364 learned_addr learned my addr 172.21.15.5:0/3332510364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:57.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.242+0000 7f3b81000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b7c0cc840 0x7f3b7c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.243+0000 7f3b81a00640 1 -- 172.21.15.5:0/3332510364 >> v2:172.21.15.80:3300/0 conn(0x7f3b7c0cc840 msgr2=0x7f3b7c06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:57.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.243+0000 7f3b81a00640 1 --2- 172.21.15.5:0/3332510364 >> v2:172.21.15.80:3300/0 conn(0x7f3b7c0cc840 0x7f3b7c06e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.243+0000 7f3b81a00640 1 -- 172.21.15.5:0/3332510364 >> v1:172.21.15.80:6789/0 conn(0x7f3b7c06e670 legacy=0x7f3b7c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.243+0000 7f3b81a00640 1 -- 172.21.15.5:0/3332510364 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0ceb30 con 0x7f3b7c15b500 2024-04-25T10:01:57.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.244+0000 7f3b81a00640 1 --2- 172.21.15.5:0/3332510364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c15b500 0x7f3b7c071000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c06b890 tx=0x7f3b7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c67743ae6628b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.244+0000 7f3b7b400640 1 -- 172.21.15.5:0/3332510364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74037070 con 0x7f3b7c15b500 2024-04-25T10:01:57.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.244+0000 7f3b7b400640 1 -- 172.21.15.5:0/3332510364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b74032440 con 0x7f3b7c15b500 2024-04-25T10:01:57.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.244+0000 7f3b7b400640 1 -- 172.21.15.5:0/3332510364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74032810 con 0x7f3b7c15b500 2024-04-25T10:01:57.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.244+0000 7f3b82400640 1 -- 172.21.15.5:0/3332510364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c15b500 msgr2=0x7f3b7c071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.244+0000 7f3b82400640 1 --2- 172.21.15.5:0/3332510364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c15b500 0x7f3b7c071000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c06b890 tx=0x7f3b7402a270 comp rx=0 tx=0).stop 2024-04-25T10:01:57.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.245+0000 7f3b82400640 1 -- 172.21.15.5:0/3332510364 shutdown_connections 2024-04-25T10:01:57.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.245+0000 7f3b82400640 1 --2- 172.21.15.5:0/3332510364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c15b500 0x7f3b7c071000 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.245+0000 7f3b82400640 1 --2- 172.21.15.5:0/3332510364 >> v2:172.21.15.80:3300/0 conn(0x7f3b7c0cc840 0x7f3b7c06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.245+0000 7f3b82400640 1 -- 172.21.15.5:0/3332510364 >> 172.21.15.5:0/3332510364 conn(0x7f3b7c066f20 msgr2=0x7f3b7c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:57.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.245+0000 7f3b82400640 1 -- 172.21.15.5:0/3332510364 shutdown_connections 2024-04-25T10:01:57.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.245+0000 7f3b82400640 1 -- 172.21.15.5:0/3332510364 wait complete. 2024-04-25T10:01:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.246+0000 7f3b82400640 1 Processor -- start 2024-04-25T10:01:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.247+0000 7f3b82400640 1 -- start start 2024-04-25T10:01:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.247+0000 7f3b82400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b7c06e670 0x7f3b7c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.247+0000 7f3b82400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 0x7f3b7c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b82400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3b7c15b500 0x7f3b7c14f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b82400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0ce1d0 con 0x7f3b7c0cc840 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b82400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0b2d20 con 0x7f3b7c06e670 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b82400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0ce350 con 0x7f3b7c15b500 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 0x7f3b7c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 0x7f3b7c150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42740/0 (socket says 172.21.15.5:42740) 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 -- 172.21.15.5:0/2140716749 learned_addr learned my addr 172.21.15.5:0/2140716749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b81000640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b7c06e670 0x7f3b7c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 -- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3b7c15b500 msgr2=0x7f3b7c14f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b81a00640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3b7c15b500 0x7f3b7c14f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3b7c15b500 0x7f3b7c14f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 -- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b7c06e670 msgr2=0x7f3b7c06dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b7c06e670 0x7f3b7c06dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b7be00640 1 -- 172.21.15.5:0/2140716749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c1574a0 con 0x7f3b7c0cc840 2024-04-25T10:01:57.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.248+0000 7f3b81a00640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3b7c15b500 0x7f3b7c14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.249+0000 7f3b7be00640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 0x7f3b7c150ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c7c0 tx=0x7f3b6400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.249+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64016020 con 0x7f3b7c0cc840 2024-04-25T10:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.249+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64006d50 con 0x7f3b7c0cc840 2024-04-25T10:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.249+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64003bb0 con 0x7f3b7c0cc840 2024-04-25T10:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.249+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c157780 con 0x7f3b7c0cc840 2024-04-25T10:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.249+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c157d40 con 0x7f3b7c0cc840 2024-04-25T10:01:57.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.250+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c068f60 con 0x7f3b7c0cc840 2024-04-25T10:01:57.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.251+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b6400a850 con 0x7f3b7c0cc840 2024-04-25T10:01:57.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.252+0000 7f3b78c00640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3b5806b4d0 0x7f3b5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:57.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.252+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3b64089f80 con 0x7f3b7c0cc840 2024-04-25T10:01:57.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.253+0000 7f3b81000640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3b5806b4d0 0x7f3b5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:57.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.254+0000 7f3b81000640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3b5806b4d0 0x7f3b5806d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009ad0 tx=0x7f3b6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:57.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.257+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b64020020 con 0x7f3b7c0cc840 2024-04-25T10:01:57.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:57.492+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b7c15b000 con 0x7f3b7c0cc840 2024-04-25T10:01:58.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.088+0000 7f3b78c00640 1 -- 172.21.15.5:0/2140716749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3b64019310 con 0x7f3b7c0cc840 2024-04-25T10:01:58.093 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:01:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3b5806b4d0 msgr2=0x7f3b5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3b5806b4d0 0x7f3b5806d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009ad0 tx=0x7f3b6c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 msgr2=0x7f3b7c150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 0x7f3b7c150ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c7c0 tx=0x7f3b6400cc80 comp rx=0 tx=0).stop 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 shutdown_connections 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3b7c15b500 0x7f3b7c14f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b7c0cc840 0x7f3b7c150ce0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3b5806b4d0 0x7f3b5806d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 --2- 172.21.15.5:0/2140716749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b7c06e670 0x7f3b7c06dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 >> 172.21.15.5:0/2140716749 conn(0x7f3b7c066f20 msgr2=0x7f3b7c167bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 shutdown_connections 2024-04-25T10:01:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.091+0000 7f3b82400640 1 -- 172.21.15.5:0/2140716749 wait complete. 2024-04-25T10:01:58.119 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T10:01:58.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T10:01:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.304+0000 7fc757000640 1 Processor -- start 2024-04-25T10:01:58.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.304+0000 7fc757000640 1 -- start start 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.305+0000 7fc757000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc75015dff0 0x7fc75015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.305+0000 7fc757000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc750071000 0x7fc750071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.305+0000 7fc757000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75015b050 con 0x7fc7500cb400 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.305+0000 7fc757000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75013fcf0 con 0x7fc75015dff0 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.305+0000 7fc757000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75015b330 con 0x7fc750071000 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.305+0000 7fc755c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc75015dff0 0x7fc75015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc75015dff0 0x7fc75015a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42756/0 (socket says 172.21.15.5:42756) 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755c00640 1 -- 172.21.15.5:0/1505430547 learned_addr learned my addr 172.21.15.5:0/1505430547 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc756600640 1 --1- 172.21.15.5:0/1505430547 >> v1:172.21.15.5:6789/0 conn(0x7fc7500cb400 0x7fc750140180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59546/0 (socket says 172.21.15.5:59546) 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755200640 1 --2- 172.21.15.5:0/1505430547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc750071000 0x7fc750071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755c00640 1 -- 172.21.15.5:0/1505430547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc750071000 msgr2=0x7fc750071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755c00640 1 --2- 172.21.15.5:0/1505430547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc750071000 0x7fc750071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755c00640 1 -- 172.21.15.5:0/1505430547 >> v1:172.21.15.5:6789/0 conn(0x7fc7500cb400 legacy=0x7fc750140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.306+0000 7fc755c00640 1 -- 172.21.15.5:0/1505430547 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75015b610 con 0x7fc75015dff0 2024-04-25T10:01:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.307+0000 7fc755c00640 1 --2- 172.21.15.5:0/1505430547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc75015a9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc744009870 tx=0x7fc74402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904a0d899a57a5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.307+0000 7fc74fe00640 1 -- 172.21.15.5:0/1505430547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744037070 con 0x7fc75015dff0 2024-04-25T10:01:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc74fe00640 1 -- 172.21.15.5:0/1505430547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744032440 con 0x7fc75015dff0 2024-04-25T10:01:58.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc74fe00640 1 -- 172.21.15.5:0/1505430547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744032810 con 0x7fc75015dff0 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 -- 172.21.15.5:0/1505430547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 msgr2=0x7fc75015a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 --2- 172.21.15.5:0/1505430547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc75015a9b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc744009870 tx=0x7fc74402a250 comp rx=0 tx=0).stop 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 -- 172.21.15.5:0/1505430547 shutdown_connections 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 --2- 172.21.15.5:0/1505430547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc750071000 0x7fc750071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 --2- 172.21.15.5:0/1505430547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc75015a9b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 -- 172.21.15.5:0/1505430547 >> 172.21.15.5:0/1505430547 conn(0x7fc750066f20 msgr2=0x7fc75006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 -- 172.21.15.5:0/1505430547 shutdown_connections 2024-04-25T10:01:58.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.308+0000 7fc757000640 1 -- 172.21.15.5:0/1505430547 wait complete. 2024-04-25T10:01:58.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.310+0000 7fc757000640 1 Processor -- start 2024-04-25T10:01:58.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.310+0000 7fc757000640 1 -- start start 2024-04-25T10:01:58.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc757000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc750071000 0x7fc75013c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc757000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7500cb400 0x7fc750141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc757000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc750141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc757000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc750140a10 con 0x7fc75015dff0 2024-04-25T10:01:58.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc757000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc750140890 con 0x7fc750071000 2024-04-25T10:01:58.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc757000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500cb890 con 0x7fc7500cb400 2024-04-25T10:01:58.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc756600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc750141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc756600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc750141a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42772/0 (socket says 172.21.15.5:42772) 2024-04-25T10:01:58.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.311+0000 7fc756600640 1 -- 172.21.15.5:0/3763625712 learned_addr learned my addr 172.21.15.5:0/3763625712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:58.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc755c00640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc750071000 0x7fc75013c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc755200640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7500cb400 0x7fc750141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc756600640 1 -- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7500cb400 msgr2=0x7fc750141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc756600640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7500cb400 0x7fc750141530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc756600640 1 -- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc750071000 msgr2=0x7fc75013c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc756600640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc750071000 0x7fc75013c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc756600640 1 -- 172.21.15.5:0/3763625712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75013e920 con 0x7fc75015dff0 2024-04-25T10:01:58.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc756600640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc750141a70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c780 tx=0x7fc74800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.312+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748017070 con 0x7fc75015dff0 2024-04-25T10:01:58.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.313+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800a640 con 0x7fc75015dff0 2024-04-25T10:01:58.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.313+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75013ec00 con 0x7fc75015dff0 2024-04-25T10:01:58.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.313+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800aa30 con 0x7fc75015dff0 2024-04-25T10:01:58.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.313+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75013f1c0 con 0x7fc75015dff0 2024-04-25T10:01:58.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.314+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc750068f60 con 0x7fc75015dff0 2024-04-25T10:01:58.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.315+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc74801a070 con 0x7fc75015dff0 2024-04-25T10:01:58.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.316+0000 7fc74d600640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73006b4d0 0x7fc73006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.316+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7480897d0 con 0x7fc75015dff0 2024-04-25T10:01:58.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.317+0000 7fc755c00640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73006b4d0 0x7fc73006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.318+0000 7fc755c00640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73006b4d0 0x7fc73006d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc744034000 tx=0x7fc74402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.321+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc748057f60 con 0x7fc75015dff0 2024-04-25T10:01:58.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:58.528+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fc7500b4920 con 0x7fc75015dff0 2024-04-25T10:01:58.790 INFO:tasks.ceph.mds.b.smithi080.stderr:2024-04-25T10:01:58.785+0000 7fa933a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:01:59.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.040+0000 7fc74d600640 1 -- 172.21.15.5:0/3763625712 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) v1 ==== 157+0+0 (secure 0 0 0) 0x7fc74805c000 con 0x7fc75015dff0 2024-04-25T10:01:59.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73006b4d0 msgr2=0x7fc73006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73006b4d0 0x7fc73006d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc744034000 tx=0x7fc74402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 msgr2=0x7fc750141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc750141a70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc74800c780 tx=0x7fc74800cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 shutdown_connections 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75015dff0 0x7fc750141a70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7500cb400 0x7fc750141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73006b4d0 0x7fc73006d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 --2- 172.21.15.5:0/3763625712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc750071000 0x7fc75013c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 >> 172.21.15.5:0/3763625712 conn(0x7fc750066f20 msgr2=0x7fc7500ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.044+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 shutdown_connections 2024-04-25T10:01:59.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.045+0000 7fc757000640 1 -- 172.21.15.5:0/3763625712 wait complete. 2024-04-25T10:01:59.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.293+0000 7fb63e200640 1 Processor -- start 2024-04-25T10:01:59.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.293+0000 7fb63e200640 1 -- start start 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63e200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb63815a4e0 0x7fb6380acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63e200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6380b41a0 con 0x7fb6380cfda0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63e200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6380b3900 con 0x7fb6380b3390 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63e200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6380b3be0 con 0x7fb63815a4e0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb63815a4e0 0x7fb6380acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63d800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb6380cfda0 0x7fb63815e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59556/0 (socket says 172.21.15.5:59556) 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb63815a4e0 0x7fb6380acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42784/0 (socket says 172.21.15.5:42784) 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.295+0000 7fb63ce00640 1 -- 172.21.15.5:0/4006285437 learned_addr learned my addr 172.21.15.5:0/4006285437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.296+0000 7fb63ce00640 1 -- 172.21.15.5:0/4006285437 >> v1:172.21.15.80:6789/0 conn(0x7fb6380b3390 legacy=0x7fb6380cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.296+0000 7fb63ce00640 1 -- 172.21.15.5:0/4006285437 >> v1:172.21.15.5:6789/0 conn(0x7fb6380cfda0 legacy=0x7fb63815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.296+0000 7fb63ce00640 1 -- 172.21.15.5:0/4006285437 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380b3ec0 con 0x7fb63815a4e0 2024-04-25T10:01:59.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.296+0000 7fb637400640 1 -- 172.21.15.5:0/4006285437 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128312628 0 0) 0x7fb6380b41a0 con 0x7fb6380cfda0 2024-04-25T10:01:59.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.297+0000 7fb63ce00640 1 --2- 172.21.15.5:0/4006285437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb63815a4e0 0x7fb6380acf80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00bb10 tx=0x7fb62c02c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1cb0daeecef4a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.297+0000 7fb637400640 1 -- 172.21.15.5:0/4006285437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c036020 con 0x7fb63815a4e0 2024-04-25T10:01:59.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.297+0000 7fb637400640 1 -- 172.21.15.5:0/4006285437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c032820 con 0x7fb63815a4e0 2024-04-25T10:01:59.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.297+0000 7fb637400640 1 -- 172.21.15.5:0/4006285437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c032bf0 con 0x7fb63815a4e0 2024-04-25T10:01:59.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 -- 172.21.15.5:0/4006285437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb63815a4e0 msgr2=0x7fb6380acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 --2- 172.21.15.5:0/4006285437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb63815a4e0 0x7fb6380acf80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00bb10 tx=0x7fb62c02c960 comp rx=0 tx=0).stop 2024-04-25T10:01:59.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 -- 172.21.15.5:0/4006285437 shutdown_connections 2024-04-25T10:01:59.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 --2- 172.21.15.5:0/4006285437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb63815a4e0 0x7fb6380acf80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 -- 172.21.15.5:0/4006285437 >> 172.21.15.5:0/4006285437 conn(0x7fb6380b12a0 msgr2=0x7fb6380b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 -- 172.21.15.5:0/4006285437 shutdown_connections 2024-04-25T10:01:59.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.298+0000 7fb63e200640 1 -- 172.21.15.5:0/4006285437 wait complete. 2024-04-25T10:01:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.301+0000 7fb63e200640 1 Processor -- start 2024-04-25T10:01:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.301+0000 7fb63e200640 1 -- start start 2024-04-25T10:01:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb63e200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6380b3390 0x7fb638149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb63e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 0x7fb638148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb63e200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63815a4e0 0x7fb63806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb63e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638072cf0 con 0x7fb6380cfda0 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb63e200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63815a8c0 con 0x7fb63815a4e0 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb63e200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb638072e70 con 0x7fb6380b3390 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb637e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 0x7fb638148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb637e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 0x7fb638148710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42786/0 (socket says 172.21.15.5:42786) 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.302+0000 7fb637e00640 1 -- 172.21.15.5:0/972794862 learned_addr learned my addr 172.21.15.5:0/972794862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb63ce00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6380b3390 0x7fb638149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb63d800640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63815a4e0 0x7fb63806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb637e00640 1 -- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6380b3390 msgr2=0x7fb638149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb637e00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6380b3390 0x7fb638149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb637e00640 1 -- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63815a4e0 msgr2=0x7fb63806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb637e00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63815a4e0 0x7fb63806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb637e00640 1 -- 172.21.15.5:0/972794862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6381effb0 con 0x7fb6380cfda0 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb637e00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 0x7fb638148710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c430 tx=0x7fb62400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.303+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011790 con 0x7fb6380cfda0 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.304+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624017070 con 0x7fb6380cfda0 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.304+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011b70 con 0x7fb6380cfda0 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.304+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6381efcd0 con 0x7fb6380cfda0 2024-04-25T10:01:59.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.304+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb638149690 con 0x7fb6380cfda0 2024-04-25T10:01:59.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.305+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb638068f60 con 0x7fb6380cfda0 2024-04-25T10:01:59.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.306+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb624011d10 con 0x7fb6380cfda0 2024-04-25T10:01:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.307+0000 7fb634c00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb61806b4d0 0x7fb61806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.307+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fb62408a1b0 con 0x7fb6380cfda0 2024-04-25T10:01:59.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.307+0000 7fb63ce00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb61806b4d0 0x7fb61806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.308+0000 7fb63ce00640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb61806b4d0 0x7fb61806d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb62c037000 tx=0x7fb62c03d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.311+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb624058ef0 con 0x7fb6380cfda0 2024-04-25T10:01:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:01:59.506+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb63806df10 con 0x7fb6380cfda0 2024-04-25T10:02:01.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.317+0000 7fb634c00640 1 -- 172.21.15.5:0/972794862 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) v1 ==== 228+0+0 (secure 0 0 0) 0x7fb6240587f0 con 0x7fb6380cfda0 2024-04-25T10:02:01.323 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' created 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb61806b4d0 msgr2=0x7fb61806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb61806b4d0 0x7fb61806d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb62c037000 tx=0x7fb62c03d040 comp rx=0 tx=0).stop 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 msgr2=0x7fb638148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 0x7fb638148710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c430 tx=0x7fb62400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 shutdown_connections 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb61806b4d0 0x7fb61806d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63815a4e0 0x7fb63806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6380cfda0 0x7fb638148710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 --2- 172.21.15.5:0/972794862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6380b3390 0x7fb638149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 >> 172.21.15.5:0/972794862 conn(0x7fb6380b12a0 msgr2=0x7fb63814f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.321+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 shutdown_connections 2024-04-25T10:02:01.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.322+0000 7fb63e200640 1 -- 172.21.15.5:0/972794862 wait complete. 2024-04-25T10:02:01.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T10:02:01.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.538+0000 7f893aa00640 1 Processor -- start 2024-04-25T10:02:01.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.538+0000 7f893aa00640 1 -- start start 2024-04-25T10:02:01.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.540+0000 7f893aa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8934161bc0 0x7f89340b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.540+0000 7f893aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934071710 0x7f89340cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.540+0000 7f893aa00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8934161fa0 con 0x7f893406da30 2024-04-25T10:02:01.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.540+0000 7f893aa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893406eac0 con 0x7f8934161bc0 2024-04-25T10:02:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.540+0000 7f893aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340cedc0 con 0x7f8934071710 2024-04-25T10:02:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.541+0000 7f8939600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8934161bc0 0x7f89340b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.541+0000 7f893a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934071710 0x7f89340cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.541+0000 7f8938c00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f893406da30 0x7f8934071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:51650/0 (socket says 172.21.15.5:51650) 2024-04-25T10:02:01.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.541+0000 7f8938c00640 1 -- 172.21.15.5:0/3975314499 learned_addr learned my addr 172.21.15.5:0/3975314499 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:01.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.542+0000 7f8933e00640 1 -- 172.21.15.5:0/3975314499 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834907552 0 0) 0x7f8934161fa0 con 0x7f893406da30 2024-04-25T10:02:01.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.542+0000 7f8933e00640 1 -- 172.21.15.5:0/3975314499 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891c003520 con 0x7f893406da30 2024-04-25T10:02:01.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.542+0000 7f8939600640 1 -- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934071710 msgr2=0x7f89340cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.542+0000 7f8939600640 1 --2- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934071710 0x7f89340cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.542+0000 7f8939600640 1 -- 172.21.15.5:0/3975314499 >> v1:172.21.15.80:6789/0 conn(0x7f893406da30 legacy=0x7f8934071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.542+0000 7f8939600640 1 -- 172.21.15.5:0/3975314499 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340b37e0 con 0x7f8934161bc0 2024-04-25T10:02:01.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.543+0000 7f8939600640 1 --2- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8934161bc0 0x7f89340b42c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89280097a0 tx=0x7f892802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cbe957cb59740ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.544+0000 7f8933e00640 1 -- 172.21.15.5:0/3975314499 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928037070 con 0x7f8934161bc0 2024-04-25T10:02:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.544+0000 7f8933e00640 1 -- 172.21.15.5:0/3975314499 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8928032440 con 0x7f8934161bc0 2024-04-25T10:02:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.544+0000 7f8933e00640 1 -- 172.21.15.5:0/3975314499 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928032810 con 0x7f8934161bc0 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.544+0000 7f893aa00640 1 -- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8934161bc0 msgr2=0x7f89340b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.544+0000 7f893aa00640 1 --2- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8934161bc0 0x7f89340b42c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89280097a0 tx=0x7f892802a180 comp rx=0 tx=0).stop 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.545+0000 7f893aa00640 1 -- 172.21.15.5:0/3975314499 shutdown_connections 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.545+0000 7f893aa00640 1 --2- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934071710 0x7f89340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.545+0000 7f893aa00640 1 --2- 172.21.15.5:0/3975314499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8934161bc0 0x7f89340b42c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.545+0000 7f893aa00640 1 -- 172.21.15.5:0/3975314499 >> 172.21.15.5:0/3975314499 conn(0x7f8934066f20 msgr2=0x7f893406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.545+0000 7f893aa00640 1 -- 172.21.15.5:0/3975314499 shutdown_connections 2024-04-25T10:02:01.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.545+0000 7f893aa00640 1 -- 172.21.15.5:0/3975314499 wait complete. 2024-04-25T10:02:01.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.547+0000 7f893aa00640 1 Processor -- start 2024-04-25T10:02:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.547+0000 7f893aa00640 1 -- start start 2024-04-25T10:02:01.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893406da30 0x7f8934163a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 0x7f893414b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934161bc0 0x7f893414b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b4800 con 0x7f8934071710 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b3da0 con 0x7f893406da30 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340cea50 con 0x7f8934161bc0 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f8938c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 0x7f893414b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f8938c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 0x7f893414b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42794/0 (socket says 172.21.15.5:42794) 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f8938c00640 1 -- 172.21.15.5:0/1618766613 learned_addr learned my addr 172.21.15.5:0/1618766613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f8939600640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893406da30 0x7f8934163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.548+0000 7f893a000640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934161bc0 0x7f893414b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8938c00640 1 -- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934161bc0 msgr2=0x7f893414b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8938c00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934161bc0 0x7f893414b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8938c00640 1 -- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893406da30 msgr2=0x7f8934163a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8938c00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893406da30 0x7f8934163a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8938c00640 1 -- 172.21.15.5:0/1618766613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934151e60 con 0x7f8934071710 2024-04-25T10:02:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f893a000640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934161bc0 0x7f893414b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:01.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8939600640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893406da30 0x7f8934163a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8938c00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 0x7f893414b330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f892000c430 tx=0x7f892000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8920017070 con 0x7f8934071710 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8920005ce0 con 0x7f8934071710 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89200072f0 con 0x7f8934071710 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8934163fe0 con 0x7f8934071710 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.549+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89341645a0 con 0x7f8934071710 2024-04-25T10:02:01.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.551+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8934068f60 con 0x7f8934071710 2024-04-25T10:02:01.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.552+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f892001a070 con 0x7f8934071710 2024-04-25T10:02:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.553+0000 7f8931600640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891406b4d0 0x7f891406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.553+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8920089a30 con 0x7f8934071710 2024-04-25T10:02:01.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.554+0000 7f8939600640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891406b4d0 0x7f891406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.555+0000 7f8939600640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891406b4d0 0x7f891406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8928034000 tx=0x7f892802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.558+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89200584f0 con 0x7f8934071710 2024-04-25T10:02:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:01.756+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f89340b2e80 con 0x7f8934071710 2024-04-25T10:02:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.322+0000 7f8931600640 1 -- 172.21.15.5:0/1618766613 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8920057df0 con 0x7f8934071710 2024-04-25T10:02:02.328 INFO:teuthology.orchestra.run.smithi005.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T10:02:02.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.328+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891406b4d0 msgr2=0x7f891406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.328+0000 7f893aa00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891406b4d0 0x7f891406d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8928034000 tx=0x7f892802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.328+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 msgr2=0x7f893414b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.328+0000 7f893aa00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 0x7f893414b330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f892000c430 tx=0x7f892000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.329+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 shutdown_connections 2024-04-25T10:02:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.329+0000 7f893aa00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8934161bc0 0x7f893414b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.329+0000 7f893aa00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8934071710 0x7f893414b330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.329+0000 7f893aa00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891406b4d0 0x7f891406d990 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8928034000 tx=0x7f892802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.329+0000 7f893aa00640 1 --2- 172.21.15.5:0/1618766613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893406da30 0x7f8934163a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.329+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 >> 172.21.15.5:0/1618766613 conn(0x7f8934066f20 msgr2=0x7f89340b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.330+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 shutdown_connections 2024-04-25T10:02:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.331+0000 7f893aa00640 1 -- 172.21.15.5:0/1618766613 wait complete. 2024-04-25T10:02:02.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T10:02:02.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.553+0000 7f57ae800640 1 Processor -- start 2024-04-25T10:02:02.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.554+0000 7f57ae800640 1 -- start start 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.555+0000 7f57ae800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57a8071490 0x7f57a815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.555+0000 7f57ae800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a815f3e0 con 0x7f57a8071000 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.555+0000 7f57ae800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a8071900 con 0x7f57a815ed20 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.555+0000 7f57ae800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a8144780 con 0x7f57a8071490 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.555+0000 7f57ade00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57a8071490 0x7f57a815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.556+0000 7f57ade00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57a8071490 0x7f57a815c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42806/0 (socket says 172.21.15.5:42806) 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.556+0000 7f57ade00640 1 -- 172.21.15.5:0/2958253854 learned_addr learned my addr 172.21.15.5:0/2958253854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.556+0000 7f57aca00640 1 --1- 172.21.15.5:0/2958253854 >> v1:172.21.15.5:6789/0 conn(0x7f57a8071000 0x7f57a815bdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59566/0 (socket says 172.21.15.5:59566) 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.556+0000 7f57ade00640 1 -- 172.21.15.5:0/2958253854 >> v1:172.21.15.80:6789/0 conn(0x7f57a815ed20 legacy=0x7f57a8144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.556+0000 7f57ade00640 1 -- 172.21.15.5:0/2958253854 >> v1:172.21.15.5:6789/0 conn(0x7f57a8071000 legacy=0x7f57a815bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.556+0000 7f57ade00640 1 -- 172.21.15.5:0/2958253854 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a815f680 con 0x7f57a8071490 2024-04-25T10:02:02.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.557+0000 7f57ade00640 1 --2- 172.21.15.5:0/2958253854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a8071490 0x7f57a815c4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57a8144900 tx=0x7f57a002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e9e493319445b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.557+0000 7f57a7e00640 1 -- 172.21.15.5:0/2958253854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0036070 con 0x7f57a8071490 2024-04-25T10:02:02.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.557+0000 7f57a7e00640 1 -- 172.21.15.5:0/2958253854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0031600 con 0x7f57a8071490 2024-04-25T10:02:02.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.557+0000 7f57a7e00640 1 -- 172.21.15.5:0/2958253854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a00319d0 con 0x7f57a8071490 2024-04-25T10:02:02.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 -- 172.21.15.5:0/2958253854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a8071490 msgr2=0x7f57a815c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 --2- 172.21.15.5:0/2958253854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a8071490 0x7f57a815c4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f57a8144900 tx=0x7f57a002a640 comp rx=0 tx=0).stop 2024-04-25T10:02:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 -- 172.21.15.5:0/2958253854 shutdown_connections 2024-04-25T10:02:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 --2- 172.21.15.5:0/2958253854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a8071490 0x7f57a815c4d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 -- 172.21.15.5:0/2958253854 >> 172.21.15.5:0/2958253854 conn(0x7f57a8066f20 msgr2=0x7f57a806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 -- 172.21.15.5:0/2958253854 shutdown_connections 2024-04-25T10:02:02.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.558+0000 7f57ae800640 1 -- 172.21.15.5:0/2958253854 wait complete. 2024-04-25T10:02:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.560+0000 7f57ae800640 1 Processor -- start 2024-04-25T10:02:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.560+0000 7f57ae800640 1 -- start start 2024-04-25T10:02:02.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ae800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57a8071000 0x7f57a8149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ae800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a8071490 0x7f57a814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ae800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 0x7f57a8139e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ae800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80cdd50 con 0x7f57a815ed20 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ae800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a815fc40 con 0x7f57a8071490 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ae800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8145060 con 0x7f57a8071000 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 0x7f57a8139e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 0x7f57a8139e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42816/0 (socket says 172.21.15.5:42816) 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.561+0000 7f57ade00640 1 -- 172.21.15.5:0/4193470466 learned_addr learned my addr 172.21.15.5:0/4193470466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57aca00640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a8071490 0x7f57a814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ad400640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57a8071000 0x7f57a8149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ade00640 1 -- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57a8071000 msgr2=0x7f57a8149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ade00640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57a8071000 0x7f57a8149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ade00640 1 -- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a8071490 msgr2=0x7f57a814c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ade00640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a8071490 0x7f57a814c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ade00640 1 -- 172.21.15.5:0/4193470466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a813de50 con 0x7f57a815ed20 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57ade00640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 0x7f57a8139e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009f40 tx=0x7f57a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0036050 con 0x7f57a815ed20 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.562+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a0042070 con 0x7f57a815ed20 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.563+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0007bd0 con 0x7f57a815ed20 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.563+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a814c830 con 0x7f57a815ed20 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.563+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a813e110 con 0x7f57a815ed20 2024-04-25T10:02:02.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.564+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8068f60 con 0x7f57a815ed20 2024-04-25T10:02:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.565+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57a0047460 con 0x7f57a815ed20 2024-04-25T10:02:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.566+0000 7f57a5600640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f578806b4d0 0x7f578806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.566+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f57a00ab690 con 0x7f57a815ed20 2024-04-25T10:02:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.566+0000 7f57ad400640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f578806b4d0 0x7f578806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.567+0000 7f57ad400640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f578806b4d0 0x7f578806d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f579c006fd0 tx=0x7f579c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.570+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57a002ae20 con 0x7f57a815ed20 2024-04-25T10:02:02.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:02.839+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f57a80720f0 con 0x7f57a815ed20 2024-04-25T10:02:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.411+0000 7f57a5600640 1 -- 172.21.15.5:0/4193470466 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f57a007a150 con 0x7f57a815ed20 2024-04-25T10:02:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:added data pool 4 to fsmap 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f578806b4d0 msgr2=0x7f578806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f578806b4d0 0x7f578806d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f579c006fd0 tx=0x7f579c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 msgr2=0x7f57a8139e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 0x7f57a8139e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009f40 tx=0x7f57a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 shutdown_connections 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57a815ed20 0x7f57a8139e90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f578806b4d0 0x7f578806d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a8071490 0x7f57a814c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 --2- 172.21.15.5:0/4193470466 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57a8071000 0x7f57a8149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 >> 172.21.15.5:0/4193470466 conn(0x7f57a8066f20 msgr2=0x7f57a806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 shutdown_connections 2024-04-25T10:02:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.415+0000 7f57ae800640 1 -- 172.21.15.5:0/4193470466 wait complete. 2024-04-25T10:02:03.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.645+0000 7fe96b200640 1 Processor -- start 2024-04-25T10:02:03.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.645+0000 7fe96b200640 1 -- start start 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe96b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c0cda10 0x7fe96c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe96b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c071d30 con 0x7fe96c141da0 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe96b200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96c06d620 con 0x7fe96c15ce00 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe96b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c142180 con 0x7fe96c0cda10 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe969e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe96c141da0 0x7fe96c06e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59572/0 (socket says 172.21.15.5:59572) 2024-04-25T10:02:03.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe969e00640 1 -- 172.21.15.5:0/4113695317 learned_addr learned my addr 172.21.15.5:0/4113695317 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946791954 0 0) 0x7fe96c071d30 con 0x7fe96c141da0 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe950003520 con 0x7fe96c141da0 2024-04-25T10:02:03.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.647+0000 7fe96a800640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c0cda10 0x7fe96c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335647463 0 0) 0x7fe950003520 con 0x7fe96c141da0 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c0cda10 msgr2=0x7fe96c15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c0cda10 0x7fe96c15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 >> v1:172.21.15.80:6789/0 conn(0x7fe96c15ce00 legacy=0x7fe96c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c06d900 con 0x7fe96c141da0 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe95c001670 con 0x7fe96c141da0 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 >> v1:172.21.15.5:6789/0 conn(0x7fe96c141da0 legacy=0x7fe96c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9500039e0 0x7fe950003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe950005210 0x7fe95000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c142180 con 0x7fe950004800 2024-04-25T10:02:03.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95000c0c0 con 0x7fe9500039e0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95000c2c0 con 0x7fe950005210 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe96a800640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c0cda10 0x7fe96c15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.648+0000 7fe969400640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969e00640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9500039e0 0x7fe950003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969400640 1 -- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe950005210 msgr2=0x7fe95000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe96a800640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe950005210 0x7fe95000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969400640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe950005210 0x7fe95000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969400640 1 -- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9500039e0 msgr2=0x7fe950003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969400640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9500039e0 0x7fe950003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969400640 1 -- 172.21.15.5:0/4113695317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950003520 con 0x7fe950004800 2024-04-25T10:02:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.649+0000 7fe969e00640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9500039e0 0x7fe950003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.650+0000 7fe969400640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe950004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe954001750 tx=0x7fe950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9843f71ecb50c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.650+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954001370 con 0x7fe950004800 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.650+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9540052d0 con 0x7fe950004800 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.650+0000 7fe968a00640 1 -- 172.21.15.5:0/4113695317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95402abe0 con 0x7fe950004800 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96b200640 1 -- 172.21.15.5:0/4113695317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 msgr2=0x7fe950004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96b200640 1 --2- 172.21.15.5:0/4113695317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe950004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe954001750 tx=0x7fe950005830 comp rx=0 tx=0).stop 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96a800640 1 -- 172.21.15.5:0/4113695317 reap_dead start 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96b200640 1 -- 172.21.15.5:0/4113695317 shutdown_connections 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96b200640 1 -- 172.21.15.5:0/4113695317 >> 172.21.15.5:0/4113695317 conn(0x7fe96c066f20 msgr2=0x7fe96c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96b200640 1 -- 172.21.15.5:0/4113695317 shutdown_connections 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.651+0000 7fe96b200640 1 -- 172.21.15.5:0/4113695317 wait complete. 2024-04-25T10:02:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.653+0000 7fe96b200640 1 Processor -- start 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.653+0000 7fe96b200640 1 -- start start 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe96b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe96c0c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe96b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c12e620 0x7fe96c12eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe96b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe96c159fb0 0x7fe96c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe96b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c15d280 con 0x7fe950004800 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe96b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c06e810 con 0x7fe96c159fb0 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe96b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96c15d400 con 0x7fe96c12e620 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe969e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe96c0c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe969e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe96c0c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42836/0 (socket says 172.21.15.5:42836) 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe969e00640 1 -- 172.21.15.5:0/316041693 learned_addr learned my addr 172.21.15.5:0/316041693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.654+0000 7fe969400640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c12e620 0x7fe96c12eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe96a800640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe96c159fb0 0x7fe96c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969e00640 1 -- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c12e620 msgr2=0x7fe96c12eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969e00640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c12e620 0x7fe96c12eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969e00640 1 -- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe96c159fb0 msgr2=0x7fe96c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969e00640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe96c159fb0 0x7fe96c15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969e00640 1 -- 172.21.15.5:0/316041693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c12f0c0 con 0x7fe950004800 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969400640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c12e620 0x7fe96c12eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe969e00640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe96c0c75b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e7b0 tx=0x7fe95c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00ee60 con 0x7fe950004800 2024-04-25T10:02:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95c018070 con 0x7fe950004800 2024-04-25T10:02:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96c12f3a0 con 0x7fe950004800 2024-04-25T10:02:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.655+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00ca70 con 0x7fe950004800 2024-04-25T10:02:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.656+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c12ca80 con 0x7fe950004800 2024-04-25T10:02:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.657+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe96c12c590 con 0x7fe950004800 2024-04-25T10:02:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.658+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe95c00cc10 con 0x7fe950004800 2024-04-25T10:02:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.658+0000 7fe962000640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe94806b550 0x7fe94806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.659+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe95c089fe0 con 0x7fe950004800 2024-04-25T10:02:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.659+0000 7fe969400640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe94806b550 0x7fe94806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.660+0000 7fe969400640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe94806b550 0x7fe94806da10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe954032000 tx=0x7fe95402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.663+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe95c022250 con 0x7fe950004800 2024-04-25T10:02:03.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.908+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe96c12cc50 con 0x7fe950004800 2024-04-25T10:02:03.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.910+0000 7fe962000640 1 -- 172.21.15.5:0/316041693 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4925 (secure 0 0 0) 0x7fe95c058b40 con 0x7fe950004800 2024-04-25T10:02:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:03.915 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:02.841787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:02:03.915 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.912+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe94806b550 msgr2=0x7fe94806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.912+0000 7fe96b200640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe94806b550 0x7fe94806da10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe954032000 tx=0x7fe95402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.912+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 msgr2=0x7fe96c0c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.912+0000 7fe96b200640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe96c0c75b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00e7b0 tx=0x7fe95c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 shutdown_connections 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe94806b550 0x7fe94806da10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe96c159fb0 0x7fe96c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe96c12e620 0x7fe96c12eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 --2- 172.21.15.5:0/316041693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe950004800 0x7fe96c0c75b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 >> 172.21.15.5:0/316041693 conn(0x7fe96c066f20 msgr2=0x7fe96c0b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 shutdown_connections 2024-04-25T10:02:03.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:03.913+0000 7fe96b200640 1 -- 172.21.15.5:0/316041693 wait complete. 2024-04-25T10:02:03.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.144+0000 7fbdf5600640 1 Processor -- start 2024-04-25T10:02:04.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.144+0000 7fbdf5600640 1 -- start start 2024-04-25T10:02:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf01652f0 0x7fbdf00cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf5600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdf00716a0 0x7fbdf006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf5600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf00cd2a0 con 0x7fbdf006e980 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf5600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf00ce340 con 0x7fbdf00716a0 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf5600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00cdeb0 con 0x7fbdf01652f0 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf4c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdf00716a0 0x7fbdf006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf4c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdf00716a0 0x7fbdf006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42844/0 (socket says 172.21.15.5:42844) 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.146+0000 7fbdf4c00640 1 -- 172.21.15.5:0/1570636365 learned_addr learned my addr 172.21.15.5:0/1570636365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.147+0000 7fbdefe00640 1 --2- 172.21.15.5:0/1570636365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf01652f0 0x7fbdf00cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.147+0000 7fbdf4c00640 1 -- 172.21.15.5:0/1570636365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf01652f0 msgr2=0x7fbdf00cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.147+0000 7fbdf4c00640 1 --2- 172.21.15.5:0/1570636365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf01652f0 0x7fbdf00cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.147+0000 7fbdf4c00640 1 -- 172.21.15.5:0/1570636365 >> v1:172.21.15.80:6789/0 conn(0x7fbdf006e980 legacy=0x7fbdf0070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.147+0000 7fbdf4c00640 1 -- 172.21.15.5:0/1570636365 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00cd580 con 0x7fbdf00716a0 2024-04-25T10:02:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.148+0000 7fbdf4c00640 1 --2- 172.21.15.5:0/1570636365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf00716a0 0x7fbdf006cff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009e50 tx=0x7fbde802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18db3a94de598938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.148+0000 7fbdeea00640 1 -- 172.21.15.5:0/1570636365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8036070 con 0x7fbdf00716a0 2024-04-25T10:02:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.148+0000 7fbdeea00640 1 -- 172.21.15.5:0/1570636365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde8031600 con 0x7fbdf00716a0 2024-04-25T10:02:04.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.148+0000 7fbdeea00640 1 -- 172.21.15.5:0/1570636365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde80319d0 con 0x7fbdf00716a0 2024-04-25T10:02:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 -- 172.21.15.5:0/1570636365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf00716a0 msgr2=0x7fbdf006cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 --2- 172.21.15.5:0/1570636365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf00716a0 0x7fbdf006cff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009e50 tx=0x7fbde802a710 comp rx=0 tx=0).stop 2024-04-25T10:02:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 -- 172.21.15.5:0/1570636365 shutdown_connections 2024-04-25T10:02:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 --2- 172.21.15.5:0/1570636365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf00716a0 0x7fbdf006cff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 --2- 172.21.15.5:0/1570636365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf01652f0 0x7fbdf00cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 -- 172.21.15.5:0/1570636365 >> 172.21.15.5:0/1570636365 conn(0x7fbdf0066f20 msgr2=0x7fbdf006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 -- 172.21.15.5:0/1570636365 shutdown_connections 2024-04-25T10:02:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.149+0000 7fbdf5600640 1 -- 172.21.15.5:0/1570636365 wait complete. 2024-04-25T10:02:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.151+0000 7fbdf5600640 1 Processor -- start 2024-04-25T10:02:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.151+0000 7fbdf5600640 1 -- start start 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 0x7fbdf01f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf00716a0 0x7fbdf01577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdf5600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdf01652f0 0x7fbdf015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdf5600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00cc8d0 con 0x7fbdf006e980 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdf5600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00cdb40 con 0x7fbdf01652f0 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdf5600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf006d530 con 0x7fbdf00716a0 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.152+0000 7fbdefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 0x7fbdf01f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 0x7fbdf01f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42858/0 (socket says 172.21.15.5:42858) 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 -- 172.21.15.5:0/2348120636 learned_addr learned my addr 172.21.15.5:0/2348120636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdef400640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf00716a0 0x7fbdf01577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdf4c00640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdf01652f0 0x7fbdf015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 -- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf00716a0 msgr2=0x7fbdf01577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf00716a0 0x7fbdf01577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 -- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdf01652f0 msgr2=0x7fbdf015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdf01652f0 0x7fbdf015d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 -- 172.21.15.5:0/2348120636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf0157d30 con 0x7fbdf006e980 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.153+0000 7fbdefe00640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 0x7fbdf01f83e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbde000d5f0 tx=0x7fbde000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.154+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00125f0 con 0x7fbdf006e980 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.154+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0019070 con 0x7fbdf006e980 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.154+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00129d0 con 0x7fbdf006e980 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.154+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf01f89c0 con 0x7fbdf006e980 2024-04-25T10:02:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.154+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf015c880 con 0x7fbdf006e980 2024-04-25T10:02:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.155+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf00cce40 con 0x7fbdf006e980 2024-04-25T10:02:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.156+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbde0012b70 con 0x7fbdf006e980 2024-04-25T10:02:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.157+0000 7fbde7e00640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd006b520 0x7fbdd006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.157+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbde008a820 con 0x7fbdf006e980 2024-04-25T10:02:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.158+0000 7fbdef400640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd006b520 0x7fbdd006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.159+0000 7fbdef400640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd006b520 0x7fbdd006d9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbdf015d840 tx=0x7fbddc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.161+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbde0059440 con 0x7fbdf006e980 2024-04-25T10:02:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.362+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdf015d5c0 con 0x7fbdf006e980 2024-04-25T10:02:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.364+0000 7fbde7e00640 1 -- 172.21.15.5:0/2348120636 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7fbde0058d40 con 0x7fbdf006e980 2024-04-25T10:02:04.369 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:04.369 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd006b520 msgr2=0x7fbdd006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd006b520 0x7fbdd006d9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbdf015d840 tx=0x7fbddc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 msgr2=0x7fbdf01f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 0x7fbdf01f83e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbde000d5f0 tx=0x7fbde000dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 shutdown_connections 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbdd006b520 0x7fbdd006d9e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdf01652f0 0x7fbdf015d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdf00716a0 0x7fbdf01577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 --2- 172.21.15.5:0/2348120636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdf006e980 0x7fbdf01f83e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.366+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 >> 172.21.15.5:0/2348120636 conn(0x7fbdf0066f20 msgr2=0x7fbdf00b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.367+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 shutdown_connections 2024-04-25T10:02:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.367+0000 7fbdf5600640 1 -- 172.21.15.5:0/2348120636 wait complete. 2024-04-25T10:02:04.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.600+0000 7f9d46800640 1 Processor -- start 2024-04-25T10:02:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.601+0000 7f9d46800640 1 -- start start 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.602+0000 7f9d46800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4015d7b0 0x7f9d4006fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.602+0000 7f9d46800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9d400b53c0 0x7f9d400b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d46800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d401618f0 con 0x7f9d4006ab60 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d46800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d40161bd0 con 0x7f9d400b53c0 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d46800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d40161eb0 con 0x7f9d4015d7b0 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9d400b53c0 0x7f9d400b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9d400b53c0 0x7f9d400b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42864/0 (socket says 172.21.15.5:42864) 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 -- 172.21.15.5:0/2020941743 learned_addr learned my addr 172.21.15.5:0/2020941743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 -- 172.21.15.5:0/2020941743 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4015d7b0 msgr2=0x7f9d4006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3f400640 1 --2- 172.21.15.5:0/2020941743 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4015d7b0 0x7f9d4006fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 --2- 172.21.15.5:0/2020941743 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4015d7b0 0x7f9d4006fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 -- 172.21.15.5:0/2020941743 >> v1:172.21.15.80:6789/0 conn(0x7f9d4006ab60 legacy=0x7f9d400b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.603+0000 7f9d3ea00640 1 -- 172.21.15.5:0/2020941743 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40162190 con 0x7f9d400b53c0 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.605+0000 7f9d3ea00640 1 --2- 172.21.15.5:0/2020941743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d400b5820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009870 tx=0x7f9d2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fe1be4abbbb284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.605+0000 7f9d3e000640 1 -- 172.21.15.5:0/2020941743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c037070 con 0x7f9d400b53c0 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.605+0000 7f9d3e000640 1 -- 172.21.15.5:0/2020941743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c032440 con 0x7f9d400b53c0 2024-04-25T10:02:04.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.605+0000 7f9d3e000640 1 -- 172.21.15.5:0/2020941743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c032810 con 0x7f9d400b53c0 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 -- 172.21.15.5:0/2020941743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 msgr2=0x7f9d400b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 --2- 172.21.15.5:0/2020941743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d400b5820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009870 tx=0x7f9d2c02a210 comp rx=0 tx=0).stop 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 -- 172.21.15.5:0/2020941743 shutdown_connections 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 --2- 172.21.15.5:0/2020941743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d400b5820 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 --2- 172.21.15.5:0/2020941743 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4015d7b0 0x7f9d4006fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 -- 172.21.15.5:0/2020941743 >> 172.21.15.5:0/2020941743 conn(0x7f9d400b2390 msgr2=0x7f9d400b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 -- 172.21.15.5:0/2020941743 shutdown_connections 2024-04-25T10:02:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.606+0000 7f9d46800640 1 -- 172.21.15.5:0/2020941743 wait complete. 2024-04-25T10:02:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.608+0000 7f9d46800640 1 Processor -- start 2024-04-25T10:02:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.608+0000 7f9d46800640 1 -- start start 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d46800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4006ab60 0x7f9d401e8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d46800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d4015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d46800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d4015d7b0 0x7f9d401e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d46800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4015ed20 con 0x7f9d400b53c0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d46800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4015eba0 con 0x7f9d4015d7b0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d46800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4006b010 con 0x7f9d4006ab60 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d4015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d4015f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42874/0 (socket says 172.21.15.5:42874) 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 -- 172.21.15.5:0/694227998 learned_addr learned my addr 172.21.15.5:0/694227998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 -- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4006ab60 msgr2=0x7f9d401e8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4006ab60 0x7f9d401e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 -- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d4015d7b0 msgr2=0x7f9d401e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d4015d7b0 0x7f9d401e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.609+0000 7f9d3ea00640 1 -- 172.21.15.5:0/694227998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4015fc70 con 0x7f9d400b53c0 2024-04-25T10:02:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.610+0000 7f9d3ea00640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d4015f4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c00 tx=0x7f9d2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.610+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c037040 con 0x7f9d400b53c0 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.610+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c042070 con 0x7f9d400b53c0 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.610+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c030bd0 con 0x7f9d400b53c0 2024-04-25T10:02:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.610+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d4015ff50 con 0x7f9d400b53c0 2024-04-25T10:02:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.610+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d40160510 con 0x7f9d400b53c0 2024-04-25T10:02:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.611+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4006b410 con 0x7f9d400b53c0 2024-04-25T10:02:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.613+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d2c047460 con 0x7f9d400b53c0 2024-04-25T10:02:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.613+0000 7f9d37400640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d2006b550 0x7f9d2006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.614+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9d2c0aaf80 con 0x7f9d400b53c0 2024-04-25T10:02:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.614+0000 7f9d3f400640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d2006b550 0x7f9d2006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.615+0000 7f9d3f400640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d2006b550 0x7f9d2006da10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d30006fd0 tx=0x7f9d30008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.618+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d4006b410 con 0x7f9d400b53c0 2024-04-25T10:02:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.820+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d40068fd0 con 0x7f9d400b53c0 2024-04-25T10:02:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.822+0000 7f9d37400640 1 -- 172.21.15.5:0/694227998 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f9d2c079c30 con 0x7f9d400b53c0 2024-04-25T10:02:04.828 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:04.828 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d2006b550 msgr2=0x7f9d2006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d2006b550 0x7f9d2006da10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9d30006fd0 tx=0x7f9d30008040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 msgr2=0x7f9d4015f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d4015f4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c000c00 tx=0x7f9d2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 shutdown_connections 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d2006b550 0x7f9d2006da10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d4015d7b0 0x7f9d401e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d400b53c0 0x7f9d4015f4d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 --2- 172.21.15.5:0/694227998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d4006ab60 0x7f9d401e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 >> 172.21.15.5:0/694227998 conn(0x7f9d400b2390 msgr2=0x7f9d4015bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 shutdown_connections 2024-04-25T10:02:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:04.826+0000 7f9d46800640 1 -- 172.21.15.5:0/694227998 wait complete. 2024-04-25T10:02:04.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.072+0000 7fb4fac00640 1 Processor -- start 2024-04-25T10:02:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.072+0000 7fb4fac00640 1 -- start start 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4fac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f41529e0 0x7fb4f40c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4fac00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb4f40c6410 0x7fb4f40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4fac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb4f4071ed0 0x7fb4f40ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4fac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f40b2340 con 0x7fb4f4071ed0 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4fac00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f415bdc0 con 0x7fb4f40c6410 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4fac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f415c060 con 0x7fb4f41529e0 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4f8c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb4f4071ed0 0x7fb4f40ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4f8c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb4f4071ed0 0x7fb4f40ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42882/0 (socket says 172.21.15.5:42882) 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4f8c00640 1 -- 172.21.15.5:0/4149904372 learned_addr learned my addr 172.21.15.5:0/4149904372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.074+0000 7fb4f3e00640 1 --2- 172.21.15.5:0/4149904372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f41529e0 0x7fb4f40c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f8c00640 1 -- 172.21.15.5:0/4149904372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f41529e0 msgr2=0x7fb4f40c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f3400640 1 --2- 172.21.15.5:0/4149904372 >> v2:172.21.15.80:3300/0 conn(0x7fb4f40c6410 0x7fb4f40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f8c00640 1 --2- 172.21.15.5:0/4149904372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f41529e0 0x7fb4f40c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f8c00640 1 -- 172.21.15.5:0/4149904372 >> v2:172.21.15.80:3300/0 conn(0x7fb4f40c6410 msgr2=0x7fb4f40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f8c00640 1 --2- 172.21.15.5:0/4149904372 >> v2:172.21.15.80:3300/0 conn(0x7fb4f40c6410 0x7fb4f40ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f8c00640 1 -- 172.21.15.5:0/4149904372 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f415c340 con 0x7fb4f4071ed0 2024-04-25T10:02:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.075+0000 7fb4f3400640 1 --2- 172.21.15.5:0/4149904372 >> v2:172.21.15.80:3300/0 conn(0x7fb4f40c6410 0x7fb4f40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.076+0000 7fb4f8c00640 1 --2- 172.21.15.5:0/4149904372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f4071ed0 0x7fb4f40ce890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40cf5a0 tx=0x7fb4ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99c6db8b3602e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.076+0000 7fb4f2a00640 1 -- 172.21.15.5:0/4149904372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec036070 con 0x7fb4f4071ed0 2024-04-25T10:02:05.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.076+0000 7fb4f2a00640 1 -- 172.21.15.5:0/4149904372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec031600 con 0x7fb4f4071ed0 2024-04-25T10:02:05.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.076+0000 7fb4f2a00640 1 -- 172.21.15.5:0/4149904372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0319d0 con 0x7fb4f4071ed0 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 -- 172.21.15.5:0/4149904372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f4071ed0 msgr2=0x7fb4f40ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 --2- 172.21.15.5:0/4149904372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f4071ed0 0x7fb4f40ce890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40cf5a0 tx=0x7fb4ec02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 -- 172.21.15.5:0/4149904372 shutdown_connections 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 --2- 172.21.15.5:0/4149904372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f4071ed0 0x7fb4f40ce890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 --2- 172.21.15.5:0/4149904372 >> v2:172.21.15.80:3300/0 conn(0x7fb4f40c6410 0x7fb4f40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 --2- 172.21.15.5:0/4149904372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f41529e0 0x7fb4f40c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 -- 172.21.15.5:0/4149904372 >> 172.21.15.5:0/4149904372 conn(0x7fb4f4071150 msgr2=0x7fb4f4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 -- 172.21.15.5:0/4149904372 shutdown_connections 2024-04-25T10:02:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.077+0000 7fb4fac00640 1 -- 172.21.15.5:0/4149904372 wait complete. 2024-04-25T10:02:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.079+0000 7fb4fac00640 1 Processor -- start 2024-04-25T10:02:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.079+0000 7fb4fac00640 1 -- start start 2024-04-25T10:02:05.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4fac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f4071ed0 0x7fb4f4148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4fac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 0x7fb4f4158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4fac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f4156d40 0x7fb4f40b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4fac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40c4660 con 0x7fb4f40c6410 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4fac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f415c900 con 0x7fb4f4071ed0 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4fac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f40c47e0 con 0x7fb4f4156d40 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 0x7fb4f4158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 0x7fb4f4158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42898/0 (socket says 172.21.15.5:42898) 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 -- 172.21.15.5:0/2230794098 learned_addr learned my addr 172.21.15.5:0/2230794098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3e00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f4071ed0 0x7fb4f4148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f8c00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f4156d40 0x7fb4f40b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 -- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f4156d40 msgr2=0x7fb4f40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f4156d40 0x7fb4f40b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 -- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f4071ed0 msgr2=0x7fb4f4148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f4071ed0 0x7fb4f4148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f3400640 1 -- 172.21.15.5:0/2230794098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f41571f0 con 0x7fb4f40c6410 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.081+0000 7fb4f8c00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f4156d40 0x7fb4f40b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.082+0000 7fb4f3400640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 0x7fb4f4158cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000c7f0 tx=0x7fb4e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.082+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0015070 con 0x7fb4f40c6410 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.082+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e0007a30 con 0x7fb4f40c6410 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.082+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0007e00 con 0x7fb4f40c6410 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.082+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f41574b0 con 0x7fb4f40c6410 2024-04-25T10:02:05.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.082+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f4157a70 con 0x7fb4f40c6410 2024-04-25T10:02:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.084+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f4156030 con 0x7fb4f40c6410 2024-04-25T10:02:05.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.085+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4e0018070 con 0x7fb4f40c6410 2024-04-25T10:02:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.086+0000 7fb4ebe00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb4d406b4d0 0x7fb4d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.086+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb4e008a590 con 0x7fb4f40c6410 2024-04-25T10:02:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.086+0000 7fb4f3e00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb4d406b4d0 0x7fb4d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.087+0000 7fb4f3e00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb4d406b4d0 0x7fb4d406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006fd0 tx=0x7fb4e4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.090+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4f4156030 con 0x7fb4f40c6410 2024-04-25T10:02:05.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.323+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4f4068fd0 con 0x7fb4f40c6410 2024-04-25T10:02:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.324+0000 7fb4ebe00640 1 -- 172.21.15.5:0/2230794098 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4925 (secure 0 0 0) 0x7fb4e0059030 con 0x7fb4f40c6410 2024-04-25T10:02:05.329 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:05.329 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:02.841787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:02:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb4d406b4d0 msgr2=0x7fb4d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb4d406b4d0 0x7fb4d406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006fd0 tx=0x7fb4e4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 msgr2=0x7fb4f4158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 0x7fb4f4158cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000c7f0 tx=0x7fb4e000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 shutdown_connections 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb4f4156d40 0x7fb4f40b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4f40c6410 0x7fb4f4158cb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb4d406b4d0 0x7fb4d406d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 --2- 172.21.15.5:0/2230794098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f4071ed0 0x7fb4f4148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.326+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 >> 172.21.15.5:0/2230794098 conn(0x7fb4f4071150 msgr2=0x7fb4f40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.327+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 shutdown_connections 2024-04-25T10:02:05.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.327+0000 7fb4fac00640 1 -- 172.21.15.5:0/2230794098 wait complete. 2024-04-25T10:02:05.351 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.556+0000 7f5b03000640 1 Processor -- start 2024-04-25T10:02:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.556+0000 7f5b03000640 1 -- start start 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b03000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc141da0 0x7f5afc06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b03000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5afc06e640 0x7f5afc15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b03000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5afc15d340 0x7f5afc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b03000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc142180 con 0x7f5afc06e640 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b03000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc071d30 con 0x7f5afc15d340 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b03000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc06d620 con 0x7f5afc141da0 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b01200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5afc06e640 0x7f5afc15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b01200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5afc06e640 0x7f5afc15ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42900/0 (socket says 172.21.15.5:42900) 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b01200640 1 -- 172.21.15.5:0/1573475178 learned_addr learned my addr 172.21.15.5:0/1573475178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b02600640 1 --2- 172.21.15.5:0/1573475178 >> v2:172.21.15.80:3300/0 conn(0x7f5afc15d340 0x7f5afc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.558+0000 7f5b01c00640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc141da0 0x7f5afc06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.559+0000 7f5b01200640 1 -- 172.21.15.5:0/1573475178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc141da0 msgr2=0x7f5afc06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.559+0000 7f5b01200640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc141da0 0x7f5afc06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.559+0000 7f5b01200640 1 -- 172.21.15.5:0/1573475178 >> v2:172.21.15.80:3300/0 conn(0x7f5afc15d340 msgr2=0x7f5afc0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.559+0000 7f5b01200640 1 --2- 172.21.15.5:0/1573475178 >> v2:172.21.15.80:3300/0 conn(0x7f5afc15d340 0x7f5afc0cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.559+0000 7f5b01200640 1 -- 172.21.15.5:0/1573475178 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc06d900 con 0x7f5afc06e640 2024-04-25T10:02:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.559+0000 7f5b01c00640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc141da0 0x7f5afc06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.560+0000 7f5b01200640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc06e640 0x7f5afc15ce00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0b2460 tx=0x7f5ae802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731c21ccab6f28ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.560+0000 7f5afbe00640 1 -- 172.21.15.5:0/1573475178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8037070 con 0x7f5afc06e640 2024-04-25T10:02:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.560+0000 7f5afbe00640 1 -- 172.21.15.5:0/1573475178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae8032440 con 0x7f5afc06e640 2024-04-25T10:02:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.560+0000 7f5afbe00640 1 -- 172.21.15.5:0/1573475178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8032810 con 0x7f5afc06e640 2024-04-25T10:02:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 -- 172.21.15.5:0/1573475178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc06e640 msgr2=0x7f5afc15ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc06e640 0x7f5afc15ce00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0b2460 tx=0x7f5ae802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 -- 172.21.15.5:0/1573475178 shutdown_connections 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 --2- 172.21.15.5:0/1573475178 >> v2:172.21.15.80:3300/0 conn(0x7f5afc15d340 0x7f5afc0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc06e640 0x7f5afc15ce00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 --2- 172.21.15.5:0/1573475178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc141da0 0x7f5afc06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 -- 172.21.15.5:0/1573475178 >> 172.21.15.5:0/1573475178 conn(0x7f5afc066f20 msgr2=0x7f5afc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 -- 172.21.15.5:0/1573475178 shutdown_connections 2024-04-25T10:02:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.561+0000 7f5b03000640 1 -- 172.21.15.5:0/1573475178 wait complete. 2024-04-25T10:02:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.563+0000 7f5b03000640 1 Processor -- start 2024-04-25T10:02:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.563+0000 7f5b03000640 1 -- start start 2024-04-25T10:02:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b03000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc06e640 0x7f5afc152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b03000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 0x7f5afc0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b03000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc15d340 0x7f5afc0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b03000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0cd9c0 con 0x7f5afc141da0 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b03000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0cd840 con 0x7f5afc15d340 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b03000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc0cdb40 con 0x7f5afc06e640 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 0x7f5afc0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 0x7f5afc0c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42906/0 (socket says 172.21.15.5:42906) 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 -- 172.21.15.5:0/1845207322 learned_addr learned my addr 172.21.15.5:0/1845207322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b02600640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc15d340 0x7f5afc0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01c00640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc06e640 0x7f5afc152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 -- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc06e640 msgr2=0x7f5afc152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc06e640 0x7f5afc152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 -- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc15d340 msgr2=0x7f5afc0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc15d340 0x7f5afc0c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b01200640 1 -- 172.21.15.5:0/1845207322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0b18c0 con 0x7f5afc141da0 2024-04-25T10:02:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.564+0000 7f5b02600640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc15d340 0x7f5afc0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.565+0000 7f5b01200640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 0x7f5afc0c4c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ae802a220 tx=0x7f5ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.565+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8041020 con 0x7f5afc141da0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.565+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae80306c0 con 0x7f5afc141da0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.565+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8037040 con 0x7f5afc141da0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.565+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc0b1ba0 con 0x7f5afc141da0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.565+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc13d670 con 0x7f5afc141da0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.566+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc068f60 con 0x7f5afc141da0 2024-04-25T10:02:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.567+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ae8044020 con 0x7f5afc141da0 2024-04-25T10:02:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.568+0000 7f5af9600640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5adc06b4d0 0x7f5adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.568+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5ae80aad60 con 0x7f5afc141da0 2024-04-25T10:02:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.569+0000 7f5b01c00640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5adc06b4d0 0x7f5adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.570+0000 7f5b01c00640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5adc06b4d0 0x7f5adc06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5af0008850 tx=0x7f5af0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.573+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ae804b030 con 0x7f5afc141da0 2024-04-25T10:02:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.775+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5afc15b180 con 0x7f5afc141da0 2024-04-25T10:02:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.777+0000 7f5af9600640 1 -- 172.21.15.5:0/1845207322 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f5ae8079890 con 0x7f5afc141da0 2024-04-25T10:02:05.783 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:05.783 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5adc06b4d0 msgr2=0x7f5adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5adc06b4d0 0x7f5adc06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5af0008850 tx=0x7f5af0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 msgr2=0x7f5afc0c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 0x7f5afc0c4c30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ae802a220 tx=0x7f5ae80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 shutdown_connections 2024-04-25T10:02:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5adc06b4d0 0x7f5adc06d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5afc15d340 0x7f5afc0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc141da0 0x7f5afc0c4c30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 --2- 172.21.15.5:0/1845207322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5afc06e640 0x7f5afc152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 >> 172.21.15.5:0/1845207322 conn(0x7f5afc066f20 msgr2=0x7f5afc15e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 shutdown_connections 2024-04-25T10:02:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:05.780+0000 7f5b03000640 1 -- 172.21.15.5:0/1845207322 wait complete. 2024-04-25T10:02:06.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:57.496056+0000', 'modified': '2024-04-25T10:02:02.841787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6837/312213316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 312213316}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 312213316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T10:02:06.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:06.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:02:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.422+0000 7fe9ce400640 1 Processor -- start 2024-04-25T10:02:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.423+0000 7fe9ce400640 1 -- start start 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.424+0000 7fe9ce400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe9c80d07d0 0x7fe9c80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.424+0000 7fe9ce400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c80cf410 con 0x7fe9c8161200 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.424+0000 7fe9ce400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c80cf6f0 con 0x7fe9c80d2380 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.424+0000 7fe9ce400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c80b28b0 con 0x7fe9c80d07d0 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.424+0000 7fe9cd000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe9c8161200 0x7fe9c80d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59580/0 (socket says 172.21.15.5:59580) 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.424+0000 7fe9cd000640 1 -- 172.21.15.5:0/4145943918 learned_addr learned my addr 172.21.15.5:0/4145943918 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/4145943918 >> v2:172.21.15.80:3300/0 conn(0x7fe9c80d07d0 0x7fe9c80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419160038 0 0) 0x7fe9c80cf410 con 0x7fe9c8161200 2024-04-25T10:02:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ac003520 con 0x7fe9c8161200 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 101406253 0 0) 0x7fe9ac003520 con 0x7fe9c8161200 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 >> v2:172.21.15.80:3300/0 conn(0x7fe9c80d07d0 msgr2=0x7fe9c80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 --2- 172.21.15.5:0/4145943918 >> v2:172.21.15.80:3300/0 conn(0x7fe9c80d07d0 0x7fe9c80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 >> v1:172.21.15.80:6789/0 conn(0x7fe9c80d2380 legacy=0x7fe9c80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80d3fd0 con 0x7fe9c8161200 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe9b0001670 con 0x7fe9c8161200 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.425+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 >> v1:172.21.15.5:6789/0 conn(0x7fe9c8161200 legacy=0x7fe9c80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7400640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac003a00 0x7fe9ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7400640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac004800 0x7fe9ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7400640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0051c0 0x7fe9ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b28b0 con 0x7fe9ac0051c0 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac00c0c0 con 0x7fe9ac004800 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac00c2c0 con 0x7fe9ac003a00 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cda00640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0051c0 0x7fe9ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cd000640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac003a00 0x7fe9ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac004800 0x7fe9ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cda00640 1 -- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac003a00 msgr2=0x7fe9ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cda00640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac003a00 0x7fe9ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cda00640 1 -- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac004800 msgr2=0x7fe9ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cda00640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac004800 0x7fe9ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.426+0000 7fe9cda00640 1 -- 172.21.15.5:0/4145943918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac003520 con 0x7fe9ac0051c0 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.427+0000 7fe9cda00640 1 -- 172.21.15.5:0/4145943918 reap_dead start 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.427+0000 7fe9cda00640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0051c0 0x7fe9ac00b9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0001750 tx=0x7fe9ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fe3e13c943f11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0001370 con 0x7fe9ac0051c0 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0005310 con 0x7fe9ac0051c0 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9c7400640 1 -- 172.21.15.5:0/4145943918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c002abe0 con 0x7fe9ac0051c0 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 -- 172.21.15.5:0/4145943918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0051c0 msgr2=0x7fe9ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0051c0 0x7fe9ac00b9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0001750 tx=0x7fe9ac005830 comp rx=0 tx=0).stop 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 -- 172.21.15.5:0/4145943918 shutdown_connections 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 --2- 172.21.15.5:0/4145943918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0051c0 0x7fe9ac00b9c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 -- 172.21.15.5:0/4145943918 >> 172.21.15.5:0/4145943918 conn(0x7fe9c808af80 msgr2=0x7fe9c80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 -- 172.21.15.5:0/4145943918 shutdown_connections 2024-04-25T10:02:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.428+0000 7fe9ce400640 1 -- 172.21.15.5:0/4145943918 wait complete. 2024-04-25T10:02:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.430+0000 7fe9ce400640 1 Processor -- start 2024-04-25T10:02:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.430+0000 7fe9ce400640 1 -- start start 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.431+0000 7fe9ce400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0051c0 0x7fe9c815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.431+0000 7fe9ce400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9c80cb7d0 0x7fe9c809d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.431+0000 7fe9ce400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 0x7fe9c809dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.431+0000 7fe9ce400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b4d30 con 0x7fe9c80cbc10 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.431+0000 7fe9ce400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b4bb0 con 0x7fe9c80cb7d0 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.431+0000 7fe9ce400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b4eb0 con 0x7fe9ac0051c0 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 0x7fe9c809dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 0x7fe9c809dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42912/0 (socket says 172.21.15.5:42912) 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 -- 172.21.15.5:0/3642373756 learned_addr learned my addr 172.21.15.5:0/3642373756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9c80cb7d0 0x7fe9c809d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cd000640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0051c0 0x7fe9c815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 -- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0051c0 msgr2=0x7fe9c815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0051c0 0x7fe9c815dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 -- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9c80cb7d0 msgr2=0x7fe9c809d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9c80cb7d0 0x7fe9c809d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 -- 172.21.15.5:0/3642373756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80cc190 con 0x7fe9c80cbc10 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9c80cb7d0 0x7fe9c809d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.432+0000 7fe9cda00640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 0x7fe9c809dbd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9c002a590 tx=0x7fe9c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.433+0000 7fe9c4c00640 1 -- 172.21.15.5:0/3642373756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0001370 con 0x7fe9c80cbc10 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.433+0000 7fe9c4c00640 1 -- 172.21.15.5:0/3642373756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c002ae20 con 0x7fe9c80cbc10 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.433+0000 7fe9c4c00640 1 -- 172.21.15.5:0/3642373756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0007a30 con 0x7fe9c80cbc10 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.433+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c80cc470 con 0x7fe9c80cbc10 2024-04-25T10:02:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.433+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c80cca30 con 0x7fe9c80cbc10 2024-04-25T10:02:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.435+0000 7fe9c4c00640 1 -- 172.21.15.5:0/3642373756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9c0007030 con 0x7fe9c80cbc10 2024-04-25T10:02:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.436+0000 7fe9c4c00640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe9a406b4d0 0x7fe9a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.436+0000 7fe9c4c00640 1 -- 172.21.15.5:0/3642373756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9c00a9960 con 0x7fe9c80cbc10 2024-04-25T10:02:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.437+0000 7fe9cd000640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe9a406b4d0 0x7fe9a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.437+0000 7fe9d2875740 1 Processor -- start 2024-04-25T10:02:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.437+0000 7fe9d2875740 1 -- start start 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c3806cb30 0x560c38094970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c3806cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c380b57a0 0x560c3806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9d2875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37fd0620 con 0x560c38094eb0 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37f79ca0 con 0x560c380b57a0 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c37fcc6c0 con 0x560c3806cb30 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9cd000640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe9a406b4d0 0x7fe9a406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001eb0 tx=0x7fe9b0000f30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c3806cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c3806cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42922/0 (socket says 172.21.15.5:42922) 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1469542045 learned_addr learned my addr 172.21.15.5:0/1469542045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9cda00640 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c380b57a0 0x560c3806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9cd000640 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c3806cb30 0x560c38094970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c3806cb30 msgr2=0x560c38094970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c3806cb30 0x560c38094970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c380b57a0 msgr2=0x560c3806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c380b57a0 0x560c3806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1469542045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c37f80f40 con 0x560c38094eb0 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.438+0000 7fe9cda00640 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c380b57a0 0x560c3806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c3806cf10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8007460 tx=0x7fe9b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c72bd83505f558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9bd600640 1 -- 172.21.15.5:0/1469542045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b801a020 con 0x560c38094eb0 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9bd600640 1 -- 172.21.15.5:0/1469542045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8016870 con 0x560c38094eb0 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9bd600640 1 -- 172.21.15.5:0/1469542045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8016c20 con 0x560c38094eb0 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9d2875740 1 -- 172.21.15.5:0/1469542045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 msgr2=0x560c3806cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c3806cf10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8007460 tx=0x7fe9b8009750 comp rx=0 tx=0).stop 2024-04-25T10:02:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9d2875740 1 -- 172.21.15.5:0/1469542045 shutdown_connections 2024-04-25T10:02:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c380b57a0 0x560c3806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c3806cf10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.439+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1469542045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c3806cb30 0x560c38094970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.440+0000 7fe9d2875740 1 -- 172.21.15.5:0/1469542045 >> 172.21.15.5:0/1469542045 conn(0x560c38069d50 msgr2=0x560c380b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.440+0000 7fe9d2875740 1 -- 172.21.15.5:0/1469542045 shutdown_connections 2024-04-25T10:02:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.440+0000 7fe9d2875740 1 -- 172.21.15.5:0/1469542045 wait complete. 2024-04-25T10:02:06.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.440+0000 7fe9d2875740 1 -- start start 2024-04-25T10:02:06.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.440+0000 7fe9d2875740 1 Processor -- start 2024-04-25T10:02:06.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.441+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.441+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38092050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.441+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c38092590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.441+0000 7fe9d2875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c380944e0 con 0x560c38094eb0 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.441+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37f6f1f0 con 0x560c3808e620 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.441+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c38093ed0 con 0x560c380b57a0 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9d2875740 10 client.0 _ll_register_callbacks cb 0x560c37f8cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42936/0 (socket says 172.21.15.5:42936) 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 -- 172.21.15.5:0/2318669189 learned_addr learned my addr 172.21.15.5:0/2318669189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38092050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 -- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 msgr2=0x560c38092050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cda00640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c38092590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38092050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 -- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 msgr2=0x560c38092590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c38092590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 -- 172.21.15.5:0/2318669189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c37f8ec50 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38092050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9cd000640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380962e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000ffe0 tx=0x7fe9b0009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24983d887bba055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.442+0000 7fe9b6000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9b6000640 1 -- 172.21.15.5:0/2318669189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0010040 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9b6000640 1 -- 172.21.15.5:0/2318669189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9b0022dc0 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b8016870 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9b6000640 1 -- 172.21.15.5:0/2318669189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b0023430 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 10 client.4395 fetch_fsmap learned FSMap version 6 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x560c37ce52d0 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9b6000640 1 -- 172.21.15.5:0/2318669189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8016870 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9b6000640 1 -- 172.21.15.5:0/2318669189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fe9b0013070 con 0x560c38094eb0 2024-04-25T10:02:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 10 client.4395 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 10 client.4395 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 20 client.4395 populate_metadata read hostname 'smithi005' 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c38099210 0x560c380995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 4 client.4395 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.443+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 --> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c37fddb60 con 0x560c38099210 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.444+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c38099210 0x560c380995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.445+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c38099210 0x560c380995f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd86e8ba52198502 server_cookie=556fc74f654ea9b7 in_seq=0 out_seq=0 2024-04-25T10:02:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.445+0000 7fe9b6000640 10 client.4395 ms_handle_connect on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9b6000640 1 -- 172.21.15.5:0/2318669189 <== mds.0 v2:172.21.15.80:6836/312213316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c37fddb60 con 0x560c38099210 2024-04-25T10:02:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9b6000640 10 client.4395 handle_command_reply: tid=0 2024-04-25T10:02:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 client.4395 shutdown 2024-04-25T10:02:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 msgr2=0x560c380962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380962e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000ffe0 tx=0x7fe9b0009070 comp rx=0 tx=0).stop 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 shutdown_connections 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c38099210 0x560c380995f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c38092590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38092050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2318669189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380962e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 >> 172.21.15.5:0/2318669189 conn(0x560c3806cb30 msgr2=0x560c380b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9b6000640 0 client.4395 ms_handle_reset on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.446+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 shutdown_connections 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.447+0000 7fe9d2875740 1 -- 172.21.15.5:0/2318669189 wait complete. 2024-04-25T10:02:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.447+0000 7fe9d2875740 20 client.4395 trim_cache size 0 max 16384 2024-04-25T10:02:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.464+0000 7fe9d2875740 1 Processor -- start 2024-04-25T10:02:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.464+0000 7fe9d2875740 1 -- start start 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 0x560c3808d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c38094eb0 0x560c38066aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c380b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9d2875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c8200930 con 0x560c3808e620 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b7c00 con 0x560c38094eb0 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c8200330 con 0x560c380b57a0 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 0x560c3808d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 0x560c3808d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42938/0 (socket says 172.21.15.5:42938) 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 -- 172.21.15.5:0/2232452885 learned_addr learned my addr 172.21.15.5:0/2232452885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 -- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 msgr2=0x560c380b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cda00640 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c380b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c38094eb0 0x560c38066aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c380b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 -- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c38094eb0 msgr2=0x560c38066aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c38094eb0 0x560c38066aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cd000640 1 -- 172.21.15.5:0/2232452885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c37daf6b0 con 0x560c3808e620 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.465+0000 7fe9cda00640 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c380b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.466+0000 7fe9cd000640 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 0x560c3808d540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0006fd0 tx=0x7fe9b0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47a447c3c6647ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.466+0000 7fe9bcc00640 1 -- 172.21.15.5:0/2232452885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0031030 con 0x560c3808e620 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.466+0000 7fe9bcc00640 1 -- 172.21.15.5:0/2232452885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0016110 con 0x560c3808e620 2024-04-25T10:02:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.466+0000 7fe9bcc00640 1 -- 172.21.15.5:0/2232452885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0022880 con 0x560c3808e620 2024-04-25T10:02:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 -- 172.21.15.5:0/2232452885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 msgr2=0x560c3808d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 0x560c3808d540 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0006fd0 tx=0x7fe9b0013000 comp rx=0 tx=0).stop 2024-04-25T10:02:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 -- 172.21.15.5:0/2232452885 shutdown_connections 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c380b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c38094eb0 0x560c38066aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 --2- 172.21.15.5:0/2232452885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c3808e620 0x560c3808d540 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 -- 172.21.15.5:0/2232452885 >> 172.21.15.5:0/2232452885 conn(0x560c3806cb30 msgr2=0x560c38063300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 -- 172.21.15.5:0/2232452885 shutdown_connections 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 -- 172.21.15.5:0/2232452885 wait complete. 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.467+0000 7fe9d2875740 1 -- start start 2024-04-25T10:02:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.468+0000 7fe9d2875740 1 Processor -- start 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c380b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38091f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c380c46b0 con 0x560c38094eb0 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c38097c20 con 0x560c3808e620 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c380c3a40 con 0x560c380b57a0 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9d2875740 10 client.0 _ll_register_callbacks cb 0x560c38094e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42942/0 (socket says 172.21.15.5:42942) 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1980032609 learned_addr learned my addr 172.21.15.5:0/1980032609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9cd000640 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c380b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 msgr2=0x560c38091f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38091f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 msgr2=0x560c380b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c380b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.469+0000 7fe9c7e00640 1 -- 172.21.15.5:0/1980032609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c3808fa50 con 0x560c38094eb0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9c7e00640 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380ccca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8018000 tx=0x7fe9b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1be60a26c89f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9b6000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9b6000640 1 -- 172.21.15.5:0/1980032609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b801a040 con 0x560c38094eb0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9b6000640 1 -- 172.21.15.5:0/1980032609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9b8003990 con 0x560c38094eb0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b0016110 con 0x560c38094eb0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9b6000640 1 -- 172.21.15.5:0/1980032609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b8003b90 con 0x560c38094eb0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9d2875740 10 client.4401 fetch_fsmap learned FSMap version 6 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.470+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x560c3811e270 con 0x560c38094eb0 2024-04-25T10:02:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9b6000640 1 -- 172.21.15.5:0/1980032609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0016110 con 0x560c38094eb0 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9b6000640 1 -- 172.21.15.5:0/1980032609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fe9b80167d0 con 0x560c38094eb0 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9d2875740 10 client.4401 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9d2875740 10 client.4401 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9d2875740 20 client.4401 populate_metadata read hostname 'smithi005' 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c3806b2f0 0x560c3806b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9d2875740 4 client.4401 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 --> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c380688c0 con 0x560c3806b2f0 2024-04-25T10:02:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.471+0000 7fe9cd000640 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c3806b2f0 0x560c3806b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.472+0000 7fe9cd000640 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c3806b2f0 0x560c3806b6d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2924c927bb9283 server_cookie=117ba4c0530c74bb in_seq=0 out_seq=0 2024-04-25T10:02:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.472+0000 7fe9b6000640 10 client.4401 ms_handle_connect on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9b6000640 1 -- 172.21.15.5:0/1980032609 <== mds.0 v2:172.21.15.80:6836/312213316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560c380688c0 con 0x560c3806b2f0 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9b6000640 10 client.4401 handle_command_reply: tid=0 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 client.4401 shutdown 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 msgr2=0x560c380ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380ccca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8018000 tx=0x7fe9b8004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 shutdown_connections 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x560c3806b2f0 0x560c3806b6d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560c380b57a0 0x560c38091f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c38094eb0 0x560c380ccca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 --2- 172.21.15.5:0/1980032609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560c3808e620 0x560c380b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 >> 172.21.15.5:0/1980032609 conn(0x560c3806cb30 msgr2=0x560c38099ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9b6000640 0 client.4401 ms_handle_reset on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 shutdown_connections 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 1 -- 172.21.15.5:0/1980032609 wait complete. 2024-04-25T10:02:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.473+0000 7fe9d2875740 20 client.4401 trim_cache size 0 max 16384 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4282, 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 8.3730170909999995, 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 6, 2024-04-25T10:02:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 26, 2024-04-25T10:02:06.483 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T10:02:06.483 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 13.684612529000001 2024-04-25T10:02:06.483 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:02:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe9a406b4d0 msgr2=0x7fe9a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe9a406b4d0 0x7fe9a406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0001eb0 tx=0x7fe9b0000f30 comp rx=0 tx=0).stop 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 msgr2=0x7fe9c809dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 0x7fe9c809dbd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9c002a590 tx=0x7fe9c0001120 comp rx=0 tx=0).stop 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 shutdown_connections 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c80cbc10 0x7fe9c809dbd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe9a406b4d0 0x7fe9a406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9c80cb7d0 0x7fe9c809d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 --2- 172.21.15.5:0/3642373756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0051c0 0x7fe9c815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 >> 172.21.15.5:0/3642373756 conn(0x7fe9c808af80 msgr2=0x7fe9c80d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 shutdown_connections 2024-04-25T10:02:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.474+0000 7fe9ce400640 1 -- 172.21.15.5:0/3642373756 wait complete. 2024-04-25T10:02:06.496 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:06.497 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:06.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.684+0000 7fca39c00640 1 Processor -- start 2024-04-25T10:02:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.684+0000 7fca39c00640 1 -- start start 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.685+0000 7fca39c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca341529e0 0x7fca340c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.685+0000 7fca39c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca340cea60 0x7fca3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.685+0000 7fca39c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3415bdc0 con 0x7fca340c6410 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.685+0000 7fca39c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca340b2340 con 0x7fca341529e0 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.685+0000 7fca39c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3415c060 con 0x7fca340cea60 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca341529e0 0x7fca340c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fca340c6410 0x7fca340ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59594/0 (socket says 172.21.15.5:59594) 2024-04-25T10:02:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32000640 1 -- 172.21.15.5:0/2488975586 learned_addr learned my addr 172.21.15.5:0/2488975586 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca33400640 1 --2- 172.21.15.5:0/2488975586 >> v2:172.21.15.80:3300/0 conn(0x7fca340cea60 0x7fca3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca31600640 1 -- 172.21.15.5:0/2488975586 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066098609 0 0) 0x7fca3415bdc0 con 0x7fca340c6410 2024-04-25T10:02:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca31600640 1 -- 172.21.15.5:0/2488975586 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca18003520 con 0x7fca340c6410 2024-04-25T10:02:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32a00640 1 -- 172.21.15.5:0/2488975586 >> v2:172.21.15.80:3300/0 conn(0x7fca340cea60 msgr2=0x7fca3406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32a00640 1 --2- 172.21.15.5:0/2488975586 >> v2:172.21.15.80:3300/0 conn(0x7fca340cea60 0x7fca3406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32a00640 1 -- 172.21.15.5:0/2488975586 >> v1:172.21.15.5:6789/0 conn(0x7fca340c6410 legacy=0x7fca340ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca32a00640 1 -- 172.21.15.5:0/2488975586 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3415c340 con 0x7fca341529e0 2024-04-25T10:02:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.686+0000 7fca33400640 1 --2- 172.21.15.5:0/2488975586 >> v2:172.21.15.80:3300/0 conn(0x7fca340cea60 0x7fca3406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.688+0000 7fca32a00640 1 --2- 172.21.15.5:0/2488975586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca341529e0 0x7fca340c5ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca340c78c0 tx=0x7fca2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1427aab09fcffed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.688+0000 7fca31600640 1 -- 172.21.15.5:0/2488975586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20037070 con 0x7fca341529e0 2024-04-25T10:02:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.688+0000 7fca31600640 1 -- 172.21.15.5:0/2488975586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20032440 con 0x7fca341529e0 2024-04-25T10:02:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.688+0000 7fca31600640 1 -- 172.21.15.5:0/2488975586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca200327f0 con 0x7fca341529e0 2024-04-25T10:02:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.688+0000 7fca39c00640 1 -- 172.21.15.5:0/2488975586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca341529e0 msgr2=0x7fca340c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.688+0000 7fca39c00640 1 --2- 172.21.15.5:0/2488975586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca341529e0 0x7fca340c5ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca340c78c0 tx=0x7fca2002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.689+0000 7fca39c00640 1 -- 172.21.15.5:0/2488975586 shutdown_connections 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.689+0000 7fca39c00640 1 --2- 172.21.15.5:0/2488975586 >> v2:172.21.15.80:3300/0 conn(0x7fca340cea60 0x7fca3406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.689+0000 7fca39c00640 1 --2- 172.21.15.5:0/2488975586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca341529e0 0x7fca340c5ed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.689+0000 7fca39c00640 1 -- 172.21.15.5:0/2488975586 >> 172.21.15.5:0/2488975586 conn(0x7fca34071150 msgr2=0x7fca34071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.689+0000 7fca39c00640 1 -- 172.21.15.5:0/2488975586 shutdown_connections 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.689+0000 7fca39c00640 1 -- 172.21.15.5:0/2488975586 wait complete. 2024-04-25T10:02:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.690+0000 7fca39c00640 1 Processor -- start 2024-04-25T10:02:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.691+0000 7fca39c00640 1 -- start start 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca39c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 0x7fca3413ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca39c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca340cea60 0x7fca340b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca39c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca340b3d60 0x7fca340c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca39c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca34071ed0 con 0x7fca340c6410 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca39c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3415c900 con 0x7fca340b3d60 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca39c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca34072050 con 0x7fca340cea60 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 0x7fca3413ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 0x7fca3413ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42954/0 (socket says 172.21.15.5:42954) 2024-04-25T10:02:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 -- 172.21.15.5:0/169899036 learned_addr learned my addr 172.21.15.5:0/169899036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 -- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca340cea60 msgr2=0x7fca340b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca340cea60 0x7fca340b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 -- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca340b3d60 msgr2=0x7fca340c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca340b3d60 0x7fca340c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.692+0000 7fca32a00640 1 -- 172.21.15.5:0/169899036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340b4300 con 0x7fca340c6410 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.693+0000 7fca32a00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 0x7fca3413ce70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fca2002a1e0 tx=0x7fca200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.693+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20037040 con 0x7fca340c6410 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.693+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca200304c0 con 0x7fca340c6410 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.693+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20030890 con 0x7fca340c6410 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.693+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca340b45e0 con 0x7fca340c6410 2024-04-25T10:02:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.693+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca340b4ba0 con 0x7fca340c6410 2024-04-25T10:02:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.694+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34068fd0 con 0x7fca340c6410 2024-04-25T10:02:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.695+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca20044020 con 0x7fca340c6410 2024-04-25T10:02:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.696+0000 7fca2aa00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca1406b520 0x7fca1406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.696+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fca200aa850 con 0x7fca340c6410 2024-04-25T10:02:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.697+0000 7fca32000640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca1406b520 0x7fca1406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.698+0000 7fca32000640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca1406b520 0x7fca1406d9e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca24000f60 tx=0x7fca240078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.701+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca20079400 con 0x7fca340c6410 2024-04-25T10:02:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.943+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca34073bf0 con 0x7fca340c6410 2024-04-25T10:02:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.944+0000 7fca2aa00640 1 -- 172.21.15.5:0/169899036 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7fca20078d00 con 0x7fca340c6410 2024-04-25T10:02:06.950 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca1406b520 msgr2=0x7fca1406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca1406b520 0x7fca1406d9e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fca24000f60 tx=0x7fca240078f0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 msgr2=0x7fca3413ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 0x7fca3413ce70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fca2002a1e0 tx=0x7fca200046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 shutdown_connections 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca1406b520 0x7fca1406d9e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca340b3d60 0x7fca340c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca340cea60 0x7fca340b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 --2- 172.21.15.5:0/169899036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca340c6410 0x7fca3413ce70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 >> 172.21.15.5:0/169899036 conn(0x7fca34071150 msgr2=0x7fca3415aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 shutdown_connections 2024-04-25T10:02:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:06.948+0000 7fca39c00640 1 -- 172.21.15.5:0/169899036 wait complete. 2024-04-25T10:02:06.972 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:06.972 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:06.972 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:02:06.972 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T10:02:06.972 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:06.973 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:06.973 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:06.973 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:06.973 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:06.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:06.973 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:02:07.023 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:02:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:02:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42599sec preferred_lft 42599sec 2024-04-25T10:02:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:02:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:07.025 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:02:07.027 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:07.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:07.028 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:07.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.458 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:07.530 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:02:07.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.534 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:02:07.586 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:02:07.587 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:07.587 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:02:07.587 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:02:07.587 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:07.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:07.588 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:07.588 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:07.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:02:07.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:02:07.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:02:07.588 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:08.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.251 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:02:08.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.310 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-04-25T10:02:08.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.367 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:08.367 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:08.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:08.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T10:02:08.367 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:08.494 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T10:02:08.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T10:02:08.494 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:08.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:08.731 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:08.731 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:08.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:02:08.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:08.732 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:08.876 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T10:02:08.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:08.876 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:08.925 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:08.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:08.925 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:08.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-04-25T10:02:09.069 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:09.128 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-04-25T10:02:09.128 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-04-25T10:02:09.128 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-04-25T10:02:09.128 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-04-25T10:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:09.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.131 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-04-25T10:02:09.188 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:09.188 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T10:02:09.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-04-25T10:02:09.263 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:09.304 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-04-25T10:02:09.304 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-04-25T10:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:09.306 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-04-25T10:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:09.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.308 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-04-25T10:02:09.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:2024-04-25T10:02:09.372+0000 7f095aae7480 -1 init, newargv = 0x561763a69260 newargc=15 2024-04-25T10:02:09.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[38732]: starting ceph client 2024-04-25T10:02:09.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[38732]: starting fuse 2024-04-25T10:02:09.393 INFO:teuthology.orchestra.run.smithi005.stdout:41 2024-04-25T10:02:09.393 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:02:09.393 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-04-25T10:02:09.394 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-04-25T10:02:09.395 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:09.395 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:02:09.395 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.admin")) 2024-04-25T10:02:09.395 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:02:09.589 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.admin.38732.asok 2024-04-25T10:02:09.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:09.606 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.38732.asok 2024-04-25T10:02:09.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.38732.asok status 2024-04-25T10:02:09.838 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "admin", 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "38732", 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4416, 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 4416 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 620657088 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:02:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 620657088 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.4416 192.168.0.1:0/620657088", 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/620657088", 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 6, 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 26, 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-04-25T10:02:09.841 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:02:09.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.854 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-04-25T10:02:09.854 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:09.912 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-04-25T10:02:09.912 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:09.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:09.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:10.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:10.007 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T10:02:10.099 INFO:teuthology.orchestra.run.smithi005.stdout:# file: . 2024-04-25T10:02:10.100 INFO:teuthology.orchestra.run.smithi005.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T10:02:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:10.101 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:02:10.101 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:10.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:10.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.351+0000 7f2d0e800640 1 Processor -- start 2024-04-25T10:02:10.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.351+0000 7f2d0e800640 1 -- start start 2024-04-25T10:02:10.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.353+0000 7f2d0e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 0x7f2d0809a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.353+0000 7f2d0e800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080cfad0 con 0x7f2d0815da20 2024-04-25T10:02:10.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.353+0000 7f2d0e800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080cfdb0 con 0x7f2d0806c330 2024-04-25T10:02:10.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.353+0000 7f2d0e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080d06e0 con 0x7f2d0806bc70 2024-04-25T10:02:10.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.353+0000 7f2d07400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2d0815da20 0x7f2d080d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59078/0 (socket says 172.21.15.5:59078) 2024-04-25T10:02:10.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.353+0000 7f2d07400640 1 -- 172.21.15.5:0/2853171620 learned_addr learned my addr 172.21.15.5:0/2853171620 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:10.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2cffe00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 0x7f2d0809a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239774599 0 0) 0x7f2d080cfad0 con 0x7f2d0815da20 2024-04-25T10:02:10.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cec003520 con 0x7f2d0815da20 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943668004 0 0) 0x7f2cec003520 con 0x7f2d0815da20 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 msgr2=0x7f2d0809a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 0x7f2d0809a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 >> v1:172.21.15.80:6789/0 conn(0x7f2d0806c330 legacy=0x7f2d080cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080d0090 con 0x7f2d0815da20 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2cf8001670 con 0x7f2d0815da20 2024-04-25T10:02:10.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.354+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 >> v1:172.21.15.5:6789/0 conn(0x7f2d0815da20 legacy=0x7f2d080d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d06a00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec0039e0 0x7f2cec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d06a00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 0x7f2cec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d06a00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cec004d40 0x7f2cec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080d06e0 con 0x7f2cec004d40 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec00baa0 con 0x7f2cec0039e0 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec00bca0 con 0x7f2d0806bc70 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d07e00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cec004d40 0x7f2cec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2d07400640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec0039e0 0x7f2cec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.355+0000 7f2cffe00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 0x7f2cec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.356+0000 7f2d07e00640 1 -- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 msgr2=0x7f2cec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.356+0000 7f2d07e00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0806bc70 0x7f2cec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.356+0000 7f2d07e00640 1 -- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec0039e0 msgr2=0x7f2cec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.356+0000 7f2d07e00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec0039e0 0x7f2cec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.356+0000 7f2d07e00640 1 -- 172.21.15.5:0/2853171620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec003520 con 0x7f2cec004d40 2024-04-25T10:02:10.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.357+0000 7f2d07e00640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cec004d40 0x7f2cec0051c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d00001e20 tx=0x7f2cec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592d92e399584255 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.358+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00001a70 con 0x7f2cec004d40 2024-04-25T10:02:10.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.358+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0002ae40 con 0x7f2cec004d40 2024-04-25T10:02:10.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.358+0000 7f2d06a00640 1 -- 172.21.15.5:0/2853171620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00003bb0 con 0x7f2cec004d40 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d0e800640 1 -- 172.21.15.5:0/2853171620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cec004d40 msgr2=0x7f2cec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d0e800640 1 --2- 172.21.15.5:0/2853171620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cec004d40 0x7f2cec0051c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d00001e20 tx=0x7f2cec005830 comp rx=0 tx=0).stop 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d07e00640 1 -- 172.21.15.5:0/2853171620 reap_dead start 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d0e800640 1 -- 172.21.15.5:0/2853171620 shutdown_connections 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d0e800640 1 -- 172.21.15.5:0/2853171620 >> 172.21.15.5:0/2853171620 conn(0x7f2d080d16f0 msgr2=0x7f2d080d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d0e800640 1 -- 172.21.15.5:0/2853171620 shutdown_connections 2024-04-25T10:02:10.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.359+0000 7f2d0e800640 1 -- 172.21.15.5:0/2853171620 wait complete. 2024-04-25T10:02:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.362+0000 7f2d0e800640 1 Processor -- start 2024-04-25T10:02:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.362+0000 7f2d0e800640 1 -- start start 2024-04-25T10:02:10.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.363+0000 7f2d0e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec004d40 0x7f2d080b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.363+0000 7f2d0e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0814cf60 0x7f2d080c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d0e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 0x7f2d0814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d0e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815de00 con 0x7f2d0814f7d0 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d0e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080d1280 con 0x7f2cec004d40 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d0e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0815df80 con 0x7f2d0814cf60 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d07e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 0x7f2d0814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d07e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 0x7f2d0814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39302/0 (socket says 172.21.15.5:39302) 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d07e00640 1 -- 172.21.15.5:0/3643451390 learned_addr learned my addr 172.21.15.5:0/3643451390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2d07400640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec004d40 0x7f2d080b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.364+0000 7f2cffe00640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0814cf60 0x7f2d080c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2d07e00640 1 -- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0814cf60 msgr2=0x7f2d080c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2d07e00640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0814cf60 0x7f2d080c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2d07e00640 1 -- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec004d40 msgr2=0x7f2d080b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2d07e00640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec004d40 0x7f2d080b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2d07e00640 1 -- 172.21.15.5:0/3643451390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b6b40 con 0x7f2d0814f7d0 2024-04-25T10:02:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2d07e00640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 0x7f2d0814fc50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2d00001f10 tx=0x7f2d000014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.365+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00041070 con 0x7f2d0814f7d0 2024-04-25T10:02:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.366+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00006b50 con 0x7f2d0814f7d0 2024-04-25T10:02:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.366+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00037040 con 0x7f2d0814f7d0 2024-04-25T10:02:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.366+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d080c4fc0 con 0x7f2d0814f7d0 2024-04-25T10:02:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.366+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080b6580 con 0x7f2d0814f7d0 2024-04-25T10:02:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.367+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d08068fd0 con 0x7f2d0814f7d0 2024-04-25T10:02:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.369+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d00044070 con 0x7f2d0814f7d0 2024-04-25T10:02:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.370+0000 7f2cff400640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ce406b520 0x7f2ce406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.370+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2d000a98b0 con 0x7f2d0814f7d0 2024-04-25T10:02:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.371+0000 7f2d07400640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ce406b520 0x7f2ce406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.373+0000 7f2d07400640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ce406b520 0x7f2ce406d9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8001eb0 tx=0x7f2cf8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.378+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d000784d0 con 0x7f2d0814f7d0 2024-04-25T10:02:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.581+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d080d2500 con 0x7f2d0814f7d0 2024-04-25T10:02:10.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.583+0000 7f2cff400640 1 -- 172.21.15.5:0/3643451390 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f2d00077dd0 con 0x7f2d0814f7d0 2024-04-25T10:02:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:10.588 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:10.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.586+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ce406b520 msgr2=0x7f2ce406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.586+0000 7f2d0e800640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ce406b520 0x7f2ce406d9e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8001eb0 tx=0x7f2cf8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:10.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.586+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 msgr2=0x7f2d0814fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.586+0000 7f2d0e800640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 0x7f2d0814fc50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2d00001f10 tx=0x7f2d000014a0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 shutdown_connections 2024-04-25T10:02:10.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d0814f7d0 0x7f2d0814fc50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d0814cf60 0x7f2d080c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ce406b520 0x7f2ce406d9e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 --2- 172.21.15.5:0/3643451390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2cec004d40 0x7f2d080b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 >> 172.21.15.5:0/3643451390 conn(0x7f2d080d16f0 msgr2=0x7f2d080d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 shutdown_connections 2024-04-25T10:02:10.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.587+0000 7f2d0e800640 1 -- 172.21.15.5:0/3643451390 wait complete. 2024-04-25T10:02:10.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:10.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.822+0000 7fb0bf200640 1 Processor -- start 2024-04-25T10:02:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.822+0000 7fb0bf200640 1 -- start start 2024-04-25T10:02:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bf200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0b80cc840 0x7fb0b806e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 0x7fb0b8071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bf200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b80ce570 con 0x7fb0b806e670 2024-04-25T10:02:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bf200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80ce850 con 0x7fb0b80cc840 2024-04-25T10:02:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80ceb30 con 0x7fb0b815b500 2024-04-25T10:02:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bd400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb0b806e670 0x7fb0b815adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59092/0 (socket says 172.21.15.5:59092) 2024-04-25T10:02:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.824+0000 7fb0bd400640 1 -- 172.21.15.5:0/2026032192 learned_addr learned my addr 172.21.15.5:0/2026032192 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bde00640 1 --2- 172.21.15.5:0/2026032192 >> v2:172.21.15.80:3300/0 conn(0x7fb0b80cc840 0x7fb0b806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0be800640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 0x7fb0b8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082788899 0 0) 0x7fb0b80ce570 con 0x7fb0b806e670 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a4005730 con 0x7fb0b806e670 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336237090 0 0) 0x7fb0a4005730 con 0x7fb0b806e670 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 msgr2=0x7fb0b8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 0x7fb0b8071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 >> v2:172.21.15.80:3300/0 conn(0x7fb0b80cc840 msgr2=0x7fb0b806e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 --2- 172.21.15.5:0/2026032192 >> v2:172.21.15.80:3300/0 conn(0x7fb0b80cc840 0x7fb0b806e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80cee10 con 0x7fb0b806e670 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb0a8001670 con 0x7fb0b806e670 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 >> v1:172.21.15.5:6789/0 conn(0x7fb0b806e670 legacy=0x7fb0b815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4005bc0 0x7fb0a4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0a4006a10 0x7fb0a4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 0x7fb0a40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80ce850 con 0x7fb0a4006a10 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80ceb30 con 0x7fb0a4005bc0 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.825+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a400dbd0 con 0x7fb0b815b500 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bd400640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0a4006a10 0x7fb0a4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bde00640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4005bc0 0x7fb0a4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0be800640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 0x7fb0a40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bd400640 1 -- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 msgr2=0x7fb0a40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bd400640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b815b500 0x7fb0a40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bd400640 1 -- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4005bc0 msgr2=0x7fb0a4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bd400640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4005bc0 0x7fb0a4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.826+0000 7fb0bd400640 1 -- 172.21.15.5:0/2026032192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4005730 con 0x7fb0a4006a10 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.827+0000 7fb0bd400640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0a4006a10 0x7fb0a4006e90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007fd0 tx=0x7fb0a4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5303f57585c17037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8007d50 con 0x7fb0a4006a10 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8005ad0 con 0x7fb0a4006a10 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bca00640 1 -- 172.21.15.5:0/2026032192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8007070 con 0x7fb0a4006a10 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bf200640 1 -- 172.21.15.5:0/2026032192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0a4006a10 msgr2=0x7fb0a4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bf200640 1 --2- 172.21.15.5:0/2026032192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0a4006a10 0x7fb0a4006e90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8007fd0 tx=0x7fb0a4007a40 comp rx=0 tx=0).stop 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0be800640 1 -- 172.21.15.5:0/2026032192 reap_dead start 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bf200640 1 -- 172.21.15.5:0/2026032192 shutdown_connections 2024-04-25T10:02:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.828+0000 7fb0bf200640 1 -- 172.21.15.5:0/2026032192 >> 172.21.15.5:0/2026032192 conn(0x7fb0b8066f20 msgr2=0x7fb0b806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.829+0000 7fb0bf200640 1 -- 172.21.15.5:0/2026032192 shutdown_connections 2024-04-25T10:02:10.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.829+0000 7fb0bf200640 1 -- 172.21.15.5:0/2026032192 wait complete. 2024-04-25T10:02:10.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.830+0000 7fb0bf200640 1 Processor -- start 2024-04-25T10:02:10.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.830+0000 7fb0bf200640 1 -- start start 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.831+0000 7fb0bf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4006a10 0x7fb0b8143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.831+0000 7fb0bf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 0x7fb0b815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.831+0000 7fb0bf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b8156220 0x7fb0b815c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.831+0000 7fb0bf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80ce350 con 0x7fb0b8155da0 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.831+0000 7fb0bf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80ce1d0 con 0x7fb0a4006a10 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.831+0000 7fb0bf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b80ccc20 con 0x7fb0b8156220 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.832+0000 7fb0bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 0x7fb0b815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.832+0000 7fb0bde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4006a10 0x7fb0b8143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.832+0000 7fb0bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 0x7fb0b815c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39320/0 (socket says 172.21.15.5:39320) 2024-04-25T10:02:10.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.832+0000 7fb0bd400640 1 -- 172.21.15.5:0/4084537261 learned_addr learned my addr 172.21.15.5:0/4084537261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.832+0000 7fb0be800640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b8156220 0x7fb0b815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bd400640 1 -- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b8156220 msgr2=0x7fb0b815c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bd400640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b8156220 0x7fb0b815c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bd400640 1 -- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4006a10 msgr2=0x7fb0b8143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bd400640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4006a10 0x7fb0b8143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bd400640 1 -- 172.21.15.5:0/4084537261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80c87a0 con 0x7fb0b8155da0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bd400640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 0x7fb0b815c400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c300 tx=0x7fb0b80b49f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8042070 con 0x7fb0b8155da0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8007480 con 0x7fb0b8155da0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8001cb0 con 0x7fb0b8155da0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b80c8a40 con 0x7fb0b8155da0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.833+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b80c9000 con 0x7fb0b8155da0 2024-04-25T10:02:10.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.835+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b8068f60 con 0x7fb0b8155da0 2024-04-25T10:02:10.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.836+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0a8045070 con 0x7fb0b8155da0 2024-04-25T10:02:10.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.837+0000 7fb0ae000640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb09c06b4d0 0x7fb09c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.837+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb0a80ab0e0 con 0x7fb0b8155da0 2024-04-25T10:02:10.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.837+0000 7fb0bde00640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb09c06b4d0 0x7fb09c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.838+0000 7fb0bde00640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb09c06b4d0 0x7fb09c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0003b80 tx=0x7fb0b000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:10.841+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0a804c030 con 0x7fb0b8155da0 2024-04-25T10:02:11.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.038+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0b8154c60 con 0x7fb0b8155da0 2024-04-25T10:02:11.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.039+0000 7fb0ae000640 1 -- 172.21.15.5:0/4084537261 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0a8079d30 con 0x7fb0b8155da0 2024-04-25T10:02:11.043 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb09c06b4d0 msgr2=0x7fb09c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb09c06b4d0 0x7fb09c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0003b80 tx=0x7fb0b000b040 comp rx=0 tx=0).stop 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 msgr2=0x7fb0b815c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 0x7fb0b815c400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c300 tx=0x7fb0b80b49f0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 shutdown_connections 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0b8156220 0x7fb0b815c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155da0 0x7fb0b815c400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb09c06b4d0 0x7fb09c06d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 --2- 172.21.15.5:0/4084537261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a4006a10 0x7fb0b8143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 >> 172.21.15.5:0/4084537261 conn(0x7fb0b8066f20 msgr2=0x7fb0b815adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 shutdown_connections 2024-04-25T10:02:11.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.042+0000 7fb0bf200640 1 -- 172.21.15.5:0/4084537261 wait complete. 2024-04-25T10:02:11.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.270+0000 7fee1ee00640 1 Processor -- start 2024-04-25T10:02:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.270+0000 7fee1ee00640 1 -- start start 2024-04-25T10:02:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee1ee00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fee180b8720 0x7fee180d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee1ee00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180d29c0 con 0x7fee1815db70 2024-04-25T10:02:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee1ee00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180d2ca0 con 0x7fee180cefb0 2024-04-25T10:02:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee1ee00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee180b6ce0 con 0x7fee180b8720 2024-04-25T10:02:11.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee17e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fee1815db70 0x7fee180ce8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59106/0 (socket says 172.21.15.5:59106) 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee17e00640 1 -- 172.21.15.5:0/2651485027 learned_addr learned my addr 172.21.15.5:0/2651485027 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.272+0000 7fee1ce00640 1 --2- 172.21.15.5:0/2651485027 >> v2:172.21.15.5:3300/0 conn(0x7fee180b8720 0x7fee180d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.273+0000 7fee16a00640 1 -- 172.21.15.5:0/2651485027 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908151032 0 0) 0x7fee180d29c0 con 0x7fee1815db70 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.273+0000 7fee16a00640 1 -- 172.21.15.5:0/2651485027 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedfc003520 con 0x7fee1815db70 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.273+0000 7fee1ce00640 1 -- 172.21.15.5:0/2651485027 >> v1:172.21.15.80:6789/0 conn(0x7fee180cefb0 legacy=0x7fee180d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.273+0000 7fee1ce00640 1 -- 172.21.15.5:0/2651485027 >> v1:172.21.15.5:6789/0 conn(0x7fee1815db70 legacy=0x7fee180ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.273+0000 7fee1ce00640 1 -- 172.21.15.5:0/2651485027 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180b6f10 con 0x7fee180b8720 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.274+0000 7fee1ce00640 1 --2- 172.21.15.5:0/2651485027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180d2380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee10009d90 tx=0x7fee1002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498dc64a4f74816f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.274+0000 7fee16a00640 1 -- 172.21.15.5:0/2651485027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10007740 con 0x7fee180b8720 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee16a00640 1 -- 172.21.15.5:0/2651485027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee100078e0 con 0x7fee180b8720 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee16a00640 1 -- 172.21.15.5:0/2651485027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10007cb0 con 0x7fee180b8720 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 -- 172.21.15.5:0/2651485027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 msgr2=0x7fee180d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 --2- 172.21.15.5:0/2651485027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180d2380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee10009d90 tx=0x7fee1002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 -- 172.21.15.5:0/2651485027 shutdown_connections 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 --2- 172.21.15.5:0/2651485027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180d2380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 -- 172.21.15.5:0/2651485027 >> 172.21.15.5:0/2651485027 conn(0x7fee180b5370 msgr2=0x7fee180b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 -- 172.21.15.5:0/2651485027 shutdown_connections 2024-04-25T10:02:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.275+0000 7fee1ee00640 1 -- 172.21.15.5:0/2651485027 wait complete. 2024-04-25T10:02:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.277+0000 7fee1ee00640 1 Processor -- start 2024-04-25T10:02:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.277+0000 7fee1ee00640 1 -- start start 2024-04-25T10:02:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee1ee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee1ee00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fee180cefb0 0x7fee1814f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee1ee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee1815db70 0x7fee18152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee1ee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180d2f80 con 0x7fee180b8720 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee1ee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1815e1a0 con 0x7fee1815db70 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee1ee00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee180b7280 con 0x7fee180cefb0 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.278+0000 7fee17e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39338/0 (socket says 172.21.15.5:39338) 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 -- 172.21.15.5:0/3410224324 learned_addr learned my addr 172.21.15.5:0/3410224324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee1ce00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee1815db70 0x7fee18152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17400640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fee180cefb0 0x7fee1814f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 -- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fee180cefb0 msgr2=0x7fee1814f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fee180cefb0 0x7fee1814f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 -- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee1815db70 msgr2=0x7fee18152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee1815db70 0x7fee18152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 -- 172.21.15.5:0/3410224324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee181525b0 con 0x7fee180b8720 2024-04-25T10:02:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.279+0000 7fee17e00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180ca930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c410 tx=0x7fee0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.280+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee08017070 con 0x7fee180b8720 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.280+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee08005ce0 con 0x7fee180b8720 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.280+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee080072f0 con 0x7fee180b8720 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.280+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee181517f0 con 0x7fee180b8720 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.280+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee18151db0 con 0x7fee180b8720 2024-04-25T10:02:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.281+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee18068f60 con 0x7fee180b8720 2024-04-25T10:02:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.282+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee0801a070 con 0x7fee180b8720 2024-04-25T10:02:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.283+0000 7fee0fe00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedf406b4d0 0x7fedf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.283+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fee08089a10 con 0x7fee180b8720 2024-04-25T10:02:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.284+0000 7fee17400640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedf406b4d0 0x7fedf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.285+0000 7fee17400640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedf406b4d0 0x7fedf406d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fee180c9130 tx=0x7fee00009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.287+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee08058600 con 0x7fee180b8720 2024-04-25T10:02:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.492+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee180d0ea0 con 0x7fee180b8720 2024-04-25T10:02:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.492+0000 7fee0fe00640 1 -- 172.21.15.5:0/3410224324 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee08057f00 con 0x7fee180b8720 2024-04-25T10:02:11.497 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedf406b4d0 msgr2=0x7fedf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedf406b4d0 0x7fedf406d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fee180c9130 tx=0x7fee00009040 comp rx=0 tx=0).stop 2024-04-25T10:02:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 msgr2=0x7fee180ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180ca930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c410 tx=0x7fee0800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 shutdown_connections 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedf406b4d0 0x7fedf406d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fee1815db70 0x7fee18152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fee180cefb0 0x7fee1814f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 --2- 172.21.15.5:0/3410224324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee180b8720 0x7fee180ca930 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 >> 172.21.15.5:0/3410224324 conn(0x7fee180b5370 msgr2=0x7fee180b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 shutdown_connections 2024-04-25T10:02:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.495+0000 7fee1ee00640 1 -- 172.21.15.5:0/3410224324 wait complete. 2024-04-25T10:02:11.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.732+0000 7fc6e1800640 1 Processor -- start 2024-04-25T10:02:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.732+0000 7fc6e1800640 1 -- start start 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6e1800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc6dc154590 0x7fc6dc15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6e1800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 0x7fc6dc0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6e1800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc15d9f0 con 0x7fc6dc071f00 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6e1800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc072370 con 0x7fc6dc154590 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6e1800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc15e5e0 con 0x7fc6dc0ae9e0 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6da000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 0x7fc6dc0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6daa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc6dc154590 0x7fc6dc15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6da000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 0x7fc6dc0719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:42984/0 (socket says 172.21.15.5:42984) 2024-04-25T10:02:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.734+0000 7fc6da000640 1 -- 172.21.15.5:0/2809940932 learned_addr learned my addr 172.21.15.5:0/2809940932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.735+0000 7fc6daa00640 1 -- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 msgr2=0x7fc6dc0719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.735+0000 7fc6daa00640 1 --2- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 0x7fc6dc0719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.735+0000 7fc6daa00640 1 -- 172.21.15.5:0/2809940932 >> v1:172.21.15.80:6789/0 conn(0x7fc6dc071f00 legacy=0x7fc6dc06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.735+0000 7fc6daa00640 1 -- 172.21.15.5:0/2809940932 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc15dcb0 con 0x7fc6dc154590 2024-04-25T10:02:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.735+0000 7fc6da000640 1 --2- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 0x7fc6dc0719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.736+0000 7fc6daa00640 1 --2- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc154590 0x7fc6dc15ae30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15e760 tx=0x7fc6cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98def9534a0c34fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.737+0000 7fc6d9600640 1 -- 172.21.15.5:0/2809940932 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc037070 con 0x7fc6dc154590 2024-04-25T10:02:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.737+0000 7fc6d9600640 1 -- 172.21.15.5:0/2809940932 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc032440 con 0x7fc6dc154590 2024-04-25T10:02:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.737+0000 7fc6d9600640 1 -- 172.21.15.5:0/2809940932 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc032810 con 0x7fc6dc154590 2024-04-25T10:02:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.737+0000 7fc6e1800640 1 -- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc154590 msgr2=0x7fc6dc15ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.737+0000 7fc6e1800640 1 --2- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc154590 0x7fc6dc15ae30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15e760 tx=0x7fc6cc02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.738+0000 7fc6e1800640 1 -- 172.21.15.5:0/2809940932 shutdown_connections 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.738+0000 7fc6e1800640 1 --2- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc0ae9e0 0x7fc6dc0719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.738+0000 7fc6e1800640 1 --2- 172.21.15.5:0/2809940932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc154590 0x7fc6dc15ae30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.738+0000 7fc6e1800640 1 -- 172.21.15.5:0/2809940932 >> 172.21.15.5:0/2809940932 conn(0x7fc6dc06ab60 msgr2=0x7fc6dc06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.738+0000 7fc6e1800640 1 -- 172.21.15.5:0/2809940932 shutdown_connections 2024-04-25T10:02:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.738+0000 7fc6e1800640 1 -- 172.21.15.5:0/2809940932 wait complete. 2024-04-25T10:02:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.740+0000 7fc6e1800640 1 Processor -- start 2024-04-25T10:02:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.740+0000 7fc6e1800640 1 -- start start 2024-04-25T10:02:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6e1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 0x7fc6dc155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6e1800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc0ae9e0 0x7fc6dc1443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6e1800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc154590 0x7fc6dc15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6e1800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc15e270 con 0x7fc6dc071f00 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6e1800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0b4b60 con 0x7fc6dc0ae9e0 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6e1800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0ced50 con 0x7fc6dc154590 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6daa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 0x7fc6dc155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6daa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 0x7fc6dc155840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39342/0 (socket says 172.21.15.5:39342) 2024-04-25T10:02:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.741+0000 7fc6daa00640 1 -- 172.21.15.5:0/1539356507 learned_addr learned my addr 172.21.15.5:0/1539356507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6da000640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc0ae9e0 0x7fc6dc1443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6db400640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc154590 0x7fc6dc15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6daa00640 1 -- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc154590 msgr2=0x7fc6dc15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6daa00640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc154590 0x7fc6dc15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6daa00640 1 -- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc0ae9e0 msgr2=0x7fc6dc1443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6daa00640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc0ae9e0 0x7fc6dc1443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6daa00640 1 -- 172.21.15.5:0/1539356507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc154a80 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.742+0000 7fc6daa00640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 0x7fc6dc155840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009890 tx=0x7fc6cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.743+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc037050 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.743+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc038040 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.743+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc154d60 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.743+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc03cb50 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.743+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc155320 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.744+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc068f60 con 0x7fc6dc071f00 2024-04-25T10:02:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.745+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6cc03ccf0 con 0x7fc6dc071f00 2024-04-25T10:02:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.746+0000 7fc6d2a00640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6bc06b500 0x7fc6bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.746+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6cc0aafe0 con 0x7fc6dc071f00 2024-04-25T10:02:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.747+0000 7fc6da000640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6bc06b500 0x7fc6bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.748+0000 7fc6da000640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6bc06b500 0x7fc6bc06d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc14c9e0 tx=0x7fc6c8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.750+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6cc079aa0 con 0x7fc6dc071f00 2024-04-25T10:02:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.948+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6dc147c60 con 0x7fc6dc071f00 2024-04-25T10:02:11.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.949+0000 7fc6d2a00640 1 -- 172.21.15.5:0/1539356507 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6cc0793a0 con 0x7fc6dc071f00 2024-04-25T10:02:11.953 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.951+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6bc06b500 msgr2=0x7fc6bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.951+0000 7fc6e1800640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6bc06b500 0x7fc6bc06d9c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc14c9e0 tx=0x7fc6c8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.951+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 msgr2=0x7fc6dc155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.951+0000 7fc6e1800640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 0x7fc6dc155840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc009890 tx=0x7fc6cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 shutdown_connections 2024-04-25T10:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6dc154590 0x7fc6dc15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6bc06b500 0x7fc6bc06d9c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6dc0ae9e0 0x7fc6dc1443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 --2- 172.21.15.5:0/1539356507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6dc071f00 0x7fc6dc155840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 >> 172.21.15.5:0/1539356507 conn(0x7fc6dc06ab60 msgr2=0x7fc6dc06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 shutdown_connections 2024-04-25T10:02:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:11.952+0000 7fc6e1800640 1 -- 172.21.15.5:0/1539356507 wait complete. 2024-04-25T10:02:11.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:12.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.183+0000 7f15bbe00640 1 Processor -- start 2024-04-25T10:02:12.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.183+0000 7f15bbe00640 1 -- start start 2024-04-25T10:02:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.185+0000 7f15bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 0x7f15bc06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.185+0000 7f15bbe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0ade10 con 0x7f15bc071710 2024-04-25T10:02:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.185+0000 7f15bbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0ae0f0 con 0x7f15bc06ec70 2024-04-25T10:02:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.185+0000 7f15bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0ae3d0 con 0x7f15bc143f30 2024-04-25T10:02:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.185+0000 7f15bb400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f15bc071710 0x7f15bc0b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59108/0 (socket says 172.21.15.5:59108) 2024-04-25T10:02:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.185+0000 7f15bb400640 1 -- 172.21.15.5:0/4137618770 learned_addr learned my addr 172.21.15.5:0/4137618770 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15baa00640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 0x7f15bc06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250556470 0 0) 0x7f15bc0ade10 con 0x7f15bc071710 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a0003520 con 0x7f15bc071710 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1173886166 0 0) 0x7f15a0003520 con 0x7f15bc071710 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 msgr2=0x7f15bc06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 0x7f15bc06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 >> v1:172.21.15.80:6789/0 conn(0x7f15bc06ec70 legacy=0x7f15bc071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0ae6b0 con 0x7f15bc071710 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f15b4001670 con 0x7f15bc071710 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 >> v1:172.21.15.5:6789/0 conn(0x7f15bc071710 legacy=0x7f15bc0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 0x7f15a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15a0003f60 0x7f15a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0ae3d0 con 0x7f15a0004d40 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a000bb50 con 0x7f15a0003f60 2024-04-25T10:02:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.186+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a000bd50 con 0x7f15bc143f30 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15bb400640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15baa00640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 0x7f15a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15ba000640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15a0003f60 0x7f15a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15bb400640 1 -- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 msgr2=0x7f15a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15bb400640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc143f30 0x7f15a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15bb400640 1 -- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15a0003f60 msgr2=0x7f15a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15bb400640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15a0003f60 0x7f15a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15bb400640 1 -- 172.21.15.5:0/4137618770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0003520 con 0x7f15a0004d40 2024-04-25T10:02:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.187+0000 7f15ba000640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15a0003f60 0x7f15a0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.188+0000 7f15bb400640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c240 tx=0x7f15a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5254513071eca491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.189+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4007d10 con 0x7f15a0004d40 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.189+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4005ad0 con 0x7f15a0004d40 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.189+0000 7f15b9600640 1 -- 172.21.15.5:0/4137618770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4007070 con 0x7f15a0004d40 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.189+0000 7f15bbe00640 1 -- 172.21.15.5:0/4137618770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 msgr2=0x7f15a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.189+0000 7f15bbe00640 1 --2- 172.21.15.5:0/4137618770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15a00051c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c240 tx=0x7f15a0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.189+0000 7f15bb400640 1 -- 172.21.15.5:0/4137618770 reap_dead start 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.190+0000 7f15bbe00640 1 -- 172.21.15.5:0/4137618770 shutdown_connections 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.190+0000 7f15bbe00640 1 -- 172.21.15.5:0/4137618770 >> 172.21.15.5:0/4137618770 conn(0x7f15bc066f20 msgr2=0x7f15bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.190+0000 7f15bbe00640 1 -- 172.21.15.5:0/4137618770 shutdown_connections 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.190+0000 7f15bbe00640 1 -- 172.21.15.5:0/4137618770 wait complete. 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.191+0000 7f15bbe00640 1 Processor -- start 2024-04-25T10:02:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.192+0000 7f15bbe00640 1 -- start start 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15bc15fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc1f3bf0 0x7f15bc1f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc1f4330 0x7f15bc1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0aed50 con 0x7f15a0004d40 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0aebd0 con 0x7f15bc1f3bf0 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0aeed0 con 0x7f15bc1f4330 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15bc15fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15bc15fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39364/0 (socket says 172.21.15.5:39364) 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 -- 172.21.15.5:0/3348466286 learned_addr learned my addr 172.21.15.5:0/3348466286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15bb400640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc1f4330 0x7f15bc1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15ba000640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc1f3bf0 0x7f15bc1f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 -- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc1f4330 msgr2=0x7f15bc1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc1f4330 0x7f15bc1613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 -- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc1f3bf0 msgr2=0x7f15bc1f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc1f3bf0 0x7f15bc1f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15baa00640 1 -- 172.21.15.5:0/3348466286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc160330 con 0x7f15a0004d40 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.193+0000 7f15ba000640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc1f3bf0 0x7f15bc1f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.194+0000 7f15baa00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15bc15fc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011e30 tx=0x7f15ac00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.194+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac015630 con 0x7f15a0004d40 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.194+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac0157d0 con 0x7f15a0004d40 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.194+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac015ba0 con 0x7f15a0004d40 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.194+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc160610 con 0x7f15a0004d40 2024-04-25T10:02:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.194+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc160bd0 con 0x7f15a0004d40 2024-04-25T10:02:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.196+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc068f60 con 0x7f15a0004d40 2024-04-25T10:02:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.196+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15ac01d400 con 0x7f15a0004d40 2024-04-25T10:02:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.197+0000 7f15b2a00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b500 0x7f159c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.197+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15ac08b730 con 0x7f15a0004d40 2024-04-25T10:02:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.198+0000 7f15ba000640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b500 0x7f159c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.199+0000 7f15ba000640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b500 0x7f159c06d9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15bc145c90 tx=0x7f15a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.202+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15ac019030 con 0x7f15a0004d40 2024-04-25T10:02:12.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.399+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f15bc06bfe0 con 0x7f15a0004d40 2024-04-25T10:02:12.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.400+0000 7f15b2a00640 1 -- 172.21.15.5:0/3348466286 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f15ac010040 con 0x7f15a0004d40 2024-04-25T10:02:12.404 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:12.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b500 msgr2=0x7f159c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b500 0x7f159c06d9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15bc145c90 tx=0x7f15a8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 msgr2=0x7f15bc15fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15bc15fc50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011e30 tx=0x7f15ac00f580 comp rx=0 tx=0).stop 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 shutdown_connections 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc1f4330 0x7f15bc1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b500 0x7f159c06d9c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc1f3bf0 0x7f15bc1f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 --2- 172.21.15.5:0/3348466286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a0004d40 0x7f15bc15fc50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 >> 172.21.15.5:0/3348466286 conn(0x7f15bc066f20 msgr2=0x7f15bc146380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 shutdown_connections 2024-04-25T10:02:12.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.403+0000 7f15bbe00640 1 -- 172.21.15.5:0/3348466286 wait complete. 2024-04-25T10:02:12.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:12.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.622+0000 7f7cada00640 1 Processor -- start 2024-04-25T10:02:12.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.622+0000 7f7cada00640 1 -- start start 2024-04-25T10:02:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7cada00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7ca815db00 0x7f7ca80d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7cada00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca80d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7cada00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ce2b0 0x7f7ca8161670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7cada00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80d1e80 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7cada00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80b5930 con 0x7f7ca815db00 2024-04-25T10:02:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7cada00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80b5c10 con 0x7f7ca80b53b0 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.624+0000 7f7ca7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ce2b0 0x7f7ca8161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ca80ce2b0 0x7f7ca8161670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39366/0 (socket says 172.21.15.5:39366) 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 -- 172.21.15.5:0/1718687588 learned_addr learned my addr 172.21.15.5:0/1718687588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7c9fe00640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca80d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca6a00640 1 --2- 172.21.15.5:0/1718687588 >> v2:172.21.15.80:3300/0 conn(0x7f7ca815db00 0x7f7ca80d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 -- 172.21.15.5:0/1718687588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 msgr2=0x7f7ca80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca80d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 -- 172.21.15.5:0/1718687588 >> v2:172.21.15.80:3300/0 conn(0x7f7ca815db00 msgr2=0x7f7ca80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 --2- 172.21.15.5:0/1718687588 >> v2:172.21.15.80:3300/0 conn(0x7f7ca815db00 0x7f7ca80d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7ca7400640 1 -- 172.21.15.5:0/1718687588 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80b5ef0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.625+0000 7f7c9fe00640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca80d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.626+0000 7f7ca7400640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca8161670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80d4940 tx=0x7f7ca002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0cd5884cfd0efb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.626+0000 7f7ca6000640 1 -- 172.21.15.5:0/1718687588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0036070 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.626+0000 7f7ca6000640 1 -- 172.21.15.5:0/1718687588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0031600 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.626+0000 7f7ca6000640 1 -- 172.21.15.5:0/1718687588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00319d0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 -- 172.21.15.5:0/1718687588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 msgr2=0x7f7ca8161670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca8161670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80d4940 tx=0x7f7ca002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 -- 172.21.15.5:0/1718687588 shutdown_connections 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca8161670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 --2- 172.21.15.5:0/1718687588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca80d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 --2- 172.21.15.5:0/1718687588 >> v2:172.21.15.80:3300/0 conn(0x7f7ca815db00 0x7f7ca80d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 -- 172.21.15.5:0/1718687588 >> 172.21.15.5:0/1718687588 conn(0x7f7ca8072290 msgr2=0x7f7ca80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 -- 172.21.15.5:0/1718687588 shutdown_connections 2024-04-25T10:02:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.627+0000 7f7cada00640 1 -- 172.21.15.5:0/1718687588 wait complete. 2024-04-25T10:02:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.629+0000 7f7cada00640 1 Processor -- start 2024-04-25T10:02:12.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.629+0000 7f7cada00640 1 -- start start 2024-04-25T10:02:12.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7cada00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca8149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7cada00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca80cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7cada00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ca815db00 0x7f7ca80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7cada00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80d3660 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7cada00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80b61d0 con 0x7f7ca815db00 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7cada00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80b6c10 con 0x7f7ca80b53b0 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca80cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.630+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca80cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39368/0 (socket says 172.21.15.5:39368) 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 -- 172.21.15.5:0/2420839645 learned_addr learned my addr 172.21.15.5:0/2420839645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7ca6a00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca8149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7ca7400640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ca815db00 0x7f7ca80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 -- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 msgr2=0x7f7ca8149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca8149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 -- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ca815db00 msgr2=0x7f7ca80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ca815db00 0x7f7ca80cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 -- 172.21.15.5:0/2420839645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca815f310 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7ca7400640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ca815db00 0x7f7ca80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.631+0000 7f7c9fe00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca80cbff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000e760 tx=0x7f7c9000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.632+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000f040 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.632+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90004230 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.632+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000ee70 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.632+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca815f5f0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.632+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca814a1e0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.633+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca8068fd0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.634+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c9000c6d0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.635+0000 7f7c9ea00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c8406b500 0x7f7c8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.635+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7c9008a370 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.635+0000 7f7ca6a00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c8406b500 0x7f7c8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.636+0000 7f7ca6a00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c8406b500 0x7f7c8406d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c98006fd0 tx=0x7f7c98008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.641+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c90017020 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.853+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ca80d16b0 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.854+0000 7f7c9ea00640 1 -- 172.21.15.5:0/2420839645 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7c90058d80 con 0x7f7ca80ce2b0 2024-04-25T10:02:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:02:12.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c8406b500 msgr2=0x7f7c8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c8406b500 0x7f7c8406d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c98006fd0 tx=0x7f7c98008040 comp rx=0 tx=0).stop 2024-04-25T10:02:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 msgr2=0x7f7ca80cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca80cbff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000e760 tx=0x7f7c9000ec20 comp rx=0 tx=0).stop 2024-04-25T10:02:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 shutdown_connections 2024-04-25T10:02:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c8406b500 0x7f7c8406d9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ca815db00 0x7f7ca80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ca80ce2b0 0x7f7ca80cbff0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 --2- 172.21.15.5:0/2420839645 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7ca80b53b0 0x7f7ca8149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 >> 172.21.15.5:0/2420839645 conn(0x7f7ca8072290 msgr2=0x7f7ca80d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 shutdown_connections 2024-04-25T10:02:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:12.857+0000 7f7cada00640 1 -- 172.21.15.5:0/2420839645 wait complete. 2024-04-25T10:02:12.880 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:02:12.880 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:12.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.146+0000 7f2a4f800640 1 Processor -- start 2024-04-25T10:02:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.146+0000 7f2a4f800640 1 -- start start 2024-04-25T10:02:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4f800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a48144100 0x7f2a480cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4f800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480cdbf0 con 0x7f2a480ced20 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4f800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480cded0 con 0x7f2a4806e970 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4f800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a481444e0 con 0x7f2a48144100 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a48144100 0x7f2a480cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a48144100 0x7f2a480cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39374/0 (socket says 172.21.15.5:39374) 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a4ce00640 1 -- 172.21.15.5:0/1496840253 learned_addr learned my addr 172.21.15.5:0/1496840253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:13.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.147+0000 7f2a47e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a480ced20 0x7f2a4806e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59116/0 (socket says 172.21.15.5:59116) 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.148+0000 7f2a4ce00640 1 -- 172.21.15.5:0/1496840253 >> v1:172.21.15.80:6789/0 conn(0x7f2a4806e970 legacy=0x7f2a4815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.148+0000 7f2a4ce00640 1 -- 172.21.15.5:0/1496840253 >> v1:172.21.15.5:6789/0 conn(0x7f2a480ced20 legacy=0x7f2a4806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.148+0000 7f2a4ce00640 1 -- 172.21.15.5:0/1496840253 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480ce1b0 con 0x7f2a48144100 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.148+0000 7f2a47400640 1 -- 172.21.15.5:0/1496840253 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546968173 0 0) 0x7f2a480cdbf0 con 0x7f2a480ced20 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.149+0000 7f2a4ce00640 1 --2- 172.21.15.5:0/1496840253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a480cd610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009870 tx=0x7f2a3c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e998f520cbaa3ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.149+0000 7f2a47400640 1 -- 172.21.15.5:0/1496840253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c037070 con 0x7f2a48144100 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a47400640 1 -- 172.21.15.5:0/1496840253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c02ae50 con 0x7f2a48144100 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a47400640 1 -- 172.21.15.5:0/1496840253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c032860 con 0x7f2a48144100 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 -- 172.21.15.5:0/1496840253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 msgr2=0x7f2a480cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 --2- 172.21.15.5:0/1496840253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a480cd610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009870 tx=0x7f2a3c02a300 comp rx=0 tx=0).stop 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 -- 172.21.15.5:0/1496840253 shutdown_connections 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 --2- 172.21.15.5:0/1496840253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a480cd610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 -- 172.21.15.5:0/1496840253 >> 172.21.15.5:0/1496840253 conn(0x7f2a48071150 msgr2=0x7f2a48071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 -- 172.21.15.5:0/1496840253 shutdown_connections 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.150+0000 7f2a4f800640 1 -- 172.21.15.5:0/1496840253 wait complete. 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.152+0000 7f2a4f800640 1 Processor -- start 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.152+0000 7f2a4f800640 1 -- start start 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4f800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4806e970 0x7f2a481388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4f800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a480ced20 0x7f2a48138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4f800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a481f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4f800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480cea00 con 0x7f2a48144100 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4f800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4815ac70 con 0x7f2a4806e970 2024-04-25T10:02:13.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4f800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4815b840 con 0x7f2a480ced20 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a481f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.153+0000 7f2a4d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a481f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39382/0 (socket says 172.21.15.5:39382) 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 -- 172.21.15.5:0/185772007 learned_addr learned my addr 172.21.15.5:0/185772007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a47e00640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a480ced20 0x7f2a48138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4ce00640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4806e970 0x7f2a481388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 -- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a480ced20 msgr2=0x7f2a48138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a480ced20 0x7f2a48138df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 -- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4806e970 msgr2=0x7f2a481388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4806e970 0x7f2a481388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 -- 172.21.15.5:0/185772007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48139330 con 0x7f2a48144100 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.154+0000 7f2a4d800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a481f68e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c730 tx=0x7f2a4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.155+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40003b70 con 0x7f2a48144100 2024-04-25T10:02:13.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.155+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a40003d10 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.155+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000aa90 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.155+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a481395e0 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.155+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a48139ba0 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.156+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a48068fd0 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.157+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a4000ac30 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.158+0000 7f2a44c00640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a2806b4d0 0x7f2a2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.158+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2a4008a3f0 con 0x7f2a48144100 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.159+0000 7f2a4ce00640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a2806b4d0 0x7f2a2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.160+0000 7f2a4ce00640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a2806b4d0 0x7f2a2806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c034000 tx=0x7f2a3c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.162+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a40058f20 con 0x7f2a48144100 2024-04-25T10:02:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.353+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a4815a850 con 0x7f2a48144100 2024-04-25T10:02:13.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.355+0000 7f2a44c00640 1 -- 172.21.15.5:0/185772007 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f2a40058820 con 0x7f2a48144100 2024-04-25T10:02:13.359 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:13.359 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.357+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a2806b4d0 msgr2=0x7f2a2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.357+0000 7f2a4f800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a2806b4d0 0x7f2a2806d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c034000 tx=0x7f2a3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 msgr2=0x7f2a481f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a481f68e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c730 tx=0x7f2a4000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 shutdown_connections 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a48144100 0x7f2a481f68e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a480ced20 0x7f2a48138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a2806b4d0 0x7f2a2806d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 --2- 172.21.15.5:0/185772007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4806e970 0x7f2a481388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 >> 172.21.15.5:0/185772007 conn(0x7f2a48071150 msgr2=0x7f2a480cb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 shutdown_connections 2024-04-25T10:02:13.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.358+0000 7f2a4f800640 1 -- 172.21.15.5:0/185772007 wait complete. 2024-04-25T10:02:13.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:13.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.587+0000 7fe9a9a00640 1 Processor -- start 2024-04-25T10:02:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.587+0000 7fe9a9a00640 1 -- start start 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.588+0000 7fe9a9a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9a40b48c0 0x7fe9a415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.588+0000 7fe9a9a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe9a40d3f90 0x7fe9a4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.588+0000 7fe9a9a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a41606e0 con 0x7fe9a4161bc0 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.588+0000 7fe9a9a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a4161ff0 con 0x7fe9a40b48c0 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.588+0000 7fe9a9a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a41609c0 con 0x7fe9a40d3f90 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9a40b48c0 0x7fe9a415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9a40b48c0 0x7fe9a415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39384/0 (socket says 172.21.15.5:39384) 2024-04-25T10:02:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 -- 172.21.15.5:0/2367748975 learned_addr learned my addr 172.21.15.5:0/2367748975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3400640 1 --2- 172.21.15.5:0/2367748975 >> v2:172.21.15.80:3300/0 conn(0x7fe9a40d3f90 0x7fe9a4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 -- 172.21.15.5:0/2367748975 >> v2:172.21.15.80:3300/0 conn(0x7fe9a40d3f90 msgr2=0x7fe9a4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 --2- 172.21.15.5:0/2367748975 >> v2:172.21.15.80:3300/0 conn(0x7fe9a40d3f90 0x7fe9a4161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 -- 172.21.15.5:0/2367748975 >> v1:172.21.15.80:6789/0 conn(0x7fe9a4161bc0 legacy=0x7fe9a40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3e00640 1 -- 172.21.15.5:0/2367748975 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a4160ca0 con 0x7fe9a40b48c0 2024-04-25T10:02:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.589+0000 7fe9a3400640 1 --2- 172.21.15.5:0/2367748975 >> v2:172.21.15.80:3300/0 conn(0x7fe9a40d3f90 0x7fe9a4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.591+0000 7fe9a3e00640 1 --2- 172.21.15.5:0/2367748975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a415fe60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40b61b0 tx=0x7fe99402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5970f6d4a934c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.591+0000 7fe9a2a00640 1 -- 172.21.15.5:0/2367748975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994036070 con 0x7fe9a40b48c0 2024-04-25T10:02:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.591+0000 7fe9a2a00640 1 -- 172.21.15.5:0/2367748975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994031600 con 0x7fe9a40b48c0 2024-04-25T10:02:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.591+0000 7fe9a2a00640 1 -- 172.21.15.5:0/2367748975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940319d0 con 0x7fe9a40b48c0 2024-04-25T10:02:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 -- 172.21.15.5:0/2367748975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 msgr2=0x7fe9a415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/2367748975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a415fe60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9a40b61b0 tx=0x7fe99402a660 comp rx=0 tx=0).stop 2024-04-25T10:02:13.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 -- 172.21.15.5:0/2367748975 shutdown_connections 2024-04-25T10:02:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/2367748975 >> v2:172.21.15.80:3300/0 conn(0x7fe9a40d3f90 0x7fe9a4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/2367748975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a415fe60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 -- 172.21.15.5:0/2367748975 >> 172.21.15.5:0/2367748975 conn(0x7fe9a408af80 msgr2=0x7fe9a40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 -- 172.21.15.5:0/2367748975 shutdown_connections 2024-04-25T10:02:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.592+0000 7fe9a9a00640 1 -- 172.21.15.5:0/2367748975 wait complete. 2024-04-25T10:02:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.594+0000 7fe9a9a00640 1 Processor -- start 2024-04-25T10:02:13.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.594+0000 7fe9a9a00640 1 -- start start 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a41f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a9a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9a40d3f90 0x7fe9a41f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a9a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a4161bc0 0x7fe9a40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a9a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a40b4ca0 con 0x7fe9a40b48c0 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a9a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a4161260 con 0x7fe9a4161bc0 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a9a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a41603a0 con 0x7fe9a40d3f90 2024-04-25T10:02:13.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a41f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a41f3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39392/0 (socket says 172.21.15.5:39392) 2024-04-25T10:02:13.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a3e00640 1 -- 172.21.15.5:0/4005097179 learned_addr learned my addr 172.21.15.5:0/4005097179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.595+0000 7fe9a3400640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9a40d3f90 0x7fe9a41f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a9000640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a4161bc0 0x7fe9a40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a3e00640 1 -- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9a40d3f90 msgr2=0x7fe9a41f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a3e00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9a40d3f90 0x7fe9a41f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a3e00640 1 -- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a4161bc0 msgr2=0x7fe9a40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a3e00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a4161bc0 0x7fe9a40cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a3e00640 1 -- 172.21.15.5:0/4005097179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a40c72b0 con 0x7fe9a40b48c0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a9000640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a4161bc0 0x7fe9a40cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a3e00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a41f3cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe994034040 tx=0x7fe994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994036050 con 0x7fe9a40b48c0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994042070 con 0x7fe9a40b48c0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994009510 con 0x7fe9a40b48c0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.596+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a40c7590 con 0x7fe9a40b48c0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.597+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a40c7b50 con 0x7fe9a40b48c0 2024-04-25T10:02:13.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.598+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a4068f60 con 0x7fe9a40b48c0 2024-04-25T10:02:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.599+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe994007c30 con 0x7fe9a40b48c0 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.600+0000 7fe99be00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98006b500 0x7fe98006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.600+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9940ab560 con 0x7fe9a40b48c0 2024-04-25T10:02:13.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.601+0000 7fe9a3400640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98006b500 0x7fe98006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.602+0000 7fe9a3400640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98006b500 0x7fe98006d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe98c006fd0 tx=0x7fe98c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.605+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe99407a090 con 0x7fe9a40b48c0 2024-04-25T10:02:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.802+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9a408d730 con 0x7fe9a40b48c0 2024-04-25T10:02:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.803+0000 7fe99be00640 1 -- 172.21.15.5:0/4005097179 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe994079990 con 0x7fe9a40b48c0 2024-04-25T10:02:13.807 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.805+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98006b500 msgr2=0x7fe98006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.805+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98006b500 0x7fe98006d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe98c006fd0 tx=0x7fe98c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 msgr2=0x7fe9a41f3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a41f3cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe994034040 tx=0x7fe994004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 shutdown_connections 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98006b500 0x7fe98006d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a4161bc0 0x7fe9a40cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9a40d3f90 0x7fe9a41f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 --2- 172.21.15.5:0/4005097179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a40b48c0 0x7fe9a41f3cb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 >> 172.21.15.5:0/4005097179 conn(0x7fe9a408af80 msgr2=0x7fe9a40d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 shutdown_connections 2024-04-25T10:02:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:13.806+0000 7fe9a9a00640 1 -- 172.21.15.5:0/4005097179 wait complete. 2024-04-25T10:02:13.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.045+0000 7ff12f000640 1 Processor -- start 2024-04-25T10:02:14.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.045+0000 7ff12f000640 1 -- start start 2024-04-25T10:02:14.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.047+0000 7ff12f000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1281448f0 0x7ff1280cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.047+0000 7ff12f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff12815bc30 0x7ff12815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.047+0000 7ff12f000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff128144cd0 con 0x7ff12815fe30 2024-04-25T10:02:14.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.047+0000 7ff12f000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12815f460 con 0x7ff1281448f0 2024-04-25T10:02:14.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.047+0000 7ff12f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815c440 con 0x7ff12815bc30 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12dc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1281448f0 0x7ff1280cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12dc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1281448f0 0x7ff1280cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:40104/0 (socket says 172.21.15.5:40104) 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12dc00640 1 -- 172.21.15.5:0/1893458514 learned_addr learned my addr 172.21.15.5:0/1893458514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12d200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff12815bc30 0x7ff12815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12d200640 1 -- 172.21.15.5:0/1893458514 >> v2:172.21.15.80:3300/0 conn(0x7ff1281448f0 msgr2=0x7ff1280cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12d200640 1 --2- 172.21.15.5:0/1893458514 >> v2:172.21.15.80:3300/0 conn(0x7ff1281448f0 0x7ff1280cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.048+0000 7ff12d200640 1 -- 172.21.15.5:0/1893458514 >> v1:172.21.15.80:6789/0 conn(0x7ff12815fe30 legacy=0x7ff12815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.049+0000 7ff12d200640 1 -- 172.21.15.5:0/1893458514 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12815c720 con 0x7ff12815bc30 2024-04-25T10:02:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.050+0000 7ff12d200640 1 --2- 172.21.15.5:0/1893458514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff12815bc30 0x7ff12815f8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff1140097a0 tx=0x7ff11402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84eacfc194326651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.050+0000 7ff127e00640 1 -- 172.21.15.5:0/1893458514 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114037070 con 0x7ff12815bc30 2024-04-25T10:02:14.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.050+0000 7ff127e00640 1 -- 172.21.15.5:0/1893458514 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114032440 con 0x7ff12815bc30 2024-04-25T10:02:14.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.050+0000 7ff127e00640 1 -- 172.21.15.5:0/1893458514 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114032810 con 0x7ff12815bc30 2024-04-25T10:02:14.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 -- 172.21.15.5:0/1893458514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff12815bc30 msgr2=0x7ff12815f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 --2- 172.21.15.5:0/1893458514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff12815bc30 0x7ff12815f8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff1140097a0 tx=0x7ff11402a180 comp rx=0 tx=0).stop 2024-04-25T10:02:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 -- 172.21.15.5:0/1893458514 shutdown_connections 2024-04-25T10:02:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 --2- 172.21.15.5:0/1893458514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff12815bc30 0x7ff12815f8f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 --2- 172.21.15.5:0/1893458514 >> v2:172.21.15.80:3300/0 conn(0x7ff1281448f0 0x7ff1280cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 -- 172.21.15.5:0/1893458514 >> 172.21.15.5:0/1893458514 conn(0x7ff128066f20 msgr2=0x7ff12806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 -- 172.21.15.5:0/1893458514 shutdown_connections 2024-04-25T10:02:14.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.051+0000 7ff12f000640 1 -- 172.21.15.5:0/1893458514 wait complete. 2024-04-25T10:02:14.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.053+0000 7ff12f000640 1 Processor -- start 2024-04-25T10:02:14.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.053+0000 7ff12f000640 1 -- start start 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1281448f0 0x7ff1281e7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 0x7ff1280c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12f000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff12815fe30 0x7ff12815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815c090 con 0x7ff12815bc30 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12f000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815d530 con 0x7ff12815fe30 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1281e5fd0 con 0x7ff1281448f0 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 0x7ff1280c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.054+0000 7ff12d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 0x7ff1280c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39408/0 (socket says 172.21.15.5:39408) 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 -- 172.21.15.5:0/2672890390 learned_addr learned my addr 172.21.15.5:0/2672890390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:14.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12dc00640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1281448f0 0x7ff1281e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12e600640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff12815fe30 0x7ff12815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 -- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1281448f0 msgr2=0x7ff1281e7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1281448f0 0x7ff1281e7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 -- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff12815fe30 msgr2=0x7ff12815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff12815fe30 0x7ff12815b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 -- 172.21.15.5:0/2672890390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1281e62b0 con 0x7ff12815bc30 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12e600640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff12815fe30 0x7ff12815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.055+0000 7ff12d200640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 0x7ff1280c4390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff114035040 tx=0x7ff1140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.056+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114037040 con 0x7ff12815bc30 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.056+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114030500 con 0x7ff12815bc30 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.056+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114030910 con 0x7ff12815bc30 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.056+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1281e6590 con 0x7ff12815bc30 2024-04-25T10:02:14.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.056+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1281e6b50 con 0x7ff12815bc30 2024-04-25T10:02:14.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.057+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff128068f60 con 0x7ff12815bc30 2024-04-25T10:02:14.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.058+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff114044020 con 0x7ff12815bc30 2024-04-25T10:02:14.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.059+0000 7ff125600640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff10806b4d0 0x7ff10806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.059+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff1140aa880 con 0x7ff12815bc30 2024-04-25T10:02:14.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.059+0000 7ff12dc00640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff10806b4d0 0x7ff10806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.061+0000 7ff12dc00640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff10806b4d0 0x7ff10806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009d80 tx=0x7ff11c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.063+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff114079320 con 0x7ff12815bc30 2024-04-25T10:02:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.268+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff12815b9a0 con 0x7ff12815bc30 2024-04-25T10:02:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.269+0000 7ff125600640 1 -- 172.21.15.5:0/2672890390 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff114078c20 con 0x7ff12815bc30 2024-04-25T10:02:14.273 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff10806b4d0 msgr2=0x7ff10806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff10806b4d0 0x7ff10806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009d80 tx=0x7ff11c00b040 comp rx=0 tx=0).stop 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 msgr2=0x7ff1280c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 0x7ff1280c4390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff114035040 tx=0x7ff1140046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 shutdown_connections 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff10806b4d0 0x7ff10806d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff12815fe30 0x7ff12815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff12815bc30 0x7ff1280c4390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.271+0000 7ff12f000640 1 --2- 172.21.15.5:0/2672890390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1281448f0 0x7ff1281e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.272+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 >> 172.21.15.5:0/2672890390 conn(0x7ff128066f20 msgr2=0x7ff1280b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.272+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 shutdown_connections 2024-04-25T10:02:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.272+0000 7ff12f000640 1 -- 172.21.15.5:0/2672890390 wait complete. 2024-04-25T10:02:14.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.510+0000 7fb7a3e00640 1 Processor -- start 2024-04-25T10:02:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.511+0000 7fb7a3e00640 1 -- start start 2024-04-25T10:02:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.512+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.512+0000 7fb7a3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7a4161bc0 0x7fb7a40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.512+0000 7fb7a3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a41606e0 con 0x7fb7a40b48c0 2024-04-25T10:02:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.512+0000 7fb7a3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a41609c0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.512+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4160ca0 con 0x7fb7a40d3f90 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.513+0000 7fb7a3400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7a4161bc0 0x7fb7a40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.513+0000 7fb7a2000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.513+0000 7fb7a2a00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb7a40b48c0 0x7fb7a415fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:40936/0 (socket says 172.21.15.5:40936) 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.513+0000 7fb7a3400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7a4161bc0 0x7fb7a40b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:40130/0 (socket says 172.21.15.5:40130) 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.513+0000 7fb7a2a00640 1 -- 172.21.15.5:0/150270891 learned_addr learned my addr 172.21.15.5:0/150270891 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.514+0000 7fb7a3400640 1 -- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 msgr2=0x7fb7a4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.514+0000 7fb7a3400640 1 --2- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a4161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.514+0000 7fb7a3400640 1 -- 172.21.15.5:0/150270891 >> v1:172.21.15.80:6789/0 conn(0x7fb7a40b48c0 legacy=0x7fb7a415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.514+0000 7fb7a3400640 1 -- 172.21.15.5:0/150270891 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4160f80 con 0x7fb7a4161bc0 2024-04-25T10:02:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.514+0000 7fb7a2000640 1 --2- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:14.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.515+0000 7fb7a3400640 1 --2- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40b2b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009d90 tx=0x7fb79c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1b1b9912f8fb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a1600640 1 -- 172.21.15.5:0/150270891 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c031600 con 0x7fb7a4161bc0 2024-04-25T10:02:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a1600640 1 -- 172.21.15.5:0/150270891 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c0317a0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a1600640 1 -- 172.21.15.5:0/150270891 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c031b70 con 0x7fb7a4161bc0 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a3e00640 1 -- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a4161bc0 msgr2=0x7fb7a40b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40b2b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009d90 tx=0x7fb79c02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a3e00640 1 -- 172.21.15.5:0/150270891 shutdown_connections 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40b2b70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/150270891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.516+0000 7fb7a3e00640 1 -- 172.21.15.5:0/150270891 >> 172.21.15.5:0/150270891 conn(0x7fb7a408af80 msgr2=0x7fb7a40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.517+0000 7fb7a3e00640 1 -- 172.21.15.5:0/150270891 shutdown_connections 2024-04-25T10:02:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.517+0000 7fb7a3e00640 1 -- 172.21.15.5:0/150270891 wait complete. 2024-04-25T10:02:14.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.518+0000 7fb7a3e00640 1 Processor -- start 2024-04-25T10:02:14.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.519+0000 7fb7a3e00640 1 -- start start 2024-04-25T10:02:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.519+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a40b48c0 0x7fb7a40c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.519+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a41f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b30b0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b4ca0 con 0x7fb7a40b48c0 2024-04-25T10:02:14.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b3230 con 0x7fb7a40d3f90 2024-04-25T10:02:14.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39424/0 (socket says 172.21.15.5:39424) 2024-04-25T10:02:14.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 -- 172.21.15.5:0/2695058484 learned_addr learned my addr 172.21.15.5:0/2695058484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:14.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a2a00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a40b48c0 0x7fb7a40c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a2000640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a41f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 -- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 msgr2=0x7fb7a41f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a41f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 -- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a40b48c0 msgr2=0x7fb7a40c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.520+0000 7fb7a3400640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a40b48c0 0x7fb7a40c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb7a3400640 1 -- 172.21.15.5:0/2695058484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a41f7db0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb7a2a00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a40b48c0 0x7fb7a40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb7a3400640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40c77d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb79c02abc0 tx=0x7fb79c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c03f070 con 0x7fb7a4161bc0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c030b60 con 0x7fb7a4161bc0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a40b24e0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c009450 con 0x7fb7a4161bc0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.521+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a40cc4a0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.522+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4068f60 con 0x7fb7a4161bc0 2024-04-25T10:02:14.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.523+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb79c03b410 con 0x7fb7a4161bc0 2024-04-25T10:02:14.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.524+0000 7fb79b400640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb78406b4d0 0x7fb78406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.524+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb79c0abab0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.525+0000 7fb7a2a00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb78406b4d0 0x7fb78406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.526+0000 7fb7a2a00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb78406b4d0 0x7fb78406d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb794000f60 tx=0x7fb794009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.529+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb7a41f7db0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.732+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7a40d0dd0 con 0x7fb7a4161bc0 2024-04-25T10:02:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.733+0000 7fb79b400640 1 -- 172.21.15.5:0/2695058484 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb79c079f00 con 0x7fb7a4161bc0 2024-04-25T10:02:14.737 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb78406b4d0 msgr2=0x7fb78406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb78406b4d0 0x7fb78406d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb794000f60 tx=0x7fb794009040 comp rx=0 tx=0).stop 2024-04-25T10:02:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 msgr2=0x7fb7a40c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40c77d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb79c02abc0 tx=0x7fb79c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 shutdown_connections 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7a4161bc0 0x7fb7a40c77d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7a40d3f90 0x7fb7a41f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb78406b4d0 0x7fb78406d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 --2- 172.21.15.5:0/2695058484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7a40b48c0 0x7fb7a40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 >> 172.21.15.5:0/2695058484 conn(0x7fb7a408af80 msgr2=0x7fb7a406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 shutdown_connections 2024-04-25T10:02:14.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.736+0000 7fb7a3e00640 1 -- 172.21.15.5:0/2695058484 wait complete. 2024-04-25T10:02:14.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:14.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.967+0000 7fbec9600640 1 Processor -- start 2024-04-25T10:02:14.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.967+0000 7fbec9600640 1 -- start start 2024-04-25T10:02:14.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.969+0000 7fbec9600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 0x7fbec40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.969+0000 7fbec9600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec4144cd0 con 0x7fbec415ce20 2024-04-25T10:02:14.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.969+0000 7fbec9600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec415c440 con 0x7fbec415bc30 2024-04-25T10:02:14.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.969+0000 7fbec9600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415f460 con 0x7fbec41448f0 2024-04-25T10:02:14.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.969+0000 7fbec8c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbec415ce20 0x7fbec4071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59130/0 (socket says 172.21.15.5:59130) 2024-04-25T10:02:14.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.969+0000 7fbec8c00640 1 -- 172.21.15.5:0/1508895487 learned_addr learned my addr 172.21.15.5:0/1508895487 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:14.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507832854 0 0) 0x7fbec4144cd0 con 0x7fbec415ce20 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea8003520 con 0x7fbec415ce20 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec3e00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 0x7fbec40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529467083 0 0) 0x7fbea8003520 con 0x7fbec415ce20 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 msgr2=0x7fbec40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 0x7fbec40cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 >> v1:172.21.15.80:6789/0 conn(0x7fbec415bc30 legacy=0x7fbec415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec415c720 con 0x7fbec415ce20 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fbebc001670 con 0x7fbec415ce20 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 >> v1:172.21.15.5:6789/0 conn(0x7fbec415ce20 legacy=0x7fbec4071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbea80039e0 0x7fbea8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 0x7fbea8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8004d40 0x7fbea80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415f460 con 0x7fbea8004d40 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea800ffb0 con 0x7fbea80039e0 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.970+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80101b0 con 0x7fbec41448f0 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8004d40 0x7fbea80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec3400640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 0x7fbea8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec3e00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbea80039e0 0x7fbea8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec8c00640 1 -- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 msgr2=0x7fbea8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbec41448f0 0x7fbea8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec8c00640 1 -- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbea80039e0 msgr2=0x7fbea8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbea80039e0 0x7fbea8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec8c00640 1 -- 172.21.15.5:0/1508895487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8003520 con 0x7fbea8004d40 2024-04-25T10:02:14.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.971+0000 7fbec3e00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbea80039e0 0x7fbea8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.973+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8004d40 0x7fbea80051c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c0e0 tx=0x7fbea8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5d507e332bc7e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.973+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007c20 con 0x7fbea8004d40 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.973+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc005af0 con 0x7fbea8004d40 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.973+0000 7fbec2a00640 1 -- 172.21.15.5:0/1508895487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007070 con 0x7fbea8004d40 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec9600640 1 -- 172.21.15.5:0/1508895487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8004d40 msgr2=0x7fbea80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec9600640 1 --2- 172.21.15.5:0/1508895487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8004d40 0x7fbea80051c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c0e0 tx=0x7fbea8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec8c00640 1 -- 172.21.15.5:0/1508895487 reap_dead start 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec9600640 1 -- 172.21.15.5:0/1508895487 shutdown_connections 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec9600640 1 -- 172.21.15.5:0/1508895487 >> 172.21.15.5:0/1508895487 conn(0x7fbec4066f20 msgr2=0x7fbec406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec9600640 1 -- 172.21.15.5:0/1508895487 shutdown_connections 2024-04-25T10:02:14.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.974+0000 7fbec9600640 1 -- 172.21.15.5:0/1508895487 wait complete. 2024-04-25T10:02:14.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.976+0000 7fbec9600640 1 Processor -- start 2024-04-25T10:02:14.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.976+0000 7fbec9600640 1 -- start start 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec9600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbea8004d40 0x7fbec40a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec9600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbec4140c00 0x7fbec415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 0x7fbec413d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec9600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415c090 con 0x7fbec40a8590 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec9600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4160000 con 0x7fbec4140c00 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec9600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40a8a10 con 0x7fbea8004d40 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 0x7fbec413d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 0x7fbec413d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39430/0 (socket says 172.21.15.5:39430) 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.977+0000 7fbec8c00640 1 -- 172.21.15.5:0/1745514806 learned_addr learned my addr 172.21.15.5:0/1745514806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec3400640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbec4140c00 0x7fbec415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec3e00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbea8004d40 0x7fbec40a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec8c00640 1 -- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbea8004d40 msgr2=0x7fbec40a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbea8004d40 0x7fbec40a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec8c00640 1 -- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbec4140c00 msgr2=0x7fbec415b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbec4140c00 0x7fbec415b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec8c00640 1 -- 172.21.15.5:0/1745514806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec413d3d0 con 0x7fbec40a8590 2024-04-25T10:02:14.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec8c00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 0x7fbec413d730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c200 tx=0x7fbec40cefa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc037040 con 0x7fbec40a8590 2024-04-25T10:02:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc041070 con 0x7fbec40a8590 2024-04-25T10:02:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0072b0 con 0x7fbec40a8590 2024-04-25T10:02:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.978+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec40a8c90 con 0x7fbec40a8590 2024-04-25T10:02:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.979+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec40a9250 con 0x7fbec40a8590 2024-04-25T10:02:14.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.980+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec404efc0 con 0x7fbec40a8590 2024-04-25T10:02:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.981+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbebc007450 con 0x7fbec40a8590 2024-04-25T10:02:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.981+0000 7fbebbe00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbea406b4d0 0x7fbea406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.982+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbebc0aa330 con 0x7fbec40a8590 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.982+0000 7fbec3e00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbea406b4d0 0x7fbea406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.983+0000 7fbec3e00640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbea406b4d0 0x7fbea406d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009fd0 tx=0x7fbeb400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:14.986+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbec404efc0 con 0x7fbec40a8590 2024-04-25T10:02:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.185+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbec4067420 con 0x7fbec40a8590 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.185+0000 7fbebbe00640 1 -- 172.21.15.5:0/1745514806 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbec4067420 con 0x7fbec40a8590 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.188+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbea406b4d0 msgr2=0x7fbea406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.188+0000 7fbec9600640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbea406b4d0 0x7fbea406d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4009fd0 tx=0x7fbeb400b040 comp rx=0 tx=0).stop 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.188+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 msgr2=0x7fbec413d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.188+0000 7fbec9600640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 0x7fbec413d730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c200 tx=0x7fbec40cefa0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 shutdown_connections 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbec40a8590 0x7fbec413d730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbea406b4d0 0x7fbea406d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbec4140c00 0x7fbec415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 --2- 172.21.15.5:0/1745514806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbea8004d40 0x7fbec40a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 >> 172.21.15.5:0/1745514806 conn(0x7fbec4066f20 msgr2=0x7fbec406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 shutdown_connections 2024-04-25T10:02:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.189+0000 7fbec9600640 1 -- 172.21.15.5:0/1745514806 wait complete. 2024-04-25T10:02:15.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.423+0000 7fea3da00640 1 Processor -- start 2024-04-25T10:02:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.423+0000 7fea3da00640 1 -- start start 2024-04-25T10:02:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.425+0000 7fea3da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 0x7fea3806e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.425+0000 7fea3da00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fea380cd410 0x7fea381622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.425+0000 7fea3da00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea38165af0 con 0x7fea380b16e0 2024-04-25T10:02:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.425+0000 7fea3da00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3815c760 con 0x7fea380cd410 2024-04-25T10:02:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.425+0000 7fea3da00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea381658a0 con 0x7fea3815c380 2024-04-25T10:02:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.426+0000 7fea36a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 0x7fea3806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.426+0000 7fea37400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fea380cd410 0x7fea381622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.426+0000 7fea36a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 0x7fea3806e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:43102/0 (socket says 172.21.15.5:43102) 2024-04-25T10:02:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.426+0000 7fea36a00640 1 -- 172.21.15.5:0/1388464863 learned_addr learned my addr 172.21.15.5:0/1388464863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.427+0000 7fea36a00640 1 -- 172.21.15.5:0/1388464863 >> v2:172.21.15.80:3300/0 conn(0x7fea380cd410 msgr2=0x7fea381622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.427+0000 7fea36a00640 1 --2- 172.21.15.5:0/1388464863 >> v2:172.21.15.80:3300/0 conn(0x7fea380cd410 0x7fea381622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.427+0000 7fea36a00640 1 -- 172.21.15.5:0/1388464863 >> v1:172.21.15.80:6789/0 conn(0x7fea380b16e0 legacy=0x7fea380ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.427+0000 7fea36a00640 1 -- 172.21.15.5:0/1388464863 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38165dd0 con 0x7fea3815c380 2024-04-25T10:02:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.428+0000 7fea36a00640 1 --2- 172.21.15.5:0/1388464863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 0x7fea3806e730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea28009870 tx=0x7fea2802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30750d7667ba3cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.429+0000 7fea36000640 1 -- 172.21.15.5:0/1388464863 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28037070 con 0x7fea3815c380 2024-04-25T10:02:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.429+0000 7fea36000640 1 -- 172.21.15.5:0/1388464863 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28032440 con 0x7fea3815c380 2024-04-25T10:02:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.429+0000 7fea36000640 1 -- 172.21.15.5:0/1388464863 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28032810 con 0x7fea3815c380 2024-04-25T10:02:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 -- 172.21.15.5:0/1388464863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 msgr2=0x7fea3806e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 --2- 172.21.15.5:0/1388464863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 0x7fea3806e730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea28009870 tx=0x7fea2802a130 comp rx=0 tx=0).stop 2024-04-25T10:02:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 -- 172.21.15.5:0/1388464863 shutdown_connections 2024-04-25T10:02:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 --2- 172.21.15.5:0/1388464863 >> v2:172.21.15.80:3300/0 conn(0x7fea380cd410 0x7fea381622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 --2- 172.21.15.5:0/1388464863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea3815c380 0x7fea3806e730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 -- 172.21.15.5:0/1388464863 >> 172.21.15.5:0/1388464863 conn(0x7fea38071150 msgr2=0x7fea38071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 -- 172.21.15.5:0/1388464863 shutdown_connections 2024-04-25T10:02:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.430+0000 7fea3da00640 1 -- 172.21.15.5:0/1388464863 wait complete. 2024-04-25T10:02:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.432+0000 7fea3da00640 1 Processor -- start 2024-04-25T10:02:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.432+0000 7fea3da00640 1 -- start start 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea3da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 0x7fea38148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea3da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea380cd410 0x7fea3814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea3da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea3815c380 0x7fea3815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea3da00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3806ec70 con 0x7fea380b16e0 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea3da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38166390 con 0x7fea3815c380 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea3da00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea381627f0 con 0x7fea380cd410 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea36a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 0x7fea38148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea36a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 0x7fea38148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39446/0 (socket says 172.21.15.5:39446) 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.433+0000 7fea36a00640 1 -- 172.21.15.5:0/1400990477 learned_addr learned my addr 172.21.15.5:0/1400990477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea2fe00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea380cd410 0x7fea3814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea37400640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea3815c380 0x7fea3815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea36a00640 1 -- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea380cd410 msgr2=0x7fea3814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea36a00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea380cd410 0x7fea3814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea36a00640 1 -- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea3815c380 msgr2=0x7fea3815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea36a00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea3815c380 0x7fea3815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea36a00640 1 -- 172.21.15.5:0/1400990477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3815e860 con 0x7fea380b16e0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea2fe00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea380cd410 0x7fea3814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea36a00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 0x7fea38148690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea2802a170 tx=0x7fea280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28041030 con 0x7fea380b16e0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28030440 con 0x7fea380b16e0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28037040 con 0x7fea380b16e0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.434+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3815eb40 con 0x7fea380b16e0 2024-04-25T10:02:15.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.435+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3815f100 con 0x7fea380b16e0 2024-04-25T10:02:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.436+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea38068fd0 con 0x7fea380b16e0 2024-04-25T10:02:15.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.437+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea28044070 con 0x7fea380b16e0 2024-04-25T10:02:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.438+0000 7fea2ea00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea1406b500 0x7fea1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.439+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea280aa970 con 0x7fea380b16e0 2024-04-25T10:02:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.439+0000 7fea2fe00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea1406b500 0x7fea1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.440+0000 7fea2fe00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea1406b500 0x7fea1406d9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea38160990 tx=0x7fea20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.443+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea2804b070 con 0x7fea380b16e0 2024-04-25T10:02:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.658+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fea381606e0 con 0x7fea380b16e0 2024-04-25T10:02:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.659+0000 7fea2ea00640 1 -- 172.21.15.5:0/1400990477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fea280794a0 con 0x7fea380b16e0 2024-04-25T10:02:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:02:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.662+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea1406b500 msgr2=0x7fea1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.662+0000 7fea3da00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea1406b500 0x7fea1406d9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea38160990 tx=0x7fea20009040 comp rx=0 tx=0).stop 2024-04-25T10:02:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.662+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 msgr2=0x7fea38148690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.662+0000 7fea3da00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 0x7fea38148690 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fea2802a170 tx=0x7fea280046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 shutdown_connections 2024-04-25T10:02:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea1406b500 0x7fea1406d9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea3815c380 0x7fea3815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea380cd410 0x7fea3814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 --2- 172.21.15.5:0/1400990477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea380b16e0 0x7fea38148690 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 >> 172.21.15.5:0/1400990477 conn(0x7fea38071150 msgr2=0x7fea38160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 shutdown_connections 2024-04-25T10:02:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:15.663+0000 7fea3da00640 1 -- 172.21.15.5:0/1400990477 wait complete. 2024-04-25T10:02:15.687 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-04-25T10:02:15.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:15.773 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[38732]: fuse finished with error 0 and tester_r 0 2024-04-25T10:02:15.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:21.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:02:21.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:21.881 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T10:02:21.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.176+0000 7fea61000640 1 Processor -- start 2024-04-25T10:02:22.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.176+0000 7fea61000640 1 -- start start 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.177+0000 7fea61000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fea5c15db00 0x7fea5c0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.177+0000 7fea61000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c0d1e80 con 0x7fea5c0b53b0 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.177+0000 7fea61000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c162490 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.177+0000 7fea61000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5c0b5930 con 0x7fea5c15db00 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea59400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fea5c0b53b0 0x7fea5c161670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59774/0 (socket says 172.21.15.5:59774) 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea59400640 1 -- 172.21.15.5:0/2349668440 learned_addr learned my addr 172.21.15.5:0/2349668440 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:22.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea59e00640 1 --2- 172.21.15.5:0/2349668440 >> v2:172.21.15.5:3300/0 conn(0x7fea5c15db00 0x7fea5c0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea58a00640 1 -- 172.21.15.5:0/2349668440 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288460813 0 0) 0x7fea5c0d1e80 con 0x7fea5c0b53b0 2024-04-25T10:02:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea58a00640 1 -- 172.21.15.5:0/2349668440 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea3c003520 con 0x7fea5c0b53b0 2024-04-25T10:02:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea59e00640 1 -- 172.21.15.5:0/2349668440 >> v1:172.21.15.80:6789/0 conn(0x7fea5c0ce2b0 legacy=0x7fea5c161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea59e00640 1 -- 172.21.15.5:0/2349668440 >> v1:172.21.15.5:6789/0 conn(0x7fea5c0b53b0 legacy=0x7fea5c161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.178+0000 7fea59e00640 1 -- 172.21.15.5:0/2349668440 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0b5c10 con 0x7fea5c15db00 2024-04-25T10:02:22.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.180+0000 7fea59e00640 1 --2- 172.21.15.5:0/2349668440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c15db00 0x7fea5c0d0340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0097a0 tx=0x7fea4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5628b2bbbe7f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.180+0000 7fea58a00640 1 -- 172.21.15.5:0/2349668440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c037070 con 0x7fea5c15db00 2024-04-25T10:02:22.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.180+0000 7fea58a00640 1 -- 172.21.15.5:0/2349668440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c032440 con 0x7fea5c15db00 2024-04-25T10:02:22.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.180+0000 7fea58a00640 1 -- 172.21.15.5:0/2349668440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c032810 con 0x7fea5c15db00 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.180+0000 7fea61000640 1 -- 172.21.15.5:0/2349668440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c15db00 msgr2=0x7fea5c0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.180+0000 7fea61000640 1 --2- 172.21.15.5:0/2349668440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c15db00 0x7fea5c0d0340 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0097a0 tx=0x7fea4c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.181+0000 7fea61000640 1 -- 172.21.15.5:0/2349668440 shutdown_connections 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.181+0000 7fea61000640 1 --2- 172.21.15.5:0/2349668440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c15db00 0x7fea5c0d0340 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.181+0000 7fea61000640 1 -- 172.21.15.5:0/2349668440 >> 172.21.15.5:0/2349668440 conn(0x7fea5c072290 msgr2=0x7fea5c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.181+0000 7fea61000640 1 -- 172.21.15.5:0/2349668440 shutdown_connections 2024-04-25T10:02:22.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.181+0000 7fea61000640 1 -- 172.21.15.5:0/2349668440 wait complete. 2024-04-25T10:02:22.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.182+0000 7fea61000640 1 Processor -- start 2024-04-25T10:02:22.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 -- start start 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea5c0b53b0 0x7fea5c0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 0x7fea5c160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea5c15db00 0x7fea5c160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0b61d0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c15dee0 con 0x7fea5c0b53b0 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.183+0000 7fea61000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0d3660 con 0x7fea5c15db00 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 0x7fea5c160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 0x7fea5c160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46656/0 (socket says 172.21.15.5:46656) 2024-04-25T10:02:22.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 -- 172.21.15.5:0/743120741 learned_addr learned my addr 172.21.15.5:0/743120741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:22.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59e00640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea5c0b53b0 0x7fea5c0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 -- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea5c15db00 msgr2=0x7fea5c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea5c15db00 0x7fea5c160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 -- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea5c0b53b0 msgr2=0x7fea5c0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea5c0b53b0 0x7fea5c0c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 -- 172.21.15.5:0/743120741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0cb930 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.184+0000 7fea59400640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 0x7fea5c160230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea4400c390 tx=0x7fea4400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.185+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44017070 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.185+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44005ce0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.185+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea440072f0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.185+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c0cbc10 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.185+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c0cc1d0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.186+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c068fd0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.187+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea4401a070 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.188+0000 7fea52000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea3806b4d0 0x7fea3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.188+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea440899f0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.189+0000 7fea59e00640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea3806b4d0 0x7fea3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.190+0000 7fea59e00640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea3806b4d0 0x7fea3806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fea4c034000 tx=0x7fea4c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.193+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea44058490 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.393+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea5c0d16b0 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.397+0000 7fea52000640 1 -- 172.21.15.5:0/743120741 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7fea44057d90 con 0x7fea5c0ce2b0 2024-04-25T10:02:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:22.401 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea3806b4d0 msgr2=0x7fea3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea3806b4d0 0x7fea3806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fea4c034000 tx=0x7fea4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 msgr2=0x7fea5c160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 0x7fea5c160230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fea4400c390 tx=0x7fea4400c850 comp rx=0 tx=0).stop 2024-04-25T10:02:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 shutdown_connections 2024-04-25T10:02:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea5c15db00 0x7fea5c160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea5c0ce2b0 0x7fea5c160230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea3806b4d0 0x7fea3806d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 --2- 172.21.15.5:0/743120741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea5c0b53b0 0x7fea5c0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.399+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 >> 172.21.15.5:0/743120741 conn(0x7fea5c072290 msgr2=0x7fea5c072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.400+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 shutdown_connections 2024-04-25T10:02:22.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.400+0000 7fea61000640 1 -- 172.21.15.5:0/743120741 wait complete. 2024-04-25T10:02:22.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.617+0000 7fd97aa00640 1 Processor -- start 2024-04-25T10:02:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.617+0000 7fd97aa00640 1 -- start start 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.618+0000 7fd97aa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd974161200 0x7fd9740b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.618+0000 7fd97aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9740d2380 0x7fd9740d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.618+0000 7fd97aa00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9740cf410 con 0x7fd9740d0600 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.618+0000 7fd97aa00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9740d27c0 con 0x7fd9740d2380 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.618+0000 7fd97aa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9740d0a70 con 0x7fd974161200 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9740d2380 0x7fd9740d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9740d2380 0x7fd9740d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46670/0 (socket says 172.21.15.5:46670) 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd97a000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd9740d0600 0x7fd9740ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59778/0 (socket says 172.21.15.5:59778) 2024-04-25T10:02:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 -- 172.21.15.5:0/2085516486 learned_addr learned my addr 172.21.15.5:0/2085516486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd979600640 1 --2- 172.21.15.5:0/2085516486 >> v2:172.21.15.80:3300/0 conn(0x7fd974161200 0x7fd9740b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 -- 172.21.15.5:0/2085516486 >> v2:172.21.15.80:3300/0 conn(0x7fd974161200 msgr2=0x7fd9740b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 --2- 172.21.15.5:0/2085516486 >> v2:172.21.15.80:3300/0 conn(0x7fd974161200 0x7fd9740b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 -- 172.21.15.5:0/2085516486 >> v1:172.21.15.5:6789/0 conn(0x7fd9740d0600 legacy=0x7fd9740ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd978c00640 1 -- 172.21.15.5:0/2085516486 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740cf6f0 con 0x7fd9740d2380 2024-04-25T10:02:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.619+0000 7fd973e00640 1 -- 172.21.15.5:0/2085516486 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628502411 0 0) 0x7fd9740cf410 con 0x7fd9740d0600 2024-04-25T10:02:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.620+0000 7fd978c00640 1 --2- 172.21.15.5:0/2085516486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d2380 0x7fd9740d00c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd97408bed0 tx=0x7fd96802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7067b44b8f3e7bde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd973e00640 1 -- 172.21.15.5:0/2085516486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968037070 con 0x7fd9740d2380 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd973e00640 1 -- 172.21.15.5:0/2085516486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968032440 con 0x7fd9740d2380 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd973e00640 1 -- 172.21.15.5:0/2085516486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968032810 con 0x7fd9740d2380 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd97aa00640 1 -- 172.21.15.5:0/2085516486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d2380 msgr2=0x7fd9740d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd97aa00640 1 --2- 172.21.15.5:0/2085516486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d2380 0x7fd9740d00c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd97408bed0 tx=0x7fd96802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd97aa00640 1 -- 172.21.15.5:0/2085516486 shutdown_connections 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd97aa00640 1 --2- 172.21.15.5:0/2085516486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740d2380 0x7fd9740d00c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd97aa00640 1 --2- 172.21.15.5:0/2085516486 >> v2:172.21.15.80:3300/0 conn(0x7fd974161200 0x7fd9740b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.621+0000 7fd97aa00640 1 -- 172.21.15.5:0/2085516486 >> 172.21.15.5:0/2085516486 conn(0x7fd97408af80 msgr2=0x7fd9740d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.622+0000 7fd97aa00640 1 -- 172.21.15.5:0/2085516486 shutdown_connections 2024-04-25T10:02:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.622+0000 7fd97aa00640 1 -- 172.21.15.5:0/2085516486 wait complete. 2024-04-25T10:02:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.623+0000 7fd97aa00640 1 Processor -- start 2024-04-25T10:02:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.623+0000 7fd97aa00640 1 -- start start 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.624+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9740d0600 0x7fd9740cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.624+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9740d2380 0x7fd974151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.624+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 0x7fd9740b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.624+0000 7fd97aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740d42b0 con 0x7fd974161200 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.624+0000 7fd97aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740b28b0 con 0x7fd9740d0600 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.624+0000 7fd97aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9740b3f90 con 0x7fd9740d2380 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 0x7fd9740b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 0x7fd9740b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46678/0 (socket says 172.21.15.5:46678) 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 -- 172.21.15.5:0/1270189722 learned_addr learned my addr 172.21.15.5:0/1270189722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd979600640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9740d0600 0x7fd9740cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 -- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9740d2380 msgr2=0x7fd974151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9740d2380 0x7fd974151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 -- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9740d0600 msgr2=0x7fd9740cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9740d0600 0x7fd9740cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 -- 172.21.15.5:0/1270189722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974151f40 con 0x7fd974161200 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.625+0000 7fd97a000640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 0x7fd9740b1410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00c7d0 tx=0x7fd96c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.626+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c018070 con 0x7fd974161200 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.626+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c00a640 con 0x7fd974161200 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.626+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00aa70 con 0x7fd974161200 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.626+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9740cc0f0 con 0x7fd974161200 2024-04-25T10:02:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.626+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9740cc6b0 con 0x7fd974161200 2024-04-25T10:02:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.627+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd974068f60 con 0x7fd974161200 2024-04-25T10:02:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.628+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd96c00ac10 con 0x7fd974161200 2024-04-25T10:02:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.629+0000 7fd971600640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd95806b500 0x7fd95806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.629+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd96c089bf0 con 0x7fd974161200 2024-04-25T10:02:22.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.630+0000 7fd979600640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd95806b500 0x7fd95806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.632+0000 7fd979600640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd95806b500 0x7fd95806d9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd964006fd0 tx=0x7fd964008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.633+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd96c058600 con 0x7fd974161200 2024-04-25T10:02:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.839+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9741602b0 con 0x7fd974161200 2024-04-25T10:02:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.842+0000 7fd971600640 1 -- 172.21.15.5:0/1270189722 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7fd96c057f00 con 0x7fd974161200 2024-04-25T10:02:22.847 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:22.847 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.845+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd95806b500 msgr2=0x7fd95806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.845+0000 7fd97aa00640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd95806b500 0x7fd95806d9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd964006fd0 tx=0x7fd964008040 comp rx=0 tx=0).stop 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.845+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 msgr2=0x7fd9740b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.845+0000 7fd97aa00640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 0x7fd9740b1410 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00c7d0 tx=0x7fd96c00cc90 comp rx=0 tx=0).stop 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 shutdown_connections 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974161200 0x7fd9740b1410 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9740d2380 0x7fd974151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd95806b500 0x7fd95806d9c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 --2- 172.21.15.5:0/1270189722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9740d0600 0x7fd9740cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 >> 172.21.15.5:0/1270189722 conn(0x7fd97408af80 msgr2=0x7fd9740b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 shutdown_connections 2024-04-25T10:02:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:22.846+0000 7fd97aa00640 1 -- 172.21.15.5:0/1270189722 wait complete. 2024-04-25T10:02:22.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.072+0000 7f91c6800640 1 Processor -- start 2024-04-25T10:02:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.072+0000 7f91c6800640 1 -- start start 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.073+0000 7f91c6800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f91c006e840 0x7f91c015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c6800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c015b330 0x7f91c0071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c6800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c00ce570 con 0x7f91c00cc840 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c6800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c00ce850 con 0x7f91c006e840 2024-04-25T10:02:23.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c6800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00ceb30 con 0x7f91c015b330 2024-04-25T10:02:23.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c4a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f91c006e840 0x7f91c015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c4a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f91c006e840 0x7f91c015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:46210/0 (socket says 172.21.15.5:46210) 2024-04-25T10:02:23.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c4a00640 1 -- 172.21.15.5:0/1182618045 learned_addr learned my addr 172.21.15.5:0/1182618045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:23.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.074+0000 7f91c5e00640 1 --2- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c015b330 0x7f91c0071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.075+0000 7f91c4a00640 1 -- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c015b330 msgr2=0x7f91c0071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.075+0000 7f91c4a00640 1 --2- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c015b330 0x7f91c0071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.075+0000 7f91c4a00640 1 -- 172.21.15.5:0/1182618045 >> v1:172.21.15.80:6789/0 conn(0x7f91c00cc840 legacy=0x7f91c006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.075+0000 7f91c4a00640 1 -- 172.21.15.5:0/1182618045 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c00cee10 con 0x7f91c006e840 2024-04-25T10:02:23.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.075+0000 7f91bfe00640 1 -- 172.21.15.5:0/1182618045 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42635383 0 0) 0x7f91c00ce570 con 0x7f91c00cc840 2024-04-25T10:02:23.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.077+0000 7f91c4a00640 1 --2- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c006e840 0x7f91c015adf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00bb10 tx=0x7f91ac02c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49759fe0ecd1abfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.077+0000 7f91bfe00640 1 -- 172.21.15.5:0/1182618045 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac036020 con 0x7f91c006e840 2024-04-25T10:02:23.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.077+0000 7f91bfe00640 1 -- 172.21.15.5:0/1182618045 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac032820 con 0x7f91c006e840 2024-04-25T10:02:23.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.077+0000 7f91bfe00640 1 -- 172.21.15.5:0/1182618045 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac032bf0 con 0x7f91c006e840 2024-04-25T10:02:23.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 -- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c006e840 msgr2=0x7f91c015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 --2- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c006e840 0x7f91c015adf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00bb10 tx=0x7f91ac02c920 comp rx=0 tx=0).stop 2024-04-25T10:02:23.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 -- 172.21.15.5:0/1182618045 shutdown_connections 2024-04-25T10:02:23.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 --2- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c015b330 0x7f91c0071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 --2- 172.21.15.5:0/1182618045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c006e840 0x7f91c015adf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 -- 172.21.15.5:0/1182618045 >> 172.21.15.5:0/1182618045 conn(0x7f91c0066f20 msgr2=0x7f91c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 -- 172.21.15.5:0/1182618045 shutdown_connections 2024-04-25T10:02:23.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.078+0000 7f91c6800640 1 -- 172.21.15.5:0/1182618045 wait complete. 2024-04-25T10:02:23.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.080+0000 7f91c6800640 1 Processor -- start 2024-04-25T10:02:23.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.080+0000 7f91c6800640 1 -- start start 2024-04-25T10:02:23.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c6800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c006e840 0x7f91c01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c6800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00cc840 0x7f91c0156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 0x7f91c014d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c6800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00ce350 con 0x7f91c015b330 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c6800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00ce1d0 con 0x7f91c00cc840 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c6800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c00ccc20 con 0x7f91c006e840 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 0x7f91c014d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 0x7f91c014d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46686/0 (socket says 172.21.15.5:46686) 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c5e00640 1 -- 172.21.15.5:0/3037990708 learned_addr learned my addr 172.21.15.5:0/3037990708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c5400640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c006e840 0x7f91c01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.082+0000 7f91c4a00640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00cc840 0x7f91c0156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c5e00640 1 -- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c006e840 msgr2=0x7f91c01550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c5e00640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c006e840 0x7f91c01550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c5e00640 1 -- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00cc840 msgr2=0x7f91c0156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c5e00640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00cc840 0x7f91c0156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c5e00640 1 -- 172.21.15.5:0/3037990708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0155680 con 0x7f91c015b330 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c5e00640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 0x7f91c014d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91b800efc0 tx=0x7f91b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8018020 con 0x7f91c015b330 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b8014480 con 0x7f91c015b330 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8014870 con 0x7f91c015b330 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.083+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c0155960 con 0x7f91c015b330 2024-04-25T10:02:23.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.084+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c0155f20 con 0x7f91c015b330 2024-04-25T10:02:23.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.085+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c0068f60 con 0x7f91c015b330 2024-04-25T10:02:23.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.086+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91b8005d10 con 0x7f91c015b330 2024-04-25T10:02:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.087+0000 7f91bd600640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f91a006b4d0 0x7f91a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.087+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f91b808afc0 con 0x7f91c015b330 2024-04-25T10:02:23.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.087+0000 7f91c5400640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f91a006b4d0 0x7f91a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.089+0000 7f91c5400640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f91a006b4d0 0x7f91a006d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91b4000f60 tx=0x7f91b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.091+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91b8022070 con 0x7f91c015b330 2024-04-25T10:02:23.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:23.242 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.332+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f91c015db50 con 0x7f91c015b330 2024-04-25T10:02:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.439+0000 7f91bd600640 1 -- 172.21.15.5:0/3037990708 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f91b8010040 con 0x7f91c015b330 2024-04-25T10:02:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.442+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f91a006b4d0 msgr2=0x7f91a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.442+0000 7f91c6800640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f91a006b4d0 0x7f91a006d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91b4000f60 tx=0x7f91b4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.442+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 msgr2=0x7f91c014d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.442+0000 7f91c6800640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 0x7f91c014d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f91b800efc0 tx=0x7f91b800c490 comp rx=0 tx=0).stop 2024-04-25T10:02:23.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 shutdown_connections 2024-04-25T10:02:23.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91c015b330 0x7f91c014d150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f91a006b4d0 0x7f91a006d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00cc840 0x7f91c0156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 --2- 172.21.15.5:0/3037990708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f91c006e840 0x7f91c01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 >> 172.21.15.5:0/3037990708 conn(0x7f91c0066f20 msgr2=0x7f91c015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 shutdown_connections 2024-04-25T10:02:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.443+0000 7f91c6800640 1 -- 172.21.15.5:0/3037990708 wait complete. 2024-04-25T10:02:23.470 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T10:02:23.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.681+0000 7f2c9f400640 1 Processor -- start 2024-04-25T10:02:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.681+0000 7f2c9f400640 1 -- start start 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.682+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c981610b0 0x7f2c9806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.682+0000 7f2c9f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2c9809a100 0x7f2c980cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.682+0000 7f2c9f400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c980cfd50 con 0x7f2c9806bd00 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.682+0000 7f2c9f400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c980d0960 con 0x7f2c9809a100 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.682+0000 7f2c9f400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9806d0d0 con 0x7f2c981610b0 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.682+0000 7f2c9d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2c9809a100 0x7f2c980cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2c9809a100 0x7f2c980cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46688/0 (socket says 172.21.15.5:46688) 2024-04-25T10:02:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9d400640 1 -- 172.21.15.5:0/425050400 learned_addr learned my addr 172.21.15.5:0/425050400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9ca00640 1 --2- 172.21.15.5:0/425050400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c981610b0 0x7f2c9806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9d400640 1 -- 172.21.15.5:0/425050400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c981610b0 msgr2=0x7f2c9806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9d400640 1 --2- 172.21.15.5:0/425050400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c981610b0 0x7f2c9806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9d400640 1 -- 172.21.15.5:0/425050400 >> v1:172.21.15.80:6789/0 conn(0x7f2c9806bd00 legacy=0x7f2c980999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.683+0000 7f2c9d400640 1 -- 172.21.15.5:0/425050400 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980d0030 con 0x7f2c9809a100 2024-04-25T10:02:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.684+0000 7f2c9d400640 1 --2- 172.21.15.5:0/425050400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9809a100 0x7f2c980cbd20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009870 tx=0x7f2c9002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca770dc5728ca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.684+0000 7f2c97400640 1 -- 172.21.15.5:0/425050400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90037070 con 0x7f2c9809a100 2024-04-25T10:02:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c97400640 1 -- 172.21.15.5:0/425050400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c90032440 con 0x7f2c9809a100 2024-04-25T10:02:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c97400640 1 -- 172.21.15.5:0/425050400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90032810 con 0x7f2c9809a100 2024-04-25T10:02:23.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 -- 172.21.15.5:0/425050400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9809a100 msgr2=0x7f2c980cbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 --2- 172.21.15.5:0/425050400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9809a100 0x7f2c980cbd20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009870 tx=0x7f2c9002a200 comp rx=0 tx=0).stop 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 -- 172.21.15.5:0/425050400 shutdown_connections 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 --2- 172.21.15.5:0/425050400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9809a100 0x7f2c980cbd20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 --2- 172.21.15.5:0/425050400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c981610b0 0x7f2c9806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 -- 172.21.15.5:0/425050400 >> 172.21.15.5:0/425050400 conn(0x7f2c980b31c0 msgr2=0x7f2c980b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 -- 172.21.15.5:0/425050400 shutdown_connections 2024-04-25T10:02:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.685+0000 7f2c9f400640 1 -- 172.21.15.5:0/425050400 wait complete. 2024-04-25T10:02:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.687+0000 7f2c9f400640 1 Processor -- start 2024-04-25T10:02:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.687+0000 7f2c9f400640 1 -- start start 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.688+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 0x7f2c981e8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.688+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c9809a100 0x7f2c981e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.688+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981610b0 0x7f2c98152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.688+0000 7f2c9f400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98161490 con 0x7f2c9806bd00 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.688+0000 7f2c9f400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c980d05f0 con 0x7f2c981610b0 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.688+0000 7f2c9f400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c980cfa00 con 0x7f2c9809a100 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 0x7f2c981e8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 0x7f2c981e8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46690/0 (socket says 172.21.15.5:46690) 2024-04-25T10:02:23.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 -- 172.21.15.5:0/370084606 learned_addr learned my addr 172.21.15.5:0/370084606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c97e00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c9809a100 0x7f2c981e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9d400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981610b0 0x7f2c98152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 -- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c9809a100 msgr2=0x7f2c981e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c9809a100 0x7f2c981e6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 -- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981610b0 msgr2=0x7f2c98152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981610b0 0x7f2c98152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.689+0000 7f2c9ca00640 1 -- 172.21.15.5:0/370084606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c981e6e80 con 0x7f2c9806bd00 2024-04-25T10:02:23.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.690+0000 7f2c9ca00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 0x7f2c981e8eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2c88007ef0 tx=0x7f2c8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.690+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88014620 con 0x7f2c9806bd00 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.690+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c88019070 con 0x7f2c9806bd00 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.690+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88014a00 con 0x7f2c9806bd00 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.690+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c981e7160 con 0x7f2c9806bd00 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.690+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c981e7720 con 0x7f2c9806bd00 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.691+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c98068f60 con 0x7f2c9806bd00 2024-04-25T10:02:23.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.692+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c88014ba0 con 0x7f2c9806bd00 2024-04-25T10:02:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.693+0000 7f2c94c00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c7806b4d0 0x7f2c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.693+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2c8808ae90 con 0x7f2c9806bd00 2024-04-25T10:02:23.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.694+0000 7f2c97e00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c7806b4d0 0x7f2c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.695+0000 7f2c97e00640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c7806b4d0 0x7f2c7806d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c84006fd0 tx=0x7f2c84008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.698+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c880599c0 con 0x7f2c9806bd00 2024-04-25T10:02:23.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:23.939+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2c980cda80 con 0x7f2c9806bd00 2024-04-25T10:02:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.453+0000 7f2c94c00640 1 -- 172.21.15.5:0/370084606 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c8808c0c0 con 0x7f2c9806bd00 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c7806b4d0 msgr2=0x7f2c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c7806b4d0 0x7f2c7806d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2c84006fd0 tx=0x7f2c84008040 comp rx=0 tx=0).stop 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 msgr2=0x7f2c981e8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 0x7f2c981e8eb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2c88007ef0 tx=0x7f2c8800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 shutdown_connections 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2c7806b4d0 0x7f2c7806d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2c981610b0 0x7f2c98152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2c9809a100 0x7f2c981e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.457+0000 7f2c9f400640 1 --2- 172.21.15.5:0/370084606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c9806bd00 0x7f2c981e8eb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.458+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 >> 172.21.15.5:0/370084606 conn(0x7f2c980b31c0 msgr2=0x7f2c980b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.458+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 shutdown_connections 2024-04-25T10:02:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.458+0000 7f2c9f400640 1 -- 172.21.15.5:0/370084606 wait complete. 2024-04-25T10:02:24.483 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:24.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:24.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.687+0000 7f4833e00640 1 Processor -- start 2024-04-25T10:02:24.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.687+0000 7f4833e00640 1 -- start start 2024-04-25T10:02:24.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.688+0000 7f4833e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f483415ce00 0x7f483415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4833e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4834071d30 con 0x7f48340cd300 2024-04-25T10:02:24.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4833e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483406d620 con 0x7f4834141da0 2024-04-25T10:02:24.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4833e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4834142180 con 0x7f483415ce00 2024-04-25T10:02:24.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4833400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f48340cd300 0x7f4834071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59792/0 (socket says 172.21.15.5:59792) 2024-04-25T10:02:24.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4833400640 1 -- 172.21.15.5:0/1616305724 learned_addr learned my addr 172.21.15.5:0/1616305724 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788991321 0 0) 0x7f4834071d30 con 0x7f48340cd300 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818003520 con 0x7f48340cd300 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.689+0000 7f4832000640 1 --2- 172.21.15.5:0/1616305724 >> v2:172.21.15.80:3300/0 conn(0x7f483415ce00 0x7f483415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436052598 0 0) 0x7f4818003520 con 0x7f48340cd300 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 >> v2:172.21.15.80:3300/0 conn(0x7f483415ce00 msgr2=0x7f483415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 --2- 172.21.15.5:0/1616305724 >> v2:172.21.15.80:3300/0 conn(0x7f483415ce00 0x7f483415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 >> v1:172.21.15.80:6789/0 conn(0x7f4834141da0 legacy=0x7f483406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483406d900 con 0x7f48340cd300 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f482c0019f0 con 0x7f48340cd300 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 >> v1:172.21.15.5:6789/0 conn(0x7f48340cd300 legacy=0x7f4834071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4818003a00 0x7f4818003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818004800 0x7f4818004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818005210 0x7f481800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834142180 con 0x7f4818003a00 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481800c160 con 0x7f4818005210 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481800c360 con 0x7f4818004800 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.690+0000 7f4832a00640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4818003a00 0x7f4818003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832000640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818004800 0x7f4818004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4833400640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818005210 0x7f481800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832a00640 1 -- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818004800 msgr2=0x7f4818004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832a00640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818004800 0x7f4818004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832a00640 1 -- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818005210 msgr2=0x7f481800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832a00640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818005210 0x7f481800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832a00640 1 -- 172.21.15.5:0/1616305724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818003520 con 0x7f4818003a00 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4832000640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818004800 0x7f4818004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4833400640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4818005210 0x7f481800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.691+0000 7f4833400640 1 -- 172.21.15.5:0/1616305724 reap_dead start 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.692+0000 7f4832a00640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4818003a00 0x7f4818003e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4824001750 tx=0x7f4818005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14655243be90d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.692+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824001370 con 0x7f4818003a00 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.692+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48240052d0 con 0x7f4818003a00 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.692+0000 7f4831600640 1 -- 172.21.15.5:0/1616305724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482402abc0 con 0x7f4818003a00 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 -- 172.21.15.5:0/1616305724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4818003a00 msgr2=0x7f4818003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4818003a00 0x7f4818003e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4824001750 tx=0x7f4818005830 comp rx=0 tx=0).stop 2024-04-25T10:02:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 -- 172.21.15.5:0/1616305724 shutdown_connections 2024-04-25T10:02:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 --2- 172.21.15.5:0/1616305724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4818003a00 0x7f4818003e20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 -- 172.21.15.5:0/1616305724 >> 172.21.15.5:0/1616305724 conn(0x7f4834066f20 msgr2=0x7f483406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 -- 172.21.15.5:0/1616305724 shutdown_connections 2024-04-25T10:02:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.693+0000 7f4833e00640 1 -- 172.21.15.5:0/1616305724 wait complete. 2024-04-25T10:02:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.695+0000 7f4833e00640 1 Processor -- start 2024-04-25T10:02:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.695+0000 7f4833e00640 1 -- start start 2024-04-25T10:02:24.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818003a00 0x7f4834139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48340aed60 0x7f48340af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 0x7f48340b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483415d400 con 0x7f48340b39f0 2024-04-25T10:02:24.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483415d280 con 0x7f48340aed60 2024-04-25T10:02:24.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483415d580 con 0x7f4818003a00 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 0x7f48340b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 0x7f48340b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46712/0 (socket says 172.21.15.5:46712) 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4833400640 1 -- 172.21.15.5:0/1997620938 learned_addr learned my addr 172.21.15.5:0/1997620938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.696+0000 7f4832a00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818003a00 0x7f4834139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4832000640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48340aed60 0x7f48340af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833400640 1 -- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818003a00 msgr2=0x7f4834139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833400640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818003a00 0x7f4834139740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833400640 1 -- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48340aed60 msgr2=0x7f48340af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833400640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48340aed60 0x7f48340af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833400640 1 -- 172.21.15.5:0/1997620938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483415ad90 con 0x7f48340b39f0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833400640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 0x7f48340b3e70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f48340cd290 tx=0x7f482c00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00b8a0 con 0x7f48340b39f0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c00ba40 con 0x7f48340b39f0 2024-04-25T10:02:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00be30 con 0x7f48340b39f0 2024-04-25T10:02:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.697+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48341f6ea0 con 0x7f48340b39f0 2024-04-25T10:02:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.698+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48341f7460 con 0x7f48340b39f0 2024-04-25T10:02:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.699+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48340af720 con 0x7f48340b39f0 2024-04-25T10:02:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.700+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f482c01b020 con 0x7f48340b39f0 2024-04-25T10:02:24.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.701+0000 7f482aa00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f481406b4d0 0x7f481406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.701+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f482c08a750 con 0x7f48340b39f0 2024-04-25T10:02:24.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.702+0000 7f4832a00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f481406b4d0 0x7f481406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.703+0000 7f4832a00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f481406b4d0 0x7f481406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4824032000 tx=0x7f482402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.706+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f482c059280 con 0x7f48340b39f0 2024-04-25T10:02:24.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.942+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4834068f60 con 0x7f48340b39f0 2024-04-25T10:02:24.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.944+0000 7f482aa00640 1 -- 172.21.15.5:0/1997620938 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7f482c058b80 con 0x7f48340b39f0 2024-04-25T10:02:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:24.949 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:24.949 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:02:24.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.946+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f481406b4d0 msgr2=0x7f481406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.946+0000 7f4833e00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f481406b4d0 0x7f481406d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4824032000 tx=0x7f482402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:24.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.946+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 msgr2=0x7f48340b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.946+0000 7f4833e00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 0x7f48340b3e70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f48340cd290 tx=0x7f482c00b0d0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 shutdown_connections 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48340b39f0 0x7f48340b3e70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f481406b4d0 0x7f481406d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48340aed60 0x7f48340af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 --2- 172.21.15.5:0/1997620938 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4818003a00 0x7f4834139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 >> 172.21.15.5:0/1997620938 conn(0x7f4834066f20 msgr2=0x7f483406e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 shutdown_connections 2024-04-25T10:02:24.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:24.947+0000 7f4833e00640 1 -- 172.21.15.5:0/1997620938 wait complete. 2024-04-25T10:02:24.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.173+0000 7f0f33600640 1 Processor -- start 2024-04-25T10:02:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.173+0000 7f0f33600640 1 -- start start 2024-04-25T10:02:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.174+0000 7f0f33600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 0x7f0f2c0d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.174+0000 7f0f33600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c0d1e80 con 0x7f0f2c15db00 2024-04-25T10:02:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f33600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c0b5930 con 0x7f0f2c0d0a50 2024-04-25T10:02:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f33600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b5c10 con 0x7f0f2c0b53b0 2024-04-25T10:02:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f30c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0f2c15db00 0x7f0f2c0d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59808/0 (socket says 172.21.15.5:59808) 2024-04-25T10:02:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f30c00640 1 -- 172.21.15.5:0/1358415933 learned_addr learned my addr 172.21.15.5:0/1358415933 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f31600640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 0x7f0f2c0d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812325206 0 0) 0x7f0f2c0d1e80 con 0x7f0f2c15db00 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.175+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f10003520 con 0x7f0f2c15db00 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943167888 0 0) 0x7f0f10003520 con 0x7f0f2c15db00 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 msgr2=0x7f0f2c0d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 0x7f0f2c0d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 >> v1:172.21.15.80:6789/0 conn(0x7f0f2c0d0a50 legacy=0x7f0f2c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0b5ef0 con 0x7f0f2c15db00 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0f1c0019f0 con 0x7f0f2c15db00 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 >> v1:172.21.15.5:6789/0 conn(0x7f0f2c15db00 legacy=0x7f0f2c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f100039e0 0x7f0f10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 0x7f0f10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f10004d40 0x7f0f100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b5c10 con 0x7f0f10004d40 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1000ba70 con 0x7f0f100039e0 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1000bc40 con 0x7f0f2c0b53b0 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f31600640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f10004d40 0x7f0f100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f30c00640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f100039e0 0x7f0f10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.176+0000 7f0f2be00640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 0x7f0f10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.177+0000 7f0f31600640 1 -- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 msgr2=0x7f0f10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.177+0000 7f0f31600640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b53b0 0x7f0f10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.177+0000 7f0f31600640 1 -- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f100039e0 msgr2=0x7f0f10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.177+0000 7f0f31600640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f100039e0 0x7f0f10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.177+0000 7f0f31600640 1 -- 172.21.15.5:0/1358415933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10003520 con 0x7f0f10004d40 2024-04-25T10:02:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.177+0000 7f0f30c00640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f100039e0 0x7f0f10003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.178+0000 7f0f31600640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f10004d40 0x7f0f100051c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ac0 tx=0x7f0f10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1935852facd688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.178+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24036070 con 0x7f0f10004d40 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.178+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f24031400 con 0x7f0f10004d40 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.178+0000 7f0f2b400640 1 -- 172.21.15.5:0/1358415933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24009690 con 0x7f0f10004d40 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f33600640 1 -- 172.21.15.5:0/1358415933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f10004d40 msgr2=0x7f0f100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f33600640 1 --2- 172.21.15.5:0/1358415933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f10004d40 0x7f0f100051c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ac0 tx=0x7f0f10005830 comp rx=0 tx=0).stop 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f31600640 1 -- 172.21.15.5:0/1358415933 reap_dead start 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f33600640 1 -- 172.21.15.5:0/1358415933 shutdown_connections 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f33600640 1 -- 172.21.15.5:0/1358415933 >> 172.21.15.5:0/1358415933 conn(0x7f0f2c072290 msgr2=0x7f0f2c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f33600640 1 -- 172.21.15.5:0/1358415933 shutdown_connections 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.179+0000 7f0f33600640 1 -- 172.21.15.5:0/1358415933 wait complete. 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.180+0000 7f0f33600640 1 Processor -- start 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.181+0000 7f0f33600640 1 -- start start 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.181+0000 7f0f33600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10004d40 0x7f0f2c0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f33600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 0x7f0f2c149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f33600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b7eb0 0x7f0f2c0b8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f33600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0d3660 con 0x7f0f2c154fd0 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f33600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b61d0 con 0x7f0f10004d40 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f33600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0ce2b0 con 0x7f0f2c0b7eb0 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 0x7f0f2c149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 0x7f0f2c149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46738/0 (socket says 172.21.15.5:46738) 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 -- 172.21.15.5:0/466536898 learned_addr learned my addr 172.21.15.5:0/466536898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f31600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b7eb0 0x7f0f2c0b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f30c00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10004d40 0x7f0f2c0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 -- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b7eb0 msgr2=0x7f0f2c0b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b7eb0 0x7f0f2c0b8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 -- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10004d40 msgr2=0x7f0f2c0ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10004d40 0x7f0f2c0ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f2be00640 1 -- 172.21.15.5:0/466536898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c155470 con 0x7f0f2c154fd0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f30c00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10004d40 0x7f0f2c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.182+0000 7f0f31600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b7eb0 0x7f0f2c0b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.183+0000 7f0f2be00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 0x7f0f2c149b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800e5a0 tx=0x7f0f1800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.183+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f180053a0 con 0x7f0f2c154fd0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.183+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f18018070 con 0x7f0f2c154fd0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.183+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c155750 con 0x7f0f2c154fd0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.183+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18003830 con 0x7f0f2c154fd0 2024-04-25T10:02:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.183+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c155d10 con 0x7f0f2c154fd0 2024-04-25T10:02:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.184+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c068fd0 con 0x7f0f2c154fd0 2024-04-25T10:02:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.185+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f180039d0 con 0x7f0f2c154fd0 2024-04-25T10:02:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.186+0000 7f0f28c00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.187+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0f1808a370 con 0x7f0f2c154fd0 2024-04-25T10:02:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.187+0000 7f0f30c00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.188+0000 7f0f30c00640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005d10 tx=0x7f0f1c001250 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.190+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f1801a310 con 0x7f0f2c154fd0 2024-04-25T10:02:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.395+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f2c0c95d0 con 0x7f0f2c154fd0 2024-04-25T10:02:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.399+0000 7f0f28c00640 1 -- 172.21.15.5:0/466536898 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f0f18081e20 con 0x7f0f2c154fd0 2024-04-25T10:02:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:25.403 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.401+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f0c06b4d0 msgr2=0x7f0f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.401+0000 7f0f33600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005d10 tx=0x7f0f1c001250 comp rx=0 tx=0).stop 2024-04-25T10:02:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.401+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 msgr2=0x7f0f2c149b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.401+0000 7f0f33600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 0x7f0f2c149b00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800e5a0 tx=0x7f0f1800ea60 comp rx=0 tx=0).stop 2024-04-25T10:02:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.401+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 shutdown_connections 2024-04-25T10:02:25.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.401+0000 7f0f33600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f2c0b7eb0 0x7f0f2c0b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.402+0000 7f0f33600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f2c154fd0 0x7f0f2c149b00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.402+0000 7f0f33600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f0c06b4d0 0x7f0f0c06d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.402+0000 7f0f33600640 1 --2- 172.21.15.5:0/466536898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f10004d40 0x7f0f2c0ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.402+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 >> 172.21.15.5:0/466536898 conn(0x7f0f2c072290 msgr2=0x7f0f2c153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.402+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 shutdown_connections 2024-04-25T10:02:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.402+0000 7f0f33600640 1 -- 172.21.15.5:0/466536898 wait complete. 2024-04-25T10:02:25.426 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:25.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:25.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.637+0000 7fb702e00640 1 Processor -- start 2024-04-25T10:02:25.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.637+0000 7fb702e00640 1 -- start start 2024-04-25T10:02:25.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6fc154a10 0x7fb6fc15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6fc0cdca0 0x7fb6fc0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc0b4890 con 0x7fb6fc070f90 2024-04-25T10:02:25.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc0cd200 con 0x7fb6fc154a10 2024-04-25T10:02:25.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc06e6f0 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb701a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6fc154a10 0x7fb6fc15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb6fc070f90 0x7fb6fc0713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59824/0 (socket says 172.21.15.5:59824) 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.639+0000 7fb702400640 1 -- 172.21.15.5:0/279152475 learned_addr learned my addr 172.21.15.5:0/279152475 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb701000640 1 --2- 172.21.15.5:0/279152475 >> v2:172.21.15.80:3300/0 conn(0x7fb6fc0cdca0 0x7fb6fc0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb6fbe00640 1 -- 172.21.15.5:0/279152475 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348578487 0 0) 0x7fb6fc0b4890 con 0x7fb6fc070f90 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb6fbe00640 1 -- 172.21.15.5:0/279152475 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6e4003520 con 0x7fb6fc070f90 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb701a00640 1 -- 172.21.15.5:0/279152475 >> v2:172.21.15.80:3300/0 conn(0x7fb6fc0cdca0 msgr2=0x7fb6fc0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb701a00640 1 --2- 172.21.15.5:0/279152475 >> v2:172.21.15.80:3300/0 conn(0x7fb6fc0cdca0 0x7fb6fc0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb701a00640 1 -- 172.21.15.5:0/279152475 >> v1:172.21.15.5:6789/0 conn(0x7fb6fc070f90 legacy=0x7fb6fc0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.640+0000 7fb701a00640 1 -- 172.21.15.5:0/279152475 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc06e9d0 con 0x7fb6fc154a10 2024-04-25T10:02:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.641+0000 7fb701a00640 1 --2- 172.21.15.5:0/279152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc154a10 0x7fb6fc15e8c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0aef00 tx=0x7fb6f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f86c397730c6877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb6fbe00640 1 -- 172.21.15.5:0/279152475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0037070 con 0x7fb6fc154a10 2024-04-25T10:02:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb6fbe00640 1 -- 172.21.15.5:0/279152475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f0032440 con 0x7fb6fc154a10 2024-04-25T10:02:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb6fbe00640 1 -- 172.21.15.5:0/279152475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f002ae60 con 0x7fb6fc154a10 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb702e00640 1 -- 172.21.15.5:0/279152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc154a10 msgr2=0x7fb6fc15e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb702e00640 1 --2- 172.21.15.5:0/279152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc154a10 0x7fb6fc15e8c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0aef00 tx=0x7fb6f002a110 comp rx=0 tx=0).stop 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb702e00640 1 -- 172.21.15.5:0/279152475 shutdown_connections 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb702e00640 1 --2- 172.21.15.5:0/279152475 >> v2:172.21.15.80:3300/0 conn(0x7fb6fc0cdca0 0x7fb6fc0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb702e00640 1 --2- 172.21.15.5:0/279152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc154a10 0x7fb6fc15e8c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.642+0000 7fb702e00640 1 -- 172.21.15.5:0/279152475 >> 172.21.15.5:0/279152475 conn(0x7fb6fc066f20 msgr2=0x7fb6fc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.643+0000 7fb702e00640 1 -- 172.21.15.5:0/279152475 shutdown_connections 2024-04-25T10:02:25.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.643+0000 7fb702e00640 1 -- 172.21.15.5:0/279152475 wait complete. 2024-04-25T10:02:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.644+0000 7fb702e00640 1 Processor -- start 2024-04-25T10:02:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.645+0000 7fb702e00640 1 -- start start 2024-04-25T10:02:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.645+0000 7fb702e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6fc070f90 0x7fb6fc147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.645+0000 7fb702e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 0x7fb6fc0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb702e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6fc154a10 0x7fb6fc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb702e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0cd780 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb702e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0cd4a0 con 0x7fb6fc070f90 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb702e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc06cff0 con 0x7fb6fc154a10 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 0x7fb6fc0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 0x7fb6fc0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46764/0 (socket says 172.21.15.5:46764) 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 -- 172.21.15.5:0/2128256741 learned_addr learned my addr 172.21.15.5:0/2128256741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701a00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6fc070f90 0x7fb6fc147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb702400640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6fc154a10 0x7fb6fc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 -- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6fc154a10 msgr2=0x7fb6fc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6fc154a10 0x7fb6fc0b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 -- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6fc070f90 msgr2=0x7fb6fc147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6fc070f90 0x7fb6fc147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.646+0000 7fb701000640 1 -- 172.21.15.5:0/2128256741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc1481b0 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.647+0000 7fb701000640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 0x7fb6fc0b1450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9b0 tx=0x7fb6e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.647+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8017070 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.647+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e800c9a0 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.647+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800cd90 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.647+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6fc0afc80 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.647+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc0b0240 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.649+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6fc071d80 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.649+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6e801a070 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.650+0000 7fb6f9600640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb6dc06b4d0 0x7fb6dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.651+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb6e808a500 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.651+0000 7fb701a00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb6dc06b4d0 0x7fb6dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.652+0000 7fb701a00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb6dc06b4d0 0x7fb6dc06d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0034000 tx=0x7fb6f002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.655+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6fc1481b0 con 0x7fb6fc0cdca0 2024-04-25T10:02:25.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.857+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6fc06e4b0 con 0x7fb6dc06b4d0 2024-04-25T10:02:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.887+0000 7fb6f9600640 1 -- 172.21.15.5:0/2128256741 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332569 (secure 0 0 0) 0x7fb6fc06e4b0 con 0x7fb6dc06b4d0 2024-04-25T10:02:25.897 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb6dc06b4d0 msgr2=0x7fb6dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb6dc06b4d0 0x7fb6dc06d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0034000 tx=0x7fb6f002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 msgr2=0x7fb6fc0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 0x7fb6fc0b1450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9b0 tx=0x7fb6e800ee70 comp rx=0 tx=0).stop 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 shutdown_connections 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6fc154a10 0x7fb6fc0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6fc0cdca0 0x7fb6fc0b1450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb6dc06b4d0 0x7fb6dc06d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 --2- 172.21.15.5:0/2128256741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6fc070f90 0x7fb6fc147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.893+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 >> 172.21.15.5:0/2128256741 conn(0x7fb6fc066f20 msgr2=0x7fb6fc0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.894+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 shutdown_connections 2024-04-25T10:02:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:25.894+0000 7fb702e00640 1 -- 172.21.15.5:0/2128256741 wait complete. 2024-04-25T10:02:25.917 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T10:02:24.618233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236148,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738572,"statfs":{"total":773094113280,"available":772852297728,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006491"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356469+0000","last_change":"2024-04-25T10:01:57.061422+0000","last_active":"2024-04-25T10:02:03.356469+0000","last_peered":"2024-04-25T10:02:03.356469+0000","last_clean":"2024-04-25T10:02:03.356469+0000","last_became_active":"2024-04-25T10:01:57.061119+0000","last_became_peered":"2024-04-25T10:01:57.061119+0000","last_unstale":"2024-04-25T10:02:03.356469+0000","last_undegraded":"2024-04-25T10:02:03.356469+0000","last_fullsized":"2024-04-25T10:02:03.356469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:26.806679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433872+0000","last_change":"2024-04-25T10:01:58.115482+0000","last_active":"2024-04-25T10:02:03.433872+0000","last_peered":"2024-04-25T10:02:03.433872+0000","last_clean":"2024-04-25T10:02:03.433872+0000","last_became_active":"2024-04-25T10:01:58.115138+0000","last_became_peered":"2024-04-25T10:01:58.115138+0000","last_unstale":"2024-04-25T10:02:03.433872+0000","last_undegraded":"2024-04-25T10:02:03.433872+0000","last_fullsized":"2024-04-25T10:02:03.433872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352821+0000","last_change":"2024-04-25T10:02:02.368709+0000","last_active":"2024-04-25T10:02:03.352821+0000","last_peered":"2024-04-25T10:02:03.352821+0000","last_clean":"2024-04-25T10:02:03.352821+0000","last_became_active":"2024-04-25T10:02:02.367787+0000","last_became_peered":"2024-04-25T10:02:02.367787+0000","last_unstale":"2024-04-25T10:02:03.352821+0000","last_undegraded":"2024-04-25T10:02:03.352821+0000","last_fullsized":"2024-04-25T10:02:03.352821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:31.444725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434120+0000","last_change":"2024-04-25T10:01:57.061756+0000","last_active":"2024-04-25T10:02:03.434120+0000","last_peered":"2024-04-25T10:02:03.434120+0000","last_clean":"2024-04-25T10:02:03.434120+0000","last_became_active":"2024-04-25T10:01:57.058627+0000","last_became_peered":"2024-04-25T10:01:57.058627+0000","last_unstale":"2024-04-25T10:02:03.434120+0000","last_undegraded":"2024-04-25T10:02:03.434120+0000","last_fullsized":"2024-04-25T10:02:03.434120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434000+0000","last_change":"2024-04-25T10:01:58.113879+0000","last_active":"2024-04-25T10:02:03.434000+0000","last_peered":"2024-04-25T10:02:03.434000+0000","last_clean":"2024-04-25T10:02:03.434000+0000","last_became_active":"2024-04-25T10:01:58.113527+0000","last_became_peered":"2024-04-25T10:01:58.113527+0000","last_unstale":"2024-04-25T10:02:03.434000+0000","last_undegraded":"2024-04-25T10:02:03.434000+0000","last_fullsized":"2024-04-25T10:02:03.434000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519097+0000","last_change":"2024-04-25T10:02:02.364833+0000","last_active":"2024-04-25T10:02:03.519097+0000","last_peered":"2024-04-25T10:02:03.519097+0000","last_clean":"2024-04-25T10:02:03.519097+0000","last_became_active":"2024-04-25T10:02:02.364420+0000","last_became_peered":"2024-04-25T10:02:02.364420+0000","last_unstale":"2024-04-25T10:02:03.519097+0000","last_undegraded":"2024-04-25T10:02:03.519097+0000","last_fullsized":"2024-04-25T10:02:03.519097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:17.582640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518857+0000","last_change":"2024-04-25T10:01:57.049323+0000","last_active":"2024-04-25T10:02:03.518857+0000","last_peered":"2024-04-25T10:02:03.518857+0000","last_clean":"2024-04-25T10:02:03.518857+0000","last_became_active":"2024-04-25T10:01:57.048997+0000","last_became_peered":"2024-04-25T10:01:57.048997+0000","last_unstale":"2024-04-25T10:02:03.518857+0000","last_undegraded":"2024-04-25T10:02:03.518857+0000","last_fullsized":"2024-04-25T10:02:03.518857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.523353+0000","last_change":"2024-04-25T10:01:58.074975+0000","last_active":"2024-04-25T10:02:03.523353+0000","last_peered":"2024-04-25T10:02:03.523353+0000","last_clean":"2024-04-25T10:02:03.523353+0000","last_became_active":"2024-04-25T10:01:58.074637+0000","last_became_peered":"2024-04-25T10:01:58.074637+0000","last_unstale":"2024-04-25T10:02:03.523353+0000","last_undegraded":"2024-04-25T10:02:03.523353+0000","last_fullsized":"2024-04-25T10:02:03.523353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522261+0000","last_change":"2024-04-25T10:02:02.367186+0000","last_active":"2024-04-25T10:02:03.522261+0000","last_peered":"2024-04-25T10:02:03.522261+0000","last_clean":"2024-04-25T10:02:03.522261+0000","last_became_active":"2024-04-25T10:02:02.366476+0000","last_became_peered":"2024-04-25T10:02:02.366476+0000","last_unstale":"2024-04-25T10:02:03.522261+0000","last_undegraded":"2024-04-25T10:02:03.522261+0000","last_fullsized":"2024-04-25T10:02:03.522261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519518+0000","last_change":"2024-04-25T10:01:57.058161+0000","last_active":"2024-04-25T10:02:03.519518+0000","last_peered":"2024-04-25T10:02:03.519518+0000","last_clean":"2024-04-25T10:02:03.519518+0000","last_became_active":"2024-04-25T10:01:57.057891+0000","last_became_peered":"2024-04-25T10:01:57.057891+0000","last_unstale":"2024-04-25T10:02:03.519518+0000","last_undegraded":"2024-04-25T10:02:03.519518+0000","last_fullsized":"2024-04-25T10:02:03.519518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:23.026096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404272+0000","last_change":"2024-04-25T10:01:58.074490+0000","last_active":"2024-04-25T10:02:03.404272+0000","last_peered":"2024-04-25T10:02:03.404272+0000","last_clean":"2024-04-25T10:02:03.404272+0000","last_became_active":"2024-04-25T10:01:58.074117+0000","last_became_peered":"2024-04-25T10:01:58.074117+0000","last_unstale":"2024-04-25T10:02:03.404272+0000","last_undegraded":"2024-04-25T10:02:03.404272+0000","last_fullsized":"2024-04-25T10:02:03.404272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:53.841219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520948+0000","last_change":"2024-04-25T10:02:02.362410+0000","last_active":"2024-04-25T10:02:03.520948+0000","last_peered":"2024-04-25T10:02:03.520948+0000","last_clean":"2024-04-25T10:02:03.520948+0000","last_became_active":"2024-04-25T10:02:02.359612+0000","last_became_peered":"2024-04-25T10:02:02.359612+0000","last_unstale":"2024-04-25T10:02:03.520948+0000","last_undegraded":"2024-04-25T10:02:03.520948+0000","last_fullsized":"2024-04-25T10:02:03.520948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.136854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348157+0000","last_change":"2024-04-25T10:01:58.073917+0000","last_active":"2024-04-25T10:02:03.348157+0000","last_peered":"2024-04-25T10:02:03.348157+0000","last_clean":"2024-04-25T10:02:03.348157+0000","last_became_active":"2024-04-25T10:01:58.073122+0000","last_became_peered":"2024-04-25T10:01:58.073122+0000","last_unstale":"2024-04-25T10:02:03.348157+0000","last_undegraded":"2024-04-25T10:02:03.348157+0000","last_fullsized":"2024-04-25T10:02:03.348157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402599+0000","last_change":"2024-04-25T10:01:57.059507+0000","last_active":"2024-04-25T10:02:03.402599+0000","last_peered":"2024-04-25T10:02:03.402599+0000","last_clean":"2024-04-25T10:02:03.402599+0000","last_became_active":"2024-04-25T10:01:57.056083+0000","last_became_peered":"2024-04-25T10:01:57.056083+0000","last_unstale":"2024-04-25T10:02:03.402599+0000","last_undegraded":"2024-04-25T10:02:03.402599+0000","last_fullsized":"2024-04-25T10:02:03.402599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:29.264217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404104+0000","last_change":"2024-04-25T10:02:02.362032+0000","last_active":"2024-04-25T10:02:03.404104+0000","last_peered":"2024-04-25T10:02:03.404104+0000","last_clean":"2024-04-25T10:02:03.404104+0000","last_became_active":"2024-04-25T10:02:02.361608+0000","last_became_peered":"2024-04-25T10:02:02.361608+0000","last_unstale":"2024-04-25T10:02:03.404104+0000","last_undegraded":"2024-04-25T10:02:03.404104+0000","last_fullsized":"2024-04-25T10:02:03.404104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:27.993920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353535+0000","last_change":"2024-04-25T10:01:58.116451+0000","last_active":"2024-04-25T10:02:03.353535+0000","last_peered":"2024-04-25T10:02:03.353535+0000","last_clean":"2024-04-25T10:02:03.353535+0000","last_became_active":"2024-04-25T10:01:58.116177+0000","last_became_peered":"2024-04-25T10:01:58.116177+0000","last_unstale":"2024-04-25T10:02:03.353535+0000","last_undegraded":"2024-04-25T10:02:03.353535+0000","last_fullsized":"2024-04-25T10:02:03.353535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518845+0000","last_change":"2024-04-25T10:01:57.062452+0000","last_active":"2024-04-25T10:02:03.518845+0000","last_peered":"2024-04-25T10:02:03.518845+0000","last_clean":"2024-04-25T10:02:03.518845+0000","last_became_active":"2024-04-25T10:01:57.062105+0000","last_became_peered":"2024-04-25T10:01:57.062105+0000","last_unstale":"2024-04-25T10:02:03.518845+0000","last_undegraded":"2024-04-25T10:02:03.518845+0000","last_fullsized":"2024-04-25T10:02:03.518845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518790+0000","last_change":"2024-04-25T10:02:02.369531+0000","last_active":"2024-04-25T10:02:03.518790+0000","last_peered":"2024-04-25T10:02:03.518790+0000","last_clean":"2024-04-25T10:02:03.518790+0000","last_became_active":"2024-04-25T10:02:02.367651+0000","last_became_peered":"2024-04-25T10:02:02.367651+0000","last_unstale":"2024-04-25T10:02:03.518790+0000","last_undegraded":"2024-04-25T10:02:03.518790+0000","last_fullsized":"2024-04-25T10:02:03.518790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:58.140005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352233+0000","last_change":"2024-04-25T10:01:58.117205+0000","last_active":"2024-04-25T10:02:03.352233+0000","last_peered":"2024-04-25T10:02:03.352233+0000","last_clean":"2024-04-25T10:02:03.352233+0000","last_became_active":"2024-04-25T10:01:58.116873+0000","last_became_peered":"2024-04-25T10:01:58.116873+0000","last_unstale":"2024-04-25T10:02:03.352233+0000","last_undegraded":"2024-04-25T10:02:03.352233+0000","last_fullsized":"2024-04-25T10:02:03.352233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:25.305583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354051+0000","last_change":"2024-04-25T10:01:57.049606+0000","last_active":"2024-04-25T10:02:03.354051+0000","last_peered":"2024-04-25T10:02:03.354051+0000","last_clean":"2024-04-25T10:02:03.354051+0000","last_became_active":"2024-04-25T10:01:57.049300+0000","last_became_peered":"2024-04-25T10:01:57.049300+0000","last_unstale":"2024-04-25T10:02:03.354051+0000","last_undegraded":"2024-04-25T10:02:03.354051+0000","last_fullsized":"2024-04-25T10:02:03.354051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:29.284211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354667+0000","last_change":"2024-04-25T10:02:02.366098+0000","last_active":"2024-04-25T10:02:03.354667+0000","last_peered":"2024-04-25T10:02:03.354667+0000","last_clean":"2024-04-25T10:02:03.354667+0000","last_became_active":"2024-04-25T10:02:02.365669+0000","last_became_peered":"2024-04-25T10:02:02.365669+0000","last_unstale":"2024-04-25T10:02:03.354667+0000","last_undegraded":"2024-04-25T10:02:03.354667+0000","last_fullsized":"2024-04-25T10:02:03.354667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:34.132211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347971+0000","last_change":"2024-04-25T10:01:58.116653+0000","last_active":"2024-04-25T10:02:03.347971+0000","last_peered":"2024-04-25T10:02:03.347971+0000","last_clean":"2024-04-25T10:02:03.347971+0000","last_became_active":"2024-04-25T10:01:58.116414+0000","last_became_peered":"2024-04-25T10:01:58.116414+0000","last_unstale":"2024-04-25T10:02:03.347971+0000","last_undegraded":"2024-04-25T10:02:03.347971+0000","last_fullsized":"2024-04-25T10:02:03.347971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:20.873677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.517442+0000","last_change":"2024-04-25T10:01:57.042210+0000","last_active":"2024-04-25T10:02:03.517442+0000","last_peered":"2024-04-25T10:02:03.517442+0000","last_clean":"2024-04-25T10:02:03.517442+0000","last_became_active":"2024-04-25T10:01:57.041890+0000","last_became_peered":"2024-04-25T10:01:57.041890+0000","last_unstale":"2024-04-25T10:02:03.517442+0000","last_undegraded":"2024-04-25T10:02:03.517442+0000","last_fullsized":"2024-04-25T10:02:03.517442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521682+0000","last_change":"2024-04-25T10:02:02.364078+0000","last_active":"2024-04-25T10:02:03.521682+0000","last_peered":"2024-04-25T10:02:03.521682+0000","last_clean":"2024-04-25T10:02:03.521682+0000","last_became_active":"2024-04-25T10:02:02.358978+0000","last_became_peered":"2024-04-25T10:02:02.358978+0000","last_unstale":"2024-04-25T10:02:03.521682+0000","last_undegraded":"2024-04-25T10:02:03.521682+0000","last_fullsized":"2024-04-25T10:02:03.521682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355780+0000","last_change":"2024-04-25T10:01:58.118220+0000","last_active":"2024-04-25T10:02:03.355780+0000","last_peered":"2024-04-25T10:02:03.355780+0000","last_clean":"2024-04-25T10:02:03.355780+0000","last_became_active":"2024-04-25T10:01:58.117911+0000","last_became_peered":"2024-04-25T10:01:58.117911+0000","last_unstale":"2024-04-25T10:02:03.355780+0000","last_undegraded":"2024-04-25T10:02:03.355780+0000","last_fullsized":"2024-04-25T10:02:03.355780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:27.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405295+0000","last_change":"2024-04-25T10:01:57.064576+0000","last_active":"2024-04-25T10:02:03.405295+0000","last_peered":"2024-04-25T10:02:03.405295+0000","last_clean":"2024-04-25T10:02:03.405295+0000","last_became_active":"2024-04-25T10:01:57.064301+0000","last_became_peered":"2024-04-25T10:01:57.064301+0000","last_unstale":"2024-04-25T10:02:03.405295+0000","last_undegraded":"2024-04-25T10:02:03.405295+0000","last_fullsized":"2024-04-25T10:02:03.405295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.258589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521777+0000","last_change":"2024-04-25T10:02:02.362701+0000","last_active":"2024-04-25T10:02:03.521777+0000","last_peered":"2024-04-25T10:02:03.521777+0000","last_clean":"2024-04-25T10:02:03.521777+0000","last_became_active":"2024-04-25T10:02:02.361742+0000","last_became_peered":"2024-04-25T10:02:02.361742+0000","last_unstale":"2024-04-25T10:02:03.521777+0000","last_undegraded":"2024-04-25T10:02:03.521777+0000","last_fullsized":"2024-04-25T10:02:03.521777+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356323+0000","last_change":"2024-04-25T10:01:58.121271+0000","last_active":"2024-04-25T10:02:03.356323+0000","last_peered":"2024-04-25T10:02:03.356323+0000","last_clean":"2024-04-25T10:02:03.356323+0000","last_became_active":"2024-04-25T10:01:58.121001+0000","last_became_peered":"2024-04-25T10:01:58.121001+0000","last_unstale":"2024-04-25T10:02:03.356323+0000","last_undegraded":"2024-04-25T10:02:03.356323+0000","last_fullsized":"2024-04-25T10:02:03.356323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:56.894928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356241+0000","last_change":"2024-04-25T10:01:57.047184+0000","last_active":"2024-04-25T10:02:03.356241+0000","last_peered":"2024-04-25T10:02:03.356241+0000","last_clean":"2024-04-25T10:02:03.356241+0000","last_became_active":"2024-04-25T10:01:57.046623+0000","last_became_peered":"2024-04-25T10:01:57.046623+0000","last_unstale":"2024-04-25T10:02:03.356241+0000","last_undegraded":"2024-04-25T10:02:03.356241+0000","last_fullsized":"2024-04-25T10:02:03.356241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:31.269009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353577+0000","last_change":"2024-04-25T10:02:02.370654+0000","last_active":"2024-04-25T10:02:03.353577+0000","last_peered":"2024-04-25T10:02:03.353577+0000","last_clean":"2024-04-25T10:02:03.353577+0000","last_became_active":"2024-04-25T10:02:02.369464+0000","last_became_peered":"2024-04-25T10:02:02.369464+0000","last_unstale":"2024-04-25T10:02:03.353577+0000","last_undegraded":"2024-04-25T10:02:03.353577+0000","last_fullsized":"2024-04-25T10:02:03.353577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:47.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519808+0000","last_change":"2024-04-25T10:01:58.123583+0000","last_active":"2024-04-25T10:02:03.519808+0000","last_peered":"2024-04-25T10:02:03.519808+0000","last_clean":"2024-04-25T10:02:03.519808+0000","last_became_active":"2024-04-25T10:01:58.122380+0000","last_became_peered":"2024-04-25T10:01:58.122380+0000","last_unstale":"2024-04-25T10:02:03.519808+0000","last_undegraded":"2024-04-25T10:02:03.519808+0000","last_fullsized":"2024-04-25T10:02:03.519808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:54.007484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405069+0000","last_change":"2024-04-25T10:01:57.060061+0000","last_active":"2024-04-25T10:02:03.405069+0000","last_peered":"2024-04-25T10:02:03.405069+0000","last_clean":"2024-04-25T10:02:03.405069+0000","last_became_active":"2024-04-25T10:01:57.056768+0000","last_became_peered":"2024-04-25T10:01:57.056768+0000","last_unstale":"2024-04-25T10:02:03.405069+0000","last_undegraded":"2024-04-25T10:02:03.405069+0000","last_fullsized":"2024-04-25T10:02:03.405069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.943464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518947+0000","last_change":"2024-04-25T10:02:02.366286+0000","last_active":"2024-04-25T10:02:03.518947+0000","last_peered":"2024-04-25T10:02:03.518947+0000","last_clean":"2024-04-25T10:02:03.518947+0000","last_became_active":"2024-04-25T10:02:02.365877+0000","last_became_peered":"2024-04-25T10:02:02.365877+0000","last_unstale":"2024-04-25T10:02:03.518947+0000","last_undegraded":"2024-04-25T10:02:03.518947+0000","last_fullsized":"2024-04-25T10:02:03.518947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521404+0000","last_change":"2024-04-25T10:01:58.126543+0000","last_active":"2024-04-25T10:02:03.521404+0000","last_peered":"2024-04-25T10:02:03.521404+0000","last_clean":"2024-04-25T10:02:03.521404+0000","last_became_active":"2024-04-25T10:01:58.126245+0000","last_became_peered":"2024-04-25T10:01:58.126245+0000","last_unstale":"2024-04-25T10:02:03.521404+0000","last_undegraded":"2024-04-25T10:02:03.521404+0000","last_fullsized":"2024-04-25T10:02:03.521404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433241+0000","last_change":"2024-04-25T10:01:57.056845+0000","last_active":"2024-04-25T10:02:03.433241+0000","last_peered":"2024-04-25T10:02:03.433241+0000","last_clean":"2024-04-25T10:02:03.433241+0000","last_became_active":"2024-04-25T10:01:57.056550+0000","last_became_peered":"2024-04-25T10:01:57.056550+0000","last_unstale":"2024-04-25T10:02:03.433241+0000","last_undegraded":"2024-04-25T10:02:03.433241+0000","last_fullsized":"2024-04-25T10:02:03.433241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518500+0000","last_change":"2024-04-25T10:02:02.360392+0000","last_active":"2024-04-25T10:02:03.518500+0000","last_peered":"2024-04-25T10:02:03.518500+0000","last_clean":"2024-04-25T10:02:03.518500+0000","last_became_active":"2024-04-25T10:02:02.359550+0000","last_became_peered":"2024-04-25T10:02:02.359550+0000","last_unstale":"2024-04-25T10:02:03.518500+0000","last_undegraded":"2024-04-25T10:02:03.518500+0000","last_fullsized":"2024-04-25T10:02:03.518500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518461+0000","last_change":"2024-04-25T10:01:58.070356+0000","last_active":"2024-04-25T10:02:03.518461+0000","last_peered":"2024-04-25T10:02:03.518461+0000","last_clean":"2024-04-25T10:02:03.518461+0000","last_became_active":"2024-04-25T10:01:58.068594+0000","last_became_peered":"2024-04-25T10:01:58.068594+0000","last_unstale":"2024-04-25T10:02:03.518461+0000","last_undegraded":"2024-04-25T10:02:03.518461+0000","last_fullsized":"2024-04-25T10:02:03.518461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402901+0000","last_change":"2024-04-25T10:01:57.061869+0000","last_active":"2024-04-25T10:02:03.402901+0000","last_peered":"2024-04-25T10:02:03.402901+0000","last_clean":"2024-04-25T10:02:03.402901+0000","last_became_active":"2024-04-25T10:01:57.059517+0000","last_became_peered":"2024-04-25T10:01:57.059517+0000","last_unstale":"2024-04-25T10:02:03.402901+0000","last_undegraded":"2024-04-25T10:02:03.402901+0000","last_fullsized":"2024-04-25T10:02:03.402901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:49.036030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519650+0000","last_change":"2024-04-25T10:02:02.364712+0000","last_active":"2024-04-25T10:02:03.519650+0000","last_peered":"2024-04-25T10:02:03.519650+0000","last_clean":"2024-04-25T10:02:03.519650+0000","last_became_active":"2024-04-25T10:02:02.363483+0000","last_became_peered":"2024-04-25T10:02:02.363483+0000","last_unstale":"2024-04-25T10:02:03.519650+0000","last_undegraded":"2024-04-25T10:02:03.519650+0000","last_fullsized":"2024-04-25T10:02:03.519650+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432126+0000","last_change":"2024-04-25T10:01:58.124129+0000","last_active":"2024-04-25T10:02:03.432126+0000","last_peered":"2024-04-25T10:02:03.432126+0000","last_clean":"2024-04-25T10:02:03.432126+0000","last_became_active":"2024-04-25T10:01:58.123815+0000","last_became_peered":"2024-04-25T10:01:58.123815+0000","last_unstale":"2024-04-25T10:02:03.432126+0000","last_undegraded":"2024-04-25T10:02:03.432126+0000","last_fullsized":"2024-04-25T10:02:03.432126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352979+0000","last_change":"2024-04-25T10:01:57.058063+0000","last_active":"2024-04-25T10:02:03.352979+0000","last_peered":"2024-04-25T10:02:03.352979+0000","last_clean":"2024-04-25T10:02:03.352979+0000","last_became_active":"2024-04-25T10:01:57.057727+0000","last_became_peered":"2024-04-25T10:01:57.057727+0000","last_unstale":"2024-04-25T10:02:03.352979+0000","last_undegraded":"2024-04-25T10:02:03.352979+0000","last_fullsized":"2024-04-25T10:02:03.352979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520229+0000","last_change":"2024-04-25T10:02:02.366966+0000","last_active":"2024-04-25T10:02:03.520229+0000","last_peered":"2024-04-25T10:02:03.520229+0000","last_clean":"2024-04-25T10:02:03.520229+0000","last_became_active":"2024-04-25T10:02:02.366563+0000","last_became_peered":"2024-04-25T10:02:02.366563+0000","last_unstale":"2024-04-25T10:02:03.520229+0000","last_undegraded":"2024-04-25T10:02:03.520229+0000","last_fullsized":"2024-04-25T10:02:03.520229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:50.524934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518446+0000","last_change":"2024-04-25T10:01:58.124316+0000","last_active":"2024-04-25T10:02:03.518446+0000","last_peered":"2024-04-25T10:02:03.518446+0000","last_clean":"2024-04-25T10:02:03.518446+0000","last_became_active":"2024-04-25T10:01:58.123995+0000","last_became_peered":"2024-04-25T10:01:58.123995+0000","last_unstale":"2024-04-25T10:02:03.518446+0000","last_undegraded":"2024-04-25T10:02:03.518446+0000","last_fullsized":"2024-04-25T10:02:03.518446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348251+0000","last_change":"2024-04-25T10:01:57.064900+0000","last_active":"2024-04-25T10:02:03.348251+0000","last_peered":"2024-04-25T10:02:03.348251+0000","last_clean":"2024-04-25T10:02:03.348251+0000","last_became_active":"2024-04-25T10:01:57.063565+0000","last_became_peered":"2024-04-25T10:01:57.063565+0000","last_unstale":"2024-04-25T10:02:03.348251+0000","last_undegraded":"2024-04-25T10:02:03.348251+0000","last_fullsized":"2024-04-25T10:02:03.348251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433722+0000","last_change":"2024-04-25T10:02:02.363528+0000","last_active":"2024-04-25T10:02:03.433722+0000","last_peered":"2024-04-25T10:02:03.433722+0000","last_clean":"2024-04-25T10:02:03.433722+0000","last_became_active":"2024-04-25T10:02:02.360337+0000","last_became_peered":"2024-04-25T10:02:02.360337+0000","last_unstale":"2024-04-25T10:02:03.433722+0000","last_undegraded":"2024-04-25T10:02:03.433722+0000","last_fullsized":"2024-04-25T10:02:03.433722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:25.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354986+0000","last_change":"2024-04-25T10:01:58.114954+0000","last_active":"2024-04-25T10:02:03.354986+0000","last_peered":"2024-04-25T10:02:03.354986+0000","last_clean":"2024-04-25T10:02:03.354986+0000","last_became_active":"2024-04-25T10:01:58.114652+0000","last_became_peered":"2024-04-25T10:01:58.114652+0000","last_unstale":"2024-04-25T10:02:03.354986+0000","last_undegraded":"2024-04-25T10:02:03.354986+0000","last_fullsized":"2024-04-25T10:02:03.354986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.396332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402951+0000","last_change":"2024-04-25T10:01:57.060078+0000","last_active":"2024-04-25T10:02:03.402951+0000","last_peered":"2024-04-25T10:02:03.402951+0000","last_clean":"2024-04-25T10:02:03.402951+0000","last_became_active":"2024-04-25T10:01:57.059811+0000","last_became_peered":"2024-04-25T10:01:57.059811+0000","last_unstale":"2024-04-25T10:02:03.402951+0000","last_undegraded":"2024-04-25T10:02:03.402951+0000","last_fullsized":"2024-04-25T10:02:03.402951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.018629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520801+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:02:03.520801+0000","last_peered":"2024-04-25T10:02:03.520801+0000","last_clean":"2024-04-25T10:02:03.520801+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:02:03.520801+0000","last_undegraded":"2024-04-25T10:02:03.520801+0000","last_fullsized":"2024-04-25T10:02:03.520801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404900+0000","last_change":"2024-04-25T10:02:02.358810+0000","last_active":"2024-04-25T10:02:03.404900+0000","last_peered":"2024-04-25T10:02:03.404900+0000","last_clean":"2024-04-25T10:02:03.404900+0000","last_became_active":"2024-04-25T10:02:02.358270+0000","last_became_peered":"2024-04-25T10:02:02.358270+0000","last_unstale":"2024-04-25T10:02:03.404900+0000","last_undegraded":"2024-04-25T10:02:03.404900+0000","last_fullsized":"2024-04-25T10:02:03.404900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:00.819451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404528+0000","last_change":"2024-04-25T10:01:58.073772+0000","last_active":"2024-04-25T10:02:03.404528+0000","last_peered":"2024-04-25T10:02:03.404528+0000","last_clean":"2024-04-25T10:02:03.404528+0000","last_became_active":"2024-04-25T10:01:58.073104+0000","last_became_peered":"2024-04-25T10:01:58.073104+0000","last_unstale":"2024-04-25T10:02:03.404528+0000","last_undegraded":"2024-04-25T10:02:03.404528+0000","last_fullsized":"2024-04-25T10:02:03.404528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.239771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520009+0000","last_change":"2024-04-25T10:01:57.054884+0000","last_active":"2024-04-25T10:02:03.520009+0000","last_peered":"2024-04-25T10:02:03.520009+0000","last_clean":"2024-04-25T10:02:03.520009+0000","last_became_active":"2024-04-25T10:01:57.054577+0000","last_became_peered":"2024-04-25T10:01:57.054577+0000","last_unstale":"2024-04-25T10:02:03.520009+0000","last_undegraded":"2024-04-25T10:02:03.520009+0000","last_fullsized":"2024-04-25T10:02:03.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521556+0000","last_change":"2024-04-25T10:02:02.364334+0000","last_active":"2024-04-25T10:02:03.521556+0000","last_peered":"2024-04-25T10:02:03.521556+0000","last_clean":"2024-04-25T10:02:03.521556+0000","last_became_active":"2024-04-25T10:02:02.363695+0000","last_became_peered":"2024-04-25T10:02:02.363695+0000","last_unstale":"2024-04-25T10:02:03.521556+0000","last_undegraded":"2024-04-25T10:02:03.521556+0000","last_fullsized":"2024-04-25T10:02:03.521556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:30.903672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519516+0000","last_change":"2024-04-25T10:01:58.090256+0000","last_active":"2024-04-25T10:02:03.519516+0000","last_peered":"2024-04-25T10:02:03.519516+0000","last_clean":"2024-04-25T10:02:03.519516+0000","last_became_active":"2024-04-25T10:01:58.089936+0000","last_became_peered":"2024-04-25T10:01:58.089936+0000","last_unstale":"2024-04-25T10:02:03.519516+0000","last_undegraded":"2024-04-25T10:02:03.519516+0000","last_fullsized":"2024-04-25T10:02:03.519516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433744+0000","last_change":"2024-04-25T10:01:57.060690+0000","last_active":"2024-04-25T10:02:03.433744+0000","last_peered":"2024-04-25T10:02:03.433744+0000","last_clean":"2024-04-25T10:02:03.433744+0000","last_became_active":"2024-04-25T10:01:57.060345+0000","last_became_peered":"2024-04-25T10:01:57.060345+0000","last_unstale":"2024-04-25T10:02:03.433744+0000","last_undegraded":"2024-04-25T10:02:03.433744+0000","last_fullsized":"2024-04-25T10:02:03.433744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433246+0000","last_change":"2024-04-25T10:02:02.371169+0000","last_active":"2024-04-25T10:02:03.433246+0000","last_peered":"2024-04-25T10:02:03.433246+0000","last_clean":"2024-04-25T10:02:03.433246+0000","last_became_active":"2024-04-25T10:02:02.370789+0000","last_became_peered":"2024-04-25T10:02:02.370789+0000","last_unstale":"2024-04-25T10:02:03.433246+0000","last_undegraded":"2024-04-25T10:02:03.433246+0000","last_fullsized":"2024-04-25T10:02:03.433246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:40.538557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520420+0000","last_change":"2024-04-25T10:01:58.072580+0000","last_active":"2024-04-25T10:02:03.520420+0000","last_peered":"2024-04-25T10:02:03.520420+0000","last_clean":"2024-04-25T10:02:03.520420+0000","last_became_active":"2024-04-25T10:01:58.072277+0000","last_became_peered":"2024-04-25T10:01:58.072277+0000","last_unstale":"2024-04-25T10:02:03.520420+0000","last_undegraded":"2024-04-25T10:02:03.520420+0000","last_fullsized":"2024-04-25T10:02:03.520420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356066+0000","last_change":"2024-04-25T10:01:57.057184+0000","last_active":"2024-04-25T10:02:03.356066+0000","last_peered":"2024-04-25T10:02:03.356066+0000","last_clean":"2024-04-25T10:02:03.356066+0000","last_became_active":"2024-04-25T10:01:57.054573+0000","last_became_peered":"2024-04-25T10:01:57.054573+0000","last_unstale":"2024-04-25T10:02:03.356066+0000","last_undegraded":"2024-04-25T10:02:03.356066+0000","last_fullsized":"2024-04-25T10:02:03.356066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:33.311253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347440+0000","last_change":"2024-04-25T10:02:02.373007+0000","last_active":"2024-04-25T10:02:03.347440+0000","last_peered":"2024-04-25T10:02:03.347440+0000","last_clean":"2024-04-25T10:02:03.347440+0000","last_became_active":"2024-04-25T10:02:02.372467+0000","last_became_peered":"2024-04-25T10:02:02.372467+0000","last_unstale":"2024-04-25T10:02:03.347440+0000","last_undegraded":"2024-04-25T10:02:03.347440+0000","last_fullsized":"2024-04-25T10:02:03.347440+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:18.003128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521943+0000","last_change":"2024-04-25T10:01:58.071062+0000","last_active":"2024-04-25T10:02:03.521943+0000","last_peered":"2024-04-25T10:02:03.521943+0000","last_clean":"2024-04-25T10:02:03.521943+0000","last_became_active":"2024-04-25T10:01:58.070721+0000","last_became_peered":"2024-04-25T10:01:58.070721+0000","last_unstale":"2024-04-25T10:02:03.521943+0000","last_undegraded":"2024-04-25T10:02:03.521943+0000","last_fullsized":"2024-04-25T10:02:03.521943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520558+0000","last_change":"2024-04-25T10:01:57.051450+0000","last_active":"2024-04-25T10:02:03.520558+0000","last_peered":"2024-04-25T10:02:03.520558+0000","last_clean":"2024-04-25T10:02:03.520558+0000","last_became_active":"2024-04-25T10:01:57.051104+0000","last_became_peered":"2024-04-25T10:01:57.051104+0000","last_unstale":"2024-04-25T10:02:03.520558+0000","last_undegraded":"2024-04-25T10:02:03.520558+0000","last_fullsized":"2024-04-25T10:02:03.520558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521721+0000","last_change":"2024-04-25T10:02:02.371242+0000","last_active":"2024-04-25T10:02:03.521721+0000","last_peered":"2024-04-25T10:02:03.521721+0000","last_clean":"2024-04-25T10:02:03.521721+0000","last_became_active":"2024-04-25T10:02:02.370975+0000","last_became_peered":"2024-04-25T10:02:02.370975+0000","last_unstale":"2024-04-25T10:02:03.521721+0000","last_undegraded":"2024-04-25T10:02:03.521721+0000","last_fullsized":"2024-04-25T10:02:03.521721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:56.622899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347953+0000","last_change":"2024-04-25T10:01:58.076461+0000","last_active":"2024-04-25T10:02:03.347953+0000","last_peered":"2024-04-25T10:02:03.347953+0000","last_clean":"2024-04-25T10:02:03.347953+0000","last_became_active":"2024-04-25T10:01:58.075822+0000","last_became_peered":"2024-04-25T10:01:58.075822+0000","last_unstale":"2024-04-25T10:02:03.347953+0000","last_undegraded":"2024-04-25T10:02:03.347953+0000","last_fullsized":"2024-04-25T10:02:03.347953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352738+0000","last_change":"2024-04-25T10:01:57.051484+0000","last_active":"2024-04-25T10:02:03.352738+0000","last_peered":"2024-04-25T10:02:03.352738+0000","last_clean":"2024-04-25T10:02:03.352738+0000","last_became_active":"2024-04-25T10:01:57.051183+0000","last_became_peered":"2024-04-25T10:01:57.051183+0000","last_unstale":"2024-04-25T10:02:03.352738+0000","last_undegraded":"2024-04-25T10:02:03.352738+0000","last_fullsized":"2024-04-25T10:02:03.352738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:48.627288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433983+0000","last_change":"2024-04-25T10:02:02.364464+0000","last_active":"2024-04-25T10:02:03.433983+0000","last_peered":"2024-04-25T10:02:03.433983+0000","last_clean":"2024-04-25T10:02:03.433983+0000","last_became_active":"2024-04-25T10:02:02.364024+0000","last_became_peered":"2024-04-25T10:02:02.364024+0000","last_unstale":"2024-04-25T10:02:03.433983+0000","last_undegraded":"2024-04-25T10:02:03.433983+0000","last_fullsized":"2024-04-25T10:02:03.433983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:39.357721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520680+0000","last_change":"2024-04-25T10:01:58.125488+0000","last_active":"2024-04-25T10:02:03.520680+0000","last_peered":"2024-04-25T10:02:03.520680+0000","last_clean":"2024-04-25T10:02:03.520680+0000","last_became_active":"2024-04-25T10:01:58.125161+0000","last_became_peered":"2024-04-25T10:01:58.125161+0000","last_unstale":"2024-04-25T10:02:03.520680+0000","last_undegraded":"2024-04-25T10:02:03.520680+0000","last_fullsized":"2024-04-25T10:02:03.520680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:10.940348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518416+0000","last_change":"2024-04-25T10:01:57.066991+0000","last_active":"2024-04-25T10:02:03.518416+0000","last_peered":"2024-04-25T10:02:03.518416+0000","last_clean":"2024-04-25T10:02:03.518416+0000","last_became_active":"2024-04-25T10:01:57.065456+0000","last_became_peered":"2024-04-25T10:01:57.065456+0000","last_unstale":"2024-04-25T10:02:03.518416+0000","last_undegraded":"2024-04-25T10:02:03.518416+0000","last_fullsized":"2024-04-25T10:02:03.518416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434030+0000","last_change":"2024-04-25T10:02:02.373840+0000","last_active":"2024-04-25T10:02:03.434030+0000","last_peered":"2024-04-25T10:02:03.434030+0000","last_clean":"2024-04-25T10:02:03.434030+0000","last_became_active":"2024-04-25T10:02:02.373426+0000","last_became_peered":"2024-04-25T10:02:02.373426+0000","last_unstale":"2024-04-25T10:02:03.434030+0000","last_undegraded":"2024-04-25T10:02:03.434030+0000","last_fullsized":"2024-04-25T10:02:03.434030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:54.471032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348267+0000","last_change":"2024-04-25T10:01:58.076731+0000","last_active":"2024-04-25T10:02:03.348267+0000","last_peered":"2024-04-25T10:02:03.348267+0000","last_clean":"2024-04-25T10:02:03.348267+0000","last_became_active":"2024-04-25T10:01:58.076430+0000","last_became_peered":"2024-04-25T10:01:58.076430+0000","last_unstale":"2024-04-25T10:02:03.348267+0000","last_undegraded":"2024-04-25T10:02:03.348267+0000","last_fullsized":"2024-04-25T10:02:03.348267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402642+0000","last_change":"2024-04-25T10:01:57.061549+0000","last_active":"2024-04-25T10:02:03.402642+0000","last_peered":"2024-04-25T10:02:03.402642+0000","last_clean":"2024-04-25T10:02:03.402642+0000","last_became_active":"2024-04-25T10:01:57.056797+0000","last_became_peered":"2024-04-25T10:01:57.056797+0000","last_unstale":"2024-04-25T10:02:03.402642+0000","last_undegraded":"2024-04-25T10:02:03.402642+0000","last_fullsized":"2024-04-25T10:02:03.402642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.994799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519398+0000","last_change":"2024-04-25T10:02:02.372845+0000","last_active":"2024-04-25T10:02:03.519398+0000","last_peered":"2024-04-25T10:02:03.519398+0000","last_clean":"2024-04-25T10:02:03.519398+0000","last_became_active":"2024-04-25T10:02:02.370004+0000","last_became_peered":"2024-04-25T10:02:02.370004+0000","last_unstale":"2024-04-25T10:02:03.519398+0000","last_undegraded":"2024-04-25T10:02:03.519398+0000","last_fullsized":"2024-04-25T10:02:03.519398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518640+0000","last_change":"2024-04-25T10:01:58.120136+0000","last_active":"2024-04-25T10:02:03.518640+0000","last_peered":"2024-04-25T10:02:03.518640+0000","last_clean":"2024-04-25T10:02:03.518640+0000","last_became_active":"2024-04-25T10:01:58.111232+0000","last_became_peered":"2024-04-25T10:01:58.111232+0000","last_unstale":"2024-04-25T10:02:03.518640+0000","last_undegraded":"2024-04-25T10:02:03.518640+0000","last_fullsized":"2024-04-25T10:02:03.518640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:17.032865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353998+0000","last_change":"2024-04-25T10:01:57.047615+0000","last_active":"2024-04-25T10:02:03.353998+0000","last_peered":"2024-04-25T10:02:03.353998+0000","last_clean":"2024-04-25T10:02:03.353998+0000","last_became_active":"2024-04-25T10:01:57.047321+0000","last_became_peered":"2024-04-25T10:01:57.047321+0000","last_unstale":"2024-04-25T10:02:03.353998+0000","last_undegraded":"2024-04-25T10:02:03.353998+0000","last_fullsized":"2024-04-25T10:02:03.353998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404368+0000","last_change":"2024-04-25T10:02:02.361172+0000","last_active":"2024-04-25T10:02:03.404368+0000","last_peered":"2024-04-25T10:02:03.404368+0000","last_clean":"2024-04-25T10:02:03.404368+0000","last_became_active":"2024-04-25T10:02:02.360349+0000","last_became_peered":"2024-04-25T10:02:02.360349+0000","last_unstale":"2024-04-25T10:02:03.404368+0000","last_undegraded":"2024-04-25T10:02:03.404368+0000","last_fullsized":"2024-04-25T10:02:03.404368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.788023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518563+0000","last_change":"2024-04-25T10:01:58.127068+0000","last_active":"2024-04-25T10:02:03.518563+0000","last_peered":"2024-04-25T10:02:03.518563+0000","last_clean":"2024-04-25T10:02:03.518563+0000","last_became_active":"2024-04-25T10:01:58.126258+0000","last_became_peered":"2024-04-25T10:01:58.126258+0000","last_unstale":"2024-04-25T10:02:03.518563+0000","last_undegraded":"2024-04-25T10:02:03.518563+0000","last_fullsized":"2024-04-25T10:02:03.518563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:15.775667+0000","last_change":"2024-04-25T10:01:57.054259+0000","last_active":"2024-04-25T10:02:15.775667+0000","last_peered":"2024-04-25T10:02:15.775667+0000","last_clean":"2024-04-25T10:02:15.775667+0000","last_became_active":"2024-04-25T10:01:57.053941+0000","last_became_peered":"2024-04-25T10:01:57.053941+0000","last_unstale":"2024-04-25T10:02:15.775667+0000","last_undegraded":"2024-04-25T10:02:15.775667+0000","last_fullsized":"2024-04-25T10:02:15.775667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:41.210591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354900+0000","last_change":"2024-04-25T10:02:02.376549+0000","last_active":"2024-04-25T10:02:03.354900+0000","last_peered":"2024-04-25T10:02:03.354900+0000","last_clean":"2024-04-25T10:02:03.354900+0000","last_became_active":"2024-04-25T10:02:02.376052+0000","last_became_peered":"2024-04-25T10:02:02.376052+0000","last_unstale":"2024-04-25T10:02:03.354900+0000","last_undegraded":"2024-04-25T10:02:03.354900+0000","last_fullsized":"2024-04-25T10:02:03.354900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:43.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.403613+0000","last_change":"2024-04-25T10:01:58.078715+0000","last_active":"2024-04-25T10:02:03.403613+0000","last_peered":"2024-04-25T10:02:03.403613+0000","last_clean":"2024-04-25T10:02:03.403613+0000","last_became_active":"2024-04-25T10:01:58.078363+0000","last_became_peered":"2024-04-25T10:01:58.078363+0000","last_unstale":"2024-04-25T10:02:03.403613+0000","last_undegraded":"2024-04-25T10:02:03.403613+0000","last_fullsized":"2024-04-25T10:02:03.403613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:25.310967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354395+0000","last_change":"2024-04-25T10:01:57.054914+0000","last_active":"2024-04-25T10:02:03.354395+0000","last_peered":"2024-04-25T10:02:03.354395+0000","last_clean":"2024-04-25T10:02:03.354395+0000","last_became_active":"2024-04-25T10:01:57.054640+0000","last_became_peered":"2024-04-25T10:01:57.054640+0000","last_unstale":"2024-04-25T10:02:03.354395+0000","last_undegraded":"2024-04-25T10:02:03.354395+0000","last_fullsized":"2024-04-25T10:02:03.354395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522376+0000","last_change":"2024-04-25T10:02:02.362949+0000","last_active":"2024-04-25T10:02:03.522376+0000","last_peered":"2024-04-25T10:02:03.522376+0000","last_clean":"2024-04-25T10:02:03.522376+0000","last_became_active":"2024-04-25T10:02:02.361660+0000","last_became_peered":"2024-04-25T10:02:02.361660+0000","last_unstale":"2024-04-25T10:02:03.522376+0000","last_undegraded":"2024-04-25T10:02:03.522376+0000","last_fullsized":"2024-04-25T10:02:03.522376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434111+0000","last_change":"2024-04-25T10:01:58.114459+0000","last_active":"2024-04-25T10:02:03.434111+0000","last_peered":"2024-04-25T10:02:03.434111+0000","last_clean":"2024-04-25T10:02:03.434111+0000","last_became_active":"2024-04-25T10:01:58.113079+0000","last_became_peered":"2024-04-25T10:01:58.113079+0000","last_unstale":"2024-04-25T10:02:03.434111+0000","last_undegraded":"2024-04-25T10:02:03.434111+0000","last_fullsized":"2024-04-25T10:02:03.434111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518456+0000","last_change":"2024-04-25T10:01:57.058749+0000","last_active":"2024-04-25T10:02:03.518456+0000","last_peered":"2024-04-25T10:02:03.518456+0000","last_clean":"2024-04-25T10:02:03.518456+0000","last_became_active":"2024-04-25T10:01:57.058498+0000","last_became_peered":"2024-04-25T10:01:57.058498+0000","last_unstale":"2024-04-25T10:02:03.518456+0000","last_undegraded":"2024-04-25T10:02:03.518456+0000","last_fullsized":"2024-04-25T10:02:03.518456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352717+0000","last_change":"2024-04-25T10:02:02.354682+0000","last_active":"2024-04-25T10:02:03.352717+0000","last_peered":"2024-04-25T10:02:03.352717+0000","last_clean":"2024-04-25T10:02:03.352717+0000","last_became_active":"2024-04-25T10:02:02.353840+0000","last_became_peered":"2024-04-25T10:02:02.353840+0000","last_unstale":"2024-04-25T10:02:03.352717+0000","last_undegraded":"2024-04-25T10:02:03.352717+0000","last_fullsized":"2024-04-25T10:02:03.352717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.306270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521836+0000","last_change":"2024-04-25T10:01:58.075136+0000","last_active":"2024-04-25T10:02:03.521836+0000","last_peered":"2024-04-25T10:02:03.521836+0000","last_clean":"2024-04-25T10:02:03.521836+0000","last_became_active":"2024-04-25T10:01:58.074823+0000","last_became_peered":"2024-04-25T10:01:58.074823+0000","last_unstale":"2024-04-25T10:02:03.521836+0000","last_undegraded":"2024-04-25T10:02:03.521836+0000","last_fullsized":"2024-04-25T10:02:03.521836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524041+0000","last_change":"2024-04-25T10:01:57.058387+0000","last_active":"2024-04-25T10:02:03.524041+0000","last_peered":"2024-04-25T10:02:03.524041+0000","last_clean":"2024-04-25T10:02:03.524041+0000","last_became_active":"2024-04-25T10:01:57.057994+0000","last_became_peered":"2024-04-25T10:01:57.057994+0000","last_unstale":"2024-04-25T10:02:03.524041+0000","last_undegraded":"2024-04-25T10:02:03.524041+0000","last_fullsized":"2024-04-25T10:02:03.524041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.171109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353666+0000","last_change":"2024-04-25T10:02:02.374647+0000","last_active":"2024-04-25T10:02:03.353666+0000","last_peered":"2024-04-25T10:02:03.353666+0000","last_clean":"2024-04-25T10:02:03.353666+0000","last_became_active":"2024-04-25T10:02:02.374168+0000","last_became_peered":"2024-04-25T10:02:02.374168+0000","last_unstale":"2024-04-25T10:02:03.353666+0000","last_undegraded":"2024-04-25T10:02:03.353666+0000","last_fullsized":"2024-04-25T10:02:03.353666+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:13.963774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356050+0000","last_change":"2024-04-25T10:02:02.374581+0000","last_active":"2024-04-25T10:02:03.356050+0000","last_peered":"2024-04-25T10:02:03.356050+0000","last_clean":"2024-04-25T10:02:03.356050+0000","last_became_active":"2024-04-25T10:02:02.373942+0000","last_became_peered":"2024-04-25T10:02:02.373942+0000","last_unstale":"2024-04-25T10:02:03.356050+0000","last_undegraded":"2024-04-25T10:02:03.356050+0000","last_fullsized":"2024-04-25T10:02:03.356050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.252346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353803+0000","last_change":"2024-04-25T10:01:58.111477+0000","last_active":"2024-04-25T10:02:03.353803+0000","last_peered":"2024-04-25T10:02:03.353803+0000","last_clean":"2024-04-25T10:02:03.353803+0000","last_became_active":"2024-04-25T10:01:58.111095+0000","last_became_peered":"2024-04-25T10:01:58.111095+0000","last_unstale":"2024-04-25T10:02:03.353803+0000","last_undegraded":"2024-04-25T10:02:03.353803+0000","last_fullsized":"2024-04-25T10:02:03.353803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:42.787884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347909+0000","last_change":"2024-04-25T10:01:57.058954+0000","last_active":"2024-04-25T10:02:03.347909+0000","last_peered":"2024-04-25T10:02:03.347909+0000","last_clean":"2024-04-25T10:02:03.347909+0000","last_became_active":"2024-04-25T10:01:57.056146+0000","last_became_peered":"2024-04-25T10:01:57.056146+0000","last_unstale":"2024-04-25T10:02:03.347909+0000","last_undegraded":"2024-04-25T10:02:03.347909+0000","last_fullsized":"2024-04-25T10:02:03.347909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404389+0000","last_change":"2024-04-25T10:02:02.360724+0000","last_active":"2024-04-25T10:02:03.404389+0000","last_peered":"2024-04-25T10:02:03.404389+0000","last_clean":"2024-04-25T10:02:03.404389+0000","last_became_active":"2024-04-25T10:02:02.360282+0000","last_became_peered":"2024-04-25T10:02:02.360282+0000","last_unstale":"2024-04-25T10:02:03.404389+0000","last_undegraded":"2024-04-25T10:02:03.404389+0000","last_fullsized":"2024-04-25T10:02:03.404389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:48.141173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432340+0000","last_change":"2024-04-25T10:01:58.126960+0000","last_active":"2024-04-25T10:02:03.432340+0000","last_peered":"2024-04-25T10:02:03.432340+0000","last_clean":"2024-04-25T10:02:03.432340+0000","last_became_active":"2024-04-25T10:01:58.126638+0000","last_became_peered":"2024-04-25T10:01:58.126638+0000","last_unstale":"2024-04-25T10:02:03.432340+0000","last_undegraded":"2024-04-25T10:02:03.432340+0000","last_fullsized":"2024-04-25T10:02:03.432340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518686+0000","last_change":"2024-04-25T10:01:57.056000+0000","last_active":"2024-04-25T10:02:03.518686+0000","last_peered":"2024-04-25T10:02:03.518686+0000","last_clean":"2024-04-25T10:02:03.518686+0000","last_became_active":"2024-04-25T10:01:57.055665+0000","last_became_peered":"2024-04-25T10:01:57.055665+0000","last_unstale":"2024-04-25T10:02:03.518686+0000","last_undegraded":"2024-04-25T10:02:03.518686+0000","last_fullsized":"2024-04-25T10:02:03.518686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402377+0000","last_change":"2024-04-25T10:01:57.064311+0000","last_active":"2024-04-25T10:02:03.402377+0000","last_peered":"2024-04-25T10:02:03.402377+0000","last_clean":"2024-04-25T10:02:03.402377+0000","last_became_active":"2024-04-25T10:01:57.063955+0000","last_became_peered":"2024-04-25T10:01:57.063955+0000","last_unstale":"2024-04-25T10:02:03.402377+0000","last_undegraded":"2024-04-25T10:02:03.402377+0000","last_fullsized":"2024-04-25T10:02:03.402377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353034+0000","last_change":"2024-04-25T10:01:58.112383+0000","last_active":"2024-04-25T10:02:03.353034+0000","last_peered":"2024-04-25T10:02:03.353034+0000","last_clean":"2024-04-25T10:02:03.353034+0000","last_became_active":"2024-04-25T10:01:58.111906+0000","last_became_peered":"2024-04-25T10:01:58.111906+0000","last_unstale":"2024-04-25T10:02:03.353034+0000","last_undegraded":"2024-04-25T10:02:03.353034+0000","last_fullsized":"2024-04-25T10:02:03.353034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:58.408032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355526+0000","last_change":"2024-04-25T10:02:02.370310+0000","last_active":"2024-04-25T10:02:03.355526+0000","last_peered":"2024-04-25T10:02:03.355526+0000","last_clean":"2024-04-25T10:02:03.355526+0000","last_became_active":"2024-04-25T10:02:02.369660+0000","last_became_peered":"2024-04-25T10:02:02.369660+0000","last_unstale":"2024-04-25T10:02:03.355526+0000","last_undegraded":"2024-04-25T10:02:03.355526+0000","last_fullsized":"2024-04-25T10:02:03.355526+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:59.232453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433292+0000","last_change":"2024-04-25T10:01:57.059797+0000","last_active":"2024-04-25T10:02:03.433292+0000","last_peered":"2024-04-25T10:02:03.433292+0000","last_clean":"2024-04-25T10:02:03.433292+0000","last_became_active":"2024-04-25T10:01:57.059483+0000","last_became_peered":"2024-04-25T10:01:57.059483+0000","last_unstale":"2024-04-25T10:02:03.433292+0000","last_undegraded":"2024-04-25T10:02:03.433292+0000","last_fullsized":"2024-04-25T10:02:03.433292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521515+0000","last_change":"2024-04-25T10:01:58.118406+0000","last_active":"2024-04-25T10:02:03.521515+0000","last_peered":"2024-04-25T10:02:03.521515+0000","last_clean":"2024-04-25T10:02:03.521515+0000","last_became_active":"2024-04-25T10:01:58.118066+0000","last_became_peered":"2024-04-25T10:01:58.118066+0000","last_unstale":"2024-04-25T10:02:03.521515+0000","last_undegraded":"2024-04-25T10:02:03.521515+0000","last_fullsized":"2024-04-25T10:02:03.521515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356020+0000","last_change":"2024-04-25T10:02:02.360301+0000","last_active":"2024-04-25T10:02:03.356020+0000","last_peered":"2024-04-25T10:02:03.356020+0000","last_clean":"2024-04-25T10:02:03.356020+0000","last_became_active":"2024-04-25T10:02:02.359873+0000","last_became_peered":"2024-04-25T10:02:02.359873+0000","last_unstale":"2024-04-25T10:02:03.356020+0000","last_undegraded":"2024-04-25T10:02:03.356020+0000","last_fullsized":"2024-04-25T10:02:03.356020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:58.571611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519243+0000","last_change":"2024-04-25T10:01:58.119106+0000","last_active":"2024-04-25T10:02:03.519243+0000","last_peered":"2024-04-25T10:02:03.519243+0000","last_clean":"2024-04-25T10:02:03.519243+0000","last_became_active":"2024-04-25T10:01:58.118822+0000","last_became_peered":"2024-04-25T10:01:58.118822+0000","last_unstale":"2024-04-25T10:02:03.519243+0000","last_undegraded":"2024-04-25T10:02:03.519243+0000","last_fullsized":"2024-04-25T10:02:03.519243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352667+0000","last_change":"2024-04-25T10:01:58.080145+0000","last_active":"2024-04-25T10:02:03.352667+0000","last_peered":"2024-04-25T10:02:03.352667+0000","last_clean":"2024-04-25T10:02:03.352667+0000","last_became_active":"2024-04-25T10:01:58.079388+0000","last_became_peered":"2024-04-25T10:01:58.079388+0000","last_unstale":"2024-04-25T10:02:03.352667+0000","last_undegraded":"2024-04-25T10:02:03.352667+0000","last_fullsized":"2024-04-25T10:02:03.352667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347995+0000","last_change":"2024-04-25T10:01:58.118742+0000","last_active":"2024-04-25T10:02:03.347995+0000","last_peered":"2024-04-25T10:02:03.347995+0000","last_clean":"2024-04-25T10:02:03.347995+0000","last_became_active":"2024-04-25T10:01:58.118407+0000","last_became_peered":"2024-04-25T10:01:58.118407+0000","last_unstale":"2024-04-25T10:02:03.347995+0000","last_undegraded":"2024-04-25T10:02:03.347995+0000","last_fullsized":"2024-04-25T10:02:03.347995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.311295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432945+0000","last_change":"2024-04-25T10:01:58.129321+0000","last_active":"2024-04-25T10:02:03.432945+0000","last_peered":"2024-04-25T10:02:03.432945+0000","last_clean":"2024-04-25T10:02:03.432945+0000","last_became_active":"2024-04-25T10:01:58.128642+0000","last_became_peered":"2024-04-25T10:01:58.128642+0000","last_unstale":"2024-04-25T10:02:03.432945+0000","last_undegraded":"2024-04-25T10:02:03.432945+0000","last_fullsized":"2024-04-25T10:02:03.432945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519722+0000","last_change":"2024-04-25T10:01:58.114750+0000","last_active":"2024-04-25T10:02:03.519722+0000","last_peered":"2024-04-25T10:02:03.519722+0000","last_clean":"2024-04-25T10:02:03.519722+0000","last_became_active":"2024-04-25T10:01:58.114410+0000","last_became_peered":"2024-04-25T10:01:58.114410+0000","last_unstale":"2024-04-25T10:02:03.519722+0000","last_undegraded":"2024-04-25T10:02:03.519722+0000","last_fullsized":"2024-04-25T10:02:03.519722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:27.145938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519712+0000","last_change":"2024-04-25T10:01:58.116458+0000","last_active":"2024-04-25T10:02:03.519712+0000","last_peered":"2024-04-25T10:02:03.519712+0000","last_clean":"2024-04-25T10:02:03.519712+0000","last_became_active":"2024-04-25T10:01:58.116051+0000","last_became_peered":"2024-04-25T10:01:58.116051+0000","last_unstale":"2024-04-25T10:02:03.519712+0000","last_undegraded":"2024-04-25T10:02:03.519712+0000","last_fullsized":"2024-04-25T10:02:03.519712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521818+0000","last_change":"2024-04-25T10:01:58.079855+0000","last_active":"2024-04-25T10:02:03.521818+0000","last_peered":"2024-04-25T10:02:03.521818+0000","last_clean":"2024-04-25T10:02:03.521818+0000","last_became_active":"2024-04-25T10:01:58.079522+0000","last_became_peered":"2024-04-25T10:01:58.079522+0000","last_unstale":"2024-04-25T10:02:03.521818+0000","last_undegraded":"2024-04-25T10:02:03.521818+0000","last_fullsized":"2024-04-25T10:02:03.521818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433705+0000","last_change":"2024-04-25T10:01:58.114808+0000","last_active":"2024-04-25T10:02:03.433705+0000","last_peered":"2024-04-25T10:02:03.433705+0000","last_clean":"2024-04-25T10:02:03.433705+0000","last_became_active":"2024-04-25T10:01:58.114656+0000","last_became_peered":"2024-04-25T10:01:58.114656+0000","last_unstale":"2024-04-25T10:02:03.433705+0000","last_undegraded":"2024-04-25T10:02:03.433705+0000","last_fullsized":"2024-04-25T10:02:03.433705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347899+0000","last_change":"2024-04-25T10:01:58.125563+0000","last_active":"2024-04-25T10:02:03.347899+0000","last_peered":"2024-04-25T10:02:03.347899+0000","last_clean":"2024-04-25T10:02:03.347899+0000","last_became_active":"2024-04-25T10:01:58.125071+0000","last_became_peered":"2024-04-25T10:01:58.125071+0000","last_unstale":"2024-04-25T10:02:03.347899+0000","last_undegraded":"2024-04-25T10:02:03.347899+0000","last_fullsized":"2024-04-25T10:02:03.347899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432367+0000","last_change":"2024-04-25T10:01:58.111903+0000","last_active":"2024-04-25T10:02:03.432367+0000","last_peered":"2024-04-25T10:02:03.432367+0000","last_clean":"2024-04-25T10:02:03.432367+0000","last_became_active":"2024-04-25T10:01:58.111366+0000","last_became_peered":"2024-04-25T10:01:58.111366+0000","last_unstale":"2024-04-25T10:02:03.432367+0000","last_undegraded":"2024-04-25T10:02:03.432367+0000","last_fullsized":"2024-04-25T10:02:03.432367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521208+0000","last_change":"2024-04-25T10:01:58.066982+0000","last_active":"2024-04-25T10:02:03.521208+0000","last_peered":"2024-04-25T10:02:03.521208+0000","last_clean":"2024-04-25T10:02:03.521208+0000","last_became_active":"2024-04-25T10:01:58.066582+0000","last_became_peered":"2024-04-25T10:01:58.066582+0000","last_unstale":"2024-04-25T10:02:03.521208+0000","last_undegraded":"2024-04-25T10:02:03.521208+0000","last_fullsized":"2024-04-25T10:02:03.521208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352779+0000","last_change":"2024-04-25T10:01:58.112990+0000","last_active":"2024-04-25T10:02:03.352779+0000","last_peered":"2024-04-25T10:02:03.352779+0000","last_clean":"2024-04-25T10:02:03.352779+0000","last_became_active":"2024-04-25T10:01:58.112724+0000","last_became_peered":"2024-04-25T10:01:58.112724+0000","last_unstale":"2024-04-25T10:02:03.352779+0000","last_undegraded":"2024-04-25T10:02:03.352779+0000","last_fullsized":"2024-04-25T10:02:03.352779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:36.503846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519412+0000","last_change":"2024-04-25T10:01:58.067953+0000","last_active":"2024-04-25T10:02:03.519412+0000","last_peered":"2024-04-25T10:02:03.519412+0000","last_clean":"2024-04-25T10:02:03.519412+0000","last_became_active":"2024-04-25T10:01:58.067363+0000","last_became_peered":"2024-04-25T10:01:58.067363+0000","last_unstale":"2024-04-25T10:02:03.519412+0000","last_undegraded":"2024-04-25T10:02:03.519412+0000","last_fullsized":"2024-04-25T10:02:03.519412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404814+0000","last_change":"2024-04-25T10:01:58.112476+0000","last_active":"2024-04-25T10:02:03.404814+0000","last_peered":"2024-04-25T10:02:03.404814+0000","last_clean":"2024-04-25T10:02:03.404814+0000","last_became_active":"2024-04-25T10:01:58.112177+0000","last_became_peered":"2024-04-25T10:01:58.112177+0000","last_unstale":"2024-04-25T10:02:03.404814+0000","last_undegraded":"2024-04-25T10:02:03.404814+0000","last_fullsized":"2024-04-25T10:02:03.404814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:50.172245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355444+0000","last_change":"2024-04-25T10:01:58.056526+0000","last_active":"2024-04-25T10:02:03.355444+0000","last_peered":"2024-04-25T10:02:03.355444+0000","last_clean":"2024-04-25T10:02:03.355444+0000","last_became_active":"2024-04-25T10:01:58.055509+0000","last_became_peered":"2024-04-25T10:01:58.055509+0000","last_unstale":"2024-04-25T10:02:03.355444+0000","last_undegraded":"2024-04-25T10:02:03.355444+0000","last_fullsized":"2024-04-25T10:02:03.355444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405090+0000","last_change":"2024-04-25T10:01:58.079032+0000","last_active":"2024-04-25T10:02:03.405090+0000","last_peered":"2024-04-25T10:02:03.405090+0000","last_clean":"2024-04-25T10:02:03.405090+0000","last_became_active":"2024-04-25T10:01:58.077635+0000","last_became_peered":"2024-04-25T10:01:58.077635+0000","last_unstale":"2024-04-25T10:02:03.405090+0000","last_undegraded":"2024-04-25T10:02:03.405090+0000","last_fullsized":"2024-04-25T10:02:03.405090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:00.440369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347561+0000","last_change":"2024-04-25T10:01:58.076733+0000","last_active":"2024-04-25T10:02:03.347561+0000","last_peered":"2024-04-25T10:02:03.347561+0000","last_clean":"2024-04-25T10:02:03.347561+0000","last_became_active":"2024-04-25T10:01:58.076476+0000","last_became_peered":"2024-04-25T10:01:58.076476+0000","last_unstale":"2024-04-25T10:02:03.347561+0000","last_undegraded":"2024-04-25T10:02:03.347561+0000","last_fullsized":"2024-04-25T10:02:03.347561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432026+0000","last_change":"2024-04-25T10:01:58.116156+0000","last_active":"2024-04-25T10:02:03.432026+0000","last_peered":"2024-04-25T10:02:03.432026+0000","last_clean":"2024-04-25T10:02:03.432026+0000","last_became_active":"2024-04-25T10:01:58.115483+0000","last_became_peered":"2024-04-25T10:01:58.115483+0000","last_unstale":"2024-04-25T10:02:03.432026+0000","last_undegraded":"2024-04-25T10:02:03.432026+0000","last_fullsized":"2024-04-25T10:02:03.432026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518554+0000","last_change":"2024-04-25T10:01:58.120123+0000","last_active":"2024-04-25T10:02:03.518554+0000","last_peered":"2024-04-25T10:02:03.518554+0000","last_clean":"2024-04-25T10:02:03.518554+0000","last_became_active":"2024-04-25T10:01:58.119852+0000","last_became_peered":"2024-04-25T10:01:58.119852+0000","last_unstale":"2024-04-25T10:02:03.518554+0000","last_undegraded":"2024-04-25T10:02:03.518554+0000","last_fullsized":"2024-04-25T10:02:03.518554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519056+0000","last_change":"2024-04-25T10:01:58.113249+0000","last_active":"2024-04-25T10:02:03.519056+0000","last_peered":"2024-04-25T10:02:03.519056+0000","last_clean":"2024-04-25T10:02:03.519056+0000","last_became_active":"2024-04-25T10:01:58.112298+0000","last_became_peered":"2024-04-25T10:01:58.112298+0000","last_unstale":"2024-04-25T10:02:03.519056+0000","last_undegraded":"2024-04-25T10:02:03.519056+0000","last_fullsized":"2024-04-25T10:02:03.519056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519272+0000","last_change":"2024-04-25T10:01:58.072097+0000","last_active":"2024-04-25T10:02:03.519272+0000","last_peered":"2024-04-25T10:02:03.519272+0000","last_clean":"2024-04-25T10:02:03.519272+0000","last_became_active":"2024-04-25T10:01:58.071662+0000","last_became_peered":"2024-04-25T10:01:58.071662+0000","last_unstale":"2024-04-25T10:02:03.519272+0000","last_undegraded":"2024-04-25T10:02:03.519272+0000","last_fullsized":"2024-04-25T10:02:03.519272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518405+0000","last_change":"2024-04-25T10:01:58.073877+0000","last_active":"2024-04-25T10:02:03.518405+0000","last_peered":"2024-04-25T10:02:03.518405+0000","last_clean":"2024-04-25T10:02:03.518405+0000","last_became_active":"2024-04-25T10:01:58.073554+0000","last_became_peered":"2024-04-25T10:01:58.073554+0000","last_unstale":"2024-04-25T10:02:03.518405+0000","last_undegraded":"2024-04-25T10:02:03.518405+0000","last_fullsized":"2024-04-25T10:02:03.518405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518311+0000","last_change":"2024-04-25T10:01:58.066768+0000","last_active":"2024-04-25T10:02:03.518311+0000","last_peered":"2024-04-25T10:02:03.518311+0000","last_clean":"2024-04-25T10:02:03.518311+0000","last_became_active":"2024-04-25T10:01:58.065944+0000","last_became_peered":"2024-04-25T10:01:58.065944+0000","last_unstale":"2024-04-25T10:02:03.518311+0000","last_undegraded":"2024-04-25T10:02:03.518311+0000","last_fullsized":"2024-04-25T10:02:03.518311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524004+0000","last_change":"2024-04-25T10:01:58.066685+0000","last_active":"2024-04-25T10:02:03.524004+0000","last_peered":"2024-04-25T10:02:03.524004+0000","last_clean":"2024-04-25T10:02:03.524004+0000","last_became_active":"2024-04-25T10:01:58.066367+0000","last_became_peered":"2024-04-25T10:01:58.066367+0000","last_unstale":"2024-04-25T10:02:03.524004+0000","last_undegraded":"2024-04-25T10:02:03.524004+0000","last_fullsized":"2024-04-25T10:02:03.524004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432058+0000","last_change":"2024-04-25T10:01:58.123950+0000","last_active":"2024-04-25T10:02:03.432058+0000","last_peered":"2024-04-25T10:02:03.432058+0000","last_clean":"2024-04-25T10:02:03.432058+0000","last_became_active":"2024-04-25T10:01:58.123668+0000","last_became_peered":"2024-04-25T10:01:58.123668+0000","last_unstale":"2024-04-25T10:02:03.432058+0000","last_undegraded":"2024-04-25T10:02:03.432058+0000","last_fullsized":"2024-04-25T10:02:03.432058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:30.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518202+0000","last_change":"2024-04-25T10:01:58.114958+0000","last_active":"2024-04-25T10:02:03.518202+0000","last_peered":"2024-04-25T10:02:03.518202+0000","last_clean":"2024-04-25T10:02:03.518202+0000","last_became_active":"2024-04-25T10:01:58.114599+0000","last_became_peered":"2024-04-25T10:01:58.114599+0000","last_unstale":"2024-04-25T10:02:03.518202+0000","last_undegraded":"2024-04-25T10:02:03.518202+0000","last_fullsized":"2024-04-25T10:02:03.518202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521335+0000","last_change":"2024-04-25T10:01:58.116405+0000","last_active":"2024-04-25T10:02:03.521335+0000","last_peered":"2024-04-25T10:02:03.521335+0000","last_clean":"2024-04-25T10:02:03.521335+0000","last_became_active":"2024-04-25T10:01:58.115946+0000","last_became_peered":"2024-04-25T10:01:58.115946+0000","last_unstale":"2024-04-25T10:02:03.521335+0000","last_undegraded":"2024-04-25T10:02:03.521335+0000","last_fullsized":"2024-04-25T10:02:03.521335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:57.669295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518323+0000","last_change":"2024-04-25T10:01:58.117287+0000","last_active":"2024-04-25T10:02:03.518323+0000","last_peered":"2024-04-25T10:02:03.518323+0000","last_clean":"2024-04-25T10:02:03.518323+0000","last_became_active":"2024-04-25T10:01:58.117024+0000","last_became_peered":"2024-04-25T10:01:58.117024+0000","last_unstale":"2024-04-25T10:02:03.518323+0000","last_undegraded":"2024-04-25T10:02:03.518323+0000","last_fullsized":"2024-04-25T10:02:03.518323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522601+0000","last_change":"2024-04-25T10:01:58.116367+0000","last_active":"2024-04-25T10:02:03.522601+0000","last_peered":"2024-04-25T10:02:03.522601+0000","last_clean":"2024-04-25T10:02:03.522601+0000","last_became_active":"2024-04-25T10:01:58.115801+0000","last_became_peered":"2024-04-25T10:01:58.115801+0000","last_unstale":"2024-04-25T10:02:03.522601+0000","last_undegraded":"2024-04-25T10:02:03.522601+0000","last_fullsized":"2024-04-25T10:02:03.522601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433436+0000","last_change":"2024-04-25T10:01:58.118411+0000","last_active":"2024-04-25T10:02:03.433436+0000","last_peered":"2024-04-25T10:02:03.433436+0000","last_clean":"2024-04-25T10:02:03.433436+0000","last_became_active":"2024-04-25T10:01:58.118131+0000","last_became_peered":"2024-04-25T10:01:58.118131+0000","last_unstale":"2024-04-25T10:02:03.433436+0000","last_undegraded":"2024-04-25T10:02:03.433436+0000","last_fullsized":"2024-04-25T10:02:03.433436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433580+0000","last_change":"2024-04-25T10:01:58.121692+0000","last_active":"2024-04-25T10:02:03.433580+0000","last_peered":"2024-04-25T10:02:03.433580+0000","last_clean":"2024-04-25T10:02:03.433580+0000","last_became_active":"2024-04-25T10:01:58.120392+0000","last_became_peered":"2024-04-25T10:01:58.120392+0000","last_unstale":"2024-04-25T10:02:03.433580+0000","last_undegraded":"2024-04-25T10:02:03.433580+0000","last_fullsized":"2024-04-25T10:02:03.433580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520287+0000","last_change":"2024-04-25T10:01:58.080365+0000","last_active":"2024-04-25T10:02:03.520287+0000","last_peered":"2024-04-25T10:02:03.520287+0000","last_clean":"2024-04-25T10:02:03.520287+0000","last_became_active":"2024-04-25T10:01:58.080055+0000","last_became_peered":"2024-04-25T10:01:58.080055+0000","last_unstale":"2024-04-25T10:02:03.520287+0000","last_undegraded":"2024-04-25T10:02:03.520287+0000","last_fullsized":"2024-04-25T10:02:03.520287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:25.923 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.171+0000 7f89ce200640 1 Processor -- start 2024-04-25T10:02:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.172+0000 7f89ce200640 1 -- start start 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.173+0000 7f89ce200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89c815d8a0 0x7f89c80d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.173+0000 7f89ce200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89c80b2f30 0x7f89c80d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.173+0000 7f89ce200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c815f410 con 0x7f89c81614e0 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.173+0000 7f89ce200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c815ffe0 con 0x7f89c80b2f30 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.173+0000 7f89ce200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c815f6b0 con 0x7f89c815d8a0 2024-04-25T10:02:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89c80b2f30 0x7f89c80d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89c80b2f30 0x7f89c80d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46780/0 (socket says 172.21.15.5:46780) 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 -- 172.21.15.5:0/3920123328 learned_addr learned my addr 172.21.15.5:0/3920123328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89cce00640 1 --2- 172.21.15.5:0/3920123328 >> v2:172.21.15.80:3300/0 conn(0x7f89c815d8a0 0x7f89c80d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 -- 172.21.15.5:0/3920123328 >> v2:172.21.15.80:3300/0 conn(0x7f89c815d8a0 msgr2=0x7f89c80d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 --2- 172.21.15.5:0/3920123328 >> v2:172.21.15.80:3300/0 conn(0x7f89c815d8a0 0x7f89c80d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 -- 172.21.15.5:0/3920123328 >> v1:172.21.15.80:6789/0 conn(0x7f89c81614e0 legacy=0x7f89c80b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.174+0000 7f89c7e00640 1 -- 172.21.15.5:0/3920123328 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c815f990 con 0x7f89c80b2f30 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.175+0000 7f89c7e00640 1 --2- 172.21.15.5:0/3920123328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c80d3f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f89c80d0c60 tx=0x7f89b002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbf6a7918d16226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.175+0000 7f89c7400640 1 -- 172.21.15.5:0/3920123328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0037070 con 0x7f89c80b2f30 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.175+0000 7f89c7400640 1 -- 172.21.15.5:0/3920123328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b002ae00 con 0x7f89c80b2f30 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.175+0000 7f89c7400640 1 -- 172.21.15.5:0/3920123328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0032660 con 0x7f89c80b2f30 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 -- 172.21.15.5:0/3920123328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 msgr2=0x7f89c80d3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 --2- 172.21.15.5:0/3920123328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c80d3f20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f89c80d0c60 tx=0x7f89b002a0f0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 -- 172.21.15.5:0/3920123328 shutdown_connections 2024-04-25T10:02:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 --2- 172.21.15.5:0/3920123328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c80d3f20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 --2- 172.21.15.5:0/3920123328 >> v2:172.21.15.80:3300/0 conn(0x7f89c815d8a0 0x7f89c80d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 -- 172.21.15.5:0/3920123328 >> 172.21.15.5:0/3920123328 conn(0x7f89c808af80 msgr2=0x7f89c80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 -- 172.21.15.5:0/3920123328 shutdown_connections 2024-04-25T10:02:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.176+0000 7f89ce200640 1 -- 172.21.15.5:0/3920123328 wait complete. 2024-04-25T10:02:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.179+0000 7f89ce200640 1 Processor -- start 2024-04-25T10:02:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.179+0000 7f89ce200640 1 -- start start 2024-04-25T10:02:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.180+0000 7f89ce200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c8152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.180+0000 7f89ce200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89c815d8a0 0x7f89c80ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.180+0000 7f89ce200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81614e0 0x7f89c80b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.180+0000 7f89ce200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c815ec90 con 0x7f89c80b2f30 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.180+0000 7f89ce200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c80d4460 con 0x7f89c815d8a0 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.180+0000 7f89ce200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c815ee10 con 0x7f89c81614e0 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c8152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c8152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46796/0 (socket says 172.21.15.5:46796) 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 -- 172.21.15.5:0/1544537691 learned_addr learned my addr 172.21.15.5:0/1544537691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89c7e00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89c815d8a0 0x7f89c80ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 -- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81614e0 msgr2=0x7f89c80b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81614e0 0x7f89c80b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 -- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89c815d8a0 msgr2=0x7f89c80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89c815d8a0 0x7f89c80ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 -- 172.21.15.5:0/1544537691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c80cac40 con 0x7f89c80b2f30 2024-04-25T10:02:26.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89c7e00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89c815d8a0 0x7f89c80ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.181+0000 7f89cce00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c8152c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ceb0 tx=0x7f89b800a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.182+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b8003b90 con 0x7f89c80b2f30 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.182+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b8003d30 con 0x7f89c80b2f30 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.182+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b800e870 con 0x7f89c80b2f30 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.182+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c80caf20 con 0x7f89c80b2f30 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.182+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c80cb4e0 con 0x7f89c80b2f30 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.183+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c8068f60 con 0x7f89c80b2f30 2024-04-25T10:02:26.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.184+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89b8009030 con 0x7f89c80b2f30 2024-04-25T10:02:26.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.185+0000 7f89c4c00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89a406b500 0x7f89a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.185+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f89b808afa0 con 0x7f89c80b2f30 2024-04-25T10:02:26.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.185+0000 7f89c7e00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89a406b500 0x7f89a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.186+0000 7f89c7e00640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89a406b500 0x7f89a406d9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89b0034000 tx=0x7f89b002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.189+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89b801d340 con 0x7f89c80b2f30 2024-04-25T10:02:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.389+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89c80d0e50 con 0x7f89a406b500 2024-04-25T10:02:26.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.415+0000 7f89c4c00640 1 -- 172.21.15.5:0/1544537691 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332569 (secure 0 0 0) 0x7f89c80d0e50 con 0x7f89a406b500 2024-04-25T10:02:26.422 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89a406b500 msgr2=0x7f89a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89a406b500 0x7f89a406d9c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f89b0034000 tx=0x7f89b002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 msgr2=0x7f89c8152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c8152c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f89b800ceb0 tx=0x7f89b800a6b0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 shutdown_connections 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89c81614e0 0x7f89c80b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89a406b500 0x7f89a406d9c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89c815d8a0 0x7f89c80ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 --2- 172.21.15.5:0/1544537691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89c80b2f30 0x7f89c8152c20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.422+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 >> 172.21.15.5:0/1544537691 conn(0x7f89c808af80 msgr2=0x7f89c806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.423+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 shutdown_connections 2024-04-25T10:02:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.423+0000 7f89ce200640 1 -- 172.21.15.5:0/1544537691 wait complete. 2024-04-25T10:02:26.448 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T10:02:24.618233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236148,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738572,"statfs":{"total":773094113280,"available":772852297728,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006491"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356469+0000","last_change":"2024-04-25T10:01:57.061422+0000","last_active":"2024-04-25T10:02:03.356469+0000","last_peered":"2024-04-25T10:02:03.356469+0000","last_clean":"2024-04-25T10:02:03.356469+0000","last_became_active":"2024-04-25T10:01:57.061119+0000","last_became_peered":"2024-04-25T10:01:57.061119+0000","last_unstale":"2024-04-25T10:02:03.356469+0000","last_undegraded":"2024-04-25T10:02:03.356469+0000","last_fullsized":"2024-04-25T10:02:03.356469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:26.806679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433872+0000","last_change":"2024-04-25T10:01:58.115482+0000","last_active":"2024-04-25T10:02:03.433872+0000","last_peered":"2024-04-25T10:02:03.433872+0000","last_clean":"2024-04-25T10:02:03.433872+0000","last_became_active":"2024-04-25T10:01:58.115138+0000","last_became_peered":"2024-04-25T10:01:58.115138+0000","last_unstale":"2024-04-25T10:02:03.433872+0000","last_undegraded":"2024-04-25T10:02:03.433872+0000","last_fullsized":"2024-04-25T10:02:03.433872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352821+0000","last_change":"2024-04-25T10:02:02.368709+0000","last_active":"2024-04-25T10:02:03.352821+0000","last_peered":"2024-04-25T10:02:03.352821+0000","last_clean":"2024-04-25T10:02:03.352821+0000","last_became_active":"2024-04-25T10:02:02.367787+0000","last_became_peered":"2024-04-25T10:02:02.367787+0000","last_unstale":"2024-04-25T10:02:03.352821+0000","last_undegraded":"2024-04-25T10:02:03.352821+0000","last_fullsized":"2024-04-25T10:02:03.352821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:31.444725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434120+0000","last_change":"2024-04-25T10:01:57.061756+0000","last_active":"2024-04-25T10:02:03.434120+0000","last_peered":"2024-04-25T10:02:03.434120+0000","last_clean":"2024-04-25T10:02:03.434120+0000","last_became_active":"2024-04-25T10:01:57.058627+0000","last_became_peered":"2024-04-25T10:01:57.058627+0000","last_unstale":"2024-04-25T10:02:03.434120+0000","last_undegraded":"2024-04-25T10:02:03.434120+0000","last_fullsized":"2024-04-25T10:02:03.434120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434000+0000","last_change":"2024-04-25T10:01:58.113879+0000","last_active":"2024-04-25T10:02:03.434000+0000","last_peered":"2024-04-25T10:02:03.434000+0000","last_clean":"2024-04-25T10:02:03.434000+0000","last_became_active":"2024-04-25T10:01:58.113527+0000","last_became_peered":"2024-04-25T10:01:58.113527+0000","last_unstale":"2024-04-25T10:02:03.434000+0000","last_undegraded":"2024-04-25T10:02:03.434000+0000","last_fullsized":"2024-04-25T10:02:03.434000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519097+0000","last_change":"2024-04-25T10:02:02.364833+0000","last_active":"2024-04-25T10:02:03.519097+0000","last_peered":"2024-04-25T10:02:03.519097+0000","last_clean":"2024-04-25T10:02:03.519097+0000","last_became_active":"2024-04-25T10:02:02.364420+0000","last_became_peered":"2024-04-25T10:02:02.364420+0000","last_unstale":"2024-04-25T10:02:03.519097+0000","last_undegraded":"2024-04-25T10:02:03.519097+0000","last_fullsized":"2024-04-25T10:02:03.519097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:17.582640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518857+0000","last_change":"2024-04-25T10:01:57.049323+0000","last_active":"2024-04-25T10:02:03.518857+0000","last_peered":"2024-04-25T10:02:03.518857+0000","last_clean":"2024-04-25T10:02:03.518857+0000","last_became_active":"2024-04-25T10:01:57.048997+0000","last_became_peered":"2024-04-25T10:01:57.048997+0000","last_unstale":"2024-04-25T10:02:03.518857+0000","last_undegraded":"2024-04-25T10:02:03.518857+0000","last_fullsized":"2024-04-25T10:02:03.518857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.523353+0000","last_change":"2024-04-25T10:01:58.074975+0000","last_active":"2024-04-25T10:02:03.523353+0000","last_peered":"2024-04-25T10:02:03.523353+0000","last_clean":"2024-04-25T10:02:03.523353+0000","last_became_active":"2024-04-25T10:01:58.074637+0000","last_became_peered":"2024-04-25T10:01:58.074637+0000","last_unstale":"2024-04-25T10:02:03.523353+0000","last_undegraded":"2024-04-25T10:02:03.523353+0000","last_fullsized":"2024-04-25T10:02:03.523353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522261+0000","last_change":"2024-04-25T10:02:02.367186+0000","last_active":"2024-04-25T10:02:03.522261+0000","last_peered":"2024-04-25T10:02:03.522261+0000","last_clean":"2024-04-25T10:02:03.522261+0000","last_became_active":"2024-04-25T10:02:02.366476+0000","last_became_peered":"2024-04-25T10:02:02.366476+0000","last_unstale":"2024-04-25T10:02:03.522261+0000","last_undegraded":"2024-04-25T10:02:03.522261+0000","last_fullsized":"2024-04-25T10:02:03.522261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519518+0000","last_change":"2024-04-25T10:01:57.058161+0000","last_active":"2024-04-25T10:02:03.519518+0000","last_peered":"2024-04-25T10:02:03.519518+0000","last_clean":"2024-04-25T10:02:03.519518+0000","last_became_active":"2024-04-25T10:01:57.057891+0000","last_became_peered":"2024-04-25T10:01:57.057891+0000","last_unstale":"2024-04-25T10:02:03.519518+0000","last_undegraded":"2024-04-25T10:02:03.519518+0000","last_fullsized":"2024-04-25T10:02:03.519518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:23.026096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404272+0000","last_change":"2024-04-25T10:01:58.074490+0000","last_active":"2024-04-25T10:02:03.404272+0000","last_peered":"2024-04-25T10:02:03.404272+0000","last_clean":"2024-04-25T10:02:03.404272+0000","last_became_active":"2024-04-25T10:01:58.074117+0000","last_became_peered":"2024-04-25T10:01:58.074117+0000","last_unstale":"2024-04-25T10:02:03.404272+0000","last_undegraded":"2024-04-25T10:02:03.404272+0000","last_fullsized":"2024-04-25T10:02:03.404272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:53.841219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520948+0000","last_change":"2024-04-25T10:02:02.362410+0000","last_active":"2024-04-25T10:02:03.520948+0000","last_peered":"2024-04-25T10:02:03.520948+0000","last_clean":"2024-04-25T10:02:03.520948+0000","last_became_active":"2024-04-25T10:02:02.359612+0000","last_became_peered":"2024-04-25T10:02:02.359612+0000","last_unstale":"2024-04-25T10:02:03.520948+0000","last_undegraded":"2024-04-25T10:02:03.520948+0000","last_fullsized":"2024-04-25T10:02:03.520948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.136854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348157+0000","last_change":"2024-04-25T10:01:58.073917+0000","last_active":"2024-04-25T10:02:03.348157+0000","last_peered":"2024-04-25T10:02:03.348157+0000","last_clean":"2024-04-25T10:02:03.348157+0000","last_became_active":"2024-04-25T10:01:58.073122+0000","last_became_peered":"2024-04-25T10:01:58.073122+0000","last_unstale":"2024-04-25T10:02:03.348157+0000","last_undegraded":"2024-04-25T10:02:03.348157+0000","last_fullsized":"2024-04-25T10:02:03.348157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402599+0000","last_change":"2024-04-25T10:01:57.059507+0000","last_active":"2024-04-25T10:02:03.402599+0000","last_peered":"2024-04-25T10:02:03.402599+0000","last_clean":"2024-04-25T10:02:03.402599+0000","last_became_active":"2024-04-25T10:01:57.056083+0000","last_became_peered":"2024-04-25T10:01:57.056083+0000","last_unstale":"2024-04-25T10:02:03.402599+0000","last_undegraded":"2024-04-25T10:02:03.402599+0000","last_fullsized":"2024-04-25T10:02:03.402599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:29.264217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404104+0000","last_change":"2024-04-25T10:02:02.362032+0000","last_active":"2024-04-25T10:02:03.404104+0000","last_peered":"2024-04-25T10:02:03.404104+0000","last_clean":"2024-04-25T10:02:03.404104+0000","last_became_active":"2024-04-25T10:02:02.361608+0000","last_became_peered":"2024-04-25T10:02:02.361608+0000","last_unstale":"2024-04-25T10:02:03.404104+0000","last_undegraded":"2024-04-25T10:02:03.404104+0000","last_fullsized":"2024-04-25T10:02:03.404104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:27.993920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353535+0000","last_change":"2024-04-25T10:01:58.116451+0000","last_active":"2024-04-25T10:02:03.353535+0000","last_peered":"2024-04-25T10:02:03.353535+0000","last_clean":"2024-04-25T10:02:03.353535+0000","last_became_active":"2024-04-25T10:01:58.116177+0000","last_became_peered":"2024-04-25T10:01:58.116177+0000","last_unstale":"2024-04-25T10:02:03.353535+0000","last_undegraded":"2024-04-25T10:02:03.353535+0000","last_fullsized":"2024-04-25T10:02:03.353535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518845+0000","last_change":"2024-04-25T10:01:57.062452+0000","last_active":"2024-04-25T10:02:03.518845+0000","last_peered":"2024-04-25T10:02:03.518845+0000","last_clean":"2024-04-25T10:02:03.518845+0000","last_became_active":"2024-04-25T10:01:57.062105+0000","last_became_peered":"2024-04-25T10:01:57.062105+0000","last_unstale":"2024-04-25T10:02:03.518845+0000","last_undegraded":"2024-04-25T10:02:03.518845+0000","last_fullsized":"2024-04-25T10:02:03.518845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518790+0000","last_change":"2024-04-25T10:02:02.369531+0000","last_active":"2024-04-25T10:02:03.518790+0000","last_peered":"2024-04-25T10:02:03.518790+0000","last_clean":"2024-04-25T10:02:03.518790+0000","last_became_active":"2024-04-25T10:02:02.367651+0000","last_became_peered":"2024-04-25T10:02:02.367651+0000","last_unstale":"2024-04-25T10:02:03.518790+0000","last_undegraded":"2024-04-25T10:02:03.518790+0000","last_fullsized":"2024-04-25T10:02:03.518790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:58.140005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352233+0000","last_change":"2024-04-25T10:01:58.117205+0000","last_active":"2024-04-25T10:02:03.352233+0000","last_peered":"2024-04-25T10:02:03.352233+0000","last_clean":"2024-04-25T10:02:03.352233+0000","last_became_active":"2024-04-25T10:01:58.116873+0000","last_became_peered":"2024-04-25T10:01:58.116873+0000","last_unstale":"2024-04-25T10:02:03.352233+0000","last_undegraded":"2024-04-25T10:02:03.352233+0000","last_fullsized":"2024-04-25T10:02:03.352233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:25.305583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354051+0000","last_change":"2024-04-25T10:01:57.049606+0000","last_active":"2024-04-25T10:02:03.354051+0000","last_peered":"2024-04-25T10:02:03.354051+0000","last_clean":"2024-04-25T10:02:03.354051+0000","last_became_active":"2024-04-25T10:01:57.049300+0000","last_became_peered":"2024-04-25T10:01:57.049300+0000","last_unstale":"2024-04-25T10:02:03.354051+0000","last_undegraded":"2024-04-25T10:02:03.354051+0000","last_fullsized":"2024-04-25T10:02:03.354051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:29.284211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354667+0000","last_change":"2024-04-25T10:02:02.366098+0000","last_active":"2024-04-25T10:02:03.354667+0000","last_peered":"2024-04-25T10:02:03.354667+0000","last_clean":"2024-04-25T10:02:03.354667+0000","last_became_active":"2024-04-25T10:02:02.365669+0000","last_became_peered":"2024-04-25T10:02:02.365669+0000","last_unstale":"2024-04-25T10:02:03.354667+0000","last_undegraded":"2024-04-25T10:02:03.354667+0000","last_fullsized":"2024-04-25T10:02:03.354667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:34.132211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347971+0000","last_change":"2024-04-25T10:01:58.116653+0000","last_active":"2024-04-25T10:02:03.347971+0000","last_peered":"2024-04-25T10:02:03.347971+0000","last_clean":"2024-04-25T10:02:03.347971+0000","last_became_active":"2024-04-25T10:01:58.116414+0000","last_became_peered":"2024-04-25T10:01:58.116414+0000","last_unstale":"2024-04-25T10:02:03.347971+0000","last_undegraded":"2024-04-25T10:02:03.347971+0000","last_fullsized":"2024-04-25T10:02:03.347971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:20.873677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.517442+0000","last_change":"2024-04-25T10:01:57.042210+0000","last_active":"2024-04-25T10:02:03.517442+0000","last_peered":"2024-04-25T10:02:03.517442+0000","last_clean":"2024-04-25T10:02:03.517442+0000","last_became_active":"2024-04-25T10:01:57.041890+0000","last_became_peered":"2024-04-25T10:01:57.041890+0000","last_unstale":"2024-04-25T10:02:03.517442+0000","last_undegraded":"2024-04-25T10:02:03.517442+0000","last_fullsized":"2024-04-25T10:02:03.517442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521682+0000","last_change":"2024-04-25T10:02:02.364078+0000","last_active":"2024-04-25T10:02:03.521682+0000","last_peered":"2024-04-25T10:02:03.521682+0000","last_clean":"2024-04-25T10:02:03.521682+0000","last_became_active":"2024-04-25T10:02:02.358978+0000","last_became_peered":"2024-04-25T10:02:02.358978+0000","last_unstale":"2024-04-25T10:02:03.521682+0000","last_undegraded":"2024-04-25T10:02:03.521682+0000","last_fullsized":"2024-04-25T10:02:03.521682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355780+0000","last_change":"2024-04-25T10:01:58.118220+0000","last_active":"2024-04-25T10:02:03.355780+0000","last_peered":"2024-04-25T10:02:03.355780+0000","last_clean":"2024-04-25T10:02:03.355780+0000","last_became_active":"2024-04-25T10:01:58.117911+0000","last_became_peered":"2024-04-25T10:01:58.117911+0000","last_unstale":"2024-04-25T10:02:03.355780+0000","last_undegraded":"2024-04-25T10:02:03.355780+0000","last_fullsized":"2024-04-25T10:02:03.355780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:27.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405295+0000","last_change":"2024-04-25T10:01:57.064576+0000","last_active":"2024-04-25T10:02:03.405295+0000","last_peered":"2024-04-25T10:02:03.405295+0000","last_clean":"2024-04-25T10:02:03.405295+0000","last_became_active":"2024-04-25T10:01:57.064301+0000","last_became_peered":"2024-04-25T10:01:57.064301+0000","last_unstale":"2024-04-25T10:02:03.405295+0000","last_undegraded":"2024-04-25T10:02:03.405295+0000","last_fullsized":"2024-04-25T10:02:03.405295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.258589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521777+0000","last_change":"2024-04-25T10:02:02.362701+0000","last_active":"2024-04-25T10:02:03.521777+0000","last_peered":"2024-04-25T10:02:03.521777+0000","last_clean":"2024-04-25T10:02:03.521777+0000","last_became_active":"2024-04-25T10:02:02.361742+0000","last_became_peered":"2024-04-25T10:02:02.361742+0000","last_unstale":"2024-04-25T10:02:03.521777+0000","last_undegraded":"2024-04-25T10:02:03.521777+0000","last_fullsized":"2024-04-25T10:02:03.521777+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356323+0000","last_change":"2024-04-25T10:01:58.121271+0000","last_active":"2024-04-25T10:02:03.356323+0000","last_peered":"2024-04-25T10:02:03.356323+0000","last_clean":"2024-04-25T10:02:03.356323+0000","last_became_active":"2024-04-25T10:01:58.121001+0000","last_became_peered":"2024-04-25T10:01:58.121001+0000","last_unstale":"2024-04-25T10:02:03.356323+0000","last_undegraded":"2024-04-25T10:02:03.356323+0000","last_fullsized":"2024-04-25T10:02:03.356323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:56.894928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356241+0000","last_change":"2024-04-25T10:01:57.047184+0000","last_active":"2024-04-25T10:02:03.356241+0000","last_peered":"2024-04-25T10:02:03.356241+0000","last_clean":"2024-04-25T10:02:03.356241+0000","last_became_active":"2024-04-25T10:01:57.046623+0000","last_became_peered":"2024-04-25T10:01:57.046623+0000","last_unstale":"2024-04-25T10:02:03.356241+0000","last_undegraded":"2024-04-25T10:02:03.356241+0000","last_fullsized":"2024-04-25T10:02:03.356241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:31.269009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353577+0000","last_change":"2024-04-25T10:02:02.370654+0000","last_active":"2024-04-25T10:02:03.353577+0000","last_peered":"2024-04-25T10:02:03.353577+0000","last_clean":"2024-04-25T10:02:03.353577+0000","last_became_active":"2024-04-25T10:02:02.369464+0000","last_became_peered":"2024-04-25T10:02:02.369464+0000","last_unstale":"2024-04-25T10:02:03.353577+0000","last_undegraded":"2024-04-25T10:02:03.353577+0000","last_fullsized":"2024-04-25T10:02:03.353577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:47.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519808+0000","last_change":"2024-04-25T10:01:58.123583+0000","last_active":"2024-04-25T10:02:03.519808+0000","last_peered":"2024-04-25T10:02:03.519808+0000","last_clean":"2024-04-25T10:02:03.519808+0000","last_became_active":"2024-04-25T10:01:58.122380+0000","last_became_peered":"2024-04-25T10:01:58.122380+0000","last_unstale":"2024-04-25T10:02:03.519808+0000","last_undegraded":"2024-04-25T10:02:03.519808+0000","last_fullsized":"2024-04-25T10:02:03.519808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:54.007484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405069+0000","last_change":"2024-04-25T10:01:57.060061+0000","last_active":"2024-04-25T10:02:03.405069+0000","last_peered":"2024-04-25T10:02:03.405069+0000","last_clean":"2024-04-25T10:02:03.405069+0000","last_became_active":"2024-04-25T10:01:57.056768+0000","last_became_peered":"2024-04-25T10:01:57.056768+0000","last_unstale":"2024-04-25T10:02:03.405069+0000","last_undegraded":"2024-04-25T10:02:03.405069+0000","last_fullsized":"2024-04-25T10:02:03.405069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.943464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518947+0000","last_change":"2024-04-25T10:02:02.366286+0000","last_active":"2024-04-25T10:02:03.518947+0000","last_peered":"2024-04-25T10:02:03.518947+0000","last_clean":"2024-04-25T10:02:03.518947+0000","last_became_active":"2024-04-25T10:02:02.365877+0000","last_became_peered":"2024-04-25T10:02:02.365877+0000","last_unstale":"2024-04-25T10:02:03.518947+0000","last_undegraded":"2024-04-25T10:02:03.518947+0000","last_fullsized":"2024-04-25T10:02:03.518947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521404+0000","last_change":"2024-04-25T10:01:58.126543+0000","last_active":"2024-04-25T10:02:03.521404+0000","last_peered":"2024-04-25T10:02:03.521404+0000","last_clean":"2024-04-25T10:02:03.521404+0000","last_became_active":"2024-04-25T10:01:58.126245+0000","last_became_peered":"2024-04-25T10:01:58.126245+0000","last_unstale":"2024-04-25T10:02:03.521404+0000","last_undegraded":"2024-04-25T10:02:03.521404+0000","last_fullsized":"2024-04-25T10:02:03.521404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433241+0000","last_change":"2024-04-25T10:01:57.056845+0000","last_active":"2024-04-25T10:02:03.433241+0000","last_peered":"2024-04-25T10:02:03.433241+0000","last_clean":"2024-04-25T10:02:03.433241+0000","last_became_active":"2024-04-25T10:01:57.056550+0000","last_became_peered":"2024-04-25T10:01:57.056550+0000","last_unstale":"2024-04-25T10:02:03.433241+0000","last_undegraded":"2024-04-25T10:02:03.433241+0000","last_fullsized":"2024-04-25T10:02:03.433241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518500+0000","last_change":"2024-04-25T10:02:02.360392+0000","last_active":"2024-04-25T10:02:03.518500+0000","last_peered":"2024-04-25T10:02:03.518500+0000","last_clean":"2024-04-25T10:02:03.518500+0000","last_became_active":"2024-04-25T10:02:02.359550+0000","last_became_peered":"2024-04-25T10:02:02.359550+0000","last_unstale":"2024-04-25T10:02:03.518500+0000","last_undegraded":"2024-04-25T10:02:03.518500+0000","last_fullsized":"2024-04-25T10:02:03.518500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518461+0000","last_change":"2024-04-25T10:01:58.070356+0000","last_active":"2024-04-25T10:02:03.518461+0000","last_peered":"2024-04-25T10:02:03.518461+0000","last_clean":"2024-04-25T10:02:03.518461+0000","last_became_active":"2024-04-25T10:01:58.068594+0000","last_became_peered":"2024-04-25T10:01:58.068594+0000","last_unstale":"2024-04-25T10:02:03.518461+0000","last_undegraded":"2024-04-25T10:02:03.518461+0000","last_fullsized":"2024-04-25T10:02:03.518461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402901+0000","last_change":"2024-04-25T10:01:57.061869+0000","last_active":"2024-04-25T10:02:03.402901+0000","last_peered":"2024-04-25T10:02:03.402901+0000","last_clean":"2024-04-25T10:02:03.402901+0000","last_became_active":"2024-04-25T10:01:57.059517+0000","last_became_peered":"2024-04-25T10:01:57.059517+0000","last_unstale":"2024-04-25T10:02:03.402901+0000","last_undegraded":"2024-04-25T10:02:03.402901+0000","last_fullsized":"2024-04-25T10:02:03.402901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:49.036030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519650+0000","last_change":"2024-04-25T10:02:02.364712+0000","last_active":"2024-04-25T10:02:03.519650+0000","last_peered":"2024-04-25T10:02:03.519650+0000","last_clean":"2024-04-25T10:02:03.519650+0000","last_became_active":"2024-04-25T10:02:02.363483+0000","last_became_peered":"2024-04-25T10:02:02.363483+0000","last_unstale":"2024-04-25T10:02:03.519650+0000","last_undegraded":"2024-04-25T10:02:03.519650+0000","last_fullsized":"2024-04-25T10:02:03.519650+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432126+0000","last_change":"2024-04-25T10:01:58.124129+0000","last_active":"2024-04-25T10:02:03.432126+0000","last_peered":"2024-04-25T10:02:03.432126+0000","last_clean":"2024-04-25T10:02:03.432126+0000","last_became_active":"2024-04-25T10:01:58.123815+0000","last_became_peered":"2024-04-25T10:01:58.123815+0000","last_unstale":"2024-04-25T10:02:03.432126+0000","last_undegraded":"2024-04-25T10:02:03.432126+0000","last_fullsized":"2024-04-25T10:02:03.432126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352979+0000","last_change":"2024-04-25T10:01:57.058063+0000","last_active":"2024-04-25T10:02:03.352979+0000","last_peered":"2024-04-25T10:02:03.352979+0000","last_clean":"2024-04-25T10:02:03.352979+0000","last_became_active":"2024-04-25T10:01:57.057727+0000","last_became_peered":"2024-04-25T10:01:57.057727+0000","last_unstale":"2024-04-25T10:02:03.352979+0000","last_undegraded":"2024-04-25T10:02:03.352979+0000","last_fullsized":"2024-04-25T10:02:03.352979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520229+0000","last_change":"2024-04-25T10:02:02.366966+0000","last_active":"2024-04-25T10:02:03.520229+0000","last_peered":"2024-04-25T10:02:03.520229+0000","last_clean":"2024-04-25T10:02:03.520229+0000","last_became_active":"2024-04-25T10:02:02.366563+0000","last_became_peered":"2024-04-25T10:02:02.366563+0000","last_unstale":"2024-04-25T10:02:03.520229+0000","last_undegraded":"2024-04-25T10:02:03.520229+0000","last_fullsized":"2024-04-25T10:02:03.520229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:50.524934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518446+0000","last_change":"2024-04-25T10:01:58.124316+0000","last_active":"2024-04-25T10:02:03.518446+0000","last_peered":"2024-04-25T10:02:03.518446+0000","last_clean":"2024-04-25T10:02:03.518446+0000","last_became_active":"2024-04-25T10:01:58.123995+0000","last_became_peered":"2024-04-25T10:01:58.123995+0000","last_unstale":"2024-04-25T10:02:03.518446+0000","last_undegraded":"2024-04-25T10:02:03.518446+0000","last_fullsized":"2024-04-25T10:02:03.518446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348251+0000","last_change":"2024-04-25T10:01:57.064900+0000","last_active":"2024-04-25T10:02:03.348251+0000","last_peered":"2024-04-25T10:02:03.348251+0000","last_clean":"2024-04-25T10:02:03.348251+0000","last_became_active":"2024-04-25T10:01:57.063565+0000","last_became_peered":"2024-04-25T10:01:57.063565+0000","last_unstale":"2024-04-25T10:02:03.348251+0000","last_undegraded":"2024-04-25T10:02:03.348251+0000","last_fullsized":"2024-04-25T10:02:03.348251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433722+0000","last_change":"2024-04-25T10:02:02.363528+0000","last_active":"2024-04-25T10:02:03.433722+0000","last_peered":"2024-04-25T10:02:03.433722+0000","last_clean":"2024-04-25T10:02:03.433722+0000","last_became_active":"2024-04-25T10:02:02.360337+0000","last_became_peered":"2024-04-25T10:02:02.360337+0000","last_unstale":"2024-04-25T10:02:03.433722+0000","last_undegraded":"2024-04-25T10:02:03.433722+0000","last_fullsized":"2024-04-25T10:02:03.433722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:25.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354986+0000","last_change":"2024-04-25T10:01:58.114954+0000","last_active":"2024-04-25T10:02:03.354986+0000","last_peered":"2024-04-25T10:02:03.354986+0000","last_clean":"2024-04-25T10:02:03.354986+0000","last_became_active":"2024-04-25T10:01:58.114652+0000","last_became_peered":"2024-04-25T10:01:58.114652+0000","last_unstale":"2024-04-25T10:02:03.354986+0000","last_undegraded":"2024-04-25T10:02:03.354986+0000","last_fullsized":"2024-04-25T10:02:03.354986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.396332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402951+0000","last_change":"2024-04-25T10:01:57.060078+0000","last_active":"2024-04-25T10:02:03.402951+0000","last_peered":"2024-04-25T10:02:03.402951+0000","last_clean":"2024-04-25T10:02:03.402951+0000","last_became_active":"2024-04-25T10:01:57.059811+0000","last_became_peered":"2024-04-25T10:01:57.059811+0000","last_unstale":"2024-04-25T10:02:03.402951+0000","last_undegraded":"2024-04-25T10:02:03.402951+0000","last_fullsized":"2024-04-25T10:02:03.402951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.018629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520801+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:02:03.520801+0000","last_peered":"2024-04-25T10:02:03.520801+0000","last_clean":"2024-04-25T10:02:03.520801+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:02:03.520801+0000","last_undegraded":"2024-04-25T10:02:03.520801+0000","last_fullsized":"2024-04-25T10:02:03.520801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404900+0000","last_change":"2024-04-25T10:02:02.358810+0000","last_active":"2024-04-25T10:02:03.404900+0000","last_peered":"2024-04-25T10:02:03.404900+0000","last_clean":"2024-04-25T10:02:03.404900+0000","last_became_active":"2024-04-25T10:02:02.358270+0000","last_became_peered":"2024-04-25T10:02:02.358270+0000","last_unstale":"2024-04-25T10:02:03.404900+0000","last_undegraded":"2024-04-25T10:02:03.404900+0000","last_fullsized":"2024-04-25T10:02:03.404900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:00.819451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404528+0000","last_change":"2024-04-25T10:01:58.073772+0000","last_active":"2024-04-25T10:02:03.404528+0000","last_peered":"2024-04-25T10:02:03.404528+0000","last_clean":"2024-04-25T10:02:03.404528+0000","last_became_active":"2024-04-25T10:01:58.073104+0000","last_became_peered":"2024-04-25T10:01:58.073104+0000","last_unstale":"2024-04-25T10:02:03.404528+0000","last_undegraded":"2024-04-25T10:02:03.404528+0000","last_fullsized":"2024-04-25T10:02:03.404528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.239771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520009+0000","last_change":"2024-04-25T10:01:57.054884+0000","last_active":"2024-04-25T10:02:03.520009+0000","last_peered":"2024-04-25T10:02:03.520009+0000","last_clean":"2024-04-25T10:02:03.520009+0000","last_became_active":"2024-04-25T10:01:57.054577+0000","last_became_peered":"2024-04-25T10:01:57.054577+0000","last_unstale":"2024-04-25T10:02:03.520009+0000","last_undegraded":"2024-04-25T10:02:03.520009+0000","last_fullsized":"2024-04-25T10:02:03.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521556+0000","last_change":"2024-04-25T10:02:02.364334+0000","last_active":"2024-04-25T10:02:03.521556+0000","last_peered":"2024-04-25T10:02:03.521556+0000","last_clean":"2024-04-25T10:02:03.521556+0000","last_became_active":"2024-04-25T10:02:02.363695+0000","last_became_peered":"2024-04-25T10:02:02.363695+0000","last_unstale":"2024-04-25T10:02:03.521556+0000","last_undegraded":"2024-04-25T10:02:03.521556+0000","last_fullsized":"2024-04-25T10:02:03.521556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:30.903672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519516+0000","last_change":"2024-04-25T10:01:58.090256+0000","last_active":"2024-04-25T10:02:03.519516+0000","last_peered":"2024-04-25T10:02:03.519516+0000","last_clean":"2024-04-25T10:02:03.519516+0000","last_became_active":"2024-04-25T10:01:58.089936+0000","last_became_peered":"2024-04-25T10:01:58.089936+0000","last_unstale":"2024-04-25T10:02:03.519516+0000","last_undegraded":"2024-04-25T10:02:03.519516+0000","last_fullsized":"2024-04-25T10:02:03.519516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433744+0000","last_change":"2024-04-25T10:01:57.060690+0000","last_active":"2024-04-25T10:02:03.433744+0000","last_peered":"2024-04-25T10:02:03.433744+0000","last_clean":"2024-04-25T10:02:03.433744+0000","last_became_active":"2024-04-25T10:01:57.060345+0000","last_became_peered":"2024-04-25T10:01:57.060345+0000","last_unstale":"2024-04-25T10:02:03.433744+0000","last_undegraded":"2024-04-25T10:02:03.433744+0000","last_fullsized":"2024-04-25T10:02:03.433744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433246+0000","last_change":"2024-04-25T10:02:02.371169+0000","last_active":"2024-04-25T10:02:03.433246+0000","last_peered":"2024-04-25T10:02:03.433246+0000","last_clean":"2024-04-25T10:02:03.433246+0000","last_became_active":"2024-04-25T10:02:02.370789+0000","last_became_peered":"2024-04-25T10:02:02.370789+0000","last_unstale":"2024-04-25T10:02:03.433246+0000","last_undegraded":"2024-04-25T10:02:03.433246+0000","last_fullsized":"2024-04-25T10:02:03.433246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:40.538557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520420+0000","last_change":"2024-04-25T10:01:58.072580+0000","last_active":"2024-04-25T10:02:03.520420+0000","last_peered":"2024-04-25T10:02:03.520420+0000","last_clean":"2024-04-25T10:02:03.520420+0000","last_became_active":"2024-04-25T10:01:58.072277+0000","last_became_peered":"2024-04-25T10:01:58.072277+0000","last_unstale":"2024-04-25T10:02:03.520420+0000","last_undegraded":"2024-04-25T10:02:03.520420+0000","last_fullsized":"2024-04-25T10:02:03.520420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356066+0000","last_change":"2024-04-25T10:01:57.057184+0000","last_active":"2024-04-25T10:02:03.356066+0000","last_peered":"2024-04-25T10:02:03.356066+0000","last_clean":"2024-04-25T10:02:03.356066+0000","last_became_active":"2024-04-25T10:01:57.054573+0000","last_became_peered":"2024-04-25T10:01:57.054573+0000","last_unstale":"2024-04-25T10:02:03.356066+0000","last_undegraded":"2024-04-25T10:02:03.356066+0000","last_fullsized":"2024-04-25T10:02:03.356066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:33.311253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347440+0000","last_change":"2024-04-25T10:02:02.373007+0000","last_active":"2024-04-25T10:02:03.347440+0000","last_peered":"2024-04-25T10:02:03.347440+0000","last_clean":"2024-04-25T10:02:03.347440+0000","last_became_active":"2024-04-25T10:02:02.372467+0000","last_became_peered":"2024-04-25T10:02:02.372467+0000","last_unstale":"2024-04-25T10:02:03.347440+0000","last_undegraded":"2024-04-25T10:02:03.347440+0000","last_fullsized":"2024-04-25T10:02:03.347440+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:18.003128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521943+0000","last_change":"2024-04-25T10:01:58.071062+0000","last_active":"2024-04-25T10:02:03.521943+0000","last_peered":"2024-04-25T10:02:03.521943+0000","last_clean":"2024-04-25T10:02:03.521943+0000","last_became_active":"2024-04-25T10:01:58.070721+0000","last_became_peered":"2024-04-25T10:01:58.070721+0000","last_unstale":"2024-04-25T10:02:03.521943+0000","last_undegraded":"2024-04-25T10:02:03.521943+0000","last_fullsized":"2024-04-25T10:02:03.521943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520558+0000","last_change":"2024-04-25T10:01:57.051450+0000","last_active":"2024-04-25T10:02:03.520558+0000","last_peered":"2024-04-25T10:02:03.520558+0000","last_clean":"2024-04-25T10:02:03.520558+0000","last_became_active":"2024-04-25T10:01:57.051104+0000","last_became_peered":"2024-04-25T10:01:57.051104+0000","last_unstale":"2024-04-25T10:02:03.520558+0000","last_undegraded":"2024-04-25T10:02:03.520558+0000","last_fullsized":"2024-04-25T10:02:03.520558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521721+0000","last_change":"2024-04-25T10:02:02.371242+0000","last_active":"2024-04-25T10:02:03.521721+0000","last_peered":"2024-04-25T10:02:03.521721+0000","last_clean":"2024-04-25T10:02:03.521721+0000","last_became_active":"2024-04-25T10:02:02.370975+0000","last_became_peered":"2024-04-25T10:02:02.370975+0000","last_unstale":"2024-04-25T10:02:03.521721+0000","last_undegraded":"2024-04-25T10:02:03.521721+0000","last_fullsized":"2024-04-25T10:02:03.521721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:56.622899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347953+0000","last_change":"2024-04-25T10:01:58.076461+0000","last_active":"2024-04-25T10:02:03.347953+0000","last_peered":"2024-04-25T10:02:03.347953+0000","last_clean":"2024-04-25T10:02:03.347953+0000","last_became_active":"2024-04-25T10:01:58.075822+0000","last_became_peered":"2024-04-25T10:01:58.075822+0000","last_unstale":"2024-04-25T10:02:03.347953+0000","last_undegraded":"2024-04-25T10:02:03.347953+0000","last_fullsized":"2024-04-25T10:02:03.347953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352738+0000","last_change":"2024-04-25T10:01:57.051484+0000","last_active":"2024-04-25T10:02:03.352738+0000","last_peered":"2024-04-25T10:02:03.352738+0000","last_clean":"2024-04-25T10:02:03.352738+0000","last_became_active":"2024-04-25T10:01:57.051183+0000","last_became_peered":"2024-04-25T10:01:57.051183+0000","last_unstale":"2024-04-25T10:02:03.352738+0000","last_undegraded":"2024-04-25T10:02:03.352738+0000","last_fullsized":"2024-04-25T10:02:03.352738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:48.627288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433983+0000","last_change":"2024-04-25T10:02:02.364464+0000","last_active":"2024-04-25T10:02:03.433983+0000","last_peered":"2024-04-25T10:02:03.433983+0000","last_clean":"2024-04-25T10:02:03.433983+0000","last_became_active":"2024-04-25T10:02:02.364024+0000","last_became_peered":"2024-04-25T10:02:02.364024+0000","last_unstale":"2024-04-25T10:02:03.433983+0000","last_undegraded":"2024-04-25T10:02:03.433983+0000","last_fullsized":"2024-04-25T10:02:03.433983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:39.357721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520680+0000","last_change":"2024-04-25T10:01:58.125488+0000","last_active":"2024-04-25T10:02:03.520680+0000","last_peered":"2024-04-25T10:02:03.520680+0000","last_clean":"2024-04-25T10:02:03.520680+0000","last_became_active":"2024-04-25T10:01:58.125161+0000","last_became_peered":"2024-04-25T10:01:58.125161+0000","last_unstale":"2024-04-25T10:02:03.520680+0000","last_undegraded":"2024-04-25T10:02:03.520680+0000","last_fullsized":"2024-04-25T10:02:03.520680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:10.940348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518416+0000","last_change":"2024-04-25T10:01:57.066991+0000","last_active":"2024-04-25T10:02:03.518416+0000","last_peered":"2024-04-25T10:02:03.518416+0000","last_clean":"2024-04-25T10:02:03.518416+0000","last_became_active":"2024-04-25T10:01:57.065456+0000","last_became_peered":"2024-04-25T10:01:57.065456+0000","last_unstale":"2024-04-25T10:02:03.518416+0000","last_undegraded":"2024-04-25T10:02:03.518416+0000","last_fullsized":"2024-04-25T10:02:03.518416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434030+0000","last_change":"2024-04-25T10:02:02.373840+0000","last_active":"2024-04-25T10:02:03.434030+0000","last_peered":"2024-04-25T10:02:03.434030+0000","last_clean":"2024-04-25T10:02:03.434030+0000","last_became_active":"2024-04-25T10:02:02.373426+0000","last_became_peered":"2024-04-25T10:02:02.373426+0000","last_unstale":"2024-04-25T10:02:03.434030+0000","last_undegraded":"2024-04-25T10:02:03.434030+0000","last_fullsized":"2024-04-25T10:02:03.434030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:54.471032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348267+0000","last_change":"2024-04-25T10:01:58.076731+0000","last_active":"2024-04-25T10:02:03.348267+0000","last_peered":"2024-04-25T10:02:03.348267+0000","last_clean":"2024-04-25T10:02:03.348267+0000","last_became_active":"2024-04-25T10:01:58.076430+0000","last_became_peered":"2024-04-25T10:01:58.076430+0000","last_unstale":"2024-04-25T10:02:03.348267+0000","last_undegraded":"2024-04-25T10:02:03.348267+0000","last_fullsized":"2024-04-25T10:02:03.348267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402642+0000","last_change":"2024-04-25T10:01:57.061549+0000","last_active":"2024-04-25T10:02:03.402642+0000","last_peered":"2024-04-25T10:02:03.402642+0000","last_clean":"2024-04-25T10:02:03.402642+0000","last_became_active":"2024-04-25T10:01:57.056797+0000","last_became_peered":"2024-04-25T10:01:57.056797+0000","last_unstale":"2024-04-25T10:02:03.402642+0000","last_undegraded":"2024-04-25T10:02:03.402642+0000","last_fullsized":"2024-04-25T10:02:03.402642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.994799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519398+0000","last_change":"2024-04-25T10:02:02.372845+0000","last_active":"2024-04-25T10:02:03.519398+0000","last_peered":"2024-04-25T10:02:03.519398+0000","last_clean":"2024-04-25T10:02:03.519398+0000","last_became_active":"2024-04-25T10:02:02.370004+0000","last_became_peered":"2024-04-25T10:02:02.370004+0000","last_unstale":"2024-04-25T10:02:03.519398+0000","last_undegraded":"2024-04-25T10:02:03.519398+0000","last_fullsized":"2024-04-25T10:02:03.519398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518640+0000","last_change":"2024-04-25T10:01:58.120136+0000","last_active":"2024-04-25T10:02:03.518640+0000","last_peered":"2024-04-25T10:02:03.518640+0000","last_clean":"2024-04-25T10:02:03.518640+0000","last_became_active":"2024-04-25T10:01:58.111232+0000","last_became_peered":"2024-04-25T10:01:58.111232+0000","last_unstale":"2024-04-25T10:02:03.518640+0000","last_undegraded":"2024-04-25T10:02:03.518640+0000","last_fullsized":"2024-04-25T10:02:03.518640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:17.032865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353998+0000","last_change":"2024-04-25T10:01:57.047615+0000","last_active":"2024-04-25T10:02:03.353998+0000","last_peered":"2024-04-25T10:02:03.353998+0000","last_clean":"2024-04-25T10:02:03.353998+0000","last_became_active":"2024-04-25T10:01:57.047321+0000","last_became_peered":"2024-04-25T10:01:57.047321+0000","last_unstale":"2024-04-25T10:02:03.353998+0000","last_undegraded":"2024-04-25T10:02:03.353998+0000","last_fullsized":"2024-04-25T10:02:03.353998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404368+0000","last_change":"2024-04-25T10:02:02.361172+0000","last_active":"2024-04-25T10:02:03.404368+0000","last_peered":"2024-04-25T10:02:03.404368+0000","last_clean":"2024-04-25T10:02:03.404368+0000","last_became_active":"2024-04-25T10:02:02.360349+0000","last_became_peered":"2024-04-25T10:02:02.360349+0000","last_unstale":"2024-04-25T10:02:03.404368+0000","last_undegraded":"2024-04-25T10:02:03.404368+0000","last_fullsized":"2024-04-25T10:02:03.404368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.788023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518563+0000","last_change":"2024-04-25T10:01:58.127068+0000","last_active":"2024-04-25T10:02:03.518563+0000","last_peered":"2024-04-25T10:02:03.518563+0000","last_clean":"2024-04-25T10:02:03.518563+0000","last_became_active":"2024-04-25T10:01:58.126258+0000","last_became_peered":"2024-04-25T10:01:58.126258+0000","last_unstale":"2024-04-25T10:02:03.518563+0000","last_undegraded":"2024-04-25T10:02:03.518563+0000","last_fullsized":"2024-04-25T10:02:03.518563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:15.775667+0000","last_change":"2024-04-25T10:01:57.054259+0000","last_active":"2024-04-25T10:02:15.775667+0000","last_peered":"2024-04-25T10:02:15.775667+0000","last_clean":"2024-04-25T10:02:15.775667+0000","last_became_active":"2024-04-25T10:01:57.053941+0000","last_became_peered":"2024-04-25T10:01:57.053941+0000","last_unstale":"2024-04-25T10:02:15.775667+0000","last_undegraded":"2024-04-25T10:02:15.775667+0000","last_fullsized":"2024-04-25T10:02:15.775667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:41.210591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354900+0000","last_change":"2024-04-25T10:02:02.376549+0000","last_active":"2024-04-25T10:02:03.354900+0000","last_peered":"2024-04-25T10:02:03.354900+0000","last_clean":"2024-04-25T10:02:03.354900+0000","last_became_active":"2024-04-25T10:02:02.376052+0000","last_became_peered":"2024-04-25T10:02:02.376052+0000","last_unstale":"2024-04-25T10:02:03.354900+0000","last_undegraded":"2024-04-25T10:02:03.354900+0000","last_fullsized":"2024-04-25T10:02:03.354900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:43.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.403613+0000","last_change":"2024-04-25T10:01:58.078715+0000","last_active":"2024-04-25T10:02:03.403613+0000","last_peered":"2024-04-25T10:02:03.403613+0000","last_clean":"2024-04-25T10:02:03.403613+0000","last_became_active":"2024-04-25T10:01:58.078363+0000","last_became_peered":"2024-04-25T10:01:58.078363+0000","last_unstale":"2024-04-25T10:02:03.403613+0000","last_undegraded":"2024-04-25T10:02:03.403613+0000","last_fullsized":"2024-04-25T10:02:03.403613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:25.310967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354395+0000","last_change":"2024-04-25T10:01:57.054914+0000","last_active":"2024-04-25T10:02:03.354395+0000","last_peered":"2024-04-25T10:02:03.354395+0000","last_clean":"2024-04-25T10:02:03.354395+0000","last_became_active":"2024-04-25T10:01:57.054640+0000","last_became_peered":"2024-04-25T10:01:57.054640+0000","last_unstale":"2024-04-25T10:02:03.354395+0000","last_undegraded":"2024-04-25T10:02:03.354395+0000","last_fullsized":"2024-04-25T10:02:03.354395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522376+0000","last_change":"2024-04-25T10:02:02.362949+0000","last_active":"2024-04-25T10:02:03.522376+0000","last_peered":"2024-04-25T10:02:03.522376+0000","last_clean":"2024-04-25T10:02:03.522376+0000","last_became_active":"2024-04-25T10:02:02.361660+0000","last_became_peered":"2024-04-25T10:02:02.361660+0000","last_unstale":"2024-04-25T10:02:03.522376+0000","last_undegraded":"2024-04-25T10:02:03.522376+0000","last_fullsized":"2024-04-25T10:02:03.522376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434111+0000","last_change":"2024-04-25T10:01:58.114459+0000","last_active":"2024-04-25T10:02:03.434111+0000","last_peered":"2024-04-25T10:02:03.434111+0000","last_clean":"2024-04-25T10:02:03.434111+0000","last_became_active":"2024-04-25T10:01:58.113079+0000","last_became_peered":"2024-04-25T10:01:58.113079+0000","last_unstale":"2024-04-25T10:02:03.434111+0000","last_undegraded":"2024-04-25T10:02:03.434111+0000","last_fullsized":"2024-04-25T10:02:03.434111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518456+0000","last_change":"2024-04-25T10:01:57.058749+0000","last_active":"2024-04-25T10:02:03.518456+0000","last_peered":"2024-04-25T10:02:03.518456+0000","last_clean":"2024-04-25T10:02:03.518456+0000","last_became_active":"2024-04-25T10:01:57.058498+0000","last_became_peered":"2024-04-25T10:01:57.058498+0000","last_unstale":"2024-04-25T10:02:03.518456+0000","last_undegraded":"2024-04-25T10:02:03.518456+0000","last_fullsized":"2024-04-25T10:02:03.518456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352717+0000","last_change":"2024-04-25T10:02:02.354682+0000","last_active":"2024-04-25T10:02:03.352717+0000","last_peered":"2024-04-25T10:02:03.352717+0000","last_clean":"2024-04-25T10:02:03.352717+0000","last_became_active":"2024-04-25T10:02:02.353840+0000","last_became_peered":"2024-04-25T10:02:02.353840+0000","last_unstale":"2024-04-25T10:02:03.352717+0000","last_undegraded":"2024-04-25T10:02:03.352717+0000","last_fullsized":"2024-04-25T10:02:03.352717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.306270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521836+0000","last_change":"2024-04-25T10:01:58.075136+0000","last_active":"2024-04-25T10:02:03.521836+0000","last_peered":"2024-04-25T10:02:03.521836+0000","last_clean":"2024-04-25T10:02:03.521836+0000","last_became_active":"2024-04-25T10:01:58.074823+0000","last_became_peered":"2024-04-25T10:01:58.074823+0000","last_unstale":"2024-04-25T10:02:03.521836+0000","last_undegraded":"2024-04-25T10:02:03.521836+0000","last_fullsized":"2024-04-25T10:02:03.521836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524041+0000","last_change":"2024-04-25T10:01:57.058387+0000","last_active":"2024-04-25T10:02:03.524041+0000","last_peered":"2024-04-25T10:02:03.524041+0000","last_clean":"2024-04-25T10:02:03.524041+0000","last_became_active":"2024-04-25T10:01:57.057994+0000","last_became_peered":"2024-04-25T10:01:57.057994+0000","last_unstale":"2024-04-25T10:02:03.524041+0000","last_undegraded":"2024-04-25T10:02:03.524041+0000","last_fullsized":"2024-04-25T10:02:03.524041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.171109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353666+0000","last_change":"2024-04-25T10:02:02.374647+0000","last_active":"2024-04-25T10:02:03.353666+0000","last_peered":"2024-04-25T10:02:03.353666+0000","last_clean":"2024-04-25T10:02:03.353666+0000","last_became_active":"2024-04-25T10:02:02.374168+0000","last_became_peered":"2024-04-25T10:02:02.374168+0000","last_unstale":"2024-04-25T10:02:03.353666+0000","last_undegraded":"2024-04-25T10:02:03.353666+0000","last_fullsized":"2024-04-25T10:02:03.353666+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:13.963774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356050+0000","last_change":"2024-04-25T10:02:02.374581+0000","last_active":"2024-04-25T10:02:03.356050+0000","last_peered":"2024-04-25T10:02:03.356050+0000","last_clean":"2024-04-25T10:02:03.356050+0000","last_became_active":"2024-04-25T10:02:02.373942+0000","last_became_peered":"2024-04-25T10:02:02.373942+0000","last_unstale":"2024-04-25T10:02:03.356050+0000","last_undegraded":"2024-04-25T10:02:03.356050+0000","last_fullsized":"2024-04-25T10:02:03.356050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.252346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353803+0000","last_change":"2024-04-25T10:01:58.111477+0000","last_active":"2024-04-25T10:02:03.353803+0000","last_peered":"2024-04-25T10:02:03.353803+0000","last_clean":"2024-04-25T10:02:03.353803+0000","last_became_active":"2024-04-25T10:01:58.111095+0000","last_became_peered":"2024-04-25T10:01:58.111095+0000","last_unstale":"2024-04-25T10:02:03.353803+0000","last_undegraded":"2024-04-25T10:02:03.353803+0000","last_fullsized":"2024-04-25T10:02:03.353803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:42.787884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347909+0000","last_change":"2024-04-25T10:01:57.058954+0000","last_active":"2024-04-25T10:02:03.347909+0000","last_peered":"2024-04-25T10:02:03.347909+0000","last_clean":"2024-04-25T10:02:03.347909+0000","last_became_active":"2024-04-25T10:01:57.056146+0000","last_became_peered":"2024-04-25T10:01:57.056146+0000","last_unstale":"2024-04-25T10:02:03.347909+0000","last_undegraded":"2024-04-25T10:02:03.347909+0000","last_fullsized":"2024-04-25T10:02:03.347909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404389+0000","last_change":"2024-04-25T10:02:02.360724+0000","last_active":"2024-04-25T10:02:03.404389+0000","last_peered":"2024-04-25T10:02:03.404389+0000","last_clean":"2024-04-25T10:02:03.404389+0000","last_became_active":"2024-04-25T10:02:02.360282+0000","last_became_peered":"2024-04-25T10:02:02.360282+0000","last_unstale":"2024-04-25T10:02:03.404389+0000","last_undegraded":"2024-04-25T10:02:03.404389+0000","last_fullsized":"2024-04-25T10:02:03.404389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:48.141173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432340+0000","last_change":"2024-04-25T10:01:58.126960+0000","last_active":"2024-04-25T10:02:03.432340+0000","last_peered":"2024-04-25T10:02:03.432340+0000","last_clean":"2024-04-25T10:02:03.432340+0000","last_became_active":"2024-04-25T10:01:58.126638+0000","last_became_peered":"2024-04-25T10:01:58.126638+0000","last_unstale":"2024-04-25T10:02:03.432340+0000","last_undegraded":"2024-04-25T10:02:03.432340+0000","last_fullsized":"2024-04-25T10:02:03.432340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518686+0000","last_change":"2024-04-25T10:01:57.056000+0000","last_active":"2024-04-25T10:02:03.518686+0000","last_peered":"2024-04-25T10:02:03.518686+0000","last_clean":"2024-04-25T10:02:03.518686+0000","last_became_active":"2024-04-25T10:01:57.055665+0000","last_became_peered":"2024-04-25T10:01:57.055665+0000","last_unstale":"2024-04-25T10:02:03.518686+0000","last_undegraded":"2024-04-25T10:02:03.518686+0000","last_fullsized":"2024-04-25T10:02:03.518686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402377+0000","last_change":"2024-04-25T10:01:57.064311+0000","last_active":"2024-04-25T10:02:03.402377+0000","last_peered":"2024-04-25T10:02:03.402377+0000","last_clean":"2024-04-25T10:02:03.402377+0000","last_became_active":"2024-04-25T10:01:57.063955+0000","last_became_peered":"2024-04-25T10:01:57.063955+0000","last_unstale":"2024-04-25T10:02:03.402377+0000","last_undegraded":"2024-04-25T10:02:03.402377+0000","last_fullsized":"2024-04-25T10:02:03.402377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353034+0000","last_change":"2024-04-25T10:01:58.112383+0000","last_active":"2024-04-25T10:02:03.353034+0000","last_peered":"2024-04-25T10:02:03.353034+0000","last_clean":"2024-04-25T10:02:03.353034+0000","last_became_active":"2024-04-25T10:01:58.111906+0000","last_became_peered":"2024-04-25T10:01:58.111906+0000","last_unstale":"2024-04-25T10:02:03.353034+0000","last_undegraded":"2024-04-25T10:02:03.353034+0000","last_fullsized":"2024-04-25T10:02:03.353034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:58.408032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355526+0000","last_change":"2024-04-25T10:02:02.370310+0000","last_active":"2024-04-25T10:02:03.355526+0000","last_peered":"2024-04-25T10:02:03.355526+0000","last_clean":"2024-04-25T10:02:03.355526+0000","last_became_active":"2024-04-25T10:02:02.369660+0000","last_became_peered":"2024-04-25T10:02:02.369660+0000","last_unstale":"2024-04-25T10:02:03.355526+0000","last_undegraded":"2024-04-25T10:02:03.355526+0000","last_fullsized":"2024-04-25T10:02:03.355526+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:59.232453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433292+0000","last_change":"2024-04-25T10:01:57.059797+0000","last_active":"2024-04-25T10:02:03.433292+0000","last_peered":"2024-04-25T10:02:03.433292+0000","last_clean":"2024-04-25T10:02:03.433292+0000","last_became_active":"2024-04-25T10:01:57.059483+0000","last_became_peered":"2024-04-25T10:01:57.059483+0000","last_unstale":"2024-04-25T10:02:03.433292+0000","last_undegraded":"2024-04-25T10:02:03.433292+0000","last_fullsized":"2024-04-25T10:02:03.433292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521515+0000","last_change":"2024-04-25T10:01:58.118406+0000","last_active":"2024-04-25T10:02:03.521515+0000","last_peered":"2024-04-25T10:02:03.521515+0000","last_clean":"2024-04-25T10:02:03.521515+0000","last_became_active":"2024-04-25T10:01:58.118066+0000","last_became_peered":"2024-04-25T10:01:58.118066+0000","last_unstale":"2024-04-25T10:02:03.521515+0000","last_undegraded":"2024-04-25T10:02:03.521515+0000","last_fullsized":"2024-04-25T10:02:03.521515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356020+0000","last_change":"2024-04-25T10:02:02.360301+0000","last_active":"2024-04-25T10:02:03.356020+0000","last_peered":"2024-04-25T10:02:03.356020+0000","last_clean":"2024-04-25T10:02:03.356020+0000","last_became_active":"2024-04-25T10:02:02.359873+0000","last_became_peered":"2024-04-25T10:02:02.359873+0000","last_unstale":"2024-04-25T10:02:03.356020+0000","last_undegraded":"2024-04-25T10:02:03.356020+0000","last_fullsized":"2024-04-25T10:02:03.356020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:58.571611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519243+0000","last_change":"2024-04-25T10:01:58.119106+0000","last_active":"2024-04-25T10:02:03.519243+0000","last_peered":"2024-04-25T10:02:03.519243+0000","last_clean":"2024-04-25T10:02:03.519243+0000","last_became_active":"2024-04-25T10:01:58.118822+0000","last_became_peered":"2024-04-25T10:01:58.118822+0000","last_unstale":"2024-04-25T10:02:03.519243+0000","last_undegraded":"2024-04-25T10:02:03.519243+0000","last_fullsized":"2024-04-25T10:02:03.519243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352667+0000","last_change":"2024-04-25T10:01:58.080145+0000","last_active":"2024-04-25T10:02:03.352667+0000","last_peered":"2024-04-25T10:02:03.352667+0000","last_clean":"2024-04-25T10:02:03.352667+0000","last_became_active":"2024-04-25T10:01:58.079388+0000","last_became_peered":"2024-04-25T10:01:58.079388+0000","last_unstale":"2024-04-25T10:02:03.352667+0000","last_undegraded":"2024-04-25T10:02:03.352667+0000","last_fullsized":"2024-04-25T10:02:03.352667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347995+0000","last_change":"2024-04-25T10:01:58.118742+0000","last_active":"2024-04-25T10:02:03.347995+0000","last_peered":"2024-04-25T10:02:03.347995+0000","last_clean":"2024-04-25T10:02:03.347995+0000","last_became_active":"2024-04-25T10:01:58.118407+0000","last_became_peered":"2024-04-25T10:01:58.118407+0000","last_unstale":"2024-04-25T10:02:03.347995+0000","last_undegraded":"2024-04-25T10:02:03.347995+0000","last_fullsized":"2024-04-25T10:02:03.347995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.311295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432945+0000","last_change":"2024-04-25T10:01:58.129321+0000","last_active":"2024-04-25T10:02:03.432945+0000","last_peered":"2024-04-25T10:02:03.432945+0000","last_clean":"2024-04-25T10:02:03.432945+0000","last_became_active":"2024-04-25T10:01:58.128642+0000","last_became_peered":"2024-04-25T10:01:58.128642+0000","last_unstale":"2024-04-25T10:02:03.432945+0000","last_undegraded":"2024-04-25T10:02:03.432945+0000","last_fullsized":"2024-04-25T10:02:03.432945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519722+0000","last_change":"2024-04-25T10:01:58.114750+0000","last_active":"2024-04-25T10:02:03.519722+0000","last_peered":"2024-04-25T10:02:03.519722+0000","last_clean":"2024-04-25T10:02:03.519722+0000","last_became_active":"2024-04-25T10:01:58.114410+0000","last_became_peered":"2024-04-25T10:01:58.114410+0000","last_unstale":"2024-04-25T10:02:03.519722+0000","last_undegraded":"2024-04-25T10:02:03.519722+0000","last_fullsized":"2024-04-25T10:02:03.519722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:27.145938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519712+0000","last_change":"2024-04-25T10:01:58.116458+0000","last_active":"2024-04-25T10:02:03.519712+0000","last_peered":"2024-04-25T10:02:03.519712+0000","last_clean":"2024-04-25T10:02:03.519712+0000","last_became_active":"2024-04-25T10:01:58.116051+0000","last_became_peered":"2024-04-25T10:01:58.116051+0000","last_unstale":"2024-04-25T10:02:03.519712+0000","last_undegraded":"2024-04-25T10:02:03.519712+0000","last_fullsized":"2024-04-25T10:02:03.519712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521818+0000","last_change":"2024-04-25T10:01:58.079855+0000","last_active":"2024-04-25T10:02:03.521818+0000","last_peered":"2024-04-25T10:02:03.521818+0000","last_clean":"2024-04-25T10:02:03.521818+0000","last_became_active":"2024-04-25T10:01:58.079522+0000","last_became_peered":"2024-04-25T10:01:58.079522+0000","last_unstale":"2024-04-25T10:02:03.521818+0000","last_undegraded":"2024-04-25T10:02:03.521818+0000","last_fullsized":"2024-04-25T10:02:03.521818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433705+0000","last_change":"2024-04-25T10:01:58.114808+0000","last_active":"2024-04-25T10:02:03.433705+0000","last_peered":"2024-04-25T10:02:03.433705+0000","last_clean":"2024-04-25T10:02:03.433705+0000","last_became_active":"2024-04-25T10:01:58.114656+0000","last_became_peered":"2024-04-25T10:01:58.114656+0000","last_unstale":"2024-04-25T10:02:03.433705+0000","last_undegraded":"2024-04-25T10:02:03.433705+0000","last_fullsized":"2024-04-25T10:02:03.433705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347899+0000","last_change":"2024-04-25T10:01:58.125563+0000","last_active":"2024-04-25T10:02:03.347899+0000","last_peered":"2024-04-25T10:02:03.347899+0000","last_clean":"2024-04-25T10:02:03.347899+0000","last_became_active":"2024-04-25T10:01:58.125071+0000","last_became_peered":"2024-04-25T10:01:58.125071+0000","last_unstale":"2024-04-25T10:02:03.347899+0000","last_undegraded":"2024-04-25T10:02:03.347899+0000","last_fullsized":"2024-04-25T10:02:03.347899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432367+0000","last_change":"2024-04-25T10:01:58.111903+0000","last_active":"2024-04-25T10:02:03.432367+0000","last_peered":"2024-04-25T10:02:03.432367+0000","last_clean":"2024-04-25T10:02:03.432367+0000","last_became_active":"2024-04-25T10:01:58.111366+0000","last_became_peered":"2024-04-25T10:01:58.111366+0000","last_unstale":"2024-04-25T10:02:03.432367+0000","last_undegraded":"2024-04-25T10:02:03.432367+0000","last_fullsized":"2024-04-25T10:02:03.432367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521208+0000","last_change":"2024-04-25T10:01:58.066982+0000","last_active":"2024-04-25T10:02:03.521208+0000","last_peered":"2024-04-25T10:02:03.521208+0000","last_clean":"2024-04-25T10:02:03.521208+0000","last_became_active":"2024-04-25T10:01:58.066582+0000","last_became_peered":"2024-04-25T10:01:58.066582+0000","last_unstale":"2024-04-25T10:02:03.521208+0000","last_undegraded":"2024-04-25T10:02:03.521208+0000","last_fullsized":"2024-04-25T10:02:03.521208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352779+0000","last_change":"2024-04-25T10:01:58.112990+0000","last_active":"2024-04-25T10:02:03.352779+0000","last_peered":"2024-04-25T10:02:03.352779+0000","last_clean":"2024-04-25T10:02:03.352779+0000","last_became_active":"2024-04-25T10:01:58.112724+0000","last_became_peered":"2024-04-25T10:01:58.112724+0000","last_unstale":"2024-04-25T10:02:03.352779+0000","last_undegraded":"2024-04-25T10:02:03.352779+0000","last_fullsized":"2024-04-25T10:02:03.352779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:36.503846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519412+0000","last_change":"2024-04-25T10:01:58.067953+0000","last_active":"2024-04-25T10:02:03.519412+0000","last_peered":"2024-04-25T10:02:03.519412+0000","last_clean":"2024-04-25T10:02:03.519412+0000","last_became_active":"2024-04-25T10:01:58.067363+0000","last_became_peered":"2024-04-25T10:01:58.067363+0000","last_unstale":"2024-04-25T10:02:03.519412+0000","last_undegraded":"2024-04-25T10:02:03.519412+0000","last_fullsized":"2024-04-25T10:02:03.519412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404814+0000","last_change":"2024-04-25T10:01:58.112476+0000","last_active":"2024-04-25T10:02:03.404814+0000","last_peered":"2024-04-25T10:02:03.404814+0000","last_clean":"2024-04-25T10:02:03.404814+0000","last_became_active":"2024-04-25T10:01:58.112177+0000","last_became_peered":"2024-04-25T10:01:58.112177+0000","last_unstale":"2024-04-25T10:02:03.404814+0000","last_undegraded":"2024-04-25T10:02:03.404814+0000","last_fullsized":"2024-04-25T10:02:03.404814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:50.172245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355444+0000","last_change":"2024-04-25T10:01:58.056526+0000","last_active":"2024-04-25T10:02:03.355444+0000","last_peered":"2024-04-25T10:02:03.355444+0000","last_clean":"2024-04-25T10:02:03.355444+0000","last_became_active":"2024-04-25T10:01:58.055509+0000","last_became_peered":"2024-04-25T10:01:58.055509+0000","last_unstale":"2024-04-25T10:02:03.355444+0000","last_undegraded":"2024-04-25T10:02:03.355444+0000","last_fullsized":"2024-04-25T10:02:03.355444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405090+0000","last_change":"2024-04-25T10:01:58.079032+0000","last_active":"2024-04-25T10:02:03.405090+0000","last_peered":"2024-04-25T10:02:03.405090+0000","last_clean":"2024-04-25T10:02:03.405090+0000","last_became_active":"2024-04-25T10:01:58.077635+0000","last_became_peered":"2024-04-25T10:01:58.077635+0000","last_unstale":"2024-04-25T10:02:03.405090+0000","last_undegraded":"2024-04-25T10:02:03.405090+0000","last_fullsized":"2024-04-25T10:02:03.405090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:00.440369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347561+0000","last_change":"2024-04-25T10:01:58.076733+0000","last_active":"2024-04-25T10:02:03.347561+0000","last_peered":"2024-04-25T10:02:03.347561+0000","last_clean":"2024-04-25T10:02:03.347561+0000","last_became_active":"2024-04-25T10:01:58.076476+0000","last_became_peered":"2024-04-25T10:01:58.076476+0000","last_unstale":"2024-04-25T10:02:03.347561+0000","last_undegraded":"2024-04-25T10:02:03.347561+0000","last_fullsized":"2024-04-25T10:02:03.347561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432026+0000","last_change":"2024-04-25T10:01:58.116156+0000","last_active":"2024-04-25T10:02:03.432026+0000","last_peered":"2024-04-25T10:02:03.432026+0000","last_clean":"2024-04-25T10:02:03.432026+0000","last_became_active":"2024-04-25T10:01:58.115483+0000","last_became_peered":"2024-04-25T10:01:58.115483+0000","last_unstale":"2024-04-25T10:02:03.432026+0000","last_undegraded":"2024-04-25T10:02:03.432026+0000","last_fullsized":"2024-04-25T10:02:03.432026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518554+0000","last_change":"2024-04-25T10:01:58.120123+0000","last_active":"2024-04-25T10:02:03.518554+0000","last_peered":"2024-04-25T10:02:03.518554+0000","last_clean":"2024-04-25T10:02:03.518554+0000","last_became_active":"2024-04-25T10:01:58.119852+0000","last_became_peered":"2024-04-25T10:01:58.119852+0000","last_unstale":"2024-04-25T10:02:03.518554+0000","last_undegraded":"2024-04-25T10:02:03.518554+0000","last_fullsized":"2024-04-25T10:02:03.518554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519056+0000","last_change":"2024-04-25T10:01:58.113249+0000","last_active":"2024-04-25T10:02:03.519056+0000","last_peered":"2024-04-25T10:02:03.519056+0000","last_clean":"2024-04-25T10:02:03.519056+0000","last_became_active":"2024-04-25T10:01:58.112298+0000","last_became_peered":"2024-04-25T10:01:58.112298+0000","last_unstale":"2024-04-25T10:02:03.519056+0000","last_undegraded":"2024-04-25T10:02:03.519056+0000","last_fullsized":"2024-04-25T10:02:03.519056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519272+0000","last_change":"2024-04-25T10:01:58.072097+0000","last_active":"2024-04-25T10:02:03.519272+0000","last_peered":"2024-04-25T10:02:03.519272+0000","last_clean":"2024-04-25T10:02:03.519272+0000","last_became_active":"2024-04-25T10:01:58.071662+0000","last_became_peered":"2024-04-25T10:01:58.071662+0000","last_unstale":"2024-04-25T10:02:03.519272+0000","last_undegraded":"2024-04-25T10:02:03.519272+0000","last_fullsized":"2024-04-25T10:02:03.519272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518405+0000","last_change":"2024-04-25T10:01:58.073877+0000","last_active":"2024-04-25T10:02:03.518405+0000","last_peered":"2024-04-25T10:02:03.518405+0000","last_clean":"2024-04-25T10:02:03.518405+0000","last_became_active":"2024-04-25T10:01:58.073554+0000","last_became_peered":"2024-04-25T10:01:58.073554+0000","last_unstale":"2024-04-25T10:02:03.518405+0000","last_undegraded":"2024-04-25T10:02:03.518405+0000","last_fullsized":"2024-04-25T10:02:03.518405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518311+0000","last_change":"2024-04-25T10:01:58.066768+0000","last_active":"2024-04-25T10:02:03.518311+0000","last_peered":"2024-04-25T10:02:03.518311+0000","last_clean":"2024-04-25T10:02:03.518311+0000","last_became_active":"2024-04-25T10:01:58.065944+0000","last_became_peered":"2024-04-25T10:01:58.065944+0000","last_unstale":"2024-04-25T10:02:03.518311+0000","last_undegraded":"2024-04-25T10:02:03.518311+0000","last_fullsized":"2024-04-25T10:02:03.518311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524004+0000","last_change":"2024-04-25T10:01:58.066685+0000","last_active":"2024-04-25T10:02:03.524004+0000","last_peered":"2024-04-25T10:02:03.524004+0000","last_clean":"2024-04-25T10:02:03.524004+0000","last_became_active":"2024-04-25T10:01:58.066367+0000","last_became_peered":"2024-04-25T10:01:58.066367+0000","last_unstale":"2024-04-25T10:02:03.524004+0000","last_undegraded":"2024-04-25T10:02:03.524004+0000","last_fullsized":"2024-04-25T10:02:03.524004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432058+0000","last_change":"2024-04-25T10:01:58.123950+0000","last_active":"2024-04-25T10:02:03.432058+0000","last_peered":"2024-04-25T10:02:03.432058+0000","last_clean":"2024-04-25T10:02:03.432058+0000","last_became_active":"2024-04-25T10:01:58.123668+0000","last_became_peered":"2024-04-25T10:01:58.123668+0000","last_unstale":"2024-04-25T10:02:03.432058+0000","last_undegraded":"2024-04-25T10:02:03.432058+0000","last_fullsized":"2024-04-25T10:02:03.432058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:30.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518202+0000","last_change":"2024-04-25T10:01:58.114958+0000","last_active":"2024-04-25T10:02:03.518202+0000","last_peered":"2024-04-25T10:02:03.518202+0000","last_clean":"2024-04-25T10:02:03.518202+0000","last_became_active":"2024-04-25T10:01:58.114599+0000","last_became_peered":"2024-04-25T10:01:58.114599+0000","last_unstale":"2024-04-25T10:02:03.518202+0000","last_undegraded":"2024-04-25T10:02:03.518202+0000","last_fullsized":"2024-04-25T10:02:03.518202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521335+0000","last_change":"2024-04-25T10:01:58.116405+0000","last_active":"2024-04-25T10:02:03.521335+0000","last_peered":"2024-04-25T10:02:03.521335+0000","last_clean":"2024-04-25T10:02:03.521335+0000","last_became_active":"2024-04-25T10:01:58.115946+0000","last_became_peered":"2024-04-25T10:01:58.115946+0000","last_unstale":"2024-04-25T10:02:03.521335+0000","last_undegraded":"2024-04-25T10:02:03.521335+0000","last_fullsized":"2024-04-25T10:02:03.521335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:57.669295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518323+0000","last_change":"2024-04-25T10:01:58.117287+0000","last_active":"2024-04-25T10:02:03.518323+0000","last_peered":"2024-04-25T10:02:03.518323+0000","last_clean":"2024-04-25T10:02:03.518323+0000","last_became_active":"2024-04-25T10:01:58.117024+0000","last_became_peered":"2024-04-25T10:01:58.117024+0000","last_unstale":"2024-04-25T10:02:03.518323+0000","last_undegraded":"2024-04-25T10:02:03.518323+0000","last_fullsized":"2024-04-25T10:02:03.518323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522601+0000","last_change":"2024-04-25T10:01:58.116367+0000","last_active":"2024-04-25T10:02:03.522601+0000","last_peered":"2024-04-25T10:02:03.522601+0000","last_clean":"2024-04-25T10:02:03.522601+0000","last_became_active":"2024-04-25T10:01:58.115801+0000","last_became_peered":"2024-04-25T10:01:58.115801+0000","last_unstale":"2024-04-25T10:02:03.522601+0000","last_undegraded":"2024-04-25T10:02:03.522601+0000","last_fullsized":"2024-04-25T10:02:03.522601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433436+0000","last_change":"2024-04-25T10:01:58.118411+0000","last_active":"2024-04-25T10:02:03.433436+0000","last_peered":"2024-04-25T10:02:03.433436+0000","last_clean":"2024-04-25T10:02:03.433436+0000","last_became_active":"2024-04-25T10:01:58.118131+0000","last_became_peered":"2024-04-25T10:01:58.118131+0000","last_unstale":"2024-04-25T10:02:03.433436+0000","last_undegraded":"2024-04-25T10:02:03.433436+0000","last_fullsized":"2024-04-25T10:02:03.433436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433580+0000","last_change":"2024-04-25T10:01:58.121692+0000","last_active":"2024-04-25T10:02:03.433580+0000","last_peered":"2024-04-25T10:02:03.433580+0000","last_clean":"2024-04-25T10:02:03.433580+0000","last_became_active":"2024-04-25T10:01:58.120392+0000","last_became_peered":"2024-04-25T10:01:58.120392+0000","last_unstale":"2024-04-25T10:02:03.433580+0000","last_undegraded":"2024-04-25T10:02:03.433580+0000","last_fullsized":"2024-04-25T10:02:03.433580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520287+0000","last_change":"2024-04-25T10:01:58.080365+0000","last_active":"2024-04-25T10:02:03.520287+0000","last_peered":"2024-04-25T10:02:03.520287+0000","last_clean":"2024-04-25T10:02:03.520287+0000","last_became_active":"2024-04-25T10:01:58.080055+0000","last_became_peered":"2024-04-25T10:01:58.080055+0000","last_unstale":"2024-04-25T10:02:03.520287+0000","last_undegraded":"2024-04-25T10:02:03.520287+0000","last_fullsized":"2024-04-25T10:02:03.520287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:26.452 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:26.453 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:02:26.453 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:02:26.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:02:26.453 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:02:26.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.704+0000 7ff6b3200640 1 Processor -- start 2024-04-25T10:02:26.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.705+0000 7ff6b3200640 1 -- start start 2024-04-25T10:02:26.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.706+0000 7ff6b3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff6b40d07d0 0x7ff6b40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.706+0000 7ff6b3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b40cf410 con 0x7ff6b4161200 2024-04-25T10:02:26.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.706+0000 7ff6b3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b40cf6f0 con 0x7ff6b40d2380 2024-04-25T10:02:26.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.706+0000 7ff6b3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b40b28b0 con 0x7ff6b40d07d0 2024-04-25T10:02:26.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b1e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff6b4161200 0x7ff6b40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59832/0 (socket says 172.21.15.5:59832) 2024-04-25T10:02:26.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b1e00640 1 -- 172.21.15.5:0/2520835599 learned_addr learned my addr 172.21.15.5:0/2520835599 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:26.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b1400640 1 --2- 172.21.15.5:0/2520835599 >> v2:172.21.15.80:3300/0 conn(0x7ff6b40d07d0 0x7ff6b40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932626606 0 0) 0x7ff6b40cf410 con 0x7ff6b4161200 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff698003520 con 0x7ff6b4161200 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3272666488 0 0) 0x7ff698003520 con 0x7ff6b4161200 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 >> v2:172.21.15.80:3300/0 conn(0x7ff6b40d07d0 msgr2=0x7ff6b40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 --2- 172.21.15.5:0/2520835599 >> v2:172.21.15.80:3300/0 conn(0x7ff6b40d07d0 0x7ff6b40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 >> v1:172.21.15.80:6789/0 conn(0x7ff6b40d2380 legacy=0x7ff6b40b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.707+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b40d3fd0 con 0x7ff6b4161200 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7ff6a4001670 con 0x7ff6b4161200 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 >> v1:172.21.15.5:6789/0 conn(0x7ff6b4161200 legacy=0x7ff6b40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980039b0 0x7ff698003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff698004800 0x7ff698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6980051c0 0x7ff69800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40b28b0 con 0x7ff6980051c0 2024-04-25T10:02:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69800c060 con 0x7ff6980039b0 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69800c260 con 0x7ff698004800 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b2800640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6980051c0 0x7ff69800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b1400640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff698004800 0x7ff698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.708+0000 7ff6b1e00640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980039b0 0x7ff698003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b2800640 1 -- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff698004800 msgr2=0x7ff698004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b2800640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff698004800 0x7ff698004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b2800640 1 -- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980039b0 msgr2=0x7ff698003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b2800640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980039b0 0x7ff698003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b2800640 1 -- 172.21.15.5:0/2520835599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698003520 con 0x7ff6980051c0 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b1e00640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980039b0 0x7ff698003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.709+0000 7ff6b2800640 1 -- 172.21.15.5:0/2520835599 reap_dead start 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.710+0000 7ff6b2800640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6980051c0 0x7ff69800b9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac001e50 tx=0x7ff698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1e12bbda4a5b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.710+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac001a70 con 0x7ff6980051c0 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.710+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac003970 con 0x7ff6980051c0 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.710+0000 7ff6b0a00640 1 -- 172.21.15.5:0/2520835599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac003d60 con 0x7ff6980051c0 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 -- 172.21.15.5:0/2520835599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6980051c0 msgr2=0x7ff69800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6980051c0 0x7ff69800b9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac001e50 tx=0x7ff698005830 comp rx=0 tx=0).stop 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 -- 172.21.15.5:0/2520835599 shutdown_connections 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 --2- 172.21.15.5:0/2520835599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6980051c0 0x7ff69800b9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 -- 172.21.15.5:0/2520835599 >> 172.21.15.5:0/2520835599 conn(0x7ff6b408af80 msgr2=0x7ff6b40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 -- 172.21.15.5:0/2520835599 shutdown_connections 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.711+0000 7ff6b3200640 1 -- 172.21.15.5:0/2520835599 wait complete. 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.713+0000 7ff6b3200640 1 Processor -- start 2024-04-25T10:02:26.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.713+0000 7ff6b3200640 1 -- start start 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980051c0 0x7ff6b4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6b40b83f0 0x7ff6b40b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 0x7ff6b409eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40b4d30 con 0x7ff6b40b6b30 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40b4bb0 con 0x7ff6980051c0 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b40b4eb0 con 0x7ff6b40b83f0 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 0x7ff6b409eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 0x7ff6b409eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46808/0 (socket says 172.21.15.5:46808) 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b2800640 1 -- 172.21.15.5:0/3541253688 learned_addr learned my addr 172.21.15.5:0/3541253688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b1400640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6b40b83f0 0x7ff6b40b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b1e00640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980051c0 0x7ff6b4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b2800640 1 -- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6b40b83f0 msgr2=0x7ff6b40b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.714+0000 7ff6b2800640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6b40b83f0 0x7ff6b40b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b2800640 1 -- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980051c0 msgr2=0x7ff6b4150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b2800640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980051c0 0x7ff6b4150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b2800640 1 -- 172.21.15.5:0/3541253688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b40b7350 con 0x7ff6b40b6b30 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b1400640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6b40b83f0 0x7ff6b40b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b2800640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 0x7ff6b409eab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02a920 tx=0x7ff6ac0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac001a70 con 0x7ff6b40b6b30 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac001250 con 0x7ff6b40b6b30 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac006b50 con 0x7ff6b40b6b30 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b40b7630 con 0x7ff6b40b6b30 2024-04-25T10:02:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.715+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b40b7bf0 con 0x7ff6b40b6b30 2024-04-25T10:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.717+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b4068f60 con 0x7ff6b40b6b30 2024-04-25T10:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.717+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6ac006cf0 con 0x7ff6b40b6b30 2024-04-25T10:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.718+0000 7ff6aa000640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff69406b4d0 0x7ff69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.719+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff6ac0a9c40 con 0x7ff6b40b6b30 2024-04-25T10:02:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.719+0000 7ff6b1e00640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff69406b4d0 0x7ff69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.720+0000 7ff6b1e00640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff69406b4d0 0x7ff69406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40017e0 tx=0x7ff6a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.723+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6ac0786e0 con 0x7ff6b40b6b30 2024-04-25T10:02:26.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.973+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff6b40d0380 con 0x7ff6b40b6b30 2024-04-25T10:02:26.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.980+0000 7ff6aa000640 1 -- 172.21.15.5:0/3541253688 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171783 (secure 0 0 0) 0x7ff6ac0a9020 con 0x7ff6b40b6b30 2024-04-25T10:02:26.987 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:26.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff69406b4d0 msgr2=0x7ff69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff69406b4d0 0x7ff69406d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40017e0 tx=0x7ff6a4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:26.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 msgr2=0x7ff6b409eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 0x7ff6b409eab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac02a920 tx=0x7ff6ac0014a0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 shutdown_connections 2024-04-25T10:02:26.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6b40b6b30 0x7ff6b409eab0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6b40b83f0 0x7ff6b40b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff69406b4d0 0x7ff69406d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.984+0000 7ff6b3200640 1 --2- 172.21.15.5:0/3541253688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6980051c0 0x7ff6b4150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.985+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 >> 172.21.15.5:0/3541253688 conn(0x7ff6b408af80 msgr2=0x7ff6b415e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.985+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 shutdown_connections 2024-04-25T10:02:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:26.985+0000 7ff6b3200640 1 -- 172.21.15.5:0/3541253688 wait complete. 2024-04-25T10:02:27.007 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"flags":0,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3633594120},{"type":"v1","addr":"172.21.15.80:6833","nonce":3633594120}]},"active_addr":"172.21.15.80:6833/3633594120","active_change":"2024-04-25T10:01:38.579438+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":223271230}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":717738954}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":1093300186}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.80:0","nonce":2508856103}]}]} 2024-04-25T10:02:27.010 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:02:27.010 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:02:27.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.209+0000 7f6bc5400640 1 Processor -- start 2024-04-25T10:02:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.210+0000 7f6bc5400640 1 -- start start 2024-04-25T10:02:27.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bc5400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6bc006e970 0x7f6bc00cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bc5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6bc015b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bc5400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc00cdbf0 con 0x7f6bc0144100 2024-04-25T10:02:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bc5400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc01444e0 con 0x7f6bc006e970 2024-04-25T10:02:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bc5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015ac70 con 0x7f6bc00ced20 2024-04-25T10:02:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bbe800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6bc0144100 0x7f6bc006e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51472/0 (socket says 172.21.15.5:51472) 2024-04-25T10:02:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.211+0000 7f6bbe800640 1 -- 172.21.15.5:0/686281752 learned_addr learned my addr 172.21.15.5:0/686281752 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:27.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbf200640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6bc015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bb7e00640 1 --2- 172.21.15.5:0/686281752 >> v2:172.21.15.80:3300/0 conn(0x7f6bc006e970 0x7f6bc00cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271658158 0 0) 0x7f6bc00cdbf0 con 0x7f6bc0144100 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba4005730 con 0x7f6bc0144100 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 46448304 0 0) 0x7f6ba4005730 con 0x7f6bc0144100 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 msgr2=0x7f6bc015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6bc015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 >> v2:172.21.15.80:3300/0 conn(0x7f6bc006e970 msgr2=0x7f6bc00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 --2- 172.21.15.5:0/686281752 >> v2:172.21.15.80:3300/0 conn(0x7f6bc006e970 0x7f6bc00cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00cded0 con 0x7f6bc0144100 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f6bb0001670 con 0x7f6bc0144100 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 >> v1:172.21.15.5:6789/0 conn(0x7f6bc0144100 legacy=0x7f6bc006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4005bc0 0x7f6ba4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6ba4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.212+0000 7f6bbde00640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ba4006f50 0x7f6ba40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc01444e0 con 0x7f6ba4005bc0 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc015ac70 con 0x7f6ba4006f50 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba400dbd0 con 0x7f6bc00ced20 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbe800640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4005bc0 0x7f6ba4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bb7e00640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6ba4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbf200640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ba4006f50 0x7f6ba40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbe800640 1 -- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 msgr2=0x7f6ba4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbe800640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6ba4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbe800640 1 -- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ba4006f50 msgr2=0x7f6ba40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbe800640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ba4006f50 0x7f6ba40073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bbe800640 1 -- 172.21.15.5:0/686281752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4005730 con 0x7f6ba4005bc0 2024-04-25T10:02:27.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.213+0000 7f6bb7e00640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6bc00ced20 0x7f6ba4006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.214+0000 7f6bbe800640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4005bc0 0x7f6ba4005fe0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c190 tx=0x7f6ba4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa55db4ccc0d9e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.214+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0007c80 con 0x7f6ba4005bc0 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.214+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb0005ad0 con 0x7f6ba4005bc0 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.214+0000 7f6bbde00640 1 -- 172.21.15.5:0/686281752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0007070 con 0x7f6ba4005bc0 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bc5400640 1 -- 172.21.15.5:0/686281752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4005bc0 msgr2=0x7f6ba4005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bc5400640 1 --2- 172.21.15.5:0/686281752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ba4005bc0 0x7f6ba4005fe0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c190 tx=0x7f6ba4007a40 comp rx=0 tx=0).stop 2024-04-25T10:02:27.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bbf200640 1 -- 172.21.15.5:0/686281752 reap_dead start 2024-04-25T10:02:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bc5400640 1 -- 172.21.15.5:0/686281752 shutdown_connections 2024-04-25T10:02:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bc5400640 1 -- 172.21.15.5:0/686281752 >> 172.21.15.5:0/686281752 conn(0x7f6bc0071150 msgr2=0x7f6bc0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bc5400640 1 -- 172.21.15.5:0/686281752 shutdown_connections 2024-04-25T10:02:27.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.215+0000 7f6bc5400640 1 -- 172.21.15.5:0/686281752 wait complete. 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.217+0000 7f6bc5400640 1 Processor -- start 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.217+0000 7f6bc5400640 1 -- start start 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.218+0000 7f6bc5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ba4005bc0 0x7f6bc01e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.218+0000 7f6bc5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 0x7f6bc00cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.218+0000 7f6bc5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6bc00cb2d0 0x7f6bc014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.218+0000 7f6bc5400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00cea00 con 0x7f6bc01e31b0 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.218+0000 7f6bc5400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00ce490 con 0x7f6bc00cb2d0 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.218+0000 7f6bc5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc006d150 con 0x7f6ba4005bc0 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 0x7f6bc00cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 0x7f6bc00cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46400/0 (socket says 172.21.15.5:46400) 2024-04-25T10:02:27.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 -- 172.21.15.5:0/2220259650 learned_addr learned my addr 172.21.15.5:0/2220259650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bbf200640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6bc00cb2d0 0x7f6bc014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bbe800640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ba4005bc0 0x7f6bc01e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 -- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ba4005bc0 msgr2=0x7f6bc01e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ba4005bc0 0x7f6bc01e2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 -- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6bc00cb2d0 msgr2=0x7f6bc014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6bc00cb2d0 0x7f6bc014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bb7e00640 1 -- 172.21.15.5:0/2220259650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc0149110 con 0x7f6bc01e31b0 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.219+0000 7f6bbf200640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6bc00cb2d0 0x7f6bc014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.220+0000 7f6bb7e00640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 0x7f6bc00cad90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80108b0 tx=0x7f6ba8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.220+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800f040 con 0x7f6bc01e31b0 2024-04-25T10:02:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.220+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba800eb60 con 0x7f6bc01e31b0 2024-04-25T10:02:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.220+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba801c400 con 0x7f6bc01e31b0 2024-04-25T10:02:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.220+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc01493f0 con 0x7f6bc01e31b0 2024-04-25T10:02:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.220+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc01dfe40 con 0x7f6bc01e31b0 2024-04-25T10:02:27.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.221+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc00cb6e0 con 0x7f6bc01e31b0 2024-04-25T10:02:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.223+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ba801c5a0 con 0x7f6bc01e31b0 2024-04-25T10:02:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.224+0000 7f6bb6a00640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b9c06b4d0 0x7f6b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.224+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ba808ab20 con 0x7f6bc01e31b0 2024-04-25T10:02:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.224+0000 7f6bbe800640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b9c06b4d0 0x7f6b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.225+0000 7f6bbe800640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b9c06b4d0 0x7f6b9c06d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0032000 tx=0x7f6bb0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.228+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bc00cb6e0 con 0x7f6bc01e31b0 2024-04-25T10:02:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.434+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bc0068fd0 con 0x7f6bc01e31b0 2024-04-25T10:02:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.437+0000 7f6bb6a00640 1 -- 172.21.15.5:0/2220259650 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f6ba8018030 con 0x7f6bc01e31b0 2024-04-25T10:02:27.443 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:27.443 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.440+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b9c06b4d0 msgr2=0x7f6b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.440+0000 7f6bc5400640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b9c06b4d0 0x7f6b9c06d990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0032000 tx=0x7f6bb0009030 comp rx=0 tx=0).stop 2024-04-25T10:02:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 msgr2=0x7f6bc00cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 0x7f6bc00cad90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80108b0 tx=0x7f6ba8010d70 comp rx=0 tx=0).stop 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 shutdown_connections 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b9c06b4d0 0x7f6b9c06d990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6bc00cb2d0 0x7f6bc014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc01e31b0 0x7f6bc00cad90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 --2- 172.21.15.5:0/2220259650 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ba4005bc0 0x7f6bc01e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 >> 172.21.15.5:0/2220259650 conn(0x7f6bc0071150 msgr2=0x7f6bc00b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 shutdown_connections 2024-04-25T10:02:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.441+0000 7f6bc5400640 1 -- 172.21.15.5:0/2220259650 wait complete. 2024-04-25T10:02:27.465 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:02:27.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:27.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.674+0000 7f898f600640 1 Processor -- start 2024-04-25T10:02:27.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.674+0000 7f898f600640 1 -- start start 2024-04-25T10:02:27.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f898f600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89880cbb50 0x7f89880ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f898f600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89880c9f90 con 0x7f89880cad50 2024-04-25T10:02:27.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f898f600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89880ca270 con 0x7f89881472c0 2024-04-25T10:02:27.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f898f600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89880ca550 con 0x7f89880cbb50 2024-04-25T10:02:27.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f898d600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f89880cad50 0x7f89880cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51486/0 (socket says 172.21.15.5:51486) 2024-04-25T10:02:27.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f898d600640 1 -- 172.21.15.5:0/2282910290 learned_addr learned my addr 172.21.15.5:0/2282910290 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:27.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.676+0000 7f897fe00640 1 --2- 172.21.15.5:0/2282910290 >> v2:172.21.15.80:3300/0 conn(0x7f89880cbb50 0x7f89880ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071286350 0 0) 0x7f89880c9f90 con 0x7f89880cad50 2024-04-25T10:02:27.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f896c003520 con 0x7f89880cad50 2024-04-25T10:02:27.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4080963166 0 0) 0x7f896c003520 con 0x7f89880cad50 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 >> v2:172.21.15.80:3300/0 conn(0x7f89880cbb50 msgr2=0x7f89880ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 --2- 172.21.15.5:0/2282910290 >> v2:172.21.15.80:3300/0 conn(0x7f89880cbb50 0x7f89880ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 >> v1:172.21.15.80:6789/0 conn(0x7f89881472c0 legacy=0x7f89880cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89880ca830 con 0x7f89880cad50 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8980001a10 con 0x7f89880cad50 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 >> v1:172.21.15.5:6789/0 conn(0x7f89880cad50 legacy=0x7f89880cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896c0039b0 0x7f896c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896c004800 0x7f896c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0052e0 0x7f896c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880ca550 con 0x7f896c0039b0 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c00c1b0 con 0x7f896c004800 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c00c3b0 con 0x7f896c0052e0 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.677+0000 7f898cc00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896c0039b0 0x7f896c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f897fe00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896c004800 0x7f896c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898d600640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0052e0 0x7f896c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898cc00640 1 -- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0052e0 msgr2=0x7f896c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898cc00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0052e0 0x7f896c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898cc00640 1 -- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896c004800 msgr2=0x7f896c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898cc00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f896c004800 0x7f896c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898cc00640 1 -- 172.21.15.5:0/2282910290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c003520 con 0x7f896c0039b0 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898d600640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0052e0 0x7f896c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.678+0000 7f898d600640 1 -- 172.21.15.5:0/2282910290 reap_dead start 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.679+0000 7f898cc00640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896c0039b0 0x7f896c003dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8978001720 tx=0x7f896c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3f4437543591fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.679+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978001370 con 0x7f896c0039b0 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.679+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89780052d0 con 0x7f896c0039b0 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.679+0000 7f8987e00640 1 -- 172.21.15.5:0/2282910290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897802ab70 con 0x7f896c0039b0 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 -- 172.21.15.5:0/2282910290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896c0039b0 msgr2=0x7f896c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896c0039b0 0x7f896c003dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8978001720 tx=0x7f896c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:27.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 -- 172.21.15.5:0/2282910290 shutdown_connections 2024-04-25T10:02:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 --2- 172.21.15.5:0/2282910290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896c0039b0 0x7f896c003dd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 -- 172.21.15.5:0/2282910290 >> 172.21.15.5:0/2282910290 conn(0x7f8988071150 msgr2=0x7f8988071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 -- 172.21.15.5:0/2282910290 shutdown_connections 2024-04-25T10:02:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.680+0000 7f898f600640 1 -- 172.21.15.5:0/2282910290 wait complete. 2024-04-25T10:02:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.682+0000 7f898f600640 1 Processor -- start 2024-04-25T10:02:27.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.682+0000 7f898f600640 1 -- start start 2024-04-25T10:02:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898f600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0039b0 0x7f89880b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898f600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898813ead0 0x7f8988144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898f600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 0x7f8988161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898f600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880cb5c0 con 0x7f8988161780 2024-04-25T10:02:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898f600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880cb440 con 0x7f898813ead0 2024-04-25T10:02:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898f600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89880cb740 con 0x7f896c0039b0 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 0x7f8988161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 0x7f8988161c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46420/0 (socket says 172.21.15.5:46420) 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.683+0000 7f898d600640 1 -- 172.21.15.5:0/1517403569 learned_addr learned my addr 172.21.15.5:0/1517403569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898cc00640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0039b0 0x7f89880b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f897fe00640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898813ead0 0x7f8988144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898d600640 1 -- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0039b0 msgr2=0x7f89880b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898d600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0039b0 0x7f89880b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898d600640 1 -- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898813ead0 msgr2=0x7f8988144010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898d600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898813ead0 0x7f8988144010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898d600640 1 -- 172.21.15.5:0/1517403569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89880b3ea0 con 0x7f8988161780 2024-04-25T10:02:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898d600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 0x7f8988161c00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f898806c580 tx=0x7f898000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980004d60 con 0x7f8988161780 2024-04-25T10:02:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898000bb80 con 0x7f8988161780 2024-04-25T10:02:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980007070 con 0x7f8988161780 2024-04-25T10:02:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.684+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89880b4180 con 0x7f8988161780 2024-04-25T10:02:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.685+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89880b4740 con 0x7f8988161780 2024-04-25T10:02:27.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.686+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8988068fd0 con 0x7f8988161780 2024-04-25T10:02:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.687+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f898001b020 con 0x7f8988161780 2024-04-25T10:02:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.688+0000 7f8985600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896406b500 0x7f896406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.688+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f898008a5b0 con 0x7f8988161780 2024-04-25T10:02:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.688+0000 7f898cc00640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896406b500 0x7f896406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.689+0000 7f898cc00640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896406b500 0x7f896406d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8978032000 tx=0x7f897802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.692+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89800590f0 con 0x7f8988161780 2024-04-25T10:02:27.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.898+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8988072390 con 0x7f8988161780 2024-04-25T10:02:27.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.902+0000 7f8985600640 1 -- 172.21.15.5:0/1517403569 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f89800589f0 con 0x7f8988161780 2024-04-25T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.904+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896406b500 msgr2=0x7f896406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.904+0000 7f898f600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896406b500 0x7f896406d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8978032000 tx=0x7f897802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 msgr2=0x7f8988161c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 0x7f8988161c00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f898806c580 tx=0x7f898000b4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 shutdown_connections 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8988161780 0x7f8988161c00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896406b500 0x7f896406d9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898813ead0 0x7f8988144010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 --2- 172.21.15.5:0/1517403569 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f896c0039b0 0x7f89880b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 >> 172.21.15.5:0/1517403569 conn(0x7f8988071150 msgr2=0x7f89880c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 shutdown_connections 2024-04-25T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:27.905+0000 7f898f600640 1 -- 172.21.15.5:0/1517403569 wait complete. 2024-04-25T10:02:27.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:02:27.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:02:27.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:02:27.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:02:27.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:02:27.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:02:27.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:02:27.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:02:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.221+0000 7f2d71200640 1 Processor -- start 2024-04-25T10:02:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.222+0000 7f2d71200640 1 -- start start 2024-04-25T10:02:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.224+0000 7f2d71200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0ce900 0x7f2d6c159940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.224+0000 7f2d71200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d6c0af9b0 0x7f2d6c15d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.224+0000 7f2d71200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d6c06f760 0x7f2d6c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.224+0000 7f2d71200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cc320 con 0x7f2d6c06f760 2024-04-25T10:02:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.224+0000 7f2d71200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cdb40 con 0x7f2d6c0af9b0 2024-04-25T10:02:28.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.224+0000 7f2d71200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cde20 con 0x7f2d6c0ce900 2024-04-25T10:02:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.225+0000 7f2d69400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d6c0af9b0 0x7f2d6c15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.225+0000 7f2d69400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d6c0af9b0 0x7f2d6c15d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44050/0 (socket says 172.21.15.5:44050) 2024-04-25T10:02:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.225+0000 7f2d69400640 1 -- 172.21.15.5:0/2438568020 learned_addr learned my addr 172.21.15.5:0/2438568020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.225+0000 7f2d6a800640 1 --2- 172.21.15.5:0/2438568020 >> v2:172.21.15.5:3300/0 conn(0x7f2d6c06f760 0x7f2d6c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.226+0000 7f2d69400640 1 -- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0ce900 msgr2=0x7f2d6c159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.227+0000 7f2d69e00640 1 --2- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0ce900 0x7f2d6c159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.227+0000 7f2d69400640 1 --2- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0ce900 0x7f2d6c159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.227+0000 7f2d69400640 1 -- 172.21.15.5:0/2438568020 >> v2:172.21.15.5:3300/0 conn(0x7f2d6c06f760 msgr2=0x7f2d6c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.227+0000 7f2d69400640 1 --2- 172.21.15.5:0/2438568020 >> v2:172.21.15.5:3300/0 conn(0x7f2d6c06f760 0x7f2d6c15d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.227+0000 7f2d69400640 1 -- 172.21.15.5:0/2438568020 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0ce100 con 0x7f2d6c0af9b0 2024-04-25T10:02:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.227+0000 7f2d6a800640 1 --2- 172.21.15.5:0/2438568020 >> v2:172.21.15.5:3300/0 conn(0x7f2d6c06f760 0x7f2d6c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.228+0000 7f99a6e00640 1 Processor -- start 2024-04-25T10:02:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.228+0000 7fa8e7200640 1 Processor -- start 2024-04-25T10:02:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.228+0000 7fa8e7200640 1 -- start start 2024-04-25T10:02:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.229+0000 7f2d69400640 1 --2- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0af9b0 0x7f2d6c15d490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c06d9b0 tx=0x7f2d64032e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecee8cb68f22dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.228+0000 7f99a6e00640 1 -- start start 2024-04-25T10:02:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.229+0000 7f2d68a00640 1 -- 172.21.15.5:0/2438568020 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64033950 con 0x7f2d6c0af9b0 2024-04-25T10:02:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.229+0000 7f2d68a00640 1 -- 172.21.15.5:0/2438568020 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64033af0 con 0x7f2d6c0af9b0 2024-04-25T10:02:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.229+0000 7f2d68a00640 1 -- 172.21.15.5:0/2438568020 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6403b3f0 con 0x7f2d6c0af9b0 2024-04-25T10:02:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7fa8e7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa8e0004a30 0x7fa8e00ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8e00df2d0 0x7fa8e004f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7fa8e7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e0054d40 con 0x7fa8e004fc20 2024-04-25T10:02:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7fa8e7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e00507a0 con 0x7fa8e0004a30 2024-04-25T10:02:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7fa8e7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00e5da0 con 0x7fa8e00df2d0 2024-04-25T10:02:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7f2d71200640 1 -- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0af9b0 msgr2=0x7f2d6c15d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.230+0000 7f2d71200640 1 --2- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0af9b0 0x7f2d6c15d490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c06d9b0 tx=0x7f2d64032e50 comp rx=0 tx=0).stop 2024-04-25T10:02:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.231+0000 7f2d71200640 1 -- 172.21.15.5:0/2438568020 shutdown_connections 2024-04-25T10:02:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.231+0000 7f2d71200640 1 --2- 172.21.15.5:0/2438568020 >> v2:172.21.15.5:3300/0 conn(0x7f2d6c06f760 0x7f2d6c15d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.231+0000 7f2d71200640 1 --2- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0af9b0 0x7f2d6c15d490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.231+0000 7f2d71200640 1 --2- 172.21.15.5:0/2438568020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0ce900 0x7f2d6c159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.231+0000 7f2d71200640 1 -- 172.21.15.5:0/2438568020 >> 172.21.15.5:0/2438568020 conn(0x7f2d6c06b400 msgr2=0x7f2d6c06d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.231+0000 7fa8e5e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa8e0004a30 0x7fa8e00ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7fa8e5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8e00df2d0 0x7fa8e004f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7fa8e6800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa8e004fc20 0x7fa8e0050090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51496/0 (socket says 172.21.15.5:51496) 2024-04-25T10:02:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7fa8e6800640 1 -- 172.21.15.5:0/1996275851 learned_addr learned my addr 172.21.15.5:0/1996275851 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7f99a6e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f99980361b0 0x7f9998039b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7f2d71200640 1 -- 172.21.15.5:0/2438568020 shutdown_connections 2024-04-25T10:02:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068450334 0 0) 0x7fa8e0054d40 con 0x7fa8e004fc20 2024-04-25T10:02:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8cc003520 con 0x7fa8e004fc20 2024-04-25T10:02:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7f99a6e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999803b340 con 0x7f99980529f0 2024-04-25T10:02:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7f99a6e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999803b5e0 con 0x7f999804a320 2024-04-25T10:02:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 993752282 0 0) 0x7fa8cc003520 con 0x7fa8e004fc20 2024-04-25T10:02:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8e00df2d0 msgr2=0x7fa8e004f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8e00df2d0 0x7fa8e004f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7f99a6e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999803b8c0 con 0x7f99980361b0 2024-04-25T10:02:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7f99a5a00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f999804a320 0x7f999803c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35466/0 (socket says 172.21.15.5:35466) 2024-04-25T10:02:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7f99a5a00640 1 -- 172.21.15.5:0/2244769946 learned_addr learned my addr 172.21.15.5:0/2244769946 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.233+0000 7f99a5000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f99980361b0 0x7f9998039b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 >> v2:172.21.15.80:3300/0 conn(0x7fa8e0004a30 msgr2=0x7fa8e00ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.232+0000 7f2d71200640 1 -- 172.21.15.5:0/2438568020 wait complete. 2024-04-25T10:02:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7f2d71200640 1 Processor -- start 2024-04-25T10:02:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 --2- 172.21.15.5:0/1996275851 >> v2:172.21.15.80:3300/0 conn(0x7fa8e0004a30 0x7fa8e00ded90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0006de0 con 0x7fa8e004fc20 2024-04-25T10:02:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.235+0000 7f2d71200640 1 -- start start 2024-04-25T10:02:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa8d4001670 con 0x7fa8e004fc20 2024-04-25T10:02:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 >> v1:172.21.15.5:6789/0 conn(0x7fa8e004fc20 legacy=0x7fa8e0050090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.235+0000 7f99a5000640 1 -- 172.21.15.5:0/2244769946 >> v1:172.21.15.80:6789/0 conn(0x7f999804a320 legacy=0x7f999803c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.235+0000 7f99a5000640 1 -- 172.21.15.5:0/2244769946 >> v1:172.21.15.5:6789/0 conn(0x7f99980529f0 legacy=0x7f9998054870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.235+0000 7f99a5000640 1 -- 172.21.15.5:0/2244769946 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999803bba0 con 0x7f99980361b0 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc0039b0 0x7fa8cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc0043a0 0x7fa8cc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc005030 0x7fa8cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00507a0 con 0x7fa8cc005030 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00e5da0 con 0x7fa8cc0039b0 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.234+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc00bfb0 con 0x7fa8cc0043a0 2024-04-25T10:02:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.236+0000 7fa8e5400640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8e00df2d0 0x7fa8e004f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.236+0000 7fa8e5e00640 1 --2- 172.21.15.5:0/1996275851 >> v2:172.21.15.80:3300/0 conn(0x7fa8e0004a30 0x7fa8e00ded90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.236+0000 7fa8e6800640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc005030 0x7fa8cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7f2d71200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6c06f760 0x7f2d6c0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.236+0000 7fa8e5400640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc0043a0 0x7fa8cc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7fa8e5e00640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc0039b0 0x7fa8cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7f2d71200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0af9b0 0x7f2d6c0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7f2d71200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0ce900 0x7f2d6c146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7f2d69e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6c06f760 0x7f2d6c0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d69e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6c06f760 0x7f2d6c0ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46458/0 (socket says 172.21.15.5:46458) 2024-04-25T10:02:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d69400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0af9b0 0x7f2d6c0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d69400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0af9b0 0x7f2d6c0ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49008/0 (socket says 172.21.15.5:49008) 2024-04-25T10:02:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7fa8e6800640 1 -- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc0043a0 msgr2=0x7fa8cc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7fa8e6800640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc0043a0 0x7fa8cc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7fa8e6800640 1 -- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc0039b0 msgr2=0x7fa8cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7fa8e6800640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc0039b0 0x7fa8cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.237+0000 7fa8e6800640 1 -- 172.21.15.5:0/1996275851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0006de0 con 0x7fa8cc005030 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d71200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cd8e0 con 0x7f2d6c06f760 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d71200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0cd760 con 0x7f2d6c0ce900 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d71200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b0ea0 con 0x7f2d6c0af9b0 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d69e00640 1 -- 172.21.15.5:0/4059927386 learned_addr learned my addr 172.21.15.5:0/4059927386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.238+0000 7f2d6a800640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0ce900 0x7f2d6c146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7fa8e6800640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc005030 0x7fa8cc00b9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c340 tx=0x7fa8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7350ca5fad152b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7f2d6a800640 1 -- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0af9b0 msgr2=0x7f2d6c0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7f2d6a800640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d6c0af9b0 0x7f2d6c0ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7f2d6a800640 1 -- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6c06f760 msgr2=0x7f2d6c0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4035070 con 0x7fa8cc005030 2024-04-25T10:02:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d4007e50 con 0x7fa8cc005030 2024-04-25T10:02:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7fa8e4a00640 1 -- 172.21.15.5:0/1996275851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005cf0 con 0x7fa8cc005030 2024-04-25T10:02:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.239+0000 7f99a5000640 1 --2- 172.21.15.5:0/2244769946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f9998039b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f998c0097a0 tx=0x7f998c02a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f4855d9affaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7f2d6a800640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6c06f760 0x7f2d6c0ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7f2d6a800640 1 -- 172.21.15.5:0/4059927386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0adfd0 con 0x7f2d6c0ce900 2024-04-25T10:02:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7f2d69e00640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d6c06f760 0x7f2d6c0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7fa8e7200640 1 -- 172.21.15.5:0/1996275851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc005030 msgr2=0x7fa8cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7fa8e7200640 1 --2- 172.21.15.5:0/1996275851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc005030 0x7fa8cc00b9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c340 tx=0x7fa8cc005830 comp rx=0 tx=0).stop 2024-04-25T10:02:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7fa8e6800640 1 -- 172.21.15.5:0/1996275851 reap_dead start 2024-04-25T10:02:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7f2d6a800640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0ce900 0x7f2d6c146e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d4d0 tx=0x7f2d5800d990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7fa8e7200640 1 -- 172.21.15.5:0/1996275851 shutdown_connections 2024-04-25T10:02:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7fa8e7200640 1 -- 172.21.15.5:0/1996275851 >> 172.21.15.5:0/1996275851 conn(0x7fa8e000a400 msgr2=0x7fa8e000a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.240+0000 7f999fe00640 1 -- 172.21.15.5:0/2244769946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c037070 con 0x7f99980361b0 2024-04-25T10:02:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7f999fe00640 1 -- 172.21.15.5:0/2244769946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c02acc0 con 0x7f99980361b0 2024-04-25T10:02:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7f999fe00640 1 -- 172.21.15.5:0/2244769946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c032660 con 0x7f99980361b0 2024-04-25T10:02:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7fca7dc00640 1 Processor -- start 2024-04-25T10:02:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d58018020 con 0x7f2d6c0ce900 2024-04-25T10:02:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58014620 con 0x7f2d6c0ce900 2024-04-25T10:02:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580149f0 con 0x7f2d6c0ce900 2024-04-25T10:02:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7fca7dc00640 1 -- start start 2024-04-25T10:02:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.241+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0ae2a0 con 0x7f2d6c0ce900 2024-04-25T10:02:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.242+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c0a8c10 con 0x7f2d6c0ce900 2024-04-25T10:02:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.242+0000 7f99a6e00640 1 -- 172.21.15.5:0/2244769946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 msgr2=0x7f9998039b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.242+0000 7f99a6e00640 1 --2- 172.21.15.5:0/2244769946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f9998039b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f998c0097a0 tx=0x7f998c02a030 comp rx=0 tx=0).stop 2024-04-25T10:02:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fa8e7200640 1 -- 172.21.15.5:0/1996275851 shutdown_connections 2024-04-25T10:02:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.242+0000 7f99a6e00640 1 -- 172.21.15.5:0/2244769946 shutdown_connections 2024-04-25T10:02:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.242+0000 7f99a6e00640 1 --2- 172.21.15.5:0/2244769946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f9998039b80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.242+0000 7f99a6e00640 1 -- 172.21.15.5:0/2244769946 >> 172.21.15.5:0/2244769946 conn(0x7f99980537b0 msgr2=0x7f9998053b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7f99a6e00640 1 -- 172.21.15.5:0/2244769946 shutdown_connections 2024-04-25T10:02:28.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fa8e7200640 1 -- 172.21.15.5:0/1996275851 wait complete. 2024-04-25T10:02:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d6c0ae7c0 con 0x7f2d6c0ce900 2024-04-25T10:02:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7f7d84400640 1 Processor -- start 2024-04-25T10:02:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fca7dc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca781652f0 0x7fca780cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fca7dc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca7806e980 0x7fca78070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fca7dc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca780cd2a0 con 0x7fca780714d0 2024-04-25T10:02:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fca7dc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca780ce340 con 0x7fca781652f0 2024-04-25T10:02:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7fca7dc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca780cdeb0 con 0x7fca7806e980 2024-04-25T10:02:28.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7f7d84400640 1 -- start start 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.244+0000 7fca77e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca781652f0 0x7fca780cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.244+0000 7fca77e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca781652f0 0x7fca780cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46474/0 (socket says 172.21.15.5:46474) 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.244+0000 7fca77e00640 1 -- 172.21.15.5:0/1999289007 learned_addr learned my addr 172.21.15.5:0/1999289007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.244+0000 7fca77400640 1 --2- 172.21.15.5:0/1999289007 >> v2:172.21.15.80:3300/0 conn(0x7fca7806e980 0x7fca78070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.245+0000 7fca77e00640 1 -- 172.21.15.5:0/1999289007 >> v2:172.21.15.80:3300/0 conn(0x7fca7806e980 msgr2=0x7fca78070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.243+0000 7f99a6e00640 1 -- 172.21.15.5:0/2244769946 wait complete. 2024-04-25T10:02:28.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.245+0000 7f99a6e00640 1 Processor -- start 2024-04-25T10:02:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.245+0000 7fca77e00640 1 --2- 172.21.15.5:0/1999289007 >> v2:172.21.15.80:3300/0 conn(0x7fca7806e980 0x7fca78070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f99a6e00640 1 -- start start 2024-04-25T10:02:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f7d84400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 0x7f7d7c0d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f7d84400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0d1e80 con 0x7f7d7c15db00 2024-04-25T10:02:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f7d84400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c0b5930 con 0x7f7d7c0d0a50 2024-04-25T10:02:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f7d84400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0b5c10 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7fca76a00640 1 -- 172.21.15.5:0/1999289007 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859639997 0 0) 0x7fca780cd2a0 con 0x7fca780714d0 2024-04-25T10:02:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7fa8e7200640 1 Processor -- start 2024-04-25T10:02:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7fa8e7200640 1 -- start start 2024-04-25T10:02:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f7d82400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 0x7f7d7c0d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7f7d82400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 0x7f7d7c0d2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49010/0 (socket says 172.21.15.5:49010) 2024-04-25T10:02:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7f7d82400640 1 -- 172.21.15.5:0/3714966949 learned_addr learned my addr 172.21.15.5:0/3714966949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7fca77e00640 1 -- 172.21.15.5:0/1999289007 >> v1:172.21.15.80:6789/0 conn(0x7fca780714d0 legacy=0x7fca78165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.246+0000 7fca77e00640 1 -- 172.21.15.5:0/1999289007 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780cd580 con 0x7fca781652f0 2024-04-25T10:02:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7f7d7be00640 1 -- 172.21.15.5:0/3714966949 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469324126 0 0) 0x7f7d7c0d1e80 con 0x7f7d7c15db00 2024-04-25T10:02:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7f7d7be00640 1 -- 172.21.15.5:0/3714966949 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d64003520 con 0x7f7d7c15db00 2024-04-25T10:02:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7f7d82400640 1 -- 172.21.15.5:0/3714966949 >> v1:172.21.15.80:6789/0 conn(0x7f7d7c0d0a50 legacy=0x7f7d7c0d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7f7d82400640 1 -- 172.21.15.5:0/3714966949 >> v1:172.21.15.5:6789/0 conn(0x7f7d7c15db00 legacy=0x7f7d7c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7f7d82400640 1 -- 172.21.15.5:0/3714966949 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0b5ef0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.248+0000 7f99a6e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f999804f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.248+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d5801b020 con 0x7f2d6c0ce900 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc005030 0x7fa8e00ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e004e520 0x7fa8e004e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8e00cea50 0x7fa8e00ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7fa8e7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0007680 con 0x7fa8e004e520 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7fa8e7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00073a0 con 0x7fa8e00cea50 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.247+0000 7fa8e7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0007ba0 con 0x7fa8cc005030 2024-04-25T10:02:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.248+0000 7f99a6e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f999804a320 0x7f99981898d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7fa8e5e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc005030 0x7fa8e00ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a6e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f99980529f0 0x7f9998189e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7fa8e5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e004e520 0x7fa8e004e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7fa8e5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e004e520 0x7fa8e004e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46490/0 (socket says 172.21.15.5:46490) 2024-04-25T10:02:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7fa8e5400640 1 -- 172.21.15.5:0/538888072 learned_addr learned my addr 172.21.15.5:0/538888072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a6e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980eb4d0 con 0x7f99980361b0 2024-04-25T10:02:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a6e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980eb350 con 0x7f999804a320 2024-04-25T10:02:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f2d62000640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d4c06b520 0x7f2d4c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a6e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9998054f80 con 0x7f99980529f0 2024-04-25T10:02:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a5a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f999804f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2d5808adc0 con 0x7f2d6c0ce900 2024-04-25T10:02:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a5a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f999804f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46498/0 (socket says 172.21.15.5:46498) 2024-04-25T10:02:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a5a00640 1 -- 172.21.15.5:0/2221016436 learned_addr learned my addr 172.21.15.5:0/2221016436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f2d62000640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f2d4c074ca0 0x7f2d4c0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.249+0000 7f99a5000640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f999804a320 0x7f99981898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f99a6400640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f99980529f0 0x7f9998189e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d4c0777b0 con 0x7f2d4c074ca0 2024-04-25T10:02:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d5808b230 con 0x7f2d6c0ce900 2024-04-25T10:02:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fa8e5400640 1 -- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc005030 msgr2=0x7fa8e00ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fca77e00640 1 --2- 172.21.15.5:0/1999289007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca781652f0 0x7fca780cc390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca780cf2d0 tx=0x7fca6802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622553267858698c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fa8e5400640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc005030 0x7fa8e00ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fa8e5400640 1 -- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8e00cea50 msgr2=0x7fa8e00ceed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fa8e6800640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8e00cea50 0x7fa8e00ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fa8e5400640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8e00cea50 0x7fa8e00ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7fa8e5400640 1 -- 172.21.15.5:0/538888072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e00d3670 con 0x7fa8e004e520 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f2d69400640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f2d4c074ca0 0x7f2d4c0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f2d69e00640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d4c06b520 0x7f2d4c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f99a5a00640 1 -- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f99980529f0 msgr2=0x7f9998189e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f99a5a00640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f99980529f0 0x7f9998189e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f99a5a00640 1 -- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f999804a320 msgr2=0x7f99981898d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f99a5a00640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f999804a320 0x7f99981898d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7f99a5a00640 1 -- 172.21.15.5:0/2221016436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999804dd70 con 0x7f99980361b0 2024-04-25T10:02:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.250+0000 7f7d82400640 1 --2- 172.21.15.5:0/3714966949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 0x7f7d7c0d2a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009e80 tx=0x7f7d7402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56d92fde3e6eca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fa8e5400640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e004e520 0x7fa8e004e9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8011830 tx=0x7fa8d8011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800f9c0 con 0x7fa8e004e520 2024-04-25T10:02:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d800fb60 con 0x7fa8e004e520 2024-04-25T10:02:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8015460 con 0x7fa8e004e520 2024-04-25T10:02:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7f7d7be00640 1 -- 172.21.15.5:0/3714966949 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74036070 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7f7d7be00640 1 -- 172.21.15.5:0/3714966949 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74031600 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7f7d7be00640 1 -- 172.21.15.5:0/3714966949 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d740319d0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fca76a00640 1 -- 172.21.15.5:0/1999289007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68036070 con 0x7fca781652f0 2024-04-25T10:02:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fca76a00640 1 -- 172.21.15.5:0/1999289007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68031600 con 0x7fca781652f0 2024-04-25T10:02:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7fca76a00640 1 -- 172.21.15.5:0/1999289007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680319d0 con 0x7fca781652f0 2024-04-25T10:02:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.251+0000 7f99a5a00640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f999804f0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f999400c7b0 tx=0x7f999400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9994018070 con 0x7f99980361b0 2024-04-25T10:02:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f593b200640 1 Processor -- start 2024-04-25T10:02:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99940072d0 con 0x7f99980361b0 2024-04-25T10:02:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940047e0 con 0x7f99980361b0 2024-04-25T10:02:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f2d69e00640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d4c06b520 0x7f2d4c06d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0cb9a0 tx=0x7f2d5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f593b200640 1 -- start start 2024-04-25T10:02:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f7d84400640 1 -- 172.21.15.5:0/3714966949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 msgr2=0x7f7d7c0d2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f7d84400640 1 --2- 172.21.15.5:0/3714966949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 0x7f7d7c0d2a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009e80 tx=0x7f7d7402a7a0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7fca7dc00640 1 -- 172.21.15.5:0/1999289007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca781652f0 msgr2=0x7fca780cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7fca7dc00640 1 --2- 172.21.15.5:0/1999289007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca781652f0 0x7fca780cc390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca780cf2d0 tx=0x7fca6802a730 comp rx=0 tx=0).stop 2024-04-25T10:02:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f2d69400640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f2d4c074ca0 0x7f2d4c0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.253+0000 7fca7dc00640 1 -- 172.21.15.5:0/1999289007 shutdown_connections 2024-04-25T10:02:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f99a6e00640 1 -- 172.21.15.5:0/2221016436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999804e050 con 0x7f99980361b0 2024-04-25T10:02:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.253+0000 7fca7dc00640 1 --2- 172.21.15.5:0/1999289007 >> v2:172.21.15.80:3300/0 conn(0x7fca7806e980 0x7fca78070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.253+0000 7fca7dc00640 1 --2- 172.21.15.5:0/1999289007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca781652f0 0x7fca780cc390 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca780cf2d0 tx=0x7fca6802a730 comp rx=0 tx=0).stop 2024-04-25T10:02:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.253+0000 7fca7dc00640 1 -- 172.21.15.5:0/1999289007 >> 172.21.15.5:0/1999289007 conn(0x7fca78066f20 msgr2=0x7fca7806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7f99a6e00640 1 -- 172.21.15.5:0/2221016436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999804e610 con 0x7f99980361b0 2024-04-25T10:02:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.253+0000 7fca7dc00640 1 -- 172.21.15.5:0/1999289007 shutdown_connections 2024-04-25T10:02:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7f99a6e00640 1 -- 172.21.15.5:0/2221016436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99980565e0 con 0x7f99980361b0 2024-04-25T10:02:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7fb61c200640 1 Processor -- start 2024-04-25T10:02:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7fca7dc00640 1 -- 172.21.15.5:0/1999289007 wait complete. 2024-04-25T10:02:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7f3225200640 1 Processor -- start 2024-04-25T10:02:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7f7d84400640 1 -- 172.21.15.5:0/3714966949 shutdown_connections 2024-04-25T10:02:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7f7d84400640 1 --2- 172.21.15.5:0/3714966949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0b53b0 0x7f7d7c0d2a40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7f7d84400640 1 -- 172.21.15.5:0/3714966949 >> 172.21.15.5:0/3714966949 conn(0x7f7d7c072290 msgr2=0x7f7d7c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2d4c0777b0 con 0x7f2d4c074ca0 2024-04-25T10:02:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.255+0000 7f7d84400640 1 -- 172.21.15.5:0/3714966949 shutdown_connections 2024-04-25T10:02:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.254+0000 7fb61c200640 1 -- start start 2024-04-25T10:02:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.255+0000 7f7d84400640 1 -- 172.21.15.5:0/3714966949 wait complete. 2024-04-25T10:02:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e004eee0 con 0x7fa8e004e520 2024-04-25T10:02:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.252+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e00d3bf0 con 0x7fa8e004e520 2024-04-25T10:02:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.255+0000 7f3225200640 1 -- start start 2024-04-25T10:02:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.255+0000 7f593b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c15d8a0 0x7f593c0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.256+0000 7f593b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593c15f410 con 0x7f593c0b2f30 2024-04-25T10:02:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.256+0000 7f593b200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593c15f6b0 con 0x7f593c1614e0 2024-04-25T10:02:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.256+0000 7f593b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c15ffe0 con 0x7f593c15d8a0 2024-04-25T10:02:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7fca7dc00640 1 Processor -- start 2024-04-25T10:02:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8c0001030 con 0x7fa8e004e520 2024-04-25T10:02:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f3225200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f322015d8a0 0x7f32200d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f3225200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32200b2f30 0x7f32200d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f7d84400640 1 Processor -- start 2024-04-25T10:02:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7fca7dc00640 1 -- start start 2024-04-25T10:02:28.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f7d84400640 1 -- start start 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f3225200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322015f410 con 0x7f32201614e0 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f3225200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322015ffe0 con 0x7f322015d8a0 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f3225200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322015f6b0 con 0x7f32200b2f30 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f321f400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f322015d8a0 0x7f32200d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f321f400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f322015d8a0 0x7f32200d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44108/0 (socket says 172.21.15.5:44108) 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f321f400640 1 -- 172.21.15.5:0/1052603332 learned_addr learned my addr 172.21.15.5:0/1052603332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f321ea00640 1 --2- 172.21.15.5:0/1052603332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32200b2f30 0x7f32200d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.256+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9994004980 con 0x7f99980361b0 2024-04-25T10:02:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7fb61c200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6140cea50 0x7fb6140ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7f5939e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c15d8a0 0x7f593c0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f593a800640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f593c1614e0 0x7f593c0b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35486/0 (socket says 172.21.15.5:35486) 2024-04-25T10:02:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f593a800640 1 -- 172.21.15.5:0/1567339001 learned_addr learned my addr 172.21.15.5:0/1567339001 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f5939400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f593c0b2f30 0x7f593c0d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51526/0 (socket says 172.21.15.5:51526) 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f7d84400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c0b53b0 0x7f7d7c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f7d84400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0d0a50 0x7f7d7c1f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f7d84400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c15db00 0x7f7d7c146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f7d84400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0d3660 con 0x7f7d7c15db00 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f7d84400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0b61d0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7f7d84400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0d2f80 con 0x7f7d7c0d0a50 2024-04-25T10:02:28.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.257+0000 7fb61c200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6140cb560 0x7fb6140cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7fb61c200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6140cbeb0 con 0x7fb614143400 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7fb61c200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61406d1f0 con 0x7fb6140cea50 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7fb61c200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61406d4d0 con 0x7fb6140cb560 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7fb619800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb614143400 0x7fb61415b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51532/0 (socket says 172.21.15.5:51532) 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.258+0000 7fb619800640 1 -- 172.21.15.5:0/4251212388 learned_addr learned my addr 172.21.15.5:0/4251212388 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7fb61a200640 1 --2- 172.21.15.5:0/4251212388 >> v2:172.21.15.80:3300/0 conn(0x7fb6140cb560 0x7fb6140cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7fb618e00640 1 --2- 172.21.15.5:0/4251212388 >> v2:172.21.15.5:3300/0 conn(0x7fb6140cea50 0x7fb6140ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f7d81a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c0b53b0 0x7f7d7c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f999d600640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f998006b4d0 0x7f998006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f7d82400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c15db00 0x7f7d7c146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f999408a120 con 0x7f99980361b0 2024-04-25T10:02:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f7d81000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0d0a50 0x7f7d7c1f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f7d81a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c0b53b0 0x7f7d7c15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44130/0 (socket says 172.21.15.5:44130) 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f7d81000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0d0a50 0x7f7d7c1f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49052/0 (socket says 172.21.15.5:49052) 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7f7d81a00640 1 -- 172.21.15.5:0/4148581774 learned_addr learned my addr 172.21.15.5:0/4148581774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7fb61a200640 1 -- 172.21.15.5:0/4251212388 >> v2:172.21.15.5:3300/0 conn(0x7fb6140cea50 msgr2=0x7fb6140ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7fb61a200640 1 --2- 172.21.15.5:0/4251212388 >> v2:172.21.15.5:3300/0 conn(0x7fb6140cea50 0x7fb6140ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7fb61a200640 1 -- 172.21.15.5:0/4251212388 >> v1:172.21.15.5:6789/0 conn(0x7fb614143400 legacy=0x7fb61415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.259+0000 7fb61a200640 1 -- 172.21.15.5:0/4251212388 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61406d7b0 con 0x7fb6140cb560 2024-04-25T10:02:28.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f321ea00640 1 -- 172.21.15.5:0/1052603332 >> v2:172.21.15.80:3300/0 conn(0x7f322015d8a0 msgr2=0x7f32200d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f321ea00640 1 --2- 172.21.15.5:0/1052603332 >> v2:172.21.15.80:3300/0 conn(0x7f322015d8a0 0x7f32200d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f321ea00640 1 -- 172.21.15.5:0/1052603332 >> v1:172.21.15.5:6789/0 conn(0x7f32201614e0 legacy=0x7f32200b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fca7dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7806e980 0x7fca78146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fca7dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca780714d0 0x7fca781534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fca7dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca78153a30 0x7fca781542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f321ea00640 1 -- 172.21.15.5:0/1052603332 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322015f990 con 0x7f32200b2f30 2024-04-25T10:02:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fca7dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780cc8d0 con 0x7fca78153a30 2024-04-25T10:02:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fca7dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca780cdb40 con 0x7fca7806e980 2024-04-25T10:02:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fca7dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7806e000 con 0x7fca780714d0 2024-04-25T10:02:28.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8d8015600 con 0x7fa8e004e520 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f5938a00640 1 -- 172.21.15.5:0/1567339001 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301615774 0 0) 0x7f593c15f6b0 con 0x7f593c1614e0 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f5938a00640 1 -- 172.21.15.5:0/1567339001 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5920003520 con 0x7f593c1614e0 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f5939e00640 1 -- 172.21.15.5:0/1567339001 >> v1:172.21.15.80:6789/0 conn(0x7f593c1614e0 legacy=0x7f593c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f5939e00640 1 -- 172.21.15.5:0/1567339001 >> v1:172.21.15.5:6789/0 conn(0x7f593c0b2f30 legacy=0x7f593c0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7fca77e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7806e980 0x7fca78146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7fca77e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7806e980 0x7fca78146600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44134/0 (socket says 172.21.15.5:44134) 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7fca77e00640 1 -- 172.21.15.5:0/1587355102 learned_addr learned my addr 172.21.15.5:0/1587355102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f999d600640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f9980074c40 0x7f9980077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7f99a5000640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f998006b4d0 0x7f998006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f999401fe50 con 0x7f9980074c40 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f999408a510 con 0x7f99980361b0 2024-04-25T10:02:28.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f5938a00640 1 -- 172.21.15.5:0/1567339001 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342885520 0 0) 0x7f593c15f410 con 0x7f593c0b2f30 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7f5939e00640 1 -- 172.21.15.5:0/1567339001 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c15f990 con 0x7f593c15d8a0 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7fca77400640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca780714d0 0x7fca781534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f7d81a00640 1 -- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0d0a50 msgr2=0x7f7d7c1f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f7d81a00640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7d7c0d0a50 0x7f7d7c1f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f7d81a00640 1 -- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c15db00 msgr2=0x7f7d7c146480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f7d81a00640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c15db00 0x7f7d7c146480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.260+0000 7f7d81a00640 1 -- 172.21.15.5:0/4148581774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c0b3610 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7fb61a200640 1 --2- 172.21.15.5:0/4251212388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6140cb560 0x7fb6140cb970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6140cc2d0 tx=0x7fb60c02a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5eee163d4661ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.261+0000 7fca7d200640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca78153a30 0x7fca781542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7f99a6400640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f9980074c40 0x7f9980077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fa8de000640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8c806b5d0 0x7fa8c806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa8d808aef0 con 0x7fa8e004e520 2024-04-25T10:02:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fa8de000640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7fa8c8074d40 0x7fa8c8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8d8024ec0 con 0x7fa8c8074d40 2024-04-25T10:02:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d8092050 con 0x7fa8e004e520 2024-04-25T10:02:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fca77e00640 1 -- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca780714d0 msgr2=0x7fca781534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fca77e00640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca780714d0 0x7fca781534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fca77e00640 1 -- 172.21.15.5:0/1587355102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca78153a30 msgr2=0x7fca781542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fca77e00640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca78153a30 0x7fca781542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fca77e00640 1 -- 172.21.15.5:0/1587355102 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca78155b60 con 0x7fca7806e980 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fca7d200640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca78153a30 0x7fca781542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7f321ea00640 1 --2- 172.21.15.5:0/1052603332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32200b2f30 0x7f32200d3f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14db7f2c59bec6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7fa8e5e00640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8c806b5d0 0x7fa8c806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.262+0000 7fb613e00640 1 -- 172.21.15.5:0/4251212388 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c032600 con 0x7fb6140cb560 2024-04-25T10:02:28.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7fb613e00640 1 -- 172.21.15.5:0/4251212388 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c0327a0 con 0x7fb6140cb560 2024-04-25T10:02:28.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7fb613e00640 1 -- 172.21.15.5:0/4251212388 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c032b90 con 0x7fb6140cb560 2024-04-25T10:02:28.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f7d82400640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c15db00 0x7f7d7c146480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7fa8e6800640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7fa8c8074d40 0x7fa8c8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f99a6400640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f9980074c40 0x7f9980077080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f321e000640 1 -- 172.21.15.5:0/1052603332 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218037070 con 0x7f32200b2f30 2024-04-25T10:02:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f321e000640 1 -- 172.21.15.5:0/1052603332 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321802adb0 con 0x7f32200b2f30 2024-04-25T10:02:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f321e000640 1 -- 172.21.15.5:0/1052603332 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218032660 con 0x7f32200b2f30 2024-04-25T10:02:28.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7fca77e00640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7806e980 0x7fca78146600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca68009f40 tx=0x7fca68004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f5939e00640 1 --2- 172.21.15.5:0/1567339001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c15d8a0 0x7f593c0d2570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5934009e10 tx=0x7f593402a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20afc13362ada3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.263+0000 7f7d81a00640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c0b53b0 0x7f7d7c15e130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c430 tx=0x7f7d7000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68041030 con 0x7fca7806e980 2024-04-25T10:02:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68031400 con 0x7fca7806e980 2024-04-25T10:02:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68036040 con 0x7fca7806e980 2024-04-25T10:02:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fca7dc00640 1 -- 172.21.15.5:0/1587355102 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca78154fe0 con 0x7fca7806e980 2024-04-25T10:02:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fa8e6800640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7fa8c8074d40 0x7fa8c8077180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fa8e5e00640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8c806b5d0 0x7fa8c806da90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8074390 tx=0x7fa8e808d040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f99a5000640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f998006b4d0 0x7f998006d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f998c034000 tx=0x7f998c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f999401fe50 con 0x7f9980074c40 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70017070 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d700052f0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70003830 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c0b38f0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c0b3eb0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fca7dc00640 1 -- 172.21.15.5:0/1587355102 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca781555a0 con 0x7fca7806e980 2024-04-25T10:02:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7fca7dc00640 1 -- 172.21.15.5:0/1587355102 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca78153ec0 con 0x7fca7806e980 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f3225200640 1 -- 172.21.15.5:0/1052603332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32200b2f30 msgr2=0x7f32200d3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7f3225200640 1 --2- 172.21.15.5:0/1052603332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32200b2f30 0x7f32200d3f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802a060 comp rx=0 tx=0).stop 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f3225200640 1 -- 172.21.15.5:0/1052603332 shutdown_connections 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f3225200640 1 --2- 172.21.15.5:0/1052603332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32200b2f30 0x7f32200d3f20 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32180097a0 tx=0x7f321802a060 comp rx=0 tx=0).stop 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f3225200640 1 --2- 172.21.15.5:0/1052603332 >> v2:172.21.15.80:3300/0 conn(0x7f322015d8a0 0x7f32200d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f3225200640 1 -- 172.21.15.5:0/1052603332 >> 172.21.15.5:0/1052603332 conn(0x7f322008af80 msgr2=0x7f32200d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fb61c200640 1 -- 172.21.15.5:0/4251212388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6140cb560 msgr2=0x7fb6140cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fb61c200640 1 --2- 172.21.15.5:0/4251212388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6140cb560 0x7fb6140cb970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6140cc2d0 tx=0x7fb60c02a850 comp rx=0 tx=0).stop 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fb61c200640 1 -- 172.21.15.5:0/4251212388 shutdown_connections 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fb61c200640 1 --2- 172.21.15.5:0/4251212388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6140cb560 0x7fb6140cb970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.264+0000 7fb61c200640 1 --2- 172.21.15.5:0/4251212388 >> v2:172.21.15.5:3300/0 conn(0x7fb6140cea50 0x7fb6140ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7fb61c200640 1 -- 172.21.15.5:0/4251212388 >> 172.21.15.5:0/4251212388 conn(0x7fb614071150 msgr2=0x7fb614071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7fb61c200640 1 -- 172.21.15.5:0/4251212388 shutdown_connections 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7fb61c200640 1 -- 172.21.15.5:0/4251212388 wait complete. 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f5938a00640 1 -- 172.21.15.5:0/1567339001 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934007740 con 0x7f593c15d8a0 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f5938a00640 1 -- 172.21.15.5:0/1567339001 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59340078e0 con 0x7f593c15d8a0 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f5938a00640 1 -- 172.21.15.5:0/1567339001 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934007cb0 con 0x7f593c15d8a0 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f3225200640 1 -- 172.21.15.5:0/1052603332 shutdown_connections 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.265+0000 7f3225200640 1 -- 172.21.15.5:0/1052603332 wait complete. 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 -- 172.21.15.5:0/1567339001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c15d8a0 msgr2=0x7f593c0d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 --2- 172.21.15.5:0/1567339001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c15d8a0 0x7f593c0d2570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5934009e10 tx=0x7f593402a5b0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 -- 172.21.15.5:0/1567339001 shutdown_connections 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 --2- 172.21.15.5:0/1567339001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c15d8a0 0x7f593c0d2570 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 -- 172.21.15.5:0/1567339001 >> 172.21.15.5:0/1567339001 conn(0x7f593c08af80 msgr2=0x7f593c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 -- 172.21.15.5:0/1567339001 shutdown_connections 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f593b200640 1 -- 172.21.15.5:0/1567339001 wait complete. 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa8d8024ec0 con 0x7fa8c8074d40 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.267+0000 7fb61c200640 1 Processor -- start 2024-04-25T10:02:28.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.267+0000 7fb61c200640 1 -- start start 2024-04-25T10:02:28.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.266+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d7c15aef0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.267+0000 7f3225200640 1 Processor -- start 2024-04-25T10:02:28.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.268+0000 7f3225200640 1 -- start start 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fb61c200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6140cb560 0x7fb614138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f3225200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f322015d8a0 0x7f32201525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f3225200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32201614e0 0x7f32200cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f3225200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32200ca6d0 0x7f322006cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f3225200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322015ec90 con 0x7f32200ca6d0 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f3225200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200d4460 con 0x7f32201614e0 2024-04-25T10:02:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f3225200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322015ee10 con 0x7f322015d8a0 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fb61c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6140cea50 0x7fb614135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f321fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32200ca6d0 0x7f322006cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f321f400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f322015d8a0 0x7f32201525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f321fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32200ca6d0 0x7f322006cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46556/0 (socket says 172.21.15.5:46556) 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f321fe00640 1 -- 172.21.15.5:0/2664922033 learned_addr learned my addr 172.21.15.5:0/2664922033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7f321ea00640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32201614e0 0x7f32200cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fb61c200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb614143400 0x7fb614135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fb61c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61415bb70 con 0x7fb6140cea50 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fb61c200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61406dd70 con 0x7fb614143400 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fb61c200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61415ae10 con 0x7fb6140cb560 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7fb619800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6140cb560 0x7fb614138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7fb619800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6140cb560 0x7fb614138df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49084/0 (socket says 172.21.15.5:49084) 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7fb619800640 1 -- 172.21.15.5:0/283114197 learned_addr learned my addr 172.21.15.5:0/283114197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7fb61a200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb614143400 0x7fb614135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7fb618e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6140cea50 0x7fb614135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7f321fe00640 1 -- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f322015d8a0 msgr2=0x7f32201525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7f321fe00640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f322015d8a0 0x7f32201525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7f321fe00640 1 -- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32201614e0 msgr2=0x7f32200cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7f321fe00640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32201614e0 0x7f32200cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7f321fe00640 1 -- 172.21.15.5:0/2664922033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200cac00 con 0x7f32200ca6d0 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7f321f400640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f322015d8a0 0x7f32201525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.269+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca68044070 con 0x7fca7806e980 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.270+0000 7fca6fe00640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca5806b4d0 0x7fca5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fca680ab810 con 0x7fca7806e980 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fca6fe00640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7fca58074bd0 0x7fca58077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca68048e70 con 0x7fca58074bd0 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca680abbc0 con 0x7fca7806e980 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fb61a200640 1 -- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6140cb560 msgr2=0x7fb614138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fb61a200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6140cb560 0x7fb614138df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fb61a200640 1 -- 172.21.15.5:0/283114197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6140cea50 msgr2=0x7fb614135a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fca77400640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca5806b4d0 0x7fca5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fb61a200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6140cea50 0x7fb614135a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7fb61a200640 1 -- 172.21.15.5:0/283114197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6140b55b0 con 0x7fb614143400 2024-04-25T10:02:28.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.271+0000 7f321fe00640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32200ca6d0 0x7f322006cca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f321000c740 tx=0x7f321000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7fb61a200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb614143400 0x7fb614135f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb60c032000 tx=0x7fb60c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7fca7d200640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7fca58074bd0 0x7fca58077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210016020 con 0x7f32200ca6d0 2024-04-25T10:02:28.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000ce50 con 0x7f32200ca6d0 2024-04-25T10:02:28.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210003970 con 0x7f32200ca6d0 2024-04-25T10:02:28.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7f593b200640 1 Processor -- start 2024-04-25T10:02:28.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c040070 con 0x7fb614143400 2024-04-25T10:02:28.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6140b4510 con 0x7fb614143400 2024-04-25T10:02:28.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.272+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6140b4ad0 con 0x7fb614143400 2024-04-25T10:02:28.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60c004240 con 0x7fb614143400 2024-04-25T10:02:28.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d7001a070 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c009450 con 0x7fb614143400 2024-04-25T10:02:28.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7fca7d200640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7fca58074bd0 0x7fca58077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32200caee0 con 0x7f32200ca6d0 2024-04-25T10:02:28.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32200cb4a0 con 0x7f32200ca6d0 2024-04-25T10:02:28.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.274+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6140b5180 con 0x7fb614143400 2024-04-25T10:02:28.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.274+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3220068f60 con 0x7f32200ca6d0 2024-04-25T10:02:28.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.274+0000 7f7d79600640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.273+0000 7fca77400640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca5806b4d0 0x7fca5806d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca780b06a0 tx=0x7fca64008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.274+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7d70089ce0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.275+0000 7f7d79600640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7f7d5c074c80 0x7f7d5c0770c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.275+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d7001feb0 con 0x7f7d5c074c80 2024-04-25T10:02:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.275+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d7008a0d0 con 0x7f7d7c0b53b0 2024-04-25T10:02:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.275+0000 7f7d81000640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.276+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb60c004630 con 0x7fb614143400 2024-04-25T10:02:28.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.276+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fca68048e70 con 0x7fca58074bd0 2024-04-25T10:02:28.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.275+0000 7f7d82400640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7f7d5c074c80 0x7f7d5c0770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.276+0000 7f7d81000640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0b4500 tx=0x7f7d68009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.276+0000 7f593b200640 1 -- start start 2024-04-25T10:02:28.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.277+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3210003b10 con 0x7f32200ca6d0 2024-04-25T10:02:28.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.277+0000 7f7d82400640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7f7d5c074c80 0x7f7d5c0770c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.277+0000 7fb611600640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb5f406b4d0 0x7fb5f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.277+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb60c0abe80 con 0x7fb614143400 2024-04-25T10:02:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f593b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f593c0b2f30 0x7f593c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f593b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c15d8a0 0x7f593c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f593b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c1614e0 0x7f593c0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f3217400640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f31fc06b520 0x7f31fc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f321008ae70 con 0x7f32200ca6d0 2024-04-25T10:02:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f3217400640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f31fc074ca0 0x7f31fc0770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7fb619800640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb5f406b4d0 0x7fb5f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7fb611600640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7fb5f4074c40 0x7fb5f4077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3210025ea0 con 0x7f31fc074ca0 2024-04-25T10:02:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f321008b220 con 0x7f32200ca6d0 2024-04-25T10:02:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb60c03fe50 con 0x7fb5f4074c40 2024-04-25T10:02:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb60c0ac270 con 0x7fb614143400 2024-04-25T10:02:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f593b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c15ee10 con 0x7f593c15d8a0 2024-04-25T10:02:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f593b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593c15ec90 con 0x7f593c0b2f30 2024-04-25T10:02:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f593b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593c161970 con 0x7f593c1614e0 2024-04-25T10:02:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f5939400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c15d8a0 0x7f593c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.278+0000 7f321f400640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f31fc06b520 0x7f31fc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.279+0000 7fb618e00640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7fb5f4074c40 0x7fb5f4077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.279+0000 7f321ea00640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f31fc074ca0 0x7f31fc0770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.280+0000 7fb619800640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb5f406b4d0 0x7fb5f406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb600009370 tx=0x7fb600009300 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.279+0000 7f5939400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c15d8a0 0x7f593c0cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46570/0 (socket says 172.21.15.5:46570) 2024-04-25T10:02:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.280+0000 7f5939400640 1 -- 172.21.15.5:0/3377198372 learned_addr learned my addr 172.21.15.5:0/3377198372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.279+0000 7f593a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c1614e0 0x7f593c0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.280+0000 7f321f400640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f31fc06b520 0x7f31fc06d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3208009ae0 tx=0x7f3208008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.279+0000 7f5939e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f593c0b2f30 0x7f593c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.280+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7d7001feb0 con 0x7f7d5c074c80 2024-04-25T10:02:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.280+0000 7fb618e00640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7fb5f4074c40 0x7fb5f4077080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.280+0000 7f321ea00640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f31fc074ca0 0x7f31fc0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5939400640 1 -- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c1614e0 msgr2=0x7f593c0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5939400640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c1614e0 0x7f593c0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5939400640 1 -- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f593c0b2f30 msgr2=0x7f593c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5939400640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f593c0b2f30 0x7f593c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5939400640 1 -- 172.21.15.5:0/3377198372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c0c8ce0 con 0x7f593c15d8a0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f593a800640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f593c1614e0 0x7f593c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5939400640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c15d8a0 0x7f593c0cbd90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f592400c4d0 tx=0x7f592400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.281+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924018070 con 0x7f593c15d8a0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.282+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59240074e0 con 0x7f593c15d8a0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.282+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924004a20 con 0x7f593c15d8a0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.282+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c0ccd00 con 0x7f593c15d8a0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.282+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593c0cd2c0 con 0x7f593c15d8a0 2024-04-25T10:02:28.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.283+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3210025ea0 con 0x7f31fc074ca0 2024-04-25T10:02:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.283+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f593c1612b0 con 0x7f593c15d8a0 2024-04-25T10:02:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.283+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb60c03fe50 con 0x7fb5f4074c40 2024-04-25T10:02:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.285+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f592401b070 con 0x7f593c15d8a0 2024-04-25T10:02:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.286+0000 7f5932000640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f591806b550 0x7f591806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.286+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5924089880 con 0x7f593c15d8a0 2024-04-25T10:02:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.287+0000 7f5932000640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f5918074cd0 0x7f5918077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.287+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5924016e70 con 0x7f5918074cd0 2024-04-25T10:02:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.287+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f592408e2a0 con 0x7f593c15d8a0 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.287+0000 7f5939e00640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f591806b550 0x7f591806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.287+0000 7f593a800640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f5918074cd0 0x7f5918077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.288+0000 7f5939e00640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f591806b550 0x7f591806da10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5934034000 tx=0x7f593402f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.291+0000 7f593a800640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f5918074cd0 0x7f5918077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.294+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5924016e70 con 0x7f5918074cd0 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.299+0000 7f99a6e00640 1 -- 172.21.15.5:0/2221016436 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9998054410 con 0x7f9980074c40 2024-04-25T10:02:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.300+0000 7f999d600640 1 -- 172.21.15.5:0/2221016436 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9998054410 con 0x7f9980074c40 2024-04-25T10:02:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.302+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f9980074c40 msgr2=0x7f9980077080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.302+0000 7f9992a00640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f9980074c40 0x7f9980077080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.304+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f998006b4d0 msgr2=0x7f998006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.304+0000 7f9992a00640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f998006b4d0 0x7f998006d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f998c034000 tx=0x7f998c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.304+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 msgr2=0x7f999804f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.304+0000 7f9992a00640 1 --2- 172.21.15.5:0/2221016436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99980361b0 0x7f999804f0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f999400c7b0 tx=0x7f999400cc70 comp rx=0 tx=0).stop 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.304+0000 7f99a6400640 1 -- 172.21.15.5:0/2221016436 reap_dead start 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32200d0e50 con 0x7f31fc074ca0 2024-04-25T10:02:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d6c0a8a30 con 0x7f2d4c074ca0 2024-04-25T10:02:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 shutdown_connections 2024-04-25T10:02:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 >> 172.21.15.5:0/2221016436 conn(0x7f99980537b0 msgr2=0x7f99980553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 shutdown_connections 2024-04-25T10:02:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f2d62000640 1 -- 172.21.15.5:0/4059927386 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2d6c0a8a30 con 0x7f2d4c074ca0 2024-04-25T10:02:28.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.305+0000 7f9992a00640 1 -- 172.21.15.5:0/2221016436 wait complete. 2024-04-25T10:02:28.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f3217400640 1 -- 172.21.15.5:0/2664922033 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f32200d0e50 con 0x7f31fc074ca0 2024-04-25T10:02:28.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f2d4c074ca0 msgr2=0x7f2d4c0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d71200640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f2d4c074ca0 0x7f2d4c0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d4c06b520 msgr2=0x7f2d4c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d71200640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d4c06b520 0x7f2d4c06d9e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0cb9a0 tx=0x7f2d5c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0ce900 msgr2=0x7f2d6c146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d71200640 1 --2- 172.21.15.5:0/4059927386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d6c0ce900 0x7f2d6c146e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800d4d0 tx=0x7f2d5800d990 comp rx=0 tx=0).stop 2024-04-25T10:02:28.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.306+0000 7f2d6a800640 1 -- 172.21.15.5:0/4059927386 reap_dead start 2024-04-25T10:02:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 shutdown_connections 2024-04-25T10:02:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 >> 172.21.15.5:0/4059927386 conn(0x7f2d6c06b400 msgr2=0x7f2d6c158370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8c0002dd0 con 0x7fa8c8074d40 2024-04-25T10:02:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f31fc074ca0 msgr2=0x7f31fc0770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f3225200640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f31fc074ca0 0x7f31fc0770e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 shutdown_connections 2024-04-25T10:02:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f2d71200640 1 -- 172.21.15.5:0/4059927386 wait complete. 2024-04-25T10:02:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f31fc06b520 msgr2=0x7f31fc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f3225200640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f31fc06b520 0x7f31fc06d9e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3208009ae0 tx=0x7f3208008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32200ca6d0 msgr2=0x7f322006cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f3225200640 1 --2- 172.21.15.5:0/2664922033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32200ca6d0 0x7f322006cca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f321000c740 tx=0x7f321000cc00 comp rx=0 tx=0).stop 2024-04-25T10:02:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.307+0000 7f321fe00640 1 -- 172.21.15.5:0/2664922033 reap_dead start 2024-04-25T10:02:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.308+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 shutdown_connections 2024-04-25T10:02:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.308+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 >> 172.21.15.5:0/2664922033 conn(0x7f322008af80 msgr2=0x7f32200b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.308+0000 7fa8de000640 1 -- 172.21.15.5:0/538888072 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8c0002dd0 con 0x7fa8c8074d40 2024-04-25T10:02:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.308+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 shutdown_connections 2024-04-25T10:02:28.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.308+0000 7f3225200640 1 -- 172.21.15.5:0/2664922033 wait complete. 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.309+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7fa8c8074d40 msgr2=0x7fa8c8077180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.309+0000 7fa8e7200640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7fa8c8074d40 0x7fa8c8077180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.310+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8c806b5d0 msgr2=0x7fa8c806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.310+0000 7fa8e7200640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8c806b5d0 0x7fa8c806da90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8074390 tx=0x7fa8e808d040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.310+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e004e520 msgr2=0x7fa8e004e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.310+0000 7fa8e7200640 1 --2- 172.21.15.5:0/538888072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e004e520 0x7fa8e004e9a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8011830 tx=0x7fa8d8011cf0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.310+0000 7fa8e6800640 1 -- 172.21.15.5:0/538888072 reap_dead start 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.311+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 shutdown_connections 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.311+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 >> 172.21.15.5:0/538888072 conn(0x7fa8e000a400 msgr2=0x7fa8e00d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.311+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 shutdown_connections 2024-04-25T10:02:28.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.311+0000 7fa8e7200640 1 -- 172.21.15.5:0/538888072 wait complete. 2024-04-25T10:02:28.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.315+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d7c146c10 con 0x7f7d5c074c80 2024-04-25T10:02:28.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.315+0000 7fca7dc00640 1 -- 172.21.15.5:0/1587355102 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca78150cc0 con 0x7fca58074bd0 2024-04-25T10:02:28.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.316+0000 7fca6fe00640 1 -- 172.21.15.5:0/1587355102 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fca70001800 con 0x7fca58074bd0 2024-04-25T10:02:28.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.316+0000 7f7d79600640 1 -- 172.21.15.5:0/4148581774 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d7c146c10 con 0x7f7d5c074c80 2024-04-25T10:02:28.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.317+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7f7d5c074c80 msgr2=0x7f7d5c0770c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.317+0000 7f7d84400640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7f7d5c074c80 0x7f7d5c0770c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.317+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7d5c06b500 msgr2=0x7f7d5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.317+0000 7f7d84400640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7d5c06b500 0x7f7d5c06d9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0b4500 tx=0x7f7d68009040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.317+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c0b53b0 msgr2=0x7f7d7c15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.317+0000 7f7d84400640 1 --2- 172.21.15.5:0/4148581774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c0b53b0 0x7f7d7c15e130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000c430 tx=0x7f7d7000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.318+0000 7f7d82400640 1 -- 172.21.15.5:0/4148581774 reap_dead start 2024-04-25T10:02:28.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.318+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 shutdown_connections 2024-04-25T10:02:28.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.318+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 >> 172.21.15.5:0/4148581774 conn(0x7f7d7c072290 msgr2=0x7f7d7c15c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.318+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 shutdown_connections 2024-04-25T10:02:28.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.318+0000 7f7d84400640 1 -- 172.21.15.5:0/4148581774 wait complete. 2024-04-25T10:02:28.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.319+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7fca58074bd0 msgr2=0x7fca58077010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.319+0000 7fca6d600640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7fca58074bd0 0x7fca58077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.319+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca5806b4d0 msgr2=0x7fca5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.319+0000 7fca6d600640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca5806b4d0 0x7fca5806d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca780b06a0 tx=0x7fca64008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.319+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7806e980 msgr2=0x7fca78146600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.319+0000 7fca6d600640 1 --2- 172.21.15.5:0/1587355102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca7806e980 0x7fca78146600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca68009f40 tx=0x7fca68004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.320+0000 7fca7d200640 1 -- 172.21.15.5:0/1587355102 reap_dead start 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.320+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 shutdown_connections 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.320+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 >> 172.21.15.5:0/1587355102 conn(0x7fca78066f20 msgr2=0x7fca78161ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.320+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 shutdown_connections 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.320+0000 7fca6d600640 1 -- 172.21.15.5:0/1587355102 wait complete. 2024-04-25T10:02:28.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.324+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb614068fd0 con 0x7fb5f4074c40 2024-04-25T10:02:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.324+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f593c068f60 con 0x7f5918074cd0 2024-04-25T10:02:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.325+0000 7f5932000640 1 -- 172.21.15.5:0/3377198372 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f593c068f60 con 0x7f5918074cd0 2024-04-25T10:02:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.325+0000 7fb611600640 1 -- 172.21.15.5:0/283114197 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb614068fd0 con 0x7fb5f4074c40 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f5918074cd0 msgr2=0x7f5918077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7fb5f4074c40 msgr2=0x7fb5f4077080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7fb61c200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7fb5f4074c40 0x7fb5f4077080 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb5f406b4d0 msgr2=0x7fb5f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7fb61c200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb5f406b4d0 0x7fb5f406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb600009370 tx=0x7fb600009300 comp rx=0 tx=0).stop 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb614143400 msgr2=0x7fb614135f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7fb61c200640 1 --2- 172.21.15.5:0/283114197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb614143400 0x7fb614135f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb60c032000 tx=0x7fb60c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7f593b200640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f5918074cd0 0x7f5918077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f591806b550 msgr2=0x7f591806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.326+0000 7f593b200640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f591806b550 0x7f591806da10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5934034000 tx=0x7f593402f000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7fb61a200640 1 -- 172.21.15.5:0/283114197 reap_dead start 2024-04-25T10:02:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 shutdown_connections 2024-04-25T10:02:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 >> 172.21.15.5:0/283114197 conn(0x7fb614071150 msgr2=0x7fb61415dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c15d8a0 msgr2=0x7f593c0cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7f593b200640 1 --2- 172.21.15.5:0/3377198372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f593c15d8a0 0x7f593c0cbd90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f592400c4d0 tx=0x7f592400c990 comp rx=0 tx=0).stop 2024-04-25T10:02:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7f593a800640 1 -- 172.21.15.5:0/3377198372 reap_dead start 2024-04-25T10:02:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 shutdown_connections 2024-04-25T10:02:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.327+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 >> 172.21.15.5:0/3377198372 conn(0x7f593c08af80 msgr2=0x7f593c0b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.328+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 shutdown_connections 2024-04-25T10:02:28.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.328+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 shutdown_connections 2024-04-25T10:02:28.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.328+0000 7f593b200640 1 -- 172.21.15.5:0/3377198372 wait complete. 2024-04-25T10:02:28.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.328+0000 7fb61c200640 1 -- 172.21.15.5:0/283114197 wait complete. 2024-04-25T10:02:28.354 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:28.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:28.355 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:28.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:28.356 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:28.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:28.357 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:28.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:28.358 INFO:teuthology.orchestra.run.smithi005.stdout:60129542157 2024-04-25T10:02:28.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:28.359 INFO:teuthology.orchestra.run.smithi005.stdout:64424509452 2024-04-25T10:02:28.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:28.360 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:28.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:28.362 INFO:teuthology.orchestra.run.smithi005.stdout:64424509452 2024-04-25T10:02:28.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:02:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.672+0000 7f55d9800640 1 Processor -- start 2024-04-25T10:02:28.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.672+0000 7f55d9800640 1 -- start start 2024-04-25T10:02:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.674+0000 7f55d9800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f55d40cea90 0x7f55d40b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.674+0000 7f55d9800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d415dbf0 con 0x7f55d40b2800 2024-04-25T10:02:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.674+0000 7f55d9800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d40b9610 con 0x7f55d415d810 2024-04-25T10:02:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.674+0000 7f55d9800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d40b7c00 con 0x7f55d40cea90 2024-04-25T10:02:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.675+0000 7f55d2000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f55d40b2800 0x7f55d40ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51538/0 (socket says 172.21.15.5:51538) 2024-04-25T10:02:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.675+0000 7f55d2000640 1 -- 172.21.15.5:0/1800244455 learned_addr learned my addr 172.21.15.5:0/1800244455 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.675+0000 7f55d3400640 1 --2- 172.21.15.5:0/1800244455 >> v2:172.21.15.80:3300/0 conn(0x7f55d40cea90 0x7f55d40b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.676+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207995956 0 0) 0x7f55d415dbf0 con 0x7f55d40b2800 2024-04-25T10:02:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.677+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55b8003520 con 0x7f55d40b2800 2024-04-25T10:02:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.677+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230606917 0 0) 0x7f55d40b9610 con 0x7f55d415d810 2024-04-25T10:02:28.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.677+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55d415dbf0 con 0x7f55d415d810 2024-04-25T10:02:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.678+0000 7f55d3400640 1 -- 172.21.15.5:0/1800244455 >> v1:172.21.15.80:6789/0 conn(0x7f55d415d810 legacy=0x7f55d40d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.678+0000 7f55d3400640 1 -- 172.21.15.5:0/1800244455 >> v1:172.21.15.5:6789/0 conn(0x7f55d40b2800 legacy=0x7f55d40ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.678+0000 7f55d3400640 1 -- 172.21.15.5:0/1800244455 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d40b72d0 con 0x7f55d40cea90 2024-04-25T10:02:28.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.681+0000 7f55d3400640 1 --2- 172.21.15.5:0/1800244455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b8f10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55c00097a0 tx=0x7f55c002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=189ede6969bca1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.681+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0037070 con 0x7f55d40cea90 2024-04-25T10:02:28.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.682+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c002ace0 con 0x7f55d40cea90 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.682+0000 7f55d9800640 1 -- 172.21.15.5:0/1800244455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 msgr2=0x7f55d40b8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d1600640 1 -- 172.21.15.5:0/1800244455 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0037070 con 0x7f55d40cea90 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d9800640 1 --2- 172.21.15.5:0/1800244455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b8f10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f55c00097a0 tx=0x7f55c002a250 comp rx=0 tx=0).stop 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d9800640 1 -- 172.21.15.5:0/1800244455 shutdown_connections 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d9800640 1 --2- 172.21.15.5:0/1800244455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b8f10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d9800640 1 -- 172.21.15.5:0/1800244455 >> 172.21.15.5:0/1800244455 conn(0x7f55d40726b0 msgr2=0x7f55d4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d9800640 1 -- 172.21.15.5:0/1800244455 shutdown_connections 2024-04-25T10:02:28.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.683+0000 7f55d9800640 1 -- 172.21.15.5:0/1800244455 wait complete. 2024-04-25T10:02:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.686+0000 7f55d9800640 1 Processor -- start 2024-04-25T10:02:28.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.686+0000 7f55d9800640 1 -- start start 2024-04-25T10:02:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f55d40b2800 0x7f55d40b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d9800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55d415d810 0x7f55d4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40b7890 con 0x7f55d415d810 2024-04-25T10:02:28.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d9800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40d20b0 con 0x7f55d40cea90 2024-04-25T10:02:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d9800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d40b8090 con 0x7f55d40b2800 2024-04-25T10:02:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.688+0000 7f55d2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f55d40b2800 0x7f55d40b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.689+0000 7f55d2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f55d40b2800 0x7f55d40b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49092/0 (socket says 172.21.15.5:49092) 2024-04-25T10:02:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.689+0000 7f55d2a00640 1 -- 172.21.15.5:0/3217327506 learned_addr learned my addr 172.21.15.5:0/3217327506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.689+0000 7f55d2000640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.689+0000 7f55d3400640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55d415d810 0x7f55d4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.691+0000 7f55d3400640 1 -- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f55d40b2800 msgr2=0x7f55d40b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.691+0000 7f55d3400640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f55d40b2800 0x7f55d40b55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.691+0000 7f55d3400640 1 -- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 msgr2=0x7f55d40b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.691+0000 7f55d3400640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.691+0000 7f55d3400640 1 -- 172.21.15.5:0/3217327506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d40b4580 con 0x7f55d415d810 2024-04-25T10:02:28.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.692+0000 7f55d3400640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55d415d810 0x7f55d4156a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55c0031000 tx=0x7f55c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.693+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0037050 con 0x7f55d415d810 2024-04-25T10:02:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.693+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c0043070 con 0x7f55d415d810 2024-04-25T10:02:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.693+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0019f0 con 0x7f55d415d810 2024-04-25T10:02:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.693+0000 7f3d53a00640 1 Processor -- start 2024-04-25T10:02:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.693+0000 7f3d53a00640 1 -- start start 2024-04-25T10:02:28.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.694+0000 7f0ccd000640 1 Processor -- start 2024-04-25T10:02:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.694+0000 7f55d9800640 1 -- 172.21.15.5:0/3217327506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d40b47f0 con 0x7f55d415d810 2024-04-25T10:02:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f0ccd000640 1 -- start start 2024-04-25T10:02:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f1852200640 1 Processor -- start 2024-04-25T10:02:28.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f1852200640 1 -- start start 2024-04-25T10:02:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f3d53a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c143400 0x7f3d4c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f3d53a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c06d1f0 con 0x7f3d4c0cf160 2024-04-25T10:02:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f3d53a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c06d4d0 con 0x7f3d4c15b9a0 2024-04-25T10:02:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.695+0000 7f3d53a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c06d7b0 con 0x7f3d4c143400 2024-04-25T10:02:28.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.696+0000 7f3d51a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3d4c0cf160 0x7f3d4c0cb560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51550/0 (socket says 172.21.15.5:51550) 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.696+0000 7f3d51a00640 1 -- 172.21.15.5:0/2570471821 learned_addr learned my addr 172.21.15.5:0/2570471821 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.696+0000 7f3d51000640 1 --2- 172.21.15.5:0/2570471821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c143400 0x7f3d4c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.696+0000 7f1852200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f184c06e980 0x7f184c070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1852200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c0714d0 0x7f184c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1852200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184c0cd2a0 con 0x7f184c1652f0 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1852200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c0ce340 con 0x7f184c06e980 2024-04-25T10:02:28.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1852200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0cdeb0 con 0x7f184c0714d0 2024-04-25T10:02:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0ccd000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0cc80cbb50 0x7f0cc80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0ccd000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0cc80cad50 0x7f0cc80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0ccd000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc80c9f90 con 0x7f0cc81472c0 2024-04-25T10:02:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0ccd000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc80ca270 con 0x7f0cc80cad50 2024-04-25T10:02:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0ccd000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc80ca550 con 0x7f0cc80cbb50 2024-04-25T10:02:28.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f55d9800640 1 -- 172.21.15.5:0/3217327506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d40b4db0 con 0x7f55d415d810 2024-04-25T10:02:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1850e00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f184c1652f0 0x7f184c0cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35524/0 (socket says 172.21.15.5:35524) 2024-04-25T10:02:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1850e00640 1 -- 172.21.15.5:0/614606400 learned_addr learned my addr 172.21.15.5:0/614606400 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0cc6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0cc80cad50 0x7f0cc80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f3d4b400640 1 -- 172.21.15.5:0/2570471821 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423901856 0 0) 0x7f3d4c06d4d0 con 0x7f3d4c15b9a0 2024-04-25T10:02:28.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f3d4b400640 1 -- 172.21.15.5:0/2570471821 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d30003520 con 0x7f3d4c15b9a0 2024-04-25T10:02:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f0cc6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0cc80cad50 0x7f0cc80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46578/0 (socket says 172.21.15.5:46578) 2024-04-25T10:02:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f0cc6800640 1 -- 172.21.15.5:0/315708741 learned_addr learned my addr 172.21.15.5:0/315708741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.697+0000 7f1851800640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c0714d0 0x7f184c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f184be00640 1 --2- 172.21.15.5:0/614606400 >> v2:172.21.15.80:3300/0 conn(0x7f184c06e980 0x7f184c070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f3d51000640 1 -- 172.21.15.5:0/2570471821 >> v1:172.21.15.80:6789/0 conn(0x7f3d4c15b9a0 legacy=0x7f3d4c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f3d51000640 1 -- 172.21.15.5:0/2570471821 >> v1:172.21.15.5:6789/0 conn(0x7f3d4c0cf160 legacy=0x7f3d4c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f3d51000640 1 -- 172.21.15.5:0/2570471821 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c06da90 con 0x7f3d4c143400 2024-04-25T10:02:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f0cc5400640 1 --2- 172.21.15.5:0/315708741 >> v2:172.21.15.80:3300/0 conn(0x7f0cc80cbb50 0x7f0cc80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f184be00640 1 -- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c0714d0 msgr2=0x7f184c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.698+0000 7f184be00640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c0714d0 0x7f184c06cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f184be00640 1 -- 172.21.15.5:0/614606400 >> v1:172.21.15.80:6789/0 conn(0x7f184c1652f0 legacy=0x7f184c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f184be00640 1 -- 172.21.15.5:0/614606400 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c0cd580 con 0x7f184c06e980 2024-04-25T10:02:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f1851800640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c0714d0 0x7f184c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f0cc4a00640 1 -- 172.21.15.5:0/315708741 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544361457 0 0) 0x7f0cc80c9f90 con 0x7f0cc81472c0 2024-04-25T10:02:28.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f0cc4a00640 1 -- 172.21.15.5:0/315708741 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cac003520 con 0x7f0cc81472c0 2024-04-25T10:02:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8005290 con 0x7f55d415d810 2024-04-25T10:02:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f0cc5400640 1 -- 172.21.15.5:0/315708741 >> v2:172.21.15.5:3300/0 conn(0x7f0cc80cad50 msgr2=0x7f0cc80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f0cc5400640 1 --2- 172.21.15.5:0/315708741 >> v2:172.21.15.5:3300/0 conn(0x7f0cc80cad50 0x7f0cc80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f0cc5400640 1 -- 172.21.15.5:0/315708741 >> v1:172.21.15.80:6789/0 conn(0x7f0cc81472c0 legacy=0x7f0cc80cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.699+0000 7f0cc5400640 1 -- 172.21.15.5:0/315708741 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80ca830 con 0x7f0cc80cbb50 2024-04-25T10:02:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f3d51000640 1 --2- 172.21.15.5:0/2570471821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c143400 0x7f3d4c15b460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3d380097d0 tx=0x7f3d3802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a716f28e0ac37fa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f184be00640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c06e980 0x7f184c070f90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18380097a0 tx=0x7f183802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaf4f4ad44070426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f184b400640 1 -- 172.21.15.5:0/614606400 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838037070 con 0x7f184c06e980 2024-04-25T10:02:28.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f3d4b400640 1 -- 172.21.15.5:0/2570471821 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d38037070 con 0x7f3d4c143400 2024-04-25T10:02:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f184b400640 1 -- 172.21.15.5:0/614606400 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183802ae40 con 0x7f184c06e980 2024-04-25T10:02:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f184b400640 1 -- 172.21.15.5:0/614606400 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838032660 con 0x7f184c06e980 2024-04-25T10:02:28.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.701+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55c002ace0 con 0x7f55d415d810 2024-04-25T10:02:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.702+0000 7f360ac00640 1 Processor -- start 2024-04-25T10:02:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.702+0000 7f360ac00640 1 -- start start 2024-04-25T10:02:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.702+0000 7f3d4b400640 1 -- 172.21.15.5:0/2570471821 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d38032440 con 0x7f3d4c143400 2024-04-25T10:02:28.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f55caa00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f55b406b5a0 0x7f55b406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.702+0000 7f0cc5400640 1 --2- 172.21.15.5:0/315708741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc80ce670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0cc806c5b0 tx=0x7f0cc002a570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bffa670ea1626d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f55c00aab20 con 0x7f55d415d810 2024-04-25T10:02:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f0cc4a00640 1 -- 172.21.15.5:0/315708741 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0007740 con 0x7f0cc80cbb50 2024-04-25T10:02:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f0cc4a00640 1 -- 172.21.15.5:0/315708741 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc00078e0 con 0x7f0cc80cbb50 2024-04-25T10:02:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f0cc4a00640 1 -- 172.21.15.5:0/315708741 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0007740 con 0x7f0cc80cbb50 2024-04-25T10:02:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f3d53a00640 1 -- 172.21.15.5:0/2570471821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c143400 msgr2=0x7f3d4c15b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f1852200640 1 -- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c06e980 msgr2=0x7f184c070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f1852200640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c06e980 0x7f184c070f90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18380097a0 tx=0x7f183802a130 comp rx=0 tx=0).stop 2024-04-25T10:02:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.703+0000 7f3d4b400640 1 -- 172.21.15.5:0/2570471821 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380327f0 con 0x7f3d4c143400 2024-04-25T10:02:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f3d53a00640 1 --2- 172.21.15.5:0/2570471821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c143400 0x7f3d4c15b460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3d380097d0 tx=0x7f3d3802a210 comp rx=0 tx=0).stop 2024-04-25T10:02:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f1852200640 1 -- 172.21.15.5:0/614606400 shutdown_connections 2024-04-25T10:02:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f1852200640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c0714d0 0x7f184c06cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f1852200640 1 --2- 172.21.15.5:0/614606400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c06e980 0x7f184c070f90 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f18380097a0 tx=0x7f183802a130 comp rx=0 tx=0).stop 2024-04-25T10:02:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f1852200640 1 -- 172.21.15.5:0/614606400 >> 172.21.15.5:0/614606400 conn(0x7f184c066f20 msgr2=0x7f184c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f3d53a00640 1 -- 172.21.15.5:0/2570471821 shutdown_connections 2024-04-25T10:02:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f1852200640 1 -- 172.21.15.5:0/614606400 shutdown_connections 2024-04-25T10:02:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f55d2a00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f55b406b5a0 0x7f55b406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f3d53a00640 1 --2- 172.21.15.5:0/2570471821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c143400 0x7f3d4c15b460 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f3d53a00640 1 -- 172.21.15.5:0/2570471821 >> 172.21.15.5:0/2570471821 conn(0x7f3d4c071150 msgr2=0x7f3d4c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f0ccd000640 1 -- 172.21.15.5:0/315708741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 msgr2=0x7f0cc80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.704+0000 7f0ccd000640 1 --2- 172.21.15.5:0/315708741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc80ce670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0cc806c5b0 tx=0x7f0cc002a570 comp rx=0 tx=0).stop 2024-04-25T10:02:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f0ccd000640 1 -- 172.21.15.5:0/315708741 shutdown_connections 2024-04-25T10:02:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f0ccd000640 1 --2- 172.21.15.5:0/315708741 >> v2:172.21.15.5:3300/0 conn(0x7f0cc80cad50 0x7f0cc80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f0ccd000640 1 --2- 172.21.15.5:0/315708741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc80ce670 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f0ccd000640 1 -- 172.21.15.5:0/315708741 >> 172.21.15.5:0/315708741 conn(0x7f0cc8071150 msgr2=0x7f0cc8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f3d53a00640 1 -- 172.21.15.5:0/2570471821 shutdown_connections 2024-04-25T10:02:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f3d53a00640 1 -- 172.21.15.5:0/2570471821 wait complete. 2024-04-25T10:02:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f360ac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360415da20 0x7f360409a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f360ac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f360406c330 0x7f360406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f360ac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040cfad0 con 0x7f360406bc70 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f360ac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360406ccd0 con 0x7f360406c330 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f360ac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36040d06e0 con 0x7f360415da20 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f3608c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f360406c330 0x7f360406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.706+0000 7f3608c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f360406c330 0x7f360406c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46592/0 (socket says 172.21.15.5:46592) 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.706+0000 7f3608c00640 1 -- 172.21.15.5:0/1695497034 learned_addr learned my addr 172.21.15.5:0/1695497034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.706+0000 7f55d2a00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f55b406b5a0 0x7f55b406da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55c4007fd0 tx=0x7f55c4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.706+0000 7f3603e00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360415da20 0x7f360409a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f3608c00640 1 -- 172.21.15.5:0/1695497034 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360415da20 msgr2=0x7f360409a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f1533200640 1 Processor -- start 2024-04-25T10:02:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f0ccd000640 1 -- 172.21.15.5:0/315708741 shutdown_connections 2024-04-25T10:02:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f3d53a00640 1 Processor -- start 2024-04-25T10:02:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.708+0000 7f3d53a00640 1 -- start start 2024-04-25T10:02:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f3608c00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360415da20 0x7f360409a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f3608c00640 1 -- 172.21.15.5:0/1695497034 >> v1:172.21.15.80:6789/0 conn(0x7f360406bc70 legacy=0x7f36040d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f3608c00640 1 -- 172.21.15.5:0/1695497034 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040cfdb0 con 0x7f360406c330 2024-04-25T10:02:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f3603e00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360415da20 0x7f360409a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.705+0000 7f1852200640 1 -- 172.21.15.5:0/614606400 wait complete. 2024-04-25T10:02:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f1852200640 1 Processor -- start 2024-04-25T10:02:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.707+0000 7f1533200640 1 -- start start 2024-04-25T10:02:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3608c00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406c330 0x7f360406c790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009e50 tx=0x7f35fc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb00336f8864afeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3d53a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d4c143400 0x7f3d4c13fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3d53a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 0x7f3d4c141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3d53a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d4c141b60 0x7f3d4c14e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3d53a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c15af90 con 0x7f3d4c143400 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3d53a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c15ae10 con 0x7f3d4c141b60 2024-04-25T10:02:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.709+0000 7f3d53a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c0cbc70 con 0x7f3d4c15b9a0 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.708+0000 7f1852200640 1 -- start start 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1533200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15341509b0 0x7f15340b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1852200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c0714d0 0x7f184c0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1852200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 0x7f184c153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1852200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c1535c0 0x7f184c162af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1852200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c06d530 con 0x7f184c1652f0 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1852200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0cdb40 con 0x7f184c0714d0 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f1852200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c06d6b0 con 0x7f184c1535c0 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f3d4be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 0x7f3d4c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f184be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 0x7f184c153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f184be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 0x7f184c153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46626/0 (socket says 172.21.15.5:46626) 2024-04-25T10:02:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f184be00640 1 -- 172.21.15.5:0/2971943057 learned_addr learned my addr 172.21.15.5:0/2971943057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1851800640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c1535c0 0x7f184c162af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f3d4be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 0x7f3d4c141620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49136/0 (socket says 172.21.15.5:49136) 2024-04-25T10:02:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1533200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340cc480 0x7f15340cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1531400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f15340ae6c0 0x7f15340cbd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51560/0 (socket says 172.21.15.5:51560) 2024-04-25T10:02:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f3d4be00640 1 -- 172.21.15.5:0/4222654752 learned_addr learned my addr 172.21.15.5:0/4222654752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f3d51a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d4c141b60 0x7f3d4c14e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f3d51000640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d4c143400 0x7f3d4c13fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1531400640 1 -- 172.21.15.5:0/4085863981 learned_addr learned my addr 172.21.15.5:0/4085863981 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153415a100 con 0x7f15340ae6c0 2024-04-25T10:02:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153415a3e0 con 0x7f15341509b0 2024-04-25T10:02:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153415a6c0 con 0x7f15340cc480 2024-04-25T10:02:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.708+0000 7f0ccd000640 1 -- 172.21.15.5:0/315708741 wait complete. 2024-04-25T10:02:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1531e00640 1 --2- 172.21.15.5:0/4085863981 >> v2:172.21.15.5:3300/0 conn(0x7f15341509b0 0x7f15340b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.711+0000 7f1532800640 1 --2- 172.21.15.5:0/4085863981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340cc480 0x7f15340cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f0ccd000640 1 Processor -- start 2024-04-25T10:02:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.710+0000 7f0ccd000640 1 -- start start 2024-04-25T10:02:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f3d51a00640 1 -- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 msgr2=0x7f3d4c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f3d51a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 0x7f3d4c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f3d51a00640 1 -- 172.21.15.5:0/4222654752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d4c143400 msgr2=0x7f3d4c13fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f0ccd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cc80cad50 0x7f0cc80c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f0ccd000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc8145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f0ccd000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0cc81472c0 0x7f0cc8158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f1530a00640 1 -- 172.21.15.5:0/4085863981 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437794031 0 0) 0x7f153415a100 con 0x7f15340ae6c0 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f184be00640 1 -- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c1535c0 msgr2=0x7f184c162af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f184be00640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c1535c0 0x7f184c162af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f184be00640 1 -- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c0714d0 msgr2=0x7f184c0adba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f3d51a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d4c143400 0x7f3d4c13fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f184be00640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c0714d0 0x7f184c0adba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f184be00640 1 -- 172.21.15.5:0/2971943057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c1639a0 con 0x7f184c1652f0 2024-04-25T10:02:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55c00796d0 con 0x7f55d415d810 2024-04-25T10:02:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0ccd000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc806d150 con 0x7f0cc80cad50 2024-04-25T10:02:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0ccd000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c9c60 con 0x7f0cc80cbb50 2024-04-25T10:02:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0ccd000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc806d2d0 con 0x7f0cc81472c0 2024-04-25T10:02:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f3d51a00640 1 -- 172.21.15.5:0/4222654752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c140580 con 0x7f3d4c141b60 2024-04-25T10:02:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f3d4be00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 0x7f3d4c141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f184be00640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 0x7f184c153080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1838035040 tx=0x7f18380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f1530a00640 1 -- 172.21.15.5:0/4085863981 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1518003520 con 0x7f15340ae6c0 2024-04-25T10:02:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0cc5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc8145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0cc5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc8145830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44208/0 (socket says 172.21.15.5:44208) 2024-04-25T10:02:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0cc5400640 1 -- 172.21.15.5:0/2593060682 learned_addr learned my addr 172.21.15.5:0/2593060682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f3d51000640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d4c143400 0x7f3d4c13fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0cc6800640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0cc81472c0 0x7f0cc8158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f3d51a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d4c141b60 0x7f3d4c14e750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0cc2f0 tx=0x7f3d3c00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.713+0000 7f0cc5e00640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cc80cad50 0x7f0cc80c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f0cc5400640 1 -- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0cc81472c0 msgr2=0x7f0cc8158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f0cc5400640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0cc81472c0 0x7f0cc8158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c017070 con 0x7f3d4c141b60 2024-04-25T10:02:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c005ce0 con 0x7f3d4c141b60 2024-04-25T10:02:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.712+0000 7f3602a00640 1 -- 172.21.15.5:0/1695497034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc036070 con 0x7f360406c330 2024-04-25T10:02:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3602a00640 1 -- 172.21.15.5:0/1695497034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc031600 con 0x7f360406c330 2024-04-25T10:02:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3602a00640 1 -- 172.21.15.5:0/1695497034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0319d0 con 0x7f360406c330 2024-04-25T10:02:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c0072f0 con 0x7f3d4c141b60 2024-04-25T10:02:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f0cc5400640 1 -- 172.21.15.5:0/2593060682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cc80cad50 msgr2=0x7f0cc80c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f0cc5400640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cc80cad50 0x7f0cc80c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f0cc5400640 1 -- 172.21.15.5:0/2593060682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80c72c0 con 0x7f0cc80cbb50 2024-04-25T10:02:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f0cc5e00640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cc80cad50 0x7f0cc80c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c140860 con 0x7f3d4c141b60 2024-04-25T10:02:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c140e20 con 0x7f3d4c141b60 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838037040 con 0x7f184c1652f0 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f1852200640 1 -- 172.21.15.5:0/2971943057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c163c80 con 0x7f184c1652f0 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f0cc5400640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc8145830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009e50 tx=0x7f0cc0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f360ac00640 1 -- 172.21.15.5:0/1695497034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406c330 msgr2=0x7f360406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f360ac00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406c330 0x7f360406c790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009e50 tx=0x7f35fc02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f360ac00640 1 -- 172.21.15.5:0/1695497034 shutdown_connections 2024-04-25T10:02:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f360ac00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406c330 0x7f360406c790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f360ac00640 1 --2- 172.21.15.5:0/1695497034 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360415da20 0x7f360409a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.715+0000 7f360ac00640 1 -- 172.21.15.5:0/1695497034 >> 172.21.15.5:0/1695497034 conn(0x7f36040d16f0 msgr2=0x7f36040d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c068fd0 con 0x7f3d4c141b60 2024-04-25T10:02:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f360ac00640 1 -- 172.21.15.5:0/1695497034 shutdown_connections 2024-04-25T10:02:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f360ac00640 1 -- 172.21.15.5:0/1695497034 wait complete. 2024-04-25T10:02:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004240 con 0x7f0cc80cbb50 2024-04-25T10:02:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc00043e0 con 0x7f0cc80cbb50 2024-04-25T10:02:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0004790 con 0x7f0cc80cbb50 2024-04-25T10:02:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f6df9c00640 1 Processor -- start 2024-04-25T10:02:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f47dc000640 1 Processor -- start 2024-04-25T10:02:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.717+0000 7f6df9c00640 1 -- start start 2024-04-25T10:02:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc80c75a0 con 0x7f0cc80cbb50 2024-04-25T10:02:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc8142f10 con 0x7f0cc80cbb50 2024-04-25T10:02:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.716+0000 7f1852200640 1 -- 172.21.15.5:0/2971943057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c164240 con 0x7f184c1652f0 2024-04-25T10:02:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.718+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1828005290 con 0x7f184c1652f0 2024-04-25T10:02:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.718+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838004020 con 0x7f184c1652f0 2024-04-25T10:02:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.718+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838006e10 con 0x7f184c1652f0 2024-04-25T10:02:28.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.718+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc8068fd0 con 0x7f0cc80cbb50 2024-04-25T10:02:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.718+0000 7f360ac00640 1 Processor -- start 2024-04-25T10:02:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f360ac00640 1 -- start start 2024-04-25T10:02:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df9c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415da50 0x7f6df40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df9c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6df415de30 0x7f6df415e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df9c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6df40d0280 0x7f6df40d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df9c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40d0c50 con 0x7f6df40d0280 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df9c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df40ce2b0 con 0x7f6df415de30 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df9c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df415e960 con 0x7f6df415da50 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df2000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6df415de30 0x7f6df415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df2000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6df415de30 0x7f6df415e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44222/0 (socket says 172.21.15.5:44222) 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f6df2000640 1 -- 172.21.15.5:0/2967277675 learned_addr learned my addr 172.21.15.5:0/2967277675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df3400640 1 --2- 172.21.15.5:0/2967277675 >> v2:172.21.15.5:3300/0 conn(0x7f6df40d0280 0x7f6df40d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f1531e00640 1 -- 172.21.15.5:0/4085863981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340cc480 msgr2=0x7f15340cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f1531e00640 1 --2- 172.21.15.5:0/4085863981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340cc480 0x7f15340cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f1531e00640 1 -- 172.21.15.5:0/4085863981 >> v1:172.21.15.5:6789/0 conn(0x7f15340ae6c0 legacy=0x7f15340cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.714+0000 7f1531e00640 1 -- 172.21.15.5:0/4085863981 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153415a9a0 con 0x7f15341509b0 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f1531e00640 1 --2- 172.21.15.5:0/4085863981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15341509b0 0x7f15340b2f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f152c0097a0 tx=0x7f152c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1322dfc860c124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df2a00640 1 --2- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415da50 0x7f6df40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df2000640 1 -- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415da50 msgr2=0x7f6df40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df2000640 1 --2- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415da50 0x7f6df40cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df2000640 1 -- 172.21.15.5:0/2967277675 >> v2:172.21.15.5:3300/0 conn(0x7f6df40d0280 msgr2=0x7f6df40d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f360ac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 0x7f36040ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df2000640 1 --2- 172.21.15.5:0/2967277675 >> v2:172.21.15.5:3300/0 conn(0x7f6df40d0280 0x7f6df40d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f6df2000640 1 -- 172.21.15.5:0/2967277675 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df415ec40 con 0x7f6df415de30 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f360ac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360406c330 0x7f360409ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f360ac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f360415da20 0x7f36040aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f360ac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36040d0370 con 0x7f360406bc70 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f360ac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360409a940 con 0x7f360415da20 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f360ac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36040d1280 con 0x7f360406c330 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f3603e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 0x7f36040ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.717+0000 7f47dc000640 1 -- start start 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f47dc000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47d415da50 0x7f47d40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f47dc000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47d415de30 0x7f47d415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f47dc000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d40ce2b0 con 0x7f47d40d0280 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f47dc000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d415e960 con 0x7f47d415de30 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f47dc000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d415ec40 con 0x7f47d415da50 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.719+0000 7f47d9600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47d415da50 0x7f47d40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f47d9600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47d415da50 0x7f47d40cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44210/0 (socket says 172.21.15.5:44210) 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f3603e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 0x7f36040ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46652/0 (socket says 172.21.15.5:46652) 2024-04-25T10:02:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f3603e00640 1 -- 172.21.15.5:0/215263732 learned_addr learned my addr 172.21.15.5:0/215263732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f47d9600640 1 -- 172.21.15.5:0/2051283280 learned_addr learned my addr 172.21.15.5:0/2051283280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f47d9600640 1 -- 172.21.15.5:0/2051283280 >> v2:172.21.15.5:3300/0 conn(0x7f47d415de30 msgr2=0x7f47d415e2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f47d9600640 1 --2- 172.21.15.5:0/2051283280 >> v2:172.21.15.5:3300/0 conn(0x7f47d415de30 0x7f47d415e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f47d9600640 1 -- 172.21.15.5:0/2051283280 >> v1:172.21.15.5:6789/0 conn(0x7f47d40d0280 legacy=0x7f47d40d06b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.720+0000 7f47d9600640 1 -- 172.21.15.5:0/2051283280 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40ce570 con 0x7f47d415da50 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f3603400640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360406c330 0x7f360409ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.721+0000 7f3608c00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f360415da20 0x7f36040aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d3c01a070 con 0x7f3d4c141b60 2024-04-25T10:02:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f6df3400640 1 --2- 172.21.15.5:0/2967277675 >> v2:172.21.15.5:3300/0 conn(0x7f6df40d0280 0x7f6df40d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f183803a070 con 0x7f184c1652f0 2024-04-25T10:02:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f47d9600640 1 --2- 172.21.15.5:0/2051283280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415da50 0x7f47d40cea90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f47d408b970 tx=0x7f47c002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb65641e8d8d59ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f6df2000640 1 --2- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df415de30 0x7f6df415e290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6df40b6430 tx=0x7f6de402a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a314e16ed380cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3603400640 1 -- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f360415da20 msgr2=0x7f36040aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3603400640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f360415da20 0x7f36040aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3603400640 1 -- 172.21.15.5:0/215263732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 msgr2=0x7f36040ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3603400640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 0x7f36040ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3603400640 1 -- 172.21.15.5:0/215263732 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040cb060 con 0x7f360406c330 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.722+0000 7f3603e00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 0x7f36040ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.723+0000 7f3603400640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360406c330 0x7f360409ee00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f35f400b3e0 tx=0x7f35f400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.723+0000 7f3d48c00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.723+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d3c0899e0 con 0x7f3d4c141b60 2024-04-25T10:02:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.723+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4015020 con 0x7f360406c330 2024-04-25T10:02:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.723+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4011830 con 0x7f360406c330 2024-04-25T10:02:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36040cb340 con 0x7f360406c330 2024-04-25T10:02:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f1848c00640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4011c00 con 0x7f360406c330 2024-04-25T10:02:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36040cb900 con 0x7f360406c330 2024-04-25T10:02:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f3d51000640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f18380aae50 con 0x7f184c1652f0 2024-04-25T10:02:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.724+0000 7f1850e00640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.725+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3604066dc0 con 0x7f360406c330 2024-04-25T10:02:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.725+0000 7f1850e00640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182c06b4d0 0x7f182c06d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18400096b0 tx=0x7f1840006b60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.725+0000 7f3d51000640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d38034000 tx=0x7f3d3802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.727+0000 7f1530a00640 1 -- 172.21.15.5:0/4085863981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c037070 con 0x7f15341509b0 2024-04-25T10:02:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.727+0000 7f1530a00640 1 -- 172.21.15.5:0/4085863981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c02ae40 con 0x7f15341509b0 2024-04-25T10:02:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.727+0000 7f1530a00640 1 -- 172.21.15.5:0/4085863981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c032660 con 0x7f15341509b0 2024-04-25T10:02:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.727+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f183804a030 con 0x7f184c1652f0 2024-04-25T10:02:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35f4018020 con 0x7f360406c330 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15341509b0 msgr2=0x7f15340b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 --2- 172.21.15.5:0/4085863981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15341509b0 0x7f15340b2f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f152c0097a0 tx=0x7f152c02a0f0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 shutdown_connections 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 --2- 172.21.15.5:0/4085863981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340cc480 0x7f15340cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 --2- 172.21.15.5:0/4085863981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15341509b0 0x7f15340b2f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 >> 172.21.15.5:0/4085863981 conn(0x7f1534066f20 msgr2=0x7f153406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 shutdown_connections 2024-04-25T10:02:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.728+0000 7f1533200640 1 -- 172.21.15.5:0/4085863981 wait complete. 2024-04-25T10:02:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.729+0000 7f35fbe00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f35e406b4d0 0x7f35e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.729+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f35f408a820 con 0x7f360406c330 2024-04-25T10:02:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.730+0000 7f3603e00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f35e406b4d0 0x7f35e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.730+0000 7f1533200640 1 Processor -- start 2024-04-25T10:02:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.730+0000 7f1533200640 1 -- start start 2024-04-25T10:02:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f3603e00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f35e406b4d0 0x7f35e406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35f0006d40 tx=0x7f35f0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1533200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 0x7f15340b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1533200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15340cc480 0x7f1534153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1533200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15341509b0 0x7f1534144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1533200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153415e430 con 0x7f15340ae6c0 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1533200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153415e2b0 con 0x7f15340cc480 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1533200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153406f080 con 0x7f15341509b0 2024-04-25T10:02:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.731+0000 7f1531e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 0x7f15340b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 0x7f15340b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46660/0 (socket says 172.21.15.5:46660) 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 -- 172.21.15.5:0/1115293197 learned_addr learned my addr 172.21.15.5:0/1115293197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531400640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15340cc480 0x7f1534153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1532800640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15341509b0 0x7f1534144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 -- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15341509b0 msgr2=0x7f1534144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15341509b0 0x7f1534144c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 -- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15340cc480 msgr2=0x7f1534153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15340cc480 0x7f1534153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 -- 172.21.15.5:0/1115293197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15341560f0 con 0x7f15340ae6c0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.732+0000 7f1531e00640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 0x7f15340b0150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f152c035040 tx=0x7f152c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.733+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c041070 con 0x7f15340ae6c0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.733+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c03c460 con 0x7f15340ae6c0 2024-04-25T10:02:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.733+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c037040 con 0x7f15340ae6c0 2024-04-25T10:02:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.734+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15341563d0 con 0x7f15340ae6c0 2024-04-25T10:02:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.734+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1534156990 con 0x7f15340ae6c0 2024-04-25T10:02:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d3c021020 con 0x7f3d4c141b60 2024-04-25T10:02:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f47d3e00640 1 -- 172.21.15.5:0/2051283280 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0037070 con 0x7f47d415da50 2024-04-25T10:02:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f47d3e00640 1 -- 172.21.15.5:0/2051283280 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0032440 con 0x7f47d415da50 2024-04-25T10:02:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f47d3e00640 1 -- 172.21.15.5:0/2051283280 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0032810 con 0x7f47d415da50 2024-04-25T10:02:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1534068f60 con 0x7f15340ae6c0 2024-04-25T10:02:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f6df1600640 1 -- 172.21.15.5:0/2967277675 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4037070 con 0x7f6df415de30 2024-04-25T10:02:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f6df1600640 1 -- 172.21.15.5:0/2967277675 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4032440 con 0x7f6df415de30 2024-04-25T10:02:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.735+0000 7f6df1600640 1 -- 172.21.15.5:0/2967277675 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4032850 con 0x7f6df415de30 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 -- 172.21.15.5:0/2051283280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415da50 msgr2=0x7f47d40cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 --2- 172.21.15.5:0/2051283280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415da50 0x7f47d40cea90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f47d408b970 tx=0x7f47c002a250 comp rx=0 tx=0).stop 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 -- 172.21.15.5:0/2051283280 shutdown_connections 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 --2- 172.21.15.5:0/2051283280 >> v2:172.21.15.5:3300/0 conn(0x7f47d415de30 0x7f47d415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 --2- 172.21.15.5:0/2051283280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415da50 0x7f47d40cea90 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f47d408b970 tx=0x7f47c002a250 comp rx=0 tx=0).stop 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 -- 172.21.15.5:0/2051283280 >> 172.21.15.5:0/2051283280 conn(0x7f47d4072290 msgr2=0x7f47d4072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f6df9c00640 1 -- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df415de30 msgr2=0x7f6df415e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df415de30 0x7f6df415e290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6df40b6430 tx=0x7f6de402a200 comp rx=0 tx=0).stop 2024-04-25T10:02:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 -- 172.21.15.5:0/2051283280 shutdown_connections 2024-04-25T10:02:28.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f6df9c00640 1 -- 172.21.15.5:0/2967277675 shutdown_connections 2024-04-25T10:02:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2967277675 >> v2:172.21.15.5:3300/0 conn(0x7f6df40d0280 0x7f6df40d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df415de30 0x7f6df415e290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.737+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2967277675 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415da50 0x7f6df40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.737+0000 7f6df9c00640 1 -- 172.21.15.5:0/2967277675 >> 172.21.15.5:0/2967277675 conn(0x7f6df4072290 msgr2=0x7f6df4072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f47dc000640 1 -- 172.21.15.5:0/2051283280 wait complete. 2024-04-25T10:02:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.738+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cc0039040 con 0x7f0cc80cbb50 2024-04-25T10:02:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.738+0000 7f6df9c00640 1 -- 172.21.15.5:0/2967277675 shutdown_connections 2024-04-25T10:02:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.736+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f152c03a070 con 0x7f15340ae6c0 2024-04-25T10:02:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.738+0000 7f152a000640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151006b500 0x7f151006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.738+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f152c0aa7c0 con 0x7f15340ae6c0 2024-04-25T10:02:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.738+0000 7f6df9c00640 1 -- 172.21.15.5:0/2967277675 wait complete. 2024-04-25T10:02:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.739+0000 7f47dc000640 1 Processor -- start 2024-04-25T10:02:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.739+0000 7f47dc000640 1 -- start start 2024-04-25T10:02:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.740+0000 7f1531400640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151006b500 0x7f151006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f0cbe000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ca806b520 0x7f0ca806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f1531400640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151006b500 0x7f151006d9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15340b20d0 tx=0x7f1524001a90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f47dc000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 0x7f47d40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f47dc000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415de30 0x7f47d40cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f47dc000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47d41536f0 0x7f47d40ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f47dc000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41617c0 con 0x7f47d40d0280 2024-04-25T10:02:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f47dc000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4161640 con 0x7f47d415de30 2024-04-25T10:02:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f47dc000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40d1d40 con 0x7f47d41536f0 2024-04-25T10:02:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f6df9c00640 1 Processor -- start 2024-04-25T10:02:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.741+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0cc00abda0 con 0x7f0cc80cbb50 2024-04-25T10:02:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.742+0000 7f6df9c00640 1 -- start start 2024-04-25T10:02:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.742+0000 7f0cc5e00640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ca806b520 0x7f0ca806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.742+0000 7f47d9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 0x7f47d40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.742+0000 7f47d9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 0x7f47d40cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46666/0 (socket says 172.21.15.5:46666) 2024-04-25T10:02:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.742+0000 7f47d9600640 1 -- 172.21.15.5:0/2547933334 learned_addr learned my addr 172.21.15.5:0/2547933334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.742+0000 7f47da000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47d41536f0 0x7f47d40ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f47d8c00640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415de30 0x7f47d40cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f6df9c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df40d0280 0x7f6df41f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f47d9600640 1 -- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47d41536f0 msgr2=0x7f47d40ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f47d9600640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47d41536f0 0x7f47d40ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f47d9600640 1 -- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415de30 msgr2=0x7f47d40cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f47d9600640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415de30 0x7f47d40cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f6df9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 0x7f6df4154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.743+0000 7f6df9c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415de30 0x7f6df40d2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f0cc5e00640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ca806b520 0x7f0ca806d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000f60 tx=0x7f0cb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 0x7f6df4154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 0x7f6df4154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46674/0 (socket says 172.21.15.5:46674) 2024-04-25T10:02:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df9c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40cf150 con 0x7f6df415da50 2024-04-25T10:02:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df9c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df40cefd0 con 0x7f6df40d0280 2024-04-25T10:02:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df9c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df4161640 con 0x7f6df415de30 2024-04-25T10:02:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df2000640 1 -- 172.21.15.5:0/2848474418 learned_addr learned my addr 172.21.15.5:0/2848474418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f47d9600640 1 -- 172.21.15.5:0/2547933334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d415aa90 con 0x7f47d40d0280 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f47da000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47d41536f0 0x7f47d40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f47d9600640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 0x7f47d40cc160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000c00 tx=0x7f47c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df2a00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df40d0280 0x7f6df41f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.744+0000 7f6df3400640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415de30 0x7f6df40d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df2000640 1 -- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415de30 msgr2=0x7f6df40d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df2000640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415de30 0x7f6df40d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df2000640 1 -- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df40d0280 msgr2=0x7f6df41f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df2000640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df40d0280 0x7f6df41f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df2000640 1 -- 172.21.15.5:0/2848474418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df40cbf20 con 0x7f6df415da50 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df3400640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415de30 0x7f6df40d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.745+0000 7f6df2000640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 0x7f6df4154c50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6de402a240 tx=0x7f6de40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.746+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3604066dc0 con 0x7f360406c330 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.747+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0041070 con 0x7f47d40d0280 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.747+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c003c870 con 0x7f47d40d0280 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.747+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0037040 con 0x7f47d40d0280 2024-04-25T10:02:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.747+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f152c04a030 con 0x7f15340ae6c0 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.747+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4041070 con 0x7f6df415da50 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.748+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de403c890 con 0x7f6df415da50 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.748+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4037040 con 0x7f6df415da50 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.748+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df40b4030 con 0x7f6df415da50 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.748+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df40b45f0 con 0x7f6df415da50 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.748+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d4153bd0 con 0x7f47d40d0280 2024-04-25T10:02:28.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.748+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d4154190 con 0x7f47d40d0280 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.750+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0cc0040030 con 0x7f0cc80cbb50 2024-04-25T10:02:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.750+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df4068fd0 con 0x7f6df415da50 2024-04-25T10:02:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.751+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d4068fd0 con 0x7f47d40d0280 2024-04-25T10:02:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.751+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47c0044070 con 0x7f47d40d0280 2024-04-25T10:02:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.752+0000 7f47d1600640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47b406b4d0 0x7f47b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.753+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f47c00778f0 con 0x7f47d40d0280 2024-04-25T10:02:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.753+0000 7f47d8c00640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47b406b4d0 0x7f47b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.753+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6de4044070 con 0x7f6df415da50 2024-04-25T10:02:28.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.754+0000 7f6deaa00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6dd406b5a0 0x7f6dd406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.755+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6de40aaee0 con 0x7f6df415da50 2024-04-25T10:02:28.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.755+0000 7f6df2a00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6dd406b5a0 0x7f6dd406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.756+0000 7f47d8c00640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47b406b4d0 0x7f47b406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47d40ca420 tx=0x7f47c8009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.756+0000 7f6df2a00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6dd406b5a0 0x7f6dd406da60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6df40b72f0 tx=0x7f6de0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.765+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47d415aa90 con 0x7f47d40d0280 2024-04-25T10:02:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:28.775+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6de4079a90 con 0x7f6df415da50 2024-04-25T10:02:29.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.145+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1828005850 con 0x7f184c1652f0 2024-04-25T10:02:29.154 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.146+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f55a8005580 con 0x7f55d415d810 2024-04-25T10:02:29.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.147+0000 7f55caa00640 1 -- 172.21.15.5:0/3217327506 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55c0078fd0 con 0x7f55d415d810 2024-04-25T10:02:29.154 INFO:teuthology.orchestra.run.smithi005.stdout:60129542156 2024-04-25T10:02:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.146+0000 7f1848c00640 1 -- 172.21.15.5:0/2971943057 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1838041030 con 0x7f184c1652f0 2024-04-25T10:02:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.150+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182c06b4d0 msgr2=0x7f182c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.150+0000 7f183e000640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182c06b4d0 0x7f182c06d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18400096b0 tx=0x7f1840006b60 comp rx=0 tx=0).stop 2024-04-25T10:02:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.150+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 msgr2=0x7f184c153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.150+0000 7f183e000640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 0x7f184c153080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1838035040 tx=0x7f18380046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.151+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 shutdown_connections 2024-04-25T10:02:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.151+0000 7f183e000640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184c1535c0 0x7f184c162af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.151+0000 7f183e000640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f184c1652f0 0x7f184c153080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.151+0000 7f183e000640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.151+0000 7f183e000640 1 --2- 172.21.15.5:0/2971943057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184c0714d0 0x7f184c0adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.151+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 >> 172.21.15.5:0/2971943057 conn(0x7f184c066f20 msgr2=0x7f184c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.152+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f55b406b5a0 msgr2=0x7f55b406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.152+0000 7f55bfe00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f55b406b5a0 0x7f55b406da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55c4007fd0 tx=0x7f55c4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.152+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55d415d810 msgr2=0x7f55d4156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.152+0000 7f55bfe00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55d415d810 0x7f55d4156a50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f55c0031000 tx=0x7f55c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.152+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 shutdown_connections 2024-04-25T10:02:29.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.152+0000 7f183e000640 1 -- 172.21.15.5:0/2971943057 wait complete. 2024-04-25T10:02:29.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.155+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 shutdown_connections 2024-04-25T10:02:29.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.155+0000 7f55bfe00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55d415d810 0x7f55d4156a50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.155+0000 7f55bfe00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f55b406b5a0 0x7f55b406da60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.155+0000 7f55bfe00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55d40cea90 0x7f55d40b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.155+0000 7f55bfe00640 1 --2- 172.21.15.5:0/3217327506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f55d40b2800 0x7f55d40b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.155+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 >> 172.21.15.5:0/3217327506 conn(0x7f55d40726b0 msgr2=0x7f55d40b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.156+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 shutdown_connections 2024-04-25T10:02:29.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.158+0000 7f55bfe00640 1 -- 172.21.15.5:0/3217327506 wait complete. 2024-04-25T10:02:29.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.177+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3d4c0cb130 con 0x7f3d4c141b60 2024-04-25T10:02:29.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.179+0000 7f3d48c00640 1 -- 172.21.15.5:0/4222654752 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3d3c058070 con 0x7f3d4c141b60 2024-04-25T10:02:29.184 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:29.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542156 for osd.4 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.181+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f15340b40a0 con 0x7f15340ae6c0 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3d2c06b4d0 msgr2=0x7f3d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d38034000 tx=0x7f3d3802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d4c141b60 msgr2=0x7f3d4c14e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d4c141b60 0x7f3d4c14e750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0cc2f0 tx=0x7f3d3c00c760 comp rx=0 tx=0).stop 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f152a000640 1 -- 172.21.15.5:0/1115293197 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f152c0792f0 con 0x7f15340ae6c0 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 shutdown_connections 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3d2c06b4d0 0x7f3d2c06d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d4c141b60 0x7f3d4c14e750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d4c15b9a0 0x7f3d4c141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 --2- 172.21.15.5:0/4222654752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d4c143400 0x7f3d4c13fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.182+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 >> 172.21.15.5:0/4222654752 conn(0x7f3d4c071150 msgr2=0x7f3d4c15cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.188 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:29.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.183+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 shutdown_connections 2024-04-25T10:02:29.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.183+0000 7f3d53a00640 1 -- 172.21.15.5:0/4222654752 wait complete. 2024-04-25T10:02:29.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.2 2024-04-25T10:02:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151006b500 msgr2=0x7f151006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151006b500 0x7f151006d9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15340b20d0 tx=0x7f1524001a90 comp rx=0 tx=0).stop 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 msgr2=0x7f15340b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 0x7f15340b0150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f152c035040 tx=0x7f152c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 shutdown_connections 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15341509b0 0x7f1534144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151006b500 0x7f151006d9c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15340cc480 0x7f1534153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 --2- 172.21.15.5:0/1115293197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15340ae6c0 0x7f15340b0150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 >> 172.21.15.5:0/1115293197 conn(0x7f1534066f20 msgr2=0x7f15340cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 shutdown_connections 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.186+0000 7f1533200640 1 -- 172.21.15.5:0/1115293197 wait complete. 2024-04-25T10:02:29.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.190+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3604068fd0 con 0x7f360406c330 2024-04-25T10:02:29.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.191+0000 7f35fbe00640 1 -- 172.21.15.5:0/215263732 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35f401f020 con 0x7f360406c330 2024-04-25T10:02:29.196 INFO:teuthology.orchestra.run.smithi005.stdout:64424509449 2024-04-25T10:02:29.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.193+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0cc8072390 con 0x7f0cc80cbb50 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f35e406b4d0 msgr2=0x7f35e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f35e406b4d0 0x7f35e406d990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35f0006d40 tx=0x7f35f0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360406c330 msgr2=0x7f360409ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360406c330 0x7f360409ee00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f35f400b3e0 tx=0x7f35f400b8a0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 shutdown_connections 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f35e406b4d0 0x7f35e406d990 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f360415da20 0x7f36040aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f360406c330 0x7f360409ee00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 --2- 172.21.15.5:0/215263732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f360406bc70 0x7f36040ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 >> 172.21.15.5:0/215263732 conn(0x7f36040d16f0 msgr2=0x7f36040cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 shutdown_connections 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f360ac00640 1 -- 172.21.15.5:0/215263732 wait complete. 2024-04-25T10:02:29.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.194+0000 7f0cbe000640 1 -- 172.21.15.5:0/2593060682 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0cc007a270 con 0x7f0cc80cbb50 2024-04-25T10:02:29.200 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:29.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ca806b520 msgr2=0x7f0ca806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ca806b520 0x7f0ca806d9e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8000f60 tx=0x7f0cb8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 msgr2=0x7f0cc8145830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc8145830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009e50 tx=0x7f0cc0004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 shutdown_connections 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0cc81472c0 0x7f0cc8158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ca806b520 0x7f0ca806d9e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80cbb50 0x7f0cc8145830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.198+0000 7f0ccd000640 1 --2- 172.21.15.5:0/2593060682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cc80cad50 0x7f0cc80c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.199+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 >> 172.21.15.5:0/2593060682 conn(0x7f0cc8071150 msgr2=0x7f0cc8161b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.199+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 shutdown_connections 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.199+0000 7f0ccd000640 1 -- 172.21.15.5:0/2593060682 wait complete. 2024-04-25T10:02:29.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.5 2024-04-25T10:02:29.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.3 2024-04-25T10:02:29.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2024-04-25T10:02:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.214+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f47d415d320 con 0x7f47d40d0280 2024-04-25T10:02:29.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.215+0000 7f47d1600640 1 -- 172.21.15.5:0/2547933334 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47c0079850 con 0x7f47d40d0280 2024-04-25T10:02:29.220 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47b406b4d0 msgr2=0x7f47b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47b406b4d0 0x7f47b406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47d40ca420 tx=0x7f47c8009060 comp rx=0 tx=0).stop 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 msgr2=0x7f47d40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 0x7f47d40cc160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000c00 tx=0x7f47c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 shutdown_connections 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47d41536f0 0x7f47d40ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47b406b4d0 0x7f47b406d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47d415de30 0x7f47d40cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 --2- 172.21.15.5:0/2547933334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d40d0280 0x7f47d40cc160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.217+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 >> 172.21.15.5:0/2547933334 conn(0x7f47d4072290 msgr2=0x7f47d4161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.0 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.220+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 shutdown_connections 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.220+0000 7f47dc000640 1 -- 172.21.15.5:0/2547933334 wait complete. 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.221+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6df40d1610 con 0x7f6df415da50 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.222+0000 7f6deaa00640 1 -- 172.21.15.5:0/2848474418 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6de4079390 con 0x7f6df415da50 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:64424509449 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.224+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6dd406b5a0 msgr2=0x7f6dd406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.224+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6dd406b5a0 0x7f6dd406da60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6df40b72f0 tx=0x7f6de0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.224+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 msgr2=0x7f6df4154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.224+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 0x7f6df4154c50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6de402a240 tx=0x7f6de40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 shutdown_connections 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6df415de30 0x7f6df40d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df415da50 0x7f6df4154c50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6dd406b5a0 0x7f6dd406da60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 --2- 172.21.15.5:0/2848474418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6df40d0280 0x7f6df41f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 >> 172.21.15.5:0/2848474418 conn(0x7f6df4072290 msgr2=0x7f6df4072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 shutdown_connections 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:29.225+0000 7f6df9c00640 1 -- 172.21.15.5:0/2848474418 wait complete. 2024-04-25T10:02:29.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.6 2024-04-25T10:02:29.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-04-25T10:02:30.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:30.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:30.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:30.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:30.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:30.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:30.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:02:30.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:30.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.463+0000 7fd4d0e00640 1 Processor -- start 2024-04-25T10:02:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.463+0000 7fd4d0e00640 1 -- start start 2024-04-25T10:02:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.465+0000 7fd4d0e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4cc0ce740 0x7fd4cc0d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.465+0000 7fd4d0e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0d2cb0 0x7fd4cc0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.466+0000 7fd4caa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4cc0ce740 0x7fd4cc0d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.466+0000 7fd4caa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4cc0ce740 0x7fd4cc0d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44290/0 (socket says 172.21.15.5:44290) 2024-04-25T10:02:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.466+0000 7fd4caa00640 1 -- 172.21.15.5:0/2221386343 learned_addr learned my addr 172.21.15.5:0/2221386343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.466+0000 7fd4cbe00640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0d2cb0 0x7fd4cc0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.466+0000 7fd4d0e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc15e140 con 0x7fd4cc15dd60 2024-04-25T10:02:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.467+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d22e0 con 0x7fd4cc0ce740 2024-04-25T10:02:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.467+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0cf2c0 con 0x7fd4cc0d2cb0 2024-04-25T10:02:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.468+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729981714 0 0) 0x7fd4cc15e140 con 0x7fd4cc15dd60 2024-04-25T10:02:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.468+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b0003520 con 0x7fd4cc15dd60 2024-04-25T10:02:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 <== mon.1 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534830250 0 0) 0x7fd4b0003520 con 0x7fd4cc15dd60 2024-04-25T10:02:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0d2cb0 msgr2=0x7fd4cc0d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0d2cb0 0x7fd4cc0d0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 >> v2:172.21.15.80:3300/0 conn(0x7fd4cc0ce740 msgr2=0x7fd4cc0d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 --2- 172.21.15.5:0/2221386343 >> v2:172.21.15.80:3300/0 conn(0x7fd4cc0ce740 0x7fd4cc0d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0ced00 con 0x7fd4cc15dd60 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 <== mon.1 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd4c40019f0 con 0x7fd4cc15dd60 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.469+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 >> v1:172.21.15.80:6789/0 conn(0x7fd4cc15dd60 legacy=0x7fd4cc162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.470+0000 7fd4ca000640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b00039d0 0x7fd4b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.470+0000 7fd4ca000640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4b0004800 0x7fd4b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.470+0000 7fd4ca000640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4b00051c0 0x7fd4b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.470+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d22e0 con 0x7fd4b00039d0 2024-04-25T10:02:30.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.470+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0cf2c0 con 0x7fd4b00051c0 2024-04-25T10:02:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.470+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b000bf50 con 0x7fd4b0004800 2024-04-25T10:02:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.471+0000 7fd4caa00640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4b0004800 0x7fd4b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.471+0000 7fd4cb400640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b00039d0 0x7fd4b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.471+0000 7fd4cbe00640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4b00051c0 0x7fd4b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.473+0000 7fd4cb400640 1 -- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4b0004800 msgr2=0x7fd4b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.473+0000 7fd4cb400640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4b0004800 0x7fd4b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.473+0000 7fd4cb400640 1 -- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4b00051c0 msgr2=0x7fd4b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.473+0000 7fd4cb400640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4b00051c0 0x7fd4b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.473+0000 7fd4cb400640 1 -- 172.21.15.5:0/2221386343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0ced00 con 0x7fd4b00039d0 2024-04-25T10:02:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.475+0000 7fd4cb400640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b00039d0 0x7fd4b0003e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c690 tx=0x7fd4b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cabd00f7558f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.476+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400c200 con 0x7fd4b00039d0 2024-04-25T10:02:30.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.476+0000 7fd4ca000640 1 -- 172.21.15.5:0/2221386343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c402f030 con 0x7fd4b00039d0 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.478+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b00039d0 msgr2=0x7fd4b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.478+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/2221386343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4b00039d0 0x7fd4b0003e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c690 tx=0x7fd4b0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.478+0000 7fd4cbe00640 1 -- 172.21.15.5:0/2221386343 reap_dead start 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.478+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 shutdown_connections 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.478+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 >> 172.21.15.5:0/2221386343 conn(0x7fd4cc06be00 msgr2=0x7fd4cc06c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.479+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 shutdown_connections 2024-04-25T10:02:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.479+0000 7fd4d0e00640 1 -- 172.21.15.5:0/2221386343 wait complete. 2024-04-25T10:02:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.482+0000 7fd4d0e00640 1 Processor -- start 2024-04-25T10:02:30.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.482+0000 7fd4d0e00640 1 -- start start 2024-04-25T10:02:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.485+0000 7fd4d0e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc200380 0x7fd4cc158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.485+0000 7fd4d0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 0x7fd4cc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.486+0000 7fd4caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 0x7fd4cc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.486+0000 7fd4caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 0x7fd4cc0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46692/0 (socket says 172.21.15.5:46692) 2024-04-25T10:02:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.486+0000 7fd4cb400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc200380 0x7fd4cc158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.486+0000 7fd4caa00640 1 -- 172.21.15.5:0/4007437223 learned_addr learned my addr 172.21.15.5:0/4007437223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.487+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0cdde0 0x7fd4cc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.487+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d0110 con 0x7fd4cc200760 2024-04-25T10:02:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.487+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d1f70 con 0x7fd4cc200380 2024-04-25T10:02:30.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.487+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc0d0290 con 0x7fd4cc0cdde0 2024-04-25T10:02:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.488+0000 7fd4caa00640 1 -- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0cdde0 msgr2=0x7fd4cc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.487+0000 7fd4cbe00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0cdde0 0x7fd4cc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.488+0000 7fd4caa00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0cdde0 0x7fd4cc156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.488+0000 7fd4caa00640 1 -- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc200380 msgr2=0x7fd4cc158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.488+0000 7fd4caa00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc200380 0x7fd4cc158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.488+0000 7fd4caa00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc200be0 con 0x7fd4cc200760 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.489+0000 7fd4caa00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 0x7fd4cc0cd8a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc011d00 tx=0x7fd4bc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4cbe00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0cdde0 0x7fd4cc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4cb400640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc200380 0x7fd4cc158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00fd20 con 0x7fd4cc200760 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc004020 con 0x7fd4cc200760 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc004410 con 0x7fd4cc200760 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc200eb0 con 0x7fd4cc200760 2024-04-25T10:02:30.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.490+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc201470 con 0x7fd4cc200760 2024-04-25T10:02:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.495+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a4005290 con 0x7fd4cc200760 2024-04-25T10:02:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.496+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4bc012040 con 0x7fd4cc200760 2024-04-25T10:02:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.497+0000 7fd4c3400640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd4ac06b5a0 0x7fd4ac06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.497+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd4bc08a7d0 con 0x7fd4cc200760 2024-04-25T10:02:30.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.503+0000 7fd4cb400640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd4ac06b5a0 0x7fd4ac06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.504+0000 7fd4cb400640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd4ac06b5a0 0x7fd4ac06da60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4033000 tx=0x7fd4c4004000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.510+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4bc019030 con 0x7fd4cc200760 2024-04-25T10:02:30.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.513+0000 7f51e3000640 1 Processor -- start 2024-04-25T10:02:30.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.513+0000 7f51e3000640 1 -- start start 2024-04-25T10:02:30.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.516+0000 7f51e3000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc0b2f30 0x7f51dc0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.516+0000 7f51e3000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51dc1614e0 0x7f51dc161990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e1c00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f51dc15d8a0 0x7f51dc0d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35546/0 (socket says 172.21.15.5:35546) 2024-04-25T10:02:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7ff078c00640 1 Processor -- start 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e3000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51dc15f410 con 0x7f51dc15d8a0 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e3000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51dc15ffe0 con 0x7f51dc1614e0 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e3000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15f6b0 con 0x7f51dc0b2f30 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e2600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51dc1614e0 0x7f51dc161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e2600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51dc1614e0 0x7f51dc161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46694/0 (socket says 172.21.15.5:46694) 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e2600640 1 -- 172.21.15.5:0/1889138159 learned_addr learned my addr 172.21.15.5:0/1889138159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.518+0000 7f51e2600640 1 -- 172.21.15.5:0/1889138159 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc0b2f30 msgr2=0x7f51dc0d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.517+0000 7f51e1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc0b2f30 0x7f51dc0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.519+0000 7f51dbe00640 1 -- 172.21.15.5:0/1889138159 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713242718 0 0) 0x7f51dc15f410 con 0x7f51dc15d8a0 2024-04-25T10:02:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.519+0000 7f51e2600640 1 --2- 172.21.15.5:0/1889138159 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc0b2f30 0x7f51dc0d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.519+0000 7f51e2600640 1 -- 172.21.15.5:0/1889138159 >> v1:172.21.15.80:6789/0 conn(0x7f51dc15d8a0 legacy=0x7f51dc0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7f51e2600640 1 -- 172.21.15.5:0/1889138159 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc15f990 con 0x7f51dc1614e0 2024-04-25T10:02:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.518+0000 7ff078c00640 1 -- start start 2024-04-25T10:02:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.519+0000 7ff078c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff06c053c50 0x7ff06c007670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7ff078c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0e79e0 con 0x7ff06c0e7600 2024-04-25T10:02:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7ff078c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0065d0 con 0x7ff06c0589d0 2024-04-25T10:02:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7ff078c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c007bb0 con 0x7ff06c053c50 2024-04-25T10:02:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7ff071400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff06c053c50 0x7ff06c007670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7ff071400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff06c053c50 0x7ff06c007670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46696/0 (socket says 172.21.15.5:46696) 2024-04-25T10:02:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7ff071400640 1 -- 172.21.15.5:0/3972762406 learned_addr learned my addr 172.21.15.5:0/3972762406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.520+0000 7f51e1200640 1 --2- 172.21.15.5:0/1889138159 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc0b2f30 0x7f51dc0d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.522+0000 7fc747200640 1 Processor -- start 2024-04-25T10:02:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.522+0000 7fc747200640 1 -- start start 2024-04-25T10:02:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.522+0000 7ff071400640 1 -- 172.21.15.5:0/3972762406 >> v1:172.21.15.80:6789/0 conn(0x7ff06c0589d0 legacy=0x7ff06c03bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.522+0000 7ff071400640 1 -- 172.21.15.5:0/3972762406 >> v1:172.21.15.5:6789/0 conn(0x7ff06c0e7600 legacy=0x7ff06c053540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.522+0000 7f51e2600640 1 --2- 172.21.15.5:0/1889138159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc1614e0 0x7f51dc161990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f51d4013a30 tx=0x7f51d4033070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299339a75b100edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.522+0000 7ff071400640 1 -- 172.21.15.5:0/3972762406 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0068b0 con 0x7ff06c053c50 2024-04-25T10:02:30.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.523+0000 7f51dbe00640 1 -- 172.21.15.5:0/1889138159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40339b0 con 0x7f51dc1614e0 2024-04-25T10:02:30.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.523+0000 7f51dbe00640 1 -- 172.21.15.5:0/1889138159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d4033b50 con 0x7f51dc1614e0 2024-04-25T10:02:30.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.523+0000 7f51dbe00640 1 -- 172.21.15.5:0/1889138159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40339b0 con 0x7f51dc1614e0 2024-04-25T10:02:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7fc747200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc7400061f0 0x7fc7400066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7fc747200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7400e36e0 con 0x7fc7400e3300 2024-04-25T10:02:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7fc747200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc740057b40 con 0x7fc740006be0 2024-04-25T10:02:30.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7fc747200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc740057e20 con 0x7fc7400061f0 2024-04-25T10:02:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7f51e3000640 1 -- 172.21.15.5:0/1889138159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc1614e0 msgr2=0x7f51dc161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7f51e3000640 1 --2- 172.21.15.5:0/1889138159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc1614e0 0x7f51dc161990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f51d4013a30 tx=0x7f51d4033070 comp rx=0 tx=0).stop 2024-04-25T10:02:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7f51e3000640 1 -- 172.21.15.5:0/1889138159 shutdown_connections 2024-04-25T10:02:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7f51e3000640 1 --2- 172.21.15.5:0/1889138159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc1614e0 0x7f51dc161990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.524+0000 7ff071400640 1 --2- 172.21.15.5:0/3972762406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c053c50 0x7ff06c007670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff060009870 tx=0x7ff06002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857c46b6ae1e28b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7f51e3000640 1 --2- 172.21.15.5:0/1889138159 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc0b2f30 0x7f51dc0d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7fc745400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc7400061f0 0x7fc7400066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7fc746800640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc740006be0 0x7fc74003c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35566/0 (socket says 172.21.15.5:35566) 2024-04-25T10:02:30.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7fc746800640 1 -- 172.21.15.5:0/3821008474 learned_addr learned my addr 172.21.15.5:0/3821008474 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:30.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7f51e3000640 1 -- 172.21.15.5:0/1889138159 >> 172.21.15.5:0/1889138159 conn(0x7f51dc08af80 msgr2=0x7f51dc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7fc744a00640 1 -- 172.21.15.5:0/3821008474 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975098113 0 0) 0x7fc740057b40 con 0x7fc740006be0 2024-04-25T10:02:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.525+0000 7fc744a00640 1 -- 172.21.15.5:0/3821008474 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc72c003520 con 0x7fc740006be0 2024-04-25T10:02:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.526+0000 7fc744a00640 1 -- 172.21.15.5:0/3821008474 <== mon.1 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099240473 0 0) 0x7fc72c003520 con 0x7fc740006be0 2024-04-25T10:02:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7fc745400640 1 -- 172.21.15.5:0/3821008474 >> v1:172.21.15.80:6789/0 conn(0x7fc740006be0 legacy=0x7fc74003c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7fc745400640 1 -- 172.21.15.5:0/3821008474 >> v1:172.21.15.5:6789/0 conn(0x7fc7400e3300 legacy=0x7fc7400572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7fc745400640 1 -- 172.21.15.5:0/3821008474 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc740058100 con 0x7fc7400061f0 2024-04-25T10:02:30.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7ff070a00640 1 -- 172.21.15.5:0/3972762406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060037070 con 0x7ff06c053c50 2024-04-25T10:02:30.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7f51e3000640 1 -- 172.21.15.5:0/1889138159 shutdown_connections 2024-04-25T10:02:30.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7ff070a00640 1 -- 172.21.15.5:0/3972762406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06002ae50 con 0x7ff06c053c50 2024-04-25T10:02:30.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7ff070a00640 1 -- 172.21.15.5:0/3972762406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060032860 con 0x7ff06c053c50 2024-04-25T10:02:30.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.527+0000 7f51e3000640 1 -- 172.21.15.5:0/1889138159 wait complete. 2024-04-25T10:02:30.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.528+0000 7ff078c00640 1 -- 172.21.15.5:0/3972762406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c053c50 msgr2=0x7ff06c007670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.528+0000 7ff078c00640 1 --2- 172.21.15.5:0/3972762406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c053c50 0x7ff06c007670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff060009870 tx=0x7ff06002a300 comp rx=0 tx=0).stop 2024-04-25T10:02:30.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.528+0000 7ff078c00640 1 -- 172.21.15.5:0/3972762406 shutdown_connections 2024-04-25T10:02:30.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.528+0000 7ff078c00640 1 --2- 172.21.15.5:0/3972762406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c053c50 0x7ff06c007670 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.528+0000 7ff078c00640 1 -- 172.21.15.5:0/3972762406 >> 172.21.15.5:0/3972762406 conn(0x7ff06c058ef0 msgr2=0x7ff06c059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.529+0000 7ff078c00640 1 -- 172.21.15.5:0/3972762406 shutdown_connections 2024-04-25T10:02:30.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.529+0000 7ff078c00640 1 -- 172.21.15.5:0/3972762406 wait complete. 2024-04-25T10:02:30.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.529+0000 7fc745400640 1 --2- 172.21.15.5:0/3821008474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400061f0 0x7fc7400066a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc7340097a0 tx=0x7fc734029f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101c044e7e4dbc5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7f51e3000640 1 Processor -- start 2024-04-25T10:02:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7f51e3000640 1 -- start start 2024-04-25T10:02:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.532+0000 7ff078c00640 1 Processor -- start 2024-04-25T10:02:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.532+0000 7ff078c00640 1 -- start start 2024-04-25T10:02:30.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.529+0000 7f5223200640 1 Processor -- start 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.530+0000 7f5223200640 1 -- start start 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.529+0000 7fc744a00640 1 -- 172.21.15.5:0/3821008474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734037070 con 0x7fc7400061f0 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.530+0000 7fc744a00640 1 -- 172.21.15.5:0/3821008474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73402a9e0 con 0x7fc7400061f0 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.530+0000 7fc747200640 1 -- 172.21.15.5:0/3821008474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400061f0 msgr2=0x7fc7400066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc744a00640 1 -- 172.21.15.5:0/3821008474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734037070 con 0x7fc7400061f0 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc747200640 1 --2- 172.21.15.5:0/3821008474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400061f0 0x7fc7400066a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc7340097a0 tx=0x7fc734029f50 comp rx=0 tx=0).stop 2024-04-25T10:02:30.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc747200640 1 -- 172.21.15.5:0/3821008474 shutdown_connections 2024-04-25T10:02:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc747200640 1 --2- 172.21.15.5:0/3821008474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400061f0 0x7fc7400066a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc747200640 1 -- 172.21.15.5:0/3821008474 >> 172.21.15.5:0/3821008474 conn(0x7fc740058ef0 msgr2=0x7fc740059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc747200640 1 -- 172.21.15.5:0/3821008474 shutdown_connections 2024-04-25T10:02:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.531+0000 7fc747200640 1 -- 172.21.15.5:0/3821008474 wait complete. 2024-04-25T10:02:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.532+0000 7f5223200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f521c0d0a50 0x7f521c0d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.533+0000 7f51e3000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc0b2f30 0x7f51dc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.533+0000 7f51e3000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 0x7f51dc0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.533+0000 7f5223200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c0d1e80 con 0x7f521c15db00 2024-04-25T10:02:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f5223200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c0b5930 con 0x7f521c0b53b0 2024-04-25T10:02:30.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f5223200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0b5c10 con 0x7f521c0d0a50 2024-04-25T10:02:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e3000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc1614e0 0x7f51dc158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e3000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15ee10 con 0x7f51dc0b2f30 2024-04-25T10:02:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e3000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc15ec90 con 0x7f51dc15d8a0 2024-04-25T10:02:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e3000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc0b2150 con 0x7f51dc1614e0 2024-04-25T10:02:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7fc747200640 1 Processor -- start 2024-04-25T10:02:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f5221200640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f521c0b53b0 0x7f521c0d2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35570/0 (socket says 172.21.15.5:35570) 2024-04-25T10:02:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f5221200640 1 -- 172.21.15.5:0/2802113303 learned_addr learned my addr 172.21.15.5:0/2802113303 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e1200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 0x7f51dc0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e1200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 0x7f51dc0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44306/0 (socket says 172.21.15.5:44306) 2024-04-25T10:02:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f51e1200640 1 -- 172.21.15.5:0/2127014621 learned_addr learned my addr 172.21.15.5:0/2127014621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7f51e1c00640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc0b2f30 0x7f51dc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff078c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff06c053c50 0x7ff06c17d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff078c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 0x7ff06c04c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.534+0000 7f521b400640 1 --2- 172.21.15.5:0/2802113303 >> v2:172.21.15.5:3300/0 conn(0x7f521c0d0a50 0x7f521c0d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff078c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c0e7600 0x7ff06c00d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff078c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c007150 con 0x7ff06c0e7600 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff071400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 0x7ff06c04c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff078c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c006e70 con 0x7ff06c0589d0 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff078c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0061f0 con 0x7ff06c053c50 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff071400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 0x7ff06c04c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:44310/0 (socket says 172.21.15.5:44310) 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7ff071400640 1 -- 172.21.15.5:0/3126447522 learned_addr learned my addr 172.21.15.5:0/3126447522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e2600640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc1614e0 0x7f51dc158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7ff072800640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c0e7600 0x7ff06c00d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635390500 0 0) 0x7f521c0b5930 con 0x7f521c0b53b0 2024-04-25T10:02:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.535+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5200003520 con 0x7f521c0b53b0 2024-04-25T10:02:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7fc747200640 1 -- start start 2024-04-25T10:02:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e1200640 1 -- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc1614e0 msgr2=0x7f51dc158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e1200640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc1614e0 0x7f51dc158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e1200640 1 -- 172.21.15.5:0/2127014621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc0b2f30 msgr2=0x7f51dc156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e1200640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc0b2f30 0x7f51dc156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e1200640 1 -- 172.21.15.5:0/2127014621 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc157430 con 0x7f51dc15d8a0 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e2600640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc1614e0 0x7f51dc158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f51e1c00640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc0b2f30 0x7f51dc156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7ff071e00640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff06c053c50 0x7ff06c17d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071400640 1 -- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff06c053c50 msgr2=0x7ff06c17d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489350230 0 0) 0x7f521c0d1e80 con 0x7f521c15db00 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f521c0b5930 con 0x7f521c15db00 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f521b400640 1 -- 172.21.15.5:0/2802113303 >> v1:172.21.15.80:6789/0 conn(0x7f521c0b53b0 legacy=0x7f521c0d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff06c053c50 0x7ff06c17d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071400640 1 -- 172.21.15.5:0/3126447522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c0e7600 msgr2=0x7ff06c00d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c0e7600 0x7ff06c00d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071400640 1 -- 172.21.15.5:0/3126447522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c00dec0 con 0x7ff06c0589d0 2024-04-25T10:02:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f521b400640 1 -- 172.21.15.5:0/2802113303 >> v1:172.21.15.5:6789/0 conn(0x7f521c15db00 legacy=0x7f521c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.536+0000 7f521b400640 1 -- 172.21.15.5:0/2802113303 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0b5ef0 con 0x7f521c0d0a50 2024-04-25T10:02:30.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff072800640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c0e7600 0x7ff06c00d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071e00640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff06c053c50 0x7ff06c17d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7f51e1200640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 0x7f51dc0cf050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c7f0 tx=0x7f51c800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7ff071400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 0x7ff06c04c690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff060009870 tx=0x7ff0600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.537+0000 7fc3dbe00640 1 Processor -- start 2024-04-25T10:02:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7fc3dbe00640 1 -- start start 2024-04-25T10:02:30.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7f521b400640 1 --2- 172.21.15.5:0/2802113303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c0d2330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f52080097a0 tx=0x7f520802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b817dff56e84277d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8015070 con 0x7f51dc15d8a0 2024-04-25T10:02:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c8007a30 con 0x7f51dc15d8a0 2024-04-25T10:02:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8007e00 con 0x7f51dc15d8a0 2024-04-25T10:02:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060042070 con 0x7ff06c0589d0 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc0cba10 con 0x7f51dc15d8a0 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.538+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc0cbfd0 con 0x7f51dc15d8a0 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7ff078c00640 1 -- 172.21.15.5:0/3126447522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c0e2650 con 0x7ff06c0589d0 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7fc747200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 0x7fc74004e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7fc747200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc740006be0 0x7fc740185550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7fc747200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400e3300 0x7fc740185a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7fc747200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740058be0 con 0x7fc7400e3300 2024-04-25T10:02:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7fc747200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc740006fc0 con 0x7fc740006be0 2024-04-25T10:02:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7fc747200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74004f800 con 0x7fc7400061f0 2024-04-25T10:02:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06003e870 con 0x7ff06c0589d0 2024-04-25T10:02:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060037040 con 0x7ff06c0589d0 2024-04-25T10:02:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.540+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51dc0720b0 con 0x7f51dc15d8a0 2024-04-25T10:02:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.539+0000 7ff078c00640 1 -- 172.21.15.5:0/3126447522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c0e2c10 con 0x7ff06c0589d0 2024-04-25T10:02:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.540+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.540+0000 7fc3dbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc3d40ca590 0x7fc3d406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.540+0000 7fc3dbe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d415f020 con 0x7fc3d40cba10 2024-04-25T10:02:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.540+0000 7fc3dbe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d40ca9c0 con 0x7fc3d40ca590 2024-04-25T10:02:30.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.540+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d406d640 con 0x7fc3d4147230 2024-04-25T10:02:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc745e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 0x7fc74004e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc745e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 0x7fc74004e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49282/0 (socket says 172.21.15.5:49282) 2024-04-25T10:02:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc745e00640 1 -- 172.21.15.5:0/3951168653 learned_addr learned my addr 172.21.15.5:0/3951168653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208037070 con 0x7f521c0d0a50 2024-04-25T10:02:30.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208032440 con 0x7f521c0d0a50 2024-04-25T10:02:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49286/0 (socket says 172.21.15.5:49286) 2024-04-25T10:02:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7f521aa00640 1 -- 172.21.15.5:0/2802113303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208032810 con 0x7f521c0d0a50 2024-04-25T10:02:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc3d9400640 1 -- 172.21.15.5:0/3663949190 learned_addr learned my addr 172.21.15.5:0/3663949190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.541+0000 7fc746800640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400e3300 0x7fc740185a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc3d9400640 1 -- 172.21.15.5:0/3663949190 >> v2:172.21.15.5:3300/0 conn(0x7fc3d40ca590 msgr2=0x7fc3d406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc3d9400640 1 --2- 172.21.15.5:0/3663949190 >> v2:172.21.15.5:3300/0 conn(0x7fc3d40ca590 0x7fc3d406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc3d9400640 1 -- 172.21.15.5:0/3663949190 >> v1:172.21.15.5:6789/0 conn(0x7fc3d40cba10 legacy=0x7fc3d40c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc745e00640 1 -- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc740006be0 msgr2=0x7fc740185550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc745e00640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc740006be0 0x7fc740185550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc745e00640 1 -- 172.21.15.5:0/3951168653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400e3300 msgr2=0x7fc740185a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc745e00640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400e3300 0x7fc740185a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc745e00640 1 -- 172.21.15.5:0/3951168653 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74003a1b0 con 0x7fc7400061f0 2024-04-25T10:02:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7f5223200640 1 -- 172.21.15.5:0/2802113303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 msgr2=0x7f521c0d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7f5223200640 1 --2- 172.21.15.5:0/2802113303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c0d2330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f52080097a0 tx=0x7f520802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.543+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51c8018070 con 0x7f51dc15d8a0 2024-04-25T10:02:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff048005290 con 0x7ff06c0589d0 2024-04-25T10:02:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.543+0000 7f5223200640 1 -- 172.21.15.5:0/2802113303 shutdown_connections 2024-04-25T10:02:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.543+0000 7f5223200640 1 --2- 172.21.15.5:0/2802113303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c0d2330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.543+0000 7f5223200640 1 -- 172.21.15.5:0/2802113303 >> 172.21.15.5:0/2802113303 conn(0x7f521c072290 msgr2=0x7f521c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.542+0000 7fc3d9400640 1 -- 172.21.15.5:0/3663949190 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40cac00 con 0x7fc3d4147230 2024-04-25T10:02:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7fc745e00640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 0x7fc74004e090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc73800c4e0 tx=0x7fc73800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7f5223200640 1 -- 172.21.15.5:0/2802113303 shutdown_connections 2024-04-25T10:02:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7f51d9600640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51bc06b4d0 0x7f51bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51c808a8a0 con 0x7f51dc15d8a0 2024-04-25T10:02:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738017070 con 0x7fc7400061f0 2024-04-25T10:02:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7380035f0 con 0x7fc7400061f0 2024-04-25T10:02:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738003a20 con 0x7fc7400061f0 2024-04-25T10:02:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.544+0000 7f5223200640 1 -- 172.21.15.5:0/2802113303 wait complete. 2024-04-25T10:02:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7fc3d9400640 1 --2- 172.21.15.5:0/3663949190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40cb4d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47dddb6f4d6c8b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74004d330 con 0x7fc7400061f0 2024-04-25T10:02:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74004db70 con 0x7fc7400061f0 2024-04-25T10:02:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7f51e1c00640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51bc06b4d0 0x7f51bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7fc3d3e00640 1 -- 172.21.15.5:0/3663949190 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8037070 con 0x7fc3d4147230 2024-04-25T10:02:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7fc3d3e00640 1 -- 172.21.15.5:0/3663949190 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8032440 con 0x7fc3d4147230 2024-04-25T10:02:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7fc3d3e00640 1 -- 172.21.15.5:0/3663949190 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8032810 con 0x7fc3d4147230 2024-04-25T10:02:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.545+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff060045070 con 0x7ff06c0589d0 2024-04-25T10:02:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.546+0000 7f51e1c00640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51bc06b4d0 0x7f51bc06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009fd0 tx=0x7f51d00094e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.546+0000 7fc3dbe00640 1 -- 172.21.15.5:0/3663949190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 msgr2=0x7fc3d40cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.546+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/3663949190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40cb4d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.546+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc740005710 con 0x7fc7400061f0 2024-04-25T10:02:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.547+0000 7fc3dbe00640 1 -- 172.21.15.5:0/3663949190 shutdown_connections 2024-04-25T10:02:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.547+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/3663949190 >> v2:172.21.15.5:3300/0 conn(0x7fc3d40ca590 0x7fc3d406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.547+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/3663949190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40cb4d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.547+0000 7fc3dbe00640 1 -- 172.21.15.5:0/3663949190 >> 172.21.15.5:0/3663949190 conn(0x7fc3d4071150 msgr2=0x7fc3d4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.547+0000 7fc3dbe00640 1 -- 172.21.15.5:0/3663949190 shutdown_connections 2024-04-25T10:02:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.547+0000 7fc3dbe00640 1 -- 172.21.15.5:0/3663949190 wait complete. 2024-04-25T10:02:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.548+0000 7f5223200640 1 Processor -- start 2024-04-25T10:02:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.548+0000 7f5223200640 1 -- start start 2024-04-25T10:02:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.549+0000 7ff06a000640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff05406b5d0 0x7ff05406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.549+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff0600aae00 con 0x7ff06c0589d0 2024-04-25T10:02:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.549+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc73801a070 con 0x7fc7400061f0 2024-04-25T10:02:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.549+0000 7ff071e00640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff05406b5d0 0x7ff05406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f5223200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f521c0b53b0 0x7f521c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f5223200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c1fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7fc3dbe00640 1 Processor -- start 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f521b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c1fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f5223200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f521c15db00 0x7f521c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f5223200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0d3660 con 0x7f521c0d0a50 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f521b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c1fbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46728/0 (socket says 172.21.15.5:46728) 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f521b400640 1 -- 172.21.15.5:0/1779502075 learned_addr learned my addr 172.21.15.5:0/1779502075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0b61d0 con 0x7f521c15db00 2024-04-25T10:02:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0d2f80 con 0x7f521c0b53b0 2024-04-25T10:02:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.550+0000 7fc3dbe00640 1 -- start start 2024-04-25T10:02:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.551+0000 7f521be00640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f521c0b53b0 0x7f521c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.551+0000 7f5221200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f521c15db00 0x7f521c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.551+0000 7ff071e00640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff05406b5d0 0x7ff05406da90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff07404fcd0 tx=0x7ff07406e040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.551+0000 7fc73e000640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc72806b5a0 0x7fc72806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.551+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc738089a30 con 0x7fc7400061f0 2024-04-25T10:02:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.551+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 0x7fc3d41e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d40cba10 0x7fc3d41e8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3dbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d406d2d0 con 0x7fc3d40ca590 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40cbe50 con 0x7fc3d40cba10 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3dbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d406e010 con 0x7fc3d4147230 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f5221200640 1 -- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f521c0b53b0 msgr2=0x7f521c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f5221200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f521c0b53b0 0x7f521c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f5221200640 1 -- 172.21.15.5:0/1779502075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 msgr2=0x7f521c1fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f5221200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c1fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f5221200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c1fc060 con 0x7f521c15db00 2024-04-25T10:02:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f521be00640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f521c0b53b0 0x7f521c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 0x7fc3d41e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 0x7fc3d41e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46740/0 (socket says 172.21.15.5:46740) 2024-04-25T10:02:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3d9400640 1 -- 172.21.15.5:0/2038745923 learned_addr learned my addr 172.21.15.5:0/2038745923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7f5221200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f521c15db00 0x7f521c15e0f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f521400ca50 tx=0x7f521400cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3d9e00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.552+0000 7fc3d8a00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d40cba10 0x7fc3d41e8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc745400640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc72806b5a0 0x7fc72806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc3d9400640 1 -- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 msgr2=0x7fc3d40717a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc3d9400640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40717a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc3d9400640 1 -- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d40cba10 msgr2=0x7fc3d41e8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc3d9400640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d40cba10 0x7fc3d41e8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc3d9400640 1 -- 172.21.15.5:0/2038745923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d41e74f0 con 0x7fc3d40ca590 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.553+0000 7fc3d9400640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 0x7fc3d41e6da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009990 tx=0x7fc3c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.554+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8041020 con 0x7fc3d40ca590 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.554+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d41e77d0 con 0x7fc3d40ca590 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.554+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8030780 con 0x7fc3d40ca590 2024-04-25T10:02:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.554+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8037040 con 0x7fc3d40ca590 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.554+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d41e7d90 con 0x7fc3d40ca590 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.555+0000 7fc745400640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc72806b5a0 0x7fc72806da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc734034000 tx=0x7fc73402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.555+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d41e80a0 con 0x7fc3d40ca590 2024-04-25T10:02:30.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.558+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3c8044020 con 0x7fc3d40ca590 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.559+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51dc0720b0 con 0x7f51dc15d8a0 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.560+0000 7fc3d1600640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc3b406b4d0 0x7fc3b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.560+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc3c80aae60 con 0x7fc3d40ca590 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.560+0000 7fc3d8a00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc3b406b4d0 0x7fc3b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.562+0000 7fc3d8a00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc3b406b4d0 0x7fc3b406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc3d41462c0 tx=0x7fc3c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.567+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc74004d330 con 0x7fc7400061f0 2024-04-25T10:02:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.568+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3c804b030 con 0x7fc3d40ca590 2024-04-25T10:02:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.571+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214017070 con 0x7f521c15db00 2024-04-25T10:02:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.572+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400abb0 con 0x7f521c15db00 2024-04-25T10:02:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.572+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214012400 con 0x7f521c15db00 2024-04-25T10:02:30.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.572+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff060079480 con 0x7ff06c0589d0 2024-04-25T10:02:30.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.572+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c0c7a20 con 0x7f521c15db00 2024-04-25T10:02:30.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.572+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0c7fe0 con 0x7f521c15db00 2024-04-25T10:02:30.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.574+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4005290 con 0x7f521c15db00 2024-04-25T10:02:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.577+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f521401a070 con 0x7f521c15db00 2024-04-25T10:02:30.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.578+0000 7f5213e00640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51fc06b520 0x7f51fc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.578+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f521408a450 con 0x7f521c15db00 2024-04-25T10:02:30.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.579+0000 7f521be00640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51fc06b520 0x7f51fc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.581+0000 7f521be00640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51fc06b520 0x7f51fc06d9e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f520c000f60 tx=0x7f520c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.584+0000 7f44eac00640 1 Processor -- start 2024-04-25T10:02:30.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.584+0000 7f44eac00640 1 -- start start 2024-04-25T10:02:30.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.586+0000 7f44eac00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44e415db00 0x7f44e40d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.586+0000 7f44eac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f44e40ce2b0 0x7f44e40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.586+0000 7f44eac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40d1e80 con 0x7f44e40b53b0 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.586+0000 7f44eac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e40b5930 con 0x7f44e40ce2b0 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.586+0000 7f44eac00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e40b5c10 con 0x7f44e415db00 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.587+0000 7f44e3400640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f44e40b53b0 0x7f44e4161670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35576/0 (socket says 172.21.15.5:35576) 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.587+0000 7f44e3400640 1 -- 172.21.15.5:0/1878356336 learned_addr learned my addr 172.21.15.5:0/1878356336 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.587+0000 7f44e3e00640 1 --2- 172.21.15.5:0/1878356336 >> v2:172.21.15.80:3300/0 conn(0x7f44e415db00 0x7f44e40d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.587+0000 7f44e8c00640 1 --2- 172.21.15.5:0/1878356336 >> v2:172.21.15.5:3300/0 conn(0x7f44e40ce2b0 0x7f44e40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.588+0000 7f44e3e00640 1 -- 172.21.15.5:0/1878356336 >> v2:172.21.15.5:3300/0 conn(0x7f44e40ce2b0 msgr2=0x7f44e40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.588+0000 7f44e3e00640 1 --2- 172.21.15.5:0/1878356336 >> v2:172.21.15.5:3300/0 conn(0x7f44e40ce2b0 0x7f44e40d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.588+0000 7f44e3e00640 1 -- 172.21.15.5:0/1878356336 >> v1:172.21.15.80:6789/0 conn(0x7f44e40b53b0 legacy=0x7f44e4161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.588+0000 7f44e3e00640 1 -- 172.21.15.5:0/1878356336 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40b5ef0 con 0x7f44e415db00 2024-04-25T10:02:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.589+0000 7f44e8c00640 1 --2- 172.21.15.5:0/1878356336 >> v2:172.21.15.5:3300/0 conn(0x7f44e40ce2b0 0x7f44e40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.589+0000 7f4239400640 1 Processor -- start 2024-04-25T10:02:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.589+0000 7f4239400640 1 -- start start 2024-04-25T10:02:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.590+0000 7f44e3e00640 1 --2- 172.21.15.5:0/1878356336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e415db00 0x7f44e40d0340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44e40d4940 tx=0x7f44d402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7b9cf89b0f8f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.590+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f521c1fc060 con 0x7f521c15db00 2024-04-25T10:02:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.591+0000 7f44e2a00640 1 -- 172.21.15.5:0/1878356336 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4037070 con 0x7f44e415db00 2024-04-25T10:02:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.591+0000 7f44e2a00640 1 -- 172.21.15.5:0/1878356336 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4032440 con 0x7f44e415db00 2024-04-25T10:02:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.591+0000 7f44e2a00640 1 -- 172.21.15.5:0/1878356336 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4032850 con 0x7f44e415db00 2024-04-25T10:02:30.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4239400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f423415c380 0x7f423406e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4239400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f42340ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4239400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4234165af0 con 0x7f42340cd240 2024-04-25T10:02:30.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4239400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423415c760 con 0x7f423415c380 2024-04-25T10:02:30.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4239400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42341658a0 con 0x7f42340b16e0 2024-04-25T10:02:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f44eac00640 1 -- 172.21.15.5:0/1878356336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e415db00 msgr2=0x7f44e40d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f44eac00640 1 --2- 172.21.15.5:0/1878356336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e415db00 0x7f44e40d0340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44e40d4940 tx=0x7f44d402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4232800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f423415c380 0x7f423406e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f4231e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f42340ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f4233200640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f42340cd240 0x7f423406d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35578/0 (socket says 172.21.15.5:35578) 2024-04-25T10:02:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f4233200640 1 -- 172.21.15.5:0/1053379749 learned_addr learned my addr 172.21.15.5:0/1053379749 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f44eac00640 1 -- 172.21.15.5:0/1878356336 shutdown_connections 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f44eac00640 1 --2- 172.21.15.5:0/1878356336 >> v2:172.21.15.5:3300/0 conn(0x7f44e40ce2b0 0x7f44e40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f44eac00640 1 --2- 172.21.15.5:0/1878356336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e415db00 0x7f44e40d0340 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.592+0000 7f44eac00640 1 -- 172.21.15.5:0/1878356336 >> 172.21.15.5:0/1878356336 conn(0x7f44e4072290 msgr2=0x7f44e40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f44eac00640 1 -- 172.21.15.5:0/1878356336 shutdown_connections 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f44eac00640 1 -- 172.21.15.5:0/1878356336 wait complete. 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f4231e00640 1 -- 172.21.15.5:0/1053379749 >> v2:172.21.15.80:3300/0 conn(0x7f423415c380 msgr2=0x7f423406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f4231e00640 1 --2- 172.21.15.5:0/1053379749 >> v2:172.21.15.80:3300/0 conn(0x7f423415c380 0x7f423406e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f4231e00640 1 -- 172.21.15.5:0/1053379749 >> v1:172.21.15.80:6789/0 conn(0x7f42340cd240 legacy=0x7f423406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.593+0000 7f4231e00640 1 -- 172.21.15.5:0/1053379749 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234165dd0 con 0x7f42340b16e0 2024-04-25T10:02:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.595+0000 7f4231e00640 1 --2- 172.21.15.5:0/1053379749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f42340ccd00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be0b3506ab62b927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.596+0000 7f44eac00640 1 Processor -- start 2024-04-25T10:02:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.596+0000 7f44eac00640 1 -- start start 2024-04-25T10:02:30.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.597+0000 7f44eac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e40b53b0 0x7f44e40c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.597+0000 7f44eac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e40ce2b0 0x7f44e40b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.597+0000 7f44eac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 0x7f44e40b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.598+0000 7f44eac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40d3660 con 0x7f44e40ce2b0 2024-04-25T10:02:30.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.598+0000 7f44eac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40b61d0 con 0x7f44e40b53b0 2024-04-25T10:02:30.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.598+0000 7f44eac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40b6c10 con 0x7f44e415db00 2024-04-25T10:02:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.598+0000 7f44e8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 0x7f44e40b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.598+0000 7f44e8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 0x7f44e40b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49328/0 (socket says 172.21.15.5:49328) 2024-04-25T10:02:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.598+0000 7f44e8c00640 1 -- 172.21.15.5:0/415637144 learned_addr learned my addr 172.21.15.5:0/415637144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f4231400640 1 -- 172.21.15.5:0/1053379749 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220037070 con 0x7f42340b16e0 2024-04-25T10:02:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f44e3e00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e40b53b0 0x7f44e40c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f4231400640 1 -- 172.21.15.5:0/1053379749 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422002ad20 con 0x7f42340b16e0 2024-04-25T10:02:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f4231400640 1 -- 172.21.15.5:0/1053379749 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220032660 con 0x7f42340b16e0 2024-04-25T10:02:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f44e8c00640 1 -- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e40b53b0 msgr2=0x7f44e40c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f44e8c00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e40b53b0 0x7f44e40c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f44e8c00640 1 -- 172.21.15.5:0/415637144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e40ce2b0 msgr2=0x7f44e40b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f44e8c00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e40ce2b0 0x7f44e40b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.599+0000 7f44e8c00640 1 -- 172.21.15.5:0/415637144 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40c7da0 con 0x7f44e415db00 2024-04-25T10:02:30.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f4239400640 1 -- 172.21.15.5:0/1053379749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 msgr2=0x7f42340ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f4239400640 1 --2- 172.21.15.5:0/1053379749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f42340ccd00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002a010 comp rx=0 tx=0).stop 2024-04-25T10:02:30.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f44e8c00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 0x7f44e40b7450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00efc0 tx=0x7f44dc009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc018030 con 0x7f44e415db00 2024-04-25T10:02:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44dc00cde0 con 0x7f44e415db00 2024-04-25T10:02:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e41f77b0 con 0x7f44e415db00 2024-04-25T10:02:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.600+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc003bd0 con 0x7f44e415db00 2024-04-25T10:02:30.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.604+0000 7f4239400640 1 -- 172.21.15.5:0/1053379749 shutdown_connections 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.604+0000 7f4239400640 1 --2- 172.21.15.5:0/1053379749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f42340ccd00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.604+0000 7f4239400640 1 --2- 172.21.15.5:0/1053379749 >> v2:172.21.15.80:3300/0 conn(0x7f423415c380 0x7f423406e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.604+0000 7f4239400640 1 -- 172.21.15.5:0/1053379749 >> 172.21.15.5:0/1053379749 conn(0x7f4234071150 msgr2=0x7f4234071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.604+0000 7f4239400640 1 -- 172.21.15.5:0/1053379749 shutdown_connections 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.604+0000 7f4239400640 1 -- 172.21.15.5:0/1053379749 wait complete. 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.601+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e40c9650 con 0x7f44e415db00 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.602+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4068fd0 con 0x7f44e415db00 2024-04-25T10:02:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.605+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44dc005d10 con 0x7f44e415db00 2024-04-25T10:02:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.606+0000 7f44dbe00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44c406b4d0 0x7f44c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.607+0000 7f4239400640 1 Processor -- start 2024-04-25T10:02:30.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.607+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f44dc08aee0 con 0x7f44e415db00 2024-04-25T10:02:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.607+0000 7f4239400640 1 -- start start 2024-04-25T10:02:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.607+0000 7f44e3e00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44c406b4d0 0x7f44c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.608+0000 7f44e3e00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44c406b4d0 0x7f44c406d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44d4034000 tx=0x7f44d402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.609+0000 7f4239400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f423414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.609+0000 7f4239400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 0x7f423406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.609+0000 7f4239400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f423415c380 0x7f423406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4239400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423406ec70 con 0x7f42340cd240 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4239400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4234166390 con 0x7f423415c380 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4239400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42341632c0 con 0x7f42340b16e0 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4231e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 0x7f423406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4232800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f423414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4231e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 0x7f423406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46762/0 (socket says 172.21.15.5:46762) 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4231e00640 1 -- 172.21.15.5:0/173548921 learned_addr learned my addr 172.21.15.5:0/173548921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4233200640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f423415c380 0x7f423406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4231e00640 1 -- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 msgr2=0x7f423414af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.610+0000 7f4231e00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f423414af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.611+0000 7f4231e00640 1 -- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f423415c380 msgr2=0x7f423406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.611+0000 7f4231e00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f423415c380 0x7f423406d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.611+0000 7f4231e00640 1 -- 172.21.15.5:0/173548921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234151a90 con 0x7f42340cd240 2024-04-25T10:02:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.611+0000 7f4231e00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 0x7f423406d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f422002a600 tx=0x7f42200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.611+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220037050 con 0x7f42340cd240 2024-04-25T10:02:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.612+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4220043070 con 0x7f42340cd240 2024-04-25T10:02:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.612+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422003ca70 con 0x7f42340cd240 2024-04-25T10:02:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.613+0000 7f4239400640 1 -- 172.21.15.5:0/173548921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4234151d70 con 0x7f42340cd240 2024-04-25T10:02:30.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.613+0000 7f4239400640 1 -- 172.21.15.5:0/173548921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4234152330 con 0x7f42340cd240 2024-04-25T10:02:30.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.615+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4208005290 con 0x7f42340cd240 2024-04-25T10:02:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.619+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f422003cc10 con 0x7f42340cd240 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.620+0000 7f422aa00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f421406b4d0 0x7f421406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.620+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f42200aba80 con 0x7f42340cd240 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.621+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44dc022070 con 0x7f44e415db00 2024-04-25T10:02:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.622+0000 7f4232800640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f421406b4d0 0x7f421406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.626+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4220079f50 con 0x7f42340cd240 2024-04-25T10:02:30.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.627+0000 7f4232800640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f421406b4d0 0x7f421406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f422400aa20 tx=0x7f4224008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.944+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd4a4005580 con 0x7fd4cc200760 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.945+0000 7fd4c3400640 1 -- 172.21.15.5:0/4007437223 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd4bc059380 con 0x7fd4cc200760 2024-04-25T10:02:30.949 INFO:teuthology.orchestra.run.smithi005.stdout:60129542157 2024-04-25T10:02:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.948+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd4ac06b5a0 msgr2=0x7fd4ac06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.949+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd4ac06b5a0 0x7fd4ac06da60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4033000 tx=0x7fd4c4004000 comp rx=0 tx=0).stop 2024-04-25T10:02:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.949+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 msgr2=0x7fd4cc0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.949+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 0x7fd4cc0cd8a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc011d00 tx=0x7fd4bc00f610 comp rx=0 tx=0).stop 2024-04-25T10:02:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 shutdown_connections 2024-04-25T10:02:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4cc0cdde0 0x7fd4cc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4cc200760 0x7fd4cc0cd8a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd4ac06b5a0 0x7fd4ac06da60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 --2- 172.21.15.5:0/4007437223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4cc200380 0x7fd4cc158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 >> 172.21.15.5:0/4007437223 conn(0x7fd4cc06be00 msgr2=0x7fd4cc160b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.950+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 shutdown_connections 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.951+0000 7fd4d0e00640 1 -- 172.21.15.5:0/4007437223 wait complete. 2024-04-25T10:02:30.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542157 for osd.4 2024-04-25T10:02:30.983 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.983+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f51dc0b9580 con 0x7f51dc15d8a0 2024-04-25T10:02:30.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.984+0000 7f51d9600640 1 -- 172.21.15.5:0/2127014621 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51c8058d70 con 0x7f51dc15d8a0 2024-04-25T10:02:30.988 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51bc06b4d0 msgr2=0x7f51bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51bc06b4d0 0x7f51bc06d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009fd0 tx=0x7f51d00094e0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 msgr2=0x7f51dc0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 0x7f51dc0cf050 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c7f0 tx=0x7f51c800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 shutdown_connections 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51dc1614e0 0x7f51dc158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51bc06b4d0 0x7f51bc06d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51dc15d8a0 0x7f51dc0cf050 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 --2- 172.21.15.5:0/2127014621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc0b2f30 0x7f51dc156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.986+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 >> 172.21.15.5:0/2127014621 conn(0x7f51dc08af80 msgr2=0x7f51dc0b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.987+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 shutdown_connections 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.987+0000 7f51e3000640 1 -- 172.21.15.5:0/2127014621 wait complete. 2024-04-25T10:02:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:30.999+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc3d4068fd0 con 0x7fc3d40ca590 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.000+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc74004c370 con 0x7fc7400061f0 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.000+0000 7fc3d1600640 1 -- 172.21.15.5:0/2038745923 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3c8079900 con 0x7fc3d40ca590 2024-04-25T10:02:31.005 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:31.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.001+0000 7fc73e000640 1 -- 172.21.15.5:0/3951168653 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7380584c0 con 0x7fc7400061f0 2024-04-25T10:02:31.006 INFO:teuthology.orchestra.run.smithi005.stdout:64424509451 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.003+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc3b406b4d0 msgr2=0x7fc3b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.003+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc3b406b4d0 0x7fc3b406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc3d41462c0 tx=0x7fc3c0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.003+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 msgr2=0x7fc3d41e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.003+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 0x7fc3d41e6da0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009990 tx=0x7fc3c80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 shutdown_connections 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc72806b5a0 msgr2=0x7fc72806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc72806b5a0 0x7fc72806da60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc734034000 tx=0x7fc73402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 msgr2=0x7fc74004e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 0x7fc74004e090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc73800c4e0 tx=0x7fc73800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.010 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc3d4147230 0x7fc3d40717a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 shutdown_connections 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7400e3300 0x7fc740185a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc72806b5a0 0x7fc72806da60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc740006be0 0x7fc740185550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 --2- 172.21.15.5:0/3951168653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc7400061f0 0x7fc74004e090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 >> 172.21.15.5:0/3951168653 conn(0x7fc740058ef0 msgr2=0x7fc74003e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 shutdown_connections 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc747200640 1 -- 172.21.15.5:0/3951168653 wait complete. 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc3b406b4d0 0x7fc3b406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3d40cba10 0x7fc3d41e8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 --2- 172.21.15.5:0/2038745923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3d40ca590 0x7fc3d41e6da0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 >> 172.21.15.5:0/2038745923 conn(0x7fc3d4071150 msgr2=0x7fc3d40cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 shutdown_connections 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.004+0000 7fc3dbe00640 1 -- 172.21.15.5:0/2038745923 wait complete. 2024-04-25T10:02:31.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.023+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff048005580 con 0x7ff06c0589d0 2024-04-25T10:02:31.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.024+0000 7ff06a000640 1 -- 172.21.15.5:0/3126447522 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff06007d520 con 0x7ff06c0589d0 2024-04-25T10:02:31.028 INFO:teuthology.orchestra.run.smithi005.stdout:64424509452 2024-04-25T10:02:31.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff05406b5d0 msgr2=0x7ff05406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff05406b5d0 0x7ff05406da90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff07404fcd0 tx=0x7ff07406e040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 msgr2=0x7ff06c04c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 0x7ff06c04c690 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff060009870 tx=0x7ff0600046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 shutdown_connections 2024-04-25T10:02:31.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff06c0e7600 0x7ff06c00d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff05406b5d0 0x7ff05406da90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff06c0589d0 0x7ff06c04c690 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 --2- 172.21.15.5:0/3126447522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff06c053c50 0x7ff06c17d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.027+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 >> 172.21.15.5:0/3126447522 conn(0x7ff06c058ef0 msgr2=0x7ff06c0e66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.028+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 shutdown_connections 2024-04-25T10:02:31.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.028+0000 7ff05f400640 1 -- 172.21.15.5:0/3126447522 wait complete. 2024-04-25T10:02:31.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.029+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51f4005850 con 0x7f521c15db00 2024-04-25T10:02:31.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.5 2024-04-25T10:02:31.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.030+0000 7f5213e00640 1 -- 172.21.15.5:0/1779502075 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5214058920 con 0x7f521c15db00 2024-04-25T10:02:31.035 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.0 2024-04-25T10:02:31.036 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:31.036 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51fc06b520 msgr2=0x7f51fc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51fc06b520 0x7f51fc06d9e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f520c000f60 tx=0x7f520c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f521c15db00 msgr2=0x7f521c15e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f521c15db00 0x7f521c15e0f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f521400ca50 tx=0x7f521400cf10 comp rx=0 tx=0).stop 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 shutdown_connections 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f51fc06b520 0x7f51fc06d9e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f521c15db00 0x7f521c15e0f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f521c0d0a50 0x7f521c1fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 --2- 172.21.15.5:0/1779502075 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f521c0b53b0 0x7f521c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 >> 172.21.15.5:0/1779502075 conn(0x7f521c072290 msgr2=0x7f521c157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 shutdown_connections 2024-04-25T10:02:31.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.034+0000 7f5223200640 1 -- 172.21.15.5:0/1779502075 wait complete. 2024-04-25T10:02:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.043+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4208005850 con 0x7f42340cd240 2024-04-25T10:02:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.044+0000 7f422aa00640 1 -- 172.21.15.5:0/173548921 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f422007dff0 con 0x7f42340cd240 2024-04-25T10:02:31.048 INFO:teuthology.orchestra.run.smithi005.stdout:64424509452 2024-04-25T10:02:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f44e40d16b0 con 0x7f44e415db00 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f421406b4d0 msgr2=0x7f421406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f421406b4d0 0x7f421406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f422400aa20 tx=0x7f4224008040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 msgr2=0x7f423406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 0x7f423406d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f422002a600 tx=0x7f42200046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 shutdown_connections 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f421406b4d0 0x7f421406d990 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f422400aa20 tx=0x7f4224008040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f423415c380 0x7f423406d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42340cd240 0x7f423406d150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 --2- 172.21.15.5:0/173548921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42340b16e0 0x7f423414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 >> 172.21.15.5:0/173548921 conn(0x7f4234071150 msgr2=0x7f4234164660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 shutdown_connections 2024-04-25T10:02:31.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.046+0000 7f421fe00640 1 -- 172.21.15.5:0/173548921 wait complete. 2024-04-25T10:02:31.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.047+0000 7f44dbe00640 1 -- 172.21.15.5:0/415637144 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44dc010040 con 0x7f44e415db00 2024-04-25T10:02:31.052 INFO:teuthology.orchestra.run.smithi005.stdout:64424509450 2024-04-25T10:02:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44c406b4d0 msgr2=0x7f44c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44c406b4d0 0x7f44c406d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44d4034000 tx=0x7f44d402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 msgr2=0x7f44e40b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 0x7f44e40b7450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f44dc00efc0 tx=0x7f44dc009f90 comp rx=0 tx=0).stop 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 shutdown_connections 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44e415db00 0x7f44e40b7450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e40ce2b0 0x7f44e40b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44c406b4d0 0x7f44c406d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 --2- 172.21.15.5:0/415637144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44e40b53b0 0x7f44e40c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 >> 172.21.15.5:0/415637144 conn(0x7f44e4072290 msgr2=0x7f44e415b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 shutdown_connections 2024-04-25T10:02:31.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.050+0000 7f44eac00640 1 -- 172.21.15.5:0/415637144 wait complete. 2024-04-25T10:02:31.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.3 2024-04-25T10:02:31.057 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.1 2024-04-25T10:02:31.061 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.6 2024-04-25T10:02:31.069 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-04-25T10:02:31.072 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:02:31.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.300+0000 7fad02000640 1 Processor -- start 2024-04-25T10:02:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.300+0000 7fad02000640 1 -- start start 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.301+0000 7fad02000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc1529e0 0x7facfc0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.301+0000 7fad02000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7facfc0c6410 0x7facfc0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.301+0000 7fad02000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7facfc071ed0 0x7facfc0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7fad02000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc0b2340 con 0x7facfc071ed0 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7fad02000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc15bdc0 con 0x7facfc0c6410 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7fad02000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc15c060 con 0x7facfc1529e0 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7facfc071ed0 0x7facfc0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7facfc071ed0 0x7facfc0ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46766/0 (socket says 172.21.15.5:46766) 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfbe00640 1 -- 172.21.15.5:0/452640897 learned_addr learned my addr 172.21.15.5:0/452640897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfaa00640 1 --2- 172.21.15.5:0/452640897 >> v2:172.21.15.80:3300/0 conn(0x7facfc0c6410 0x7facfc0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfb400640 1 --2- 172.21.15.5:0/452640897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc1529e0 0x7facfc0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfbe00640 1 -- 172.21.15.5:0/452640897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc1529e0 msgr2=0x7facfc0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.302+0000 7facfbe00640 1 --2- 172.21.15.5:0/452640897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc1529e0 0x7facfc0c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.303+0000 7facfbe00640 1 -- 172.21.15.5:0/452640897 >> v2:172.21.15.80:3300/0 conn(0x7facfc0c6410 msgr2=0x7facfc0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.303+0000 7facfbe00640 1 --2- 172.21.15.5:0/452640897 >> v2:172.21.15.80:3300/0 conn(0x7facfc0c6410 0x7facfc0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.303+0000 7facfbe00640 1 -- 172.21.15.5:0/452640897 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc15c340 con 0x7facfc071ed0 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.304+0000 7facfbe00640 1 --2- 172.21.15.5:0/452640897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc0ce890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7facfc0cf5a0 tx=0x7facf402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960d532431681b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.304+0000 7facfa000640 1 -- 172.21.15.5:0/452640897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4036070 con 0x7facfc071ed0 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.304+0000 7facfa000640 1 -- 172.21.15.5:0/452640897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf4031600 con 0x7facfc071ed0 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.304+0000 7facfa000640 1 -- 172.21.15.5:0/452640897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40319d0 con 0x7facfc071ed0 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 -- 172.21.15.5:0/452640897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 msgr2=0x7facfc0ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 --2- 172.21.15.5:0/452640897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc0ce890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7facfc0cf5a0 tx=0x7facf402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 -- 172.21.15.5:0/452640897 shutdown_connections 2024-04-25T10:02:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 --2- 172.21.15.5:0/452640897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc0ce890 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 --2- 172.21.15.5:0/452640897 >> v2:172.21.15.80:3300/0 conn(0x7facfc0c6410 0x7facfc0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 --2- 172.21.15.5:0/452640897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc1529e0 0x7facfc0c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 -- 172.21.15.5:0/452640897 >> 172.21.15.5:0/452640897 conn(0x7facfc071150 msgr2=0x7facfc071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 -- 172.21.15.5:0/452640897 shutdown_connections 2024-04-25T10:02:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.305+0000 7fad02000640 1 -- 172.21.15.5:0/452640897 wait complete. 2024-04-25T10:02:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.306+0000 7fad02000640 1 Processor -- start 2024-04-25T10:02:31.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.307+0000 7fad02000640 1 -- start start 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7fad02000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7fad02000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facfc0c6410 0x7facfc1f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7fad02000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc153e20 0x7facfc0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7fad02000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0c4660 con 0x7facfc071ed0 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7fad02000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc15c900 con 0x7facfc0c6410 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7fad02000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0c47e0 con 0x7facfc153e20 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46770/0 (socket says 172.21.15.5:46770) 2024-04-25T10:02:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 -- 172.21.15.5:0/3327564328 learned_addr learned my addr 172.21.15.5:0/3327564328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 -- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc153e20 msgr2=0x7facfc0b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc153e20 0x7facfc0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 -- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facfc0c6410 msgr2=0x7facfc1f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfaa00640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facfc0c6410 0x7facfc1f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facfc0c6410 0x7facfc1f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.308+0000 7facfb400640 1 -- 172.21.15.5:0/3327564328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc1542b0 con 0x7facfc071ed0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.309+0000 7facfb400640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc155350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facec00d620 tx=0x7facec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.309+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00b620 con 0x7facfc071ed0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.309+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec019070 con 0x7facfc071ed0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.309+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00ba00 con 0x7facfc071ed0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.309+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc154590 con 0x7facfc071ed0 2024-04-25T10:02:31.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.309+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc154b50 con 0x7facfc071ed0 2024-04-25T10:02:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.310+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc068fd0 con 0x7facfc071ed0 2024-04-25T10:02:31.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.311+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facec00bba0 con 0x7facfc071ed0 2024-04-25T10:02:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.312+0000 7facf3400640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7facd806b4d0 0x7facd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.312+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7facec08a9f0 con 0x7facfc071ed0 2024-04-25T10:02:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.313+0000 7facfaa00640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7facd806b4d0 0x7facd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.314+0000 7facfaa00640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7facd806b4d0 0x7facd806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facfc159dd0 tx=0x7face4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.316+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facec059400 con 0x7facfc071ed0 2024-04-25T10:02:31.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.501+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facfc0ce830 con 0x7facd806b4d0 2024-04-25T10:02:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.534+0000 7facf3400640 1 -- 172.21.15.5:0/3327564328 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7facfc0ce830 con 0x7facd806b4d0 2024-04-25T10:02:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:02:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.539+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7facd806b4d0 msgr2=0x7facd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.539+0000 7fad02000640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7facd806b4d0 0x7facd806d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7facfc159dd0 tx=0x7face4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.539+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 msgr2=0x7facfc155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.539+0000 7fad02000640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc155350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7facec00d620 tx=0x7facec00dae0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 shutdown_connections 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7facfc153e20 0x7facfc0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7facd806b4d0 0x7facd806d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facfc0c6410 0x7facfc1f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 --2- 172.21.15.5:0/3327564328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facfc071ed0 0x7facfc155350 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 >> 172.21.15.5:0/3327564328 conn(0x7facfc071150 msgr2=0x7facfc0c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 shutdown_connections 2024-04-25T10:02:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.540+0000 7fad02000640 1 -- 172.21.15.5:0/3327564328 wait complete. 2024-04-25T10:02:31.562 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T10:02:30.621258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236148,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738572,"statfs":{"total":773094113280,"available":772852297728,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005987"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356469+0000","last_change":"2024-04-25T10:01:57.061422+0000","last_active":"2024-04-25T10:02:03.356469+0000","last_peered":"2024-04-25T10:02:03.356469+0000","last_clean":"2024-04-25T10:02:03.356469+0000","last_became_active":"2024-04-25T10:01:57.061119+0000","last_became_peered":"2024-04-25T10:01:57.061119+0000","last_unstale":"2024-04-25T10:02:03.356469+0000","last_undegraded":"2024-04-25T10:02:03.356469+0000","last_fullsized":"2024-04-25T10:02:03.356469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:26.806679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433872+0000","last_change":"2024-04-25T10:01:58.115482+0000","last_active":"2024-04-25T10:02:03.433872+0000","last_peered":"2024-04-25T10:02:03.433872+0000","last_clean":"2024-04-25T10:02:03.433872+0000","last_became_active":"2024-04-25T10:01:58.115138+0000","last_became_peered":"2024-04-25T10:01:58.115138+0000","last_unstale":"2024-04-25T10:02:03.433872+0000","last_undegraded":"2024-04-25T10:02:03.433872+0000","last_fullsized":"2024-04-25T10:02:03.433872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352821+0000","last_change":"2024-04-25T10:02:02.368709+0000","last_active":"2024-04-25T10:02:03.352821+0000","last_peered":"2024-04-25T10:02:03.352821+0000","last_clean":"2024-04-25T10:02:03.352821+0000","last_became_active":"2024-04-25T10:02:02.367787+0000","last_became_peered":"2024-04-25T10:02:02.367787+0000","last_unstale":"2024-04-25T10:02:03.352821+0000","last_undegraded":"2024-04-25T10:02:03.352821+0000","last_fullsized":"2024-04-25T10:02:03.352821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:31.444725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434120+0000","last_change":"2024-04-25T10:01:57.061756+0000","last_active":"2024-04-25T10:02:03.434120+0000","last_peered":"2024-04-25T10:02:03.434120+0000","last_clean":"2024-04-25T10:02:03.434120+0000","last_became_active":"2024-04-25T10:01:57.058627+0000","last_became_peered":"2024-04-25T10:01:57.058627+0000","last_unstale":"2024-04-25T10:02:03.434120+0000","last_undegraded":"2024-04-25T10:02:03.434120+0000","last_fullsized":"2024-04-25T10:02:03.434120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434000+0000","last_change":"2024-04-25T10:01:58.113879+0000","last_active":"2024-04-25T10:02:03.434000+0000","last_peered":"2024-04-25T10:02:03.434000+0000","last_clean":"2024-04-25T10:02:03.434000+0000","last_became_active":"2024-04-25T10:01:58.113527+0000","last_became_peered":"2024-04-25T10:01:58.113527+0000","last_unstale":"2024-04-25T10:02:03.434000+0000","last_undegraded":"2024-04-25T10:02:03.434000+0000","last_fullsized":"2024-04-25T10:02:03.434000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519097+0000","last_change":"2024-04-25T10:02:02.364833+0000","last_active":"2024-04-25T10:02:03.519097+0000","last_peered":"2024-04-25T10:02:03.519097+0000","last_clean":"2024-04-25T10:02:03.519097+0000","last_became_active":"2024-04-25T10:02:02.364420+0000","last_became_peered":"2024-04-25T10:02:02.364420+0000","last_unstale":"2024-04-25T10:02:03.519097+0000","last_undegraded":"2024-04-25T10:02:03.519097+0000","last_fullsized":"2024-04-25T10:02:03.519097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:17.582640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518857+0000","last_change":"2024-04-25T10:01:57.049323+0000","last_active":"2024-04-25T10:02:03.518857+0000","last_peered":"2024-04-25T10:02:03.518857+0000","last_clean":"2024-04-25T10:02:03.518857+0000","last_became_active":"2024-04-25T10:01:57.048997+0000","last_became_peered":"2024-04-25T10:01:57.048997+0000","last_unstale":"2024-04-25T10:02:03.518857+0000","last_undegraded":"2024-04-25T10:02:03.518857+0000","last_fullsized":"2024-04-25T10:02:03.518857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.523353+0000","last_change":"2024-04-25T10:01:58.074975+0000","last_active":"2024-04-25T10:02:03.523353+0000","last_peered":"2024-04-25T10:02:03.523353+0000","last_clean":"2024-04-25T10:02:03.523353+0000","last_became_active":"2024-04-25T10:01:58.074637+0000","last_became_peered":"2024-04-25T10:01:58.074637+0000","last_unstale":"2024-04-25T10:02:03.523353+0000","last_undegraded":"2024-04-25T10:02:03.523353+0000","last_fullsized":"2024-04-25T10:02:03.523353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522261+0000","last_change":"2024-04-25T10:02:02.367186+0000","last_active":"2024-04-25T10:02:03.522261+0000","last_peered":"2024-04-25T10:02:03.522261+0000","last_clean":"2024-04-25T10:02:03.522261+0000","last_became_active":"2024-04-25T10:02:02.366476+0000","last_became_peered":"2024-04-25T10:02:02.366476+0000","last_unstale":"2024-04-25T10:02:03.522261+0000","last_undegraded":"2024-04-25T10:02:03.522261+0000","last_fullsized":"2024-04-25T10:02:03.522261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519518+0000","last_change":"2024-04-25T10:01:57.058161+0000","last_active":"2024-04-25T10:02:03.519518+0000","last_peered":"2024-04-25T10:02:03.519518+0000","last_clean":"2024-04-25T10:02:03.519518+0000","last_became_active":"2024-04-25T10:01:57.057891+0000","last_became_peered":"2024-04-25T10:01:57.057891+0000","last_unstale":"2024-04-25T10:02:03.519518+0000","last_undegraded":"2024-04-25T10:02:03.519518+0000","last_fullsized":"2024-04-25T10:02:03.519518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:23.026096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404272+0000","last_change":"2024-04-25T10:01:58.074490+0000","last_active":"2024-04-25T10:02:03.404272+0000","last_peered":"2024-04-25T10:02:03.404272+0000","last_clean":"2024-04-25T10:02:03.404272+0000","last_became_active":"2024-04-25T10:01:58.074117+0000","last_became_peered":"2024-04-25T10:01:58.074117+0000","last_unstale":"2024-04-25T10:02:03.404272+0000","last_undegraded":"2024-04-25T10:02:03.404272+0000","last_fullsized":"2024-04-25T10:02:03.404272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:53.841219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520948+0000","last_change":"2024-04-25T10:02:02.362410+0000","last_active":"2024-04-25T10:02:03.520948+0000","last_peered":"2024-04-25T10:02:03.520948+0000","last_clean":"2024-04-25T10:02:03.520948+0000","last_became_active":"2024-04-25T10:02:02.359612+0000","last_became_peered":"2024-04-25T10:02:02.359612+0000","last_unstale":"2024-04-25T10:02:03.520948+0000","last_undegraded":"2024-04-25T10:02:03.520948+0000","last_fullsized":"2024-04-25T10:02:03.520948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.136854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348157+0000","last_change":"2024-04-25T10:01:58.073917+0000","last_active":"2024-04-25T10:02:03.348157+0000","last_peered":"2024-04-25T10:02:03.348157+0000","last_clean":"2024-04-25T10:02:03.348157+0000","last_became_active":"2024-04-25T10:01:58.073122+0000","last_became_peered":"2024-04-25T10:01:58.073122+0000","last_unstale":"2024-04-25T10:02:03.348157+0000","last_undegraded":"2024-04-25T10:02:03.348157+0000","last_fullsized":"2024-04-25T10:02:03.348157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402599+0000","last_change":"2024-04-25T10:01:57.059507+0000","last_active":"2024-04-25T10:02:03.402599+0000","last_peered":"2024-04-25T10:02:03.402599+0000","last_clean":"2024-04-25T10:02:03.402599+0000","last_became_active":"2024-04-25T10:01:57.056083+0000","last_became_peered":"2024-04-25T10:01:57.056083+0000","last_unstale":"2024-04-25T10:02:03.402599+0000","last_undegraded":"2024-04-25T10:02:03.402599+0000","last_fullsized":"2024-04-25T10:02:03.402599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:29.264217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404104+0000","last_change":"2024-04-25T10:02:02.362032+0000","last_active":"2024-04-25T10:02:03.404104+0000","last_peered":"2024-04-25T10:02:03.404104+0000","last_clean":"2024-04-25T10:02:03.404104+0000","last_became_active":"2024-04-25T10:02:02.361608+0000","last_became_peered":"2024-04-25T10:02:02.361608+0000","last_unstale":"2024-04-25T10:02:03.404104+0000","last_undegraded":"2024-04-25T10:02:03.404104+0000","last_fullsized":"2024-04-25T10:02:03.404104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:27.993920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353535+0000","last_change":"2024-04-25T10:01:58.116451+0000","last_active":"2024-04-25T10:02:03.353535+0000","last_peered":"2024-04-25T10:02:03.353535+0000","last_clean":"2024-04-25T10:02:03.353535+0000","last_became_active":"2024-04-25T10:01:58.116177+0000","last_became_peered":"2024-04-25T10:01:58.116177+0000","last_unstale":"2024-04-25T10:02:03.353535+0000","last_undegraded":"2024-04-25T10:02:03.353535+0000","last_fullsized":"2024-04-25T10:02:03.353535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518845+0000","last_change":"2024-04-25T10:01:57.062452+0000","last_active":"2024-04-25T10:02:03.518845+0000","last_peered":"2024-04-25T10:02:03.518845+0000","last_clean":"2024-04-25T10:02:03.518845+0000","last_became_active":"2024-04-25T10:01:57.062105+0000","last_became_peered":"2024-04-25T10:01:57.062105+0000","last_unstale":"2024-04-25T10:02:03.518845+0000","last_undegraded":"2024-04-25T10:02:03.518845+0000","last_fullsized":"2024-04-25T10:02:03.518845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518790+0000","last_change":"2024-04-25T10:02:02.369531+0000","last_active":"2024-04-25T10:02:03.518790+0000","last_peered":"2024-04-25T10:02:03.518790+0000","last_clean":"2024-04-25T10:02:03.518790+0000","last_became_active":"2024-04-25T10:02:02.367651+0000","last_became_peered":"2024-04-25T10:02:02.367651+0000","last_unstale":"2024-04-25T10:02:03.518790+0000","last_undegraded":"2024-04-25T10:02:03.518790+0000","last_fullsized":"2024-04-25T10:02:03.518790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:58.140005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352233+0000","last_change":"2024-04-25T10:01:58.117205+0000","last_active":"2024-04-25T10:02:03.352233+0000","last_peered":"2024-04-25T10:02:03.352233+0000","last_clean":"2024-04-25T10:02:03.352233+0000","last_became_active":"2024-04-25T10:01:58.116873+0000","last_became_peered":"2024-04-25T10:01:58.116873+0000","last_unstale":"2024-04-25T10:02:03.352233+0000","last_undegraded":"2024-04-25T10:02:03.352233+0000","last_fullsized":"2024-04-25T10:02:03.352233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:25.305583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354051+0000","last_change":"2024-04-25T10:01:57.049606+0000","last_active":"2024-04-25T10:02:03.354051+0000","last_peered":"2024-04-25T10:02:03.354051+0000","last_clean":"2024-04-25T10:02:03.354051+0000","last_became_active":"2024-04-25T10:01:57.049300+0000","last_became_peered":"2024-04-25T10:01:57.049300+0000","last_unstale":"2024-04-25T10:02:03.354051+0000","last_undegraded":"2024-04-25T10:02:03.354051+0000","last_fullsized":"2024-04-25T10:02:03.354051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:29.284211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354667+0000","last_change":"2024-04-25T10:02:02.366098+0000","last_active":"2024-04-25T10:02:03.354667+0000","last_peered":"2024-04-25T10:02:03.354667+0000","last_clean":"2024-04-25T10:02:03.354667+0000","last_became_active":"2024-04-25T10:02:02.365669+0000","last_became_peered":"2024-04-25T10:02:02.365669+0000","last_unstale":"2024-04-25T10:02:03.354667+0000","last_undegraded":"2024-04-25T10:02:03.354667+0000","last_fullsized":"2024-04-25T10:02:03.354667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:34.132211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347971+0000","last_change":"2024-04-25T10:01:58.116653+0000","last_active":"2024-04-25T10:02:03.347971+0000","last_peered":"2024-04-25T10:02:03.347971+0000","last_clean":"2024-04-25T10:02:03.347971+0000","last_became_active":"2024-04-25T10:01:58.116414+0000","last_became_peered":"2024-04-25T10:01:58.116414+0000","last_unstale":"2024-04-25T10:02:03.347971+0000","last_undegraded":"2024-04-25T10:02:03.347971+0000","last_fullsized":"2024-04-25T10:02:03.347971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:20.873677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.517442+0000","last_change":"2024-04-25T10:01:57.042210+0000","last_active":"2024-04-25T10:02:03.517442+0000","last_peered":"2024-04-25T10:02:03.517442+0000","last_clean":"2024-04-25T10:02:03.517442+0000","last_became_active":"2024-04-25T10:01:57.041890+0000","last_became_peered":"2024-04-25T10:01:57.041890+0000","last_unstale":"2024-04-25T10:02:03.517442+0000","last_undegraded":"2024-04-25T10:02:03.517442+0000","last_fullsized":"2024-04-25T10:02:03.517442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521682+0000","last_change":"2024-04-25T10:02:02.364078+0000","last_active":"2024-04-25T10:02:03.521682+0000","last_peered":"2024-04-25T10:02:03.521682+0000","last_clean":"2024-04-25T10:02:03.521682+0000","last_became_active":"2024-04-25T10:02:02.358978+0000","last_became_peered":"2024-04-25T10:02:02.358978+0000","last_unstale":"2024-04-25T10:02:03.521682+0000","last_undegraded":"2024-04-25T10:02:03.521682+0000","last_fullsized":"2024-04-25T10:02:03.521682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355780+0000","last_change":"2024-04-25T10:01:58.118220+0000","last_active":"2024-04-25T10:02:03.355780+0000","last_peered":"2024-04-25T10:02:03.355780+0000","last_clean":"2024-04-25T10:02:03.355780+0000","last_became_active":"2024-04-25T10:01:58.117911+0000","last_became_peered":"2024-04-25T10:01:58.117911+0000","last_unstale":"2024-04-25T10:02:03.355780+0000","last_undegraded":"2024-04-25T10:02:03.355780+0000","last_fullsized":"2024-04-25T10:02:03.355780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:27.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405295+0000","last_change":"2024-04-25T10:01:57.064576+0000","last_active":"2024-04-25T10:02:03.405295+0000","last_peered":"2024-04-25T10:02:03.405295+0000","last_clean":"2024-04-25T10:02:03.405295+0000","last_became_active":"2024-04-25T10:01:57.064301+0000","last_became_peered":"2024-04-25T10:01:57.064301+0000","last_unstale":"2024-04-25T10:02:03.405295+0000","last_undegraded":"2024-04-25T10:02:03.405295+0000","last_fullsized":"2024-04-25T10:02:03.405295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.258589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521777+0000","last_change":"2024-04-25T10:02:02.362701+0000","last_active":"2024-04-25T10:02:03.521777+0000","last_peered":"2024-04-25T10:02:03.521777+0000","last_clean":"2024-04-25T10:02:03.521777+0000","last_became_active":"2024-04-25T10:02:02.361742+0000","last_became_peered":"2024-04-25T10:02:02.361742+0000","last_unstale":"2024-04-25T10:02:03.521777+0000","last_undegraded":"2024-04-25T10:02:03.521777+0000","last_fullsized":"2024-04-25T10:02:03.521777+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356323+0000","last_change":"2024-04-25T10:01:58.121271+0000","last_active":"2024-04-25T10:02:03.356323+0000","last_peered":"2024-04-25T10:02:03.356323+0000","last_clean":"2024-04-25T10:02:03.356323+0000","last_became_active":"2024-04-25T10:01:58.121001+0000","last_became_peered":"2024-04-25T10:01:58.121001+0000","last_unstale":"2024-04-25T10:02:03.356323+0000","last_undegraded":"2024-04-25T10:02:03.356323+0000","last_fullsized":"2024-04-25T10:02:03.356323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:56.894928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356241+0000","last_change":"2024-04-25T10:01:57.047184+0000","last_active":"2024-04-25T10:02:03.356241+0000","last_peered":"2024-04-25T10:02:03.356241+0000","last_clean":"2024-04-25T10:02:03.356241+0000","last_became_active":"2024-04-25T10:01:57.046623+0000","last_became_peered":"2024-04-25T10:01:57.046623+0000","last_unstale":"2024-04-25T10:02:03.356241+0000","last_undegraded":"2024-04-25T10:02:03.356241+0000","last_fullsized":"2024-04-25T10:02:03.356241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:31.269009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353577+0000","last_change":"2024-04-25T10:02:02.370654+0000","last_active":"2024-04-25T10:02:03.353577+0000","last_peered":"2024-04-25T10:02:03.353577+0000","last_clean":"2024-04-25T10:02:03.353577+0000","last_became_active":"2024-04-25T10:02:02.369464+0000","last_became_peered":"2024-04-25T10:02:02.369464+0000","last_unstale":"2024-04-25T10:02:03.353577+0000","last_undegraded":"2024-04-25T10:02:03.353577+0000","last_fullsized":"2024-04-25T10:02:03.353577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:47.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519808+0000","last_change":"2024-04-25T10:01:58.123583+0000","last_active":"2024-04-25T10:02:03.519808+0000","last_peered":"2024-04-25T10:02:03.519808+0000","last_clean":"2024-04-25T10:02:03.519808+0000","last_became_active":"2024-04-25T10:01:58.122380+0000","last_became_peered":"2024-04-25T10:01:58.122380+0000","last_unstale":"2024-04-25T10:02:03.519808+0000","last_undegraded":"2024-04-25T10:02:03.519808+0000","last_fullsized":"2024-04-25T10:02:03.519808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:54.007484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405069+0000","last_change":"2024-04-25T10:01:57.060061+0000","last_active":"2024-04-25T10:02:03.405069+0000","last_peered":"2024-04-25T10:02:03.405069+0000","last_clean":"2024-04-25T10:02:03.405069+0000","last_became_active":"2024-04-25T10:01:57.056768+0000","last_became_peered":"2024-04-25T10:01:57.056768+0000","last_unstale":"2024-04-25T10:02:03.405069+0000","last_undegraded":"2024-04-25T10:02:03.405069+0000","last_fullsized":"2024-04-25T10:02:03.405069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.943464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518947+0000","last_change":"2024-04-25T10:02:02.366286+0000","last_active":"2024-04-25T10:02:03.518947+0000","last_peered":"2024-04-25T10:02:03.518947+0000","last_clean":"2024-04-25T10:02:03.518947+0000","last_became_active":"2024-04-25T10:02:02.365877+0000","last_became_peered":"2024-04-25T10:02:02.365877+0000","last_unstale":"2024-04-25T10:02:03.518947+0000","last_undegraded":"2024-04-25T10:02:03.518947+0000","last_fullsized":"2024-04-25T10:02:03.518947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521404+0000","last_change":"2024-04-25T10:01:58.126543+0000","last_active":"2024-04-25T10:02:03.521404+0000","last_peered":"2024-04-25T10:02:03.521404+0000","last_clean":"2024-04-25T10:02:03.521404+0000","last_became_active":"2024-04-25T10:01:58.126245+0000","last_became_peered":"2024-04-25T10:01:58.126245+0000","last_unstale":"2024-04-25T10:02:03.521404+0000","last_undegraded":"2024-04-25T10:02:03.521404+0000","last_fullsized":"2024-04-25T10:02:03.521404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433241+0000","last_change":"2024-04-25T10:01:57.056845+0000","last_active":"2024-04-25T10:02:03.433241+0000","last_peered":"2024-04-25T10:02:03.433241+0000","last_clean":"2024-04-25T10:02:03.433241+0000","last_became_active":"2024-04-25T10:01:57.056550+0000","last_became_peered":"2024-04-25T10:01:57.056550+0000","last_unstale":"2024-04-25T10:02:03.433241+0000","last_undegraded":"2024-04-25T10:02:03.433241+0000","last_fullsized":"2024-04-25T10:02:03.433241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518500+0000","last_change":"2024-04-25T10:02:02.360392+0000","last_active":"2024-04-25T10:02:03.518500+0000","last_peered":"2024-04-25T10:02:03.518500+0000","last_clean":"2024-04-25T10:02:03.518500+0000","last_became_active":"2024-04-25T10:02:02.359550+0000","last_became_peered":"2024-04-25T10:02:02.359550+0000","last_unstale":"2024-04-25T10:02:03.518500+0000","last_undegraded":"2024-04-25T10:02:03.518500+0000","last_fullsized":"2024-04-25T10:02:03.518500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518461+0000","last_change":"2024-04-25T10:01:58.070356+0000","last_active":"2024-04-25T10:02:03.518461+0000","last_peered":"2024-04-25T10:02:03.518461+0000","last_clean":"2024-04-25T10:02:03.518461+0000","last_became_active":"2024-04-25T10:01:58.068594+0000","last_became_peered":"2024-04-25T10:01:58.068594+0000","last_unstale":"2024-04-25T10:02:03.518461+0000","last_undegraded":"2024-04-25T10:02:03.518461+0000","last_fullsized":"2024-04-25T10:02:03.518461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402901+0000","last_change":"2024-04-25T10:01:57.061869+0000","last_active":"2024-04-25T10:02:03.402901+0000","last_peered":"2024-04-25T10:02:03.402901+0000","last_clean":"2024-04-25T10:02:03.402901+0000","last_became_active":"2024-04-25T10:01:57.059517+0000","last_became_peered":"2024-04-25T10:01:57.059517+0000","last_unstale":"2024-04-25T10:02:03.402901+0000","last_undegraded":"2024-04-25T10:02:03.402901+0000","last_fullsized":"2024-04-25T10:02:03.402901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:49.036030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519650+0000","last_change":"2024-04-25T10:02:02.364712+0000","last_active":"2024-04-25T10:02:03.519650+0000","last_peered":"2024-04-25T10:02:03.519650+0000","last_clean":"2024-04-25T10:02:03.519650+0000","last_became_active":"2024-04-25T10:02:02.363483+0000","last_became_peered":"2024-04-25T10:02:02.363483+0000","last_unstale":"2024-04-25T10:02:03.519650+0000","last_undegraded":"2024-04-25T10:02:03.519650+0000","last_fullsized":"2024-04-25T10:02:03.519650+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432126+0000","last_change":"2024-04-25T10:01:58.124129+0000","last_active":"2024-04-25T10:02:03.432126+0000","last_peered":"2024-04-25T10:02:03.432126+0000","last_clean":"2024-04-25T10:02:03.432126+0000","last_became_active":"2024-04-25T10:01:58.123815+0000","last_became_peered":"2024-04-25T10:01:58.123815+0000","last_unstale":"2024-04-25T10:02:03.432126+0000","last_undegraded":"2024-04-25T10:02:03.432126+0000","last_fullsized":"2024-04-25T10:02:03.432126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352979+0000","last_change":"2024-04-25T10:01:57.058063+0000","last_active":"2024-04-25T10:02:03.352979+0000","last_peered":"2024-04-25T10:02:03.352979+0000","last_clean":"2024-04-25T10:02:03.352979+0000","last_became_active":"2024-04-25T10:01:57.057727+0000","last_became_peered":"2024-04-25T10:01:57.057727+0000","last_unstale":"2024-04-25T10:02:03.352979+0000","last_undegraded":"2024-04-25T10:02:03.352979+0000","last_fullsized":"2024-04-25T10:02:03.352979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520229+0000","last_change":"2024-04-25T10:02:02.366966+0000","last_active":"2024-04-25T10:02:03.520229+0000","last_peered":"2024-04-25T10:02:03.520229+0000","last_clean":"2024-04-25T10:02:03.520229+0000","last_became_active":"2024-04-25T10:02:02.366563+0000","last_became_peered":"2024-04-25T10:02:02.366563+0000","last_unstale":"2024-04-25T10:02:03.520229+0000","last_undegraded":"2024-04-25T10:02:03.520229+0000","last_fullsized":"2024-04-25T10:02:03.520229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:50.524934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518446+0000","last_change":"2024-04-25T10:01:58.124316+0000","last_active":"2024-04-25T10:02:03.518446+0000","last_peered":"2024-04-25T10:02:03.518446+0000","last_clean":"2024-04-25T10:02:03.518446+0000","last_became_active":"2024-04-25T10:01:58.123995+0000","last_became_peered":"2024-04-25T10:01:58.123995+0000","last_unstale":"2024-04-25T10:02:03.518446+0000","last_undegraded":"2024-04-25T10:02:03.518446+0000","last_fullsized":"2024-04-25T10:02:03.518446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348251+0000","last_change":"2024-04-25T10:01:57.064900+0000","last_active":"2024-04-25T10:02:03.348251+0000","last_peered":"2024-04-25T10:02:03.348251+0000","last_clean":"2024-04-25T10:02:03.348251+0000","last_became_active":"2024-04-25T10:01:57.063565+0000","last_became_peered":"2024-04-25T10:01:57.063565+0000","last_unstale":"2024-04-25T10:02:03.348251+0000","last_undegraded":"2024-04-25T10:02:03.348251+0000","last_fullsized":"2024-04-25T10:02:03.348251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433722+0000","last_change":"2024-04-25T10:02:02.363528+0000","last_active":"2024-04-25T10:02:03.433722+0000","last_peered":"2024-04-25T10:02:03.433722+0000","last_clean":"2024-04-25T10:02:03.433722+0000","last_became_active":"2024-04-25T10:02:02.360337+0000","last_became_peered":"2024-04-25T10:02:02.360337+0000","last_unstale":"2024-04-25T10:02:03.433722+0000","last_undegraded":"2024-04-25T10:02:03.433722+0000","last_fullsized":"2024-04-25T10:02:03.433722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:25.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354986+0000","last_change":"2024-04-25T10:01:58.114954+0000","last_active":"2024-04-25T10:02:03.354986+0000","last_peered":"2024-04-25T10:02:03.354986+0000","last_clean":"2024-04-25T10:02:03.354986+0000","last_became_active":"2024-04-25T10:01:58.114652+0000","last_became_peered":"2024-04-25T10:01:58.114652+0000","last_unstale":"2024-04-25T10:02:03.354986+0000","last_undegraded":"2024-04-25T10:02:03.354986+0000","last_fullsized":"2024-04-25T10:02:03.354986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.396332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402951+0000","last_change":"2024-04-25T10:01:57.060078+0000","last_active":"2024-04-25T10:02:03.402951+0000","last_peered":"2024-04-25T10:02:03.402951+0000","last_clean":"2024-04-25T10:02:03.402951+0000","last_became_active":"2024-04-25T10:01:57.059811+0000","last_became_peered":"2024-04-25T10:01:57.059811+0000","last_unstale":"2024-04-25T10:02:03.402951+0000","last_undegraded":"2024-04-25T10:02:03.402951+0000","last_fullsized":"2024-04-25T10:02:03.402951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.018629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520801+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:02:03.520801+0000","last_peered":"2024-04-25T10:02:03.520801+0000","last_clean":"2024-04-25T10:02:03.520801+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:02:03.520801+0000","last_undegraded":"2024-04-25T10:02:03.520801+0000","last_fullsized":"2024-04-25T10:02:03.520801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404900+0000","last_change":"2024-04-25T10:02:02.358810+0000","last_active":"2024-04-25T10:02:03.404900+0000","last_peered":"2024-04-25T10:02:03.404900+0000","last_clean":"2024-04-25T10:02:03.404900+0000","last_became_active":"2024-04-25T10:02:02.358270+0000","last_became_peered":"2024-04-25T10:02:02.358270+0000","last_unstale":"2024-04-25T10:02:03.404900+0000","last_undegraded":"2024-04-25T10:02:03.404900+0000","last_fullsized":"2024-04-25T10:02:03.404900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:00.819451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404528+0000","last_change":"2024-04-25T10:01:58.073772+0000","last_active":"2024-04-25T10:02:03.404528+0000","last_peered":"2024-04-25T10:02:03.404528+0000","last_clean":"2024-04-25T10:02:03.404528+0000","last_became_active":"2024-04-25T10:01:58.073104+0000","last_became_peered":"2024-04-25T10:01:58.073104+0000","last_unstale":"2024-04-25T10:02:03.404528+0000","last_undegraded":"2024-04-25T10:02:03.404528+0000","last_fullsized":"2024-04-25T10:02:03.404528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.239771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520009+0000","last_change":"2024-04-25T10:01:57.054884+0000","last_active":"2024-04-25T10:02:03.520009+0000","last_peered":"2024-04-25T10:02:03.520009+0000","last_clean":"2024-04-25T10:02:03.520009+0000","last_became_active":"2024-04-25T10:01:57.054577+0000","last_became_peered":"2024-04-25T10:01:57.054577+0000","last_unstale":"2024-04-25T10:02:03.520009+0000","last_undegraded":"2024-04-25T10:02:03.520009+0000","last_fullsized":"2024-04-25T10:02:03.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521556+0000","last_change":"2024-04-25T10:02:02.364334+0000","last_active":"2024-04-25T10:02:03.521556+0000","last_peered":"2024-04-25T10:02:03.521556+0000","last_clean":"2024-04-25T10:02:03.521556+0000","last_became_active":"2024-04-25T10:02:02.363695+0000","last_became_peered":"2024-04-25T10:02:02.363695+0000","last_unstale":"2024-04-25T10:02:03.521556+0000","last_undegraded":"2024-04-25T10:02:03.521556+0000","last_fullsized":"2024-04-25T10:02:03.521556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:30.903672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519516+0000","last_change":"2024-04-25T10:01:58.090256+0000","last_active":"2024-04-25T10:02:03.519516+0000","last_peered":"2024-04-25T10:02:03.519516+0000","last_clean":"2024-04-25T10:02:03.519516+0000","last_became_active":"2024-04-25T10:01:58.089936+0000","last_became_peered":"2024-04-25T10:01:58.089936+0000","last_unstale":"2024-04-25T10:02:03.519516+0000","last_undegraded":"2024-04-25T10:02:03.519516+0000","last_fullsized":"2024-04-25T10:02:03.519516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433744+0000","last_change":"2024-04-25T10:01:57.060690+0000","last_active":"2024-04-25T10:02:03.433744+0000","last_peered":"2024-04-25T10:02:03.433744+0000","last_clean":"2024-04-25T10:02:03.433744+0000","last_became_active":"2024-04-25T10:01:57.060345+0000","last_became_peered":"2024-04-25T10:01:57.060345+0000","last_unstale":"2024-04-25T10:02:03.433744+0000","last_undegraded":"2024-04-25T10:02:03.433744+0000","last_fullsized":"2024-04-25T10:02:03.433744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433246+0000","last_change":"2024-04-25T10:02:02.371169+0000","last_active":"2024-04-25T10:02:03.433246+0000","last_peered":"2024-04-25T10:02:03.433246+0000","last_clean":"2024-04-25T10:02:03.433246+0000","last_became_active":"2024-04-25T10:02:02.370789+0000","last_became_peered":"2024-04-25T10:02:02.370789+0000","last_unstale":"2024-04-25T10:02:03.433246+0000","last_undegraded":"2024-04-25T10:02:03.433246+0000","last_fullsized":"2024-04-25T10:02:03.433246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:40.538557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520420+0000","last_change":"2024-04-25T10:01:58.072580+0000","last_active":"2024-04-25T10:02:03.520420+0000","last_peered":"2024-04-25T10:02:03.520420+0000","last_clean":"2024-04-25T10:02:03.520420+0000","last_became_active":"2024-04-25T10:01:58.072277+0000","last_became_peered":"2024-04-25T10:01:58.072277+0000","last_unstale":"2024-04-25T10:02:03.520420+0000","last_undegraded":"2024-04-25T10:02:03.520420+0000","last_fullsized":"2024-04-25T10:02:03.520420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356066+0000","last_change":"2024-04-25T10:01:57.057184+0000","last_active":"2024-04-25T10:02:03.356066+0000","last_peered":"2024-04-25T10:02:03.356066+0000","last_clean":"2024-04-25T10:02:03.356066+0000","last_became_active":"2024-04-25T10:01:57.054573+0000","last_became_peered":"2024-04-25T10:01:57.054573+0000","last_unstale":"2024-04-25T10:02:03.356066+0000","last_undegraded":"2024-04-25T10:02:03.356066+0000","last_fullsized":"2024-04-25T10:02:03.356066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:33.311253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347440+0000","last_change":"2024-04-25T10:02:02.373007+0000","last_active":"2024-04-25T10:02:03.347440+0000","last_peered":"2024-04-25T10:02:03.347440+0000","last_clean":"2024-04-25T10:02:03.347440+0000","last_became_active":"2024-04-25T10:02:02.372467+0000","last_became_peered":"2024-04-25T10:02:02.372467+0000","last_unstale":"2024-04-25T10:02:03.347440+0000","last_undegraded":"2024-04-25T10:02:03.347440+0000","last_fullsized":"2024-04-25T10:02:03.347440+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:18.003128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521943+0000","last_change":"2024-04-25T10:01:58.071062+0000","last_active":"2024-04-25T10:02:03.521943+0000","last_peered":"2024-04-25T10:02:03.521943+0000","last_clean":"2024-04-25T10:02:03.521943+0000","last_became_active":"2024-04-25T10:01:58.070721+0000","last_became_peered":"2024-04-25T10:01:58.070721+0000","last_unstale":"2024-04-25T10:02:03.521943+0000","last_undegraded":"2024-04-25T10:02:03.521943+0000","last_fullsized":"2024-04-25T10:02:03.521943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520558+0000","last_change":"2024-04-25T10:01:57.051450+0000","last_active":"2024-04-25T10:02:03.520558+0000","last_peered":"2024-04-25T10:02:03.520558+0000","last_clean":"2024-04-25T10:02:03.520558+0000","last_became_active":"2024-04-25T10:01:57.051104+0000","last_became_peered":"2024-04-25T10:01:57.051104+0000","last_unstale":"2024-04-25T10:02:03.520558+0000","last_undegraded":"2024-04-25T10:02:03.520558+0000","last_fullsized":"2024-04-25T10:02:03.520558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521721+0000","last_change":"2024-04-25T10:02:02.371242+0000","last_active":"2024-04-25T10:02:03.521721+0000","last_peered":"2024-04-25T10:02:03.521721+0000","last_clean":"2024-04-25T10:02:03.521721+0000","last_became_active":"2024-04-25T10:02:02.370975+0000","last_became_peered":"2024-04-25T10:02:02.370975+0000","last_unstale":"2024-04-25T10:02:03.521721+0000","last_undegraded":"2024-04-25T10:02:03.521721+0000","last_fullsized":"2024-04-25T10:02:03.521721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:56.622899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347953+0000","last_change":"2024-04-25T10:01:58.076461+0000","last_active":"2024-04-25T10:02:03.347953+0000","last_peered":"2024-04-25T10:02:03.347953+0000","last_clean":"2024-04-25T10:02:03.347953+0000","last_became_active":"2024-04-25T10:01:58.075822+0000","last_became_peered":"2024-04-25T10:01:58.075822+0000","last_unstale":"2024-04-25T10:02:03.347953+0000","last_undegraded":"2024-04-25T10:02:03.347953+0000","last_fullsized":"2024-04-25T10:02:03.347953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352738+0000","last_change":"2024-04-25T10:01:57.051484+0000","last_active":"2024-04-25T10:02:03.352738+0000","last_peered":"2024-04-25T10:02:03.352738+0000","last_clean":"2024-04-25T10:02:03.352738+0000","last_became_active":"2024-04-25T10:01:57.051183+0000","last_became_peered":"2024-04-25T10:01:57.051183+0000","last_unstale":"2024-04-25T10:02:03.352738+0000","last_undegraded":"2024-04-25T10:02:03.352738+0000","last_fullsized":"2024-04-25T10:02:03.352738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:48.627288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433983+0000","last_change":"2024-04-25T10:02:02.364464+0000","last_active":"2024-04-25T10:02:03.433983+0000","last_peered":"2024-04-25T10:02:03.433983+0000","last_clean":"2024-04-25T10:02:03.433983+0000","last_became_active":"2024-04-25T10:02:02.364024+0000","last_became_peered":"2024-04-25T10:02:02.364024+0000","last_unstale":"2024-04-25T10:02:03.433983+0000","last_undegraded":"2024-04-25T10:02:03.433983+0000","last_fullsized":"2024-04-25T10:02:03.433983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:39.357721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520680+0000","last_change":"2024-04-25T10:01:58.125488+0000","last_active":"2024-04-25T10:02:03.520680+0000","last_peered":"2024-04-25T10:02:03.520680+0000","last_clean":"2024-04-25T10:02:03.520680+0000","last_became_active":"2024-04-25T10:01:58.125161+0000","last_became_peered":"2024-04-25T10:01:58.125161+0000","last_unstale":"2024-04-25T10:02:03.520680+0000","last_undegraded":"2024-04-25T10:02:03.520680+0000","last_fullsized":"2024-04-25T10:02:03.520680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:10.940348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518416+0000","last_change":"2024-04-25T10:01:57.066991+0000","last_active":"2024-04-25T10:02:03.518416+0000","last_peered":"2024-04-25T10:02:03.518416+0000","last_clean":"2024-04-25T10:02:03.518416+0000","last_became_active":"2024-04-25T10:01:57.065456+0000","last_became_peered":"2024-04-25T10:01:57.065456+0000","last_unstale":"2024-04-25T10:02:03.518416+0000","last_undegraded":"2024-04-25T10:02:03.518416+0000","last_fullsized":"2024-04-25T10:02:03.518416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434030+0000","last_change":"2024-04-25T10:02:02.373840+0000","last_active":"2024-04-25T10:02:03.434030+0000","last_peered":"2024-04-25T10:02:03.434030+0000","last_clean":"2024-04-25T10:02:03.434030+0000","last_became_active":"2024-04-25T10:02:02.373426+0000","last_became_peered":"2024-04-25T10:02:02.373426+0000","last_unstale":"2024-04-25T10:02:03.434030+0000","last_undegraded":"2024-04-25T10:02:03.434030+0000","last_fullsized":"2024-04-25T10:02:03.434030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:54.471032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348267+0000","last_change":"2024-04-25T10:01:58.076731+0000","last_active":"2024-04-25T10:02:03.348267+0000","last_peered":"2024-04-25T10:02:03.348267+0000","last_clean":"2024-04-25T10:02:03.348267+0000","last_became_active":"2024-04-25T10:01:58.076430+0000","last_became_peered":"2024-04-25T10:01:58.076430+0000","last_unstale":"2024-04-25T10:02:03.348267+0000","last_undegraded":"2024-04-25T10:02:03.348267+0000","last_fullsized":"2024-04-25T10:02:03.348267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402642+0000","last_change":"2024-04-25T10:01:57.061549+0000","last_active":"2024-04-25T10:02:03.402642+0000","last_peered":"2024-04-25T10:02:03.402642+0000","last_clean":"2024-04-25T10:02:03.402642+0000","last_became_active":"2024-04-25T10:01:57.056797+0000","last_became_peered":"2024-04-25T10:01:57.056797+0000","last_unstale":"2024-04-25T10:02:03.402642+0000","last_undegraded":"2024-04-25T10:02:03.402642+0000","last_fullsized":"2024-04-25T10:02:03.402642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.994799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519398+0000","last_change":"2024-04-25T10:02:02.372845+0000","last_active":"2024-04-25T10:02:03.519398+0000","last_peered":"2024-04-25T10:02:03.519398+0000","last_clean":"2024-04-25T10:02:03.519398+0000","last_became_active":"2024-04-25T10:02:02.370004+0000","last_became_peered":"2024-04-25T10:02:02.370004+0000","last_unstale":"2024-04-25T10:02:03.519398+0000","last_undegraded":"2024-04-25T10:02:03.519398+0000","last_fullsized":"2024-04-25T10:02:03.519398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518640+0000","last_change":"2024-04-25T10:01:58.120136+0000","last_active":"2024-04-25T10:02:03.518640+0000","last_peered":"2024-04-25T10:02:03.518640+0000","last_clean":"2024-04-25T10:02:03.518640+0000","last_became_active":"2024-04-25T10:01:58.111232+0000","last_became_peered":"2024-04-25T10:01:58.111232+0000","last_unstale":"2024-04-25T10:02:03.518640+0000","last_undegraded":"2024-04-25T10:02:03.518640+0000","last_fullsized":"2024-04-25T10:02:03.518640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:17.032865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353998+0000","last_change":"2024-04-25T10:01:57.047615+0000","last_active":"2024-04-25T10:02:03.353998+0000","last_peered":"2024-04-25T10:02:03.353998+0000","last_clean":"2024-04-25T10:02:03.353998+0000","last_became_active":"2024-04-25T10:01:57.047321+0000","last_became_peered":"2024-04-25T10:01:57.047321+0000","last_unstale":"2024-04-25T10:02:03.353998+0000","last_undegraded":"2024-04-25T10:02:03.353998+0000","last_fullsized":"2024-04-25T10:02:03.353998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404368+0000","last_change":"2024-04-25T10:02:02.361172+0000","last_active":"2024-04-25T10:02:03.404368+0000","last_peered":"2024-04-25T10:02:03.404368+0000","last_clean":"2024-04-25T10:02:03.404368+0000","last_became_active":"2024-04-25T10:02:02.360349+0000","last_became_peered":"2024-04-25T10:02:02.360349+0000","last_unstale":"2024-04-25T10:02:03.404368+0000","last_undegraded":"2024-04-25T10:02:03.404368+0000","last_fullsized":"2024-04-25T10:02:03.404368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.788023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518563+0000","last_change":"2024-04-25T10:01:58.127068+0000","last_active":"2024-04-25T10:02:03.518563+0000","last_peered":"2024-04-25T10:02:03.518563+0000","last_clean":"2024-04-25T10:02:03.518563+0000","last_became_active":"2024-04-25T10:01:58.126258+0000","last_became_peered":"2024-04-25T10:01:58.126258+0000","last_unstale":"2024-04-25T10:02:03.518563+0000","last_undegraded":"2024-04-25T10:02:03.518563+0000","last_fullsized":"2024-04-25T10:02:03.518563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:15.775667+0000","last_change":"2024-04-25T10:01:57.054259+0000","last_active":"2024-04-25T10:02:15.775667+0000","last_peered":"2024-04-25T10:02:15.775667+0000","last_clean":"2024-04-25T10:02:15.775667+0000","last_became_active":"2024-04-25T10:01:57.053941+0000","last_became_peered":"2024-04-25T10:01:57.053941+0000","last_unstale":"2024-04-25T10:02:15.775667+0000","last_undegraded":"2024-04-25T10:02:15.775667+0000","last_fullsized":"2024-04-25T10:02:15.775667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:41.210591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354900+0000","last_change":"2024-04-25T10:02:02.376549+0000","last_active":"2024-04-25T10:02:03.354900+0000","last_peered":"2024-04-25T10:02:03.354900+0000","last_clean":"2024-04-25T10:02:03.354900+0000","last_became_active":"2024-04-25T10:02:02.376052+0000","last_became_peered":"2024-04-25T10:02:02.376052+0000","last_unstale":"2024-04-25T10:02:03.354900+0000","last_undegraded":"2024-04-25T10:02:03.354900+0000","last_fullsized":"2024-04-25T10:02:03.354900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:43.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.403613+0000","last_change":"2024-04-25T10:01:58.078715+0000","last_active":"2024-04-25T10:02:03.403613+0000","last_peered":"2024-04-25T10:02:03.403613+0000","last_clean":"2024-04-25T10:02:03.403613+0000","last_became_active":"2024-04-25T10:01:58.078363+0000","last_became_peered":"2024-04-25T10:01:58.078363+0000","last_unstale":"2024-04-25T10:02:03.403613+0000","last_undegraded":"2024-04-25T10:02:03.403613+0000","last_fullsized":"2024-04-25T10:02:03.403613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:25.310967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354395+0000","last_change":"2024-04-25T10:01:57.054914+0000","last_active":"2024-04-25T10:02:03.354395+0000","last_peered":"2024-04-25T10:02:03.354395+0000","last_clean":"2024-04-25T10:02:03.354395+0000","last_became_active":"2024-04-25T10:01:57.054640+0000","last_became_peered":"2024-04-25T10:01:57.054640+0000","last_unstale":"2024-04-25T10:02:03.354395+0000","last_undegraded":"2024-04-25T10:02:03.354395+0000","last_fullsized":"2024-04-25T10:02:03.354395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522376+0000","last_change":"2024-04-25T10:02:02.362949+0000","last_active":"2024-04-25T10:02:03.522376+0000","last_peered":"2024-04-25T10:02:03.522376+0000","last_clean":"2024-04-25T10:02:03.522376+0000","last_became_active":"2024-04-25T10:02:02.361660+0000","last_became_peered":"2024-04-25T10:02:02.361660+0000","last_unstale":"2024-04-25T10:02:03.522376+0000","last_undegraded":"2024-04-25T10:02:03.522376+0000","last_fullsized":"2024-04-25T10:02:03.522376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434111+0000","last_change":"2024-04-25T10:01:58.114459+0000","last_active":"2024-04-25T10:02:03.434111+0000","last_peered":"2024-04-25T10:02:03.434111+0000","last_clean":"2024-04-25T10:02:03.434111+0000","last_became_active":"2024-04-25T10:01:58.113079+0000","last_became_peered":"2024-04-25T10:01:58.113079+0000","last_unstale":"2024-04-25T10:02:03.434111+0000","last_undegraded":"2024-04-25T10:02:03.434111+0000","last_fullsized":"2024-04-25T10:02:03.434111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518456+0000","last_change":"2024-04-25T10:01:57.058749+0000","last_active":"2024-04-25T10:02:03.518456+0000","last_peered":"2024-04-25T10:02:03.518456+0000","last_clean":"2024-04-25T10:02:03.518456+0000","last_became_active":"2024-04-25T10:01:57.058498+0000","last_became_peered":"2024-04-25T10:01:57.058498+0000","last_unstale":"2024-04-25T10:02:03.518456+0000","last_undegraded":"2024-04-25T10:02:03.518456+0000","last_fullsized":"2024-04-25T10:02:03.518456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352717+0000","last_change":"2024-04-25T10:02:02.354682+0000","last_active":"2024-04-25T10:02:03.352717+0000","last_peered":"2024-04-25T10:02:03.352717+0000","last_clean":"2024-04-25T10:02:03.352717+0000","last_became_active":"2024-04-25T10:02:02.353840+0000","last_became_peered":"2024-04-25T10:02:02.353840+0000","last_unstale":"2024-04-25T10:02:03.352717+0000","last_undegraded":"2024-04-25T10:02:03.352717+0000","last_fullsized":"2024-04-25T10:02:03.352717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.306270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521836+0000","last_change":"2024-04-25T10:01:58.075136+0000","last_active":"2024-04-25T10:02:03.521836+0000","last_peered":"2024-04-25T10:02:03.521836+0000","last_clean":"2024-04-25T10:02:03.521836+0000","last_became_active":"2024-04-25T10:01:58.074823+0000","last_became_peered":"2024-04-25T10:01:58.074823+0000","last_unstale":"2024-04-25T10:02:03.521836+0000","last_undegraded":"2024-04-25T10:02:03.521836+0000","last_fullsized":"2024-04-25T10:02:03.521836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524041+0000","last_change":"2024-04-25T10:01:57.058387+0000","last_active":"2024-04-25T10:02:03.524041+0000","last_peered":"2024-04-25T10:02:03.524041+0000","last_clean":"2024-04-25T10:02:03.524041+0000","last_became_active":"2024-04-25T10:01:57.057994+0000","last_became_peered":"2024-04-25T10:01:57.057994+0000","last_unstale":"2024-04-25T10:02:03.524041+0000","last_undegraded":"2024-04-25T10:02:03.524041+0000","last_fullsized":"2024-04-25T10:02:03.524041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.171109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353666+0000","last_change":"2024-04-25T10:02:02.374647+0000","last_active":"2024-04-25T10:02:03.353666+0000","last_peered":"2024-04-25T10:02:03.353666+0000","last_clean":"2024-04-25T10:02:03.353666+0000","last_became_active":"2024-04-25T10:02:02.374168+0000","last_became_peered":"2024-04-25T10:02:02.374168+0000","last_unstale":"2024-04-25T10:02:03.353666+0000","last_undegraded":"2024-04-25T10:02:03.353666+0000","last_fullsized":"2024-04-25T10:02:03.353666+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:13.963774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356050+0000","last_change":"2024-04-25T10:02:02.374581+0000","last_active":"2024-04-25T10:02:03.356050+0000","last_peered":"2024-04-25T10:02:03.356050+0000","last_clean":"2024-04-25T10:02:03.356050+0000","last_became_active":"2024-04-25T10:02:02.373942+0000","last_became_peered":"2024-04-25T10:02:02.373942+0000","last_unstale":"2024-04-25T10:02:03.356050+0000","last_undegraded":"2024-04-25T10:02:03.356050+0000","last_fullsized":"2024-04-25T10:02:03.356050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.252346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353803+0000","last_change":"2024-04-25T10:01:58.111477+0000","last_active":"2024-04-25T10:02:03.353803+0000","last_peered":"2024-04-25T10:02:03.353803+0000","last_clean":"2024-04-25T10:02:03.353803+0000","last_became_active":"2024-04-25T10:01:58.111095+0000","last_became_peered":"2024-04-25T10:01:58.111095+0000","last_unstale":"2024-04-25T10:02:03.353803+0000","last_undegraded":"2024-04-25T10:02:03.353803+0000","last_fullsized":"2024-04-25T10:02:03.353803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:42.787884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347909+0000","last_change":"2024-04-25T10:01:57.058954+0000","last_active":"2024-04-25T10:02:03.347909+0000","last_peered":"2024-04-25T10:02:03.347909+0000","last_clean":"2024-04-25T10:02:03.347909+0000","last_became_active":"2024-04-25T10:01:57.056146+0000","last_became_peered":"2024-04-25T10:01:57.056146+0000","last_unstale":"2024-04-25T10:02:03.347909+0000","last_undegraded":"2024-04-25T10:02:03.347909+0000","last_fullsized":"2024-04-25T10:02:03.347909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404389+0000","last_change":"2024-04-25T10:02:02.360724+0000","last_active":"2024-04-25T10:02:03.404389+0000","last_peered":"2024-04-25T10:02:03.404389+0000","last_clean":"2024-04-25T10:02:03.404389+0000","last_became_active":"2024-04-25T10:02:02.360282+0000","last_became_peered":"2024-04-25T10:02:02.360282+0000","last_unstale":"2024-04-25T10:02:03.404389+0000","last_undegraded":"2024-04-25T10:02:03.404389+0000","last_fullsized":"2024-04-25T10:02:03.404389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:48.141173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432340+0000","last_change":"2024-04-25T10:01:58.126960+0000","last_active":"2024-04-25T10:02:03.432340+0000","last_peered":"2024-04-25T10:02:03.432340+0000","last_clean":"2024-04-25T10:02:03.432340+0000","last_became_active":"2024-04-25T10:01:58.126638+0000","last_became_peered":"2024-04-25T10:01:58.126638+0000","last_unstale":"2024-04-25T10:02:03.432340+0000","last_undegraded":"2024-04-25T10:02:03.432340+0000","last_fullsized":"2024-04-25T10:02:03.432340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518686+0000","last_change":"2024-04-25T10:01:57.056000+0000","last_active":"2024-04-25T10:02:03.518686+0000","last_peered":"2024-04-25T10:02:03.518686+0000","last_clean":"2024-04-25T10:02:03.518686+0000","last_became_active":"2024-04-25T10:01:57.055665+0000","last_became_peered":"2024-04-25T10:01:57.055665+0000","last_unstale":"2024-04-25T10:02:03.518686+0000","last_undegraded":"2024-04-25T10:02:03.518686+0000","last_fullsized":"2024-04-25T10:02:03.518686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402377+0000","last_change":"2024-04-25T10:01:57.064311+0000","last_active":"2024-04-25T10:02:03.402377+0000","last_peered":"2024-04-25T10:02:03.402377+0000","last_clean":"2024-04-25T10:02:03.402377+0000","last_became_active":"2024-04-25T10:01:57.063955+0000","last_became_peered":"2024-04-25T10:01:57.063955+0000","last_unstale":"2024-04-25T10:02:03.402377+0000","last_undegraded":"2024-04-25T10:02:03.402377+0000","last_fullsized":"2024-04-25T10:02:03.402377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353034+0000","last_change":"2024-04-25T10:01:58.112383+0000","last_active":"2024-04-25T10:02:03.353034+0000","last_peered":"2024-04-25T10:02:03.353034+0000","last_clean":"2024-04-25T10:02:03.353034+0000","last_became_active":"2024-04-25T10:01:58.111906+0000","last_became_peered":"2024-04-25T10:01:58.111906+0000","last_unstale":"2024-04-25T10:02:03.353034+0000","last_undegraded":"2024-04-25T10:02:03.353034+0000","last_fullsized":"2024-04-25T10:02:03.353034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:58.408032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355526+0000","last_change":"2024-04-25T10:02:02.370310+0000","last_active":"2024-04-25T10:02:03.355526+0000","last_peered":"2024-04-25T10:02:03.355526+0000","last_clean":"2024-04-25T10:02:03.355526+0000","last_became_active":"2024-04-25T10:02:02.369660+0000","last_became_peered":"2024-04-25T10:02:02.369660+0000","last_unstale":"2024-04-25T10:02:03.355526+0000","last_undegraded":"2024-04-25T10:02:03.355526+0000","last_fullsized":"2024-04-25T10:02:03.355526+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:59.232453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433292+0000","last_change":"2024-04-25T10:01:57.059797+0000","last_active":"2024-04-25T10:02:03.433292+0000","last_peered":"2024-04-25T10:02:03.433292+0000","last_clean":"2024-04-25T10:02:03.433292+0000","last_became_active":"2024-04-25T10:01:57.059483+0000","last_became_peered":"2024-04-25T10:01:57.059483+0000","last_unstale":"2024-04-25T10:02:03.433292+0000","last_undegraded":"2024-04-25T10:02:03.433292+0000","last_fullsized":"2024-04-25T10:02:03.433292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521515+0000","last_change":"2024-04-25T10:01:58.118406+0000","last_active":"2024-04-25T10:02:03.521515+0000","last_peered":"2024-04-25T10:02:03.521515+0000","last_clean":"2024-04-25T10:02:03.521515+0000","last_became_active":"2024-04-25T10:01:58.118066+0000","last_became_peered":"2024-04-25T10:01:58.118066+0000","last_unstale":"2024-04-25T10:02:03.521515+0000","last_undegraded":"2024-04-25T10:02:03.521515+0000","last_fullsized":"2024-04-25T10:02:03.521515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356020+0000","last_change":"2024-04-25T10:02:02.360301+0000","last_active":"2024-04-25T10:02:03.356020+0000","last_peered":"2024-04-25T10:02:03.356020+0000","last_clean":"2024-04-25T10:02:03.356020+0000","last_became_active":"2024-04-25T10:02:02.359873+0000","last_became_peered":"2024-04-25T10:02:02.359873+0000","last_unstale":"2024-04-25T10:02:03.356020+0000","last_undegraded":"2024-04-25T10:02:03.356020+0000","last_fullsized":"2024-04-25T10:02:03.356020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:58.571611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519243+0000","last_change":"2024-04-25T10:01:58.119106+0000","last_active":"2024-04-25T10:02:03.519243+0000","last_peered":"2024-04-25T10:02:03.519243+0000","last_clean":"2024-04-25T10:02:03.519243+0000","last_became_active":"2024-04-25T10:01:58.118822+0000","last_became_peered":"2024-04-25T10:01:58.118822+0000","last_unstale":"2024-04-25T10:02:03.519243+0000","last_undegraded":"2024-04-25T10:02:03.519243+0000","last_fullsized":"2024-04-25T10:02:03.519243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352667+0000","last_change":"2024-04-25T10:01:58.080145+0000","last_active":"2024-04-25T10:02:03.352667+0000","last_peered":"2024-04-25T10:02:03.352667+0000","last_clean":"2024-04-25T10:02:03.352667+0000","last_became_active":"2024-04-25T10:01:58.079388+0000","last_became_peered":"2024-04-25T10:01:58.079388+0000","last_unstale":"2024-04-25T10:02:03.352667+0000","last_undegraded":"2024-04-25T10:02:03.352667+0000","last_fullsized":"2024-04-25T10:02:03.352667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347995+0000","last_change":"2024-04-25T10:01:58.118742+0000","last_active":"2024-04-25T10:02:03.347995+0000","last_peered":"2024-04-25T10:02:03.347995+0000","last_clean":"2024-04-25T10:02:03.347995+0000","last_became_active":"2024-04-25T10:01:58.118407+0000","last_became_peered":"2024-04-25T10:01:58.118407+0000","last_unstale":"2024-04-25T10:02:03.347995+0000","last_undegraded":"2024-04-25T10:02:03.347995+0000","last_fullsized":"2024-04-25T10:02:03.347995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.311295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432945+0000","last_change":"2024-04-25T10:01:58.129321+0000","last_active":"2024-04-25T10:02:03.432945+0000","last_peered":"2024-04-25T10:02:03.432945+0000","last_clean":"2024-04-25T10:02:03.432945+0000","last_became_active":"2024-04-25T10:01:58.128642+0000","last_became_peered":"2024-04-25T10:01:58.128642+0000","last_unstale":"2024-04-25T10:02:03.432945+0000","last_undegraded":"2024-04-25T10:02:03.432945+0000","last_fullsized":"2024-04-25T10:02:03.432945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519722+0000","last_change":"2024-04-25T10:01:58.114750+0000","last_active":"2024-04-25T10:02:03.519722+0000","last_peered":"2024-04-25T10:02:03.519722+0000","last_clean":"2024-04-25T10:02:03.519722+0000","last_became_active":"2024-04-25T10:01:58.114410+0000","last_became_peered":"2024-04-25T10:01:58.114410+0000","last_unstale":"2024-04-25T10:02:03.519722+0000","last_undegraded":"2024-04-25T10:02:03.519722+0000","last_fullsized":"2024-04-25T10:02:03.519722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:27.145938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519712+0000","last_change":"2024-04-25T10:01:58.116458+0000","last_active":"2024-04-25T10:02:03.519712+0000","last_peered":"2024-04-25T10:02:03.519712+0000","last_clean":"2024-04-25T10:02:03.519712+0000","last_became_active":"2024-04-25T10:01:58.116051+0000","last_became_peered":"2024-04-25T10:01:58.116051+0000","last_unstale":"2024-04-25T10:02:03.519712+0000","last_undegraded":"2024-04-25T10:02:03.519712+0000","last_fullsized":"2024-04-25T10:02:03.519712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521818+0000","last_change":"2024-04-25T10:01:58.079855+0000","last_active":"2024-04-25T10:02:03.521818+0000","last_peered":"2024-04-25T10:02:03.521818+0000","last_clean":"2024-04-25T10:02:03.521818+0000","last_became_active":"2024-04-25T10:01:58.079522+0000","last_became_peered":"2024-04-25T10:01:58.079522+0000","last_unstale":"2024-04-25T10:02:03.521818+0000","last_undegraded":"2024-04-25T10:02:03.521818+0000","last_fullsized":"2024-04-25T10:02:03.521818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433705+0000","last_change":"2024-04-25T10:01:58.114808+0000","last_active":"2024-04-25T10:02:03.433705+0000","last_peered":"2024-04-25T10:02:03.433705+0000","last_clean":"2024-04-25T10:02:03.433705+0000","last_became_active":"2024-04-25T10:01:58.114656+0000","last_became_peered":"2024-04-25T10:01:58.114656+0000","last_unstale":"2024-04-25T10:02:03.433705+0000","last_undegraded":"2024-04-25T10:02:03.433705+0000","last_fullsized":"2024-04-25T10:02:03.433705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347899+0000","last_change":"2024-04-25T10:01:58.125563+0000","last_active":"2024-04-25T10:02:03.347899+0000","last_peered":"2024-04-25T10:02:03.347899+0000","last_clean":"2024-04-25T10:02:03.347899+0000","last_became_active":"2024-04-25T10:01:58.125071+0000","last_became_peered":"2024-04-25T10:01:58.125071+0000","last_unstale":"2024-04-25T10:02:03.347899+0000","last_undegraded":"2024-04-25T10:02:03.347899+0000","last_fullsized":"2024-04-25T10:02:03.347899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432367+0000","last_change":"2024-04-25T10:01:58.111903+0000","last_active":"2024-04-25T10:02:03.432367+0000","last_peered":"2024-04-25T10:02:03.432367+0000","last_clean":"2024-04-25T10:02:03.432367+0000","last_became_active":"2024-04-25T10:01:58.111366+0000","last_became_peered":"2024-04-25T10:01:58.111366+0000","last_unstale":"2024-04-25T10:02:03.432367+0000","last_undegraded":"2024-04-25T10:02:03.432367+0000","last_fullsized":"2024-04-25T10:02:03.432367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521208+0000","last_change":"2024-04-25T10:01:58.066982+0000","last_active":"2024-04-25T10:02:03.521208+0000","last_peered":"2024-04-25T10:02:03.521208+0000","last_clean":"2024-04-25T10:02:03.521208+0000","last_became_active":"2024-04-25T10:01:58.066582+0000","last_became_peered":"2024-04-25T10:01:58.066582+0000","last_unstale":"2024-04-25T10:02:03.521208+0000","last_undegraded":"2024-04-25T10:02:03.521208+0000","last_fullsized":"2024-04-25T10:02:03.521208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352779+0000","last_change":"2024-04-25T10:01:58.112990+0000","last_active":"2024-04-25T10:02:03.352779+0000","last_peered":"2024-04-25T10:02:03.352779+0000","last_clean":"2024-04-25T10:02:03.352779+0000","last_became_active":"2024-04-25T10:01:58.112724+0000","last_became_peered":"2024-04-25T10:01:58.112724+0000","last_unstale":"2024-04-25T10:02:03.352779+0000","last_undegraded":"2024-04-25T10:02:03.352779+0000","last_fullsized":"2024-04-25T10:02:03.352779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:36.503846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519412+0000","last_change":"2024-04-25T10:01:58.067953+0000","last_active":"2024-04-25T10:02:03.519412+0000","last_peered":"2024-04-25T10:02:03.519412+0000","last_clean":"2024-04-25T10:02:03.519412+0000","last_became_active":"2024-04-25T10:01:58.067363+0000","last_became_peered":"2024-04-25T10:01:58.067363+0000","last_unstale":"2024-04-25T10:02:03.519412+0000","last_undegraded":"2024-04-25T10:02:03.519412+0000","last_fullsized":"2024-04-25T10:02:03.519412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404814+0000","last_change":"2024-04-25T10:01:58.112476+0000","last_active":"2024-04-25T10:02:03.404814+0000","last_peered":"2024-04-25T10:02:03.404814+0000","last_clean":"2024-04-25T10:02:03.404814+0000","last_became_active":"2024-04-25T10:01:58.112177+0000","last_became_peered":"2024-04-25T10:01:58.112177+0000","last_unstale":"2024-04-25T10:02:03.404814+0000","last_undegraded":"2024-04-25T10:02:03.404814+0000","last_fullsized":"2024-04-25T10:02:03.404814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:50.172245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355444+0000","last_change":"2024-04-25T10:01:58.056526+0000","last_active":"2024-04-25T10:02:03.355444+0000","last_peered":"2024-04-25T10:02:03.355444+0000","last_clean":"2024-04-25T10:02:03.355444+0000","last_became_active":"2024-04-25T10:01:58.055509+0000","last_became_peered":"2024-04-25T10:01:58.055509+0000","last_unstale":"2024-04-25T10:02:03.355444+0000","last_undegraded":"2024-04-25T10:02:03.355444+0000","last_fullsized":"2024-04-25T10:02:03.355444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405090+0000","last_change":"2024-04-25T10:01:58.079032+0000","last_active":"2024-04-25T10:02:03.405090+0000","last_peered":"2024-04-25T10:02:03.405090+0000","last_clean":"2024-04-25T10:02:03.405090+0000","last_became_active":"2024-04-25T10:01:58.077635+0000","last_became_peered":"2024-04-25T10:01:58.077635+0000","last_unstale":"2024-04-25T10:02:03.405090+0000","last_undegraded":"2024-04-25T10:02:03.405090+0000","last_fullsized":"2024-04-25T10:02:03.405090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:00.440369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347561+0000","last_change":"2024-04-25T10:01:58.076733+0000","last_active":"2024-04-25T10:02:03.347561+0000","last_peered":"2024-04-25T10:02:03.347561+0000","last_clean":"2024-04-25T10:02:03.347561+0000","last_became_active":"2024-04-25T10:01:58.076476+0000","last_became_peered":"2024-04-25T10:01:58.076476+0000","last_unstale":"2024-04-25T10:02:03.347561+0000","last_undegraded":"2024-04-25T10:02:03.347561+0000","last_fullsized":"2024-04-25T10:02:03.347561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432026+0000","last_change":"2024-04-25T10:01:58.116156+0000","last_active":"2024-04-25T10:02:03.432026+0000","last_peered":"2024-04-25T10:02:03.432026+0000","last_clean":"2024-04-25T10:02:03.432026+0000","last_became_active":"2024-04-25T10:01:58.115483+0000","last_became_peered":"2024-04-25T10:01:58.115483+0000","last_unstale":"2024-04-25T10:02:03.432026+0000","last_undegraded":"2024-04-25T10:02:03.432026+0000","last_fullsized":"2024-04-25T10:02:03.432026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518554+0000","last_change":"2024-04-25T10:01:58.120123+0000","last_active":"2024-04-25T10:02:03.518554+0000","last_peered":"2024-04-25T10:02:03.518554+0000","last_clean":"2024-04-25T10:02:03.518554+0000","last_became_active":"2024-04-25T10:01:58.119852+0000","last_became_peered":"2024-04-25T10:01:58.119852+0000","last_unstale":"2024-04-25T10:02:03.518554+0000","last_undegraded":"2024-04-25T10:02:03.518554+0000","last_fullsized":"2024-04-25T10:02:03.518554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519056+0000","last_change":"2024-04-25T10:01:58.113249+0000","last_active":"2024-04-25T10:02:03.519056+0000","last_peered":"2024-04-25T10:02:03.519056+0000","last_clean":"2024-04-25T10:02:03.519056+0000","last_became_active":"2024-04-25T10:01:58.112298+0000","last_became_peered":"2024-04-25T10:01:58.112298+0000","last_unstale":"2024-04-25T10:02:03.519056+0000","last_undegraded":"2024-04-25T10:02:03.519056+0000","last_fullsized":"2024-04-25T10:02:03.519056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519272+0000","last_change":"2024-04-25T10:01:58.072097+0000","last_active":"2024-04-25T10:02:03.519272+0000","last_peered":"2024-04-25T10:02:03.519272+0000","last_clean":"2024-04-25T10:02:03.519272+0000","last_became_active":"2024-04-25T10:01:58.071662+0000","last_became_peered":"2024-04-25T10:01:58.071662+0000","last_unstale":"2024-04-25T10:02:03.519272+0000","last_undegraded":"2024-04-25T10:02:03.519272+0000","last_fullsized":"2024-04-25T10:02:03.519272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518405+0000","last_change":"2024-04-25T10:01:58.073877+0000","last_active":"2024-04-25T10:02:03.518405+0000","last_peered":"2024-04-25T10:02:03.518405+0000","last_clean":"2024-04-25T10:02:03.518405+0000","last_became_active":"2024-04-25T10:01:58.073554+0000","last_became_peered":"2024-04-25T10:01:58.073554+0000","last_unstale":"2024-04-25T10:02:03.518405+0000","last_undegraded":"2024-04-25T10:02:03.518405+0000","last_fullsized":"2024-04-25T10:02:03.518405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518311+0000","last_change":"2024-04-25T10:01:58.066768+0000","last_active":"2024-04-25T10:02:03.518311+0000","last_peered":"2024-04-25T10:02:03.518311+0000","last_clean":"2024-04-25T10:02:03.518311+0000","last_became_active":"2024-04-25T10:01:58.065944+0000","last_became_peered":"2024-04-25T10:01:58.065944+0000","last_unstale":"2024-04-25T10:02:03.518311+0000","last_undegraded":"2024-04-25T10:02:03.518311+0000","last_fullsized":"2024-04-25T10:02:03.518311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524004+0000","last_change":"2024-04-25T10:01:58.066685+0000","last_active":"2024-04-25T10:02:03.524004+0000","last_peered":"2024-04-25T10:02:03.524004+0000","last_clean":"2024-04-25T10:02:03.524004+0000","last_became_active":"2024-04-25T10:01:58.066367+0000","last_became_peered":"2024-04-25T10:01:58.066367+0000","last_unstale":"2024-04-25T10:02:03.524004+0000","last_undegraded":"2024-04-25T10:02:03.524004+0000","last_fullsized":"2024-04-25T10:02:03.524004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432058+0000","last_change":"2024-04-25T10:01:58.123950+0000","last_active":"2024-04-25T10:02:03.432058+0000","last_peered":"2024-04-25T10:02:03.432058+0000","last_clean":"2024-04-25T10:02:03.432058+0000","last_became_active":"2024-04-25T10:01:58.123668+0000","last_became_peered":"2024-04-25T10:01:58.123668+0000","last_unstale":"2024-04-25T10:02:03.432058+0000","last_undegraded":"2024-04-25T10:02:03.432058+0000","last_fullsized":"2024-04-25T10:02:03.432058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:30.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518202+0000","last_change":"2024-04-25T10:01:58.114958+0000","last_active":"2024-04-25T10:02:03.518202+0000","last_peered":"2024-04-25T10:02:03.518202+0000","last_clean":"2024-04-25T10:02:03.518202+0000","last_became_active":"2024-04-25T10:01:58.114599+0000","last_became_peered":"2024-04-25T10:01:58.114599+0000","last_unstale":"2024-04-25T10:02:03.518202+0000","last_undegraded":"2024-04-25T10:02:03.518202+0000","last_fullsized":"2024-04-25T10:02:03.518202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521335+0000","last_change":"2024-04-25T10:01:58.116405+0000","last_active":"2024-04-25T10:02:03.521335+0000","last_peered":"2024-04-25T10:02:03.521335+0000","last_clean":"2024-04-25T10:02:03.521335+0000","last_became_active":"2024-04-25T10:01:58.115946+0000","last_became_peered":"2024-04-25T10:01:58.115946+0000","last_unstale":"2024-04-25T10:02:03.521335+0000","last_undegraded":"2024-04-25T10:02:03.521335+0000","last_fullsized":"2024-04-25T10:02:03.521335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:57.669295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518323+0000","last_change":"2024-04-25T10:01:58.117287+0000","last_active":"2024-04-25T10:02:03.518323+0000","last_peered":"2024-04-25T10:02:03.518323+0000","last_clean":"2024-04-25T10:02:03.518323+0000","last_became_active":"2024-04-25T10:01:58.117024+0000","last_became_peered":"2024-04-25T10:01:58.117024+0000","last_unstale":"2024-04-25T10:02:03.518323+0000","last_undegraded":"2024-04-25T10:02:03.518323+0000","last_fullsized":"2024-04-25T10:02:03.518323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522601+0000","last_change":"2024-04-25T10:01:58.116367+0000","last_active":"2024-04-25T10:02:03.522601+0000","last_peered":"2024-04-25T10:02:03.522601+0000","last_clean":"2024-04-25T10:02:03.522601+0000","last_became_active":"2024-04-25T10:01:58.115801+0000","last_became_peered":"2024-04-25T10:01:58.115801+0000","last_unstale":"2024-04-25T10:02:03.522601+0000","last_undegraded":"2024-04-25T10:02:03.522601+0000","last_fullsized":"2024-04-25T10:02:03.522601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433436+0000","last_change":"2024-04-25T10:01:58.118411+0000","last_active":"2024-04-25T10:02:03.433436+0000","last_peered":"2024-04-25T10:02:03.433436+0000","last_clean":"2024-04-25T10:02:03.433436+0000","last_became_active":"2024-04-25T10:01:58.118131+0000","last_became_peered":"2024-04-25T10:01:58.118131+0000","last_unstale":"2024-04-25T10:02:03.433436+0000","last_undegraded":"2024-04-25T10:02:03.433436+0000","last_fullsized":"2024-04-25T10:02:03.433436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433580+0000","last_change":"2024-04-25T10:01:58.121692+0000","last_active":"2024-04-25T10:02:03.433580+0000","last_peered":"2024-04-25T10:02:03.433580+0000","last_clean":"2024-04-25T10:02:03.433580+0000","last_became_active":"2024-04-25T10:01:58.120392+0000","last_became_peered":"2024-04-25T10:01:58.120392+0000","last_unstale":"2024-04-25T10:02:03.433580+0000","last_undegraded":"2024-04-25T10:02:03.433580+0000","last_fullsized":"2024-04-25T10:02:03.433580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520287+0000","last_change":"2024-04-25T10:01:58.080365+0000","last_active":"2024-04-25T10:02:03.520287+0000","last_peered":"2024-04-25T10:02:03.520287+0000","last_clean":"2024-04-25T10:02:03.520287+0000","last_became_active":"2024-04-25T10:01:58.080055+0000","last_became_peered":"2024-04-25T10:01:58.080055+0000","last_unstale":"2024-04-25T10:02:03.520287+0000","last_undegraded":"2024-04-25T10:02:03.520287+0000","last_fullsized":"2024-04-25T10:02:03.520287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:31.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.807+0000 7f03cd200640 1 Processor -- start 2024-04-25T10:02:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.808+0000 7f03cd200640 1 -- start start 2024-04-25T10:02:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.809+0000 7f03cd200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03c80d0aa0 0x7f03c8161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.809+0000 7f03cd200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03c81622b0 0x7f03c80ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.809+0000 7f03cd200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c815e8a0 con 0x7f03c815da60 2024-04-25T10:02:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.809+0000 7f03cd200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c815eb80 con 0x7f03c81622b0 2024-04-25T10:02:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.809+0000 7f03cd200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c815ee60 con 0x7f03c80d0aa0 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.809+0000 7f03c6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03c81622b0 0x7f03c80ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03c81622b0 0x7f03c80ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46784/0 (socket says 172.21.15.5:46784) 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c6800640 1 -- 172.21.15.5:0/2436806506 learned_addr learned my addr 172.21.15.5:0/2436806506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c5400640 1 --2- 172.21.15.5:0/2436806506 >> v2:172.21.15.80:3300/0 conn(0x7f03c80d0aa0 0x7f03c8161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c6800640 1 -- 172.21.15.5:0/2436806506 >> v2:172.21.15.80:3300/0 conn(0x7f03c80d0aa0 msgr2=0x7f03c8161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c6800640 1 --2- 172.21.15.5:0/2436806506 >> v2:172.21.15.80:3300/0 conn(0x7f03c80d0aa0 0x7f03c8161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c6800640 1 -- 172.21.15.5:0/2436806506 >> v1:172.21.15.80:6789/0 conn(0x7f03c815da60 legacy=0x7f03c815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.810+0000 7f03c6800640 1 -- 172.21.15.5:0/2436806506 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8161850 con 0x7f03c81622b0 2024-04-25T10:02:31.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.811+0000 7f03c6800640 1 --2- 172.21.15.5:0/2436806506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80ce520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f03c80b3260 tx=0x7f03c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654df0f5aad6f79f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.811+0000 7f03c4a00640 1 -- 172.21.15.5:0/2436806506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0037070 con 0x7f03c81622b0 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.811+0000 7f03c4a00640 1 -- 172.21.15.5:0/2436806506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c0032440 con 0x7f03c81622b0 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.811+0000 7f03c4a00640 1 -- 172.21.15.5:0/2436806506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0032810 con 0x7f03c81622b0 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 -- 172.21.15.5:0/2436806506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 msgr2=0x7f03c80ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 --2- 172.21.15.5:0/2436806506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80ce520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f03c80b3260 tx=0x7f03c002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 -- 172.21.15.5:0/2436806506 shutdown_connections 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 --2- 172.21.15.5:0/2436806506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80ce520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 --2- 172.21.15.5:0/2436806506 >> v2:172.21.15.80:3300/0 conn(0x7f03c80d0aa0 0x7f03c8161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 -- 172.21.15.5:0/2436806506 >> 172.21.15.5:0/2436806506 conn(0x7f03c806bfe0 msgr2=0x7f03c806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 -- 172.21.15.5:0/2436806506 shutdown_connections 2024-04-25T10:02:31.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.812+0000 7f03cd200640 1 -- 172.21.15.5:0/2436806506 wait complete. 2024-04-25T10:02:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.814+0000 7f03cd200640 1 Processor -- start 2024-04-25T10:02:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.814+0000 7f03cd200640 1 -- start start 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03cd200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03c80d0aa0 0x7f03c8156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03cd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03c815da60 0x7f03c81568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03cd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03cd200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80cebe0 con 0x7f03c81622b0 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03cd200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80cea60 con 0x7f03c815da60 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03cd200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80ced60 con 0x7f03c80d0aa0 2024-04-25T10:02:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03c6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03c6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46790/0 (socket says 172.21.15.5:46790) 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.815+0000 7f03c6800640 1 -- 172.21.15.5:0/1827249257 learned_addr learned my addr 172.21.15.5:0/1827249257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c5400640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03c815da60 0x7f03c81568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c6800640 1 -- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03c80d0aa0 msgr2=0x7f03c8156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c5e00640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03c80d0aa0 0x7f03c8156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c6800640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03c80d0aa0 0x7f03c8156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c6800640 1 -- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03c815da60 msgr2=0x7f03c81568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c6800640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03c815da60 0x7f03c81568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c6800640 1 -- 172.21.15.5:0/1827249257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c80b4330 con 0x7f03c81622b0 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c5400640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03c815da60 0x7f03c81568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03c6800640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80b69f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f03c00090b0 tx=0x7f03c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0041020 con 0x7f03c81622b0 2024-04-25T10:02:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00306c0 con 0x7f03c81622b0 2024-04-25T10:02:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0037040 con 0x7f03c81622b0 2024-04-25T10:02:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.816+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c81e7f30 con 0x7f03c81622b0 2024-04-25T10:02:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.817+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c81e7be0 con 0x7f03c81622b0 2024-04-25T10:02:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.818+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8068fd0 con 0x7f03c81622b0 2024-04-25T10:02:31.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.819+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03c0044020 con 0x7f03c81622b0 2024-04-25T10:02:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.820+0000 7f03be000640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f03a406b4d0 0x7f03a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.820+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f03c00aadf0 con 0x7f03c81622b0 2024-04-25T10:02:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.820+0000 7f03c5e00640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f03a406b4d0 0x7f03a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.821+0000 7f03c5e00640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f03a406b4d0 0x7f03a406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f03b8007bf0 tx=0x7f03b8007240 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:31.824+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03c004b030 con 0x7f03c81622b0 2024-04-25T10:02:32.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.008+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03c80680c0 con 0x7f03a406b4d0 2024-04-25T10:02:32.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.032+0000 7f03be000640 1 -- 172.21.15.5:0/1827249257 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f03b800c9d0 con 0x7f03a406b4d0 2024-04-25T10:02:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:32.043 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f03a406b4d0 msgr2=0x7f03a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f03a406b4d0 0x7f03a406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f03b8007bf0 tx=0x7f03b8007240 comp rx=0 tx=0).stop 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 msgr2=0x7f03c80b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80b69f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f03c00090b0 tx=0x7f03c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 shutdown_connections 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03c81622b0 0x7f03c80b69f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f03a406b4d0 0x7f03a406d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03c815da60 0x7f03c81568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 --2- 172.21.15.5:0/1827249257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03c80d0aa0 0x7f03c8156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 >> 172.21.15.5:0/1827249257 conn(0x7f03c806bfe0 msgr2=0x7f03c8073700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 shutdown_connections 2024-04-25T10:02:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.039+0000 7f03cd200640 1 -- 172.21.15.5:0/1827249257 wait complete. 2024-04-25T10:02:32.061 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T10:02:30.621258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236148,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738572,"statfs":{"total":773094113280,"available":772852297728,"internally_reserved":0,"allocated":3416064,"data_stored":2034928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005987"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356469+0000","last_change":"2024-04-25T10:01:57.061422+0000","last_active":"2024-04-25T10:02:03.356469+0000","last_peered":"2024-04-25T10:02:03.356469+0000","last_clean":"2024-04-25T10:02:03.356469+0000","last_became_active":"2024-04-25T10:01:57.061119+0000","last_became_peered":"2024-04-25T10:01:57.061119+0000","last_unstale":"2024-04-25T10:02:03.356469+0000","last_undegraded":"2024-04-25T10:02:03.356469+0000","last_fullsized":"2024-04-25T10:02:03.356469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:26.806679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433872+0000","last_change":"2024-04-25T10:01:58.115482+0000","last_active":"2024-04-25T10:02:03.433872+0000","last_peered":"2024-04-25T10:02:03.433872+0000","last_clean":"2024-04-25T10:02:03.433872+0000","last_became_active":"2024-04-25T10:01:58.115138+0000","last_became_peered":"2024-04-25T10:01:58.115138+0000","last_unstale":"2024-04-25T10:02:03.433872+0000","last_undegraded":"2024-04-25T10:02:03.433872+0000","last_fullsized":"2024-04-25T10:02:03.433872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352821+0000","last_change":"2024-04-25T10:02:02.368709+0000","last_active":"2024-04-25T10:02:03.352821+0000","last_peered":"2024-04-25T10:02:03.352821+0000","last_clean":"2024-04-25T10:02:03.352821+0000","last_became_active":"2024-04-25T10:02:02.367787+0000","last_became_peered":"2024-04-25T10:02:02.367787+0000","last_unstale":"2024-04-25T10:02:03.352821+0000","last_undegraded":"2024-04-25T10:02:03.352821+0000","last_fullsized":"2024-04-25T10:02:03.352821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:31.444725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434120+0000","last_change":"2024-04-25T10:01:57.061756+0000","last_active":"2024-04-25T10:02:03.434120+0000","last_peered":"2024-04-25T10:02:03.434120+0000","last_clean":"2024-04-25T10:02:03.434120+0000","last_became_active":"2024-04-25T10:01:57.058627+0000","last_became_peered":"2024-04-25T10:01:57.058627+0000","last_unstale":"2024-04-25T10:02:03.434120+0000","last_undegraded":"2024-04-25T10:02:03.434120+0000","last_fullsized":"2024-04-25T10:02:03.434120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434000+0000","last_change":"2024-04-25T10:01:58.113879+0000","last_active":"2024-04-25T10:02:03.434000+0000","last_peered":"2024-04-25T10:02:03.434000+0000","last_clean":"2024-04-25T10:02:03.434000+0000","last_became_active":"2024-04-25T10:01:58.113527+0000","last_became_peered":"2024-04-25T10:01:58.113527+0000","last_unstale":"2024-04-25T10:02:03.434000+0000","last_undegraded":"2024-04-25T10:02:03.434000+0000","last_fullsized":"2024-04-25T10:02:03.434000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519097+0000","last_change":"2024-04-25T10:02:02.364833+0000","last_active":"2024-04-25T10:02:03.519097+0000","last_peered":"2024-04-25T10:02:03.519097+0000","last_clean":"2024-04-25T10:02:03.519097+0000","last_became_active":"2024-04-25T10:02:02.364420+0000","last_became_peered":"2024-04-25T10:02:02.364420+0000","last_unstale":"2024-04-25T10:02:03.519097+0000","last_undegraded":"2024-04-25T10:02:03.519097+0000","last_fullsized":"2024-04-25T10:02:03.519097+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:17.582640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518857+0000","last_change":"2024-04-25T10:01:57.049323+0000","last_active":"2024-04-25T10:02:03.518857+0000","last_peered":"2024-04-25T10:02:03.518857+0000","last_clean":"2024-04-25T10:02:03.518857+0000","last_became_active":"2024-04-25T10:01:57.048997+0000","last_became_peered":"2024-04-25T10:01:57.048997+0000","last_unstale":"2024-04-25T10:02:03.518857+0000","last_undegraded":"2024-04-25T10:02:03.518857+0000","last_fullsized":"2024-04-25T10:02:03.518857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.523353+0000","last_change":"2024-04-25T10:01:58.074975+0000","last_active":"2024-04-25T10:02:03.523353+0000","last_peered":"2024-04-25T10:02:03.523353+0000","last_clean":"2024-04-25T10:02:03.523353+0000","last_became_active":"2024-04-25T10:01:58.074637+0000","last_became_peered":"2024-04-25T10:01:58.074637+0000","last_unstale":"2024-04-25T10:02:03.523353+0000","last_undegraded":"2024-04-25T10:02:03.523353+0000","last_fullsized":"2024-04-25T10:02:03.523353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522261+0000","last_change":"2024-04-25T10:02:02.367186+0000","last_active":"2024-04-25T10:02:03.522261+0000","last_peered":"2024-04-25T10:02:03.522261+0000","last_clean":"2024-04-25T10:02:03.522261+0000","last_became_active":"2024-04-25T10:02:02.366476+0000","last_became_peered":"2024-04-25T10:02:02.366476+0000","last_unstale":"2024-04-25T10:02:03.522261+0000","last_undegraded":"2024-04-25T10:02:03.522261+0000","last_fullsized":"2024-04-25T10:02:03.522261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519518+0000","last_change":"2024-04-25T10:01:57.058161+0000","last_active":"2024-04-25T10:02:03.519518+0000","last_peered":"2024-04-25T10:02:03.519518+0000","last_clean":"2024-04-25T10:02:03.519518+0000","last_became_active":"2024-04-25T10:01:57.057891+0000","last_became_peered":"2024-04-25T10:01:57.057891+0000","last_unstale":"2024-04-25T10:02:03.519518+0000","last_undegraded":"2024-04-25T10:02:03.519518+0000","last_fullsized":"2024-04-25T10:02:03.519518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:23.026096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404272+0000","last_change":"2024-04-25T10:01:58.074490+0000","last_active":"2024-04-25T10:02:03.404272+0000","last_peered":"2024-04-25T10:02:03.404272+0000","last_clean":"2024-04-25T10:02:03.404272+0000","last_became_active":"2024-04-25T10:01:58.074117+0000","last_became_peered":"2024-04-25T10:01:58.074117+0000","last_unstale":"2024-04-25T10:02:03.404272+0000","last_undegraded":"2024-04-25T10:02:03.404272+0000","last_fullsized":"2024-04-25T10:02:03.404272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:53.841219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520948+0000","last_change":"2024-04-25T10:02:02.362410+0000","last_active":"2024-04-25T10:02:03.520948+0000","last_peered":"2024-04-25T10:02:03.520948+0000","last_clean":"2024-04-25T10:02:03.520948+0000","last_became_active":"2024-04-25T10:02:02.359612+0000","last_became_peered":"2024-04-25T10:02:02.359612+0000","last_unstale":"2024-04-25T10:02:03.520948+0000","last_undegraded":"2024-04-25T10:02:03.520948+0000","last_fullsized":"2024-04-25T10:02:03.520948+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.136854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348157+0000","last_change":"2024-04-25T10:01:58.073917+0000","last_active":"2024-04-25T10:02:03.348157+0000","last_peered":"2024-04-25T10:02:03.348157+0000","last_clean":"2024-04-25T10:02:03.348157+0000","last_became_active":"2024-04-25T10:01:58.073122+0000","last_became_peered":"2024-04-25T10:01:58.073122+0000","last_unstale":"2024-04-25T10:02:03.348157+0000","last_undegraded":"2024-04-25T10:02:03.348157+0000","last_fullsized":"2024-04-25T10:02:03.348157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402599+0000","last_change":"2024-04-25T10:01:57.059507+0000","last_active":"2024-04-25T10:02:03.402599+0000","last_peered":"2024-04-25T10:02:03.402599+0000","last_clean":"2024-04-25T10:02:03.402599+0000","last_became_active":"2024-04-25T10:01:57.056083+0000","last_became_peered":"2024-04-25T10:01:57.056083+0000","last_unstale":"2024-04-25T10:02:03.402599+0000","last_undegraded":"2024-04-25T10:02:03.402599+0000","last_fullsized":"2024-04-25T10:02:03.402599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:29.264217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404104+0000","last_change":"2024-04-25T10:02:02.362032+0000","last_active":"2024-04-25T10:02:03.404104+0000","last_peered":"2024-04-25T10:02:03.404104+0000","last_clean":"2024-04-25T10:02:03.404104+0000","last_became_active":"2024-04-25T10:02:02.361608+0000","last_became_peered":"2024-04-25T10:02:02.361608+0000","last_unstale":"2024-04-25T10:02:03.404104+0000","last_undegraded":"2024-04-25T10:02:03.404104+0000","last_fullsized":"2024-04-25T10:02:03.404104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:27.993920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353535+0000","last_change":"2024-04-25T10:01:58.116451+0000","last_active":"2024-04-25T10:02:03.353535+0000","last_peered":"2024-04-25T10:02:03.353535+0000","last_clean":"2024-04-25T10:02:03.353535+0000","last_became_active":"2024-04-25T10:01:58.116177+0000","last_became_peered":"2024-04-25T10:01:58.116177+0000","last_unstale":"2024-04-25T10:02:03.353535+0000","last_undegraded":"2024-04-25T10:02:03.353535+0000","last_fullsized":"2024-04-25T10:02:03.353535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518845+0000","last_change":"2024-04-25T10:01:57.062452+0000","last_active":"2024-04-25T10:02:03.518845+0000","last_peered":"2024-04-25T10:02:03.518845+0000","last_clean":"2024-04-25T10:02:03.518845+0000","last_became_active":"2024-04-25T10:01:57.062105+0000","last_became_peered":"2024-04-25T10:01:57.062105+0000","last_unstale":"2024-04-25T10:02:03.518845+0000","last_undegraded":"2024-04-25T10:02:03.518845+0000","last_fullsized":"2024-04-25T10:02:03.518845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518790+0000","last_change":"2024-04-25T10:02:02.369531+0000","last_active":"2024-04-25T10:02:03.518790+0000","last_peered":"2024-04-25T10:02:03.518790+0000","last_clean":"2024-04-25T10:02:03.518790+0000","last_became_active":"2024-04-25T10:02:02.367651+0000","last_became_peered":"2024-04-25T10:02:02.367651+0000","last_unstale":"2024-04-25T10:02:03.518790+0000","last_undegraded":"2024-04-25T10:02:03.518790+0000","last_fullsized":"2024-04-25T10:02:03.518790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:58.140005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352233+0000","last_change":"2024-04-25T10:01:58.117205+0000","last_active":"2024-04-25T10:02:03.352233+0000","last_peered":"2024-04-25T10:02:03.352233+0000","last_clean":"2024-04-25T10:02:03.352233+0000","last_became_active":"2024-04-25T10:01:58.116873+0000","last_became_peered":"2024-04-25T10:01:58.116873+0000","last_unstale":"2024-04-25T10:02:03.352233+0000","last_undegraded":"2024-04-25T10:02:03.352233+0000","last_fullsized":"2024-04-25T10:02:03.352233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:25.305583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354051+0000","last_change":"2024-04-25T10:01:57.049606+0000","last_active":"2024-04-25T10:02:03.354051+0000","last_peered":"2024-04-25T10:02:03.354051+0000","last_clean":"2024-04-25T10:02:03.354051+0000","last_became_active":"2024-04-25T10:01:57.049300+0000","last_became_peered":"2024-04-25T10:01:57.049300+0000","last_unstale":"2024-04-25T10:02:03.354051+0000","last_undegraded":"2024-04-25T10:02:03.354051+0000","last_fullsized":"2024-04-25T10:02:03.354051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:29.284211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354667+0000","last_change":"2024-04-25T10:02:02.366098+0000","last_active":"2024-04-25T10:02:03.354667+0000","last_peered":"2024-04-25T10:02:03.354667+0000","last_clean":"2024-04-25T10:02:03.354667+0000","last_became_active":"2024-04-25T10:02:02.365669+0000","last_became_peered":"2024-04-25T10:02:02.365669+0000","last_unstale":"2024-04-25T10:02:03.354667+0000","last_undegraded":"2024-04-25T10:02:03.354667+0000","last_fullsized":"2024-04-25T10:02:03.354667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:34.132211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347971+0000","last_change":"2024-04-25T10:01:58.116653+0000","last_active":"2024-04-25T10:02:03.347971+0000","last_peered":"2024-04-25T10:02:03.347971+0000","last_clean":"2024-04-25T10:02:03.347971+0000","last_became_active":"2024-04-25T10:01:58.116414+0000","last_became_peered":"2024-04-25T10:01:58.116414+0000","last_unstale":"2024-04-25T10:02:03.347971+0000","last_undegraded":"2024-04-25T10:02:03.347971+0000","last_fullsized":"2024-04-25T10:02:03.347971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:20.873677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.517442+0000","last_change":"2024-04-25T10:01:57.042210+0000","last_active":"2024-04-25T10:02:03.517442+0000","last_peered":"2024-04-25T10:02:03.517442+0000","last_clean":"2024-04-25T10:02:03.517442+0000","last_became_active":"2024-04-25T10:01:57.041890+0000","last_became_peered":"2024-04-25T10:01:57.041890+0000","last_unstale":"2024-04-25T10:02:03.517442+0000","last_undegraded":"2024-04-25T10:02:03.517442+0000","last_fullsized":"2024-04-25T10:02:03.517442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521682+0000","last_change":"2024-04-25T10:02:02.364078+0000","last_active":"2024-04-25T10:02:03.521682+0000","last_peered":"2024-04-25T10:02:03.521682+0000","last_clean":"2024-04-25T10:02:03.521682+0000","last_became_active":"2024-04-25T10:02:02.358978+0000","last_became_peered":"2024-04-25T10:02:02.358978+0000","last_unstale":"2024-04-25T10:02:03.521682+0000","last_undegraded":"2024-04-25T10:02:03.521682+0000","last_fullsized":"2024-04-25T10:02:03.521682+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355780+0000","last_change":"2024-04-25T10:01:58.118220+0000","last_active":"2024-04-25T10:02:03.355780+0000","last_peered":"2024-04-25T10:02:03.355780+0000","last_clean":"2024-04-25T10:02:03.355780+0000","last_became_active":"2024-04-25T10:01:58.117911+0000","last_became_peered":"2024-04-25T10:01:58.117911+0000","last_unstale":"2024-04-25T10:02:03.355780+0000","last_undegraded":"2024-04-25T10:02:03.355780+0000","last_fullsized":"2024-04-25T10:02:03.355780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:27.829945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405295+0000","last_change":"2024-04-25T10:01:57.064576+0000","last_active":"2024-04-25T10:02:03.405295+0000","last_peered":"2024-04-25T10:02:03.405295+0000","last_clean":"2024-04-25T10:02:03.405295+0000","last_became_active":"2024-04-25T10:01:57.064301+0000","last_became_peered":"2024-04-25T10:01:57.064301+0000","last_unstale":"2024-04-25T10:02:03.405295+0000","last_undegraded":"2024-04-25T10:02:03.405295+0000","last_fullsized":"2024-04-25T10:02:03.405295+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:53.258589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521777+0000","last_change":"2024-04-25T10:02:02.362701+0000","last_active":"2024-04-25T10:02:03.521777+0000","last_peered":"2024-04-25T10:02:03.521777+0000","last_clean":"2024-04-25T10:02:03.521777+0000","last_became_active":"2024-04-25T10:02:02.361742+0000","last_became_peered":"2024-04-25T10:02:02.361742+0000","last_unstale":"2024-04-25T10:02:03.521777+0000","last_undegraded":"2024-04-25T10:02:03.521777+0000","last_fullsized":"2024-04-25T10:02:03.521777+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356323+0000","last_change":"2024-04-25T10:01:58.121271+0000","last_active":"2024-04-25T10:02:03.356323+0000","last_peered":"2024-04-25T10:02:03.356323+0000","last_clean":"2024-04-25T10:02:03.356323+0000","last_became_active":"2024-04-25T10:01:58.121001+0000","last_became_peered":"2024-04-25T10:01:58.121001+0000","last_unstale":"2024-04-25T10:02:03.356323+0000","last_undegraded":"2024-04-25T10:02:03.356323+0000","last_fullsized":"2024-04-25T10:02:03.356323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:56.894928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356241+0000","last_change":"2024-04-25T10:01:57.047184+0000","last_active":"2024-04-25T10:02:03.356241+0000","last_peered":"2024-04-25T10:02:03.356241+0000","last_clean":"2024-04-25T10:02:03.356241+0000","last_became_active":"2024-04-25T10:01:57.046623+0000","last_became_peered":"2024-04-25T10:01:57.046623+0000","last_unstale":"2024-04-25T10:02:03.356241+0000","last_undegraded":"2024-04-25T10:02:03.356241+0000","last_fullsized":"2024-04-25T10:02:03.356241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:31.269009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353577+0000","last_change":"2024-04-25T10:02:02.370654+0000","last_active":"2024-04-25T10:02:03.353577+0000","last_peered":"2024-04-25T10:02:03.353577+0000","last_clean":"2024-04-25T10:02:03.353577+0000","last_became_active":"2024-04-25T10:02:02.369464+0000","last_became_peered":"2024-04-25T10:02:02.369464+0000","last_unstale":"2024-04-25T10:02:03.353577+0000","last_undegraded":"2024-04-25T10:02:03.353577+0000","last_fullsized":"2024-04-25T10:02:03.353577+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:47.460473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519808+0000","last_change":"2024-04-25T10:01:58.123583+0000","last_active":"2024-04-25T10:02:03.519808+0000","last_peered":"2024-04-25T10:02:03.519808+0000","last_clean":"2024-04-25T10:02:03.519808+0000","last_became_active":"2024-04-25T10:01:58.122380+0000","last_became_peered":"2024-04-25T10:01:58.122380+0000","last_unstale":"2024-04-25T10:02:03.519808+0000","last_undegraded":"2024-04-25T10:02:03.519808+0000","last_fullsized":"2024-04-25T10:02:03.519808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:54.007484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405069+0000","last_change":"2024-04-25T10:01:57.060061+0000","last_active":"2024-04-25T10:02:03.405069+0000","last_peered":"2024-04-25T10:02:03.405069+0000","last_clean":"2024-04-25T10:02:03.405069+0000","last_became_active":"2024-04-25T10:01:57.056768+0000","last_became_peered":"2024-04-25T10:01:57.056768+0000","last_unstale":"2024-04-25T10:02:03.405069+0000","last_undegraded":"2024-04-25T10:02:03.405069+0000","last_fullsized":"2024-04-25T10:02:03.405069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:23.943464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518947+0000","last_change":"2024-04-25T10:02:02.366286+0000","last_active":"2024-04-25T10:02:03.518947+0000","last_peered":"2024-04-25T10:02:03.518947+0000","last_clean":"2024-04-25T10:02:03.518947+0000","last_became_active":"2024-04-25T10:02:02.365877+0000","last_became_peered":"2024-04-25T10:02:02.365877+0000","last_unstale":"2024-04-25T10:02:03.518947+0000","last_undegraded":"2024-04-25T10:02:03.518947+0000","last_fullsized":"2024-04-25T10:02:03.518947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:05.307980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521404+0000","last_change":"2024-04-25T10:01:58.126543+0000","last_active":"2024-04-25T10:02:03.521404+0000","last_peered":"2024-04-25T10:02:03.521404+0000","last_clean":"2024-04-25T10:02:03.521404+0000","last_became_active":"2024-04-25T10:01:58.126245+0000","last_became_peered":"2024-04-25T10:01:58.126245+0000","last_unstale":"2024-04-25T10:02:03.521404+0000","last_undegraded":"2024-04-25T10:02:03.521404+0000","last_fullsized":"2024-04-25T10:02:03.521404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433241+0000","last_change":"2024-04-25T10:01:57.056845+0000","last_active":"2024-04-25T10:02:03.433241+0000","last_peered":"2024-04-25T10:02:03.433241+0000","last_clean":"2024-04-25T10:02:03.433241+0000","last_became_active":"2024-04-25T10:01:57.056550+0000","last_became_peered":"2024-04-25T10:01:57.056550+0000","last_unstale":"2024-04-25T10:02:03.433241+0000","last_undegraded":"2024-04-25T10:02:03.433241+0000","last_fullsized":"2024-04-25T10:02:03.433241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518500+0000","last_change":"2024-04-25T10:02:02.360392+0000","last_active":"2024-04-25T10:02:03.518500+0000","last_peered":"2024-04-25T10:02:03.518500+0000","last_clean":"2024-04-25T10:02:03.518500+0000","last_became_active":"2024-04-25T10:02:02.359550+0000","last_became_peered":"2024-04-25T10:02:02.359550+0000","last_unstale":"2024-04-25T10:02:03.518500+0000","last_undegraded":"2024-04-25T10:02:03.518500+0000","last_fullsized":"2024-04-25T10:02:03.518500+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:52.460860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518461+0000","last_change":"2024-04-25T10:01:58.070356+0000","last_active":"2024-04-25T10:02:03.518461+0000","last_peered":"2024-04-25T10:02:03.518461+0000","last_clean":"2024-04-25T10:02:03.518461+0000","last_became_active":"2024-04-25T10:01:58.068594+0000","last_became_peered":"2024-04-25T10:01:58.068594+0000","last_unstale":"2024-04-25T10:02:03.518461+0000","last_undegraded":"2024-04-25T10:02:03.518461+0000","last_fullsized":"2024-04-25T10:02:03.518461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402901+0000","last_change":"2024-04-25T10:01:57.061869+0000","last_active":"2024-04-25T10:02:03.402901+0000","last_peered":"2024-04-25T10:02:03.402901+0000","last_clean":"2024-04-25T10:02:03.402901+0000","last_became_active":"2024-04-25T10:01:57.059517+0000","last_became_peered":"2024-04-25T10:01:57.059517+0000","last_unstale":"2024-04-25T10:02:03.402901+0000","last_undegraded":"2024-04-25T10:02:03.402901+0000","last_fullsized":"2024-04-25T10:02:03.402901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:49.036030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519650+0000","last_change":"2024-04-25T10:02:02.364712+0000","last_active":"2024-04-25T10:02:03.519650+0000","last_peered":"2024-04-25T10:02:03.519650+0000","last_clean":"2024-04-25T10:02:03.519650+0000","last_became_active":"2024-04-25T10:02:02.363483+0000","last_became_peered":"2024-04-25T10:02:02.363483+0000","last_unstale":"2024-04-25T10:02:03.519650+0000","last_undegraded":"2024-04-25T10:02:03.519650+0000","last_fullsized":"2024-04-25T10:02:03.519650+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432126+0000","last_change":"2024-04-25T10:01:58.124129+0000","last_active":"2024-04-25T10:02:03.432126+0000","last_peered":"2024-04-25T10:02:03.432126+0000","last_clean":"2024-04-25T10:02:03.432126+0000","last_became_active":"2024-04-25T10:01:58.123815+0000","last_became_peered":"2024-04-25T10:01:58.123815+0000","last_unstale":"2024-04-25T10:02:03.432126+0000","last_undegraded":"2024-04-25T10:02:03.432126+0000","last_fullsized":"2024-04-25T10:02:03.432126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352979+0000","last_change":"2024-04-25T10:01:57.058063+0000","last_active":"2024-04-25T10:02:03.352979+0000","last_peered":"2024-04-25T10:02:03.352979+0000","last_clean":"2024-04-25T10:02:03.352979+0000","last_became_active":"2024-04-25T10:01:57.057727+0000","last_became_peered":"2024-04-25T10:01:57.057727+0000","last_unstale":"2024-04-25T10:02:03.352979+0000","last_undegraded":"2024-04-25T10:02:03.352979+0000","last_fullsized":"2024-04-25T10:02:03.352979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:56.302921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520229+0000","last_change":"2024-04-25T10:02:02.366966+0000","last_active":"2024-04-25T10:02:03.520229+0000","last_peered":"2024-04-25T10:02:03.520229+0000","last_clean":"2024-04-25T10:02:03.520229+0000","last_became_active":"2024-04-25T10:02:02.366563+0000","last_became_peered":"2024-04-25T10:02:02.366563+0000","last_unstale":"2024-04-25T10:02:03.520229+0000","last_undegraded":"2024-04-25T10:02:03.520229+0000","last_fullsized":"2024-04-25T10:02:03.520229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:50.524934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518446+0000","last_change":"2024-04-25T10:01:58.124316+0000","last_active":"2024-04-25T10:02:03.518446+0000","last_peered":"2024-04-25T10:02:03.518446+0000","last_clean":"2024-04-25T10:02:03.518446+0000","last_became_active":"2024-04-25T10:01:58.123995+0000","last_became_peered":"2024-04-25T10:01:58.123995+0000","last_unstale":"2024-04-25T10:02:03.518446+0000","last_undegraded":"2024-04-25T10:02:03.518446+0000","last_fullsized":"2024-04-25T10:02:03.518446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:50.050265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348251+0000","last_change":"2024-04-25T10:01:57.064900+0000","last_active":"2024-04-25T10:02:03.348251+0000","last_peered":"2024-04-25T10:02:03.348251+0000","last_clean":"2024-04-25T10:02:03.348251+0000","last_became_active":"2024-04-25T10:01:57.063565+0000","last_became_peered":"2024-04-25T10:01:57.063565+0000","last_unstale":"2024-04-25T10:02:03.348251+0000","last_undegraded":"2024-04-25T10:02:03.348251+0000","last_fullsized":"2024-04-25T10:02:03.348251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433722+0000","last_change":"2024-04-25T10:02:02.363528+0000","last_active":"2024-04-25T10:02:03.433722+0000","last_peered":"2024-04-25T10:02:03.433722+0000","last_clean":"2024-04-25T10:02:03.433722+0000","last_became_active":"2024-04-25T10:02:02.360337+0000","last_became_peered":"2024-04-25T10:02:02.360337+0000","last_unstale":"2024-04-25T10:02:03.433722+0000","last_undegraded":"2024-04-25T10:02:03.433722+0000","last_fullsized":"2024-04-25T10:02:03.433722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:25.172463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354986+0000","last_change":"2024-04-25T10:01:58.114954+0000","last_active":"2024-04-25T10:02:03.354986+0000","last_peered":"2024-04-25T10:02:03.354986+0000","last_clean":"2024-04-25T10:02:03.354986+0000","last_became_active":"2024-04-25T10:01:58.114652+0000","last_became_peered":"2024-04-25T10:01:58.114652+0000","last_unstale":"2024-04-25T10:02:03.354986+0000","last_undegraded":"2024-04-25T10:02:03.354986+0000","last_fullsized":"2024-04-25T10:02:03.354986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.396332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402951+0000","last_change":"2024-04-25T10:01:57.060078+0000","last_active":"2024-04-25T10:02:03.402951+0000","last_peered":"2024-04-25T10:02:03.402951+0000","last_clean":"2024-04-25T10:02:03.402951+0000","last_became_active":"2024-04-25T10:01:57.059811+0000","last_became_peered":"2024-04-25T10:01:57.059811+0000","last_unstale":"2024-04-25T10:02:03.402951+0000","last_undegraded":"2024-04-25T10:02:03.402951+0000","last_fullsized":"2024-04-25T10:02:03.402951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:16.018629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520801+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:02:03.520801+0000","last_peered":"2024-04-25T10:02:03.520801+0000","last_clean":"2024-04-25T10:02:03.520801+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:02:03.520801+0000","last_undegraded":"2024-04-25T10:02:03.520801+0000","last_fullsized":"2024-04-25T10:02:03.520801+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404900+0000","last_change":"2024-04-25T10:02:02.358810+0000","last_active":"2024-04-25T10:02:03.404900+0000","last_peered":"2024-04-25T10:02:03.404900+0000","last_clean":"2024-04-25T10:02:03.404900+0000","last_became_active":"2024-04-25T10:02:02.358270+0000","last_became_peered":"2024-04-25T10:02:02.358270+0000","last_unstale":"2024-04-25T10:02:03.404900+0000","last_undegraded":"2024-04-25T10:02:03.404900+0000","last_fullsized":"2024-04-25T10:02:03.404900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:00.819451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404528+0000","last_change":"2024-04-25T10:01:58.073772+0000","last_active":"2024-04-25T10:02:03.404528+0000","last_peered":"2024-04-25T10:02:03.404528+0000","last_clean":"2024-04-25T10:02:03.404528+0000","last_became_active":"2024-04-25T10:01:58.073104+0000","last_became_peered":"2024-04-25T10:01:58.073104+0000","last_unstale":"2024-04-25T10:02:03.404528+0000","last_undegraded":"2024-04-25T10:02:03.404528+0000","last_fullsized":"2024-04-25T10:02:03.404528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.239771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520009+0000","last_change":"2024-04-25T10:01:57.054884+0000","last_active":"2024-04-25T10:02:03.520009+0000","last_peered":"2024-04-25T10:02:03.520009+0000","last_clean":"2024-04-25T10:02:03.520009+0000","last_became_active":"2024-04-25T10:01:57.054577+0000","last_became_peered":"2024-04-25T10:01:57.054577+0000","last_unstale":"2024-04-25T10:02:03.520009+0000","last_undegraded":"2024-04-25T10:02:03.520009+0000","last_fullsized":"2024-04-25T10:02:03.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521556+0000","last_change":"2024-04-25T10:02:02.364334+0000","last_active":"2024-04-25T10:02:03.521556+0000","last_peered":"2024-04-25T10:02:03.521556+0000","last_clean":"2024-04-25T10:02:03.521556+0000","last_became_active":"2024-04-25T10:02:02.363695+0000","last_became_peered":"2024-04-25T10:02:02.363695+0000","last_unstale":"2024-04-25T10:02:03.521556+0000","last_undegraded":"2024-04-25T10:02:03.521556+0000","last_fullsized":"2024-04-25T10:02:03.521556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:30.903672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519516+0000","last_change":"2024-04-25T10:01:58.090256+0000","last_active":"2024-04-25T10:02:03.519516+0000","last_peered":"2024-04-25T10:02:03.519516+0000","last_clean":"2024-04-25T10:02:03.519516+0000","last_became_active":"2024-04-25T10:01:58.089936+0000","last_became_peered":"2024-04-25T10:01:58.089936+0000","last_unstale":"2024-04-25T10:02:03.519516+0000","last_undegraded":"2024-04-25T10:02:03.519516+0000","last_fullsized":"2024-04-25T10:02:03.519516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:48.162074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433744+0000","last_change":"2024-04-25T10:01:57.060690+0000","last_active":"2024-04-25T10:02:03.433744+0000","last_peered":"2024-04-25T10:02:03.433744+0000","last_clean":"2024-04-25T10:02:03.433744+0000","last_became_active":"2024-04-25T10:01:57.060345+0000","last_became_peered":"2024-04-25T10:01:57.060345+0000","last_unstale":"2024-04-25T10:02:03.433744+0000","last_undegraded":"2024-04-25T10:02:03.433744+0000","last_fullsized":"2024-04-25T10:02:03.433744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433246+0000","last_change":"2024-04-25T10:02:02.371169+0000","last_active":"2024-04-25T10:02:03.433246+0000","last_peered":"2024-04-25T10:02:03.433246+0000","last_clean":"2024-04-25T10:02:03.433246+0000","last_became_active":"2024-04-25T10:02:02.370789+0000","last_became_peered":"2024-04-25T10:02:02.370789+0000","last_unstale":"2024-04-25T10:02:03.433246+0000","last_undegraded":"2024-04-25T10:02:03.433246+0000","last_fullsized":"2024-04-25T10:02:03.433246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:40.538557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520420+0000","last_change":"2024-04-25T10:01:58.072580+0000","last_active":"2024-04-25T10:02:03.520420+0000","last_peered":"2024-04-25T10:02:03.520420+0000","last_clean":"2024-04-25T10:02:03.520420+0000","last_became_active":"2024-04-25T10:01:58.072277+0000","last_became_peered":"2024-04-25T10:01:58.072277+0000","last_unstale":"2024-04-25T10:02:03.520420+0000","last_undegraded":"2024-04-25T10:02:03.520420+0000","last_fullsized":"2024-04-25T10:02:03.520420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.846454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356066+0000","last_change":"2024-04-25T10:01:57.057184+0000","last_active":"2024-04-25T10:02:03.356066+0000","last_peered":"2024-04-25T10:02:03.356066+0000","last_clean":"2024-04-25T10:02:03.356066+0000","last_became_active":"2024-04-25T10:01:57.054573+0000","last_became_peered":"2024-04-25T10:01:57.054573+0000","last_unstale":"2024-04-25T10:02:03.356066+0000","last_undegraded":"2024-04-25T10:02:03.356066+0000","last_fullsized":"2024-04-25T10:02:03.356066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:33.311253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347440+0000","last_change":"2024-04-25T10:02:02.373007+0000","last_active":"2024-04-25T10:02:03.347440+0000","last_peered":"2024-04-25T10:02:03.347440+0000","last_clean":"2024-04-25T10:02:03.347440+0000","last_became_active":"2024-04-25T10:02:02.372467+0000","last_became_peered":"2024-04-25T10:02:02.372467+0000","last_unstale":"2024-04-25T10:02:03.347440+0000","last_undegraded":"2024-04-25T10:02:03.347440+0000","last_fullsized":"2024-04-25T10:02:03.347440+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:18.003128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521943+0000","last_change":"2024-04-25T10:01:58.071062+0000","last_active":"2024-04-25T10:02:03.521943+0000","last_peered":"2024-04-25T10:02:03.521943+0000","last_clean":"2024-04-25T10:02:03.521943+0000","last_became_active":"2024-04-25T10:01:58.070721+0000","last_became_peered":"2024-04-25T10:01:58.070721+0000","last_unstale":"2024-04-25T10:02:03.521943+0000","last_undegraded":"2024-04-25T10:02:03.521943+0000","last_fullsized":"2024-04-25T10:02:03.521943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520558+0000","last_change":"2024-04-25T10:01:57.051450+0000","last_active":"2024-04-25T10:02:03.520558+0000","last_peered":"2024-04-25T10:02:03.520558+0000","last_clean":"2024-04-25T10:02:03.520558+0000","last_became_active":"2024-04-25T10:01:57.051104+0000","last_became_peered":"2024-04-25T10:01:57.051104+0000","last_unstale":"2024-04-25T10:02:03.520558+0000","last_undegraded":"2024-04-25T10:02:03.520558+0000","last_fullsized":"2024-04-25T10:02:03.520558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521721+0000","last_change":"2024-04-25T10:02:02.371242+0000","last_active":"2024-04-25T10:02:03.521721+0000","last_peered":"2024-04-25T10:02:03.521721+0000","last_clean":"2024-04-25T10:02:03.521721+0000","last_became_active":"2024-04-25T10:02:02.370975+0000","last_became_peered":"2024-04-25T10:02:02.370975+0000","last_unstale":"2024-04-25T10:02:03.521721+0000","last_undegraded":"2024-04-25T10:02:03.521721+0000","last_fullsized":"2024-04-25T10:02:03.521721+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:56.622899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347953+0000","last_change":"2024-04-25T10:01:58.076461+0000","last_active":"2024-04-25T10:02:03.347953+0000","last_peered":"2024-04-25T10:02:03.347953+0000","last_clean":"2024-04-25T10:02:03.347953+0000","last_became_active":"2024-04-25T10:01:58.075822+0000","last_became_peered":"2024-04-25T10:01:58.075822+0000","last_unstale":"2024-04-25T10:02:03.347953+0000","last_undegraded":"2024-04-25T10:02:03.347953+0000","last_fullsized":"2024-04-25T10:02:03.347953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352738+0000","last_change":"2024-04-25T10:01:57.051484+0000","last_active":"2024-04-25T10:02:03.352738+0000","last_peered":"2024-04-25T10:02:03.352738+0000","last_clean":"2024-04-25T10:02:03.352738+0000","last_became_active":"2024-04-25T10:01:57.051183+0000","last_became_peered":"2024-04-25T10:01:57.051183+0000","last_unstale":"2024-04-25T10:02:03.352738+0000","last_undegraded":"2024-04-25T10:02:03.352738+0000","last_fullsized":"2024-04-25T10:02:03.352738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:48.627288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433983+0000","last_change":"2024-04-25T10:02:02.364464+0000","last_active":"2024-04-25T10:02:03.433983+0000","last_peered":"2024-04-25T10:02:03.433983+0000","last_clean":"2024-04-25T10:02:03.433983+0000","last_became_active":"2024-04-25T10:02:02.364024+0000","last_became_peered":"2024-04-25T10:02:02.364024+0000","last_unstale":"2024-04-25T10:02:03.433983+0000","last_undegraded":"2024-04-25T10:02:03.433983+0000","last_fullsized":"2024-04-25T10:02:03.433983+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:39.357721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520680+0000","last_change":"2024-04-25T10:01:58.125488+0000","last_active":"2024-04-25T10:02:03.520680+0000","last_peered":"2024-04-25T10:02:03.520680+0000","last_clean":"2024-04-25T10:02:03.520680+0000","last_became_active":"2024-04-25T10:01:58.125161+0000","last_became_peered":"2024-04-25T10:01:58.125161+0000","last_unstale":"2024-04-25T10:02:03.520680+0000","last_undegraded":"2024-04-25T10:02:03.520680+0000","last_fullsized":"2024-04-25T10:02:03.520680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:10.940348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518416+0000","last_change":"2024-04-25T10:01:57.066991+0000","last_active":"2024-04-25T10:02:03.518416+0000","last_peered":"2024-04-25T10:02:03.518416+0000","last_clean":"2024-04-25T10:02:03.518416+0000","last_became_active":"2024-04-25T10:01:57.065456+0000","last_became_peered":"2024-04-25T10:01:57.065456+0000","last_unstale":"2024-04-25T10:02:03.518416+0000","last_undegraded":"2024-04-25T10:02:03.518416+0000","last_fullsized":"2024-04-25T10:02:03.518416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:09.926112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434030+0000","last_change":"2024-04-25T10:02:02.373840+0000","last_active":"2024-04-25T10:02:03.434030+0000","last_peered":"2024-04-25T10:02:03.434030+0000","last_clean":"2024-04-25T10:02:03.434030+0000","last_became_active":"2024-04-25T10:02:02.373426+0000","last_became_peered":"2024-04-25T10:02:02.373426+0000","last_unstale":"2024-04-25T10:02:03.434030+0000","last_undegraded":"2024-04-25T10:02:03.434030+0000","last_fullsized":"2024-04-25T10:02:03.434030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:54.471032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.348267+0000","last_change":"2024-04-25T10:01:58.076731+0000","last_active":"2024-04-25T10:02:03.348267+0000","last_peered":"2024-04-25T10:02:03.348267+0000","last_clean":"2024-04-25T10:02:03.348267+0000","last_became_active":"2024-04-25T10:01:58.076430+0000","last_became_peered":"2024-04-25T10:01:58.076430+0000","last_unstale":"2024-04-25T10:02:03.348267+0000","last_undegraded":"2024-04-25T10:02:03.348267+0000","last_fullsized":"2024-04-25T10:02:03.348267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.058934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402642+0000","last_change":"2024-04-25T10:01:57.061549+0000","last_active":"2024-04-25T10:02:03.402642+0000","last_peered":"2024-04-25T10:02:03.402642+0000","last_clean":"2024-04-25T10:02:03.402642+0000","last_became_active":"2024-04-25T10:01:57.056797+0000","last_became_peered":"2024-04-25T10:01:57.056797+0000","last_unstale":"2024-04-25T10:02:03.402642+0000","last_undegraded":"2024-04-25T10:02:03.402642+0000","last_fullsized":"2024-04-25T10:02:03.402642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:22.994799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519398+0000","last_change":"2024-04-25T10:02:02.372845+0000","last_active":"2024-04-25T10:02:03.519398+0000","last_peered":"2024-04-25T10:02:03.519398+0000","last_clean":"2024-04-25T10:02:03.519398+0000","last_became_active":"2024-04-25T10:02:02.370004+0000","last_became_peered":"2024-04-25T10:02:02.370004+0000","last_unstale":"2024-04-25T10:02:03.519398+0000","last_undegraded":"2024-04-25T10:02:03.519398+0000","last_fullsized":"2024-04-25T10:02:03.519398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:32.967789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518640+0000","last_change":"2024-04-25T10:01:58.120136+0000","last_active":"2024-04-25T10:02:03.518640+0000","last_peered":"2024-04-25T10:02:03.518640+0000","last_clean":"2024-04-25T10:02:03.518640+0000","last_became_active":"2024-04-25T10:01:58.111232+0000","last_became_peered":"2024-04-25T10:01:58.111232+0000","last_unstale":"2024-04-25T10:02:03.518640+0000","last_undegraded":"2024-04-25T10:02:03.518640+0000","last_fullsized":"2024-04-25T10:02:03.518640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:17.032865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353998+0000","last_change":"2024-04-25T10:01:57.047615+0000","last_active":"2024-04-25T10:02:03.353998+0000","last_peered":"2024-04-25T10:02:03.353998+0000","last_clean":"2024-04-25T10:02:03.353998+0000","last_became_active":"2024-04-25T10:01:57.047321+0000","last_became_peered":"2024-04-25T10:01:57.047321+0000","last_unstale":"2024-04-25T10:02:03.353998+0000","last_undegraded":"2024-04-25T10:02:03.353998+0000","last_fullsized":"2024-04-25T10:02:03.353998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404368+0000","last_change":"2024-04-25T10:02:02.361172+0000","last_active":"2024-04-25T10:02:03.404368+0000","last_peered":"2024-04-25T10:02:03.404368+0000","last_clean":"2024-04-25T10:02:03.404368+0000","last_became_active":"2024-04-25T10:02:02.360349+0000","last_became_peered":"2024-04-25T10:02:02.360349+0000","last_unstale":"2024-04-25T10:02:03.404368+0000","last_undegraded":"2024-04-25T10:02:03.404368+0000","last_fullsized":"2024-04-25T10:02:03.404368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:13.788023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518563+0000","last_change":"2024-04-25T10:01:58.127068+0000","last_active":"2024-04-25T10:02:03.518563+0000","last_peered":"2024-04-25T10:02:03.518563+0000","last_clean":"2024-04-25T10:02:03.518563+0000","last_became_active":"2024-04-25T10:01:58.126258+0000","last_became_peered":"2024-04-25T10:01:58.126258+0000","last_unstale":"2024-04-25T10:02:03.518563+0000","last_undegraded":"2024-04-25T10:02:03.518563+0000","last_fullsized":"2024-04-25T10:02:03.518563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:15.775667+0000","last_change":"2024-04-25T10:01:57.054259+0000","last_active":"2024-04-25T10:02:15.775667+0000","last_peered":"2024-04-25T10:02:15.775667+0000","last_clean":"2024-04-25T10:02:15.775667+0000","last_became_active":"2024-04-25T10:01:57.053941+0000","last_became_peered":"2024-04-25T10:01:57.053941+0000","last_unstale":"2024-04-25T10:02:15.775667+0000","last_undegraded":"2024-04-25T10:02:15.775667+0000","last_fullsized":"2024-04-25T10:02:15.775667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:41.210591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354900+0000","last_change":"2024-04-25T10:02:02.376549+0000","last_active":"2024-04-25T10:02:03.354900+0000","last_peered":"2024-04-25T10:02:03.354900+0000","last_clean":"2024-04-25T10:02:03.354900+0000","last_became_active":"2024-04-25T10:02:02.376052+0000","last_became_peered":"2024-04-25T10:02:02.376052+0000","last_unstale":"2024-04-25T10:02:03.354900+0000","last_undegraded":"2024-04-25T10:02:03.354900+0000","last_fullsized":"2024-04-25T10:02:03.354900+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:43.653929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.403613+0000","last_change":"2024-04-25T10:01:58.078715+0000","last_active":"2024-04-25T10:02:03.403613+0000","last_peered":"2024-04-25T10:02:03.403613+0000","last_clean":"2024-04-25T10:02:03.403613+0000","last_became_active":"2024-04-25T10:01:58.078363+0000","last_became_peered":"2024-04-25T10:01:58.078363+0000","last_unstale":"2024-04-25T10:02:03.403613+0000","last_undegraded":"2024-04-25T10:02:03.403613+0000","last_fullsized":"2024-04-25T10:02:03.403613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:25.310967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.354395+0000","last_change":"2024-04-25T10:01:57.054914+0000","last_active":"2024-04-25T10:02:03.354395+0000","last_peered":"2024-04-25T10:02:03.354395+0000","last_clean":"2024-04-25T10:02:03.354395+0000","last_became_active":"2024-04-25T10:01:57.054640+0000","last_became_peered":"2024-04-25T10:01:57.054640+0000","last_unstale":"2024-04-25T10:02:03.354395+0000","last_undegraded":"2024-04-25T10:02:03.354395+0000","last_fullsized":"2024-04-25T10:02:03.354395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522376+0000","last_change":"2024-04-25T10:02:02.362949+0000","last_active":"2024-04-25T10:02:03.522376+0000","last_peered":"2024-04-25T10:02:03.522376+0000","last_clean":"2024-04-25T10:02:03.522376+0000","last_became_active":"2024-04-25T10:02:02.361660+0000","last_became_peered":"2024-04-25T10:02:02.361660+0000","last_unstale":"2024-04-25T10:02:03.522376+0000","last_undegraded":"2024-04-25T10:02:03.522376+0000","last_fullsized":"2024-04-25T10:02:03.522376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:14.300818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.434111+0000","last_change":"2024-04-25T10:01:58.114459+0000","last_active":"2024-04-25T10:02:03.434111+0000","last_peered":"2024-04-25T10:02:03.434111+0000","last_clean":"2024-04-25T10:02:03.434111+0000","last_became_active":"2024-04-25T10:01:58.113079+0000","last_became_peered":"2024-04-25T10:01:58.113079+0000","last_unstale":"2024-04-25T10:02:03.434111+0000","last_undegraded":"2024-04-25T10:02:03.434111+0000","last_fullsized":"2024-04-25T10:02:03.434111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518456+0000","last_change":"2024-04-25T10:01:57.058749+0000","last_active":"2024-04-25T10:02:03.518456+0000","last_peered":"2024-04-25T10:02:03.518456+0000","last_clean":"2024-04-25T10:02:03.518456+0000","last_became_active":"2024-04-25T10:01:57.058498+0000","last_became_peered":"2024-04-25T10:01:57.058498+0000","last_unstale":"2024-04-25T10:02:03.518456+0000","last_undegraded":"2024-04-25T10:02:03.518456+0000","last_fullsized":"2024-04-25T10:02:03.518456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:02.458150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352717+0000","last_change":"2024-04-25T10:02:02.354682+0000","last_active":"2024-04-25T10:02:03.352717+0000","last_peered":"2024-04-25T10:02:03.352717+0000","last_clean":"2024-04-25T10:02:03.352717+0000","last_became_active":"2024-04-25T10:02:02.353840+0000","last_became_peered":"2024-04-25T10:02:02.353840+0000","last_unstale":"2024-04-25T10:02:03.352717+0000","last_undegraded":"2024-04-25T10:02:03.352717+0000","last_fullsized":"2024-04-25T10:02:03.352717+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:58.306270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521836+0000","last_change":"2024-04-25T10:01:58.075136+0000","last_active":"2024-04-25T10:02:03.521836+0000","last_peered":"2024-04-25T10:02:03.521836+0000","last_clean":"2024-04-25T10:02:03.521836+0000","last_became_active":"2024-04-25T10:01:58.074823+0000","last_became_peered":"2024-04-25T10:01:58.074823+0000","last_unstale":"2024-04-25T10:02:03.521836+0000","last_undegraded":"2024-04-25T10:02:03.521836+0000","last_fullsized":"2024-04-25T10:02:03.521836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524041+0000","last_change":"2024-04-25T10:01:57.058387+0000","last_active":"2024-04-25T10:02:03.524041+0000","last_peered":"2024-04-25T10:02:03.524041+0000","last_clean":"2024-04-25T10:02:03.524041+0000","last_became_active":"2024-04-25T10:01:57.057994+0000","last_became_peered":"2024-04-25T10:01:57.057994+0000","last_unstale":"2024-04-25T10:02:03.524041+0000","last_undegraded":"2024-04-25T10:02:03.524041+0000","last_fullsized":"2024-04-25T10:02:03.524041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.171109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353666+0000","last_change":"2024-04-25T10:02:02.374647+0000","last_active":"2024-04-25T10:02:03.353666+0000","last_peered":"2024-04-25T10:02:03.353666+0000","last_clean":"2024-04-25T10:02:03.353666+0000","last_became_active":"2024-04-25T10:02:02.374168+0000","last_became_peered":"2024-04-25T10:02:02.374168+0000","last_unstale":"2024-04-25T10:02:03.353666+0000","last_undegraded":"2024-04-25T10:02:03.353666+0000","last_fullsized":"2024-04-25T10:02:03.353666+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:13.963774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356050+0000","last_change":"2024-04-25T10:02:02.374581+0000","last_active":"2024-04-25T10:02:03.356050+0000","last_peered":"2024-04-25T10:02:03.356050+0000","last_clean":"2024-04-25T10:02:03.356050+0000","last_became_active":"2024-04-25T10:02:02.373942+0000","last_became_peered":"2024-04-25T10:02:02.373942+0000","last_unstale":"2024-04-25T10:02:03.356050+0000","last_undegraded":"2024-04-25T10:02:03.356050+0000","last_fullsized":"2024-04-25T10:02:03.356050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:30.252346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353803+0000","last_change":"2024-04-25T10:01:58.111477+0000","last_active":"2024-04-25T10:02:03.353803+0000","last_peered":"2024-04-25T10:02:03.353803+0000","last_clean":"2024-04-25T10:02:03.353803+0000","last_became_active":"2024-04-25T10:01:58.111095+0000","last_became_peered":"2024-04-25T10:01:58.111095+0000","last_unstale":"2024-04-25T10:02:03.353803+0000","last_undegraded":"2024-04-25T10:02:03.353803+0000","last_fullsized":"2024-04-25T10:02:03.353803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:42.787884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347909+0000","last_change":"2024-04-25T10:01:57.058954+0000","last_active":"2024-04-25T10:02:03.347909+0000","last_peered":"2024-04-25T10:02:03.347909+0000","last_clean":"2024-04-25T10:02:03.347909+0000","last_became_active":"2024-04-25T10:01:57.056146+0000","last_became_peered":"2024-04-25T10:01:57.056146+0000","last_unstale":"2024-04-25T10:02:03.347909+0000","last_undegraded":"2024-04-25T10:02:03.347909+0000","last_fullsized":"2024-04-25T10:02:03.347909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:05.741605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404389+0000","last_change":"2024-04-25T10:02:02.360724+0000","last_active":"2024-04-25T10:02:03.404389+0000","last_peered":"2024-04-25T10:02:03.404389+0000","last_clean":"2024-04-25T10:02:03.404389+0000","last_became_active":"2024-04-25T10:02:02.360282+0000","last_became_peered":"2024-04-25T10:02:02.360282+0000","last_unstale":"2024-04-25T10:02:03.404389+0000","last_undegraded":"2024-04-25T10:02:03.404389+0000","last_fullsized":"2024-04-25T10:02:03.404389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:48.141173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432340+0000","last_change":"2024-04-25T10:01:58.126960+0000","last_active":"2024-04-25T10:02:03.432340+0000","last_peered":"2024-04-25T10:02:03.432340+0000","last_clean":"2024-04-25T10:02:03.432340+0000","last_became_active":"2024-04-25T10:01:58.126638+0000","last_became_peered":"2024-04-25T10:01:58.126638+0000","last_unstale":"2024-04-25T10:02:03.432340+0000","last_undegraded":"2024-04-25T10:02:03.432340+0000","last_fullsized":"2024-04-25T10:02:03.432340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:01.009194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518686+0000","last_change":"2024-04-25T10:01:57.056000+0000","last_active":"2024-04-25T10:02:03.518686+0000","last_peered":"2024-04-25T10:02:03.518686+0000","last_clean":"2024-04-25T10:02:03.518686+0000","last_became_active":"2024-04-25T10:01:57.055665+0000","last_became_peered":"2024-04-25T10:01:57.055665+0000","last_unstale":"2024-04-25T10:02:03.518686+0000","last_undegraded":"2024-04-25T10:02:03.518686+0000","last_fullsized":"2024-04-25T10:02:03.518686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:21.922114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.402377+0000","last_change":"2024-04-25T10:01:57.064311+0000","last_active":"2024-04-25T10:02:03.402377+0000","last_peered":"2024-04-25T10:02:03.402377+0000","last_clean":"2024-04-25T10:02:03.402377+0000","last_became_active":"2024-04-25T10:01:57.063955+0000","last_became_peered":"2024-04-25T10:01:57.063955+0000","last_unstale":"2024-04-25T10:02:03.402377+0000","last_undegraded":"2024-04-25T10:02:03.402377+0000","last_fullsized":"2024-04-25T10:02:03.402377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:35.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.353034+0000","last_change":"2024-04-25T10:01:58.112383+0000","last_active":"2024-04-25T10:02:03.353034+0000","last_peered":"2024-04-25T10:02:03.353034+0000","last_clean":"2024-04-25T10:02:03.353034+0000","last_became_active":"2024-04-25T10:01:58.111906+0000","last_became_peered":"2024-04-25T10:01:58.111906+0000","last_unstale":"2024-04-25T10:02:03.353034+0000","last_undegraded":"2024-04-25T10:02:03.353034+0000","last_fullsized":"2024-04-25T10:02:03.353034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:58.408032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355526+0000","last_change":"2024-04-25T10:02:02.370310+0000","last_active":"2024-04-25T10:02:03.355526+0000","last_peered":"2024-04-25T10:02:03.355526+0000","last_clean":"2024-04-25T10:02:03.355526+0000","last_became_active":"2024-04-25T10:02:02.369660+0000","last_became_peered":"2024-04-25T10:02:02.369660+0000","last_unstale":"2024-04-25T10:02:03.355526+0000","last_undegraded":"2024-04-25T10:02:03.355526+0000","last_fullsized":"2024-04-25T10:02:03.355526+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:59.232453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433292+0000","last_change":"2024-04-25T10:01:57.059797+0000","last_active":"2024-04-25T10:02:03.433292+0000","last_peered":"2024-04-25T10:02:03.433292+0000","last_clean":"2024-04-25T10:02:03.433292+0000","last_became_active":"2024-04-25T10:01:57.059483+0000","last_became_peered":"2024-04-25T10:01:57.059483+0000","last_unstale":"2024-04-25T10:02:03.433292+0000","last_undegraded":"2024-04-25T10:02:03.433292+0000","last_fullsized":"2024-04-25T10:02:03.433292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:55.993638+0000","last_clean_scrub_stamp":"2024-04-25T10:01:55.993638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:03.608708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521515+0000","last_change":"2024-04-25T10:01:58.118406+0000","last_active":"2024-04-25T10:02:03.521515+0000","last_peered":"2024-04-25T10:02:03.521515+0000","last_clean":"2024-04-25T10:02:03.521515+0000","last_became_active":"2024-04-25T10:01:58.118066+0000","last_became_peered":"2024-04-25T10:01:58.118066+0000","last_unstale":"2024-04-25T10:02:03.521515+0000","last_undegraded":"2024-04-25T10:02:03.521515+0000","last_fullsized":"2024-04-25T10:02:03.521515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.356020+0000","last_change":"2024-04-25T10:02:02.360301+0000","last_active":"2024-04-25T10:02:03.356020+0000","last_peered":"2024-04-25T10:02:03.356020+0000","last_clean":"2024-04-25T10:02:03.356020+0000","last_became_active":"2024-04-25T10:02:02.359873+0000","last_became_peered":"2024-04-25T10:02:02.359873+0000","last_unstale":"2024-04-25T10:02:03.356020+0000","last_undegraded":"2024-04-25T10:02:03.356020+0000","last_fullsized":"2024-04-25T10:02:03.356020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:01.306660+0000","last_clean_scrub_stamp":"2024-04-25T10:02:01.306660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:58.571611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519243+0000","last_change":"2024-04-25T10:01:58.119106+0000","last_active":"2024-04-25T10:02:03.519243+0000","last_peered":"2024-04-25T10:02:03.519243+0000","last_clean":"2024-04-25T10:02:03.519243+0000","last_became_active":"2024-04-25T10:01:58.118822+0000","last_became_peered":"2024-04-25T10:01:58.118822+0000","last_unstale":"2024-04-25T10:02:03.519243+0000","last_undegraded":"2024-04-25T10:02:03.519243+0000","last_fullsized":"2024-04-25T10:02:03.519243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352667+0000","last_change":"2024-04-25T10:01:58.080145+0000","last_active":"2024-04-25T10:02:03.352667+0000","last_peered":"2024-04-25T10:02:03.352667+0000","last_clean":"2024-04-25T10:02:03.352667+0000","last_became_active":"2024-04-25T10:01:58.079388+0000","last_became_peered":"2024-04-25T10:01:58.079388+0000","last_unstale":"2024-04-25T10:02:03.352667+0000","last_undegraded":"2024-04-25T10:02:03.352667+0000","last_fullsized":"2024-04-25T10:02:03.352667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347995+0000","last_change":"2024-04-25T10:01:58.118742+0000","last_active":"2024-04-25T10:02:03.347995+0000","last_peered":"2024-04-25T10:02:03.347995+0000","last_clean":"2024-04-25T10:02:03.347995+0000","last_became_active":"2024-04-25T10:01:58.118407+0000","last_became_peered":"2024-04-25T10:01:58.118407+0000","last_unstale":"2024-04-25T10:02:03.347995+0000","last_undegraded":"2024-04-25T10:02:03.347995+0000","last_fullsized":"2024-04-25T10:02:03.347995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:10.311295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432945+0000","last_change":"2024-04-25T10:01:58.129321+0000","last_active":"2024-04-25T10:02:03.432945+0000","last_peered":"2024-04-25T10:02:03.432945+0000","last_clean":"2024-04-25T10:02:03.432945+0000","last_became_active":"2024-04-25T10:01:58.128642+0000","last_became_peered":"2024-04-25T10:01:58.128642+0000","last_unstale":"2024-04-25T10:02:03.432945+0000","last_undegraded":"2024-04-25T10:02:03.432945+0000","last_fullsized":"2024-04-25T10:02:03.432945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519722+0000","last_change":"2024-04-25T10:01:58.114750+0000","last_active":"2024-04-25T10:02:03.519722+0000","last_peered":"2024-04-25T10:02:03.519722+0000","last_clean":"2024-04-25T10:02:03.519722+0000","last_became_active":"2024-04-25T10:01:58.114410+0000","last_became_peered":"2024-04-25T10:01:58.114410+0000","last_unstale":"2024-04-25T10:02:03.519722+0000","last_undegraded":"2024-04-25T10:02:03.519722+0000","last_fullsized":"2024-04-25T10:02:03.519722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:27.145938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519712+0000","last_change":"2024-04-25T10:01:58.116458+0000","last_active":"2024-04-25T10:02:03.519712+0000","last_peered":"2024-04-25T10:02:03.519712+0000","last_clean":"2024-04-25T10:02:03.519712+0000","last_became_active":"2024-04-25T10:01:58.116051+0000","last_became_peered":"2024-04-25T10:01:58.116051+0000","last_unstale":"2024-04-25T10:02:03.519712+0000","last_undegraded":"2024-04-25T10:02:03.519712+0000","last_fullsized":"2024-04-25T10:02:03.519712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521818+0000","last_change":"2024-04-25T10:01:58.079855+0000","last_active":"2024-04-25T10:02:03.521818+0000","last_peered":"2024-04-25T10:02:03.521818+0000","last_clean":"2024-04-25T10:02:03.521818+0000","last_became_active":"2024-04-25T10:01:58.079522+0000","last_became_peered":"2024-04-25T10:01:58.079522+0000","last_unstale":"2024-04-25T10:02:03.521818+0000","last_undegraded":"2024-04-25T10:02:03.521818+0000","last_fullsized":"2024-04-25T10:02:03.521818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:24.009035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433705+0000","last_change":"2024-04-25T10:01:58.114808+0000","last_active":"2024-04-25T10:02:03.433705+0000","last_peered":"2024-04-25T10:02:03.433705+0000","last_clean":"2024-04-25T10:02:03.433705+0000","last_became_active":"2024-04-25T10:01:58.114656+0000","last_became_peered":"2024-04-25T10:01:58.114656+0000","last_unstale":"2024-04-25T10:02:03.433705+0000","last_undegraded":"2024-04-25T10:02:03.433705+0000","last_fullsized":"2024-04-25T10:02:03.433705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347899+0000","last_change":"2024-04-25T10:01:58.125563+0000","last_active":"2024-04-25T10:02:03.347899+0000","last_peered":"2024-04-25T10:02:03.347899+0000","last_clean":"2024-04-25T10:02:03.347899+0000","last_became_active":"2024-04-25T10:01:58.125071+0000","last_became_peered":"2024-04-25T10:01:58.125071+0000","last_unstale":"2024-04-25T10:02:03.347899+0000","last_undegraded":"2024-04-25T10:02:03.347899+0000","last_fullsized":"2024-04-25T10:02:03.347899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:28.669003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432367+0000","last_change":"2024-04-25T10:01:58.111903+0000","last_active":"2024-04-25T10:02:03.432367+0000","last_peered":"2024-04-25T10:02:03.432367+0000","last_clean":"2024-04-25T10:02:03.432367+0000","last_became_active":"2024-04-25T10:01:58.111366+0000","last_became_peered":"2024-04-25T10:01:58.111366+0000","last_unstale":"2024-04-25T10:02:03.432367+0000","last_undegraded":"2024-04-25T10:02:03.432367+0000","last_fullsized":"2024-04-25T10:02:03.432367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521208+0000","last_change":"2024-04-25T10:01:58.066982+0000","last_active":"2024-04-25T10:02:03.521208+0000","last_peered":"2024-04-25T10:02:03.521208+0000","last_clean":"2024-04-25T10:02:03.521208+0000","last_became_active":"2024-04-25T10:01:58.066582+0000","last_became_peered":"2024-04-25T10:01:58.066582+0000","last_unstale":"2024-04-25T10:02:03.521208+0000","last_undegraded":"2024-04-25T10:02:03.521208+0000","last_fullsized":"2024-04-25T10:02:03.521208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:36.574586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.352779+0000","last_change":"2024-04-25T10:01:58.112990+0000","last_active":"2024-04-25T10:02:03.352779+0000","last_peered":"2024-04-25T10:02:03.352779+0000","last_clean":"2024-04-25T10:02:03.352779+0000","last_became_active":"2024-04-25T10:01:58.112724+0000","last_became_peered":"2024-04-25T10:01:58.112724+0000","last_unstale":"2024-04-25T10:02:03.352779+0000","last_undegraded":"2024-04-25T10:02:03.352779+0000","last_fullsized":"2024-04-25T10:02:03.352779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:36.503846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519412+0000","last_change":"2024-04-25T10:01:58.067953+0000","last_active":"2024-04-25T10:02:03.519412+0000","last_peered":"2024-04-25T10:02:03.519412+0000","last_clean":"2024-04-25T10:02:03.519412+0000","last_became_active":"2024-04-25T10:01:58.067363+0000","last_became_peered":"2024-04-25T10:01:58.067363+0000","last_unstale":"2024-04-25T10:02:03.519412+0000","last_undegraded":"2024-04-25T10:02:03.519412+0000","last_fullsized":"2024-04-25T10:02:03.519412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:13.283853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.404814+0000","last_change":"2024-04-25T10:01:58.112476+0000","last_active":"2024-04-25T10:02:03.404814+0000","last_peered":"2024-04-25T10:02:03.404814+0000","last_clean":"2024-04-25T10:02:03.404814+0000","last_became_active":"2024-04-25T10:01:58.112177+0000","last_became_peered":"2024-04-25T10:01:58.112177+0000","last_unstale":"2024-04-25T10:02:03.404814+0000","last_undegraded":"2024-04-25T10:02:03.404814+0000","last_fullsized":"2024-04-25T10:02:03.404814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:50.172245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.355444+0000","last_change":"2024-04-25T10:01:58.056526+0000","last_active":"2024-04-25T10:02:03.355444+0000","last_peered":"2024-04-25T10:02:03.355444+0000","last_clean":"2024-04-25T10:02:03.355444+0000","last_became_active":"2024-04-25T10:01:58.055509+0000","last_became_peered":"2024-04-25T10:01:58.055509+0000","last_unstale":"2024-04-25T10:02:03.355444+0000","last_undegraded":"2024-04-25T10:02:03.355444+0000","last_fullsized":"2024-04-25T10:02:03.355444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.405090+0000","last_change":"2024-04-25T10:01:58.079032+0000","last_active":"2024-04-25T10:02:03.405090+0000","last_peered":"2024-04-25T10:02:03.405090+0000","last_clean":"2024-04-25T10:02:03.405090+0000","last_became_active":"2024-04-25T10:01:58.077635+0000","last_became_peered":"2024-04-25T10:01:58.077635+0000","last_unstale":"2024-04-25T10:02:03.405090+0000","last_undegraded":"2024-04-25T10:02:03.405090+0000","last_fullsized":"2024-04-25T10:02:03.405090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:00.440369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.347561+0000","last_change":"2024-04-25T10:01:58.076733+0000","last_active":"2024-04-25T10:02:03.347561+0000","last_peered":"2024-04-25T10:02:03.347561+0000","last_clean":"2024-04-25T10:02:03.347561+0000","last_became_active":"2024-04-25T10:01:58.076476+0000","last_became_peered":"2024-04-25T10:01:58.076476+0000","last_unstale":"2024-04-25T10:02:03.347561+0000","last_undegraded":"2024-04-25T10:02:03.347561+0000","last_fullsized":"2024-04-25T10:02:03.347561+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432026+0000","last_change":"2024-04-25T10:01:58.116156+0000","last_active":"2024-04-25T10:02:03.432026+0000","last_peered":"2024-04-25T10:02:03.432026+0000","last_clean":"2024-04-25T10:02:03.432026+0000","last_became_active":"2024-04-25T10:01:58.115483+0000","last_became_peered":"2024-04-25T10:01:58.115483+0000","last_unstale":"2024-04-25T10:02:03.432026+0000","last_undegraded":"2024-04-25T10:02:03.432026+0000","last_fullsized":"2024-04-25T10:02:03.432026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.272824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518554+0000","last_change":"2024-04-25T10:01:58.120123+0000","last_active":"2024-04-25T10:02:03.518554+0000","last_peered":"2024-04-25T10:02:03.518554+0000","last_clean":"2024-04-25T10:02:03.518554+0000","last_became_active":"2024-04-25T10:01:58.119852+0000","last_became_peered":"2024-04-25T10:01:58.119852+0000","last_unstale":"2024-04-25T10:02:03.518554+0000","last_undegraded":"2024-04-25T10:02:03.518554+0000","last_fullsized":"2024-04-25T10:02:03.518554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.324112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519056+0000","last_change":"2024-04-25T10:01:58.113249+0000","last_active":"2024-04-25T10:02:03.519056+0000","last_peered":"2024-04-25T10:02:03.519056+0000","last_clean":"2024-04-25T10:02:03.519056+0000","last_became_active":"2024-04-25T10:01:58.112298+0000","last_became_peered":"2024-04-25T10:01:58.112298+0000","last_unstale":"2024-04-25T10:02:03.519056+0000","last_undegraded":"2024-04-25T10:02:03.519056+0000","last_fullsized":"2024-04-25T10:02:03.519056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:29.833424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.519272+0000","last_change":"2024-04-25T10:01:58.072097+0000","last_active":"2024-04-25T10:02:03.519272+0000","last_peered":"2024-04-25T10:02:03.519272+0000","last_clean":"2024-04-25T10:02:03.519272+0000","last_became_active":"2024-04-25T10:01:58.071662+0000","last_became_peered":"2024-04-25T10:01:58.071662+0000","last_unstale":"2024-04-25T10:02:03.519272+0000","last_undegraded":"2024-04-25T10:02:03.519272+0000","last_fullsized":"2024-04-25T10:02:03.519272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518405+0000","last_change":"2024-04-25T10:01:58.073877+0000","last_active":"2024-04-25T10:02:03.518405+0000","last_peered":"2024-04-25T10:02:03.518405+0000","last_clean":"2024-04-25T10:02:03.518405+0000","last_became_active":"2024-04-25T10:01:58.073554+0000","last_became_peered":"2024-04-25T10:01:58.073554+0000","last_unstale":"2024-04-25T10:02:03.518405+0000","last_undegraded":"2024-04-25T10:02:03.518405+0000","last_fullsized":"2024-04-25T10:02:03.518405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518311+0000","last_change":"2024-04-25T10:01:58.066768+0000","last_active":"2024-04-25T10:02:03.518311+0000","last_peered":"2024-04-25T10:02:03.518311+0000","last_clean":"2024-04-25T10:02:03.518311+0000","last_became_active":"2024-04-25T10:01:58.065944+0000","last_became_peered":"2024-04-25T10:01:58.065944+0000","last_unstale":"2024-04-25T10:02:03.518311+0000","last_undegraded":"2024-04-25T10:02:03.518311+0000","last_fullsized":"2024-04-25T10:02:03.518311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:10.002031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.524004+0000","last_change":"2024-04-25T10:01:58.066685+0000","last_active":"2024-04-25T10:02:03.524004+0000","last_peered":"2024-04-25T10:02:03.524004+0000","last_clean":"2024-04-25T10:02:03.524004+0000","last_became_active":"2024-04-25T10:01:58.066367+0000","last_became_peered":"2024-04-25T10:01:58.066367+0000","last_unstale":"2024-04-25T10:02:03.524004+0000","last_undegraded":"2024-04-25T10:02:03.524004+0000","last_fullsized":"2024-04-25T10:02:03.524004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:17.229453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.432058+0000","last_change":"2024-04-25T10:01:58.123950+0000","last_active":"2024-04-25T10:02:03.432058+0000","last_peered":"2024-04-25T10:02:03.432058+0000","last_clean":"2024-04-25T10:02:03.432058+0000","last_became_active":"2024-04-25T10:01:58.123668+0000","last_became_peered":"2024-04-25T10:01:58.123668+0000","last_unstale":"2024-04-25T10:02:03.432058+0000","last_undegraded":"2024-04-25T10:02:03.432058+0000","last_fullsized":"2024-04-25T10:02:03.432058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:30.278453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518202+0000","last_change":"2024-04-25T10:01:58.114958+0000","last_active":"2024-04-25T10:02:03.518202+0000","last_peered":"2024-04-25T10:02:03.518202+0000","last_clean":"2024-04-25T10:02:03.518202+0000","last_became_active":"2024-04-25T10:01:58.114599+0000","last_became_peered":"2024-04-25T10:01:58.114599+0000","last_unstale":"2024-04-25T10:02:03.518202+0000","last_undegraded":"2024-04-25T10:02:03.518202+0000","last_fullsized":"2024-04-25T10:02:03.518202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:19.251951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.521335+0000","last_change":"2024-04-25T10:01:58.116405+0000","last_active":"2024-04-25T10:02:03.521335+0000","last_peered":"2024-04-25T10:02:03.521335+0000","last_clean":"2024-04-25T10:02:03.521335+0000","last_became_active":"2024-04-25T10:01:58.115946+0000","last_became_peered":"2024-04-25T10:01:58.115946+0000","last_unstale":"2024-04-25T10:02:03.521335+0000","last_undegraded":"2024-04-25T10:02:03.521335+0000","last_fullsized":"2024-04-25T10:02:03.521335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:57.669295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.518323+0000","last_change":"2024-04-25T10:01:58.117287+0000","last_active":"2024-04-25T10:02:03.518323+0000","last_peered":"2024-04-25T10:02:03.518323+0000","last_clean":"2024-04-25T10:02:03.518323+0000","last_became_active":"2024-04-25T10:01:58.117024+0000","last_became_peered":"2024-04-25T10:01:58.117024+0000","last_unstale":"2024-04-25T10:02:03.518323+0000","last_undegraded":"2024-04-25T10:02:03.518323+0000","last_fullsized":"2024-04-25T10:02:03.518323+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:26.604886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.522601+0000","last_change":"2024-04-25T10:01:58.116367+0000","last_active":"2024-04-25T10:02:03.522601+0000","last_peered":"2024-04-25T10:02:03.522601+0000","last_clean":"2024-04-25T10:02:03.522601+0000","last_became_active":"2024-04-25T10:01:58.115801+0000","last_became_peered":"2024-04-25T10:01:58.115801+0000","last_unstale":"2024-04-25T10:02:03.522601+0000","last_undegraded":"2024-04-25T10:02:03.522601+0000","last_fullsized":"2024-04-25T10:02:03.522601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:24.957699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433436+0000","last_change":"2024-04-25T10:01:58.118411+0000","last_active":"2024-04-25T10:02:03.433436+0000","last_peered":"2024-04-25T10:02:03.433436+0000","last_clean":"2024-04-25T10:02:03.433436+0000","last_became_active":"2024-04-25T10:01:58.118131+0000","last_became_peered":"2024-04-25T10:01:58.118131+0000","last_unstale":"2024-04-25T10:02:03.433436+0000","last_undegraded":"2024-04-25T10:02:03.433436+0000","last_fullsized":"2024-04-25T10:02:03.433436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:54.838068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.433580+0000","last_change":"2024-04-25T10:01:58.121692+0000","last_active":"2024-04-25T10:02:03.433580+0000","last_peered":"2024-04-25T10:02:03.433580+0000","last_clean":"2024-04-25T10:02:03.433580+0000","last_became_active":"2024-04-25T10:01:58.120392+0000","last_became_peered":"2024-04-25T10:01:58.120392+0000","last_unstale":"2024-04-25T10:02:03.433580+0000","last_undegraded":"2024-04-25T10:02:03.433580+0000","last_fullsized":"2024-04-25T10:02:03.433580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:54.694272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T10:02:03.520287+0000","last_change":"2024-04-25T10:01:58.080365+0000","last_active":"2024-04-25T10:02:03.520287+0000","last_peered":"2024-04-25T10:02:03.520287+0000","last_clean":"2024-04-25T10:02:03.520287+0000","last_became_active":"2024-04-25T10:01:58.080055+0000","last_became_peered":"2024-04-25T10:01:58.080055+0000","last_unstale":"2024-04-25T10:02:03.520287+0000","last_undegraded":"2024-04-25T10:02:03.520287+0000","last_fullsized":"2024-04-25T10:02:03.520287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:57.007873+0000","last_clean_scrub_stamp":"2024-04-25T10:01:57.007873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:46.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":105738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":696074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":274432,"data_stored":106196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":109456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":868352,"data_stored":696052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":105670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":110128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:32.066 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:02:32.066 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:02:32.066 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:02:32.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:02:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.302+0000 7fb8b7200640 1 Processor -- start 2024-04-25T10:02:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.303+0000 7fb8b7200640 1 -- start start 2024-04-25T10:02:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb8b806f450 0x7fb8b8070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb8b80cc560 0x7fb8b80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b806fa30 con 0x7fb8b815b780 2024-04-25T10:02:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b80ccef0 con 0x7fb8b806f450 2024-04-25T10:02:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b806fd10 con 0x7fb8b80cc560 2024-04-25T10:02:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b5e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb8b815b780 0x7fb8b809a8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51610/0 (socket says 172.21.15.5:51610) 2024-04-25T10:02:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b5e00640 1 -- 172.21.15.5:0/102507274 learned_addr learned my addr 172.21.15.5:0/102507274 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.304+0000 7fb8b5400640 1 --2- 172.21.15.5:0/102507274 >> v2:172.21.15.5:3300/0 conn(0x7fb8b806f450 0x7fb8b8070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b6800640 1 --2- 172.21.15.5:0/102507274 >> v2:172.21.15.80:3300/0 conn(0x7fb8b80cc560 0x7fb8b80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b4a00640 1 -- 172.21.15.5:0/102507274 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903145252 0 0) 0x7fb8b806fa30 con 0x7fb8b815b780 2024-04-25T10:02:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b4a00640 1 -- 172.21.15.5:0/102507274 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb89c003520 con 0x7fb8b815b780 2024-04-25T10:02:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b5400640 1 -- 172.21.15.5:0/102507274 >> v2:172.21.15.80:3300/0 conn(0x7fb8b80cc560 msgr2=0x7fb8b80cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b5400640 1 --2- 172.21.15.5:0/102507274 >> v2:172.21.15.80:3300/0 conn(0x7fb8b80cc560 0x7fb8b80cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b5400640 1 -- 172.21.15.5:0/102507274 >> v1:172.21.15.5:6789/0 conn(0x7fb8b815b780 legacy=0x7fb8b809a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b5400640 1 -- 172.21.15.5:0/102507274 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b806fff0 con 0x7fb8b806f450 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.305+0000 7fb8b6800640 1 --2- 172.21.15.5:0/102507274 >> v2:172.21.15.80:3300/0 conn(0x7fb8b80cc560 0x7fb8b80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.306+0000 7fb8b5400640 1 --2- 172.21.15.5:0/102507274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b8070ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80cdd30 tx=0x7fb8a402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a170c369611666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.306+0000 7fb8b4a00640 1 -- 172.21.15.5:0/102507274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4007740 con 0x7fb8b806f450 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.306+0000 7fb8b4a00640 1 -- 172.21.15.5:0/102507274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a40078e0 con 0x7fb8b806f450 2024-04-25T10:02:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.306+0000 7fb8b4a00640 1 -- 172.21.15.5:0/102507274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4007cb0 con 0x7fb8b806f450 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 -- 172.21.15.5:0/102507274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 msgr2=0x7fb8b8070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 --2- 172.21.15.5:0/102507274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b8070ad0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80cdd30 tx=0x7fb8a402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 -- 172.21.15.5:0/102507274 shutdown_connections 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 --2- 172.21.15.5:0/102507274 >> v2:172.21.15.80:3300/0 conn(0x7fb8b80cc560 0x7fb8b80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 --2- 172.21.15.5:0/102507274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b8070ad0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 -- 172.21.15.5:0/102507274 >> 172.21.15.5:0/102507274 conn(0x7fb8b80b2ff0 msgr2=0x7fb8b80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 -- 172.21.15.5:0/102507274 shutdown_connections 2024-04-25T10:02:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.307+0000 7fb8b7200640 1 -- 172.21.15.5:0/102507274 wait complete. 2024-04-25T10:02:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.309+0000 7fb8b7200640 1 Processor -- start 2024-04-25T10:02:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.309+0000 7fb8b7200640 1 -- start start 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b80b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8b80cc560 0x7fb8b80b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb8b815b780 0x7fb8b80b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8071010 con 0x7fb8b806f450 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80705b0 con 0x7fb8b80cc560 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80c7990 con 0x7fb8b815b780 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b80b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b80b6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46818/0 (socket says 172.21.15.5:46818) 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 -- 172.21.15.5:0/471651929 learned_addr learned my addr 172.21.15.5:0/471651929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 -- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb8b815b780 msgr2=0x7fb8b80b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b6800640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb8b815b780 0x7fb8b80b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5400640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8b80cc560 0x7fb8b80b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb8b815b780 0x7fb8b80b1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 -- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8b80cc560 msgr2=0x7fb8b80b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8b80cc560 0x7fb8b80b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.310+0000 7fb8b5e00640 1 -- 172.21.15.5:0/471651929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80b4800 con 0x7fb8b806f450 2024-04-25T10:02:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.311+0000 7fb8b5e00640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b80b6e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c390 tx=0x7fb8a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.311+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8017070 con 0x7fb8b806f450 2024-04-25T10:02:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.311+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8005ce0 con 0x7fb8b806f450 2024-04-25T10:02:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.311+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80072f0 con 0x7fb8b806f450 2024-04-25T10:02:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.311+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b80c70f0 con 0x7fb8b806f450 2024-04-25T10:02:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.311+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b80c76b0 con 0x7fb8b806f450 2024-04-25T10:02:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.312+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b8068f60 con 0x7fb8b806f450 2024-04-25T10:02:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.314+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8a801a070 con 0x7fb8b806f450 2024-04-25T10:02:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.314+0000 7fb8ae000640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.315+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb8a8089d50 con 0x7fb8b806f450 2024-04-25T10:02:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.315+0000 7fb8b5400640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.316+0000 7fb8b5400640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb89806b4d0 0x7fb89806d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4034000 tx=0x7fb8a402f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.318+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8a8021020 con 0x7fb8b806f450 2024-04-25T10:02:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.547+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb8b806bd00 con 0x7fb8b806f450 2024-04-25T10:02:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.547+0000 7fb8ae000640 1 -- 172.21.15.5:0/471651929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb8a8058900 con 0x7fb8b806f450 2024-04-25T10:02:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:32.552 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:02:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb89806b4d0 msgr2=0x7fb89806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb89806b4d0 0x7fb89806d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4034000 tx=0x7fb8a402f000 comp rx=0 tx=0).stop 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 msgr2=0x7fb8b80b6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b80b6e60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c390 tx=0x7fb8a800c850 comp rx=0 tx=0).stop 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 shutdown_connections 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb8b815b780 0x7fb8b80b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb89806b4d0 0x7fb89806d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8b80cc560 0x7fb8b80b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 --2- 172.21.15.5:0/471651929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8b806f450 0x7fb8b80b6e60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 >> 172.21.15.5:0/471651929 conn(0x7fb8b80b2ff0 msgr2=0x7fb8b80cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 shutdown_connections 2024-04-25T10:02:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.550+0000 7fb8b7200640 1 -- 172.21.15.5:0/471651929 wait complete. 2024-04-25T10:02:32.571 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:02:32.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.771+0000 7f88cb200640 1 Processor -- start 2024-04-25T10:02:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.771+0000 7f88cb200640 1 -- start start 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.772+0000 7f88cb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 0x7f88cc0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.772+0000 7f88cb200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88cc15da40 0x7f88cc0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.772+0000 7f88cb200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc0ce610 con 0x7f88cc0d06a0 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.772+0000 7f88cb200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc06bfc0 con 0x7f88cc15da40 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.772+0000 7f88cb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0cf8e0 con 0x7f88cc0cdcf0 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88ca800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f88cc0d06a0 0x7f88cc0999f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51624/0 (socket says 172.21.15.5:51624) 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88ca800640 1 -- 172.21.15.5:0/1655413295 learned_addr learned my addr 172.21.15.5:0/1655413295 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c9400640 1 --2- 172.21.15.5:0/1655413295 >> v2:172.21.15.80:3300/0 conn(0x7f88cc15da40 0x7f88cc0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c9e00640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 0x7f88cc0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346791236 0 0) 0x7f88cc0ce610 con 0x7f88cc0d06a0 2024-04-25T10:02:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b0003520 con 0x7f88cc0d06a0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3680001314 0 0) 0x7f88b0003520 con 0x7f88cc0d06a0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 msgr2=0x7f88cc0b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 0x7f88cc0b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 >> v2:172.21.15.80:3300/0 conn(0x7f88cc15da40 msgr2=0x7f88cc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 --2- 172.21.15.5:0/1655413295 >> v2:172.21.15.80:3300/0 conn(0x7f88cc15da40 0x7f88cc0ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0cfbc0 con 0x7f88cc0d06a0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.773+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f88c4001670 con 0x7f88cc0d06a0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 >> v1:172.21.15.5:6789/0 conn(0x7f88cc0d06a0 legacy=0x7f88cc0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88b00039b0 0x7f88b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 0x7f88b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b0004d40 0x7f88b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0cf8e0 con 0x7f88b0004d40 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b000b9c0 con 0x7f88b00039b0 2024-04-25T10:02:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b000bb70 con 0x7f88cc0cdcf0 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88ca800640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b0004d40 0x7f88b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c9e00640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88b00039b0 0x7f88b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88c9400640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 0x7f88b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88ca800640 1 -- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 msgr2=0x7f88b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88ca800640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88cc0cdcf0 0x7f88b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88ca800640 1 -- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88b00039b0 msgr2=0x7f88b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88ca800640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88b00039b0 0x7f88b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.774+0000 7f88ca800640 1 -- 172.21.15.5:0/1655413295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0003520 con 0x7f88b0004d40 2024-04-25T10:02:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.775+0000 7f88ca800640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b0004d40 0x7f88b00051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c190 tx=0x7f88b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38900ed81169280 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4007c80 con 0x7f88b0004d40 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4005ad0 con 0x7f88b0004d40 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88c8a00640 1 -- 172.21.15.5:0/1655413295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4007070 con 0x7f88b0004d40 2024-04-25T10:02:32.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88cb200640 1 -- 172.21.15.5:0/1655413295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b0004d40 msgr2=0x7f88b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88cb200640 1 --2- 172.21.15.5:0/1655413295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88b0004d40 0x7f88b00051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c190 tx=0x7f88b0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88ca800640 1 -- 172.21.15.5:0/1655413295 reap_dead start 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88cb200640 1 -- 172.21.15.5:0/1655413295 shutdown_connections 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.776+0000 7f88cb200640 1 -- 172.21.15.5:0/1655413295 >> 172.21.15.5:0/1655413295 conn(0x7f88cc0d1470 msgr2=0x7f88cc0d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.777+0000 7f88cb200640 1 -- 172.21.15.5:0/1655413295 shutdown_connections 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.777+0000 7f88cb200640 1 -- 172.21.15.5:0/1655413295 wait complete. 2024-04-25T10:02:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.778+0000 7f88cb200640 1 Processor -- start 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.779+0000 7f88cb200640 1 -- start start 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.779+0000 7f88cb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88b0004d40 0x7f88cc1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.779+0000 7f88cb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88cc1602e0 0x7f88cc1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.779+0000 7f88cb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 0x7f88cc0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88cb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0cf590 con 0x7f88cc0b6560 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88cb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0d0180 con 0x7f88cc1602e0 2024-04-25T10:02:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88cb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0b2a50 con 0x7f88b0004d40 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 0x7f88cc0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 0x7f88cc0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46844/0 (socket says 172.21.15.5:46844) 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 -- 172.21.15.5:0/1290175230 learned_addr learned my addr 172.21.15.5:0/1290175230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88c9e00640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88b0004d40 0x7f88cc1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88c9400640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88cc1602e0 0x7f88cc1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 -- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88b0004d40 msgr2=0x7f88cc1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88b0004d40 0x7f88cc1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 -- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88cc1602e0 msgr2=0x7f88cc1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88cc1602e0 0x7f88cc1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88ca800640 1 -- 172.21.15.5:0/1290175230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc154be0 con 0x7f88cc0b6560 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.780+0000 7f88c9400640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88cc1602e0 0x7f88cc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.781+0000 7f88ca800640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 0x7f88cc0b69e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c190 tx=0x7f88c4001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.781+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4037040 con 0x7f88cc0b6560 2024-04-25T10:02:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.781+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4041070 con 0x7f88cc0b6560 2024-04-25T10:02:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.781+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4001bc0 con 0x7f88cc0b6560 2024-04-25T10:02:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.781+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc154ea0 con 0x7f88cc0b6560 2024-04-25T10:02:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.781+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc155460 con 0x7f88cc0b6560 2024-04-25T10:02:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.782+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc068f60 con 0x7f88cc0b6560 2024-04-25T10:02:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.783+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88c4007070 con 0x7f88cc0b6560 2024-04-25T10:02:32.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.784+0000 7f88c2000640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88a806b500 0x7f88a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.785+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f88c40aa3a0 con 0x7f88cc0b6560 2024-04-25T10:02:32.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.785+0000 7f88c9e00640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88a806b500 0x7f88a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.786+0000 7f88c9e00640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88a806b500 0x7f88a806d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0c83f0 tx=0x7f88b4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.789+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88c4005aa0 con 0x7f88cc0b6560 2024-04-25T10:02:32.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.987+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88cc0b15a0 con 0x7f88cc0b6560 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.989+0000 7f88c2000640 1 -- 172.21.15.5:0/1290175230 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f88c4078ed0 con 0x7f88cc0b6560 2024-04-25T10:02:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:32.995 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.992+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88a806b500 msgr2=0x7f88a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.992+0000 7f88cb200640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88a806b500 0x7f88a806d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0c83f0 tx=0x7f88b4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.992+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 msgr2=0x7f88cc0b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.992+0000 7f88cb200640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 0x7f88cc0b69e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c190 tx=0x7f88c4001970 comp rx=0 tx=0).stop 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 shutdown_connections 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88cc0b6560 0x7f88cc0b69e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88a806b500 0x7f88a806d9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88cc1602e0 0x7f88cc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 --2- 172.21.15.5:0/1290175230 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88b0004d40 0x7f88cc1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 >> 172.21.15.5:0/1290175230 conn(0x7f88cc0d1470 msgr2=0x7f88cc09a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 shutdown_connections 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:32.993+0000 7f88cb200640 1 -- 172.21.15.5:0/1290175230 wait complete. 2024-04-25T10:02:33.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:33.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:33.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.224+0000 7fc996400640 1 Processor -- start 2024-04-25T10:02:33.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.225+0000 7fc996400640 1 -- start start 2024-04-25T10:02:33.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.226+0000 7fc996400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9900cc840 0x7fc99006e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.226+0000 7fc996400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc99006e670 0x7fc99015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.226+0000 7fc996400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc990072010 con 0x7fc99015b330 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.226+0000 7fc996400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99006ead0 con 0x7fc99006e670 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.226+0000 7fc996400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900ce570 con 0x7fc9900cc840 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc99006e670 0x7fc99015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc99006e670 0x7fc99015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46848/0 (socket says 172.21.15.5:46848) 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 -- 172.21.15.5:0/847801840 learned_addr learned my addr 172.21.15.5:0/847801840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc995000640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9900cc840 0x7fc99006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 -- 172.21.15.5:0/847801840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9900cc840 msgr2=0x7fc99006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9900cc840 0x7fc99006e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 -- 172.21.15.5:0/847801840 >> v1:172.21.15.80:6789/0 conn(0x7fc99015b330 legacy=0x7fc99015e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc98fe00640 1 -- 172.21.15.5:0/847801840 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900ce850 con 0x7fc99006e670 2024-04-25T10:02:33.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.227+0000 7fc995000640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9900cc840 0x7fc99006e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.228+0000 7fc98fe00640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99006e670 0x7fc99015adf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009870 tx=0x7fc97c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d34c4e23201930e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc98f400640 1 -- 172.21.15.5:0/847801840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c037070 con 0x7fc99006e670 2024-04-25T10:02:33.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc98f400640 1 -- 172.21.15.5:0/847801840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c032440 con 0x7fc99006e670 2024-04-25T10:02:33.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc98f400640 1 -- 172.21.15.5:0/847801840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c032810 con 0x7fc99006e670 2024-04-25T10:02:33.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc996400640 1 -- 172.21.15.5:0/847801840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99006e670 msgr2=0x7fc99015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc996400640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99006e670 0x7fc99015adf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009870 tx=0x7fc97c02a200 comp rx=0 tx=0).stop 2024-04-25T10:02:33.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc996400640 1 -- 172.21.15.5:0/847801840 shutdown_connections 2024-04-25T10:02:33.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc996400640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99006e670 0x7fc99015adf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.229+0000 7fc996400640 1 --2- 172.21.15.5:0/847801840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9900cc840 0x7fc99006e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.230+0000 7fc996400640 1 -- 172.21.15.5:0/847801840 >> 172.21.15.5:0/847801840 conn(0x7fc990066f20 msgr2=0x7fc99006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.230+0000 7fc996400640 1 -- 172.21.15.5:0/847801840 shutdown_connections 2024-04-25T10:02:33.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.230+0000 7fc996400640 1 -- 172.21.15.5:0/847801840 wait complete. 2024-04-25T10:02:33.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.232+0000 7fc996400640 1 Processor -- start 2024-04-25T10:02:33.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.232+0000 7fc996400640 1 -- start start 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc996400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc99006e670 0x7fc9901441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc996400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 0x7fc99014eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc996400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99015b330 0x7fc9901550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc996400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b2d20 con 0x7fc9900cc840 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc996400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900cee10 con 0x7fc99006e670 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc996400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900ce1d0 con 0x7fc99015b330 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 0x7fc99014eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 0x7fc99014eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46850/0 (socket says 172.21.15.5:46850) 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.233+0000 7fc98fe00640 1 -- 172.21.15.5:0/4063613511 learned_addr learned my addr 172.21.15.5:0/4063613511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc995a00640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99015b330 0x7fc9901550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc995000640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc99006e670 0x7fc9901441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98fe00640 1 -- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99015b330 msgr2=0x7fc9901550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98fe00640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99015b330 0x7fc9901550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98fe00640 1 -- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc99006e670 msgr2=0x7fc9901441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98fe00640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc99006e670 0x7fc9901441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98fe00640 1 -- 172.21.15.5:0/4063613511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990148150 con 0x7fc9900cc840 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc995000640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc99006e670 0x7fc9901441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc995a00640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99015b330 0x7fc9901550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98fe00640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 0x7fc99014eb50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009870 tx=0x7fc97c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c041020 con 0x7fc9900cc840 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.234+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c030970 con 0x7fc9900cc840 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.235+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c037040 con 0x7fc9900cc840 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.235+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc990148430 con 0x7fc9900cc840 2024-04-25T10:02:33.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.235+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc990148cd0 con 0x7fc9900cc840 2024-04-25T10:02:33.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.236+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99015de00 con 0x7fc9900cc840 2024-04-25T10:02:33.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.237+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc97c044020 con 0x7fc9900cc840 2024-04-25T10:02:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.238+0000 7fc98cc00640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.238+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc97c0ab070 con 0x7fc9900cc840 2024-04-25T10:02:33.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.239+0000 7fc995000640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.240+0000 7fc995000640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc984006fd0 tx=0x7fc984008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.242+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc97c04b030 con 0x7fc9900cc840 2024-04-25T10:02:33.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.433+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc990068f60 con 0x7fc9900cc840 2024-04-25T10:02:33.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.434+0000 7fc98cc00640 1 -- 172.21.15.5:0/4063613511 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc97c079a80 con 0x7fc9900cc840 2024-04-25T10:02:33.438 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:33.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 msgr2=0x7fc97006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc984006fd0 tx=0x7fc984008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 msgr2=0x7fc99014eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 0x7fc99014eb50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc97c009870 tx=0x7fc97c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 shutdown_connections 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99015b330 0x7fc9901550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9900cc840 0x7fc99014eb50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 --2- 172.21.15.5:0/4063613511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc99006e670 0x7fc9901441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 >> 172.21.15.5:0/4063613511 conn(0x7fc990066f20 msgr2=0x7fc99015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 shutdown_connections 2024-04-25T10:02:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.436+0000 7fc996400640 1 -- 172.21.15.5:0/4063613511 wait complete. 2024-04-25T10:02:33.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.668+0000 7fb577e00640 1 Processor -- start 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.668+0000 7fb577e00640 1 -- start start 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.669+0000 7fb577e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb57815b4a0 0x7fb57815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.669+0000 7fb577e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb5780713e0 0x7fb57806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.669+0000 7fb577e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57815b880 con 0x7fb57806b0a0 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.669+0000 7fb577e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57806d6c0 con 0x7fb5780713e0 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.669+0000 7fb577e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57815edf0 con 0x7fb57815b4a0 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb56fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb5780713e0 0x7fb57806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb56fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb5780713e0 0x7fb57806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46860/0 (socket says 172.21.15.5:46860) 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb56fe00640 1 -- 172.21.15.5:0/2319300905 learned_addr learned my addr 172.21.15.5:0/2319300905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb576a00640 1 --2- 172.21.15.5:0/2319300905 >> v2:172.21.15.80:3300/0 conn(0x7fb57815b4a0 0x7fb57815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb56fe00640 1 -- 172.21.15.5:0/2319300905 >> v2:172.21.15.80:3300/0 conn(0x7fb57815b4a0 msgr2=0x7fb57815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb56fe00640 1 --2- 172.21.15.5:0/2319300905 >> v2:172.21.15.80:3300/0 conn(0x7fb57815b4a0 0x7fb57815e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.670+0000 7fb56fe00640 1 -- 172.21.15.5:0/2319300905 >> v1:172.21.15.80:6789/0 conn(0x7fb57806b0a0 legacy=0x7fb578161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.671+0000 7fb56fe00640 1 -- 172.21.15.5:0/2319300905 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5780719a0 con 0x7fb5780713e0 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.671+0000 7fb576a00640 1 --2- 172.21.15.5:0/2319300905 >> v2:172.21.15.80:3300/0 conn(0x7fb57815b4a0 0x7fb57815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.672+0000 7fb56fe00640 1 --2- 172.21.15.5:0/2319300905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5780713e0 0x7fb57806ab60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb578072580 tx=0x7fb56002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99d5db20e72ed38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.672+0000 7fb576000640 1 -- 172.21.15.5:0/2319300905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560037070 con 0x7fb5780713e0 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.672+0000 7fb576000640 1 -- 172.21.15.5:0/2319300905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb560032440 con 0x7fb5780713e0 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.672+0000 7fb576000640 1 -- 172.21.15.5:0/2319300905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560032810 con 0x7fb5780713e0 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 -- 172.21.15.5:0/2319300905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5780713e0 msgr2=0x7fb57806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 --2- 172.21.15.5:0/2319300905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5780713e0 0x7fb57806ab60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb578072580 tx=0x7fb56002a230 comp rx=0 tx=0).stop 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 -- 172.21.15.5:0/2319300905 shutdown_connections 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 --2- 172.21.15.5:0/2319300905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5780713e0 0x7fb57806ab60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 --2- 172.21.15.5:0/2319300905 >> v2:172.21.15.80:3300/0 conn(0x7fb57815b4a0 0x7fb57815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 -- 172.21.15.5:0/2319300905 >> 172.21.15.5:0/2319300905 conn(0x7fb578066f80 msgr2=0x7fb5780b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 -- 172.21.15.5:0/2319300905 shutdown_connections 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.673+0000 7fb577e00640 1 -- 172.21.15.5:0/2319300905 wait complete. 2024-04-25T10:02:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.675+0000 7fb577e00640 1 Processor -- start 2024-04-25T10:02:33.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.675+0000 7fb577e00640 1 -- start start 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 0x7fb578159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb5780713e0 0x7fb57814f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb57815b4a0 0x7fb578153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578072240 con 0x7fb57806b0a0 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb578071f60 con 0x7fb57815b4a0 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57806eb60 con 0x7fb5780713e0 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb576a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 0x7fb578159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb576a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 0x7fb578159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46864/0 (socket says 172.21.15.5:46864) 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb576a00640 1 -- 172.21.15.5:0/549062872 learned_addr learned my addr 172.21.15.5:0/549062872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb56fe00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb5780713e0 0x7fb57814f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb576a00640 1 -- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb5780713e0 msgr2=0x7fb57814f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.676+0000 7fb577400640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb57815b4a0 0x7fb578153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb576a00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb5780713e0 0x7fb57814f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb576a00640 1 -- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb57815b4a0 msgr2=0x7fb578153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb576a00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb57815b4a0 0x7fb578153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb576a00640 1 -- 172.21.15.5:0/549062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578153eb0 con 0x7fb57806b0a0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb577400640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb57815b4a0 0x7fb578153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb576a00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 0x7fb578159800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb56800c7f0 tx=0x7fb56800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568015070 con 0x7fb57806b0a0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb568007820 con 0x7fb57806b0a0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568007bd0 con 0x7fb57806b0a0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb578154100 con 0x7fb57806b0a0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.677+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5781543c0 con 0x7fb57806b0a0 2024-04-25T10:02:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.679+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb578068f60 con 0x7fb57806b0a0 2024-04-25T10:02:33.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.679+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb568018070 con 0x7fb57806b0a0 2024-04-25T10:02:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.680+0000 7fb56ea00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.680+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb56808a700 con 0x7fb57806b0a0 2024-04-25T10:02:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.681+0000 7fb56fe00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.682+0000 7fb56fe00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb55406b4d0 0x7fb55406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb560034000 tx=0x7fb56002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.685+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb578153eb0 con 0x7fb57806b0a0 2024-04-25T10:02:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.884+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5780cee30 con 0x7fb57806b0a0 2024-04-25T10:02:33.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.885+0000 7fb56ea00640 1 -- 172.21.15.5:0/549062872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5680591a0 con 0x7fb57806b0a0 2024-04-25T10:02:33.889 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb55406b4d0 msgr2=0x7fb55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb55406b4d0 0x7fb55406d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb560034000 tx=0x7fb56002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 msgr2=0x7fb578159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 0x7fb578159800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb56800c7f0 tx=0x7fb56800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 shutdown_connections 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb57815b4a0 0x7fb578153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb5780713e0 0x7fb57814f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 --2- 172.21.15.5:0/549062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb57806b0a0 0x7fb578159800 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 >> 172.21.15.5:0/549062872 conn(0x7fb578066f80 msgr2=0x7fb57815a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 shutdown_connections 2024-04-25T10:02:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:33.888+0000 7fb577e00640 1 -- 172.21.15.5:0/549062872 wait complete. 2024-04-25T10:02:33.914 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:34.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.131+0000 7f6644200640 1 Processor -- start 2024-04-25T10:02:34.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.131+0000 7f6644200640 1 -- start start 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6644200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6644200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f663c15f1c0 0x7f663c0d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6644200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f663c0ce830 0x7f663c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6644200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c0ce380 con 0x7f663c15f1c0 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6644200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c15f000 con 0x7f663c0ce830 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6644200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0d2190 con 0x7f663c0d3760 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6640e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f663c15f1c0 0x7f663c0d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6640e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f663c15f1c0 0x7f663c0d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46870/0 (socket says 172.21.15.5:46870) 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.133+0000 7f6640e00640 1 -- 172.21.15.5:0/1121101460 learned_addr learned my addr 172.21.15.5:0/1121101460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6641800640 1 --2- 172.21.15.5:0/1121101460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6642200640 1 --2- 172.21.15.5:0/1121101460 >> v2:172.21.15.80:3300/0 conn(0x7f663c0ce830 0x7f663c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6640e00640 1 -- 172.21.15.5:0/1121101460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 msgr2=0x7f663c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6640e00640 1 --2- 172.21.15.5:0/1121101460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6640e00640 1 -- 172.21.15.5:0/1121101460 >> v2:172.21.15.80:3300/0 conn(0x7f663c0ce830 msgr2=0x7f663c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6640e00640 1 --2- 172.21.15.5:0/1121101460 >> v2:172.21.15.80:3300/0 conn(0x7f663c0ce830 0x7f663c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.134+0000 7f6640e00640 1 -- 172.21.15.5:0/1121101460 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0d4080 con 0x7f663c15f1c0 2024-04-25T10:02:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.135+0000 7f6640e00640 1 --2- 172.21.15.5:0/1121101460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c0d3b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f663c0d1760 tx=0x7f662802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7a55b39f3b55b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.135+0000 7f663be00640 1 -- 172.21.15.5:0/1121101460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628037070 con 0x7f663c15f1c0 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.135+0000 7f663be00640 1 -- 172.21.15.5:0/1121101460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6628032440 con 0x7f663c15f1c0 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.135+0000 7f663be00640 1 -- 172.21.15.5:0/1121101460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6628032810 con 0x7f663c15f1c0 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 -- 172.21.15.5:0/1121101460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 msgr2=0x7f663c0d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 --2- 172.21.15.5:0/1121101460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c0d3b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f663c0d1760 tx=0x7f662802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 -- 172.21.15.5:0/1121101460 shutdown_connections 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 --2- 172.21.15.5:0/1121101460 >> v2:172.21.15.80:3300/0 conn(0x7f663c0ce830 0x7f663c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 --2- 172.21.15.5:0/1121101460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c0d3b40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 --2- 172.21.15.5:0/1121101460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 -- 172.21.15.5:0/1121101460 >> 172.21.15.5:0/1121101460 conn(0x7f663c072680 msgr2=0x7f663c072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 -- 172.21.15.5:0/1121101460 shutdown_connections 2024-04-25T10:02:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.136+0000 7f6644200640 1 -- 172.21.15.5:0/1121101460 wait complete. 2024-04-25T10:02:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.138+0000 7f6644200640 1 Processor -- start 2024-04-25T10:02:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.138+0000 7f6644200640 1 -- start start 2024-04-25T10:02:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6644200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f663c0ce830 0x7f663c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6644200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6644200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6644200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0d1e20 con 0x7f663c15f1c0 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6644200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0d1b40 con 0x7f663c0ce830 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6644200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c15de10 con 0x7f663c0d3760 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.139+0000 7f6642200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c15c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46880/0 (socket says 172.21.15.5:46880) 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 -- 172.21.15.5:0/2926999217 learned_addr learned my addr 172.21.15.5:0/2926999217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6641800640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f663c0ce830 0x7f663c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6640e00640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 -- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 msgr2=0x7f663c15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 -- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f663c0ce830 msgr2=0x7f663c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f663c0ce830 0x7f663c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 -- 172.21.15.5:0/2926999217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c158cb0 con 0x7f663c15f1c0 2024-04-25T10:02:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.140+0000 7f6642200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c15c4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6634007e30 tx=0x7f663400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.141+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634014420 con 0x7f663c15f1c0 2024-04-25T10:02:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.141+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634019070 con 0x7f663c15f1c0 2024-04-25T10:02:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.141+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634014800 con 0x7f663c15f1c0 2024-04-25T10:02:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.141+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c158f90 con 0x7f663c15f1c0 2024-04-25T10:02:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.141+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c159550 con 0x7f663c15f1c0 2024-04-25T10:02:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.142+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c068f60 con 0x7f663c15f1c0 2024-04-25T10:02:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.143+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66340149a0 con 0x7f663c15f1c0 2024-04-25T10:02:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.144+0000 7f6639600640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f661c06b550 0x7f661c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.144+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f663408ac50 con 0x7f663c15f1c0 2024-04-25T10:02:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.145+0000 7f6641800640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f661c06b550 0x7f661c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.146+0000 7f6641800640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f661c06b550 0x7f661c06da10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.148+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6634059800 con 0x7f663c15f1c0 2024-04-25T10:02:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.353+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f663c0ce610 con 0x7f663c15f1c0 2024-04-25T10:02:34.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.354+0000 7f6639600640 1 -- 172.21.15.5:0/2926999217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6634059100 con 0x7f663c15f1c0 2024-04-25T10:02:34.359 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f661c06b550 msgr2=0x7f661c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f661c06b550 0x7f661c06da10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f6630008040 comp rx=0 tx=0).stop 2024-04-25T10:02:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 msgr2=0x7f663c15c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c15c4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6634007e30 tx=0x7f663400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 shutdown_connections 2024-04-25T10:02:34.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f663c15f1c0 0x7f663c15c4c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f663c0d3760 0x7f663c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f661c06b550 0x7f661c06da10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.357+0000 7f6644200640 1 --2- 172.21.15.5:0/2926999217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f663c0ce830 0x7f663c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.358+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 >> 172.21.15.5:0/2926999217 conn(0x7f663c072680 msgr2=0x7f663c0b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.358+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 shutdown_connections 2024-04-25T10:02:34.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.358+0000 7f6644200640 1 -- 172.21.15.5:0/2926999217 wait complete. 2024-04-25T10:02:34.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.598+0000 7f8097200640 1 Processor -- start 2024-04-25T10:02:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.599+0000 7f8097200640 1 -- start start 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.600+0000 7f8097200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80900cdaa0 0x7f809015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.600+0000 7f8097200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f809015fe30 0x7f809015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.600+0000 7f8097200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8090144cd0 con 0x7f80901448f0 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.600+0000 7f8097200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80900cdf30 con 0x7f80900cdaa0 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.600+0000 7f8097200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809015f460 con 0x7f809015fe30 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80900cdaa0 0x7f809015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80900cdaa0 0x7f809015f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46896/0 (socket says 172.21.15.5:46896) 2024-04-25T10:02:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 -- 172.21.15.5:0/2120942671 learned_addr learned my addr 172.21.15.5:0/2120942671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8096800640 1 --2- 172.21.15.5:0/2120942671 >> v2:172.21.15.80:3300/0 conn(0x7f809015fe30 0x7f809015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 -- 172.21.15.5:0/2120942671 >> v2:172.21.15.80:3300/0 conn(0x7f809015fe30 msgr2=0x7f809015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 --2- 172.21.15.5:0/2120942671 >> v2:172.21.15.80:3300/0 conn(0x7f809015fe30 0x7f809015ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 -- 172.21.15.5:0/2120942671 >> v1:172.21.15.80:6789/0 conn(0x7f80901448f0 legacy=0x7f809015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.601+0000 7f8095400640 1 -- 172.21.15.5:0/2120942671 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809015c440 con 0x7f80900cdaa0 2024-04-25T10:02:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.602+0000 7f8095400640 1 --2- 172.21.15.5:0/2120942671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809015f8f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f809006bc00 tx=0x7f808002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34418feea7d1fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8094a00640 1 -- 172.21.15.5:0/2120942671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080037070 con 0x7f80900cdaa0 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8094a00640 1 -- 172.21.15.5:0/2120942671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080032440 con 0x7f80900cdaa0 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8094a00640 1 -- 172.21.15.5:0/2120942671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080032810 con 0x7f80900cdaa0 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8097200640 1 -- 172.21.15.5:0/2120942671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 msgr2=0x7f809015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8097200640 1 --2- 172.21.15.5:0/2120942671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809015f8f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f809006bc00 tx=0x7f808002a230 comp rx=0 tx=0).stop 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8097200640 1 -- 172.21.15.5:0/2120942671 shutdown_connections 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8097200640 1 --2- 172.21.15.5:0/2120942671 >> v2:172.21.15.80:3300/0 conn(0x7f809015fe30 0x7f809015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8097200640 1 --2- 172.21.15.5:0/2120942671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809015f8f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.603+0000 7f8097200640 1 -- 172.21.15.5:0/2120942671 >> 172.21.15.5:0/2120942671 conn(0x7f8090066f20 msgr2=0x7f809006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.604+0000 7f8097200640 1 -- 172.21.15.5:0/2120942671 shutdown_connections 2024-04-25T10:02:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.604+0000 7f8097200640 1 -- 172.21.15.5:0/2120942671 wait complete. 2024-04-25T10:02:34.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.606+0000 7f8097200640 1 Processor -- start 2024-04-25T10:02:34.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.606+0000 7f8097200640 1 -- start start 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8097200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809013ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8097200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80901448f0 0x7f80900a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8097200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809015fe30 0x7f80900a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8097200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809015d360 con 0x7f80900cdaa0 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8097200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809015ca00 con 0x7f80901448f0 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8097200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809015d4e0 con 0x7f809015fe30 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8095e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809013ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8095e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809013ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46910/0 (socket says 172.21.15.5:46910) 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.607+0000 7f8095e00640 1 -- 172.21.15.5:0/1502540302 learned_addr learned my addr 172.21.15.5:0/1502540302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:34.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095400640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80901448f0 0x7f80900a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8096800640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809015fe30 0x7f80900a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095e00640 1 -- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809015fe30 msgr2=0x7f80900a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095e00640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809015fe30 0x7f80900a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095e00640 1 -- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80901448f0 msgr2=0x7f80900a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095e00640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80901448f0 0x7f80900a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095e00640 1 -- 172.21.15.5:0/1502540302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80901f2150 con 0x7f80900cdaa0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095400640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80901448f0 0x7f80900a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f8095e00640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809013ed80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f808400c410 tx=0x7f808400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.608+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084017070 con 0x7f80900cdaa0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.609+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80840035f0 con 0x7f80900cdaa0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.609+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809013a650 con 0x7f80900cdaa0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.609+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084003a20 con 0x7f80900cdaa0 2024-04-25T10:02:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.609+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809013ac10 con 0x7f80900cdaa0 2024-04-25T10:02:34.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.610+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8090068f60 con 0x7f80900cdaa0 2024-04-25T10:02:34.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.611+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f808401a070 con 0x7f80900cdaa0 2024-04-25T10:02:34.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.612+0000 7f808a000640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807406b4d0 0x7f807406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.612+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8084089ac0 con 0x7f80900cdaa0 2024-04-25T10:02:34.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.613+0000 7f8095400640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807406b4d0 0x7f807406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.614+0000 7f8095400640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807406b4d0 0x7f807406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8080034000 tx=0x7f808002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.616+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8084058560 con 0x7f80900cdaa0 2024-04-25T10:02:34.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.814+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8090072180 con 0x7f80900cdaa0 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.815+0000 7f808a000640 1 -- 172.21.15.5:0/1502540302 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8084089e20 con 0x7f80900cdaa0 2024-04-25T10:02:34.820 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807406b4d0 msgr2=0x7f807406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807406b4d0 0x7f807406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8080034000 tx=0x7f808002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 msgr2=0x7f809013ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809013ed80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f808400c410 tx=0x7f808400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 shutdown_connections 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809015fe30 0x7f80900a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807406b4d0 0x7f807406d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80901448f0 0x7f80900a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 --2- 172.21.15.5:0/1502540302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80900cdaa0 0x7f809013ed80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 >> 172.21.15.5:0/1502540302 conn(0x7f8090066f20 msgr2=0x7f80900af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 shutdown_connections 2024-04-25T10:02:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:34.818+0000 7f8097200640 1 -- 172.21.15.5:0/1502540302 wait complete. 2024-04-25T10:02:34.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.044+0000 7fa82f200640 1 Processor -- start 2024-04-25T10:02:35.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.044+0000 7fa82f200640 1 -- start start 2024-04-25T10:02:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa82f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa830141da0 0x7fa83006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa82f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa83006e640 0x7fa83015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa82f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa8300cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa82f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa830142180 con 0x7fa83006e640 2024-04-25T10:02:35.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa82f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa830071d30 con 0x7fa830141da0 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa82f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83006d620 con 0x7fa83015d340 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.046+0000 7fa827e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa83006e640 0x7fa83015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa83006e640 0x7fa83015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46926/0 (socket says 172.21.15.5:46926) 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 -- 172.21.15.5:0/3825400443 learned_addr learned my addr 172.21.15.5:0/3825400443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa82de00640 1 --2- 172.21.15.5:0/3825400443 >> v2:172.21.15.80:3300/0 conn(0x7fa830141da0 0x7fa83006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa82e800640 1 --2- 172.21.15.5:0/3825400443 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa8300cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 -- 172.21.15.5:0/3825400443 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 msgr2=0x7fa8300cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 --2- 172.21.15.5:0/3825400443 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa8300cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 -- 172.21.15.5:0/3825400443 >> v2:172.21.15.80:3300/0 conn(0x7fa830141da0 msgr2=0x7fa83006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 --2- 172.21.15.5:0/3825400443 >> v2:172.21.15.80:3300/0 conn(0x7fa830141da0 0x7fa83006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.047+0000 7fa827e00640 1 -- 172.21.15.5:0/3825400443 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83006d900 con 0x7fa83006e640 2024-04-25T10:02:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.048+0000 7fa827e00640 1 --2- 172.21.15.5:0/3825400443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83006e640 0x7fa83015ce00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa83006bbb0 tx=0x7fa81802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e51ba8716493b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.048+0000 7fa82d400640 1 -- 172.21.15.5:0/3825400443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818037070 con 0x7fa83006e640 2024-04-25T10:02:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.048+0000 7fa82d400640 1 -- 172.21.15.5:0/3825400443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818032440 con 0x7fa83006e640 2024-04-25T10:02:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.048+0000 7fa82d400640 1 -- 172.21.15.5:0/3825400443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818032810 con 0x7fa83006e640 2024-04-25T10:02:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 -- 172.21.15.5:0/3825400443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83006e640 msgr2=0x7fa83015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 --2- 172.21.15.5:0/3825400443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83006e640 0x7fa83015ce00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa83006bbb0 tx=0x7fa81802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 -- 172.21.15.5:0/3825400443 shutdown_connections 2024-04-25T10:02:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 --2- 172.21.15.5:0/3825400443 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa8300cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 --2- 172.21.15.5:0/3825400443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83006e640 0x7fa83015ce00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 --2- 172.21.15.5:0/3825400443 >> v2:172.21.15.80:3300/0 conn(0x7fa830141da0 0x7fa83006e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.049+0000 7fa82f200640 1 -- 172.21.15.5:0/3825400443 >> 172.21.15.5:0/3825400443 conn(0x7fa830066f20 msgr2=0x7fa83006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.050+0000 7fa82f200640 1 -- 172.21.15.5:0/3825400443 shutdown_connections 2024-04-25T10:02:35.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.050+0000 7fa82f200640 1 -- 172.21.15.5:0/3825400443 wait complete. 2024-04-25T10:02:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.051+0000 7fa82f200640 1 Processor -- start 2024-04-25T10:02:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.051+0000 7fa82f200640 1 -- start start 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.052+0000 7fa82f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa83006e640 0x7fa83013c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.052+0000 7fa82f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 0x7fa8300a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.052+0000 7fa82f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa830159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.052+0000 7fa82f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300cd9c0 con 0x7fa830141da0 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.052+0000 7fa82f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300cd840 con 0x7fa83006e640 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.052+0000 7fa82f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300cdb40 con 0x7fa83015d340 2024-04-25T10:02:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 0x7fa8300a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 0x7fa8300a9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46932/0 (socket says 172.21.15.5:46932) 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 -- 172.21.15.5:0/3477187195 learned_addr learned my addr 172.21.15.5:0/3477187195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa82de00640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa83006e640 0x7fa83013c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa82e800640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa830159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 -- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 msgr2=0x7fa830159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa830159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 -- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa83006e640 msgr2=0x7fa83013c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa83006e640 0x7fa83013c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.053+0000 7fa827e00640 1 -- 172.21.15.5:0/3477187195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300aeeb0 con 0x7fa830141da0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.054+0000 7fa827e00640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 0x7fa8300a9f10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa81802a220 tx=0x7fa8180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.054+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818041020 con 0x7fa830141da0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.054+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8180306c0 con 0x7fa830141da0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.054+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818037040 con 0x7fa830141da0 2024-04-25T10:02:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.054+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8300af190 con 0x7fa830141da0 2024-04-25T10:02:35.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.054+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8300af750 con 0x7fa830141da0 2024-04-25T10:02:35.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.055+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa830068f60 con 0x7fa830141da0 2024-04-25T10:02:35.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.056+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa818044020 con 0x7fa830141da0 2024-04-25T10:02:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.058+0000 7fa826000640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b4d0 0x7fa80c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.058+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa8180aae80 con 0x7fa830141da0 2024-04-25T10:02:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.058+0000 7fa82de00640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b4d0 0x7fa80c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.059+0000 7fa82de00640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b4d0 0x7fa80c06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.062+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa81804b030 con 0x7fa830141da0 2024-04-25T10:02:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.300+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa83015ccb0 con 0x7fa830141da0 2024-04-25T10:02:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.302+0000 7fa826000640 1 -- 172.21.15.5:0/3477187195 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7fa818079890 con 0x7fa830141da0 2024-04-25T10:02:35.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:35.306 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b4d0 msgr2=0x7fa80c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b4d0 0x7fa80c06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820008040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 msgr2=0x7fa8300a9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 0x7fa8300a9f10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa81802a220 tx=0x7fa8180046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 shutdown_connections 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa83015d340 0x7fa830159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830141da0 0x7fa8300a9f10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b4d0 0x7fa80c06d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 --2- 172.21.15.5:0/3477187195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa83006e640 0x7fa83013c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.304+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 >> 172.21.15.5:0/3477187195 conn(0x7fa830066f20 msgr2=0x7fa8300b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.305+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 shutdown_connections 2024-04-25T10:02:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.305+0000 7fa82f200640 1 -- 172.21.15.5:0/3477187195 wait complete. 2024-04-25T10:02:35.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.521+0000 7ff815400640 1 Processor -- start 2024-04-25T10:02:35.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.522+0000 7ff815400640 1 -- start start 2024-04-25T10:02:35.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.523+0000 7ff815400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff81015db00 0x7ff8100d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.523+0000 7ff815400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8100b53b0 0x7ff8100d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.523+0000 7ff815400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff8100ce2b0 0x7ff810161670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.523+0000 7ff815400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100d1e80 con 0x7ff81015db00 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.523+0000 7ff815400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100b5930 con 0x7ff8100ce2b0 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.523+0000 7ff815400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100b5c10 con 0x7ff8100b53b0 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff81015db00 0x7ff8100d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff81015db00 0x7ff8100d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46940/0 (socket says 172.21.15.5:46940) 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 -- 172.21.15.5:0/139859567 learned_addr learned my addr 172.21.15.5:0/139859567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80f200640 1 --2- 172.21.15.5:0/139859567 >> v2:172.21.15.80:3300/0 conn(0x7ff8100ce2b0 0x7ff810161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 -- 172.21.15.5:0/139859567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8100b53b0 msgr2=0x7ff8100d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80de00640 1 --2- 172.21.15.5:0/139859567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8100b53b0 0x7ff8100d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 --2- 172.21.15.5:0/139859567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8100b53b0 0x7ff8100d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 -- 172.21.15.5:0/139859567 >> v2:172.21.15.80:3300/0 conn(0x7ff8100ce2b0 msgr2=0x7ff810161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 --2- 172.21.15.5:0/139859567 >> v2:172.21.15.80:3300/0 conn(0x7ff8100ce2b0 0x7ff810161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80e800640 1 -- 172.21.15.5:0/139859567 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100b5ef0 con 0x7ff81015db00 2024-04-25T10:02:35.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.524+0000 7ff80f200640 1 --2- 172.21.15.5:0/139859567 >> v2:172.21.15.80:3300/0 conn(0x7ff8100ce2b0 0x7ff810161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.525+0000 7ff80e800640 1 --2- 172.21.15.5:0/139859567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81015db00 0x7ff8100d0340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff8100ce010 tx=0x7ff80002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37757c4d26ca7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.526+0000 7ff80d400640 1 -- 172.21.15.5:0/139859567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800037070 con 0x7ff81015db00 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.526+0000 7ff80d400640 1 -- 172.21.15.5:0/139859567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800032440 con 0x7ff81015db00 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.526+0000 7ff80d400640 1 -- 172.21.15.5:0/139859567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800032810 con 0x7ff81015db00 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.526+0000 7ff815400640 1 -- 172.21.15.5:0/139859567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81015db00 msgr2=0x7ff8100d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.526+0000 7ff815400640 1 --2- 172.21.15.5:0/139859567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81015db00 0x7ff8100d0340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff8100ce010 tx=0x7ff80002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 -- 172.21.15.5:0/139859567 shutdown_connections 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 --2- 172.21.15.5:0/139859567 >> v2:172.21.15.80:3300/0 conn(0x7ff8100ce2b0 0x7ff810161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 --2- 172.21.15.5:0/139859567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8100b53b0 0x7ff8100d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 --2- 172.21.15.5:0/139859567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff81015db00 0x7ff8100d0340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 -- 172.21.15.5:0/139859567 >> 172.21.15.5:0/139859567 conn(0x7ff810072290 msgr2=0x7ff8100726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 -- 172.21.15.5:0/139859567 shutdown_connections 2024-04-25T10:02:35.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.527+0000 7ff815400640 1 -- 172.21.15.5:0/139859567 wait complete. 2024-04-25T10:02:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.529+0000 7ff815400640 1 Processor -- start 2024-04-25T10:02:35.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.529+0000 7ff815400640 1 -- start start 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff815400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8100b53b0 0x7ff81015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff815400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 0x7ff810151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff815400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff81015db00 0x7ff8100caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff815400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100d3660 con 0x7ff8100ce2b0 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff815400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100b61d0 con 0x7ff8100b53b0 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff815400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100b6c10 con 0x7ff81015db00 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.530+0000 7ff80de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 0x7ff810151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 0x7ff810151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46942/0 (socket says 172.21.15.5:46942) 2024-04-25T10:02:35.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 -- 172.21.15.5:0/1447150989 learned_addr learned my addr 172.21.15.5:0/1447150989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80e800640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8100b53b0 0x7ff81015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80f200640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff81015db00 0x7ff8100caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 -- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff81015db00 msgr2=0x7ff8100caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff81015db00 0x7ff8100caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 -- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8100b53b0 msgr2=0x7ff81015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8100b53b0 0x7ff81015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 -- 172.21.15.5:0/1447150989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8101547d0 con 0x7ff8100ce2b0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80e800640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8100b53b0 0x7ff81015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.531+0000 7ff80de00640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 0x7ff810151d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c7f0 tx=0x7ff7fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.532+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc015070 con 0x7ff8100ce2b0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.532+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc004240 con 0x7ff8100ce2b0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.532+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc007620 con 0x7ff8100ce2b0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.532+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810154ab0 con 0x7ff8100ce2b0 2024-04-25T10:02:35.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.532+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff810155070 con 0x7ff8100ce2b0 2024-04-25T10:02:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.533+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff810068fd0 con 0x7ff8100ce2b0 2024-04-25T10:02:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.534+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7fc0079d0 con 0x7ff8100ce2b0 2024-04-25T10:02:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.535+0000 7ff806a00640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7f006b4d0 0x7ff7f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.535+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff7fc08a640 con 0x7ff8100ce2b0 2024-04-25T10:02:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.535+0000 7ff80e800640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7f006b4d0 0x7ff7f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.536+0000 7ff80e800640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7f006b4d0 0x7ff7f006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff800034000 tx=0x7ff80002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.539+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7fc059180 con 0x7ff8100ce2b0 2024-04-25T10:02:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.735+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8100d16b0 con 0x7ff8100ce2b0 2024-04-25T10:02:35.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.738+0000 7ff806a00640 1 -- 172.21.15.5:0/1447150989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7ff7fc058a80 con 0x7ff8100ce2b0 2024-04-25T10:02:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:35.743 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.740+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7f006b4d0 msgr2=0x7ff7f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.740+0000 7ff815400640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7f006b4d0 0x7ff7f006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff800034000 tx=0x7ff80002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.740+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 msgr2=0x7ff810151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.740+0000 7ff815400640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 0x7ff810151d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00c7f0 tx=0x7ff7fc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 shutdown_connections 2024-04-25T10:02:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff81015db00 0x7ff8100caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8100ce2b0 0x7ff810151d20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7f006b4d0 0x7ff7f006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 --2- 172.21.15.5:0/1447150989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8100b53b0 0x7ff81015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 >> 172.21.15.5:0/1447150989 conn(0x7ff810072290 msgr2=0x7ff8100c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 shutdown_connections 2024-04-25T10:02:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.741+0000 7ff815400640 1 -- 172.21.15.5:0/1447150989 wait complete. 2024-04-25T10:02:35.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:35.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.968+0000 7efd8e800640 1 Processor -- start 2024-04-25T10:02:35.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.968+0000 7efd8e800640 1 -- start start 2024-04-25T10:02:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.969+0000 7efd8e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 0x7efd8806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.969+0000 7efd8e800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd881453c0 con 0x7efd8806d130 2024-04-25T10:02:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd8e800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8806d6a0 con 0x7efd88071840 2024-04-25T10:02:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd8e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880a8560 con 0x7efd88144fe0 2024-04-25T10:02:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd86a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efd8806d130 0x7efd88071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51628/0 (socket says 172.21.15.5:51628) 2024-04-25T10:02:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd86a00640 1 -- 172.21.15.5:0/2155653002 learned_addr learned my addr 172.21.15.5:0/2155653002 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd87400640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 0x7efd8806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968161065 0 0) 0x7efd881453c0 con 0x7efd8806d130 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.970+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd6c003520 con 0x7efd8806d130 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1540241955 0 0) 0x7efd6c003520 con 0x7efd8806d130 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 msgr2=0x7efd8806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 0x7efd8806e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 >> v1:172.21.15.80:6789/0 conn(0x7efd88071840 legacy=0x7efd8815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8806d980 con 0x7efd8806d130 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7efd780019f0 con 0x7efd8806d130 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 >> v1:172.21.15.5:6789/0 conn(0x7efd8806d130 legacy=0x7efd88071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6c0039e0 0x7efd6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 0x7efd6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c00ba70 con 0x7efd6c0039e0 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c00bc20 con 0x7efd6c004800 2024-04-25T10:02:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c00be20 con 0x7efd88144fe0 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.971+0000 7efd87400640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6c0039e0 0x7efd6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd86a00640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd87e00640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 0x7efd6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd87400640 1 -- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 msgr2=0x7efd6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd87400640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd88144fe0 0x7efd6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd87400640 1 -- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd6c004800 msgr2=0x7efd6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd87400640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd6c004800 0x7efd6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.972+0000 7efd87400640 1 -- 172.21.15.5:0/2155653002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c003520 con 0x7efd6c0039e0 2024-04-25T10:02:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.973+0000 7efd87400640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6c0039e0 0x7efd6c003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd70009a90 tx=0x7efd6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6b58779b50ceb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.973+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70036070 con 0x7efd6c0039e0 2024-04-25T10:02:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.973+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd70031400 con 0x7efd6c0039e0 2024-04-25T10:02:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.973+0000 7efd86000640 1 -- 172.21.15.5:0/2155653002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70009690 con 0x7efd6c0039e0 2024-04-25T10:02:35.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd8e800640 1 -- 172.21.15.5:0/2155653002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6c0039e0 msgr2=0x7efd6c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd8e800640 1 --2- 172.21.15.5:0/2155653002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd6c0039e0 0x7efd6c003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efd70009a90 tx=0x7efd6c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd87e00640 1 -- 172.21.15.5:0/2155653002 reap_dead start 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd8e800640 1 -- 172.21.15.5:0/2155653002 shutdown_connections 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd8e800640 1 -- 172.21.15.5:0/2155653002 >> 172.21.15.5:0/2155653002 conn(0x7efd8806ab60 msgr2=0x7efd8806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd8e800640 1 -- 172.21.15.5:0/2155653002 shutdown_connections 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.974+0000 7efd8e800640 1 -- 172.21.15.5:0/2155653002 wait complete. 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.976+0000 7efd8e800640 1 Processor -- start 2024-04-25T10:02:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.976+0000 7efd8e800640 1 -- start start 2024-04-25T10:02:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.977+0000 7efd8e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd6c0039e0 0x7efd880b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.977+0000 7efd8e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 0x7efd881e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.977+0000 7efd8e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd88159c50 0x7efd881e7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.977+0000 7efd8e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8815be50 con 0x7efd881411b0 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.977+0000 7efd8e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8806df40 con 0x7efd88159c50 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.977+0000 7efd8e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8806e560 con 0x7efd6c0039e0 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 0x7efd881e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 0x7efd881e5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46950/0 (socket says 172.21.15.5:46950) 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 -- 172.21.15.5:0/3303744896 learned_addr learned my addr 172.21.15.5:0/3303744896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd87400640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd6c0039e0 0x7efd880b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd87e00640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd88159c50 0x7efd881e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 -- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd6c0039e0 msgr2=0x7efd880b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd6c0039e0 0x7efd880b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 -- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd88159c50 msgr2=0x7efd881e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd88159c50 0x7efd881e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 -- 172.21.15.5:0/3303744896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880af690 con 0x7efd881411b0 2024-04-25T10:02:35.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.978+0000 7efd86a00640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 0x7efd881e5e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd78007a90 tx=0x7efd7800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.979+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800edf0 con 0x7efd881411b0 2024-04-25T10:02:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.979+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78018070 con 0x7efd881411b0 2024-04-25T10:02:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.979+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78013660 con 0x7efd881411b0 2024-04-25T10:02:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.979+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd880af970 con 0x7efd881411b0 2024-04-25T10:02:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.979+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd881e6370 con 0x7efd881411b0 2024-04-25T10:02:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.980+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd88068f60 con 0x7efd881411b0 2024-04-25T10:02:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.981+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd78013800 con 0x7efd881411b0 2024-04-25T10:02:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.982+0000 7efd7f400640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd6406b500 0x7efd6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.982+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd7808ace0 con 0x7efd881411b0 2024-04-25T10:02:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.982+0000 7efd87400640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd6406b500 0x7efd6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.984+0000 7efd87400640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd6406b500 0x7efd6406d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efd70033000 tx=0x7efd70008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:35.986+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd780590a0 con 0x7efd881411b0 2024-04-25T10:02:36.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.182+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd880b3360 con 0x7efd881411b0 2024-04-25T10:02:36.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.183+0000 7efd7f400640 1 -- 172.21.15.5:0/3303744896 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd7805d140 con 0x7efd881411b0 2024-04-25T10:02:36.187 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd6406b500 msgr2=0x7efd6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd6406b500 0x7efd6406d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efd70033000 tx=0x7efd70008030 comp rx=0 tx=0).stop 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 msgr2=0x7efd881e5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 0x7efd881e5e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd78007a90 tx=0x7efd7800e530 comp rx=0 tx=0).stop 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 shutdown_connections 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd6406b500 0x7efd6406d9c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd88159c50 0x7efd881e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd881411b0 0x7efd881e5e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 --2- 172.21.15.5:0/3303744896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd6c0039e0 0x7efd880b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 >> 172.21.15.5:0/3303744896 conn(0x7efd8806ab60 msgr2=0x7efd8813f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 shutdown_connections 2024-04-25T10:02:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.185+0000 7efd8e800640 1 -- 172.21.15.5:0/3303744896 wait complete. 2024-04-25T10:02:36.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.413+0000 7f898be00640 1 Processor -- start 2024-04-25T10:02:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.413+0000 7f898be00640 1 -- start start 2024-04-25T10:02:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f898be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f898be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f898c0d3f90 0x7f898c161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f898be00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898c1606e0 con 0x7f898c161bc0 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f898be00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c1609c0 con 0x7f898c0d3f90 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f898be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c160ca0 con 0x7f898c0b48c0 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f8982000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f898c0d3f90 0x7f898c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f8982000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f898c0d3f90 0x7f898c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46956/0 (socket says 172.21.15.5:46956) 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.415+0000 7f8982000640 1 -- 172.21.15.5:0/1728018761 learned_addr learned my addr 172.21.15.5:0/1728018761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.416+0000 7f898aa00640 1 --2- 172.21.15.5:0/1728018761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.416+0000 7f8982000640 1 -- 172.21.15.5:0/1728018761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 msgr2=0x7f898c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.416+0000 7f8982000640 1 --2- 172.21.15.5:0/1728018761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c15fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.416+0000 7f8982000640 1 -- 172.21.15.5:0/1728018761 >> v1:172.21.15.80:6789/0 conn(0x7f898c161bc0 legacy=0x7f898c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.416+0000 7f8982000640 1 -- 172.21.15.5:0/1728018761 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c160f80 con 0x7f898c0d3f90 2024-04-25T10:02:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.417+0000 7f8982000640 1 --2- 172.21.15.5:0/1728018761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c0d3f90 0x7f898c161680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8974009850 tx=0x7f897402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda062fe6307a8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.417+0000 7f898a000640 1 -- 172.21.15.5:0/1728018761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974004030 con 0x7f898c0d3f90 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898a000640 1 -- 172.21.15.5:0/1728018761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89740041d0 con 0x7f898c0d3f90 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898a000640 1 -- 172.21.15.5:0/1728018761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974003610 con 0x7f898c0d3f90 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898be00640 1 -- 172.21.15.5:0/1728018761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c0d3f90 msgr2=0x7f898c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898be00640 1 --2- 172.21.15.5:0/1728018761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c0d3f90 0x7f898c161680 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8974009850 tx=0x7f897402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898be00640 1 -- 172.21.15.5:0/1728018761 shutdown_connections 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898be00640 1 --2- 172.21.15.5:0/1728018761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c0d3f90 0x7f898c161680 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898be00640 1 --2- 172.21.15.5:0/1728018761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c15fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.418+0000 7f898be00640 1 -- 172.21.15.5:0/1728018761 >> 172.21.15.5:0/1728018761 conn(0x7f898c08af80 msgr2=0x7f898c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.419+0000 7f898be00640 1 -- 172.21.15.5:0/1728018761 shutdown_connections 2024-04-25T10:02:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.419+0000 7f898be00640 1 -- 172.21.15.5:0/1728018761 wait complete. 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.421+0000 7f898be00640 1 Processor -- start 2024-04-25T10:02:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.421+0000 7f898be00640 1 -- start start 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c14d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898c0d3f90 0x7f898c1e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 0x7f898c0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c1603a0 con 0x7f898c161bc0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0b4ca0 con 0x7f898c0d3f90 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0b3b80 con 0x7f898c0b48c0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 0x7f898c0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 0x7f898c0cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46970/0 (socket says 172.21.15.5:46970) 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.422+0000 7f898b400640 1 -- 172.21.15.5:0/1079885652 learned_addr learned my addr 172.21.15.5:0/1079885652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f8982000640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898c0d3f90 0x7f898c1e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898aa00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c14d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898b400640 1 -- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 msgr2=0x7f898c14d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898b400640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c14d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898b400640 1 -- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898c0d3f90 msgr2=0x7f898c1e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898b400640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898c0d3f90 0x7f898c1e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898b400640 1 -- 172.21.15.5:0/1079885652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0c93a0 con 0x7f898c161bc0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.423+0000 7f898b400640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 0x7f898c0cf6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f898400c390 tx=0x7f898400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.424+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984017070 con 0x7f898c161bc0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.424+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89840052d0 con 0x7f898c161bc0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.424+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984003810 con 0x7f898c161bc0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.424+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c0c9680 con 0x7f898c161bc0 2024-04-25T10:02:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.424+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c0c9c40 con 0x7f898c161bc0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.425+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c0c9f80 con 0x7f898c161bc0 2024-04-25T10:02:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.426+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f898401a070 con 0x7f898c161bc0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.428+0000 7f8983400640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.428+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f89840897f0 con 0x7f898c161bc0 2024-04-25T10:02:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.428+0000 7f898aa00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.429+0000 7f898aa00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896806b4d0 0x7f896806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f897c003b80 tx=0x7f897c0098d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.432+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8984057cd0 con 0x7f898c161bc0 2024-04-25T10:02:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.633+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f898c068f60 con 0x7f898c161bc0 2024-04-25T10:02:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.633+0000 7f8983400640 1 -- 172.21.15.5:0/1079885652 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f898405bd70 con 0x7f898c161bc0 2024-04-25T10:02:36.638 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896806b4d0 msgr2=0x7f896806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896806b4d0 0x7f896806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f897c003b80 tx=0x7f897c0098d0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 msgr2=0x7f898c0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 0x7f898c0cf6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f898400c390 tx=0x7f898400c850 comp rx=0 tx=0).stop 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 shutdown_connections 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f898c161bc0 0x7f898c0cf6c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f896806b4d0 0x7f896806d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f898c0d3f90 0x7f898c1e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 --2- 172.21.15.5:0/1079885652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f898c0b48c0 0x7f898c14d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 >> 172.21.15.5:0/1079885652 conn(0x7f898c08af80 msgr2=0x7f898c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 shutdown_connections 2024-04-25T10:02:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.636+0000 7f898be00640 1 -- 172.21.15.5:0/1079885652 wait complete. 2024-04-25T10:02:36.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.869+0000 7f5eb3200640 1 Processor -- start 2024-04-25T10:02:36.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.870+0000 7f5eb3200640 1 -- start start 2024-04-25T10:02:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.871+0000 7f5eb3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5eb41448f0 0x7f5eb40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb4144cd0 con 0x7f5eb415bc30 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb3200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb415f460 con 0x7f5eb41448f0 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb415c440 con 0x7f5eb415ce20 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5eb41448f0 0x7f5eb40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5eb41448f0 0x7f5eb40cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58554/0 (socket says 172.21.15.5:58554) 2024-04-25T10:02:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 -- 172.21.15.5:0/2691233026 learned_addr learned my addr 172.21.15.5:0/2691233026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 -- 172.21.15.5:0/2691233026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 msgr2=0x7f5eb415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb2800640 1 --2- 172.21.15.5:0/2691233026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 --2- 172.21.15.5:0/2691233026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb415d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 -- 172.21.15.5:0/2691233026 >> v1:172.21.15.80:6789/0 conn(0x7f5eb415bc30 legacy=0x7f5eb415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.872+0000 7f5eb1e00640 1 -- 172.21.15.5:0/2691233026 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb415c720 con 0x7f5eb41448f0 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.874+0000 7f5eb1e00640 1 --2- 172.21.15.5:0/2691233026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb41448f0 0x7f5eb40cdaa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009870 tx=0x7f5ea402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d298a98dfee8158e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.874+0000 7f5eb1400640 1 -- 172.21.15.5:0/2691233026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4037070 con 0x7f5eb41448f0 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.874+0000 7f5eb1400640 1 -- 172.21.15.5:0/2691233026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea4032440 con 0x7f5eb41448f0 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.874+0000 7f5eb1400640 1 -- 172.21.15.5:0/2691233026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4032810 con 0x7f5eb41448f0 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 -- 172.21.15.5:0/2691233026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb41448f0 msgr2=0x7f5eb40cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2691233026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb41448f0 0x7f5eb40cdaa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009870 tx=0x7f5ea402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 -- 172.21.15.5:0/2691233026 shutdown_connections 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2691233026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb415d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2691233026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb41448f0 0x7f5eb40cdaa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 -- 172.21.15.5:0/2691233026 >> 172.21.15.5:0/2691233026 conn(0x7f5eb4066f20 msgr2=0x7f5eb406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 -- 172.21.15.5:0/2691233026 shutdown_connections 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.875+0000 7f5eb3200640 1 -- 172.21.15.5:0/2691233026 wait complete. 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.877+0000 7f5eb3200640 1 Processor -- start 2024-04-25T10:02:36.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.877+0000 7f5eb3200640 1 -- start start 2024-04-25T10:02:36.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5eb41448f0 0x7f5eb41415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 0x7f5eb40acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb40ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4160000 con 0x7f5eb415bc30 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb415c0c0 con 0x7f5eb41448f0 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb4160180 con 0x7f5eb415ce20 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 0x7f5eb40acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 0x7f5eb40acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58568/0 (socket says 172.21.15.5:58568) 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eabe00640 1 -- 172.21.15.5:0/2960444519 learned_addr learned my addr 172.21.15.5:0/2960444519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb2800640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb40ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eb1e00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5eb41448f0 0x7f5eb41415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eabe00640 1 -- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 msgr2=0x7f5eb40ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.878+0000 7f5eabe00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb40ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eabe00640 1 -- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5eb41448f0 msgr2=0x7f5eb41415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eabe00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5eb41448f0 0x7f5eb41415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eabe00640 1 -- 172.21.15.5:0/2960444519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb41f3640 con 0x7f5eb415bc30 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eb1e00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5eb41448f0 0x7f5eb41415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eabe00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 0x7f5eb40acf40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c430 tx=0x7f5e9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c011790 con 0x7f5eb415bc30 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c017070 con 0x7f5eb415bc30 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb41e8670 con 0x7f5eb415bc30 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c011bb0 con 0x7f5eb415bc30 2024-04-25T10:02:36.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.879+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb41f3bc0 con 0x7f5eb415bc30 2024-04-25T10:02:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.881+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb41f3dc0 con 0x7f5eb415bc30 2024-04-25T10:02:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.882+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e9c01c460 con 0x7f5eb415bc30 2024-04-25T10:02:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.883+0000 7f5eaa000640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e9006b4d0 0x7f5e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.883+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5e9c08a350 con 0x7f5eb415bc30 2024-04-25T10:02:36.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.883+0000 7f5eb1e00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e9006b4d0 0x7f5e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.885+0000 7f5eb1e00640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e9006b4d0 0x7f5e9006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4034000 tx=0x7f5ea402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:36.887+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5eb41f3dc0 con 0x7f5eb415bc30 2024-04-25T10:02:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.089+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5eb4072180 con 0x7f5eb415bc30 2024-04-25T10:02:37.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.089+0000 7f5eaa000640 1 -- 172.21.15.5:0/2960444519 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e9c058e80 con 0x7f5eb415bc30 2024-04-25T10:02:37.094 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e9006b4d0 msgr2=0x7f5e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e9006b4d0 0x7f5e9006d990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4034000 tx=0x7f5ea402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 msgr2=0x7f5eb40acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 0x7f5eb40acf40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c430 tx=0x7f5e9c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 shutdown_connections 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5eb415ce20 0x7f5eb40ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5eb415bc30 0x7f5eb40acf40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e9006b4d0 0x7f5e9006d990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 --2- 172.21.15.5:0/2960444519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5eb41448f0 0x7f5eb41415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.092+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 >> 172.21.15.5:0/2960444519 conn(0x7f5eb4066f20 msgr2=0x7f5eb40ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.093+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 shutdown_connections 2024-04-25T10:02:37.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.093+0000 7f5eb3200640 1 -- 172.21.15.5:0/2960444519 wait complete. 2024-04-25T10:02:37.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:37.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.320+0000 7fe15b000640 1 Processor -- start 2024-04-25T10:02:37.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.321+0000 7fe15b000640 1 -- start start 2024-04-25T10:02:37.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe15b000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe15415b500 0x7fe1541651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe15b000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe154072010 con 0x7fe1540cc840 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe15b000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540ce570 con 0x7fe15406e840 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe15b000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15415b910 con 0x7fe15415b500 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe159c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe1540cc840 0x7fe15406e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46894/0 (socket says 172.21.15.5:46894) 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe159c00640 1 -- 172.21.15.5:0/1206865647 learned_addr learned my addr 172.21.15.5:0/1206865647 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.322+0000 7fe15a600640 1 --2- 172.21.15.5:0/1206865647 >> v2:172.21.15.5:3300/0 conn(0x7fe15415b500 0x7fe1541651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.323+0000 7fe153e00640 1 -- 172.21.15.5:0/1206865647 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365281499 0 0) 0x7fe154072010 con 0x7fe1540cc840 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.323+0000 7fe153e00640 1 -- 172.21.15.5:0/1206865647 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c005730 con 0x7fe1540cc840 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.323+0000 7fe15a600640 1 -- 172.21.15.5:0/1206865647 >> v1:172.21.15.80:6789/0 conn(0x7fe15406e840 legacy=0x7fe15415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.323+0000 7fe15a600640 1 -- 172.21.15.5:0/1206865647 >> v1:172.21.15.5:6789/0 conn(0x7fe1540cc840 legacy=0x7fe15406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.323+0000 7fe15a600640 1 -- 172.21.15.5:0/1206865647 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540ce850 con 0x7fe15415b500 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.324+0000 7fe15a600640 1 --2- 172.21.15.5:0/1206865647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541651a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009d90 tx=0x7fe14c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee78808f98a25eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.324+0000 7fe153e00640 1 -- 172.21.15.5:0/1206865647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c007740 con 0x7fe15415b500 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.324+0000 7fe153e00640 1 -- 172.21.15.5:0/1206865647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c0078e0 con 0x7fe15415b500 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.324+0000 7fe153e00640 1 -- 172.21.15.5:0/1206865647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c007cb0 con 0x7fe15415b500 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 -- 172.21.15.5:0/1206865647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 msgr2=0x7fe1541651a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 --2- 172.21.15.5:0/1206865647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541651a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009d90 tx=0x7fe14c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 -- 172.21.15.5:0/1206865647 shutdown_connections 2024-04-25T10:02:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 --2- 172.21.15.5:0/1206865647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541651a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 -- 172.21.15.5:0/1206865647 >> 172.21.15.5:0/1206865647 conn(0x7fe154066f20 msgr2=0x7fe15406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 -- 172.21.15.5:0/1206865647 shutdown_connections 2024-04-25T10:02:37.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.325+0000 7fe15b000640 1 -- 172.21.15.5:0/1206865647 wait complete. 2024-04-25T10:02:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.327+0000 7fe15b000640 1 Processor -- start 2024-04-25T10:02:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.327+0000 7fe15b000640 1 -- start start 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.328+0000 7fe15b000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe15406e840 0x7fe1540ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.328+0000 7fe15b000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1540cc840 0x7fe15414f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.328+0000 7fe15b000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.328+0000 7fe15b000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b2d20 con 0x7fe15415b500 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.328+0000 7fe15b000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540cee10 con 0x7fe1540cc840 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.328+0000 7fe15b000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540ce1d0 con 0x7fe15406e840 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58592/0 (socket says 172.21.15.5:58592) 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 -- 172.21.15.5:0/3299268455 learned_addr learned my addr 172.21.15.5:0/3299268455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe159c00640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe15406e840 0x7fe1540ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe159200640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1540cc840 0x7fe15414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 -- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe15406e840 msgr2=0x7fe1540ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe15406e840 0x7fe1540ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 -- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1540cc840 msgr2=0x7fe15414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1540cc840 0x7fe15414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 -- 172.21.15.5:0/3299268455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540ae130 con 0x7fe15415b500 2024-04-25T10:02:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.329+0000 7fe15a600640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541f66d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009eb0 tx=0x7fe14c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.330+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c03f070 con 0x7fe15415b500 2024-04-25T10:02:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.330+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c031b40 con 0x7fe15415b500 2024-04-25T10:02:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.330+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c044460 con 0x7fe15415b500 2024-04-25T10:02:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.330+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1540ad090 con 0x7fe15415b500 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.330+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1540ad930 con 0x7fe15415b500 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.331+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe154068f60 con 0x7fe15415b500 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.332+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe14c03b410 con 0x7fe15415b500 2024-04-25T10:02:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.334+0000 7fe151600640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe13406b4d0 0x7fe13406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.334+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe14c0abc40 con 0x7fe15415b500 2024-04-25T10:02:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.334+0000 7fe159c00640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe13406b4d0 0x7fe13406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.335+0000 7fe159c00640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe13406b4d0 0x7fe13406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe148005880 tx=0x7fe148009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.338+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe14c07a120 con 0x7fe15415b500 2024-04-25T10:02:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.537+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1540acda0 con 0x7fe15415b500 2024-04-25T10:02:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.538+0000 7fe151600640 1 -- 172.21.15.5:0/3299268455 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe14c0ab0a0 con 0x7fe15415b500 2024-04-25T10:02:37.543 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe13406b4d0 msgr2=0x7fe13406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe13406b4d0 0x7fe13406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe148005880 tx=0x7fe148009040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 msgr2=0x7fe1541f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541f66d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009eb0 tx=0x7fe14c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 shutdown_connections 2024-04-25T10:02:37.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe15415b500 0x7fe1541f66d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe13406b4d0 0x7fe13406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1540cc840 0x7fe15414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 --2- 172.21.15.5:0/3299268455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe15406e840 0x7fe1540ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.541+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 >> 172.21.15.5:0/3299268455 conn(0x7fe154066f20 msgr2=0x7fe154158600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.542+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 shutdown_connections 2024-04-25T10:02:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.542+0000 7fe15b000640 1 -- 172.21.15.5:0/3299268455 wait complete. 2024-04-25T10:02:37.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.771+0000 7f79c3e00640 1 Processor -- start 2024-04-25T10:02:37.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.771+0000 7f79c3e00640 1 -- start start 2024-04-25T10:02:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.773+0000 7f79c3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 0x7f79c40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.773+0000 7f79c3e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c415bb60 con 0x7f79c415b780 2024-04-25T10:02:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.773+0000 7f79c3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c40cc230 con 0x7f79c40cf8c0 2024-04-25T10:02:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.773+0000 7f79c3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40cc510 con 0x7f79c40cfd50 2024-04-25T10:02:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.773+0000 7f79c2a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f79c415b780 0x7f79c40b2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46896/0 (socket says 172.21.15.5:46896) 2024-04-25T10:02:37.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.773+0000 7f79c2a00640 1 -- 172.21.15.5:0/2385506632 learned_addr learned my addr 172.21.15.5:0/2385506632 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:37.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c3400640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 0x7f79c40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672488665 0 0) 0x7f79c415bb60 con 0x7f79c415b780 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79a8003520 con 0x7f79c415b780 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 258480306 0 0) 0x7f79a8003520 con 0x7f79c415b780 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 msgr2=0x7f79c40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 0x7f79c40cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 >> v1:172.21.15.80:6789/0 conn(0x7f79c40cf8c0 legacy=0x7f79c40cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40cc7f0 con 0x7f79c415b780 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f79b4001670 con 0x7f79c415b780 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 >> v1:172.21.15.5:6789/0 conn(0x7f79c415b780 legacy=0x7f79c40b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 0x7f79a80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a8003f60 0x7f79a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a8004d40 0x7f79a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40cc510 con 0x7f79a8004d40 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a800ff40 con 0x7f79a8003f60 2024-04-25T10:02:37.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.774+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8010140 con 0x7f79c40cfd50 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c3400640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a8004d40 0x7f79a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c2000640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a8003f60 0x7f79a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c2a00640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 0x7f79a80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c3400640 1 -- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 msgr2=0x7f79a80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c3400640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c40cfd50 0x7f79a80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c3400640 1 -- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a8003f60 msgr2=0x7f79a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c3400640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79a8003f60 0x7f79a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.775+0000 7f79c3400640 1 -- 172.21.15.5:0/2385506632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8003520 con 0x7f79a8004d40 2024-04-25T10:02:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.776+0000 7f79c3400640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a8004d40 0x7f79a80051c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009fc0 tx=0x7f79a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dba9aaa1518621f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.776+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc009c10 con 0x7f79a8004d40 2024-04-25T10:02:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.776+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc036070 con 0x7f79a8004d40 2024-04-25T10:02:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c1600640 1 -- 172.21.15.5:0/2385506632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc031880 con 0x7f79a8004d40 2024-04-25T10:02:37.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3e00640 1 -- 172.21.15.5:0/2385506632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a8004d40 msgr2=0x7f79a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3e00640 1 --2- 172.21.15.5:0/2385506632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79a8004d40 0x7f79a80051c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009fc0 tx=0x7f79a8005830 comp rx=0 tx=0).stop 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3400640 1 -- 172.21.15.5:0/2385506632 reap_dead start 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3e00640 1 -- 172.21.15.5:0/2385506632 shutdown_connections 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3e00640 1 -- 172.21.15.5:0/2385506632 >> 172.21.15.5:0/2385506632 conn(0x7f79c40d1470 msgr2=0x7f79c40d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3e00640 1 -- 172.21.15.5:0/2385506632 shutdown_connections 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.777+0000 7f79c3e00640 1 -- 172.21.15.5:0/2385506632 wait complete. 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.779+0000 7f79c3e00640 1 Processor -- start 2024-04-25T10:02:37.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.779+0000 7f79c3e00640 1 -- start start 2024-04-25T10:02:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.780+0000 7f79c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 0x7f79c40b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.780+0000 7f79c3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c406c2c0 0x7f79c41eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.780+0000 7f79c3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c4157560 0x7f79c41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.780+0000 7f79c3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40d0130 con 0x7f79c406bee0 2024-04-25T10:02:37.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.780+0000 7f79c3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40ccdb0 con 0x7f79c4157560 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.780+0000 7f79c3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40cbe50 con 0x7f79c406c2c0 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 0x7f79c40b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 0x7f79c40b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58606/0 (socket says 172.21.15.5:58606) 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 -- 172.21.15.5:0/8445187 learned_addr learned my addr 172.21.15.5:0/8445187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2000640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c406c2c0 0x7f79c41eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c3400640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c4157560 0x7f79c41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 -- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c406c2c0 msgr2=0x7f79c41eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c406c2c0 0x7f79c41eef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 -- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c4157560 msgr2=0x7f79c41579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c4157560 0x7f79c41579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 -- 172.21.15.5:0/8445187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c41ef470 con 0x7f79c406bee0 2024-04-25T10:02:37.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c3400640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c4157560 0x7f79c41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.781+0000 7f79c2a00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 0x7f79c40b4f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79b400eaa0 tx=0x7f79b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.782+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400c8a0 con 0x7f79c406bee0 2024-04-25T10:02:37.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.782+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4018070 con 0x7f79c406bee0 2024-04-25T10:02:37.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.782+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400cca0 con 0x7f79c406bee0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.782+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c41ef710 con 0x7f79c406bee0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.782+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c41efcd0 con 0x7f79c406bee0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.783+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c4068f60 con 0x7f79c406bee0 2024-04-25T10:02:37.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.784+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79b401d460 con 0x7f79c406bee0 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.785+0000 7f79baa00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79a006b4d0 0x7f79a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.785+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f79b408a3e0 con 0x7f79c406bee0 2024-04-25T10:02:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.786+0000 7f79c2000640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79a006b4d0 0x7f79a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.787+0000 7f79c2000640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79a006b4d0 0x7f79a006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79ac007fd0 tx=0x7f79ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:37.790+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79b4058fd0 con 0x7f79c406bee0 2024-04-25T10:02:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.034+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79c41596b0 con 0x7f79c406bee0 2024-04-25T10:02:38.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.035+0000 7f79baa00640 1 -- 172.21.15.5:0/8445187 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7f79b40588d0 con 0x7f79c406bee0 2024-04-25T10:02:38.039 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:38.039 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:38.040 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:02:38.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79a006b4d0 msgr2=0x7f79a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79a006b4d0 0x7f79a006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f79ac007fd0 tx=0x7f79ac009040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 msgr2=0x7f79c40b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 0x7f79c40b4f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79b400eaa0 tx=0x7f79b400ef60 comp rx=0 tx=0).stop 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 shutdown_connections 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79a006b4d0 0x7f79a006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c4157560 0x7f79c41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79c406c2c0 0x7f79c41eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 --2- 172.21.15.5:0/8445187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79c406bee0 0x7f79c40b4f60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 >> 172.21.15.5:0/8445187 conn(0x7f79c40d1470 msgr2=0x7f79c4159eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 shutdown_connections 2024-04-25T10:02:38.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.038+0000 7f79c3e00640 1 -- 172.21.15.5:0/8445187 wait complete. 2024-04-25T10:02:38.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.271+0000 7f81c3400640 1 Processor -- start 2024-04-25T10:02:38.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.271+0000 7f81c3400640 1 -- start start 2024-04-25T10:02:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c3400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc071ed0 0x7f81bc0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f81bc0ce890 0x7f81bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c3400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc15bdc0 con 0x7f81bc1529e0 2024-04-25T10:02:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c3400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0b2340 con 0x7f81bc0ce890 2024-04-25T10:02:38.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c3400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc15c060 con 0x7f81bc071ed0 2024-04-25T10:02:38.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f81bc0ce890 0x7f81bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f81bc0ce890 0x7f81bc06d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58614/0 (socket says 172.21.15.5:58614) 2024-04-25T10:02:38.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81c1400640 1 -- 172.21.15.5:0/532026502 learned_addr learned my addr 172.21.15.5:0/532026502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:38.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.273+0000 7f81b3e00640 1 --2- 172.21.15.5:0/532026502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc071ed0 0x7f81bc0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.274+0000 7f81c1400640 1 -- 172.21.15.5:0/532026502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc071ed0 msgr2=0x7f81bc0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.274+0000 7f81c1400640 1 --2- 172.21.15.5:0/532026502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc071ed0 0x7f81bc0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.274+0000 7f81c1400640 1 -- 172.21.15.5:0/532026502 >> v1:172.21.15.80:6789/0 conn(0x7f81bc1529e0 legacy=0x7f81bc0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.274+0000 7f81c1400640 1 -- 172.21.15.5:0/532026502 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc15c340 con 0x7f81bc0ce890 2024-04-25T10:02:38.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.275+0000 7f81c1400640 1 --2- 172.21.15.5:0/532026502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc06d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009e50 tx=0x7f81b402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fcbec984b0ff79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.275+0000 7f81bbe00640 1 -- 172.21.15.5:0/532026502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4036070 con 0x7f81bc0ce890 2024-04-25T10:02:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.275+0000 7f81bbe00640 1 -- 172.21.15.5:0/532026502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b4031600 con 0x7f81bc0ce890 2024-04-25T10:02:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.275+0000 7f81bbe00640 1 -- 172.21.15.5:0/532026502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40319d0 con 0x7f81bc0ce890 2024-04-25T10:02:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 -- 172.21.15.5:0/532026502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 msgr2=0x7f81bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 --2- 172.21.15.5:0/532026502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc06d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009e50 tx=0x7f81b402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 -- 172.21.15.5:0/532026502 shutdown_connections 2024-04-25T10:02:38.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 --2- 172.21.15.5:0/532026502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc06d150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 --2- 172.21.15.5:0/532026502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc071ed0 0x7f81bc0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 -- 172.21.15.5:0/532026502 >> 172.21.15.5:0/532026502 conn(0x7f81bc071150 msgr2=0x7f81bc071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 -- 172.21.15.5:0/532026502 shutdown_connections 2024-04-25T10:02:38.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.276+0000 7f81c3400640 1 -- 172.21.15.5:0/532026502 wait complete. 2024-04-25T10:02:38.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.278+0000 7f81c3400640 1 Processor -- start 2024-04-25T10:02:38.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.278+0000 7f81c3400640 1 -- start start 2024-04-25T10:02:38.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c3400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81bc071ed0 0x7f81bc156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c3400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc15ee30 0x7f81bc15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c3400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c65e0 con 0x7f81bc0ce890 2024-04-25T10:02:38.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c3400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc15c900 con 0x7f81bc071ed0 2024-04-25T10:02:38.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c3400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc06d690 con 0x7f81bc15ee30 2024-04-25T10:02:38.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc146000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58620/0 (socket says 172.21.15.5:58620) 2024-04-25T10:02:38.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81b3e00640 1 -- 172.21.15.5:0/3612024290 learned_addr learned my addr 172.21.15.5:0/3612024290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c1400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc15ee30 0x7f81bc15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.279+0000 7f81c0a00640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81bc071ed0 0x7f81bc156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b3e00640 1 -- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc15ee30 msgr2=0x7f81bc15f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b3e00640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc15ee30 0x7f81bc15f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b3e00640 1 -- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81bc071ed0 msgr2=0x7f81bc156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b3e00640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81bc071ed0 0x7f81bc156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b3e00640 1 -- 172.21.15.5:0/3612024290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc1572a0 con 0x7f81bc0ce890 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b3e00640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc146000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d7d0 tx=0x7f81a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400b6c0 con 0x7f81bc0ce890 2024-04-25T10:02:38.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4019070 con 0x7f81bc0ce890 2024-04-25T10:02:38.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400bac0 con 0x7f81bc0ce890 2024-04-25T10:02:38.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc146540 con 0x7f81bc0ce890 2024-04-25T10:02:38.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.280+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc157820 con 0x7f81bc0ce890 2024-04-25T10:02:38.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.282+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc068fd0 con 0x7f81bc0ce890 2024-04-25T10:02:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.283+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81a400bc60 con 0x7f81bc0ce890 2024-04-25T10:02:38.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.283+0000 7f81b9600640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.284+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f81a408aa40 con 0x7f81bc0ce890 2024-04-25T10:02:38.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.284+0000 7f81c0a00640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.285+0000 7f81c0a00640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f819806b4d0 0x7f819806d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81ac006fd0 tx=0x7f81ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.288+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81a4059580 con 0x7f81bc0ce890 2024-04-25T10:02:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.492+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81bc06c1a0 con 0x7f81bc0ce890 2024-04-25T10:02:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.494+0000 7f81b9600640 1 -- 172.21.15.5:0/3612024290 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f81a4058e80 con 0x7f81bc0ce890 2024-04-25T10:02:38.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:38.499 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:02.841579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f819806b4d0 msgr2=0x7f819806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f819806b4d0 0x7f819806d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81ac006fd0 tx=0x7f81ac008040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 msgr2=0x7f81bc146000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc146000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d7d0 tx=0x7f81a400dc90 comp rx=0 tx=0).stop 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 shutdown_connections 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81bc15ee30 0x7f81bc15f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc0ce890 0x7f81bc146000 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f819806b4d0 0x7f819806d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 --2- 172.21.15.5:0/3612024290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81bc071ed0 0x7f81bc156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 >> 172.21.15.5:0/3612024290 conn(0x7f81bc071150 msgr2=0x7f81bc15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 shutdown_connections 2024-04-25T10:02:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.497+0000 7f81c3400640 1 -- 172.21.15.5:0/3612024290 wait complete. 2024-04-25T10:02:38.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.728+0000 7ffa6a200640 1 Processor -- start 2024-04-25T10:02:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.728+0000 7ffa6a200640 1 -- start start 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.730+0000 7ffa6a200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa64154a10 0x7ffa6415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.730+0000 7ffa6a200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa640cdca0 0x7ffa640b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.730+0000 7ffa6a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa640713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.730+0000 7ffa6a200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa64071900 con 0x7ffa64154a10 2024-04-25T10:02:38.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.730+0000 7ffa6a200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa640b4890 con 0x7ffa640cdca0 2024-04-25T10:02:38.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.730+0000 7ffa6a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640cd200 con 0x7ffa64070f90 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa64154a10 0x7ffa6415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa64154a10 0x7ffa6415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58626/0 (socket says 172.21.15.5:58626) 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 -- 172.21.15.5:0/400768885 learned_addr learned my addr 172.21.15.5:0/400768885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa63e00640 1 --2- 172.21.15.5:0/400768885 >> v2:172.21.15.80:3300/0 conn(0x7ffa640cdca0 0x7ffa640b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa69800640 1 --2- 172.21.15.5:0/400768885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa640713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 -- 172.21.15.5:0/400768885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 msgr2=0x7ffa640713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 --2- 172.21.15.5:0/400768885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa640713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 -- 172.21.15.5:0/400768885 >> v2:172.21.15.80:3300/0 conn(0x7ffa640cdca0 msgr2=0x7ffa640b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 --2- 172.21.15.5:0/400768885 >> v2:172.21.15.80:3300/0 conn(0x7ffa640cdca0 0x7ffa640b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.731+0000 7ffa68e00640 1 -- 172.21.15.5:0/400768885 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6406e6f0 con 0x7ffa64154a10 2024-04-25T10:02:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.733+0000 7ffa68e00640 1 --2- 172.21.15.5:0/400768885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa6415e8c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa640aef00 tx=0x7ffa5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cae4a69ba7788c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.733+0000 7ffa63400640 1 -- 172.21.15.5:0/400768885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58037070 con 0x7ffa64154a10 2024-04-25T10:02:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.733+0000 7ffa63400640 1 -- 172.21.15.5:0/400768885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58032440 con 0x7ffa64154a10 2024-04-25T10:02:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.733+0000 7ffa63400640 1 -- 172.21.15.5:0/400768885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58032810 con 0x7ffa64154a10 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 -- 172.21.15.5:0/400768885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 msgr2=0x7ffa6415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 --2- 172.21.15.5:0/400768885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa6415e8c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa640aef00 tx=0x7ffa5802a140 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 -- 172.21.15.5:0/400768885 shutdown_connections 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 --2- 172.21.15.5:0/400768885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa640713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 --2- 172.21.15.5:0/400768885 >> v2:172.21.15.80:3300/0 conn(0x7ffa640cdca0 0x7ffa640b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 --2- 172.21.15.5:0/400768885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa6415e8c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 -- 172.21.15.5:0/400768885 >> 172.21.15.5:0/400768885 conn(0x7ffa64066f20 msgr2=0x7ffa6406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.734+0000 7ffa6a200640 1 -- 172.21.15.5:0/400768885 shutdown_connections 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.735+0000 7ffa6a200640 1 -- 172.21.15.5:0/400768885 wait complete. 2024-04-25T10:02:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.737+0000 7ffa6a200640 1 Processor -- start 2024-04-25T10:02:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.737+0000 7ffa6a200640 1 -- start start 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa6a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa641587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa6a200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa640cdca0 0x7ffa6415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa6a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa64155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa6a200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640cd4a0 con 0x7ffa64154a10 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa6a200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6406ecb0 con 0x7ffa640cdca0 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa6a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640cd780 con 0x7ffa64070f90 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa69800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa64155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa69800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa64155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58632/0 (socket says 172.21.15.5:58632) 2024-04-25T10:02:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.738+0000 7ffa69800640 1 -- 172.21.15.5:0/3598584133 learned_addr learned my addr 172.21.15.5:0/3598584133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa68e00640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa641587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa63e00640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa640cdca0 0x7ffa6415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa69800640 1 -- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 msgr2=0x7ffa641587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa69800640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa641587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa69800640 1 -- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa640cdca0 msgr2=0x7ffa6415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa69800640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa640cdca0 0x7ffa6415e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa69800640 1 -- 172.21.15.5:0/3598584133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6415f2c0 con 0x7ffa64154a10 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa68e00640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa641587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.739+0000 7ffa69800640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa64155bd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007e60 tx=0x7ffa5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.740+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c014620 con 0x7ffa64154a10 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.740+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c019070 con 0x7ffa64154a10 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.740+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c014a20 con 0x7ffa64154a10 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.740+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6415f5a0 con 0x7ffa64154a10 2024-04-25T10:02:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.740+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6415fb60 con 0x7ffa64154a10 2024-04-25T10:02:38.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.741+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6406b090 con 0x7ffa64154a10 2024-04-25T10:02:38.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.742+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa5c014bc0 con 0x7ffa64154a10 2024-04-25T10:02:38.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.743+0000 7ffa60c00640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4406b4d0 0x7ffa4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.743+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffa5c08ae00 con 0x7ffa64154a10 2024-04-25T10:02:38.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.743+0000 7ffa68e00640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4406b4d0 0x7ffa4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.745+0000 7ffa68e00640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4406b4d0 0x7ffa4406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa58034000 tx=0x7ffa5802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.749+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa6406b090 con 0x7ffa64154a10 2024-04-25T10:02:38.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.988+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6406d7c0 con 0x7ffa64154a10 2024-04-25T10:02:38.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.990+0000 7ffa60c00640 1 -- 172.21.15.5:0/3598584133 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7ffa5c0598a0 con 0x7ffa64154a10 2024-04-25T10:02:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:38.995 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:02:38.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4406b4d0 msgr2=0x7ffa4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4406b4d0 0x7ffa4406d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa58034000 tx=0x7ffa5802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 msgr2=0x7ffa64155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa64155bd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c007e60 tx=0x7ffa5c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 shutdown_connections 2024-04-25T10:02:38.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa64154a10 0x7ffa64155bd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.992+0000 7ffa6a200640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4406b4d0 0x7ffa4406d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.993+0000 7ffa6a200640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa640cdca0 0x7ffa6415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.993+0000 7ffa6a200640 1 --2- 172.21.15.5:0/3598584133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa64070f90 0x7ffa641587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.993+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 >> 172.21.15.5:0/3598584133 conn(0x7ffa64066f20 msgr2=0x7ffa640b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.993+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 shutdown_connections 2024-04-25T10:02:38.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:38.993+0000 7ffa6a200640 1 -- 172.21.15.5:0/3598584133 wait complete. 2024-04-25T10:02:39.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:01:57.496056+0000', 'modified': '2024-04-25T10:02:23.941309+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6837/312213316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 312213316}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 312213316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:39.420 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:39.421 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:02:39.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.643+0000 7faeb1e00640 1 Processor -- start 2024-04-25T10:02:39.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.643+0000 7faeb1e00640 1 -- start start 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.645+0000 7faeb1e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faeac0cdca0 0x7faeac15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.645+0000 7faeb1e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac0cd200 con 0x7faeac070f90 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.645+0000 7faeb1e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac06e6f0 con 0x7faeac154a10 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.645+0000 7faeb1e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac06e9d0 con 0x7faeac0cdca0 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.645+0000 7faeb1400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faeac070f90 0x7faeac06cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46904/0 (socket says 172.21.15.5:46904) 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.645+0000 7faeb1400640 1 -- 172.21.15.5:0/4209761069 learned_addr learned my addr 172.21.15.5:0/4209761069 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeabe00640 1 --2- 172.21.15.5:0/4209761069 >> v2:172.21.15.80:3300/0 conn(0x7faeac0cdca0 0x7faeac15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623969543 0 0) 0x7faeac0cd200 con 0x7faeac070f90 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae94003520 con 0x7faeac070f90 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4053801404 0 0) 0x7fae94003520 con 0x7faeac070f90 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 >> v2:172.21.15.80:3300/0 conn(0x7faeac0cdca0 msgr2=0x7faeac15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 --2- 172.21.15.5:0/4209761069 >> v2:172.21.15.80:3300/0 conn(0x7faeac0cdca0 0x7faeac15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 >> v1:172.21.15.80:6789/0 conn(0x7faeac154a10 legacy=0x7faeac0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac06ecb0 con 0x7faeac070f90 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7faea4001a10 con 0x7faeac070f90 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 >> v1:172.21.15.5:6789/0 conn(0x7faeac070f90 legacy=0x7faeac06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940039b0 0x7fae94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae94004800 0x7fae94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae940051c0 0x7fae9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac06e9d0 con 0x7fae940051c0 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9400c160 con 0x7fae940039b0 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.646+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9400c360 con 0x7fae94004800 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb1400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae940051c0 0x7fae9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb0a00640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940039b0 0x7fae94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeabe00640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae94004800 0x7fae94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb1400640 1 -- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae94004800 msgr2=0x7fae94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb1400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae94004800 0x7fae94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb1400640 1 -- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940039b0 msgr2=0x7fae94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb1400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940039b0 0x7fae94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.647+0000 7faeb1400640 1 -- 172.21.15.5:0/4209761069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94003520 con 0x7fae940051c0 2024-04-25T10:02:39.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.648+0000 7faeb1400640 1 -- 172.21.15.5:0/4209761069 reap_dead start 2024-04-25T10:02:39.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.648+0000 7faeb1400640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae940051c0 0x7fae9400b9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faea400a850 tx=0x7fae94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4c415069787704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.648+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400a4a0 con 0x7fae940051c0 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.648+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea402f070 con 0x7fae940051c0 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.648+0000 7faeab400640 1 -- 172.21.15.5:0/4209761069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea402bcb0 con 0x7fae940051c0 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 -- 172.21.15.5:0/4209761069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae940051c0 msgr2=0x7fae9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae940051c0 0x7fae9400b9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faea400a850 tx=0x7fae94005830 comp rx=0 tx=0).stop 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 -- 172.21.15.5:0/4209761069 shutdown_connections 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 --2- 172.21.15.5:0/4209761069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae940051c0 0x7fae9400b9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 -- 172.21.15.5:0/4209761069 >> 172.21.15.5:0/4209761069 conn(0x7faeac066f20 msgr2=0x7faeac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 -- 172.21.15.5:0/4209761069 shutdown_connections 2024-04-25T10:02:39.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.649+0000 7faeb1e00640 1 -- 172.21.15.5:0/4209761069 wait complete. 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.651+0000 7faeb1e00640 1 Processor -- start 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.651+0000 7faeb1e00640 1 -- start start 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940051c0 0x7faeac15ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeac15c3c0 0x7faeac1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 0x7faeac155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac15e8c0 con 0x7faeac1559b0 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cd780 con 0x7fae940051c0 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac15ea40 con 0x7faeac15c3c0 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.652+0000 7faeb1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 0x7faeac155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 0x7faeac155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58654/0 (socket says 172.21.15.5:58654) 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 -- 172.21.15.5:0/3423505472 learned_addr learned my addr 172.21.15.5:0/3423505472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb0a00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940051c0 0x7faeac15ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeabe00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeac15c3c0 0x7faeac1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 -- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeac15c3c0 msgr2=0x7faeac1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeac15c3c0 0x7faeac1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 -- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940051c0 msgr2=0x7faeac15ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940051c0 0x7faeac15ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 -- 172.21.15.5:0/3423505472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac15f4e0 con 0x7faeac1559b0 2024-04-25T10:02:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.653+0000 7faeb1400640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 0x7faeac155e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faea402b500 tx=0x7faea402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.654+0000 7faea8c00640 1 -- 172.21.15.5:0/3423505472 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea402bc60 con 0x7faeac1559b0 2024-04-25T10:02:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.654+0000 7faea8c00640 1 -- 172.21.15.5:0/3423505472 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea402be00 con 0x7faeac1559b0 2024-04-25T10:02:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.654+0000 7faea8c00640 1 -- 172.21.15.5:0/3423505472 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4034690 con 0x7faeac1559b0 2024-04-25T10:02:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.654+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac15f7c0 con 0x7faeac1559b0 2024-04-25T10:02:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.654+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac15fd80 con 0x7faeac1559b0 2024-04-25T10:02:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.656+0000 7faea8c00640 1 -- 172.21.15.5:0/3423505472 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faea4034830 con 0x7faeac1559b0 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.658+0000 7faea8c00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae8c06b500 0x7fae8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.658+0000 7faea8c00640 1 -- 172.21.15.5:0/3423505472 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6590+0+0 (secure 0 0 0) 0x7faea40aa830 con 0x7faeac1559b0 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.658+0000 7faeb0a00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae8c06b500 0x7fae8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.659+0000 7faeb6216740 1 Processor -- start 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.659+0000 7faeb6216740 1 -- start start 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae8c06b500 0x7fae8c06d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faeac160a80 tx=0x7faea0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb6216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 0x564e991b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991b3eb0 0x564e9918bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e9918a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb6216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e990ef620 con 0x564e9918bb30 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e99098ca0 con 0x564e991b3eb0 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e990eb6c0 con 0x564e991d47a0 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 0x564e991b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 0x564e991b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58670/0 (socket says 172.21.15.5:58670) 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 -- 172.21.15.5:0/1346513508 learned_addr learned my addr 172.21.15.5:0/1346513508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb1400640 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e9918a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 -- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 msgr2=0x564e9918a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeabe00640 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991b3eb0 0x564e9918bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e9918a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.660+0000 7faeb0a00640 1 -- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991b3eb0 msgr2=0x564e9918bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.661+0000 7faeb0a00640 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991b3eb0 0x564e9918bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.661+0000 7faeb0a00640 1 -- 172.21.15.5:0/1346513508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e9909ff40 con 0x564e9918bb30 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.661+0000 7faeb0a00640 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 0x564e991b3970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faea0005620 tx=0x7faea0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d498e1f9b332be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.661+0000 7fae9d200640 1 -- 172.21.15.5:0/1346513508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0027070 con 0x564e9918bb30 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.661+0000 7fae9d200640 1 -- 172.21.15.5:0/1346513508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea0014780 con 0x564e9918bb30 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.661+0000 7fae9d200640 1 -- 172.21.15.5:0/1346513508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0014b50 con 0x564e9918bb30 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 -- 172.21.15.5:0/1346513508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 msgr2=0x564e991b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 0x564e991b3970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faea0005620 tx=0x7faea0013000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 -- 172.21.15.5:0/1346513508 shutdown_connections 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e9918a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991b3eb0 0x564e9918bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 --2- 172.21.15.5:0/1346513508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e9918bb30 0x564e991b3970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 -- 172.21.15.5:0/1346513508 >> 172.21.15.5:0/1346513508 conn(0x564e99188d50 msgr2=0x564e991d1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 -- 172.21.15.5:0/1346513508 shutdown_connections 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.662+0000 7faeb6216740 1 -- 172.21.15.5:0/1346513508 wait complete. 2024-04-25T10:02:39.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.663+0000 7faeb6216740 1 -- start start 2024-04-25T10:02:39.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.663+0000 7faeb6216740 1 Processor -- start 2024-04-25T10:02:39.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e991ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991aeb20 0x564e991b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e991b34e0 con 0x564e991b3eb0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e9908e1f0 con 0x564e991d47a0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e991b2ed0 con 0x564e991aeb20 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb6216740 10 client.0 _ll_register_callbacks cb 0x564e990abad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58678/0 (socket says 172.21.15.5:58678) 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.664+0000 7faeb0a00640 1 -- 172.21.15.5:0/1590524982 learned_addr learned my addr 172.21.15.5:0/1590524982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb1400640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991aeb20 0x564e991b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeabe00640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e991ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb0a00640 1 -- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991aeb20 msgr2=0x564e991b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb0a00640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991aeb20 0x564e991b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb0a00640 1 -- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 msgr2=0x564e991ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb0a00640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e991ae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb0a00640 1 -- 172.21.15.5:0/1590524982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e990adc50 con 0x564e991b3eb0 2024-04-25T10:02:39.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb1400640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991aeb20 0x564e991b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7faeb0a00640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b7430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faea00055f0 tx=0x7faea0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293fa8b7ffbd9acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7fae91600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.665+0000 7fae91600640 1 -- 172.21.15.5:0/1590524982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0032060 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7fae91600640 1 -- 172.21.15.5:0/1590524982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6590+0+0 (secure 0 0 0) 0x7faea00072f0 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faea0014780 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7fae91600640 1 -- 172.21.15.5:0/1590524982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea0027040 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 10 client.4815 fetch_fsmap learned FSMap version 8 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564e98e042d0 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7fae91600640 1 -- 172.21.15.5:0/1590524982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea0014780 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7fae91600640 1 -- 172.21.15.5:0/1590524982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7faea00074f0 con 0x564e991b3eb0 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 10 client.4815 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 10 client.4815 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 20 client.4815 populate_metadata read hostname 'smithi005' 2024-04-25T10:02:39.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e991edb50 0x564e991eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 4 client.4815 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.666+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 --> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e990fcb60 con 0x564e991edb50 2024-04-25T10:02:39.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.667+0000 7faeabe00640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e991edb50 0x564e991eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.668+0000 7faeabe00640 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e991edb50 0x564e991eff40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da76b95b672ed787 server_cookie=af3f0c7a6b9f0bdc in_seq=0 out_seq=0 2024-04-25T10:02:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.668+0000 7fae91600640 10 client.4815 ms_handle_connect on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7fae91600640 1 -- 172.21.15.5:0/1590524982 <== mds.0 v2:172.21.15.80:6836/312213316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e990fcb60 con 0x564e991edb50 2024-04-25T10:02:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7fae91600640 10 client.4815 handle_command_reply: tid=0 2024-04-25T10:02:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 client.4815 shutdown 2024-04-25T10:02:39.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 msgr2=0x564e991b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b7430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faea00055f0 tx=0x7faea0001190 comp rx=0 tx=0).stop 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 shutdown_connections 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e991edb50 0x564e991eff40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991aeb20 0x564e991b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7fae91600640 0 client.4815 ms_handle_reset on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e991ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 --2- 172.21.15.5:0/1590524982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b7430 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 >> 172.21.15.5:0/1590524982 conn(0x564e9918bb30 msgr2=0x564e9918a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 shutdown_connections 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.670+0000 7faeb6216740 1 -- 172.21.15.5:0/1590524982 wait complete. 2024-04-25T10:02:39.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.671+0000 7faeb6216740 20 client.4815 trim_cache size 0 max 16384 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.688+0000 7faeb6216740 1 Processor -- start 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.688+0000 7faeb6216740 1 -- start start 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991aeb20 0x564e991d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb6216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e991b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb6216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac200950 con 0x564e991b3eb0 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac1ffd80 con 0x564e991aeb20 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac200350 con 0x564e991d47a0 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58682/0 (socket says 172.21.15.5:58682) 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991aeb20 0x564e991d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeabe00640 1 -- 172.21.15.5:0/3614024993 learned_addr learned my addr 172.21.15.5:0/3614024993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.689+0000 7faeb1400640 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e991b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7faeabe00640 1 -- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 msgr2=0x564e991b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7faeabe00640 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e991b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7faeabe00640 1 -- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991aeb20 msgr2=0x564e991d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7faeabe00640 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991aeb20 0x564e991d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7faeabe00640 1 -- 172.21.15.5:0/3614024993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e98ece6b0 con 0x564e991b3eb0 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7faeabe00640 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b4290 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fae98009ca0 tx=0x7fae98008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb5abe4c37c4b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7fae93e00640 1 -- 172.21.15.5:0/3614024993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98023070 con 0x564e991b3eb0 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7fae93e00640 1 -- 172.21.15.5:0/3614024993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9801fb70 con 0x564e991b3eb0 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.690+0000 7fae93e00640 1 -- 172.21.15.5:0/3614024993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98004020 con 0x564e991b3eb0 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 -- 172.21.15.5:0/3614024993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 msgr2=0x564e991b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b4290 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fae98009ca0 tx=0x7fae98008040 comp rx=0 tx=0).stop 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 -- 172.21.15.5:0/3614024993 shutdown_connections 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991d47a0 0x564e991b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991b3eb0 0x564e991b4290 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 --2- 172.21.15.5:0/3614024993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991aeb20 0x564e991d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 -- 172.21.15.5:0/3614024993 >> 172.21.15.5:0/3614024993 conn(0x564e9918bb30 msgr2=0x564e99182200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 -- 172.21.15.5:0/3614024993 shutdown_connections 2024-04-25T10:02:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.691+0000 7faeb6216740 1 -- 172.21.15.5:0/3614024993 wait complete. 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.692+0000 7faeb6216740 1 -- start start 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.692+0000 7faeb6216740 1 Processor -- start 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 0x564e991b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991b3eb0 0x564e991b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e9923c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e991b5ac0 con 0x564e991aeb20 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e991b1ea0 con 0x564e991d47a0 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564e991b5780 con 0x564e991b3eb0 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.693+0000 7faeb6216740 10 client.0 _ll_register_callbacks cb 0x7faea0025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 0x564e991b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 0x564e991b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58690/0 (socket says 172.21.15.5:58690) 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 -- 172.21.15.5:0/3151723553 learned_addr learned my addr 172.21.15.5:0/3151723553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb1400640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e9923c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeabe00640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991b3eb0 0x564e991b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 -- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991b3eb0 msgr2=0x564e991b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991b3eb0 0x564e991b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 -- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 msgr2=0x564e9923c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e9923c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 -- 172.21.15.5:0/3151723553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e991f0530 con 0x564e991aeb20 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb1400640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e9923c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7faeb0a00640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 0x564e991b4290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faea0015fd0 tx=0x7faea000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b1ac8832a7c8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.694+0000 7fae91600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7fae91600640 1 -- 172.21.15.5:0/3151723553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0045060 con 0x564e991aeb20 2024-04-25T10:02:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7fae91600640 1 -- 172.21.15.5:0/3151723553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6590+0+0 (secure 0 0 0) 0x7faea0040460 con 0x564e991aeb20 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae9801fb70 con 0x564e991aeb20 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7fae91600640 1 -- 172.21.15.5:0/3151723553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7faea0032050 con 0x564e991aeb20 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 10 client.4821 fetch_fsmap learned FSMap version 8 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x564e9923d970 con 0x564e991aeb20 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7fae91600640 1 -- 172.21.15.5:0/3151723553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9801fb70 con 0x564e991aeb20 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7fae91600640 1 -- 172.21.15.5:0/3151723553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7faea0046280 con 0x564e991aeb20 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 10 client.4821 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 10 client.4821 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 20 client.4821 populate_metadata read hostname 'smithi005' 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e9923cdc0 0x564e9918bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 4 client.4821 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.695+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 --> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564e990f6eb0 con 0x564e9923cdc0 2024-04-25T10:02:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.696+0000 7faeabe00640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e9923cdc0 0x564e9918bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.697+0000 7faeabe00640 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e9923cdc0 0x564e9918bf10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f1b8e5c37d0ee36 server_cookie=2b86d5c0945bfb17 in_seq=0 out_seq=0 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.697+0000 7fae91600640 10 client.4821 ms_handle_connect on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7fae91600640 1 -- 172.21.15.5:0/3151723553 <== mds.0 v2:172.21.15.80:6836/312213316 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x564e990f6eb0 con 0x564e9923cdc0 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7fae91600640 10 client.4821 handle_command_reply: tid=0 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 client.4821 shutdown 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 msgr2=0x564e991b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 0x564e991b4290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faea0015fd0 tx=0x7faea000c000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 shutdown_connections 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:6836/312213316,v1:172.21.15.80:6837/312213316] conn(0x564e9923cdc0 0x564e9918bf10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564e991d47a0 0x564e9923c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564e991b3eb0 0x564e991b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 --2- 172.21.15.5:0/3151723553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e991aeb20 0x564e991b4290 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 >> 172.21.15.5:0/3151723553 conn(0x564e9918bb30 msgr2=0x564e991b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7fae91600640 0 client.4821 ms_handle_reset on v2:172.21.15.80:6836/312213316 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 shutdown_connections 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 1 -- 172.21.15.5:0/3151723553 wait complete. 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.698+0000 7faeb6216740 20 client.4821 trim_cache size 0 max 16384 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4282, 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:02:39.706 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 41.596839299999999, 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 8, 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 27, 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 46.908445751000002 2024-04-25T10:02:39.707 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae8c06b500 msgr2=0x7fae8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae8c06b500 0x7fae8c06d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faeac160a80 tx=0x7faea0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 msgr2=0x7faeac155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 0x7faeac155e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faea402b500 tx=0x7faea402d070 comp rx=0 tx=0).stop 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 shutdown_connections 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeac1559b0 0x7faeac155e30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeac15c3c0 0x7faeac1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae8c06b500 0x7fae8c06d9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 --2- 172.21.15.5:0/3423505472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae940051c0 0x7faeac15ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 >> 172.21.15.5:0/3423505472 conn(0x7faeac066f20 msgr2=0x7faeac0713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 shutdown_connections 2024-04-25T10:02:39.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:39.699+0000 7faeb1e00640 1 -- 172.21.15.5:0/3423505472 wait complete. 2024-04-25T10:02:39.722 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:02:39.736 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:02:39.736 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:02:39.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.736 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:02:39.787 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T10:02:39.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:39.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.005 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:02:40.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.176 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:40.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:40.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.180 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T10:02:40.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.241 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.332 INFO:teuthology.orchestra.run.smithi080.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:40.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:40.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.336 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T10:02:40.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.386 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.472 INFO:teuthology.orchestra.run.smithi080.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:40.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:40.476 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T10:02:40.476 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:40.476 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:40.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.699+0000 7fae95e00640 1 Processor -- start 2024-04-25T10:02:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.699+0000 7fae95e00640 1 -- start start 2024-04-25T10:02:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae95e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae900cdca0 0x7fae9015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae95e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90070f90 0x7fae90071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae95e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae900cd200 con 0x7fae90154a10 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae95e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae90071940 con 0x7fae900cdca0 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae95e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9006e6f0 con 0x7fae90070f90 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae8fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae900cdca0 0x7fae9015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae8fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae900cdca0 0x7fae9015e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58696/0 (socket says 172.21.15.5:58696) 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae8fe00640 1 -- 172.21.15.5:0/10463449 learned_addr learned my addr 172.21.15.5:0/10463449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.701+0000 7fae95400640 1 --2- 172.21.15.5:0/10463449 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90070f90 0x7fae90071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.702+0000 7fae8fe00640 1 -- 172.21.15.5:0/10463449 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90070f90 msgr2=0x7fae90071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.702+0000 7fae8fe00640 1 --2- 172.21.15.5:0/10463449 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90070f90 0x7fae90071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.702+0000 7fae8fe00640 1 -- 172.21.15.5:0/10463449 >> v1:172.21.15.80:6789/0 conn(0x7fae90154a10 legacy=0x7fae900b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.702+0000 7fae8fe00640 1 -- 172.21.15.5:0/10463449 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9006e9d0 con 0x7fae900cdca0 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.703+0000 7fae8fe00640 1 --2- 172.21.15.5:0/10463449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9015e8c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009870 tx=0x7fae7c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b8187fb3c2a3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.703+0000 7fae8f400640 1 -- 172.21.15.5:0/10463449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c037070 con 0x7fae900cdca0 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.703+0000 7fae8f400640 1 -- 172.21.15.5:0/10463449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c032440 con 0x7fae900cdca0 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.703+0000 7fae8f400640 1 -- 172.21.15.5:0/10463449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c032810 con 0x7fae900cdca0 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 -- 172.21.15.5:0/10463449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 msgr2=0x7fae9015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 --2- 172.21.15.5:0/10463449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9015e8c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009870 tx=0x7fae7c02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 -- 172.21.15.5:0/10463449 shutdown_connections 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 --2- 172.21.15.5:0/10463449 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90070f90 0x7fae90071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 --2- 172.21.15.5:0/10463449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9015e8c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 -- 172.21.15.5:0/10463449 >> 172.21.15.5:0/10463449 conn(0x7fae90066f20 msgr2=0x7fae9006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 -- 172.21.15.5:0/10463449 shutdown_connections 2024-04-25T10:02:40.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.704+0000 7fae95e00640 1 -- 172.21.15.5:0/10463449 wait complete. 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.706+0000 7fae95e00640 1 Processor -- start 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.707+0000 7fae95e00640 1 -- start start 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.707+0000 7fae95e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae90070f90 0x7fae901539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.707+0000 7fae95e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae95e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90154a10 0x7fae9015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae95e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900cd780 con 0x7fae900cdca0 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae95e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae900cd4a0 con 0x7fae90070f90 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae95e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9006cff0 con 0x7fae90154a10 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9014daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58710/0 (socket says 172.21.15.5:58710) 2024-04-25T10:02:40.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 -- 172.21.15.5:0/4192893969 learned_addr learned my addr 172.21.15.5:0/4192893969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae94a00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae90070f90 0x7fae901539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae95400640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90154a10 0x7fae9015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 -- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90154a10 msgr2=0x7fae9015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90154a10 0x7fae9015c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 -- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae90070f90 msgr2=0x7fae901539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae90070f90 0x7fae901539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae8fe00640 1 -- 172.21.15.5:0/4192893969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae90153f30 con 0x7fae900cdca0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.708+0000 7fae94a00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae90070f90 0x7fae901539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.709+0000 7fae8fe00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9014daa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009870 tx=0x7fae7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.709+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c041020 con 0x7fae900cdca0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.709+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c0306c0 con 0x7fae900cdca0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.709+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c037040 con 0x7fae900cdca0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.709+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae90152950 con 0x7fae900cdca0 2024-04-25T10:02:40.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.709+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae90152f10 con 0x7fae900cdca0 2024-04-25T10:02:40.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.711+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae90154330 con 0x7fae900cdca0 2024-04-25T10:02:40.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.712+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae7c044020 con 0x7fae900cdca0 2024-04-25T10:02:40.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.712+0000 7fae8cc00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae7006b520 0x7fae7006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.713+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fae7c0aa9a0 con 0x7fae900cdca0 2024-04-25T10:02:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.713+0000 7fae94a00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae7006b520 0x7fae7006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.714+0000 7fae94a00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae7006b520 0x7fae7006d9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fae900718b0 tx=0x7fae84008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.717+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae7c04b070 con 0x7fae900cdca0 2024-04-25T10:02:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.963+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae9015d850 con 0x7fae900cdca0 2024-04-25T10:02:40.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.964+0000 7fae8cc00640 1 -- 172.21.15.5:0/4192893969 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fae7c079520 con 0x7fae900cdca0 2024-04-25T10:02:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae7006b520 msgr2=0x7fae7006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae7006b520 0x7fae7006d9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fae900718b0 tx=0x7fae84008040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 msgr2=0x7fae9014daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9014daa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009870 tx=0x7fae7c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 shutdown_connections 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae90154a10 0x7fae9015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae900cdca0 0x7fae9014daa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae7006b520 0x7fae7006d9e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 --2- 172.21.15.5:0/4192893969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae90070f90 0x7fae901539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 >> 172.21.15.5:0/4192893969 conn(0x7fae90066f20 msgr2=0x7fae900af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.968+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 shutdown_connections 2024-04-25T10:02:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:40.969+0000 7fae95e00640 1 -- 172.21.15.5:0/4192893969 wait complete. 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:40.993 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:40.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.993 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:02:41.047 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.048 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42565sec preferred_lft 42565sec 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:41.049 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:02:41.050 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:41.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:41.051 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:41.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.241 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:41.307 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:02:41.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.311 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:02:41.361 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:02:41.362 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:41.362 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:02:41.362 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:02:41.362 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:41.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.362 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:41.362 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:41.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:02:41.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:02:41.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:02:41.363 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:41.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.516 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:02:41.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.572 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-04-25T10:02:41.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:41.753 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:02:41.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.753 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:41.753 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:41.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:02:41.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:41.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:02:41.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:02:41.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:02:41.754 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:41.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.960 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:41.960 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:41.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:02:41.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:41.960 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:42.104 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:42.105 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.156 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:42.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:42.156 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.202 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:42.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.203 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:42.377 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:42.378 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:42.378 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:02:43.114 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:43.115 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:43.115 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:02:43.115 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:43.115 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:43.149 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:43.149 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:43.149 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:02:43.150 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:43.150 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:43.150 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:02:43.150 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:02:43.150 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:02:43.152 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:02:43.152 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:43.153 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:43.153 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:43.153 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:43.157 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:43.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:43.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.245 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:43.304 INFO:teuthology.orchestra.run.smithi005.stdout:client4830 2024-04-25T10:02:43.305 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:02:43.305 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:43.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.358 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout:client4830 2024-04-25T10:02:43.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.417 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout:client4830 2024-04-25T10:02:43.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.476 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b6dc2a7-3300-467c-ae7a-376f479db0dd.client4830/status') 2024-04-25T10:02:43.558 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.4830 (3)192.168.0.1:0/2440568360 2024-04-25T10:02:43.559 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-04-25T10:02:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-04-25T10:02:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-04-25T10:02:43.559 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 6.7851e-05 s, 1.0 MB/s 2024-04-25T10:02:43.563 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T10:02:43.563 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:43.563 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:43.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:43.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.762+0000 7f5ba1600640 1 Processor -- start 2024-04-25T10:02:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.762+0000 7f5ba1600640 1 -- start start 2024-04-25T10:02:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.764+0000 7f5ba1600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 0x7f5b9c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.764+0000 7f5ba1600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c071d30 con 0x7f5b9c06e640 2024-04-25T10:02:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.764+0000 7f5ba1600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c06d620 con 0x7f5b9c15d510 2024-04-25T10:02:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.764+0000 7f5ba1600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c142180 con 0x7f5b9c141da0 2024-04-25T10:02:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.764+0000 7f5b93e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5b9c06e640 0x7f5b9c15ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46918/0 (socket says 172.21.15.5:46918) 2024-04-25T10:02:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.764+0000 7f5b93e00640 1 -- 172.21.15.5:0/1562347331 learned_addr learned my addr 172.21.15.5:0/1562347331 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9be00640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 0x7f5b9c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699953283 0 0) 0x7f5b9c071d30 con 0x7f5b9c06e640 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b80003520 con 0x7f5b9c06e640 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468940627 0 0) 0x7f5b80003520 con 0x7f5b9c06e640 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 msgr2=0x7f5b9c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 0x7f5b9c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 >> v1:172.21.15.80:6789/0 conn(0x7f5b9c15d510 legacy=0x7f5b9c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c06d900 con 0x7f5b9c06e640 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f5b84001670 con 0x7f5b9c06e640 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.765+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 >> v1:172.21.15.5:6789/0 conn(0x7f5b9c06e640 legacy=0x7f5b9c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9b400640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9b400640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b80004800 0x7f5b80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9b400640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 0x7f5b800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c142180 con 0x7f5b800039e0 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8000baa0 con 0x7f5b80004800 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8000bca0 con 0x7f5b9c141da0 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9be00640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b93e00640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b80004800 0x7f5b80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9be00640 1 -- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 msgr2=0x7f5b800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9be00640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c141da0 0x7f5b800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9be00640 1 -- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b80004800 msgr2=0x7f5b80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9be00640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b80004800 0x7f5b80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.766+0000 7f5b9be00640 1 -- 172.21.15.5:0/1562347331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80003520 con 0x7f5b800039e0 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.767+0000 7f5b9be00640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b80003e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009a90 tx=0x7f5b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d11a7b59009c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c036070 con 0x7f5b800039e0 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c031400 con 0x7f5b800039e0 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5b9b400640 1 -- 172.21.15.5:0/1562347331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c009690 con 0x7f5b800039e0 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5ba1600640 1 -- 172.21.15.5:0/1562347331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 msgr2=0x7f5b80003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5ba1600640 1 --2- 172.21.15.5:0/1562347331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b80003e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009a90 tx=0x7f5b80005830 comp rx=0 tx=0).stop 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5ba0c00640 1 -- 172.21.15.5:0/1562347331 reap_dead start 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5ba1600640 1 -- 172.21.15.5:0/1562347331 shutdown_connections 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5ba1600640 1 -- 172.21.15.5:0/1562347331 >> 172.21.15.5:0/1562347331 conn(0x7f5b9c066f20 msgr2=0x7f5b9c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.768+0000 7f5ba1600640 1 -- 172.21.15.5:0/1562347331 shutdown_connections 2024-04-25T10:02:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.769+0000 7f5ba1600640 1 -- 172.21.15.5:0/1562347331 wait complete. 2024-04-25T10:02:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.770+0000 7f5ba1600640 1 Processor -- start 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.770+0000 7f5ba1600640 1 -- start start 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.771+0000 7f5ba1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b9c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.771+0000 7f5ba1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b9c13c5f0 0x7f5b9c157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.771+0000 7f5ba1600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c13ca50 0x7f5b9c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.771+0000 7f5ba1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0cdb90 con 0x7f5b800039e0 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.771+0000 7f5ba1600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0cda10 con 0x7f5b9c13c5f0 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.771+0000 7f5ba1600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c06e100 con 0x7f5b9c13ca50 2024-04-25T10:02:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b9c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b9c1571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58730/0 (socket says 172.21.15.5:58730) 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 -- 172.21.15.5:0/2333725021 learned_addr learned my addr 172.21.15.5:0/2333725021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5ba0c00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c13ca50 0x7f5b9c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b93e00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b9c13c5f0 0x7f5b9c157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 -- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c13ca50 msgr2=0x7f5b9c1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c13ca50 0x7f5b9c1580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 -- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b9c13c5f0 msgr2=0x7f5b9c157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b9c13c5f0 0x7f5b9c157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5b9be00640 1 -- 172.21.15.5:0/2333725021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c13d3e0 con 0x7f5b800039e0 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.772+0000 7f5ba0c00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c13ca50 0x7f5b9c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.773+0000 7f5b9be00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b9c1571e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c034040 tx=0x7f5b8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.773+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c040020 con 0x7f5b800039e0 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.773+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c003bc0 con 0x7f5b800039e0 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.773+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c036040 con 0x7f5b800039e0 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.773+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c13d720 con 0x7f5b800039e0 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.773+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c13f860 con 0x7f5b800039e0 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.774+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c072220 con 0x7f5b800039e0 2024-04-25T10:02:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.775+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b8c043020 con 0x7f5b800039e0 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.776+0000 7f5b98c00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b7806b4d0 0x7f5b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.776+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f5b8c0aa680 con 0x7f5b800039e0 2024-04-25T10:02:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.777+0000 7f5b93e00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b7806b4d0 0x7f5b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.778+0000 7f5b93e00640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b7806b4d0 0x7f5b7806d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b84001050 tx=0x7f5b84009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:43.781+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b8c04a070 con 0x7f5b800039e0 2024-04-25T10:02:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.023+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b9c068f60 con 0x7f5b800039e0 2024-04-25T10:02:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.024+0000 7f5b98c00640 1 -- 172.21.15.5:0/2333725021 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5b9c13d720 con 0x7f5b800039e0 2024-04-25T10:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b7806b4d0 msgr2=0x7f5b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b7806b4d0 0x7f5b7806d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5b84001050 tx=0x7f5b84009040 comp rx=0 tx=0).stop 2024-04-25T10:02:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 msgr2=0x7f5b9c1571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b9c1571e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c034040 tx=0x7f5b8c004730 comp rx=0 tx=0).stop 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 shutdown_connections 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b9c13ca50 0x7f5b9c1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b7806b4d0 0x7f5b7806d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b9c13c5f0 0x7f5b9c157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 --2- 172.21.15.5:0/2333725021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b800039e0 0x7f5b9c1571e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.026+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 >> 172.21.15.5:0/2333725021 conn(0x7f5b9c066f20 msgr2=0x7f5b9c15f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.027+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 shutdown_connections 2024-04-25T10:02:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:44.027+0000 7f5ba1600640 1 -- 172.21.15.5:0/2333725021 wait complete. 2024-04-25T10:02:44.050 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:44.050 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:44.050 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:02:44.051 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:02:44.051 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:44.051 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:44.051 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:44.051 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:44.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:44.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.051 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:02:44.102 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:44.102 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:44.102 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:44.102 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.102 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:44.102 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42562sec preferred_lft 42562sec 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:02:44.103 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:02:44.104 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.105 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:44.183 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:02:44.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.187 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:02:44.238 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:02:44.238 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:44.238 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:02:44.238 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:02:44.238 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:44.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.239 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:44.239 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:44.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:02:44.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:02:44.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:02:44.239 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:44.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.394 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:02:44.446 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:44.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.448 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-04-25T10:02:44.502 INFO:teuthology.orchestra.run.smithi005.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:02:44.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.503 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:44.503 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:44.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:44.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:02:44.503 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:44.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 2e:91:ff:b9:8b:09 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2c91:ffff:feb9:8b09/64 scope link 2024-04-25T10:02:44.708 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 2e:91:ff:b9:8b:09 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:44.803 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:44.804 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.804 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2c91:ffff:feb9:8b09/64 scope link 2024-04-25T10:02:44.804 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.807 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:02:44.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:02:44.808 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:45.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.040 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:02:45.040 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:02:45.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.1 up 2024-04-25T10:02:45.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:45.040 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:02:45.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:45.194 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:45.239 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:45.239 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:45.239 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:45.337 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:45.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.474 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.474 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:02:45.474 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:02:45.496 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.496 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:02:45.497 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:02:45.497 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:45.497 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:02:45.527 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:02:45.530 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T10:02:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:45.535 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:45.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:45.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.616 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:45.679 INFO:teuthology.orchestra.run.smithi005.stdout:client4708 2024-04-25T10:02:45.679 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:02:45.679 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:45.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.731 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:45.791 INFO:teuthology.orchestra.run.smithi005.stdout:client4708 2024-04-25T10:02:45.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.792 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:45.853 INFO:teuthology.orchestra.run.smithi005.stdout:client4708 2024-04-25T10:02:45.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.854 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b6dc2a7-3300-467c-ae7a-376f479db0dd.client4708/status') 2024-04-25T10:02:45.938 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.4708 (3)172.21.15.5:0/3152121078 2024-04-25T10:02:45.938 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-04-25T10:02:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-04-25T10:02:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-04-25T10:02:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 7.5458e-05 s, 914 kB/s 2024-04-25T10:02:45.942 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T10:02:45.942 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:45.942 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:45.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.150+0000 7fe3bcc00640 1 Processor -- start 2024-04-25T10:02:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.150+0000 7fe3bcc00640 1 -- start start 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3bcc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe3b80b16e0 0x7fe3b80ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3bcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe3b81622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3bcc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b8165af0 con 0x7fe3b815c380 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3bcc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b815c760 con 0x7fe3b80b16e0 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3bcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81658a0 con 0x7fe3b80cd240 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3b5e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe3b815c380 0x7fe3b806e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46934/0 (socket says 172.21.15.5:46934) 2024-04-25T10:02:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.152+0000 7fe3b5e00640 1 -- 172.21.15.5:0/206530344 learned_addr learned my addr 172.21.15.5:0/206530344 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3ad400640 1 --2- 172.21.15.5:0/206530344 >> v2:172.21.15.80:3300/0 conn(0x7fe3b80b16e0 0x7fe3b80ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b6800640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe3b81622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078118389 0 0) 0x7fe3b8165af0 con 0x7fe3b815c380 2024-04-25T10:02:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c003520 con 0x7fe3b815c380 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968763585 0 0) 0x7fe39c003520 con 0x7fe3b815c380 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 msgr2=0x7fe3b81622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe3b81622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 >> v2:172.21.15.80:3300/0 conn(0x7fe3b80b16e0 msgr2=0x7fe3b80ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.153+0000 7fe3b5400640 1 --2- 172.21.15.5:0/206530344 >> v2:172.21.15.80:3300/0 conn(0x7fe3b80b16e0 0x7fe3b80ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b8165dd0 con 0x7fe3b815c380 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe3a8001670 con 0x7fe3b815c380 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 >> v1:172.21.15.5:6789/0 conn(0x7fe3b815c380 legacy=0x7fe3b806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe39c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b815c760 con 0x7fe39c004800 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81658a0 con 0x7fe39c0039b0 2024-04-25T10:02:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00baa0 con 0x7fe3b80cd240 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3ad400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b5e00640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.154+0000 7fe3b6800640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe39c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.155+0000 7fe3ad400640 1 -- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 msgr2=0x7fe39c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.155+0000 7fe3ad400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe39c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.155+0000 7fe3ad400640 1 -- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe39c0039b0 msgr2=0x7fe39c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.155+0000 7fe3ad400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.155+0000 7fe3ad400640 1 -- 172.21.15.5:0/206530344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c003520 con 0x7fe39c004800 2024-04-25T10:02:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.155+0000 7fe3b6800640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3b80cd240 0x7fe39c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.156+0000 7fe3ad400640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80b1b50 tx=0x7fe39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff80717c95379d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.156+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0036070 con 0x7fe39c004800 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.156+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0031400 con 0x7fe39c004800 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.156+0000 7fe3b5400640 1 -- 172.21.15.5:0/206530344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0009690 con 0x7fe39c004800 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.156+0000 7fe3bcc00640 1 -- 172.21.15.5:0/206530344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39c004800 msgr2=0x7fe39c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.156+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/206530344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80b1b50 tx=0x7fe39c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.157+0000 7fe3b6800640 1 -- 172.21.15.5:0/206530344 reap_dead start 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.157+0000 7fe3bcc00640 1 -- 172.21.15.5:0/206530344 shutdown_connections 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.157+0000 7fe3bcc00640 1 -- 172.21.15.5:0/206530344 >> 172.21.15.5:0/206530344 conn(0x7fe3b8071150 msgr2=0x7fe3b8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.157+0000 7fe3bcc00640 1 -- 172.21.15.5:0/206530344 shutdown_connections 2024-04-25T10:02:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.157+0000 7fe3bcc00640 1 -- 172.21.15.5:0/206530344 wait complete. 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.159+0000 7fe3bcc00640 1 Processor -- start 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.159+0000 7fe3bcc00640 1 -- start start 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3bcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe39c004800 0x7fe3b81f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3bcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 0x7fe3b81550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3bcc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3b8155620 0x7fe3b8150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3bcc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81622b0 con 0x7fe3b813f6b0 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3bcc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8166390 con 0x7fe3b8155620 2024-04-25T10:02:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3bcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8162430 con 0x7fe39c004800 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 0x7fe3b81550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 0x7fe3b81550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58754/0 (socket says 172.21.15.5:58754) 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.160+0000 7fe3ad400640 1 -- 172.21.15.5:0/1834713575 learned_addr learned my addr 172.21.15.5:0/1834713575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3b5e00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe39c004800 0x7fe3b81f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3b6800640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3b8155620 0x7fe3b8150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3ad400640 1 -- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe39c004800 msgr2=0x7fe3b81f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe39c004800 0x7fe3b81f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3ad400640 1 -- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3b8155620 msgr2=0x7fe3b8150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3b8155620 0x7fe3b8150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3ad400640 1 -- 172.21.15.5:0/1834713575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80b42b0 con 0x7fe3b813f6b0 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3b6800640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3b8155620 0x7fe3b8150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3b5e00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe39c004800 0x7fe3b81f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3ad400640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 0x7fe3b81550e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002ab50 tx=0x7fe3a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0040020 con 0x7fe3b813f6b0 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a00038d0 con 0x7fe3b813f6b0 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0036040 con 0x7fe3b813f6b0 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.161+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b80b4590 con 0x7fe3b813f6b0 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.162+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b80b4b50 con 0x7fe3b813f6b0 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.163+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b81582c0 con 0x7fe3b813f6b0 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.163+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3a0043020 con 0x7fe3b813f6b0 2024-04-25T10:02:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.164+0000 7fe3aea00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe39406b4d0 0x7fe39406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.164+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fe3a00aa770 con 0x7fe3b813f6b0 2024-04-25T10:02:46.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.165+0000 7fe3b5e00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe39406b4d0 0x7fe39406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:46.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.166+0000 7fe3b5e00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe39406b4d0 0x7fe39406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8000fb0 tx=0x7fe3a8001080 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.169+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3b81582c0 con 0x7fe3b813f6b0 2024-04-25T10:02:46.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.410+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3b8068fd0 con 0x7fe3b813f6b0 2024-04-25T10:02:46.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.411+0000 7fe3aea00640 1 -- 172.21.15.5:0/1834713575 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe3a004a070 con 0x7fe3b813f6b0 2024-04-25T10:02:46.416 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe39406b4d0 msgr2=0x7fe39406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe39406b4d0 0x7fe39406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8000fb0 tx=0x7fe3a8001080 comp rx=0 tx=0).stop 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 msgr2=0x7fe3b81550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 0x7fe3b81550e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3a002ab50 tx=0x7fe3a0004730 comp rx=0 tx=0).stop 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 shutdown_connections 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe39406b4d0 0x7fe39406d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3b8155620 0x7fe3b8150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b813f6b0 0x7fe3b81550e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 --2- 172.21.15.5:0/1834713575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe39c004800 0x7fe3b81f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 >> 172.21.15.5:0/1834713575 conn(0x7fe3b8071150 msgr2=0x7fe3b8157620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 shutdown_connections 2024-04-25T10:02:46.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:46.414+0000 7fe3bcc00640 1 -- 172.21.15.5:0/1834713575 wait complete. 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:46.440 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:46.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.441 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.497 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T10:02:46.498 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.499 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:46.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set ceph-brx up 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:46.500 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:46.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.929 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:46.998 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T10:02:47.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.001 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T10:02:47.061 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T10:02:47.062 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:47.062 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:47.062 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:47.062 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:47.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.062 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:47.062 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:47.062 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:47.063 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:47.063 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:47.063 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:47.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.776 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T10:02:47.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.831 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list-id 2024-04-25T10:02:47.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.883 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:47.883 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:47.883 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:47.883 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:02:47.883 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:48.001 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:02:48.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:02:48.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:02:48.003 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:48.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.231 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:48.231 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:48.232 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T10:02:48.232 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:48.232 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:48.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:48.370 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:48.411 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:48.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:48.412 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:48.501 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:48.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.502 DEBUG:teuthology.orchestra.run.smithi080:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:48.639 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:48.639 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:02:48.639 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-25T10:02:49.343 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:49.344 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:02:49.344 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-25T10:02:49.344 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:49.344 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:49.384 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:49.384 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:02:49.384 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-25T10:02:49.385 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:49.385 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:49.385 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:02:49.385 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:02:49.385 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:02:49.390 INFO:teuthology.orchestra.run.smithi080.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T10:02:49.390 INFO:teuthology.orchestra.run.smithi080.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:49.390 INFO:teuthology.orchestra.run.smithi080.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:49.390 INFO:teuthology.orchestra.run.smithi080.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:49.390 INFO:teuthology.orchestra.run.smithi080.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:49.395 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:49.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.395 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:49.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.476 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:49.529 INFO:teuthology.orchestra.run.smithi080.stdout:client4845 2024-04-25T10:02:49.529 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:02:49.529 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:49.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.580 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:49.637 INFO:teuthology.orchestra.run.smithi080.stdout:client4845 2024-04-25T10:02:49.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.638 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:49.696 INFO:teuthology.orchestra.run.smithi080.stdout:client4845 2024-04-25T10:02:49.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.697 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b6dc2a7-3300-467c-ae7a-376f479db0dd.client4845/status') 2024-04-25T10:02:49.777 INFO:teuthology.orchestra.run.smithi080.stdout:instance: client.4845 (3)172.21.15.80:0/2612166852 2024-04-25T10:02:49.777 INFO:teuthology.orchestra.run.smithi080.stdout:blocklisted: false 2024-04-25T10:02:49.777 INFO:teuthology.orchestra.run.smithi080.stderr:0+1 records in 2024-04-25T10:02:49.777 INFO:teuthology.orchestra.run.smithi080.stderr:0+1 records out 2024-04-25T10:02:49.777 INFO:teuthology.orchestra.run.smithi080.stderr:70 bytes copied, 9.8247e-05 s, 712 kB/s 2024-04-25T10:02:49.781 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T10:02:49.781 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:49.781 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:49.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.997+0000 7f13dc800640 1 Processor -- start 2024-04-25T10:02:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.997+0000 7f13dc800640 1 -- start start 2024-04-25T10:02:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13dc800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f13d415db00 0x7f13d40d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13dc800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f13d40b53b0 0x7f13d40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13dc800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40d1e80 con 0x7f13d40ce2b0 2024-04-25T10:02:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13dc800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40b5930 con 0x7f13d40b53b0 2024-04-25T10:02:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13dc800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40b5c10 con 0x7f13d415db00 2024-04-25T10:02:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13d9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f13d40b53b0 0x7f13d40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13da800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f13d40ce2b0 0x7f13d4161670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39812/0 (socket says 172.21.15.5:39812) 2024-04-25T10:02:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:49.999+0000 7f13da800640 1 -- 172.21.15.5:0/118814805 learned_addr learned my addr 172.21.15.5:0/118814805 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.000+0000 7f13d9e00640 1 --2- 172.21.15.5:0/118814805 >> v2:172.21.15.80:3300/0 conn(0x7f13d415db00 0x7f13d40d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.000+0000 7f13d9400640 1 -- 172.21.15.5:0/118814805 >> v2:172.21.15.80:3300/0 conn(0x7f13d415db00 msgr2=0x7f13d40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.000+0000 7f13d9400640 1 --2- 172.21.15.5:0/118814805 >> v2:172.21.15.80:3300/0 conn(0x7f13d415db00 0x7f13d40d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.000+0000 7f13d9400640 1 -- 172.21.15.5:0/118814805 >> v1:172.21.15.5:6789/0 conn(0x7f13d40ce2b0 legacy=0x7f13d4161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.000+0000 7f13d9400640 1 -- 172.21.15.5:0/118814805 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40b5ef0 con 0x7f13d40b53b0 2024-04-25T10:02:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.000+0000 7f13d8a00640 1 -- 172.21.15.5:0/118814805 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913369485 0 0) 0x7f13d40d1e80 con 0x7f13d40ce2b0 2024-04-25T10:02:50.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.001+0000 7f13d9400640 1 --2- 172.21.15.5:0/118814805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40b53b0 0x7f13d40d0880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f13d40d4940 tx=0x7f13c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc9c9c8fe97e7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.001+0000 7f13d8a00640 1 -- 172.21.15.5:0/118814805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4037070 con 0x7f13d40b53b0 2024-04-25T10:02:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.001+0000 7f13d8a00640 1 -- 172.21.15.5:0/118814805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4032440 con 0x7f13d40b53b0 2024-04-25T10:02:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.001+0000 7f13d8a00640 1 -- 172.21.15.5:0/118814805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4032810 con 0x7f13d40b53b0 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 -- 172.21.15.5:0/118814805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40b53b0 msgr2=0x7f13d40d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 --2- 172.21.15.5:0/118814805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40b53b0 0x7f13d40d0880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f13d40d4940 tx=0x7f13c402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 -- 172.21.15.5:0/118814805 shutdown_connections 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 --2- 172.21.15.5:0/118814805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40b53b0 0x7f13d40d0880 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 --2- 172.21.15.5:0/118814805 >> v2:172.21.15.80:3300/0 conn(0x7f13d415db00 0x7f13d40d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 -- 172.21.15.5:0/118814805 >> 172.21.15.5:0/118814805 conn(0x7f13d4072290 msgr2=0x7f13d40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 -- 172.21.15.5:0/118814805 shutdown_connections 2024-04-25T10:02:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.002+0000 7f13dc800640 1 -- 172.21.15.5:0/118814805 wait complete. 2024-04-25T10:02:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.004+0000 7f13dc800640 1 Processor -- start 2024-04-25T10:02:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.004+0000 7f13dc800640 1 -- start start 2024-04-25T10:02:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13dc800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f13d40b53b0 0x7f13d414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13dc800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 0x7f13d40c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13dc800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d415db00 0x7f13d40c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13dc800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40d3660 con 0x7f13d40ce2b0 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13dc800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40b61d0 con 0x7f13d415db00 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13dc800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40b6c10 con 0x7f13d40b53b0 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 0x7f13d40c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 0x7f13d40c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49524/0 (socket says 172.21.15.5:49524) 2024-04-25T10:02:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.005+0000 7f13d9400640 1 -- 172.21.15.5:0/3178132160 learned_addr learned my addr 172.21.15.5:0/3178132160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13da800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d415db00 0x7f13d40c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13d9400640 1 -- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f13d40b53b0 msgr2=0x7f13d414ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13d9400640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f13d40b53b0 0x7f13d414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13d9400640 1 -- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d415db00 msgr2=0x7f13d40c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13d9400640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d415db00 0x7f13d40c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13d9400640 1 -- 172.21.15.5:0/3178132160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4156290 con 0x7f13d40ce2b0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13da800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d415db00 0x7f13d40c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.006+0000 7f13d9400640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 0x7f13d40c8dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13c4000c00 tx=0x7f13c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.007+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4037040 con 0x7f13d40ce2b0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.007+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4042070 con 0x7f13d40ce2b0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.007+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4030bd0 con 0x7f13d40ce2b0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.007+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d415e340 con 0x7f13d40ce2b0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.007+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d415e900 con 0x7f13d40ce2b0 2024-04-25T10:02:50.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.008+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d4068fd0 con 0x7f13d40ce2b0 2024-04-25T10:02:50.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.009+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13c4047460 con 0x7f13d40ce2b0 2024-04-25T10:02:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.010+0000 7f13ca000640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13b806b4d0 0x7f13b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:50.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.010+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f13c40ab2a0 con 0x7f13d40ce2b0 2024-04-25T10:02:50.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.011+0000 7f13d9e00640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13b806b4d0 0x7f13b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:50.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.012+0000 7f13d9e00640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13b806b4d0 0x7f13b806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0070b0 tx=0x7f13cc007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:50.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.015+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13c4032e20 con 0x7f13d40ce2b0 2024-04-25T10:02:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.253+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f13d40d16b0 con 0x7f13d40ce2b0 2024-04-25T10:02:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.253+0000 7f13ca000640 1 -- 172.21.15.5:0/3178132160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f13c4079c80 con 0x7f13d40ce2b0 2024-04-25T10:02:50.259 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13b806b4d0 msgr2=0x7f13b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13b806b4d0 0x7f13b806d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0070b0 tx=0x7f13cc007040 comp rx=0 tx=0).stop 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 msgr2=0x7f13d40c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 0x7f13d40c8dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13c4000c00 tx=0x7f13c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 shutdown_connections 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13b806b4d0 0x7f13b806d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d415db00 0x7f13d40c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d40ce2b0 0x7f13d40c8dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 --2- 172.21.15.5:0/3178132160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f13d40b53b0 0x7f13d414ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 >> 172.21.15.5:0/3178132160 conn(0x7f13d4072290 msgr2=0x7f13d40d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 shutdown_connections 2024-04-25T10:02:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:50.256+0000 7f13dc800640 1 -- 172.21.15.5:0/3178132160 wait complete. 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:50.282 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:50.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.283 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T10:02:50.338 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.339 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42575sec preferred_lft 42575sec 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether f6:8c:cb:f3:47:47 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::7468:aaff:fe9c:e958/64 scope link 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether f6:8c:cb:f3:47:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:50.340 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::f48c:cbff:fef3:4747/64 scope link 2024-04-25T10:02:50.341 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.342 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:50.417 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T10:02:50.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.421 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T10:02:50.468 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T10:02:50.468 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:50.468 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:50.469 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:50.469 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:50.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.469 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:50.469 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:50.469 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:50.469 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:50.469 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:50.469 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:50.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.664 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T10:02:50.713 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:02:50.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.714 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list-id 2024-04-25T10:02:50.764 INFO:teuthology.orchestra.run.smithi080.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:02:50.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.765 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:50.765 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:50.765 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:50.765 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:02:50.765 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:50.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.880 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:50.956 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:50.956 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:50.956 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:50.956 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.956 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 2e:10:cc:77:bc:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::2c10:ccff:fe77:bca2/64 scope link 2024-04-25T10:02:50.957 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.960 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 2e:10:cc:77:bc:a2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:51.038 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:51.039 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:51.039 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::2c10:ccff:fe77:bca2/64 scope link 2024-04-25T10:02:51.039 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:51.041 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:02:51.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:02:51.042 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:51.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.256 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec bash -c ' 2024-04-25T10:02:51.256 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T10:02:51.256 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.1 up 2024-04-25T10:02:51.256 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:51.256 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T10:02:51.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:51.399 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.444 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:51.444 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:51.444 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.537 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:51.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.537 DEBUG:teuthology.orchestra.run.smithi080:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.670 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.670 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:02:51.670 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-25T10:02:51.691 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.691 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:02:51.692 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-25T10:02:51.692 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:51.692 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:invalid new device string format 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:02:51.717 INFO:teuthology.orchestra.run.smithi080.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:02:51.719 INFO:teuthology.orchestra.run.smithi080.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T10:02:51.719 INFO:teuthology.orchestra.run.smithi080.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:51.720 INFO:teuthology.orchestra.run.smithi080.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:51.720 INFO:teuthology.orchestra.run.smithi080.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:51.720 INFO:teuthology.orchestra.run.smithi080.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:51.724 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:51.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.724 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.794 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi080.stdout:client4556 2024-04-25T10:02:51.847 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:02:51.847 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:51.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.895 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:51.996 INFO:teuthology.orchestra.run.smithi080.stdout:client4556 2024-04-25T10:02:51.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.997 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:52.055 INFO:teuthology.orchestra.run.smithi080.stdout:client4556 2024-04-25T10:02:52.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:52.055 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8b6dc2a7-3300-467c-ae7a-376f479db0dd.client4556/status') 2024-04-25T10:02:52.132 INFO:teuthology.orchestra.run.smithi080.stdout:instance: client.4556 (3)192.168.0.2:0/4228601868 2024-04-25T10:02:52.132 INFO:teuthology.orchestra.run.smithi080.stdout:blocklisted: false 2024-04-25T10:02:52.132 INFO:teuthology.orchestra.run.smithi080.stderr:0+1 records in 2024-04-25T10:02:52.133 INFO:teuthology.orchestra.run.smithi080.stderr:0+1 records out 2024-04-25T10:02:52.133 INFO:teuthology.orchestra.run.smithi080.stderr:69 bytes copied, 7.9996e-05 s, 863 kB/s 2024-04-25T10:02:52.136 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:02:52.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.372+0000 7f75bca00640 1 Processor -- start 2024-04-25T10:02:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.373+0000 7f75bca00640 1 -- start start 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75bca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75b8161200 0x7f75b80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75bca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f75b80d07d0 0x7f75b80b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75bca00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b80cf410 con 0x7f75b80d2380 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75bca00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b80d27c0 con 0x7f75b8161200 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75bca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b80b28b0 con 0x7f75b80d07d0 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75b6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75b8161200 0x7f75b80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75b6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75b8161200 0x7f75b80b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49538/0 (socket says 172.21.15.5:49538) 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.374+0000 7f75b6800640 1 -- 172.21.15.5:0/840186787 learned_addr learned my addr 172.21.15.5:0/840186787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.375+0000 7f75b7200640 1 --2- 172.21.15.5:0/840186787 >> v2:172.21.15.80:3300/0 conn(0x7f75b80d07d0 0x7f75b80b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.375+0000 7f75b6800640 1 -- 172.21.15.5:0/840186787 >> v2:172.21.15.80:3300/0 conn(0x7f75b80d07d0 msgr2=0x7f75b80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.375+0000 7f75b6800640 1 --2- 172.21.15.5:0/840186787 >> v2:172.21.15.80:3300/0 conn(0x7f75b80d07d0 0x7f75b80b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.375+0000 7f75b6800640 1 -- 172.21.15.5:0/840186787 >> v1:172.21.15.80:6789/0 conn(0x7f75b80d2380 legacy=0x7f75b80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.375+0000 7f75b6800640 1 -- 172.21.15.5:0/840186787 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b80cf6f0 con 0x7f75b8161200 2024-04-25T10:02:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.376+0000 7f75b6800640 1 --2- 172.21.15.5:0/840186787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b8161200 0x7f75b80b4bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75b80cfc80 tx=0x7f75a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be3c4397d417990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.376+0000 7f75b5400640 1 -- 172.21.15.5:0/840186787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8037070 con 0x7f75b8161200 2024-04-25T10:02:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.376+0000 7f75b5400640 1 -- 172.21.15.5:0/840186787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8032440 con 0x7f75b8161200 2024-04-25T10:02:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.376+0000 7f75b5400640 1 -- 172.21.15.5:0/840186787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8032810 con 0x7f75b8161200 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 -- 172.21.15.5:0/840186787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b8161200 msgr2=0x7f75b80b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 --2- 172.21.15.5:0/840186787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b8161200 0x7f75b80b4bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75b80cfc80 tx=0x7f75a802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 -- 172.21.15.5:0/840186787 shutdown_connections 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 --2- 172.21.15.5:0/840186787 >> v2:172.21.15.80:3300/0 conn(0x7f75b80d07d0 0x7f75b80b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 --2- 172.21.15.5:0/840186787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b8161200 0x7f75b80b4bb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 -- 172.21.15.5:0/840186787 >> 172.21.15.5:0/840186787 conn(0x7f75b808af80 msgr2=0x7f75b80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 -- 172.21.15.5:0/840186787 shutdown_connections 2024-04-25T10:02:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.377+0000 7f75bca00640 1 -- 172.21.15.5:0/840186787 wait complete. 2024-04-25T10:02:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.379+0000 7f75bca00640 1 Processor -- start 2024-04-25T10:02:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.379+0000 7f75bca00640 1 -- start start 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75bca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b80d07d0 0x7f75b8156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75bca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 0x7f75b80b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75bca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b8161200 0x7f75b80b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75bca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80b34c0 con 0x7f75b80d2380 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75bca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80d42b0 con 0x7f75b8161200 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75bca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b80b3640 con 0x7f75b80d07d0 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 0x7f75b80b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 0x7f75b80b78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49554/0 (socket says 172.21.15.5:49554) 2024-04-25T10:02:52.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.380+0000 7f75b5e00640 1 -- 172.21.15.5:0/2447142302 learned_addr learned my addr 172.21.15.5:0/2447142302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b7200640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b8161200 0x7f75b80b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b6800640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b80d07d0 0x7f75b8156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b5e00640 1 -- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b80d07d0 msgr2=0x7f75b8156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b5e00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b80d07d0 0x7f75b8156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b5e00640 1 -- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b8161200 msgr2=0x7f75b80b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b5e00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b8161200 0x7f75b80b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b5e00640 1 -- 172.21.15.5:0/2447142302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b81571c0 con 0x7f75b80d2380 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75b5e00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 0x7f75b80b78c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c460 tx=0x7f75a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4016020 con 0x7f75b80d2380 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.381+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4012870 con 0x7f75b80d2380 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.382+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4012c60 con 0x7f75b80d2380 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.382+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b809d720 con 0x7f75b80d2380 2024-04-25T10:02:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.382+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b809dce0 con 0x7f75b80d2380 2024-04-25T10:02:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.383+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b8068f60 con 0x7f75b80d2380 2024-04-25T10:02:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.384+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75a4019020 con 0x7f75b80d2380 2024-04-25T10:02:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.385+0000 7f75aea00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f759806b4d0 0x7f759806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.385+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f75a4089ff0 con 0x7f75b80d2380 2024-04-25T10:02:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.386+0000 7f75b6800640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f759806b4d0 0x7f759806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.387+0000 7f75b6800640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f759806b4d0 0x7f759806d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f75a8034000 tx=0x7f75a802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.389+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75a4020070 con 0x7f75b80d2380 2024-04-25T10:02:52.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.593+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75b80b1b80 con 0x7f75b80d2380 2024-04-25T10:02:52.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.595+0000 7f75aea00640 1 -- 172.21.15.5:0/2447142302 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f75a4058b00 con 0x7f75b80d2380 2024-04-25T10:02:52.600 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:52.601 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.597+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f759806b4d0 msgr2=0x7f759806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.597+0000 7f75bca00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f759806b4d0 0x7f759806d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f75a8034000 tx=0x7f75a802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.597+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 msgr2=0x7f75b80b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.597+0000 7f75bca00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 0x7f75b80b78c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c460 tx=0x7f75a400c920 comp rx=0 tx=0).stop 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 shutdown_connections 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f759806b4d0 0x7f759806d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b8161200 0x7f75b80b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b80d2380 0x7f75b80b78c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 --2- 172.21.15.5:0/2447142302 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b80d07d0 0x7f75b8156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 >> 172.21.15.5:0/2447142302 conn(0x7f75b808af80 msgr2=0x7f75b80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 shutdown_connections 2024-04-25T10:02:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.598+0000 7f75bca00640 1 -- 172.21.15.5:0/2447142302 wait complete. 2024-04-25T10:02:52.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:52.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.815+0000 7efd14e00640 1 Processor -- start 2024-04-25T10:02:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.816+0000 7efd14e00640 1 -- start start 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.817+0000 7efd14e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd1006e840 0x7efd1015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.817+0000 7efd14e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efd1015b330 0x7efd10071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.817+0000 7efd14e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd100ce570 con 0x7efd100cc840 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.817+0000 7efd14e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd100ce850 con 0x7efd1015b330 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.817+0000 7efd14e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd100ceb30 con 0x7efd1006e840 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.818+0000 7efd0f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efd100cc840 0x7efd1006e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39816/0 (socket says 172.21.15.5:39816) 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.818+0000 7efd0f400640 1 -- 172.21.15.5:0/2059237268 learned_addr learned my addr 172.21.15.5:0/2059237268 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.818+0000 7efd0fe00640 1 --2- 172.21.15.5:0/2059237268 >> v2:172.21.15.5:3300/0 conn(0x7efd1015b330 0x7efd10071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.818+0000 7efd0ea00640 1 --2- 172.21.15.5:0/2059237268 >> v2:172.21.15.80:3300/0 conn(0x7efd1006e840 0x7efd1015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.818+0000 7efd0e000640 1 -- 172.21.15.5:0/2059237268 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302155734 0 0) 0x7efd100ce570 con 0x7efd100cc840 2024-04-25T10:02:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.818+0000 7efd0e000640 1 -- 172.21.15.5:0/2059237268 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf4003520 con 0x7efd100cc840 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.819+0000 7efd0fe00640 1 -- 172.21.15.5:0/2059237268 >> v2:172.21.15.80:3300/0 conn(0x7efd1006e840 msgr2=0x7efd1015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.819+0000 7efd0fe00640 1 --2- 172.21.15.5:0/2059237268 >> v2:172.21.15.80:3300/0 conn(0x7efd1006e840 0x7efd1015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.819+0000 7efd0fe00640 1 -- 172.21.15.5:0/2059237268 >> v1:172.21.15.5:6789/0 conn(0x7efd100cc840 legacy=0x7efd1006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.819+0000 7efd0fe00640 1 -- 172.21.15.5:0/2059237268 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100cee10 con 0x7efd1015b330 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.820+0000 7efd0fe00640 1 --2- 172.21.15.5:0/2059237268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd10071000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd1006ba20 tx=0x7efd0802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923c0603fc702ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.820+0000 7efd0e000640 1 -- 172.21.15.5:0/2059237268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08007740 con 0x7efd1015b330 2024-04-25T10:02:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.820+0000 7efd0e000640 1 -- 172.21.15.5:0/2059237268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd080078e0 con 0x7efd1015b330 2024-04-25T10:02:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.820+0000 7efd0e000640 1 -- 172.21.15.5:0/2059237268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08007cb0 con 0x7efd1015b330 2024-04-25T10:02:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 -- 172.21.15.5:0/2059237268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 msgr2=0x7efd10071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 --2- 172.21.15.5:0/2059237268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd10071000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efd1006ba20 tx=0x7efd0802a680 comp rx=0 tx=0).stop 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 -- 172.21.15.5:0/2059237268 shutdown_connections 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 --2- 172.21.15.5:0/2059237268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd10071000 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 --2- 172.21.15.5:0/2059237268 >> v2:172.21.15.80:3300/0 conn(0x7efd1006e840 0x7efd1015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 -- 172.21.15.5:0/2059237268 >> 172.21.15.5:0/2059237268 conn(0x7efd10066f20 msgr2=0x7efd1006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 -- 172.21.15.5:0/2059237268 shutdown_connections 2024-04-25T10:02:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.821+0000 7efd14e00640 1 -- 172.21.15.5:0/2059237268 wait complete. 2024-04-25T10:02:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.823+0000 7efd14e00640 1 Processor -- start 2024-04-25T10:02:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.823+0000 7efd14e00640 1 -- start start 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd14e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd1006e840 0x7efd100b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd14e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd100cc840 0x7efd1013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd14e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd1013c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd14e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ce350 con 0x7efd1015b330 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd14e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ce1d0 con 0x7efd1006e840 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd14e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ccc20 con 0x7efd100cc840 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd1013c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd1013c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49572/0 (socket says 172.21.15.5:49572) 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.824+0000 7efd0fe00640 1 -- 172.21.15.5:0/62449155 learned_addr learned my addr 172.21.15.5:0/62449155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0f400640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd1006e840 0x7efd100b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0ea00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd100cc840 0x7efd1013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0fe00640 1 -- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd100cc840 msgr2=0x7efd1013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0fe00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd100cc840 0x7efd1013bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0fe00640 1 -- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd1006e840 msgr2=0x7efd100b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0fe00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd1006e840 0x7efd100b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0fe00640 1 -- 172.21.15.5:0/62449155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100b0ad0 con 0x7efd1015b330 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.825+0000 7efd0fe00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd1013c210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd0802ac70 tx=0x7efd08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.826+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0803f070 con 0x7efd1015b330 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.826+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08031bc0 con 0x7efd1015b330 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.826+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08009450 con 0x7efd1015b330 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.826+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1013c9c0 con 0x7efd1015b330 2024-04-25T10:02:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.826+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1013cf80 con 0x7efd1015b330 2024-04-25T10:02:52.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.827+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10068f60 con 0x7efd1015b330 2024-04-25T10:02:52.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.828+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd0803b410 con 0x7efd1015b330 2024-04-25T10:02:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.829+0000 7efd07400640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.829+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7efd080abb30 con 0x7efd1015b330 2024-04-25T10:02:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.830+0000 7efd0f400640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.831+0000 7efd0f400640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcf006b4d0 0x7efcf006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd00005880 tx=0x7efd00009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:52.837+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd0807a6d0 con 0x7efd1015b330 2024-04-25T10:02:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.043+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd1006e410 con 0x7efd1015b330 2024-04-25T10:02:53.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.043+0000 7efd07400640 1 -- 172.21.15.5:0/62449155 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd08079fd0 con 0x7efd1015b330 2024-04-25T10:02:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcf006b4d0 msgr2=0x7efcf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcf006b4d0 0x7efcf006d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efd00005880 tx=0x7efd00009040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 msgr2=0x7efd1013c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd1013c210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efd0802ac70 tx=0x7efd08004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 shutdown_connections 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd1015b330 0x7efd1013c210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd100cc840 0x7efd1013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 --2- 172.21.15.5:0/62449155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd1006e840 0x7efd100b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.046+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 >> 172.21.15.5:0/62449155 conn(0x7efd10066f20 msgr2=0x7efd10158530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.047+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 shutdown_connections 2024-04-25T10:02:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.047+0000 7efd14e00640 1 -- 172.21.15.5:0/62449155 wait complete. 2024-04-25T10:02:53.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.249+0000 7f5fabe00640 1 Processor -- start 2024-04-25T10:02:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.249+0000 7f5fabe00640 1 -- start start 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.250+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac15b7f0 0x7f5fac0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.250+0000 7f5fabe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5fac0b27c0 0x7f5fac0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.250+0000 7f5fabe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5fac0cfd70 0x7f5fac0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.250+0000 7f5fabe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac15bbd0 con 0x7f5fac0b27c0 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.250+0000 7f5fabe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fac0cc510 con 0x7f5fac0cfd70 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.250+0000 7f5fabe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0cc7f0 con 0x7f5fac15b7f0 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5fac0b27c0 0x7f5fac0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5fac0b27c0 0x7f5fac0cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49578/0 (socket says 172.21.15.5:49578) 2024-04-25T10:02:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 -- 172.21.15.5:0/907043584 learned_addr learned my addr 172.21.15.5:0/907043584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5fab400640 1 --2- 172.21.15.5:0/907043584 >> v2:172.21.15.80:3300/0 conn(0x7f5fac0cfd70 0x7f5fac0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faaa00640 1 --2- 172.21.15.5:0/907043584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac15b7f0 0x7f5fac0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 -- 172.21.15.5:0/907043584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac15b7f0 msgr2=0x7f5fac0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 --2- 172.21.15.5:0/907043584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac15b7f0 0x7f5fac0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 -- 172.21.15.5:0/907043584 >> v2:172.21.15.80:3300/0 conn(0x7f5fac0cfd70 msgr2=0x7f5fac0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 --2- 172.21.15.5:0/907043584 >> v2:172.21.15.80:3300/0 conn(0x7f5fac0cfd70 0x7f5fac0999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5faa000640 1 -- 172.21.15.5:0/907043584 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0ccad0 con 0x7f5fac0b27c0 2024-04-25T10:02:53.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.251+0000 7f5fab400640 1 --2- 172.21.15.5:0/907043584 >> v2:172.21.15.80:3300/0 conn(0x7f5fac0cfd70 0x7f5fac0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.252+0000 7f5faa000640 1 --2- 172.21.15.5:0/907043584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0b27c0 0x7f5fac0cf830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0d2550 tx=0x7f5f9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6520189c27e0a111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fa9600640 1 -- 172.21.15.5:0/907043584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98037070 con 0x7f5fac0b27c0 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fa9600640 1 -- 172.21.15.5:0/907043584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f98032440 con 0x7f5fac0b27c0 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fa9600640 1 -- 172.21.15.5:0/907043584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98032810 con 0x7f5fac0b27c0 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 -- 172.21.15.5:0/907043584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0b27c0 msgr2=0x7f5fac0cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 --2- 172.21.15.5:0/907043584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0b27c0 0x7f5fac0cf830 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0d2550 tx=0x7f5f9802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 -- 172.21.15.5:0/907043584 shutdown_connections 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 --2- 172.21.15.5:0/907043584 >> v2:172.21.15.80:3300/0 conn(0x7f5fac0cfd70 0x7f5fac0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 --2- 172.21.15.5:0/907043584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0b27c0 0x7f5fac0cf830 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 --2- 172.21.15.5:0/907043584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac15b7f0 0x7f5fac0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 -- 172.21.15.5:0/907043584 >> 172.21.15.5:0/907043584 conn(0x7f5fac0d1470 msgr2=0x7f5fac0d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 -- 172.21.15.5:0/907043584 shutdown_connections 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.253+0000 7f5fabe00640 1 -- 172.21.15.5:0/907043584 wait complete. 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.255+0000 7f5fabe00640 1 Processor -- start 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.255+0000 7f5fabe00640 1 -- start start 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac0b27c0 0x7f5fac15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 0x7f5fac06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5fac15b7f0 0x7f5fac1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5fabe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0b2c20 con 0x7f5fac0cfd70 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5fabe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0cd090 con 0x7f5fac15b7f0 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5fabe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac15d250 con 0x7f5fac0b27c0 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.256+0000 7f5faa000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 0x7f5fac06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 0x7f5fac06cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49594/0 (socket says 172.21.15.5:49594) 2024-04-25T10:02:53.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 -- 172.21.15.5:0/2146679958 learned_addr learned my addr 172.21.15.5:0/2146679958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faaa00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac0b27c0 0x7f5fac15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5fab400640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5fac15b7f0 0x7f5fac1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 -- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac0b27c0 msgr2=0x7f5fac15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac0b27c0 0x7f5fac15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 -- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5fac15b7f0 msgr2=0x7f5fac1f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5fac15b7f0 0x7f5fac1f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 -- 172.21.15.5:0/2146679958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac15c9b0 con 0x7f5fac0cfd70 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5fab400640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5fac15b7f0 0x7f5fac1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.257+0000 7f5faa000640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 0x7f5fac06cff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000c00 tx=0x7f5f980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.258+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98041070 con 0x7f5fac0cfd70 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.258+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9803c870 con 0x7f5fac0cfd70 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.258+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac15cc90 con 0x7f5fac0cfd70 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.258+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98037040 con 0x7f5fac0cfd70 2024-04-25T10:02:53.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.258+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac15ebb0 con 0x7f5fac0cfd70 2024-04-25T10:02:53.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.259+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac068f60 con 0x7f5fac0cfd70 2024-04-25T10:02:53.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.260+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f98044070 con 0x7f5fac0cfd70 2024-04-25T10:02:53.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.261+0000 7f5fa2a00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5f8c06b4d0 0x7f5f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.261+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f5f980ab170 con 0x7f5fac0cfd70 2024-04-25T10:02:53.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.262+0000 7f5faaa00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5f8c06b4d0 0x7f5f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.263+0000 7f5faaa00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5f8c06b4d0 0x7f5f8c06d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006fd0 tx=0x7f5f9c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.265+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f98079d10 con 0x7f5fac0cfd70 2024-04-25T10:02:53.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:53.370 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.459+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fac0cb570 con 0x7f5fac0cfd70 2024-04-25T10:02:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.460+0000 7f5fa2a00640 1 -- 172.21.15.5:0/2146679958 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f98079610 con 0x7f5fac0cfd70 2024-04-25T10:02:53.465 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5f8c06b4d0 msgr2=0x7f5f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5f8c06b4d0 0x7f5f8c06d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006fd0 tx=0x7f5f9c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 msgr2=0x7f5fac06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 0x7f5fac06cff0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5f98000c00 tx=0x7f5f980046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 shutdown_connections 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5f8c06b4d0 0x7f5f8c06d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5fac15b7f0 0x7f5fac1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fac0cfd70 0x7f5fac06cff0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 --2- 172.21.15.5:0/2146679958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5fac0b27c0 0x7f5fac15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 >> 172.21.15.5:0/2146679958 conn(0x7f5fac0d1470 msgr2=0x7f5fac070030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 shutdown_connections 2024-04-25T10:02:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.462+0000 7f5fabe00640 1 -- 172.21.15.5:0/2146679958 wait complete. 2024-04-25T10:02:53.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.677+0000 7f16aa800640 1 Processor -- start 2024-04-25T10:02:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.677+0000 7f16aa800640 1 -- start start 2024-04-25T10:02:53.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.678+0000 7f16aa800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f16a415deb0 0x7f16a40b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.678+0000 7f16aa800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16a40ce030 0x7f16a40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.678+0000 7f16aa800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a40ce4c0 con 0x7f16a40b6f20 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.678+0000 7f16aa800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40d1a40 con 0x7f16a40ce030 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.678+0000 7f16aa800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40d1d20 con 0x7f16a415deb0 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16a40ce030 0x7f16a40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16a40ce030 0x7f16a40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49608/0 (socket says 172.21.15.5:49608) 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 -- 172.21.15.5:0/3529891986 learned_addr learned my addr 172.21.15.5:0/3529891986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a3400640 1 --2- 172.21.15.5:0/3529891986 >> v2:172.21.15.80:3300/0 conn(0x7f16a415deb0 0x7f16a40b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 -- 172.21.15.5:0/3529891986 >> v2:172.21.15.80:3300/0 conn(0x7f16a415deb0 msgr2=0x7f16a40b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 --2- 172.21.15.5:0/3529891986 >> v2:172.21.15.80:3300/0 conn(0x7f16a415deb0 0x7f16a40b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 -- 172.21.15.5:0/3529891986 >> v1:172.21.15.80:6789/0 conn(0x7f16a40b6f20 legacy=0x7f16a40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.679+0000 7f16a2a00640 1 -- 172.21.15.5:0/3529891986 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40d2000 con 0x7f16a40ce030 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.680+0000 7f16a2a00640 1 --2- 172.21.15.5:0/3529891986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a40ce030 0x7f16a40b69e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16a40d43f0 tx=0x7f168c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd6d51be529aca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.681+0000 7f16a2000640 1 -- 172.21.15.5:0/3529891986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c037070 con 0x7f16a40ce030 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.681+0000 7f16a2000640 1 -- 172.21.15.5:0/3529891986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c032440 con 0x7f16a40ce030 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.681+0000 7f16a2000640 1 -- 172.21.15.5:0/3529891986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c032810 con 0x7f16a40ce030 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.681+0000 7f16aa800640 1 -- 172.21.15.5:0/3529891986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a40ce030 msgr2=0x7f16a40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.681+0000 7f16aa800640 1 --2- 172.21.15.5:0/3529891986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a40ce030 0x7f16a40b69e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16a40d43f0 tx=0x7f168c02a230 comp rx=0 tx=0).stop 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.682+0000 7f16aa800640 1 -- 172.21.15.5:0/3529891986 shutdown_connections 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.682+0000 7f16aa800640 1 --2- 172.21.15.5:0/3529891986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a40ce030 0x7f16a40b69e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.682+0000 7f16aa800640 1 --2- 172.21.15.5:0/3529891986 >> v2:172.21.15.80:3300/0 conn(0x7f16a415deb0 0x7f16a40b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.682+0000 7f16aa800640 1 -- 172.21.15.5:0/3529891986 >> 172.21.15.5:0/3529891986 conn(0x7f16a40726b0 msgr2=0x7f16a4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.682+0000 7f16aa800640 1 -- 172.21.15.5:0/3529891986 shutdown_connections 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.682+0000 7f16aa800640 1 -- 172.21.15.5:0/3529891986 wait complete. 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.683+0000 7f16aa800640 1 Processor -- start 2024-04-25T10:02:53.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.684+0000 7f16aa800640 1 -- start start 2024-04-25T10:02:53.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.684+0000 7f16aa800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16a40b6f20 0x7f16a40ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.684+0000 7f16aa800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16a40ce030 0x7f16a40b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.684+0000 7f16aa800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 0x7f16a40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16aa800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40d3e70 con 0x7f16a415deb0 2024-04-25T10:02:53.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16aa800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40d25c0 con 0x7f16a40b6f20 2024-04-25T10:02:53.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16aa800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40d3ff0 con 0x7f16a40ce030 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 0x7f16a40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 0x7f16a40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49612/0 (socket says 172.21.15.5:49612) 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16a3e00640 1 -- 172.21.15.5:0/473090035 learned_addr learned my addr 172.21.15.5:0/473090035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16a3400640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16a40b6f20 0x7f16a40ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.685+0000 7f16a2a00640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16a40ce030 0x7f16a40b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a3e00640 1 -- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16a40ce030 msgr2=0x7f16a40b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a3e00640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16a40ce030 0x7f16a40b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a3e00640 1 -- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16a40b6f20 msgr2=0x7f16a40ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a3e00640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16a40b6f20 0x7f16a40ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a3e00640 1 -- 172.21.15.5:0/473090035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4155680 con 0x7f16a415deb0 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a2a00640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16a40ce030 0x7f16a40b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16a3e00640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 0x7f16a40cb5d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c410 tx=0x7f169c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c011790 con 0x7f16a415deb0 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c017070 con 0x7f16a415deb0 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c011bb0 con 0x7f16a415deb0 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.686+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a40cd620 con 0x7f16a415deb0 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.687+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a40cadd0 con 0x7f16a415deb0 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.688+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a4068f60 con 0x7f16a415deb0 2024-04-25T10:02:53.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.689+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169c01c460 con 0x7f16a415deb0 2024-04-25T10:02:53.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.689+0000 7f169b400640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f168006b4d0 0x7f168006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.690+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f169c08a1a0 con 0x7f16a415deb0 2024-04-25T10:02:53.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.690+0000 7f16a3400640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f168006b4d0 0x7f168006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.691+0000 7f16a3400640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f168006b4d0 0x7f168006d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f16940099c0 tx=0x7f1694003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.694+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f169c021020 con 0x7f16a415deb0 2024-04-25T10:02:53.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.900+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16a40b9040 con 0x7f16a415deb0 2024-04-25T10:02:53.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.901+0000 7f169b400640 1 -- 172.21.15.5:0/473090035 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f169c058ca0 con 0x7f16a415deb0 2024-04-25T10:02:53.906 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:53.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.903+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f168006b4d0 msgr2=0x7f168006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.903+0000 7f16aa800640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f168006b4d0 0x7f168006d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f16940099c0 tx=0x7f1694003990 comp rx=0 tx=0).stop 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.903+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 msgr2=0x7f16a40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.903+0000 7f16aa800640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 0x7f16a40cb5d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c410 tx=0x7f169c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 shutdown_connections 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a415deb0 0x7f16a40cb5d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16a40ce030 0x7f16a40b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f168006b4d0 0x7f168006d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 --2- 172.21.15.5:0/473090035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16a40b6f20 0x7f16a40ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 >> 172.21.15.5:0/473090035 conn(0x7f16a40726b0 msgr2=0x7f16a40d3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 shutdown_connections 2024-04-25T10:02:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:53.904+0000 7f16aa800640 1 -- 172.21.15.5:0/473090035 wait complete. 2024-04-25T10:02:53.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.133+0000 7f5d32e00640 1 Processor -- start 2024-04-25T10:02:54.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.133+0000 7f5d32e00640 1 -- start start 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.134+0000 7f5d32e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c15ec80 0x7f5d2c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d32e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d2c0d4000 0x7f5d2c160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d32e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c15ffa0 con 0x7f5d2c161240 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d32e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2c160240 con 0x7f5d2c0d4000 2024-04-25T10:02:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d32e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c160520 con 0x7f5d2c15ec80 2024-04-25T10:02:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d31000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d2c0d4000 0x7f5d2c160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d31000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d2c0d4000 0x7f5d2c160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:41916/0 (socket says 172.21.15.5:41916) 2024-04-25T10:02:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.135+0000 7f5d31000640 1 -- 172.21.15.5:0/1689189684 learned_addr learned my addr 172.21.15.5:0/1689189684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.136+0000 7f5d31a00640 1 --2- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c15ec80 0x7f5d2c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.136+0000 7f5d31000640 1 -- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c15ec80 msgr2=0x7f5d2c0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.136+0000 7f5d31000640 1 --2- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c15ec80 0x7f5d2c0b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.136+0000 7f5d31000640 1 -- 172.21.15.5:0/1689189684 >> v1:172.21.15.80:6789/0 conn(0x7f5d2c161240 legacy=0x7f5d2c1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.136+0000 7f5d31000640 1 -- 172.21.15.5:0/1689189684 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c160800 con 0x7f5d2c0d4000 2024-04-25T10:02:54.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.138+0000 7f5d31000640 1 --2- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c160d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009850 tx=0x7f5d1802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadb0cf8faf8ee04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.139+0000 7f5d2be00640 1 -- 172.21.15.5:0/1689189684 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18004030 con 0x7f5d2c0d4000 2024-04-25T10:02:54.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.139+0000 7f5d2be00640 1 -- 172.21.15.5:0/1689189684 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d180041d0 con 0x7f5d2c0d4000 2024-04-25T10:02:54.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.139+0000 7f5d2be00640 1 -- 172.21.15.5:0/1689189684 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18003610 con 0x7f5d2c0d4000 2024-04-25T10:02:54.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 -- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 msgr2=0x7f5d2c160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 --2- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c160d00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009850 tx=0x7f5d1802a230 comp rx=0 tx=0).stop 2024-04-25T10:02:54.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 -- 172.21.15.5:0/1689189684 shutdown_connections 2024-04-25T10:02:54.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 --2- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c160d00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 --2- 172.21.15.5:0/1689189684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c15ec80 0x7f5d2c0b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 -- 172.21.15.5:0/1689189684 >> 172.21.15.5:0/1689189684 conn(0x7f5d2c08af80 msgr2=0x7f5d2c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 -- 172.21.15.5:0/1689189684 shutdown_connections 2024-04-25T10:02:54.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.140+0000 7f5d32e00640 1 -- 172.21.15.5:0/1689189684 wait complete. 2024-04-25T10:02:54.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.141+0000 7f5d32e00640 1 Processor -- start 2024-04-25T10:02:54.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.142+0000 7f5d32e00640 1 -- start start 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 0x7f5d2c156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c161240 0x7f5d2c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0d4460 con 0x7f5d2c15ec80 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0b4c50 con 0x7f5d2c0d4000 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c161e00 con 0x7f5d2c161240 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 0x7f5d2c156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 0x7f5d2c156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49618/0 (socket says 172.21.15.5:49618) 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 -- 172.21.15.5:0/3541887200 learned_addr learned my addr 172.21.15.5:0/3541887200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31a00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d32400640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c161240 0x7f5d2c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 -- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c161240 msgr2=0x7f5d2c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c161240 0x7f5d2c157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 -- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 msgr2=0x7f5d2c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.143+0000 7f5d31000640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d31000640 1 -- 172.21.15.5:0/3541887200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c153ec0 con 0x7f5d2c15ec80 2024-04-25T10:02:54.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d32400640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c161240 0x7f5d2c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d31000640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 0x7f5d2c156ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009970 tx=0x7f5d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18041030 con 0x7f5d2c15ec80 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d18003bd0 con 0x7f5d2c15ec80 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c0cce70 con 0x7f5d2c15ec80 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d18004030 con 0x7f5d2c15ec80 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.144+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c0cd430 con 0x7f5d2c15ec80 2024-04-25T10:02:54.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.146+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c068f60 con 0x7f5d2c15ec80 2024-04-25T10:02:54.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.147+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d18044070 con 0x7f5d2c15ec80 2024-04-25T10:02:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.148+0000 7f5d29600640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0c06b4d0 0x7f5d0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.148+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f5d180ab0b0 con 0x7f5d2c15ec80 2024-04-25T10:02:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.149+0000 7f5d31a00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0c06b4d0 0x7f5d0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.150+0000 7f5d31a00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0c06b4d0 0x7f5d0c06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009ac0 tx=0x7f5d20008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.153+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d18079b20 con 0x7f5d2c15ec80 2024-04-25T10:02:54.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.342+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5d2c0d0e50 con 0x7f5d2c15ec80 2024-04-25T10:02:54.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.343+0000 7f5d29600640 1 -- 172.21.15.5:0/3541887200 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5d18079420 con 0x7f5d2c15ec80 2024-04-25T10:02:54.349 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0c06b4d0 msgr2=0x7f5d0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0c06b4d0 0x7f5d0c06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009ac0 tx=0x7f5d20008040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 msgr2=0x7f5d2c156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 0x7f5d2c156ee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d18009970 tx=0x7f5d180046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 shutdown_connections 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d2c161240 0x7f5d2c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2c15ec80 0x7f5d2c156ee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0c06b4d0 0x7f5d0c06d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 --2- 172.21.15.5:0/3541887200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2c0d4000 0x7f5d2c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.347+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 >> 172.21.15.5:0/3541887200 conn(0x7f5d2c08af80 msgr2=0x7f5d2c0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.348+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 shutdown_connections 2024-04-25T10:02:54.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.348+0000 7f5d32e00640 1 -- 172.21.15.5:0/3541887200 wait complete. 2024-04-25T10:02:54.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.572+0000 7f1a9f200640 1 Processor -- start 2024-04-25T10:02:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.573+0000 7f1a9f200640 1 -- start start 2024-04-25T10:02:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.574+0000 7f1a9f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 0x7f1aa006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.574+0000 7f1a9f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa0071d30 con 0x7f1aa006e640 2024-04-25T10:02:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.574+0000 7f1a9f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa006d620 con 0x7f1aa015d510 2024-04-25T10:02:54.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.574+0000 7f1a9f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0142180 con 0x7f1aa0141da0 2024-04-25T10:02:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.574+0000 7f1a9d400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1aa006e640 0x7f1aa015ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39822/0 (socket says 172.21.15.5:39822) 2024-04-25T10:02:54.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.574+0000 7f1a9d400640 1 -- 172.21.15.5:0/1121555677 learned_addr learned my addr 172.21.15.5:0/1121555677 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:54.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9de00640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 0x7f1aa006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216549032 0 0) 0x7f1aa0071d30 con 0x7f1aa006e640 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a80003520 con 0x7f1aa006e640 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993691873 0 0) 0x7f1a80003520 con 0x7f1aa006e640 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 msgr2=0x7f1aa006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 0x7f1aa006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 >> v1:172.21.15.80:6789/0 conn(0x7f1aa015d510 legacy=0x7f1aa00cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa006d900 con 0x7f1aa006e640 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1a880019f0 con 0x7f1aa006e640 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 >> v1:172.21.15.5:6789/0 conn(0x7f1aa006e640 legacy=0x7f1aa015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 0x7f1a800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80003fb0 0x7f1a80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0142180 con 0x7f1a80004d40 2024-04-25T10:02:54.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8000bb00 con 0x7f1a80003fb0 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.575+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8000bd00 con 0x7f1aa0141da0 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9e800640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9d400640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80003fb0 0x7f1a80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9de00640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 0x7f1a800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9e800640 1 -- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 msgr2=0x7f1a800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9e800640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa0141da0 0x7f1a800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9e800640 1 -- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80003fb0 msgr2=0x7f1a80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9e800640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80003fb0 0x7f1a80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.576+0000 7f1a9e800640 1 -- 172.21.15.5:0/1121555677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80003520 con 0x7f1a80004d40 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.577+0000 7f1a9e800640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1a98001750 tx=0x7f1a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1541cb37d002bda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.577+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98001370 con 0x7f1a80004d40 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.577+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a980052d0 con 0x7f1a80004d40 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.577+0000 7f1a9ca00640 1 -- 172.21.15.5:0/1121555677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9802abe0 con 0x7f1a80004d40 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9f200640 1 -- 172.21.15.5:0/1121555677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a80004d40 msgr2=0x7f1a800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9f200640 1 --2- 172.21.15.5:0/1121555677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a80004d40 0x7f1a800051c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1a98001750 tx=0x7f1a80005830 comp rx=0 tx=0).stop 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9e800640 1 -- 172.21.15.5:0/1121555677 reap_dead start 2024-04-25T10:02:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9f200640 1 -- 172.21.15.5:0/1121555677 shutdown_connections 2024-04-25T10:02:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9f200640 1 -- 172.21.15.5:0/1121555677 >> 172.21.15.5:0/1121555677 conn(0x7f1aa0066f20 msgr2=0x7f1aa006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9f200640 1 -- 172.21.15.5:0/1121555677 shutdown_connections 2024-04-25T10:02:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.578+0000 7f1a9f200640 1 -- 172.21.15.5:0/1121555677 wait complete. 2024-04-25T10:02:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.580+0000 7f1a9f200640 1 Processor -- start 2024-04-25T10:02:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.580+0000 7f1a9f200640 1 -- start start 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80004d40 0x7f1aa0155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa012e7e0 0x7f1aa01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 0x7f1aa0159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00cdb90 con 0x7f1aa012ec60 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa00cda10 con 0x7f1a80004d40 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa006e100 con 0x7f1aa012e7e0 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 0x7f1aa0159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 0x7f1aa0159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49632/0 (socket says 172.21.15.5:49632) 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9e800640 1 -- 172.21.15.5:0/2235366123 learned_addr learned my addr 172.21.15.5:0/2235366123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.581+0000 7f1a9d400640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa012e7e0 0x7f1aa01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9de00640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80004d40 0x7f1aa0155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9e800640 1 -- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa012e7e0 msgr2=0x7f1aa01578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9e800640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa012e7e0 0x7f1aa01578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9e800640 1 -- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80004d40 msgr2=0x7f1aa0155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9e800640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80004d40 0x7f1aa0155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9e800640 1 -- 172.21.15.5:0/2235366123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa0158380 con 0x7f1aa012ec60 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9de00640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80004d40 0x7f1aa0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9e800640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 0x7f1aa0159be0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1a98001870 tx=0x7f1a98001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98037040 con 0x7f1aa012ec60 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a98040070 con 0x7f1aa012ec60 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a98007a30 con 0x7f1aa012ec60 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.582+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa0158660 con 0x7f1aa012ec60 2024-04-25T10:02:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.583+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa01e54a0 con 0x7f1aa012ec60 2024-04-25T10:02:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.584+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa01e5670 con 0x7f1aa012ec60 2024-04-25T10:02:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.585+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a980052d0 con 0x7f1aa012ec60 2024-04-25T10:02:54.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.586+0000 7f1a96000640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a7c06b4d0 0x7f1a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.586+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f1a980a9d30 con 0x7f1aa012ec60 2024-04-25T10:02:54.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.587+0000 7f1a9de00640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a7c06b4d0 0x7f1a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.588+0000 7f1a9de00640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a7c06b4d0 0x7f1a7c06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009a00 tx=0x7f1a90008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.591+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a98078830 con 0x7f1aa012ec60 2024-04-25T10:02:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.783+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aa015cfa0 con 0x7f1aa012ec60 2024-04-25T10:02:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.785+0000 7f1a96000640 1 -- 172.21.15.5:0/2235366123 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f1a98078130 con 0x7f1aa012ec60 2024-04-25T10:02:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:54.791 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a7c06b4d0 msgr2=0x7f1a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a7c06b4d0 0x7f1a7c06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009a00 tx=0x7f1a90008040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 msgr2=0x7f1aa0159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 0x7f1aa0159be0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1a98001870 tx=0x7f1a98001120 comp rx=0 tx=0).stop 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 shutdown_connections 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa012ec60 0x7f1aa0159be0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1aa012e7e0 0x7f1aa01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a7c06b4d0 0x7f1a7c06d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 --2- 172.21.15.5:0/2235366123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a80004d40 0x7f1aa0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 >> 172.21.15.5:0/2235366123 conn(0x7f1aa0066f20 msgr2=0x7f1aa00af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 shutdown_connections 2024-04-25T10:02:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:54.788+0000 7f1a9f200640 1 -- 172.21.15.5:0/2235366123 wait complete. 2024-04-25T10:02:54.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:54.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:55.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.014+0000 7f80d1000640 1 Processor -- start 2024-04-25T10:02:55.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.014+0000 7f80d1000640 1 -- start start 2024-04-25T10:02:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80d1000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80d1000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80cc15b330 0x7f80cc071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80d1000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0ce570 con 0x7f80cc0cc840 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80d1000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc0ce850 con 0x7f80cc15b330 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80d1000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0ceb30 con 0x7f80cc06e840 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cb400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f80cc0cc840 0x7f80cc06e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39826/0 (socket says 172.21.15.5:39826) 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cb400640 1 -- 172.21.15.5:0/1985991886 learned_addr learned my addr 172.21.15.5:0/1985991886 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cbe00640 1 --2- 172.21.15.5:0/1985991886 >> v2:172.21.15.5:3300/0 conn(0x7f80cc15b330 0x7f80cc071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80caa00640 1 --2- 172.21.15.5:0/1985991886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80ca000640 1 -- 172.21.15.5:0/1985991886 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940244032 0 0) 0x7f80cc0ce570 con 0x7f80cc0cc840 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80ca000640 1 -- 172.21.15.5:0/1985991886 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b0003520 con 0x7f80cc0cc840 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cbe00640 1 -- 172.21.15.5:0/1985991886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 msgr2=0x7f80cc15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cbe00640 1 --2- 172.21.15.5:0/1985991886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cbe00640 1 -- 172.21.15.5:0/1985991886 >> v1:172.21.15.5:6789/0 conn(0x7f80cc0cc840 legacy=0x7f80cc06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.016+0000 7f80cbe00640 1 -- 172.21.15.5:0/1985991886 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0cee10 con 0x7f80cc15b330 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.017+0000 7f80cbe00640 1 --2- 172.21.15.5:0/1985991886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc071000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009d90 tx=0x7f80c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14886c00beb41201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.017+0000 7f80ca000640 1 -- 172.21.15.5:0/1985991886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4007740 con 0x7f80cc15b330 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.017+0000 7f80ca000640 1 -- 172.21.15.5:0/1985991886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c40078e0 con 0x7f80cc15b330 2024-04-25T10:02:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.017+0000 7f80ca000640 1 -- 172.21.15.5:0/1985991886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4007cb0 con 0x7f80cc15b330 2024-04-25T10:02:55.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 -- 172.21.15.5:0/1985991886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 msgr2=0x7f80cc071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 --2- 172.21.15.5:0/1985991886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc071000 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009d90 tx=0x7f80c402a660 comp rx=0 tx=0).stop 2024-04-25T10:02:55.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 -- 172.21.15.5:0/1985991886 shutdown_connections 2024-04-25T10:02:55.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 --2- 172.21.15.5:0/1985991886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc071000 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 --2- 172.21.15.5:0/1985991886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 -- 172.21.15.5:0/1985991886 >> 172.21.15.5:0/1985991886 conn(0x7f80cc066f20 msgr2=0x7f80cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 -- 172.21.15.5:0/1985991886 shutdown_connections 2024-04-25T10:02:55.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.019+0000 7f80d1000640 1 -- 172.21.15.5:0/1985991886 wait complete. 2024-04-25T10:02:55.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.020+0000 7f80d1000640 1 Processor -- start 2024-04-25T10:02:55.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.020+0000 7f80d1000640 1 -- start start 2024-04-25T10:02:55.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80d1000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80d1000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80cc0cc840 0x7f80cc0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80d1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80d1000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0ce350 con 0x7f80cc15b330 2024-04-25T10:02:55.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80d1000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0ce1d0 con 0x7f80cc0cc840 2024-04-25T10:02:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80d1000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0ccc20 con 0x7f80cc06e840 2024-04-25T10:02:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc14ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49646/0 (socket says 172.21.15.5:49646) 2024-04-25T10:02:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 -- 172.21.15.5:0/1258625468 learned_addr learned my addr 172.21.15.5:0/1258625468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80caa00640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80cc0cc840 0x7f80cc0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cb400640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 -- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 msgr2=0x7f80cc152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 -- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80cc0cc840 msgr2=0x7f80cc0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80cc0cc840 0x7f80cc0b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.021+0000 7f80cbe00640 1 -- 172.21.15.5:0/1258625468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc154e00 con 0x7f80cc15b330 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.022+0000 7f80cbe00640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc14ba20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009eb0 tx=0x7f80c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.022+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c403f070 con 0x7f80cc15b330 2024-04-25T10:02:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.022+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c4031bc0 con 0x7f80cc15b330 2024-04-25T10:02:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.022+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4009450 con 0x7f80cc15b330 2024-04-25T10:02:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.022+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc1550e0 con 0x7f80cc15b330 2024-04-25T10:02:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.022+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1556a0 con 0x7f80cc15b330 2024-04-25T10:02:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.023+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc068f60 con 0x7f80cc15b330 2024-04-25T10:02:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.024+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80c403b410 con 0x7f80cc15b330 2024-04-25T10:02:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.025+0000 7f80c3400640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f80ac06b4d0 0x7f80ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.025+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f80c40abda0 con 0x7f80cc15b330 2024-04-25T10:02:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.026+0000 7f80cb400640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f80ac06b4d0 0x7f80ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.027+0000 7f80cb400640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f80ac06b4d0 0x7f80ac06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80bc000f60 tx=0x7f80bc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.030+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80c407a810 con 0x7f80cc15b330 2024-04-25T10:02:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.225+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80cc151290 con 0x7f80cc15b330 2024-04-25T10:02:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.226+0000 7f80c3400640 1 -- 172.21.15.5:0/1258625468 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80c407a110 con 0x7f80cc15b330 2024-04-25T10:02:55.232 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f80ac06b4d0 msgr2=0x7f80ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f80ac06b4d0 0x7f80ac06d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f80bc000f60 tx=0x7f80bc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 msgr2=0x7f80cc14ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc14ba20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009eb0 tx=0x7f80c4004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 shutdown_connections 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80cc15b330 0x7f80cc14ba20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f80ac06b4d0 0x7f80ac06d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80cc0cc840 0x7f80cc0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 --2- 172.21.15.5:0/1258625468 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80cc06e840 0x7f80cc152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 >> 172.21.15.5:0/1258625468 conn(0x7f80cc066f20 msgr2=0x7f80cc0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 shutdown_connections 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.229+0000 7f80d1000640 1 -- 172.21.15.5:0/1258625468 wait complete. 2024-04-25T10:02:55.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.469+0000 7fbdb9800640 1 Processor -- start 2024-04-25T10:02:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.469+0000 7fbdb9800640 1 -- start start 2024-04-25T10:02:55.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb9800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbdb40d3f90 0x7fbdb4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb9800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb41606e0 con 0x7fbdb40b48c0 2024-04-25T10:02:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb9800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb41609c0 con 0x7fbdb4161bc0 2024-04-25T10:02:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb9800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb4161ff0 con 0x7fbdb40d3f90 2024-04-25T10:02:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb3e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbdb40b48c0 0x7fbdb415fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39836/0 (socket says 172.21.15.5:39836) 2024-04-25T10:02:55.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb3e00640 1 -- 172.21.15.5:0/3097476970 learned_addr learned my addr 172.21.15.5:0/3097476970 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdabe00640 1 --2- 172.21.15.5:0/3097476970 >> v2:172.21.15.5:3300/0 conn(0x7fbdb40d3f90 0x7fbdb4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb3400640 1 -- 172.21.15.5:0/3097476970 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760797385 0 0) 0x7fbdb41606e0 con 0x7fbdb40b48c0 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdb3400640 1 -- 172.21.15.5:0/3097476970 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd98003520 con 0x7fbdb40b48c0 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdabe00640 1 -- 172.21.15.5:0/3097476970 >> v1:172.21.15.80:6789/0 conn(0x7fbdb4161bc0 legacy=0x7fbdb40b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdabe00640 1 -- 172.21.15.5:0/3097476970 >> v1:172.21.15.5:6789/0 conn(0x7fbdb40b48c0 legacy=0x7fbdb415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.471+0000 7fbdabe00640 1 -- 172.21.15.5:0/3097476970 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4160ca0 con 0x7fbdb40d3f90 2024-04-25T10:02:55.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.472+0000 7fbdabe00640 1 --2- 172.21.15.5:0/3097476970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb40d3f90 0x7fbdb4161680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009790 tx=0x7fbd9c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fd12b079b87646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.473+0000 7fbdb3400640 1 -- 172.21.15.5:0/3097476970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c037070 con 0x7fbdb40d3f90 2024-04-25T10:02:55.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.473+0000 7fbdb3400640 1 -- 172.21.15.5:0/3097476970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c032440 con 0x7fbdb40d3f90 2024-04-25T10:02:55.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.473+0000 7fbdb3400640 1 -- 172.21.15.5:0/3097476970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c032810 con 0x7fbdb40d3f90 2024-04-25T10:02:55.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.473+0000 7fbdb9800640 1 -- 172.21.15.5:0/3097476970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb40d3f90 msgr2=0x7fbdb4161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.473+0000 7fbdb9800640 1 --2- 172.21.15.5:0/3097476970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb40d3f90 0x7fbdb4161680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009790 tx=0x7fbd9c02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:55.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.474+0000 7fbdb9800640 1 -- 172.21.15.5:0/3097476970 shutdown_connections 2024-04-25T10:02:55.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.474+0000 7fbdb9800640 1 --2- 172.21.15.5:0/3097476970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb40d3f90 0x7fbdb4161680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.474+0000 7fbdb9800640 1 -- 172.21.15.5:0/3097476970 >> 172.21.15.5:0/3097476970 conn(0x7fbdb408af80 msgr2=0x7fbdb40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.474+0000 7fbdb9800640 1 -- 172.21.15.5:0/3097476970 shutdown_connections 2024-04-25T10:02:55.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.474+0000 7fbdb9800640 1 -- 172.21.15.5:0/3097476970 wait complete. 2024-04-25T10:02:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.476+0000 7fbdb9800640 1 Processor -- start 2024-04-25T10:02:55.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 -- start start 2024-04-25T10:02:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdb40b48c0 0x7fbdb41588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdb40d3f90 0x7fbdb40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 0x7fbdb415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40b4ca0 con 0x7fbdb4161bc0 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb4161260 con 0x7fbdb40d3f90 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb9800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb40b3280 con 0x7fbdb40b48c0 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 0x7fbdb415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 0x7fbdb415dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49674/0 (socket says 172.21.15.5:49674) 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 -- 172.21.15.5:0/1329953843 learned_addr learned my addr 172.21.15.5:0/1329953843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb3e00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdb40b48c0 0x7fbdb41588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdabe00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdb40d3f90 0x7fbdb40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 -- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdb40b48c0 msgr2=0x7fbdb41588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdb40b48c0 0x7fbdb41588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 -- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdb40d3f90 msgr2=0x7fbdb40b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdb40d3f90 0x7fbdb40b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdb8e00640 1 -- 172.21.15.5:0/1329953843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb40c7510 con 0x7fbdb4161bc0 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.477+0000 7fbdabe00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdb40d3f90 0x7fbdb40b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.478+0000 7fbdb8e00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 0x7fbdb415dca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c760 tx=0x7fbdac00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.478+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac016020 con 0x7fbdb4161bc0 2024-04-25T10:02:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.478+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac006d50 con 0x7fbdb4161bc0 2024-04-25T10:02:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.478+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac003bb0 con 0x7fbdb4161bc0 2024-04-25T10:02:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.478+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb40c77f0 con 0x7fbdb4161bc0 2024-04-25T10:02:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.478+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb40cbd00 con 0x7fbdb4161bc0 2024-04-25T10:02:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.479+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb4068f60 con 0x7fbdb4161bc0 2024-04-25T10:02:55.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.480+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdac0050d0 con 0x7fbdb4161bc0 2024-04-25T10:02:55.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.482+0000 7fbdb0c00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbd9006b500 0x7fbd9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.482+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fbdac08a170 con 0x7fbdb4161bc0 2024-04-25T10:02:55.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.482+0000 7fbdb3e00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbd9006b500 0x7fbd9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.483+0000 7fbdb3e00640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbd9006b500 0x7fbd9006d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbda40011e0 tx=0x7fbda4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.486+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdac020020 con 0x7fbdb4161bc0 2024-04-25T10:02:55.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.690+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbdb40d0dd0 con 0x7fbdb4161bc0 2024-04-25T10:02:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.690+0000 7fbdb0c00640 1 -- 172.21.15.5:0/1329953843 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbdac058c70 con 0x7fbdb4161bc0 2024-04-25T10:02:55.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbd9006b500 msgr2=0x7fbd9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbd9006b500 0x7fbd9006d9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbda40011e0 tx=0x7fbda4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 msgr2=0x7fbdb415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 0x7fbdb415dca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c760 tx=0x7fbdac00cb60 comp rx=0 tx=0).stop 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 shutdown_connections 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb4161bc0 0x7fbdb415dca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbd9006b500 0x7fbd9006d9c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbdb40d3f90 0x7fbdb40b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 --2- 172.21.15.5:0/1329953843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbdb40b48c0 0x7fbdb41588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 >> 172.21.15.5:0/1329953843 conn(0x7fbdb408af80 msgr2=0x7fbdb40d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 shutdown_connections 2024-04-25T10:02:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.694+0000 7fbdb9800640 1 -- 172.21.15.5:0/1329953843 wait complete. 2024-04-25T10:02:55.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.931+0000 7fbbbf000640 1 Processor -- start 2024-04-25T10:02:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.931+0000 7fbbbf000640 1 -- start start 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.932+0000 7fbbbf000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbbb8162010 0x7fbbb80b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbbf000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb80b2d60 con 0x7fbbb8161c30 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbbf000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb80cd4c0 con 0x7fbbb8162da0 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbbf000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb80b4bd0 con 0x7fbbb8162010 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbb7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbbb8161c30 0x7fbbb80b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39840/0 (socket says 172.21.15.5:39840) 2024-04-25T10:02:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbb7e00640 1 -- 172.21.15.5:0/4189912355 learned_addr learned my addr 172.21.15.5:0/4189912355 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbb7400640 1 --2- 172.21.15.5:0/4189912355 >> v2:172.21.15.80:3300/0 conn(0x7fbbb8162010 0x7fbbb80b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442529622 0 0) 0x7fbbb80b2d60 con 0x7fbbb8161c30 2024-04-25T10:02:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.933+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb9c003520 con 0x7fbbb8161c30 2024-04-25T10:02:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348703779 0 0) 0x7fbb9c003520 con 0x7fbbb8161c30 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 >> v2:172.21.15.80:3300/0 conn(0x7fbbb8162010 msgr2=0x7fbbb80b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 --2- 172.21.15.5:0/4189912355 >> v2:172.21.15.80:3300/0 conn(0x7fbbb8162010 0x7fbbb80b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 >> v1:172.21.15.80:6789/0 conn(0x7fbbb8162da0 legacy=0x7fbbb8071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb80cd760 con 0x7fbbb8161c30 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fbba8001670 con 0x7fbbb8161c30 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 >> v1:172.21.15.5:6789/0 conn(0x7fbbb8161c30 legacy=0x7fbbb80b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb9c0039b0 0x7fbb9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c004800 0x7fbb9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c0051c0 0x7fbb9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b4bd0 con 0x7fbb9c0051c0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c00c160 con 0x7fbb9c004800 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c00c360 con 0x7fbb9c0039b0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbbd000640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c0051c0 0x7fbb9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb7400640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c004800 0x7fbb9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.934+0000 7fbbb7e00640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb9c0039b0 0x7fbb9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.935+0000 7fbbbd000640 1 -- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb9c0039b0 msgr2=0x7fbb9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.935+0000 7fbbbd000640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb9c0039b0 0x7fbb9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.935+0000 7fbbbd000640 1 -- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c004800 msgr2=0x7fbb9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.935+0000 7fbbbd000640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c004800 0x7fbb9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.935+0000 7fbbbd000640 1 -- 172.21.15.5:0/4189912355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9c003520 con 0x7fbb9c0051c0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.936+0000 7fbbbd000640 1 -- 172.21.15.5:0/4189912355 reap_dead start 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.936+0000 7fbbbd000640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c0051c0 0x7fbb9c00b9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0001e50 tx=0x7fbb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1d5800b0887a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.936+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0001a70 con 0x7fbb9c0051c0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.936+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0003970 con 0x7fbb9c0051c0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.936+0000 7fbbb6a00640 1 -- 172.21.15.5:0/4189912355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0003d60 con 0x7fbb9c0051c0 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 -- 172.21.15.5:0/4189912355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c0051c0 msgr2=0x7fbb9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c0051c0 0x7fbb9c00b9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0001e50 tx=0x7fbb9c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 -- 172.21.15.5:0/4189912355 shutdown_connections 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 --2- 172.21.15.5:0/4189912355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb9c0051c0 0x7fbb9c00b9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 -- 172.21.15.5:0/4189912355 >> 172.21.15.5:0/4189912355 conn(0x7fbbb806ab60 msgr2=0x7fbbb806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 -- 172.21.15.5:0/4189912355 shutdown_connections 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.937+0000 7fbbbf000640 1 -- 172.21.15.5:0/4189912355 wait complete. 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.939+0000 7fbbbf000640 1 Processor -- start 2024-04-25T10:02:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.939+0000 7fbbbf000640 1 -- start start 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbf000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c0051c0 0x7fbbb81e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbf000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 0x7fbbb8150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbf000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbbb8150a60 0x7fbbb8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbf000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb8163250 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbf000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80cda00 con 0x7fbb9c0051c0 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbf000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb80b4010 con 0x7fbbb8150a60 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbb7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 0x7fbbb8150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbb7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 0x7fbbb8150520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49694/0 (socket says 172.21.15.5:49694) 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbb7400640 1 -- 172.21.15.5:0/1984023338 learned_addr learned my addr 172.21.15.5:0/1984023338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbbd000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbbb8150a60 0x7fbbb8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.940+0000 7fbbb7e00640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c0051c0 0x7fbbb81e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbb7400640 1 -- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbbb8150a60 msgr2=0x7fbbb8150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbb7400640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbbb8150a60 0x7fbbb8150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbb7400640 1 -- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c0051c0 msgr2=0x7fbbb81e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbb7400640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c0051c0 0x7fbbb81e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbb7400640 1 -- 172.21.15.5:0/1984023338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb81e86c0 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbb7400640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 0x7fbbb8150520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbba40118e0 tx=0x7fbba4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400faf0 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400fc90 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb8151f40 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4014440 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.941+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb8152500 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.943+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb8068f60 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.944+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbba4012040 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.945+0000 7fbbafe00640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb9806b4d0 0x7fbb9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.945+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fbba408ad20 con 0x7fbbb81e8ae0 2024-04-25T10:02:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.945+0000 7fbbb7e00640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb9806b4d0 0x7fbb9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.946+0000 7fbbb7e00640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb9806b4d0 0x7fbb9806d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbba8001eb0 tx=0x7fbba8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:55.949+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbba4019030 con 0x7fbbb81e8ae0 2024-04-25T10:02:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.149+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbb8067c30 con 0x7fbbb81e8ae0 2024-04-25T10:02:56.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.150+0000 7fbbafe00640 1 -- 172.21.15.5:0/1984023338 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbba4059140 con 0x7fbbb81e8ae0 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.152+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb9806b4d0 msgr2=0x7fbb9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.152+0000 7fbbbf000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb9806b4d0 0x7fbb9806d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbba8001eb0 tx=0x7fbba8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 msgr2=0x7fbbb8150520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 0x7fbbb8150520 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbba40118e0 tx=0x7fbba4011da0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 shutdown_connections 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbbb8150a60 0x7fbbb8150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb81e8ae0 0x7fbbb8150520 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb9806b4d0 0x7fbb9806d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 --2- 172.21.15.5:0/1984023338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb9c0051c0 0x7fbbb81e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 >> 172.21.15.5:0/1984023338 conn(0x7fbbb806ab60 msgr2=0x7fbbb8071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 shutdown_connections 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.153+0000 7fbbbf000640 1 -- 172.21.15.5:0/1984023338 wait complete. 2024-04-25T10:02:56.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:56.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.390+0000 7fa220a00640 1 Processor -- start 2024-04-25T10:02:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.390+0000 7fa220a00640 1 -- start start 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa220a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa21c0d0780 0x7fa21c0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa220a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa21c0b9130 0x7fa21c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa220a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9590 0x7fa21c0d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa220a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0d10a0 con 0x7fa21c0b9130 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa220a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c0b7ef0 con 0x7fa21c0d0780 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa220a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0b81d0 con 0x7fa21c0b9590 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa219400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa21c0b9130 0x7fa21c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa219400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa21c0b9130 0x7fa21c0d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49704/0 (socket says 172.21.15.5:49704) 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa219400640 1 -- 172.21.15.5:0/429600800 learned_addr learned my addr 172.21.15.5:0/429600800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.392+0000 7fa21a800640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9590 0x7fa21c0d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa219e00640 1 --2- 172.21.15.5:0/429600800 >> v2:172.21.15.80:3300/0 conn(0x7fa21c0d0780 0x7fa21c0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa219400640 1 -- 172.21.15.5:0/429600800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9590 msgr2=0x7fa21c0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa219400640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9590 0x7fa21c0d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa219400640 1 -- 172.21.15.5:0/429600800 >> v2:172.21.15.80:3300/0 conn(0x7fa21c0d0780 msgr2=0x7fa21c0d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa219400640 1 --2- 172.21.15.5:0/429600800 >> v2:172.21.15.80:3300/0 conn(0x7fa21c0d0780 0x7fa21c0d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa219400640 1 -- 172.21.15.5:0/429600800 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0b84b0 con 0x7fa21c0b9130 2024-04-25T10:02:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.393+0000 7fa21a800640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9590 0x7fa21c0d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:56.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.394+0000 7fa219400640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9130 0x7fa21c0d1c30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b6180 tx=0x7fa20802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16886aee52590d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.395+0000 7fa218a00640 1 -- 172.21.15.5:0/429600800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208037070 con 0x7fa21c0b9130 2024-04-25T10:02:56.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.395+0000 7fa218a00640 1 -- 172.21.15.5:0/429600800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208032440 con 0x7fa21c0b9130 2024-04-25T10:02:56.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.395+0000 7fa218a00640 1 -- 172.21.15.5:0/429600800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208032810 con 0x7fa21c0b9130 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.395+0000 7fa220a00640 1 -- 172.21.15.5:0/429600800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9130 msgr2=0x7fa21c0d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.395+0000 7fa220a00640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9130 0x7fa21c0d1c30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0b6180 tx=0x7fa20802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 -- 172.21.15.5:0/429600800 shutdown_connections 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9590 0x7fa21c0d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 --2- 172.21.15.5:0/429600800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9130 0x7fa21c0d1c30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 --2- 172.21.15.5:0/429600800 >> v2:172.21.15.80:3300/0 conn(0x7fa21c0d0780 0x7fa21c0d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 -- 172.21.15.5:0/429600800 >> 172.21.15.5:0/429600800 conn(0x7fa21c0726b0 msgr2=0x7fa21c072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 -- 172.21.15.5:0/429600800 shutdown_connections 2024-04-25T10:02:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.396+0000 7fa220a00640 1 -- 172.21.15.5:0/429600800 wait complete. 2024-04-25T10:02:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.398+0000 7fa220a00640 1 Processor -- start 2024-04-25T10:02:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.398+0000 7fa220a00640 1 -- start start 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa220a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9130 0x7fa21c0c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa220a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 0x7fa21c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa220a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa21c0d0780 0x7fa21c0d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa220a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c1611b0 con 0x7fa21c0b9590 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa220a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0b8a70 con 0x7fa21c0d0780 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa220a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c161330 con 0x7fa21c0b9130 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa219400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 0x7fa21c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa219400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 0x7fa21c0d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49714/0 (socket says 172.21.15.5:49714) 2024-04-25T10:02:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa219400640 1 -- 172.21.15.5:0/164013726 learned_addr learned my addr 172.21.15.5:0/164013726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa219e00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9130 0x7fa21c0c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.399+0000 7fa21a800640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa21c0d0780 0x7fa21c0d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa219400640 1 -- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9130 msgr2=0x7fa21c0c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa219400640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9130 0x7fa21c0c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa219400640 1 -- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa21c0d0780 msgr2=0x7fa21c0d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa219400640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa21c0d0780 0x7fa21c0d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa219400640 1 -- 172.21.15.5:0/164013726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c1f4820 con 0x7fa21c0b9590 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa219400640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 0x7fa21c0d3630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa20802a220 tx=0x7fa2080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.400+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208037040 con 0x7fa21c0b9590 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.401+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208042070 con 0x7fa21c0b9590 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.401+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208030880 con 0x7fa21c0b9590 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.401+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c1f3a60 con 0x7fa21c0b9590 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.401+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c1f4020 con 0x7fa21c0b9590 2024-04-25T10:02:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.402+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c0685d0 con 0x7fa21c0b9590 2024-04-25T10:02:56.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.403+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa208030a20 con 0x7fa21c0b9590 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.404+0000 7fa212000640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.404+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fa2080aadd0 con 0x7fa21c0b9590 2024-04-25T10:02:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.405+0000 7fa219e00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.405+0000 7fa219e00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0cc290 tx=0x7fa20c009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.408+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa208079840 con 0x7fa21c0b9590 2024-04-25T10:02:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.607+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa21c068f60 con 0x7fa21c0b9590 2024-04-25T10:02:56.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.608+0000 7fa212000640 1 -- 172.21.15.5:0/164013726 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa20807f020 con 0x7fa21c0b9590 2024-04-25T10:02:56.613 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1fc06b4d0 msgr2=0x7fa1fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0cc290 tx=0x7fa20c009190 comp rx=0 tx=0).stop 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 msgr2=0x7fa21c0d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 0x7fa21c0d3630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa20802a220 tx=0x7fa2080046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 shutdown_connections 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1fc06b4d0 0x7fa1fc06d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa21c0d0780 0x7fa21c0d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c0b9590 0x7fa21c0d3630 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 --2- 172.21.15.5:0/164013726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa21c0b9130 0x7fa21c0c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.611+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 >> 172.21.15.5:0/164013726 conn(0x7fa21c0726b0 msgr2=0x7fa21c0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.612+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 shutdown_connections 2024-04-25T10:02:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.612+0000 7fa220a00640 1 -- 172.21.15.5:0/164013726 wait complete. 2024-04-25T10:02:56.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.844+0000 7f18a7200640 1 Processor -- start 2024-04-25T10:02:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.844+0000 7f18a7200640 1 -- start start 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18a8141da0 0x7f18a806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18a806e640 0x7f18a815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a815d340 0x7f18a80cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a8142180 con 0x7f18a8141da0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a8071d30 con 0x7f18a806e640 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a806d620 con 0x7f18a815d340 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18a8141da0 0x7f18a806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18a8141da0 0x7f18a806e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60060/0 (socket says 172.21.15.5:60060) 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.846+0000 7f18a5e00640 1 -- 172.21.15.5:0/3062206726 learned_addr learned my addr 172.21.15.5:0/3062206726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5400640 1 --2- 172.21.15.5:0/3062206726 >> v2:172.21.15.80:3300/0 conn(0x7f18a806e640 0x7f18a815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a6800640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a815d340 0x7f18a80cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5e00640 1 -- 172.21.15.5:0/3062206726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a815d340 msgr2=0x7f18a80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5e00640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a815d340 0x7f18a80cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5e00640 1 -- 172.21.15.5:0/3062206726 >> v2:172.21.15.80:3300/0 conn(0x7f18a806e640 msgr2=0x7f18a815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5e00640 1 --2- 172.21.15.5:0/3062206726 >> v2:172.21.15.80:3300/0 conn(0x7f18a806e640 0x7f18a815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5e00640 1 -- 172.21.15.5:0/3062206726 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a806d900 con 0x7f18a8141da0 2024-04-25T10:02:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a5400640 1 --2- 172.21.15.5:0/3062206726 >> v2:172.21.15.80:3300/0 conn(0x7f18a806e640 0x7f18a815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.847+0000 7f18a6800640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a815d340 0x7f18a80cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.848+0000 7f18a5e00640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a8141da0 0x7f18a806e100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18a806c800 tx=0x7f189802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d8f9663a59492e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.848+0000 7f18a4a00640 1 -- 172.21.15.5:0/3062206726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898037070 con 0x7f18a8141da0 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.848+0000 7f18a4a00640 1 -- 172.21.15.5:0/3062206726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898032440 con 0x7f18a8141da0 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.848+0000 7f18a4a00640 1 -- 172.21.15.5:0/3062206726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898032810 con 0x7f18a8141da0 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 -- 172.21.15.5:0/3062206726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a8141da0 msgr2=0x7f18a806e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a8141da0 0x7f18a806e100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f18a806c800 tx=0x7f189802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 -- 172.21.15.5:0/3062206726 shutdown_connections 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a815d340 0x7f18a80cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 --2- 172.21.15.5:0/3062206726 >> v2:172.21.15.80:3300/0 conn(0x7f18a806e640 0x7f18a815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 --2- 172.21.15.5:0/3062206726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a8141da0 0x7f18a806e100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 -- 172.21.15.5:0/3062206726 >> 172.21.15.5:0/3062206726 conn(0x7f18a8066f20 msgr2=0x7f18a806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 -- 172.21.15.5:0/3062206726 shutdown_connections 2024-04-25T10:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.849+0000 7f18a7200640 1 -- 172.21.15.5:0/3062206726 wait complete. 2024-04-25T10:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.851+0000 7f18a7200640 1 Processor -- start 2024-04-25T10:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.851+0000 7f18a7200640 1 -- start start 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a806e640 0x7f18a8140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a8141da0 0x7f18a8143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 0x7f18a8144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cd9c0 con 0x7f18a815d340 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cd840 con 0x7f18a806e640 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cdb40 con 0x7f18a8141da0 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 0x7f18a8144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 0x7f18a8144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60076/0 (socket says 172.21.15.5:60076) 2024-04-25T10:02:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.852+0000 7f18a6800640 1 -- 172.21.15.5:0/3875080266 learned_addr learned my addr 172.21.15.5:0/3875080266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a5e00640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a806e640 0x7f18a8140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a6800640 1 -- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a8141da0 msgr2=0x7f18a8143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a5400640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a8141da0 0x7f18a8143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a6800640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a8141da0 0x7f18a8143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a6800640 1 -- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a806e640 msgr2=0x7f18a8140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a6800640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a806e640 0x7f18a8140490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a6800640 1 -- 172.21.15.5:0/3875080266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80b3210 con 0x7f18a815d340 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a6800640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 0x7f18a8144110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f18a000cef0 tx=0x7f18a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a001a070 con 0x7f18a815d340 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a000d860 con 0x7f18a815d340 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000dc50 con 0x7f18a815d340 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.853+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a80b34f0 con 0x7f18a815d340 2024-04-25T10:02:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.854+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a80b3ab0 con 0x7f18a815d340 2024-04-25T10:02:56.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.855+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a80b3eb0 con 0x7f18a815d340 2024-04-25T10:02:56.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.856+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18a001d070 con 0x7f18a815d340 2024-04-25T10:02:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.857+0000 7f189e000640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f188806b500 0x7f188806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.857+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f18a008ae10 con 0x7f18a815d340 2024-04-25T10:02:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.857+0000 7f18a5e00640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f188806b500 0x7f188806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.858+0000 7f18a5e00640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f188806b500 0x7f188806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1898034000 tx=0x7f189802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:56.861+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18a80b3eb0 con 0x7f18a815d340 2024-04-25T10:02:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.062+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18a80ccae0 con 0x7f18a815d340 2024-04-25T10:02:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.065+0000 7f189e000640 1 -- 172.21.15.5:0/3875080266 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f18a008e030 con 0x7f18a815d340 2024-04-25T10:02:57.070 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:57.070 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.067+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f188806b500 msgr2=0x7f188806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.067+0000 7f18a7200640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f188806b500 0x7f188806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1898034000 tx=0x7f189802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.067+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 msgr2=0x7f18a8144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.067+0000 7f18a7200640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 0x7f18a8144110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f18a000cef0 tx=0x7f18a000a710 comp rx=0 tx=0).stop 2024-04-25T10:02:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 shutdown_connections 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a815d340 0x7f18a8144110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18a8141da0 0x7f18a8143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f188806b500 0x7f188806d9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 --2- 172.21.15.5:0/3875080266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a806e640 0x7f18a8140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 >> 172.21.15.5:0/3875080266 conn(0x7f18a8066f20 msgr2=0x7f18a815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 shutdown_connections 2024-04-25T10:02:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.068+0000 7f18a7200640 1 -- 172.21.15.5:0/3875080266 wait complete. 2024-04-25T10:02:57.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:57.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.298+0000 7f196aa00640 1 Processor -- start 2024-04-25T10:02:57.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.298+0000 7f196aa00640 1 -- start start 2024-04-25T10:02:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f196aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f19640cf750 0x7f196406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f196aa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f19640d0d60 0x7f196406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f196aa00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196406c120 con 0x7f196415da00 2024-04-25T10:02:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f196aa00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640d08d0 con 0x7f19640cf750 2024-04-25T10:02:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f196aa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640cfcc0 con 0x7f19640d0d60 2024-04-25T10:02:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f1968c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f19640cf750 0x7f196406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f1968c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f19640cf750 0x7f196406c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60086/0 (socket says 172.21.15.5:60086) 2024-04-25T10:02:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.300+0000 7f1968c00640 1 -- 172.21.15.5:0/3297292924 learned_addr learned my addr 172.21.15.5:0/3297292924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:57.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.301+0000 7f196a000640 1 --2- 172.21.15.5:0/3297292924 >> v2:172.21.15.80:3300/0 conn(0x7f19640d0d60 0x7f196406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.301+0000 7f1968c00640 1 -- 172.21.15.5:0/3297292924 >> v2:172.21.15.80:3300/0 conn(0x7f19640d0d60 msgr2=0x7f196406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.301+0000 7f1968c00640 1 --2- 172.21.15.5:0/3297292924 >> v2:172.21.15.80:3300/0 conn(0x7f19640d0d60 0x7f196406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.301+0000 7f1968c00640 1 -- 172.21.15.5:0/3297292924 >> v1:172.21.15.80:6789/0 conn(0x7f196415da00 legacy=0x7f1964161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.301+0000 7f1968c00640 1 -- 172.21.15.5:0/3297292924 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640cffa0 con 0x7f19640cf750 2024-04-25T10:02:57.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.302+0000 7f1968c00640 1 --2- 172.21.15.5:0/3297292924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19640cf750 0x7f196406c600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f19640cbaf0 tx=0x7f195002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbd5db393b19ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.302+0000 7f1963e00640 1 -- 172.21.15.5:0/3297292924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950037070 con 0x7f19640cf750 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.302+0000 7f1963e00640 1 -- 172.21.15.5:0/3297292924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1950032440 con 0x7f19640cf750 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.302+0000 7f1963e00640 1 -- 172.21.15.5:0/3297292924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950032810 con 0x7f19640cf750 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 -- 172.21.15.5:0/3297292924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19640cf750 msgr2=0x7f196406c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 --2- 172.21.15.5:0/3297292924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19640cf750 0x7f196406c600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f19640cbaf0 tx=0x7f195002a270 comp rx=0 tx=0).stop 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 -- 172.21.15.5:0/3297292924 shutdown_connections 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 --2- 172.21.15.5:0/3297292924 >> v2:172.21.15.80:3300/0 conn(0x7f19640d0d60 0x7f196406cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 --2- 172.21.15.5:0/3297292924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19640cf750 0x7f196406c600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 -- 172.21.15.5:0/3297292924 >> 172.21.15.5:0/3297292924 conn(0x7f19640b3000 msgr2=0x7f19640b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.303+0000 7f196aa00640 1 -- 172.21.15.5:0/3297292924 shutdown_connections 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.304+0000 7f196aa00640 1 -- 172.21.15.5:0/3297292924 wait complete. 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.305+0000 7f196aa00640 1 Processor -- start 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.305+0000 7f196aa00640 1 -- start start 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.306+0000 7f196aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19640cf750 0x7f19641f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.306+0000 7f196aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f19640d0d60 0x7f19640b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.306+0000 7f196aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 0x7f19640cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.306+0000 7f196aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196415dde0 con 0x7f196415da00 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.306+0000 7f196aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19640d0560 con 0x7f19640cf750 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.306+0000 7f196aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19640b0e70 con 0x7f19640d0d60 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 0x7f19640cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 0x7f19640cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60096/0 (socket says 172.21.15.5:60096) 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 -- 172.21.15.5:0/4077566080 learned_addr learned my addr 172.21.15.5:0/4077566080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 -- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f19640d0d60 msgr2=0x7f19640b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f19640d0d60 0x7f19640b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 -- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19640cf750 msgr2=0x7f19641f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19640cf750 0x7f19641f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 -- 172.21.15.5:0/4077566080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640b70d0 con 0x7f196415da00 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.307+0000 7f196a000640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 0x7f19640cc6e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f195c007e70 tx=0x7f195c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.308+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c014620 con 0x7f196415da00 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.308+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c019070 con 0x7f196415da00 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.308+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c014a20 con 0x7f196415da00 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.308+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19640c4580 con 0x7f196415da00 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.308+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19640c4e20 con 0x7f196415da00 2024-04-25T10:02:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.309+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19640c4970 con 0x7f196415da00 2024-04-25T10:02:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.310+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f195c014bc0 con 0x7f196415da00 2024-04-25T10:02:57.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.311+0000 7f1961600640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f194406b4d0 0x7f194406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.311+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f195c08aec0 con 0x7f196415da00 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.312+0000 7f1969600640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f194406b4d0 0x7f194406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.313+0000 7f1969600640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f194406b4d0 0x7f194406d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1954000f60 tx=0x7f19540079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.315+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f195c059930 con 0x7f196415da00 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.519+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1964068f60 con 0x7f196415da00 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.519+0000 7f1961600640 1 -- 172.21.15.5:0/4077566080 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f195c059230 con 0x7f196415da00 2024-04-25T10:02:57.524 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f194406b4d0 msgr2=0x7f194406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f194406b4d0 0x7f194406d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1954000f60 tx=0x7f19540079b0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 msgr2=0x7f19640cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 0x7f19640cc6e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f195c007e70 tx=0x7f195c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 shutdown_connections 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f196415da00 0x7f19640cc6e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f19640d0d60 0x7f19640b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f194406b4d0 0x7f194406d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 --2- 172.21.15.5:0/4077566080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19640cf750 0x7f19641f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 >> 172.21.15.5:0/4077566080 conn(0x7f19640b3000 msgr2=0x7f19640b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 shutdown_connections 2024-04-25T10:02:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.522+0000 7f196aa00640 1 -- 172.21.15.5:0/4077566080 wait complete. 2024-04-25T10:02:57.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.745+0000 7f23a6000640 1 Processor -- start 2024-04-25T10:02:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.745+0000 7f23a6000640 1 -- start start 2024-04-25T10:02:57.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a6000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f23a006ec70 0x7f23a0071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a6000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 0x7f23a0071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a6000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00ade10 con 0x7f23a0143f30 2024-04-25T10:02:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a6000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00ae0f0 con 0x7f23a006ec70 2024-04-25T10:02:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a6000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00ae3d0 con 0x7f23a00aebd0 2024-04-25T10:02:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a4c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f23a0143f30 0x7f23a006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53142/0 (socket says 172.21.15.5:53142) 2024-04-25T10:02:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.747+0000 7f23a4c00640 1 -- 172.21.15.5:0/867844852 learned_addr learned my addr 172.21.15.5:0/867844852 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239fe00640 1 --2- 172.21.15.5:0/867844852 >> v2:172.21.15.80:3300/0 conn(0x7f23a006ec70 0x7f23a0071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f23a5600640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 0x7f23a0071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634496939 0 0) 0x7f23a00ade10 con 0x7f23a0143f30 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2388003520 con 0x7f23a0143f30 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707579961 0 0) 0x7f2388003520 con 0x7f23a0143f30 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 msgr2=0x7f23a0071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 0x7f23a0071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 >> v2:172.21.15.80:3300/0 conn(0x7f23a006ec70 msgr2=0x7f23a0071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 --2- 172.21.15.5:0/867844852 >> v2:172.21.15.80:3300/0 conn(0x7f23a006ec70 0x7f23a0071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00ae6b0 con 0x7f23a0143f30 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2394001670 con 0x7f23a0143f30 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 >> v1:172.21.15.5:6789/0 conn(0x7f23a0143f30 legacy=0x7f23a006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f2388003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2388004800 0x7f2388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 0x7f23880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00ae0f0 con 0x7f23880039b0 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00ae3d0 con 0x7f2388004800 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.748+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238800ba40 con 0x7f23a00aebd0 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f23a4c00640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f2388003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f23a4c00640 1 -- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 msgr2=0x7f23880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f23a4c00640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00aebd0 0x7f23880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f23a4c00640 1 -- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2388004800 msgr2=0x7f2388004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f239fe00640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2388004800 0x7f2388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f23a4c00640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2388004800 0x7f2388004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.749+0000 7f23a4c00640 1 -- 172.21.15.5:0/867844852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388003520 con 0x7f23880039b0 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.750+0000 7f23a4c00640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f2388003dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f239400c160 tx=0x7f2388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da77d67af9686123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.751+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394007c80 con 0x7f23880039b0 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.751+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394005ad0 con 0x7f23880039b0 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.751+0000 7f239f400640 1 -- 172.21.15.5:0/867844852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394007070 con 0x7f23880039b0 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.751+0000 7f23a6000640 1 -- 172.21.15.5:0/867844852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 msgr2=0x7f2388003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.751+0000 7f23a6000640 1 --2- 172.21.15.5:0/867844852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f2388003dd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f239400c160 tx=0x7f2388005830 comp rx=0 tx=0).stop 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.751+0000 7f23a5600640 1 -- 172.21.15.5:0/867844852 reap_dead start 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.752+0000 7f23a6000640 1 -- 172.21.15.5:0/867844852 shutdown_connections 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.752+0000 7f23a6000640 1 -- 172.21.15.5:0/867844852 >> 172.21.15.5:0/867844852 conn(0x7f23a0066f20 msgr2=0x7f23a006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.752+0000 7f23a6000640 1 -- 172.21.15.5:0/867844852 shutdown_connections 2024-04-25T10:02:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.752+0000 7f23a6000640 1 -- 172.21.15.5:0/867844852 wait complete. 2024-04-25T10:02:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.753+0000 7f23a6000640 1 Processor -- start 2024-04-25T10:02:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.754+0000 7f23a6000640 1 -- start start 2024-04-25T10:02:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f23a00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23a0143480 0x7f23a015d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a6000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00ca110 0x7f23a00b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a6000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00716c0 con 0x7f23880039b0 2024-04-25T10:02:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a6000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0071540 con 0x7f23a0143480 2024-04-25T10:02:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a6000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0071840 con 0x7f23a00ca110 2024-04-25T10:02:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f23a00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f23a00c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60120/0 (socket says 172.21.15.5:60120) 2024-04-25T10:02:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 -- 172.21.15.5:0/3075811666 learned_addr learned my addr 172.21.15.5:0/3075811666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a5600640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00ca110 0x7f23a00b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f239fe00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23a0143480 0x7f23a015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 -- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00ca110 msgr2=0x7f23a00b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00ca110 0x7f23a00b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 -- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23a0143480 msgr2=0x7f23a015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23a0143480 0x7f23a015d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a4c00640 1 -- 172.21.15.5:0/3075811666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0135430 con 0x7f23880039b0 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.755+0000 7f23a5600640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00ca110 0x7f23a00b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.756+0000 7f23a4c00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f23a00c7210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f239400c280 tx=0x7f23a015dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.756+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940013b0 con 0x7f23880039b0 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.756+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2394042070 con 0x7f23880039b0 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.756+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a0135710 con 0x7f23880039b0 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.756+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394007070 con 0x7f23880039b0 2024-04-25T10:02:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.756+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0135cd0 con 0x7f23880039b0 2024-04-25T10:02:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.758+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0068f60 con 0x7f23880039b0 2024-04-25T10:02:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.758+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2394007210 con 0x7f23880039b0 2024-04-25T10:02:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.759+0000 7f239cc00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f238006b500 0x7f238006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.759+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f23940aa440 con 0x7f23880039b0 2024-04-25T10:02:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.760+0000 7f239fe00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f238006b500 0x7f238006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.761+0000 7f239fe00640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f238006b500 0x7f238006d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f23a00c6df0 tx=0x7f238c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.763+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2394078fd0 con 0x7f23880039b0 2024-04-25T10:02:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.961+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23a006bfe0 con 0x7f23880039b0 2024-04-25T10:02:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.962+0000 7f239cc00640 1 -- 172.21.15.5:0/3075811666 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2394005aa0 con 0x7f23880039b0 2024-04-25T10:02:57.967 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f238006b500 msgr2=0x7f238006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f238006b500 0x7f238006d9c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f23a00c6df0 tx=0x7f238c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 msgr2=0x7f23a00c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f23a00c7210 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f239400c280 tx=0x7f23a015dd70 comp rx=0 tx=0).stop 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 shutdown_connections 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f23a00ca110 0x7f23a00b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f238006b500 0x7f238006d9c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23a0143480 0x7f23a015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 --2- 172.21.15.5:0/3075811666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23880039b0 0x7f23a00c7210 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 >> 172.21.15.5:0/3075811666 conn(0x7f23a0066f20 msgr2=0x7f23a0145b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 shutdown_connections 2024-04-25T10:02:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:57.965+0000 7f23a6000640 1 -- 172.21.15.5:0/3075811666 wait complete. 2024-04-25T10:02:57.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.191+0000 7f9c95800640 1 Processor -- start 2024-04-25T10:02:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.191+0000 7f9c95800640 1 -- start start 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c95800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c900b9000 0x7f9c900ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c95800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9c900ce980 0x7f9c900b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c95800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900d2280 con 0x7f9c9015d980 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c95800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c900cedf0 con 0x7f9c900b9000 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c95800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c900d2560 con 0x7f9c900ce980 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c87e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c900b9000 0x7f9c900ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c87e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c900b9000 0x7f9c900ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60124/0 (socket says 172.21.15.5:60124) 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c87e00640 1 -- 172.21.15.5:0/2323551655 learned_addr learned my addr 172.21.15.5:0/2323551655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c94e00640 1 --2- 172.21.15.5:0/2323551655 >> v2:172.21.15.80:3300/0 conn(0x7f9c900ce980 0x7f9c900b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c87e00640 1 -- 172.21.15.5:0/2323551655 >> v2:172.21.15.80:3300/0 conn(0x7f9c900ce980 msgr2=0x7f9c900b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c87e00640 1 --2- 172.21.15.5:0/2323551655 >> v2:172.21.15.80:3300/0 conn(0x7f9c900ce980 0x7f9c900b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.193+0000 7f9c87e00640 1 -- 172.21.15.5:0/2323551655 >> v1:172.21.15.80:6789/0 conn(0x7f9c9015d980 legacy=0x7f9c900d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.194+0000 7f9c87e00640 1 -- 172.21.15.5:0/2323551655 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900d2840 con 0x7f9c900b9000 2024-04-25T10:02:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.195+0000 7f9c87e00640 1 --2- 172.21.15.5:0/2323551655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c900b9000 0x7f9c900ce440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9c900b6210 tx=0x7f9c7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b43bbbbead11d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.195+0000 7f9c8f400640 1 -- 172.21.15.5:0/2323551655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78037070 con 0x7f9c900b9000 2024-04-25T10:02:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.195+0000 7f9c8f400640 1 -- 172.21.15.5:0/2323551655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78032440 con 0x7f9c900b9000 2024-04-25T10:02:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.195+0000 7f9c8f400640 1 -- 172.21.15.5:0/2323551655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78032810 con 0x7f9c900b9000 2024-04-25T10:02:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 -- 172.21.15.5:0/2323551655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c900b9000 msgr2=0x7f9c900ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 --2- 172.21.15.5:0/2323551655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c900b9000 0x7f9c900ce440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9c900b6210 tx=0x7f9c7802a270 comp rx=0 tx=0).stop 2024-04-25T10:02:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 -- 172.21.15.5:0/2323551655 shutdown_connections 2024-04-25T10:02:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 --2- 172.21.15.5:0/2323551655 >> v2:172.21.15.80:3300/0 conn(0x7f9c900ce980 0x7f9c900b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 --2- 172.21.15.5:0/2323551655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c900b9000 0x7f9c900ce440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 -- 172.21.15.5:0/2323551655 >> 172.21.15.5:0/2323551655 conn(0x7f9c900728c0 msgr2=0x7f9c9006bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 -- 172.21.15.5:0/2323551655 shutdown_connections 2024-04-25T10:02:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.196+0000 7f9c95800640 1 -- 172.21.15.5:0/2323551655 wait complete. 2024-04-25T10:02:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.198+0000 7f9c95800640 1 Processor -- start 2024-04-25T10:02:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.198+0000 7f9c95800640 1 -- start start 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c95800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c900b9000 0x7f9c9014bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c95800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c900ce980 0x7f9c900cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c95800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 0x7f9c900ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c95800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9015dd60 con 0x7f9c9015d980 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c95800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900d2e00 con 0x7f9c900b9000 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c95800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c900d3320 con 0x7f9c900ce980 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c94e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 0x7f9c900ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c94e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 0x7f9c900ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60140/0 (socket says 172.21.15.5:60140) 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.199+0000 7f9c94e00640 1 -- 172.21.15.5:0/766704203 learned_addr learned my addr 172.21.15.5:0/766704203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:58.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c8fe00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c900b9000 0x7f9c9014bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c87e00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c900ce980 0x7f9c900cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c94e00640 1 -- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c900ce980 msgr2=0x7f9c900cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c94e00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c900ce980 0x7f9c900cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c94e00640 1 -- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c900b9000 msgr2=0x7f9c9014bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c94e00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c900b9000 0x7f9c9014bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c94e00640 1 -- 172.21.15.5:0/766704203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900cb410 con 0x7f9c9015d980 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c94e00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 0x7f9c900ccef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c88007e90 tx=0x7f9c8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88014420 con 0x7f9c9015d980 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88019070 con 0x7f9c9015d980 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88014800 con 0x7f9c9015d980 2024-04-25T10:02:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c900cb6f0 con 0x7f9c9015d980 2024-04-25T10:02:58.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.200+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c900cbcb0 con 0x7f9c9015d980 2024-04-25T10:02:58.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.202+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c90068f60 con 0x7f9c9015d980 2024-04-25T10:02:58.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.203+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c880149a0 con 0x7f9c9015d980 2024-04-25T10:02:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.204+0000 7f9c8cc00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.204+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f9c8808ae40 con 0x7f9c9015d980 2024-04-25T10:02:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.204+0000 7f9c8fe00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.205+0000 7f9c8fe00640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000f60 tx=0x7f9c80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.208+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c880598b0 con 0x7f9c9015d980 2024-04-25T10:02:58.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.419+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c9006ad60 con 0x7f9c9015d980 2024-04-25T10:02:58.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.420+0000 7f9c8cc00640 1 -- 172.21.15.5:0/766704203 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c880591b0 con 0x7f9c9015d980 2024-04-25T10:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:02:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6c06b4d0 msgr2=0x7f9c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000f60 tx=0x7f9c80009040 comp rx=0 tx=0).stop 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 msgr2=0x7f9c900ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 0x7f9c900ccef0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c88007e90 tx=0x7f9c8800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 shutdown_connections 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c9015d980 0x7f9c900ccef0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c900ce980 0x7f9c900cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6c06b4d0 0x7f9c6c06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 --2- 172.21.15.5:0/766704203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c900b9000 0x7f9c9014bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 >> 172.21.15.5:0/766704203 conn(0x7f9c900728c0 msgr2=0x7f9c900b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 shutdown_connections 2024-04-25T10:02:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.423+0000 7f9c95800640 1 -- 172.21.15.5:0/766704203 wait complete. 2024-04-25T10:02:58.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.657+0000 7fae5a600640 1 Processor -- start 2024-04-25T10:02:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.657+0000 7fae5a600640 1 -- start start 2024-04-25T10:02:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae5a600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fae540d3f90 0x7fae54161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae5a600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fae54161bc0 0x7fae540b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae5a600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae541606e0 con 0x7fae540b48c0 2024-04-25T10:02:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae5a600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae54161ff0 con 0x7fae54161bc0 2024-04-25T10:02:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae5a600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae541609c0 con 0x7fae540d3f90 2024-04-25T10:02:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae59200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fae540b48c0 0x7fae5415fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53156/0 (socket says 172.21.15.5:53156) 2024-04-25T10:02:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae59200640 1 -- 172.21.15.5:0/763832979 learned_addr learned my addr 172.21.15.5:0/763832979 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae59c00640 1 --2- 172.21.15.5:0/763832979 >> v2:172.21.15.5:3300/0 conn(0x7fae54161bc0 0x7fae540b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.659+0000 7fae53400640 1 -- 172.21.15.5:0/763832979 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215072015 0 0) 0x7fae541606e0 con 0x7fae540b48c0 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae53400640 1 -- 172.21.15.5:0/763832979 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae3c003520 con 0x7fae540b48c0 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae53e00640 1 --2- 172.21.15.5:0/763832979 >> v2:172.21.15.80:3300/0 conn(0x7fae540d3f90 0x7fae54161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae59c00640 1 -- 172.21.15.5:0/763832979 >> v2:172.21.15.80:3300/0 conn(0x7fae540d3f90 msgr2=0x7fae54161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae59c00640 1 --2- 172.21.15.5:0/763832979 >> v2:172.21.15.80:3300/0 conn(0x7fae540d3f90 0x7fae54161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae59c00640 1 -- 172.21.15.5:0/763832979 >> v1:172.21.15.5:6789/0 conn(0x7fae540b48c0 legacy=0x7fae5415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae59c00640 1 -- 172.21.15.5:0/763832979 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54160ca0 con 0x7fae54161bc0 2024-04-25T10:02:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.660+0000 7fae53e00640 1 --2- 172.21.15.5:0/763832979 >> v2:172.21.15.80:3300/0 conn(0x7fae540d3f90 0x7fae54161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.661+0000 7fae59c00640 1 --2- 172.21.15.5:0/763832979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b2b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fae540b61b0 tx=0x7fae4c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77817a74cd1907a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.661+0000 7fae53400640 1 -- 172.21.15.5:0/763832979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c007740 con 0x7fae54161bc0 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.661+0000 7fae53400640 1 -- 172.21.15.5:0/763832979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c0078e0 con 0x7fae54161bc0 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.661+0000 7fae53400640 1 -- 172.21.15.5:0/763832979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c007cb0 con 0x7fae54161bc0 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 -- 172.21.15.5:0/763832979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 msgr2=0x7fae540b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 --2- 172.21.15.5:0/763832979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b2b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fae540b61b0 tx=0x7fae4c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 -- 172.21.15.5:0/763832979 shutdown_connections 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 --2- 172.21.15.5:0/763832979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b2b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 --2- 172.21.15.5:0/763832979 >> v2:172.21.15.80:3300/0 conn(0x7fae540d3f90 0x7fae54161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 -- 172.21.15.5:0/763832979 >> 172.21.15.5:0/763832979 conn(0x7fae5408af80 msgr2=0x7fae540d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 -- 172.21.15.5:0/763832979 shutdown_connections 2024-04-25T10:02:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.662+0000 7fae5a600640 1 -- 172.21.15.5:0/763832979 wait complete. 2024-04-25T10:02:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.664+0000 7fae5a600640 1 Processor -- start 2024-04-25T10:02:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.664+0000 7fae5a600640 1 -- start start 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.665+0000 7fae5a600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae540b48c0 0x7fae5414b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.665+0000 7fae5a600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae540d3f90 0x7fae540b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.665+0000 7fae5a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.665+0000 7fae5a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae540b4ca0 con 0x7fae54161bc0 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.665+0000 7fae5a600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae54161260 con 0x7fae540d3f90 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.665+0000 7fae5a600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae540b30b0 con 0x7fae540b48c0 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60152/0 (socket says 172.21.15.5:60152) 2024-04-25T10:02:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 -- 172.21.15.5:0/3483245417 learned_addr learned my addr 172.21.15.5:0/3483245417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:58.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae53e00640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae540d3f90 0x7fae540b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59200640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae540b48c0 0x7fae5414b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 -- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae540b48c0 msgr2=0x7fae5414b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae540b48c0 0x7fae5414b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 -- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae540d3f90 msgr2=0x7fae540b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae540d3f90 0x7fae540b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.666+0000 7fae59c00640 1 -- 172.21.15.5:0/3483245417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae540c7d10 con 0x7fae54161bc0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.667+0000 7fae59c00640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b1c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fae4c02abe0 tx=0x7fae4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.667+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c03f070 con 0x7fae54161bc0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.667+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c031bc0 con 0x7fae54161bc0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.667+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c009450 con 0x7fae54161bc0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.667+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae540c7ff0 con 0x7fae54161bc0 2024-04-25T10:02:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.667+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae540c85b0 con 0x7fae54161bc0 2024-04-25T10:02:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.668+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae54068f60 con 0x7fae54161bc0 2024-04-25T10:02:58.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.669+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae4c03b410 con 0x7fae54161bc0 2024-04-25T10:02:58.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.670+0000 7fae50c00640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae3406b4d0 0x7fae3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.670+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fae4c0abe50 con 0x7fae54161bc0 2024-04-25T10:02:58.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.671+0000 7fae59200640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae3406b4d0 0x7fae3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.672+0000 7fae59200640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae3406b4d0 0x7fae3406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae48005880 tx=0x7fae48009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.675+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae4c07a830 con 0x7fae54161bc0 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.883+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fae5408d730 con 0x7fae54161bc0 2024-04-25T10:02:58.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.884+0000 7fae50c00640 1 -- 172.21.15.5:0/3483245417 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fae4c07a130 con 0x7fae54161bc0 2024-04-25T10:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae3406b4d0 msgr2=0x7fae3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae3406b4d0 0x7fae3406d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae48005880 tx=0x7fae48009040 comp rx=0 tx=0).stop 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 msgr2=0x7fae540b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b1c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fae4c02abe0 tx=0x7fae4c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 shutdown_connections 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae54161bc0 0x7fae540b1c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fae3406b4d0 0x7fae3406d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae540d3f90 0x7fae540b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 --2- 172.21.15.5:0/3483245417 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae540b48c0 0x7fae5414b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 >> 172.21.15.5:0/3483245417 conn(0x7fae5408af80 msgr2=0x7fae5406d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 shutdown_connections 2024-04-25T10:02:58.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:58.887+0000 7fae5a600640 1 -- 172.21.15.5:0/3483245417 wait complete. 2024-04-25T10:02:58.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.108+0000 7f5b62a00640 1 Processor -- start 2024-04-25T10:02:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.108+0000 7f5b62a00640 1 -- start start 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.110+0000 7f5b62a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b5c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.110+0000 7f5b62a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b5c070f90 0x7f5b5c071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.110+0000 7f5b62a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c0cd200 con 0x7f5b5c0cdca0 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.110+0000 7f5b62a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c071940 con 0x7f5b5c070f90 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.110+0000 7f5b62a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c06e6f0 con 0x7f5b5c154a10 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.111+0000 7f5b60c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5b5c0cdca0 0x7f5b5c0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53164/0 (socket says 172.21.15.5:53164) 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.111+0000 7f5b60c00640 1 -- 172.21.15.5:0/760301037 learned_addr learned my addr 172.21.15.5:0/760301037 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.111+0000 7f5b61600640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b5c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.111+0000 7f5b62000640 1 --2- 172.21.15.5:0/760301037 >> v2:172.21.15.80:3300/0 conn(0x7f5b5c070f90 0x7f5b5c071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.111+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191250516 0 0) 0x7f5b5c0cd200 con 0x7f5b5c0cdca0 2024-04-25T10:02:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.111+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b44003520 con 0x7f5b5c0cdca0 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3250698359 0 0) 0x7f5b44003520 con 0x7f5b5c0cdca0 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 msgr2=0x7f5b5c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b5c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 >> v2:172.21.15.80:3300/0 conn(0x7f5b5c070f90 msgr2=0x7f5b5c071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 --2- 172.21.15.5:0/760301037 >> v2:172.21.15.80:3300/0 conn(0x7f5b5c070f90 0x7f5b5c071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c06e9d0 con 0x7f5b5c0cdca0 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f5b4c001670 con 0x7f5b5c0cdca0 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 >> v1:172.21.15.5:6789/0 conn(0x7f5b5c0cdca0 legacy=0x7f5b5c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b44004d40 0x7f5b440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c071940 con 0x7f5b440039b0 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c06e6f0 con 0x7f5b44004d40 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4400b9c0 con 0x7f5b5c154a10 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b61600640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b60c00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.112+0000 7f5b62000640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b44004d40 0x7f5b440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.113+0000 7f5b61600640 1 -- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 msgr2=0x7f5b44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.113+0000 7f5b61600640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.113+0000 7f5b61600640 1 -- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b44004d40 msgr2=0x7f5b440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.113+0000 7f5b61600640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b44004d40 0x7f5b440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.113+0000 7f5b61600640 1 -- 172.21.15.5:0/760301037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44003520 con 0x7f5b440039b0 2024-04-25T10:02:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.113+0000 7f5b60c00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c154a10 0x7f5b44004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.114+0000 7f5b61600640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b44003dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009ac0 tx=0x7f5b44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef37cab6b2f23a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.114+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50036070 con 0x7f5b440039b0 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.114+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50031400 con 0x7f5b440039b0 2024-04-25T10:02:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.114+0000 7f5b5be00640 1 -- 172.21.15.5:0/760301037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50009690 con 0x7f5b440039b0 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62a00640 1 -- 172.21.15.5:0/760301037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 msgr2=0x7f5b44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62a00640 1 --2- 172.21.15.5:0/760301037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b44003dd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009ac0 tx=0x7f5b44005830 comp rx=0 tx=0).stop 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62000640 1 -- 172.21.15.5:0/760301037 reap_dead start 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62a00640 1 -- 172.21.15.5:0/760301037 shutdown_connections 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62a00640 1 -- 172.21.15.5:0/760301037 >> 172.21.15.5:0/760301037 conn(0x7f5b5c066f20 msgr2=0x7f5b5c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62a00640 1 -- 172.21.15.5:0/760301037 shutdown_connections 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.115+0000 7f5b62a00640 1 -- 172.21.15.5:0/760301037 wait complete. 2024-04-25T10:02:59.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.117+0000 7f5b62a00640 1 Processor -- start 2024-04-25T10:02:59.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.117+0000 7f5b62a00640 1 -- start start 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.118+0000 7f5b62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b5c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.118+0000 7f5b62a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c14d9f0 0x7f5b5c14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.118+0000 7f5b62a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0af2b0 0x7f5b5c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.118+0000 7f5b62a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0cd780 con 0x7f5b440039b0 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.118+0000 7f5b62a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0cd4a0 con 0x7f5b5c14d9f0 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.118+0000 7f5b62a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c15e8c0 con 0x7f5b5c0af2b0 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b5c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b5c0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60166/0 (socket says 172.21.15.5:60166) 2024-04-25T10:02:59.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 -- 172.21.15.5:0/1060794542 learned_addr learned my addr 172.21.15.5:0/1060794542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 -- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0af2b0 msgr2=0x7f5b5c0af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0af2b0 0x7f5b5c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 -- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c14d9f0 msgr2=0x7f5b5c14de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c14d9f0 0x7f5b5c14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.119+0000 7f5b61600640 1 -- 172.21.15.5:0/1060794542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c1f4b00 con 0x7f5b440039b0 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.120+0000 7f5b61600640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b5c0adf70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009ac0 tx=0x7f5b50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.120+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50040020 con 0x7f5b440039b0 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.120+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50003b40 con 0x7f5b440039b0 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.120+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50036040 con 0x7f5b440039b0 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.120+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c1f3940 con 0x7f5b440039b0 2024-04-25T10:02:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.120+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c1f3ee0 con 0x7f5b440039b0 2024-04-25T10:02:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.121+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c067420 con 0x7f5b440039b0 2024-04-25T10:02:59.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.122+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b50043020 con 0x7f5b440039b0 2024-04-25T10:02:59.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.124+0000 7f5b59600640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b4006b4d0 0x7f5b4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.124+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f5b500aa6b0 con 0x7f5b440039b0 2024-04-25T10:02:59.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.124+0000 7f5b60c00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b4006b4d0 0x7f5b4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.125+0000 7f5b60c00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b4006b4d0 0x7f5b4006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c001eb0 tx=0x7f5b4c000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.128+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b5c067420 con 0x7f5b440039b0 2024-04-25T10:02:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.328+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b5c068f60 con 0x7f5b440039b0 2024-04-25T10:02:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.331+0000 7f5b59600640 1 -- 172.21.15.5:0/1060794542 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f5b5004a070 con 0x7f5b440039b0 2024-04-25T10:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b4006b4d0 msgr2=0x7f5b4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b4006b4d0 0x7f5b4006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c001eb0 tx=0x7f5b4c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 msgr2=0x7f5b5c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b5c0adf70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009ac0 tx=0x7f5b50004730 comp rx=0 tx=0).stop 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 shutdown_connections 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0af2b0 0x7f5b5c0af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b4006b4d0 0x7f5b4006d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c14d9f0 0x7f5b5c14de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 --2- 172.21.15.5:0/1060794542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b440039b0 0x7f5b5c0adf70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 >> 172.21.15.5:0/1060794542 conn(0x7f5b5c066f20 msgr2=0x7f5b5c161250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 shutdown_connections 2024-04-25T10:02:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.334+0000 7f5b62a00640 1 -- 172.21.15.5:0/1060794542 wait complete. 2024-04-25T10:02:59.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:59.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.575+0000 7fd6ad600640 1 Processor -- start 2024-04-25T10:02:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.576+0000 7fd6ad600640 1 -- start start 2024-04-25T10:02:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6ad600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd6a80d3760 0x7fd6a815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6ad600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6a80ce830 0x7fd6a80b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6ad600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a80ced10 con 0x7fd6a815f1c0 2024-04-25T10:02:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6ad600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a80ce380 con 0x7fd6a80ce830 2024-04-25T10:02:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6ad600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a815f000 con 0x7fd6a80d3760 2024-04-25T10:02:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6a6000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd6a815f1c0 0x7fd6a80d3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53170/0 (socket says 172.21.15.5:53170) 2024-04-25T10:02:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6a6000640 1 -- 172.21.15.5:0/3292476194 learned_addr learned my addr 172.21.15.5:0/3292476194 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.577+0000 7fd6a7400640 1 --2- 172.21.15.5:0/3292476194 >> v2:172.21.15.5:3300/0 conn(0x7fd6a80ce830 0x7fd6a80b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a6a00640 1 --2- 172.21.15.5:0/3292476194 >> v2:172.21.15.80:3300/0 conn(0x7fd6a80d3760 0x7fd6a815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a5600640 1 -- 172.21.15.5:0/3292476194 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1387737559 0 0) 0x7fd6a80ced10 con 0x7fd6a815f1c0 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a5600640 1 -- 172.21.15.5:0/3292476194 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd68c003520 con 0x7fd6a815f1c0 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a7400640 1 -- 172.21.15.5:0/3292476194 >> v2:172.21.15.80:3300/0 conn(0x7fd6a80d3760 msgr2=0x7fd6a815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a7400640 1 --2- 172.21.15.5:0/3292476194 >> v2:172.21.15.80:3300/0 conn(0x7fd6a80d3760 0x7fd6a815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a7400640 1 -- 172.21.15.5:0/3292476194 >> v1:172.21.15.5:6789/0 conn(0x7fd6a815f1c0 legacy=0x7fd6a80d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.578+0000 7fd6a7400640 1 -- 172.21.15.5:0/3292476194 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a80d1b40 con 0x7fd6a80ce830 2024-04-25T10:02:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.579+0000 7fd6a7400640 1 --2- 172.21.15.5:0/3292476194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a80b2240 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80b6320 tx=0x7fd6a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7819c8fbed32a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.580+0000 7fd6a5600640 1 -- 172.21.15.5:0/3292476194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0037070 con 0x7fd6a80ce830 2024-04-25T10:02:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.580+0000 7fd6a5600640 1 -- 172.21.15.5:0/3292476194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a0032440 con 0x7fd6a80ce830 2024-04-25T10:02:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.580+0000 7fd6a5600640 1 -- 172.21.15.5:0/3292476194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0032810 con 0x7fd6a80ce830 2024-04-25T10:02:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.580+0000 7fd6ad600640 1 -- 172.21.15.5:0/3292476194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 msgr2=0x7fd6a80b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.580+0000 7fd6ad600640 1 --2- 172.21.15.5:0/3292476194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a80b2240 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80b6320 tx=0x7fd6a002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.581+0000 7fd6ad600640 1 -- 172.21.15.5:0/3292476194 shutdown_connections 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.581+0000 7fd6ad600640 1 --2- 172.21.15.5:0/3292476194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a80b2240 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.581+0000 7fd6ad600640 1 --2- 172.21.15.5:0/3292476194 >> v2:172.21.15.80:3300/0 conn(0x7fd6a80d3760 0x7fd6a815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.581+0000 7fd6ad600640 1 -- 172.21.15.5:0/3292476194 >> 172.21.15.5:0/3292476194 conn(0x7fd6a8072680 msgr2=0x7fd6a8072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.581+0000 7fd6ad600640 1 -- 172.21.15.5:0/3292476194 shutdown_connections 2024-04-25T10:02:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.581+0000 7fd6ad600640 1 -- 172.21.15.5:0/3292476194 wait complete. 2024-04-25T10:02:59.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.582+0000 7fd6ad600640 1 Processor -- start 2024-04-25T10:02:59.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.583+0000 7fd6ad600640 1 -- start start 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6ad600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a8149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6ad600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6a80d3760 0x7fd6a81f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6ad600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6a815f1c0 0x7fd6a81f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6ad600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a815de10 con 0x7fd6a80ce830 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6ad600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a80d2190 con 0x7fd6a815f1c0 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6ad600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a80b2780 con 0x7fd6a80d3760 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a8149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a8149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60176/0 (socket says 172.21.15.5:60176) 2024-04-25T10:02:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 -- 172.21.15.5:0/161465404 learned_addr learned my addr 172.21.15.5:0/161465404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6000640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6a80d3760 0x7fd6a81f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 -- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6a80d3760 msgr2=0x7fd6a81f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6a80d3760 0x7fd6a81f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a7400640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6a815f1c0 0x7fd6a81f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 -- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6a815f1c0 msgr2=0x7fd6a81f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6a815f1c0 0x7fd6a81f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.584+0000 7fd6a6a00640 1 -- 172.21.15.5:0/161465404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a815cce0 con 0x7fd6a80ce830 2024-04-25T10:02:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.585+0000 7fd6a6a00640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a8149aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd698009db0 tx=0x7fd69800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.585+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698018020 con 0x7fd6a80ce830 2024-04-25T10:02:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.585+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd698014480 con 0x7fd6a80ce830 2024-04-25T10:02:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.585+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698014850 con 0x7fd6a80ce830 2024-04-25T10:02:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.585+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a81f7780 con 0x7fd6a80ce830 2024-04-25T10:02:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.585+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a814a3a0 con 0x7fd6a80ce830 2024-04-25T10:02:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.586+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a8161230 con 0x7fd6a80ce830 2024-04-25T10:02:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.587+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd698005d10 con 0x7fd6a80ce830 2024-04-25T10:02:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.588+0000 7fd69ea00640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd68406b4d0 0x7fd68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.588+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fd69808b4b0 con 0x7fd6a80ce830 2024-04-25T10:02:59.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.589+0000 7fd6a6000640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd68406b4d0 0x7fd68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.590+0000 7fd6a6000640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd68406b4d0 0x7fd68406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80ce330 tx=0x7fd690009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.592+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6a8161230 con 0x7fd6a80ce830 2024-04-25T10:02:59.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.790+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6a8068f60 con 0x7fd6a80ce830 2024-04-25T10:02:59.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.793+0000 7fd69ea00640 1 -- 172.21.15.5:0/161465404 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd698022030 con 0x7fd6a80ce830 2024-04-25T10:02:59.798 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd68406b4d0 msgr2=0x7fd68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd68406b4d0 0x7fd68406d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80ce330 tx=0x7fd690009040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 msgr2=0x7fd6a8149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a8149aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd698009db0 tx=0x7fd69800c490 comp rx=0 tx=0).stop 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 shutdown_connections 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd68406b4d0 0x7fd68406d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6a815f1c0 0x7fd6a81f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6a80d3760 0x7fd6a81f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 --2- 172.21.15.5:0/161465404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a80ce830 0x7fd6a8149aa0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 >> 172.21.15.5:0/161465404 conn(0x7fd6a8072680 msgr2=0x7fd6a8072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 shutdown_connections 2024-04-25T10:02:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:02:59.796+0000 7fd6ad600640 1 -- 172.21.15.5:0/161465404 wait complete. 2024-04-25T10:02:59.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.029+0000 7f310dc00640 1 Processor -- start 2024-04-25T10:03:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.029+0000 7f310dc00640 1 -- start start 2024-04-25T10:03:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.031+0000 7f310dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f310815f800 0x7f3108162ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.031+0000 7f310dc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3108071000 0x7f3108071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.031+0000 7f310dc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31081630e0 con 0x7f31080b2420 2024-04-25T10:03:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.031+0000 7f310dc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31080719a0 con 0x7f3108071000 2024-04-25T10:03:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.031+0000 7f310dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310806ded0 con 0x7f310815f800 2024-04-25T10:03:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f30ff400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3108071000 0x7f3108071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f3107e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f310815f800 0x7f3108162ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f310d200640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f31080b2420 0x7f31080b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:43608/0 (socket says 172.21.15.5:43608) 2024-04-25T10:03:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f310d200640 1 -- 172.21.15.5:0/792272013 learned_addr learned my addr 172.21.15.5:0/792272013 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f30ff400640 1 -- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f310815f800 msgr2=0x7f3108162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f30ff400640 1 --2- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f310815f800 0x7f3108162ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f30ff400640 1 -- 172.21.15.5:0/792272013 >> v1:172.21.15.80:6789/0 conn(0x7f31080b2420 legacy=0x7f31080b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f30ff400640 1 -- 172.21.15.5:0/792272013 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310806d020 con 0x7f3108071000 2024-04-25T10:03:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.032+0000 7f3107e00640 1 --2- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f310815f800 0x7f3108162ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.033+0000 7f30ff400640 1 --2- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f3108071460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f310806e050 tx=0x7f30f002a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8c0fc3f89096c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.034+0000 7f3107400640 1 -- 172.21.15.5:0/792272013 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f002ae60 con 0x7f3108071000 2024-04-25T10:03:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.034+0000 7f3107400640 1 -- 172.21.15.5:0/792272013 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0007740 con 0x7f3108071000 2024-04-25T10:03:00.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.034+0000 7f3107400640 1 -- 172.21.15.5:0/792272013 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0007b10 con 0x7f3108071000 2024-04-25T10:03:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.034+0000 7f310dc00640 1 -- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 msgr2=0x7f3108071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.034+0000 7f310dc00640 1 --2- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f3108071460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f310806e050 tx=0x7f30f002a640 comp rx=0 tx=0).stop 2024-04-25T10:03:00.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.035+0000 7f310dc00640 1 -- 172.21.15.5:0/792272013 shutdown_connections 2024-04-25T10:03:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.035+0000 7f310dc00640 1 --2- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f3108071460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.035+0000 7f310dc00640 1 --2- 172.21.15.5:0/792272013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f310815f800 0x7f3108162ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.035+0000 7f310dc00640 1 -- 172.21.15.5:0/792272013 >> 172.21.15.5:0/792272013 conn(0x7f3108066f20 msgr2=0x7f310806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.035+0000 7f310dc00640 1 -- 172.21.15.5:0/792272013 shutdown_connections 2024-04-25T10:03:00.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.035+0000 7f310dc00640 1 -- 172.21.15.5:0/792272013 wait complete. 2024-04-25T10:03:00.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.037+0000 7f310dc00640 1 Processor -- start 2024-04-25T10:03:00.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.037+0000 7f310dc00640 1 -- start start 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f31080ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f31080b2420 0x7f3108160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 0x7f31081613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310806d5a0 con 0x7f310815f800 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31080b4850 con 0x7f3108071000 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310806d880 con 0x7f31080b2420 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 0x7f31081613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 0x7f31081613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60188/0 (socket says 172.21.15.5:60188) 2024-04-25T10:03:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f310d200640 1 -- 172.21.15.5:0/2298877044 learned_addr learned my addr 172.21.15.5:0/2298877044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f3107e00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f31080ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.038+0000 7f30ff400640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f31080b2420 0x7f3108160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310d200640 1 -- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f31080b2420 msgr2=0x7f3108160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310d200640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f31080b2420 0x7f3108160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310d200640 1 -- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 msgr2=0x7f31080ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310d200640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f31080ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310d200640 1 -- 172.21.15.5:0/2298877044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31080ae3b0 con 0x7f310815f800 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310d200640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 0x7f31081613d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f310000c420 tx=0x7f310000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100017070 con 0x7f310815f800 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100005ce0 con 0x7f310815f800 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31080b0980 con 0x7f310815f800 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.039+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31000072d0 con 0x7f310815f800 2024-04-25T10:03:00.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.040+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31081496b0 con 0x7f310815f800 2024-04-25T10:03:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.041+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3108068f60 con 0x7f310815f800 2024-04-25T10:03:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.042+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f310001a070 con 0x7f310815f800 2024-04-25T10:03:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.043+0000 7f3104c00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30e406b4d0 0x7f30e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.043+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f3100089870 con 0x7f310815f800 2024-04-25T10:03:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.043+0000 7f3107e00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30e406b4d0 0x7f30e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.044+0000 7f3107e00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30e406b4d0 0x7f30e406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3108158cd0 tx=0x7f30f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.047+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31081630e0 con 0x7f310815f800 2024-04-25T10:03:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.250+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3108163460 con 0x7f310815f800 2024-04-25T10:03:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.251+0000 7f3104c00640 1 -- 172.21.15.5:0/2298877044 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3100057c90 con 0x7f310815f800 2024-04-25T10:03:00.256 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30e406b4d0 msgr2=0x7f30e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30e406b4d0 0x7f30e406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3108158cd0 tx=0x7f30f8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 msgr2=0x7f31081613d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 0x7f31081613d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f310000c420 tx=0x7f310000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 shutdown_connections 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f310815f800 0x7f31081613d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f31080b2420 0x7f3108160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f30e406b4d0 0x7f30e406d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 --2- 172.21.15.5:0/2298877044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3108071000 0x7f31080ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.254+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 >> 172.21.15.5:0/2298877044 conn(0x7f3108066f20 msgr2=0x7f3108164af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.255+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 shutdown_connections 2024-04-25T10:03:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.255+0000 7f310dc00640 1 -- 172.21.15.5:0/2298877044 wait complete. 2024-04-25T10:03:00.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.492+0000 7fba14000640 1 Processor -- start 2024-04-25T10:03:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.492+0000 7fba14000640 1 -- start start 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.493+0000 7fba14000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba0c144fe0 0x7fba0c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba14000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c06d130 0x7fba0c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba14000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba0c071670 0x7fba0c0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba14000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0a8560 con 0x7fba0c144fe0 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba14000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c1453c0 con 0x7fba0c071670 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba14000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c06d6a0 con 0x7fba0c06d130 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba11600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba0c144fe0 0x7fba0c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba11600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fba0c144fe0 0x7fba0c06e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60192/0 (socket says 172.21.15.5:60192) 2024-04-25T10:03:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba11600640 1 -- 172.21.15.5:0/1251252076 learned_addr learned my addr 172.21.15.5:0/1251252076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba11600640 1 -- 172.21.15.5:0/1251252076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c06d130 msgr2=0x7fba0c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba12000640 1 --2- 172.21.15.5:0/1251252076 >> v2:172.21.15.80:3300/0 conn(0x7fba0c071670 0x7fba0c0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba10c00640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c06d130 0x7fba0c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.494+0000 7fba11600640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c06d130 0x7fba0c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.495+0000 7fba11600640 1 -- 172.21.15.5:0/1251252076 >> v2:172.21.15.80:3300/0 conn(0x7fba0c071670 msgr2=0x7fba0c0cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.495+0000 7fba11600640 1 --2- 172.21.15.5:0/1251252076 >> v2:172.21.15.80:3300/0 conn(0x7fba0c071670 0x7fba0c0cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.495+0000 7fba11600640 1 -- 172.21.15.5:0/1251252076 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c06d980 con 0x7fba0c144fe0 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.495+0000 7fba10c00640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c06d130 0x7fba0c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.496+0000 7fba11600640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c06e560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fba0c15bd40 tx=0x7fb9fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229fd7164d45152c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.496+0000 7fba0be00640 1 -- 172.21.15.5:0/1251252076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc037070 con 0x7fba0c144fe0 2024-04-25T10:03:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.496+0000 7fba0be00640 1 -- 172.21.15.5:0/1251252076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc032440 con 0x7fba0c144fe0 2024-04-25T10:03:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.496+0000 7fba0be00640 1 -- 172.21.15.5:0/1251252076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc032810 con 0x7fba0c144fe0 2024-04-25T10:03:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 -- 172.21.15.5:0/1251252076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 msgr2=0x7fba0c06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c06e560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fba0c15bd40 tx=0x7fb9fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 -- 172.21.15.5:0/1251252076 shutdown_connections 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 --2- 172.21.15.5:0/1251252076 >> v2:172.21.15.80:3300/0 conn(0x7fba0c071670 0x7fba0c0cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c06d130 0x7fba0c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 --2- 172.21.15.5:0/1251252076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c06e560 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 -- 172.21.15.5:0/1251252076 >> 172.21.15.5:0/1251252076 conn(0x7fba0c06ab60 msgr2=0x7fba0c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 -- 172.21.15.5:0/1251252076 shutdown_connections 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.497+0000 7fba14000640 1 -- 172.21.15.5:0/1251252076 wait complete. 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.499+0000 7fba14000640 1 Processor -- start 2024-04-25T10:03:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.499+0000 7fba14000640 1 -- start start 2024-04-25T10:03:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba14000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba0c06d130 0x7fba0c142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba14000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c071670 0x7fba0c1e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba14000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba14000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c15be50 con 0x7fba0c144fe0 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba14000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c06df40 con 0x7fba0c06d130 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba14000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c06eaa0 con 0x7fba0c071670 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba12000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba12000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c0ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60194/0 (socket says 172.21.15.5:60194) 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.500+0000 7fba12000640 1 -- 172.21.15.5:0/2647281053 learned_addr learned my addr 172.21.15.5:0/2647281053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba11600640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba0c06d130 0x7fba0c142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba10c00640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c071670 0x7fba0c1e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba12000640 1 -- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c071670 msgr2=0x7fba0c1e1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba12000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c071670 0x7fba0c1e1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba12000640 1 -- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba0c06d130 msgr2=0x7fba0c142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba12000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba0c06d130 0x7fba0c142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba12000640 1 -- 172.21.15.5:0/2647281053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c1e2620 con 0x7fba0c144fe0 2024-04-25T10:03:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba12000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c0ad2d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba0400cda0 tx=0x7fba0400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04003b90 con 0x7fba0c144fe0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04003d30 con 0x7fba0c144fe0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04004240 con 0x7fba0c144fe0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.501+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c1e2900 con 0x7fba0c144fe0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.502+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c1e2ec0 con 0x7fba0c144fe0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.503+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c068f60 con 0x7fba0c144fe0 2024-04-25T10:03:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.504+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba040043e0 con 0x7fba0c144fe0 2024-04-25T10:03:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.505+0000 7fba09600640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9f006b4d0 0x7fb9f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.505+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fba0408b100 con 0x7fba0c144fe0 2024-04-25T10:03:00.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.506+0000 7fba11600640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9f006b4d0 0x7fb9f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.507+0000 7fba11600640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9f006b4d0 0x7fb9f006d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc034000 tx=0x7fb9fc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.509+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba0401d340 con 0x7fba0c144fe0 2024-04-25T10:03:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.710+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba0c1e3100 con 0x7fba0c144fe0 2024-04-25T10:03:00.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.711+0000 7fba09600640 1 -- 172.21.15.5:0/2647281053 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba04059c00 con 0x7fba0c144fe0 2024-04-25T10:03:00.716 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:00.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.713+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9f006b4d0 msgr2=0x7fb9f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.713+0000 7fba14000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9f006b4d0 0x7fb9f006d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc034000 tx=0x7fb9fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:00.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.713+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 msgr2=0x7fba0c0ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.713+0000 7fba14000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c0ad2d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fba0400cda0 tx=0x7fba0400a660 comp rx=0 tx=0).stop 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 shutdown_connections 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba0c144fe0 0x7fba0c0ad2d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba0c071670 0x7fba0c1e1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9f006b4d0 0x7fb9f006d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 --2- 172.21.15.5:0/2647281053 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba0c06d130 0x7fba0c142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 >> 172.21.15.5:0/2647281053 conn(0x7fba0c06ab60 msgr2=0x7fba0c144290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 shutdown_connections 2024-04-25T10:03:00.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.714+0000 7fba14000640 1 -- 172.21.15.5:0/2647281053 wait complete. 2024-04-25T10:03:00.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.944+0000 7f753ae00640 1 Processor -- start 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.944+0000 7f753ae00640 1 -- start start 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.945+0000 7f753ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340b48c0 0x7f753415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.945+0000 7f753ae00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7534161d90 0x7f75340b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.945+0000 7f753ae00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75341606e0 con 0x7f75340d3f90 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.945+0000 7f753ae00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75341609c0 con 0x7f7534161d90 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.945+0000 7f753ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7534160ca0 con 0x7f75340b48c0 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.946+0000 7f7539000640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f75340d3f90 0x7f7534161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:43620/0 (socket says 172.21.15.5:43620) 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.946+0000 7f7539000640 1 -- 172.21.15.5:0/300178387 learned_addr learned my addr 172.21.15.5:0/300178387 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.946+0000 7f7539a00640 1 --2- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340b48c0 0x7f753415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.946+0000 7f753a400640 1 --2- 172.21.15.5:0/300178387 >> v2:172.21.15.80:3300/0 conn(0x7f7534161d90 0x7f75340b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.947+0000 7f753a400640 1 -- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340b48c0 msgr2=0x7f753415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.947+0000 7f753a400640 1 --2- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340b48c0 0x7f753415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.947+0000 7f753a400640 1 -- 172.21.15.5:0/300178387 >> v1:172.21.15.80:6789/0 conn(0x7f75340d3f90 legacy=0x7f7534161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.947+0000 7f753a400640 1 -- 172.21.15.5:0/300178387 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534160f80 con 0x7f7534161d90 2024-04-25T10:03:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.948+0000 7f753a400640 1 --2- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b2b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f752c009d90 tx=0x7f752c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d05167dfe3382e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f7533e00640 1 -- 172.21.15.5:0/300178387 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c031600 con 0x7f7534161d90 2024-04-25T10:03:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f7533e00640 1 -- 172.21.15.5:0/300178387 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c0317a0 con 0x7f7534161d90 2024-04-25T10:03:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f7533e00640 1 -- 172.21.15.5:0/300178387 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c031b70 con 0x7f7534161d90 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f753ae00640 1 -- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 msgr2=0x7f75340b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f753ae00640 1 --2- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b2b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f752c009d90 tx=0x7f752c02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f753ae00640 1 -- 172.21.15.5:0/300178387 shutdown_connections 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f753ae00640 1 --2- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b2b70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f753ae00640 1 --2- 172.21.15.5:0/300178387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340b48c0 0x7f753415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.949+0000 7f753ae00640 1 -- 172.21.15.5:0/300178387 >> 172.21.15.5:0/300178387 conn(0x7f753408af80 msgr2=0x7f75340d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.950+0000 7f753ae00640 1 -- 172.21.15.5:0/300178387 shutdown_connections 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.950+0000 7f753ae00640 1 -- 172.21.15.5:0/300178387 wait complete. 2024-04-25T10:03:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.951+0000 7f753ae00640 1 Processor -- start 2024-04-25T10:03:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.952+0000 7f753ae00640 1 -- start start 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 0x7f75340cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340d3f90 0x7f75340b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75341603a0 con 0x7f75340b48c0 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340b4ca0 con 0x7f7534161d90 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75340b30b0 con 0x7f75340d3f90 2024-04-25T10:03:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f7539a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 0x7f75340cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f7539a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 0x7f75340cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60210/0 (socket says 172.21.15.5:60210) 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f7539a00640 1 -- 172.21.15.5:0/529667966 learned_addr learned my addr 172.21.15.5:0/529667966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f7539000640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340d3f90 0x7f75340b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.953+0000 7f753a400640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7539a00640 1 -- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340d3f90 msgr2=0x7f75340b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7539a00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340d3f90 0x7f75340b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7539a00640 1 -- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 msgr2=0x7f75340b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7539a00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7539a00640 1 -- 172.21.15.5:0/529667966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340c7cd0 con 0x7f75340b48c0 2024-04-25T10:03:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7539a00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 0x7f75340cc130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f752400e9b0 tx=0x7f752400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524017070 con 0x7f75340b48c0 2024-04-25T10:03:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400c9a0 con 0x7f75340b48c0 2024-04-25T10:03:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75340cc8a0 con 0x7f75340b48c0 2024-04-25T10:03:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400cd90 con 0x7f75340b48c0 2024-04-25T10:03:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.954+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75340cce60 con 0x7f75340b48c0 2024-04-25T10:03:00.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.955+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7534068f60 con 0x7f75340b48c0 2024-04-25T10:03:00.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.956+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f752401a070 con 0x7f75340b48c0 2024-04-25T10:03:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.957+0000 7f7531600640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f751406b4d0 0x7f751406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.958+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f752408a550 con 0x7f75340b48c0 2024-04-25T10:03:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.959+0000 7f7539000640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f751406b4d0 0x7f751406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.960+0000 7f7539000640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f751406b4d0 0x7f751406d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75340b2850 tx=0x7f7520009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:00.962+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7524059050 con 0x7f75340b48c0 2024-04-25T10:03:01.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.162+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f75340d0dd0 con 0x7f75340b48c0 2024-04-25T10:03:01.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.163+0000 7f7531600640 1 -- 172.21.15.5:0/529667966 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7524058950 con 0x7f75340b48c0 2024-04-25T10:03:01.167 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f751406b4d0 msgr2=0x7f751406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f751406b4d0 0x7f751406d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75340b2850 tx=0x7f7520009040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 msgr2=0x7f75340cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 0x7f75340cc130 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f752400e9b0 tx=0x7f752400ee70 comp rx=0 tx=0).stop 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 shutdown_connections 2024-04-25T10:03:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f751406b4d0 0x7f751406d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7534161d90 0x7f75340b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75340d3f90 0x7f75340b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 --2- 172.21.15.5:0/529667966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75340b48c0 0x7f75340cc130 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 >> 172.21.15.5:0/529667966 conn(0x7f753408af80 msgr2=0x7f753415f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.165+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 shutdown_connections 2024-04-25T10:03:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.166+0000 7f753ae00640 1 -- 172.21.15.5:0/529667966 wait complete. 2024-04-25T10:03:01.196 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:03:01.197 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.389+0000 7f2aaf200640 1 Processor -- start 2024-04-25T10:03:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.389+0000 7f2aaf200640 1 -- start start 2024-04-25T10:03:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aaf200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ab015eda0 0x7f2ab00ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab00cecd0 0x7f2ab015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aaf200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00b4880 con 0x7f2ab0143d30 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aaf200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00b45a0 con 0x7f2ab015eda0 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aaf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab015e2c0 con 0x7f2ab00cecd0 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aad400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ab015eda0 0x7f2ab00ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aade00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2ab0143d30 0x7f2ab00b3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53186/0 (socket says 172.21.15.5:53186) 2024-04-25T10:03:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.391+0000 7f2aade00640 1 -- 172.21.15.5:0/4018872213 learned_addr learned my addr 172.21.15.5:0/4018872213 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.392+0000 7f2aad400640 1 -- 172.21.15.5:0/4018872213 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab00cecd0 msgr2=0x7f2ab015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.392+0000 7f2aae800640 1 --2- 172.21.15.5:0/4018872213 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab00cecd0 0x7f2ab015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.392+0000 7f2aad400640 1 --2- 172.21.15.5:0/4018872213 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab00cecd0 0x7f2ab015abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.392+0000 7f2aad400640 1 -- 172.21.15.5:0/4018872213 >> v1:172.21.15.5:6789/0 conn(0x7f2ab0143d30 legacy=0x7f2ab00b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.392+0000 7f2aad400640 1 -- 172.21.15.5:0/4018872213 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab015e5a0 con 0x7f2ab015eda0 2024-04-25T10:03:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.392+0000 7f2aaca00640 1 -- 172.21.15.5:0/4018872213 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951539514 0 0) 0x7f2ab00b4880 con 0x7f2ab0143d30 2024-04-25T10:03:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.393+0000 7f2aad400640 1 --2- 172.21.15.5:0/4018872213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab015eda0 0x7f2ab00ce790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0097a0 tx=0x7f2a9c02c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e4f1ca9deae641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.393+0000 7f2aaca00640 1 -- 172.21.15.5:0/4018872213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c039070 con 0x7f2ab015eda0 2024-04-25T10:03:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.393+0000 7f2aaca00640 1 -- 172.21.15.5:0/4018872213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c034640 con 0x7f2ab015eda0 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.393+0000 7f2aaca00640 1 -- 172.21.15.5:0/4018872213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c02ce60 con 0x7f2ab015eda0 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 -- 172.21.15.5:0/4018872213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab015eda0 msgr2=0x7f2ab00ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 --2- 172.21.15.5:0/4018872213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab015eda0 0x7f2ab00ce790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0097a0 tx=0x7f2a9c02c310 comp rx=0 tx=0).stop 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 -- 172.21.15.5:0/4018872213 shutdown_connections 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 --2- 172.21.15.5:0/4018872213 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab00cecd0 0x7f2ab015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 --2- 172.21.15.5:0/4018872213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab015eda0 0x7f2ab00ce790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 -- 172.21.15.5:0/4018872213 >> 172.21.15.5:0/4018872213 conn(0x7f2ab0066f20 msgr2=0x7f2ab006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 -- 172.21.15.5:0/4018872213 shutdown_connections 2024-04-25T10:03:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.394+0000 7f2aaf200640 1 -- 172.21.15.5:0/4018872213 wait complete. 2024-04-25T10:03:01.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.396+0000 7f2aaf200640 1 Processor -- start 2024-04-25T10:03:01.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.396+0000 7f2aaf200640 1 -- start start 2024-04-25T10:03:01.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab00cecd0 0x7f2ab013e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 0x7f2ab015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aaf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab015eda0 0x7f2ab013c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aaf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00b41f0 con 0x7f2ab0143d30 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aaf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0144110 con 0x7f2ab00cecd0 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aaf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab013d350 con 0x7f2ab015eda0 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 0x7f2ab015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 0x7f2ab015ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60216/0 (socket says 172.21.15.5:60216) 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 -- 172.21.15.5:0/113543180 learned_addr learned my addr 172.21.15.5:0/113543180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aae800640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab015eda0 0x7f2ab013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aade00640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab00cecd0 0x7f2ab013e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 -- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab015eda0 msgr2=0x7f2ab013c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab015eda0 0x7f2ab013c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 -- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab00cecd0 msgr2=0x7f2ab013e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab00cecd0 0x7f2ab013e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.397+0000 7f2aad400640 1 -- 172.21.15.5:0/113543180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab013d630 con 0x7f2ab0143d30 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.398+0000 7f2aad400640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 0x7f2ab015ce90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c037040 tx=0x7f2a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.398+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c043020 con 0x7f2ab0143d30 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.398+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c032960 con 0x7f2ab0143d30 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.398+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab013d910 con 0x7f2ab0143d30 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.398+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c039070 con 0x7f2ab0143d30 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.398+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab013c230 con 0x7f2ab0143d30 2024-04-25T10:03:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.399+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab013dae0 con 0x7f2ab0143d30 2024-04-25T10:03:01.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.401+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a9c03f440 con 0x7f2ab0143d30 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.402+0000 7f2aa6000640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a9006b4d0 0x7f2a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.402+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f2a9c0aaf20 con 0x7f2ab0143d30 2024-04-25T10:03:01.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.402+0000 7f2aade00640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a9006b4d0 0x7f2a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.403+0000 7f2aade00640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a9006b4d0 0x7f2a9006d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000f60 tx=0x7f2aa0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.407+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a9c04d020 con 0x7f2ab0143d30 2024-04-25T10:03:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.608+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ab006ab60 con 0x7f2ab0143d30 2024-04-25T10:03:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.611+0000 7f2aa6000640 1 -- 172.21.15.5:0/113543180 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f2ab006ab60 con 0x7f2ab0143d30 2024-04-25T10:03:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:01.617 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a9006b4d0 msgr2=0x7f2a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a9006b4d0 0x7f2a9006d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0000f60 tx=0x7f2aa0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 msgr2=0x7f2ab015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 0x7f2ab015ce90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c037040 tx=0x7f2a9c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 shutdown_connections 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab015eda0 0x7f2ab013c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab0143d30 0x7f2ab015ce90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a9006b4d0 0x7f2a9006d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 --2- 172.21.15.5:0/113543180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab00cecd0 0x7f2ab013e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.614+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 >> 172.21.15.5:0/113543180 conn(0x7f2ab0066f20 msgr2=0x7f2ab01594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.615+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 shutdown_connections 2024-04-25T10:03:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.615+0000 7f2aaf200640 1 -- 172.21.15.5:0/113543180 wait complete. 2024-04-25T10:03:01.638 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:01.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:01.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.845+0000 7fd33ec00640 1 Processor -- start 2024-04-25T10:03:01.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.846+0000 7fd33ec00640 1 -- start start 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.847+0000 7fd33ec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 0x7fd3381689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.847+0000 7fd33ec00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3381606e0 con 0x7fd3380b48c0 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.847+0000 7fd33ec00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3381609c0 con 0x7fd3380d3f90 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.847+0000 7fd33ec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338160ca0 con 0x7fd338161d90 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.848+0000 7fd33d800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd3380b48c0 0x7fd33815fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53196/0 (socket says 172.21.15.5:53196) 2024-04-25T10:03:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.848+0000 7fd33d800640 1 -- 172.21.15.5:0/348184430 learned_addr learned my addr 172.21.15.5:0/348184430 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.848+0000 7fd33e200640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 0x7fd3381689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.848+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471064975 0 0) 0x7fd3381606e0 con 0x7fd3380b48c0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.848+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd320003520 con 0x7fd3380b48c0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4154313414 0 0) 0x7fd320003520 con 0x7fd3380b48c0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 msgr2=0x7fd3381689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 0x7fd3381689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 >> v1:172.21.15.80:6789/0 conn(0x7fd3380d3f90 legacy=0x7fd338161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd338160f80 con 0x7fd3380b48c0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd32c001670 con 0x7fd3380b48c0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 >> v1:172.21.15.5:6789/0 conn(0x7fd3380b48c0 legacy=0x7fd33815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 0x7fd3200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd320003fb0 0x7fd320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd320004d40 0x7fd3200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338160ca0 con 0x7fd320004d40 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32000bbb0 con 0x7fd320003fb0 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32000bdb0 con 0x7fd338161d90 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.849+0000 7fd33e200640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd320004d40 0x7fd3200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33ce00640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd320003fb0 0x7fd320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33d800640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 0x7fd3200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33e200640 1 -- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 msgr2=0x7fd3200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33e200640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd338161d90 0x7fd3200039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33e200640 1 -- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd320003fb0 msgr2=0x7fd320004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33e200640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd320003fb0 0x7fd320004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.850+0000 7fd33e200640 1 -- 172.21.15.5:0/348184430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320003520 con 0x7fd320004d40 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.851+0000 7fd33e200640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd320004d40 0x7fd3200051c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd33808d300 tx=0x7fd320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4548926029e2bd6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.851+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330037070 con 0x7fd320004d40 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.851+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33002e050 con 0x7fd320004d40 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.851+0000 7fd337e00640 1 -- 172.21.15.5:0/348184430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330009c10 con 0x7fd320004d40 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33ec00640 1 -- 172.21.15.5:0/348184430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd320004d40 msgr2=0x7fd3200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33ec00640 1 --2- 172.21.15.5:0/348184430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd320004d40 0x7fd3200051c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd33808d300 tx=0x7fd320005830 comp rx=0 tx=0).stop 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33e200640 1 -- 172.21.15.5:0/348184430 reap_dead start 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33ec00640 1 -- 172.21.15.5:0/348184430 shutdown_connections 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33ec00640 1 -- 172.21.15.5:0/348184430 >> 172.21.15.5:0/348184430 conn(0x7fd33808af80 msgr2=0x7fd3380d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33ec00640 1 -- 172.21.15.5:0/348184430 shutdown_connections 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.852+0000 7fd33ec00640 1 -- 172.21.15.5:0/348184430 wait complete. 2024-04-25T10:03:01.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.854+0000 7fd33ec00640 1 Processor -- start 2024-04-25T10:03:01.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.854+0000 7fd33ec00640 1 -- start start 2024-04-25T10:03:01.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.855+0000 7fd33ec00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd33815be40 0x7fd33806c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.855+0000 7fd33ec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd33815c220 0x7fd33806c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.855+0000 7fd33ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 0x7fd3380cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.855+0000 7fd33ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380b3b80 con 0x7fd3380cbc60 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.855+0000 7fd33ec00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380b4ca0 con 0x7fd33815be40 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.855+0000 7fd33ec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380b3d00 con 0x7fd33815c220 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 0x7fd3380cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 0x7fd3380cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60230/0 (socket says 172.21.15.5:60230) 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 -- 172.21.15.5:0/1889382021 learned_addr learned my addr 172.21.15.5:0/1889382021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33d800640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd33815be40 0x7fd33806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33ce00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd33815c220 0x7fd33806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 -- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd33815c220 msgr2=0x7fd33806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd33815c220 0x7fd33806c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 -- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd33815be40 msgr2=0x7fd33806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd33815be40 0x7fd33806c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 -- 172.21.15.5:0/1889382021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380cc9d0 con 0x7fd3380cbc60 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33d800640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd33815be40 0x7fd33806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.856+0000 7fd33e200640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 0x7fd3380cc110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd330035040 tx=0x7fd330009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.857+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330037040 con 0x7fd3380cbc60 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.857+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd330044070 con 0x7fd3380cbc60 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.857+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330009c10 con 0x7fd3380cbc60 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.857+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3380cccb0 con 0x7fd3380cbc60 2024-04-25T10:03:01.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.857+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3380cd270 con 0x7fd3380cbc60 2024-04-25T10:03:01.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.858+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd338068f60 con 0x7fd3380cbc60 2024-04-25T10:03:01.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.859+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd330005d10 con 0x7fd3380cbc60 2024-04-25T10:03:01.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.860+0000 7fd335600640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd31c06b4d0 0x7fd31c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.860+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fd3300aad60 con 0x7fd3380cbc60 2024-04-25T10:03:01.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.861+0000 7fd33d800640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd31c06b4d0 0x7fd31c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.862+0000 7fd33d800640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd31c06b4d0 0x7fd31c06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd32c001eb0 tx=0x7fd32c000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:01.864+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd33004d020 con 0x7fd3380cbc60 2024-04-25T10:03:02.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.061+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd33806db60 con 0x7fd3380cbc60 2024-04-25T10:03:02.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.061+0000 7fd335600640 1 -- 172.21.15.5:0/1889382021 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3300797d0 con 0x7fd3380cbc60 2024-04-25T10:03:02.066 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd31c06b4d0 msgr2=0x7fd31c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd31c06b4d0 0x7fd31c06d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd32c001eb0 tx=0x7fd32c000ed0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 msgr2=0x7fd3380cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 0x7fd3380cc110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd330035040 tx=0x7fd330009750 comp rx=0 tx=0).stop 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 shutdown_connections 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3380cbc60 0x7fd3380cc110 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd33815c220 0x7fd33806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd31c06b4d0 0x7fd31c06d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 --2- 172.21.15.5:0/1889382021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd33815be40 0x7fd33806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 >> 172.21.15.5:0/1889382021 conn(0x7fd33808af80 msgr2=0x7fd3380b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 shutdown_connections 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.064+0000 7fd33ec00640 1 -- 172.21.15.5:0/1889382021 wait complete. 2024-04-25T10:03:02.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:02.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.291+0000 7f4aca600640 1 Processor -- start 2024-04-25T10:03:02.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.292+0000 7f4aca600640 1 -- start start 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.293+0000 7f4aca600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406f7d0 0x7f4ac41620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.293+0000 7f4aca600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ac406ab60 0x7f4ac406fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.293+0000 7f4aca600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ac40b2950 0x7f4ac406e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.293+0000 7f4aca600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac4161c10 con 0x7f4ac406ab60 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.293+0000 7f4aca600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac41625e0 con 0x7f4ac40b2950 2024-04-25T10:03:02.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.293+0000 7f4aca600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40700f0 con 0x7f4ac406f7d0 2024-04-25T10:03:02.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ac406ab60 0x7f4ac406fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ac406ab60 0x7f4ac406fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60236/0 (socket says 172.21.15.5:60236) 2024-04-25T10:03:02.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 -- 172.21.15.5:0/3387034826 learned_addr learned my addr 172.21.15.5:0/3387034826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac3400640 1 --2- 172.21.15.5:0/3387034826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406f7d0 0x7f4ac41620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac3e00640 1 --2- 172.21.15.5:0/3387034826 >> v2:172.21.15.80:3300/0 conn(0x7f4ac40b2950 0x7f4ac406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 -- 172.21.15.5:0/3387034826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406f7d0 msgr2=0x7f4ac41620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 --2- 172.21.15.5:0/3387034826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406f7d0 0x7f4ac41620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 -- 172.21.15.5:0/3387034826 >> v2:172.21.15.80:3300/0 conn(0x7f4ac40b2950 msgr2=0x7f4ac406e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 --2- 172.21.15.5:0/3387034826 >> v2:172.21.15.80:3300/0 conn(0x7f4ac40b2950 0x7f4ac406e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.294+0000 7f4ac2a00640 1 -- 172.21.15.5:0/3387034826 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac406b0d0 con 0x7f4ac406ab60 2024-04-25T10:03:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.296+0000 7f4ac2a00640 1 --2- 172.21.15.5:0/3387034826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac406ab60 0x7f4ac406fbb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009e50 tx=0x7f4aac02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da68a030ac578a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.296+0000 7f4ac2000640 1 -- 172.21.15.5:0/3387034826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac036070 con 0x7f4ac406ab60 2024-04-25T10:03:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.296+0000 7f4ac2000640 1 -- 172.21.15.5:0/3387034826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aac031600 con 0x7f4ac406ab60 2024-04-25T10:03:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.296+0000 7f4ac2000640 1 -- 172.21.15.5:0/3387034826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0319d0 con 0x7f4ac406ab60 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 -- 172.21.15.5:0/3387034826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac406ab60 msgr2=0x7f4ac406fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 --2- 172.21.15.5:0/3387034826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac406ab60 0x7f4ac406fbb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009e50 tx=0x7f4aac02a760 comp rx=0 tx=0).stop 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 -- 172.21.15.5:0/3387034826 shutdown_connections 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 --2- 172.21.15.5:0/3387034826 >> v2:172.21.15.80:3300/0 conn(0x7f4ac40b2950 0x7f4ac406e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 --2- 172.21.15.5:0/3387034826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac406ab60 0x7f4ac406fbb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 --2- 172.21.15.5:0/3387034826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406f7d0 0x7f4ac41620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 -- 172.21.15.5:0/3387034826 >> 172.21.15.5:0/3387034826 conn(0x7f4ac40999f0 msgr2=0x7f4ac40b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 -- 172.21.15.5:0/3387034826 shutdown_connections 2024-04-25T10:03:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.297+0000 7f4aca600640 1 -- 172.21.15.5:0/3387034826 wait complete. 2024-04-25T10:03:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.299+0000 7f4aca600640 1 Processor -- start 2024-04-25T10:03:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.299+0000 7f4aca600640 1 -- start start 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4aca600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406ab60 0x7f4ac415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4aca600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ac406f7d0 0x7f4ac40ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4aca600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 0x7f4ac40ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4aca600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40afd90 con 0x7f4ac40b2950 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4aca600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac41618a0 con 0x7f4ac406f7d0 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4aca600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40b2de0 con 0x7f4ac406ab60 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4ac3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 0x7f4ac40ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4ac3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 0x7f4ac40ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60250/0 (socket says 172.21.15.5:60250) 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4ac3e00640 1 -- 172.21.15.5:0/1398974942 learned_addr learned my addr 172.21.15.5:0/1398974942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4ac2a00640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ac406f7d0 0x7f4ac40ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.300+0000 7f4ac3400640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406ab60 0x7f4ac415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3e00640 1 -- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406ab60 msgr2=0x7f4ac415ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3e00640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406ab60 0x7f4ac415ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3e00640 1 -- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ac406f7d0 msgr2=0x7f4ac40ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3e00640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ac406f7d0 0x7f4ac40ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3e00640 1 -- 172.21.15.5:0/1398974942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac41613c0 con 0x7f4ac40b2950 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3400640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406ab60 0x7f4ac415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.301+0000 7f4ac3e00640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 0x7f4ac40ae8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00d5d0 tx=0x7f4abc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.302+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0125f0 con 0x7f4ac40b2950 2024-04-25T10:03:02.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.302+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc019070 con 0x7f4ac40b2950 2024-04-25T10:03:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.302+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0129f0 con 0x7f4ac40b2950 2024-04-25T10:03:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.302+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac40b4790 con 0x7f4ac40b2950 2024-04-25T10:03:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.302+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac4160b20 con 0x7f4ac40b2950 2024-04-25T10:03:02.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.303+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4068f60 con 0x7f4ac40b2950 2024-04-25T10:03:02.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.305+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4abc012b90 con 0x7f4ac40b2950 2024-04-25T10:03:02.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.306+0000 7f4abb400640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4aa006b4d0 0x7f4aa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.306+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f4abc08a950 con 0x7f4ac40b2950 2024-04-25T10:03:02.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.306+0000 7f4ac3400640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4aa006b4d0 0x7f4aa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.308+0000 7f4ac3400640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4aa006b4d0 0x7f4aa006d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006fd0 tx=0x7f4ab4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.311+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac41613c0 con 0x7f4ac40b2950 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.515+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ac40ce3a0 con 0x7f4ac40b2950 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.516+0000 7f4abb400640 1 -- 172.21.15.5:0/1398974942 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4abc0593d0 con 0x7f4ac40b2950 2024-04-25T10:03:02.521 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4aa006b4d0 msgr2=0x7f4aa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4aa006b4d0 0x7f4aa006d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4006fd0 tx=0x7f4ab4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 msgr2=0x7f4ac40ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 0x7f4ac40ae8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00d5d0 tx=0x7f4abc00da90 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 shutdown_connections 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac40b2950 0x7f4ac40ae8f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4aa006b4d0 0x7f4aa006d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ac406f7d0 0x7f4ac40ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 --2- 172.21.15.5:0/1398974942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ac406ab60 0x7f4ac415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 >> 172.21.15.5:0/1398974942 conn(0x7f4ac40999f0 msgr2=0x7f4ac40cad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 shutdown_connections 2024-04-25T10:03:02.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.519+0000 7f4aca600640 1 -- 172.21.15.5:0/1398974942 wait complete. 2024-04-25T10:03:02.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:02.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.754+0000 7f98df200640 1 Processor -- start 2024-04-25T10:03:02.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.754+0000 7f98df200640 1 -- start start 2024-04-25T10:03:02.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98df200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f98e0154380 0x7f98e00b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98df200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e015e6b0 con 0x7f98e0071fa0 2024-04-25T10:03:02.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98df200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e015a310 con 0x7f98e015e200 2024-04-25T10:03:02.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98df200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e015a5f0 con 0x7f98e0154380 2024-04-25T10:03:02.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98de800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f98e0071fa0 0x7f98e006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53212/0 (socket says 172.21.15.5:53212) 2024-04-25T10:03:02.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98de800640 1 -- 172.21.15.5:0/3529944046 learned_addr learned my addr 172.21.15.5:0/3529944046 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:02.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.756+0000 7f98dde00640 1 --2- 172.21.15.5:0/3529944046 >> v2:172.21.15.80:3300/0 conn(0x7f98e0154380 0x7f98e00b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379717859 0 0) 0x7f98e015e6b0 con 0x7f98e0071fa0 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c4005730 con 0x7f98e0071fa0 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2525778195 0 0) 0x7f98c4005730 con 0x7f98e0071fa0 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 >> v2:172.21.15.80:3300/0 conn(0x7f98e0154380 msgr2=0x7f98e00b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 --2- 172.21.15.5:0/3529944046 >> v2:172.21.15.80:3300/0 conn(0x7f98e0154380 0x7f98e00b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 >> v1:172.21.15.80:6789/0 conn(0x7f98e015e200 legacy=0x7f98e0071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e015a8d0 con 0x7f98e0071fa0 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f98d80019f0 con 0x7f98e0071fa0 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 >> v1:172.21.15.5:6789/0 conn(0x7f98e0071fa0 legacy=0x7f98e006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98c4005bc0 0x7f98c4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c4006a10 0x7f98c4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.757+0000 7f98dca00640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c40073d0 0x7f98c400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e015a5f0 con 0x7f98c40073d0 2024-04-25T10:03:02.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c400e270 con 0x7f98c4006a10 2024-04-25T10:03:02.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c400e470 con 0x7f98c4005bc0 2024-04-25T10:03:02.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98de800640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c40073d0 0x7f98c400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98dd400640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c4006a10 0x7f98c4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98dde00640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98c4005bc0 0x7f98c4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98de800640 1 -- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98c4005bc0 msgr2=0x7f98c4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98de800640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98c4005bc0 0x7f98c4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98de800640 1 -- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c4006a10 msgr2=0x7f98c4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.758+0000 7f98de800640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c4006a10 0x7f98c4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.759+0000 7f98de800640 1 -- 172.21.15.5:0/3529944046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4005730 con 0x7f98c40073d0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.759+0000 7f98dde00640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98c4005bc0 0x7f98c4005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.760+0000 7f98de800640 1 -- 172.21.15.5:0/3529944046 reap_dead start 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.760+0000 7f98de800640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c40073d0 0x7f98c400dbd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f98d800a830 tx=0x7f98c4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3726ded6d826b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.760+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800a480 con 0x7f98c40073d0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.760+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d802f070 con 0x7f98c40073d0 2024-04-25T10:03:02.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.760+0000 7f98dca00640 1 -- 172.21.15.5:0/3529944046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d802bc90 con 0x7f98c40073d0 2024-04-25T10:03:02.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 -- 172.21.15.5:0/3529944046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c40073d0 msgr2=0x7f98c400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c40073d0 0x7f98c400dbd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f98d800a830 tx=0x7f98c4007a40 comp rx=0 tx=0).stop 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 -- 172.21.15.5:0/3529944046 shutdown_connections 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 --2- 172.21.15.5:0/3529944046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c40073d0 0x7f98c400dbd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 -- 172.21.15.5:0/3529944046 >> 172.21.15.5:0/3529944046 conn(0x7f98e0066f20 msgr2=0x7f98e006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 -- 172.21.15.5:0/3529944046 shutdown_connections 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.761+0000 7f98df200640 1 -- 172.21.15.5:0/3529944046 wait complete. 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.763+0000 7f98df200640 1 Processor -- start 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.763+0000 7f98df200640 1 -- start start 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98df200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c40073d0 0x7f98e00afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98df200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 0x7f98e014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98df200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98e00b15b0 0x7f98e00b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98df200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00cdae0 con 0x7f98e014ee60 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98df200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00cd960 con 0x7f98c40073d0 2024-04-25T10:03:02.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98df200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00b3e70 con 0x7f98e00b15b0 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98dd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 0x7f98e014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98dd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 0x7f98e014c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60270/0 (socket says 172.21.15.5:60270) 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.764+0000 7f98dd400640 1 -- 172.21.15.5:0/4024791014 learned_addr learned my addr 172.21.15.5:0/4024791014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dde00640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c40073d0 0x7f98e00afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98de800640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98e00b15b0 0x7f98e00b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dd400640 1 -- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98e00b15b0 msgr2=0x7f98e00b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dd400640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98e00b15b0 0x7f98e00b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dd400640 1 -- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c40073d0 msgr2=0x7f98e00afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dd400640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c40073d0 0x7f98e00afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dd400640 1 -- 172.21.15.5:0/4024791014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00b0210 con 0x7f98e014ee60 2024-04-25T10:03:02.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.765+0000 7f98dd400640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 0x7f98e014c140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00f5f0 tx=0x7f98cc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.766+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0052e0 con 0x7f98e014ee60 2024-04-25T10:03:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.766+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc018070 con 0x7f98e014ee60 2024-04-25T10:03:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.766+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc003810 con 0x7f98e014ee60 2024-04-25T10:03:02.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.766+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e00b04f0 con 0x7f98e014ee60 2024-04-25T10:03:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.766+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e01548e0 con 0x7f98e014ee60 2024-04-25T10:03:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.767+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e006e090 con 0x7f98e014ee60 2024-04-25T10:03:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.768+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98cc0039b0 con 0x7f98e014ee60 2024-04-25T10:03:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.769+0000 7f98d6000640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f98c006b500 0x7f98c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.769+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f98cc08a290 con 0x7f98e014ee60 2024-04-25T10:03:02.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.769+0000 7f98dde00640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f98c006b500 0x7f98c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.771+0000 7f98dde00640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f98c006b500 0x7f98c006d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009990 tx=0x7f98d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.773+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98e006e090 con 0x7f98e014ee60 2024-04-25T10:03:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.975+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98e0068f60 con 0x7f98e014ee60 2024-04-25T10:03:02.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.976+0000 7f98d6000640 1 -- 172.21.15.5:0/4024791014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98cc058d90 con 0x7f98e014ee60 2024-04-25T10:03:02.981 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f98c006b500 msgr2=0x7f98c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f98c006b500 0x7f98c006d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009990 tx=0x7f98d0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 msgr2=0x7f98e014c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 0x7f98e014c140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00f5f0 tx=0x7f98cc00fab0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 shutdown_connections 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98e00b15b0 0x7f98e00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e014ee60 0x7f98e014c140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f98c006b500 0x7f98c006d9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 --2- 172.21.15.5:0/4024791014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c40073d0 0x7f98e00afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 >> 172.21.15.5:0/4024791014 conn(0x7f98e0066f20 msgr2=0x7f98e0157e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.979+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 shutdown_connections 2024-04-25T10:03:02.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:02.980+0000 7f98df200640 1 -- 172.21.15.5:0/4024791014 wait complete. 2024-04-25T10:03:03.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.220+0000 7f0fb5800640 1 Processor -- start 2024-04-25T10:03:03.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.220+0000 7f0fb5800640 1 -- start start 2024-04-25T10:03:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.221+0000 7f0fb5800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 0x7f0fb01685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.221+0000 7f0fb5800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb00b7ef0 con 0x7f0fb00d0780 2024-04-25T10:03:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.221+0000 7f0fb5800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb00b81d0 con 0x7f0fb00d2c40 2024-04-25T10:03:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.221+0000 7f0fb5800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b9840 con 0x7f0fb00b78c0 2024-04-25T10:03:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0faea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0fb00d0780 0x7f0fb00d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53218/0 (socket says 172.21.15.5:53218) 2024-04-25T10:03:03.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0faea00640 1 -- 172.21.15.5:0/2432900041 learned_addr learned my addr 172.21.15.5:0/2432900041 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0faf400640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 0x7f0fb01685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224142114 0 0) 0x7f0fb00b7ef0 con 0x7f0fb00d0780 2024-04-25T10:03:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f90003520 con 0x7f0fb00d0780 2024-04-25T10:03:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362190827 0 0) 0x7f0f90003520 con 0x7f0fb00d0780 2024-04-25T10:03:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 msgr2=0x7f0fb01685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.222+0000 7f0fae000640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 0x7f0fb01685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 >> v1:172.21.15.80:6789/0 conn(0x7f0fb00d2c40 legacy=0x7f0fb00b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00b84b0 con 0x7f0fb00d0780 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0fa0001670 con 0x7f0fb00d0780 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 >> v1:172.21.15.5:6789/0 conn(0x7f0fb00d0780 legacy=0x7f0fb00d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f900039e0 0x7f0f90003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 0x7f0f90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f90004d40 0x7f0f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b9840 con 0x7f0f90004d40 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9000bad0 con 0x7f0f900039e0 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9000bcd0 con 0x7f0fb00b78c0 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0faf400640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f90004d40 0x7f0f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0faea00640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f900039e0 0x7f0f90003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.223+0000 7f0fa7e00640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 0x7f0f90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.224+0000 7f0faf400640 1 -- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 msgr2=0x7f0f90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.224+0000 7f0faf400640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fb00b78c0 0x7f0f90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.224+0000 7f0faf400640 1 -- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f900039e0 msgr2=0x7f0f90003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.224+0000 7f0faf400640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f900039e0 0x7f0f90003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.224+0000 7f0faf400640 1 -- 172.21.15.5:0/2432900041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f90003520 con 0x7f0f90004d40 2024-04-25T10:03:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0faf400640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f90004d40 0x7f0f900051c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009f90 tx=0x7f0f90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deefa28d119a7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8009c10 con 0x7f0f90004d40 2024-04-25T10:03:03.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa8036070 con 0x7f0f90004d40 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0fae000640 1 -- 172.21.15.5:0/2432900041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8031850 con 0x7f0f90004d40 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0fb5800640 1 -- 172.21.15.5:0/2432900041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f90004d40 msgr2=0x7f0f900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0fb5800640 1 --2- 172.21.15.5:0/2432900041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f90004d40 0x7f0f900051c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009f90 tx=0x7f0f90005830 comp rx=0 tx=0).stop 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.225+0000 7f0faf400640 1 -- 172.21.15.5:0/2432900041 reap_dead start 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.226+0000 7f0fb5800640 1 -- 172.21.15.5:0/2432900041 shutdown_connections 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.226+0000 7f0fb5800640 1 -- 172.21.15.5:0/2432900041 >> 172.21.15.5:0/2432900041 conn(0x7f0fb00726b0 msgr2=0x7f0fb0072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.226+0000 7f0fb5800640 1 -- 172.21.15.5:0/2432900041 shutdown_connections 2024-04-25T10:03:03.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.226+0000 7f0fb5800640 1 -- 172.21.15.5:0/2432900041 wait complete. 2024-04-25T10:03:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.227+0000 7f0fb5800640 1 Processor -- start 2024-04-25T10:03:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.228+0000 7f0fb5800640 1 -- start start 2024-04-25T10:03:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.228+0000 7f0fb5800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f90004d40 0x7f0fb0156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.228+0000 7f0fb5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fb014cc50 0x7f0fb00ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0fb5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 0x7f0fb00d3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0fb5800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb01611b0 con 0x7f0fb00d3100 2024-04-25T10:03:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0fb5800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb00b8a70 con 0x7f0fb014cc50 2024-04-25T10:03:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0fb5800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0161330 con 0x7f0f90004d40 2024-04-25T10:03:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 0x7f0fb00d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 0x7f0fb00d3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60290/0 (socket says 172.21.15.5:60290) 2024-04-25T10:03:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 -- 172.21.15.5:0/78288674 learned_addr learned my addr 172.21.15.5:0/78288674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0fa7e00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fb014cc50 0x7f0fb00ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faea00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f90004d40 0x7f0fb0156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 -- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f90004d40 msgr2=0x7f0fb0156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f90004d40 0x7f0fb0156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 -- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fb014cc50 msgr2=0x7f0fb00ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fb014cc50 0x7f0fb00ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faf400640 1 -- 172.21.15.5:0/78288674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb014d0b0 con 0x7f0fb00d3100 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0fa7e00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fb014cc50 0x7f0fb00ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.229+0000 7f0faea00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f90004d40 0x7f0fb0156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.230+0000 7f0faf400640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 0x7f0fb00d3580 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8033040 tx=0x7f0fa8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.230+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8009c10 con 0x7f0fb00d3100 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.230+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa8003990 con 0x7f0fb00d3100 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.230+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8003d80 con 0x7f0fb00d3100 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.230+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb00cd220 con 0x7f0fb00d3100 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.230+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb00cbc80 con 0x7f0fb00d3100 2024-04-25T10:03:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.231+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb00685d0 con 0x7f0fb00d3100 2024-04-25T10:03:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.232+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fa8005d10 con 0x7f0fb00d3100 2024-04-25T10:03:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.234+0000 7f0fa6a00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f8c06b4d0 0x7f0f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.234+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f0fa80ab910 con 0x7f0fb00d3100 2024-04-25T10:03:03.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.234+0000 7f0faea00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f8c06b4d0 0x7f0f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.236+0000 7f0faea00640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f8c06b4d0 0x7f0f8c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001c00 tx=0x7f0fa0000f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.238+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fb014d0b0 con 0x7f0fb00d3100 2024-04-25T10:03:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.437+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0fb0068f60 con 0x7f0fb00d3100 2024-04-25T10:03:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.438+0000 7f0fa6a00640 1 -- 172.21.15.5:0/78288674 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0fa802e050 con 0x7f0fb00d3100 2024-04-25T10:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f8c06b4d0 msgr2=0x7f0f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f8c06b4d0 0x7f0f8c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001c00 tx=0x7f0fa0000f90 comp rx=0 tx=0).stop 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 msgr2=0x7f0fb00d3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 0x7f0fb00d3580 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8033040 tx=0x7f0fa8009750 comp rx=0 tx=0).stop 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 shutdown_connections 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fb00d3100 0x7f0fb00d3580 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f8c06b4d0 0x7f0f8c06d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fb014cc50 0x7f0fb00ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 --2- 172.21.15.5:0/78288674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f90004d40 0x7f0fb0156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 >> 172.21.15.5:0/78288674 conn(0x7f0fb00726b0 msgr2=0x7f0fb00ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 shutdown_connections 2024-04-25T10:03:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.441+0000 7f0fb5800640 1 -- 172.21.15.5:0/78288674 wait complete. 2024-04-25T10:03:03.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:03:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.689+0000 7fa71a000640 1 Processor -- start 2024-04-25T10:03:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.690+0000 7fa71a000640 1 -- start start 2024-04-25T10:03:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.691+0000 7fa71a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa714143d30 0x7fa71415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.691+0000 7fa71a000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa7140ceea0 0x7fa7140b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.691+0000 7fa71a000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7140b4880 con 0x7fa7140b3ae0 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.691+0000 7fa71a000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7140b45a0 con 0x7fa714143d30 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.691+0000 7fa71a000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71415e2c0 con 0x7fa7140ceea0 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.691+0000 7fa718c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa714143d30 0x7fa71415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa718c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa714143d30 0x7fa71415eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60304/0 (socket says 172.21.15.5:60304) 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa718c00640 1 -- 172.21.15.5:0/635887895 learned_addr learned my addr 172.21.15.5:0/635887895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa719600640 1 --2- 172.21.15.5:0/635887895 >> v2:172.21.15.80:3300/0 conn(0x7fa7140ceea0 0x7fa7140b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa718c00640 1 -- 172.21.15.5:0/635887895 >> v2:172.21.15.80:3300/0 conn(0x7fa7140ceea0 msgr2=0x7fa7140b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa718c00640 1 --2- 172.21.15.5:0/635887895 >> v2:172.21.15.80:3300/0 conn(0x7fa7140ceea0 0x7fa7140b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa718c00640 1 -- 172.21.15.5:0/635887895 >> v1:172.21.15.80:6789/0 conn(0x7fa7140b3ae0 legacy=0x7fa7140ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.692+0000 7fa718c00640 1 -- 172.21.15.5:0/635887895 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71415e5a0 con 0x7fa714143d30 2024-04-25T10:03:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.693+0000 7fa718c00640 1 --2- 172.21.15.5:0/635887895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714143d30 0x7fa71415eda0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa71406f4a0 tx=0x7fa70802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13834b71538f38fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.693+0000 7fa713400640 1 -- 172.21.15.5:0/635887895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080325a0 con 0x7fa714143d30 2024-04-25T10:03:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa713400640 1 -- 172.21.15.5:0/635887895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708038070 con 0x7fa714143d30 2024-04-25T10:03:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa713400640 1 -- 172.21.15.5:0/635887895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708032960 con 0x7fa714143d30 2024-04-25T10:03:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa71a000640 1 -- 172.21.15.5:0/635887895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714143d30 msgr2=0x7fa71415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa71a000640 1 --2- 172.21.15.5:0/635887895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714143d30 0x7fa71415eda0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa71406f4a0 tx=0x7fa70802c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa71a000640 1 -- 172.21.15.5:0/635887895 shutdown_connections 2024-04-25T10:03:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa71a000640 1 --2- 172.21.15.5:0/635887895 >> v2:172.21.15.80:3300/0 conn(0x7fa7140ceea0 0x7fa7140b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa71a000640 1 --2- 172.21.15.5:0/635887895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714143d30 0x7fa71415eda0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.694+0000 7fa71a000640 1 -- 172.21.15.5:0/635887895 >> 172.21.15.5:0/635887895 conn(0x7fa714066f20 msgr2=0x7fa71406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.695+0000 7fa71a000640 1 -- 172.21.15.5:0/635887895 shutdown_connections 2024-04-25T10:03:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.695+0000 7fa71a000640 1 -- 172.21.15.5:0/635887895 wait complete. 2024-04-25T10:03:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.697+0000 7fa71a000640 1 Processor -- start 2024-04-25T10:03:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.697+0000 7fa71a000640 1 -- start start 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.698+0000 7fa71a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 0x7fa7141ef600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.698+0000 7fa71a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7140ceea0 0x7fa7141f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.698+0000 7fa71a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa714143d30 0x7fa7140caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.698+0000 7fa71a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714144290 con 0x7fa7140b3ae0 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.698+0000 7fa71a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714144110 con 0x7fa714143d30 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.698+0000 7fa71a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71406e430 con 0x7fa7140ceea0 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 0x7fa7141ef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 0x7fa7141ef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60320/0 (socket says 172.21.15.5:60320) 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 -- 172.21.15.5:0/4205905300 learned_addr learned my addr 172.21.15.5:0/4205905300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa713e00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7140ceea0 0x7fa7141f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 -- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7140ceea0 msgr2=0x7fa7141f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7140ceea0 0x7fa7141f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa719600640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa714143d30 0x7fa7140caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 -- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa714143d30 msgr2=0x7fa7140caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa714143d30 0x7fa7140caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 -- 172.21.15.5:0/4205905300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7140c8660 con 0x7fa7140b3ae0 2024-04-25T10:03:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.699+0000 7fa718c00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 0x7fa7141ef600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa70802c920 tx=0x7fa7080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.700+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708033840 con 0x7fa7140b3ae0 2024-04-25T10:03:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.700+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708033a00 con 0x7fa7140b3ae0 2024-04-25T10:03:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.700+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708033df0 con 0x7fa7140b3ae0 2024-04-25T10:03:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.700+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7140c8940 con 0x7fa7140b3ae0 2024-04-25T10:03:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.700+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7140c8f00 con 0x7fa7140b3ae0 2024-04-25T10:03:03.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.701+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714068f60 con 0x7fa7140b3ae0 2024-04-25T10:03:03.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.702+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa70803c3f0 con 0x7fa7140b3ae0 2024-04-25T10:03:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.703+0000 7fa710c00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6f406b4d0 0x7fa6f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.703+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fa708030030 con 0x7fa7140b3ae0 2024-04-25T10:03:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.704+0000 7fa713e00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6f406b4d0 0x7fa6f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.705+0000 7fa713e00640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6f406b4d0 0x7fa6f406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7140ca8d0 tx=0x7fa700009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.708+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa708041070 con 0x7fa7140b3ae0 2024-04-25T10:03:03.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:03.981+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fa714158df0 con 0x7fa7140b3ae0 2024-04-25T10:03:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.827+0000 7fa710c00640 1 -- 172.21.15.5:0/4205905300 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v86) v1 ==== 145+0+0 (secure 0 0 0) 0x7fa70807c320 con 0x7fa7140b3ae0 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6f406b4d0 msgr2=0x7fa6f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6f406b4d0 0x7fa6f406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7140ca8d0 tx=0x7fa700009040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 msgr2=0x7fa7141ef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 0x7fa7141ef600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa70802c920 tx=0x7fa7080046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 shutdown_connections 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6f406b4d0 0x7fa6f406d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa714143d30 0x7fa7140caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7140ceea0 0x7fa7141f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 --2- 172.21.15.5:0/4205905300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7140b3ae0 0x7fa7141ef600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.831+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 >> 172.21.15.5:0/4205905300 conn(0x7fa714066f20 msgr2=0x7fa714159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.832+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 shutdown_connections 2024-04-25T10:03:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:04.832+0000 7fa71a000640 1 -- 172.21.15.5:0/4205905300 wait complete. 2024-04-25T10:03:04.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:03:05.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.067+0000 7f02de800640 1 Processor -- start 2024-04-25T10:03:05.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.067+0000 7f02de800640 1 -- start start 2024-04-25T10:03:05.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02de800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02d8071460 0x7f02d80b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02de800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d806d020 con 0x7f02d815f800 2024-04-25T10:03:05.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02de800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d806d2c0 con 0x7f02d8071000 2024-04-25T10:03:05.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02de800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d806ded0 con 0x7f02d8071460 2024-04-25T10:03:05.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02dd400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f02d815f800 0x7f02d8162ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53228/0 (socket says 172.21.15.5:53228) 2024-04-25T10:03:05.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02dd400640 1 -- 172.21.15.5:0/2661978944 learned_addr learned my addr 172.21.15.5:0/2661978944 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:05.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.069+0000 7f02dde00640 1 --2- 172.21.15.5:0/2661978944 >> v2:172.21.15.5:3300/0 conn(0x7f02d8071460 0x7f02d80b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.070+0000 7f02d7e00640 1 -- 172.21.15.5:0/2661978944 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545491078 0 0) 0x7f02d806d020 con 0x7f02d815f800 2024-04-25T10:03:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.070+0000 7f02d7e00640 1 -- 172.21.15.5:0/2661978944 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c0003520 con 0x7f02d815f800 2024-04-25T10:03:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.070+0000 7f02dde00640 1 -- 172.21.15.5:0/2661978944 >> v1:172.21.15.80:6789/0 conn(0x7f02d8071000 legacy=0x7f02d80b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.070+0000 7f02dde00640 1 -- 172.21.15.5:0/2661978944 >> v1:172.21.15.5:6789/0 conn(0x7f02d815f800 legacy=0x7f02d8162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.070+0000 7f02dde00640 1 -- 172.21.15.5:0/2661978944 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d806d5a0 con 0x7f02d8071460 2024-04-25T10:03:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.071+0000 7f02dde00640 1 --2- 172.21.15.5:0/2661978944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071460 0x7f02d80b3f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009d90 tx=0x7f02d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4be6f331201bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02d7e00640 1 -- 172.21.15.5:0/2661978944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0007740 con 0x7f02d8071460 2024-04-25T10:03:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02d7e00640 1 -- 172.21.15.5:0/2661978944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d00078e0 con 0x7f02d8071460 2024-04-25T10:03:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02d7e00640 1 -- 172.21.15.5:0/2661978944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0007c90 con 0x7f02d8071460 2024-04-25T10:03:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 -- 172.21.15.5:0/2661978944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071460 msgr2=0x7f02d80b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 --2- 172.21.15.5:0/2661978944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071460 0x7f02d80b3f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f02d0009d90 tx=0x7f02d002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 -- 172.21.15.5:0/2661978944 shutdown_connections 2024-04-25T10:03:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 --2- 172.21.15.5:0/2661978944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071460 0x7f02d80b3f50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 -- 172.21.15.5:0/2661978944 >> 172.21.15.5:0/2661978944 conn(0x7f02d8066f20 msgr2=0x7f02d806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 -- 172.21.15.5:0/2661978944 shutdown_connections 2024-04-25T10:03:05.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.072+0000 7f02de800640 1 -- 172.21.15.5:0/2661978944 wait complete. 2024-04-25T10:03:05.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.074+0000 7f02de800640 1 Processor -- start 2024-04-25T10:03:05.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.074+0000 7f02de800640 1 -- start start 2024-04-25T10:03:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.075+0000 7f02de800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 0x7f02d814a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.075+0000 7f02de800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02d8071460 0x7f02d815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.075+0000 7f02de800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d815f800 0x7f02d81583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.075+0000 7f02de800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d8071840 con 0x7f02d8071000 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.075+0000 7f02de800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d806db60 con 0x7f02d815f800 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.075+0000 7f02de800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d806e960 con 0x7f02d8071460 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 0x7f02d814a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 0x7f02d814a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60340/0 (socket says 172.21.15.5:60340) 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 -- 172.21.15.5:0/4108511107 learned_addr learned my addr 172.21.15.5:0/4108511107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 -- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02d8071460 msgr2=0x7f02d815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02d8071460 0x7f02d815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 -- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d815f800 msgr2=0x7f02d81583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d815f800 0x7f02d81583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 -- 172.21.15.5:0/4108511107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8157350 con 0x7f02d8071000 2024-04-25T10:03:05.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.076+0000 7f02dd400640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 0x7f02d814a520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f02c800c440 tx=0x7f02c800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.077+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8017070 con 0x7f02d8071000 2024-04-25T10:03:05.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.077+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c8005ce0 con 0x7f02d8071000 2024-04-25T10:03:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.077+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c80072b0 con 0x7f02d8071000 2024-04-25T10:03:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.077+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d8157630 con 0x7f02d8071000 2024-04-25T10:03:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.077+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8157bf0 con 0x7f02d8071000 2024-04-25T10:03:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.078+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d8068f60 con 0x7f02d8071000 2024-04-25T10:03:05.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.079+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02c801a070 con 0x7f02d8071000 2024-04-25T10:03:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.080+0000 7f02d5600640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02bc06b500 0x7f02bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.080+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f02c8089960 con 0x7f02d8071000 2024-04-25T10:03:05.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.081+0000 7f02dca00640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02bc06b500 0x7f02bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.082+0000 7f02dca00640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02bc06b500 0x7f02bc06d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02cc000f60 tx=0x7f02cc0079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.084+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02c8058400 con 0x7f02d8071000 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.284+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f02d80cc9d0 con 0x7f02d8071000 2024-04-25T10:03:05.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.291+0000 7f02d5600640 1 -- 172.21.15.5:0/4108511107 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02c8057d00 con 0x7f02d8071000 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02bc06b500 msgr2=0x7f02bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02bc06b500 0x7f02bc06d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f02cc000f60 tx=0x7f02cc0079b0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 msgr2=0x7f02d814a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 0x7f02d814a520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f02c800c440 tx=0x7f02c800c900 comp rx=0 tx=0).stop 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 shutdown_connections 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02bc06b500 0x7f02bc06d9c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02d815f800 0x7f02d81583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02d8071460 0x7f02d815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 --2- 172.21.15.5:0/4108511107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02d8071000 0x7f02d814a520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.294+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 >> 172.21.15.5:0/4108511107 conn(0x7f02d8066f20 msgr2=0x7f02d8071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.295+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 shutdown_connections 2024-04-25T10:03:05.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.295+0000 7f02de800640 1 -- 172.21.15.5:0/4108511107 wait complete. 2024-04-25T10:03:05.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:05.319 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:05.432 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:05.554 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:05.554 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:05.554 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:05.616 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:05.617 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:05.675 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:05.675 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:05.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.949+0000 7f342ca00640 1 Processor -- start 2024-04-25T10:03:05.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.949+0000 7f342ca00640 1 -- start start 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.951+0000 7f342ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3428161200 0x7f34280b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.951+0000 7f342ca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f34280d2380 0x7f34280d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.951+0000 7f342ca00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34280cf410 con 0x7f34280d0600 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.951+0000 7f342ca00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34280d27c0 con 0x7f34280d2380 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.951+0000 7f342ca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34280d0a70 con 0x7f3428161200 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f34280d2380 0x7f34280d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f34280d2380 0x7f34280d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60348/0 (socket says 172.21.15.5:60348) 2024-04-25T10:03:05.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 -- 172.21.15.5:0/4085177245 learned_addr learned my addr 172.21.15.5:0/4085177245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f3426800640 1 --2- 172.21.15.5:0/4085177245 >> v2:172.21.15.80:3300/0 conn(0x7f3428161200 0x7f34280b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 -- 172.21.15.5:0/4085177245 >> v2:172.21.15.80:3300/0 conn(0x7f3428161200 msgr2=0x7f34280b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 --2- 172.21.15.5:0/4085177245 >> v2:172.21.15.80:3300/0 conn(0x7f3428161200 0x7f34280b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 -- 172.21.15.5:0/4085177245 >> v1:172.21.15.80:6789/0 conn(0x7f34280d0600 legacy=0x7f34280ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.952+0000 7f341de00640 1 -- 172.21.15.5:0/4085177245 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280cf6f0 con 0x7f34280d2380 2024-04-25T10:03:05.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.953+0000 7f341de00640 1 --2- 172.21.15.5:0/4085177245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d2380 0x7f34280d00c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f342808bed0 tx=0x7f341002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b86403ea7db070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f3425e00640 1 -- 172.21.15.5:0/4085177245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410037070 con 0x7f34280d2380 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f3425e00640 1 -- 172.21.15.5:0/4085177245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410032440 con 0x7f34280d2380 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f3425e00640 1 -- 172.21.15.5:0/4085177245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410032810 con 0x7f34280d2380 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f342ca00640 1 -- 172.21.15.5:0/4085177245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d2380 msgr2=0x7f34280d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f342ca00640 1 --2- 172.21.15.5:0/4085177245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d2380 0x7f34280d00c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f342808bed0 tx=0x7f341002a270 comp rx=0 tx=0).stop 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f342ca00640 1 -- 172.21.15.5:0/4085177245 shutdown_connections 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f342ca00640 1 --2- 172.21.15.5:0/4085177245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d2380 0x7f34280d00c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f342ca00640 1 --2- 172.21.15.5:0/4085177245 >> v2:172.21.15.80:3300/0 conn(0x7f3428161200 0x7f34280b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.954+0000 7f342ca00640 1 -- 172.21.15.5:0/4085177245 >> 172.21.15.5:0/4085177245 conn(0x7f342808af80 msgr2=0x7f34280d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.955+0000 7f342ca00640 1 -- 172.21.15.5:0/4085177245 shutdown_connections 2024-04-25T10:03:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.955+0000 7f342ca00640 1 -- 172.21.15.5:0/4085177245 wait complete. 2024-04-25T10:03:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.956+0000 7f342ca00640 1 Processor -- start 2024-04-25T10:03:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.956+0000 7f342ca00640 1 -- start start 2024-04-25T10:03:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.957+0000 7f342ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 0x7f342815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.957+0000 7f342ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f34280d2380 0x7f34281f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.957+0000 7f342ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3428161200 0x7f34280c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.957+0000 7f342ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280d42b0 con 0x7f34280d0600 2024-04-25T10:03:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.957+0000 7f342ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280b28b0 con 0x7f3428161200 2024-04-25T10:03:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.957+0000 7f342ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34280b3f90 con 0x7f34280d2380 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 0x7f342815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 0x7f342815b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60356/0 (socket says 172.21.15.5:60356) 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 -- 172.21.15.5:0/2454157062 learned_addr learned my addr 172.21.15.5:0/2454157062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f341de00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f34280d2380 0x7f34281f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 -- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f34280d2380 msgr2=0x7f34281f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f34280d2380 0x7f34281f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 -- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3428161200 msgr2=0x7f34280c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3428161200 0x7f34280c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 -- 172.21.15.5:0/2454157062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34281f4630 con 0x7f34280d0600 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f341de00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f34280d2380 0x7f34281f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.958+0000 7f3426800640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 0x7f342815b550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f341800e8a0 tx=0x7f341800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.959+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418017070 con 0x7f34280d0600 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.959+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341800c940 con 0x7f34280d0600 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.959+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341800cd30 con 0x7f34280d0600 2024-04-25T10:03:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.959+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34280c7410 con 0x7f34280d0600 2024-04-25T10:03:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.959+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34280cb7e0 con 0x7f34280d0600 2024-04-25T10:03:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.960+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3428068f60 con 0x7f34280d0600 2024-04-25T10:03:05.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.961+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f341801a070 con 0x7f34280d0600 2024-04-25T10:03:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.962+0000 7f341f400640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f340406b500 0x7f340406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.962+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f341808a3b0 con 0x7f34280d0600 2024-04-25T10:03:05.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.963+0000 7f341de00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f340406b500 0x7f340406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.964+0000 7f341de00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f340406b500 0x7f340406d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3410034000 tx=0x7f341002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:05.967+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34281f4630 con 0x7f34280d0600 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.168+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34280cb590 con 0x7f34280d0600 2024-04-25T10:03:06.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.170+0000 7f341f400640 1 -- 172.21.15.5:0/2454157062 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f3418058f50 con 0x7f34280d0600 2024-04-25T10:03:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:06.175 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f340406b500 msgr2=0x7f340406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f340406b500 0x7f340406d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3410034000 tx=0x7f341002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 msgr2=0x7f342815b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 0x7f342815b550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f341800e8a0 tx=0x7f341800eca0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 shutdown_connections 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f340406b500 0x7f340406d9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3428161200 0x7f34280c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f34280d2380 0x7f34281f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 --2- 172.21.15.5:0/2454157062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34280d0600 0x7f342815b550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 >> 172.21.15.5:0/2454157062 conn(0x7f342808af80 msgr2=0x7f34280b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.173+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 shutdown_connections 2024-04-25T10:03:06.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.174+0000 7f342ca00640 1 -- 172.21.15.5:0/2454157062 wait complete. 2024-04-25T10:03:06.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:06.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.410+0000 7fb288e00640 1 Processor -- start 2024-04-25T10:03:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.410+0000 7fb288e00640 1 -- start start 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb288e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb284144fe0 0x7fb28406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb288e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb284071840 0x7fb28415c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb288e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2841453c0 con 0x7fb28406d130 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb288e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840a8560 con 0x7fb284144fe0 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb288e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28406d6a0 con 0x7fb284071840 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb281e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb284144fe0 0x7fb28406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb281400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb28406d130 0x7fb284071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53240/0 (socket says 172.21.15.5:53240) 2024-04-25T10:03:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.412+0000 7fb281400640 1 -- 172.21.15.5:0/2103935622 learned_addr learned my addr 172.21.15.5:0/2103935622 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.413+0000 7fb282800640 1 --2- 172.21.15.5:0/2103935622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb284071840 0x7fb28415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.413+0000 7fb281e00640 1 -- 172.21.15.5:0/2103935622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb284071840 msgr2=0x7fb28415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.413+0000 7fb281e00640 1 --2- 172.21.15.5:0/2103935622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb284071840 0x7fb28415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.413+0000 7fb281e00640 1 -- 172.21.15.5:0/2103935622 >> v1:172.21.15.5:6789/0 conn(0x7fb28406d130 legacy=0x7fb284071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.413+0000 7fb281e00640 1 -- 172.21.15.5:0/2103935622 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28406d980 con 0x7fb284144fe0 2024-04-25T10:03:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.413+0000 7fb280a00640 1 -- 172.21.15.5:0/2103935622 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892488500 0 0) 0x7fb2841453c0 con 0x7fb28406d130 2024-04-25T10:03:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.414+0000 7fb281e00640 1 --2- 172.21.15.5:0/2103935622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb28406e560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb2740097a0 tx=0x7fb27402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a565a7fad47c0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.414+0000 7fb280a00640 1 -- 172.21.15.5:0/2103935622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274037070 con 0x7fb284144fe0 2024-04-25T10:03:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.414+0000 7fb280a00640 1 -- 172.21.15.5:0/2103935622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274032440 con 0x7fb284144fe0 2024-04-25T10:03:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.414+0000 7fb280a00640 1 -- 172.21.15.5:0/2103935622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274032810 con 0x7fb284144fe0 2024-04-25T10:03:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 -- 172.21.15.5:0/2103935622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 msgr2=0x7fb28406e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 --2- 172.21.15.5:0/2103935622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb28406e560 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb2740097a0 tx=0x7fb27402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 -- 172.21.15.5:0/2103935622 shutdown_connections 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 --2- 172.21.15.5:0/2103935622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb284071840 0x7fb28415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 --2- 172.21.15.5:0/2103935622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb28406e560 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 -- 172.21.15.5:0/2103935622 >> 172.21.15.5:0/2103935622 conn(0x7fb28406ab60 msgr2=0x7fb28406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 -- 172.21.15.5:0/2103935622 shutdown_connections 2024-04-25T10:03:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.415+0000 7fb288e00640 1 -- 172.21.15.5:0/2103935622 wait complete. 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.417+0000 7fb288e00640 1 Processor -- start 2024-04-25T10:03:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.417+0000 7fb288e00640 1 -- start start 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb288e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb28406d130 0x7fb2840b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb288e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb284071840 0x7fb284141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb288e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb2841e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb288e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28415be50 con 0x7fb284144fe0 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb288e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28406df40 con 0x7fb284071840 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb288e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28406eaa0 con 0x7fb28406d130 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb282800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb2841e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb282800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb2841e58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60374/0 (socket says 172.21.15.5:60374) 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.418+0000 7fb282800640 1 -- 172.21.15.5:0/11350979 learned_addr learned my addr 172.21.15.5:0/11350979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb281e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb28406d130 0x7fb2840b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb281400640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb284071840 0x7fb284141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb282800640 1 -- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb28406d130 msgr2=0x7fb2840b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb282800640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb28406d130 0x7fb2840b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb282800640 1 -- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb284071840 msgr2=0x7fb284141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb282800640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb284071840 0x7fb284141f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb282800640 1 -- 172.21.15.5:0/11350979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840b3080 con 0x7fb284144fe0 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb281e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb28406d130 0x7fb2840b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb282800640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb2841e58b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb27c007e70 tx=0x7fb27c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.419+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c014620 con 0x7fb284144fe0 2024-04-25T10:03:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.420+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c019070 con 0x7fb284144fe0 2024-04-25T10:03:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.420+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c014a20 con 0x7fb284144fe0 2024-04-25T10:03:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.420+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840b3350 con 0x7fb284144fe0 2024-04-25T10:03:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.420+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2840b3910 con 0x7fb284144fe0 2024-04-25T10:03:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.421+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284068f60 con 0x7fb284144fe0 2024-04-25T10:03:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.422+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb27c014bc0 con 0x7fb284144fe0 2024-04-25T10:03:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.423+0000 7fb27a000640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb26006b4d0 0x7fb26006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.423+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fb27c08ae30 con 0x7fb284144fe0 2024-04-25T10:03:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.424+0000 7fb281e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb26006b4d0 0x7fb26006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.425+0000 7fb281e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb26006b4d0 0x7fb26006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb274034000 tx=0x7fb27402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.427+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb27c059930 con 0x7fb284144fe0 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.626+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb284071380 con 0x7fb284144fe0 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.627+0000 7fb27a000640 1 -- 172.21.15.5:0/11350979 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb27c059230 con 0x7fb284144fe0 2024-04-25T10:03:06.631 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb26006b4d0 msgr2=0x7fb26006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb26006b4d0 0x7fb26006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb274034000 tx=0x7fb27402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 msgr2=0x7fb2841e58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb2841e58b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb27c007e70 tx=0x7fb27c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 shutdown_connections 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb284144fe0 0x7fb2841e58b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb26006b4d0 0x7fb26006d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb284071840 0x7fb284141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 --2- 172.21.15.5:0/11350979 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb28406d130 0x7fb2840b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 >> 172.21.15.5:0/11350979 conn(0x7fb28406ab60 msgr2=0x7fb284143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 shutdown_connections 2024-04-25T10:03:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.630+0000 7fb288e00640 1 -- 172.21.15.5:0/11350979 wait complete. 2024-04-25T10:03:06.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.867+0000 7f71d2600640 1 Processor -- start 2024-04-25T10:03:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.867+0000 7f71d2600640 1 -- start start 2024-04-25T10:03:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.868+0000 7f71d2600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 0x7f71cc15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.868+0000 7f71d2600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71cc0ce830 0x7f71cc0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.868+0000 7f71d2600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0ced10 con 0x7f71cc0d3760 2024-04-25T10:03:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.868+0000 7f71d2600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc0ce380 con 0x7f71cc0ce830 2024-04-25T10:03:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.868+0000 7f71d2600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15f000 con 0x7f71cc15f1c0 2024-04-25T10:03:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.869+0000 7f71cb400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f71cc0d3760 0x7f71cc0d3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56128/0 (socket says 172.21.15.5:56128) 2024-04-25T10:03:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.869+0000 7f71cb400640 1 -- 172.21.15.5:0/4198190387 learned_addr learned my addr 172.21.15.5:0/4198190387 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.869+0000 7f71cbe00640 1 --2- 172.21.15.5:0/4198190387 >> v2:172.21.15.80:3300/0 conn(0x7f71cc0ce830 0x7f71cc0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.869+0000 7f71c2a00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 0x7f71cc15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.869+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178011982 0 0) 0x7f71cc0ced10 con 0x7f71cc0d3760 2024-04-25T10:03:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.869+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b0003520 con 0x7f71cc0d3760 2024-04-25T10:03:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1761988836 0 0) 0x7f71b0003520 con 0x7f71cc0d3760 2024-04-25T10:03:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 msgr2=0x7f71cc15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 0x7f71cc15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 >> v2:172.21.15.80:3300/0 conn(0x7f71cc0ce830 msgr2=0x7f71cc0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 --2- 172.21.15.5:0/4198190387 >> v2:172.21.15.80:3300/0 conn(0x7f71cc0ce830 0x7f71cc0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0d1b40 con 0x7f71cc0d3760 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f71bc0019f0 con 0x7f71cc0d3760 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 >> v1:172.21.15.5:6789/0 conn(0x7f71cc0d3760 legacy=0x7f71cc0d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71b00039b0 0x7f71b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b0004800 0x7f71b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 0x7f71b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15f000 con 0x7f71b00039b0 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b000b9c0 con 0x7f71b0004800 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b000bb40 con 0x7f71cc15f1c0 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.870+0000 7f71cb400640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71b00039b0 0x7f71b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71c2a00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b0004800 0x7f71b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71cb400640 1 -- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 msgr2=0x7f71b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71cb400640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc15f1c0 0x7f71b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71cb400640 1 -- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b0004800 msgr2=0x7f71b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71cb400640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b0004800 0x7f71b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71cb400640 1 -- 172.21.15.5:0/4198190387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0003520 con 0x7f71b00039b0 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.871+0000 7f71c2a00640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71b0004800 0x7f71b0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.872+0000 7f71cb400640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71b00039b0 0x7f71b0003dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00a7b0 tx=0x7f71b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc31d38fa5ad49fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.872+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00a3d0 con 0x7f71b00039b0 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.872+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc02f070 con 0x7f71b00039b0 2024-04-25T10:03:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.872+0000 7f71caa00640 1 -- 172.21.15.5:0/4198190387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc02bc10 con 0x7f71b00039b0 2024-04-25T10:03:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71d2600640 1 -- 172.21.15.5:0/4198190387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71b00039b0 msgr2=0x7f71b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71d2600640 1 --2- 172.21.15.5:0/4198190387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71b00039b0 0x7f71b0003dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00a7b0 tx=0x7f71b0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71cbe00640 1 -- 172.21.15.5:0/4198190387 reap_dead start 2024-04-25T10:03:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71d2600640 1 -- 172.21.15.5:0/4198190387 shutdown_connections 2024-04-25T10:03:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71d2600640 1 -- 172.21.15.5:0/4198190387 >> 172.21.15.5:0/4198190387 conn(0x7f71cc072680 msgr2=0x7f71cc072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71d2600640 1 -- 172.21.15.5:0/4198190387 shutdown_connections 2024-04-25T10:03:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.873+0000 7f71d2600640 1 -- 172.21.15.5:0/4198190387 wait complete. 2024-04-25T10:03:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.875+0000 7f71d2600640 1 Processor -- start 2024-04-25T10:03:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.875+0000 7f71d2600640 1 -- start start 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71d2600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71b00039b0 0x7f71cc157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71d2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 0x7f71cc156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71d2600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0c8140 0x7f71cc0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71d2600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15de10 con 0x7f71cc0c93d0 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71d2600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0d2190 con 0x7f71cc0c8140 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71d2600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15eac0 con 0x7f71b00039b0 2024-04-25T10:03:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 0x7f71cc156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71c2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 0x7f71cc156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37652/0 (socket says 172.21.15.5:37652) 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71c2a00640 1 -- 172.21.15.5:0/4043593429 learned_addr learned my addr 172.21.15.5:0/4043593429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.876+0000 7f71cb400640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71b00039b0 0x7f71cc157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71cbe00640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0c8140 0x7f71cc0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c2a00640 1 -- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71b00039b0 msgr2=0x7f71cc157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c2a00640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71b00039b0 0x7f71cc157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c2a00640 1 -- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0c8140 msgr2=0x7f71cc0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c2a00640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0c8140 0x7f71cc0c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c2a00640 1 -- 172.21.15.5:0/4043593429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc158f80 con 0x7f71cc0c93d0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c2a00640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 0x7f71cc156390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71b4010880 tx=0x7f71b4010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b400f040 con 0x7f71cc0c93d0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b400eb60 con 0x7f71cc0c93d0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b401c400 con 0x7f71cc0c93d0 2024-04-25T10:03:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc158400 con 0x7f71cc0c93d0 2024-04-25T10:03:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.877+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc1589c0 con 0x7f71cc0c93d0 2024-04-25T10:03:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.879+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc068f60 con 0x7f71cc0c93d0 2024-04-25T10:03:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.880+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71b401c5a0 con 0x7f71cc0c93d0 2024-04-25T10:03:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.881+0000 7f71c3e00640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71a806b4d0 0x7f71a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.881+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f71b408ac50 con 0x7f71cc0c93d0 2024-04-25T10:03:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.881+0000 7f71cb400640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71a806b4d0 0x7f71a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.883+0000 7f71cb400640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71a806b4d0 0x7f71a806d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71bc033000 tx=0x7f71bc007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:06.885+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71b4018030 con 0x7f71cc0c93d0 2024-04-25T10:03:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.090+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71cc0cf220 con 0x7f71cc0c93d0 2024-04-25T10:03:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.091+0000 7f71c3e00640 1 -- 172.21.15.5:0/4043593429 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f71b4059750 con 0x7f71cc0c93d0 2024-04-25T10:03:07.096 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71a806b4d0 msgr2=0x7f71a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71a806b4d0 0x7f71a806d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71bc033000 tx=0x7f71bc007000 comp rx=0 tx=0).stop 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 msgr2=0x7f71cc156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 0x7f71cc156390 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71b4010880 tx=0x7f71b4010d40 comp rx=0 tx=0).stop 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 shutdown_connections 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71a806b4d0 0x7f71a806d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0c8140 0x7f71cc0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc0c93d0 0x7f71cc156390 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 --2- 172.21.15.5:0/4043593429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71b00039b0 0x7f71cc157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 >> 172.21.15.5:0/4043593429 conn(0x7f71cc072680 msgr2=0x7f71cc1607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 shutdown_connections 2024-04-25T10:03:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.094+0000 7f71d2600640 1 -- 172.21.15.5:0/4043593429 wait complete. 2024-04-25T10:03:07.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:07.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.331+0000 7f0d47000640 1 Processor -- start 2024-04-25T10:03:07.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.332+0000 7f0d47000640 1 -- start start 2024-04-25T10:03:07.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d47000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d40161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d47000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d40161bc0 0x7f0d400b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d47000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d401606e0 con 0x7f0d400b48c0 2024-04-25T10:03:07.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d47000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d401609c0 con 0x7f0d40161bc0 2024-04-25T10:03:07.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d47000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d40160ca0 con 0x7f0d400d3f90 2024-04-25T10:03:07.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d46600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0d40161bc0 0x7f0d400b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d45c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0d400b48c0 0x7f0d4015fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56140/0 (socket says 172.21.15.5:56140) 2024-04-25T10:03:07.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d45c00640 1 -- 172.21.15.5:0/1748452714 learned_addr learned my addr 172.21.15.5:0/1748452714 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.334+0000 7f0d45200640 1 --2- 172.21.15.5:0/1748452714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d40161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.335+0000 7f0d46600640 1 -- 172.21.15.5:0/1748452714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 msgr2=0x7f0d40161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.335+0000 7f0d46600640 1 --2- 172.21.15.5:0/1748452714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d40161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.335+0000 7f0d46600640 1 -- 172.21.15.5:0/1748452714 >> v1:172.21.15.5:6789/0 conn(0x7f0d400b48c0 legacy=0x7f0d4015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.335+0000 7f0d46600640 1 -- 172.21.15.5:0/1748452714 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40160f80 con 0x7f0d40161bc0 2024-04-25T10:03:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.335+0000 7f0d3fe00640 1 -- 172.21.15.5:0/1748452714 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999791095 0 0) 0x7f0d401606e0 con 0x7f0d400b48c0 2024-04-25T10:03:07.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.336+0000 7f0d46600640 1 --2- 172.21.15.5:0/1748452714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d40161bc0 0x7f0d400b2b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d380097a0 tx=0x7f0d3802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd751acef09d554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.336+0000 7f0d3fe00640 1 -- 172.21.15.5:0/1748452714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38037070 con 0x7f0d40161bc0 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.336+0000 7f0d3fe00640 1 -- 172.21.15.5:0/1748452714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3802ae50 con 0x7f0d40161bc0 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.336+0000 7f0d3fe00640 1 -- 172.21.15.5:0/1748452714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38032660 con 0x7f0d40161bc0 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 -- 172.21.15.5:0/1748452714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d40161bc0 msgr2=0x7f0d400b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 --2- 172.21.15.5:0/1748452714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d40161bc0 0x7f0d400b2b70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d380097a0 tx=0x7f0d3802a100 comp rx=0 tx=0).stop 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 -- 172.21.15.5:0/1748452714 shutdown_connections 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 --2- 172.21.15.5:0/1748452714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d40161bc0 0x7f0d400b2b70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 --2- 172.21.15.5:0/1748452714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d40161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 -- 172.21.15.5:0/1748452714 >> 172.21.15.5:0/1748452714 conn(0x7f0d4008af80 msgr2=0x7f0d400d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 -- 172.21.15.5:0/1748452714 shutdown_connections 2024-04-25T10:03:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.337+0000 7f0d47000640 1 -- 172.21.15.5:0/1748452714 wait complete. 2024-04-25T10:03:07.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.339+0000 7f0d47000640 1 Processor -- start 2024-04-25T10:03:07.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.339+0000 7f0d47000640 1 -- start start 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d47000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 0x7f0d40155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d47000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d400cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d47000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d40161bc0 0x7f0d400b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d47000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400b30b0 con 0x7f0d400b48c0 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d47000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400b4ca0 con 0x7f0d40161bc0 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d47000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d400b3230 con 0x7f0d400d3f90 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d45c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 0x7f0d40155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d45c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 0x7f0d40155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37674/0 (socket says 172.21.15.5:37674) 2024-04-25T10:03:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.340+0000 7f0d45c00640 1 -- 172.21.15.5:0/4231982581 learned_addr learned my addr 172.21.15.5:0/4231982581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:07.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d45c00640 1 -- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 msgr2=0x7f0d400cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d45c00640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d400cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d45c00640 1 -- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d40161bc0 msgr2=0x7f0d400b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d45c00640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d40161bc0 0x7f0d400b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d45c00640 1 -- 172.21.15.5:0/4231982581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400c7f70 con 0x7f0d400b48c0 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d45c00640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 0x7f0d40155990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c890 tx=0x7f0d3400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34018070 con 0x7f0d400b48c0 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3400a640 con 0x7f0d400b48c0 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d400c8250 con 0x7f0d400b48c0 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.341+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400aa30 con 0x7f0d400b48c0 2024-04-25T10:03:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.342+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d400c8810 con 0x7f0d400b48c0 2024-04-25T10:03:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.343+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d40068f60 con 0x7f0d400b48c0 2024-04-25T10:03:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.344+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d3401b070 con 0x7f0d400b48c0 2024-04-25T10:03:07.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.345+0000 7f0d3d600640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d2006b500 0x7f0d2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.345+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f0d34089c10 con 0x7f0d400b48c0 2024-04-25T10:03:07.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.345+0000 7f0d45200640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d2006b500 0x7f0d2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.347+0000 7f0d45200640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d2006b500 0x7f0d2006d9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c006e20 tx=0x7f0d2c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.349+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d34058720 con 0x7f0d400b48c0 2024-04-25T10:03:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.554+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d400d07b0 con 0x7f0d400b48c0 2024-04-25T10:03:07.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.555+0000 7f0d3d600640 1 -- 172.21.15.5:0/4231982581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d34058020 con 0x7f0d400b48c0 2024-04-25T10:03:07.559 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.557+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d2006b500 msgr2=0x7f0d2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.557+0000 7f0d47000640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d2006b500 0x7f0d2006d9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c006e20 tx=0x7f0d2c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.557+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 msgr2=0x7f0d40155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.557+0000 7f0d47000640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 0x7f0d40155990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c890 tx=0x7f0d3400cd50 comp rx=0 tx=0).stop 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 shutdown_connections 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d2006b500 0x7f0d2006d9c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d40161bc0 0x7f0d400b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d400d3f90 0x7f0d400cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 --2- 172.21.15.5:0/4231982581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d400b48c0 0x7f0d40155990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 >> 172.21.15.5:0/4231982581 conn(0x7f0d4008af80 msgr2=0x7f0d4006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 shutdown_connections 2024-04-25T10:03:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.558+0000 7f0d47000640 1 -- 172.21.15.5:0/4231982581 wait complete. 2024-04-25T10:03:07.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:07.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.791+0000 7fd8a1400640 1 Processor -- start 2024-04-25T10:03:07.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.791+0000 7fd8a1400640 1 -- start start 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.793+0000 7fd8a1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c1448f0 0x7fd89c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.793+0000 7fd8a1400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd89c15bc30 0x7fd89c15f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.793+0000 7fd8a1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd89c15fe30 0x7fd89c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.793+0000 7fd8a1400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c15f460 con 0x7fd89c15fe30 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.793+0000 7fd8a1400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c144cd0 con 0x7fd89c15bc30 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.793+0000 7fd8a1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c15c440 con 0x7fd89c1448f0 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd89c15fe30 0x7fd89c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd89c15fe30 0x7fd89c15ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37676/0 (socket says 172.21.15.5:37676) 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 -- 172.21.15.5:0/673851361 learned_addr learned my addr 172.21.15.5:0/673851361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd89be00640 1 --2- 172.21.15.5:0/673851361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c1448f0 0x7fd89c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd89b400640 1 --2- 172.21.15.5:0/673851361 >> v2:172.21.15.80:3300/0 conn(0x7fd89c15bc30 0x7fd89c15f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 -- 172.21.15.5:0/673851361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c1448f0 msgr2=0x7fd89c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 --2- 172.21.15.5:0/673851361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c1448f0 0x7fd89c0cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 -- 172.21.15.5:0/673851361 >> v2:172.21.15.80:3300/0 conn(0x7fd89c15bc30 msgr2=0x7fd89c15f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 --2- 172.21.15.5:0/673851361 >> v2:172.21.15.80:3300/0 conn(0x7fd89c15bc30 0x7fd89c15f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd8a0a00640 1 -- 172.21.15.5:0/673851361 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c15c720 con 0x7fd89c15fe30 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.794+0000 7fd89b400640 1 --2- 172.21.15.5:0/673851361 >> v2:172.21.15.80:3300/0 conn(0x7fd89c15bc30 0x7fd89c15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.795+0000 7fd8a0a00640 1 --2- 172.21.15.5:0/673851361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15fe30 0x7fd89c15ce20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0b4890 tx=0x7fd89402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4440924cc588bc38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd89aa00640 1 -- 172.21.15.5:0/673851361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894036070 con 0x7fd89c15fe30 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd89aa00640 1 -- 172.21.15.5:0/673851361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd894031600 con 0x7fd89c15fe30 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd89aa00640 1 -- 172.21.15.5:0/673851361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940319d0 con 0x7fd89c15fe30 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 -- 172.21.15.5:0/673851361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15fe30 msgr2=0x7fd89c15ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 --2- 172.21.15.5:0/673851361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15fe30 0x7fd89c15ce20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0b4890 tx=0x7fd89402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 -- 172.21.15.5:0/673851361 shutdown_connections 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 --2- 172.21.15.5:0/673851361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15fe30 0x7fd89c15ce20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 --2- 172.21.15.5:0/673851361 >> v2:172.21.15.80:3300/0 conn(0x7fd89c15bc30 0x7fd89c15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 --2- 172.21.15.5:0/673851361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c1448f0 0x7fd89c0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.796+0000 7fd8a1400640 1 -- 172.21.15.5:0/673851361 >> 172.21.15.5:0/673851361 conn(0x7fd89c066f20 msgr2=0x7fd89c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.797+0000 7fd8a1400640 1 -- 172.21.15.5:0/673851361 shutdown_connections 2024-04-25T10:03:07.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.797+0000 7fd8a1400640 1 -- 172.21.15.5:0/673851361 wait complete. 2024-04-25T10:03:07.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.798+0000 7fd8a1400640 1 Processor -- start 2024-04-25T10:03:07.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.799+0000 7fd8a1400640 1 -- start start 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.799+0000 7fd8a1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1448f0 0x7fd89c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd8a1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 0x7fd89c13e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd8a1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c15fe30 0x7fd89c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd8a1400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c15c090 con 0x7fd89c15bc30 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd8a1400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c15d360 con 0x7fd89c1448f0 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd8a1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c13f1b0 con 0x7fd89c15fe30 2024-04-25T10:03:07.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 0x7fd89c13e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 0x7fd89c13e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37680/0 (socket says 172.21.15.5:37680) 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 -- 172.21.15.5:0/3067268297 learned_addr learned my addr 172.21.15.5:0/3067268297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89be00640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1448f0 0x7fd89c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd8a0a00640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c15fe30 0x7fd89c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 -- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c15fe30 msgr2=0x7fd89c0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c15fe30 0x7fd89c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 -- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1448f0 msgr2=0x7fd89c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1448f0 0x7fd89c0cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.800+0000 7fd89b400640 1 -- 172.21.15.5:0/3067268297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c13f490 con 0x7fd89c15bc30 2024-04-25T10:03:07.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.801+0000 7fd89b400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 0x7fd89c13e8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c7f0 tx=0x7fd88800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.801+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888015070 con 0x7fd89c15bc30 2024-04-25T10:03:07.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.801+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd888007a30 con 0x7fd89c15bc30 2024-04-25T10:03:07.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.801+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888007e00 con 0x7fd89c15bc30 2024-04-25T10:03:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.801+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c13f770 con 0x7fd89c15bc30 2024-04-25T10:03:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.801+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c13edf0 con 0x7fd89c15bc30 2024-04-25T10:03:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.802+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c068f60 con 0x7fd89c15bc30 2024-04-25T10:03:07.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.803+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd888018070 con 0x7fd89c15bc30 2024-04-25T10:03:07.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.804+0000 7fd893e00640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd87c06b500 0x7fd87c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.805+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fd88808a530 con 0x7fd89c15bc30 2024-04-25T10:03:07.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.805+0000 7fd89be00640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd87c06b500 0x7fd87c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.806+0000 7fd89be00640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd87c06b500 0x7fd87c06d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd88c006fd0 tx=0x7fd88c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:07.809+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd888059030 con 0x7fd89c15bc30 2024-04-25T10:03:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.010+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd89c072180 con 0x7fd89c15bc30 2024-04-25T10:03:08.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.011+0000 7fd893e00640 1 -- 172.21.15.5:0/3067268297 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd888058930 con 0x7fd89c15bc30 2024-04-25T10:03:08.016 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.014+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd87c06b500 msgr2=0x7fd87c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.014+0000 7fd8a1400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd87c06b500 0x7fd87c06d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd88c006fd0 tx=0x7fd88c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.014+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 msgr2=0x7fd89c13e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.014+0000 7fd8a1400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 0x7fd89c13e8b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd88800c7f0 tx=0x7fd88800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 shutdown_connections 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd89c15fe30 0x7fd89c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd89c15bc30 0x7fd89c13e8b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd87c06b500 0x7fd87c06d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 --2- 172.21.15.5:0/3067268297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c1448f0 0x7fd89c0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 >> 172.21.15.5:0/3067268297 conn(0x7fd89c066f20 msgr2=0x7fd89c15cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 shutdown_connections 2024-04-25T10:03:08.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.015+0000 7fd8a1400640 1 -- 172.21.15.5:0/3067268297 wait complete. 2024-04-25T10:03:08.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.246+0000 7f8a97200640 1 Processor -- start 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.246+0000 7f8a97200640 1 -- start start 2024-04-25T10:03:08.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.247+0000 7f8a97200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 0x7f8a9815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.247+0000 7f8a97200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a98071d30 con 0x7f8a98141da0 2024-04-25T10:03:08.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.247+0000 7f8a97200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9806d620 con 0x7f8a980cd300 2024-04-25T10:03:08.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.247+0000 7f8a97200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98142180 con 0x7f8a9815ce00 2024-04-25T10:03:08.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.248+0000 7f8a95e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8a98141da0 0x7f8a9806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56154/0 (socket says 172.21.15.5:56154) 2024-04-25T10:03:08.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.248+0000 7f8a95e00640 1 -- 172.21.15.5:0/338241390 learned_addr learned my addr 172.21.15.5:0/338241390 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:08.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.248+0000 7f8a95400640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 0x7f8a9815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.248+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622964253 0 0) 0x7f8a98071d30 con 0x7f8a98141da0 2024-04-25T10:03:08.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.248+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a7c003520 con 0x7f8a98141da0 2024-04-25T10:03:08.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3713154174 0 0) 0x7f8a7c003520 con 0x7f8a98141da0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 msgr2=0x7f8a9815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 0x7f8a9815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 >> v1:172.21.15.80:6789/0 conn(0x7f8a980cd300 legacy=0x7f8a98071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9806d900 con 0x7f8a98141da0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8a88001670 con 0x7f8a98141da0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 >> v1:172.21.15.5:6789/0 conn(0x7f8a98141da0 legacy=0x7f8a9806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 0x7f8a7c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7c003fb0 0x7f8a7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7c004d40 0x7f8a7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98142180 con 0x7f8a7c004d40 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c00bb00 con 0x7f8a7c003fb0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c00bd00 con 0x7f8a9815ce00 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.249+0000 7f8a96800640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7c004d40 0x7f8a7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a95400640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7c003fb0 0x7f8a7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a95e00640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 0x7f8a7c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a96800640 1 -- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 msgr2=0x7f8a7c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a96800640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a9815ce00 0x7f8a7c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a96800640 1 -- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7c003fb0 msgr2=0x7f8a7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a96800640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7c003fb0 0x7f8a7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.250+0000 7f8a96800640 1 -- 172.21.15.5:0/338241390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c003520 con 0x7f8a7c004d40 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.251+0000 7f8a96800640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7c004d40 0x7f8a7c0051c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a90001e50 tx=0x7f8a7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc74e31449ffefed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90001a70 con 0x7f8a7c004d40 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90003970 con 0x7f8a7c004d40 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a94a00640 1 -- 172.21.15.5:0/338241390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90003d60 con 0x7f8a7c004d40 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a97200640 1 -- 172.21.15.5:0/338241390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7c004d40 msgr2=0x7f8a7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a97200640 1 --2- 172.21.15.5:0/338241390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7c004d40 0x7f8a7c0051c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a90001e50 tx=0x7f8a7c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a96800640 1 -- 172.21.15.5:0/338241390 reap_dead start 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a97200640 1 -- 172.21.15.5:0/338241390 shutdown_connections 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a97200640 1 -- 172.21.15.5:0/338241390 >> 172.21.15.5:0/338241390 conn(0x7f8a98066f20 msgr2=0x7f8a9806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a97200640 1 -- 172.21.15.5:0/338241390 shutdown_connections 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.252+0000 7f8a97200640 1 -- 172.21.15.5:0/338241390 wait complete. 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.254+0000 7f8a97200640 1 Processor -- start 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.254+0000 7f8a97200640 1 -- start start 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.255+0000 7f8a97200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a7c004d40 0x7f8a98144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.255+0000 7f8a97200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a980b1990 0x7f8a98152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.255+0000 7f8a97200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 0x7f8a980b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.255+0000 7f8a97200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98071710 con 0x7f8a980b3630 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.255+0000 7f8a97200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9806e810 con 0x7f8a980b1990 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.255+0000 7f8a97200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98071890 con 0x7f8a7c004d40 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 0x7f8a980b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 0x7f8a980b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37700/0 (socket says 172.21.15.5:37700) 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 -- 172.21.15.5:0/1391089876 learned_addr learned my addr 172.21.15.5:0/1391089876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a95400640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a980b1990 0x7f8a98152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a95e00640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a7c004d40 0x7f8a98144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 -- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a7c004d40 msgr2=0x7f8a98144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a7c004d40 0x7f8a98144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 -- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a980b1990 msgr2=0x7f8a98152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a980b1990 0x7f8a98152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 -- 172.21.15.5:0/1391089876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98144da0 con 0x7f8a980b3630 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.256+0000 7f8a96800640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 0x7f8a980b3ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a90005040 tx=0x7f8a900014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.257+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90037040 con 0x7f8a980b3630 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.257+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90003e50 con 0x7f8a980b3630 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.257+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90001a70 con 0x7f8a980b3630 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.257+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a98145080 con 0x7f8a980b3630 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.257+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a980af0c0 con 0x7f8a980b3630 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.258+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a98068f60 con 0x7f8a980b3630 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.259+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a90042020 con 0x7f8a980b3630 2024-04-25T10:03:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.260+0000 7f8a8e000640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7806b550 0x7f8a7806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.261+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f8a900a9c20 con 0x7f8a980b3630 2024-04-25T10:03:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.261+0000 7f8a95e00640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7806b550 0x7f8a7806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.262+0000 7f8a95e00640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7806b550 0x7f8a7806da10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a88001bf0 tx=0x7f8a88000f10 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.265+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a90078710 con 0x7f8a980b3630 2024-04-25T10:03:08.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.482+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a980ccae0 con 0x7f8a980b3630 2024-04-25T10:03:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.483+0000 7f8a8e000640 1 -- 172.21.15.5:0/1391089876 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a90078010 con 0x7f8a980b3630 2024-04-25T10:03:08.488 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:03:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7806b550 msgr2=0x7f8a7806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7806b550 0x7f8a7806da10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8a88001bf0 tx=0x7f8a88000f10 comp rx=0 tx=0).stop 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 msgr2=0x7f8a980b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 0x7f8a980b3ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a90005040 tx=0x7f8a900014a0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 shutdown_connections 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a980b3630 0x7f8a980b3ae0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7806b550 0x7f8a7806da10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a980b1990 0x7f8a98152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 --2- 172.21.15.5:0/1391089876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a7c004d40 0x7f8a98144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 >> 172.21.15.5:0/1391089876 conn(0x7f8a98066f20 msgr2=0x7f8a98073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 shutdown_connections 2024-04-25T10:03:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:08.486+0000 7f8a97200640 1 -- 172.21.15.5:0/1391089876 wait complete. 2024-04-25T10:03:08.510 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:08.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:03:08.602 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:03:08.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.603 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:08.653 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:03:08.654 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.711 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:03:08.711 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.762 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:03:08.762 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.022+0000 7f0351a00640 1 Processor -- start 2024-04-25T10:03:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.022+0000 7f0351a00640 1 -- start start 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0351a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f034c15ce00 0x7f034c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0351a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f034c0cd300 0x7f034c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0351a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c071d30 con 0x7f034c141da0 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0351a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c0cdcd0 con 0x7f034c15ce00 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0351a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c142180 con 0x7f034c0cd300 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0343e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f034c15ce00 0x7f034c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0343e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f034c15ce00 0x7f034c15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37704/0 (socket says 172.21.15.5:37704) 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.024+0000 7f0343e00640 1 -- 172.21.15.5:0/2273489462 learned_addr learned my addr 172.21.15.5:0/2273489462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.025+0000 7f0351000640 1 --2- 172.21.15.5:0/2273489462 >> v2:172.21.15.80:3300/0 conn(0x7f034c0cd300 0x7f034c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.025+0000 7f0343e00640 1 -- 172.21.15.5:0/2273489462 >> v2:172.21.15.80:3300/0 conn(0x7f034c0cd300 msgr2=0x7f034c0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.025+0000 7f0343e00640 1 --2- 172.21.15.5:0/2273489462 >> v2:172.21.15.80:3300/0 conn(0x7f034c0cd300 0x7f034c0cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.025+0000 7f0343e00640 1 -- 172.21.15.5:0/2273489462 >> v1:172.21.15.5:6789/0 conn(0x7f034c141da0 legacy=0x7f034c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.025+0000 7f0343e00640 1 -- 172.21.15.5:0/2273489462 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c06d620 con 0x7f034c15ce00 2024-04-25T10:03:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.026+0000 7f0343e00640 1 --2- 172.21.15.5:0/2273489462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c15ce00 0x7f034c15d280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f034c15f220 tx=0x7f033402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10ee61e6e133530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f034b400640 1 -- 172.21.15.5:0/2273489462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334037070 con 0x7f034c15ce00 2024-04-25T10:03:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f034b400640 1 -- 172.21.15.5:0/2273489462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0334032440 con 0x7f034c15ce00 2024-04-25T10:03:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f034b400640 1 -- 172.21.15.5:0/2273489462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0334032810 con 0x7f034c15ce00 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f0351a00640 1 -- 172.21.15.5:0/2273489462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c15ce00 msgr2=0x7f034c15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f0351a00640 1 --2- 172.21.15.5:0/2273489462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c15ce00 0x7f034c15d280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f034c15f220 tx=0x7f033402a270 comp rx=0 tx=0).stop 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f0351a00640 1 -- 172.21.15.5:0/2273489462 shutdown_connections 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f0351a00640 1 --2- 172.21.15.5:0/2273489462 >> v2:172.21.15.80:3300/0 conn(0x7f034c0cd300 0x7f034c0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f0351a00640 1 --2- 172.21.15.5:0/2273489462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c15ce00 0x7f034c15d280 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.027+0000 7f0351a00640 1 -- 172.21.15.5:0/2273489462 >> 172.21.15.5:0/2273489462 conn(0x7f034c066f20 msgr2=0x7f034c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.028+0000 7f0351a00640 1 -- 172.21.15.5:0/2273489462 shutdown_connections 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.028+0000 7f0351a00640 1 -- 172.21.15.5:0/2273489462 wait complete. 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.029+0000 7f0351a00640 1 Processor -- start 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.029+0000 7f0351a00640 1 -- start start 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.030+0000 7f0351a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 0x7f034c13aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.030+0000 7f0351a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f034c141da0 0x7f034c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.030+0000 7f0351a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f034c15ce00 0x7f034c13d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.030+0000 7f0351a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c06e810 con 0x7f034c0cd300 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.030+0000 7f0351a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c06dbe0 con 0x7f034c15ce00 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.030+0000 7f0351a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c071000 con 0x7f034c141da0 2024-04-25T10:03:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 0x7f034c13aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 0x7f034c13aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37718/0 (socket says 172.21.15.5:37718) 2024-04-25T10:03:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 -- 172.21.15.5:0/1901531339 learned_addr learned my addr 172.21.15.5:0/1901531339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f0343e00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f034c141da0 0x7f034c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f0351000640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f034c15ce00 0x7f034c13d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 -- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f034c141da0 msgr2=0x7f034c1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f034c141da0 0x7f034c1561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 -- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f034c15ce00 msgr2=0x7f034c13d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f034c15ce00 0x7f034c13d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.031+0000 7f034be00640 1 -- 172.21.15.5:0/1901531339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c1424a0 con 0x7f034c0cd300 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.032+0000 7f034be00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 0x7f034c13aca0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c400 tx=0x7f033c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.032+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0052d0 con 0x7f034c0cd300 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.032+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c005490 con 0x7f034c0cd300 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.032+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c007260 con 0x7f034c0cd300 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.032+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c142780 con 0x7f034c0cd300 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.032+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c142d40 con 0x7f034c0cd300 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.033+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c068f60 con 0x7f034c0cd300 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.034+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f033c007400 con 0x7f034c0cd300 2024-04-25T10:03:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.035+0000 7f0348c00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f032806b4d0 0x7f032806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.035+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f033c089ad0 con 0x7f034c0cd300 2024-04-25T10:03:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.036+0000 7f0343e00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f032806b4d0 0x7f032806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.037+0000 7f0343e00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f032806b4d0 0x7f032806d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0334034000 tx=0x7f033402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.040+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f033c0585d0 con 0x7f034c0cd300 2024-04-25T10:03:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.246+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f034c0ccae0 con 0x7f034c0cd300 2024-04-25T10:03:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.248+0000 7f0348c00640 1 -- 172.21.15.5:0/1901531339 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f033c057ed0 con 0x7f034c0cd300 2024-04-25T10:03:09.253 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:09.253 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f032806b4d0 msgr2=0x7f032806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f032806b4d0 0x7f032806d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0334034000 tx=0x7f033402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 msgr2=0x7f034c13aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 0x7f034c13aca0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f033c00c400 tx=0x7f033c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 shutdown_connections 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f032806b4d0 0x7f032806d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f034c15ce00 0x7f034c13d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f034c141da0 0x7f034c1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 --2- 172.21.15.5:0/1901531339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c0cd300 0x7f034c13aca0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 >> 172.21.15.5:0/1901531339 conn(0x7f034c066f20 msgr2=0x7f034c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 shutdown_connections 2024-04-25T10:03:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.251+0000 7f0351a00640 1 -- 172.21.15.5:0/1901531339 wait complete. 2024-04-25T10:03:09.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:09.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:09.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.492+0000 7fe1da200640 1 Processor -- start 2024-04-25T10:03:09.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.492+0000 7fe1da200640 1 -- start start 2024-04-25T10:03:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1da200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1d415ec80 0x7fe1d40b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1da200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1d40d4000 0x7fe1d4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1da200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d4161240 0x7fe1d41616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1da200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d415ffa0 con 0x7fe1d40d4000 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1da200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d4160240 con 0x7fe1d415ec80 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1da200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4160520 con 0x7fe1d4161240 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1cbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1d40d4000 0x7fe1d4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1cbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1d40d4000 0x7fe1d4160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37734/0 (socket says 172.21.15.5:37734) 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1cbe00640 1 -- 172.21.15.5:0/2143815040 learned_addr learned my addr 172.21.15.5:0/2143815040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:09.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1d8e00640 1 --2- 172.21.15.5:0/2143815040 >> v2:172.21.15.80:3300/0 conn(0x7fe1d415ec80 0x7fe1d40b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.494+0000 7fe1d9800640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d4161240 0x7fe1d41616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1cbe00640 1 -- 172.21.15.5:0/2143815040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d4161240 msgr2=0x7fe1d41616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1cbe00640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d4161240 0x7fe1d41616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1cbe00640 1 -- 172.21.15.5:0/2143815040 >> v2:172.21.15.80:3300/0 conn(0x7fe1d415ec80 msgr2=0x7fe1d40b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1cbe00640 1 --2- 172.21.15.5:0/2143815040 >> v2:172.21.15.80:3300/0 conn(0x7fe1d415ec80 0x7fe1d40b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1cbe00640 1 -- 172.21.15.5:0/2143815040 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4160800 con 0x7fe1d40d4000 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1d9800640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d4161240 0x7fe1d41616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.495+0000 7fe1d8e00640 1 --2- 172.21.15.5:0/2143815040 >> v2:172.21.15.80:3300/0 conn(0x7fe1d415ec80 0x7fe1d40b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.496+0000 7fe1cbe00640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d40d4000 0x7fe1d4160d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1d408b840 tx=0x7fe1c402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed44f5eae019de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.496+0000 7fe1d3e00640 1 -- 172.21.15.5:0/2143815040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4004030 con 0x7fe1d40d4000 2024-04-25T10:03:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.496+0000 7fe1d3e00640 1 -- 172.21.15.5:0/2143815040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c40041d0 con 0x7fe1d40d4000 2024-04-25T10:03:09.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.496+0000 7fe1d3e00640 1 -- 172.21.15.5:0/2143815040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4003610 con 0x7fe1d40d4000 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 -- 172.21.15.5:0/2143815040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d40d4000 msgr2=0x7fe1d4160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d40d4000 0x7fe1d4160d00 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1d408b840 tx=0x7fe1c402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 -- 172.21.15.5:0/2143815040 shutdown_connections 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d4161240 0x7fe1d41616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 --2- 172.21.15.5:0/2143815040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d40d4000 0x7fe1d4160d00 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 --2- 172.21.15.5:0/2143815040 >> v2:172.21.15.80:3300/0 conn(0x7fe1d415ec80 0x7fe1d40b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 -- 172.21.15.5:0/2143815040 >> 172.21.15.5:0/2143815040 conn(0x7fe1d408af80 msgr2=0x7fe1d40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 -- 172.21.15.5:0/2143815040 shutdown_connections 2024-04-25T10:03:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.497+0000 7fe1da200640 1 -- 172.21.15.5:0/2143815040 wait complete. 2024-04-25T10:03:09.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.499+0000 7fe1da200640 1 Processor -- start 2024-04-25T10:03:09.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.499+0000 7fe1da200640 1 -- start start 2024-04-25T10:03:09.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1da200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d40d4000 0x7fe1d4148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1da200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d415ec80 0x7fe1d40b8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1da200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 0x7fe1d4145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1da200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40d4490 con 0x7fe1d4161240 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1da200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40b4c50 con 0x7fe1d40d4000 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1da200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4161c30 con 0x7fe1d415ec80 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.500+0000 7fe1d9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 0x7fe1d4145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 0x7fe1d4145470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37746/0 (socket says 172.21.15.5:37746) 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 -- 172.21.15.5:0/2830221484 learned_addr learned my addr 172.21.15.5:0/2830221484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d8e00640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d40d4000 0x7fe1d4148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1cbe00640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d415ec80 0x7fe1d40b8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 -- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d415ec80 msgr2=0x7fe1d40b8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d415ec80 0x7fe1d40b8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 -- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d40d4000 msgr2=0x7fe1d4148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d40d4000 0x7fe1d4148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 -- 172.21.15.5:0/2830221484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4156a30 con 0x7fe1d4161240 2024-04-25T10:03:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.501+0000 7fe1d9800640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 0x7fe1d4145470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00cdb0 tx=0x7fe1cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.502+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00f680 con 0x7fe1d4161240 2024-04-25T10:03:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.502+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc00f840 con 0x7fe1d4161240 2024-04-25T10:03:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.502+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00fc30 con 0x7fe1d4161240 2024-04-25T10:03:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.502+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d4156d10 con 0x7fe1d4161240 2024-04-25T10:03:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.502+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d41572d0 con 0x7fe1d4161240 2024-04-25T10:03:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.503+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d4068f60 con 0x7fe1d4161240 2024-04-25T10:03:09.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.504+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1cc00e3e0 con 0x7fe1d4161240 2024-04-25T10:03:09.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.505+0000 7fe1d1600640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1b006b4d0 0x7fe1b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.505+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fe1cc08b340 con 0x7fe1d4161240 2024-04-25T10:03:09.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.506+0000 7fe1d8e00640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1b006b4d0 0x7fe1b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.507+0000 7fe1d8e00640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1b006b4d0 0x7fe1b006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc006ab0 tx=0x7fe1bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.509+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1cc014030 con 0x7fe1d4161240 2024-04-25T10:03:09.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.707+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1d40d0e50 con 0x7fe1d4161240 2024-04-25T10:03:09.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.708+0000 7fe1d1600640 1 -- 172.21.15.5:0/2830221484 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1cc059e40 con 0x7fe1d4161240 2024-04-25T10:03:09.713 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.710+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1b006b4d0 msgr2=0x7fe1b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.710+0000 7fe1da200640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1b006b4d0 0x7fe1b006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc006ab0 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.710+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 msgr2=0x7fe1d4145470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.710+0000 7fe1da200640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 0x7fe1d4145470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00cdb0 tx=0x7fe1cc00a650 comp rx=0 tx=0).stop 2024-04-25T10:03:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 shutdown_connections 2024-04-25T10:03:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d4161240 0x7fe1d4145470 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1d415ec80 0x7fe1d40b8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1b006b4d0 0x7fe1b006d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 --2- 172.21.15.5:0/2830221484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d40d4000 0x7fe1d4148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 >> 172.21.15.5:0/2830221484 conn(0x7fe1d408af80 msgr2=0x7fe1d40ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 shutdown_connections 2024-04-25T10:03:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.711+0000 7fe1da200640 1 -- 172.21.15.5:0/2830221484 wait complete. 2024-04-25T10:03:09.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:09.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.950+0000 7ff769600640 1 Processor -- start 2024-04-25T10:03:09.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.950+0000 7ff769600640 1 -- start start 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff769600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff76406b270 0x7ff76406fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff769600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff7640b2cd0 0x7ff76415d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff769600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76415e150 con 0x7ff764162240 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff769600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7640b2aa0 con 0x7ff7640b2cd0 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff769600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76406ff60 con 0x7ff76406b270 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff763400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff7640b2cd0 0x7ff76415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff763400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff7640b2cd0 0x7ff76415d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37752/0 (socket says 172.21.15.5:37752) 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff763400640 1 -- 172.21.15.5:0/391230366 learned_addr learned my addr 172.21.15.5:0/391230366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.952+0000 7ff762000640 1 --2- 172.21.15.5:0/391230366 >> v2:172.21.15.80:3300/0 conn(0x7ff76406b270 0x7ff76406fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.953+0000 7ff763400640 1 -- 172.21.15.5:0/391230366 >> v2:172.21.15.80:3300/0 conn(0x7ff76406b270 msgr2=0x7ff76406fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.953+0000 7ff763400640 1 --2- 172.21.15.5:0/391230366 >> v2:172.21.15.80:3300/0 conn(0x7ff76406b270 0x7ff76406fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.953+0000 7ff763400640 1 -- 172.21.15.5:0/391230366 >> v1:172.21.15.80:6789/0 conn(0x7ff764162240 legacy=0x7ff76406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.953+0000 7ff763400640 1 -- 172.21.15.5:0/391230366 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff764161630 con 0x7ff7640b2cd0 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.954+0000 7ff763400640 1 --2- 172.21.15.5:0/391230366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7640b2cd0 0x7ff76415d240 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff7640d16c0 tx=0x7ff75c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b26d6d7b946f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.954+0000 7ff761600640 1 -- 172.21.15.5:0/391230366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c037070 con 0x7ff7640b2cd0 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.954+0000 7ff761600640 1 -- 172.21.15.5:0/391230366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c032440 con 0x7ff7640b2cd0 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.954+0000 7ff761600640 1 -- 172.21.15.5:0/391230366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c032810 con 0x7ff7640b2cd0 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 -- 172.21.15.5:0/391230366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7640b2cd0 msgr2=0x7ff76415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 --2- 172.21.15.5:0/391230366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7640b2cd0 0x7ff76415d240 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff7640d16c0 tx=0x7ff75c02a230 comp rx=0 tx=0).stop 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 -- 172.21.15.5:0/391230366 shutdown_connections 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 --2- 172.21.15.5:0/391230366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7640b2cd0 0x7ff76415d240 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 --2- 172.21.15.5:0/391230366 >> v2:172.21.15.80:3300/0 conn(0x7ff76406b270 0x7ff76406fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 -- 172.21.15.5:0/391230366 >> 172.21.15.5:0/391230366 conn(0x7ff7640999f0 msgr2=0x7ff7640b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 -- 172.21.15.5:0/391230366 shutdown_connections 2024-04-25T10:03:09.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.955+0000 7ff769600640 1 -- 172.21.15.5:0/391230366 wait complete. 2024-04-25T10:03:09.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.957+0000 7ff769600640 1 Processor -- start 2024-04-25T10:03:09.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.957+0000 7ff769600640 1 -- start start 2024-04-25T10:03:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff769600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 0x7ff7640b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff769600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7640b2cd0 0x7ff7641480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff769600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff764162240 0x7ff7640c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff769600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76415e6d0 con 0x7ff76406b270 2024-04-25T10:03:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff769600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76415e3f0 con 0x7ff7640b2cd0 2024-04-25T10:03:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff769600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76415e9b0 con 0x7ff764162240 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff762a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 0x7ff7640b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff762a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 0x7ff7640b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37754/0 (socket says 172.21.15.5:37754) 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.958+0000 7ff762a00640 1 -- 172.21.15.5:0/2524230154 learned_addr learned my addr 172.21.15.5:0/2524230154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762000640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7640b2cd0 0x7ff7641480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762a00640 1 -- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff764162240 msgr2=0x7ff7640c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762a00640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff764162240 0x7ff7640c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762a00640 1 -- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7640b2cd0 msgr2=0x7ff7641480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762a00640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7640b2cd0 0x7ff7641480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762a00640 1 -- 172.21.15.5:0/2524230154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff764148630 con 0x7ff76406b270 2024-04-25T10:03:09.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff762a00640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 0x7ff7640b5fa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff75400ba50 tx=0x7ff75400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754019070 con 0x7ff76406b270 2024-04-25T10:03:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400cc20 con 0x7ff76406b270 2024-04-25T10:03:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754014680 con 0x7ff76406b270 2024-04-25T10:03:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.959+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7640b4f00 con 0x7ff76406b270 2024-04-25T10:03:09.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.960+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7640b54c0 con 0x7ff76406b270 2024-04-25T10:03:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.961+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff764068f60 con 0x7ff76406b270 2024-04-25T10:03:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.962+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff75401c070 con 0x7ff76406b270 2024-04-25T10:03:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.962+0000 7ff75aa00640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff74006b500 0x7ff74006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.963+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7ff75408a230 con 0x7ff76406b270 2024-04-25T10:03:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.963+0000 7ff762000640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff74006b500 0x7ff74006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.964+0000 7ff762000640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff74006b500 0x7ff74006d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff76415fc80 tx=0x7ff74c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:09.967+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff754058650 con 0x7ff76406b270 2024-04-25T10:03:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.171+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff76406b690 con 0x7ff76406b270 2024-04-25T10:03:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.172+0000 7ff75aa00640 1 -- 172.21.15.5:0/2524230154 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff75405c6f0 con 0x7ff76406b270 2024-04-25T10:03:10.177 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.174+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff74006b500 msgr2=0x7ff74006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.174+0000 7ff769600640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff74006b500 0x7ff74006d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff76415fc80 tx=0x7ff74c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 msgr2=0x7ff7640b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 0x7ff7640b5fa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff75400ba50 tx=0x7ff75400bf10 comp rx=0 tx=0).stop 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 shutdown_connections 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff764162240 0x7ff7640c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff74006b500 0x7ff74006d9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7640b2cd0 0x7ff7641480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 --2- 172.21.15.5:0/2524230154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76406b270 0x7ff7640b5fa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 >> 172.21.15.5:0/2524230154 conn(0x7ff7640999f0 msgr2=0x7ff7640b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 shutdown_connections 2024-04-25T10:03:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.175+0000 7ff769600640 1 -- 172.21.15.5:0/2524230154 wait complete. 2024-04-25T10:03:10.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.401+0000 7f6055800640 1 Processor -- start 2024-04-25T10:03:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.401+0000 7f6055800640 1 -- start start 2024-04-25T10:03:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.403+0000 7f6055800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f605006bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.403+0000 7f6055800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60501622b0 con 0x7f60500c4b50 2024-04-25T10:03:10.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.403+0000 7f6055800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6050162590 con 0x7f605006d970 2024-04-25T10:03:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.403+0000 7f6055800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6050162ec0 con 0x7f605015f5a0 2024-04-25T10:03:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.403+0000 7f6054e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f60500c4b50 0x7f60500b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56176/0 (socket says 172.21.15.5:56176) 2024-04-25T10:03:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.403+0000 7f6054e00640 1 -- 172.21.15.5:0/2479006440 learned_addr learned my addr 172.21.15.5:0/2479006440 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604fe00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f605006bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332205338 0 0) 0x7f60501622b0 con 0x7f60500c4b50 2024-04-25T10:03:10.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6034003520 con 0x7f60500c4b50 2024-04-25T10:03:10.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3437631791 0 0) 0x7f6034003520 con 0x7f60500c4b50 2024-04-25T10:03:10.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 msgr2=0x7f605006bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f605006bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 >> v1:172.21.15.80:6789/0 conn(0x7f605006d970 legacy=0x7f60500c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050162870 con 0x7f60500c4b50 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f60480019f0 con 0x7f60500c4b50 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.404+0000 7f604fe00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f605006bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 >> v1:172.21.15.5:6789/0 conn(0x7f60500c4b50 legacy=0x7f60500b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60340039e0 0x7f6034003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6034004800 0x7f6034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f60340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050162ec0 con 0x7f60340039e0 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603400ba70 con 0x7f6034004800 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603400bc70 con 0x7f605015f5a0 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604fe00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60340039e0 0x7f6034003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f6054e00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f60340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f6047400640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6034004800 0x7f6034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604fe00640 1 -- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 msgr2=0x7f60340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604fe00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f605015f5a0 0x7f60340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604fe00640 1 -- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6034004800 msgr2=0x7f6034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604fe00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6034004800 0x7f6034004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.405+0000 7f604fe00640 1 -- 172.21.15.5:0/2479006440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034003520 con 0x7f60340039e0 2024-04-25T10:03:10.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.406+0000 7f6047400640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6034004800 0x7f6034004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f604fe00640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60340039e0 0x7f6034003e00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6040009c60 tx=0x7f6034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d153b59a5834a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040036070 con 0x7f60340039e0 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040031400 con 0x7f60340039e0 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f604f400640 1 -- 172.21.15.5:0/2479006440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040009860 con 0x7f60340039e0 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f6055800640 1 -- 172.21.15.5:0/2479006440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60340039e0 msgr2=0x7f6034003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f6055800640 1 --2- 172.21.15.5:0/2479006440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60340039e0 0x7f6034003e00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6040009c60 tx=0x7f6034005830 comp rx=0 tx=0).stop 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.407+0000 7f6054e00640 1 -- 172.21.15.5:0/2479006440 reap_dead start 2024-04-25T10:03:10.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.408+0000 7f6055800640 1 -- 172.21.15.5:0/2479006440 shutdown_connections 2024-04-25T10:03:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.408+0000 7f6055800640 1 -- 172.21.15.5:0/2479006440 >> 172.21.15.5:0/2479006440 conn(0x7f60500d1470 msgr2=0x7f60500d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.408+0000 7f6055800640 1 -- 172.21.15.5:0/2479006440 shutdown_connections 2024-04-25T10:03:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.408+0000 7f6055800640 1 -- 172.21.15.5:0/2479006440 wait complete. 2024-04-25T10:03:10.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.410+0000 7f6055800640 1 Processor -- start 2024-04-25T10:03:10.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.410+0000 7f6055800640 1 -- start start 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6055800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60340039e0 0x7f60500caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6055800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 0x7f6050158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6055800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60500ae1e0 0x7f60500c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6055800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605015fb00 con 0x7f6050158800 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6055800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605015f980 con 0x7f60500ae1e0 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6055800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60500cbac0 con 0x7f60340039e0 2024-04-25T10:03:10.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6047400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 0x7f6050158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6047400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 0x7f6050158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37780/0 (socket says 172.21.15.5:37780) 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f6047400640 1 -- 172.21.15.5:0/570274401 learned_addr learned my addr 172.21.15.5:0/570274401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.412+0000 7f604fe00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60340039e0 0x7f60500caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6054e00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60500ae1e0 0x7f60500c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6047400640 1 -- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60340039e0 msgr2=0x7f60500caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6047400640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60340039e0 0x7f60500caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6047400640 1 -- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60500ae1e0 msgr2=0x7f60500c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6047400640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60500ae1e0 0x7f60500c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6047400640 1 -- 172.21.15.5:0/570274401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60500c9890 con 0x7f6050158800 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f604fe00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60340039e0 0x7f60500caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6054e00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60500ae1e0 0x7f60500c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6047400640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 0x7f6050158090 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f603800d7a0 tx=0x7f603800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038005070 con 0x7f6050158800 2024-04-25T10:03:10.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6038005210 con 0x7f6050158800 2024-04-25T10:03:10.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038007030 con 0x7f6050158800 2024-04-25T10:03:10.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60501585d0 con 0x7f6050158800 2024-04-25T10:03:10.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.413+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605015e640 con 0x7f6050158800 2024-04-25T10:03:10.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.415+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6050068f60 con 0x7f6050158800 2024-04-25T10:03:10.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.416+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f603801a020 con 0x7f6050158800 2024-04-25T10:03:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.417+0000 7f604cc00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f602c06b4d0 0x7f602c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.417+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f6038089dc0 con 0x7f6050158800 2024-04-25T10:03:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.417+0000 7f604fe00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f602c06b4d0 0x7f602c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.419+0000 7f604fe00640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f602c06b4d0 0x7f602c06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6040009c60 tx=0x7f6040008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.421+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60380588c0 con 0x7f6050158800 2024-04-25T10:03:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.621+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f605009a3c0 con 0x7f6050158800 2024-04-25T10:03:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.622+0000 7f604cc00640 1 -- 172.21.15.5:0/570274401 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f60380581c0 con 0x7f6050158800 2024-04-25T10:03:10.627 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f602c06b4d0 msgr2=0x7f602c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f602c06b4d0 0x7f602c06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6040009c60 tx=0x7f6040008030 comp rx=0 tx=0).stop 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 msgr2=0x7f6050158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 0x7f6050158090 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f603800d7a0 tx=0x7f603800dc60 comp rx=0 tx=0).stop 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 shutdown_connections 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f602c06b4d0 0x7f602c06d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60500ae1e0 0x7f60500c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6050158800 0x7f6050158090 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 --2- 172.21.15.5:0/570274401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60340039e0 0x7f60500caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 >> 172.21.15.5:0/570274401 conn(0x7f60500d1470 msgr2=0x7f60500cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 shutdown_connections 2024-04-25T10:03:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.625+0000 7f6055800640 1 -- 172.21.15.5:0/570274401 wait complete. 2024-04-25T10:03:10.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.860+0000 7f2181e00640 1 Processor -- start 2024-04-25T10:03:10.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.860+0000 7f2181e00640 1 -- start start 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.862+0000 7f2181e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 0x7f217c15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.862+0000 7f2181e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f217c0713e0 0x7f217c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.862+0000 7f2181e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c15b880 con 0x7f217c06b0a0 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.862+0000 7f2181e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c06d6c0 con 0x7f217c0713e0 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.862+0000 7f2181e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c15edf0 con 0x7f217c15b4a0 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.863+0000 7f2181400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f217c06b0a0 0x7f217c161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56190/0 (socket says 172.21.15.5:56190) 2024-04-25T10:03:10.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.863+0000 7f2181400640 1 -- 172.21.15.5:0/1335960546 learned_addr learned my addr 172.21.15.5:0/1335960546 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.863+0000 7f2180a00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 0x7f217c15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.863+0000 7f217be00640 1 --2- 172.21.15.5:0/1335960546 >> v2:172.21.15.80:3300/0 conn(0x7f217c0713e0 0x7f217c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.863+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127252837 0 0) 0x7f217c15b880 con 0x7f217c06b0a0 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.863+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2164003520 con 0x7f217c06b0a0 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603702722 0 0) 0x7f2164003520 con 0x7f217c06b0a0 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 msgr2=0x7f217c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 0x7f217c15e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 >> v2:172.21.15.80:3300/0 conn(0x7f217c0713e0 msgr2=0x7f217c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 --2- 172.21.15.5:0/1335960546 >> v2:172.21.15.80:3300/0 conn(0x7f217c0713e0 0x7f217c06ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0719a0 con 0x7f217c06b0a0 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f21740019f0 con 0x7f217c06b0a0 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 >> v1:172.21.15.5:6789/0 conn(0x7f217c06b0a0 legacy=0x7f217c161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21640039b0 0x7f2164003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2164004800 0x7f2164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 0x7f21640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c06d6c0 con 0x7f2164004800 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c15edf0 con 0x7f21640039b0 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216400b9c0 con 0x7f217c15b4a0 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217be00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2164004800 0x7f2164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f2180a00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21640039b0 0x7f2164003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217be00640 1 -- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 msgr2=0x7f21640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217be00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15b4a0 0x7f21640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217be00640 1 -- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21640039b0 msgr2=0x7f2164003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.864+0000 7f217be00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21640039b0 0x7f2164003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.865+0000 7f217be00640 1 -- 172.21.15.5:0/1335960546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164003520 con 0x7f2164004800 2024-04-25T10:03:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.865+0000 7f2180a00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21640039b0 0x7f2164003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.866+0000 7f217be00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2164004800 0x7f2164004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2168009ac0 tx=0x7f2164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe14e15a21beb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.866+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168036070 con 0x7f2164004800 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.866+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168031400 con 0x7f2164004800 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.866+0000 7f217b400640 1 -- 172.21.15.5:0/1335960546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168009690 con 0x7f2164004800 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.866+0000 7f2181e00640 1 -- 172.21.15.5:0/1335960546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2164004800 msgr2=0x7f2164004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.866+0000 7f2181e00640 1 --2- 172.21.15.5:0/1335960546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2164004800 0x7f2164004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2168009ac0 tx=0x7f2164005830 comp rx=0 tx=0).stop 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.867+0000 7f2181400640 1 -- 172.21.15.5:0/1335960546 reap_dead start 2024-04-25T10:03:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.867+0000 7f2181e00640 1 -- 172.21.15.5:0/1335960546 shutdown_connections 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.867+0000 7f2181e00640 1 -- 172.21.15.5:0/1335960546 >> 172.21.15.5:0/1335960546 conn(0x7f217c066f80 msgr2=0x7f217c0b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.867+0000 7f2181e00640 1 -- 172.21.15.5:0/1335960546 shutdown_connections 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.867+0000 7f2181e00640 1 -- 172.21.15.5:0/1335960546 wait complete. 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.869+0000 7f2181e00640 1 Processor -- start 2024-04-25T10:03:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.869+0000 7f2181e00640 1 -- start start 2024-04-25T10:03:10.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2164004800 0x7f217c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15cb40 0x7f217c155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 0x7f217c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c072240 con 0x7f217c15cfa0 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c071f60 con 0x7f2164004800 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c06eb60 con 0x7f217c15cb40 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 0x7f217c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 0x7f217c1f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37790/0 (socket says 172.21.15.5:37790) 2024-04-25T10:03:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.870+0000 7f2181400640 1 -- 172.21.15.5:0/1124494619 learned_addr learned my addr 172.21.15.5:0/1124494619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f217be00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15cb40 0x7f217c155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2180a00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2164004800 0x7f217c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181400640 1 -- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15cb40 msgr2=0x7f217c155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181400640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15cb40 0x7f217c155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181400640 1 -- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2164004800 msgr2=0x7f217c154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181400640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2164004800 0x7f217c154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181400640 1 -- 172.21.15.5:0/1124494619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c1537c0 con 0x7f217c15cfa0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181400640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 0x7f217c1f36d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f217c0d0740 tx=0x7f217400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400a5b0 con 0x7f217c15cfa0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217400a750 con 0x7f217c15cfa0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400ab00 con 0x7f217c15cfa0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c153aa0 con 0x7f217c15cfa0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.871+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c1502d0 con 0x7f217c15cfa0 2024-04-25T10:03:10.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.873+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c1541c0 con 0x7f217c15cfa0 2024-04-25T10:03:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.874+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2174011070 con 0x7f217c15cfa0 2024-04-25T10:03:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.875+0000 7f2178c00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f215c06b4d0 0x7f215c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.875+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f2174089470 con 0x7f217c15cfa0 2024-04-25T10:03:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.875+0000 7f2180a00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f215c06b4d0 0x7f215c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.877+0000 7f2180a00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f215c06b4d0 0x7f215c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21700098f0 tx=0x7f2170008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:10.879+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2174057ee0 con 0x7f217c15cfa0 2024-04-25T10:03:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.079+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f217c068f60 con 0x7f217c15cfa0 2024-04-25T10:03:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.080+0000 7f2178c00640 1 -- 172.21.15.5:0/1124494619 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f21740577e0 con 0x7f217c15cfa0 2024-04-25T10:03:11.085 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:11.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f215c06b4d0 msgr2=0x7f215c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f215c06b4d0 0x7f215c06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21700098f0 tx=0x7f2170008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 msgr2=0x7f217c1f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 0x7f217c1f36d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f217c0d0740 tx=0x7f217400cc00 comp rx=0 tx=0).stop 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 shutdown_connections 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f217c15cfa0 0x7f217c1f36d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f217c15cb40 0x7f217c155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f215c06b4d0 0x7f215c06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 --2- 172.21.15.5:0/1124494619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2164004800 0x7f217c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 >> 172.21.15.5:0/1124494619 conn(0x7f217c066f80 msgr2=0x7f217c15f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 shutdown_connections 2024-04-25T10:03:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.083+0000 7f2181e00640 1 -- 172.21.15.5:0/1124494619 wait complete. 2024-04-25T10:03:11.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:11.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.317+0000 7f4abb800640 1 Processor -- start 2024-04-25T10:03:11.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.317+0000 7f4abb800640 1 -- start start 2024-04-25T10:03:11.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4abb800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ab40cea60 0x7f4ab4166420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4abb800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab415bdc0 con 0x7f4ab4071ed0 2024-04-25T10:03:11.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4abb800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab415c060 con 0x7f4ab41529e0 2024-04-25T10:03:11.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4abb800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab40b2340 con 0x7f4ab40cea60 2024-04-25T10:03:11.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4ab3e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4ab4071ed0 0x7f4ab40ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56202/0 (socket says 172.21.15.5:56202) 2024-04-25T10:03:11.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4ab3e00640 1 -- 172.21.15.5:0/1495122866 learned_addr learned my addr 172.21.15.5:0/1495122866 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.319+0000 7f4ab9800640 1 --2- 172.21.15.5:0/1495122866 >> v2:172.21.15.80:3300/0 conn(0x7f4ab40cea60 0x7f4ab4166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588217867 0 0) 0x7f4ab415bdc0 con 0x7f4ab4071ed0 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a98003520 con 0x7f4ab4071ed0 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112419825 0 0) 0x7f4a98003520 con 0x7f4ab4071ed0 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 >> v2:172.21.15.80:3300/0 conn(0x7f4ab40cea60 msgr2=0x7f4ab4166420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 --2- 172.21.15.5:0/1495122866 >> v2:172.21.15.80:3300/0 conn(0x7f4ab40cea60 0x7f4ab4166420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 >> v1:172.21.15.80:6789/0 conn(0x7f4ab41529e0 legacy=0x7f4ab40c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab415c340 con 0x7f4ab4071ed0 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4a9c001670 con 0x7f4ab4071ed0 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 >> v1:172.21.15.5:6789/0 conn(0x7f4ab4071ed0 legacy=0x7f4ab40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4a98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a980051c0 0x7f4a9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40b2340 con 0x7f4a98003a00 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9800c0c0 con 0x7f4a98004800 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.320+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9800c2c0 con 0x7f4a980051c0 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4a98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab3e00640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a980051c0 msgr2=0x7f4a9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a980051c0 0x7f4a9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98004800 msgr2=0x7f4a98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1495122866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98003520 con 0x7f4a98003a00 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab3e00640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.321+0000 7f4ab9800640 1 -- 172.21.15.5:0/1495122866 reap_dead start 2024-04-25T10:03:11.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.322+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4a98003e20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4001720 tx=0x7f4a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babd6a6f683a9f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.322+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4001370 con 0x7f4a98003a00 2024-04-25T10:03:11.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa40052d0 con 0x7f4a98003a00 2024-04-25T10:03:11.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4ab3400640 1 -- 172.21.15.5:0/1495122866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa402abc0 con 0x7f4a98003a00 2024-04-25T10:03:11.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 -- 172.21.15.5:0/1495122866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 msgr2=0x7f4a98003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4a98003e20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4001720 tx=0x7f4a98005830 comp rx=0 tx=0).stop 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 -- 172.21.15.5:0/1495122866 shutdown_connections 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 --2- 172.21.15.5:0/1495122866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4a98003e20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 -- 172.21.15.5:0/1495122866 >> 172.21.15.5:0/1495122866 conn(0x7f4ab4071150 msgr2=0x7f4ab4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 -- 172.21.15.5:0/1495122866 shutdown_connections 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.323+0000 7f4abb800640 1 -- 172.21.15.5:0/1495122866 wait complete. 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.325+0000 7f4abb800640 1 Processor -- start 2024-04-25T10:03:11.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.325+0000 7f4abb800640 1 -- start start 2024-04-25T10:03:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.326+0000 7f4abb800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4ab40b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.326+0000 7f4abb800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ab4141ea0 0x7f4ab415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.326+0000 7f4abb800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ab415b0f0 0x7f4ab41447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.326+0000 7f4abb800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40ce350 con 0x7f4a98003a00 2024-04-25T10:03:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.326+0000 7f4abb800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab415c900 con 0x7f4ab4141ea0 2024-04-25T10:03:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.326+0000 7f4abb800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40ce4d0 con 0x7f4ab415b0f0 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4ab40b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4ab40b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37820/0 (socket says 172.21.15.5:37820) 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1533263485 learned_addr learned my addr 172.21.15.5:0/1533263485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab3e00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ab4141ea0 0x7f4ab415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab9800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ab415b0f0 0x7f4ab41447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ab415b0f0 msgr2=0x7f4ab41447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ab415b0f0 0x7f4ab41447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ab4141ea0 msgr2=0x7f4ab415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ab4141ea0 0x7f4ab415abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.327+0000 7f4ab8e00640 1 -- 172.21.15.5:0/1533263485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab41454f0 con 0x7f4a98003a00 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.328+0000 7f4ab8e00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4ab40b52d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4001840 tx=0x7f4aa4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.328+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4007a50 con 0x7f4a98003a00 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.328+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4040070 con 0x7f4a98003a00 2024-04-25T10:03:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.328+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4001370 con 0x7f4a98003a00 2024-04-25T10:03:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.328+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab40b47f0 con 0x7f4a98003a00 2024-04-25T10:03:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.328+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab40b4db0 con 0x7f4a98003a00 2024-04-25T10:03:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.329+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab4068fd0 con 0x7f4a98003a00 2024-04-25T10:03:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.330+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4aa40052d0 con 0x7f4a98003a00 2024-04-25T10:03:11.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.331+0000 7f4ab0c00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a9006b4d0 0x7f4a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.331+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f4aa40aa650 con 0x7f4a98003a00 2024-04-25T10:03:11.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.332+0000 7f4ab3e00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a9006b4d0 0x7f4a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.333+0000 7f4ab3e00640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a9006b4d0 0x7f4a9006d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4148e80 tx=0x7f4a9c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.335+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4aa40790c0 con 0x7f4a98003a00 2024-04-25T10:03:11.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.545+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ab415b4d0 con 0x7f4a98003a00 2024-04-25T10:03:11.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.546+0000 7f4ab0c00640 1 -- 172.21.15.5:0/1533263485 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4aa40789c0 con 0x7f4a98003a00 2024-04-25T10:03:11.550 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a9006b4d0 msgr2=0x7f4a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a9006b4d0 0x7f4a9006d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4148e80 tx=0x7f4a9c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 msgr2=0x7f4ab40b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4ab40b52d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4001840 tx=0x7f4aa4001120 comp rx=0 tx=0).stop 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 shutdown_connections 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ab415b0f0 0x7f4ab41447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a9006b4d0 0x7f4a9006d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ab4141ea0 0x7f4ab415abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.548+0000 7f4abb800640 1 --2- 172.21.15.5:0/1533263485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a98003a00 0x7f4ab40b52d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.549+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 >> 172.21.15.5:0/1533263485 conn(0x7f4ab4071150 msgr2=0x7f4ab4156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.549+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 shutdown_connections 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:11.549+0000 7f4abb800640 1 -- 172.21.15.5:0/1533263485 wait complete. 2024-04-25T10:03:11.571 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:03:11.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:11.571 DEBUG:teuthology.orchestra.run.smithi080:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:03:11.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:03:11.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:11.663 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:03:11.718 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:11.718 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:11.775 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:11.775 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:11.824 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:11.824 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:11.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:12.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.147+0000 7fab7ca00640 1 Processor -- start 2024-04-25T10:03:12.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.147+0000 7fab7ca00640 1 -- start start 2024-04-25T10:03:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.148+0000 7fab7ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fab7815de30 0x7fab7815e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.148+0000 7fab7ca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab780d0280 0x7fab780d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.148+0000 7fab7ca00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab780d0bf0 con 0x7fab7815da50 2024-04-25T10:03:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.148+0000 7fab7ca00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab780cf1a0 con 0x7fab780d0280 2024-04-25T10:03:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.148+0000 7fab7ca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab780ce2b0 con 0x7fab7815de30 2024-04-25T10:03:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab780d0280 0x7fab780d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fab780d0280 0x7fab780d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37822/0 (socket says 172.21.15.5:37822) 2024-04-25T10:03:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 -- 172.21.15.5:0/2357127574 learned_addr learned my addr 172.21.15.5:0/2357127574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab75400640 1 --2- 172.21.15.5:0/2357127574 >> v2:172.21.15.80:3300/0 conn(0x7fab7815de30 0x7fab7815e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 -- 172.21.15.5:0/2357127574 >> v2:172.21.15.80:3300/0 conn(0x7fab7815de30 msgr2=0x7fab7815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 --2- 172.21.15.5:0/2357127574 >> v2:172.21.15.80:3300/0 conn(0x7fab7815de30 0x7fab7815e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 -- 172.21.15.5:0/2357127574 >> v1:172.21.15.5:6789/0 conn(0x7fab7815da50 legacy=0x7fab780cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab76800640 1 -- 172.21.15.5:0/2357127574 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7815e960 con 0x7fab780d0280 2024-04-25T10:03:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.149+0000 7fab75400640 1 --2- 172.21.15.5:0/2357127574 >> v2:172.21.15.80:3300/0 conn(0x7fab7815de30 0x7fab7815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.150+0000 7fab76800640 1 --2- 172.21.15.5:0/2357127574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab780d0280 0x7fab780d06b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fab7808b9a0 tx=0x7fab7002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e4902289d512a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.151+0000 7fab74a00640 1 -- 172.21.15.5:0/2357127574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70036070 con 0x7fab780d0280 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.151+0000 7fab74a00640 1 -- 172.21.15.5:0/2357127574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70031600 con 0x7fab780d0280 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.151+0000 7fab74a00640 1 -- 172.21.15.5:0/2357127574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700319d0 con 0x7fab780d0280 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.151+0000 7fab7ca00640 1 -- 172.21.15.5:0/2357127574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab780d0280 msgr2=0x7fab780d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.151+0000 7fab7ca00640 1 --2- 172.21.15.5:0/2357127574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab780d0280 0x7fab780d06b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fab7808b9a0 tx=0x7fab7002a730 comp rx=0 tx=0).stop 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.152+0000 7fab7ca00640 1 -- 172.21.15.5:0/2357127574 shutdown_connections 2024-04-25T10:03:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.152+0000 7fab7ca00640 1 --2- 172.21.15.5:0/2357127574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab780d0280 0x7fab780d06b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.152+0000 7fab7ca00640 1 --2- 172.21.15.5:0/2357127574 >> v2:172.21.15.80:3300/0 conn(0x7fab7815de30 0x7fab7815e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.152+0000 7fab7ca00640 1 -- 172.21.15.5:0/2357127574 >> 172.21.15.5:0/2357127574 conn(0x7fab78072290 msgr2=0x7fab78072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.152+0000 7fab7ca00640 1 -- 172.21.15.5:0/2357127574 shutdown_connections 2024-04-25T10:03:12.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.152+0000 7fab7ca00640 1 -- 172.21.15.5:0/2357127574 wait complete. 2024-04-25T10:03:12.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.154+0000 7fab7ca00640 1 Processor -- start 2024-04-25T10:03:12.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.154+0000 7fab7ca00640 1 -- start start 2024-04-25T10:03:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab7ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fab780d0280 0x7fab780b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab7ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 0x7fab7814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab7ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab7815de30 0x7fab780aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab7ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78161640 con 0x7fab7815da50 2024-04-25T10:03:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab7ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab780ce570 con 0x7fab7815de30 2024-04-25T10:03:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab7ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab781617c0 con 0x7fab780d0280 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 0x7fab7814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 0x7fab7814f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37836/0 (socket says 172.21.15.5:37836) 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 -- 172.21.15.5:0/3737160706 learned_addr learned my addr 172.21.15.5:0/3737160706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 -- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fab780d0280 msgr2=0x7fab780b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75e00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fab780d0280 0x7fab780b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab76800640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab7815de30 0x7fab780aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fab780d0280 0x7fab780b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 -- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab7815de30 msgr2=0x7fab780aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab7815de30 0x7fab780aa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75400640 1 -- 172.21.15.5:0/3737160706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780b5e20 con 0x7fab7815da50 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.155+0000 7fab75e00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fab780d0280 0x7fab780b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.156+0000 7fab75400640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 0x7fab7814f650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fab6400c7f0 tx=0x7fab6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.156+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64015070 con 0x7fab7815da50 2024-04-25T10:03:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.156+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab64007820 con 0x7fab7815da50 2024-04-25T10:03:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.156+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64007bd0 con 0x7fab7815da50 2024-04-25T10:03:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.156+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab780ab340 con 0x7fab7815da50 2024-04-25T10:03:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.156+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab780ab900 con 0x7fab7815da50 2024-04-25T10:03:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.157+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab780d1610 con 0x7fab7815da50 2024-04-25T10:03:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.159+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab64018070 con 0x7fab7815da50 2024-04-25T10:03:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.160+0000 7fab6e000640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fab5806b4d0 0x7fab5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.160+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fab6408a740 con 0x7fab7815da50 2024-04-25T10:03:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.160+0000 7fab75e00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fab5806b4d0 0x7fab5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.161+0000 7fab75e00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fab5806b4d0 0x7fab5806d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab68001680 tx=0x7fab68009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.164+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab780b5e20 con 0x7fab7815da50 2024-04-25T10:03:12.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.365+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab780b7e30 con 0x7fab7815da50 2024-04-25T10:03:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.369+0000 7fab6e000640 1 -- 172.21.15.5:0/3737160706 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7fab640591b0 con 0x7fab7815da50 2024-04-25T10:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fab5806b4d0 msgr2=0x7fab5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fab5806b4d0 0x7fab5806d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab68001680 tx=0x7fab68009040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 msgr2=0x7fab7814f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 0x7fab7814f650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fab6400c7f0 tx=0x7fab6400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 shutdown_connections 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fab5806b4d0 0x7fab5806d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab7815de30 0x7fab780aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab7815da50 0x7fab7814f650 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 --2- 172.21.15.5:0/3737160706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fab780d0280 0x7fab780b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 >> 172.21.15.5:0/3737160706 conn(0x7fab78072290 msgr2=0x7fab780c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 shutdown_connections 2024-04-25T10:03:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.372+0000 7fab7ca00640 1 -- 172.21.15.5:0/3737160706 wait complete. 2024-04-25T10:03:12.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:12.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:12.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.604+0000 7f171f200640 1 Processor -- start 2024-04-25T10:03:12.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.604+0000 7f171f200640 1 -- start start 2024-04-25T10:03:12.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17201448f0 0x7f17200cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f172015bc30 0x7f172015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1720144cd0 con 0x7f172015fe30 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172015f460 con 0x7f17201448f0 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172015c440 con 0x7f172015bc30 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171e800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f172015fe30 0x7f172015ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56228/0 (socket says 172.21.15.5:56228) 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171e800640 1 -- 172.21.15.5:0/583526933 learned_addr learned my addr 172.21.15.5:0/583526933 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17201448f0 0x7f17200cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.606+0000 7f171d400640 1 --2- 172.21.15.5:0/583526933 >> v2:172.21.15.80:3300/0 conn(0x7f172015bc30 0x7f172015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.607+0000 7f171de00640 1 -- 172.21.15.5:0/583526933 >> v2:172.21.15.80:3300/0 conn(0x7f172015bc30 msgr2=0x7f172015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.607+0000 7f171de00640 1 --2- 172.21.15.5:0/583526933 >> v2:172.21.15.80:3300/0 conn(0x7f172015bc30 0x7f172015f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.607+0000 7f171de00640 1 -- 172.21.15.5:0/583526933 >> v1:172.21.15.5:6789/0 conn(0x7f172015fe30 legacy=0x7f172015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.607+0000 7f171de00640 1 -- 172.21.15.5:0/583526933 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172015c720 con 0x7f17201448f0 2024-04-25T10:03:12.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.608+0000 7f171de00640 1 --2- 172.21.15.5:0/583526933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17201448f0 0x7f17200cdaa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17200b4980 tx=0x7f171002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27865c5c2e1a726f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.608+0000 7f171ca00640 1 -- 172.21.15.5:0/583526933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710007740 con 0x7f17201448f0 2024-04-25T10:03:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.608+0000 7f171ca00640 1 -- 172.21.15.5:0/583526933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17100078e0 con 0x7f17201448f0 2024-04-25T10:03:12.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.608+0000 7f171ca00640 1 -- 172.21.15.5:0/583526933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710007cb0 con 0x7f17201448f0 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 -- 172.21.15.5:0/583526933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17201448f0 msgr2=0x7f17200cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 --2- 172.21.15.5:0/583526933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17201448f0 0x7f17200cdaa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17200b4980 tx=0x7f171002a680 comp rx=0 tx=0).stop 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 -- 172.21.15.5:0/583526933 shutdown_connections 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 --2- 172.21.15.5:0/583526933 >> v2:172.21.15.80:3300/0 conn(0x7f172015bc30 0x7f172015f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 --2- 172.21.15.5:0/583526933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17201448f0 0x7f17200cdaa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 -- 172.21.15.5:0/583526933 >> 172.21.15.5:0/583526933 conn(0x7f1720066f20 msgr2=0x7f172006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 -- 172.21.15.5:0/583526933 shutdown_connections 2024-04-25T10:03:12.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.609+0000 7f171f200640 1 -- 172.21.15.5:0/583526933 wait complete. 2024-04-25T10:03:12.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.611+0000 7f171f200640 1 Processor -- start 2024-04-25T10:03:12.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.611+0000 7f171f200640 1 -- start start 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201448f0 0x7f1720150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f172015bc30 0x7f172013b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 0x7f172013b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172015d530 con 0x7f172015fe30 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172015c0c0 con 0x7f172015bc30 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172015d6b0 con 0x7f17201448f0 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.612+0000 7f171e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 0x7f172013b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 0x7f172013b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37856/0 (socket says 172.21.15.5:37856) 2024-04-25T10:03:12.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 -- 172.21.15.5:0/2114136422 learned_addr learned my addr 172.21.15.5:0/2114136422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:12.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171de00640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201448f0 0x7f1720150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171d400640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f172015bc30 0x7f172013b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 -- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201448f0 msgr2=0x7f1720150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201448f0 0x7f1720150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 -- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f172015bc30 msgr2=0x7f172013b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f172015bc30 0x7f172013b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 -- 172.21.15.5:0/2114136422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172013a690 con 0x7f172015fe30 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.613+0000 7f171e800640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 0x7f172013b7c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f171800c370 tx=0x7f171800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.614+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718017070 con 0x7f172015fe30 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.614+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1718005ce0 con 0x7f172015fe30 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.614+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17180072d0 con 0x7f172015fe30 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.614+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17200a86b0 con 0x7f172015fe30 2024-04-25T10:03:12.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.614+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17200a8c70 con 0x7f172015fe30 2024-04-25T10:03:12.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.615+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172006b090 con 0x7f172015fe30 2024-04-25T10:03:12.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.616+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f171801a070 con 0x7f172015fe30 2024-04-25T10:03:12.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.617+0000 7f1716000640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.617+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f17180897a0 con 0x7f172015fe30 2024-04-25T10:03:12.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.618+0000 7f171de00640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.619+0000 7f171de00640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1710034000 tx=0x7f171002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.621+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f172006b090 con 0x7f172015fe30 2024-04-25T10:03:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.820+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1720068f60 con 0x7f172015fe30 2024-04-25T10:03:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.821+0000 7f1716000640 1 -- 172.21.15.5:0/2114136422 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1718057c60 con 0x7f172015fe30 2024-04-25T10:03:12.826 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 msgr2=0x7f170006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1710034000 tx=0x7f171002f000 comp rx=0 tx=0).stop 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 msgr2=0x7f172013b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 0x7f172013b7c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f171800c370 tx=0x7f171800c830 comp rx=0 tx=0).stop 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 shutdown_connections 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f172015fe30 0x7f172013b7c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f172015bc30 0x7f172013b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 --2- 172.21.15.5:0/2114136422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201448f0 0x7f1720150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 >> 172.21.15.5:0/2114136422 conn(0x7f1720066f20 msgr2=0x7f17200afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 shutdown_connections 2024-04-25T10:03:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:12.824+0000 7f171f200640 1 -- 172.21.15.5:0/2114136422 wait complete. 2024-04-25T10:03:12.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.056+0000 7f57fac00640 1 Processor -- start 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.056+0000 7f57fac00640 1 -- start start 2024-04-25T10:03:13.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f406e840 0x7f57f415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57f415b330 0x7f57f4071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40ce570 con 0x7f57f40cc840 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f40ce850 con 0x7f57f415b330 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ceb30 con 0x7f57f406e840 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fa200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57f415b330 0x7f57f4071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fa200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57f415b330 0x7f57f4071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37868/0 (socket says 172.21.15.5:37868) 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57fa200640 1 -- 172.21.15.5:0/3992448183 learned_addr learned my addr 172.21.15.5:0/3992448183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.058+0000 7f57f8e00640 1 --2- 172.21.15.5:0/3992448183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f406e840 0x7f57f415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.059+0000 7f57fa200640 1 -- 172.21.15.5:0/3992448183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f406e840 msgr2=0x7f57f415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.059+0000 7f57fa200640 1 --2- 172.21.15.5:0/3992448183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f406e840 0x7f57f415adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.059+0000 7f57fa200640 1 -- 172.21.15.5:0/3992448183 >> v1:172.21.15.80:6789/0 conn(0x7f57f40cc840 legacy=0x7f57f406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.059+0000 7f57fa200640 1 -- 172.21.15.5:0/3992448183 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40cee10 con 0x7f57f415b330 2024-04-25T10:03:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.060+0000 7f57fa200640 1 --2- 172.21.15.5:0/3992448183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f415b330 0x7f57f4071000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e50 tx=0x7f57ec02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddb0397d4278ade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.060+0000 7f57f3e00640 1 -- 172.21.15.5:0/3992448183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec036070 con 0x7f57f415b330 2024-04-25T10:03:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.060+0000 7f57f3e00640 1 -- 172.21.15.5:0/3992448183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec031600 con 0x7f57f415b330 2024-04-25T10:03:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.060+0000 7f57f3e00640 1 -- 172.21.15.5:0/3992448183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0319d0 con 0x7f57f415b330 2024-04-25T10:03:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 -- 172.21.15.5:0/3992448183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f415b330 msgr2=0x7f57f4071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 --2- 172.21.15.5:0/3992448183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f415b330 0x7f57f4071000 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e50 tx=0x7f57ec02a710 comp rx=0 tx=0).stop 2024-04-25T10:03:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 -- 172.21.15.5:0/3992448183 shutdown_connections 2024-04-25T10:03:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 --2- 172.21.15.5:0/3992448183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f415b330 0x7f57f4071000 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 --2- 172.21.15.5:0/3992448183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f406e840 0x7f57f415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 -- 172.21.15.5:0/3992448183 >> 172.21.15.5:0/3992448183 conn(0x7f57f4066f20 msgr2=0x7f57f406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 -- 172.21.15.5:0/3992448183 shutdown_connections 2024-04-25T10:03:13.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.061+0000 7f57fac00640 1 -- 172.21.15.5:0/3992448183 wait complete. 2024-04-25T10:03:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.063+0000 7f57fac00640 1 Processor -- start 2024-04-25T10:03:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.063+0000 7f57fac00640 1 -- start start 2024-04-25T10:03:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57fac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 0x7f57f4150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57fac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40cc840 0x7f57f406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57fac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f415b330 0x7f57f406e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57fac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ce350 con 0x7f57f406e840 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57fac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ce1d0 con 0x7f57f40cc840 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57fac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ccc20 con 0x7f57f415b330 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57f9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 0x7f57f4150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57f9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 0x7f57f4150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37882/0 (socket says 172.21.15.5:37882) 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.064+0000 7f57f9800640 1 -- 172.21.15.5:0/705684926 learned_addr learned my addr 172.21.15.5:0/705684926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f8e00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40cc840 0x7f57f406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57fa200640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f415b330 0x7f57f406e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f9800640 1 -- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f415b330 msgr2=0x7f57f406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f9800640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f415b330 0x7f57f406e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f9800640 1 -- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40cc840 msgr2=0x7f57f406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f9800640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40cc840 0x7f57f406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f9800640 1 -- 172.21.15.5:0/705684926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f41f51d0 con 0x7f57f406e840 2024-04-25T10:03:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f9800640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 0x7f57f4150010 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f57e800c480 tx=0x7f57e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.065+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80052d0 con 0x7f57f406e840 2024-04-25T10:03:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.066+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e8005470 con 0x7f57f406e840 2024-04-25T10:03:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.066+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8012870 con 0x7f57f406e840 2024-04-25T10:03:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.066+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f41f1130 con 0x7f57f406e840 2024-04-25T10:03:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.066+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f41f16f0 con 0x7f57f406e840 2024-04-25T10:03:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.067+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f41f0f40 con 0x7f57f406e840 2024-04-25T10:03:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.068+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57e8012a10 con 0x7f57f406e840 2024-04-25T10:03:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.069+0000 7f57f1600640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f57d406b4d0 0x7f57d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.069+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f57e808a210 con 0x7f57f406e840 2024-04-25T10:03:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.070+0000 7f57f8e00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f57d406b4d0 0x7f57d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.071+0000 7f57f8e00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f57d406b4d0 0x7f57d406d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57f4150a40 tx=0x7f57e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.073+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57e8058d10 con 0x7f57f406e840 2024-04-25T10:03:13.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.289+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57f4068f60 con 0x7f57f406e840 2024-04-25T10:03:13.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.289+0000 7f57f1600640 1 -- 172.21.15.5:0/705684926 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57e8058610 con 0x7f57f406e840 2024-04-25T10:03:13.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f57d406b4d0 msgr2=0x7f57d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f57d406b4d0 0x7f57d406d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f57f4150a40 tx=0x7f57e0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 msgr2=0x7f57f4150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 0x7f57f4150010 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f57e800c480 tx=0x7f57e800c940 comp rx=0 tx=0).stop 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 shutdown_connections 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57f415b330 0x7f57f406e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f57d406b4d0 0x7f57d406d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40cc840 0x7f57f406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 --2- 172.21.15.5:0/705684926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57f406e840 0x7f57f4150010 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 >> 172.21.15.5:0/705684926 conn(0x7f57f4066f20 msgr2=0x7f57f40b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 shutdown_connections 2024-04-25T10:03:13.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.292+0000 7f57fac00640 1 -- 172.21.15.5:0/705684926 wait complete. 2024-04-25T10:03:13.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.526+0000 7f8e5f200640 1 Processor -- start 2024-04-25T10:03:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.527+0000 7f8e5f200640 1 -- start start 2024-04-25T10:03:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.528+0000 7f8e5f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8e6006ec70 0x7f8e60071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e600ade10 con 0x7f8e60143f30 2024-04-25T10:03:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e600ae0f0 con 0x7f8e600aebd0 2024-04-25T10:03:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e600ae3d0 con 0x7f8e6006ec70 2024-04-25T10:03:13.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5de00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8e60143f30 0x7f8e6006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56234/0 (socket says 172.21.15.5:56234) 2024-04-25T10:03:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5de00640 1 -- 172.21.15.5:0/725860420 learned_addr learned my addr 172.21.15.5:0/725860420 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e55400640 1 --2- 172.21.15.5:0/725860420 >> v2:172.21.15.80:3300/0 conn(0x7f8e6006ec70 0x7f8e60071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508519540 0 0) 0x7f8e600ade10 con 0x7f8e60143f30 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.529+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e44003520 con 0x7f8e60143f30 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342177358 0 0) 0x7f8e44003520 con 0x7f8e60143f30 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 >> v2:172.21.15.80:3300/0 conn(0x7f8e6006ec70 msgr2=0x7f8e60071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 --2- 172.21.15.5:0/725860420 >> v2:172.21.15.80:3300/0 conn(0x7f8e6006ec70 0x7f8e60071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 >> v1:172.21.15.80:6789/0 conn(0x7f8e600aebd0 legacy=0x7f8e600b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600ae6b0 con 0x7f8e60143f30 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8e50001670 con 0x7f8e60143f30 2024-04-25T10:03:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 >> v1:172.21.15.5:6789/0 conn(0x7f8e60143f30 legacy=0x7f8e6006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039b0 0x7f8e44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e440051c0 0x7f8e4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e600ae3d0 con 0x7f8e44004800 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4400c160 con 0x7f8e440039b0 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4400c360 con 0x7f8e440051c0 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.530+0000 7f8e55400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e5de00640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039b0 0x7f8e44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e55400640 1 -- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e440051c0 msgr2=0x7f8e4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e55400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e440051c0 0x7f8e4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e55400640 1 -- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039b0 msgr2=0x7f8e44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e55400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039b0 0x7f8e44003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e55400640 1 -- 172.21.15.5:0/725860420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44003520 con 0x7f8e44004800 2024-04-25T10:03:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.531+0000 7f8e5e800640 1 -- 172.21.15.5:0/725860420 reap_dead start 2024-04-25T10:03:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.532+0000 7f8e55400640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44004800 0x7f8e44004c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009a90 tx=0x7f8e44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db00d324ba4c68d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.532+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48036070 con 0x7f8e44004800 2024-04-25T10:03:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.532+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e48031400 con 0x7f8e44004800 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.532+0000 7f8e5d400640 1 -- 172.21.15.5:0/725860420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e48009690 con 0x7f8e44004800 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 -- 172.21.15.5:0/725860420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44004800 msgr2=0x7f8e44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44004800 0x7f8e44004c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009a90 tx=0x7f8e44005830 comp rx=0 tx=0).stop 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 -- 172.21.15.5:0/725860420 shutdown_connections 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 --2- 172.21.15.5:0/725860420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44004800 0x7f8e44004c80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 -- 172.21.15.5:0/725860420 >> 172.21.15.5:0/725860420 conn(0x7f8e60066f20 msgr2=0x7f8e6006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 -- 172.21.15.5:0/725860420 shutdown_connections 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.533+0000 7f8e5f200640 1 -- 172.21.15.5:0/725860420 wait complete. 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.535+0000 7f8e5f200640 1 Processor -- start 2024-04-25T10:03:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.535+0000 7f8e5f200640 1 -- start start 2024-04-25T10:03:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e44004800 0x7f8e601f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e601e71a0 0x7f8e601f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 0x7f8e600b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6006dbb0 con 0x7f8e600b3750 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6006da30 con 0x7f8e44004800 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6006dd30 con 0x7f8e601e71a0 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 0x7f8e600b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 0x7f8e600b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37912/0 (socket says 172.21.15.5:37912) 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.536+0000 7f8e5e800640 1 -- 172.21.15.5:0/4253877786 learned_addr learned my addr 172.21.15.5:0/4253877786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5de00640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e44004800 0x7f8e601f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e55400640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e601e71a0 0x7f8e601f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5e800640 1 -- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e601e71a0 msgr2=0x7f8e601f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5e800640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e601e71a0 0x7f8e601f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5e800640 1 -- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e44004800 msgr2=0x7f8e601f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5e800640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e44004800 0x7f8e601f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5e800640 1 -- 172.21.15.5:0/4253877786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e601f4270 con 0x7f8e600b3750 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e5e800640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 0x7f8e600b3c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8e600723d0 tx=0x7f8e5800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.537+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e58017070 con 0x7f8e600b3750 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.538+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e58005d40 con 0x7f8e600b3750 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.538+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e601e7660 con 0x7f8e600b3750 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.538+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800a850 con 0x7f8e600b3750 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.538+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e601e7c20 con 0x7f8e600b3750 2024-04-25T10:03:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.539+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e60068f60 con 0x7f8e600b3750 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.540+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e5801a070 con 0x7f8e600b3750 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.541+0000 7f8e56a00640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e3c06b4d0 0x7f8e3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.541+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f8e58089830 con 0x7f8e600b3750 2024-04-25T10:03:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.542+0000 7f8e5de00640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e3c06b4d0 0x7f8e3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.543+0000 7f8e5de00640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e3c06b4d0 0x7f8e3c06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e50001eb0 tx=0x7f8e50009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.546+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e601f4270 con 0x7f8e600b3750 2024-04-25T10:03:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.742+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e600716d0 con 0x7f8e600b3750 2024-04-25T10:03:13.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.743+0000 7f8e56a00640 1 -- 172.21.15.5:0/4253877786 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e58058360 con 0x7f8e600b3750 2024-04-25T10:03:13.748 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e3c06b4d0 msgr2=0x7f8e3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e3c06b4d0 0x7f8e3c06d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e50001eb0 tx=0x7f8e50009040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 msgr2=0x7f8e600b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 0x7f8e600b3c00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8e600723d0 tx=0x7f8e5800cb90 comp rx=0 tx=0).stop 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 shutdown_connections 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e600b3750 0x7f8e600b3c00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8e601e71a0 0x7f8e601f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8e3c06b4d0 0x7f8e3c06d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 --2- 172.21.15.5:0/4253877786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e44004800 0x7f8e601f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 >> 172.21.15.5:0/4253877786 conn(0x7f8e60066f20 msgr2=0x7f8e60144f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 shutdown_connections 2024-04-25T10:03:13.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.746+0000 7f8e5f200640 1 -- 172.21.15.5:0/4253877786 wait complete. 2024-04-25T10:03:13.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.979+0000 7ffb14200640 1 Processor -- start 2024-04-25T10:03:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.980+0000 7ffb14200640 1 -- start start 2024-04-25T10:03:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.981+0000 7ffb14200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffb0c06d130 0x7ffb0c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.981+0000 7ffb14200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb0c071670 0x7ffb0c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.981+0000 7ffb14200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c1453c0 con 0x7ffb0c144fe0 2024-04-25T10:03:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.981+0000 7ffb14200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c0a8560 con 0x7ffb0c06d130 2024-04-25T10:03:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.981+0000 7ffb14200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c06d6a0 con 0x7ffb0c071670 2024-04-25T10:03:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.982+0000 7ffb10e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffb0c06d130 0x7ffb0c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.982+0000 7ffb10e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffb0c06d130 0x7ffb0c071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37916/0 (socket says 172.21.15.5:37916) 2024-04-25T10:03:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.982+0000 7ffb10e00640 1 -- 172.21.15.5:0/3274087648 learned_addr learned my addr 172.21.15.5:0/3274087648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.982+0000 7ffb12200640 1 --2- 172.21.15.5:0/3274087648 >> v2:172.21.15.80:3300/0 conn(0x7ffb0c071670 0x7ffb0c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.983+0000 7ffb10e00640 1 -- 172.21.15.5:0/3274087648 >> v2:172.21.15.80:3300/0 conn(0x7ffb0c071670 msgr2=0x7ffb0c15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.983+0000 7ffb10e00640 1 --2- 172.21.15.5:0/3274087648 >> v2:172.21.15.80:3300/0 conn(0x7ffb0c071670 0x7ffb0c15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.983+0000 7ffb10e00640 1 -- 172.21.15.5:0/3274087648 >> v1:172.21.15.80:6789/0 conn(0x7ffb0c144fe0 legacy=0x7ffb0c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.983+0000 7ffb10e00640 1 -- 172.21.15.5:0/3274087648 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c06d980 con 0x7ffb0c06d130 2024-04-25T10:03:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.984+0000 7ffb10e00640 1 --2- 172.21.15.5:0/3274087648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c06d130 0x7ffb0c071130 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c15bd10 tx=0x7ffaf802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eed319330efe50c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb0be00640 1 -- 172.21.15.5:0/3274087648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8037070 con 0x7ffb0c06d130 2024-04-25T10:03:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb0be00640 1 -- 172.21.15.5:0/3274087648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8032440 con 0x7ffb0c06d130 2024-04-25T10:03:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb0be00640 1 -- 172.21.15.5:0/3274087648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8032810 con 0x7ffb0c06d130 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb14200640 1 -- 172.21.15.5:0/3274087648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c06d130 msgr2=0x7ffb0c071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb14200640 1 --2- 172.21.15.5:0/3274087648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c06d130 0x7ffb0c071130 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c15bd10 tx=0x7ffaf802a230 comp rx=0 tx=0).stop 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb14200640 1 -- 172.21.15.5:0/3274087648 shutdown_connections 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb14200640 1 --2- 172.21.15.5:0/3274087648 >> v2:172.21.15.80:3300/0 conn(0x7ffb0c071670 0x7ffb0c15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb14200640 1 --2- 172.21.15.5:0/3274087648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c06d130 0x7ffb0c071130 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.985+0000 7ffb14200640 1 -- 172.21.15.5:0/3274087648 >> 172.21.15.5:0/3274087648 conn(0x7ffb0c06ab60 msgr2=0x7ffb0c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.986+0000 7ffb14200640 1 -- 172.21.15.5:0/3274087648 shutdown_connections 2024-04-25T10:03:13.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.986+0000 7ffb14200640 1 -- 172.21.15.5:0/3274087648 wait complete. 2024-04-25T10:03:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.987+0000 7ffb14200640 1 Processor -- start 2024-04-25T10:03:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.987+0000 7ffb14200640 1 -- start start 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.988+0000 7ffb14200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb0c06d130 0x7ffb0c13e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.988+0000 7ffb14200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 0x7ffb0c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.988+0000 7ffb14200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb0c144fe0 0x7ffb0c133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.988+0000 7ffb14200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c15be50 con 0x7ffb0c071670 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.988+0000 7ffb14200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c06df40 con 0x7ffb0c144fe0 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.988+0000 7ffb14200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c15c8b0 con 0x7ffb0c06d130 2024-04-25T10:03:13.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 0x7ffb0c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 0x7ffb0c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37920/0 (socket says 172.21.15.5:37920) 2024-04-25T10:03:13.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 -- 172.21.15.5:0/1825704691 learned_addr learned my addr 172.21.15.5:0/1825704691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 -- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb0c06d130 msgr2=0x7ffb0c13e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb11800640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb0c06d130 0x7ffb0c13e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb0c06d130 0x7ffb0c13e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 -- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb0c144fe0 msgr2=0x7ffb0c133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb12200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb0c144fe0 0x7ffb0c133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb0c144fe0 0x7ffb0c133bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.989+0000 7ffb10e00640 1 -- 172.21.15.5:0/1825704691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0b2860 con 0x7ffb0c071670 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.990+0000 7ffb10e00640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 0x7ffb0c0b3900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000c00 tx=0x7ffaf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.990+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8041020 con 0x7ffb0c071670 2024-04-25T10:03:13.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.990+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf80306c0 con 0x7ffb0c071670 2024-04-25T10:03:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.990+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8037040 con 0x7ffb0c071670 2024-04-25T10:03:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.990+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0b2b40 con 0x7ffb0c071670 2024-04-25T10:03:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.990+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c0b3100 con 0x7ffb0c071670 2024-04-25T10:03:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.991+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c068f60 con 0x7ffb0c071670 2024-04-25T10:03:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.992+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaf8044020 con 0x7ffb0c071670 2024-04-25T10:03:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.993+0000 7ffb09600640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.993+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7ffaf80aae60 con 0x7ffb0c071670 2024-04-25T10:03:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.994+0000 7ffb11800640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.995+0000 7ffb11800640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffaec06b4d0 0x7ffaec06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000a750 tx=0x7ffb00008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:13.998+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaf804b030 con 0x7ffb0c071670 2024-04-25T10:03:14.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.195+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ffb0c06b3a0 con 0x7ffb0c071670 2024-04-25T10:03:14.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.195+0000 7ffb09600640 1 -- 172.21.15.5:0/1825704691 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffaf80798d0 con 0x7ffb0c071670 2024-04-25T10:03:14.200 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffaec06b4d0 msgr2=0x7ffaec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffaec06b4d0 0x7ffaec06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000a750 tx=0x7ffb00008040 comp rx=0 tx=0).stop 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 msgr2=0x7ffb0c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 0x7ffb0c0b3900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000c00 tx=0x7ffaf80046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 shutdown_connections 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffaec06b4d0 0x7ffaec06d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb0c144fe0 0x7ffb0c133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb0c071670 0x7ffb0c0b3900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 --2- 172.21.15.5:0/1825704691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb0c06d130 0x7ffb0c13e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 >> 172.21.15.5:0/1825704691 conn(0x7ffb0c06ab60 msgr2=0x7ffb0c1434b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 shutdown_connections 2024-04-25T10:03:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.198+0000 7ffb14200640 1 -- 172.21.15.5:0/1825704691 wait complete. 2024-04-25T10:03:14.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.440+0000 7fd7a2a00640 1 Processor -- start 2024-04-25T10:03:14.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.440+0000 7fd7a2a00640 1 -- start start 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd7a2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 0x7fd79c0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd7a2a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd79c06e970 0x7fd79c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd7a2a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0cdbf0 con 0x7fd79c0ced20 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd7a2a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c1444e0 con 0x7fd79c06e970 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd7a2a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c15ac70 con 0x7fd79c144100 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd79b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd79c0ced20 0x7fd79c06e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56246/0 (socket says 172.21.15.5:56246) 2024-04-25T10:03:14.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.442+0000 7fd79b400640 1 -- 172.21.15.5:0/342610803 learned_addr learned my addr 172.21.15.5:0/342610803 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd79be00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 0x7fd79c0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd7a0a00640 1 --2- 172.21.15.5:0/342610803 >> v2:172.21.15.80:3300/0 conn(0x7fd79c06e970 0x7fd79c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958036469 0 0) 0x7fd79c0cdbf0 con 0x7fd79c0ced20 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd780003520 con 0x7fd79c0ced20 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1191367449 0 0) 0x7fd780003520 con 0x7fd79c0ced20 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 msgr2=0x7fd79c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.443+0000 7fd79aa00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 0x7fd79c0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 >> v2:172.21.15.80:3300/0 conn(0x7fd79c06e970 msgr2=0x7fd79c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 --2- 172.21.15.5:0/342610803 >> v2:172.21.15.80:3300/0 conn(0x7fd79c06e970 0x7fd79c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0cded0 con 0x7fd79c0ced20 2024-04-25T10:03:14.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd788001670 con 0x7fd79c0ced20 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 >> v1:172.21.15.5:6789/0 conn(0x7fd79c0ced20 legacy=0x7fd79c06e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7800039b0 0x7fd780003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 0x7fd780004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd780004d40 0x7fd7800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c1444e0 con 0x7fd7800039b0 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c15ac70 con 0x7fd780004d40 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78000b9c0 con 0x7fd79c144100 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79be00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7800039b0 0x7fd780003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79b400640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 0x7fd780004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd7a0a00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd780004d40 0x7fd7800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79be00640 1 -- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 msgr2=0x7fd780004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.444+0000 7fd79be00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd79c144100 0x7fd780004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.445+0000 7fd79be00640 1 -- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd780004d40 msgr2=0x7fd7800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.445+0000 7fd79be00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd780004d40 0x7fd7800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.445+0000 7fd79be00640 1 -- 172.21.15.5:0/342610803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780003520 con 0x7fd7800039b0 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.445+0000 7fd7a0a00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd780004d40 0x7fd7800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd79be00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7800039b0 0x7fd780003dd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009650 tx=0x7fd780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1972ccef1c82f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c038070 con 0x7fd7800039b0 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c033400 con 0x7fd7800039b0 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd79aa00640 1 -- 172.21.15.5:0/342610803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c009690 con 0x7fd7800039b0 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd7a2a00640 1 -- 172.21.15.5:0/342610803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7800039b0 msgr2=0x7fd780003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/342610803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7800039b0 0x7fd780003dd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009650 tx=0x7fd780005830 comp rx=0 tx=0).stop 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.446+0000 7fd7a0a00640 1 -- 172.21.15.5:0/342610803 reap_dead start 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.447+0000 7fd7a2a00640 1 -- 172.21.15.5:0/342610803 shutdown_connections 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.447+0000 7fd7a2a00640 1 -- 172.21.15.5:0/342610803 >> 172.21.15.5:0/342610803 conn(0x7fd79c071150 msgr2=0x7fd79c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.447+0000 7fd7a2a00640 1 -- 172.21.15.5:0/342610803 shutdown_connections 2024-04-25T10:03:14.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.447+0000 7fd7a2a00640 1 -- 172.21.15.5:0/342610803 wait complete. 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.448+0000 7fd7a2a00640 1 Processor -- start 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.449+0000 7fd7a2a00640 1 -- start start 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7800039b0 0x7fd79c13cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 0x7fd79c0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd79c0b37a0 0x7fd79c1e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a2a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0cea00 con 0x7fd79c1f6af0 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a2a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0ce490 con 0x7fd79c0b37a0 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a2a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0cd610 con 0x7fd7800039b0 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 0x7fd79c0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 0x7fd79c0b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37944/0 (socket says 172.21.15.5:37944) 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 -- 172.21.15.5:0/1079859651 learned_addr learned my addr 172.21.15.5:0/1079859651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd7a0a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd79c0b37a0 0x7fd79c1e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 -- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7800039b0 msgr2=0x7fd79c13cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79be00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7800039b0 0x7fd79c13cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7800039b0 0x7fd79c13cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 -- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd79c0b37a0 msgr2=0x7fd79c1e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd79c0b37a0 0x7fd79c1e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.450+0000 7fd79b400640 1 -- 172.21.15.5:0/1079859651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0b5140 con 0x7fd79c1f6af0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.451+0000 7fd79b400640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 0x7fd79c0b3260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e7f0 tx=0x7fd78800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.451+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800c870 con 0x7fd79c1f6af0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.451+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788018070 con 0x7fd79c1f6af0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.451+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800cc50 con 0x7fd79c1f6af0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.451+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c0b53e0 con 0x7fd79c1f6af0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.451+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c0b2d50 con 0x7fd79c1f6af0 2024-04-25T10:03:14.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.452+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c0b2f50 con 0x7fd79c1f6af0 2024-04-25T10:03:14.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.453+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd78801d460 con 0x7fd79c1f6af0 2024-04-25T10:03:14.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.454+0000 7fd793e00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd77c06b500 0x7fd77c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.454+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fd78808a5b0 con 0x7fd79c1f6af0 2024-04-25T10:03:14.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.455+0000 7fd79be00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd77c06b500 0x7fd77c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.456+0000 7fd79be00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd77c06b500 0x7fd77c06d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd78c035000 tx=0x7fd78c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.459+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd79c0b2f50 con 0x7fd79c1f6af0 2024-04-25T10:03:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.671+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd79c068fd0 con 0x7fd79c1f6af0 2024-04-25T10:03:14.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.672+0000 7fd793e00640 1 -- 172.21.15.5:0/1079859651 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd788059020 con 0x7fd79c1f6af0 2024-04-25T10:03:14.677 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd77c06b500 msgr2=0x7fd77c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd77c06b500 0x7fd77c06d9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd78c035000 tx=0x7fd78c008030 comp rx=0 tx=0).stop 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 msgr2=0x7fd79c0b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 0x7fd79c0b3260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e7f0 tx=0x7fd78800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 shutdown_connections 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd77c06b500 0x7fd77c06d9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd79c0b37a0 0x7fd79c1e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd79c1f6af0 0x7fd79c0b3260 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 --2- 172.21.15.5:0/1079859651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7800039b0 0x7fd79c13cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 >> 172.21.15.5:0/1079859651 conn(0x7fd79c071150 msgr2=0x7fd79c15a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 shutdown_connections 2024-04-25T10:03:14.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:14.675+0000 7fd7a2a00640 1 -- 172.21.15.5:0/1079859651 wait complete. 2024-04-25T10:03:14.698 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:03:14.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:14.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:03:14.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:03:14.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:14.791 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:03:14.847 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:03:14.847 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:14.907 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:03:14.907 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:14.955 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:03:14.955 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:15.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:15.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.269+0000 7f782d000640 1 Processor -- start 2024-04-25T10:03:15.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.270+0000 7f782d000640 1 -- start start 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.271+0000 7f782d000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78280d07d0 0x7f78280b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.271+0000 7f782d000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78280d2380 0x7f78280b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.271+0000 7f782d000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78280cf410 con 0x7f7828161200 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.271+0000 7f782d000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280b28b0 con 0x7f78280d2380 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.271+0000 7f782d000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280cf6f0 con 0x7f78280d07d0 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.271+0000 7f7827e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78280d2380 0x7f78280b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f7827e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78280d2380 0x7f78280b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37950/0 (socket says 172.21.15.5:37950) 2024-04-25T10:03:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f7827e00640 1 -- 172.21.15.5:0/4075296086 learned_addr learned my addr 172.21.15.5:0/4075296086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f781fe00640 1 --2- 172.21.15.5:0/4075296086 >> v2:172.21.15.80:3300/0 conn(0x7f78280d07d0 0x7f78280b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f7827e00640 1 -- 172.21.15.5:0/4075296086 >> v2:172.21.15.80:3300/0 conn(0x7f78280d07d0 msgr2=0x7f78280b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f7827e00640 1 --2- 172.21.15.5:0/4075296086 >> v2:172.21.15.80:3300/0 conn(0x7f78280d07d0 0x7f78280b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f7827e00640 1 -- 172.21.15.5:0/4075296086 >> v1:172.21.15.80:6789/0 conn(0x7f7828161200 legacy=0x7f78280d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.272+0000 7f7827e00640 1 -- 172.21.15.5:0/4075296086 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280d3fd0 con 0x7f78280d2380 2024-04-25T10:03:15.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.274+0000 7f7827e00640 1 --2- 172.21.15.5:0/4075296086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78280d2380 0x7f78280b2f80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f782808bb30 tx=0x7f782002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb1cf84bc9a2e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.274+0000 7f7826a00640 1 -- 172.21.15.5:0/4075296086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820036070 con 0x7f78280d2380 2024-04-25T10:03:15.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.274+0000 7f7826a00640 1 -- 172.21.15.5:0/4075296086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820031600 con 0x7f78280d2380 2024-04-25T10:03:15.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.274+0000 7f7826a00640 1 -- 172.21.15.5:0/4075296086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200319d0 con 0x7f78280d2380 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 -- 172.21.15.5:0/4075296086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78280d2380 msgr2=0x7f78280b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 --2- 172.21.15.5:0/4075296086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78280d2380 0x7f78280b2f80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f782808bb30 tx=0x7f782002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 -- 172.21.15.5:0/4075296086 shutdown_connections 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 --2- 172.21.15.5:0/4075296086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78280d2380 0x7f78280b2f80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 --2- 172.21.15.5:0/4075296086 >> v2:172.21.15.80:3300/0 conn(0x7f78280d07d0 0x7f78280b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 -- 172.21.15.5:0/4075296086 >> 172.21.15.5:0/4075296086 conn(0x7f782808af80 msgr2=0x7f78280d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 -- 172.21.15.5:0/4075296086 shutdown_connections 2024-04-25T10:03:15.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.275+0000 7f782d000640 1 -- 172.21.15.5:0/4075296086 wait complete. 2024-04-25T10:03:15.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.277+0000 7f782d000640 1 Processor -- start 2024-04-25T10:03:15.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.277+0000 7f782d000640 1 -- start start 2024-04-25T10:03:15.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f782d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78280d07d0 0x7f78280cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f782d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78280d2380 0x7f78280b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f782d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 0x7f78280b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f782d000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b3640 con 0x7f7828161200 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f782d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b34c0 con 0x7f78280d07d0 2024-04-25T10:03:15.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f782d000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78280b37c0 con 0x7f78280d2380 2024-04-25T10:03:15.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f7827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 0x7f78280b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f7827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 0x7f78280b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37960/0 (socket says 172.21.15.5:37960) 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.278+0000 7f7827e00640 1 -- 172.21.15.5:0/3941431098 learned_addr learned my addr 172.21.15.5:0/3941431098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f781fe00640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78280d2380 0x7f78280b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827400640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78280d07d0 0x7f78280cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827e00640 1 -- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78280d2380 msgr2=0x7f78280b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827e00640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78280d2380 0x7f78280b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827e00640 1 -- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78280d07d0 msgr2=0x7f78280cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827e00640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78280d07d0 0x7f78280cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827e00640 1 -- 172.21.15.5:0/3941431098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280cba70 con 0x7f7828161200 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827400640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78280d07d0 0x7f78280cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f781fe00640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78280d2380 0x7f78280b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.279+0000 7f7827e00640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 0x7f78280b7850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7820034040 tx=0x7f7820004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.280+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820036050 con 0x7f7828161200 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.280+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820042070 con 0x7f7828161200 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.280+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78280cbd50 con 0x7f7828161200 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.280+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820009510 con 0x7f7828161200 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.280+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78280c74a0 con 0x7f7828161200 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.281+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7828068f60 con 0x7f7828161200 2024-04-25T10:03:15.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.282+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7820007bd0 con 0x7f7828161200 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.283+0000 7f781f400640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f780406b4d0 0x7f780406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.283+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f78200ab520 con 0x7f7828161200 2024-04-25T10:03:15.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.284+0000 7f7827400640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f780406b4d0 0x7f780406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.285+0000 7f7827400640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f780406b4d0 0x7f780406d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7818006fd0 tx=0x7f7818008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.287+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7820079f90 con 0x7f7828161200 2024-04-25T10:03:15.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.488+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78280c7670 con 0x7f7828161200 2024-04-25T10:03:15.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.490+0000 7f781f400640 1 -- 172.21.15.5:0/3941431098 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f7820079890 con 0x7f7828161200 2024-04-25T10:03:15.495 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:15.496 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f780406b4d0 msgr2=0x7f780406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f780406b4d0 0x7f780406d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7818006fd0 tx=0x7f7818008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 msgr2=0x7f78280b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 0x7f78280b7850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7820034040 tx=0x7f7820004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 shutdown_connections 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7828161200 0x7f78280b7850 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78280d2380 0x7f78280b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f780406b4d0 0x7f780406d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 --2- 172.21.15.5:0/3941431098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78280d07d0 0x7f78280cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 >> 172.21.15.5:0/3941431098 conn(0x7f782808af80 msgr2=0x7f782815ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 shutdown_connections 2024-04-25T10:03:15.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.494+0000 7f782d000640 1 -- 172.21.15.5:0/3941431098 wait complete. 2024-04-25T10:03:15.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:15.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:15.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.716+0000 7fcb71800640 1 Processor -- start 2024-04-25T10:03:15.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.717+0000 7fcb71800640 1 -- start start 2024-04-25T10:03:15.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.718+0000 7fcb71800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcb6c06d130 0x7fcb6c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.718+0000 7fcb71800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c1453c0 con 0x7fcb6c071670 2024-04-25T10:03:15.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.718+0000 7fcb71800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c06d6a0 con 0x7fcb6c144fe0 2024-04-25T10:03:15.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.718+0000 7fcb71800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c0a8560 con 0x7fcb6c06d130 2024-04-25T10:03:15.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb6b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcb6c071670 0x7fcb6c15c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56252/0 (socket says 172.21.15.5:56252) 2024-04-25T10:03:15.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb6b400640 1 -- 172.21.15.5:0/2173159669 learned_addr learned my addr 172.21.15.5:0/2173159669 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb6a000640 1 --2- 172.21.15.5:0/2173159669 >> v2:172.21.15.80:3300/0 conn(0x7fcb6c06d130 0x7fcb6c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262310876 0 0) 0x7fcb6c1453c0 con 0x7fcb6c071670 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb50003520 con 0x7fcb6c071670 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365742030 0 0) 0x7fcb50003520 con 0x7fcb6c071670 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 >> v2:172.21.15.80:3300/0 conn(0x7fcb6c06d130 msgr2=0x7fcb6c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb69600640 1 --2- 172.21.15.5:0/2173159669 >> v2:172.21.15.80:3300/0 conn(0x7fcb6c06d130 0x7fcb6c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.719+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 >> v1:172.21.15.80:6789/0 conn(0x7fcb6c144fe0 legacy=0x7fcb6c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c06d980 con 0x7fcb6c071670 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fcb64001670 con 0x7fcb6c071670 2024-04-25T10:03:15.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 >> v1:172.21.15.5:6789/0 conn(0x7fcb6c071670 legacy=0x7fcb6c15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb50003a00 0x7fcb50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb500051c0 0x7fcb5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5000c090 con 0x7fcb50004800 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5000c240 con 0x7fcb50003a00 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5000c440 con 0x7fcb500051c0 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb6a000640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb6a000640 1 -- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb500051c0 msgr2=0x7fcb5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb6a000640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb500051c0 0x7fcb5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb6aa00640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb50003a00 0x7fcb50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb6a000640 1 -- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb50003a00 msgr2=0x7fcb50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.720+0000 7fcb6a000640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb50003a00 0x7fcb50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.721+0000 7fcb6a000640 1 -- 172.21.15.5:0/2173159669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50003520 con 0x7fcb50004800 2024-04-25T10:03:15.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.721+0000 7fcb6b400640 1 -- 172.21.15.5:0/2173159669 reap_dead start 2024-04-25T10:03:15.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.721+0000 7fcb6aa00640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb50003a00 0x7fcb50003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.722+0000 7fcb6a000640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009fc0 tx=0x7fcb50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b599fda5cb520b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.722+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c009c10 con 0x7fcb50004800 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.722+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c036070 con 0x7fcb50004800 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.722+0000 7fcb69600640 1 -- 172.21.15.5:0/2173159669 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c031880 con 0x7fcb50004800 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.722+0000 7fcb71800640 1 -- 172.21.15.5:0/2173159669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 msgr2=0x7fcb50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.722+0000 7fcb71800640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009fc0 tx=0x7fcb50005830 comp rx=0 tx=0).stop 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.723+0000 7fcb71800640 1 -- 172.21.15.5:0/2173159669 shutdown_connections 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.723+0000 7fcb71800640 1 --2- 172.21.15.5:0/2173159669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb50004c80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.723+0000 7fcb71800640 1 -- 172.21.15.5:0/2173159669 >> 172.21.15.5:0/2173159669 conn(0x7fcb6c06ab60 msgr2=0x7fcb6c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.723+0000 7fcb71800640 1 -- 172.21.15.5:0/2173159669 shutdown_connections 2024-04-25T10:03:15.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.723+0000 7fcb71800640 1 -- 172.21.15.5:0/2173159669 wait complete. 2024-04-25T10:03:15.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.725+0000 7fcb71800640 1 Processor -- start 2024-04-25T10:03:15.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.725+0000 7fcb71800640 1 -- start start 2024-04-25T10:03:15.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb71800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb6c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb71800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb6c143a50 0x7fcb6c143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb71800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb6c0b3160 0x7fcb6c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb71800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c15be50 con 0x7fcb50004800 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb71800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c06df40 con 0x7fcb6c0b3160 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb71800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c071130 con 0x7fcb6c143a50 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb6aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb6c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb6aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb6c0cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37978/0 (socket says 172.21.15.5:37978) 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.726+0000 7fcb6aa00640 1 -- 172.21.15.5:0/687073529 learned_addr learned my addr 172.21.15.5:0/687073529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6b400640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb6c0b3160 0x7fcb6c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6a000640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb6c143a50 0x7fcb6c143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6aa00640 1 -- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb6c143a50 msgr2=0x7fcb6c143eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6aa00640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb6c143a50 0x7fcb6c143eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6aa00640 1 -- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb6c0b3160 msgr2=0x7fcb6c0b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6aa00640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb6c0b3160 0x7fcb6c0b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6aa00640 1 -- 172.21.15.5:0/687073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c1443f0 con 0x7fcb50004800 2024-04-25T10:03:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6b400640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb6c0b3160 0x7fcb6c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.727+0000 7fcb6aa00640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb6c0cafb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d400 tx=0x7fcb5400d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.728+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54018070 con 0x7fcb50004800 2024-04-25T10:03:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.728+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54003610 con 0x7fcb50004800 2024-04-25T10:03:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.728+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c0b3e20 con 0x7fcb50004800 2024-04-25T10:03:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.728+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54003a40 con 0x7fcb50004800 2024-04-25T10:03:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.728+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c1312d0 con 0x7fcb50004800 2024-04-25T10:03:15.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.729+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c068f60 con 0x7fcb50004800 2024-04-25T10:03:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.730+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb5401b070 con 0x7fcb50004800 2024-04-25T10:03:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.731+0000 7fcb62a00640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb4806b500 0x7fcb4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.731+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fcb54089d40 con 0x7fcb50004800 2024-04-25T10:03:15.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.732+0000 7fcb6a000640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb4806b500 0x7fcb4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.733+0000 7fcb6a000640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb4806b500 0x7fcb4806d9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c035000 tx=0x7fcb5c02f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.735+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb54058160 con 0x7fcb50004800 2024-04-25T10:03:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.935+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb6c0cb4f0 con 0x7fcb50004800 2024-04-25T10:03:15.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.936+0000 7fcb62a00640 1 -- 172.21.15.5:0/687073529 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb5405c200 con 0x7fcb50004800 2024-04-25T10:03:15.940 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb4806b500 msgr2=0x7fcb4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb4806b500 0x7fcb4806d9c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c035000 tx=0x7fcb5c02f000 comp rx=0 tx=0).stop 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 msgr2=0x7fcb6c0cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb6c0cafb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d400 tx=0x7fcb5400d8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 shutdown_connections 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb4806b500 0x7fcb4806d9c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb6c0b3160 0x7fcb6c0b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb6c143a50 0x7fcb6c143eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 --2- 172.21.15.5:0/687073529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb50004800 0x7fcb6c0cafb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 >> 172.21.15.5:0/687073529 conn(0x7fcb6c06ab60 msgr2=0x7fcb6c0a9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 shutdown_connections 2024-04-25T10:03:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:15.939+0000 7fcb71800640 1 -- 172.21.15.5:0/687073529 wait complete. 2024-04-25T10:03:15.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.166+0000 7f4ef6000640 1 Processor -- start 2024-04-25T10:03:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.166+0000 7f4ef6000640 1 -- start start 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.168+0000 7f4ef6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ef01448f0 0x7f4ef00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.168+0000 7f4ef6000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ef015ce20 0x7f4ef015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.168+0000 7f4ef6000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef0144cd0 con 0x7f4ef015bc30 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.168+0000 7f4ef6000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef015f460 con 0x7f4ef01448f0 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.168+0000 7f4ef6000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef015c440 con 0x7f4ef015ce20 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ef01448f0 0x7f4ef00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ef01448f0 0x7f4ef00cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37988/0 (socket says 172.21.15.5:37988) 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 -- 172.21.15.5:0/3060197708 learned_addr learned my addr 172.21.15.5:0/3060197708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef5600640 1 --2- 172.21.15.5:0/3060197708 >> v2:172.21.15.80:3300/0 conn(0x7f4ef015ce20 0x7f4ef015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 -- 172.21.15.5:0/3060197708 >> v2:172.21.15.80:3300/0 conn(0x7f4ef015ce20 msgr2=0x7f4ef015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 --2- 172.21.15.5:0/3060197708 >> v2:172.21.15.80:3300/0 conn(0x7f4ef015ce20 0x7f4ef015d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 -- 172.21.15.5:0/3060197708 >> v1:172.21.15.80:6789/0 conn(0x7f4ef015bc30 legacy=0x7f4ef015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.169+0000 7f4ef4c00640 1 -- 172.21.15.5:0/3060197708 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef015c720 con 0x7f4ef01448f0 2024-04-25T10:03:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.170+0000 7f4ef4c00640 1 --2- 172.21.15.5:0/3060197708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef01448f0 0x7f4ef00cdaa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4ef006bc00 tx=0x7f4ee402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ee1eaa1a899a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.171+0000 7f4eef400640 1 -- 172.21.15.5:0/3060197708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4037070 con 0x7f4ef01448f0 2024-04-25T10:03:16.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.171+0000 7f4eef400640 1 -- 172.21.15.5:0/3060197708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4032440 con 0x7f4ef01448f0 2024-04-25T10:03:16.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.171+0000 7f4eef400640 1 -- 172.21.15.5:0/3060197708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4032810 con 0x7f4ef01448f0 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 -- 172.21.15.5:0/3060197708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef01448f0 msgr2=0x7f4ef00cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 --2- 172.21.15.5:0/3060197708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef01448f0 0x7f4ef00cdaa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4ef006bc00 tx=0x7f4ee402a230 comp rx=0 tx=0).stop 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 -- 172.21.15.5:0/3060197708 shutdown_connections 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 --2- 172.21.15.5:0/3060197708 >> v2:172.21.15.80:3300/0 conn(0x7f4ef015ce20 0x7f4ef015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 --2- 172.21.15.5:0/3060197708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef01448f0 0x7f4ef00cdaa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 -- 172.21.15.5:0/3060197708 >> 172.21.15.5:0/3060197708 conn(0x7f4ef0066f20 msgr2=0x7f4ef006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 -- 172.21.15.5:0/3060197708 shutdown_connections 2024-04-25T10:03:16.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.172+0000 7f4ef6000640 1 -- 172.21.15.5:0/3060197708 wait complete. 2024-04-25T10:03:16.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.174+0000 7f4ef6000640 1 Processor -- start 2024-04-25T10:03:16.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.174+0000 7f4ef6000640 1 -- start start 2024-04-25T10:03:16.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01448f0 0x7f4ef0130640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 0x7f4ef01f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef6000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ef015ce20 0x7f4ef01f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef6000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0160000 con 0x7f4ef015bc30 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef6000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef015c0c0 con 0x7f4ef01448f0 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef6000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef0160180 con 0x7f4ef015ce20 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4eefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 0x7f4ef01f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4eefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 0x7f4ef01f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37996/0 (socket says 172.21.15.5:37996) 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4eefe00640 1 -- 172.21.15.5:0/723601380 learned_addr learned my addr 172.21.15.5:0/723601380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:16.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef4c00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01448f0 0x7f4ef0130640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4eefe00640 1 -- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ef015ce20 msgr2=0x7f4ef01f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.176+0000 7f4ef5600640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ef015ce20 0x7f4ef01f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eefe00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ef015ce20 0x7f4ef01f75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eefe00640 1 -- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01448f0 msgr2=0x7f4ef0130640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eefe00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01448f0 0x7f4ef0130640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eefe00640 1 -- 172.21.15.5:0/723601380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0141b00 con 0x7f4ef015bc30 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4ef4c00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01448f0 0x7f4ef0130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eefe00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 0x7f4ef01f70a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c410 tx=0x7f4edc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc017070 con 0x7f4ef015bc30 2024-04-25T10:03:16.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4edc0035f0 con 0x7f4ef015bc30 2024-04-25T10:03:16.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc0039e0 con 0x7f4ef015bc30 2024-04-25T10:03:16.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef00b0230 con 0x7f4ef015bc30 2024-04-25T10:03:16.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.177+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef00b07f0 con 0x7f4ef015bc30 2024-04-25T10:03:16.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.179+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef0068f60 con 0x7f4ef015bc30 2024-04-25T10:03:16.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.180+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4edc003b80 con 0x7f4ef015bc30 2024-04-25T10:03:16.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.181+0000 7f4eecc00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4ed006b4d0 0x7f4ed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.181+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f4edc089920 con 0x7f4ef015bc30 2024-04-25T10:03:16.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.181+0000 7f4ef4c00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4ed006b4d0 0x7f4ed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.183+0000 7f4ef4c00640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4ed006b4d0 0x7f4ed006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4034000 tx=0x7f4ee402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.186+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4edc058390 con 0x7f4ef015bc30 2024-04-25T10:03:16.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.387+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ef0072180 con 0x7f4ef015bc30 2024-04-25T10:03:16.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.388+0000 7f4eecc00640 1 -- 172.21.15.5:0/723601380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4edc0010c0 con 0x7f4ef015bc30 2024-04-25T10:03:16.392 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4ed006b4d0 msgr2=0x7f4ed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4ed006b4d0 0x7f4ed006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4034000 tx=0x7f4ee402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 msgr2=0x7f4ef01f70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 0x7f4ef01f70a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00c410 tx=0x7f4edc00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 shutdown_connections 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ef015ce20 0x7f4ef01f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef015bc30 0x7f4ef01f70a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4ed006b4d0 0x7f4ed006d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 --2- 172.21.15.5:0/723601380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef01448f0 0x7f4ef0130640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.390+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 >> 172.21.15.5:0/723601380 conn(0x7f4ef0066f20 msgr2=0x7f4ef00cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.391+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 shutdown_connections 2024-04-25T10:03:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.391+0000 7f4ef6000640 1 -- 172.21.15.5:0/723601380 wait complete. 2024-04-25T10:03:16.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:16.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.622+0000 7f0955800640 1 Processor -- start 2024-04-25T10:03:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.622+0000 7f0955800640 1 -- start start 2024-04-25T10:03:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.623+0000 7f0955800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09500b48c0 0x7f095015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.623+0000 7f0955800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0950161d90 0x7f09500b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.623+0000 7f0955800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09501606e0 con 0x7f09500d3f90 2024-04-25T10:03:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.623+0000 7f0955800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09501609c0 con 0x7f09500b48c0 2024-04-25T10:03:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.623+0000 7f0955800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0950160ca0 con 0x7f0950161d90 2024-04-25T10:03:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.624+0000 7f094fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09500b48c0 0x7f095015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.624+0000 7f094fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09500b48c0 0x7f095015fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:57044/0 (socket says 172.21.15.5:57044) 2024-04-25T10:03:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.624+0000 7f094fe00640 1 -- 172.21.15.5:0/1254270947 learned_addr learned my addr 172.21.15.5:0/1254270947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.624+0000 7f0954e00640 1 --2- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0950161d90 0x7f09500b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.625+0000 7f094fe00640 1 -- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0950161d90 msgr2=0x7f09500b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.625+0000 7f094fe00640 1 --2- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0950161d90 0x7f09500b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.625+0000 7f094fe00640 1 -- 172.21.15.5:0/1254270947 >> v1:172.21.15.80:6789/0 conn(0x7f09500d3f90 legacy=0x7f0950161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.625+0000 7f094fe00640 1 -- 172.21.15.5:0/1254270947 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950160f80 con 0x7f09500b48c0 2024-04-25T10:03:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.626+0000 7f094fe00640 1 --2- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500b48c0 0x7f095015fe60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0940009870 tx=0x7f094002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9091d7386b75877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.627+0000 7f094f400640 1 -- 172.21.15.5:0/1254270947 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940037070 con 0x7f09500b48c0 2024-04-25T10:03:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.627+0000 7f094f400640 1 -- 172.21.15.5:0/1254270947 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940032440 con 0x7f09500b48c0 2024-04-25T10:03:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.627+0000 7f094f400640 1 -- 172.21.15.5:0/1254270947 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940032810 con 0x7f09500b48c0 2024-04-25T10:03:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 -- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500b48c0 msgr2=0x7f095015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 --2- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500b48c0 0x7f095015fe60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0940009870 tx=0x7f094002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 -- 172.21.15.5:0/1254270947 shutdown_connections 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 --2- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0950161d90 0x7f09500b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 --2- 172.21.15.5:0/1254270947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500b48c0 0x7f095015fe60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 -- 172.21.15.5:0/1254270947 >> 172.21.15.5:0/1254270947 conn(0x7f095008af80 msgr2=0x7f09500d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 -- 172.21.15.5:0/1254270947 shutdown_connections 2024-04-25T10:03:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.628+0000 7f0955800640 1 -- 172.21.15.5:0/1254270947 wait complete. 2024-04-25T10:03:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.630+0000 7f0955800640 1 Processor -- start 2024-04-25T10:03:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.630+0000 7f0955800640 1 -- start start 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0955800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09500b48c0 0x7f09500c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0955800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500d3f90 0x7f0950149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0955800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 0x7f0950149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0955800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09501603a0 con 0x7f0950161d90 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0955800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500b4ca0 con 0x7f09500d3f90 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0955800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500b30b0 con 0x7f09500b48c0 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0954e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 0x7f0950149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0954e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 0x7f0950149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38002/0 (socket says 172.21.15.5:38002) 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0954e00640 1 -- 172.21.15.5:0/3160561685 learned_addr learned my addr 172.21.15.5:0/3160561685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f094fe00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09500b48c0 0x7f09500c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0954e00640 1 -- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09500b48c0 msgr2=0x7f09500c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.631+0000 7f0954e00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09500b48c0 0x7f09500c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0954e00640 1 -- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500d3f90 msgr2=0x7f0950149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0947400640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500d3f90 0x7f0950149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0954e00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500d3f90 0x7f0950149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0954e00640 1 -- 172.21.15.5:0/3160561685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500cbfe0 con 0x7f0950161d90 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0947400640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500d3f90 0x7f0950149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0954e00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 0x7f0950149e60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f094800ef90 tx=0x7f094800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948018020 con 0x7f0950161d90 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0948014480 con 0x7f0950161d90 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948009e60 con 0x7f0950161d90 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09500c9480 con 0x7f0950161d90 2024-04-25T10:03:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.632+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09500c9a40 con 0x7f0950161d90 2024-04-25T10:03:16.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.634+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950068f60 con 0x7f0950161d90 2024-04-25T10:03:16.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.634+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0948005d10 con 0x7f0950161d90 2024-04-25T10:03:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.636+0000 7f094cc00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f092c06b4d0 0x7f092c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.636+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f094808b2f0 con 0x7f0950161d90 2024-04-25T10:03:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.637+0000 7f094fe00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f092c06b4d0 0x7f092c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.638+0000 7f094fe00640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f092c06b4d0 0x7f092c06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0940034000 tx=0x7f094002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.641+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0948022030 con 0x7f0950161d90 2024-04-25T10:03:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.841+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09500c6d80 con 0x7f0950161d90 2024-04-25T10:03:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.842+0000 7f094cc00640 1 -- 172.21.15.5:0/3160561685 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0948010040 con 0x7f0950161d90 2024-04-25T10:03:16.847 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f092c06b4d0 msgr2=0x7f092c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f092c06b4d0 0x7f092c06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0940034000 tx=0x7f094002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 msgr2=0x7f0950149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 0x7f0950149e60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f094800ef90 tx=0x7f094800c490 comp rx=0 tx=0).stop 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 shutdown_connections 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0950161d90 0x7f0950149e60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f092c06b4d0 0x7f092c06d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09500d3f90 0x7f0950149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 --2- 172.21.15.5:0/3160561685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09500b48c0 0x7f09500c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 >> 172.21.15.5:0/3160561685 conn(0x7f095008af80 msgr2=0x7f095006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 shutdown_connections 2024-04-25T10:03:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:16.845+0000 7f0955800640 1 -- 172.21.15.5:0/3160561685 wait complete. 2024-04-25T10:03:16.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.074+0000 7f42b3e00640 1 Processor -- start 2024-04-25T10:03:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.075+0000 7f42b3e00640 1 -- start start 2024-04-25T10:03:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.076+0000 7f42b3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42b40d3f90 0x7f42b4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.076+0000 7f42b3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f42b4161bc0 0x7f42b40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.076+0000 7f42b3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b41606e0 con 0x7f42b40b48c0 2024-04-25T10:03:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.076+0000 7f42b3e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b4161ff0 con 0x7f42b4161bc0 2024-04-25T10:03:17.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.076+0000 7f42b3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b41609c0 con 0x7f42b40d3f90 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f42b4161bc0 0x7f42b40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f42b4161bc0 0x7f42b40b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44600/0 (socket says 172.21.15.5:44600) 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 -- 172.21.15.5:0/348173513 learned_addr learned my addr 172.21.15.5:0/348173513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42abe00640 1 --2- 172.21.15.5:0/348173513 >> v2:172.21.15.80:3300/0 conn(0x7f42b40d3f90 0x7f42b4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 -- 172.21.15.5:0/348173513 >> v2:172.21.15.80:3300/0 conn(0x7f42b40d3f90 msgr2=0x7f42b4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 --2- 172.21.15.5:0/348173513 >> v2:172.21.15.80:3300/0 conn(0x7f42b40d3f90 0x7f42b4161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 -- 172.21.15.5:0/348173513 >> v1:172.21.15.80:6789/0 conn(0x7f42b40b48c0 legacy=0x7f42b415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.077+0000 7f42b3400640 1 -- 172.21.15.5:0/348173513 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b4160ca0 con 0x7f42b4161bc0 2024-04-25T10:03:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.079+0000 7f42b3400640 1 --2- 172.21.15.5:0/348173513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b4161bc0 0x7f42b40b2b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f42b40b61b0 tx=0x7f42ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70102b80ab24dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.079+0000 7f42b2000640 1 -- 172.21.15.5:0/348173513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac036070 con 0x7f42b4161bc0 2024-04-25T10:03:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.079+0000 7f42b2000640 1 -- 172.21.15.5:0/348173513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac031600 con 0x7f42b4161bc0 2024-04-25T10:03:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.079+0000 7f42b2000640 1 -- 172.21.15.5:0/348173513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0319d0 con 0x7f42b4161bc0 2024-04-25T10:03:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.079+0000 7f42b3e00640 1 -- 172.21.15.5:0/348173513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b4161bc0 msgr2=0x7f42b40b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.079+0000 7f42b3e00640 1 --2- 172.21.15.5:0/348173513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b4161bc0 0x7f42b40b2b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f42b40b61b0 tx=0x7f42ac02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.080+0000 7f42b3e00640 1 -- 172.21.15.5:0/348173513 shutdown_connections 2024-04-25T10:03:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.080+0000 7f42b3e00640 1 --2- 172.21.15.5:0/348173513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b4161bc0 0x7f42b40b2b70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.080+0000 7f42b3e00640 1 --2- 172.21.15.5:0/348173513 >> v2:172.21.15.80:3300/0 conn(0x7f42b40d3f90 0x7f42b4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.080+0000 7f42b3e00640 1 -- 172.21.15.5:0/348173513 >> 172.21.15.5:0/348173513 conn(0x7f42b408af80 msgr2=0x7f42b40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.080+0000 7f42b3e00640 1 -- 172.21.15.5:0/348173513 shutdown_connections 2024-04-25T10:03:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.080+0000 7f42b3e00640 1 -- 172.21.15.5:0/348173513 wait complete. 2024-04-25T10:03:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.081+0000 7f42b3e00640 1 Processor -- start 2024-04-25T10:03:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.081+0000 7f42b3e00640 1 -- start start 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.082+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 0x7f42b40b8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.082+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42b40d3f90 0x7f42b40c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.082+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42b4161bc0 0x7f42b415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.082+0000 7f42b3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40b4ca0 con 0x7f42b40b48c0 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.082+0000 7f42b3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b4161260 con 0x7f42b4161bc0 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.082+0000 7f42b3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b40b30b0 con 0x7f42b40d3f90 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 0x7f42b40b8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 0x7f42b40b8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44606/0 (socket says 172.21.15.5:44606) 2024-04-25T10:03:17.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 -- 172.21.15.5:0/1962989828 learned_addr learned my addr 172.21.15.5:0/1962989828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b3400640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42b4161bc0 0x7f42b415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42abe00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42b40d3f90 0x7f42b40c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 -- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42b40d3f90 msgr2=0x7f42b40c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42b40d3f90 0x7f42b40c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 -- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42b4161bc0 msgr2=0x7f42b415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42b4161bc0 0x7f42b415e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.083+0000 7f42b2a00640 1 -- 172.21.15.5:0/1962989828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b40c8490 con 0x7f42b40b48c0 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.084+0000 7f42b2a00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 0x7f42b40b8860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c450 tx=0x7f42a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.084+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40052d0 con 0x7f42b40b48c0 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.084+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4005470 con 0x7f42b40b48c0 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.084+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4012870 con 0x7f42b40b48c0 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.084+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b40c8770 con 0x7f42b40b48c0 2024-04-25T10:03:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.084+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b40c8d30 con 0x7f42b40b48c0 2024-04-25T10:03:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.085+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b4068f60 con 0x7f42b40b48c0 2024-04-25T10:03:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.086+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42a4012a10 con 0x7f42b40b48c0 2024-04-25T10:03:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.087+0000 7f42aaa00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f429006b4d0 0x7f429006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.087+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f42a408a1c0 con 0x7f42b40b48c0 2024-04-25T10:03:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.088+0000 7f42abe00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f429006b4d0 0x7f429006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.089+0000 7f42abe00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f429006b4d0 0x7f429006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f42b415f4c0 tx=0x7f429c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.091+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42a4058cc0 con 0x7f42b40b48c0 2024-04-25T10:03:17.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.286+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f42b408d730 con 0x7f42b40b48c0 2024-04-25T10:03:17.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.287+0000 7f42aaa00640 1 -- 172.21.15.5:0/1962989828 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f42a40585c0 con 0x7f42b40b48c0 2024-04-25T10:03:17.291 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f429006b4d0 msgr2=0x7f429006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f429006b4d0 0x7f429006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f42b415f4c0 tx=0x7f429c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 msgr2=0x7f42b40b8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 0x7f42b40b8860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c450 tx=0x7f42a400c910 comp rx=0 tx=0).stop 2024-04-25T10:03:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 shutdown_connections 2024-04-25T10:03:17.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f429006b4d0 0x7f429006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42b4161bc0 0x7f42b415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42b40d3f90 0x7f42b40c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 --2- 172.21.15.5:0/1962989828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42b40b48c0 0x7f42b40b8860 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.289+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 >> 172.21.15.5:0/1962989828 conn(0x7f42b408af80 msgr2=0x7f42b40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.290+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 shutdown_connections 2024-04-25T10:03:17.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.290+0000 7f42b3e00640 1 -- 172.21.15.5:0/1962989828 wait complete. 2024-04-25T10:03:17.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.511+0000 7f3a42800640 1 Processor -- start 2024-04-25T10:03:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.511+0000 7f3a42800640 1 -- start start 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a42800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a3c1529e0 0x7f3a3c0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a42800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a42800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c15bdc0 con 0x7f3a3c071ed0 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a42800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c0b2340 con 0x7f3a3c1529e0 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a42800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c15c060 con 0x7f3a3c0c6410 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a3b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a3c1529e0 0x7f3a3c0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a3b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a3c1529e0 0x7f3a3c0c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44620/0 (socket says 172.21.15.5:44620) 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a3b400640 1 -- 172.21.15.5:0/1131243408 learned_addr learned my addr 172.21.15.5:0/1131243408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a3b400640 1 -- 172.21.15.5:0/1131243408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 msgr2=0x7f3a3c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a3aa00640 1 --2- 172.21.15.5:0/1131243408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.513+0000 7f3a3b400640 1 --2- 172.21.15.5:0/1131243408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.514+0000 7f3a3b400640 1 -- 172.21.15.5:0/1131243408 >> v1:172.21.15.80:6789/0 conn(0x7f3a3c071ed0 legacy=0x7f3a3c0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.514+0000 7f3a3b400640 1 -- 172.21.15.5:0/1131243408 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c15c340 con 0x7f3a3c1529e0 2024-04-25T10:03:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.515+0000 7f3a3b400640 1 --2- 172.21.15.5:0/1131243408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1529e0 0x7f3a3c0c5ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009870 tx=0x7f3a2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb89f1606c6788aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.515+0000 7f3a3a000640 1 -- 172.21.15.5:0/1131243408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c037070 con 0x7f3a3c1529e0 2024-04-25T10:03:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.515+0000 7f3a3a000640 1 -- 172.21.15.5:0/1131243408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c032440 con 0x7f3a3c1529e0 2024-04-25T10:03:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.515+0000 7f3a3a000640 1 -- 172.21.15.5:0/1131243408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c032810 con 0x7f3a3c1529e0 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 -- 172.21.15.5:0/1131243408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1529e0 msgr2=0x7f3a3c0c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 --2- 172.21.15.5:0/1131243408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1529e0 0x7f3a3c0c5ed0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c009870 tx=0x7f3a2c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 -- 172.21.15.5:0/1131243408 shutdown_connections 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 --2- 172.21.15.5:0/1131243408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 --2- 172.21.15.5:0/1131243408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1529e0 0x7f3a3c0c5ed0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 -- 172.21.15.5:0/1131243408 >> 172.21.15.5:0/1131243408 conn(0x7f3a3c071150 msgr2=0x7f3a3c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 -- 172.21.15.5:0/1131243408 shutdown_connections 2024-04-25T10:03:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.516+0000 7f3a42800640 1 -- 172.21.15.5:0/1131243408 wait complete. 2024-04-25T10:03:17.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.518+0000 7f3a42800640 1 Processor -- start 2024-04-25T10:03:17.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.519+0000 7f3a42800640 1 -- start start 2024-04-25T10:03:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a42800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3c071ed0 0x7f3a3c158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a42800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a42800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 0x7f3a3c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a42800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c06e160 con 0x7f3a3c1544d0 2024-04-25T10:03:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a42800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c15c900 con 0x7f3a3c071ed0 2024-04-25T10:03:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a42800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c06e2e0 con 0x7f3a3c0c6410 2024-04-25T10:03:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 0x7f3a3c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 0x7f3a3c156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44632/0 (socket says 172.21.15.5:44632) 2024-04-25T10:03:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a3be00640 1 -- 172.21.15.5:0/3417756193 learned_addr learned my addr 172.21.15.5:0/3417756193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:17.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a3b400640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3c071ed0 0x7f3a3c158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.520+0000 7f3a3aa00640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a3be00640 1 -- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 msgr2=0x7f3a3c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a3be00640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a3be00640 1 -- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3c071ed0 msgr2=0x7f3a3c158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a3be00640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3c071ed0 0x7f3a3c158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a3be00640 1 -- 172.21.15.5:0/3417756193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0b2970 con 0x7f3a3c1544d0 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a3be00640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 0x7f3a3c156350 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b9e0 tx=0x7f3a3400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34003b70 con 0x7f3a3c1544d0 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a34003d10 con 0x7f3a3c1544d0 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400cb00 con 0x7f3a3c1544d0 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0b2c50 con 0x7f3a3c1544d0 2024-04-25T10:03:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.521+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0b3210 con 0x7f3a3c1544d0 2024-04-25T10:03:17.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.523+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c068fd0 con 0x7f3a3c1544d0 2024-04-25T10:03:17.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.523+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a3400cca0 con 0x7f3a3c1544d0 2024-04-25T10:03:17.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.524+0000 7f3a33400640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.525+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f3a3408a3e0 con 0x7f3a3c1544d0 2024-04-25T10:03:17.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.525+0000 7f3a3b400640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.526+0000 7f3a3b400640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c034000 tx=0x7f3a2c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.528+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a34058e50 con 0x7f3a3c1544d0 2024-04-25T10:03:17.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.730+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a3c0b1810 con 0x7f3a3c1544d0 2024-04-25T10:03:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.730+0000 7f3a33400640 1 -- 172.21.15.5:0/3417756193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3a34058750 con 0x7f3a3c1544d0 2024-04-25T10:03:17.735 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 msgr2=0x7f3a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c034000 tx=0x7f3a2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 msgr2=0x7f3a3c156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 0x7f3a3c156350 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b9e0 tx=0x7f3a3400bea0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 shutdown_connections 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3c1544d0 0x7f3a3c156350 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a3c0c6410 0x7f3a3c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 --2- 172.21.15.5:0/3417756193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3c071ed0 0x7f3a3c158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 >> 172.21.15.5:0/3417756193 conn(0x7f3a3c071150 msgr2=0x7f3a3c1598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 shutdown_connections 2024-04-25T10:03:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:17.733+0000 7f3a42800640 1 -- 172.21.15.5:0/3417756193 wait complete. 2024-04-25T10:03:17.756 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:03:17.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:17.757 DEBUG:teuthology.orchestra.run.smithi080:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:03:17.854 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:03:17.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:17.854 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:03:17.902 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:03:17.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.113+0000 7fc6bf200640 1 Processor -- start 2024-04-25T10:03:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.114+0000 7fc6bf200640 1 -- start start 2024-04-25T10:03:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.115+0000 7fc6bf200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6c00b4bb0 0x7fc6c00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.115+0000 7fc6bf200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00cf410 con 0x7fc6c00d07d0 2024-04-25T10:03:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.115+0000 7fc6bf200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00cf6f0 con 0x7fc6c0161200 2024-04-25T10:03:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.115+0000 7fc6bf200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c00b28b0 con 0x7fc6c00b4bb0 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.116+0000 7fc6bd400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc6c00d07d0 0x7fc6c00b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58816/0 (socket says 172.21.15.5:58816) 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.116+0000 7fc6bd400640 1 -- 172.21.15.5:0/767767174 learned_addr learned my addr 172.21.15.5:0/767767174 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.116+0000 7fc6be800640 1 --2- 172.21.15.5:0/767767174 >> v2:172.21.15.5:3300/0 conn(0x7fc6c00b4bb0 0x7fc6c00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.117+0000 7fc6be800640 1 -- 172.21.15.5:0/767767174 >> v1:172.21.15.80:6789/0 conn(0x7fc6c0161200 legacy=0x7fc6c00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.117+0000 7fc6be800640 1 -- 172.21.15.5:0/767767174 >> v1:172.21.15.5:6789/0 conn(0x7fc6c00d07d0 legacy=0x7fc6c00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.117+0000 7fc6bca00640 1 -- 172.21.15.5:0/767767174 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159363500 0 0) 0x7fc6c00cf410 con 0x7fc6c00d07d0 2024-04-25T10:03:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.117+0000 7fc6be800640 1 -- 172.21.15.5:0/767767174 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00d3fd0 con 0x7fc6c00b4bb0 2024-04-25T10:03:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.118+0000 7fc6be800640 1 --2- 172.21.15.5:0/767767174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c00b4bb0 0x7fc6c00d2380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80097a0 tx=0x7fc6b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507e55b06a5f6faf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bca00640 1 -- 172.21.15.5:0/767767174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8037070 con 0x7fc6c00b4bb0 2024-04-25T10:03:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bca00640 1 -- 172.21.15.5:0/767767174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8032440 con 0x7fc6c00b4bb0 2024-04-25T10:03:18.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bca00640 1 -- 172.21.15.5:0/767767174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8032810 con 0x7fc6c00b4bb0 2024-04-25T10:03:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bf200640 1 -- 172.21.15.5:0/767767174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c00b4bb0 msgr2=0x7fc6c00d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bf200640 1 --2- 172.21.15.5:0/767767174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c00b4bb0 0x7fc6c00d2380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80097a0 tx=0x7fc6b802a230 comp rx=0 tx=0).stop 2024-04-25T10:03:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bf200640 1 -- 172.21.15.5:0/767767174 shutdown_connections 2024-04-25T10:03:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bf200640 1 --2- 172.21.15.5:0/767767174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c00b4bb0 0x7fc6c00d2380 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.119+0000 7fc6bf200640 1 -- 172.21.15.5:0/767767174 >> 172.21.15.5:0/767767174 conn(0x7fc6c008af80 msgr2=0x7fc6c00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.120+0000 7fc6bf200640 1 -- 172.21.15.5:0/767767174 shutdown_connections 2024-04-25T10:03:18.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.120+0000 7fc6bf200640 1 -- 172.21.15.5:0/767767174 wait complete. 2024-04-25T10:03:18.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.121+0000 7fc6bf200640 1 Processor -- start 2024-04-25T10:03:18.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.122+0000 7fc6bf200640 1 -- start start 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6bf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6c00b4bb0 0x7fc6c015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6bf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d07d0 0x7fc6c00c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6bf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 0x7fc6c006bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6bf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00b3810 con 0x7fc6c0161200 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6bf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00b3690 con 0x7fc6c00d07d0 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6bf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00b3990 con 0x7fc6c00b4bb0 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 0x7fc6c006bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 0x7fc6c006bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44650/0 (socket says 172.21.15.5:44650) 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 -- 172.21.15.5:0/2368703526 learned_addr learned my addr 172.21.15.5:0/2368703526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 -- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6c00b4bb0 msgr2=0x7fc6c015c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6c00b4bb0 0x7fc6c015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 -- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d07d0 msgr2=0x7fc6c00c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d07d0 0x7fc6c00c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.123+0000 7fc6be800640 1 -- 172.21.15.5:0/2368703526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c015c5f0 con 0x7fc6c0161200 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.124+0000 7fc6be800640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 0x7fc6c006bdd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80098c0 tx=0x7fc6b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.124+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8037040 con 0x7fc6c0161200 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.124+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8042070 con 0x7fc6c0161200 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.124+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8030b30 con 0x7fc6c0161200 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.124+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c00ca1a0 con 0x7fc6c0161200 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.124+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c00cbab0 con 0x7fc6c0161200 2024-04-25T10:03:18.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.126+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0068f60 con 0x7fc6c0161200 2024-04-25T10:03:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.126+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6b8030cd0 con 0x7fc6c0161200 2024-04-25T10:03:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.127+0000 7fc6b6000640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc69c06b550 0x7fc69c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.127+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fc6b80ab170 con 0x7fc6c0161200 2024-04-25T10:03:18.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.128+0000 7fc6bde00640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc69c06b550 0x7fc69c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.129+0000 7fc6bde00640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc69c06b550 0x7fc69c06da10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000f60 tx=0x7fc6b00079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.132+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6b8045360 con 0x7fc6c0161200 2024-04-25T10:03:18.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.374+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c00b5030 con 0x7fc6c0161200 2024-04-25T10:03:18.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.375+0000 7fc6b6000640 1 -- 172.21.15.5:0/2368703526 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7fc6b8079c60 con 0x7fc6c0161200 2024-04-25T10:03:18.380 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:18.380 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:18.380 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc69c06b550 msgr2=0x7fc69c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc69c06b550 0x7fc69c06da10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0000f60 tx=0x7fc6b00079b0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 msgr2=0x7fc6c006bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 0x7fc6c006bdd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80098c0 tx=0x7fc6b80046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 shutdown_connections 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6c0161200 0x7fc6c006bdd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc69c06b550 0x7fc69c06da10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d07d0 0x7fc6c00c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 --2- 172.21.15.5:0/2368703526 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc6c00b4bb0 0x7fc6c015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 >> 172.21.15.5:0/2368703526 conn(0x7fc6c008af80 msgr2=0x7fc6c00ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 shutdown_connections 2024-04-25T10:03:18.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.378+0000 7fc6bf200640 1 -- 172.21.15.5:0/2368703526 wait complete. 2024-04-25T10:03:18.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.603+0000 7f70e9400640 1 Processor -- start 2024-04-25T10:03:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.604+0000 7f70e9400640 1 -- start start 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e9400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f70e40b5e60 0x7f70e406bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70e40b6a40 0x7f70e406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e9400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e406ca30 con 0x7f70e415dd00 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e9400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e415e0e0 con 0x7f70e40b6a40 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e9400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e406cfe0 con 0x7f70e40b5e60 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70e40b6a40 0x7f70e406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.605+0000 7f70e2800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f70e415dd00 0x7f70e40b5750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58826/0 (socket says 172.21.15.5:58826) 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.606+0000 7f70e2800640 1 -- 172.21.15.5:0/2439252519 learned_addr learned my addr 172.21.15.5:0/2439252519 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.606+0000 7f70e3200640 1 -- 172.21.15.5:0/2439252519 >> v2:172.21.15.80:3300/0 conn(0x7f70e40b5e60 msgr2=0x7f70e406bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.606+0000 7f70e3200640 1 --2- 172.21.15.5:0/2439252519 >> v2:172.21.15.80:3300/0 conn(0x7f70e40b5e60 0x7f70e406bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.606+0000 7f70e3200640 1 -- 172.21.15.5:0/2439252519 >> v1:172.21.15.5:6789/0 conn(0x7f70e415dd00 legacy=0x7f70e40b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.606+0000 7f70e3200640 1 -- 172.21.15.5:0/2439252519 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e406cd00 con 0x7f70e40b6a40 2024-04-25T10:03:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.607+0000 7f70e3200640 1 --2- 172.21.15.5:0/2439252519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e406c4f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f70e408c540 tx=0x7f70dc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39235be99c6b98a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.607+0000 7f70e1400640 1 -- 172.21.15.5:0/2439252519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc007740 con 0x7f70e40b6a40 2024-04-25T10:03:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e1400640 1 -- 172.21.15.5:0/2439252519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0078e0 con 0x7f70e40b6a40 2024-04-25T10:03:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e1400640 1 -- 172.21.15.5:0/2439252519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc007cb0 con 0x7f70e40b6a40 2024-04-25T10:03:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 -- 172.21.15.5:0/2439252519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 msgr2=0x7f70e406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 --2- 172.21.15.5:0/2439252519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e406c4f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f70e408c540 tx=0x7f70dc02a710 comp rx=0 tx=0).stop 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 -- 172.21.15.5:0/2439252519 shutdown_connections 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 --2- 172.21.15.5:0/2439252519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e406c4f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 --2- 172.21.15.5:0/2439252519 >> v2:172.21.15.80:3300/0 conn(0x7f70e40b5e60 0x7f70e406bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 -- 172.21.15.5:0/2439252519 >> 172.21.15.5:0/2439252519 conn(0x7f70e40d07b0 msgr2=0x7f70e40d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 -- 172.21.15.5:0/2439252519 shutdown_connections 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.608+0000 7f70e9400640 1 -- 172.21.15.5:0/2439252519 wait complete. 2024-04-25T10:03:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.610+0000 7f70e9400640 1 Processor -- start 2024-04-25T10:03:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.610+0000 7f70e9400640 1 -- start start 2024-04-25T10:03:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e9400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70e40b5e60 0x7f70e4152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70e415dd00 0x7f70e415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e9400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40d2260 con 0x7f70e40b6a40 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e9400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40d1f80 con 0x7f70e415dd00 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e9400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40d2780 con 0x7f70e40b5e60 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.611+0000 7f70e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44664/0 (socket says 172.21.15.5:44664) 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 -- 172.21.15.5:0/3935844336 learned_addr learned my addr 172.21.15.5:0/3935844336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e2800640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70e40b5e60 0x7f70e4152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e3200640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70e415dd00 0x7f70e415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 -- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70e40b5e60 msgr2=0x7f70e4152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70e40b5e60 0x7f70e4152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 -- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70e415dd00 msgr2=0x7f70e415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70e415dd00 0x7f70e415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 -- 172.21.15.5:0/3935844336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4155b80 con 0x7f70e40b6a40 2024-04-25T10:03:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.612+0000 7f70e1e00640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e41533f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b4d0 tx=0x7f70cc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.613+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc014070 con 0x7f70e40b6a40 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.613+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc010870 con 0x7f70e40b6a40 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.613+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc010c60 con 0x7f70e40b6a40 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.613+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e4155e60 con 0x7f70e40b6a40 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.613+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e4156420 con 0x7f70e40b6a40 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.614+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e4068fd0 con 0x7f70e40b6a40 2024-04-25T10:03:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.615+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70cc017070 con 0x7f70e40b6a40 2024-04-25T10:03:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.616+0000 7f70daa00640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f70c006b4d0 0x7f70c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.616+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f70cc08a390 con 0x7f70e40b6a40 2024-04-25T10:03:18.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.617+0000 7f70e2800640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f70c006b4d0 0x7f70c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.618+0000 7f70e2800640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f70c006b4d0 0x7f70c006d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000f60 tx=0x7f70d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.621+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70cc058e90 con 0x7f70e40b6a40 2024-04-25T10:03:18.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.821+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70e40d1620 con 0x7f70e40b6a40 2024-04-25T10:03:18.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.825+0000 7f70daa00640 1 -- 172.21.15.5:0/3935844336 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f70cc058790 con 0x7f70e40b6a40 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:18.830 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f70c006b4d0 msgr2=0x7f70c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f70c006b4d0 0x7f70c006d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000f60 tx=0x7f70d4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 msgr2=0x7f70e41533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e41533f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b4d0 tx=0x7f70cc00b990 comp rx=0 tx=0).stop 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 shutdown_connections 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f70c006b4d0 0x7f70c006d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70e415dd00 0x7f70e415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e40b6a40 0x7f70e41533f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 --2- 172.21.15.5:0/3935844336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70e40b5e60 0x7f70e4152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 >> 172.21.15.5:0/3935844336 conn(0x7f70e40d07b0 msgr2=0x7f70e40d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 shutdown_connections 2024-04-25T10:03:18.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:18.828+0000 7f70e9400640 1 -- 172.21.15.5:0/3935844336 wait complete. 2024-04-25T10:03:18.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:01:55.847408+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:01:56.507966+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:00.365944+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:18.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.068+0000 7f1391000640 1 Processor -- start 2024-04-25T10:03:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.069+0000 7f1391000640 1 -- start start 2024-04-25T10:03:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.070+0000 7f1391000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f138c143f30 0x7f138c06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.070+0000 7f1391000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c0ade10 con 0x7f138c071710 2024-04-25T10:03:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f1391000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c0ae0f0 con 0x7f138c06ec70 2024-04-25T10:03:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f1391000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c0ae3d0 con 0x7f138c143f30 2024-04-25T10:03:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f138b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f138c143f30 0x7f138c06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f138be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f138c071710 0x7f138c0b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58828/0 (socket says 172.21.15.5:58828) 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f138be00640 1 -- 172.21.15.5:0/601577911 learned_addr learned my addr 172.21.15.5:0/601577911 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f138aa00640 1 -- 172.21.15.5:0/601577911 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268500071 0 0) 0x7f138c0ade10 con 0x7f138c071710 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.071+0000 7f138aa00640 1 -- 172.21.15.5:0/601577911 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1370003520 con 0x7f138c071710 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.072+0000 7f138b400640 1 -- 172.21.15.5:0/601577911 >> v1:172.21.15.80:6789/0 conn(0x7f138c06ec70 legacy=0x7f138c071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.072+0000 7f138b400640 1 -- 172.21.15.5:0/601577911 >> v1:172.21.15.5:6789/0 conn(0x7f138c071710 legacy=0x7f138c0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.072+0000 7f138b400640 1 -- 172.21.15.5:0/601577911 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0ae6b0 con 0x7f138c143f30 2024-04-25T10:03:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.073+0000 7f138b400640 1 --2- 172.21.15.5:0/601577911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c143f30 0x7f138c06da30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f137c0097a0 tx=0x7f137c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53671a39de74103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.073+0000 7f138aa00640 1 -- 172.21.15.5:0/601577911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c037070 con 0x7f138c143f30 2024-04-25T10:03:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.073+0000 7f138aa00640 1 -- 172.21.15.5:0/601577911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c032440 con 0x7f138c143f30 2024-04-25T10:03:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.073+0000 7f138aa00640 1 -- 172.21.15.5:0/601577911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c032810 con 0x7f138c143f30 2024-04-25T10:03:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 -- 172.21.15.5:0/601577911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c143f30 msgr2=0x7f138c06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 --2- 172.21.15.5:0/601577911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c143f30 0x7f138c06da30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f137c0097a0 tx=0x7f137c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 -- 172.21.15.5:0/601577911 shutdown_connections 2024-04-25T10:03:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 --2- 172.21.15.5:0/601577911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c143f30 0x7f138c06da30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 -- 172.21.15.5:0/601577911 >> 172.21.15.5:0/601577911 conn(0x7f138c066f20 msgr2=0x7f138c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 -- 172.21.15.5:0/601577911 shutdown_connections 2024-04-25T10:03:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.074+0000 7f1391000640 1 -- 172.21.15.5:0/601577911 wait complete. 2024-04-25T10:03:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.076+0000 7f1391000640 1 Processor -- start 2024-04-25T10:03:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.076+0000 7f1391000640 1 -- start start 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f1391000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 0x7f138c132c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f1391000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f138c071710 0x7f138c1331b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f1391000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138c143f30 0x7f138c1f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f1391000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c06e0f0 con 0x7f138c06ec70 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f1391000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c06df70 con 0x7f138c143f30 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f1391000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0aebd0 con 0x7f138c071710 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f138b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 0x7f138c132c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f138b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 0x7f138c132c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.077+0000 7f138b400640 1 -- 172.21.15.5:0/3100323949 learned_addr learned my addr 172.21.15.5:0/3100323949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f1383e00640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f138c071710 0x7f138c1331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138be00640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138c143f30 0x7f138c1f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138b400640 1 -- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f138c071710 msgr2=0x7f138c1331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138b400640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f138c071710 0x7f138c1331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138b400640 1 -- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138c143f30 msgr2=0x7f138c1f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138b400640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138c143f30 0x7f138c1f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138b400640 1 -- 172.21.15.5:0/3100323949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c1e7180 con 0x7f138c06ec70 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138be00640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138c143f30 0x7f138c1f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.078+0000 7f138b400640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 0x7f138c132c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f137c000c00 tx=0x7f137c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.079+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c037040 con 0x7f138c06ec70 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.079+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c042070 con 0x7f138c06ec70 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.079+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c030940 con 0x7f138c06ec70 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.079+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c1e7460 con 0x7f138c06ec70 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.079+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c130ce0 con 0x7f138c06ec70 2024-04-25T10:03:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.080+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c068f60 con 0x7f138c06ec70 2024-04-25T10:03:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.081+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f137c030ae0 con 0x7f138c06ec70 2024-04-25T10:03:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.082+0000 7f1383400640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f136806b4d0 0x7f136806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.082+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f137c0aaf50 con 0x7f138c06ec70 2024-04-25T10:03:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.082+0000 7f1383e00640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f136806b4d0 0x7f136806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.084+0000 7f1383e00640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f136806b4d0 0x7f136806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f138c144fb0 tx=0x7f1374009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.086+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f137c045360 con 0x7f138c06ec70 2024-04-25T10:03:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.286+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f138c0b3b60 con 0x7f138c06ec70 2024-04-25T10:03:19.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.287+0000 7f1383400640 1 -- 172.21.15.5:0/3100323949 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f137c0799c0 con 0x7f138c06ec70 2024-04-25T10:03:19.292 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:19.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f136806b4d0 msgr2=0x7f136806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f136806b4d0 0x7f136806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f138c144fb0 tx=0x7f1374009040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 msgr2=0x7f138c132c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 0x7f138c132c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f137c000c00 tx=0x7f137c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 shutdown_connections 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f136806b4d0 0x7f136806d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138c143f30 0x7f138c1f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f138c071710 0x7f138c1331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 --2- 172.21.15.5:0/3100323949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f138c06ec70 0x7f138c132c70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.290+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 >> 172.21.15.5:0/3100323949 conn(0x7f138c066f20 msgr2=0x7f138c145220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.291+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 shutdown_connections 2024-04-25T10:03:19.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.291+0000 7f1391000640 1 -- 172.21.15.5:0/3100323949 wait complete. 2024-04-25T10:03:19.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:19.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.531+0000 7f1dde600640 1 Processor -- start 2024-04-25T10:03:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.531+0000 7f1dde600640 1 -- start start 2024-04-25T10:03:19.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.532+0000 7f1dde600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dd8166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.532+0000 7f1dde600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd80cc510 con 0x7f1dd80cbe50 2024-04-25T10:03:19.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.532+0000 7f1dde600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd80cc7f0 con 0x7f1dd815b7f0 2024-04-25T10:03:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.532+0000 7f1dde600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd815bbd0 con 0x7f1dd80cff40 2024-04-25T10:03:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.533+0000 7f1dd7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1dd80cbe50 0x7f1dd80cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58830/0 (socket says 172.21.15.5:58830) 2024-04-25T10:03:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.533+0000 7f1dd7e00640 1 -- 172.21.15.5:0/3113833924 learned_addr learned my addr 172.21.15.5:0/3113833924 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.533+0000 7f1dddc00640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dd8166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.533+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059645905 0 0) 0x7f1dd80cc510 con 0x7f1dd80cbe50 2024-04-25T10:03:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.533+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dbc003520 con 0x7f1dd80cbe50 2024-04-25T10:03:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.533+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 314849460 0 0) 0x7f1dbc003520 con 0x7f1dd80cbe50 2024-04-25T10:03:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 msgr2=0x7f1dd8166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dd8166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 >> v1:172.21.15.80:6789/0 conn(0x7f1dd815b7f0 legacy=0x7f1dd80b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80ccad0 con 0x7f1dd80cbe50 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1dc80019f0 con 0x7f1dd80cbe50 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 >> v1:172.21.15.5:6789/0 conn(0x7f1dd80cbe50 legacy=0x7f1dd80cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dbc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc003f60 0x7f1dbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dbc004d40 0x7f1dbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd815bbd0 con 0x7f1dbc004d40 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc00feb0 con 0x7f1dbc003f60 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dbc0100b0 con 0x7f1dd80cff40 2024-04-25T10:03:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dddc00640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dbc004d40 0x7f1dbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dd7e00640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc003f60 0x7f1dbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1dddc00640 1 -- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 msgr2=0x7f1dbc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.534+0000 7f1ddd200640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dbc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.535+0000 7f1dddc00640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dbc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.535+0000 7f1dddc00640 1 -- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc003f60 msgr2=0x7f1dbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.535+0000 7f1dddc00640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dbc003f60 0x7f1dbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.535+0000 7f1dddc00640 1 -- 172.21.15.5:0/3113833924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc003520 con 0x7f1dbc004d40 2024-04-25T10:03:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.535+0000 7f1ddd200640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dd80cff40 0x7f1dbc0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.536+0000 7f1dddc00640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dbc004d40 0x7f1dbc0051c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009ac0 tx=0x7f1dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903dab8c409edb9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.536+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0036070 con 0x7f1dbc004d40 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.536+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd0031400 con 0x7f1dbc004d40 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.536+0000 7f1dd7400640 1 -- 172.21.15.5:0/3113833924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0009690 con 0x7f1dbc004d40 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.536+0000 7f1dde600640 1 -- 172.21.15.5:0/3113833924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dbc004d40 msgr2=0x7f1dbc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.537+0000 7f1dde600640 1 --2- 172.21.15.5:0/3113833924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dbc004d40 0x7f1dbc0051c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009ac0 tx=0x7f1dbc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.537+0000 7f1dddc00640 1 -- 172.21.15.5:0/3113833924 reap_dead start 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.537+0000 7f1dde600640 1 -- 172.21.15.5:0/3113833924 shutdown_connections 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.537+0000 7f1dde600640 1 -- 172.21.15.5:0/3113833924 >> 172.21.15.5:0/3113833924 conn(0x7f1dd80d1470 msgr2=0x7f1dd80d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.537+0000 7f1dde600640 1 -- 172.21.15.5:0/3113833924 shutdown_connections 2024-04-25T10:03:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.537+0000 7f1dde600640 1 -- 172.21.15.5:0/3113833924 wait complete. 2024-04-25T10:03:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.538+0000 7f1dde600640 1 Processor -- start 2024-04-25T10:03:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.539+0000 7f1dde600640 1 -- start start 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dde600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dbc004d40 0x7f1dd80c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dde600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dd80c54b0 0x7f1dd80c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dde600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 0x7f1dd8158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dde600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80cd090 con 0x7f1dd8158110 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dde600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80ce5c0 con 0x7f1dd80c54b0 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dde600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd80ce740 con 0x7f1dbc004d40 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 0x7f1dd8158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 0x7f1dd8158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44704/0 (socket says 172.21.15.5:44704) 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 -- 172.21.15.5:0/2349746691 learned_addr learned my addr 172.21.15.5:0/2349746691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dd7e00640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dd80c54b0 0x7f1dd80c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1ddd200640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dbc004d40 0x7f1dd80c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 -- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dbc004d40 msgr2=0x7f1dd80c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dbc004d40 0x7f1dd80c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 -- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dd80c54b0 msgr2=0x7f1dd80c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dd80c54b0 0x7f1dd80c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dddc00640 1 -- 172.21.15.5:0/2349746691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80c5a20 con 0x7f1dd8158110 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1dd7e00640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dd80c54b0 0x7f1dd80c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.540+0000 7f1ddd200640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dbc004d40 0x7f1dd80c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.541+0000 7f1dddc00640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 0x7f1dd8158590 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009be0 tx=0x7f1dd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.541+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0036040 con 0x7f1dd8158110 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.541+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd0041070 con 0x7f1dd8158110 2024-04-25T10:03:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.541+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd80c5d00 con 0x7f1dd8158110 2024-04-25T10:03:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.541+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0009690 con 0x7f1dd8158110 2024-04-25T10:03:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.541+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd80c62c0 con 0x7f1dd8158110 2024-04-25T10:03:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.543+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd8068f60 con 0x7f1dd8158110 2024-04-25T10:03:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.543+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1dd0039070 con 0x7f1dd8158110 2024-04-25T10:03:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.544+0000 7f1dd4c00640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1db406b4d0 0x7f1db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.545+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f1dd00a9d80 con 0x7f1dd8158110 2024-04-25T10:03:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.545+0000 7f1ddd200640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1db406b4d0 0x7f1db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.546+0000 7f1ddd200640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1db406b4d0 0x7f1db406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00016f0 tx=0x7f1dc0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.549+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1dd004a020 con 0x7f1dd8158110 2024-04-25T10:03:19.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.747+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1dd80b15d0 con 0x7f1dd8158110 2024-04-25T10:03:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.748+0000 7f1dd4c00640 1 -- 172.21.15.5:0/2349746691 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1dd0078760 con 0x7f1dd8158110 2024-04-25T10:03:19.753 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1db406b4d0 msgr2=0x7f1db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1db406b4d0 0x7f1db406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00016f0 tx=0x7f1dc0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 msgr2=0x7f1dd8158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 0x7f1dd8158590 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009be0 tx=0x7f1dd0004730 comp rx=0 tx=0).stop 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 shutdown_connections 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dd8158110 0x7f1dd8158590 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1db406b4d0 0x7f1db406d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1dd80c54b0 0x7f1dd80c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 --2- 172.21.15.5:0/2349746691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1dbc004d40 0x7f1dd80c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 >> 172.21.15.5:0/2349746691 conn(0x7f1dd80d1470 msgr2=0x7f1dd80b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 shutdown_connections 2024-04-25T10:03:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.751+0000 7f1dde600640 1 -- 172.21.15.5:0/2349746691 wait complete. 2024-04-25T10:03:19.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:19.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.989+0000 7f3847200640 1 Processor -- start 2024-04-25T10:03:19.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.989+0000 7f3847200640 1 -- start start 2024-04-25T10:03:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.991+0000 7f3847200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38481448f0 0x7f38480cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.991+0000 7f3847200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3848144cd0 con 0x7f384815bc30 2024-04-25T10:03:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.991+0000 7f3847200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384815c440 con 0x7f384815ce20 2024-04-25T10:03:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.991+0000 7f3847200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384815f460 con 0x7f38481448f0 2024-04-25T10:03:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.991+0000 7f3845400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f384815bc30 0x7f384815f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58838/0 (socket says 172.21.15.5:58838) 2024-04-25T10:03:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.991+0000 7f3845400640 1 -- 172.21.15.5:0/2157204423 learned_addr learned my addr 172.21.15.5:0/2157204423 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3845e00640 1 --2- 172.21.15.5:0/2157204423 >> v2:172.21.15.80:3300/0 conn(0x7f38481448f0 0x7f38480cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924533966 0 0) 0x7f3848144cd0 con 0x7f384815bc30 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f382c003520 con 0x7f384815bc30 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1139737665 0 0) 0x7f382c003520 con 0x7f384815bc30 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 >> v2:172.21.15.80:3300/0 conn(0x7f38481448f0 msgr2=0x7f38480cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 --2- 172.21.15.5:0/2157204423 >> v2:172.21.15.80:3300/0 conn(0x7f38481448f0 0x7f38480cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 >> v1:172.21.15.80:6789/0 conn(0x7f384815ce20 legacy=0x7f3848071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384815c720 con 0x7f384815bc30 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f3834001670 con 0x7f384815bc30 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 >> v1:172.21.15.5:6789/0 conn(0x7f384815bc30 legacy=0x7f384815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f382c003a00 0x7f382c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f382c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c0051c0 0x7f382c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815f460 con 0x7f382c004800 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c00c0c0 con 0x7f382c0051c0 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.992+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c00c2c0 con 0x7f382c003a00 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845400640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f382c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845400640 1 -- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f382c003a00 msgr2=0x7f382c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845e00640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f382c003a00 0x7f382c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3846800640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c0051c0 0x7f382c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845400640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f382c003a00 0x7f382c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845400640 1 -- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c0051c0 msgr2=0x7f382c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845400640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c0051c0 0x7f382c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3845400640 1 -- 172.21.15.5:0/2157204423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c003520 con 0x7f382c004800 2024-04-25T10:03:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.993+0000 7f3846800640 1 -- 172.21.15.5:0/2157204423 reap_dead start 2024-04-25T10:03:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.994+0000 7f3845400640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f382c004c80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f383400c0e0 tx=0x7f382c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c613b74dd8830091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.995+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834007c20 con 0x7f382c004800 2024-04-25T10:03:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.995+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834005af0 con 0x7f382c004800 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.995+0000 7f3844a00640 1 -- 172.21.15.5:0/2157204423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834007070 con 0x7f382c004800 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.995+0000 7f3847200640 1 -- 172.21.15.5:0/2157204423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 msgr2=0x7f382c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.995+0000 7f3847200640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f382c004c80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f383400c0e0 tx=0x7f382c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.995+0000 7f3847200640 1 -- 172.21.15.5:0/2157204423 shutdown_connections 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.996+0000 7f3847200640 1 --2- 172.21.15.5:0/2157204423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f382c004c80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.996+0000 7f3847200640 1 -- 172.21.15.5:0/2157204423 >> 172.21.15.5:0/2157204423 conn(0x7f3848066f20 msgr2=0x7f384806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.996+0000 7f3847200640 1 -- 172.21.15.5:0/2157204423 shutdown_connections 2024-04-25T10:03:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.996+0000 7f3847200640 1 -- 172.21.15.5:0/2157204423 wait complete. 2024-04-25T10:03:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.997+0000 7f3847200640 1 Processor -- start 2024-04-25T10:03:20.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.998+0000 7f3847200640 1 -- start start 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3847200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f384813af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3847200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38480b33f0 0x7f38480b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3847200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38480ad4a0 0x7f38480ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3847200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815d2b0 con 0x7f382c004800 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3847200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815c0c0 con 0x7f38480ad4a0 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3847200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384815d430 con 0x7f38480b33f0 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f384813af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f384813af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44736/0 (socket says 172.21.15.5:44736) 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 -- 172.21.15.5:0/3323947806 learned_addr learned my addr 172.21.15.5:0/3323947806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3846800640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38480ad4a0 0x7f38480ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 -- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38480b33f0 msgr2=0x7f38480b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38480b33f0 0x7f38480b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 -- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38480ad4a0 msgr2=0x7f38480ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38480ad4a0 0x7f38480ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:19.999+0000 7f3845e00640 1 -- 172.21.15.5:0/3323947806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384813b680 con 0x7f382c004800 2024-04-25T10:03:20.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.000+0000 7f3845e00640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f384813af30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f38380107c0 tx=0x7f3838010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.000+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383800f040 con 0x7f382c004800 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.000+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383800eb60 con 0x7f382c004800 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.000+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383801c400 con 0x7f382c004800 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.000+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384813b960 con 0x7f382c004800 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.000+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480ade60 con 0x7f382c004800 2024-04-25T10:03:20.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.001+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3848068f60 con 0x7f382c004800 2024-04-25T10:03:20.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.003+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f383801c5a0 con 0x7f382c004800 2024-04-25T10:03:20.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.004+0000 7f383e000640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f382806b500 0x7f382806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.004+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f383808aad0 con 0x7f382c004800 2024-04-25T10:03:20.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.004+0000 7f3845400640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f382806b500 0x7f382806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.006+0000 7f3845400640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f382806b500 0x7f382806d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3834032000 tx=0x7f3834007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.008+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3838018030 con 0x7f382c004800 2024-04-25T10:03:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.204+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3848144cd0 con 0x7f382c004800 2024-04-25T10:03:20.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.206+0000 7f383e000640 1 -- 172.21.15.5:0/3323947806 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38380595d0 con 0x7f382c004800 2024-04-25T10:03:20.210 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f382806b500 msgr2=0x7f382806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f382806b500 0x7f382806d9c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3834032000 tx=0x7f3834007000 comp rx=0 tx=0).stop 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 msgr2=0x7f384813af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f384813af30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f38380107c0 tx=0x7f3838010c80 comp rx=0 tx=0).stop 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 shutdown_connections 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f382806b500 0x7f382806d9c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38480ad4a0 0x7f38480ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38480b33f0 0x7f38480b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 --2- 172.21.15.5:0/3323947806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f382c004800 0x7f384813af30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 >> 172.21.15.5:0/3323947806 conn(0x7f3848066f20 msgr2=0x7f38480b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.208+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 shutdown_connections 2024-04-25T10:03:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.209+0000 7f3847200640 1 -- 172.21.15.5:0/3323947806 wait complete. 2024-04-25T10:03:20.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.434+0000 7fe1a8c00640 1 Processor -- start 2024-04-25T10:03:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.434+0000 7fe1a8c00640 1 -- start start 2024-04-25T10:03:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.435+0000 7fe1a8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 0x7fe1a406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.435+0000 7fe1a8c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a41453c0 con 0x7fe1a4071840 2024-04-25T10:03:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.435+0000 7fe1a8c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a406d6a0 con 0x7fe1a406d130 2024-04-25T10:03:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.435+0000 7fe1a8c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40a8560 con 0x7fe1a4144fe0 2024-04-25T10:03:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.436+0000 7fe1a2800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe1a4071840 0x7fe1a415c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58842/0 (socket says 172.21.15.5:58842) 2024-04-25T10:03:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.436+0000 7fe1a2800640 1 -- 172.21.15.5:0/836770547 learned_addr learned my addr 172.21.15.5:0/836770547 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.436+0000 7fe1a1e00640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 0x7fe1a406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.436+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359194901 0 0) 0x7fe1a41453c0 con 0x7fe1a4071840 2024-04-25T10:03:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.436+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe188003520 con 0x7fe1a4071840 2024-04-25T10:03:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322702923 0 0) 0x7fe188003520 con 0x7fe1a4071840 2024-04-25T10:03:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 msgr2=0x7fe1a406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 0x7fe1a406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 >> v1:172.21.15.80:6789/0 conn(0x7fe1a406d130 legacy=0x7fe1a4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a406d980 con 0x7fe1a4071840 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe19c0019f0 con 0x7fe1a4071840 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 >> v1:172.21.15.5:6789/0 conn(0x7fe1a4071840 legacy=0x7fe1a415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 0x7fe1880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188003f60 0x7fe188004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188004d40 0x7fe1880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18800ba70 con 0x7fe188003f60 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18800bc20 con 0x7fe188004d40 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18800be20 con 0x7fe1a4144fe0 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.437+0000 7fe199400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188003f60 0x7fe188004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe1a2800640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188004d40 0x7fe1880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe1a1e00640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 0x7fe1880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe199400640 1 -- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 msgr2=0x7fe1880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe199400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a4144fe0 0x7fe1880039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe199400640 1 -- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188004d40 msgr2=0x7fe1880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe199400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188004d40 0x7fe1880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe199400640 1 -- 172.21.15.5:0/836770547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188003520 con 0x7fe188003f60 2024-04-25T10:03:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.438+0000 7fe1a2800640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188004d40 0x7fe1880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.439+0000 7fe199400640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188003f60 0x7fe188004800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001750 tx=0x7fe188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592c193757a8129c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.439+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c001370 con 0x7fe188003f60 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.439+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c0052d0 con 0x7fe188003f60 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.439+0000 7fe1a1400640 1 -- 172.21.15.5:0/836770547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c02abe0 con 0x7fe188003f60 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a8c00640 1 -- 172.21.15.5:0/836770547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188003f60 msgr2=0x7fe188004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/836770547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188003f60 0x7fe188004800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe18c001750 tx=0x7fe188005830 comp rx=0 tx=0).stop 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a2800640 1 -- 172.21.15.5:0/836770547 reap_dead start 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a8c00640 1 -- 172.21.15.5:0/836770547 shutdown_connections 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a8c00640 1 -- 172.21.15.5:0/836770547 >> 172.21.15.5:0/836770547 conn(0x7fe1a406ab60 msgr2=0x7fe1a406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a8c00640 1 -- 172.21.15.5:0/836770547 shutdown_connections 2024-04-25T10:03:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.440+0000 7fe1a8c00640 1 -- 172.21.15.5:0/836770547 wait complete. 2024-04-25T10:03:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.442+0000 7fe1a8c00640 1 Processor -- start 2024-04-25T10:03:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.442+0000 7fe1a8c00640 1 -- start start 2024-04-25T10:03:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188003f60 0x7fe1a41f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a413aed0 0x7fe1a40a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 0x7fe1a40a8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a8c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a415be50 con 0x7fe1a40aa130 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a8c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a406df40 con 0x7fe188003f60 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a8c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a406e560 con 0x7fe1a413aed0 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 0x7fe1a40a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 0x7fe1a40a8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44756/0 (socket says 172.21.15.5:44756) 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.443+0000 7fe1a2800640 1 -- 172.21.15.5:0/982668448 learned_addr learned my addr 172.21.15.5:0/982668448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a1e00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188003f60 0x7fe1a41f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe199400640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a413aed0 0x7fe1a40a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a2800640 1 -- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a413aed0 msgr2=0x7fe1a40a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a2800640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a413aed0 0x7fe1a40a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a2800640 1 -- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188003f60 msgr2=0x7fe1a41f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a2800640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188003f60 0x7fe1a41f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a2800640 1 -- 172.21.15.5:0/982668448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a413b350 con 0x7fe1a40aa130 2024-04-25T10:03:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe1a2800640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 0x7fe1a40a8790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4072180 tx=0x7fe19c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c004d60 con 0x7fe1a40aa130 2024-04-25T10:03:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.444+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c00bb20 con 0x7fe1a40aa130 2024-04-25T10:03:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.445+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c007070 con 0x7fe1a40aa130 2024-04-25T10:03:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.445+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a40a9500 con 0x7fe1a40aa130 2024-04-25T10:03:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.445+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a413f340 con 0x7fe1a40aa130 2024-04-25T10:03:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.446+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a4068f60 con 0x7fe1a40aa130 2024-04-25T10:03:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.447+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe19c01b020 con 0x7fe1a40aa130 2024-04-25T10:03:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.448+0000 7fe19aa00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe18006b500 0x7fe18006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.448+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fe19c08a6f0 con 0x7fe1a40aa130 2024-04-25T10:03:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.448+0000 7fe1a1e00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe18006b500 0x7fe18006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.450+0000 7fe1a1e00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe18006b500 0x7fe18006d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe194009a10 tx=0x7fe194008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.452+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe19c001a10 con 0x7fe1a40aa130 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.653+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1a41441d0 con 0x7fe1a40aa130 2024-04-25T10:03:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.654+0000 7fe19aa00640 1 -- 172.21.15.5:0/982668448 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe19c059160 con 0x7fe1a40aa130 2024-04-25T10:03:20.659 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe18006b500 msgr2=0x7fe18006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe18006b500 0x7fe18006d9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe194009a10 tx=0x7fe194008040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 msgr2=0x7fe1a40a8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 0x7fe1a40a8790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4072180 tx=0x7fe19c00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 shutdown_connections 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1a40aa130 0x7fe1a40a8790 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1a413aed0 0x7fe1a40a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe18006b500 0x7fe18006d9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 --2- 172.21.15.5:0/982668448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188003f60 0x7fe1a41f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 >> 172.21.15.5:0/982668448 conn(0x7fe1a406ab60 msgr2=0x7fe1a40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 shutdown_connections 2024-04-25T10:03:20.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.657+0000 7fe1a8c00640 1 -- 172.21.15.5:0/982668448 wait complete. 2024-04-25T10:03:20.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.895+0000 7f6012800640 1 Processor -- start 2024-04-25T10:03:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.895+0000 7f6012800640 1 -- start start 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f6012800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f600c1472c0 0x7f600c0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f6012800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f600c0cad50 0x7f600c0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f6012800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f6012800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0c9f90 con 0x7f600c0cad50 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f6012800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0ca270 con 0x7f600c1472c0 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f6012800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0ca550 con 0x7f600c0ce670 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f600aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f600c0cad50 0x7f600c0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f600aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f600c0cad50 0x7f600c0cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44768/0 (socket says 172.21.15.5:44768) 2024-04-25T10:03:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.897+0000 7f600aa00640 1 -- 172.21.15.5:0/506203065 learned_addr learned my addr 172.21.15.5:0/506203065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:20.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600be00640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600aa00640 1 -- 172.21.15.5:0/506203065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 msgr2=0x7f600c0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600b400640 1 --2- 172.21.15.5:0/506203065 >> v2:172.21.15.80:3300/0 conn(0x7f600c1472c0 0x7f600c0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600aa00640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600aa00640 1 -- 172.21.15.5:0/506203065 >> v2:172.21.15.80:3300/0 conn(0x7f600c1472c0 msgr2=0x7f600c0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600aa00640 1 --2- 172.21.15.5:0/506203065 >> v2:172.21.15.80:3300/0 conn(0x7f600c1472c0 0x7f600c0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600aa00640 1 -- 172.21.15.5:0/506203065 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0ca830 con 0x7f600c0cad50 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600be00640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.898+0000 7f600b400640 1 --2- 172.21.15.5:0/506203065 >> v2:172.21.15.80:3300/0 conn(0x7f600c1472c0 0x7f600c0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.899+0000 7f600aa00640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c0cad50 0x7f600c0cb980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f600c0b58a0 tx=0x7f5ff802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39588a96e28412c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.899+0000 7f600a000640 1 -- 172.21.15.5:0/506203065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037070 con 0x7f600c0cad50 2024-04-25T10:03:20.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f600a000640 1 -- 172.21.15.5:0/506203065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff802ae00 con 0x7f600c0cad50 2024-04-25T10:03:20.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f600a000640 1 -- 172.21.15.5:0/506203065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8032660 con 0x7f600c0cad50 2024-04-25T10:03:20.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 -- 172.21.15.5:0/506203065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c0cad50 msgr2=0x7f600c0cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c0cad50 0x7f600c0cb980 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f600c0b58a0 tx=0x7f5ff802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 -- 172.21.15.5:0/506203065 shutdown_connections 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 --2- 172.21.15.5:0/506203065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c0cad50 0x7f600c0cb980 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 --2- 172.21.15.5:0/506203065 >> v2:172.21.15.80:3300/0 conn(0x7f600c1472c0 0x7f600c0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 -- 172.21.15.5:0/506203065 >> 172.21.15.5:0/506203065 conn(0x7f600c071150 msgr2=0x7f600c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 -- 172.21.15.5:0/506203065 shutdown_connections 2024-04-25T10:03:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.900+0000 7f6012800640 1 -- 172.21.15.5:0/506203065 wait complete. 2024-04-25T10:03:20.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.902+0000 7f6012800640 1 Processor -- start 2024-04-25T10:03:20.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.902+0000 7f6012800640 1 -- start start 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.903+0000 7f6012800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f600c0cad50 0x7f600c138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.903+0000 7f6012800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.903+0000 7f6012800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 0x7f600c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.903+0000 7f6012800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0cf180 con 0x7f600c1472c0 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.903+0000 7f6012800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0cf000 con 0x7f600c0cad50 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.903+0000 7f6012800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c1614f0 con 0x7f600c0ce670 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 0x7f600c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 0x7f600c0b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44770/0 (socket says 172.21.15.5:44770) 2024-04-25T10:03:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 -- 172.21.15.5:0/953672274 learned_addr learned my addr 172.21.15.5:0/953672274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600b400640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f600c0cad50 0x7f600c138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600aa00640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 -- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 msgr2=0x7f600c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 -- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f600c0cad50 msgr2=0x7f600c138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f600c0cad50 0x7f600c138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.904+0000 7f600be00640 1 -- 172.21.15.5:0/953672274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c142990 con 0x7f600c1472c0 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.905+0000 7f600be00640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 0x7f600c0b4720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f600400cef0 tx=0x7f600400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.905+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600401a070 con 0x7f600c1472c0 2024-04-25T10:03:20.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.905+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600400d860 con 0x7f600c1472c0 2024-04-25T10:03:20.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.905+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c142c70 con 0x7f600c1472c0 2024-04-25T10:03:20.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.905+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400dc50 con 0x7f600c1472c0 2024-04-25T10:03:20.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.905+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c143230 con 0x7f600c1472c0 2024-04-25T10:03:20.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.906+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c068fd0 con 0x7f600c1472c0 2024-04-25T10:03:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.907+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f600401d070 con 0x7f600c1472c0 2024-04-25T10:03:20.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.908+0000 7f6003400640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5fec06b4d0 0x7f5fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.908+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f600408ae90 con 0x7f600c1472c0 2024-04-25T10:03:20.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.909+0000 7f600b400640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5fec06b4d0 0x7f5fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.910+0000 7f600b400640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5fec06b4d0 0x7f5fec06d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00a8d0 tx=0x7f5ffc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:20.912+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60040599c0 con 0x7f600c1472c0 2024-04-25T10:03:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.153+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f600c072390 con 0x7f600c1472c0 2024-04-25T10:03:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.154+0000 7f6003400640 1 -- 172.21.15.5:0/953672274 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7f600408c090 con 0x7f600c1472c0 2024-04-25T10:03:21.159 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:21.159 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:03:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.156+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5fec06b4d0 msgr2=0x7f5fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.156+0000 7f6012800640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5fec06b4d0 0x7f5fec06d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00a8d0 tx=0x7f5ffc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.156+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 msgr2=0x7f600c0b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.156+0000 7f6012800640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 0x7f600c0b4720 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f600400cef0 tx=0x7f600400a710 comp rx=0 tx=0).stop 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 shutdown_connections 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f600c1472c0 0x7f600c0b4720 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f600c0ce670 0x7f600c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5fec06b4d0 0x7f5fec06d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 --2- 172.21.15.5:0/953672274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f600c0cad50 0x7f600c138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 >> 172.21.15.5:0/953672274 conn(0x7f600c071150 msgr2=0x7f600c0cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 shutdown_connections 2024-04-25T10:03:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.157+0000 7f6012800640 1 -- 172.21.15.5:0/953672274 wait complete. 2024-04-25T10:03:21.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.387+0000 7f3571e00640 1 Processor -- start 2024-04-25T10:03:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.387+0000 7f3571e00640 1 -- start start 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f356c0cdca0 0x7f356c0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c0b4890 con 0x7f356c070f90 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c0cd200 con 0x7f356c0cdca0 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c06e6f0 con 0x7f356c154a10 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f356be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f356c0cdca0 0x7f356c0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f356c070f90 0x7f356c06cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58844/0 (socket says 172.21.15.5:58844) 2024-04-25T10:03:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.389+0000 7f3571400640 1 -- 172.21.15.5:0/3887030019 learned_addr learned my addr 172.21.15.5:0/3887030019 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.390+0000 7f3570a00640 1 --2- 172.21.15.5:0/3887030019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.390+0000 7f356be00640 1 -- 172.21.15.5:0/3887030019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 msgr2=0x7f356c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.390+0000 7f356be00640 1 --2- 172.21.15.5:0/3887030019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.390+0000 7f356be00640 1 -- 172.21.15.5:0/3887030019 >> v1:172.21.15.5:6789/0 conn(0x7f356c070f90 legacy=0x7f356c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.390+0000 7f356be00640 1 -- 172.21.15.5:0/3887030019 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c06e9d0 con 0x7f356c0cdca0 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.391+0000 7f356be00640 1 --2- 172.21.15.5:0/3887030019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c0cdca0 0x7f356c0b4350 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f35580097a0 tx=0x7f355802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e7b2c787792cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.391+0000 7f356b400640 1 -- 172.21.15.5:0/3887030019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558037070 con 0x7f356c0cdca0 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f356b400640 1 -- 172.21.15.5:0/3887030019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355802ae40 con 0x7f356c0cdca0 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f356b400640 1 -- 172.21.15.5:0/3887030019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558032660 con 0x7f356c0cdca0 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 -- 172.21.15.5:0/3887030019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c0cdca0 msgr2=0x7f356c0b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 --2- 172.21.15.5:0/3887030019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c0cdca0 0x7f356c0b4350 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f35580097a0 tx=0x7f355802a0f0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 -- 172.21.15.5:0/3887030019 shutdown_connections 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 --2- 172.21.15.5:0/3887030019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c0cdca0 0x7f356c0b4350 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 --2- 172.21.15.5:0/3887030019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 -- 172.21.15.5:0/3887030019 >> 172.21.15.5:0/3887030019 conn(0x7f356c066f20 msgr2=0x7f356c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 -- 172.21.15.5:0/3887030019 shutdown_connections 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.392+0000 7f3571e00640 1 -- 172.21.15.5:0/3887030019 wait complete. 2024-04-25T10:03:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.394+0000 7f3571e00640 1 Processor -- start 2024-04-25T10:03:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.394+0000 7f3571e00640 1 -- start start 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3571e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 0x7f356c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3571e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f356c0cdca0 0x7f356c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3571e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3571e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0cd780 con 0x7f356c070f90 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3571e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0cd4a0 con 0x7f356c0cdca0 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3571e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c06d700 con 0x7f356c154a10 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3570a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 0x7f356c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.395+0000 7f3570a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 0x7f356c157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44798/0 (socket says 172.21.15.5:44798) 2024-04-25T10:03:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 -- 172.21.15.5:0/4006860555 learned_addr learned my addr 172.21.15.5:0/4006860555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f356be00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f356c0cdca0 0x7f356c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3571400640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 -- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 msgr2=0x7f356c15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 -- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f356c0cdca0 msgr2=0x7f356c154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f356c0cdca0 0x7f356c154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 -- 172.21.15.5:0/4006860555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c155330 con 0x7f356c070f90 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.396+0000 7f3570a00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 0x7f356c157940 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f356000d620 tx=0x7f356000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.397+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000b620 con 0x7f356c070f90 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.397+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3560019070 con 0x7f356c070f90 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.397+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000ba00 con 0x7f356c070f90 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.397+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c1f6f60 con 0x7f356c070f90 2024-04-25T10:03:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.397+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c1f7520 con 0x7f356c070f90 2024-04-25T10:03:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.398+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c068f60 con 0x7f356c070f90 2024-04-25T10:03:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.399+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f356000bba0 con 0x7f356c070f90 2024-04-25T10:03:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.400+0000 7f3568c00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f354c06b4d0 0x7f354c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.400+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f356008aa50 con 0x7f356c070f90 2024-04-25T10:03:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.401+0000 7f356be00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f354c06b4d0 0x7f354c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.402+0000 7f356be00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f354c06b4d0 0x7f354c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3558034000 tx=0x7f355802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.404+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35600594c0 con 0x7f356c070f90 2024-04-25T10:03:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.605+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f356c06ac60 con 0x7f356c070f90 2024-04-25T10:03:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.606+0000 7f3568c00640 1 -- 172.21.15.5:0/4006860555 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f3560058dc0 con 0x7f356c070f90 2024-04-25T10:03:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:21.614 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f354c06b4d0 msgr2=0x7f354c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f354c06b4d0 0x7f354c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3558034000 tx=0x7f355802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:21.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 msgr2=0x7f356c157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 0x7f356c157940 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f356000d620 tx=0x7f356000dae0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 shutdown_connections 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f356c154a10 0x7f356c15f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f354c06b4d0 0x7f354c06d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f356c0cdca0 0x7f356c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 --2- 172.21.15.5:0/4006860555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f356c070f90 0x7f356c157940 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.609+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 >> 172.21.15.5:0/4006860555 conn(0x7f356c066f20 msgr2=0x7f356c0af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.610+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 shutdown_connections 2024-04-25T10:03:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.610+0000 7f3571e00640 1 -- 172.21.15.5:0/4006860555 wait complete. 2024-04-25T10:03:21.635 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:21.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.842+0000 7fec4b400640 1 Processor -- start 2024-04-25T10:03:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.842+0000 7fec4b400640 1 -- start start 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.844+0000 7fec4b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec441529e0 0x7fec440c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.844+0000 7fec4b400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec440cea60 0x7fec4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.844+0000 7fec4b400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4415bdc0 con 0x7fec440c6410 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.844+0000 7fec4b400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440b2340 con 0x7fec440cea60 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.844+0000 7fec4b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415c060 con 0x7fec441529e0 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.845+0000 7fec43e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fec440c6410 0x7fec440ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58854/0 (socket says 172.21.15.5:58854) 2024-04-25T10:03:21.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.845+0000 7fec43e00640 1 -- 172.21.15.5:0/963532383 learned_addr learned my addr 172.21.15.5:0/963532383 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.845+0000 7fec49400640 1 --2- 172.21.15.5:0/963532383 >> v2:172.21.15.80:3300/0 conn(0x7fec440cea60 0x7fec4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.845+0000 7fec48a00640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec441529e0 0x7fec440c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.845+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987974272 0 0) 0x7fec4415bdc0 con 0x7fec440c6410 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.845+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec2c003520 con 0x7fec440c6410 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3918099553 0 0) 0x7fec2c003520 con 0x7fec440c6410 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec441529e0 msgr2=0x7fec440c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec441529e0 0x7fec440c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 >> v2:172.21.15.80:3300/0 conn(0x7fec440cea60 msgr2=0x7fec4406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 --2- 172.21.15.5:0/963532383 >> v2:172.21.15.80:3300/0 conn(0x7fec440cea60 0x7fec4406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4415c340 con 0x7fec440c6410 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fec30001690 con 0x7fec440c6410 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 >> v1:172.21.15.5:6789/0 conn(0x7fec440c6410 legacy=0x7fec440ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec2c004800 0x7fec2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec2c005180 0x7fec2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b2340 con 0x7fec2c0039b0 2024-04-25T10:03:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415c060 con 0x7fec2c005180 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c004600 con 0x7fec2c004800 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec48a00640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.846+0000 7fec43e00640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec2c004800 0x7fec2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.847+0000 7fec49400640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec2c005180 0x7fec2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.847+0000 7fec48a00640 1 -- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec2c004800 msgr2=0x7fec2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.847+0000 7fec48a00640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec2c004800 0x7fec2c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.847+0000 7fec48a00640 1 -- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec2c005180 msgr2=0x7fec2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.847+0000 7fec48a00640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec2c005180 0x7fec2c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.847+0000 7fec48a00640 1 -- 172.21.15.5:0/963532383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c003520 con 0x7fec2c0039b0 2024-04-25T10:03:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.848+0000 7fec48a00640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fec38009ac0 tx=0x7fec2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14abd285666581f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.848+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38036070 con 0x7fec2c0039b0 2024-04-25T10:03:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.848+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38031400 con 0x7fec2c0039b0 2024-04-25T10:03:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.848+0000 7fec43400640 1 -- 172.21.15.5:0/963532383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38009690 con 0x7fec2c0039b0 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec4b400640 1 -- 172.21.15.5:0/963532383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 msgr2=0x7fec2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec4b400640 1 --2- 172.21.15.5:0/963532383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec2c003dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fec38009ac0 tx=0x7fec2c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec49400640 1 -- 172.21.15.5:0/963532383 reap_dead start 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec4b400640 1 -- 172.21.15.5:0/963532383 shutdown_connections 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec4b400640 1 -- 172.21.15.5:0/963532383 >> 172.21.15.5:0/963532383 conn(0x7fec44071150 msgr2=0x7fec44071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec4b400640 1 -- 172.21.15.5:0/963532383 shutdown_connections 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.849+0000 7fec4b400640 1 -- 172.21.15.5:0/963532383 wait complete. 2024-04-25T10:03:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.851+0000 7fec4b400640 1 Processor -- start 2024-04-25T10:03:21.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.851+0000 7fec4b400640 1 -- start start 2024-04-25T10:03:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec4b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec44158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec4b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec440b5400 0x7fec44156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec4b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec440b27d0 0x7fec44156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec4b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44071ed0 con 0x7fec2c0039b0 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec4b400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415c900 con 0x7fec440b5400 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec4b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec44072050 con 0x7fec440b27d0 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec48a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec44158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec48a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec44158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44802/0 (socket says 172.21.15.5:44802) 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec48a00640 1 -- 172.21.15.5:0/1821565998 learned_addr learned my addr 172.21.15.5:0/1821565998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.852+0000 7fec43e00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec440b5400 0x7fec44156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec49400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec440b27d0 0x7fec44156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec48a00640 1 -- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec440b27d0 msgr2=0x7fec44156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec48a00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec440b27d0 0x7fec44156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec48a00640 1 -- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec440b5400 msgr2=0x7fec44156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec48a00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec440b5400 0x7fec44156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec48a00640 1 -- 172.21.15.5:0/1821565998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4414ee50 con 0x7fec2c0039b0 2024-04-25T10:03:21.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec43e00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec440b5400 0x7fec44156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:21.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec49400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec440b27d0 0x7fec44156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:21.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec48a00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec44158c00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fec3802abb0 tx=0x7fec38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38040020 con 0x7fec2c0039b0 2024-04-25T10:03:21.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38003940 con 0x7fec2c0039b0 2024-04-25T10:03:21.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38036040 con 0x7fec2c0039b0 2024-04-25T10:03:21.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.853+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec44159140 con 0x7fec2c0039b0 2024-04-25T10:03:21.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.854+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec44149020 con 0x7fec2c0039b0 2024-04-25T10:03:21.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.855+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44068fd0 con 0x7fec2c0039b0 2024-04-25T10:03:21.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.856+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec38043020 con 0x7fec2c0039b0 2024-04-25T10:03:21.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.857+0000 7fec40c00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2406b4d0 0x7fec2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.857+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fec380aab30 con 0x7fec2c0039b0 2024-04-25T10:03:21.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.857+0000 7fec43e00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2406b4d0 0x7fec2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.859+0000 7fec43e00640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2406b4d0 0x7fec2406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec4415b150 tx=0x7fec30000ed0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:21.861+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec3804a030 con 0x7fec2c0039b0 2024-04-25T10:03:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.100+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec44156030 con 0x7fec2c0039b0 2024-04-25T10:03:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.102+0000 7fec40c00640 1 -- 172.21.15.5:0/1821565998 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7fec380795a0 con 0x7fec2c0039b0 2024-04-25T10:03:22.107 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:22.107 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.104+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2406b4d0 msgr2=0x7fec2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.104+0000 7fec4b400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2406b4d0 0x7fec2406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec4415b150 tx=0x7fec30000ed0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.104+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 msgr2=0x7fec44158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.104+0000 7fec4b400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec44158c00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fec3802abb0 tx=0x7fec38004730 comp rx=0 tx=0).stop 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 shutdown_connections 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec440b27d0 0x7fec44156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2406b4d0 0x7fec2406d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec440b5400 0x7fec44156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 --2- 172.21.15.5:0/1821565998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec2c0039b0 0x7fec44158c00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 >> 172.21.15.5:0/1821565998 conn(0x7fec44071150 msgr2=0x7fec440cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 shutdown_connections 2024-04-25T10:03:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.105+0000 7fec4b400640 1 -- 172.21.15.5:0/1821565998 wait complete. 2024-04-25T10:03:22.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.338+0000 7f06aba00640 1 Processor -- start 2024-04-25T10:03:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.338+0000 7f06aba00640 1 -- start start 2024-04-25T10:03:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.340+0000 7f06aba00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06a4071ed0 0x7f06a40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.340+0000 7f06aba00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06a40ce890 0x7f06a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.340+0000 7f06aba00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a415bdc0 con 0x7f06a41529e0 2024-04-25T10:03:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.340+0000 7f06aba00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40b2340 con 0x7f06a4071ed0 2024-04-25T10:03:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.340+0000 7f06aba00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40cecc0 con 0x7f06a40ce890 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06a4071ed0 0x7f06a40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a9a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06a40ce890 0x7f06a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06a4071ed0 0x7f06a40ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44816/0 (socket says 172.21.15.5:44816) 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 -- 172.21.15.5:0/2490630819 learned_addr learned my addr 172.21.15.5:0/2490630819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 -- 172.21.15.5:0/2490630819 >> v2:172.21.15.80:3300/0 conn(0x7f06a40ce890 msgr2=0x7f06a406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 --2- 172.21.15.5:0/2490630819 >> v2:172.21.15.80:3300/0 conn(0x7f06a40ce890 0x7f06a406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 -- 172.21.15.5:0/2490630819 >> v1:172.21.15.80:6789/0 conn(0x7f06a41529e0 legacy=0x7f06a40c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.341+0000 7f06a3e00640 1 -- 172.21.15.5:0/2490630819 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a415c060 con 0x7f06a4071ed0 2024-04-25T10:03:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.343+0000 7f06a3e00640 1 --2- 172.21.15.5:0/2490630819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a4071ed0 0x7f06a40ce350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f06a40c78c0 tx=0x7f069402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743636e955c928b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.343+0000 7f06a3400640 1 -- 172.21.15.5:0/2490630819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694037070 con 0x7f06a4071ed0 2024-04-25T10:03:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.343+0000 7f06a3400640 1 -- 172.21.15.5:0/2490630819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694032440 con 0x7f06a4071ed0 2024-04-25T10:03:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.343+0000 7f06a3400640 1 -- 172.21.15.5:0/2490630819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694032810 con 0x7f06a4071ed0 2024-04-25T10:03:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.343+0000 7f06aba00640 1 -- 172.21.15.5:0/2490630819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a4071ed0 msgr2=0x7f06a40ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.343+0000 7f06aba00640 1 --2- 172.21.15.5:0/2490630819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a4071ed0 0x7f06a40ce350 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f06a40c78c0 tx=0x7f069402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.344+0000 7f06aba00640 1 -- 172.21.15.5:0/2490630819 shutdown_connections 2024-04-25T10:03:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.344+0000 7f06aba00640 1 --2- 172.21.15.5:0/2490630819 >> v2:172.21.15.80:3300/0 conn(0x7f06a40ce890 0x7f06a406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.344+0000 7f06aba00640 1 --2- 172.21.15.5:0/2490630819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a4071ed0 0x7f06a40ce350 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.344+0000 7f06aba00640 1 -- 172.21.15.5:0/2490630819 >> 172.21.15.5:0/2490630819 conn(0x7f06a4071150 msgr2=0x7f06a4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.344+0000 7f06aba00640 1 -- 172.21.15.5:0/2490630819 shutdown_connections 2024-04-25T10:03:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.344+0000 7f06aba00640 1 -- 172.21.15.5:0/2490630819 wait complete. 2024-04-25T10:03:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.345+0000 7f06aba00640 1 Processor -- start 2024-04-25T10:03:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.346+0000 7f06aba00640 1 -- start start 2024-04-25T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.346+0000 7f06aba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06a4071ed0 0x7f06a414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.346+0000 7f06aba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 0x7f06a40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.346+0000 7f06aba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06a40b4570 0x7f06a4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.346+0000 7f06aba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a415c900 con 0x7f06a40ce890 2024-04-25T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06aba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a415c620 con 0x7f06a4071ed0 2024-04-25T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06aba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40c65e0 con 0x7f06a40b4570 2024-04-25T10:03:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 0x7f06a40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 0x7f06a40b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44818/0 (socket says 172.21.15.5:44818) 2024-04-25T10:03:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 -- 172.21.15.5:0/365366923 learned_addr learned my addr 172.21.15.5:0/365366923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 -- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06a40b4570 msgr2=0x7f06a4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06a40b4570 0x7f06a4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 -- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06a4071ed0 msgr2=0x7f06a414acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06a4071ed0 0x7f06a414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.347+0000 7f06a3e00640 1 -- 172.21.15.5:0/365366923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a41546d0 con 0x7f06a40ce890 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.348+0000 7f06a3e00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 0x7f06a40b4db0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0694000c00 tx=0x7f06940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.348+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694037040 con 0x7f06a40ce890 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.348+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694042070 con 0x7f06a40ce890 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.348+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069403b870 con 0x7f06a40ce890 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.348+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a41549b0 con 0x7f06a40ce890 2024-04-25T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.348+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a4154f70 con 0x7f06a40ce890 2024-04-25T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.349+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a4068fd0 con 0x7f06a40ce890 2024-04-25T10:03:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.350+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f069403ba10 con 0x7f06a40ce890 2024-04-25T10:03:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.351+0000 7f06a0c00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.352+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f06940aae60 con 0x7f06a40ce890 2024-04-25T10:03:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.352+0000 7f06a9000640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.353+0000 7f06a9000640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f068806b4d0 0x7f068806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06a415bf90 tx=0x7f0698007970 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.357+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0694079840 con 0x7f06a40ce890 2024-04-25T10:03:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.605+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06a40737a0 con 0x7f06a40ce890 2024-04-25T10:03:22.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.606+0000 7f06a0c00640 1 -- 172.21.15.5:0/365366923 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4925 (secure 0 0 0) 0x7f0694079140 con 0x7f06a40ce890 2024-04-25T10:03:22.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:22.611 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3168204646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3168204646},{"type":"v1","addr":"172.21.15.5:6835","nonce":3168204646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2945764629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2945764629},{"type":"v1","addr":"172.21.15.80:6835","nonce":2945764629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4157906725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4157906725},{"type":"v1","addr":"172.21.15.5:6834","nonce":4157906725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:01:57.496056+0000","modified":"2024-04-25T10:02:23.941309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/312213316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":312213316},{"type":"v1","addr":"172.21.15.80:6837","nonce":312213316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:22.612 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f068806b4d0 msgr2=0x7f068806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f068806b4d0 0x7f068806d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f06a415bf90 tx=0x7f0698007970 comp rx=0 tx=0).stop 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 msgr2=0x7f06a40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 0x7f06a40b4db0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0694000c00 tx=0x7f06940046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 shutdown_connections 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06a40b4570 0x7f06a4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06a40ce890 0x7f06a40b4db0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 --2- 172.21.15.5:0/365366923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06a4071ed0 0x7f06a414acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 >> 172.21.15.5:0/365366923 conn(0x7f06a4071150 msgr2=0x7f06a40cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.609+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 shutdown_connections 2024-04-25T10:03:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.610+0000 7f06aba00640 1 -- 172.21.15.5:0/365366923 wait complete. 2024-04-25T10:03:22.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.842+0000 7f020b200640 1 Processor -- start 2024-04-25T10:03:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.843+0000 7f020b200640 1 -- start start 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.844+0000 7f020b200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f020c0b4bb0 0x7f020c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.844+0000 7f020b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0cf410 con 0x7f020c161200 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.844+0000 7f020b200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c0cf6f0 con 0x7f020c0d07d0 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.844+0000 7f020b200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0b28b0 con 0x7f020c0b4bb0 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.844+0000 7f0209e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f020c161200 0x7f020c0d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58866/0 (socket says 172.21.15.5:58866) 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0209e00640 1 -- 172.21.15.5:0/4119982145 learned_addr learned my addr 172.21.15.5:0/4119982145 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f020a800640 1 --2- 172.21.15.5:0/4119982145 >> v2:172.21.15.80:3300/0 conn(0x7f020c0b4bb0 0x7f020c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273664335 0 0) 0x7f020c0cf410 con 0x7f020c161200 2024-04-25T10:03:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f0003520 con 0x7f020c161200 2024-04-25T10:03:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1377464387 0 0) 0x7f01f0003520 con 0x7f020c161200 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 >> v2:172.21.15.80:3300/0 conn(0x7f020c0b4bb0 msgr2=0x7f020c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 --2- 172.21.15.5:0/4119982145 >> v2:172.21.15.80:3300/0 conn(0x7f020c0b4bb0 0x7f020c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 >> v1:172.21.15.80:6789/0 conn(0x7f020c0d07d0 legacy=0x7f020c0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0d3fd0 con 0x7f020c161200 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.845+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f01fc001670 con 0x7f020c161200 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 >> v1:172.21.15.5:6789/0 conn(0x7f020c161200 legacy=0x7f020c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f01f0003a00 0x7f01f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0b28b0 con 0x7f01f00051c0 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f000c0c0 con 0x7f01f0004800 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f000c2c0 con 0x7f01f0003a00 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f020a800640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0209e00640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f01f0003a00 0x7f01f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f0209400640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.846+0000 7f020a800640 1 -- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f01f0003a00 msgr2=0x7f01f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.847+0000 7f020a800640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f01f0003a00 0x7f01f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.847+0000 7f020a800640 1 -- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01f0004800 msgr2=0x7f01f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.847+0000 7f020a800640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01f0004800 0x7f01f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.847+0000 7f020a800640 1 -- 172.21.15.5:0/4119982145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0003520 con 0x7f01f00051c0 2024-04-25T10:03:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.847+0000 7f0209e00640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f01f0003a00 0x7f01f0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.847+0000 7f020a800640 1 -- 172.21.15.5:0/4119982145 reap_dead start 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.848+0000 7f020a800640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0204009fc0 tx=0x7f01f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923cecf5f2e0c2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.848+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204035020 con 0x7f01f00051c0 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.848+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0204003bd0 con 0x7f01f00051c0 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.848+0000 7f0208a00640 1 -- 172.21.15.5:0/4119982145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204009c10 con 0x7f01f00051c0 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.848+0000 7f020b200640 1 -- 172.21.15.5:0/4119982145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 msgr2=0x7f01f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.848+0000 7f020b200640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0204009fc0 tx=0x7f01f0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.849+0000 7f020b200640 1 -- 172.21.15.5:0/4119982145 shutdown_connections 2024-04-25T10:03:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.849+0000 7f020b200640 1 --2- 172.21.15.5:0/4119982145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f01f000b9c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.849+0000 7f020b200640 1 -- 172.21.15.5:0/4119982145 >> 172.21.15.5:0/4119982145 conn(0x7f020c08af80 msgr2=0x7f020c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.849+0000 7f020b200640 1 -- 172.21.15.5:0/4119982145 shutdown_connections 2024-04-25T10:03:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.849+0000 7f020b200640 1 -- 172.21.15.5:0/4119982145 wait complete. 2024-04-25T10:03:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.850+0000 7f020b200640 1 Processor -- start 2024-04-25T10:03:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.851+0000 7f020b200640 1 -- start start 2024-04-25T10:03:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.851+0000 7f020b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f020c0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f020b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f020c14c690 0x7f020c0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f020b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f020c14ef90 0x7f020c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f020b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0d2500 con 0x7f01f00051c0 2024-04-25T10:03:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f020b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0d2380 con 0x7f020c14ef90 2024-04-25T10:03:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f020b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0d2680 con 0x7f020c14c690 2024-04-25T10:03:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f020c0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f020c0ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44844/0 (socket says 172.21.15.5:44844) 2024-04-25T10:03:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 -- 172.21.15.5:0/2288501453 learned_addr learned my addr 172.21.15.5:0/2288501453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 -- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f020c14c690 msgr2=0x7f020c0b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f020c14c690 0x7f020c0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f020a800640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f020c14ef90 0x7f020c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 -- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f020c14ef90 msgr2=0x7f020c14f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f020c14ef90 0x7f020c14f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.852+0000 7f0209e00640 1 -- 172.21.15.5:0/2288501453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0cacc0 con 0x7f01f00051c0 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.853+0000 7f0209e00640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f020c0ca780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d7d0 tx=0x7f01fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.853+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc019070 con 0x7f01f00051c0 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.853+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc0074e0 con 0x7f01f00051c0 2024-04-25T10:03:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.853+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c14d110 con 0x7f01f00051c0 2024-04-25T10:03:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.853+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc004a00 con 0x7f01f00051c0 2024-04-25T10:03:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.853+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c14d6d0 con 0x7f01f00051c0 2024-04-25T10:03:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.854+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0d2940 con 0x7f01f00051c0 2024-04-25T10:03:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.855+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01fc01c070 con 0x7f01f00051c0 2024-04-25T10:03:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.856+0000 7f0202000640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f01ec06b500 0x7f01ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.856+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7f01fc089fe0 con 0x7f01f00051c0 2024-04-25T10:03:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.857+0000 7f0209400640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f01ec06b500 0x7f01ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.858+0000 7f0209400640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f01ec06b500 0x7f01ec06d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007fd0 tx=0x7f01f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:22.862+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f020c0d2940 con 0x7f01f00051c0 2024-04-25T10:03:23.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.065+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020c068f60 con 0x7f01f00051c0 2024-04-25T10:03:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.067+0000 7f0202000640 1 -- 172.21.15.5:0/2288501453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18506 (secure 0 0 0) 0x7f01fc058a90 con 0x7f01f00051c0 2024-04-25T10:03:23.072 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:23.073 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:02:40.579034+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:01:55.847408+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:01:56.507966+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:00.365944+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3b","mappings":[{"from":0,"to":7}]},{"pgid":"4.10","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f01ec06b500 msgr2=0x7f01ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f01ec06b500 0x7f01ec06d9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f01f8007fd0 tx=0x7f01f8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 msgr2=0x7f020c0ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f020c0ca780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d7d0 tx=0x7f01fc00dc90 comp rx=0 tx=0).stop 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 shutdown_connections 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f01ec06b500 0x7f01ec06d9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f020c14ef90 0x7f020c14f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f020c14c690 0x7f020c0b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 --2- 172.21.15.5:0/2288501453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f00051c0 0x7f020c0ca780 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.070+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 >> 172.21.15.5:0/2288501453 conn(0x7f020c08af80 msgr2=0x7f020c15e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.071+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 shutdown_connections 2024-04-25T10:03:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.071+0000 7f020b200640 1 -- 172.21.15.5:0/2288501453 wait complete. 2024-04-25T10:03:23.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.306+0000 7fe927e00640 1 Processor -- start 2024-04-25T10:03:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.306+0000 7fe927e00640 1 -- start start 2024-04-25T10:03:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe927e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe928141da0 0x7fe92806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe927e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe928071d30 con 0x7fe92815d510 2024-04-25T10:03:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe927e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92806d620 con 0x7fe92806e640 2024-04-25T10:03:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe927e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe928142180 con 0x7fe928141da0 2024-04-25T10:03:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe927400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe92815d510 0x7fe9280cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58878/0 (socket says 172.21.15.5:58878) 2024-04-25T10:03:23.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe927400640 1 -- 172.21.15.5:0/3537798524 learned_addr learned my addr 172.21.15.5:0/3537798524 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.308+0000 7fe926a00640 1 --2- 172.21.15.5:0/3537798524 >> v2:172.21.15.80:3300/0 conn(0x7fe928141da0 0x7fe92806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845913465 0 0) 0x7fe928071d30 con 0x7fe92815d510 2024-04-25T10:03:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90c003520 con 0x7fe92815d510 2024-04-25T10:03:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147471994 0 0) 0x7fe90c003520 con 0x7fe92815d510 2024-04-25T10:03:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 >> v2:172.21.15.80:3300/0 conn(0x7fe928141da0 msgr2=0x7fe92806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 --2- 172.21.15.5:0/3537798524 >> v2:172.21.15.80:3300/0 conn(0x7fe928141da0 0x7fe92806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 >> v1:172.21.15.80:6789/0 conn(0x7fe92806e640 legacy=0x7fe92815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92806d900 con 0x7fe92815d510 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe9200019f0 con 0x7fe92815d510 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 >> v1:172.21.15.5:6789/0 conn(0x7fe92815d510 legacy=0x7fe9280cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0039b0 0x7fe90c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c004800 0x7fe90c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.309+0000 7fe926000640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c0051c0 0x7fe90c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe928142180 con 0x7fe90c004800 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00c060 con 0x7fe90c0051c0 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c00c260 con 0x7fe90c0039b0 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe91fe00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c004800 0x7fe90c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe926a00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0039b0 0x7fe90c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe927400640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c0051c0 0x7fe90c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe91fe00640 1 -- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0039b0 msgr2=0x7fe90c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe91fe00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0039b0 0x7fe90c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe91fe00640 1 -- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c0051c0 msgr2=0x7fe90c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe91fe00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c0051c0 0x7fe90c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe91fe00640 1 -- 172.21.15.5:0/3537798524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c003520 con 0x7fe90c004800 2024-04-25T10:03:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe927400640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c0051c0 0x7fe90c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.310+0000 7fe927400640 1 -- 172.21.15.5:0/3537798524 reap_dead start 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.311+0000 7fe91fe00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c004800 0x7fe90c004c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe910001720 tx=0x7fe90c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40866dfef1264fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910001370 con 0x7fe90c004800 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9100052d0 con 0x7fe90c004800 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe926000640 1 -- 172.21.15.5:0/3537798524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91002ab90 con 0x7fe90c004800 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe927e00640 1 -- 172.21.15.5:0/3537798524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c004800 msgr2=0x7fe90c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe927e00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c004800 0x7fe90c004c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe910001720 tx=0x7fe90c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe927e00640 1 -- 172.21.15.5:0/3537798524 shutdown_connections 2024-04-25T10:03:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe927e00640 1 --2- 172.21.15.5:0/3537798524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c004800 0x7fe90c004c80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe927e00640 1 -- 172.21.15.5:0/3537798524 >> 172.21.15.5:0/3537798524 conn(0x7fe928066f20 msgr2=0x7fe92806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.312+0000 7fe927e00640 1 -- 172.21.15.5:0/3537798524 shutdown_connections 2024-04-25T10:03:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.313+0000 7fe927e00640 1 -- 172.21.15.5:0/3537798524 wait complete. 2024-04-25T10:03:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.315+0000 7fe927e00640 1 Processor -- start 2024-04-25T10:03:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.315+0000 7fe927e00640 1 -- start start 2024-04-25T10:03:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c004800 0x7fe92814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 0x7fe9280cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe92814c5c0 0x7fe9280abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92806e280 con 0x7fe92814d9e0 2024-04-25T10:03:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92806e100 con 0x7fe92814c5c0 2024-04-25T10:03:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92806e400 con 0x7fe90c004800 2024-04-25T10:03:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe91fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 0x7fe9280cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe91fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 0x7fe9280cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44862/0 (socket says 172.21.15.5:44862) 2024-04-25T10:03:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe91fe00640 1 -- 172.21.15.5:0/1214856639 learned_addr learned my addr 172.21.15.5:0/1214856639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe926a00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c004800 0x7fe92814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.316+0000 7fe927400640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe92814c5c0 0x7fe9280abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91fe00640 1 -- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c004800 msgr2=0x7fe92814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91fe00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c004800 0x7fe92814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91fe00640 1 -- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe92814c5c0 msgr2=0x7fe9280abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91fe00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe92814c5c0 0x7fe9280abe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91fe00640 1 -- 172.21.15.5:0/1214856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92814b520 con 0x7fe92814d9e0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91fe00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 0x7fe9280cc150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe910000c00 tx=0x7fe910001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91003f030 con 0x7fe92814d9e0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910005480 con 0x7fe92814d9e0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910037040 con 0x7fe92814d9e0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.317+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92814b800 con 0x7fe92814d9e0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.318+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92814bdc0 con 0x7fe92814d9e0 2024-04-25T10:03:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.319+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92815b380 con 0x7fe92814d9e0 2024-04-25T10:03:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.320+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe910042070 con 0x7fe92814d9e0 2024-04-25T10:03:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.321+0000 7fe91ea00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe90406b4d0 0x7fe90406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.321+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6590+0+0 (secure 0 0 0) 0x7fe9100a9bb0 con 0x7fe92814d9e0 2024-04-25T10:03:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.321+0000 7fe926a00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe90406b4d0 0x7fe90406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.323+0000 7fe926a00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe90406b4d0 0x7fe90406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe92806ecf0 tx=0x7fe918008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.325+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe92815b380 con 0x7fe92814d9e0 2024-04-25T10:03:23.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:23.503 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.566+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe928068f60 con 0x7fe92814d9e0 2024-04-25T10:03:23.572 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:03:23.567+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.570+0000 7fe91ea00640 1 -- 172.21.15.5:0/1214856639 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe910078590 con 0x7fe92814d9e0 2024-04-25T10:03:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe90406b4d0 msgr2=0x7fe90406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe90406b4d0 0x7fe90406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe92806ecf0 tx=0x7fe918008040 comp rx=0 tx=0).stop 2024-04-25T10:03:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 msgr2=0x7fe9280cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 0x7fe9280cc150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe910000c00 tx=0x7fe910001120 comp rx=0 tx=0).stop 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 shutdown_connections 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe90406b4d0 0x7fe90406d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe92814c5c0 0x7fe9280abe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92814d9e0 0x7fe9280cc150 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 --2- 172.21.15.5:0/1214856639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c004800 0x7fe92814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.573+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 >> 172.21.15.5:0/1214856639 conn(0x7fe928066f20 msgr2=0x7fe9280b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.574+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 shutdown_connections 2024-04-25T10:03:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.574+0000 7fe927e00640 1 -- 172.21.15.5:0/1214856639 wait complete. 2024-04-25T10:03:23.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:23.616 INFO:tasks.ceph.mds.b.smithi080.stderr: -6370> 2024-04-25T10:01:58.785+0000 7fa933a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:23.706 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:03:23.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.808+0000 7f619b000640 1 Processor -- start 2024-04-25T10:03:23.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.808+0000 7f619b000640 1 -- start start 2024-04-25T10:03:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f619b000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f61940b3bb0 0x7f6194071190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f619b000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61940b2df0 con 0x7f61940716d0 2024-04-25T10:03:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f619b000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61940b30d0 con 0x7f619406e130 2024-04-25T10:03:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f619b000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61941440c0 con 0x7f61940b3bb0 2024-04-25T10:03:23.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f619a600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f61940716d0 0x7f619415a9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2024-04-25T10:03:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f619a600640 1 -- 172.21.15.5:0/145496215 learned_addr learned my addr 172.21.15.5:0/145496215 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.810+0000 7f6199200640 1 --2- 172.21.15.5:0/145496215 >> v2:172.21.15.80:3300/0 conn(0x7f61940b3bb0 0x7f6194071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.811+0000 7f6193e00640 1 -- 172.21.15.5:0/145496215 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804120397 0 0) 0x7f61940b2df0 con 0x7f61940716d0 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.811+0000 7f6193e00640 1 -- 172.21.15.5:0/145496215 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f617c005730 con 0x7f61940716d0 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.811+0000 7f6199200640 1 -- 172.21.15.5:0/145496215 >> v1:172.21.15.80:6789/0 conn(0x7f619406e130 legacy=0x7f619406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.811+0000 7f6199200640 1 -- 172.21.15.5:0/145496215 >> v1:172.21.15.5:6789/0 conn(0x7f61940716d0 legacy=0x7f619415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.811+0000 7f6199200640 1 -- 172.21.15.5:0/145496215 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940b33b0 con 0x7f61940b3bb0 2024-04-25T10:03:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.812+0000 7f6199200640 1 --2- 172.21.15.5:0/145496215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940b3bb0 0x7f6194071190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61840097a0 tx=0x7f618402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3064b23f847e93b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.813+0000 7f6193e00640 1 -- 172.21.15.5:0/145496215 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184037070 con 0x7f61940b3bb0 2024-04-25T10:03:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.813+0000 7f6193e00640 1 -- 172.21.15.5:0/145496215 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6184032440 con 0x7f61940b3bb0 2024-04-25T10:03:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.813+0000 7f6193e00640 1 -- 172.21.15.5:0/145496215 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840327f0 con 0x7f61940b3bb0 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.813+0000 7f619b000640 1 -- 172.21.15.5:0/145496215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940b3bb0 msgr2=0x7f6194071190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.813+0000 7f619b000640 1 --2- 172.21.15.5:0/145496215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940b3bb0 0x7f6194071190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61840097a0 tx=0x7f618402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.814+0000 7f619b000640 1 -- 172.21.15.5:0/145496215 shutdown_connections 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.814+0000 7f619b000640 1 --2- 172.21.15.5:0/145496215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940b3bb0 0x7f6194071190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.814+0000 7f619b000640 1 -- 172.21.15.5:0/145496215 >> 172.21.15.5:0/145496215 conn(0x7f6194066f20 msgr2=0x7f619406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.814+0000 7f619b000640 1 -- 172.21.15.5:0/145496215 shutdown_connections 2024-04-25T10:03:23.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.814+0000 7f619b000640 1 -- 172.21.15.5:0/145496215 wait complete. 2024-04-25T10:03:23.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.815+0000 7f619b000640 1 Processor -- start 2024-04-25T10:03:23.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.816+0000 7f619b000640 1 -- start start 2024-04-25T10:03:23.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.816+0000 7f619b000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f619406e130 0x7f61941f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619b000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940716d0 0x7f61941e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619b000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 0x7f619415c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619b000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194071b20 con 0x7f61940b3bb0 2024-04-25T10:03:23.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619b000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61940b4010 con 0x7f61940716d0 2024-04-25T10:03:23.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619b000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6194071ca0 con 0x7f619406e130 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 0x7f619415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 0x7f619415c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44866/0 (socket says 172.21.15.5:44866) 2024-04-25T10:03:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 -- 172.21.15.5:0/3940113495 learned_addr learned my addr 172.21.15.5:0/3940113495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f6199200640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940716d0 0x7f61941e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 -- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f619406e130 msgr2=0x7f61941f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f6199c00640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f619406e130 0x7f61941f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f619406e130 0x7f61941f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 -- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940716d0 msgr2=0x7f61941e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940716d0 0x7f61941e89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.817+0000 7f619a600640 1 -- 172.21.15.5:0/3940113495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61941e60a0 con 0x7f61940b3bb0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.818+0000 7f619a600640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 0x7f619415c540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f618c00cb40 tx=0x7f618c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.818+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c017070 con 0x7f61940b3bb0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.818+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c00aca0 con 0x7f61940b3bb0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.818+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c012620 con 0x7f61940b3bb0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.818+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61941e6380 con 0x7f61940b3bb0 2024-04-25T10:03:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.818+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61941e6940 con 0x7f61940b3bb0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.819+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61940cd4c0 con 0x7f61940b3bb0 2024-04-25T10:03:23.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.820+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f618c01a070 con 0x7f61940b3bb0 2024-04-25T10:03:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.822+0000 7f6191600640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f617806b4d0 0x7f617806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.822+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6676+0+0 (secure 0 0 0) 0x7f618c08a500 con 0x7f61940b3bb0 2024-04-25T10:03:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.822+0000 7f6199c00640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f617806b4d0 0x7f617806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.823+0000 7f6199c00640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f617806b4d0 0x7f617806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6188000f60 tx=0x7f6188009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:23.826+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61940cd4c0 con 0x7f61940b3bb0 2024-04-25T10:03:24.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.069+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6194068f60 con 0x7f61940b3bb0 2024-04-25T10:03:24.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.574+0000 7f6191600640 1 -- 172.21.15.5:0/3940113495 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f618c058f10 con 0x7f61940b3bb0 2024-04-25T10:03:24.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f617806b4d0 msgr2=0x7f617806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f617806b4d0 0x7f617806d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6188000f60 tx=0x7f6188009040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 msgr2=0x7f619415c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 0x7f619415c540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f618c00cb40 tx=0x7f618c00a060 comp rx=0 tx=0).stop 2024-04-25T10:03:24.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 shutdown_connections 2024-04-25T10:03:24.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61940b3bb0 0x7f619415c540 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f617806b4d0 0x7f617806d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61940716d0 0x7f61941e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 --2- 172.21.15.5:0/3940113495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f619406e130 0x7f61941f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.577+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 >> 172.21.15.5:0/3940113495 conn(0x7f6194066f20 msgr2=0x7f619413f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.578+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 shutdown_connections 2024-04-25T10:03:24.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.578+0000 7f619b000640 1 -- 172.21.15.5:0/3940113495 wait complete. 2024-04-25T10:03:24.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:24.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.809+0000 7fe2e8400640 1 Processor -- start 2024-04-25T10:03:24.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.809+0000 7fe2e8400640 1 -- start start 2024-04-25T10:03:24.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.810+0000 7fe2e8400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2e015b8c0 0x7fe2e00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.810+0000 7fe2e8400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e00d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.810+0000 7fe2e8400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00cc730 con 0x7fe2e00999f0 2024-04-25T10:03:24.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.810+0000 7fe2e8400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00d0670 con 0x7fe2e015b8c0 2024-04-25T10:03:24.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.810+0000 7fe2e8400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00cca10 con 0x7fe2e00cc250 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.810+0000 7fe2e5a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2e015b8c0 0x7fe2e00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e6400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe2e00999f0 0x7fe2e00c5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58904/0 (socket says 172.21.15.5:58904) 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e6400640 1 -- 172.21.15.5:0/2249893013 learned_addr learned my addr 172.21.15.5:0/2249893013 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e5000640 1 --2- 172.21.15.5:0/2249893013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e00d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e5a00640 1 -- 172.21.15.5:0/2249893013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 msgr2=0x7fe2e00d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e5a00640 1 --2- 172.21.15.5:0/2249893013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e00d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e5a00640 1 -- 172.21.15.5:0/2249893013 >> v1:172.21.15.5:6789/0 conn(0x7fe2e00999f0 legacy=0x7fe2e00c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2e5a00640 1 -- 172.21.15.5:0/2249893013 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00cccf0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.811+0000 7fe2dfe00640 1 -- 172.21.15.5:0/2249893013 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574425066 0 0) 0x7fe2e00cc730 con 0x7fe2e00999f0 2024-04-25T10:03:24.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.812+0000 7fe2e5a00640 1 --2- 172.21.15.5:0/2249893013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e00cd7d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009d90 tx=0x7fe2d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327acec96f5199d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2dfe00640 1 -- 172.21.15.5:0/2249893013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0007740 con 0x7fe2e015b8c0 2024-04-25T10:03:24.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2dfe00640 1 -- 172.21.15.5:0/2249893013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00078e0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2dfe00640 1 -- 172.21.15.5:0/2249893013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0007cb0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 -- 172.21.15.5:0/2249893013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 msgr2=0x7fe2e00cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 --2- 172.21.15.5:0/2249893013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e00cd7d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009d90 tx=0x7fe2d002a660 comp rx=0 tx=0).stop 2024-04-25T10:03:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 -- 172.21.15.5:0/2249893013 shutdown_connections 2024-04-25T10:03:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 --2- 172.21.15.5:0/2249893013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e00d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 --2- 172.21.15.5:0/2249893013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e00cd7d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 -- 172.21.15.5:0/2249893013 >> 172.21.15.5:0/2249893013 conn(0x7fe2e00b3130 msgr2=0x7fe2e00b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 -- 172.21.15.5:0/2249893013 shutdown_connections 2024-04-25T10:03:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.813+0000 7fe2e8400640 1 -- 172.21.15.5:0/2249893013 wait complete. 2024-04-25T10:03:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.815+0000 7fe2e8400640 1 Processor -- start 2024-04-25T10:03:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.815+0000 7fe2e8400640 1 -- start start 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.816+0000 7fe2e8400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2e00999f0 0x7fe2e00b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.816+0000 7fe2e8400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.816+0000 7fe2e8400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.816+0000 7fe2e8400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00cf480 con 0x7fe2e015b8c0 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.816+0000 7fe2e8400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00cd2b0 con 0x7fe2e00999f0 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.816+0000 7fe2e8400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e015bca0 con 0x7fe2e00cc250 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e014f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44890/0 (socket says 172.21.15.5:44890) 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 -- 172.21.15.5:0/1509641988 learned_addr learned my addr 172.21.15.5:0/1509641988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e5a00640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2e00999f0 0x7fe2e00b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e5000640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 -- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 msgr2=0x7fe2e015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e015bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 -- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2e00999f0 msgr2=0x7fe2e00b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2e00999f0 0x7fe2e00b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.817+0000 7fe2e6400640 1 -- 172.21.15.5:0/1509641988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00c9a50 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.818+0000 7fe2e6400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e014f9c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c410 tx=0x7fe2d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.818+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d80074e0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.818+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d80047e0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.818+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8004bd0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.818+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00c9d30 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.818+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00ca2f0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.819+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e0069040 con 0x7fe2e015b8c0 2024-04-25T10:03:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.820+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2d8011070 con 0x7fe2e015b8c0 2024-04-25T10:03:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.821+0000 7fe2dd600640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.821+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6676+0+0 (secure 0 0 0) 0x7fe2d808a1d0 con 0x7fe2e015b8c0 2024-04-25T10:03:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.822+0000 7fe2e5a00640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.823+0000 7fe2e5a00640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe2c006b500 0x7fe2c006d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:24.826+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2d801a2b0 con 0x7fe2e015b8c0 2024-04-25T10:03:25.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.025+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2e00ca4c0 con 0x7fe2e015b8c0 2024-04-25T10:03:25.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.968+0000 7fe2dd600640 1 -- 172.21.15.5:0/1509641988 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe2d8058be0 con 0x7fe2e015b8c0 2024-04-25T10:03:25.973 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.973+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe2c006b500 msgr2=0x7fe2c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.973+0000 7fe2e8400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe2c006b500 0x7fe2c006d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d002f000 comp rx=0 tx=0).stop 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.973+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 msgr2=0x7fe2e014f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.973+0000 7fe2e8400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e014f9c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800c410 tx=0x7fe2d800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.974+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 shutdown_connections 2024-04-25T10:03:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.974+0000 7fe2e8400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2e015b8c0 0x7fe2e014f9c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.974+0000 7fe2e8400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2e00cc250 0x7fe2e015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.974+0000 7fe2e8400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe2c006b500 0x7fe2c006d9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.974+0000 7fe2e8400640 1 --2- 172.21.15.5:0/1509641988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2e00999f0 0x7fe2e00b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.974+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 >> 172.21.15.5:0/1509641988 conn(0x7fe2e00b3130 msgr2=0x7fe2e0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.975+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 shutdown_connections 2024-04-25T10:03:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:25.975+0000 7fe2e8400640 1 -- 172.21.15.5:0/1509641988 wait complete. 2024-04-25T10:03:26.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:26.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.206+0000 7fb597200640 1 Processor -- start 2024-04-25T10:03:26.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.206+0000 7fb597200640 1 -- start start 2024-04-25T10:03:26.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.208+0000 7fb597200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb59015c870 0x7fb590071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.208+0000 7fb597200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb590071540 0x7fb5900cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.208+0000 7fb597200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59006e850 con 0x7fb590144e20 2024-04-25T10:03:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.208+0000 7fb597200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5900719b0 con 0x7fb590071540 2024-04-25T10:03:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.208+0000 7fb597200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59015fdf0 con 0x7fb59015c870 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb590071540 0x7fb5900cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb590071540 0x7fb5900cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44896/0 (socket says 172.21.15.5:44896) 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 -- 172.21.15.5:0/2358678254 learned_addr learned my addr 172.21.15.5:0/2358678254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb595400640 1 --2- 172.21.15.5:0/2358678254 >> v2:172.21.15.80:3300/0 conn(0x7fb59015c870 0x7fb590071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 -- 172.21.15.5:0/2358678254 >> v2:172.21.15.80:3300/0 conn(0x7fb59015c870 msgr2=0x7fb590071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 --2- 172.21.15.5:0/2358678254 >> v2:172.21.15.80:3300/0 conn(0x7fb59015c870 0x7fb590071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 -- 172.21.15.5:0/2358678254 >> v1:172.21.15.80:6789/0 conn(0x7fb590144e20 legacy=0x7fb59015f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.209+0000 7fb596800640 1 -- 172.21.15.5:0/2358678254 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5901600d0 con 0x7fb590071540 2024-04-25T10:03:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.210+0000 7fb596800640 1 --2- 172.21.15.5:0/2358678254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590071540 0x7fb5900cbd60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb59006e420 tx=0x7fb58c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8366820573fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb594a00640 1 -- 172.21.15.5:0/2358678254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c036070 con 0x7fb590071540 2024-04-25T10:03:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb594a00640 1 -- 172.21.15.5:0/2358678254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c031600 con 0x7fb590071540 2024-04-25T10:03:26.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb594a00640 1 -- 172.21.15.5:0/2358678254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0319d0 con 0x7fb590071540 2024-04-25T10:03:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb597200640 1 -- 172.21.15.5:0/2358678254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590071540 msgr2=0x7fb5900cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb597200640 1 --2- 172.21.15.5:0/2358678254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590071540 0x7fb5900cbd60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb59006e420 tx=0x7fb58c02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb597200640 1 -- 172.21.15.5:0/2358678254 shutdown_connections 2024-04-25T10:03:26.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb597200640 1 --2- 172.21.15.5:0/2358678254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590071540 0x7fb5900cbd60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.211+0000 7fb597200640 1 --2- 172.21.15.5:0/2358678254 >> v2:172.21.15.80:3300/0 conn(0x7fb59015c870 0x7fb590071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.212+0000 7fb597200640 1 -- 172.21.15.5:0/2358678254 >> 172.21.15.5:0/2358678254 conn(0x7fb590066f20 msgr2=0x7fb59006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.212+0000 7fb597200640 1 -- 172.21.15.5:0/2358678254 shutdown_connections 2024-04-25T10:03:26.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.212+0000 7fb597200640 1 -- 172.21.15.5:0/2358678254 wait complete. 2024-04-25T10:03:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.213+0000 7fb597200640 1 Processor -- start 2024-04-25T10:03:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.213+0000 7fb597200640 1 -- start start 2024-04-25T10:03:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.214+0000 7fb597200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb590071540 0x7fb5901404d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.214+0000 7fb597200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 0x7fb5901e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.214+0000 7fb597200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb59015c870 0x7fb590142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.214+0000 7fb597200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb590145200 con 0x7fb590144e20 2024-04-25T10:03:26.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.214+0000 7fb597200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015c070 con 0x7fb59015c870 2024-04-25T10:03:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.214+0000 7fb597200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb590145380 con 0x7fb590071540 2024-04-25T10:03:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 0x7fb5901e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 0x7fb5901e59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44900/0 (socket says 172.21.15.5:44900) 2024-04-25T10:03:26.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 -- 172.21.15.5:0/3343369578 learned_addr learned my addr 172.21.15.5:0/3343369578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb596800640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb59015c870 0x7fb590142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 -- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb590071540 msgr2=0x7fb5901404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb590071540 0x7fb5901404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 -- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb59015c870 msgr2=0x7fb590142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb59015c870 0x7fb590142400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.215+0000 7fb595400640 1 -- 172.21.15.5:0/3343369578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59013e9e0 con 0x7fb590144e20 2024-04-25T10:03:26.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.216+0000 7fb595400640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 0x7fb5901e59c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d5f0 tx=0x7fb58400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.216+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840125f0 con 0x7fb590144e20 2024-04-25T10:03:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.216+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb584019070 con 0x7fb590144e20 2024-04-25T10:03:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.216+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5840129d0 con 0x7fb590144e20 2024-04-25T10:03:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.216+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5901e4fb0 con 0x7fb590144e20 2024-04-25T10:03:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.216+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5901e5570 con 0x7fb590144e20 2024-04-25T10:03:26.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.217+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb590067420 con 0x7fb590144e20 2024-04-25T10:03:26.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.218+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb584012b70 con 0x7fb590144e20 2024-04-25T10:03:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.220+0000 7fb58a000640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb57406b500 0x7fb57406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.220+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fb58408a870 con 0x7fb590144e20 2024-04-25T10:03:26.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.220+0000 7fb595e00640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb57406b500 0x7fb57406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.221+0000 7fb595e00640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb57406b500 0x7fb57406d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb580006fd0 tx=0x7fb580006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.225+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb590067420 con 0x7fb590144e20 2024-04-25T10:03:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.431+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb590068f60 con 0x7fb590144e20 2024-04-25T10:03:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.980+0000 7fb58a000640 1 -- 172.21.15.5:0/3343369578 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb584059440 con 0x7fb590144e20 2024-04-25T10:03:26.985 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.985+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb57406b500 msgr2=0x7fb57406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.985+0000 7fb597200640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb57406b500 0x7fb57406d9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb580006fd0 tx=0x7fb580006bc0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.986+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 msgr2=0x7fb5901e59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.986+0000 7fb597200640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 0x7fb5901e59c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d5f0 tx=0x7fb58400dab0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 shutdown_connections 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb57406b500 0x7fb57406d9c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb59015c870 0x7fb590142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb590144e20 0x7fb5901e59c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 --2- 172.21.15.5:0/3343369578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb590071540 0x7fb5901404d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 >> 172.21.15.5:0/3343369578 conn(0x7fb590066f20 msgr2=0x7fb5900b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.989+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 shutdown_connections 2024-04-25T10:03:26.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:26.990+0000 7fb597200640 1 -- 172.21.15.5:0/3343369578 wait complete. 2024-04-25T10:03:27.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T10:03:27.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.220+0000 7fea97000640 1 Processor -- start 2024-04-25T10:03:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.221+0000 7fea97000640 1 -- start start 2024-04-25T10:03:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.222+0000 7fea97000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fea9006f340 0x7fea9015d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.222+0000 7fea97000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9015de60 0x7fea9006abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.222+0000 7fea97000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea900aded0 con 0x7fea9006ef60 2024-04-25T10:03:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.222+0000 7fea97000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea900ae1b0 con 0x7fea9006f340 2024-04-25T10:03:27.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.222+0000 7fea97000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900ae490 con 0x7fea9015de60 2024-04-25T10:03:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.223+0000 7fea8f400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fea9006f340 0x7fea9015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.223+0000 7fea8fe00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fea9006ef60 0x7fea900ad6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:58688/0 (socket says 172.21.15.5:58688) 2024-04-25T10:03:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.223+0000 7fea8fe00640 1 -- 172.21.15.5:0/3260604800 learned_addr learned my addr 172.21.15.5:0/3260604800 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:27.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.224+0000 7fea95000640 1 --2- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9015de60 0x7fea9006abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.225+0000 7fea8f400640 1 -- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9015de60 msgr2=0x7fea9006abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:27.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.225+0000 7fea8f400640 1 --2- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9015de60 0x7fea9006abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.225+0000 7fea8f400640 1 -- 172.21.15.5:0/3260604800 >> v1:172.21.15.80:6789/0 conn(0x7fea9006ef60 legacy=0x7fea900ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.225+0000 7fea8f400640 1 -- 172.21.15.5:0/3260604800 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900ae770 con 0x7fea9006f340 2024-04-25T10:03:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.227+0000 7fea8f400640 1 --2- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea9015d920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea7800ba40 tx=0x7fea7802c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44d2afb1b50aa48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.227+0000 7fea8ea00640 1 -- 172.21.15.5:0/3260604800 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78031710 con 0x7fea9006f340 2024-04-25T10:03:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.227+0000 7fea8ea00640 1 -- 172.21.15.5:0/3260604800 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea780318b0 con 0x7fea9006f340 2024-04-25T10:03:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.227+0000 7fea8ea00640 1 -- 172.21.15.5:0/3260604800 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78031ca0 con 0x7fea9006f340 2024-04-25T10:03:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 -- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 msgr2=0x7fea9015d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 --2- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea9015d920 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea7800ba40 tx=0x7fea7802c7c0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 -- 172.21.15.5:0/3260604800 shutdown_connections 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 --2- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9015de60 0x7fea9006abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 --2- 172.21.15.5:0/3260604800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea9015d920 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 -- 172.21.15.5:0/3260604800 >> 172.21.15.5:0/3260604800 conn(0x7fea900b2300 msgr2=0x7fea900cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 -- 172.21.15.5:0/3260604800 shutdown_connections 2024-04-25T10:03:27.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.228+0000 7fea97000640 1 -- 172.21.15.5:0/3260604800 wait complete. 2024-04-25T10:03:27.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.230+0000 7fea97000640 1 Processor -- start 2024-04-25T10:03:27.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.230+0000 7fea97000640 1 -- start start 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea97000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9006ef60 0x7fea90151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea97000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea90151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea97000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 0x7fea901f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea97000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90066a70 con 0x7fea9015de60 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea97000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea900668f0 con 0x7fea9006f340 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea97000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea90066bf0 con 0x7fea9006ef60 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea95000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 0x7fea901f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea95000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 0x7fea901f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41366/0 (socket says 172.21.15.5:41366) 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea95000640 1 -- 172.21.15.5:0/3075734723 learned_addr learned my addr 172.21.15.5:0/3075734723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea8f400640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea90151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.231+0000 7fea8fe00640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9006ef60 0x7fea90151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea95000640 1 -- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9006ef60 msgr2=0x7fea90151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea95000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9006ef60 0x7fea90151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea95000640 1 -- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 msgr2=0x7fea90151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea95000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea90151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea95000640 1 -- 172.21.15.5:0/3075734723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90152710 con 0x7fea9015de60 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea95000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 0x7fea901f3790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea88009d30 tx=0x7fea8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.232+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88018020 con 0x7fea9015de60 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.233+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88014480 con 0x7fea9015de60 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.233+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88014870 con 0x7fea9015de60 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.233+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea90152150 con 0x7fea9015de60 2024-04-25T10:03:27.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.233+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea90155ff0 con 0x7fea9015de60 2024-04-25T10:03:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.234+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90069040 con 0x7fea9015de60 2024-04-25T10:03:27.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.235+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea88005d10 con 0x7fea9015de60 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.236+0000 7fea87e00640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea6c06b500 0x7fea6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.236+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fea8808ab30 con 0x7fea9015de60 2024-04-25T10:03:27.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.237+0000 7fea8fe00640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea6c06b500 0x7fea6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.238+0000 7fea8fe00640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea6c06b500 0x7fea6c06d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea80005850 tx=0x7fea80001000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.241+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea88022070 con 0x7fea9015de60 2024-04-25T10:03:27.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.447+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fea9006e160 con 0x7fea9015de60 2024-04-25T10:03:27.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.984+0000 7fea87e00640 1 -- 172.21.15.5:0/3075734723 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7fea88010040 con 0x7fea9015de60 2024-04-25T10:03:27.989 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' removed 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.988+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea6c06b500 msgr2=0x7fea6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.988+0000 7fea97000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea6c06b500 0x7fea6c06d9c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea80005850 tx=0x7fea80001000 comp rx=0 tx=0).stop 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.988+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 msgr2=0x7fea901f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.988+0000 7fea97000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 0x7fea901f3790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fea88009d30 tx=0x7fea8800c490 comp rx=0 tx=0).stop 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.989+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 shutdown_connections 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.989+0000 7fea97000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea9015de60 0x7fea901f3790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.989+0000 7fea97000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea6c06b500 0x7fea6c06d9c0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea80005850 tx=0x7fea80001000 comp rx=0 tx=0).stop 2024-04-25T10:03:27.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.989+0000 7fea97000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9006f340 0x7fea90151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.989+0000 7fea97000640 1 --2- 172.21.15.5:0/3075734723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9006ef60 0x7fea90151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.989+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 >> 172.21.15.5:0/3075734723 conn(0x7fea900b2300 msgr2=0x7fea90158980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.990+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 shutdown_connections 2024-04-25T10:03:27.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:27.990+0000 7fea97000640 1 -- 172.21.15.5:0/3075734723 wait complete. 2024-04-25T10:03:28.019 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:03:28.019 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:03:28.019 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:28.019 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:28.020 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:03:28.020 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:03:28.020 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:28.020 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:28.020 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:03:28.020 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:03:28.021 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:28.021 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:28.021 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:03:28.021 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:03:28.021 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:28.021 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:28.044 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:28.044 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:03:28.045 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:28.045 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:03:28.046 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:28.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:03:28.054 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:28.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:03:28.092 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:03:28.092 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:28.097 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:03:28.098 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:28.102 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:03:28.102 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:28.108 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:03:28.109 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:28.109 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:03:28.242 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:03:28.242 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:03:28.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:03:28.255 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:03:28.257 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:03:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.295+0000 7f0087200640 1 Processor -- start 2024-04-25T10:03:28.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.295+0000 7f0087200640 1 -- start start 2024-04-25T10:03:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0087200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800d0a50 0x7f00800d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0087200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00800b53b0 0x7f00800d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0087200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00800d1e80 con 0x7f008015db00 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0087200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00800b5930 con 0x7f00800b53b0 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0087200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800b5c10 con 0x7f00800d0a50 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0085200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00800b53b0 0x7f00800d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f007fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f008015db00 0x7f00800d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47870/0 (socket says 172.21.15.5:47870) 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f007fe00640 1 -- 172.21.15.5:0/81881209 learned_addr learned my addr 172.21.15.5:0/81881209 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:28.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0077e00640 1 --2- 172.21.15.5:0/81881209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800d0a50 0x7f00800d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0085200640 1 -- 172.21.15.5:0/81881209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800d0a50 msgr2=0x7f00800d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0085200640 1 --2- 172.21.15.5:0/81881209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800d0a50 0x7f00800d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.297+0000 7f0085200640 1 -- 172.21.15.5:0/81881209 >> v1:172.21.15.5:6789/0 conn(0x7f008015db00 legacy=0x7f00800d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.298+0000 7f0085200640 1 -- 172.21.15.5:0/81881209 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800b5ef0 con 0x7f00800b53b0 2024-04-25T10:03:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.298+0000 7f007f400640 1 -- 172.21.15.5:0/81881209 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248998755 0 0) 0x7f00800d1e80 con 0x7f008015db00 2024-04-25T10:03:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.299+0000 7f0085200640 1 --2- 172.21.15.5:0/81881209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00800b53b0 0x7f00800d2870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0078009d90 tx=0x7f007802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468bd14df086b982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.299+0000 7f007f400640 1 -- 172.21.15.5:0/81881209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078007740 con 0x7f00800b53b0 2024-04-25T10:03:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.299+0000 7f007f400640 1 -- 172.21.15.5:0/81881209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00780078e0 con 0x7f00800b53b0 2024-04-25T10:03:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.299+0000 7f007f400640 1 -- 172.21.15.5:0/81881209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078007cb0 con 0x7f00800b53b0 2024-04-25T10:03:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.299+0000 7f0087200640 1 -- 172.21.15.5:0/81881209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00800b53b0 msgr2=0x7f00800d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.299+0000 7f0087200640 1 --2- 172.21.15.5:0/81881209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00800b53b0 0x7f00800d2870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0078009d90 tx=0x7f007802a660 comp rx=0 tx=0).stop 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.300+0000 7f0087200640 1 -- 172.21.15.5:0/81881209 shutdown_connections 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.300+0000 7f0087200640 1 --2- 172.21.15.5:0/81881209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00800b53b0 0x7f00800d2870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.300+0000 7f0087200640 1 --2- 172.21.15.5:0/81881209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800d0a50 0x7f00800d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.300+0000 7f0087200640 1 -- 172.21.15.5:0/81881209 >> 172.21.15.5:0/81881209 conn(0x7f0080072290 msgr2=0x7f00800726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.300+0000 7f0087200640 1 -- 172.21.15.5:0/81881209 shutdown_connections 2024-04-25T10:03:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.300+0000 7f0087200640 1 -- 172.21.15.5:0/81881209 wait complete. 2024-04-25T10:03:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.301+0000 7f0087200640 1 Processor -- start 2024-04-25T10:03:28.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.302+0000 7f0087200640 1 -- start start 2024-04-25T10:03:28.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0087200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800b53b0 0x7f00801f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0087200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00800d0a50 0x7f00801f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0087200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 0x7f008015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0087200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800d3660 con 0x7f008015db00 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0087200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00800b61d0 con 0x7f00800d0a50 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0087200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00800ce2b0 con 0x7f00800b53b0 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0085200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 0x7f008015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0085200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 0x7f008015fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41418/0 (socket says 172.21.15.5:41418) 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0085200640 1 -- 172.21.15.5:0/3714479588 learned_addr learned my addr 172.21.15.5:0/3714479588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f007fe00640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800b53b0 0x7f00801f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0077e00640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00800d0a50 0x7f00801f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0085200640 1 -- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800b53b0 msgr2=0x7f00801f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.303+0000 7f0085200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800b53b0 0x7f00801f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0085200640 1 -- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00800d0a50 msgr2=0x7f00801f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0085200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00800d0a50 0x7f00801f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0085200640 1 -- 172.21.15.5:0/3714479588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00801f6b90 con 0x7f008015db00 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0077e00640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00800d0a50 0x7f00801f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:28.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0085200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 0x7f008015fb50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f007802ac50 tx=0x7f0078004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078040070 con 0x7f008015db00 2024-04-25T10:03:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0078031bc0 con 0x7f008015db00 2024-04-25T10:03:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078045460 con 0x7f008015db00 2024-04-25T10:03:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008015eab0 con 0x7f008015db00 2024-04-25T10:03:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.304+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008015f070 con 0x7f008015db00 2024-04-25T10:03:28.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.305+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0080068fd0 con 0x7f008015db00 2024-04-25T10:03:28.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.307+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0078039040 con 0x7f008015db00 2024-04-25T10:03:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.307+0000 7f007cc00640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f005c06b4d0 0x7f005c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.308+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f00780ab990 con 0x7f008015db00 2024-04-25T10:03:28.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.308+0000 7f007fe00640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f005c06b4d0 0x7f005c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.309+0000 7f007fe00640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f005c06b4d0 0x7f005c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0070007fd0 tx=0x7f0070007de0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.312+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f007807a970 con 0x7f008015db00 2024-04-25T10:03:28.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:28.529+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f00800d16b0 con 0x7f008015db00 2024-04-25T10:03:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.027+0000 7f007cc00640 1 -- 172.21.15.5:0/3714479588 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f007807a270 con 0x7f008015db00 2024-04-25T10:03:29.032 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:03:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.032+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f005c06b4d0 msgr2=0x7f005c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.032+0000 7f0087200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f005c06b4d0 0x7f005c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0070007fd0 tx=0x7f0070007de0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.032+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 msgr2=0x7f008015fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.032+0000 7f0087200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 0x7f008015fb50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f007802ac50 tx=0x7f0078004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 shutdown_connections 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008015db00 0x7f008015fb50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f005c06b4d0 0x7f005c06d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00800d0a50 0x7f00801f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 --2- 172.21.15.5:0/3714479588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00800b53b0 0x7f00801f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 >> 172.21.15.5:0/3714479588 conn(0x7f0080072290 msgr2=0x7f008015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 shutdown_connections 2024-04-25T10:03:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.033+0000 7f0087200640 1 -- 172.21.15.5:0/3714479588 wait complete. 2024-04-25T10:03:29.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:03:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.245+0000 7f2696c00640 1 Processor -- start 2024-04-25T10:03:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.246+0000 7f2696c00640 1 -- start start 2024-04-25T10:03:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f2696c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26900ce510 0x7f26900ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f2696c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26900d2290 0x7f26900d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f2696c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2690161fe0 con 0x7f269015d930 2024-04-25T10:03:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f2696c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26900ceeb0 con 0x7f26900ce510 2024-04-25T10:03:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f2696c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900d1e00 con 0x7f26900d2290 2024-04-25T10:03:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f268f400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26900ce510 0x7f26900ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.248+0000 7f2694c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26900d2290 0x7f26900d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.249+0000 7f268f400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26900ce510 0x7f26900ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:56702/0 (socket says 172.21.15.5:56702) 2024-04-25T10:03:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.249+0000 7f268f400640 1 -- 172.21.15.5:0/1217879574 learned_addr learned my addr 172.21.15.5:0/1217879574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.250+0000 7f268f400640 1 -- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26900d2290 msgr2=0x7f26900d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.250+0000 7f268f400640 1 --2- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26900d2290 0x7f26900d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.250+0000 7f268f400640 1 -- 172.21.15.5:0/1217879574 >> v1:172.21.15.80:6789/0 conn(0x7f269015d930 legacy=0x7f2690161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.250+0000 7f268f400640 1 -- 172.21.15.5:0/1217879574 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26901622c0 con 0x7f26900ce510 2024-04-25T10:03:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.250+0000 7f2694c00640 1 --2- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26900d2290 0x7f26900d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.251+0000 7f268f400640 1 --2- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900ce510 0x7f26900ce970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f267c009e50 tx=0x7f267c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e836da7531f052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.251+0000 7f268ea00640 1 -- 172.21.15.5:0/1217879574 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c036070 con 0x7f26900ce510 2024-04-25T10:03:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.251+0000 7f268ea00640 1 -- 172.21.15.5:0/1217879574 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c031600 con 0x7f26900ce510 2024-04-25T10:03:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.251+0000 7f268ea00640 1 -- 172.21.15.5:0/1217879574 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0319d0 con 0x7f26900ce510 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.252+0000 7f2696c00640 1 -- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900ce510 msgr2=0x7f26900ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.252+0000 7f2696c00640 1 --2- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900ce510 0x7f26900ce970 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f267c009e50 tx=0x7f267c02a710 comp rx=0 tx=0).stop 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.253+0000 7f2696c00640 1 -- 172.21.15.5:0/1217879574 shutdown_connections 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.253+0000 7f2696c00640 1 --2- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26900d2290 0x7f26900d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.253+0000 7f2696c00640 1 --2- 172.21.15.5:0/1217879574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900ce510 0x7f26900ce970 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.253+0000 7f2696c00640 1 -- 172.21.15.5:0/1217879574 >> 172.21.15.5:0/1217879574 conn(0x7f26900d06e0 msgr2=0x7f26900d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.253+0000 7f2696c00640 1 -- 172.21.15.5:0/1217879574 shutdown_connections 2024-04-25T10:03:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.253+0000 7f2696c00640 1 -- 172.21.15.5:0/1217879574 wait complete. 2024-04-25T10:03:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.255+0000 7f2696c00640 1 Processor -- start 2024-04-25T10:03:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.255+0000 7f2696c00640 1 -- start start 2024-04-25T10:03:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.256+0000 7f2696c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 0x7f26900c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.256+0000 7f2696c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900d2290 0x7f269015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.256+0000 7f2696c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269015d930 0x7f26901607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.256+0000 7f2696c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269015dd10 con 0x7f26900ce510 2024-04-25T10:03:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.256+0000 7f2696c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2690161ca0 con 0x7f26900d2290 2024-04-25T10:03:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.256+0000 7f2696c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269015de90 con 0x7f269015d930 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 0x7f26900c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 0x7f26900c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41422/0 (socket says 172.21.15.5:41422) 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 -- 172.21.15.5:0/3426012247 learned_addr learned my addr 172.21.15.5:0/3426012247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268f400640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900d2290 0x7f269015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f2694c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269015d930 0x7f26901607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 -- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269015d930 msgr2=0x7f26901607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269015d930 0x7f26901607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 -- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900d2290 msgr2=0x7f269015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900d2290 0x7f269015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f268fe00640 1 -- 172.21.15.5:0/3426012247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269015ec60 con 0x7f26900ce510 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.257+0000 7f2694c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269015d930 0x7f26901607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:29.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.258+0000 7f268fe00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 0x7f26900c9920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f268000c3c0 tx=0x7f268000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.258+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800052d0 con 0x7f26900ce510 2024-04-25T10:03:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.258+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2680005490 con 0x7f26900ce510 2024-04-25T10:03:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.258+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269015ef00 con 0x7f26900ce510 2024-04-25T10:03:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.258+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2680007260 con 0x7f26900ce510 2024-04-25T10:03:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.258+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269015f4c0 con 0x7f26900ce510 2024-04-25T10:03:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.259+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2690074bb0 con 0x7f26900ce510 2024-04-25T10:03:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.260+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2680007400 con 0x7f26900ce510 2024-04-25T10:03:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.261+0000 7f2687e00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267006b500 0x7f267006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.261+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f26800894a0 con 0x7f26900ce510 2024-04-25T10:03:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.262+0000 7f268f400640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267006b500 0x7f267006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.263+0000 7f268f400640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267006b500 0x7f267006d9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f267c033000 tx=0x7f267c009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.267+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26800584d0 con 0x7f26900ce510 2024-04-25T10:03:29.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.535+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f26900d14a0 con 0x7f26900ce510 2024-04-25T10:03:29.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.536+0000 7f2687e00640 1 -- 172.21.15.5:0/3426012247 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2680057dd0 con 0x7f26900ce510 2024-04-25T10:03:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:29.540 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:03:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:03:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD9KSpm8JjREhAATmw7p7smlJebuGBPe5edNw==", 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:03:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD9KSpmZqLREhAAzvCd9GI7uVaXMqbFb122nQ==", 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD9KSpmW63REhAA1ALabbThLpMvRhXvZmM2dA==", 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD9KSpmv7XREhAAt9JOZRmcnhZra3d7T47HVw==", 2024-04-25T10:03:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD9KSpmKb7REhAAWjKx9WCo7rlIqDFYckLv9Q==", 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD9KSpmusbREhAAKkjQIEF0NSwsrZFYdYFuwQ==", 2024-04-25T10:03:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:03:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:03:29.556 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:03:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.538+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267006b500 msgr2=0x7f267006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.538+0000 7f2696c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267006b500 0x7f267006d9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f267c033000 tx=0x7f267c009000 comp rx=0 tx=0).stop 2024-04-25T10:03:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.538+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 msgr2=0x7f26900c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 0x7f26900c9920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f268000c3c0 tx=0x7f268000c880 comp rx=0 tx=0).stop 2024-04-25T10:03:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 shutdown_connections 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269015d930 0x7f26901607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267006b500 0x7f267006d9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26900d2290 0x7f269015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 --2- 172.21.15.5:0/3426012247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26900ce510 0x7f26900c9920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 >> 172.21.15.5:0/3426012247 conn(0x7f26900d06e0 msgr2=0x7f26900d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 shutdown_connections 2024-04-25T10:03:29.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.539+0000 7f2696c00640 1 -- 172.21.15.5:0/3426012247 wait complete. 2024-04-25T10:03:29.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:03:29.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.751+0000 7fe214c00640 1 Processor -- start 2024-04-25T10:03:29.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.751+0000 7fe214c00640 1 -- start start 2024-04-25T10:03:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.752+0000 7fe214c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2101441c0 0x7fe2100b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.752+0000 7fe214c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe210071170 0x7fe2100cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.752+0000 7fe214c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe210071600 con 0x7fe21006e2a0 2024-04-25T10:03:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.753+0000 7fe214c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2100718e0 con 0x7fe2101441c0 2024-04-25T10:03:29.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.753+0000 7fe214c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe210071bc0 con 0x7fe210071170 2024-04-25T10:03:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.753+0000 7fe20de00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2101441c0 0x7fe2100b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.753+0000 7fe20de00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2101441c0 0x7fe2100b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:56718/0 (socket says 172.21.15.5:56718) 2024-04-25T10:03:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.753+0000 7fe20de00640 1 -- 172.21.15.5:0/4130405485 learned_addr learned my addr 172.21.15.5:0/4130405485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.753+0000 7fe207e00640 1 --2- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe210071170 0x7fe2100cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.754+0000 7fe20de00640 1 -- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe210071170 msgr2=0x7fe2100cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.754+0000 7fe20de00640 1 --2- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe210071170 0x7fe2100cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.754+0000 7fe20de00640 1 -- 172.21.15.5:0/4130405485 >> v1:172.21.15.80:6789/0 conn(0x7fe21006e2a0 legacy=0x7fe21015e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.754+0000 7fe20de00640 1 -- 172.21.15.5:0/4130405485 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210071ea0 con 0x7fe2101441c0 2024-04-25T10:03:29.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.755+0000 7fe20de00640 1 --2- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2101441c0 0x7fe2100b2c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe200009870 tx=0x7fe20002c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e71e89a372ffa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.756+0000 7fe20d400640 1 -- 172.21.15.5:0/4130405485 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200038070 con 0x7fe2101441c0 2024-04-25T10:03:29.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.756+0000 7fe20d400640 1 -- 172.21.15.5:0/4130405485 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe200031710 con 0x7fe2101441c0 2024-04-25T10:03:29.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.756+0000 7fe20d400640 1 -- 172.21.15.5:0/4130405485 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200031ae0 con 0x7fe2101441c0 2024-04-25T10:03:29.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 -- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2101441c0 msgr2=0x7fe2100b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 --2- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2101441c0 0x7fe2100b2c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe200009870 tx=0x7fe20002c460 comp rx=0 tx=0).stop 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 -- 172.21.15.5:0/4130405485 shutdown_connections 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 --2- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe210071170 0x7fe2100cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 --2- 172.21.15.5:0/4130405485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2101441c0 0x7fe2100b2c30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 -- 172.21.15.5:0/4130405485 >> 172.21.15.5:0/4130405485 conn(0x7fe210066fd0 msgr2=0x7fe21006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 -- 172.21.15.5:0/4130405485 shutdown_connections 2024-04-25T10:03:29.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.757+0000 7fe214c00640 1 -- 172.21.15.5:0/4130405485 wait complete. 2024-04-25T10:03:29.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.758+0000 7fe214c00640 1 Processor -- start 2024-04-25T10:03:29.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.759+0000 7fe214c00640 1 -- start start 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe214c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21006e2a0 0x7fe210141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe214c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2101441c0 0x7fe2101578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe214c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 0x7fe21013a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe214c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21006e890 con 0x7fe21013a540 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe214c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21006e710 con 0x7fe21006e2a0 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe214c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21006ea10 con 0x7fe2101441c0 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe20e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 0x7fe21013a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe20e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 0x7fe21013a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41432/0 (socket says 172.21.15.5:41432) 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe20e800640 1 -- 172.21.15.5:0/429222557 learned_addr learned my addr 172.21.15.5:0/429222557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:29.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe20de00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21006e2a0 0x7fe210141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.760+0000 7fe207e00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2101441c0 0x7fe2101578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe20e800640 1 -- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2101441c0 msgr2=0x7fe2101578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe20e800640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2101441c0 0x7fe2101578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe20e800640 1 -- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21006e2a0 msgr2=0x7fe210141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe20e800640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21006e2a0 0x7fe210141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe20e800640 1 -- 172.21.15.5:0/429222557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210141b50 con 0x7fe21013a540 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe20e800640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 0x7fe21013a9f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c7d0 tx=0x7fe20800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208003b70 con 0x7fe21013a540 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208003d30 con 0x7fe21013a540 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800aa90 con 0x7fe21013a540 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe210141e30 con 0x7fe21013a540 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.761+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2101423f0 con 0x7fe21013a540 2024-04-25T10:03:29.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.763+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210074bb0 con 0x7fe21013a540 2024-04-25T10:03:29.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.764+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe20800ac30 con 0x7fe21013a540 2024-04-25T10:03:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.764+0000 7fe206000640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.765+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe208089f70 con 0x7fe21013a540 2024-04-25T10:03:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.765+0000 7fe20de00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.766+0000 7fe20de00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe200035000 tx=0x7fe200008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:29.769+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe208059040 con 0x7fe21013a540 2024-04-25T10:03:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.045+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe2100b2a10 con 0x7fe21013a540 2024-04-25T10:03:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.049+0000 7fe206000640 1 -- 172.21.15.5:0/429222557 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe208058940 con 0x7fe21013a540 2024-04-25T10:03:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1ec06b4d0 msgr2=0x7fe1ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe200035000 tx=0x7fe200008030 comp rx=0 tx=0).stop 2024-04-25T10:03:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 msgr2=0x7fe21013a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 0x7fe21013a9f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe20800c7d0 tx=0x7fe20800cc90 comp rx=0 tx=0).stop 2024-04-25T10:03:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 shutdown_connections 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe21013a540 0x7fe21013a9f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2101441c0 0x7fe2101578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1ec06b4d0 0x7fe1ec06d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 --2- 172.21.15.5:0/429222557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe21006e2a0 0x7fe210141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 >> 172.21.15.5:0/429222557 conn(0x7fe210066fd0 msgr2=0x7fe21006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 shutdown_connections 2024-04-25T10:03:30.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.052+0000 7fe214c00640 1 -- 172.21.15.5:0/429222557 wait complete. 2024-04-25T10:03:30.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:03:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.273+0000 7fd927200640 1 Processor -- start 2024-04-25T10:03:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.273+0000 7fd927200640 1 -- start start 2024-04-25T10:03:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.275+0000 7fd927200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92815c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.275+0000 7fd927200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9280cc460 0x7fd9280cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.275+0000 7fd927200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92806d2b0 con 0x7fd92806d930 2024-04-25T10:03:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.275+0000 7fd927200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd928142180 con 0x7fd9280cc460 2024-04-25T10:03:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.275+0000 7fd927200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92815f310 con 0x7fd928141da0 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9280cc460 0x7fd9280cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9280cc460 0x7fd9280cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41438/0 (socket says 172.21.15.5:41438) 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 -- 172.21.15.5:0/1345129052 learned_addr learned my addr 172.21.15.5:0/1345129052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd925e00640 1 --2- 172.21.15.5:0/1345129052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 -- 172.21.15.5:0/1345129052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 msgr2=0x7fd92815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 --2- 172.21.15.5:0/1345129052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 -- 172.21.15.5:0/1345129052 >> v1:172.21.15.80:6789/0 conn(0x7fd92806d930 legacy=0x7fd9280cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.276+0000 7fd926800640 1 -- 172.21.15.5:0/1345129052 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92815f5f0 con 0x7fd9280cc460 2024-04-25T10:03:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.277+0000 7fd926800640 1 --2- 172.21.15.5:0/1345129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9280cc460 0x7fd9280cd6c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd920009e50 tx=0x7fd92002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bfe7441cd46375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd925400640 1 -- 172.21.15.5:0/1345129052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920036070 con 0x7fd9280cc460 2024-04-25T10:03:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd925400640 1 -- 172.21.15.5:0/1345129052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920031600 con 0x7fd9280cc460 2024-04-25T10:03:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd925400640 1 -- 172.21.15.5:0/1345129052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200319d0 con 0x7fd9280cc460 2024-04-25T10:03:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd927200640 1 -- 172.21.15.5:0/1345129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9280cc460 msgr2=0x7fd9280cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd927200640 1 --2- 172.21.15.5:0/1345129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9280cc460 0x7fd9280cd6c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd920009e50 tx=0x7fd92002a710 comp rx=0 tx=0).stop 2024-04-25T10:03:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd927200640 1 -- 172.21.15.5:0/1345129052 shutdown_connections 2024-04-25T10:03:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd927200640 1 --2- 172.21.15.5:0/1345129052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9280cc460 0x7fd9280cd6c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd927200640 1 --2- 172.21.15.5:0/1345129052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.278+0000 7fd927200640 1 -- 172.21.15.5:0/1345129052 >> 172.21.15.5:0/1345129052 conn(0x7fd9280b4f00 msgr2=0x7fd92806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.279+0000 7fd927200640 1 -- 172.21.15.5:0/1345129052 shutdown_connections 2024-04-25T10:03:30.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.279+0000 7fd927200640 1 -- 172.21.15.5:0/1345129052 wait complete. 2024-04-25T10:03:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.280+0000 7fd927200640 1 Processor -- start 2024-04-25T10:03:30.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.281+0000 7fd927200640 1 -- start start 2024-04-25T10:03:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd927200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 0x7fd928137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd927200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9280cc460 0x7fd92813d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd927200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd927200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92815fe90 con 0x7fd92806d930 2024-04-25T10:03:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd927200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92815fbb0 con 0x7fd9280cc460 2024-04-25T10:03:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd927200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280cdc00 con 0x7fd928141da0 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd925e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 0x7fd928137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd925e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 0x7fd928137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41442/0 (socket says 172.21.15.5:41442) 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd925e00640 1 -- 172.21.15.5:0/1718602587 learned_addr learned my addr 172.21.15.5:0/1718602587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd926800640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd925e00640 1 -- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 msgr2=0x7fd92813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd925e00640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92813ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd925e00640 1 -- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9280cc460 msgr2=0x7fd92813d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.282+0000 7fd91d400640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9280cc460 0x7fd92813d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd925e00640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9280cc460 0x7fd92813d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd925e00640 1 -- 172.21.15.5:0/1718602587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280c9510 con 0x7fd92806d930 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd91d400640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9280cc460 0x7fd92813d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd925e00640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 0x7fd928137b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd91800d5f0 tx=0x7fd91800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180125f0 con 0x7fd92806d930 2024-04-25T10:03:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918019070 con 0x7fd92806d930 2024-04-25T10:03:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180129d0 con 0x7fd92806d930 2024-04-25T10:03:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9280c97f0 con 0x7fd92806d930 2024-04-25T10:03:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.283+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9280c9db0 con 0x7fd92806d930 2024-04-25T10:03:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.285+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd928139010 con 0x7fd92806d930 2024-04-25T10:03:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.286+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd918012b70 con 0x7fd92806d930 2024-04-25T10:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.287+0000 7fd91ea00640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd90406b4d0 0x7fd90406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.287+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd91808a560 con 0x7fd92806d930 2024-04-25T10:03:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.287+0000 7fd91d400640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd90406b4d0 0x7fd90406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.289+0000 7fd91d400640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd90406b4d0 0x7fd90406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9280caef0 tx=0x7fd910009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.291+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd928139010 con 0x7fd92806d930 2024-04-25T10:03:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.572+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd928068f60 con 0x7fd92806d930 2024-04-25T10:03:30.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.577+0000 7fd91ea00640 1 -- 172.21.15.5:0/1718602587 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd918059470 con 0x7fd92806d930 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.580+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd90406b4d0 msgr2=0x7fd90406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.580+0000 7fd927200640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd90406b4d0 0x7fd90406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9280caef0 tx=0x7fd910009040 comp rx=0 tx=0).stop 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.580+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 msgr2=0x7fd928137b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.580+0000 7fd927200640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 0x7fd928137b00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd91800d5f0 tx=0x7fd91800dab0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 shutdown_connections 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd928141da0 0x7fd92813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd90406b4d0 0x7fd90406d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9280cc460 0x7fd92813d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 --2- 172.21.15.5:0/1718602587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd92806d930 0x7fd928137b00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 >> 172.21.15.5:0/1718602587 conn(0x7fd9280b4f00 msgr2=0x7fd92806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 shutdown_connections 2024-04-25T10:03:30.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.581+0000 7fd927200640 1 -- 172.21.15.5:0/1718602587 wait complete. 2024-04-25T10:03:30.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:03:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.806+0000 7fe1c6000640 1 Processor -- start 2024-04-25T10:03:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.806+0000 7fe1c6000640 1 -- start start 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.807+0000 7fe1c6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1c015ca50 0x7fe1c015e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.807+0000 7fe1c6000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1c015edd0 0x7fe1c00cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.807+0000 7fe1c6000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c015d7f0 con 0x7fe1c01451d0 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1c6000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c00a85f0 con 0x7fe1c015ca50 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1c6000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c015e400 con 0x7fe1c015edd0 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1c015ca50 0x7fe1c015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1c015ca50 0x7fe1c015e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41458/0 (socket says 172.21.15.5:41458) 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 -- 172.21.15.5:0/1774677559 learned_addr learned my addr 172.21.15.5:0/1774677559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bfe00640 1 --2- 172.21.15.5:0/1774677559 >> v2:172.21.15.80:3300/0 conn(0x7fe1c015edd0 0x7fe1c00cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 -- 172.21.15.5:0/1774677559 >> v2:172.21.15.80:3300/0 conn(0x7fe1c015edd0 msgr2=0x7fe1c00cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 --2- 172.21.15.5:0/1774677559 >> v2:172.21.15.80:3300/0 conn(0x7fe1c015edd0 0x7fe1c00cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 -- 172.21.15.5:0/1774677559 >> v1:172.21.15.80:6789/0 conn(0x7fe1c01451d0 legacy=0x7fe1c015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.808+0000 7fe1bea00640 1 -- 172.21.15.5:0/1774677559 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c015dad0 con 0x7fe1c015ca50 2024-04-25T10:03:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.810+0000 7fe1bea00640 1 --2- 172.21.15.5:0/1774677559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015ca50 0x7fe1c015e890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00b4ac0 tx=0x7fe1ac02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853e83168128bc39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.810+0000 7fe1be000640 1 -- 172.21.15.5:0/1774677559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac037070 con 0x7fe1c015ca50 2024-04-25T10:03:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.810+0000 7fe1be000640 1 -- 172.21.15.5:0/1774677559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac032440 con 0x7fe1c015ca50 2024-04-25T10:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.810+0000 7fe1be000640 1 -- 172.21.15.5:0/1774677559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac032810 con 0x7fe1c015ca50 2024-04-25T10:03:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 -- 172.21.15.5:0/1774677559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015ca50 msgr2=0x7fe1c015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 --2- 172.21.15.5:0/1774677559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015ca50 0x7fe1c015e890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00b4ac0 tx=0x7fe1ac02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 -- 172.21.15.5:0/1774677559 shutdown_connections 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 --2- 172.21.15.5:0/1774677559 >> v2:172.21.15.80:3300/0 conn(0x7fe1c015edd0 0x7fe1c00cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 --2- 172.21.15.5:0/1774677559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015ca50 0x7fe1c015e890 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 -- 172.21.15.5:0/1774677559 >> 172.21.15.5:0/1774677559 conn(0x7fe1c0066fd0 msgr2=0x7fe1c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 -- 172.21.15.5:0/1774677559 shutdown_connections 2024-04-25T10:03:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.811+0000 7fe1c6000640 1 -- 172.21.15.5:0/1774677559 wait complete. 2024-04-25T10:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.813+0000 7fe1c6000640 1 Processor -- start 2024-04-25T10:03:30.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.813+0000 7fe1c6000640 1 -- start start 2024-04-25T10:03:30.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1c6000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1c01451d0 0x7fe1c0138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1c6000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1c015ca50 0x7fe1c015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1c6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 0x7fe1c015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1c6000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00cc4b0 con 0x7fe1c015edd0 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1c6000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c015e090 con 0x7fe1c015ca50 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1c6000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c00cc630 con 0x7fe1c01451d0 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 0x7fe1c015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 0x7fe1c015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41474/0 (socket says 172.21.15.5:41474) 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.814+0000 7fe1bfe00640 1 -- 172.21.15.5:0/167781998 learned_addr learned my addr 172.21.15.5:0/167781998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bea00640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1c015ca50 0x7fe1c015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bf400640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1c01451d0 0x7fe1c0138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bfe00640 1 -- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1c01451d0 msgr2=0x7fe1c0138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bfe00640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1c01451d0 0x7fe1c0138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bfe00640 1 -- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1c015ca50 msgr2=0x7fe1c015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bfe00640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1c015ca50 0x7fe1c015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bfe00640 1 -- 172.21.15.5:0/167781998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c01e6000 con 0x7fe1c015edd0 2024-04-25T10:03:30.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bf400640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1c01451d0 0x7fe1c0138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:30.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1bfe00640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 0x7fe1c015b250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8007e60 tx=0x7fe1b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.815+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8014420 con 0x7fe1c015edd0 2024-04-25T10:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.816+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8019070 con 0x7fe1c015edd0 2024-04-25T10:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.816+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8014820 con 0x7fe1c015edd0 2024-04-25T10:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.816+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c01e62e0 con 0x7fe1c015edd0 2024-04-25T10:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.816+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c01e68a0 con 0x7fe1c015edd0 2024-04-25T10:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.817+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c0074bb0 con 0x7fe1c015edd0 2024-04-25T10:03:30.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.818+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1b80149c0 con 0x7fe1c015edd0 2024-04-25T10:03:30.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.819+0000 7fe1b7400640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1a006b5a0 0x7fe1a006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.819+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe1b808a8e0 con 0x7fe1c015edd0 2024-04-25T10:03:30.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.820+0000 7fe1bf400640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1a006b5a0 0x7fe1a006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.821+0000 7fe1bf400640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1a006b5a0 0x7fe1a006da60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0003eb0 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:30.823+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1b8059900 con 0x7fe1c015edd0 2024-04-25T10:03:31.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.096+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe1c015c1b0 con 0x7fe1c015edd0 2024-04-25T10:03:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.100+0000 7fe1b7400640 1 -- 172.21.15.5:0/167781998 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe1b8059200 con 0x7fe1c015edd0 2024-04-25T10:03:31.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.102+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1a006b5a0 msgr2=0x7fe1a006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.102+0000 7fe1c6000640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1a006b5a0 0x7fe1a006da60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0003eb0 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:31.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.102+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 msgr2=0x7fe1c015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.102+0000 7fe1c6000640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 0x7fe1c015b250 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8007e60 tx=0x7fe1b800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 shutdown_connections 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c015edd0 0x7fe1c015b250 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe1a006b5a0 0x7fe1a006da60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1c015ca50 0x7fe1c015ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 --2- 172.21.15.5:0/167781998 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1c01451d0 0x7fe1c0138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 >> 172.21.15.5:0/167781998 conn(0x7fe1c0066fd0 msgr2=0x7fe1c00adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 shutdown_connections 2024-04-25T10:03:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.103+0000 7fe1c6000640 1 -- 172.21.15.5:0/167781998 wait complete. 2024-04-25T10:03:31.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:03:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.321+0000 7f973e800640 1 Processor -- start 2024-04-25T10:03:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.322+0000 7f973e800640 1 -- start start 2024-04-25T10:03:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.323+0000 7f973e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973815d6a0 0x7f973806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.323+0000 7f973e800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f973806f0f0 0x7f97380cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.323+0000 7f973e800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97380cc6e0 con 0x7f97380cc2d0 2024-04-25T10:03:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.323+0000 7f973e800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97380cd420 con 0x7f973806f0f0 2024-04-25T10:03:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.323+0000 7f973e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9738161550 con 0x7f973815d6a0 2024-04-25T10:03:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973ca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f973806f0f0 0x7f97380cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973de00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f97380cc2d0 0x7f9738070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47876/0 (socket says 172.21.15.5:47876) 2024-04-25T10:03:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973de00640 1 -- 172.21.15.5:0/93717120 learned_addr learned my addr 172.21.15.5:0/93717120 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973d400640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973815d6a0 0x7f973806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973ca00640 1 -- 172.21.15.5:0/93717120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973815d6a0 msgr2=0x7f973806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973ca00640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973815d6a0 0x7f973806ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973ca00640 1 -- 172.21.15.5:0/93717120 >> v1:172.21.15.5:6789/0 conn(0x7f97380cc2d0 legacy=0x7f9738070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973ca00640 1 -- 172.21.15.5:0/93717120 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738161830 con 0x7f973806f0f0 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.324+0000 7f973d400640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973815d6a0 0x7f973806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.325+0000 7f973ca00640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973806f0f0 0x7f97380cbd90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f97240097a0 tx=0x7f972402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213b77c7897c5b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f9737e00640 1 -- 172.21.15.5:0/93717120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724037070 con 0x7f973806f0f0 2024-04-25T10:03:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f9737e00640 1 -- 172.21.15.5:0/93717120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972402ae40 con 0x7f973806f0f0 2024-04-25T10:03:31.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f9737e00640 1 -- 172.21.15.5:0/93717120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724032660 con 0x7f973806f0f0 2024-04-25T10:03:31.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 -- 172.21.15.5:0/93717120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973806f0f0 msgr2=0x7f97380cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973806f0f0 0x7f97380cbd90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f97240097a0 tx=0x7f972402a0f0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 -- 172.21.15.5:0/93717120 shutdown_connections 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973806f0f0 0x7f97380cbd90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 --2- 172.21.15.5:0/93717120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973815d6a0 0x7f973806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 -- 172.21.15.5:0/93717120 >> 172.21.15.5:0/93717120 conn(0x7f97380b4f00 msgr2=0x7f973806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.326+0000 7f973e800640 1 -- 172.21.15.5:0/93717120 shutdown_connections 2024-04-25T10:03:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.327+0000 7f973e800640 1 -- 172.21.15.5:0/93717120 wait complete. 2024-04-25T10:03:31.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.328+0000 7f973e800640 1 Processor -- start 2024-04-25T10:03:31.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.328+0000 7f973e800640 1 -- start start 2024-04-25T10:03:31.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.329+0000 7f973e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973806f0f0 0x7f973815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.329+0000 7f973e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97380cc2d0 0x7f9738148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.329+0000 7f973e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 0x7f97380afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97381620d0 con 0x7f973815d6a0 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9738161df0 con 0x7f97380cc2d0 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97380cefe0 con 0x7f973806f0f0 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 0x7f97380afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 0x7f97380afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41484/0 (socket says 172.21.15.5:41484) 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 -- 172.21.15.5:0/2307076231 learned_addr learned my addr 172.21.15.5:0/2307076231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973d400640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973806f0f0 0x7f973815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973ca00640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97380cc2d0 0x7f9738148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 -- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973806f0f0 msgr2=0x7f973815fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973806f0f0 0x7f973815fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 -- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97380cc2d0 msgr2=0x7f9738148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97380cc2d0 0x7f9738148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973de00640 1 -- 172.21.15.5:0/2307076231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97381602b0 con 0x7f973815d6a0 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.330+0000 7f973d400640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973806f0f0 0x7f973815fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.331+0000 7f973de00640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 0x7f97380afda0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f973000caa0 tx=0x7f973000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.331+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730017070 con 0x7f973815d6a0 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.331+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973000ac00 con 0x7f973815d6a0 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.331+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9730012620 con 0x7f973815d6a0 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.331+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973815e5c0 con 0x7f973815d6a0 2024-04-25T10:03:31.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.331+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973815eb40 con 0x7f973815d6a0 2024-04-25T10:03:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.332+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9738068f60 con 0x7f973815d6a0 2024-04-25T10:03:31.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.333+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f973001a070 con 0x7f973815d6a0 2024-04-25T10:03:31.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.335+0000 7f9735600640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.335+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9730089e00 con 0x7f973815d6a0 2024-04-25T10:03:31.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.335+0000 7f973d400640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.337+0000 7f973d400640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f971806b4d0 0x7f971806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f972c006fd0 tx=0x7f972c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.339+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97380cc6e0 con 0x7f973815d6a0 2024-04-25T10:03:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.601+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f97380cd200 con 0x7f973815d6a0 2024-04-25T10:03:31.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.605+0000 7f9735600640 1 -- 172.21.15.5:0/2307076231 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9730058da0 con 0x7f973815d6a0 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f971806b4d0 msgr2=0x7f971806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f971806b4d0 0x7f971806d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f972c006fd0 tx=0x7f972c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 msgr2=0x7f97380afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 0x7f97380afda0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f973000caa0 tx=0x7f973000cf60 comp rx=0 tx=0).stop 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 shutdown_connections 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f973815d6a0 0x7f97380afda0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f971806b4d0 0x7f971806d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97380cc2d0 0x7f9738148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 --2- 172.21.15.5:0/2307076231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f973806f0f0 0x7f973815fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.608+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 >> 172.21.15.5:0/2307076231 conn(0x7f97380b4f00 msgr2=0x7f97380ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.609+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 shutdown_connections 2024-04-25T10:03:31.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.609+0000 7f973e800640 1 -- 172.21.15.5:0/2307076231 wait complete. 2024-04-25T10:03:31.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:03:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.825+0000 7f5c5be00640 1 Processor -- start 2024-04-25T10:03:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.825+0000 7f5c5be00640 1 -- start start 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5c5c141da0 0x7f5c5c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c5c06e640 0x7f5c5c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5be00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5c071d30 con 0x7f5c5c15d340 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5be00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5c142180 con 0x7f5c5c06e640 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5c06d620 con 0x7f5c5c141da0 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c5c06e640 0x7f5c5c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c5c15d340 0x7f5c5c0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47880/0 (socket says 172.21.15.5:47880) 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5b400640 1 -- 172.21.15.5:0/3114004840 learned_addr learned my addr 172.21.15.5:0/3114004840 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.827+0000 7f5c5aa00640 1 --2- 172.21.15.5:0/3114004840 >> v2:172.21.15.80:3300/0 conn(0x7f5c5c141da0 0x7f5c5c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.828+0000 7f5c5a000640 1 -- 172.21.15.5:0/3114004840 >> v2:172.21.15.80:3300/0 conn(0x7f5c5c141da0 msgr2=0x7f5c5c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.828+0000 7f5c5a000640 1 --2- 172.21.15.5:0/3114004840 >> v2:172.21.15.80:3300/0 conn(0x7f5c5c141da0 0x7f5c5c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.828+0000 7f5c5a000640 1 -- 172.21.15.5:0/3114004840 >> v1:172.21.15.5:6789/0 conn(0x7f5c5c15d340 legacy=0x7f5c5c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.828+0000 7f5c5a000640 1 -- 172.21.15.5:0/3114004840 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c06d900 con 0x7f5c5c06e640 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.828+0000 7f5c59600640 1 -- 172.21.15.5:0/3114004840 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366841622 0 0) 0x7f5c5c071d30 con 0x7f5c5c15d340 2024-04-25T10:03:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.829+0000 7f5c5a000640 1 --2- 172.21.15.5:0/3114004840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06e640 0x7f5c5c15ce00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c06bbb0 tx=0x7f5c4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72af3548ceedcee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.829+0000 7f5c59600640 1 -- 172.21.15.5:0/3114004840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48037070 con 0x7f5c5c06e640 2024-04-25T10:03:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.829+0000 7f5c59600640 1 -- 172.21.15.5:0/3114004840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c48032440 con 0x7f5c5c06e640 2024-04-25T10:03:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.829+0000 7f5c59600640 1 -- 172.21.15.5:0/3114004840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48032810 con 0x7f5c5c06e640 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 -- 172.21.15.5:0/3114004840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06e640 msgr2=0x7f5c5c15ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 --2- 172.21.15.5:0/3114004840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06e640 0x7f5c5c15ce00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c06bbb0 tx=0x7f5c4802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 -- 172.21.15.5:0/3114004840 shutdown_connections 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 --2- 172.21.15.5:0/3114004840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c06e640 0x7f5c5c15ce00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 --2- 172.21.15.5:0/3114004840 >> v2:172.21.15.80:3300/0 conn(0x7f5c5c141da0 0x7f5c5c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 -- 172.21.15.5:0/3114004840 >> 172.21.15.5:0/3114004840 conn(0x7f5c5c066f20 msgr2=0x7f5c5c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 -- 172.21.15.5:0/3114004840 shutdown_connections 2024-04-25T10:03:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.830+0000 7f5c5be00640 1 -- 172.21.15.5:0/3114004840 wait complete. 2024-04-25T10:03:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.832+0000 7f5c5be00640 1 Processor -- start 2024-04-25T10:03:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.832+0000 7f5c5be00640 1 -- start start 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c5c06e640 0x7f5c5c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c5c141da0 0x7f5c5c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 0x7f5c5c1f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0cdb90 con 0x7f5c5c15d340 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0cda10 con 0x7f5c5c141da0 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c071000 con 0x7f5c5c06e640 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 0x7f5c5c1f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 0x7f5c5c1f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41504/0 (socket says 172.21.15.5:41504) 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.833+0000 7f5c5b400640 1 -- 172.21.15.5:0/2784475222 learned_addr learned my addr 172.21.15.5:0/2784475222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5aa00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c5c06e640 0x7f5c5c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5b400640 1 -- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c5c06e640 msgr2=0x7f5c5c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5b400640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c5c06e640 0x7f5c5c15d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5b400640 1 -- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c5c141da0 msgr2=0x7f5c5c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5a000640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c5c141da0 0x7f5c5c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5b400640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c5c141da0 0x7f5c5c15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5b400640 1 -- 172.21.15.5:0/2784475222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c1f4970 con 0x7f5c5c15d340 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c5b400640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 0x7f5c5c1f3bf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c410 tx=0x7f5c5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c540074e0 con 0x7f5c5c15d340 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c540047e0 con 0x7f5c5c15d340 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.834+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54004bd0 con 0x7f5c5c15d340 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.835+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c1f4c50 con 0x7f5c5c15d340 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.835+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c0b08b0 con 0x7f5c5c15d340 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.836+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c068f60 con 0x7f5c5c15d340 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.837+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c54011070 con 0x7f5c5c15d340 2024-04-25T10:03:31.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.838+0000 7f5c53400640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c3c06b500 0x7f5c3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.838+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5c54089be0 con 0x7f5c5c15d340 2024-04-25T10:03:31.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.838+0000 7f5c5aa00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c3c06b500 0x7f5c3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.840+0000 7f5c5aa00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c3c06b500 0x7f5c3c06d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009f30 tx=0x7f5c4c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:31.842+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c54058c10 con 0x7f5c5c15d340 2024-04-25T10:03:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.104+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5c5c0ccae0 con 0x7f5c5c15d340 2024-04-25T10:03:32.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.108+0000 7f5c53400640 1 -- 172.21.15.5:0/2784475222 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5c54058510 con 0x7f5c5c15d340 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c3c06b500 msgr2=0x7f5c3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c3c06b500 0x7f5c3c06d9c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009f30 tx=0x7f5c4c00b040 comp rx=0 tx=0).stop 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 msgr2=0x7f5c5c1f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 0x7f5c5c1f3bf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c410 tx=0x7f5c5400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 shutdown_connections 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c5c15d340 0x7f5c5c1f3bf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c3c06b500 0x7f5c3c06d9c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c5c141da0 0x7f5c5c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 --2- 172.21.15.5:0/2784475222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c5c06e640 0x7f5c5c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 >> 172.21.15.5:0/2784475222 conn(0x7f5c5c066f20 msgr2=0x7f5c5c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 shutdown_connections 2024-04-25T10:03:32.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.111+0000 7f5c5be00640 1 -- 172.21.15.5:0/2784475222 wait complete. 2024-04-25T10:03:32.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:03:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.327+0000 7fa3c8c00640 1 Processor -- start 2024-04-25T10:03:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.328+0000 7fa3c8c00640 1 -- start start 2024-04-25T10:03:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.329+0000 7fa3c8c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa3c40cc220 0x7fa3c406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.329+0000 7fa3c8c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c406f2c0 con 0x7fa3c415d6a0 2024-04-25T10:03:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.329+0000 7fa3c8c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c4161550 con 0x7fa3c40cbd90 2024-04-25T10:03:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.329+0000 7fa3c8c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c40cd420 con 0x7fa3c40cc220 2024-04-25T10:03:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.330+0000 7fa3c3400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa3c415d6a0 0x7fa3c406ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47892/0 (socket says 172.21.15.5:47892) 2024-04-25T10:03:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.330+0000 7fa3c3400640 1 -- 172.21.15.5:0/2843647023 learned_addr learned my addr 172.21.15.5:0/2843647023 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.330+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823447634 0 0) 0x7fa3c406f2c0 con 0x7fa3c415d6a0 2024-04-25T10:03:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.330+0000 7fa3c3e00640 1 --2- 172.21.15.5:0/2843647023 >> v2:172.21.15.80:3300/0 conn(0x7fa3c40cc220 0x7fa3c406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.330+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3a8003520 con 0x7fa3c415d6a0 2024-04-25T10:03:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1057943375 0 0) 0x7fa3a8003520 con 0x7fa3c415d6a0 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 >> v2:172.21.15.80:3300/0 conn(0x7fa3c40cc220 msgr2=0x7fa3c406dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 --2- 172.21.15.5:0/2843647023 >> v2:172.21.15.80:3300/0 conn(0x7fa3c40cc220 0x7fa3c406dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 >> v1:172.21.15.80:6789/0 conn(0x7fa3c40cbd90 legacy=0x7fa3c406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4161830 con 0x7fa3c415d6a0 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa3b4001670 con 0x7fa3c415d6a0 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 >> v1:172.21.15.5:6789/0 conn(0x7fa3c415d6a0 legacy=0x7fa3c406ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3a8004800 0x7fa3a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800c1d0 con 0x7fa3a80052e0 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800c380 con 0x7fa3a8003a00 2024-04-25T10:03:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a800c580 con 0x7fa3a8004800 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.331+0000 7fa3c3e00640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.332+0000 7fa3c3e00640 1 -- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3a8004800 msgr2=0x7fa3a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.332+0000 7fa3c3e00640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3a8004800 0x7fa3a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.332+0000 7fa3c3e00640 1 -- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a8003a00 msgr2=0x7fa3a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.332+0000 7fa3c3400640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.332+0000 7fa3c3e00640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3a8003a00 0x7fa3a8003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.332+0000 7fa3c3e00640 1 -- 172.21.15.5:0/2843647023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8003520 con 0x7fa3a80052e0 2024-04-25T10:03:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c3e00640 1 -- 172.21.15.5:0/2843647023 reap_dead start 2024-04-25T10:03:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c3e00640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3a800b9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc007360 tx=0x7fa3a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd57c532454af992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc009be0 con 0x7fa3a80052e0 2024-04-25T10:03:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc003bd0 con 0x7fa3a80052e0 2024-04-25T10:03:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c2000640 1 -- 172.21.15.5:0/2843647023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc004020 con 0x7fa3a80052e0 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2843647023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 msgr2=0x7fa3a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.333+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3a800b9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc007360 tx=0x7fa3a8005830 comp rx=0 tx=0).stop 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.334+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2843647023 shutdown_connections 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.334+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/2843647023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3a800b9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.334+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2843647023 >> 172.21.15.5:0/2843647023 conn(0x7fa3c40b4f00 msgr2=0x7fa3c406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.334+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2843647023 shutdown_connections 2024-04-25T10:03:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.334+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2843647023 wait complete. 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.335+0000 7fa3c8c00640 1 Processor -- start 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.336+0000 7fa3c8c00640 1 -- start start 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.336+0000 7fa3c8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3c41f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.336+0000 7fa3c8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3c40b0980 0x7fa3c41556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.336+0000 7fa3c8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3c415c4f0 0x7fa3c41f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.336+0000 7fa3c8c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c41620d0 con 0x7fa3a80052e0 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c8c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4161df0 con 0x7fa3c40b0980 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c8c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40cc630 con 0x7fa3c415c4f0 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3c41f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3c41f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41516/0 (socket says 172.21.15.5:41516) 2024-04-25T10:03:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 -- 172.21.15.5:0/3246246710 learned_addr learned my addr 172.21.15.5:0/3246246710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c2a00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3c40b0980 0x7fa3c41556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 -- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3c415c4f0 msgr2=0x7fa3c41f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3c415c4f0 0x7fa3c41f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 -- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3c40b0980 msgr2=0x7fa3c41556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3c40b0980 0x7fa3c41556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.337+0000 7fa3c3400640 1 -- 172.21.15.5:0/3246246710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4155c10 con 0x7fa3a80052e0 2024-04-25T10:03:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.338+0000 7fa3c3400640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3c41f36b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400d7a0 tx=0x7fa3b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.338+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4019070 con 0x7fa3a80052e0 2024-04-25T10:03:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.338+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b400de50 con 0x7fa3a80052e0 2024-04-25T10:03:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.338+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c4155eb0 con 0x7fa3a80052e0 2024-04-25T10:03:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.338+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40047e0 con 0x7fa3a80052e0 2024-04-25T10:03:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.338+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c40aeff0 con 0x7fa3a80052e0 2024-04-25T10:03:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.339+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c4068f60 con 0x7fa3a80052e0 2024-04-25T10:03:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.340+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3b4004980 con 0x7fa3a80052e0 2024-04-25T10:03:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.341+0000 7fa3bb400640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3a406b500 0x7fa3a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.341+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa3b4089ae0 con 0x7fa3a80052e0 2024-04-25T10:03:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.342+0000 7fa3c2a00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3a406b500 0x7fa3a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.343+0000 7fa3c2a00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3a406b500 0x7fa3a406d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3c414e4c0 tx=0x7fa3b0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.345+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3b4058970 con 0x7fa3a80052e0 2024-04-25T10:03:32.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.615+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa3c40b33b0 con 0x7fa3a80052e0 2024-04-25T10:03:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.620+0000 7fa3bb400640 1 -- 172.21.15.5:0/3246246710 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa3b4058270 con 0x7fa3a80052e0 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3a406b500 msgr2=0x7fa3a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3a406b500 0x7fa3a406d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa3c414e4c0 tx=0x7fa3b0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 msgr2=0x7fa3c41f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3c41f36b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400d7a0 tx=0x7fa3b400dc60 comp rx=0 tx=0).stop 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 shutdown_connections 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3c415c4f0 0x7fa3c41f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3a406b500 0x7fa3a406d9c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3c40b0980 0x7fa3c41556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 --2- 172.21.15.5:0/3246246710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a80052e0 0x7fa3c41f36b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 >> 172.21.15.5:0/3246246710 conn(0x7fa3c40b4f00 msgr2=0x7fa3c406e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.622+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 shutdown_connections 2024-04-25T10:03:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.623+0000 7fa3c8c00640 1 -- 172.21.15.5:0/3246246710 wait complete. 2024-04-25T10:03:32.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:32.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.851+0000 7f17a7200640 1 Processor -- start 2024-04-25T10:03:32.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.851+0000 7f17a7200640 1 -- start start 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17a81652f0 0x7f17a80cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a806e980 0x7f17a8070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f17a80714d0 0x7f17a8165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80ce340 con 0x7f17a81652f0 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a80cdeb0 con 0x7f17a80714d0 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80cd2a0 con 0x7f17a806e980 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17a81652f0 0x7f17a80cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17a81652f0 0x7f17a80cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41518/0 (socket says 172.21.15.5:41518) 2024-04-25T10:03:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.853+0000 7f17a5e00640 1 -- 172.21.15.5:0/1982239987 learned_addr learned my addr 172.21.15.5:0/1982239987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a5400640 1 --2- 172.21.15.5:0/1982239987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a806e980 0x7f17a8070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a6800640 1 --2- 172.21.15.5:0/1982239987 >> v2:172.21.15.80:3300/0 conn(0x7f17a80714d0 0x7f17a8165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a5e00640 1 -- 172.21.15.5:0/1982239987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a806e980 msgr2=0x7f17a8070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a5e00640 1 --2- 172.21.15.5:0/1982239987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a806e980 0x7f17a8070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a5e00640 1 -- 172.21.15.5:0/1982239987 >> v2:172.21.15.80:3300/0 conn(0x7f17a80714d0 msgr2=0x7f17a8165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a5e00640 1 --2- 172.21.15.5:0/1982239987 >> v2:172.21.15.80:3300/0 conn(0x7f17a80714d0 0x7f17a8165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.854+0000 7f17a5e00640 1 -- 172.21.15.5:0/1982239987 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80cd580 con 0x7f17a81652f0 2024-04-25T10:03:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.856+0000 7f17a5e00640 1 --2- 172.21.15.5:0/1982239987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81652f0 0x7f17a80cc390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f17a80b2790 tx=0x7f179802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9971e86024d8f656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.856+0000 7f17a4a00640 1 -- 172.21.15.5:0/1982239987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798037070 con 0x7f17a81652f0 2024-04-25T10:03:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.856+0000 7f17a4a00640 1 -- 172.21.15.5:0/1982239987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1798032440 con 0x7f17a81652f0 2024-04-25T10:03:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.856+0000 7f17a4a00640 1 -- 172.21.15.5:0/1982239987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798032810 con 0x7f17a81652f0 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 -- 172.21.15.5:0/1982239987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81652f0 msgr2=0x7f17a80cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 --2- 172.21.15.5:0/1982239987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81652f0 0x7f17a80cc390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f17a80b2790 tx=0x7f179802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 -- 172.21.15.5:0/1982239987 shutdown_connections 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 --2- 172.21.15.5:0/1982239987 >> v2:172.21.15.80:3300/0 conn(0x7f17a80714d0 0x7f17a8165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 --2- 172.21.15.5:0/1982239987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a806e980 0x7f17a8070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 --2- 172.21.15.5:0/1982239987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a81652f0 0x7f17a80cc390 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 -- 172.21.15.5:0/1982239987 >> 172.21.15.5:0/1982239987 conn(0x7f17a8066f20 msgr2=0x7f17a806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 -- 172.21.15.5:0/1982239987 shutdown_connections 2024-04-25T10:03:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.857+0000 7f17a7200640 1 -- 172.21.15.5:0/1982239987 wait complete. 2024-04-25T10:03:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.859+0000 7f17a7200640 1 Processor -- start 2024-04-25T10:03:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.859+0000 7f17a7200640 1 -- start start 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 0x7f17a81ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a80714d0 0x7f17a8156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a81652f0 0x7f17a81434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80cc8d0 con 0x7f17a806e980 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a80cdb40 con 0x7f17a81652f0 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8165d90 con 0x7f17a80714d0 2024-04-25T10:03:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 0x7f17a81ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 0x7f17a81ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41526/0 (socket says 172.21.15.5:41526) 2024-04-25T10:03:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.860+0000 7f17a5e00640 1 -- 172.21.15.5:0/932548772 learned_addr learned my addr 172.21.15.5:0/932548772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f17a5e00640 1 -- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a80714d0 msgr2=0x7f17a8156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f17a5e00640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a80714d0 0x7f17a8156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f17a5e00640 1 -- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a81652f0 msgr2=0x7f17a81434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f17a5e00640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a81652f0 0x7f17a81434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f17a5e00640 1 -- 172.21.15.5:0/932548772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a81edf30 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f17a5e00640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 0x7f17a81ec680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f179802a220 tx=0x7f17980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.861+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798041070 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.862+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179803c870 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.862+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798037040 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.862+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a81ecbc0 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.862+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a81ed450 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.863+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a80cce40 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.864+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1798044070 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.865+0000 7f179e000640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f178406b4d0 0x7f178406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.865+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f17980aae50 con 0x7f17a806e980 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.865+0000 7f17a5400640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f178406b4d0 0x7f178406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.867+0000 7f17a5400640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f178406b4d0 0x7f178406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17a80cc490 tx=0x7f1790008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:32.869+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1798079f10 con 0x7f17a806e980 2024-04-25T10:03:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.070+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17a806e230 con 0x7f17a806e980 2024-04-25T10:03:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.072+0000 7f179e000640 1 -- 172.21.15.5:0/932548772 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13056 (secure 0 0 0) 0x7f1798079810 con 0x7f17a806e980 2024-04-25T10:03:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:33.079 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:29.040689+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.074+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f178406b4d0 msgr2=0x7f178406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.074+0000 7f17a7200640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f178406b4d0 0x7f178406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f17a80cc490 tx=0x7f1790008040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 msgr2=0x7f17a81ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 0x7f17a81ec680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f179802a220 tx=0x7f17980046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 shutdown_connections 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f178406b4d0 0x7f178406d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a81652f0 0x7f17a81434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17a80714d0 0x7f17a8156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 --2- 172.21.15.5:0/932548772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17a806e980 0x7f17a81ec680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 >> 172.21.15.5:0/932548772 conn(0x7f17a8066f20 msgr2=0x7f17a8160470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 shutdown_connections 2024-04-25T10:03:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.075+0000 7f17a7200640 1 -- 172.21.15.5:0/932548772 wait complete. 2024-04-25T10:03:33.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:33.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.308+0000 7f5ceca00640 1 Processor -- start 2024-04-25T10:03:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.308+0000 7f5ceca00640 1 -- start start 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.310+0000 7f5ceca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce80b48c0 0x7f5ce815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.310+0000 7f5ceca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ce80d3f90 0x7f5ce8161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.310+0000 7f5ceca00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce81606e0 con 0x7f5ce8161bc0 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.310+0000 7f5ceca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce81609c0 con 0x7f5ce80d3f90 2024-04-25T10:03:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.310+0000 7f5ceca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce8160ca0 con 0x7f5ce80b48c0 2024-04-25T10:03:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.311+0000 7f5ce7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5ce8161bc0 0x7f5ce80d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47902/0 (socket says 172.21.15.5:47902) 2024-04-25T10:03:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.311+0000 7f5ce7e00640 1 -- 172.21.15.5:0/399681960 learned_addr learned my addr 172.21.15.5:0/399681960 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:33.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.311+0000 7f5ce6a00640 1 --2- 172.21.15.5:0/399681960 >> v2:172.21.15.80:3300/0 conn(0x7f5ce80d3f90 0x7f5ce8161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.311+0000 7f5ce7400640 1 --2- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce80b48c0 0x7f5ce815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.311+0000 7f5ce6000640 1 -- 172.21.15.5:0/399681960 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951355187 0 0) 0x7f5ce81606e0 con 0x7f5ce8161bc0 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.311+0000 7f5ce6000640 1 -- 172.21.15.5:0/399681960 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ccc003520 con 0x7f5ce8161bc0 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.312+0000 7f5ce6a00640 1 -- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce80b48c0 msgr2=0x7f5ce815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.312+0000 7f5ce6000640 1 -- 172.21.15.5:0/399681960 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136519890 0 0) 0x7f5ccc003520 con 0x7f5ce8161bc0 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.312+0000 7f5ce6a00640 1 --2- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce80b48c0 0x7f5ce815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.312+0000 7f5ce6a00640 1 -- 172.21.15.5:0/399681960 >> v1:172.21.15.5:6789/0 conn(0x7f5ce8161bc0 legacy=0x7f5ce80d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.312+0000 7f5ce6a00640 1 -- 172.21.15.5:0/399681960 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce8160f80 con 0x7f5ce80d3f90 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.313+0000 7f5ce6a00640 1 --2- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80d3f90 0x7f5ce8161680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009790 tx=0x7f5cd402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a2cbdf7b63a97e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.314+0000 7f5ce6000640 1 -- 172.21.15.5:0/399681960 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4037070 con 0x7f5ce80d3f90 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.314+0000 7f5ce6000640 1 -- 172.21.15.5:0/399681960 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd402ab10 con 0x7f5ce80d3f90 2024-04-25T10:03:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.314+0000 7f5ce6000640 1 -- 172.21.15.5:0/399681960 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce00019f0 con 0x7f5ce80d3f90 2024-04-25T10:03:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.314+0000 7f5ceca00640 1 -- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80d3f90 msgr2=0x7f5ce8161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.314+0000 7f5ceca00640 1 --2- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80d3f90 0x7f5ce8161680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009790 tx=0x7f5cd402a080 comp rx=0 tx=0).stop 2024-04-25T10:03:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.315+0000 7f5ceca00640 1 -- 172.21.15.5:0/399681960 shutdown_connections 2024-04-25T10:03:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.315+0000 7f5ceca00640 1 --2- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80d3f90 0x7f5ce8161680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.315+0000 7f5ceca00640 1 --2- 172.21.15.5:0/399681960 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce80b48c0 0x7f5ce815fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.315+0000 7f5ceca00640 1 -- 172.21.15.5:0/399681960 >> 172.21.15.5:0/399681960 conn(0x7f5ce808af80 msgr2=0x7f5ce80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.315+0000 7f5ceca00640 1 -- 172.21.15.5:0/399681960 shutdown_connections 2024-04-25T10:03:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.315+0000 7f5ceca00640 1 -- 172.21.15.5:0/399681960 wait complete. 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.317+0000 7f5ceca00640 1 Processor -- start 2024-04-25T10:03:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.317+0000 7f5ceca00640 1 -- start start 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ceca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80b48c0 0x7f5ce80cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ceca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 0x7f5ce80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ceca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce8161bc0 0x7f5ce80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ceca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81603a0 con 0x7f5ce80d3f90 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ceca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80b4ca0 con 0x7f5ce80b48c0 2024-04-25T10:03:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ceca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80b3b80 con 0x7f5ce8161bc0 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ce6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 0x7f5ce80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ce6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 0x7f5ce80ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41530/0 (socket says 172.21.15.5:41530) 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.319+0000 7f5ce6a00640 1 -- 172.21.15.5:0/726865654 learned_addr learned my addr 172.21.15.5:0/726865654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce7400640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80b48c0 0x7f5ce80cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce7e00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce8161bc0 0x7f5ce80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce6a00640 1 -- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce8161bc0 msgr2=0x7f5ce80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce6a00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce8161bc0 0x7f5ce80cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce6a00640 1 -- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80b48c0 msgr2=0x7f5ce80cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce6a00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80b48c0 0x7f5ce80cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce6a00640 1 -- 172.21.15.5:0/726865654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce8159c90 con 0x7f5ce80d3f90 2024-04-25T10:03:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce7400640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80b48c0 0x7f5ce80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5ce6a00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 0x7f5ce80ca6d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4035040 tx=0x7f5cd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.320+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4037040 con 0x7f5ce80d3f90 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.321+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd4005bb0 con 0x7f5ce80d3f90 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.321+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4006030 con 0x7f5ce80d3f90 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.321+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce8159f50 con 0x7f5ce80d3f90 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.321+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce815a510 con 0x7f5ce80d3f90 2024-04-25T10:03:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.322+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce8068f60 con 0x7f5ce80d3f90 2024-04-25T10:03:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.323+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cd40061d0 con 0x7f5ce80d3f90 2024-04-25T10:03:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.324+0000 7f5cdf400640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5cc806b4d0 0x7f5cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.324+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5cd40aab60 con 0x7f5ce80d3f90 2024-04-25T10:03:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.325+0000 7f5ce7400640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5cc806b4d0 0x7f5cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.326+0000 7f5ce7400640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5cc806b4d0 0x7f5cc806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009ac0 tx=0x7f5cd8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.331+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cd4079b90 con 0x7f5ce80d3f90 2024-04-25T10:03:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.523+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ce808d730 con 0x7f5ce80d3f90 2024-04-25T10:03:33.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.524+0000 7f5cdf400640 1 -- 172.21.15.5:0/726865654 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cd4079490 con 0x7f5ce80d3f90 2024-04-25T10:03:33.530 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5cc806b4d0 msgr2=0x7f5cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5cc806b4d0 0x7f5cc806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009ac0 tx=0x7f5cd8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 msgr2=0x7f5ce80ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 0x7f5ce80ca6d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4035040 tx=0x7f5cd40046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 shutdown_connections 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5ce8161bc0 0x7f5ce80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce80d3f90 0x7f5ce80ca6d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5cc806b4d0 0x7f5cc806d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 --2- 172.21.15.5:0/726865654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce80b48c0 0x7f5ce80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 >> 172.21.15.5:0/726865654 conn(0x7f5ce808af80 msgr2=0x7f5ce80d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 shutdown_connections 2024-04-25T10:03:33.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.527+0000 7f5ceca00640 1 -- 172.21.15.5:0/726865654 wait complete. 2024-04-25T10:03:33.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:03:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.754+0000 7f7821400640 1 Processor -- start 2024-04-25T10:03:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.755+0000 7f7821400640 1 -- start start 2024-04-25T10:03:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.756+0000 7f7821400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d1590 0x7f781c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.756+0000 7f7821400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f781c15e4c0 0x7f781c1613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.756+0000 7f7821400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781c0ce6f0 con 0x7f781c0d0570 2024-04-25T10:03:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.756+0000 7f7821400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781c0cea20 con 0x7f781c15e4c0 2024-04-25T10:03:33.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.756+0000 7f7821400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c15ea10 con 0x7f781c0d1590 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f781c15e4c0 0x7f781c1613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f781c15e4c0 0x7f781c1613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41542/0 (socket says 172.21.15.5:41542) 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 -- 172.21.15.5:0/3789848931 learned_addr learned my addr 172.21.15.5:0/3789848931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f781be00640 1 --2- 172.21.15.5:0/3789848931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d1590 0x7f781c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 -- 172.21.15.5:0/3789848931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d1590 msgr2=0x7f781c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 --2- 172.21.15.5:0/3789848931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d1590 0x7f781c0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 -- 172.21.15.5:0/3789848931 >> v1:172.21.15.80:6789/0 conn(0x7f781c0d0570 legacy=0x7f781c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.757+0000 7f7820a00640 1 -- 172.21.15.5:0/3789848931 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c15ecf0 con 0x7f781c15e4c0 2024-04-25T10:03:33.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.758+0000 7f7820a00640 1 --2- 172.21.15.5:0/3789848931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c1613c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7814009870 tx=0x7f781402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b635cc466c029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.759+0000 7f781aa00640 1 -- 172.21.15.5:0/3789848931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814037070 con 0x7f781c15e4c0 2024-04-25T10:03:33.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.759+0000 7f781aa00640 1 -- 172.21.15.5:0/3789848931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7814032440 con 0x7f781c15e4c0 2024-04-25T10:03:33.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.759+0000 7f781aa00640 1 -- 172.21.15.5:0/3789848931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814032810 con 0x7f781c15e4c0 2024-04-25T10:03:33.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.759+0000 7f7821400640 1 -- 172.21.15.5:0/3789848931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 msgr2=0x7f781c1613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.759+0000 7f7821400640 1 --2- 172.21.15.5:0/3789848931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c1613c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7814009870 tx=0x7f781402a210 comp rx=0 tx=0).stop 2024-04-25T10:03:33.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.760+0000 7f7821400640 1 -- 172.21.15.5:0/3789848931 shutdown_connections 2024-04-25T10:03:33.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.760+0000 7f7821400640 1 --2- 172.21.15.5:0/3789848931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c1613c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.760+0000 7f7821400640 1 --2- 172.21.15.5:0/3789848931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d1590 0x7f781c0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.760+0000 7f7821400640 1 -- 172.21.15.5:0/3789848931 >> 172.21.15.5:0/3789848931 conn(0x7f781c0726e0 msgr2=0x7f781c072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.760+0000 7f7821400640 1 -- 172.21.15.5:0/3789848931 shutdown_connections 2024-04-25T10:03:33.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.760+0000 7f7821400640 1 -- 172.21.15.5:0/3789848931 wait complete. 2024-04-25T10:03:33.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.761+0000 7f7821400640 1 Processor -- start 2024-04-25T10:03:33.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.762+0000 7f7821400640 1 -- start start 2024-04-25T10:03:33.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7821400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d0570 0x7f781c14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7821400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0d1590 0x7f781c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7821400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0d31c0 con 0x7f781c15e4c0 2024-04-25T10:03:33.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7821400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0d3040 con 0x7f781c0d1590 2024-04-25T10:03:33.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7821400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c161900 con 0x7f781c0d0570 2024-04-25T10:03:33.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41556/0 (socket says 172.21.15.5:41556) 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 -- 172.21.15.5:0/1194722380 learned_addr learned my addr 172.21.15.5:0/1194722380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f781b400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0d1590 0x7f781c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f781be00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d0570 0x7f781c14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 -- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d0570 msgr2=0x7f781c14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d0570 0x7f781c14eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 -- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0d1590 msgr2=0x7f781c0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0d1590 0x7f781c0c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f7820a00640 1 -- 172.21.15.5:0/1194722380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0b5a80 con 0x7f781c15e4c0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.763+0000 7f781b400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0d1590 0x7f781c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.764+0000 7f7820a00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c0c94c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f781402a250 tx=0x7f78140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.764+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814041020 con 0x7f781c15e4c0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.764+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78140306c0 con 0x7f781c15e4c0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.764+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814037040 con 0x7f781c15e4c0 2024-04-25T10:03:33.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.764+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c0b57a0 con 0x7f781c15e4c0 2024-04-25T10:03:33.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.764+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781c0c7070 con 0x7f781c15e4c0 2024-04-25T10:03:33.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.765+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781c068080 con 0x7f781c15e4c0 2024-04-25T10:03:33.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.766+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7814044020 con 0x7f781c15e4c0 2024-04-25T10:03:33.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.768+0000 7f7813e00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f77f806b4d0 0x7f77f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.768+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f78140aa850 con 0x7f781c15e4c0 2024-04-25T10:03:33.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.769+0000 7f781be00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f77f806b4d0 0x7f77f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.770+0000 7f781be00640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f77f806b4d0 0x7f77f806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7804006fd0 tx=0x7f7804008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.773+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f781404b030 con 0x7f781c15e4c0 2024-04-25T10:03:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.976+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f781c068f60 con 0x7f781c15e4c0 2024-04-25T10:03:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.977+0000 7f7813e00640 1 -- 172.21.15.5:0/1194722380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7814079880 con 0x7f781c15e4c0 2024-04-25T10:03:33.982 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:33.982 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:03:33.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:03:33.983 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:03:33.983 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:03:33.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:03:33.983 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:03:33.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.979+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f77f806b4d0 msgr2=0x7f77f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.979+0000 7f7821400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f77f806b4d0 0x7f77f806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7804006fd0 tx=0x7f7804008040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.979+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 msgr2=0x7f781c0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.979+0000 7f7821400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c0c94c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f781402a250 tx=0x7f78140046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 shutdown_connections 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c15e4c0 0x7f781c0c94c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f77f806b4d0 0x7f77f806d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0d1590 0x7f781c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 --2- 172.21.15.5:0/1194722380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0d0570 0x7f781c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 >> 172.21.15.5:0/1194722380 conn(0x7f781c0726e0 msgr2=0x7f781c15da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 shutdown_connections 2024-04-25T10:03:33.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:33.980+0000 7f7821400640 1 -- 172.21.15.5:0/1194722380 wait complete. 2024-04-25T10:03:34.004 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:03:34.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:03:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.214+0000 7fe8aaa00640 1 Processor -- start 2024-04-25T10:03:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.215+0000 7fe8aaa00640 1 -- start start 2024-04-25T10:03:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.216+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 0x7fe8a415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.216+0000 7fe8aaa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe8a4071460 0x7fe8a4140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.216+0000 7fe8aaa00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a415b050 con 0x7fe8a4071000 2024-04-25T10:03:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.216+0000 7fe8aaa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a413fcf0 con 0x7fe8a4071460 2024-04-25T10:03:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.216+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415b330 con 0x7fe8a415dff0 2024-04-25T10:03:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8a8c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe8a4071000 0x7fe8a40cb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47912/0 (socket says 172.21.15.5:47912) 2024-04-25T10:03:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8a8c00640 1 -- 172.21.15.5:0/2776311173 learned_addr learned my addr 172.21.15.5:0/2776311173 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8a9600640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 0x7fe8a415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830699825 0 0) 0x7fe8a415b050 con 0x7fe8a4071000 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8aa000640 1 --2- 172.21.15.5:0/2776311173 >> v2:172.21.15.80:3300/0 conn(0x7fe8a4071460 0x7fe8a4140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe88c003520 con 0x7fe8a4071000 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.217+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987491957 0 0) 0x7fe88c003520 con 0x7fe8a4071000 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 msgr2=0x7fe8a415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 0x7fe8a415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 >> v2:172.21.15.80:3300/0 conn(0x7fe8a4071460 msgr2=0x7fe8a4140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/2776311173 >> v2:172.21.15.80:3300/0 conn(0x7fe8a4071460 0x7fe8a4140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a415b610 con 0x7fe8a4071000 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe890001670 con 0x7fe8a4071000 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 >> v1:172.21.15.5:6789/0 conn(0x7fe8a4071000 legacy=0x7fe8a40cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe88c0039b0 0x7fe88c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 0x7fe88c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe88c004d40 0x7fe88c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a413fcf0 con 0x7fe88c004d40 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a415b330 con 0x7fe88c0039b0 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c00b9c0 con 0x7fe8a415dff0 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.218+0000 7fe8aa000640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe88c004d40 0x7fe88c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8a9600640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe88c0039b0 0x7fe88c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8a8c00640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 0x7fe88c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8aa000640 1 -- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 msgr2=0x7fe88c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8aa000640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8a415dff0 0x7fe88c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8aa000640 1 -- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe88c0039b0 msgr2=0x7fe88c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8aa000640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe88c0039b0 0x7fe88c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8aa000640 1 -- 172.21.15.5:0/2776311173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c003520 con 0x7fe88c004d40 2024-04-25T10:03:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.219+0000 7fe8a9600640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe88c0039b0 0x7fe88c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.220+0000 7fe8aa000640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe88c004d40 0x7fe88c0051c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00bf70 tx=0x7fe88c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e13bd272b85d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c00bc10 con 0x7fe88c004d40 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c038070 con 0x7fe88c004d40 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8a3e00640 1 -- 172.21.15.5:0/2776311173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c033850 con 0x7fe88c004d40 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aaa00640 1 -- 172.21.15.5:0/2776311173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe88c004d40 msgr2=0x7fe88c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/2776311173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe88c004d40 0x7fe88c0051c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00bf70 tx=0x7fe88c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aa000640 1 -- 172.21.15.5:0/2776311173 reap_dead start 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aaa00640 1 -- 172.21.15.5:0/2776311173 shutdown_connections 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aaa00640 1 -- 172.21.15.5:0/2776311173 >> 172.21.15.5:0/2776311173 conn(0x7fe8a4066f20 msgr2=0x7fe8a406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aaa00640 1 -- 172.21.15.5:0/2776311173 shutdown_connections 2024-04-25T10:03:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.221+0000 7fe8aaa00640 1 -- 172.21.15.5:0/2776311173 wait complete. 2024-04-25T10:03:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.223+0000 7fe8aaa00640 1 Processor -- start 2024-04-25T10:03:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.223+0000 7fe8aaa00640 1 -- start start 2024-04-25T10:03:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.224+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe88c004d40 0x7fe8a414daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.224+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a415d4e0 0x7fe8a415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.224+0000 7fe8aaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 0x7fe8a40cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.224+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40cbc90 con 0x7fe8a40cbe70 2024-04-25T10:03:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.224+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40cbb10 con 0x7fe8a415d4e0 2024-04-25T10:03:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.224+0000 7fe8aaa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a41406c0 con 0x7fe88c004d40 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 0x7fe8a40cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 0x7fe8a40cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41582/0 (socket says 172.21.15.5:41582) 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 -- 172.21.15.5:0/3982945883 learned_addr learned my addr 172.21.15.5:0/3982945883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8a9600640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe88c004d40 0x7fe8a414daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8a8c00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a415d4e0 0x7fe8a415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 -- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe88c004d40 msgr2=0x7fe8a414daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe88c004d40 0x7fe8a414daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 -- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a415d4e0 msgr2=0x7fe8a415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a415d4e0 0x7fe8a415d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.225+0000 7fe8aa000640 1 -- 172.21.15.5:0/3982945883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a415cf40 con 0x7fe8a40cbe70 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.226+0000 7fe8aa000640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 0x7fe8a40cc2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe89c035040 tx=0x7fe89c00b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.226+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c042030 con 0x7fe8a40cbe70 2024-04-25T10:03:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.226+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c003990 con 0x7fe8a40cbe70 2024-04-25T10:03:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.226+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a415bea0 con 0x7fe8a40cbe70 2024-04-25T10:03:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.226+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003d60 con 0x7fe8a40cbe70 2024-04-25T10:03:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.226+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a415c460 con 0x7fe8a40cbe70 2024-04-25T10:03:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.227+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4068f60 con 0x7fe8a40cbe70 2024-04-25T10:03:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.228+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe89c005d10 con 0x7fe8a40cbe70 2024-04-25T10:03:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.229+0000 7fe8a1600640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88406b500 0x7fe88406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.230+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe89c0aa8b0 con 0x7fe8a40cbe70 2024-04-25T10:03:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.230+0000 7fe8a9600640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88406b500 0x7fe88406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.231+0000 7fe8a9600640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88406b500 0x7fe88406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe898009990 tx=0x7fe898008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.234+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe89c04c070 con 0x7fe8a40cbe70 2024-04-25T10:03:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:34.435+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe8a40b4920 con 0x7fe8a40cbe70 2024-04-25T10:03:34.969 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:03:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.052+0000 7fe8a1600640 1 -- 172.21.15.5:0/3982945883 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe89c030050 con 0x7fe8a40cbe70 2024-04-25T10:03:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:03:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.056+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88406b500 msgr2=0x7fe88406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.056+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88406b500 0x7fe88406d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe898009990 tx=0x7fe898008040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.056+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 msgr2=0x7fe8a40cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.056+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 0x7fe8a40cc2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe89c035040 tx=0x7fe89c00b750 comp rx=0 tx=0).stop 2024-04-25T10:03:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 shutdown_connections 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8a40cbe70 0x7fe8a40cc2f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88406b500 0x7fe88406d9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8a415d4e0 0x7fe8a415d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 --2- 172.21.15.5:0/3982945883 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe88c004d40 0x7fe8a414daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 >> 172.21.15.5:0/3982945883 conn(0x7fe8a4066f20 msgr2=0x7fe8a413da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 shutdown_connections 2024-04-25T10:03:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.057+0000 7fe8aaa00640 1 -- 172.21.15.5:0/3982945883 wait complete. 2024-04-25T10:03:35.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:03:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.286+0000 7f3cd9400640 1 Processor -- start 2024-04-25T10:03:35.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.286+0000 7f3cd9400640 1 -- start start 2024-04-25T10:03:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.287+0000 7f3cd9400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3cd40b48c0 0x7f3cd415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3cd4161d90 0x7f3cd40b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd9400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd41606e0 con 0x7f3cd40d3f90 2024-04-25T10:03:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd9400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd41609c0 con 0x7f3cd4161d90 2024-04-25T10:03:35.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd9400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd4160ca0 con 0x7f3cd40b48c0 2024-04-25T10:03:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd8a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3cd4161d90 0x7f3cd40b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd8a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3cd4161d90 0x7f3cd40b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41596/0 (socket says 172.21.15.5:41596) 2024-04-25T10:03:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd8a00640 1 -- 172.21.15.5:0/2526674665 learned_addr learned my addr 172.21.15.5:0/2526674665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.288+0000 7f3cd3e00640 1 --2- 172.21.15.5:0/2526674665 >> v2:172.21.15.80:3300/0 conn(0x7f3cd40b48c0 0x7f3cd415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.289+0000 7f3cd8a00640 1 -- 172.21.15.5:0/2526674665 >> v2:172.21.15.80:3300/0 conn(0x7f3cd40b48c0 msgr2=0x7f3cd415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.289+0000 7f3cd8a00640 1 --2- 172.21.15.5:0/2526674665 >> v2:172.21.15.80:3300/0 conn(0x7f3cd40b48c0 0x7f3cd415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.289+0000 7f3cd8a00640 1 -- 172.21.15.5:0/2526674665 >> v1:172.21.15.80:6789/0 conn(0x7f3cd40d3f90 legacy=0x7f3cd4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.289+0000 7f3cd8a00640 1 -- 172.21.15.5:0/2526674665 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4160f80 con 0x7f3cd4161d90 2024-04-25T10:03:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.290+0000 7f3cd8a00640 1 --2- 172.21.15.5:0/2526674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40b2b70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cd408ba40 tx=0x7f3ccc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75c12ff7e59fbcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.290+0000 7f3cd3400640 1 -- 172.21.15.5:0/2526674665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc036070 con 0x7f3cd4161d90 2024-04-25T10:03:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.290+0000 7f3cd3400640 1 -- 172.21.15.5:0/2526674665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc031600 con 0x7f3cd4161d90 2024-04-25T10:03:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.290+0000 7f3cd3400640 1 -- 172.21.15.5:0/2526674665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0319d0 con 0x7f3cd4161d90 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 -- 172.21.15.5:0/2526674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 msgr2=0x7f3cd40b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 --2- 172.21.15.5:0/2526674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40b2b70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3cd408ba40 tx=0x7f3ccc02a660 comp rx=0 tx=0).stop 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 -- 172.21.15.5:0/2526674665 shutdown_connections 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 --2- 172.21.15.5:0/2526674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40b2b70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 --2- 172.21.15.5:0/2526674665 >> v2:172.21.15.80:3300/0 conn(0x7f3cd40b48c0 0x7f3cd415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 -- 172.21.15.5:0/2526674665 >> 172.21.15.5:0/2526674665 conn(0x7f3cd408af80 msgr2=0x7f3cd40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 -- 172.21.15.5:0/2526674665 shutdown_connections 2024-04-25T10:03:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.291+0000 7f3cd9400640 1 -- 172.21.15.5:0/2526674665 wait complete. 2024-04-25T10:03:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.293+0000 7f3cd9400640 1 Processor -- start 2024-04-25T10:03:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.293+0000 7f3cd9400640 1 -- start start 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.294+0000 7f3cd9400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cd40b48c0 0x7f3cd40cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.294+0000 7f3cd9400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cd40d3f90 0x7f3cd40c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.294+0000 7f3cd9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.294+0000 7f3cd9400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41603a0 con 0x7f3cd4161d90 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.294+0000 7f3cd9400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40b4ca0 con 0x7f3cd40b48c0 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.294+0000 7f3cd9400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40b30b0 con 0x7f3cd40d3f90 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41602/0 (socket says 172.21.15.5:41602) 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 -- 172.21.15.5:0/86147151 learned_addr learned my addr 172.21.15.5:0/86147151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd3e00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cd40b48c0 0x7f3cd40cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3ccbe00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cd40d3f90 0x7f3cd40c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 -- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cd40d3f90 msgr2=0x7f3cd40c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cd40d3f90 0x7f3cd40c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 -- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cd40b48c0 msgr2=0x7f3cd40cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cd40b48c0 0x7f3cd40cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd8a00640 1 -- 172.21.15.5:0/86147151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40cd4e0 con 0x7f3cd4161d90 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3cd3e00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cd40b48c0 0x7f3cd40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.295+0000 7f3ccbe00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cd40d3f90 0x7f3cd40c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.296+0000 7f3cd8a00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40ccd90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009f40 tx=0x7f3ccc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.296+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc036050 con 0x7f3cd4161d90 2024-04-25T10:03:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.296+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc042070 con 0x7f3cd4161d90 2024-04-25T10:03:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.296+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40c7be0 con 0x7f3cd4161d90 2024-04-25T10:03:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.296+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc009510 con 0x7f3cd4161d90 2024-04-25T10:03:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.296+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd40c81a0 con 0x7f3cd4161d90 2024-04-25T10:03:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.297+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd4068f60 con 0x7f3cd4161d90 2024-04-25T10:03:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.298+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ccc007bd0 con 0x7f3cd4161d90 2024-04-25T10:03:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.299+0000 7f3cd0c00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cb006b4d0 0x7f3cb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.299+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f3ccc0ab120 con 0x7f3cd4161d90 2024-04-25T10:03:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.300+0000 7f3cd3e00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cb006b4d0 0x7f3cb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.301+0000 7f3cd3e00640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cb006b4d0 0x7f3cb006d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.304+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ccc045360 con 0x7f3cd4161d90 2024-04-25T10:03:35.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:35.509+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3cd40d0dd0 con 0x7f3cd4161d90 2024-04-25T10:03:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.059+0000 7f3cd0c00640 1 -- 172.21.15.5:0/86147151 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3ccc079f50 con 0x7f3cd4161d90 2024-04-25T10:03:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:03:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.063+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cb006b4d0 msgr2=0x7f3cb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.063+0000 7f3cd9400640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cb006b4d0 0x7f3cb006d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.063+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 msgr2=0x7f3cd40ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.063+0000 7f3cd9400640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40ccd90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009f40 tx=0x7f3ccc004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 shutdown_connections 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4161d90 0x7f3cd40ccd90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cd40d3f90 0x7f3cd40c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cb006b4d0 0x7f3cb006d990 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 --2- 172.21.15.5:0/86147151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cd40b48c0 0x7f3cd40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 >> 172.21.15.5:0/86147151 conn(0x7f3cd408af80 msgr2=0x7f3cd415ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 shutdown_connections 2024-04-25T10:03:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.064+0000 7f3cd9400640 1 -- 172.21.15.5:0/86147151 wait complete. 2024-04-25T10:03:36.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:03:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.287+0000 7f3fed800640 1 Processor -- start 2024-04-25T10:03:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.287+0000 7f3fed800640 1 -- start start 2024-04-25T10:03:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.288+0000 7f3fed800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe815a940 0x7f3fe80cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.288+0000 7f3fed800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3fe815f040 0x7f3fe80b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.288+0000 7f3fed800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3fe80ccfd0 0x7f3fe80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.288+0000 7f3fed800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80cb0d0 con 0x7f3fe815f040 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.288+0000 7f3fed800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80cd960 con 0x7f3fe80ccfd0 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.288+0000 7f3fed800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80b3030 con 0x7f3fe815a940 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3fe815f040 0x7f3fe80b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3fe815f040 0x7f3fe80b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41612/0 (socket says 172.21.15.5:41612) 2024-04-25T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 -- 172.21.15.5:0/2925538788 learned_addr learned my addr 172.21.15.5:0/2925538788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6a00640 1 --2- 172.21.15.5:0/2925538788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe815a940 0x7f3fe80cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe7400640 1 --2- 172.21.15.5:0/2925538788 >> v2:172.21.15.80:3300/0 conn(0x7f3fe80ccfd0 0x7f3fe80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 -- 172.21.15.5:0/2925538788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe815a940 msgr2=0x7f3fe80cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 --2- 172.21.15.5:0/2925538788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe815a940 0x7f3fe80cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 -- 172.21.15.5:0/2925538788 >> v2:172.21.15.80:3300/0 conn(0x7f3fe80ccfd0 msgr2=0x7f3fe80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.289+0000 7f3fe6000640 1 --2- 172.21.15.5:0/2925538788 >> v2:172.21.15.80:3300/0 conn(0x7f3fe80ccfd0 0x7f3fe80cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.290+0000 7f3fe6000640 1 -- 172.21.15.5:0/2925538788 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe815ad20 con 0x7f3fe815f040 2024-04-25T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.290+0000 7f3fe7400640 1 --2- 172.21.15.5:0/2925538788 >> v2:172.21.15.80:3300/0 conn(0x7f3fe80ccfd0 0x7f3fe80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.291+0000 7f3fe6000640 1 --2- 172.21.15.5:0/2925538788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe80b2800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fe809a0d0 tx=0x7f3fd402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db781255621a7ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.291+0000 7f3fe5600640 1 -- 172.21.15.5:0/2925538788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4037070 con 0x7f3fe815f040 2024-04-25T10:03:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.291+0000 7f3fe5600640 1 -- 172.21.15.5:0/2925538788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4032440 con 0x7f3fe815f040 2024-04-25T10:03:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.291+0000 7f3fe5600640 1 -- 172.21.15.5:0/2925538788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4032810 con 0x7f3fe815f040 2024-04-25T10:03:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 -- 172.21.15.5:0/2925538788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 msgr2=0x7f3fe80b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 --2- 172.21.15.5:0/2925538788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe80b2800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fe809a0d0 tx=0x7f3fd402a140 comp rx=0 tx=0).stop 2024-04-25T10:03:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 -- 172.21.15.5:0/2925538788 shutdown_connections 2024-04-25T10:03:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 --2- 172.21.15.5:0/2925538788 >> v2:172.21.15.80:3300/0 conn(0x7f3fe80ccfd0 0x7f3fe80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 --2- 172.21.15.5:0/2925538788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe80b2800 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 --2- 172.21.15.5:0/2925538788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe815a940 0x7f3fe80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 -- 172.21.15.5:0/2925538788 >> 172.21.15.5:0/2925538788 conn(0x7f3fe80999f0 msgr2=0x7f3fe80b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 -- 172.21.15.5:0/2925538788 shutdown_connections 2024-04-25T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.292+0000 7f3fed800640 1 -- 172.21.15.5:0/2925538788 wait complete. 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.294+0000 7f3fed800640 1 Processor -- start 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.294+0000 7f3fed800640 1 -- start start 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fed800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe80ccfd0 0x7f3fe81e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fed800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe815a940 0x7f3fe815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fed800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80cb8c0 con 0x7f3fe815f040 2024-04-25T10:03:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fed800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80cb5e0 con 0x7f3fe815a940 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fed800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80cbba0 con 0x7f3fe80ccfd0 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fe7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fe7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe815d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41620/0 (socket says 172.21.15.5:41620) 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fe7400640 1 -- 172.21.15.5:0/2125142013 learned_addr learned my addr 172.21.15.5:0/2125142013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fe6000640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe815a940 0x7f3fe815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.296+0000 7f3fe6a00640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe80ccfd0 0x7f3fe81e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fe7400640 1 -- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe80ccfd0 msgr2=0x7f3fe81e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fe7400640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe80ccfd0 0x7f3fe81e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fe7400640 1 -- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe815a940 msgr2=0x7f3fe815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fe7400640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe815a940 0x7f3fe815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fe7400640 1 -- 172.21.15.5:0/2125142013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80c5b20 con 0x7f3fe815f040 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fe7400640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe815d9d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000cd50 tx=0x7f3fe000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000f680 con 0x7f3fe815f040 2024-04-25T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe000f840 con 0x7f3fe815f040 2024-04-25T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe80c5e00 con 0x7f3fe815f040 2024-04-25T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.297+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000fc30 con 0x7f3fe815f040 2024-04-25T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.298+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe80c63c0 con 0x7f3fe815f040 2024-04-25T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.299+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe8068f60 con 0x7f3fe815f040 2024-04-25T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.300+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fe00213f0 con 0x7f3fe815f040 2024-04-25T10:03:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.301+0000 7f3fdea00640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fc806b4d0 0x7f3fc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.301+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f3fe008b170 con 0x7f3fe815f040 2024-04-25T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.302+0000 7f3fe6a00640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fc806b4d0 0x7f3fc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.303+0000 7f3fe6a00640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fc806b4d0 0x7f3fc806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006fd0 tx=0x7f3fd8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.306+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fe0014030 con 0x7f3fe815f040 2024-04-25T10:03:36.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:36.553+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3fe80d03a0 con 0x7f3fe815f040 2024-04-25T10:03:37.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.103+0000 7f3fdea00640 1 -- 172.21.15.5:0/2125142013 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3fe0059eb0 con 0x7f3fe815f040 2024-04-25T10:03:37.112 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T10:03:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.113+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fc806b4d0 msgr2=0x7f3fc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.113+0000 7f3fed800640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fc806b4d0 0x7f3fc806d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8006fd0 tx=0x7f3fd8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.113+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 msgr2=0x7f3fe815d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.113+0000 7f3fed800640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe815d9d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000cd50 tx=0x7f3fe000a650 comp rx=0 tx=0).stop 2024-04-25T10:03:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.117+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 shutdown_connections 2024-04-25T10:03:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.117+0000 7f3fed800640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe815f040 0x7f3fe815d9d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.117+0000 7f3fed800640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fc806b4d0 0x7f3fc806d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.117+0000 7f3fed800640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe815a940 0x7f3fe815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.117+0000 7f3fed800640 1 --2- 172.21.15.5:0/2125142013 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe80ccfd0 0x7f3fe81e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.117+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 >> 172.21.15.5:0/2125142013 conn(0x7f3fe80999f0 msgr2=0x7f3fe80ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.118+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 shutdown_connections 2024-04-25T10:03:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.120+0000 7f3fed800640 1 -- 172.21.15.5:0/2125142013 wait complete. 2024-04-25T10:03:37.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.335+0000 7ffb90800640 1 Processor -- start 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.335+0000 7ffb90800640 1 -- start start 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.336+0000 7ffb90800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb880cbb50 0x7ffb880ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.336+0000 7ffb90800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cad50 0x7ffb880cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.336+0000 7ffb90800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb880c9f90 con 0x7ffb881472c0 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.336+0000 7ffb90800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb880ca270 con 0x7ffb880cbb50 2024-04-25T10:03:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.336+0000 7ffb90800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb880ca550 con 0x7ffb880cad50 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.337+0000 7ffb8e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cad50 0x7ffb880cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.337+0000 7ffb8d400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb880cbb50 0x7ffb880ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.337+0000 7ffb8d400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb880cbb50 0x7ffb880ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:32900/0 (socket says 172.21.15.5:32900) 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.337+0000 7ffb8d400640 1 -- 172.21.15.5:0/2984447536 learned_addr learned my addr 172.21.15.5:0/2984447536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.337+0000 7ffb8e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cad50 0x7ffb880cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:53806/0 (socket says 172.21.15.5:53806) 2024-04-25T10:03:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.338+0000 7ffb8d400640 1 -- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cad50 msgr2=0x7ffb880cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.338+0000 7ffb8d400640 1 --2- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cad50 0x7ffb880cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.338+0000 7ffb8d400640 1 -- 172.21.15.5:0/2984447536 >> v1:172.21.15.80:6789/0 conn(0x7ffb881472c0 legacy=0x7ffb880cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.338+0000 7ffb8d400640 1 -- 172.21.15.5:0/2984447536 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb880ca830 con 0x7ffb880cbb50 2024-04-25T10:03:37.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.339+0000 7ffb8d400640 1 --2- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb880cbb50 0x7ffb880ce670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009870 tx=0x7ffb7802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b2acafec050337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.340+0000 7ffb8ca00640 1 -- 172.21.15.5:0/2984447536 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78037070 con 0x7ffb880cbb50 2024-04-25T10:03:37.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.340+0000 7ffb8ca00640 1 -- 172.21.15.5:0/2984447536 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78032440 con 0x7ffb880cbb50 2024-04-25T10:03:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.340+0000 7ffb8ca00640 1 -- 172.21.15.5:0/2984447536 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78032810 con 0x7ffb880cbb50 2024-04-25T10:03:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 -- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb880cbb50 msgr2=0x7ffb880ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 --2- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb880cbb50 0x7ffb880ce670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009870 tx=0x7ffb7802a130 comp rx=0 tx=0).stop 2024-04-25T10:03:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 -- 172.21.15.5:0/2984447536 shutdown_connections 2024-04-25T10:03:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 --2- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cad50 0x7ffb880cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 --2- 172.21.15.5:0/2984447536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb880cbb50 0x7ffb880ce670 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 -- 172.21.15.5:0/2984447536 >> 172.21.15.5:0/2984447536 conn(0x7ffb88071150 msgr2=0x7ffb88071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 -- 172.21.15.5:0/2984447536 shutdown_connections 2024-04-25T10:03:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.341+0000 7ffb90800640 1 -- 172.21.15.5:0/2984447536 wait complete. 2024-04-25T10:03:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.343+0000 7ffb90800640 1 Processor -- start 2024-04-25T10:03:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.343+0000 7ffb90800640 1 -- start start 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb90800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 0x7ffb881584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb90800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cbb50 0x7ffb880b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb90800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb881472c0 0x7ffb880b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb90800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8806d150 con 0x7ffb880cad50 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb90800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb880c9c60 con 0x7ffb881472c0 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb90800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8806d2d0 con 0x7ffb880cbb50 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb8de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 0x7ffb881584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb8de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 0x7ffb881584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35970/0 (socket says 172.21.15.5:35970) 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.344+0000 7ffb8de00640 1 -- 172.21.15.5:0/1068446406 learned_addr learned my addr 172.21.15.5:0/1068446406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8e800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb881472c0 0x7ffb880b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8d400640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cbb50 0x7ffb880b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8de00640 1 -- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cbb50 msgr2=0x7ffb880b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8de00640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cbb50 0x7ffb880b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8de00640 1 -- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb881472c0 msgr2=0x7ffb880b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8de00640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb881472c0 0x7ffb880b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8de00640 1 -- 172.21.15.5:0/1068446406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb881f3ca0 con 0x7ffb880cad50 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.345+0000 7ffb8de00640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 0x7ffb881584c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c340 tx=0x7ffb7c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.346+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c017070 con 0x7ffb880cad50 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.346+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0052d0 con 0x7ffb880cad50 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.346+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003850 con 0x7ffb880cad50 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.346+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb880b3940 con 0x7ffb880cad50 2024-04-25T10:03:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.346+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb880b3f00 con 0x7ffb880cad50 2024-04-25T10:03:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.347+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb88160820 con 0x7ffb880cad50 2024-04-25T10:03:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.348+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb7c01a070 con 0x7ffb880cad50 2024-04-25T10:03:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.349+0000 7ffb82000640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb6c06b4d0 0x7ffb6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.349+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb7c089740 con 0x7ffb880cad50 2024-04-25T10:03:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.350+0000 7ffb8d400640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb6c06b4d0 0x7ffb6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.351+0000 7ffb8d400640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb6c06b4d0 0x7ffb6c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb78034000 tx=0x7ffb7802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.353+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb88160820 con 0x7ffb880cad50 2024-04-25T10:03:37.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.551+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb88068fd0 con 0x7ffb880cad50 2024-04-25T10:03:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.554+0000 7ffb82000640 1 -- 172.21.15.5:0/1068446406 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16847 (secure 0 0 0) 0x7ffb7c058390 con 0x7ffb880cad50 2024-04-25T10:03:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:37.561 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:36.555860+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb6c06b4d0 msgr2=0x7ffb6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb6c06b4d0 0x7ffb6c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb78034000 tx=0x7ffb7802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 msgr2=0x7ffb881584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 0x7ffb881584c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c340 tx=0x7ffb7c00c800 comp rx=0 tx=0).stop 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 shutdown_connections 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb6c06b4d0 0x7ffb6c06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb881472c0 0x7ffb880b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffb880cbb50 0x7ffb880b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 --2- 172.21.15.5:0/1068446406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb880cad50 0x7ffb881584c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.557+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 >> 172.21.15.5:0/1068446406 conn(0x7ffb88071150 msgr2=0x7ffb880c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.558+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 shutdown_connections 2024-04-25T10:03:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.558+0000 7ffb90800640 1 -- 172.21.15.5:0/1068446406 wait complete. 2024-04-25T10:03:37.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.785+0000 7f0bda000640 1 Processor -- start 2024-04-25T10:03:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.785+0000 7f0bda000640 1 -- start start 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bda000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0bd40b4bb0 0x7f0bd40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bda000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40cf410 con 0x7f0bd4161200 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bda000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd40cf6f0 con 0x7f0bd40d07d0 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bda000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd40b28b0 con 0x7f0bd40b4bb0 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bd8c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0bd4161200 0x7f0bd40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40488/0 (socket says 172.21.15.5:40488) 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bd8c00640 1 -- 172.21.15.5:0/3840937097 learned_addr learned my addr 172.21.15.5:0/3840937097 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.787+0000 7f0bd9600640 1 --2- 172.21.15.5:0/3840937097 >> v2:172.21.15.5:3300/0 conn(0x7f0bd40b4bb0 0x7f0bd40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.788+0000 7f0bd3400640 1 -- 172.21.15.5:0/3840937097 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846336089 0 0) 0x7f0bd40cf410 con 0x7f0bd4161200 2024-04-25T10:03:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.788+0000 7f0bd3400640 1 -- 172.21.15.5:0/3840937097 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bbc003520 con 0x7f0bd4161200 2024-04-25T10:03:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.788+0000 7f0bd9600640 1 -- 172.21.15.5:0/3840937097 >> v1:172.21.15.80:6789/0 conn(0x7f0bd40d07d0 legacy=0x7f0bd40b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.788+0000 7f0bd9600640 1 -- 172.21.15.5:0/3840937097 >> v1:172.21.15.5:6789/0 conn(0x7f0bd4161200 legacy=0x7f0bd40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.788+0000 7f0bd9600640 1 -- 172.21.15.5:0/3840937097 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd40d3fd0 con 0x7f0bd40b4bb0 2024-04-25T10:03:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.789+0000 7f0bd9600640 1 --2- 172.21.15.5:0/3840937097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd40b4bb0 0x7f0bd40d2380 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d90 tx=0x7f0bcc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ebd4e3301eba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.789+0000 7f0bd3400640 1 -- 172.21.15.5:0/3840937097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007740 con 0x7f0bd40b4bb0 2024-04-25T10:03:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.789+0000 7f0bd3400640 1 -- 172.21.15.5:0/3840937097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc0078e0 con 0x7f0bd40b4bb0 2024-04-25T10:03:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.789+0000 7f0bd3400640 1 -- 172.21.15.5:0/3840937097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc007cb0 con 0x7f0bd40b4bb0 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 -- 172.21.15.5:0/3840937097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd40b4bb0 msgr2=0x7f0bd40d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 --2- 172.21.15.5:0/3840937097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd40b4bb0 0x7f0bd40d2380 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009d90 tx=0x7f0bcc02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 -- 172.21.15.5:0/3840937097 shutdown_connections 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 --2- 172.21.15.5:0/3840937097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd40b4bb0 0x7f0bd40d2380 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 -- 172.21.15.5:0/3840937097 >> 172.21.15.5:0/3840937097 conn(0x7f0bd408af80 msgr2=0x7f0bd40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 -- 172.21.15.5:0/3840937097 shutdown_connections 2024-04-25T10:03:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.790+0000 7f0bda000640 1 -- 172.21.15.5:0/3840937097 wait complete. 2024-04-25T10:03:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.792+0000 7f0bda000640 1 Processor -- start 2024-04-25T10:03:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.792+0000 7f0bda000640 1 -- start start 2024-04-25T10:03:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bda000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bd40b4bb0 0x7f0bd40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bda000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd40d07d0 0x7f0bd4150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bda000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 0x7f0bd409d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bda000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40b3810 con 0x7f0bd4161200 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bda000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40b3690 con 0x7f0bd40d07d0 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bda000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40b3990 con 0x7f0bd40b4bb0 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bd9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 0x7f0bd409d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bd9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 0x7f0bd409d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35988/0 (socket says 172.21.15.5:35988) 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.793+0000 7f0bd9600640 1 -- 172.21.15.5:0/1475577481 learned_addr learned my addr 172.21.15.5:0/1475577481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd8c00640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bd40b4bb0 0x7f0bd40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd3e00640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd40d07d0 0x7f0bd4150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd9600640 1 -- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bd40b4bb0 msgr2=0x7f0bd40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd9600640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bd40b4bb0 0x7f0bd40cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd9600640 1 -- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd40d07d0 msgr2=0x7f0bd4150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd9600640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd40d07d0 0x7f0bd4150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd9600640 1 -- 172.21.15.5:0/1475577481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd409da10 con 0x7f0bd4161200 2024-04-25T10:03:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.794+0000 7f0bd9600640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 0x7f0bd409d4d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009eb0 tx=0x7f0bcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.795+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc040070 con 0x7f0bd4161200 2024-04-25T10:03:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.795+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc031bc0 con 0x7f0bd4161200 2024-04-25T10:03:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.795+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc009450 con 0x7f0bd4161200 2024-04-25T10:03:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.795+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd40cb970 con 0x7f0bd4161200 2024-04-25T10:03:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.795+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd40cbf30 con 0x7f0bd4161200 2024-04-25T10:03:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.796+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4068f60 con 0x7f0bd4161200 2024-04-25T10:03:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.797+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bcc039040 con 0x7f0bd4161200 2024-04-25T10:03:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.798+0000 7f0bd0c00640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0bb406b4d0 0x7f0bb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.798+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bcc0abd60 con 0x7f0bd4161200 2024-04-25T10:03:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.799+0000 7f0bd8c00640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0bb406b4d0 0x7f0bb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.800+0000 7f0bd8c00640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0bb406b4d0 0x7f0bb406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80011e0 tx=0x7f0bc8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:37.802+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bcc07a9b0 con 0x7f0bd4161200 2024-04-25T10:03:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.010+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bd40ceb10 con 0x7f0bd4161200 2024-04-25T10:03:38.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.013+0000 7f0bd0c00640 1 -- 172.21.15.5:0/1475577481 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16847 (secure 0 0 0) 0x7f0bcc07a2b0 con 0x7f0bd4161200 2024-04-25T10:03:38.019 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:38.019 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:36.555860+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:38.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0bb406b4d0 msgr2=0x7f0bb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0bb406b4d0 0x7f0bb406d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80011e0 tx=0x7f0bc8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 msgr2=0x7f0bd409d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 0x7f0bd409d4d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009eb0 tx=0x7f0bcc004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 shutdown_connections 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bd4161200 0x7f0bd409d4d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0bb406b4d0 0x7f0bb406d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bd40d07d0 0x7f0bd4150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 --2- 172.21.15.5:0/1475577481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bd40b4bb0 0x7f0bd40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 >> 172.21.15.5:0/1475577481 conn(0x7f0bd408af80 msgr2=0x7f0bd415fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 shutdown_connections 2024-04-25T10:03:38.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.016+0000 7f0bda000640 1 -- 172.21.15.5:0/1475577481 wait complete. 2024-04-25T10:03:38.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:03:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.251+0000 7f1a2a600640 1 Processor -- start 2024-04-25T10:03:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.252+0000 7f1a2a600640 1 -- start start 2024-04-25T10:03:38.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.253+0000 7f1a2a600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a24143fb0 0x7f1a240b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.253+0000 7f1a2a600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a240ced20 0x7f1a2406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.253+0000 7f1a2a600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a240b2ee0 con 0x7f1a2406e240 2024-04-25T10:03:38.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.253+0000 7f1a2a600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2406ebe0 con 0x7f1a240ced20 2024-04-25T10:03:38.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.253+0000 7f1a2a600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240b3150 con 0x7f1a24143fb0 2024-04-25T10:03:38.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.254+0000 7f1a1aa00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1a2406e240 0x7f1a240ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35876/0 (socket says 172.21.15.5:35876) 2024-04-25T10:03:38.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.254+0000 7f1a1aa00640 1 -- 172.21.15.5:0/1935936987 learned_addr learned my addr 172.21.15.5:0/1935936987 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:38.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.254+0000 7f1a23e00640 1 --2- 172.21.15.5:0/1935936987 >> v2:172.21.15.80:3300/0 conn(0x7f1a240ced20 0x7f1a2406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.254+0000 7f1a23400640 1 --2- 172.21.15.5:0/1935936987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a24143fb0 0x7f1a240b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.255+0000 7f1a22a00640 1 -- 172.21.15.5:0/1935936987 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901028958 0 0) 0x7f1a240b2ee0 con 0x7f1a2406e240 2024-04-25T10:03:38.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.255+0000 7f1a22a00640 1 -- 172.21.15.5:0/1935936987 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a08003520 con 0x7f1a2406e240 2024-04-25T10:03:38.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.255+0000 7f1a23400640 1 -- 172.21.15.5:0/1935936987 >> v2:172.21.15.80:3300/0 conn(0x7f1a240ced20 msgr2=0x7f1a2406e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:38.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.255+0000 7f1a23400640 1 --2- 172.21.15.5:0/1935936987 >> v2:172.21.15.80:3300/0 conn(0x7f1a240ced20 0x7f1a2406e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.255+0000 7f1a23400640 1 -- 172.21.15.5:0/1935936987 >> v1:172.21.15.80:6789/0 conn(0x7f1a2406e240 legacy=0x7f1a240ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.255+0000 7f1a23400640 1 -- 172.21.15.5:0/1935936987 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240b3430 con 0x7f1a24143fb0 2024-04-25T10:03:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.257+0000 7f1a23400640 1 --2- 172.21.15.5:0/1935936987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a24143fb0 0x7f1a240b3c30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a140097a0 tx=0x7f1a1402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab283d8d114f46c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.257+0000 7f1a22a00640 1 -- 172.21.15.5:0/1935936987 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14037070 con 0x7f1a24143fb0 2024-04-25T10:03:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.257+0000 7f1a22a00640 1 -- 172.21.15.5:0/1935936987 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a14032440 con 0x7f1a24143fb0 2024-04-25T10:03:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.257+0000 7f1a22a00640 1 -- 172.21.15.5:0/1935936987 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14032810 con 0x7f1a24143fb0 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 -- 172.21.15.5:0/1935936987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a24143fb0 msgr2=0x7f1a240b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 --2- 172.21.15.5:0/1935936987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a24143fb0 0x7f1a240b3c30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1a140097a0 tx=0x7f1a1402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 -- 172.21.15.5:0/1935936987 shutdown_connections 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 --2- 172.21.15.5:0/1935936987 >> v2:172.21.15.80:3300/0 conn(0x7f1a240ced20 0x7f1a2406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 --2- 172.21.15.5:0/1935936987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a24143fb0 0x7f1a240b3c30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 -- 172.21.15.5:0/1935936987 >> 172.21.15.5:0/1935936987 conn(0x7f1a2406ab60 msgr2=0x7f1a2406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 -- 172.21.15.5:0/1935936987 shutdown_connections 2024-04-25T10:03:38.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.258+0000 7f1a2a600640 1 -- 172.21.15.5:0/1935936987 wait complete. 2024-04-25T10:03:38.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.261+0000 7f1a2a600640 1 Processor -- start 2024-04-25T10:03:38.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.261+0000 7f1a2a600640 1 -- start start 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a2a600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a2406e240 0x7f1a241f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a2a600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a240ced20 0x7f1a241f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a2a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 0x7f1a241f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a2a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a240cf2f0 con 0x7f1a24143fb0 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a2a600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a24144390 con 0x7f1a240ced20 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a2a600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2415ac70 con 0x7f1a2406e240 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.262+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 0x7f1a241f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 0x7f1a241f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36002/0 (socket says 172.21.15.5:36002) 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 -- 172.21.15.5:0/2744022224 learned_addr learned my addr 172.21.15.5:0/2744022224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23400640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a2406e240 0x7f1a241f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a1aa00640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a240ced20 0x7f1a241f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 -- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a2406e240 msgr2=0x7f1a241f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a2406e240 0x7f1a241f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 -- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a240ced20 msgr2=0x7f1a241f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a240ced20 0x7f1a241f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 -- 172.21.15.5:0/2744022224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240af510 con 0x7f1a24143fb0 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.263+0000 7f1a23e00640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 0x7f1a241f6ef0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00e930 tx=0x7f1a1c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.264+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c017070 con 0x7f1a24143fb0 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.264+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1c00c980 con 0x7f1a24143fb0 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.264+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240af7f0 con 0x7f1a24143fb0 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.264+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c00cd70 con 0x7f1a24143fb0 2024-04-25T10:03:38.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.264+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a240afdb0 con 0x7f1a24143fb0 2024-04-25T10:03:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.265+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a240affb0 con 0x7f1a24143fb0 2024-04-25T10:03:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.266+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a1c01a070 con 0x7f1a24143fb0 2024-04-25T10:03:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.267+0000 7f1a1be00640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a0006b4d0 0x7f1a0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:38.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.267+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1a1c08a800 con 0x7f1a24143fb0 2024-04-25T10:03:38.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.268+0000 7f1a23400640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a0006b4d0 0x7f1a0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:38.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.269+0000 7f1a23400640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a0006b4d0 0x7f1a0006d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a14034000 tx=0x7f1a1402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:38.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.271+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a240affb0 con 0x7f1a24143fb0 2024-04-25T10:03:38.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:38.515+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1a24068f60 con 0x7f1a24143fb0 2024-04-25T10:03:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.123+0000 7f1a1be00640 1 -- 172.21.15.5:0/2744022224 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1a1c059510 con 0x7f1a24143fb0 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a0006b4d0 msgr2=0x7f1a0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a0006b4d0 0x7f1a0006d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a14034000 tx=0x7f1a1402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 msgr2=0x7f1a241f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 0x7f1a241f6ef0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c00e930 tx=0x7f1a1c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 shutdown_connections 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a24143fb0 0x7f1a241f6ef0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a0006b4d0 0x7f1a0006d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a240ced20 0x7f1a241f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 --2- 172.21.15.5:0/2744022224 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1a2406e240 0x7f1a241f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 >> 172.21.15.5:0/2744022224 conn(0x7f1a2406ab60 msgr2=0x7f1a240add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.127+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 shutdown_connections 2024-04-25T10:03:39.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.128+0000 7f1a2a600640 1 -- 172.21.15.5:0/2744022224 wait complete. 2024-04-25T10:03:39.155 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:03:39.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:03:39.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.365+0000 7f2bba200640 1 Processor -- start 2024-04-25T10:03:39.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.365+0000 7f2bba200640 1 -- start start 2024-04-25T10:03:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.366+0000 7f2bba200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2bb40b78c0 0x7f2bb41685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.366+0000 7f2bba200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b7ef0 con 0x7f2bb40d0780 2024-04-25T10:03:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.366+0000 7f2bba200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b81d0 con 0x7f2bb40d2c40 2024-04-25T10:03:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.366+0000 7f2bba200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40b9840 con 0x7f2bb40b78c0 2024-04-25T10:03:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.367+0000 7f2bb3400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2bb40d0780 0x7f2bb40d0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40502/0 (socket says 172.21.15.5:40502) 2024-04-25T10:03:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.367+0000 7f2bb3400640 1 -- 172.21.15.5:0/2727488032 learned_addr learned my addr 172.21.15.5:0/2727488032 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.367+0000 7f2bb3e00640 1 --2- 172.21.15.5:0/2727488032 >> v2:172.21.15.5:3300/0 conn(0x7f2bb40b78c0 0x7f2bb41685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.367+0000 7f2bb2000640 1 -- 172.21.15.5:0/2727488032 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780334119 0 0) 0x7f2bb40b7ef0 con 0x7f2bb40d0780 2024-04-25T10:03:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.368+0000 7f2bb2000640 1 -- 172.21.15.5:0/2727488032 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b98003520 con 0x7f2bb40d0780 2024-04-25T10:03:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.368+0000 7f2bb3e00640 1 -- 172.21.15.5:0/2727488032 >> v1:172.21.15.80:6789/0 conn(0x7f2bb40d2c40 legacy=0x7f2bb40b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.368+0000 7f2bb3e00640 1 -- 172.21.15.5:0/2727488032 >> v1:172.21.15.5:6789/0 conn(0x7f2bb40d0780 legacy=0x7f2bb40d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.368+0000 7f2bb3e00640 1 -- 172.21.15.5:0/2727488032 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40b84b0 con 0x7f2bb40b78c0 2024-04-25T10:03:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.369+0000 7f2bb3e00640 1 --2- 172.21.15.5:0/2727488032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40b78c0 0x7f2bb41685d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009d90 tx=0x7f2bac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2673cea260374b91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.369+0000 7f2bb2000640 1 -- 172.21.15.5:0/2727488032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac007740 con 0x7f2bb40b78c0 2024-04-25T10:03:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.369+0000 7f2bb2000640 1 -- 172.21.15.5:0/2727488032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac0078e0 con 0x7f2bb40b78c0 2024-04-25T10:03:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.369+0000 7f2bb2000640 1 -- 172.21.15.5:0/2727488032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac007c90 con 0x7f2bb40b78c0 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 -- 172.21.15.5:0/2727488032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40b78c0 msgr2=0x7f2bb41685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 --2- 172.21.15.5:0/2727488032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40b78c0 0x7f2bb41685d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009d90 tx=0x7f2bac02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 -- 172.21.15.5:0/2727488032 shutdown_connections 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 --2- 172.21.15.5:0/2727488032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40b78c0 0x7f2bb41685d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 -- 172.21.15.5:0/2727488032 >> 172.21.15.5:0/2727488032 conn(0x7f2bb40726b0 msgr2=0x7f2bb4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 -- 172.21.15.5:0/2727488032 shutdown_connections 2024-04-25T10:03:39.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.370+0000 7f2bba200640 1 -- 172.21.15.5:0/2727488032 wait complete. 2024-04-25T10:03:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.373+0000 7f2bba200640 1 Processor -- start 2024-04-25T10:03:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.373+0000 7f2bba200640 1 -- start start 2024-04-25T10:03:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.374+0000 7f2bba200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bb40b78c0 0x7f2bb40cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.374+0000 7f2bba200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bb40d0780 0x7f2bb40cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.374+0000 7f2bba200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 0x7f2bb40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.374+0000 7f2bba200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41611b0 con 0x7f2bb40d2c40 2024-04-25T10:03:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.374+0000 7f2bba200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40b8a70 con 0x7f2bb40b78c0 2024-04-25T10:03:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.374+0000 7f2bba200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4161330 con 0x7f2bb40d0780 2024-04-25T10:03:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 0x7f2bb40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 0x7f2bb40b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36018/0 (socket says 172.21.15.5:36018) 2024-04-25T10:03:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 -- 172.21.15.5:0/1756340400 learned_addr learned my addr 172.21.15.5:0/1756340400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb2a00640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bb40d0780 0x7f2bb40cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3400640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bb40b78c0 0x7f2bb40cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 -- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bb40d0780 msgr2=0x7f2bb40cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bb40d0780 0x7f2bb40cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 -- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bb40b78c0 msgr2=0x7f2bb40cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bb40b78c0 0x7f2bb40cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3e00640 1 -- 172.21.15.5:0/1756340400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40cdbe0 con 0x7f2bb40d2c40 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.375+0000 7f2bb3400640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bb40b78c0 0x7f2bb40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.376+0000 7f2bb3e00640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 0x7f2bb40b3b90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009eb0 tx=0x7f2bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.376+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac040070 con 0x7f2bb40d2c40 2024-04-25T10:03:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.376+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac031bc0 con 0x7f2bb40d2c40 2024-04-25T10:03:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.376+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac009450 con 0x7f2bb40d2c40 2024-04-25T10:03:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.376+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb40b30f0 con 0x7f2bb40d2c40 2024-04-25T10:03:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.376+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb40cd900 con 0x7f2bb40d2c40 2024-04-25T10:03:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.377+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb4068f60 con 0x7f2bb40d2c40 2024-04-25T10:03:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.378+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bac039040 con 0x7f2bb40d2c40 2024-04-25T10:03:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.379+0000 7f2bab400640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2b9006b4d0 0x7f2b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.379+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bac0abc80 con 0x7f2bb40d2c40 2024-04-25T10:03:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.380+0000 7f2bb3400640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2b9006b4d0 0x7f2b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.381+0000 7f2bb3400640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2b9006b4d0 0x7f2b9006d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40011e0 tx=0x7f2ba4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.385+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bac043360 con 0x7f2bb40d2c40 2024-04-25T10:03:39.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:39.638+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2bb4072e20 con 0x7f2bb40d2c40 2024-04-25T10:03:40.047 INFO:tasks.ceph.mds.b.smithi080.stderr:2024-04-25T10:03:40.043+0000 7f1f2d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:40.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.126+0000 7f2bab400640 1 -- 172.21.15.5:0/1756340400 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2bac07a960 con 0x7f2bb40d2c40 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2b9006b4d0 msgr2=0x7f2b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2b9006b4d0 0x7f2b9006d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40011e0 tx=0x7f2ba4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 msgr2=0x7f2bb40b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 0x7f2bb40b3b90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009eb0 tx=0x7f2bac004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 shutdown_connections 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bb40d2c40 0x7f2bb40b3b90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bb40d0780 0x7f2bb40cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2b9006b4d0 0x7f2b9006d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 --2- 172.21.15.5:0/1756340400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bb40b78c0 0x7f2bb40cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 >> 172.21.15.5:0/1756340400 conn(0x7f2bb40726b0 msgr2=0x7f2bb40d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 shutdown_connections 2024-04-25T10:03:40.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.131+0000 7f2bba200640 1 -- 172.21.15.5:0/1756340400 wait complete. 2024-04-25T10:03:40.162 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:03:40.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:40.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.378+0000 7f2d79c00640 1 Processor -- start 2024-04-25T10:03:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.378+0000 7f2d79c00640 1 -- start start 2024-04-25T10:03:40.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.379+0000 7f2d79c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d7406e970 0x7f2d740cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.379+0000 7f2d79c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740cdbf0 con 0x7f2d74144100 2024-04-25T10:03:40.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.379+0000 7f2d79c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740cded0 con 0x7f2d740ced20 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.379+0000 7f2d79c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d741444e0 con 0x7f2d7406e970 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2d74144100 0x7f2d7406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40512/0 (socket says 172.21.15.5:40512) 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72a00640 1 -- 172.21.15.5:0/2826958413 learned_addr learned my addr 172.21.15.5:0/2826958413 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006156002 0 0) 0x7f2d740cdbf0 con 0x7f2d74144100 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58005730 con 0x7f2d74144100 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d6be00640 1 --2- 172.21.15.5:0/2826958413 >> v2:172.21.15.80:3300/0 conn(0x7f2d7406e970 0x7f2d740cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 814443838 0 0) 0x7f2d58005730 con 0x7f2d74144100 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 >> v2:172.21.15.80:3300/0 conn(0x7f2d7406e970 msgr2=0x7f2d740cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.380+0000 7f2d72000640 1 --2- 172.21.15.5:0/2826958413 >> v2:172.21.15.80:3300/0 conn(0x7f2d7406e970 0x7f2d740cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 >> v1:172.21.15.80:6789/0 conn(0x7f2d740ced20 legacy=0x7f2d7415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740ce1b0 con 0x7f2d74144100 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2d64001670 con 0x7f2d74144100 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 >> v1:172.21.15.5:6789/0 conn(0x7f2d74144100 legacy=0x7f2d7406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58005bc0 0x7f2d58005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d58006a10 0x7f2d58006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d580074f0 0x7f2d5800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741444e0 con 0x7f2d580074f0 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800e3c0 con 0x7f2d58006a10 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800e5c0 con 0x7f2d58005bc0 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d73400640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d580074f0 0x7f2d5800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d6be00640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d58006a10 0x7f2d58006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d72a00640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58005bc0 0x7f2d58005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.381+0000 7f2d73400640 1 -- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58005bc0 msgr2=0x7f2d58005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.382+0000 7f2d73400640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58005bc0 0x7f2d58005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.382+0000 7f2d73400640 1 -- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d58006a10 msgr2=0x7f2d58006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.382+0000 7f2d73400640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d58006a10 0x7f2d58006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.382+0000 7f2d73400640 1 -- 172.21.15.5:0/2826958413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58005730 con 0x7f2d580074f0 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.382+0000 7f2d72a00640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58005bc0 0x7f2d58005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.382+0000 7f2d73400640 1 -- 172.21.15.5:0/2826958413 reap_dead start 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.383+0000 7f2d73400640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d580074f0 0x7f2d5800dbd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001e20 tx=0x7f2d58007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cdf68d70b70bde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.383+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c001a70 con 0x7f2d580074f0 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.383+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c02ae40 con 0x7f2d580074f0 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.383+0000 7f2d72000640 1 -- 172.21.15.5:0/2826958413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c003bb0 con 0x7f2d580074f0 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.383+0000 7f2d79c00640 1 -- 172.21.15.5:0/2826958413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d580074f0 msgr2=0x7f2d5800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.383+0000 7f2d79c00640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d580074f0 0x7f2d5800dbd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c001e20 tx=0x7f2d58007a40 comp rx=0 tx=0).stop 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.384+0000 7f2d79c00640 1 -- 172.21.15.5:0/2826958413 shutdown_connections 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.384+0000 7f2d79c00640 1 --2- 172.21.15.5:0/2826958413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d580074f0 0x7f2d5800dbd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.384+0000 7f2d79c00640 1 -- 172.21.15.5:0/2826958413 >> 172.21.15.5:0/2826958413 conn(0x7f2d74071150 msgr2=0x7f2d74071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.384+0000 7f2d79c00640 1 -- 172.21.15.5:0/2826958413 shutdown_connections 2024-04-25T10:03:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.384+0000 7f2d79c00640 1 -- 172.21.15.5:0/2826958413 wait complete. 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.385+0000 7f2d79c00640 1 Processor -- start 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.386+0000 7f2d79c00640 1 -- start start 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d79c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580074f0 0x7f2d74159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d79c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 0x7f2d741e7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d79c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d741e7ac0 0x7f2d741e4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d79c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740cea00 con 0x7f2d741f37d0 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d79c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7415ac70 con 0x7f2d580074f0 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d79c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740cd610 con 0x7f2d741e7ac0 2024-04-25T10:03:40.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 0x7f2d741e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 0x7f2d741e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36024/0 (socket says 172.21.15.5:36024) 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 -- 172.21.15.5:0/3111326558 learned_addr learned my addr 172.21.15.5:0/3111326558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d73400640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d741e7ac0 0x7f2d741e4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d72a00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580074f0 0x7f2d74159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 -- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d741e7ac0 msgr2=0x7f2d741e4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d741e7ac0 0x7f2d741e4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 -- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580074f0 msgr2=0x7f2d74159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.387+0000 7f2d6be00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580074f0 0x7f2d74159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d6be00640 1 -- 172.21.15.5:0/3111326558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d741e4d60 con 0x7f2d741f37d0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d6be00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 0x7f2d741e7580 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00f580 tx=0x7f2d5c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c013810 con 0x7f2d741f37d0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c01a070 con 0x7f2d741f37d0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c013bb0 con 0x7f2d741f37d0 2024-04-25T10:03:40.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d741e5040 con 0x7f2d741f37d0 2024-04-25T10:03:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.388+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d741e5600 con 0x7f2d741f37d0 2024-04-25T10:03:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.389+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d741e7ed0 con 0x7f2d741f37d0 2024-04-25T10:03:40.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.391+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d5c012070 con 0x7f2d741f37d0 2024-04-25T10:03:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.392+0000 7f2d6aa00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.392+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d5c08a410 con 0x7f2d741f37d0 2024-04-25T10:03:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.393+0000 7f2d72a00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.394+0000 7f2d72a00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000f60 tx=0x7f2d64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.396+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d741e7ed0 con 0x7f2d741f37d0 2024-04-25T10:03:40.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.637+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d740cc1e0 con 0x7f2d741f37d0 2024-04-25T10:03:40.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.639+0000 7f2d6aa00640 1 -- 172.21.15.5:0/3111326558 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f2d5c01c310 con 0x7f2d741f37d0 2024-04-25T10:03:40.650 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:40.650 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:03:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 msgr2=0x7f2d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000f60 tx=0x7f2d64009040 comp rx=0 tx=0).stop 2024-04-25T10:03:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 msgr2=0x7f2d741e7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 0x7f2d741e7580 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00f580 tx=0x7f2d5c00fa40 comp rx=0 tx=0).stop 2024-04-25T10:03:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 shutdown_connections 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d741e7ac0 0x7f2d741e4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d741f37d0 0x7f2d741e7580 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 --2- 172.21.15.5:0/3111326558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580074f0 0x7f2d74159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 >> 172.21.15.5:0/3111326558 conn(0x7f2d74071150 msgr2=0x7f2d7415cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 shutdown_connections 2024-04-25T10:03:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.642+0000 7f2d79c00640 1 -- 172.21.15.5:0/3111326558 wait complete. 2024-04-25T10:03:40.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.872+0000 7f5b5be00640 1 Processor -- start 2024-04-25T10:03:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.872+0000 7f5b5be00640 1 -- start start 2024-04-25T10:03:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.873+0000 7f5b5be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b5c0d07d0 0x7f5b5c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.873+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.873+0000 7f5b5be00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b5c0cf410 con 0x7f5b5c161200 2024-04-25T10:03:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.873+0000 7f5b5be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b28b0 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.873+0000 7f5b5be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0cf6f0 con 0x7f5b5c0d2380 2024-04-25T10:03:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.874+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.874+0000 7f5b5a000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b5c0d07d0 0x7f5b5c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.874+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:53872/0 (socket says 172.21.15.5:53872) 2024-04-25T10:03:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.874+0000 7f5b5b400640 1 -- 172.21.15.5:0/2132014774 learned_addr learned my addr 172.21.15.5:0/2132014774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.874+0000 7f5b5a000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b5c0d07d0 0x7f5b5c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:32980/0 (socket says 172.21.15.5:32980) 2024-04-25T10:03:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.875+0000 7f5b5b400640 1 -- 172.21.15.5:0/2132014774 >> v2:172.21.15.80:3300/0 conn(0x7f5b5c0d07d0 msgr2=0x7f5b5c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.875+0000 7f5b5b400640 1 --2- 172.21.15.5:0/2132014774 >> v2:172.21.15.80:3300/0 conn(0x7f5b5c0d07d0 0x7f5b5c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.875+0000 7f5b5b400640 1 -- 172.21.15.5:0/2132014774 >> v1:172.21.15.80:6789/0 conn(0x7f5b5c161200 legacy=0x7f5b5c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.875+0000 7f5b5b400640 1 -- 172.21.15.5:0/2132014774 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0d3fd0 con 0x7f5b5c0d2380 2024-04-25T10:03:40.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.877+0000 7f5b5b400640 1 --2- 172.21.15.5:0/2132014774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b2f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009e50 tx=0x7f5b5402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2458216f1a961faa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.877+0000 7f5b59600640 1 -- 172.21.15.5:0/2132014774 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b54036070 con 0x7f5b5c0d2380 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b59600640 1 -- 172.21.15.5:0/2132014774 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b54031600 con 0x7f5b5c0d2380 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b59600640 1 -- 172.21.15.5:0/2132014774 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b540319d0 con 0x7f5b5c0d2380 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 -- 172.21.15.5:0/2132014774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 msgr2=0x7f5b5c0b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 --2- 172.21.15.5:0/2132014774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b2f80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009e50 tx=0x7f5b5402a710 comp rx=0 tx=0).stop 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 -- 172.21.15.5:0/2132014774 shutdown_connections 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 --2- 172.21.15.5:0/2132014774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b2f80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 --2- 172.21.15.5:0/2132014774 >> v2:172.21.15.80:3300/0 conn(0x7f5b5c0d07d0 0x7f5b5c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 -- 172.21.15.5:0/2132014774 >> 172.21.15.5:0/2132014774 conn(0x7f5b5c08af80 msgr2=0x7f5b5c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 -- 172.21.15.5:0/2132014774 shutdown_connections 2024-04-25T10:03:40.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.878+0000 7f5b5be00640 1 -- 172.21.15.5:0/2132014774 wait complete. 2024-04-25T10:03:40.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.880+0000 7f5b5be00640 1 Processor -- start 2024-04-25T10:03:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.880+0000 7f5b5be00640 1 -- start start 2024-04-25T10:03:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 0x7f5b5c152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c161200 0x7f5b5c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b3640 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b34c0 con 0x7f5b5c161200 2024-04-25T10:03:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b37c0 con 0x7f5b5c0d2380 2024-04-25T10:03:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 0x7f5b5c152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.881+0000 7f5b5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 0x7f5b5c152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36032/0 (socket says 172.21.15.5:36032) 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 -- 172.21.15.5:0/3697172930 learned_addr learned my addr 172.21.15.5:0/3697172930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5b400640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c161200 0x7f5b5c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5a000640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 -- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 msgr2=0x7f5b5c0b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 -- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c161200 msgr2=0x7f5b5c0c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c161200 0x7f5b5c0c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 -- 172.21.15.5:0/3697172930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0c71c0 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.882+0000 7f5b5aa00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 0x7f5b5c152a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00c3c0 tx=0x7f5b4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.883+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c0052d0 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.883+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c005490 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.883+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c007260 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.883+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c152f50 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.883+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c0c7740 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.884+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c068f60 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.885+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b4c007400 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.886+0000 7f5b52a00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b3c06b4d0 0x7f5b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.886+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b4c089910 con 0x7f5b5c0d07d0 2024-04-25T10:03:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.887+0000 7f5b5a000640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b3c06b4d0 0x7f5b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.888+0000 7f5b5a000640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b3c06b4d0 0x7f5b3c06d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0c8ff0 tx=0x7f5b48008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:40.891+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b4c058560 con 0x7f5b5c0d07d0 2024-04-25T10:03:41.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.092+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b5c0b09f0 con 0x7f5b5c0d07d0 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.095+0000 7f5b52a00640 1 -- 172.21.15.5:0/3697172930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f5b4c057e60 con 0x7f5b5c0d07d0 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:38.104579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b3c06b4d0 msgr2=0x7f5b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b3c06b4d0 0x7f5b3c06d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0c8ff0 tx=0x7f5b48008040 comp rx=0 tx=0).stop 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 msgr2=0x7f5b5c152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 0x7f5b5c152a10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00c3c0 tx=0x7f5b4c00c880 comp rx=0 tx=0).stop 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 shutdown_connections 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b3c06b4d0 0x7f5b3c06d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5c161200 0x7f5b5c0c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b5c0d2380 0x7f5b5c0b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 --2- 172.21.15.5:0/3697172930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b5c0d07d0 0x7f5b5c152a10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 >> 172.21.15.5:0/3697172930 conn(0x7f5b5c08af80 msgr2=0x7f5b5c0d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 shutdown_connections 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.098+0000 7f5b5be00640 1 -- 172.21.15.5:0/3697172930 wait complete. 2024-04-25T10:03:41.124 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:03:41.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.326+0000 7f15bb200640 1 Processor -- start 2024-04-25T10:03:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.326+0000 7f15bb200640 1 -- start start 2024-04-25T10:03:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15bb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc15db30 0x7f15bc0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15bb200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f15bc0d0f30 0x7f15bc06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15bb200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15bc0cfe60 con 0x7f15bc06f100 2024-04-25T10:03:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15bb200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15bc06f570 con 0x7f15bc0d0f30 2024-04-25T10:03:41.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15bb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0cf9d0 con 0x7f15bc15db30 2024-04-25T10:03:41.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15b9400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f15bc0d0f30 0x7f15bc06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15b9400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f15bc0d0f30 0x7f15bc06ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:32994/0 (socket says 172.21.15.5:32994) 2024-04-25T10:03:41.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.328+0000 7f15b9400640 1 -- 172.21.15.5:0/725362132 learned_addr learned my addr 172.21.15.5:0/725362132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:41.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.329+0000 7f15b9e00640 1 --2- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc15db30 0x7f15bc0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.329+0000 7f15b9400640 1 -- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc15db30 msgr2=0x7f15bc0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.329+0000 7f15b9400640 1 --2- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc15db30 0x7f15bc0ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.329+0000 7f15b9400640 1 -- 172.21.15.5:0/725362132 >> v1:172.21.15.80:6789/0 conn(0x7f15bc06f100 legacy=0x7f15bc15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.329+0000 7f15b9400640 1 -- 172.21.15.5:0/725362132 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0cedc0 con 0x7f15bc0d0f30 2024-04-25T10:03:41.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.331+0000 7f15b9400640 1 --2- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc0d0f30 0x7f15bc06ebc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009e50 tx=0x7f15a802c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4fa7a491d36795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.331+0000 7f15b8a00640 1 -- 172.21.15.5:0/725362132 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8032610 con 0x7f15bc0d0f30 2024-04-25T10:03:41.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.331+0000 7f15b8a00640 1 -- 172.21.15.5:0/725362132 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a8037070 con 0x7f15bc0d0f30 2024-04-25T10:03:41.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.331+0000 7f15b8a00640 1 -- 172.21.15.5:0/725362132 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a80329d0 con 0x7f15bc0d0f30 2024-04-25T10:03:41.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 -- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc0d0f30 msgr2=0x7f15bc06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 --2- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc0d0f30 0x7f15bc06ebc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009e50 tx=0x7f15a802c920 comp rx=0 tx=0).stop 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 -- 172.21.15.5:0/725362132 shutdown_connections 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 --2- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc0d0f30 0x7f15bc06ebc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 --2- 172.21.15.5:0/725362132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc15db30 0x7f15bc0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 -- 172.21.15.5:0/725362132 >> 172.21.15.5:0/725362132 conn(0x7f15bc0999f0 msgr2=0x7f15bc0d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 -- 172.21.15.5:0/725362132 shutdown_connections 2024-04-25T10:03:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.332+0000 7f15bb200640 1 -- 172.21.15.5:0/725362132 wait complete. 2024-04-25T10:03:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.334+0000 7f15bb200640 1 Processor -- start 2024-04-25T10:03:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.334+0000 7f15bb200640 1 -- start start 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15bb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 0x7f15bc14e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15bb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc0d0f30 0x7f15bc0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15bb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc15db30 0x7f15bc0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15bb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0cf660 con 0x7f15bc06f100 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15bb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc0cf380 con 0x7f15bc15db30 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15bb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15bc15df10 con 0x7f15bc0d0f30 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 0x7f15bc14e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 0x7f15bc14e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36048/0 (socket says 172.21.15.5:36048) 2024-04-25T10:03:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.335+0000 7f15b9e00640 1 -- 172.21.15.5:0/1411945421 learned_addr learned my addr 172.21.15.5:0/1411945421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15ba800640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc15db30 0x7f15bc0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9400640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc0d0f30 0x7f15bc0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9e00640 1 -- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc0d0f30 msgr2=0x7f15bc0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9e00640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc0d0f30 0x7f15bc0b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9e00640 1 -- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc15db30 msgr2=0x7f15bc0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9e00640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc15db30 0x7f15bc0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9e00640 1 -- 172.21.15.5:0/1411945421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0b04a0 con 0x7f15bc06f100 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15ba800640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc15db30 0x7f15bc0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b9e00640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 0x7f15bc14e010 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e870 tx=0x7f15ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac017070 con 0x7f15bc06f100 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac00c8e0 con 0x7f15bc06f100 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00ccd0 con 0x7f15bc06f100 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.336+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc0b0780 con 0x7f15bc06f100 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.337+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15bc0b0d40 con 0x7f15bc06f100 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.338+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15bc0d0970 con 0x7f15bc06f100 2024-04-25T10:03:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.339+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15ac01a070 con 0x7f15bc06f100 2024-04-25T10:03:41.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.340+0000 7f15b2000640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b4d0 0x7f159c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.340+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f15ac08a610 con 0x7f15bc06f100 2024-04-25T10:03:41.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.341+0000 7f15b9400640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b4d0 0x7f159c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.342+0000 7f15b9400640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b4d0 0x7f159c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15a8036000 tx=0x7f15a8009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.344+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15bc0d0970 con 0x7f15bc06f100 2024-04-25T10:03:41.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.531+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15bc068f60 con 0x7f159c06b4d0 2024-04-25T10:03:41.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.549+0000 7f15b2000640 1 -- 172.21.15.5:0/1411945421 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242409 (secure 0 0 0) 0x7f15bc068f60 con 0x7f159c06b4d0 2024-04-25T10:03:41.560 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:03:41.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b4d0 msgr2=0x7f159c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b4d0 0x7f159c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f15a8036000 tx=0x7f15a8009000 comp rx=0 tx=0).stop 2024-04-25T10:03:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 msgr2=0x7f15bc14e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 0x7f15bc14e010 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00e870 tx=0x7f15ac00ed30 comp rx=0 tx=0).stop 2024-04-25T10:03:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 shutdown_connections 2024-04-25T10:03:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f159c06b4d0 0x7f159c06d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15bc15db30 0x7f15bc0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15bc0d0f30 0x7f15bc0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 --2- 172.21.15.5:0/1411945421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15bc06f100 0x7f15bc14e010 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.554+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 >> 172.21.15.5:0/1411945421 conn(0x7f15bc0999f0 msgr2=0x7f15bc0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.555+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 shutdown_connections 2024-04-25T10:03:41.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.555+0000 7f15bb200640 1 -- 172.21.15.5:0/1411945421 wait complete. 2024-04-25T10:03:41.579 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T10:03:40.648398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237272,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737448,"statfs":{"total":773094113280,"available":772851146752,"internally_reserved":0,"allocated":4562944,"data_stored":2582388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.590299"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207506+0000","last_change":"2024-04-25T10:03:37.134249+0000","last_active":"2024-04-25T10:03:38.207506+0000","last_peered":"2024-04-25T10:03:38.207506+0000","last_clean":"2024-04-25T10:03:38.207506+0000","last_became_active":"2024-04-25T10:03:37.133378+0000","last_became_peered":"2024-04-25T10:03:37.133378+0000","last_unstale":"2024-04-25T10:03:38.207506+0000","last_undegraded":"2024-04-25T10:03:38.207506+0000","last_fullsized":"2024-04-25T10:03:38.207506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.585105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206496+0000","last_change":"2024-04-25T10:03:37.133516+0000","last_active":"2024-04-25T10:03:38.206496+0000","last_peered":"2024-04-25T10:03:38.206496+0000","last_clean":"2024-04-25T10:03:38.206496+0000","last_became_active":"2024-04-25T10:03:37.132370+0000","last_became_peered":"2024-04-25T10:03:37.132370+0000","last_unstale":"2024-04-25T10:03:38.206496+0000","last_undegraded":"2024-04-25T10:03:38.206496+0000","last_fullsized":"2024-04-25T10:03:38.206496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:33.505774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206619+0000","last_change":"2024-04-25T10:03:37.131079+0000","last_active":"2024-04-25T10:03:38.206619+0000","last_peered":"2024-04-25T10:03:38.206619+0000","last_clean":"2024-04-25T10:03:38.206619+0000","last_became_active":"2024-04-25T10:03:37.130786+0000","last_became_peered":"2024-04-25T10:03:37.130786+0000","last_unstale":"2024-04-25T10:03:38.206619+0000","last_undegraded":"2024-04-25T10:03:38.206619+0000","last_fullsized":"2024-04-25T10:03:38.206619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:11.177455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205696+0000","last_change":"2024-04-25T10:03:37.133051+0000","last_active":"2024-04-25T10:03:38.205696+0000","last_peered":"2024-04-25T10:03:38.205696+0000","last_clean":"2024-04-25T10:03:38.205696+0000","last_became_active":"2024-04-25T10:03:37.131855+0000","last_became_peered":"2024-04-25T10:03:37.131855+0000","last_unstale":"2024-04-25T10:03:38.205696+0000","last_undegraded":"2024-04-25T10:03:38.205696+0000","last_fullsized":"2024-04-25T10:03:38.205696+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:52.525576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206099+0000","last_change":"2024-04-25T10:03:37.130876+0000","last_active":"2024-04-25T10:03:38.206099+0000","last_peered":"2024-04-25T10:03:38.206099+0000","last_clean":"2024-04-25T10:03:38.206099+0000","last_became_active":"2024-04-25T10:03:37.130587+0000","last_became_peered":"2024-04-25T10:03:37.130587+0000","last_unstale":"2024-04-25T10:03:38.206099+0000","last_undegraded":"2024-04-25T10:03:38.206099+0000","last_fullsized":"2024-04-25T10:03:38.206099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.764236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206607+0000","last_change":"2024-04-25T10:03:37.130321+0000","last_active":"2024-04-25T10:03:38.206607+0000","last_peered":"2024-04-25T10:03:38.206607+0000","last_clean":"2024-04-25T10:03:38.206607+0000","last_became_active":"2024-04-25T10:03:37.129117+0000","last_became_peered":"2024-04-25T10:03:37.129117+0000","last_unstale":"2024-04-25T10:03:38.206607+0000","last_undegraded":"2024-04-25T10:03:38.206607+0000","last_fullsized":"2024-04-25T10:03:38.206607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:39.586705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206229+0000","last_change":"2024-04-25T10:03:37.132694+0000","last_active":"2024-04-25T10:03:38.206229+0000","last_peered":"2024-04-25T10:03:38.206229+0000","last_clean":"2024-04-25T10:03:38.206229+0000","last_became_active":"2024-04-25T10:03:37.131287+0000","last_became_peered":"2024-04-25T10:03:37.131287+0000","last_unstale":"2024-04-25T10:03:38.206229+0000","last_undegraded":"2024-04-25T10:03:38.206229+0000","last_fullsized":"2024-04-25T10:03:38.206229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:00.454471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062709+0000","last_change":"2024-04-25T10:03:35.062709+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062709+0000","last_undegraded":"2024-04-25T10:03:35.062709+0000","last_fullsized":"2024-04-25T10:03:35.062709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075811+0000","last_change":"2024-04-25T10:03:35.075811+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075811+0000","last_undegraded":"2024-04-25T10:03:35.075811+0000","last_fullsized":"2024-04-25T10:03:35.075811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206789+0000","last_change":"2024-04-25T10:03:36.105651+0000","last_active":"2024-04-25T10:03:38.206789+0000","last_peered":"2024-04-25T10:03:38.206789+0000","last_clean":"2024-04-25T10:03:38.206789+0000","last_became_active":"2024-04-25T10:03:36.105203+0000","last_became_peered":"2024-04-25T10:03:36.105203+0000","last_unstale":"2024-04-25T10:03:38.206789+0000","last_undegraded":"2024-04-25T10:03:38.206789+0000","last_fullsized":"2024-04-25T10:03:38.206789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:09.488962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062261+0000","last_change":"2024-04-25T10:03:35.062261+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062261+0000","last_undegraded":"2024-04-25T10:03:35.062261+0000","last_fullsized":"2024-04-25T10:03:35.062261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206661+0000","last_change":"2024-04-25T10:03:37.115687+0000","last_active":"2024-04-25T10:03:38.206661+0000","last_peered":"2024-04-25T10:03:38.206661+0000","last_clean":"2024-04-25T10:03:38.206661+0000","last_became_active":"2024-04-25T10:03:37.115368+0000","last_became_peered":"2024-04-25T10:03:37.115368+0000","last_unstale":"2024-04-25T10:03:38.206661+0000","last_undegraded":"2024-04-25T10:03:38.206661+0000","last_fullsized":"2024-04-25T10:03:38.206661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:56.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062315+0000","last_change":"2024-04-25T10:03:35.062315+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062315+0000","last_undegraded":"2024-04-25T10:03:35.062315+0000","last_fullsized":"2024-04-25T10:03:35.062315+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065569+0000","last_change":"2024-04-25T10:03:35.065569+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065569+0000","last_undegraded":"2024-04-25T10:03:35.065569+0000","last_fullsized":"2024-04-25T10:03:35.065569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205709+0000","last_change":"2024-04-25T10:03:36.095557+0000","last_active":"2024-04-25T10:03:38.205709+0000","last_peered":"2024-04-25T10:03:38.205709+0000","last_clean":"2024-04-25T10:03:38.205709+0000","last_became_active":"2024-04-25T10:03:36.095265+0000","last_became_peered":"2024-04-25T10:03:36.095265+0000","last_unstale":"2024-04-25T10:03:38.205709+0000","last_undegraded":"2024-04-25T10:03:38.205709+0000","last_fullsized":"2024-04-25T10:03:38.205709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.385227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065358+0000","last_change":"2024-04-25T10:03:35.065358+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065358+0000","last_undegraded":"2024-04-25T10:03:35.065358+0000","last_fullsized":"2024-04-25T10:03:35.065358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207045+0000","last_change":"2024-04-25T10:03:37.130899+0000","last_active":"2024-04-25T10:03:38.207045+0000","last_peered":"2024-04-25T10:03:38.207045+0000","last_clean":"2024-04-25T10:03:38.207045+0000","last_became_active":"2024-04-25T10:03:37.130432+0000","last_became_peered":"2024-04-25T10:03:37.130432+0000","last_unstale":"2024-04-25T10:03:38.207045+0000","last_undegraded":"2024-04-25T10:03:38.207045+0000","last_fullsized":"2024-04-25T10:03:38.207045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.079554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207233+0000","last_change":"2024-04-25T10:03:36.098558+0000","last_active":"2024-04-25T10:03:38.207233+0000","last_peered":"2024-04-25T10:03:38.207233+0000","last_clean":"2024-04-25T10:03:38.207233+0000","last_became_active":"2024-04-25T10:03:36.098223+0000","last_became_peered":"2024-04-25T10:03:36.098223+0000","last_unstale":"2024-04-25T10:03:38.207233+0000","last_undegraded":"2024-04-25T10:03:38.207233+0000","last_fullsized":"2024-04-25T10:03:38.207233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:51.024456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074550+0000","last_change":"2024-04-25T10:03:35.074550+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074550+0000","last_undegraded":"2024-04-25T10:03:35.074550+0000","last_fullsized":"2024-04-25T10:03:35.074550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207030+0000","last_change":"2024-04-25T10:03:37.138930+0000","last_active":"2024-04-25T10:03:38.207030+0000","last_peered":"2024-04-25T10:03:38.207030+0000","last_clean":"2024-04-25T10:03:38.207030+0000","last_became_active":"2024-04-25T10:03:37.138643+0000","last_became_peered":"2024-04-25T10:03:37.138643+0000","last_unstale":"2024-04-25T10:03:38.207030+0000","last_undegraded":"2024-04-25T10:03:38.207030+0000","last_fullsized":"2024-04-25T10:03:38.207030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.100516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.069611+0000","last_change":"2024-04-25T10:03:35.069611+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.069611+0000","last_undegraded":"2024-04-25T10:03:35.069611+0000","last_fullsized":"2024-04-25T10:03:35.069611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207294+0000","last_change":"2024-04-25T10:03:36.098166+0000","last_active":"2024-04-25T10:03:38.207294+0000","last_peered":"2024-04-25T10:03:38.207294+0000","last_clean":"2024-04-25T10:03:38.207294+0000","last_became_active":"2024-04-25T10:03:36.097774+0000","last_became_peered":"2024-04-25T10:03:36.097774+0000","last_unstale":"2024-04-25T10:03:38.207294+0000","last_undegraded":"2024-04-25T10:03:38.207294+0000","last_fullsized":"2024-04-25T10:03:38.207294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:32.511676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207141+0000","last_change":"2024-04-25T10:03:37.119860+0000","last_active":"2024-04-25T10:03:38.207141+0000","last_peered":"2024-04-25T10:03:38.207141+0000","last_clean":"2024-04-25T10:03:38.207141+0000","last_became_active":"2024-04-25T10:03:37.118103+0000","last_became_peered":"2024-04-25T10:03:37.118103+0000","last_unstale":"2024-04-25T10:03:38.207141+0000","last_undegraded":"2024-04-25T10:03:38.207141+0000","last_fullsized":"2024-04-25T10:03:38.207141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:06.878310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207283+0000","last_change":"2024-04-25T10:03:36.104310+0000","last_active":"2024-04-25T10:03:38.207283+0000","last_peered":"2024-04-25T10:03:38.207283+0000","last_clean":"2024-04-25T10:03:38.207283+0000","last_became_active":"2024-04-25T10:03:36.103982+0000","last_became_peered":"2024-04-25T10:03:36.103982+0000","last_unstale":"2024-04-25T10:03:38.207283+0000","last_undegraded":"2024-04-25T10:03:38.207283+0000","last_fullsized":"2024-04-25T10:03:38.207283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:33.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.076755+0000","last_change":"2024-04-25T10:03:35.076755+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.076755+0000","last_undegraded":"2024-04-25T10:03:35.076755+0000","last_fullsized":"2024-04-25T10:03:35.076755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.072257+0000","last_change":"2024-04-25T10:03:35.072257+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.072257+0000","last_undegraded":"2024-04-25T10:03:35.072257+0000","last_fullsized":"2024-04-25T10:03:35.072257+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062252+0000","last_change":"2024-04-25T10:03:35.062252+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062252+0000","last_undegraded":"2024-04-25T10:03:35.062252+0000","last_fullsized":"2024-04-25T10:03:35.062252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065348+0000","last_change":"2024-04-25T10:03:35.065348+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065348+0000","last_undegraded":"2024-04-25T10:03:35.065348+0000","last_fullsized":"2024-04-25T10:03:35.065348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.073828+0000","last_change":"2024-04-25T10:03:35.073828+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.073828+0000","last_undegraded":"2024-04-25T10:03:35.073828+0000","last_fullsized":"2024-04-25T10:03:35.073828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.067731+0000","last_change":"2024-04-25T10:03:35.067731+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.067731+0000","last_undegraded":"2024-04-25T10:03:35.067731+0000","last_fullsized":"2024-04-25T10:03:35.067731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074734+0000","last_change":"2024-04-25T10:03:35.074734+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074734+0000","last_undegraded":"2024-04-25T10:03:35.074734+0000","last_fullsized":"2024-04-25T10:03:35.074734+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074177+0000","last_change":"2024-04-25T10:03:35.074177+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074177+0000","last_undegraded":"2024-04-25T10:03:35.074177+0000","last_fullsized":"2024-04-25T10:03:35.074177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205441+0000","last_change":"2024-04-25T10:03:37.139337+0000","last_active":"2024-04-25T10:03:38.205441+0000","last_peered":"2024-04-25T10:03:38.205441+0000","last_clean":"2024-04-25T10:03:38.205441+0000","last_became_active":"2024-04-25T10:03:37.137532+0000","last_became_peered":"2024-04-25T10:03:37.137532+0000","last_unstale":"2024-04-25T10:03:38.205441+0000","last_undegraded":"2024-04-25T10:03:38.205441+0000","last_fullsized":"2024-04-25T10:03:38.205441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:56.223185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075811+0000","last_change":"2024-04-25T10:03:35.075811+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075811+0000","last_undegraded":"2024-04-25T10:03:35.075811+0000","last_fullsized":"2024-04-25T10:03:35.075811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065355+0000","last_change":"2024-04-25T10:03:35.065355+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065355+0000","last_undegraded":"2024-04-25T10:03:35.065355+0000","last_fullsized":"2024-04-25T10:03:35.065355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207050+0000","last_change":"2024-04-25T10:03:37.123190+0000","last_active":"2024-04-25T10:03:38.207050+0000","last_peered":"2024-04-25T10:03:38.207050+0000","last_clean":"2024-04-25T10:03:38.207050+0000","last_became_active":"2024-04-25T10:03:37.122906+0000","last_became_peered":"2024-04-25T10:03:37.122906+0000","last_unstale":"2024-04-25T10:03:38.207050+0000","last_undegraded":"2024-04-25T10:03:38.207050+0000","last_fullsized":"2024-04-25T10:03:38.207050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.154712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075675+0000","last_change":"2024-04-25T10:03:35.075675+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075675+0000","last_undegraded":"2024-04-25T10:03:35.075675+0000","last_fullsized":"2024-04-25T10:03:35.075675+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.073206+0000","last_change":"2024-04-25T10:03:35.073206+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.073206+0000","last_undegraded":"2024-04-25T10:03:35.073206+0000","last_fullsized":"2024-04-25T10:03:35.073206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.064189+0000","last_change":"2024-04-25T10:03:35.064189+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.064189+0000","last_undegraded":"2024-04-25T10:03:35.064189+0000","last_fullsized":"2024-04-25T10:03:35.064189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.064903+0000","last_change":"2024-04-25T10:03:35.064903+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.064903+0000","last_undegraded":"2024-04-25T10:03:35.064903+0000","last_fullsized":"2024-04-25T10:03:35.064903+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207633+0000","last_change":"2024-04-25T10:03:36.102517+0000","last_active":"2024-04-25T10:03:38.207633+0000","last_peered":"2024-04-25T10:03:38.207633+0000","last_clean":"2024-04-25T10:03:38.207633+0000","last_became_active":"2024-04-25T10:03:36.102006+0000","last_became_peered":"2024-04-25T10:03:36.102006+0000","last_unstale":"2024-04-25T10:03:38.207633+0000","last_undegraded":"2024-04-25T10:03:38.207633+0000","last_fullsized":"2024-04-25T10:03:38.207633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:22.682970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.069045+0000","last_change":"2024-04-25T10:03:35.069045+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.069045+0000","last_undegraded":"2024-04-25T10:03:35.069045+0000","last_fullsized":"2024-04-25T10:03:35.069045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.071523+0000","last_change":"2024-04-25T10:03:35.071523+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.071523+0000","last_undegraded":"2024-04-25T10:03:35.071523+0000","last_fullsized":"2024-04-25T10:03:35.071523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206932+0000","last_change":"2024-04-25T10:03:36.090034+0000","last_active":"2024-04-25T10:03:38.206932+0000","last_peered":"2024-04-25T10:03:38.206932+0000","last_clean":"2024-04-25T10:03:38.206932+0000","last_became_active":"2024-04-25T10:03:36.089136+0000","last_became_peered":"2024-04-25T10:03:36.089136+0000","last_unstale":"2024-04-25T10:03:38.206932+0000","last_undegraded":"2024-04-25T10:03:38.206932+0000","last_fullsized":"2024-04-25T10:03:38.206932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.068912+0000","last_change":"2024-04-25T10:03:35.068912+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.068912+0000","last_undegraded":"2024-04-25T10:03:35.068912+0000","last_fullsized":"2024-04-25T10:03:35.068912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:35.066540+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:03:35.066540+0000","last_peered":"2024-04-25T10:03:35.066540+0000","last_clean":"2024-04-25T10:03:35.066540+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:03:35.066540+0000","last_undegraded":"2024-04-25T10:03:35.066540+0000","last_fullsized":"2024-04-25T10:03:35.066540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":764344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":764344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":196238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:41.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:41.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.828+0000 7f16faa00640 1 Processor -- start 2024-04-25T10:03:41.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.828+0000 7f16faa00640 1 -- start start 2024-04-25T10:03:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.829+0000 7f16faa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f16f40b4bb0 0x7f16f40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.829+0000 7f16faa00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f40cf410 con 0x7f16f40d07d0 2024-04-25T10:03:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.829+0000 7f16faa00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f40cf6f0 con 0x7f16f4161200 2024-04-25T10:03:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.829+0000 7f16faa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f40b28b0 con 0x7f16f40b4bb0 2024-04-25T10:03:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.830+0000 7f16f8c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f16f40d07d0 0x7f16f40b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40522/0 (socket says 172.21.15.5:40522) 2024-04-25T10:03:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.830+0000 7f16f8c00640 1 -- 172.21.15.5:0/1843332386 learned_addr learned my addr 172.21.15.5:0/1843332386 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:41.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.830+0000 7f16fa000640 1 --2- 172.21.15.5:0/1843332386 >> v2:172.21.15.80:3300/0 conn(0x7f16f40b4bb0 0x7f16f40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.830+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610797320 0 0) 0x7f16f40cf410 con 0x7f16f40d07d0 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.830+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16dc003520 con 0x7f16f40d07d0 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1175453133 0 0) 0x7f16dc003520 con 0x7f16f40d07d0 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 >> v2:172.21.15.80:3300/0 conn(0x7f16f40b4bb0 msgr2=0x7f16f40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 --2- 172.21.15.5:0/1843332386 >> v2:172.21.15.80:3300/0 conn(0x7f16f40b4bb0 0x7f16f40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 >> v1:172.21.15.80:6789/0 conn(0x7f16f4161200 legacy=0x7f16f40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40d3fd0 con 0x7f16f40d07d0 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f16e0001670 con 0x7f16f40d07d0 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 >> v1:172.21.15.5:6789/0 conn(0x7f16f40d07d0 legacy=0x7f16f40b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16dc0039b0 0x7f16dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40b28b0 con 0x7f16dc0051c0 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc00c060 con 0x7f16dc004800 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16dc00c260 con 0x7f16dc0039b0 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16fa000640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.831+0000 7f16f9600640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16dc0039b0 0x7f16dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.832+0000 7f16f8c00640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.832+0000 7f16fa000640 1 -- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16dc0039b0 msgr2=0x7f16dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.832+0000 7f16fa000640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16dc0039b0 0x7f16dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.832+0000 7f16fa000640 1 -- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16dc004800 msgr2=0x7f16dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.832+0000 7f16fa000640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16dc004800 0x7f16dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.832+0000 7f16fa000640 1 -- 172.21.15.5:0/1843332386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc003520 con 0x7f16dc0051c0 2024-04-25T10:03:41.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.833+0000 7f16fa000640 1 -- 172.21.15.5:0/1843332386 reap_dead start 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.833+0000 7f16fa000640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009fc0 tx=0x7f16dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a28c1dccb34960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.833+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec035020 con 0x7f16dc0051c0 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec003bd0 con 0x7f16dc0051c0 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16f3e00640 1 -- 172.21.15.5:0/1843332386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec009c10 con 0x7f16dc0051c0 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 -- 172.21.15.5:0/1843332386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 msgr2=0x7f16dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009fc0 tx=0x7f16dc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 -- 172.21.15.5:0/1843332386 shutdown_connections 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 --2- 172.21.15.5:0/1843332386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16dc00b9c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 -- 172.21.15.5:0/1843332386 >> 172.21.15.5:0/1843332386 conn(0x7f16f408af80 msgr2=0x7f16f40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 -- 172.21.15.5:0/1843332386 shutdown_connections 2024-04-25T10:03:41.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.834+0000 7f16faa00640 1 -- 172.21.15.5:0/1843332386 wait complete. 2024-04-25T10:03:41.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.836+0000 7f16faa00640 1 Processor -- start 2024-04-25T10:03:41.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.836+0000 7f16faa00640 1 -- start start 2024-04-25T10:03:41.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.837+0000 7f16faa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16f4153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.837+0000 7f16faa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16f40b18a0 0x7f16f40b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.837+0000 7f16faa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16f41f6e30 0x7f16f41f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.837+0000 7f16faa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40d2500 con 0x7f16dc0051c0 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.837+0000 7f16faa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40d2380 con 0x7f16f41f6e30 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.837+0000 7f16faa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f40d2680 con 0x7f16f40b18a0 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16f4153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16f4153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36068/0 (socket says 172.21.15.5:36068) 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 -- 172.21.15.5:0/2366504486 learned_addr learned my addr 172.21.15.5:0/2366504486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 -- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16f40b18a0 msgr2=0x7f16f40b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16f40b18a0 0x7f16f40b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 -- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16f41f6e30 msgr2=0x7f16f41f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16f41f6e30 0x7f16f41f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 -- 172.21.15.5:0/2366504486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40b62d0 con 0x7f16dc0051c0 2024-04-25T10:03:41.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f9600640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16f4153de0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e4b0 tx=0x7f16e800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.838+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e80052e0 con 0x7f16dc0051c0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.839+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e8018070 con 0x7f16dc0051c0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.839+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f4154790 con 0x7f16dc0051c0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.839+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8003830 con 0x7f16dc0051c0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.839+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f4154d30 con 0x7f16dc0051c0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.840+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4068f60 con 0x7f16dc0051c0 2024-04-25T10:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.841+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16e80039d0 con 0x7f16dc0051c0 2024-04-25T10:03:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.842+0000 7f16f1600640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16d406b500 0x7f16d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.842+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16e8089f40 con 0x7f16dc0051c0 2024-04-25T10:03:41.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.842+0000 7f16f8c00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16d406b500 0x7f16d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.844+0000 7f16f8c00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16d406b500 0x7f16d406d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16e0001200 tx=0x7f16e0001110 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:41.846+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16e8058b90 con 0x7f16dc0051c0 2024-04-25T10:03:42.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.042+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16f40b2260 con 0x7f16d406b500 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.060+0000 7f16f1600640 1 -- 172.21.15.5:0/2366504486 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242409 (secure 0 0 0) 0x7f16e000e7b0 con 0x7f16d406b500 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:42.071 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:03:42.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16d406b500 msgr2=0x7f16d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16d406b500 0x7f16d406d9c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f16e0001200 tx=0x7f16e0001110 comp rx=0 tx=0).stop 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 msgr2=0x7f16f4153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16f4153de0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e4b0 tx=0x7f16e800e970 comp rx=0 tx=0).stop 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 shutdown_connections 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16d406b500 0x7f16d406d9c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16f41f6e30 0x7f16f41f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16f40b18a0 0x7f16f40b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 --2- 172.21.15.5:0/2366504486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16dc0051c0 0x7f16f4153de0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 >> 172.21.15.5:0/2366504486 conn(0x7f16f408af80 msgr2=0x7f16f4160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 shutdown_connections 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.065+0000 7f16faa00640 1 -- 172.21.15.5:0/2366504486 wait complete. 2024-04-25T10:03:42.094 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T10:03:40.648398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237272,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737448,"statfs":{"total":773094113280,"available":772851146752,"internally_reserved":0,"allocated":4562944,"data_stored":2582388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.590299"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207506+0000","last_change":"2024-04-25T10:03:37.134249+0000","last_active":"2024-04-25T10:03:38.207506+0000","last_peered":"2024-04-25T10:03:38.207506+0000","last_clean":"2024-04-25T10:03:38.207506+0000","last_became_active":"2024-04-25T10:03:37.133378+0000","last_became_peered":"2024-04-25T10:03:37.133378+0000","last_unstale":"2024-04-25T10:03:38.207506+0000","last_undegraded":"2024-04-25T10:03:38.207506+0000","last_fullsized":"2024-04-25T10:03:38.207506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.585105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206496+0000","last_change":"2024-04-25T10:03:37.133516+0000","last_active":"2024-04-25T10:03:38.206496+0000","last_peered":"2024-04-25T10:03:38.206496+0000","last_clean":"2024-04-25T10:03:38.206496+0000","last_became_active":"2024-04-25T10:03:37.132370+0000","last_became_peered":"2024-04-25T10:03:37.132370+0000","last_unstale":"2024-04-25T10:03:38.206496+0000","last_undegraded":"2024-04-25T10:03:38.206496+0000","last_fullsized":"2024-04-25T10:03:38.206496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:33.505774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206619+0000","last_change":"2024-04-25T10:03:37.131079+0000","last_active":"2024-04-25T10:03:38.206619+0000","last_peered":"2024-04-25T10:03:38.206619+0000","last_clean":"2024-04-25T10:03:38.206619+0000","last_became_active":"2024-04-25T10:03:37.130786+0000","last_became_peered":"2024-04-25T10:03:37.130786+0000","last_unstale":"2024-04-25T10:03:38.206619+0000","last_undegraded":"2024-04-25T10:03:38.206619+0000","last_fullsized":"2024-04-25T10:03:38.206619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:11.177455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205696+0000","last_change":"2024-04-25T10:03:37.133051+0000","last_active":"2024-04-25T10:03:38.205696+0000","last_peered":"2024-04-25T10:03:38.205696+0000","last_clean":"2024-04-25T10:03:38.205696+0000","last_became_active":"2024-04-25T10:03:37.131855+0000","last_became_peered":"2024-04-25T10:03:37.131855+0000","last_unstale":"2024-04-25T10:03:38.205696+0000","last_undegraded":"2024-04-25T10:03:38.205696+0000","last_fullsized":"2024-04-25T10:03:38.205696+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:52.525576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206099+0000","last_change":"2024-04-25T10:03:37.130876+0000","last_active":"2024-04-25T10:03:38.206099+0000","last_peered":"2024-04-25T10:03:38.206099+0000","last_clean":"2024-04-25T10:03:38.206099+0000","last_became_active":"2024-04-25T10:03:37.130587+0000","last_became_peered":"2024-04-25T10:03:37.130587+0000","last_unstale":"2024-04-25T10:03:38.206099+0000","last_undegraded":"2024-04-25T10:03:38.206099+0000","last_fullsized":"2024-04-25T10:03:38.206099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.764236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206607+0000","last_change":"2024-04-25T10:03:37.130321+0000","last_active":"2024-04-25T10:03:38.206607+0000","last_peered":"2024-04-25T10:03:38.206607+0000","last_clean":"2024-04-25T10:03:38.206607+0000","last_became_active":"2024-04-25T10:03:37.129117+0000","last_became_peered":"2024-04-25T10:03:37.129117+0000","last_unstale":"2024-04-25T10:03:38.206607+0000","last_undegraded":"2024-04-25T10:03:38.206607+0000","last_fullsized":"2024-04-25T10:03:38.206607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:39.586705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206229+0000","last_change":"2024-04-25T10:03:37.132694+0000","last_active":"2024-04-25T10:03:38.206229+0000","last_peered":"2024-04-25T10:03:38.206229+0000","last_clean":"2024-04-25T10:03:38.206229+0000","last_became_active":"2024-04-25T10:03:37.131287+0000","last_became_peered":"2024-04-25T10:03:37.131287+0000","last_unstale":"2024-04-25T10:03:38.206229+0000","last_undegraded":"2024-04-25T10:03:38.206229+0000","last_fullsized":"2024-04-25T10:03:38.206229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:00.454471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062709+0000","last_change":"2024-04-25T10:03:35.062709+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062709+0000","last_undegraded":"2024-04-25T10:03:35.062709+0000","last_fullsized":"2024-04-25T10:03:35.062709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075811+0000","last_change":"2024-04-25T10:03:35.075811+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075811+0000","last_undegraded":"2024-04-25T10:03:35.075811+0000","last_fullsized":"2024-04-25T10:03:35.075811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206789+0000","last_change":"2024-04-25T10:03:36.105651+0000","last_active":"2024-04-25T10:03:38.206789+0000","last_peered":"2024-04-25T10:03:38.206789+0000","last_clean":"2024-04-25T10:03:38.206789+0000","last_became_active":"2024-04-25T10:03:36.105203+0000","last_became_peered":"2024-04-25T10:03:36.105203+0000","last_unstale":"2024-04-25T10:03:38.206789+0000","last_undegraded":"2024-04-25T10:03:38.206789+0000","last_fullsized":"2024-04-25T10:03:38.206789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:09.488962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062261+0000","last_change":"2024-04-25T10:03:35.062261+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062261+0000","last_undegraded":"2024-04-25T10:03:35.062261+0000","last_fullsized":"2024-04-25T10:03:35.062261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206661+0000","last_change":"2024-04-25T10:03:37.115687+0000","last_active":"2024-04-25T10:03:38.206661+0000","last_peered":"2024-04-25T10:03:38.206661+0000","last_clean":"2024-04-25T10:03:38.206661+0000","last_became_active":"2024-04-25T10:03:37.115368+0000","last_became_peered":"2024-04-25T10:03:37.115368+0000","last_unstale":"2024-04-25T10:03:38.206661+0000","last_undegraded":"2024-04-25T10:03:38.206661+0000","last_fullsized":"2024-04-25T10:03:38.206661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:56.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062315+0000","last_change":"2024-04-25T10:03:35.062315+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062315+0000","last_undegraded":"2024-04-25T10:03:35.062315+0000","last_fullsized":"2024-04-25T10:03:35.062315+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065569+0000","last_change":"2024-04-25T10:03:35.065569+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065569+0000","last_undegraded":"2024-04-25T10:03:35.065569+0000","last_fullsized":"2024-04-25T10:03:35.065569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205709+0000","last_change":"2024-04-25T10:03:36.095557+0000","last_active":"2024-04-25T10:03:38.205709+0000","last_peered":"2024-04-25T10:03:38.205709+0000","last_clean":"2024-04-25T10:03:38.205709+0000","last_became_active":"2024-04-25T10:03:36.095265+0000","last_became_peered":"2024-04-25T10:03:36.095265+0000","last_unstale":"2024-04-25T10:03:38.205709+0000","last_undegraded":"2024-04-25T10:03:38.205709+0000","last_fullsized":"2024-04-25T10:03:38.205709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.385227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065358+0000","last_change":"2024-04-25T10:03:35.065358+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065358+0000","last_undegraded":"2024-04-25T10:03:35.065358+0000","last_fullsized":"2024-04-25T10:03:35.065358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207045+0000","last_change":"2024-04-25T10:03:37.130899+0000","last_active":"2024-04-25T10:03:38.207045+0000","last_peered":"2024-04-25T10:03:38.207045+0000","last_clean":"2024-04-25T10:03:38.207045+0000","last_became_active":"2024-04-25T10:03:37.130432+0000","last_became_peered":"2024-04-25T10:03:37.130432+0000","last_unstale":"2024-04-25T10:03:38.207045+0000","last_undegraded":"2024-04-25T10:03:38.207045+0000","last_fullsized":"2024-04-25T10:03:38.207045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.079554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207233+0000","last_change":"2024-04-25T10:03:36.098558+0000","last_active":"2024-04-25T10:03:38.207233+0000","last_peered":"2024-04-25T10:03:38.207233+0000","last_clean":"2024-04-25T10:03:38.207233+0000","last_became_active":"2024-04-25T10:03:36.098223+0000","last_became_peered":"2024-04-25T10:03:36.098223+0000","last_unstale":"2024-04-25T10:03:38.207233+0000","last_undegraded":"2024-04-25T10:03:38.207233+0000","last_fullsized":"2024-04-25T10:03:38.207233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:51.024456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074550+0000","last_change":"2024-04-25T10:03:35.074550+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074550+0000","last_undegraded":"2024-04-25T10:03:35.074550+0000","last_fullsized":"2024-04-25T10:03:35.074550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207030+0000","last_change":"2024-04-25T10:03:37.138930+0000","last_active":"2024-04-25T10:03:38.207030+0000","last_peered":"2024-04-25T10:03:38.207030+0000","last_clean":"2024-04-25T10:03:38.207030+0000","last_became_active":"2024-04-25T10:03:37.138643+0000","last_became_peered":"2024-04-25T10:03:37.138643+0000","last_unstale":"2024-04-25T10:03:38.207030+0000","last_undegraded":"2024-04-25T10:03:38.207030+0000","last_fullsized":"2024-04-25T10:03:38.207030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.100516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.069611+0000","last_change":"2024-04-25T10:03:35.069611+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.069611+0000","last_undegraded":"2024-04-25T10:03:35.069611+0000","last_fullsized":"2024-04-25T10:03:35.069611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207294+0000","last_change":"2024-04-25T10:03:36.098166+0000","last_active":"2024-04-25T10:03:38.207294+0000","last_peered":"2024-04-25T10:03:38.207294+0000","last_clean":"2024-04-25T10:03:38.207294+0000","last_became_active":"2024-04-25T10:03:36.097774+0000","last_became_peered":"2024-04-25T10:03:36.097774+0000","last_unstale":"2024-04-25T10:03:38.207294+0000","last_undegraded":"2024-04-25T10:03:38.207294+0000","last_fullsized":"2024-04-25T10:03:38.207294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:32.511676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207141+0000","last_change":"2024-04-25T10:03:37.119860+0000","last_active":"2024-04-25T10:03:38.207141+0000","last_peered":"2024-04-25T10:03:38.207141+0000","last_clean":"2024-04-25T10:03:38.207141+0000","last_became_active":"2024-04-25T10:03:37.118103+0000","last_became_peered":"2024-04-25T10:03:37.118103+0000","last_unstale":"2024-04-25T10:03:38.207141+0000","last_undegraded":"2024-04-25T10:03:38.207141+0000","last_fullsized":"2024-04-25T10:03:38.207141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:06.878310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207283+0000","last_change":"2024-04-25T10:03:36.104310+0000","last_active":"2024-04-25T10:03:38.207283+0000","last_peered":"2024-04-25T10:03:38.207283+0000","last_clean":"2024-04-25T10:03:38.207283+0000","last_became_active":"2024-04-25T10:03:36.103982+0000","last_became_peered":"2024-04-25T10:03:36.103982+0000","last_unstale":"2024-04-25T10:03:38.207283+0000","last_undegraded":"2024-04-25T10:03:38.207283+0000","last_fullsized":"2024-04-25T10:03:38.207283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:33.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.076755+0000","last_change":"2024-04-25T10:03:35.076755+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.076755+0000","last_undegraded":"2024-04-25T10:03:35.076755+0000","last_fullsized":"2024-04-25T10:03:35.076755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.072257+0000","last_change":"2024-04-25T10:03:35.072257+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.072257+0000","last_undegraded":"2024-04-25T10:03:35.072257+0000","last_fullsized":"2024-04-25T10:03:35.072257+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062252+0000","last_change":"2024-04-25T10:03:35.062252+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062252+0000","last_undegraded":"2024-04-25T10:03:35.062252+0000","last_fullsized":"2024-04-25T10:03:35.062252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065348+0000","last_change":"2024-04-25T10:03:35.065348+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065348+0000","last_undegraded":"2024-04-25T10:03:35.065348+0000","last_fullsized":"2024-04-25T10:03:35.065348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.073828+0000","last_change":"2024-04-25T10:03:35.073828+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.073828+0000","last_undegraded":"2024-04-25T10:03:35.073828+0000","last_fullsized":"2024-04-25T10:03:35.073828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.067731+0000","last_change":"2024-04-25T10:03:35.067731+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.067731+0000","last_undegraded":"2024-04-25T10:03:35.067731+0000","last_fullsized":"2024-04-25T10:03:35.067731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074734+0000","last_change":"2024-04-25T10:03:35.074734+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074734+0000","last_undegraded":"2024-04-25T10:03:35.074734+0000","last_fullsized":"2024-04-25T10:03:35.074734+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074177+0000","last_change":"2024-04-25T10:03:35.074177+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074177+0000","last_undegraded":"2024-04-25T10:03:35.074177+0000","last_fullsized":"2024-04-25T10:03:35.074177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205441+0000","last_change":"2024-04-25T10:03:37.139337+0000","last_active":"2024-04-25T10:03:38.205441+0000","last_peered":"2024-04-25T10:03:38.205441+0000","last_clean":"2024-04-25T10:03:38.205441+0000","last_became_active":"2024-04-25T10:03:37.137532+0000","last_became_peered":"2024-04-25T10:03:37.137532+0000","last_unstale":"2024-04-25T10:03:38.205441+0000","last_undegraded":"2024-04-25T10:03:38.205441+0000","last_fullsized":"2024-04-25T10:03:38.205441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:56.223185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075811+0000","last_change":"2024-04-25T10:03:35.075811+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075811+0000","last_undegraded":"2024-04-25T10:03:35.075811+0000","last_fullsized":"2024-04-25T10:03:35.075811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065355+0000","last_change":"2024-04-25T10:03:35.065355+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065355+0000","last_undegraded":"2024-04-25T10:03:35.065355+0000","last_fullsized":"2024-04-25T10:03:35.065355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207050+0000","last_change":"2024-04-25T10:03:37.123190+0000","last_active":"2024-04-25T10:03:38.207050+0000","last_peered":"2024-04-25T10:03:38.207050+0000","last_clean":"2024-04-25T10:03:38.207050+0000","last_became_active":"2024-04-25T10:03:37.122906+0000","last_became_peered":"2024-04-25T10:03:37.122906+0000","last_unstale":"2024-04-25T10:03:38.207050+0000","last_undegraded":"2024-04-25T10:03:38.207050+0000","last_fullsized":"2024-04-25T10:03:38.207050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.154712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075675+0000","last_change":"2024-04-25T10:03:35.075675+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075675+0000","last_undegraded":"2024-04-25T10:03:35.075675+0000","last_fullsized":"2024-04-25T10:03:35.075675+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.073206+0000","last_change":"2024-04-25T10:03:35.073206+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.073206+0000","last_undegraded":"2024-04-25T10:03:35.073206+0000","last_fullsized":"2024-04-25T10:03:35.073206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.064189+0000","last_change":"2024-04-25T10:03:35.064189+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.064189+0000","last_undegraded":"2024-04-25T10:03:35.064189+0000","last_fullsized":"2024-04-25T10:03:35.064189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.064903+0000","last_change":"2024-04-25T10:03:35.064903+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.064903+0000","last_undegraded":"2024-04-25T10:03:35.064903+0000","last_fullsized":"2024-04-25T10:03:35.064903+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207633+0000","last_change":"2024-04-25T10:03:36.102517+0000","last_active":"2024-04-25T10:03:38.207633+0000","last_peered":"2024-04-25T10:03:38.207633+0000","last_clean":"2024-04-25T10:03:38.207633+0000","last_became_active":"2024-04-25T10:03:36.102006+0000","last_became_peered":"2024-04-25T10:03:36.102006+0000","last_unstale":"2024-04-25T10:03:38.207633+0000","last_undegraded":"2024-04-25T10:03:38.207633+0000","last_fullsized":"2024-04-25T10:03:38.207633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:22.682970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.069045+0000","last_change":"2024-04-25T10:03:35.069045+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.069045+0000","last_undegraded":"2024-04-25T10:03:35.069045+0000","last_fullsized":"2024-04-25T10:03:35.069045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.071523+0000","last_change":"2024-04-25T10:03:35.071523+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.071523+0000","last_undegraded":"2024-04-25T10:03:35.071523+0000","last_fullsized":"2024-04-25T10:03:35.071523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206932+0000","last_change":"2024-04-25T10:03:36.090034+0000","last_active":"2024-04-25T10:03:38.206932+0000","last_peered":"2024-04-25T10:03:38.206932+0000","last_clean":"2024-04-25T10:03:38.206932+0000","last_became_active":"2024-04-25T10:03:36.089136+0000","last_became_peered":"2024-04-25T10:03:36.089136+0000","last_unstale":"2024-04-25T10:03:38.206932+0000","last_undegraded":"2024-04-25T10:03:38.206932+0000","last_fullsized":"2024-04-25T10:03:38.206932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.068912+0000","last_change":"2024-04-25T10:03:35.068912+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.068912+0000","last_undegraded":"2024-04-25T10:03:35.068912+0000","last_fullsized":"2024-04-25T10:03:35.068912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:35.066540+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:03:35.066540+0000","last_peered":"2024-04-25T10:03:35.066540+0000","last_clean":"2024-04-25T10:03:35.066540+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:03:35.066540+0000","last_undegraded":"2024-04-25T10:03:35.066540+0000","last_fullsized":"2024-04-25T10:03:35.066540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":764344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":764344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":196238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:42.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.353+0000 7f09c2000640 1 Processor -- start 2024-04-25T10:03:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.353+0000 7f09c2000640 1 -- start start 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.354+0000 7f09c2000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09bc161200 0x7f09bc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.354+0000 7f09c2000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc0cf410 con 0x7f09bc0d07d0 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.354+0000 7f09c2000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09bc0cf6f0 con 0x7f09bc0d2380 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.354+0000 7f09c2000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09bc0d27c0 con 0x7f09bc161200 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.355+0000 7f09c0c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09bc161200 0x7f09bc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.355+0000 7f09c0c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09bc161200 0x7f09bc0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36080/0 (socket says 172.21.15.5:36080) 2024-04-25T10:03:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.355+0000 7f09c0c00640 1 -- 172.21.15.5:0/3529981941 learned_addr learned my addr 172.21.15.5:0/3529981941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.355+0000 7f09c0c00640 1 -- 172.21.15.5:0/3529981941 >> v1:172.21.15.80:6789/0 conn(0x7f09bc0d2380 legacy=0x7f09bc0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.355+0000 7f09c0c00640 1 -- 172.21.15.5:0/3529981941 >> v1:172.21.15.5:6789/0 conn(0x7f09bc0d07d0 legacy=0x7f09bc0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.355+0000 7f09c0c00640 1 -- 172.21.15.5:0/3529981941 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc0d3fd0 con 0x7f09bc161200 2024-04-25T10:03:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.356+0000 7f09c0c00640 1 --2- 172.21.15.5:0/3529981941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc0b4bb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009870 tx=0x7f09ac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d340b6a9ae30f2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09bb400640 1 -- 172.21.15.5:0/3529981941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac037070 con 0x7f09bc161200 2024-04-25T10:03:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09bb400640 1 -- 172.21.15.5:0/3529981941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac032440 con 0x7f09bc161200 2024-04-25T10:03:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09bb400640 1 -- 172.21.15.5:0/3529981941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac032810 con 0x7f09bc161200 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09c2000640 1 -- 172.21.15.5:0/3529981941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 msgr2=0x7f09bc0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09c2000640 1 --2- 172.21.15.5:0/3529981941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc0b4bb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009870 tx=0x7f09ac02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09c2000640 1 -- 172.21.15.5:0/3529981941 shutdown_connections 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09c2000640 1 --2- 172.21.15.5:0/3529981941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc0b4bb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.357+0000 7f09c2000640 1 -- 172.21.15.5:0/3529981941 >> 172.21.15.5:0/3529981941 conn(0x7f09bc08af80 msgr2=0x7f09bc0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.358+0000 7f09c2000640 1 -- 172.21.15.5:0/3529981941 shutdown_connections 2024-04-25T10:03:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.358+0000 7f09c2000640 1 -- 172.21.15.5:0/3529981941 wait complete. 2024-04-25T10:03:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.359+0000 7f09c2000640 1 Processor -- start 2024-04-25T10:03:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.359+0000 7f09c2000640 1 -- start start 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.360+0000 7f09c2000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09bc0d07d0 0x7f09bc1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.360+0000 7f09c2000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09bc0d2380 0x7f09bc06d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.360+0000 7f09c2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.360+0000 7f09c2000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0b3690 con 0x7f09bc161200 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.360+0000 7f09c2000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0b28b0 con 0x7f09bc0d07d0 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.360+0000 7f09c2000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09bc0b3810 con 0x7f09bc0d2380 2024-04-25T10:03:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36090/0 (socket says 172.21.15.5:36090) 2024-04-25T10:03:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 -- 172.21.15.5:0/3923691425 learned_addr learned my addr 172.21.15.5:0/3923691425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c0c00640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09bc0d07d0 0x7f09bc1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09bbe00640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09bc0d2380 0x7f09bc06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 -- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09bc0d2380 msgr2=0x7f09bc06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09bc0d2380 0x7f09bc06d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 -- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09bc0d07d0 msgr2=0x7f09bc1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09bc0d07d0 0x7f09bc1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 -- 172.21.15.5:0/3923691425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc0cc630 con 0x7f09bc161200 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c0c00640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09bc0d07d0 0x7f09bc1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.361+0000 7f09c1600640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc06d590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c730 tx=0x7f09b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.362+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4003b70 con 0x7f09bc161200 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.362+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4003d10 con 0x7f09bc161200 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.362+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09bc0cc910 con 0x7f09bc161200 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.362+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b400aa90 con 0x7f09bc161200 2024-04-25T10:03:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.362+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09bc0cced0 con 0x7f09bc161200 2024-04-25T10:03:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.363+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09bc068f60 con 0x7f09bc161200 2024-04-25T10:03:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.364+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09b400ac30 con 0x7f09bc161200 2024-04-25T10:03:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.365+0000 7f09b8c00640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f099806b4d0 0x7f099806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.365+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09b408a310 con 0x7f09bc161200 2024-04-25T10:03:42.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.366+0000 7f09c0c00640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f099806b4d0 0x7f099806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.367+0000 7f09c0c00640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f099806b4d0 0x7f099806d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09ac034000 tx=0x7f09ac02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.369+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09b4058ff0 con 0x7f09bc161200 2024-04-25T10:03:42.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.569+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09bc06acf0 con 0x7f099806b4d0 2024-04-25T10:03:42.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.587+0000 7f09b8c00640 1 -- 172.21.15.5:0/3923691425 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242409 (secure 0 0 0) 0x7f09bc06acf0 con 0x7f099806b4d0 2024-04-25T10:03:42.595 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:42.597 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:03:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.593+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f099806b4d0 msgr2=0x7f099806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.593+0000 7f09c2000640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f099806b4d0 0x7f099806d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f09ac034000 tx=0x7f09ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.593+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 msgr2=0x7f09bc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.593+0000 7f09c2000640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc06d590 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f09b400c730 tx=0x7f09b400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 shutdown_connections 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc161200 0x7f09bc06d590 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09bc0d2380 0x7f09bc06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f099806b4d0 0x7f099806d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 --2- 172.21.15.5:0/3923691425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09bc0d07d0 0x7f09bc1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 >> 172.21.15.5:0/3923691425 conn(0x7f09bc08af80 msgr2=0x7f09bc0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 shutdown_connections 2024-04-25T10:03:42.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.594+0000 7f09c2000640 1 -- 172.21.15.5:0/3923691425 wait complete. 2024-04-25T10:03:42.618 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2024-04-25T10:03:40.648398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592014,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":171,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":92,"acting":92,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237272,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737448,"statfs":{"total":773094113280,"available":772851146752,"internally_reserved":0,"allocated":4562944,"data_stored":2582388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.590299"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207506+0000","last_change":"2024-04-25T10:03:37.134249+0000","last_active":"2024-04-25T10:03:38.207506+0000","last_peered":"2024-04-25T10:03:38.207506+0000","last_clean":"2024-04-25T10:03:38.207506+0000","last_became_active":"2024-04-25T10:03:37.133378+0000","last_became_peered":"2024-04-25T10:03:37.133378+0000","last_unstale":"2024-04-25T10:03:38.207506+0000","last_undegraded":"2024-04-25T10:03:38.207506+0000","last_fullsized":"2024-04-25T10:03:38.207506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.585105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206496+0000","last_change":"2024-04-25T10:03:37.133516+0000","last_active":"2024-04-25T10:03:38.206496+0000","last_peered":"2024-04-25T10:03:38.206496+0000","last_clean":"2024-04-25T10:03:38.206496+0000","last_became_active":"2024-04-25T10:03:37.132370+0000","last_became_peered":"2024-04-25T10:03:37.132370+0000","last_unstale":"2024-04-25T10:03:38.206496+0000","last_undegraded":"2024-04-25T10:03:38.206496+0000","last_fullsized":"2024-04-25T10:03:38.206496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:33.505774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206619+0000","last_change":"2024-04-25T10:03:37.131079+0000","last_active":"2024-04-25T10:03:38.206619+0000","last_peered":"2024-04-25T10:03:38.206619+0000","last_clean":"2024-04-25T10:03:38.206619+0000","last_became_active":"2024-04-25T10:03:37.130786+0000","last_became_peered":"2024-04-25T10:03:37.130786+0000","last_unstale":"2024-04-25T10:03:38.206619+0000","last_undegraded":"2024-04-25T10:03:38.206619+0000","last_fullsized":"2024-04-25T10:03:38.206619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:11.177455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205696+0000","last_change":"2024-04-25T10:03:37.133051+0000","last_active":"2024-04-25T10:03:38.205696+0000","last_peered":"2024-04-25T10:03:38.205696+0000","last_clean":"2024-04-25T10:03:38.205696+0000","last_became_active":"2024-04-25T10:03:37.131855+0000","last_became_peered":"2024-04-25T10:03:37.131855+0000","last_unstale":"2024-04-25T10:03:38.205696+0000","last_undegraded":"2024-04-25T10:03:38.205696+0000","last_fullsized":"2024-04-25T10:03:38.205696+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:52.525576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206099+0000","last_change":"2024-04-25T10:03:37.130876+0000","last_active":"2024-04-25T10:03:38.206099+0000","last_peered":"2024-04-25T10:03:38.206099+0000","last_clean":"2024-04-25T10:03:38.206099+0000","last_became_active":"2024-04-25T10:03:37.130587+0000","last_became_peered":"2024-04-25T10:03:37.130587+0000","last_unstale":"2024-04-25T10:03:38.206099+0000","last_undegraded":"2024-04-25T10:03:38.206099+0000","last_fullsized":"2024-04-25T10:03:38.206099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.764236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206607+0000","last_change":"2024-04-25T10:03:37.130321+0000","last_active":"2024-04-25T10:03:38.206607+0000","last_peered":"2024-04-25T10:03:38.206607+0000","last_clean":"2024-04-25T10:03:38.206607+0000","last_became_active":"2024-04-25T10:03:37.129117+0000","last_became_peered":"2024-04-25T10:03:37.129117+0000","last_unstale":"2024-04-25T10:03:38.206607+0000","last_undegraded":"2024-04-25T10:03:38.206607+0000","last_fullsized":"2024-04-25T10:03:38.206607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:39.586705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206229+0000","last_change":"2024-04-25T10:03:37.132694+0000","last_active":"2024-04-25T10:03:38.206229+0000","last_peered":"2024-04-25T10:03:38.206229+0000","last_clean":"2024-04-25T10:03:38.206229+0000","last_became_active":"2024-04-25T10:03:37.131287+0000","last_became_peered":"2024-04-25T10:03:37.131287+0000","last_unstale":"2024-04-25T10:03:38.206229+0000","last_undegraded":"2024-04-25T10:03:38.206229+0000","last_fullsized":"2024-04-25T10:03:38.206229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:00.454471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062709+0000","last_change":"2024-04-25T10:03:35.062709+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062709+0000","last_undegraded":"2024-04-25T10:03:35.062709+0000","last_fullsized":"2024-04-25T10:03:35.062709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075811+0000","last_change":"2024-04-25T10:03:35.075811+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075811+0000","last_undegraded":"2024-04-25T10:03:35.075811+0000","last_fullsized":"2024-04-25T10:03:35.075811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206789+0000","last_change":"2024-04-25T10:03:36.105651+0000","last_active":"2024-04-25T10:03:38.206789+0000","last_peered":"2024-04-25T10:03:38.206789+0000","last_clean":"2024-04-25T10:03:38.206789+0000","last_became_active":"2024-04-25T10:03:36.105203+0000","last_became_peered":"2024-04-25T10:03:36.105203+0000","last_unstale":"2024-04-25T10:03:38.206789+0000","last_undegraded":"2024-04-25T10:03:38.206789+0000","last_fullsized":"2024-04-25T10:03:38.206789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:09.488962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062261+0000","last_change":"2024-04-25T10:03:35.062261+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062261+0000","last_undegraded":"2024-04-25T10:03:35.062261+0000","last_fullsized":"2024-04-25T10:03:35.062261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206661+0000","last_change":"2024-04-25T10:03:37.115687+0000","last_active":"2024-04-25T10:03:38.206661+0000","last_peered":"2024-04-25T10:03:38.206661+0000","last_clean":"2024-04-25T10:03:38.206661+0000","last_became_active":"2024-04-25T10:03:37.115368+0000","last_became_peered":"2024-04-25T10:03:37.115368+0000","last_unstale":"2024-04-25T10:03:38.206661+0000","last_undegraded":"2024-04-25T10:03:38.206661+0000","last_fullsized":"2024-04-25T10:03:38.206661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:56.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062315+0000","last_change":"2024-04-25T10:03:35.062315+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062315+0000","last_undegraded":"2024-04-25T10:03:35.062315+0000","last_fullsized":"2024-04-25T10:03:35.062315+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065569+0000","last_change":"2024-04-25T10:03:35.065569+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065569+0000","last_undegraded":"2024-04-25T10:03:35.065569+0000","last_fullsized":"2024-04-25T10:03:35.065569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205709+0000","last_change":"2024-04-25T10:03:36.095557+0000","last_active":"2024-04-25T10:03:38.205709+0000","last_peered":"2024-04-25T10:03:38.205709+0000","last_clean":"2024-04-25T10:03:38.205709+0000","last_became_active":"2024-04-25T10:03:36.095265+0000","last_became_peered":"2024-04-25T10:03:36.095265+0000","last_unstale":"2024-04-25T10:03:38.205709+0000","last_undegraded":"2024-04-25T10:03:38.205709+0000","last_fullsized":"2024-04-25T10:03:38.205709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.385227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065358+0000","last_change":"2024-04-25T10:03:35.065358+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065358+0000","last_undegraded":"2024-04-25T10:03:35.065358+0000","last_fullsized":"2024-04-25T10:03:35.065358+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207045+0000","last_change":"2024-04-25T10:03:37.130899+0000","last_active":"2024-04-25T10:03:38.207045+0000","last_peered":"2024-04-25T10:03:38.207045+0000","last_clean":"2024-04-25T10:03:38.207045+0000","last_became_active":"2024-04-25T10:03:37.130432+0000","last_became_peered":"2024-04-25T10:03:37.130432+0000","last_unstale":"2024-04-25T10:03:38.207045+0000","last_undegraded":"2024-04-25T10:03:38.207045+0000","last_fullsized":"2024-04-25T10:03:38.207045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.079554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207233+0000","last_change":"2024-04-25T10:03:36.098558+0000","last_active":"2024-04-25T10:03:38.207233+0000","last_peered":"2024-04-25T10:03:38.207233+0000","last_clean":"2024-04-25T10:03:38.207233+0000","last_became_active":"2024-04-25T10:03:36.098223+0000","last_became_peered":"2024-04-25T10:03:36.098223+0000","last_unstale":"2024-04-25T10:03:38.207233+0000","last_undegraded":"2024-04-25T10:03:38.207233+0000","last_fullsized":"2024-04-25T10:03:38.207233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:51.024456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074550+0000","last_change":"2024-04-25T10:03:35.074550+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074550+0000","last_undegraded":"2024-04-25T10:03:35.074550+0000","last_fullsized":"2024-04-25T10:03:35.074550+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207030+0000","last_change":"2024-04-25T10:03:37.138930+0000","last_active":"2024-04-25T10:03:38.207030+0000","last_peered":"2024-04-25T10:03:38.207030+0000","last_clean":"2024-04-25T10:03:38.207030+0000","last_became_active":"2024-04-25T10:03:37.138643+0000","last_became_peered":"2024-04-25T10:03:37.138643+0000","last_unstale":"2024-04-25T10:03:38.207030+0000","last_undegraded":"2024-04-25T10:03:38.207030+0000","last_fullsized":"2024-04-25T10:03:38.207030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.100516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.069611+0000","last_change":"2024-04-25T10:03:35.069611+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.069611+0000","last_undegraded":"2024-04-25T10:03:35.069611+0000","last_fullsized":"2024-04-25T10:03:35.069611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207294+0000","last_change":"2024-04-25T10:03:36.098166+0000","last_active":"2024-04-25T10:03:38.207294+0000","last_peered":"2024-04-25T10:03:38.207294+0000","last_clean":"2024-04-25T10:03:38.207294+0000","last_became_active":"2024-04-25T10:03:36.097774+0000","last_became_peered":"2024-04-25T10:03:36.097774+0000","last_unstale":"2024-04-25T10:03:38.207294+0000","last_undegraded":"2024-04-25T10:03:38.207294+0000","last_fullsized":"2024-04-25T10:03:38.207294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:32.511676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207141+0000","last_change":"2024-04-25T10:03:37.119860+0000","last_active":"2024-04-25T10:03:38.207141+0000","last_peered":"2024-04-25T10:03:38.207141+0000","last_clean":"2024-04-25T10:03:38.207141+0000","last_became_active":"2024-04-25T10:03:37.118103+0000","last_became_peered":"2024-04-25T10:03:37.118103+0000","last_unstale":"2024-04-25T10:03:38.207141+0000","last_undegraded":"2024-04-25T10:03:38.207141+0000","last_fullsized":"2024-04-25T10:03:38.207141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:06.878310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207283+0000","last_change":"2024-04-25T10:03:36.104310+0000","last_active":"2024-04-25T10:03:38.207283+0000","last_peered":"2024-04-25T10:03:38.207283+0000","last_clean":"2024-04-25T10:03:38.207283+0000","last_became_active":"2024-04-25T10:03:36.103982+0000","last_became_peered":"2024-04-25T10:03:36.103982+0000","last_unstale":"2024-04-25T10:03:38.207283+0000","last_undegraded":"2024-04-25T10:03:38.207283+0000","last_fullsized":"2024-04-25T10:03:38.207283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:33.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.076755+0000","last_change":"2024-04-25T10:03:35.076755+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.076755+0000","last_undegraded":"2024-04-25T10:03:35.076755+0000","last_fullsized":"2024-04-25T10:03:35.076755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.072257+0000","last_change":"2024-04-25T10:03:35.072257+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.072257+0000","last_undegraded":"2024-04-25T10:03:35.072257+0000","last_fullsized":"2024-04-25T10:03:35.072257+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.062252+0000","last_change":"2024-04-25T10:03:35.062252+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.062252+0000","last_undegraded":"2024-04-25T10:03:35.062252+0000","last_fullsized":"2024-04-25T10:03:35.062252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065348+0000","last_change":"2024-04-25T10:03:35.065348+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065348+0000","last_undegraded":"2024-04-25T10:03:35.065348+0000","last_fullsized":"2024-04-25T10:03:35.065348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.073828+0000","last_change":"2024-04-25T10:03:35.073828+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.073828+0000","last_undegraded":"2024-04-25T10:03:35.073828+0000","last_fullsized":"2024-04-25T10:03:35.073828+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.067731+0000","last_change":"2024-04-25T10:03:35.067731+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.067731+0000","last_undegraded":"2024-04-25T10:03:35.067731+0000","last_fullsized":"2024-04-25T10:03:35.067731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074734+0000","last_change":"2024-04-25T10:03:35.074734+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074734+0000","last_undegraded":"2024-04-25T10:03:35.074734+0000","last_fullsized":"2024-04-25T10:03:35.074734+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.074177+0000","last_change":"2024-04-25T10:03:35.074177+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.074177+0000","last_undegraded":"2024-04-25T10:03:35.074177+0000","last_fullsized":"2024-04-25T10:03:35.074177+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205441+0000","last_change":"2024-04-25T10:03:37.139337+0000","last_active":"2024-04-25T10:03:38.205441+0000","last_peered":"2024-04-25T10:03:38.205441+0000","last_clean":"2024-04-25T10:03:38.205441+0000","last_became_active":"2024-04-25T10:03:37.137532+0000","last_became_peered":"2024-04-25T10:03:37.137532+0000","last_unstale":"2024-04-25T10:03:38.205441+0000","last_undegraded":"2024-04-25T10:03:38.205441+0000","last_fullsized":"2024-04-25T10:03:38.205441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:56.223185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075811+0000","last_change":"2024-04-25T10:03:35.075811+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075811+0000","last_undegraded":"2024-04-25T10:03:35.075811+0000","last_fullsized":"2024-04-25T10:03:35.075811+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.065355+0000","last_change":"2024-04-25T10:03:35.065355+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.065355+0000","last_undegraded":"2024-04-25T10:03:35.065355+0000","last_fullsized":"2024-04-25T10:03:35.065355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207050+0000","last_change":"2024-04-25T10:03:37.123190+0000","last_active":"2024-04-25T10:03:38.207050+0000","last_peered":"2024-04-25T10:03:38.207050+0000","last_clean":"2024-04-25T10:03:38.207050+0000","last_became_active":"2024-04-25T10:03:37.122906+0000","last_became_peered":"2024-04-25T10:03:37.122906+0000","last_unstale":"2024-04-25T10:03:38.207050+0000","last_undegraded":"2024-04-25T10:03:38.207050+0000","last_fullsized":"2024-04-25T10:03:38.207050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.154712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.075675+0000","last_change":"2024-04-25T10:03:35.075675+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.075675+0000","last_undegraded":"2024-04-25T10:03:35.075675+0000","last_fullsized":"2024-04-25T10:03:35.075675+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.073206+0000","last_change":"2024-04-25T10:03:35.073206+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.073206+0000","last_undegraded":"2024-04-25T10:03:35.073206+0000","last_fullsized":"2024-04-25T10:03:35.073206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.064189+0000","last_change":"2024-04-25T10:03:35.064189+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.064189+0000","last_undegraded":"2024-04-25T10:03:35.064189+0000","last_fullsized":"2024-04-25T10:03:35.064189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.064903+0000","last_change":"2024-04-25T10:03:35.064903+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.064903+0000","last_undegraded":"2024-04-25T10:03:35.064903+0000","last_fullsized":"2024-04-25T10:03:35.064903+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207633+0000","last_change":"2024-04-25T10:03:36.102517+0000","last_active":"2024-04-25T10:03:38.207633+0000","last_peered":"2024-04-25T10:03:38.207633+0000","last_clean":"2024-04-25T10:03:38.207633+0000","last_became_active":"2024-04-25T10:03:36.102006+0000","last_became_peered":"2024-04-25T10:03:36.102006+0000","last_unstale":"2024-04-25T10:03:38.207633+0000","last_undegraded":"2024-04-25T10:03:38.207633+0000","last_fullsized":"2024-04-25T10:03:38.207633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:22.682970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.069045+0000","last_change":"2024-04-25T10:03:35.069045+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.069045+0000","last_undegraded":"2024-04-25T10:03:35.069045+0000","last_fullsized":"2024-04-25T10:03:35.069045+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.071523+0000","last_change":"2024-04-25T10:03:35.071523+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.071523+0000","last_undegraded":"2024-04-25T10:03:35.071523+0000","last_fullsized":"2024-04-25T10:03:35.071523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206932+0000","last_change":"2024-04-25T10:03:36.090034+0000","last_active":"2024-04-25T10:03:38.206932+0000","last_peered":"2024-04-25T10:03:38.206932+0000","last_clean":"2024-04-25T10:03:38.206932+0000","last_became_active":"2024-04-25T10:03:36.089136+0000","last_became_peered":"2024-04-25T10:03:36.089136+0000","last_unstale":"2024-04-25T10:03:38.206932+0000","last_undegraded":"2024-04-25T10:03:38.206932+0000","last_fullsized":"2024-04-25T10:03:38.206932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:03:36.053810+0000","last_change":"2024-04-25T10:03:36.053810+0000","last_active":"2024-04-25T10:03:36.053810+0000","last_peered":"2024-04-25T10:03:36.053810+0000","last_clean":"2024-04-25T10:03:36.053810+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:36.053810+0000","last_undegraded":"2024-04-25T10:03:36.053810+0000","last_fullsized":"2024-04-25T10:03:36.053810+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-25T10:03:35.068912+0000","last_change":"2024-04-25T10:03:35.068912+0000","last_active":"2024-04-25T10:03:35.041419+0000","last_peered":"2024-04-25T10:03:35.041419+0000","last_clean":"2024-04-25T10:03:35.041419+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:03:35.068912+0000","last_undegraded":"2024-04-25T10:03:35.068912+0000","last_fullsized":"2024-04-25T10:03:35.068912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-25T10:03:35.066540+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:03:35.066540+0000","last_peered":"2024-04-25T10:03:35.066540+0000","last_clean":"2024-04-25T10:03:35.066540+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:03:35.066540+0000","last_undegraded":"2024-04-25T10:03:35.066540+0000","last_fullsized":"2024-04-25T10:03:35.066540+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":1},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1011712,"data_stored":764344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":417792,"data_stored":173976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":764344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":405504,"data_stored":167767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":196238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:42.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-25T10:03:42.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-25T10:03:42.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-25T10:03:42.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-25T10:03:42.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-25T10:03:42.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-25T10:03:42.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-25T10:03:42.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.062709+0000', 'last_change': '2024-04-25T10:03:35.062709+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.062709+0000', 'last_undegraded': '2024-04-25T10:03:35.062709+0000', 'last_fullsized': '2024-04-25T10:03:35.062709+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.075811+0000', 'last_change': '2024-04-25T10:03:35.075811+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.075811+0000', 'last_undegraded': '2024-04-25T10:03:35.075811+0000', 'last_fullsized': '2024-04-25T10:03:35.075811+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-25T10:03:42.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.062261+0000', 'last_change': '2024-04-25T10:03:35.062261+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.062261+0000', 'last_undegraded': '2024-04-25T10:03:35.062261+0000', 'last_fullsized': '2024-04-25T10:03:35.062261+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.062315+0000', 'last_change': '2024-04-25T10:03:35.062315+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.062315+0000', 'last_undegraded': '2024-04-25T10:03:35.062315+0000', 'last_fullsized': '2024-04-25T10:03:35.062315+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.065569+0000', 'last_change': '2024-04-25T10:03:35.065569+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.065569+0000', 'last_undegraded': '2024-04-25T10:03:35.065569+0000', 'last_fullsized': '2024-04-25T10:03:35.065569+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-25T10:03:42.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.065358+0000', 'last_change': '2024-04-25T10:03:35.065358+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.065358+0000', 'last_undegraded': '2024-04-25T10:03:35.065358+0000', 'last_fullsized': '2024-04-25T10:03:35.065358+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.074550+0000', 'last_change': '2024-04-25T10:03:35.074550+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.074550+0000', 'last_undegraded': '2024-04-25T10:03:35.074550+0000', 'last_fullsized': '2024-04-25T10:03:35.074550+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.069611+0000', 'last_change': '2024-04-25T10:03:35.069611+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.069611+0000', 'last_undegraded': '2024-04-25T10:03:35.069611+0000', 'last_fullsized': '2024-04-25T10:03:35.069611+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-25T10:03:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.076755+0000', 'last_change': '2024-04-25T10:03:35.076755+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.076755+0000', 'last_undegraded': '2024-04-25T10:03:35.076755+0000', 'last_fullsized': '2024-04-25T10:03:35.076755+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.072257+0000', 'last_change': '2024-04-25T10:03:35.072257+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.072257+0000', 'last_undegraded': '2024-04-25T10:03:35.072257+0000', 'last_fullsized': '2024-04-25T10:03:35.072257+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.062252+0000', 'last_change': '2024-04-25T10:03:35.062252+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.062252+0000', 'last_undegraded': '2024-04-25T10:03:35.062252+0000', 'last_fullsized': '2024-04-25T10:03:35.062252+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-25T10:03:42.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.065348+0000', 'last_change': '2024-04-25T10:03:35.065348+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.065348+0000', 'last_undegraded': '2024-04-25T10:03:35.065348+0000', 'last_fullsized': '2024-04-25T10:03:35.065348+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.073828+0000', 'last_change': '2024-04-25T10:03:35.073828+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.073828+0000', 'last_undegraded': '2024-04-25T10:03:35.073828+0000', 'last_fullsized': '2024-04-25T10:03:35.073828+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.067731+0000', 'last_change': '2024-04-25T10:03:35.067731+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.067731+0000', 'last_undegraded': '2024-04-25T10:03:35.067731+0000', 'last_fullsized': '2024-04-25T10:03:35.067731+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:42.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.074734+0000', 'last_change': '2024-04-25T10:03:35.074734+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.074734+0000', 'last_undegraded': '2024-04-25T10:03:35.074734+0000', 'last_fullsized': '2024-04-25T10:03:35.074734+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.074177+0000', 'last_change': '2024-04-25T10:03:35.074177+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.074177+0000', 'last_undegraded': '2024-04-25T10:03:35.074177+0000', 'last_fullsized': '2024-04-25T10:03:35.074177+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.075811+0000', 'last_change': '2024-04-25T10:03:35.075811+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.075811+0000', 'last_undegraded': '2024-04-25T10:03:35.075811+0000', 'last_fullsized': '2024-04-25T10:03:35.075811+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-25T10:03:42.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.065355+0000', 'last_change': '2024-04-25T10:03:35.065355+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.065355+0000', 'last_undegraded': '2024-04-25T10:03:35.065355+0000', 'last_fullsized': '2024-04-25T10:03:35.065355+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.075675+0000', 'last_change': '2024-04-25T10:03:35.075675+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.075675+0000', 'last_undegraded': '2024-04-25T10:03:35.075675+0000', 'last_fullsized': '2024-04-25T10:03:35.075675+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.073206+0000', 'last_change': '2024-04-25T10:03:35.073206+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.073206+0000', 'last_undegraded': '2024-04-25T10:03:35.073206+0000', 'last_fullsized': '2024-04-25T10:03:35.073206+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T10:03:42.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.064189+0000', 'last_change': '2024-04-25T10:03:35.064189+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.064189+0000', 'last_undegraded': '2024-04-25T10:03:35.064189+0000', 'last_fullsized': '2024-04-25T10:03:35.064189+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.064903+0000', 'last_change': '2024-04-25T10:03:35.064903+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.064903+0000', 'last_undegraded': '2024-04-25T10:03:35.064903+0000', 'last_fullsized': '2024-04-25T10:03:35.064903+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.069045+0000', 'last_change': '2024-04-25T10:03:35.069045+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.069045+0000', 'last_undegraded': '2024-04-25T10:03:35.069045+0000', 'last_fullsized': '2024-04-25T10:03:35.069045+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:03:42.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.071523+0000', 'last_change': '2024-04-25T10:03:35.071523+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.071523+0000', 'last_undegraded': '2024-04-25T10:03:35.071523+0000', 'last_fullsized': '2024-04-25T10:03:35.071523+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:03:36.053810+0000', 'last_change': '2024-04-25T10:03:36.053810+0000', 'last_active': '2024-04-25T10:03:36.053810+0000', 'last_peered': '2024-04-25T10:03:36.053810+0000', 'last_clean': '2024-04-25T10:03:36.053810+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:36.053810+0000', 'last_undegraded': '2024-04-25T10:03:36.053810+0000', 'last_fullsized': '2024-04-25T10:03:36.053810+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:36.053810+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T10:03:42.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:03:35.068912+0000', 'last_change': '2024-04-25T10:03:35.068912+0000', 'last_active': '2024-04-25T10:03:35.041419+0000', 'last_peered': '2024-04-25T10:03:35.041419+0000', 'last_clean': '2024-04-25T10:03:35.041419+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:03:35.068912+0000', 'last_undegraded': '2024-04-25T10:03:35.068912+0000', 'last_fullsized': '2024-04-25T10:03:35.068912+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'last_clean_scrub_stamp': '2024-04-25T10:03:35.041419+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:03:42.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:03:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.869+0000 7fcde3c00640 1 Processor -- start 2024-04-25T10:03:42.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.869+0000 7fcde3c00640 1 -- start start 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc0714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde3c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcddc0cd2c0 0x7fcddc0cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde3c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcddc06d170 0x7fcddc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde3c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc071a20 con 0x7fcddc06d170 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde3c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0cdd80 con 0x7fcddc0cd2c0 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde3c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0ce060 con 0x7fcddc071100 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde1c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcddc06d170 0x7fcddc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde1c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcddc06d170 0x7fcddc06d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36100/0 (socket says 172.21.15.5:36100) 2024-04-25T10:03:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.871+0000 7fcde1c00640 1 -- 172.21.15.5:0/3049303757 learned_addr learned my addr 172.21.15.5:0/3049303757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcddbe00640 1 --2- 172.21.15.5:0/3049303757 >> v2:172.21.15.80:3300/0 conn(0x7fcddc0cd2c0 0x7fcddc0cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcde1200640 1 --2- 172.21.15.5:0/3049303757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc0714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcde1c00640 1 -- 172.21.15.5:0/3049303757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 msgr2=0x7fcddc0714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcde1c00640 1 --2- 172.21.15.5:0/3049303757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc0714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcde1c00640 1 -- 172.21.15.5:0/3049303757 >> v2:172.21.15.80:3300/0 conn(0x7fcddc0cd2c0 msgr2=0x7fcddc0cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcde1c00640 1 --2- 172.21.15.5:0/3049303757 >> v2:172.21.15.80:3300/0 conn(0x7fcddc0cd2c0 0x7fcddc0cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcde1c00640 1 -- 172.21.15.5:0/3049303757 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc15c090 con 0x7fcddc06d170 2024-04-25T10:03:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.872+0000 7fcddbe00640 1 --2- 172.21.15.5:0/3049303757 >> v2:172.21.15.80:3300/0 conn(0x7fcddc0cd2c0 0x7fcddc0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.873+0000 7fcde1c00640 1 --2- 172.21.15.5:0/3049303757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc06d600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcddc06cac0 tx=0x7fcdd402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780678d801b739cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.874+0000 7fcddb400640 1 -- 172.21.15.5:0/3049303757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4037070 con 0x7fcddc06d170 2024-04-25T10:03:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.874+0000 7fcddb400640 1 -- 172.21.15.5:0/3049303757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd4032440 con 0x7fcddc06d170 2024-04-25T10:03:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.874+0000 7fcddb400640 1 -- 172.21.15.5:0/3049303757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4032810 con 0x7fcddc06d170 2024-04-25T10:03:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.874+0000 7fcde3c00640 1 -- 172.21.15.5:0/3049303757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 msgr2=0x7fcddc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.874+0000 7fcde3c00640 1 --2- 172.21.15.5:0/3049303757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc06d600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcddc06cac0 tx=0x7fcdd402a140 comp rx=0 tx=0).stop 2024-04-25T10:03:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.874+0000 7fcde3c00640 1 -- 172.21.15.5:0/3049303757 shutdown_connections 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.875+0000 7fcde3c00640 1 --2- 172.21.15.5:0/3049303757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc06d600 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.875+0000 7fcde3c00640 1 --2- 172.21.15.5:0/3049303757 >> v2:172.21.15.80:3300/0 conn(0x7fcddc0cd2c0 0x7fcddc0cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.875+0000 7fcde3c00640 1 --2- 172.21.15.5:0/3049303757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc0714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.875+0000 7fcde3c00640 1 -- 172.21.15.5:0/3049303757 >> 172.21.15.5:0/3049303757 conn(0x7fcddc06ab60 msgr2=0x7fcddc06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.875+0000 7fcde3c00640 1 -- 172.21.15.5:0/3049303757 shutdown_connections 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.875+0000 7fcde3c00640 1 -- 172.21.15.5:0/3049303757 wait complete. 2024-04-25T10:03:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.877+0000 7fcde3c00640 1 Processor -- start 2024-04-25T10:03:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.877+0000 7fcde3c00640 1 -- start start 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc13aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcddc0cd2c0 0x7fcddc0c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde3c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0cc260 con 0x7fcddc06d170 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde3c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc15bd80 con 0x7fcddc0cd2c0 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde3c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc145270 con 0x7fcddc071100 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc0c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36110/0 (socket says 172.21.15.5:36110) 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 -- 172.21.15.5:0/1098855237 learned_addr learned my addr 172.21.15.5:0/1098855237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcddbe00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc13aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcddc0cd2c0 0x7fcddc0c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 -- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 msgr2=0x7fcddc13aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc13aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 -- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcddc0cd2c0 msgr2=0x7fcddc0c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcddc0cd2c0 0x7fcddc0c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.878+0000 7fcde1200640 1 -- 172.21.15.5:0/1098855237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc142280 con 0x7fcddc06d170 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.879+0000 7fcde1200640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc0c51e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007e60 tx=0x7fcdcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.879+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc014620 con 0x7fcddc06d170 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.879+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc019070 con 0x7fcddc06d170 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.879+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc014a00 con 0x7fcddc06d170 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.879+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc0cb2a0 con 0x7fcddc06d170 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.879+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc0b0600 con 0x7fcddc06d170 2024-04-25T10:03:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.880+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc15e950 con 0x7fcddc06d170 2024-04-25T10:03:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.883+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdcc014ba0 con 0x7fcddc06d170 2024-04-25T10:03:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.883+0000 7fcdd8c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdb806b4d0 0x7fcdb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.883+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdcc08ac10 con 0x7fcddc06d170 2024-04-25T10:03:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.883+0000 7fcddbe00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdb806b4d0 0x7fcdb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.884+0000 7fcddbe00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdb806b4d0 0x7fcdb806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcddc142550 tx=0x7fcdc4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:42.887+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcddc15e950 con 0x7fcddc06d170 2024-04-25T10:03:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.157+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcddc0b0110 con 0x7fcddc06d170 2024-04-25T10:03:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.158+0000 7fcdd8c00640 1 -- 172.21.15.5:0/1098855237 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1013 (secure 0 0 0) 0x7fcddc142280 con 0x7fcddc06d170 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:43.164 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":129,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039305,"num_in_osds":8,"osd_in_since":1714039296,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":243298304,"bytes_avail":772850814976,"bytes_total":773094113280,"write_bytes_sec":2176,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5005}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-25T10:03:04.635657+0000","services":{}},"progress_events":{}} 2024-04-25T10:03:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdb806b4d0 msgr2=0x7fcdb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdb806b4d0 0x7fcdb806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcddc142550 tx=0x7fcdc4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 msgr2=0x7fcddc0c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc0c51e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007e60 tx=0x7fcdcc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 shutdown_connections 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdb806b4d0 0x7fcdb806d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcddc0cd2c0 0x7fcddc0c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcddc071100 0x7fcddc13aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 --2- 172.21.15.5:0/1098855237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcddc06d170 0x7fcddc0c51e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 >> 172.21.15.5:0/1098855237 conn(0x7fcddc06ab60 msgr2=0x7fcddc0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 shutdown_connections 2024-04-25T10:03:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.161+0000 7fcde3c00640 1 -- 172.21.15.5:0/1098855237 wait complete. 2024-04-25T10:03:43.184 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:03:43.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:43.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.382+0000 7fa503200640 1 Processor -- start 2024-04-25T10:03:43.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.382+0000 7fa503200640 1 -- start start 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa503200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa504154a10 0x7fa50415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa503200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5040cd200 con 0x7fa504070f90 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa503200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50406e6f0 con 0x7fa5040cdca0 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa503200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50406e9d0 con 0x7fa504154a10 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa502800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa504070f90 0x7fa50406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40544/0 (socket says 172.21.15.5:40544) 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa502800640 1 -- 172.21.15.5:0/2201011945 learned_addr learned my addr 172.21.15.5:0/2201011945 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423179926 0 0) 0x7fa5040cd200 con 0x7fa504070f90 2024-04-25T10:03:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501e00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa504154a10 0x7fa50415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e8003520 con 0x7fa504070f90 2024-04-25T10:03:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295836975 0 0) 0x7fa4e8003520 con 0x7fa504070f90 2024-04-25T10:03:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa504154a10 msgr2=0x7fa50415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa504154a10 0x7fa50415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 >> v1:172.21.15.80:6789/0 conn(0x7fa5040cdca0 legacy=0x7fa5040b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50406ecb0 con 0x7fa504070f90 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.384+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa4fc0019f0 con 0x7fa504070f90 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 >> v1:172.21.15.5:6789/0 conn(0x7fa504070f90 legacy=0x7fa50406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e80039e0 0x7fa4e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e80052e0 0x7fa4e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50406e9d0 con 0x7fa4e8004800 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e800c160 con 0x7fa4e80052e0 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e800c360 con 0x7fa4e80039e0 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501e00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa504154a10 0x7fa50415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa4fbe00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa502800640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e80052e0 0x7fa4e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa4fbe00640 1 -- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e80039e0 msgr2=0x7fa4e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa501e00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e80039e0 0x7fa4e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa4fbe00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e80039e0 0x7fa4e8003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa4fbe00640 1 -- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e80052e0 msgr2=0x7fa4e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa4fbe00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e80052e0 0x7fa4e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.385+0000 7fa4fbe00640 1 -- 172.21.15.5:0/2201011945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8003520 con 0x7fa4e8004800 2024-04-25T10:03:43.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.386+0000 7fa502800640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4e80052e0 0x7fa4e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa4fbe00640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec001720 tx=0x7fa4e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1df85abac071fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec001370 con 0x7fa4e8004800 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0052d0 con 0x7fa4e8004800 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa501400640 1 -- 172.21.15.5:0/2201011945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec02abc0 con 0x7fa4e8004800 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa503200640 1 -- 172.21.15.5:0/2201011945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e8004800 msgr2=0x7fa4e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa503200640 1 --2- 172.21.15.5:0/2201011945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4e8004800 0x7fa4e8004c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec001720 tx=0x7fa4e8005830 comp rx=0 tx=0).stop 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa502800640 1 -- 172.21.15.5:0/2201011945 reap_dead start 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa503200640 1 -- 172.21.15.5:0/2201011945 shutdown_connections 2024-04-25T10:03:43.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa503200640 1 -- 172.21.15.5:0/2201011945 >> 172.21.15.5:0/2201011945 conn(0x7fa504066f20 msgr2=0x7fa50406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa503200640 1 -- 172.21.15.5:0/2201011945 shutdown_connections 2024-04-25T10:03:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.388+0000 7fa503200640 1 -- 172.21.15.5:0/2201011945 wait complete. 2024-04-25T10:03:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.389+0000 7fa503200640 1 Processor -- start 2024-04-25T10:03:43.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.390+0000 7fa503200640 1 -- start start 2024-04-25T10:03:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.390+0000 7fa503200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e8004800 0x7fa504152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa503200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa504143410 0x7fa5041e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa503200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 0x7fa50415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa503200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50406d700 con 0x7fa50415e850 2024-04-25T10:03:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa503200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040cd780 con 0x7fa504143410 2024-04-25T10:03:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa503200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa50406d880 con 0x7fa4e8004800 2024-04-25T10:03:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 0x7fa50415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 0x7fa50415ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36118/0 (socket says 172.21.15.5:36118) 2024-04-25T10:03:43.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 -- 172.21.15.5:0/4273694370 learned_addr learned my addr 172.21.15.5:0/4273694370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa501e00640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e8004800 0x7fa504152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 -- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e8004800 msgr2=0x7fa504152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa4fbe00640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa504143410 0x7fa5041e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e8004800 0x7fa504152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 -- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa504143410 msgr2=0x7fa5041e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa504143410 0x7fa5041e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa502800640 1 -- 172.21.15.5:0/4273694370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50415fa10 con 0x7fa50415e850 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa501e00640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e8004800 0x7fa504152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.391+0000 7fa4fbe00640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa504143410 0x7fa5041e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.392+0000 7fa502800640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 0x7fa50415ecd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ccdf0 tx=0x7fa4fc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.392+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00dea0 con 0x7fa50415e850 2024-04-25T10:03:43.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.392+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc00bb50 con 0x7fa50415e850 2024-04-25T10:03:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.392+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc007110 con 0x7fa50415e850 2024-04-25T10:03:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.392+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50415fcf0 con 0x7fa50415e850 2024-04-25T10:03:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.392+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5041e65a0 con 0x7fa50415e850 2024-04-25T10:03:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.394+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa504068f60 con 0x7fa50415e850 2024-04-25T10:03:43.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.394+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4fc01b020 con 0x7fa50415e850 2024-04-25T10:03:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.395+0000 7fa4fa000640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa4e006b4d0 0x7fa4e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.395+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4fc08a490 con 0x7fa50415e850 2024-04-25T10:03:43.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.396+0000 7fa501e00640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa4e006b4d0 0x7fa4e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.397+0000 7fa501e00640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa4e006b4d0 0x7fa4e006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ccb70 tx=0x7fa4f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.400+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4fc059210 con 0x7fa50415e850 2024-04-25T10:03:43.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.598+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5040cce40 con 0x7fa4e006b4d0 2024-04-25T10:03:43.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.617+0000 7fa4fa000640 1 -- 172.21.15.5:0/4273694370 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251957 (secure 0 0 0) 0x7fa5040cce40 con 0x7fa4e006b4d0 2024-04-25T10:03:43.627 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:43.628 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:03:43.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa4e006b4d0 msgr2=0x7fa4e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa4e006b4d0 0x7fa4e006d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ccb70 tx=0x7fa4f4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:43.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 msgr2=0x7fa50415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 0x7fa50415ecd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ccdf0 tx=0x7fa4fc00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 shutdown_connections 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50415e850 0x7fa50415ecd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa4e006b4d0 0x7fa4e006d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa504143410 0x7fa5041e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 --2- 172.21.15.5:0/4273694370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4e8004800 0x7fa504152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.623+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 >> 172.21.15.5:0/4273694370 conn(0x7fa504066f20 msgr2=0x7fa5040713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.625+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 shutdown_connections 2024-04-25T10:03:43.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:43.625+0000 7fa503200640 1 -- 172.21.15.5:0/4273694370 wait complete. 2024-04-25T10:03:43.650 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-04-25T10:03:42.649247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237596,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737124,"statfs":{"total":773094113280,"available":772850814976,"internally_reserved":0,"allocated":4890624,"data_stored":2741572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.585636"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207506+0000","last_change":"2024-04-25T10:03:37.134249+0000","last_active":"2024-04-25T10:03:38.207506+0000","last_peered":"2024-04-25T10:03:38.207506+0000","last_clean":"2024-04-25T10:03:38.207506+0000","last_became_active":"2024-04-25T10:03:37.133378+0000","last_became_peered":"2024-04-25T10:03:37.133378+0000","last_unstale":"2024-04-25T10:03:38.207506+0000","last_undegraded":"2024-04-25T10:03:38.207506+0000","last_fullsized":"2024-04-25T10:03:38.207506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.585105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122100+0000","last_change":"2024-04-25T10:03:37.118315+0000","last_active":"2024-04-25T10:03:38.122100+0000","last_peered":"2024-04-25T10:03:38.122100+0000","last_clean":"2024-04-25T10:03:38.122100+0000","last_became_active":"2024-04-25T10:03:37.116557+0000","last_became_peered":"2024-04-25T10:03:37.116557+0000","last_unstale":"2024-04-25T10:03:38.122100+0000","last_undegraded":"2024-04-25T10:03:38.122100+0000","last_fullsized":"2024-04-25T10:03:38.122100+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:07.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206496+0000","last_change":"2024-04-25T10:03:37.133516+0000","last_active":"2024-04-25T10:03:38.206496+0000","last_peered":"2024-04-25T10:03:38.206496+0000","last_clean":"2024-04-25T10:03:38.206496+0000","last_became_active":"2024-04-25T10:03:37.132370+0000","last_became_peered":"2024-04-25T10:03:37.132370+0000","last_unstale":"2024-04-25T10:03:38.206496+0000","last_undegraded":"2024-04-25T10:03:38.206496+0000","last_fullsized":"2024-04-25T10:03:38.206496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:33.505774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206619+0000","last_change":"2024-04-25T10:03:37.131079+0000","last_active":"2024-04-25T10:03:38.206619+0000","last_peered":"2024-04-25T10:03:38.206619+0000","last_clean":"2024-04-25T10:03:38.206619+0000","last_became_active":"2024-04-25T10:03:37.130786+0000","last_became_peered":"2024-04-25T10:03:37.130786+0000","last_unstale":"2024-04-25T10:03:38.206619+0000","last_undegraded":"2024-04-25T10:03:38.206619+0000","last_fullsized":"2024-04-25T10:03:38.206619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:11.177455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122957+0000","last_change":"2024-04-25T10:03:37.126163+0000","last_active":"2024-04-25T10:03:38.122957+0000","last_peered":"2024-04-25T10:03:38.122957+0000","last_clean":"2024-04-25T10:03:38.122957+0000","last_became_active":"2024-04-25T10:03:37.125858+0000","last_became_peered":"2024-04-25T10:03:37.125858+0000","last_unstale":"2024-04-25T10:03:38.122957+0000","last_undegraded":"2024-04-25T10:03:38.122957+0000","last_fullsized":"2024-04-25T10:03:38.122957+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:09.118521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120997+0000","last_change":"2024-04-25T10:03:37.123236+0000","last_active":"2024-04-25T10:03:38.120997+0000","last_peered":"2024-04-25T10:03:38.120997+0000","last_clean":"2024-04-25T10:03:38.120997+0000","last_became_active":"2024-04-25T10:03:37.122935+0000","last_became_peered":"2024-04-25T10:03:37.122935+0000","last_unstale":"2024-04-25T10:03:38.120997+0000","last_undegraded":"2024-04-25T10:03:38.120997+0000","last_fullsized":"2024-04-25T10:03:38.120997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.800549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205727+0000","last_change":"2024-04-25T10:03:37.135271+0000","last_active":"2024-04-25T10:03:38.205727+0000","last_peered":"2024-04-25T10:03:38.205727+0000","last_clean":"2024-04-25T10:03:38.205727+0000","last_became_active":"2024-04-25T10:03:37.134973+0000","last_became_peered":"2024-04-25T10:03:37.134973+0000","last_unstale":"2024-04-25T10:03:38.205727+0000","last_undegraded":"2024-04-25T10:03:38.205727+0000","last_fullsized":"2024-04-25T10:03:38.205727+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:45.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205696+0000","last_change":"2024-04-25T10:03:37.133051+0000","last_active":"2024-04-25T10:03:38.205696+0000","last_peered":"2024-04-25T10:03:38.205696+0000","last_clean":"2024-04-25T10:03:38.205696+0000","last_became_active":"2024-04-25T10:03:37.131855+0000","last_became_peered":"2024-04-25T10:03:37.131855+0000","last_unstale":"2024-04-25T10:03:38.205696+0000","last_undegraded":"2024-04-25T10:03:38.205696+0000","last_fullsized":"2024-04-25T10:03:38.205696+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:52.525576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.123240+0000","last_change":"2024-04-25T10:03:37.124102+0000","last_active":"2024-04-25T10:03:38.123240+0000","last_peered":"2024-04-25T10:03:38.123240+0000","last_clean":"2024-04-25T10:03:38.123240+0000","last_became_active":"2024-04-25T10:03:37.123828+0000","last_became_peered":"2024-04-25T10:03:37.123828+0000","last_unstale":"2024-04-25T10:03:38.123240+0000","last_undegraded":"2024-04-25T10:03:38.123240+0000","last_fullsized":"2024-04-25T10:03:38.123240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:09.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207449+0000","last_change":"2024-04-25T10:03:37.130211+0000","last_active":"2024-04-25T10:03:38.207449+0000","last_peered":"2024-04-25T10:03:38.207449+0000","last_clean":"2024-04-25T10:03:38.207449+0000","last_became_active":"2024-04-25T10:03:37.129922+0000","last_became_peered":"2024-04-25T10:03:37.129922+0000","last_unstale":"2024-04-25T10:03:38.207449+0000","last_undegraded":"2024-04-25T10:03:38.207449+0000","last_fullsized":"2024-04-25T10:03:38.207449+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:12.937361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128800+0000","last_change":"2024-04-25T10:03:37.137081+0000","last_active":"2024-04-25T10:03:38.128800+0000","last_peered":"2024-04-25T10:03:38.128800+0000","last_clean":"2024-04-25T10:03:38.128800+0000","last_became_active":"2024-04-25T10:03:37.136596+0000","last_became_peered":"2024-04-25T10:03:37.136596+0000","last_unstale":"2024-04-25T10:03:38.128800+0000","last_undegraded":"2024-04-25T10:03:38.128800+0000","last_fullsized":"2024-04-25T10:03:38.128800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:04.198530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122470+0000","last_change":"2024-04-25T10:03:37.119080+0000","last_active":"2024-04-25T10:03:38.122470+0000","last_peered":"2024-04-25T10:03:38.122470+0000","last_clean":"2024-04-25T10:03:38.122470+0000","last_became_active":"2024-04-25T10:03:37.118820+0000","last_became_peered":"2024-04-25T10:03:37.118820+0000","last_unstale":"2024-04-25T10:03:38.122470+0000","last_undegraded":"2024-04-25T10:03:38.122470+0000","last_fullsized":"2024-04-25T10:03:38.122470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:45.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207440+0000","last_change":"2024-04-25T10:03:37.137142+0000","last_active":"2024-04-25T10:03:38.207440+0000","last_peered":"2024-04-25T10:03:38.207440+0000","last_clean":"2024-04-25T10:03:38.207440+0000","last_became_active":"2024-04-25T10:03:37.136836+0000","last_became_peered":"2024-04-25T10:03:37.136836+0000","last_unstale":"2024-04-25T10:03:38.207440+0000","last_undegraded":"2024-04-25T10:03:38.207440+0000","last_fullsized":"2024-04-25T10:03:38.207440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:09.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206131+0000","last_change":"2024-04-25T10:03:37.118976+0000","last_active":"2024-04-25T10:03:38.206131+0000","last_peered":"2024-04-25T10:03:38.206131+0000","last_clean":"2024-04-25T10:03:38.206131+0000","last_became_active":"2024-04-25T10:03:37.118677+0000","last_became_peered":"2024-04-25T10:03:37.118677+0000","last_unstale":"2024-04-25T10:03:38.206131+0000","last_undegraded":"2024-04-25T10:03:38.206131+0000","last_fullsized":"2024-04-25T10:03:38.206131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.448427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.118844+0000","last_change":"2024-04-25T10:03:37.131996+0000","last_active":"2024-04-25T10:03:38.118844+0000","last_peered":"2024-04-25T10:03:38.118844+0000","last_clean":"2024-04-25T10:03:38.118844+0000","last_became_active":"2024-04-25T10:03:37.131626+0000","last_became_peered":"2024-04-25T10:03:37.131626+0000","last_unstale":"2024-04-25T10:03:38.118844+0000","last_undegraded":"2024-04-25T10:03:38.118844+0000","last_fullsized":"2024-04-25T10:03:38.118844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:30.215722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206196+0000","last_change":"2024-04-25T10:03:37.133361+0000","last_active":"2024-04-25T10:03:38.206196+0000","last_peered":"2024-04-25T10:03:38.206196+0000","last_clean":"2024-04-25T10:03:38.206196+0000","last_became_active":"2024-04-25T10:03:37.132367+0000","last_became_peered":"2024-04-25T10:03:37.132367+0000","last_unstale":"2024-04-25T10:03:38.206196+0000","last_undegraded":"2024-04-25T10:03:38.206196+0000","last_fullsized":"2024-04-25T10:03:38.206196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:09.118521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119711+0000","last_change":"2024-04-25T10:03:37.125481+0000","last_active":"2024-04-25T10:03:38.119711+0000","last_peered":"2024-04-25T10:03:38.119711+0000","last_clean":"2024-04-25T10:03:38.119711+0000","last_became_active":"2024-04-25T10:03:37.125143+0000","last_became_peered":"2024-04-25T10:03:37.125143+0000","last_unstale":"2024-04-25T10:03:38.119711+0000","last_undegraded":"2024-04-25T10:03:38.119711+0000","last_fullsized":"2024-04-25T10:03:38.119711+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:41.471971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119706+0000","last_change":"2024-04-25T10:03:37.128357+0000","last_active":"2024-04-25T10:03:38.119706+0000","last_peered":"2024-04-25T10:03:38.119706+0000","last_clean":"2024-04-25T10:03:38.119706+0000","last_became_active":"2024-04-25T10:03:37.128055+0000","last_became_peered":"2024-04-25T10:03:37.128055+0000","last_unstale":"2024-04-25T10:03:38.119706+0000","last_undegraded":"2024-04-25T10:03:38.119706+0000","last_fullsized":"2024-04-25T10:03:38.119706+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:04.587320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207836+0000","last_change":"2024-04-25T10:03:37.135093+0000","last_active":"2024-04-25T10:03:38.207836+0000","last_peered":"2024-04-25T10:03:38.207836+0000","last_clean":"2024-04-25T10:03:38.207836+0000","last_became_active":"2024-04-25T10:03:37.133581+0000","last_became_peered":"2024-04-25T10:03:37.133581+0000","last_unstale":"2024-04-25T10:03:38.207836+0000","last_undegraded":"2024-04-25T10:03:38.207836+0000","last_fullsized":"2024-04-25T10:03:38.207836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.287757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128915+0000","last_change":"2024-04-25T10:03:37.129037+0000","last_active":"2024-04-25T10:03:38.128915+0000","last_peered":"2024-04-25T10:03:38.128915+0000","last_clean":"2024-04-25T10:03:38.128915+0000","last_became_active":"2024-04-25T10:03:37.128676+0000","last_became_peered":"2024-04-25T10:03:37.128676+0000","last_unstale":"2024-04-25T10:03:38.128915+0000","last_undegraded":"2024-04-25T10:03:38.128915+0000","last_fullsized":"2024-04-25T10:03:38.128915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.732827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128161+0000","last_change":"2024-04-25T10:03:37.137258+0000","last_active":"2024-04-25T10:03:38.128161+0000","last_peered":"2024-04-25T10:03:38.128161+0000","last_clean":"2024-04-25T10:03:38.128161+0000","last_became_active":"2024-04-25T10:03:37.136954+0000","last_became_peered":"2024-04-25T10:03:37.136954+0000","last_unstale":"2024-04-25T10:03:38.128161+0000","last_undegraded":"2024-04-25T10:03:38.128161+0000","last_fullsized":"2024-04-25T10:03:38.128161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.800549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128274+0000","last_change":"2024-04-25T10:03:37.133419+0000","last_active":"2024-04-25T10:03:38.128274+0000","last_peered":"2024-04-25T10:03:38.128274+0000","last_clean":"2024-04-25T10:03:38.128274+0000","last_became_active":"2024-04-25T10:03:37.133142+0000","last_became_peered":"2024-04-25T10:03:37.133142+0000","last_unstale":"2024-04-25T10:03:38.128274+0000","last_undegraded":"2024-04-25T10:03:38.128274+0000","last_fullsized":"2024-04-25T10:03:38.128274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.282184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119352+0000","last_change":"2024-04-25T10:03:37.110215+0000","last_active":"2024-04-25T10:03:38.119352+0000","last_peered":"2024-04-25T10:03:38.119352+0000","last_clean":"2024-04-25T10:03:38.119352+0000","last_became_active":"2024-04-25T10:03:37.109942+0000","last_became_peered":"2024-04-25T10:03:37.109942+0000","last_unstale":"2024-04-25T10:03:38.119352+0000","last_undegraded":"2024-04-25T10:03:38.119352+0000","last_fullsized":"2024-04-25T10:03:38.119352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:51.627770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206099+0000","last_change":"2024-04-25T10:03:37.130876+0000","last_active":"2024-04-25T10:03:38.206099+0000","last_peered":"2024-04-25T10:03:38.206099+0000","last_clean":"2024-04-25T10:03:38.206099+0000","last_became_active":"2024-04-25T10:03:37.130587+0000","last_became_peered":"2024-04-25T10:03:37.130587+0000","last_unstale":"2024-04-25T10:03:38.206099+0000","last_undegraded":"2024-04-25T10:03:38.206099+0000","last_fullsized":"2024-04-25T10:03:38.206099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.764236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207490+0000","last_change":"2024-04-25T10:03:37.121862+0000","last_active":"2024-04-25T10:03:38.207490+0000","last_peered":"2024-04-25T10:03:38.207490+0000","last_clean":"2024-04-25T10:03:38.207490+0000","last_became_active":"2024-04-25T10:03:37.121573+0000","last_became_peered":"2024-04-25T10:03:37.121573+0000","last_unstale":"2024-04-25T10:03:38.207490+0000","last_undegraded":"2024-04-25T10:03:38.207490+0000","last_fullsized":"2024-04-25T10:03:38.207490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:25.600517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206607+0000","last_change":"2024-04-25T10:03:37.130321+0000","last_active":"2024-04-25T10:03:38.206607+0000","last_peered":"2024-04-25T10:03:38.206607+0000","last_clean":"2024-04-25T10:03:38.206607+0000","last_became_active":"2024-04-25T10:03:37.129117+0000","last_became_peered":"2024-04-25T10:03:37.129117+0000","last_unstale":"2024-04-25T10:03:38.206607+0000","last_undegraded":"2024-04-25T10:03:38.206607+0000","last_fullsized":"2024-04-25T10:03:38.206607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:39.586705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128553+0000","last_change":"2024-04-25T10:03:37.126657+0000","last_active":"2024-04-25T10:03:38.128553+0000","last_peered":"2024-04-25T10:03:38.128553+0000","last_clean":"2024-04-25T10:03:38.128553+0000","last_became_active":"2024-04-25T10:03:37.126059+0000","last_became_peered":"2024-04-25T10:03:37.126059+0000","last_unstale":"2024-04-25T10:03:38.128553+0000","last_undegraded":"2024-04-25T10:03:38.128553+0000","last_fullsized":"2024-04-25T10:03:38.128553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:36.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207911+0000","last_change":"2024-04-25T10:03:37.131836+0000","last_active":"2024-04-25T10:03:38.207911+0000","last_peered":"2024-04-25T10:03:38.207911+0000","last_clean":"2024-04-25T10:03:38.207911+0000","last_became_active":"2024-04-25T10:03:37.131545+0000","last_became_peered":"2024-04-25T10:03:37.131545+0000","last_unstale":"2024-04-25T10:03:38.207911+0000","last_undegraded":"2024-04-25T10:03:38.207911+0000","last_fullsized":"2024-04-25T10:03:38.207911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:12.992054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119150+0000","last_change":"2024-04-25T10:03:37.112482+0000","last_active":"2024-04-25T10:03:38.119150+0000","last_peered":"2024-04-25T10:03:38.119150+0000","last_clean":"2024-04-25T10:03:38.119150+0000","last_became_active":"2024-04-25T10:03:37.112177+0000","last_became_peered":"2024-04-25T10:03:37.112177+0000","last_unstale":"2024-04-25T10:03:38.119150+0000","last_undegraded":"2024-04-25T10:03:38.119150+0000","last_fullsized":"2024-04-25T10:03:38.119150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:48.221643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119128+0000","last_change":"2024-04-25T10:03:37.114246+0000","last_active":"2024-04-25T10:03:38.119128+0000","last_peered":"2024-04-25T10:03:38.119128+0000","last_clean":"2024-04-25T10:03:38.119128+0000","last_became_active":"2024-04-25T10:03:37.113632+0000","last_became_peered":"2024-04-25T10:03:37.113632+0000","last_unstale":"2024-04-25T10:03:38.119128+0000","last_undegraded":"2024-04-25T10:03:38.119128+0000","last_fullsized":"2024-04-25T10:03:38.119128+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:47.504490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205903+0000","last_change":"2024-04-25T10:03:37.138112+0000","last_active":"2024-04-25T10:03:38.205903+0000","last_peered":"2024-04-25T10:03:38.205903+0000","last_clean":"2024-04-25T10:03:38.205903+0000","last_became_active":"2024-04-25T10:03:37.137565+0000","last_became_peered":"2024-04-25T10:03:37.137565+0000","last_unstale":"2024-04-25T10:03:38.205903+0000","last_undegraded":"2024-04-25T10:03:38.205903+0000","last_fullsized":"2024-04-25T10:03:38.205903+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.886320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206229+0000","last_change":"2024-04-25T10:03:37.132694+0000","last_active":"2024-04-25T10:03:38.206229+0000","last_peered":"2024-04-25T10:03:38.206229+0000","last_clean":"2024-04-25T10:03:38.206229+0000","last_became_active":"2024-04-25T10:03:37.131287+0000","last_became_peered":"2024-04-25T10:03:37.131287+0000","last_unstale":"2024-04-25T10:03:38.206229+0000","last_undegraded":"2024-04-25T10:03:38.206229+0000","last_fullsized":"2024-04-25T10:03:38.206229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:00.454471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119397+0000","last_change":"2024-04-25T10:03:37.115528+0000","last_active":"2024-04-25T10:03:38.119397+0000","last_peered":"2024-04-25T10:03:38.119397+0000","last_clean":"2024-04-25T10:03:38.119397+0000","last_became_active":"2024-04-25T10:03:37.115228+0000","last_became_peered":"2024-04-25T10:03:37.115228+0000","last_unstale":"2024-04-25T10:03:38.119397+0000","last_undegraded":"2024-04-25T10:03:38.119397+0000","last_fullsized":"2024-04-25T10:03:38.119397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:03.968382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207636+0000","last_change":"2024-04-25T10:03:36.084480+0000","last_active":"2024-04-25T10:03:38.207636+0000","last_peered":"2024-04-25T10:03:38.207636+0000","last_clean":"2024-04-25T10:03:38.207636+0000","last_became_active":"2024-04-25T10:03:36.082127+0000","last_became_peered":"2024-04-25T10:03:36.082127+0000","last_unstale":"2024-04-25T10:03:38.207636+0000","last_undegraded":"2024-04-25T10:03:38.207636+0000","last_fullsized":"2024-04-25T10:03:38.207636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121584+0000","last_change":"2024-04-25T10:03:37.127210+0000","last_active":"2024-04-25T10:03:38.121584+0000","last_peered":"2024-04-25T10:03:38.121584+0000","last_clean":"2024-04-25T10:03:38.121584+0000","last_became_active":"2024-04-25T10:03:37.126836+0000","last_became_peered":"2024-04-25T10:03:37.126836+0000","last_unstale":"2024-04-25T10:03:38.121584+0000","last_undegraded":"2024-04-25T10:03:38.121584+0000","last_fullsized":"2024-04-25T10:03:38.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:13.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121818+0000","last_change":"2024-04-25T10:03:36.101720+0000","last_active":"2024-04-25T10:03:38.121818+0000","last_peered":"2024-04-25T10:03:38.121818+0000","last_clean":"2024-04-25T10:03:38.121818+0000","last_became_active":"2024-04-25T10:03:36.101275+0000","last_became_peered":"2024-04-25T10:03:36.101275+0000","last_unstale":"2024-04-25T10:03:38.121818+0000","last_undegraded":"2024-04-25T10:03:38.121818+0000","last_fullsized":"2024-04-25T10:03:38.121818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:21.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119983+0000","last_change":"2024-04-25T10:03:37.122717+0000","last_active":"2024-04-25T10:03:38.119983+0000","last_peered":"2024-04-25T10:03:38.119983+0000","last_clean":"2024-04-25T10:03:38.119983+0000","last_became_active":"2024-04-25T10:03:37.122408+0000","last_became_peered":"2024-04-25T10:03:37.122408+0000","last_unstale":"2024-04-25T10:03:38.119983+0000","last_undegraded":"2024-04-25T10:03:38.119983+0000","last_fullsized":"2024-04-25T10:03:38.119983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.124102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206789+0000","last_change":"2024-04-25T10:03:36.105651+0000","last_active":"2024-04-25T10:03:38.206789+0000","last_peered":"2024-04-25T10:03:38.206789+0000","last_clean":"2024-04-25T10:03:38.206789+0000","last_became_active":"2024-04-25T10:03:36.105203+0000","last_became_peered":"2024-04-25T10:03:36.105203+0000","last_unstale":"2024-04-25T10:03:38.206789+0000","last_undegraded":"2024-04-25T10:03:38.206789+0000","last_fullsized":"2024-04-25T10:03:38.206789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:09.488962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122199+0000","last_change":"2024-04-25T10:03:37.128030+0000","last_active":"2024-04-25T10:03:38.122199+0000","last_peered":"2024-04-25T10:03:38.122199+0000","last_clean":"2024-04-25T10:03:38.122199+0000","last_became_active":"2024-04-25T10:03:37.127757+0000","last_became_peered":"2024-04-25T10:03:37.127757+0000","last_unstale":"2024-04-25T10:03:38.122199+0000","last_undegraded":"2024-04-25T10:03:38.122199+0000","last_fullsized":"2024-04-25T10:03:38.122199+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:36.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206528+0000","last_change":"2024-04-25T10:03:36.095927+0000","last_active":"2024-04-25T10:03:38.206528+0000","last_peered":"2024-04-25T10:03:38.206528+0000","last_clean":"2024-04-25T10:03:38.206528+0000","last_became_active":"2024-04-25T10:03:36.095607+0000","last_became_peered":"2024-04-25T10:03:36.095607+0000","last_unstale":"2024-04-25T10:03:38.206528+0000","last_undegraded":"2024-04-25T10:03:38.206528+0000","last_fullsized":"2024-04-25T10:03:38.206528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:08.106131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206661+0000","last_change":"2024-04-25T10:03:37.115687+0000","last_active":"2024-04-25T10:03:38.206661+0000","last_peered":"2024-04-25T10:03:38.206661+0000","last_clean":"2024-04-25T10:03:38.206661+0000","last_became_active":"2024-04-25T10:03:37.115368+0000","last_became_peered":"2024-04-25T10:03:37.115368+0000","last_unstale":"2024-04-25T10:03:38.206661+0000","last_undegraded":"2024-04-25T10:03:38.206661+0000","last_fullsized":"2024-04-25T10:03:38.206661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:56.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206039+0000","last_change":"2024-04-25T10:03:36.098272+0000","last_active":"2024-04-25T10:03:38.206039+0000","last_peered":"2024-04-25T10:03:38.206039+0000","last_clean":"2024-04-25T10:03:38.206039+0000","last_became_active":"2024-04-25T10:03:36.098002+0000","last_became_peered":"2024-04-25T10:03:36.098002+0000","last_unstale":"2024-04-25T10:03:38.206039+0000","last_undegraded":"2024-04-25T10:03:38.206039+0000","last_fullsized":"2024-04-25T10:03:38.206039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:33.873929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128271+0000","last_change":"2024-04-25T10:03:37.124128+0000","last_active":"2024-04-25T10:03:38.128271+0000","last_peered":"2024-04-25T10:03:38.128271+0000","last_clean":"2024-04-25T10:03:38.128271+0000","last_became_active":"2024-04-25T10:03:37.123683+0000","last_became_peered":"2024-04-25T10:03:37.123683+0000","last_unstale":"2024-04-25T10:03:38.128271+0000","last_undegraded":"2024-04-25T10:03:38.128271+0000","last_fullsized":"2024-04-25T10:03:38.128271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:00.157878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206697+0000","last_change":"2024-04-25T10:03:36.098328+0000","last_active":"2024-04-25T10:03:38.206697+0000","last_peered":"2024-04-25T10:03:38.206697+0000","last_clean":"2024-04-25T10:03:38.206697+0000","last_became_active":"2024-04-25T10:03:36.098059+0000","last_became_peered":"2024-04-25T10:03:36.098059+0000","last_unstale":"2024-04-25T10:03:38.206697+0000","last_undegraded":"2024-04-25T10:03:38.206697+0000","last_fullsized":"2024-04-25T10:03:38.206697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:56.798289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.118934+0000","last_change":"2024-04-25T10:03:37.126428+0000","last_active":"2024-04-25T10:03:38.118934+0000","last_peered":"2024-04-25T10:03:38.118934+0000","last_clean":"2024-04-25T10:03:38.118934+0000","last_became_active":"2024-04-25T10:03:37.126162+0000","last_became_peered":"2024-04-25T10:03:37.126162+0000","last_unstale":"2024-04-25T10:03:38.118934+0000","last_undegraded":"2024-04-25T10:03:38.118934+0000","last_fullsized":"2024-04-25T10:03:38.118934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:31.987695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205709+0000","last_change":"2024-04-25T10:03:36.095557+0000","last_active":"2024-04-25T10:03:38.205709+0000","last_peered":"2024-04-25T10:03:38.205709+0000","last_clean":"2024-04-25T10:03:38.205709+0000","last_became_active":"2024-04-25T10:03:36.095265+0000","last_became_peered":"2024-04-25T10:03:36.095265+0000","last_unstale":"2024-04-25T10:03:38.205709+0000","last_undegraded":"2024-04-25T10:03:38.205709+0000","last_fullsized":"2024-04-25T10:03:38.205709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.385227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121471+0000","last_change":"2024-04-25T10:03:37.125429+0000","last_active":"2024-04-25T10:03:38.121471+0000","last_peered":"2024-04-25T10:03:38.121471+0000","last_clean":"2024-04-25T10:03:38.121471+0000","last_became_active":"2024-04-25T10:03:37.124914+0000","last_became_peered":"2024-04-25T10:03:37.124914+0000","last_unstale":"2024-04-25T10:03:38.121471+0000","last_undegraded":"2024-04-25T10:03:38.121471+0000","last_fullsized":"2024-04-25T10:03:38.121471+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.282184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206409+0000","last_change":"2024-04-25T10:03:36.100796+0000","last_active":"2024-04-25T10:03:38.206409+0000","last_peered":"2024-04-25T10:03:38.206409+0000","last_clean":"2024-04-25T10:03:38.206409+0000","last_became_active":"2024-04-25T10:03:36.098651+0000","last_became_peered":"2024-04-25T10:03:36.098651+0000","last_unstale":"2024-04-25T10:03:38.206409+0000","last_undegraded":"2024-04-25T10:03:38.206409+0000","last_fullsized":"2024-04-25T10:03:38.206409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:44.787754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207045+0000","last_change":"2024-04-25T10:03:37.130899+0000","last_active":"2024-04-25T10:03:38.207045+0000","last_peered":"2024-04-25T10:03:38.207045+0000","last_clean":"2024-04-25T10:03:38.207045+0000","last_became_active":"2024-04-25T10:03:37.130432+0000","last_became_peered":"2024-04-25T10:03:37.130432+0000","last_unstale":"2024-04-25T10:03:38.207045+0000","last_undegraded":"2024-04-25T10:03:38.207045+0000","last_fullsized":"2024-04-25T10:03:38.207045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.079554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207233+0000","last_change":"2024-04-25T10:03:36.098558+0000","last_active":"2024-04-25T10:03:38.207233+0000","last_peered":"2024-04-25T10:03:38.207233+0000","last_clean":"2024-04-25T10:03:38.207233+0000","last_became_active":"2024-04-25T10:03:36.098223+0000","last_became_peered":"2024-04-25T10:03:36.098223+0000","last_unstale":"2024-04-25T10:03:38.207233+0000","last_undegraded":"2024-04-25T10:03:38.207233+0000","last_fullsized":"2024-04-25T10:03:38.207233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:51.024456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207091+0000","last_change":"2024-04-25T10:03:37.129554+0000","last_active":"2024-04-25T10:03:38.207091+0000","last_peered":"2024-04-25T10:03:38.207091+0000","last_clean":"2024-04-25T10:03:38.207091+0000","last_became_active":"2024-04-25T10:03:37.126357+0000","last_became_peered":"2024-04-25T10:03:37.126357+0000","last_unstale":"2024-04-25T10:03:38.207091+0000","last_undegraded":"2024-04-25T10:03:38.207091+0000","last_fullsized":"2024-04-25T10:03:38.207091+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:36.646130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119632+0000","last_change":"2024-04-25T10:03:36.104706+0000","last_active":"2024-04-25T10:03:38.119632+0000","last_peered":"2024-04-25T10:03:38.119632+0000","last_clean":"2024-04-25T10:03:38.119632+0000","last_became_active":"2024-04-25T10:03:36.104438+0000","last_became_peered":"2024-04-25T10:03:36.104438+0000","last_unstale":"2024-04-25T10:03:38.119632+0000","last_undegraded":"2024-04-25T10:03:38.119632+0000","last_fullsized":"2024-04-25T10:03:38.119632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:55.720436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207030+0000","last_change":"2024-04-25T10:03:37.138930+0000","last_active":"2024-04-25T10:03:38.207030+0000","last_peered":"2024-04-25T10:03:38.207030+0000","last_clean":"2024-04-25T10:03:38.207030+0000","last_became_active":"2024-04-25T10:03:37.138643+0000","last_became_peered":"2024-04-25T10:03:37.138643+0000","last_unstale":"2024-04-25T10:03:38.207030+0000","last_undegraded":"2024-04-25T10:03:38.207030+0000","last_fullsized":"2024-04-25T10:03:38.207030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.100516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119359+0000","last_change":"2024-04-25T10:03:36.103445+0000","last_active":"2024-04-25T10:03:38.119359+0000","last_peered":"2024-04-25T10:03:38.119359+0000","last_clean":"2024-04-25T10:03:38.119359+0000","last_became_active":"2024-04-25T10:03:36.103130+0000","last_became_peered":"2024-04-25T10:03:36.103130+0000","last_unstale":"2024-04-25T10:03:38.119359+0000","last_undegraded":"2024-04-25T10:03:38.119359+0000","last_fullsized":"2024-04-25T10:03:38.119359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:32.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.208396+0000","last_change":"2024-04-25T10:03:37.133150+0000","last_active":"2024-04-25T10:03:38.208396+0000","last_peered":"2024-04-25T10:03:38.208396+0000","last_clean":"2024-04-25T10:03:38.208396+0000","last_became_active":"2024-04-25T10:03:37.132839+0000","last_became_peered":"2024-04-25T10:03:37.132839+0000","last_unstale":"2024-04-25T10:03:38.208396+0000","last_undegraded":"2024-04-25T10:03:38.208396+0000","last_fullsized":"2024-04-25T10:03:38.208396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.448427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207294+0000","last_change":"2024-04-25T10:03:36.098166+0000","last_active":"2024-04-25T10:03:38.207294+0000","last_peered":"2024-04-25T10:03:38.207294+0000","last_clean":"2024-04-25T10:03:38.207294+0000","last_became_active":"2024-04-25T10:03:36.097774+0000","last_became_peered":"2024-04-25T10:03:36.097774+0000","last_unstale":"2024-04-25T10:03:38.207294+0000","last_undegraded":"2024-04-25T10:03:38.207294+0000","last_fullsized":"2024-04-25T10:03:38.207294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:32.511676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207141+0000","last_change":"2024-04-25T10:03:37.119860+0000","last_active":"2024-04-25T10:03:38.207141+0000","last_peered":"2024-04-25T10:03:38.207141+0000","last_clean":"2024-04-25T10:03:38.207141+0000","last_became_active":"2024-04-25T10:03:37.118103+0000","last_became_peered":"2024-04-25T10:03:37.118103+0000","last_unstale":"2024-04-25T10:03:38.207141+0000","last_undegraded":"2024-04-25T10:03:38.207141+0000","last_fullsized":"2024-04-25T10:03:38.207141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:06.878310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207283+0000","last_change":"2024-04-25T10:03:36.104310+0000","last_active":"2024-04-25T10:03:38.207283+0000","last_peered":"2024-04-25T10:03:38.207283+0000","last_clean":"2024-04-25T10:03:38.207283+0000","last_became_active":"2024-04-25T10:03:36.103982+0000","last_became_peered":"2024-04-25T10:03:36.103982+0000","last_unstale":"2024-04-25T10:03:38.207283+0000","last_undegraded":"2024-04-25T10:03:38.207283+0000","last_fullsized":"2024-04-25T10:03:38.207283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:33.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120603+0000","last_change":"2024-04-25T10:03:37.125640+0000","last_active":"2024-04-25T10:03:38.120603+0000","last_peered":"2024-04-25T10:03:38.120603+0000","last_clean":"2024-04-25T10:03:38.120603+0000","last_became_active":"2024-04-25T10:03:37.125345+0000","last_became_peered":"2024-04-25T10:03:37.125345+0000","last_unstale":"2024-04-25T10:03:38.120603+0000","last_undegraded":"2024-04-25T10:03:38.120603+0000","last_fullsized":"2024-04-25T10:03:38.120603+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.411564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120905+0000","last_change":"2024-04-25T10:03:36.099035+0000","last_active":"2024-04-25T10:03:38.120905+0000","last_peered":"2024-04-25T10:03:38.120905+0000","last_clean":"2024-04-25T10:03:38.120905+0000","last_became_active":"2024-04-25T10:03:36.097751+0000","last_became_peered":"2024-04-25T10:03:36.097751+0000","last_unstale":"2024-04-25T10:03:38.120905+0000","last_undegraded":"2024-04-25T10:03:38.120905+0000","last_fullsized":"2024-04-25T10:03:38.120905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:11.924970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128491+0000","last_change":"2024-04-25T10:03:37.120699+0000","last_active":"2024-04-25T10:03:38.128491+0000","last_peered":"2024-04-25T10:03:38.128491+0000","last_clean":"2024-04-25T10:03:38.128491+0000","last_became_active":"2024-04-25T10:03:37.120413+0000","last_became_peered":"2024-04-25T10:03:37.120413+0000","last_unstale":"2024-04-25T10:03:38.128491+0000","last_undegraded":"2024-04-25T10:03:38.128491+0000","last_fullsized":"2024-04-25T10:03:38.128491+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:07.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119044+0000","last_change":"2024-04-25T10:03:36.094470+0000","last_active":"2024-04-25T10:03:38.119044+0000","last_peered":"2024-04-25T10:03:38.119044+0000","last_clean":"2024-04-25T10:03:38.119044+0000","last_became_active":"2024-04-25T10:03:36.094193+0000","last_became_peered":"2024-04-25T10:03:36.094193+0000","last_unstale":"2024-04-25T10:03:38.119044+0000","last_undegraded":"2024-04-25T10:03:38.119044+0000","last_fullsized":"2024-04-25T10:03:38.119044+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:43.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.123441+0000","last_change":"2024-04-25T10:03:37.124806+0000","last_active":"2024-04-25T10:03:38.123441+0000","last_peered":"2024-04-25T10:03:38.123441+0000","last_clean":"2024-04-25T10:03:38.123441+0000","last_became_active":"2024-04-25T10:03:37.124505+0000","last_became_peered":"2024-04-25T10:03:37.124505+0000","last_unstale":"2024-04-25T10:03:38.123441+0000","last_undegraded":"2024-04-25T10:03:38.123441+0000","last_fullsized":"2024-04-25T10:03:38.123441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:12.937361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205625+0000","last_change":"2024-04-25T10:03:36.091559+0000","last_active":"2024-04-25T10:03:38.205625+0000","last_peered":"2024-04-25T10:03:38.205625+0000","last_clean":"2024-04-25T10:03:38.205625+0000","last_became_active":"2024-04-25T10:03:36.091201+0000","last_became_peered":"2024-04-25T10:03:36.091201+0000","last_unstale":"2024-04-25T10:03:38.205625+0000","last_undegraded":"2024-04-25T10:03:38.205625+0000","last_fullsized":"2024-04-25T10:03:38.205625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:44.681404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121351+0000","last_change":"2024-04-25T10:03:37.127691+0000","last_active":"2024-04-25T10:03:38.121351+0000","last_peered":"2024-04-25T10:03:38.121351+0000","last_clean":"2024-04-25T10:03:38.121351+0000","last_became_active":"2024-04-25T10:03:37.124821+0000","last_became_peered":"2024-04-25T10:03:37.124821+0000","last_unstale":"2024-04-25T10:03:38.121351+0000","last_undegraded":"2024-04-25T10:03:38.121351+0000","last_fullsized":"2024-04-25T10:03:38.121351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:00.157878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.208138+0000","last_change":"2024-04-25T10:03:36.096846+0000","last_active":"2024-04-25T10:03:38.208138+0000","last_peered":"2024-04-25T10:03:38.208138+0000","last_clean":"2024-04-25T10:03:38.208138+0000","last_became_active":"2024-04-25T10:03:36.096537+0000","last_became_peered":"2024-04-25T10:03:36.096537+0000","last_unstale":"2024-04-25T10:03:38.208138+0000","last_undegraded":"2024-04-25T10:03:38.208138+0000","last_fullsized":"2024-04-25T10:03:38.208138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:11.422492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.208360+0000","last_change":"2024-04-25T10:03:37.137465+0000","last_active":"2024-04-25T10:03:38.208360+0000","last_peered":"2024-04-25T10:03:38.208360+0000","last_clean":"2024-04-25T10:03:38.208360+0000","last_became_active":"2024-04-25T10:03:37.137196+0000","last_became_peered":"2024-04-25T10:03:37.137196+0000","last_unstale":"2024-04-25T10:03:38.208360+0000","last_undegraded":"2024-04-25T10:03:38.208360+0000","last_fullsized":"2024-04-25T10:03:38.208360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:18.938228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119852+0000","last_change":"2024-04-25T10:03:36.104201+0000","last_active":"2024-04-25T10:03:38.119852+0000","last_peered":"2024-04-25T10:03:38.119852+0000","last_clean":"2024-04-25T10:03:38.119852+0000","last_became_active":"2024-04-25T10:03:36.103744+0000","last_became_peered":"2024-04-25T10:03:36.103744+0000","last_unstale":"2024-04-25T10:03:38.119852+0000","last_undegraded":"2024-04-25T10:03:38.119852+0000","last_fullsized":"2024-04-25T10:03:38.119852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:52.298743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207877+0000","last_change":"2024-04-25T10:03:37.136929+0000","last_active":"2024-04-25T10:03:38.207877+0000","last_peered":"2024-04-25T10:03:38.207877+0000","last_clean":"2024-04-25T10:03:38.207877+0000","last_became_active":"2024-04-25T10:03:37.136641+0000","last_became_peered":"2024-04-25T10:03:37.136641+0000","last_unstale":"2024-04-25T10:03:38.207877+0000","last_undegraded":"2024-04-25T10:03:38.207877+0000","last_fullsized":"2024-04-25T10:03:38.207877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:02.282914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122649+0000","last_change":"2024-04-25T10:03:36.092770+0000","last_active":"2024-04-25T10:03:38.122649+0000","last_peered":"2024-04-25T10:03:38.122649+0000","last_clean":"2024-04-25T10:03:38.122649+0000","last_became_active":"2024-04-25T10:03:36.092466+0000","last_became_peered":"2024-04-25T10:03:36.092466+0000","last_unstale":"2024-04-25T10:03:38.122649+0000","last_undegraded":"2024-04-25T10:03:38.122649+0000","last_fullsized":"2024-04-25T10:03:38.122649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:11.422492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207925+0000","last_change":"2024-04-25T10:03:37.133883+0000","last_active":"2024-04-25T10:03:38.207925+0000","last_peered":"2024-04-25T10:03:38.207925+0000","last_clean":"2024-04-25T10:03:38.207925+0000","last_became_active":"2024-04-25T10:03:37.133559+0000","last_became_peered":"2024-04-25T10:03:37.133559+0000","last_unstale":"2024-04-25T10:03:38.207925+0000","last_undegraded":"2024-04-25T10:03:38.207925+0000","last_fullsized":"2024-04-25T10:03:38.207925+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.411564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205377+0000","last_change":"2024-04-25T10:03:36.094541+0000","last_active":"2024-04-25T10:03:38.205377+0000","last_peered":"2024-04-25T10:03:38.205377+0000","last_clean":"2024-04-25T10:03:38.205377+0000","last_became_active":"2024-04-25T10:03:36.094179+0000","last_became_peered":"2024-04-25T10:03:36.094179+0000","last_unstale":"2024-04-25T10:03:38.205377+0000","last_undegraded":"2024-04-25T10:03:38.205377+0000","last_fullsized":"2024-04-25T10:03:38.205377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:21.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119041+0000","last_change":"2024-04-25T10:03:37.117486+0000","last_active":"2024-04-25T10:03:38.119041+0000","last_peered":"2024-04-25T10:03:38.119041+0000","last_clean":"2024-04-25T10:03:38.119041+0000","last_became_active":"2024-04-25T10:03:37.117194+0000","last_became_peered":"2024-04-25T10:03:37.117194+0000","last_unstale":"2024-04-25T10:03:38.119041+0000","last_undegraded":"2024-04-25T10:03:38.119041+0000","last_fullsized":"2024-04-25T10:03:38.119041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:26.125538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207054+0000","last_change":"2024-04-25T10:03:36.097091+0000","last_active":"2024-04-25T10:03:38.207054+0000","last_peered":"2024-04-25T10:03:38.207054+0000","last_clean":"2024-04-25T10:03:38.207054+0000","last_became_active":"2024-04-25T10:03:36.096820+0000","last_became_peered":"2024-04-25T10:03:36.096820+0000","last_unstale":"2024-04-25T10:03:38.207054+0000","last_undegraded":"2024-04-25T10:03:38.207054+0000","last_fullsized":"2024-04-25T10:03:38.207054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:17.925837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205441+0000","last_change":"2024-04-25T10:03:37.139337+0000","last_active":"2024-04-25T10:03:38.205441+0000","last_peered":"2024-04-25T10:03:38.205441+0000","last_clean":"2024-04-25T10:03:38.205441+0000","last_became_active":"2024-04-25T10:03:37.137532+0000","last_became_peered":"2024-04-25T10:03:37.137532+0000","last_unstale":"2024-04-25T10:03:38.205441+0000","last_undegraded":"2024-04-25T10:03:38.205441+0000","last_fullsized":"2024-04-25T10:03:38.205441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:56.223185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121088+0000","last_change":"2024-04-25T10:03:36.100566+0000","last_active":"2024-04-25T10:03:38.121088+0000","last_peered":"2024-04-25T10:03:38.121088+0000","last_clean":"2024-04-25T10:03:38.121088+0000","last_became_active":"2024-04-25T10:03:36.100186+0000","last_became_peered":"2024-04-25T10:03:36.100186+0000","last_unstale":"2024-04-25T10:03:38.121088+0000","last_undegraded":"2024-04-25T10:03:38.121088+0000","last_fullsized":"2024-04-25T10:03:38.121088+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:01.270523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128434+0000","last_change":"2024-04-25T10:03:37.132271+0000","last_active":"2024-04-25T10:03:38.128434+0000","last_peered":"2024-04-25T10:03:38.128434+0000","last_clean":"2024-04-25T10:03:38.128434+0000","last_became_active":"2024-04-25T10:03:37.131960+0000","last_became_peered":"2024-04-25T10:03:37.131960+0000","last_unstale":"2024-04-25T10:03:38.128434+0000","last_undegraded":"2024-04-25T10:03:38.128434+0000","last_fullsized":"2024-04-25T10:03:38.128434+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:13.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205956+0000","last_change":"2024-04-25T10:03:36.096821+0000","last_active":"2024-04-25T10:03:38.205956+0000","last_peered":"2024-04-25T10:03:38.205956+0000","last_clean":"2024-04-25T10:03:38.205956+0000","last_became_active":"2024-04-25T10:03:36.096490+0000","last_became_peered":"2024-04-25T10:03:36.096490+0000","last_unstale":"2024-04-25T10:03:38.205956+0000","last_undegraded":"2024-04-25T10:03:38.205956+0000","last_fullsized":"2024-04-25T10:03:38.205956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:55.146532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207050+0000","last_change":"2024-04-25T10:03:37.123190+0000","last_active":"2024-04-25T10:03:38.207050+0000","last_peered":"2024-04-25T10:03:38.207050+0000","last_clean":"2024-04-25T10:03:38.207050+0000","last_became_active":"2024-04-25T10:03:37.122906+0000","last_became_peered":"2024-04-25T10:03:37.122906+0000","last_unstale":"2024-04-25T10:03:38.207050+0000","last_undegraded":"2024-04-25T10:03:38.207050+0000","last_fullsized":"2024-04-25T10:03:38.207050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.154712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120280+0000","last_change":"2024-04-25T10:03:36.103569+0000","last_active":"2024-04-25T10:03:38.120280+0000","last_peered":"2024-04-25T10:03:38.120280+0000","last_clean":"2024-04-25T10:03:38.120280+0000","last_became_active":"2024-04-25T10:03:36.103282+0000","last_became_peered":"2024-04-25T10:03:36.103282+0000","last_unstale":"2024-04-25T10:03:38.120280+0000","last_undegraded":"2024-04-25T10:03:38.120280+0000","last_fullsized":"2024-04-25T10:03:38.120280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122773+0000","last_change":"2024-04-25T10:03:37.125608+0000","last_active":"2024-04-25T10:03:38.122773+0000","last_peered":"2024-04-25T10:03:38.122773+0000","last_clean":"2024-04-25T10:03:38.122773+0000","last_became_active":"2024-04-25T10:03:37.125241+0000","last_became_peered":"2024-04-25T10:03:37.125241+0000","last_unstale":"2024-04-25T10:03:38.122773+0000","last_undegraded":"2024-04-25T10:03:38.122773+0000","last_fullsized":"2024-04-25T10:03:38.122773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.886320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119810+0000","last_change":"2024-04-25T10:03:36.107003+0000","last_active":"2024-04-25T10:03:38.119810+0000","last_peered":"2024-04-25T10:03:38.119810+0000","last_clean":"2024-04-25T10:03:38.119810+0000","last_became_active":"2024-04-25T10:03:36.106571+0000","last_became_peered":"2024-04-25T10:03:36.106571+0000","last_unstale":"2024-04-25T10:03:38.119810+0000","last_undegraded":"2024-04-25T10:03:38.119810+0000","last_fullsized":"2024-04-25T10:03:38.119810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:35.650886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128779+0000","last_change":"2024-04-25T10:03:37.130908+0000","last_active":"2024-04-25T10:03:38.128779+0000","last_peered":"2024-04-25T10:03:38.128779+0000","last_clean":"2024-04-25T10:03:38.128779+0000","last_became_active":"2024-04-25T10:03:37.130557+0000","last_became_peered":"2024-04-25T10:03:37.130557+0000","last_unstale":"2024-04-25T10:03:38.128779+0000","last_undegraded":"2024-04-25T10:03:38.128779+0000","last_fullsized":"2024-04-25T10:03:38.128779+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:44.620298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.123032+0000","last_change":"2024-04-25T10:03:36.096706+0000","last_active":"2024-04-25T10:03:38.123032+0000","last_peered":"2024-04-25T10:03:38.123032+0000","last_clean":"2024-04-25T10:03:38.123032+0000","last_became_active":"2024-04-25T10:03:36.095723+0000","last_became_peered":"2024-04-25T10:03:36.095723+0000","last_unstale":"2024-04-25T10:03:38.123032+0000","last_undegraded":"2024-04-25T10:03:38.123032+0000","last_fullsized":"2024-04-25T10:03:38.123032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:12.071145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207819+0000","last_change":"2024-04-25T10:03:37.120789+0000","last_active":"2024-04-25T10:03:38.207819+0000","last_peered":"2024-04-25T10:03:38.207819+0000","last_clean":"2024-04-25T10:03:38.207819+0000","last_became_active":"2024-04-25T10:03:37.120515+0000","last_became_peered":"2024-04-25T10:03:37.120515+0000","last_unstale":"2024-04-25T10:03:38.207819+0000","last_undegraded":"2024-04-25T10:03:38.207819+0000","last_fullsized":"2024-04-25T10:03:38.207819+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:53.342578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122819+0000","last_change":"2024-04-25T10:03:36.104229+0000","last_active":"2024-04-25T10:03:38.122819+0000","last_peered":"2024-04-25T10:03:38.122819+0000","last_clean":"2024-04-25T10:03:38.122819+0000","last_became_active":"2024-04-25T10:03:36.103025+0000","last_became_peered":"2024-04-25T10:03:36.103025+0000","last_unstale":"2024-04-25T10:03:38.122819+0000","last_undegraded":"2024-04-25T10:03:38.122819+0000","last_fullsized":"2024-04-25T10:03:38.122819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:54.628483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122140+0000","last_change":"2024-04-25T10:03:37.132482+0000","last_active":"2024-04-25T10:03:38.122140+0000","last_peered":"2024-04-25T10:03:38.122140+0000","last_clean":"2024-04-25T10:03:38.122140+0000","last_became_active":"2024-04-25T10:03:37.131368+0000","last_became_peered":"2024-04-25T10:03:37.131368+0000","last_unstale":"2024-04-25T10:03:38.122140+0000","last_undegraded":"2024-04-25T10:03:38.122140+0000","last_fullsized":"2024-04-25T10:03:38.122140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:29.886643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207633+0000","last_change":"2024-04-25T10:03:36.102517+0000","last_active":"2024-04-25T10:03:38.207633+0000","last_peered":"2024-04-25T10:03:38.207633+0000","last_clean":"2024-04-25T10:03:38.207633+0000","last_became_active":"2024-04-25T10:03:36.102006+0000","last_became_peered":"2024-04-25T10:03:36.102006+0000","last_unstale":"2024-04-25T10:03:38.207633+0000","last_undegraded":"2024-04-25T10:03:38.207633+0000","last_fullsized":"2024-04-25T10:03:38.207633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:22.682970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120665+0000","last_change":"2024-04-25T10:03:37.122086+0000","last_active":"2024-04-25T10:03:38.120665+0000","last_peered":"2024-04-25T10:03:38.120665+0000","last_clean":"2024-04-25T10:03:38.120665+0000","last_became_active":"2024-04-25T10:03:37.121786+0000","last_became_peered":"2024-04-25T10:03:37.121786+0000","last_unstale":"2024-04-25T10:03:38.120665+0000","last_undegraded":"2024-04-25T10:03:38.120665+0000","last_fullsized":"2024-04-25T10:03:38.120665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.177091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128507+0000","last_change":"2024-04-25T10:03:36.102101+0000","last_active":"2024-04-25T10:03:38.128507+0000","last_peered":"2024-04-25T10:03:38.128507+0000","last_clean":"2024-04-25T10:03:38.128507+0000","last_became_active":"2024-04-25T10:03:36.101810+0000","last_became_peered":"2024-04-25T10:03:36.101810+0000","last_unstale":"2024-04-25T10:03:38.128507+0000","last_undegraded":"2024-04-25T10:03:38.128507+0000","last_fullsized":"2024-04-25T10:03:38.128507+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:01.270523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119305+0000","last_change":"2024-04-25T10:03:37.116902+0000","last_active":"2024-04-25T10:03:38.119305+0000","last_peered":"2024-04-25T10:03:38.119305+0000","last_clean":"2024-04-25T10:03:38.119305+0000","last_became_active":"2024-04-25T10:03:37.116410+0000","last_became_peered":"2024-04-25T10:03:37.116410+0000","last_unstale":"2024-04-25T10:03:38.119305+0000","last_undegraded":"2024-04-25T10:03:38.119305+0000","last_fullsized":"2024-04-25T10:03:38.119305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:40.483272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119192+0000","last_change":"2024-04-25T10:03:36.097695+0000","last_active":"2024-04-25T10:03:38.119192+0000","last_peered":"2024-04-25T10:03:38.119192+0000","last_clean":"2024-04-25T10:03:38.119192+0000","last_became_active":"2024-04-25T10:03:36.097467+0000","last_became_peered":"2024-04-25T10:03:36.097467+0000","last_unstale":"2024-04-25T10:03:38.119192+0000","last_undegraded":"2024-04-25T10:03:38.119192+0000","last_fullsized":"2024-04-25T10:03:38.119192+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:03.186140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128651+0000","last_change":"2024-04-25T10:03:37.124824+0000","last_active":"2024-04-25T10:03:38.128651+0000","last_peered":"2024-04-25T10:03:38.128651+0000","last_clean":"2024-04-25T10:03:38.128651+0000","last_became_active":"2024-04-25T10:03:37.124402+0000","last_became_peered":"2024-04-25T10:03:37.124402+0000","last_unstale":"2024-04-25T10:03:38.128651+0000","last_undegraded":"2024-04-25T10:03:38.128651+0000","last_fullsized":"2024-04-25T10:03:38.128651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.144462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206932+0000","last_change":"2024-04-25T10:03:36.090034+0000","last_active":"2024-04-25T10:03:38.206932+0000","last_peered":"2024-04-25T10:03:38.206932+0000","last_clean":"2024-04-25T10:03:38.206932+0000","last_became_active":"2024-04-25T10:03:36.089136+0000","last_became_peered":"2024-04-25T10:03:36.089136+0000","last_unstale":"2024-04-25T10:03:38.206932+0000","last_undegraded":"2024-04-25T10:03:38.206932+0000","last_fullsized":"2024-04-25T10:03:38.206932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119734+0000","last_change":"2024-04-25T10:03:37.111090+0000","last_active":"2024-04-25T10:03:38.119734+0000","last_peered":"2024-04-25T10:03:38.119734+0000","last_clean":"2024-04-25T10:03:38.119734+0000","last_became_active":"2024-04-25T10:03:37.110832+0000","last_became_peered":"2024-04-25T10:03:37.110832+0000","last_unstale":"2024-04-25T10:03:38.119734+0000","last_undegraded":"2024-04-25T10:03:38.119734+0000","last_fullsized":"2024-04-25T10:03:38.119734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:15.856124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128468+0000","last_change":"2024-04-25T10:03:36.103532+0000","last_active":"2024-04-25T10:03:38.128468+0000","last_peered":"2024-04-25T10:03:38.128468+0000","last_clean":"2024-04-25T10:03:38.128468+0000","last_became_active":"2024-04-25T10:03:36.102663+0000","last_became_peered":"2024-04-25T10:03:36.102663+0000","last_unstale":"2024-04-25T10:03:38.128468+0000","last_undegraded":"2024-04-25T10:03:38.128468+0000","last_fullsized":"2024-04-25T10:03:38.128468+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:11.924970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206411+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:03:38.206411+0000","last_peered":"2024-04-25T10:03:38.206411+0000","last_clean":"2024-04-25T10:03:38.206411+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:03:38.206411+0000","last_undegraded":"2024-04-25T10:03:38.206411+0000","last_fullsized":"2024-04-25T10:03:38.206411+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":785452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":194592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":471040,"data_stored":196696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":784892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509466,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":462848,"data_stored":194594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":196238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:46.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.875+0000 7f71eda00640 1 Processor -- start 2024-04-25T10:03:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.875+0000 7f71eda00640 1 -- start start 2024-04-25T10:03:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.876+0000 7f71eda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71e8143d30 0x7f71e815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.876+0000 7f71eda00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71e80ceea0 0x7f71e80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.876+0000 7f71eda00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e80b4880 con 0x7f71e80b3ae0 2024-04-25T10:03:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.876+0000 7f71eda00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e80b45a0 con 0x7f71e80ceea0 2024-04-25T10:03:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.876+0000 7f71eda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e815e2c0 con 0x7f71e8143d30 2024-04-25T10:03:46.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.877+0000 7f71df400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f71e80b3ae0 0x7f71e80ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43898/0 (socket says 172.21.15.5:43898) 2024-04-25T10:03:46.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.877+0000 7f71df400640 1 -- 172.21.15.5:0/2672714629 learned_addr learned my addr 172.21.15.5:0/2672714629 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.877+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082937725 0 0) 0x7f71e80b4880 con 0x7f71e80b3ae0 2024-04-25T10:03:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.877+0000 7f71ed000640 1 --2- 172.21.15.5:0/2672714629 >> v2:172.21.15.80:3300/0 conn(0x7f71e80ceea0 0x7f71e80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.877+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71cc003520 con 0x7f71e80b3ae0 2024-04-25T10:03:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.877+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71e8143d30 0x7f71e815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4190227401 0 0) 0x7f71cc003520 con 0x7f71e80b3ae0 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71e8143d30 msgr2=0x7f71e815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71e8143d30 0x7f71e815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 >> v2:172.21.15.80:3300/0 conn(0x7f71e80ceea0 msgr2=0x7f71e80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 --2- 172.21.15.5:0/2672714629 >> v2:172.21.15.80:3300/0 conn(0x7f71e80ceea0 0x7f71e80b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e815e5a0 con 0x7f71e80b3ae0 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f71d0001670 con 0x7f71e80b3ae0 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 >> v1:172.21.15.5:6789/0 conn(0x7f71e80b3ae0 legacy=0x7f71e80ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc0039b0 0x7f71cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0043a0 0x7f71cc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc004fa0 0x7f71cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e80b45a0 con 0x7f71cc004fa0 2024-04-25T10:03:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e815e2c0 con 0x7f71cc0043a0 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc00bf30 con 0x7f71cc0039b0 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71e8143d30 0x7f71e815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.878+0000 7f71ed000640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc004fa0 0x7f71cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71df400640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0043a0 0x7f71cc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc0039b0 0x7f71cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71ed000640 1 -- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc0039b0 msgr2=0x7f71cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71ed000640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc0039b0 0x7f71cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71ed000640 1 -- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0043a0 msgr2=0x7f71cc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71ed000640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0043a0 0x7f71cc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71ed000640 1 -- 172.21.15.5:0/2672714629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc003520 con 0x7f71cc004fa0 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71df400640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71cc0043a0 0x7f71cc004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.879+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc0039b0 0x7f71cc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:46.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.880+0000 7f71ed000640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc004fa0 0x7f71cc00b9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ac0 tx=0x7f71cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a42c71aa3327686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.880+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0036070 con 0x7f71cc004fa0 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.880+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0031400 con 0x7f71cc004fa0 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.880+0000 7f71e7400640 1 -- 172.21.15.5:0/2672714629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0009690 con 0x7f71cc004fa0 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71eda00640 1 -- 172.21.15.5:0/2672714629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc004fa0 msgr2=0x7f71cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71eda00640 1 --2- 172.21.15.5:0/2672714629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71cc004fa0 0x7f71cc00b9c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ac0 tx=0x7f71cc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71ed000640 1 -- 172.21.15.5:0/2672714629 reap_dead start 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71eda00640 1 -- 172.21.15.5:0/2672714629 shutdown_connections 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71eda00640 1 -- 172.21.15.5:0/2672714629 >> 172.21.15.5:0/2672714629 conn(0x7f71e8066f20 msgr2=0x7f71e806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71eda00640 1 -- 172.21.15.5:0/2672714629 shutdown_connections 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.881+0000 7f71eda00640 1 -- 172.21.15.5:0/2672714629 wait complete. 2024-04-25T10:03:46.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.883+0000 7f71eda00640 1 Processor -- start 2024-04-25T10:03:46.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.883+0000 7f71eda00640 1 -- start start 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71eda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc004fa0 0x7f71e81f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71eda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71e8142780 0x7f71e8141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71eda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 0x7f71e80a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71eda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8144290 con 0x7f71e80a8de0 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71eda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8144110 con 0x7f71e8142780 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71eda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e806e430 con 0x7f71cc004fa0 2024-04-25T10:03:46.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 0x7f71e80a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.884+0000 7f71ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 0x7f71e80a9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49476/0 (socket says 172.21.15.5:49476) 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 -- 172.21.15.5:0/2296229438 learned_addr learned my addr 172.21.15.5:0/2296229438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71df400640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71e8142780 0x7f71e8141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc004fa0 0x7f71e81f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 -- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc004fa0 msgr2=0x7f71e81f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc004fa0 0x7f71e81f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 -- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71e8142780 msgr2=0x7f71e8141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71e8142780 0x7f71e8141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 -- 172.21.15.5:0/2296229438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80a8580 con 0x7f71e80a8de0 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc004fa0 0x7f71e81f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71df400640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71e8142780 0x7f71e8141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.885+0000 7f71ed000640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 0x7f71e80a9260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ac0 tx=0x7f71e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.886+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0036040 con 0x7f71e80a8de0 2024-04-25T10:03:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.886+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0041070 con 0x7f71e80a8de0 2024-04-25T10:03:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.886+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0009690 con 0x7f71e80a8de0 2024-04-25T10:03:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.886+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e81f75c0 con 0x7f71e80a8de0 2024-04-25T10:03:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.886+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e815ad20 con 0x7f71e80a8de0 2024-04-25T10:03:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.887+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e80a8a60 con 0x7f71e80a8de0 2024-04-25T10:03:46.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.888+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71e0039070 con 0x7f71e80a8de0 2024-04-25T10:03:46.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.889+0000 7f71e4c00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71c406b4d0 0x7f71c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.890+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71e00a9a30 con 0x7f71e80a8de0 2024-04-25T10:03:46.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.890+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71c406b4d0 0x7f71c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.891+0000 7f71e7e00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71c406b4d0 0x7f71c406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009540 tx=0x7f71d800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:46.894+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71e80a8a60 con 0x7f71e80a8de0 2024-04-25T10:03:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.089+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71e8068f60 con 0x7f71c406b4d0 2024-04-25T10:03:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.109+0000 7f71e4c00640 1 -- 172.21.15.5:0/2296229438 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251845 (secure 0 0 0) 0x7f71e8068f60 con 0x7f71c406b4d0 2024-04-25T10:03:47.117 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.113+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71c406b4d0 msgr2=0x7f71c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.113+0000 7f71eda00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71c406b4d0 0x7f71c406d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009540 tx=0x7f71d800b040 comp rx=0 tx=0).stop 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.113+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 msgr2=0x7f71e80a9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.113+0000 7f71eda00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 0x7f71e80a9260 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ac0 tx=0x7f71e0004730 comp rx=0 tx=0).stop 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 shutdown_connections 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e80a8de0 0x7f71e80a9260 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71c406b4d0 0x7f71c406d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71e8142780 0x7f71e8141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 --2- 172.21.15.5:0/2296229438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71cc004fa0 0x7f71e81f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 >> 172.21.15.5:0/2296229438 conn(0x7f71e8066f20 msgr2=0x7f71e806e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 shutdown_connections 2024-04-25T10:03:47.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.114+0000 7f71eda00640 1 -- 172.21.15.5:0/2296229438 wait complete. 2024-04-25T10:03:47.137 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-04-25T10:03:46.650877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237596,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737124,"statfs":{"total":773094113280,"available":772850814976,"internally_reserved":0,"allocated":4890624,"data_stored":2741572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.563843"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207506+0000","last_change":"2024-04-25T10:03:37.134249+0000","last_active":"2024-04-25T10:03:38.207506+0000","last_peered":"2024-04-25T10:03:38.207506+0000","last_clean":"2024-04-25T10:03:38.207506+0000","last_became_active":"2024-04-25T10:03:37.133378+0000","last_became_peered":"2024-04-25T10:03:37.133378+0000","last_unstale":"2024-04-25T10:03:38.207506+0000","last_undegraded":"2024-04-25T10:03:38.207506+0000","last_fullsized":"2024-04-25T10:03:38.207506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:20.585105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122100+0000","last_change":"2024-04-25T10:03:37.118315+0000","last_active":"2024-04-25T10:03:38.122100+0000","last_peered":"2024-04-25T10:03:38.122100+0000","last_clean":"2024-04-25T10:03:38.122100+0000","last_became_active":"2024-04-25T10:03:37.116557+0000","last_became_peered":"2024-04-25T10:03:37.116557+0000","last_unstale":"2024-04-25T10:03:38.122100+0000","last_undegraded":"2024-04-25T10:03:38.122100+0000","last_fullsized":"2024-04-25T10:03:38.122100+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:07.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206496+0000","last_change":"2024-04-25T10:03:37.133516+0000","last_active":"2024-04-25T10:03:38.206496+0000","last_peered":"2024-04-25T10:03:38.206496+0000","last_clean":"2024-04-25T10:03:38.206496+0000","last_became_active":"2024-04-25T10:03:37.132370+0000","last_became_peered":"2024-04-25T10:03:37.132370+0000","last_unstale":"2024-04-25T10:03:38.206496+0000","last_undegraded":"2024-04-25T10:03:38.206496+0000","last_fullsized":"2024-04-25T10:03:38.206496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:33.505774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206619+0000","last_change":"2024-04-25T10:03:37.131079+0000","last_active":"2024-04-25T10:03:38.206619+0000","last_peered":"2024-04-25T10:03:38.206619+0000","last_clean":"2024-04-25T10:03:38.206619+0000","last_became_active":"2024-04-25T10:03:37.130786+0000","last_became_peered":"2024-04-25T10:03:37.130786+0000","last_unstale":"2024-04-25T10:03:38.206619+0000","last_undegraded":"2024-04-25T10:03:38.206619+0000","last_fullsized":"2024-04-25T10:03:38.206619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:11.177455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122957+0000","last_change":"2024-04-25T10:03:37.126163+0000","last_active":"2024-04-25T10:03:38.122957+0000","last_peered":"2024-04-25T10:03:38.122957+0000","last_clean":"2024-04-25T10:03:38.122957+0000","last_became_active":"2024-04-25T10:03:37.125858+0000","last_became_peered":"2024-04-25T10:03:37.125858+0000","last_unstale":"2024-04-25T10:03:38.122957+0000","last_undegraded":"2024-04-25T10:03:38.122957+0000","last_fullsized":"2024-04-25T10:03:38.122957+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:09.118521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120997+0000","last_change":"2024-04-25T10:03:37.123236+0000","last_active":"2024-04-25T10:03:38.120997+0000","last_peered":"2024-04-25T10:03:38.120997+0000","last_clean":"2024-04-25T10:03:38.120997+0000","last_became_active":"2024-04-25T10:03:37.122935+0000","last_became_peered":"2024-04-25T10:03:37.122935+0000","last_unstale":"2024-04-25T10:03:38.120997+0000","last_undegraded":"2024-04-25T10:03:38.120997+0000","last_fullsized":"2024-04-25T10:03:38.120997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.800549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205727+0000","last_change":"2024-04-25T10:03:37.135271+0000","last_active":"2024-04-25T10:03:38.205727+0000","last_peered":"2024-04-25T10:03:38.205727+0000","last_clean":"2024-04-25T10:03:38.205727+0000","last_became_active":"2024-04-25T10:03:37.134973+0000","last_became_peered":"2024-04-25T10:03:37.134973+0000","last_unstale":"2024-04-25T10:03:38.205727+0000","last_undegraded":"2024-04-25T10:03:38.205727+0000","last_fullsized":"2024-04-25T10:03:38.205727+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:45.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205696+0000","last_change":"2024-04-25T10:03:37.133051+0000","last_active":"2024-04-25T10:03:38.205696+0000","last_peered":"2024-04-25T10:03:38.205696+0000","last_clean":"2024-04-25T10:03:38.205696+0000","last_became_active":"2024-04-25T10:03:37.131855+0000","last_became_peered":"2024-04-25T10:03:37.131855+0000","last_unstale":"2024-04-25T10:03:38.205696+0000","last_undegraded":"2024-04-25T10:03:38.205696+0000","last_fullsized":"2024-04-25T10:03:38.205696+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:52.525576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.123240+0000","last_change":"2024-04-25T10:03:37.124102+0000","last_active":"2024-04-25T10:03:38.123240+0000","last_peered":"2024-04-25T10:03:38.123240+0000","last_clean":"2024-04-25T10:03:38.123240+0000","last_became_active":"2024-04-25T10:03:37.123828+0000","last_became_peered":"2024-04-25T10:03:37.123828+0000","last_unstale":"2024-04-25T10:03:38.123240+0000","last_undegraded":"2024-04-25T10:03:38.123240+0000","last_fullsized":"2024-04-25T10:03:38.123240+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:09.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207449+0000","last_change":"2024-04-25T10:03:37.130211+0000","last_active":"2024-04-25T10:03:38.207449+0000","last_peered":"2024-04-25T10:03:38.207449+0000","last_clean":"2024-04-25T10:03:38.207449+0000","last_became_active":"2024-04-25T10:03:37.129922+0000","last_became_peered":"2024-04-25T10:03:37.129922+0000","last_unstale":"2024-04-25T10:03:38.207449+0000","last_undegraded":"2024-04-25T10:03:38.207449+0000","last_fullsized":"2024-04-25T10:03:38.207449+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:12.937361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128800+0000","last_change":"2024-04-25T10:03:37.137081+0000","last_active":"2024-04-25T10:03:38.128800+0000","last_peered":"2024-04-25T10:03:38.128800+0000","last_clean":"2024-04-25T10:03:38.128800+0000","last_became_active":"2024-04-25T10:03:37.136596+0000","last_became_peered":"2024-04-25T10:03:37.136596+0000","last_unstale":"2024-04-25T10:03:38.128800+0000","last_undegraded":"2024-04-25T10:03:38.128800+0000","last_fullsized":"2024-04-25T10:03:38.128800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:04.198530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122470+0000","last_change":"2024-04-25T10:03:37.119080+0000","last_active":"2024-04-25T10:03:38.122470+0000","last_peered":"2024-04-25T10:03:38.122470+0000","last_clean":"2024-04-25T10:03:38.122470+0000","last_became_active":"2024-04-25T10:03:37.118820+0000","last_became_peered":"2024-04-25T10:03:37.118820+0000","last_unstale":"2024-04-25T10:03:38.122470+0000","last_undegraded":"2024-04-25T10:03:38.122470+0000","last_fullsized":"2024-04-25T10:03:38.122470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:45.693794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207440+0000","last_change":"2024-04-25T10:03:37.137142+0000","last_active":"2024-04-25T10:03:38.207440+0000","last_peered":"2024-04-25T10:03:38.207440+0000","last_clean":"2024-04-25T10:03:38.207440+0000","last_became_active":"2024-04-25T10:03:37.136836+0000","last_became_peered":"2024-04-25T10:03:37.136836+0000","last_unstale":"2024-04-25T10:03:38.207440+0000","last_undegraded":"2024-04-25T10:03:38.207440+0000","last_fullsized":"2024-04-25T10:03:38.207440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:09.559578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206131+0000","last_change":"2024-04-25T10:03:37.118976+0000","last_active":"2024-04-25T10:03:38.206131+0000","last_peered":"2024-04-25T10:03:38.206131+0000","last_clean":"2024-04-25T10:03:38.206131+0000","last_became_active":"2024-04-25T10:03:37.118677+0000","last_became_peered":"2024-04-25T10:03:37.118677+0000","last_unstale":"2024-04-25T10:03:38.206131+0000","last_undegraded":"2024-04-25T10:03:38.206131+0000","last_fullsized":"2024-04-25T10:03:38.206131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.448427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.118844+0000","last_change":"2024-04-25T10:03:37.131996+0000","last_active":"2024-04-25T10:03:38.118844+0000","last_peered":"2024-04-25T10:03:38.118844+0000","last_clean":"2024-04-25T10:03:38.118844+0000","last_became_active":"2024-04-25T10:03:37.131626+0000","last_became_peered":"2024-04-25T10:03:37.131626+0000","last_unstale":"2024-04-25T10:03:38.118844+0000","last_undegraded":"2024-04-25T10:03:38.118844+0000","last_fullsized":"2024-04-25T10:03:38.118844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:30.215722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206196+0000","last_change":"2024-04-25T10:03:37.133361+0000","last_active":"2024-04-25T10:03:38.206196+0000","last_peered":"2024-04-25T10:03:38.206196+0000","last_clean":"2024-04-25T10:03:38.206196+0000","last_became_active":"2024-04-25T10:03:37.132367+0000","last_became_peered":"2024-04-25T10:03:37.132367+0000","last_unstale":"2024-04-25T10:03:38.206196+0000","last_undegraded":"2024-04-25T10:03:38.206196+0000","last_fullsized":"2024-04-25T10:03:38.206196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:09.118521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119711+0000","last_change":"2024-04-25T10:03:37.125481+0000","last_active":"2024-04-25T10:03:38.119711+0000","last_peered":"2024-04-25T10:03:38.119711+0000","last_clean":"2024-04-25T10:03:38.119711+0000","last_became_active":"2024-04-25T10:03:37.125143+0000","last_became_peered":"2024-04-25T10:03:37.125143+0000","last_unstale":"2024-04-25T10:03:38.119711+0000","last_undegraded":"2024-04-25T10:03:38.119711+0000","last_fullsized":"2024-04-25T10:03:38.119711+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:41.471971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119706+0000","last_change":"2024-04-25T10:03:37.128357+0000","last_active":"2024-04-25T10:03:38.119706+0000","last_peered":"2024-04-25T10:03:38.119706+0000","last_clean":"2024-04-25T10:03:38.119706+0000","last_became_active":"2024-04-25T10:03:37.128055+0000","last_became_peered":"2024-04-25T10:03:37.128055+0000","last_unstale":"2024-04-25T10:03:38.119706+0000","last_undegraded":"2024-04-25T10:03:38.119706+0000","last_fullsized":"2024-04-25T10:03:38.119706+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:04.587320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207836+0000","last_change":"2024-04-25T10:03:37.135093+0000","last_active":"2024-04-25T10:03:38.207836+0000","last_peered":"2024-04-25T10:03:38.207836+0000","last_clean":"2024-04-25T10:03:38.207836+0000","last_became_active":"2024-04-25T10:03:37.133581+0000","last_became_peered":"2024-04-25T10:03:37.133581+0000","last_unstale":"2024-04-25T10:03:38.207836+0000","last_undegraded":"2024-04-25T10:03:38.207836+0000","last_fullsized":"2024-04-25T10:03:38.207836+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:52.287757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128915+0000","last_change":"2024-04-25T10:03:37.129037+0000","last_active":"2024-04-25T10:03:38.128915+0000","last_peered":"2024-04-25T10:03:38.128915+0000","last_clean":"2024-04-25T10:03:38.128915+0000","last_became_active":"2024-04-25T10:03:37.128676+0000","last_became_peered":"2024-04-25T10:03:37.128676+0000","last_unstale":"2024-04-25T10:03:38.128915+0000","last_undegraded":"2024-04-25T10:03:38.128915+0000","last_fullsized":"2024-04-25T10:03:38.128915+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:56.732827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128161+0000","last_change":"2024-04-25T10:03:37.137258+0000","last_active":"2024-04-25T10:03:38.128161+0000","last_peered":"2024-04-25T10:03:38.128161+0000","last_clean":"2024-04-25T10:03:38.128161+0000","last_became_active":"2024-04-25T10:03:37.136954+0000","last_became_peered":"2024-04-25T10:03:37.136954+0000","last_unstale":"2024-04-25T10:03:38.128161+0000","last_undegraded":"2024-04-25T10:03:38.128161+0000","last_fullsized":"2024-04-25T10:03:38.128161+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.800549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128274+0000","last_change":"2024-04-25T10:03:37.133419+0000","last_active":"2024-04-25T10:03:38.128274+0000","last_peered":"2024-04-25T10:03:38.128274+0000","last_clean":"2024-04-25T10:03:38.128274+0000","last_became_active":"2024-04-25T10:03:37.133142+0000","last_became_peered":"2024-04-25T10:03:37.133142+0000","last_unstale":"2024-04-25T10:03:38.128274+0000","last_undegraded":"2024-04-25T10:03:38.128274+0000","last_fullsized":"2024-04-25T10:03:38.128274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.282184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119352+0000","last_change":"2024-04-25T10:03:37.110215+0000","last_active":"2024-04-25T10:03:38.119352+0000","last_peered":"2024-04-25T10:03:38.119352+0000","last_clean":"2024-04-25T10:03:38.119352+0000","last_became_active":"2024-04-25T10:03:37.109942+0000","last_became_peered":"2024-04-25T10:03:37.109942+0000","last_unstale":"2024-04-25T10:03:38.119352+0000","last_undegraded":"2024-04-25T10:03:38.119352+0000","last_fullsized":"2024-04-25T10:03:38.119352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:51.627770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206099+0000","last_change":"2024-04-25T10:03:37.130876+0000","last_active":"2024-04-25T10:03:38.206099+0000","last_peered":"2024-04-25T10:03:38.206099+0000","last_clean":"2024-04-25T10:03:38.206099+0000","last_became_active":"2024-04-25T10:03:37.130587+0000","last_became_peered":"2024-04-25T10:03:37.130587+0000","last_unstale":"2024-04-25T10:03:38.206099+0000","last_undegraded":"2024-04-25T10:03:38.206099+0000","last_fullsized":"2024-04-25T10:03:38.206099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:57.764236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207490+0000","last_change":"2024-04-25T10:03:37.121862+0000","last_active":"2024-04-25T10:03:38.207490+0000","last_peered":"2024-04-25T10:03:38.207490+0000","last_clean":"2024-04-25T10:03:38.207490+0000","last_became_active":"2024-04-25T10:03:37.121573+0000","last_became_peered":"2024-04-25T10:03:37.121573+0000","last_unstale":"2024-04-25T10:03:38.207490+0000","last_undegraded":"2024-04-25T10:03:38.207490+0000","last_fullsized":"2024-04-25T10:03:38.207490+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:25.600517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206607+0000","last_change":"2024-04-25T10:03:37.130321+0000","last_active":"2024-04-25T10:03:38.206607+0000","last_peered":"2024-04-25T10:03:38.206607+0000","last_clean":"2024-04-25T10:03:38.206607+0000","last_became_active":"2024-04-25T10:03:37.129117+0000","last_became_peered":"2024-04-25T10:03:37.129117+0000","last_unstale":"2024-04-25T10:03:38.206607+0000","last_undegraded":"2024-04-25T10:03:38.206607+0000","last_fullsized":"2024-04-25T10:03:38.206607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:39.586705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128553+0000","last_change":"2024-04-25T10:03:37.126657+0000","last_active":"2024-04-25T10:03:38.128553+0000","last_peered":"2024-04-25T10:03:38.128553+0000","last_clean":"2024-04-25T10:03:38.128553+0000","last_became_active":"2024-04-25T10:03:37.126059+0000","last_became_peered":"2024-04-25T10:03:37.126059+0000","last_unstale":"2024-04-25T10:03:38.128553+0000","last_undegraded":"2024-04-25T10:03:38.128553+0000","last_fullsized":"2024-04-25T10:03:38.128553+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:36.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207911+0000","last_change":"2024-04-25T10:03:37.131836+0000","last_active":"2024-04-25T10:03:38.207911+0000","last_peered":"2024-04-25T10:03:38.207911+0000","last_clean":"2024-04-25T10:03:38.207911+0000","last_became_active":"2024-04-25T10:03:37.131545+0000","last_became_peered":"2024-04-25T10:03:37.131545+0000","last_unstale":"2024-04-25T10:03:38.207911+0000","last_undegraded":"2024-04-25T10:03:38.207911+0000","last_fullsized":"2024-04-25T10:03:38.207911+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:12.992054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119150+0000","last_change":"2024-04-25T10:03:37.112482+0000","last_active":"2024-04-25T10:03:38.119150+0000","last_peered":"2024-04-25T10:03:38.119150+0000","last_clean":"2024-04-25T10:03:38.119150+0000","last_became_active":"2024-04-25T10:03:37.112177+0000","last_became_peered":"2024-04-25T10:03:37.112177+0000","last_unstale":"2024-04-25T10:03:38.119150+0000","last_undegraded":"2024-04-25T10:03:38.119150+0000","last_fullsized":"2024-04-25T10:03:38.119150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:48.221643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119128+0000","last_change":"2024-04-25T10:03:37.114246+0000","last_active":"2024-04-25T10:03:38.119128+0000","last_peered":"2024-04-25T10:03:38.119128+0000","last_clean":"2024-04-25T10:03:38.119128+0000","last_became_active":"2024-04-25T10:03:37.113632+0000","last_became_peered":"2024-04-25T10:03:37.113632+0000","last_unstale":"2024-04-25T10:03:38.119128+0000","last_undegraded":"2024-04-25T10:03:38.119128+0000","last_fullsized":"2024-04-25T10:03:38.119128+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:47.504490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205903+0000","last_change":"2024-04-25T10:03:37.138112+0000","last_active":"2024-04-25T10:03:38.205903+0000","last_peered":"2024-04-25T10:03:38.205903+0000","last_clean":"2024-04-25T10:03:38.205903+0000","last_became_active":"2024-04-25T10:03:37.137565+0000","last_became_peered":"2024-04-25T10:03:37.137565+0000","last_unstale":"2024-04-25T10:03:38.205903+0000","last_undegraded":"2024-04-25T10:03:38.205903+0000","last_fullsized":"2024-04-25T10:03:38.205903+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.886320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206229+0000","last_change":"2024-04-25T10:03:37.132694+0000","last_active":"2024-04-25T10:03:38.206229+0000","last_peered":"2024-04-25T10:03:38.206229+0000","last_clean":"2024-04-25T10:03:38.206229+0000","last_became_active":"2024-04-25T10:03:37.131287+0000","last_became_peered":"2024-04-25T10:03:37.131287+0000","last_unstale":"2024-04-25T10:03:38.206229+0000","last_undegraded":"2024-04-25T10:03:38.206229+0000","last_fullsized":"2024-04-25T10:03:38.206229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:00.454471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119397+0000","last_change":"2024-04-25T10:03:37.115528+0000","last_active":"2024-04-25T10:03:38.119397+0000","last_peered":"2024-04-25T10:03:38.119397+0000","last_clean":"2024-04-25T10:03:38.119397+0000","last_became_active":"2024-04-25T10:03:37.115228+0000","last_became_peered":"2024-04-25T10:03:37.115228+0000","last_unstale":"2024-04-25T10:03:38.119397+0000","last_undegraded":"2024-04-25T10:03:38.119397+0000","last_fullsized":"2024-04-25T10:03:38.119397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:03.968382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207636+0000","last_change":"2024-04-25T10:03:36.084480+0000","last_active":"2024-04-25T10:03:38.207636+0000","last_peered":"2024-04-25T10:03:38.207636+0000","last_clean":"2024-04-25T10:03:38.207636+0000","last_became_active":"2024-04-25T10:03:36.082127+0000","last_became_peered":"2024-04-25T10:03:36.082127+0000","last_unstale":"2024-04-25T10:03:38.207636+0000","last_undegraded":"2024-04-25T10:03:38.207636+0000","last_fullsized":"2024-04-25T10:03:38.207636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121584+0000","last_change":"2024-04-25T10:03:37.127210+0000","last_active":"2024-04-25T10:03:38.121584+0000","last_peered":"2024-04-25T10:03:38.121584+0000","last_clean":"2024-04-25T10:03:38.121584+0000","last_became_active":"2024-04-25T10:03:37.126836+0000","last_became_peered":"2024-04-25T10:03:37.126836+0000","last_unstale":"2024-04-25T10:03:38.121584+0000","last_undegraded":"2024-04-25T10:03:38.121584+0000","last_fullsized":"2024-04-25T10:03:38.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:13.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121818+0000","last_change":"2024-04-25T10:03:36.101720+0000","last_active":"2024-04-25T10:03:38.121818+0000","last_peered":"2024-04-25T10:03:38.121818+0000","last_clean":"2024-04-25T10:03:38.121818+0000","last_became_active":"2024-04-25T10:03:36.101275+0000","last_became_peered":"2024-04-25T10:03:36.101275+0000","last_unstale":"2024-04-25T10:03:38.121818+0000","last_undegraded":"2024-04-25T10:03:38.121818+0000","last_fullsized":"2024-04-25T10:03:38.121818+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:21.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119983+0000","last_change":"2024-04-25T10:03:37.122717+0000","last_active":"2024-04-25T10:03:38.119983+0000","last_peered":"2024-04-25T10:03:38.119983+0000","last_clean":"2024-04-25T10:03:38.119983+0000","last_became_active":"2024-04-25T10:03:37.122408+0000","last_became_peered":"2024-04-25T10:03:37.122408+0000","last_unstale":"2024-04-25T10:03:38.119983+0000","last_undegraded":"2024-04-25T10:03:38.119983+0000","last_fullsized":"2024-04-25T10:03:38.119983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:36.124102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206789+0000","last_change":"2024-04-25T10:03:36.105651+0000","last_active":"2024-04-25T10:03:38.206789+0000","last_peered":"2024-04-25T10:03:38.206789+0000","last_clean":"2024-04-25T10:03:38.206789+0000","last_became_active":"2024-04-25T10:03:36.105203+0000","last_became_peered":"2024-04-25T10:03:36.105203+0000","last_unstale":"2024-04-25T10:03:38.206789+0000","last_undegraded":"2024-04-25T10:03:38.206789+0000","last_fullsized":"2024-04-25T10:03:38.206789+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:09.488962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122199+0000","last_change":"2024-04-25T10:03:37.128030+0000","last_active":"2024-04-25T10:03:38.122199+0000","last_peered":"2024-04-25T10:03:38.122199+0000","last_clean":"2024-04-25T10:03:38.122199+0000","last_became_active":"2024-04-25T10:03:37.127757+0000","last_became_peered":"2024-04-25T10:03:37.127757+0000","last_unstale":"2024-04-25T10:03:38.122199+0000","last_undegraded":"2024-04-25T10:03:38.122199+0000","last_fullsized":"2024-04-25T10:03:38.122199+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:36.663277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206528+0000","last_change":"2024-04-25T10:03:36.095927+0000","last_active":"2024-04-25T10:03:38.206528+0000","last_peered":"2024-04-25T10:03:38.206528+0000","last_clean":"2024-04-25T10:03:38.206528+0000","last_became_active":"2024-04-25T10:03:36.095607+0000","last_became_peered":"2024-04-25T10:03:36.095607+0000","last_unstale":"2024-04-25T10:03:38.206528+0000","last_undegraded":"2024-04-25T10:03:38.206528+0000","last_fullsized":"2024-04-25T10:03:38.206528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:08.106131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206661+0000","last_change":"2024-04-25T10:03:37.115687+0000","last_active":"2024-04-25T10:03:38.206661+0000","last_peered":"2024-04-25T10:03:38.206661+0000","last_clean":"2024-04-25T10:03:38.206661+0000","last_became_active":"2024-04-25T10:03:37.115368+0000","last_became_peered":"2024-04-25T10:03:37.115368+0000","last_unstale":"2024-04-25T10:03:38.206661+0000","last_undegraded":"2024-04-25T10:03:38.206661+0000","last_fullsized":"2024-04-25T10:03:38.206661+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:56.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206039+0000","last_change":"2024-04-25T10:03:36.098272+0000","last_active":"2024-04-25T10:03:38.206039+0000","last_peered":"2024-04-25T10:03:38.206039+0000","last_clean":"2024-04-25T10:03:38.206039+0000","last_became_active":"2024-04-25T10:03:36.098002+0000","last_became_peered":"2024-04-25T10:03:36.098002+0000","last_unstale":"2024-04-25T10:03:38.206039+0000","last_undegraded":"2024-04-25T10:03:38.206039+0000","last_fullsized":"2024-04-25T10:03:38.206039+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:33.873929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128271+0000","last_change":"2024-04-25T10:03:37.124128+0000","last_active":"2024-04-25T10:03:38.128271+0000","last_peered":"2024-04-25T10:03:38.128271+0000","last_clean":"2024-04-25T10:03:38.128271+0000","last_became_active":"2024-04-25T10:03:37.123683+0000","last_became_peered":"2024-04-25T10:03:37.123683+0000","last_unstale":"2024-04-25T10:03:38.128271+0000","last_undegraded":"2024-04-25T10:03:38.128271+0000","last_fullsized":"2024-04-25T10:03:38.128271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:00.157878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206697+0000","last_change":"2024-04-25T10:03:36.098328+0000","last_active":"2024-04-25T10:03:38.206697+0000","last_peered":"2024-04-25T10:03:38.206697+0000","last_clean":"2024-04-25T10:03:38.206697+0000","last_became_active":"2024-04-25T10:03:36.098059+0000","last_became_peered":"2024-04-25T10:03:36.098059+0000","last_unstale":"2024-04-25T10:03:38.206697+0000","last_undegraded":"2024-04-25T10:03:38.206697+0000","last_fullsized":"2024-04-25T10:03:38.206697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:56.798289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.118934+0000","last_change":"2024-04-25T10:03:37.126428+0000","last_active":"2024-04-25T10:03:38.118934+0000","last_peered":"2024-04-25T10:03:38.118934+0000","last_clean":"2024-04-25T10:03:38.118934+0000","last_became_active":"2024-04-25T10:03:37.126162+0000","last_became_peered":"2024-04-25T10:03:37.126162+0000","last_unstale":"2024-04-25T10:03:38.118934+0000","last_undegraded":"2024-04-25T10:03:38.118934+0000","last_fullsized":"2024-04-25T10:03:38.118934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:31.987695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205709+0000","last_change":"2024-04-25T10:03:36.095557+0000","last_active":"2024-04-25T10:03:38.205709+0000","last_peered":"2024-04-25T10:03:38.205709+0000","last_clean":"2024-04-25T10:03:38.205709+0000","last_became_active":"2024-04-25T10:03:36.095265+0000","last_became_peered":"2024-04-25T10:03:36.095265+0000","last_unstale":"2024-04-25T10:03:38.205709+0000","last_undegraded":"2024-04-25T10:03:38.205709+0000","last_fullsized":"2024-04-25T10:03:38.205709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:49.385227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121471+0000","last_change":"2024-04-25T10:03:37.125429+0000","last_active":"2024-04-25T10:03:38.121471+0000","last_peered":"2024-04-25T10:03:38.121471+0000","last_clean":"2024-04-25T10:03:38.121471+0000","last_became_active":"2024-04-25T10:03:37.124914+0000","last_became_peered":"2024-04-25T10:03:37.124914+0000","last_unstale":"2024-04-25T10:03:38.121471+0000","last_undegraded":"2024-04-25T10:03:38.121471+0000","last_fullsized":"2024-04-25T10:03:38.121471+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:00.282184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206409+0000","last_change":"2024-04-25T10:03:36.100796+0000","last_active":"2024-04-25T10:03:38.206409+0000","last_peered":"2024-04-25T10:03:38.206409+0000","last_clean":"2024-04-25T10:03:38.206409+0000","last_became_active":"2024-04-25T10:03:36.098651+0000","last_became_peered":"2024-04-25T10:03:36.098651+0000","last_unstale":"2024-04-25T10:03:38.206409+0000","last_undegraded":"2024-04-25T10:03:38.206409+0000","last_fullsized":"2024-04-25T10:03:38.206409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:44.787754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207045+0000","last_change":"2024-04-25T10:03:37.130899+0000","last_active":"2024-04-25T10:03:38.207045+0000","last_peered":"2024-04-25T10:03:38.207045+0000","last_clean":"2024-04-25T10:03:38.207045+0000","last_became_active":"2024-04-25T10:03:37.130432+0000","last_became_peered":"2024-04-25T10:03:37.130432+0000","last_unstale":"2024-04-25T10:03:38.207045+0000","last_undegraded":"2024-04-25T10:03:38.207045+0000","last_fullsized":"2024-04-25T10:03:38.207045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:56.079554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207233+0000","last_change":"2024-04-25T10:03:36.098558+0000","last_active":"2024-04-25T10:03:38.207233+0000","last_peered":"2024-04-25T10:03:38.207233+0000","last_clean":"2024-04-25T10:03:38.207233+0000","last_became_active":"2024-04-25T10:03:36.098223+0000","last_became_peered":"2024-04-25T10:03:36.098223+0000","last_unstale":"2024-04-25T10:03:38.207233+0000","last_undegraded":"2024-04-25T10:03:38.207233+0000","last_fullsized":"2024-04-25T10:03:38.207233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:51.024456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207091+0000","last_change":"2024-04-25T10:03:37.129554+0000","last_active":"2024-04-25T10:03:38.207091+0000","last_peered":"2024-04-25T10:03:38.207091+0000","last_clean":"2024-04-25T10:03:38.207091+0000","last_became_active":"2024-04-25T10:03:37.126357+0000","last_became_peered":"2024-04-25T10:03:37.126357+0000","last_unstale":"2024-04-25T10:03:38.207091+0000","last_undegraded":"2024-04-25T10:03:38.207091+0000","last_fullsized":"2024-04-25T10:03:38.207091+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:36.646130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119632+0000","last_change":"2024-04-25T10:03:36.104706+0000","last_active":"2024-04-25T10:03:38.119632+0000","last_peered":"2024-04-25T10:03:38.119632+0000","last_clean":"2024-04-25T10:03:38.119632+0000","last_became_active":"2024-04-25T10:03:36.104438+0000","last_became_peered":"2024-04-25T10:03:36.104438+0000","last_unstale":"2024-04-25T10:03:38.119632+0000","last_undegraded":"2024-04-25T10:03:38.119632+0000","last_fullsized":"2024-04-25T10:03:38.119632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:55.720436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207030+0000","last_change":"2024-04-25T10:03:37.138930+0000","last_active":"2024-04-25T10:03:38.207030+0000","last_peered":"2024-04-25T10:03:38.207030+0000","last_clean":"2024-04-25T10:03:38.207030+0000","last_became_active":"2024-04-25T10:03:37.138643+0000","last_became_peered":"2024-04-25T10:03:37.138643+0000","last_unstale":"2024-04-25T10:03:38.207030+0000","last_undegraded":"2024-04-25T10:03:38.207030+0000","last_fullsized":"2024-04-25T10:03:38.207030+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:22.100516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119359+0000","last_change":"2024-04-25T10:03:36.103445+0000","last_active":"2024-04-25T10:03:38.119359+0000","last_peered":"2024-04-25T10:03:38.119359+0000","last_clean":"2024-04-25T10:03:38.119359+0000","last_became_active":"2024-04-25T10:03:36.103130+0000","last_became_peered":"2024-04-25T10:03:36.103130+0000","last_unstale":"2024-04-25T10:03:38.119359+0000","last_undegraded":"2024-04-25T10:03:38.119359+0000","last_fullsized":"2024-04-25T10:03:38.119359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:32.132072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.208396+0000","last_change":"2024-04-25T10:03:37.133150+0000","last_active":"2024-04-25T10:03:38.208396+0000","last_peered":"2024-04-25T10:03:38.208396+0000","last_clean":"2024-04-25T10:03:38.208396+0000","last_became_active":"2024-04-25T10:03:37.132839+0000","last_became_peered":"2024-04-25T10:03:37.132839+0000","last_unstale":"2024-04-25T10:03:38.208396+0000","last_undegraded":"2024-04-25T10:03:38.208396+0000","last_fullsized":"2024-04-25T10:03:38.208396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.448427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207294+0000","last_change":"2024-04-25T10:03:36.098166+0000","last_active":"2024-04-25T10:03:38.207294+0000","last_peered":"2024-04-25T10:03:38.207294+0000","last_clean":"2024-04-25T10:03:38.207294+0000","last_became_active":"2024-04-25T10:03:36.097774+0000","last_became_peered":"2024-04-25T10:03:36.097774+0000","last_unstale":"2024-04-25T10:03:38.207294+0000","last_undegraded":"2024-04-25T10:03:38.207294+0000","last_fullsized":"2024-04-25T10:03:38.207294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:32.511676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207141+0000","last_change":"2024-04-25T10:03:37.119860+0000","last_active":"2024-04-25T10:03:38.207141+0000","last_peered":"2024-04-25T10:03:38.207141+0000","last_clean":"2024-04-25T10:03:38.207141+0000","last_became_active":"2024-04-25T10:03:37.118103+0000","last_became_peered":"2024-04-25T10:03:37.118103+0000","last_unstale":"2024-04-25T10:03:38.207141+0000","last_undegraded":"2024-04-25T10:03:38.207141+0000","last_fullsized":"2024-04-25T10:03:38.207141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:06.878310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207283+0000","last_change":"2024-04-25T10:03:36.104310+0000","last_active":"2024-04-25T10:03:38.207283+0000","last_peered":"2024-04-25T10:03:38.207283+0000","last_clean":"2024-04-25T10:03:38.207283+0000","last_became_active":"2024-04-25T10:03:36.103982+0000","last_became_peered":"2024-04-25T10:03:36.103982+0000","last_unstale":"2024-04-25T10:03:38.207283+0000","last_undegraded":"2024-04-25T10:03:38.207283+0000","last_fullsized":"2024-04-25T10:03:38.207283+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:33.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120603+0000","last_change":"2024-04-25T10:03:37.125640+0000","last_active":"2024-04-25T10:03:38.120603+0000","last_peered":"2024-04-25T10:03:38.120603+0000","last_clean":"2024-04-25T10:03:38.120603+0000","last_became_active":"2024-04-25T10:03:37.125345+0000","last_became_peered":"2024-04-25T10:03:37.125345+0000","last_unstale":"2024-04-25T10:03:38.120603+0000","last_undegraded":"2024-04-25T10:03:38.120603+0000","last_fullsized":"2024-04-25T10:03:38.120603+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.411564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120905+0000","last_change":"2024-04-25T10:03:36.099035+0000","last_active":"2024-04-25T10:03:38.120905+0000","last_peered":"2024-04-25T10:03:38.120905+0000","last_clean":"2024-04-25T10:03:38.120905+0000","last_became_active":"2024-04-25T10:03:36.097751+0000","last_became_peered":"2024-04-25T10:03:36.097751+0000","last_unstale":"2024-04-25T10:03:38.120905+0000","last_undegraded":"2024-04-25T10:03:38.120905+0000","last_fullsized":"2024-04-25T10:03:38.120905+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:11.924970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128491+0000","last_change":"2024-04-25T10:03:37.120699+0000","last_active":"2024-04-25T10:03:38.128491+0000","last_peered":"2024-04-25T10:03:38.128491+0000","last_clean":"2024-04-25T10:03:38.128491+0000","last_became_active":"2024-04-25T10:03:37.120413+0000","last_became_peered":"2024-04-25T10:03:37.120413+0000","last_unstale":"2024-04-25T10:03:38.128491+0000","last_undegraded":"2024-04-25T10:03:38.128491+0000","last_fullsized":"2024-04-25T10:03:38.128491+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:07.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119044+0000","last_change":"2024-04-25T10:03:36.094470+0000","last_active":"2024-04-25T10:03:38.119044+0000","last_peered":"2024-04-25T10:03:38.119044+0000","last_clean":"2024-04-25T10:03:38.119044+0000","last_became_active":"2024-04-25T10:03:36.094193+0000","last_became_peered":"2024-04-25T10:03:36.094193+0000","last_unstale":"2024-04-25T10:03:38.119044+0000","last_undegraded":"2024-04-25T10:03:38.119044+0000","last_fullsized":"2024-04-25T10:03:38.119044+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:43.607907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.123441+0000","last_change":"2024-04-25T10:03:37.124806+0000","last_active":"2024-04-25T10:03:38.123441+0000","last_peered":"2024-04-25T10:03:38.123441+0000","last_clean":"2024-04-25T10:03:38.123441+0000","last_became_active":"2024-04-25T10:03:37.124505+0000","last_became_peered":"2024-04-25T10:03:37.124505+0000","last_unstale":"2024-04-25T10:03:38.123441+0000","last_undegraded":"2024-04-25T10:03:38.123441+0000","last_fullsized":"2024-04-25T10:03:38.123441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:12.937361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205625+0000","last_change":"2024-04-25T10:03:36.091559+0000","last_active":"2024-04-25T10:03:38.205625+0000","last_peered":"2024-04-25T10:03:38.205625+0000","last_clean":"2024-04-25T10:03:38.205625+0000","last_became_active":"2024-04-25T10:03:36.091201+0000","last_became_peered":"2024-04-25T10:03:36.091201+0000","last_unstale":"2024-04-25T10:03:38.205625+0000","last_undegraded":"2024-04-25T10:03:38.205625+0000","last_fullsized":"2024-04-25T10:03:38.205625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:44.681404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121351+0000","last_change":"2024-04-25T10:03:37.127691+0000","last_active":"2024-04-25T10:03:38.121351+0000","last_peered":"2024-04-25T10:03:38.121351+0000","last_clean":"2024-04-25T10:03:38.121351+0000","last_became_active":"2024-04-25T10:03:37.124821+0000","last_became_peered":"2024-04-25T10:03:37.124821+0000","last_unstale":"2024-04-25T10:03:38.121351+0000","last_undegraded":"2024-04-25T10:03:38.121351+0000","last_fullsized":"2024-04-25T10:03:38.121351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:00.157878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.208138+0000","last_change":"2024-04-25T10:03:36.096846+0000","last_active":"2024-04-25T10:03:38.208138+0000","last_peered":"2024-04-25T10:03:38.208138+0000","last_clean":"2024-04-25T10:03:38.208138+0000","last_became_active":"2024-04-25T10:03:36.096537+0000","last_became_peered":"2024-04-25T10:03:36.096537+0000","last_unstale":"2024-04-25T10:03:38.208138+0000","last_undegraded":"2024-04-25T10:03:38.208138+0000","last_fullsized":"2024-04-25T10:03:38.208138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:11.422492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.208360+0000","last_change":"2024-04-25T10:03:37.137465+0000","last_active":"2024-04-25T10:03:38.208360+0000","last_peered":"2024-04-25T10:03:38.208360+0000","last_clean":"2024-04-25T10:03:38.208360+0000","last_became_active":"2024-04-25T10:03:37.137196+0000","last_became_peered":"2024-04-25T10:03:37.137196+0000","last_unstale":"2024-04-25T10:03:38.208360+0000","last_undegraded":"2024-04-25T10:03:38.208360+0000","last_fullsized":"2024-04-25T10:03:38.208360+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:18.938228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119852+0000","last_change":"2024-04-25T10:03:36.104201+0000","last_active":"2024-04-25T10:03:38.119852+0000","last_peered":"2024-04-25T10:03:38.119852+0000","last_clean":"2024-04-25T10:03:38.119852+0000","last_became_active":"2024-04-25T10:03:36.103744+0000","last_became_peered":"2024-04-25T10:03:36.103744+0000","last_unstale":"2024-04-25T10:03:38.119852+0000","last_undegraded":"2024-04-25T10:03:38.119852+0000","last_fullsized":"2024-04-25T10:03:38.119852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:52.298743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207877+0000","last_change":"2024-04-25T10:03:37.136929+0000","last_active":"2024-04-25T10:03:38.207877+0000","last_peered":"2024-04-25T10:03:38.207877+0000","last_clean":"2024-04-25T10:03:38.207877+0000","last_became_active":"2024-04-25T10:03:37.136641+0000","last_became_peered":"2024-04-25T10:03:37.136641+0000","last_unstale":"2024-04-25T10:03:38.207877+0000","last_undegraded":"2024-04-25T10:03:38.207877+0000","last_fullsized":"2024-04-25T10:03:38.207877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:02.282914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122649+0000","last_change":"2024-04-25T10:03:36.092770+0000","last_active":"2024-04-25T10:03:38.122649+0000","last_peered":"2024-04-25T10:03:38.122649+0000","last_clean":"2024-04-25T10:03:38.122649+0000","last_became_active":"2024-04-25T10:03:36.092466+0000","last_became_peered":"2024-04-25T10:03:36.092466+0000","last_unstale":"2024-04-25T10:03:38.122649+0000","last_undegraded":"2024-04-25T10:03:38.122649+0000","last_fullsized":"2024-04-25T10:03:38.122649+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:11.422492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207925+0000","last_change":"2024-04-25T10:03:37.133883+0000","last_active":"2024-04-25T10:03:38.207925+0000","last_peered":"2024-04-25T10:03:38.207925+0000","last_clean":"2024-04-25T10:03:38.207925+0000","last_became_active":"2024-04-25T10:03:37.133559+0000","last_became_peered":"2024-04-25T10:03:37.133559+0000","last_unstale":"2024-04-25T10:03:38.207925+0000","last_undegraded":"2024-04-25T10:03:38.207925+0000","last_fullsized":"2024-04-25T10:03:38.207925+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:50.411564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205377+0000","last_change":"2024-04-25T10:03:36.094541+0000","last_active":"2024-04-25T10:03:38.205377+0000","last_peered":"2024-04-25T10:03:38.205377+0000","last_clean":"2024-04-25T10:03:38.205377+0000","last_became_active":"2024-04-25T10:03:36.094179+0000","last_became_peered":"2024-04-25T10:03:36.094179+0000","last_unstale":"2024-04-25T10:03:38.205377+0000","last_undegraded":"2024-04-25T10:03:38.205377+0000","last_fullsized":"2024-04-25T10:03:38.205377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:21.436037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119041+0000","last_change":"2024-04-25T10:03:37.117486+0000","last_active":"2024-04-25T10:03:38.119041+0000","last_peered":"2024-04-25T10:03:38.119041+0000","last_clean":"2024-04-25T10:03:38.119041+0000","last_became_active":"2024-04-25T10:03:37.117194+0000","last_became_peered":"2024-04-25T10:03:37.117194+0000","last_unstale":"2024-04-25T10:03:38.119041+0000","last_undegraded":"2024-04-25T10:03:38.119041+0000","last_fullsized":"2024-04-25T10:03:38.119041+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:26.125538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207054+0000","last_change":"2024-04-25T10:03:36.097091+0000","last_active":"2024-04-25T10:03:38.207054+0000","last_peered":"2024-04-25T10:03:38.207054+0000","last_clean":"2024-04-25T10:03:38.207054+0000","last_became_active":"2024-04-25T10:03:36.096820+0000","last_became_peered":"2024-04-25T10:03:36.096820+0000","last_unstale":"2024-04-25T10:03:38.207054+0000","last_undegraded":"2024-04-25T10:03:38.207054+0000","last_fullsized":"2024-04-25T10:03:38.207054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:17.925837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205441+0000","last_change":"2024-04-25T10:03:37.139337+0000","last_active":"2024-04-25T10:03:38.205441+0000","last_peered":"2024-04-25T10:03:38.205441+0000","last_clean":"2024-04-25T10:03:38.205441+0000","last_became_active":"2024-04-25T10:03:37.137532+0000","last_became_peered":"2024-04-25T10:03:37.137532+0000","last_unstale":"2024-04-25T10:03:38.205441+0000","last_undegraded":"2024-04-25T10:03:38.205441+0000","last_fullsized":"2024-04-25T10:03:38.205441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:56.223185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.121088+0000","last_change":"2024-04-25T10:03:36.100566+0000","last_active":"2024-04-25T10:03:38.121088+0000","last_peered":"2024-04-25T10:03:38.121088+0000","last_clean":"2024-04-25T10:03:38.121088+0000","last_became_active":"2024-04-25T10:03:36.100186+0000","last_became_peered":"2024-04-25T10:03:36.100186+0000","last_unstale":"2024-04-25T10:03:38.121088+0000","last_undegraded":"2024-04-25T10:03:38.121088+0000","last_fullsized":"2024-04-25T10:03:38.121088+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:01.270523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128434+0000","last_change":"2024-04-25T10:03:37.132271+0000","last_active":"2024-04-25T10:03:38.128434+0000","last_peered":"2024-04-25T10:03:38.128434+0000","last_clean":"2024-04-25T10:03:38.128434+0000","last_became_active":"2024-04-25T10:03:37.131960+0000","last_became_peered":"2024-04-25T10:03:37.131960+0000","last_unstale":"2024-04-25T10:03:38.128434+0000","last_undegraded":"2024-04-25T10:03:38.128434+0000","last_fullsized":"2024-04-25T10:03:38.128434+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:13.557419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.205956+0000","last_change":"2024-04-25T10:03:36.096821+0000","last_active":"2024-04-25T10:03:38.205956+0000","last_peered":"2024-04-25T10:03:38.205956+0000","last_clean":"2024-04-25T10:03:38.205956+0000","last_became_active":"2024-04-25T10:03:36.096490+0000","last_became_peered":"2024-04-25T10:03:36.096490+0000","last_unstale":"2024-04-25T10:03:38.205956+0000","last_undegraded":"2024-04-25T10:03:38.205956+0000","last_fullsized":"2024-04-25T10:03:38.205956+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:55.146532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207050+0000","last_change":"2024-04-25T10:03:37.123190+0000","last_active":"2024-04-25T10:03:38.207050+0000","last_peered":"2024-04-25T10:03:38.207050+0000","last_clean":"2024-04-25T10:03:38.207050+0000","last_became_active":"2024-04-25T10:03:37.122906+0000","last_became_peered":"2024-04-25T10:03:37.122906+0000","last_unstale":"2024-04-25T10:03:38.207050+0000","last_undegraded":"2024-04-25T10:03:38.207050+0000","last_fullsized":"2024-04-25T10:03:38.207050+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.154712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120280+0000","last_change":"2024-04-25T10:03:36.103569+0000","last_active":"2024-04-25T10:03:38.120280+0000","last_peered":"2024-04-25T10:03:38.120280+0000","last_clean":"2024-04-25T10:03:38.120280+0000","last_became_active":"2024-04-25T10:03:36.103282+0000","last_became_peered":"2024-04-25T10:03:36.103282+0000","last_unstale":"2024-04-25T10:03:38.120280+0000","last_undegraded":"2024-04-25T10:03:38.120280+0000","last_fullsized":"2024-04-25T10:03:38.120280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.547187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122773+0000","last_change":"2024-04-25T10:03:37.125608+0000","last_active":"2024-04-25T10:03:38.122773+0000","last_peered":"2024-04-25T10:03:38.122773+0000","last_clean":"2024-04-25T10:03:38.122773+0000","last_became_active":"2024-04-25T10:03:37.125241+0000","last_became_peered":"2024-04-25T10:03:37.125241+0000","last_unstale":"2024-04-25T10:03:38.122773+0000","last_undegraded":"2024-04-25T10:03:38.122773+0000","last_fullsized":"2024-04-25T10:03:38.122773+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.886320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119810+0000","last_change":"2024-04-25T10:03:36.107003+0000","last_active":"2024-04-25T10:03:38.119810+0000","last_peered":"2024-04-25T10:03:38.119810+0000","last_clean":"2024-04-25T10:03:38.119810+0000","last_became_active":"2024-04-25T10:03:36.106571+0000","last_became_peered":"2024-04-25T10:03:36.106571+0000","last_unstale":"2024-04-25T10:03:38.119810+0000","last_undegraded":"2024-04-25T10:03:38.119810+0000","last_fullsized":"2024-04-25T10:03:38.119810+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:35.650886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128779+0000","last_change":"2024-04-25T10:03:37.130908+0000","last_active":"2024-04-25T10:03:38.128779+0000","last_peered":"2024-04-25T10:03:38.128779+0000","last_clean":"2024-04-25T10:03:38.128779+0000","last_became_active":"2024-04-25T10:03:37.130557+0000","last_became_peered":"2024-04-25T10:03:37.130557+0000","last_unstale":"2024-04-25T10:03:38.128779+0000","last_undegraded":"2024-04-25T10:03:38.128779+0000","last_fullsized":"2024-04-25T10:03:38.128779+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:44.620298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.123032+0000","last_change":"2024-04-25T10:03:36.096706+0000","last_active":"2024-04-25T10:03:38.123032+0000","last_peered":"2024-04-25T10:03:38.123032+0000","last_clean":"2024-04-25T10:03:38.123032+0000","last_became_active":"2024-04-25T10:03:36.095723+0000","last_became_peered":"2024-04-25T10:03:36.095723+0000","last_unstale":"2024-04-25T10:03:38.123032+0000","last_undegraded":"2024-04-25T10:03:38.123032+0000","last_fullsized":"2024-04-25T10:03:38.123032+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:12.071145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207819+0000","last_change":"2024-04-25T10:03:37.120789+0000","last_active":"2024-04-25T10:03:38.207819+0000","last_peered":"2024-04-25T10:03:38.207819+0000","last_clean":"2024-04-25T10:03:38.207819+0000","last_became_active":"2024-04-25T10:03:37.120515+0000","last_became_peered":"2024-04-25T10:03:37.120515+0000","last_unstale":"2024-04-25T10:03:38.207819+0000","last_undegraded":"2024-04-25T10:03:38.207819+0000","last_fullsized":"2024-04-25T10:03:38.207819+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:53.342578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122819+0000","last_change":"2024-04-25T10:03:36.104229+0000","last_active":"2024-04-25T10:03:38.122819+0000","last_peered":"2024-04-25T10:03:38.122819+0000","last_clean":"2024-04-25T10:03:38.122819+0000","last_became_active":"2024-04-25T10:03:36.103025+0000","last_became_peered":"2024-04-25T10:03:36.103025+0000","last_unstale":"2024-04-25T10:03:38.122819+0000","last_undegraded":"2024-04-25T10:03:38.122819+0000","last_fullsized":"2024-04-25T10:03:38.122819+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:54.628483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.122140+0000","last_change":"2024-04-25T10:03:37.132482+0000","last_active":"2024-04-25T10:03:38.122140+0000","last_peered":"2024-04-25T10:03:38.122140+0000","last_clean":"2024-04-25T10:03:38.122140+0000","last_became_active":"2024-04-25T10:03:37.131368+0000","last_became_peered":"2024-04-25T10:03:37.131368+0000","last_unstale":"2024-04-25T10:03:38.122140+0000","last_undegraded":"2024-04-25T10:03:38.122140+0000","last_fullsized":"2024-04-25T10:03:38.122140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:29.886643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.207633+0000","last_change":"2024-04-25T10:03:36.102517+0000","last_active":"2024-04-25T10:03:38.207633+0000","last_peered":"2024-04-25T10:03:38.207633+0000","last_clean":"2024-04-25T10:03:38.207633+0000","last_became_active":"2024-04-25T10:03:36.102006+0000","last_became_peered":"2024-04-25T10:03:36.102006+0000","last_unstale":"2024-04-25T10:03:38.207633+0000","last_undegraded":"2024-04-25T10:03:38.207633+0000","last_fullsized":"2024-04-25T10:03:38.207633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:22.682970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.120665+0000","last_change":"2024-04-25T10:03:37.122086+0000","last_active":"2024-04-25T10:03:38.120665+0000","last_peered":"2024-04-25T10:03:38.120665+0000","last_clean":"2024-04-25T10:03:38.120665+0000","last_became_active":"2024-04-25T10:03:37.121786+0000","last_became_peered":"2024-04-25T10:03:37.121786+0000","last_unstale":"2024-04-25T10:03:38.120665+0000","last_undegraded":"2024-04-25T10:03:38.120665+0000","last_fullsized":"2024-04-25T10:03:38.120665+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:40.177091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128507+0000","last_change":"2024-04-25T10:03:36.102101+0000","last_active":"2024-04-25T10:03:38.128507+0000","last_peered":"2024-04-25T10:03:38.128507+0000","last_clean":"2024-04-25T10:03:38.128507+0000","last_became_active":"2024-04-25T10:03:36.101810+0000","last_became_peered":"2024-04-25T10:03:36.101810+0000","last_unstale":"2024-04-25T10:03:38.128507+0000","last_undegraded":"2024-04-25T10:03:38.128507+0000","last_fullsized":"2024-04-25T10:03:38.128507+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:01.270523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119305+0000","last_change":"2024-04-25T10:03:37.116902+0000","last_active":"2024-04-25T10:03:38.119305+0000","last_peered":"2024-04-25T10:03:38.119305+0000","last_clean":"2024-04-25T10:03:38.119305+0000","last_became_active":"2024-04-25T10:03:37.116410+0000","last_became_peered":"2024-04-25T10:03:37.116410+0000","last_unstale":"2024-04-25T10:03:38.119305+0000","last_undegraded":"2024-04-25T10:03:38.119305+0000","last_fullsized":"2024-04-25T10:03:38.119305+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:40.483272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119192+0000","last_change":"2024-04-25T10:03:36.097695+0000","last_active":"2024-04-25T10:03:38.119192+0000","last_peered":"2024-04-25T10:03:38.119192+0000","last_clean":"2024-04-25T10:03:38.119192+0000","last_became_active":"2024-04-25T10:03:36.097467+0000","last_became_peered":"2024-04-25T10:03:36.097467+0000","last_unstale":"2024-04-25T10:03:38.119192+0000","last_undegraded":"2024-04-25T10:03:38.119192+0000","last_fullsized":"2024-04-25T10:03:38.119192+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:03.186140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128651+0000","last_change":"2024-04-25T10:03:37.124824+0000","last_active":"2024-04-25T10:03:38.128651+0000","last_peered":"2024-04-25T10:03:38.128651+0000","last_clean":"2024-04-25T10:03:38.128651+0000","last_became_active":"2024-04-25T10:03:37.124402+0000","last_became_peered":"2024-04-25T10:03:37.124402+0000","last_unstale":"2024-04-25T10:03:38.128651+0000","last_undegraded":"2024-04-25T10:03:38.128651+0000","last_fullsized":"2024-04-25T10:03:38.128651+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:33.144462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206932+0000","last_change":"2024-04-25T10:03:36.090034+0000","last_active":"2024-04-25T10:03:38.206932+0000","last_peered":"2024-04-25T10:03:38.206932+0000","last_clean":"2024-04-25T10:03:38.206932+0000","last_became_active":"2024-04-25T10:03:36.089136+0000","last_became_peered":"2024-04-25T10:03:36.089136+0000","last_unstale":"2024-04-25T10:03:38.206932+0000","last_undegraded":"2024-04-25T10:03:38.206932+0000","last_fullsized":"2024-04-25T10:03:38.206932+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.703806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.119734+0000","last_change":"2024-04-25T10:03:37.111090+0000","last_active":"2024-04-25T10:03:38.119734+0000","last_peered":"2024-04-25T10:03:38.119734+0000","last_clean":"2024-04-25T10:03:38.119734+0000","last_became_active":"2024-04-25T10:03:37.110832+0000","last_became_peered":"2024-04-25T10:03:37.110832+0000","last_unstale":"2024-04-25T10:03:38.119734+0000","last_undegraded":"2024-04-25T10:03:38.119734+0000","last_fullsized":"2024-04-25T10:03:38.119734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:36.053810+0000","last_clean_scrub_stamp":"2024-04-25T10:03:36.053810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:15.856124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.128468+0000","last_change":"2024-04-25T10:03:36.103532+0000","last_active":"2024-04-25T10:03:38.128468+0000","last_peered":"2024-04-25T10:03:38.128468+0000","last_clean":"2024-04-25T10:03:38.128468+0000","last_became_active":"2024-04-25T10:03:36.102663+0000","last_became_peered":"2024-04-25T10:03:36.102663+0000","last_unstale":"2024-04-25T10:03:38.128468+0000","last_undegraded":"2024-04-25T10:03:38.128468+0000","last_fullsized":"2024-04-25T10:03:38.128468+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:35.041419+0000","last_clean_scrub_stamp":"2024-04-25T10:03:35.041419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:11.924970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T10:03:38.206411+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:03:38.206411+0000","last_peered":"2024-04-25T10:03:38.206411+0000","last_clean":"2024-04-25T10:03:38.206411+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:03:38.206411+0000","last_undegraded":"2024-04-25T10:03:38.206411+0000","last_fullsized":"2024-04-25T10:03:38.206411+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":785452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":458752,"data_stored":194592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":471040,"data_stored":196696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1052672,"data_stored":784892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":462848,"data_stored":194594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":475136,"data_stored":196238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:47.141 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:03:47.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:03:47.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.401+0000 7f4466200640 1 Processor -- start 2024-04-25T10:03:47.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.401+0000 7f4466200640 1 -- start start 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.403+0000 7f4466200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460144fe0 0x7f446006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.403+0000 7f4466200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f446006d130 0x7f4460071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.403+0000 7f4466200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44601453c0 con 0x7f4460071670 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.403+0000 7f4466200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44600a8560 con 0x7f446006d130 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.403+0000 7f4466200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446006d6a0 con 0x7f4460144fe0 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f446006d130 0x7f4460071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f446006d130 0x7f4460071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49490/0 (socket says 172.21.15.5:49490) 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 -- 172.21.15.5:0/3862803741 learned_addr learned my addr 172.21.15.5:0/3862803741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445f400640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460144fe0 0x7f446006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 -- 172.21.15.5:0/3862803741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460144fe0 msgr2=0x7f446006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460144fe0 0x7f446006e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 -- 172.21.15.5:0/3862803741 >> v1:172.21.15.80:6789/0 conn(0x7f4460071670 legacy=0x7f44600cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445ea00640 1 -- 172.21.15.5:0/3862803741 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446006d980 con 0x7f446006d130 2024-04-25T10:03:47.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.404+0000 7f445f400640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460144fe0 0x7f446006e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.405+0000 7f445ea00640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f4460071130 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f444c009870 tx=0x7f444c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ffdd178c1ba3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f445e000640 1 -- 172.21.15.5:0/3862803741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c037070 con 0x7f446006d130 2024-04-25T10:03:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f445e000640 1 -- 172.21.15.5:0/3862803741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c032440 con 0x7f446006d130 2024-04-25T10:03:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f445e000640 1 -- 172.21.15.5:0/3862803741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c032810 con 0x7f446006d130 2024-04-25T10:03:47.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f4466200640 1 -- 172.21.15.5:0/3862803741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 msgr2=0x7f4460071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f4466200640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f4460071130 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f444c009870 tx=0x7f444c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f4466200640 1 -- 172.21.15.5:0/3862803741 shutdown_connections 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f4466200640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f4460071130 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f4466200640 1 --2- 172.21.15.5:0/3862803741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460144fe0 0x7f446006e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.406+0000 7f4466200640 1 -- 172.21.15.5:0/3862803741 >> 172.21.15.5:0/3862803741 conn(0x7f446006ab60 msgr2=0x7f446006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.407+0000 7f4466200640 1 -- 172.21.15.5:0/3862803741 shutdown_connections 2024-04-25T10:03:47.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.407+0000 7f4466200640 1 -- 172.21.15.5:0/3862803741 wait complete. 2024-04-25T10:03:47.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.408+0000 7f4466200640 1 Processor -- start 2024-04-25T10:03:47.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.408+0000 7f4466200640 1 -- start start 2024-04-25T10:03:47.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.409+0000 7f4466200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f446013d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.409+0000 7f4466200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460071670 0x7f446013dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.409+0000 7f4466200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4460144fe0 0x7f44600a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.409+0000 7f4466200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446015be50 con 0x7f446006d130 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.409+0000 7f4466200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446006df40 con 0x7f4460144fe0 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.409+0000 7f4466200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446006eaa0 con 0x7f4460071670 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f446013d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f446013d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49494/0 (socket says 172.21.15.5:49494) 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 -- 172.21.15.5:0/584251689 learned_addr learned my addr 172.21.15.5:0/584251689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445fe00640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4460144fe0 0x7f44600a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445ea00640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460071670 0x7f446013dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 -- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460071670 msgr2=0x7f446013dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460071670 0x7f446013dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 -- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4460144fe0 msgr2=0x7f44600a86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4460144fe0 0x7f44600a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.410+0000 7f445f400640 1 -- 172.21.15.5:0/584251689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446013e240 con 0x7f446006d130 2024-04-25T10:03:47.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.411+0000 7f445f400640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f446013d7c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f445000c7f0 tx=0x7f445000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.411+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450015070 con 0x7f446006d130 2024-04-25T10:03:47.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.411+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4450007820 con 0x7f446006d130 2024-04-25T10:03:47.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.411+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450007bd0 con 0x7f446006d130 2024-04-25T10:03:47.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.411+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446013e4f0 con 0x7f446006d130 2024-04-25T10:03:47.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.411+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446013fca0 con 0x7f446006d130 2024-04-25T10:03:47.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.412+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44600b0a00 con 0x7f446006d130 2024-04-25T10:03:47.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.413+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4450018070 con 0x7f446006d130 2024-04-25T10:03:47.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.415+0000 7f4457400640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f444006b4d0 0x7f444006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.415+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f445008a560 con 0x7f446006d130 2024-04-25T10:03:47.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.415+0000 7f445ea00640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f444006b4d0 0x7f444006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.416+0000 7f445ea00640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f444006b4d0 0x7f444006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f444c034000 tx=0x7f444c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.419+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44600b0a00 con 0x7f446006d130 2024-04-25T10:03:47.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.698+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f446006b170 con 0x7f446006d130 2024-04-25T10:03:47.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.704+0000 7f4457400640 1 -- 172.21.15.5:0/584251689 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f446006b170 con 0x7f446006d130 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f444006b4d0 msgr2=0x7f444006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f444006b4d0 0x7f444006d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f444c034000 tx=0x7f444c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 msgr2=0x7f446013d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f446013d7c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f445000c7f0 tx=0x7f445000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 shutdown_connections 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f444006b4d0 0x7f444006d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4460144fe0 0x7f44600a86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4460071670 0x7f446013dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 --2- 172.21.15.5:0/584251689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f446006d130 0x7f446013d7c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.706+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 >> 172.21.15.5:0/584251689 conn(0x7f446006ab60 msgr2=0x7f44600b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.707+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 shutdown_connections 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.707+0000 7f4466200640 1 -- 172.21.15.5:0/584251689 wait complete. 2024-04-25T10:03:47.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:47.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.924+0000 7efcfce00640 1 Processor -- start 2024-04-25T10:03:47.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.924+0000 7efcfce00640 1 -- start start 2024-04-25T10:03:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcfce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf8144100 0x7efcf80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcfce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efcf806e970 0x7efcf815b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcfce00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf80cdbf0 con 0x7efcf80ced20 2024-04-25T10:03:47.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcfce00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf81444e0 con 0x7efcf806e970 2024-04-25T10:03:47.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcfce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815ac70 con 0x7efcf8144100 2024-04-25T10:03:47.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcf5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efcf80ced20 0x7efcf806e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43910/0 (socket says 172.21.15.5:43910) 2024-04-25T10:03:47.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcf5400640 1 -- 172.21.15.5:0/2451151348 learned_addr learned my addr 172.21.15.5:0/2451151348 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.926+0000 7efcf6800640 1 --2- 172.21.15.5:0/2451151348 >> v2:172.21.15.5:3300/0 conn(0x7efcf806e970 0x7efcf815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf5e00640 1 --2- 172.21.15.5:0/2451151348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf8144100 0x7efcf80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf4a00640 1 -- 172.21.15.5:0/2451151348 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855194861 0 0) 0x7efcf80cdbf0 con 0x7efcf80ced20 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf4a00640 1 -- 172.21.15.5:0/2451151348 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcdc005730 con 0x7efcf80ced20 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf6800640 1 -- 172.21.15.5:0/2451151348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf8144100 msgr2=0x7efcf80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf6800640 1 --2- 172.21.15.5:0/2451151348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf8144100 0x7efcf80cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf6800640 1 -- 172.21.15.5:0/2451151348 >> v1:172.21.15.5:6789/0 conn(0x7efcf80ced20 legacy=0x7efcf806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.927+0000 7efcf6800640 1 -- 172.21.15.5:0/2451151348 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80cded0 con 0x7efcf806e970 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.928+0000 7efcf6800640 1 --2- 172.21.15.5:0/2451151348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf806e970 0x7efcf815b130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009d90 tx=0x7efcf002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ed5948a14e9ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.928+0000 7efcf4a00640 1 -- 172.21.15.5:0/2451151348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0007740 con 0x7efcf806e970 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcf4a00640 1 -- 172.21.15.5:0/2451151348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf00078e0 con 0x7efcf806e970 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcf4a00640 1 -- 172.21.15.5:0/2451151348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0007cb0 con 0x7efcf806e970 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 -- 172.21.15.5:0/2451151348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf806e970 msgr2=0x7efcf815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 --2- 172.21.15.5:0/2451151348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf806e970 0x7efcf815b130 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009d90 tx=0x7efcf002a660 comp rx=0 tx=0).stop 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 -- 172.21.15.5:0/2451151348 shutdown_connections 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 --2- 172.21.15.5:0/2451151348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf806e970 0x7efcf815b130 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 --2- 172.21.15.5:0/2451151348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf8144100 0x7efcf80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 -- 172.21.15.5:0/2451151348 >> 172.21.15.5:0/2451151348 conn(0x7efcf8071150 msgr2=0x7efcf8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 -- 172.21.15.5:0/2451151348 shutdown_connections 2024-04-25T10:03:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.929+0000 7efcfce00640 1 -- 172.21.15.5:0/2451151348 wait complete. 2024-04-25T10:03:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.931+0000 7efcfce00640 1 Processor -- start 2024-04-25T10:03:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.931+0000 7efcfce00640 1 -- start start 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcfce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf806e970 0x7efcf8141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcfce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 0x7efcf81f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcfce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8144100 0x7efcf813af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcfce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cea00 con 0x7efcf80ced20 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcfce00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80ce490 con 0x7efcf8144100 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcfce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf815b670 con 0x7efcf806e970 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 0x7efcf81f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 0x7efcf81f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49512/0 (socket says 172.21.15.5:49512) 2024-04-25T10:03:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcf5400640 1 -- 172.21.15.5:0/3742358498 learned_addr learned my addr 172.21.15.5:0/3742358498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.932+0000 7efcf5e00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf806e970 0x7efcf8141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf6800640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8144100 0x7efcf813af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf5400640 1 -- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf806e970 msgr2=0x7efcf8141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf5400640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf806e970 0x7efcf8141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf5400640 1 -- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8144100 msgr2=0x7efcf813af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf5400640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8144100 0x7efcf813af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf5400640 1 -- 172.21.15.5:0/3742358498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8140810 con 0x7efcf80ced20 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcf5400640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 0x7efcf81f35c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efce400c460 tx=0x7efce400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4017070 con 0x7efcf80ced20 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce4005ce0 con 0x7efcf80ced20 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf8140af0 con 0x7efcf80ced20 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4007290 con 0x7efcf80ced20 2024-04-25T10:03:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.933+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf813b460 con 0x7efcf80ced20 2024-04-25T10:03:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.934+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf81ef670 con 0x7efcf80ced20 2024-04-25T10:03:47.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.935+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efce401a070 con 0x7efcf80ced20 2024-04-25T10:03:47.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.937+0000 7efcee000640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcd806b4d0 0x7efcd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.937+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efce4089720 con 0x7efcf80ced20 2024-04-25T10:03:47.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.937+0000 7efcf5e00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcd806b4d0 0x7efcd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.938+0000 7efcf5e00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcd806b4d0 0x7efcd806d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efce8009a30 tx=0x7efce8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:47.941+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efce4058400 con 0x7efcf80ced20 2024-04-25T10:03:48.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.170+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcf8068fd0 con 0x7efcf80ced20 2024-04-25T10:03:48.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.171+0000 7efcee000640 1 -- 172.21.15.5:0/3742358498 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7efce4057d00 con 0x7efcf80ced20 2024-04-25T10:03:48.176 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:48.176 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:03:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.173+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcd806b4d0 msgr2=0x7efcd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.173+0000 7efcfce00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcd806b4d0 0x7efcd806d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efce8009a30 tx=0x7efce8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.173+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 msgr2=0x7efcf81f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.173+0000 7efcfce00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 0x7efcf81f35c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efce400c460 tx=0x7efce400c920 comp rx=0 tx=0).stop 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 shutdown_connections 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efcd806b4d0 0x7efcd806d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8144100 0x7efcf813af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf80ced20 0x7efcf81f35c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 --2- 172.21.15.5:0/3742358498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efcf806e970 0x7efcf8141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 >> 172.21.15.5:0/3742358498 conn(0x7efcf8071150 msgr2=0x7efcf806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 shutdown_connections 2024-04-25T10:03:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.174+0000 7efcfce00640 1 -- 172.21.15.5:0/3742358498 wait complete. 2024-04-25T10:03:48.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:03:36.556737+0000', 'modified': '2024-04-25T10:03:39.640872+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5005}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5005': {'gid': 5005, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.80:6837/3920492243', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6835', 'nonce': 3920492243}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 3920492243}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:48.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:48.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:03:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.820+0000 7fa09fa00640 1 Processor -- start 2024-04-25T10:03:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.820+0000 7fa09fa00640 1 -- start start 2024-04-25T10:03:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.822+0000 7fa09fa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa09806e970 0x7fa0980cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.822+0000 7fa09fa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09815b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.822+0000 7fa09fa00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0980cdbf0 con 0x7fa098144100 2024-04-25T10:03:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.822+0000 7fa09fa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0981444e0 con 0x7fa09806e970 2024-04-25T10:03:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.822+0000 7fa09fa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09815ac70 con 0x7fa0980ced20 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.823+0000 7fa08fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa09806e970 0x7fa0980cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.823+0000 7fa09da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.823+0000 7fa08fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa09806e970 0x7fa0980cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:34624/0 (socket says 172.21.15.5:34624) 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.823+0000 7fa08fe00640 1 -- 172.21.15.5:0/703245896 learned_addr learned my addr 172.21.15.5:0/703245896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.824+0000 7fa09da00640 1 -- 172.21.15.5:0/703245896 >> v2:172.21.15.80:3300/0 conn(0x7fa09806e970 msgr2=0x7fa0980cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.824+0000 7fa09da00640 1 --2- 172.21.15.5:0/703245896 >> v2:172.21.15.80:3300/0 conn(0x7fa09806e970 0x7fa0980cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.824+0000 7fa09da00640 1 -- 172.21.15.5:0/703245896 >> v1:172.21.15.80:6789/0 conn(0x7fa098144100 legacy=0x7fa09806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.824+0000 7fa09da00640 1 -- 172.21.15.5:0/703245896 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0980cded0 con 0x7fa0980ced20 2024-04-25T10:03:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.825+0000 7fa09da00640 1 --2- 172.21.15.5:0/703245896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09815b130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa090009d90 tx=0x7fa09002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6a60a78638027e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.826+0000 7fa097e00640 1 -- 172.21.15.5:0/703245896 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090007740 con 0x7fa0980ced20 2024-04-25T10:03:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.826+0000 7fa097e00640 1 -- 172.21.15.5:0/703245896 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0900078e0 con 0x7fa0980ced20 2024-04-25T10:03:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.826+0000 7fa097e00640 1 -- 172.21.15.5:0/703245896 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090007cb0 con 0x7fa0980ced20 2024-04-25T10:03:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 -- 172.21.15.5:0/703245896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 msgr2=0x7fa09815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 --2- 172.21.15.5:0/703245896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09815b130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa090009d90 tx=0x7fa09002a660 comp rx=0 tx=0).stop 2024-04-25T10:03:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 -- 172.21.15.5:0/703245896 shutdown_connections 2024-04-25T10:03:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 --2- 172.21.15.5:0/703245896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09815b130 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 --2- 172.21.15.5:0/703245896 >> v2:172.21.15.80:3300/0 conn(0x7fa09806e970 0x7fa0980cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 -- 172.21.15.5:0/703245896 >> 172.21.15.5:0/703245896 conn(0x7fa098071150 msgr2=0x7fa098071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 -- 172.21.15.5:0/703245896 shutdown_connections 2024-04-25T10:03:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.827+0000 7fa09fa00640 1 -- 172.21.15.5:0/703245896 wait complete. 2024-04-25T10:03:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.828+0000 7fa09fa00640 1 Processor -- start 2024-04-25T10:03:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.829+0000 7fa09fa00640 1 -- start start 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09fa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa09806e970 0x7fa098139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09fa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09813d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09fa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 0x7fa09813d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09fa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0980cea00 con 0x7fa098144100 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09fa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0980ce490 con 0x7fa09806e970 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09fa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09815b670 con 0x7fa0980ced20 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 0x7fa09813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 0x7fa09813d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49526/0 (socket says 172.21.15.5:49526) 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09da00640 1 -- 172.21.15.5:0/1944961546 learned_addr learned my addr 172.21.15.5:0/1944961546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa08fe00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09813d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.830+0000 7fa09d000640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa09806e970 0x7fa098139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09da00640 1 -- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 msgr2=0x7fa09813d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09da00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09813d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09da00640 1 -- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa09806e970 msgr2=0x7fa098139480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09da00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa09806e970 0x7fa098139480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09da00640 1 -- 172.21.15.5:0/1944961546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa098139c70 con 0x7fa098144100 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09da00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 0x7fa09813d580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa09002ac50 tx=0x7fa090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa095600640 1 -- 172.21.15.5:0/1944961546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090040070 con 0x7fa098144100 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa095600640 1 -- 172.21.15.5:0/1944961546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa090031b40 con 0x7fa098144100 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa095600640 1 -- 172.21.15.5:0/1944961546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090045460 con 0x7fa098144100 2024-04-25T10:03:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.831+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09813ca20 con 0x7fa098144100 2024-04-25T10:03:48.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.832+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa098139f10 con 0x7fa098144100 2024-04-25T10:03:48.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.835+0000 7fa095600640 1 -- 172.21.15.5:0/1944961546 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa090039040 con 0x7fa098144100 2024-04-25T10:03:48.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.835+0000 7fa095600640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa07406b500 0x7fa07406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.836+0000 7fa095600640 1 -- 172.21.15.5:0/1944961546 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0900abbb0 con 0x7fa098144100 2024-04-25T10:03:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.836+0000 7fa0a2845740 1 Processor -- start 2024-04-25T10:03:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.836+0000 7fa09d000640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa07406b500 0x7fa07406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.836+0000 7fa0a2845740 1 -- start start 2024-04-25T10:03:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe0a950 0x55a2afe32790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe0ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe09110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa0a2845740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afd6e620 con 0x55a2afe535c0 2024-04-25T10:03:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afd17ca0 con 0x55a2afe32cd0 2024-04-25T10:03:48.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2afd6a6c0 con 0x55a2afe0a950 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.837+0000 7fa09d000640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa07406b500 0x7fa07406d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa088007fd0 tx=0x7fa088009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe09110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe09110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49530/0 (socket says 172.21.15.5:49530) 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 -- 172.21.15.5:0/1776282692 learned_addr learned my addr 172.21.15.5:0/1776282692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09d000640 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe0a950 0x55a2afe32790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa08fe00640 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe0ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 -- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe0a950 msgr2=0x55a2afe32790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe0a950 0x55a2afe32790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 -- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 msgr2=0x55a2afe0ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe0ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 -- 172.21.15.5:0/1776282692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2afd1ef40 con 0x55a2afe535c0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.838+0000 7fa09da00640 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe09110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa0900a9df0 tx=0x7fa090009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b49f07db97162a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.839+0000 7fa08d600640 1 -- 172.21.15.5:0/1776282692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09004a040 con 0x55a2afe535c0 2024-04-25T10:03:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.839+0000 7fa08d600640 1 -- 172.21.15.5:0/1776282692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09007a220 con 0x55a2afe535c0 2024-04-25T10:03:48.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.839+0000 7fa08d600640 1 -- 172.21.15.5:0/1776282692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09007e170 con 0x55a2afe535c0 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.839+0000 7fa0a2845740 1 -- 172.21.15.5:0/1776282692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 msgr2=0x55a2afe09110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.839+0000 7fa0a2845740 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe09110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa0900a9df0 tx=0x7fa090009750 comp rx=0 tx=0).stop 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.839+0000 7fa0a2845740 1 -- 172.21.15.5:0/1776282692 shutdown_connections 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe09110 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe0ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 --2- 172.21.15.5:0/1776282692 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe0a950 0x55a2afe32790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 -- 172.21.15.5:0/1776282692 >> 172.21.15.5:0/1776282692 conn(0x55a2afe07b70 msgr2=0x55a2afe50550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 -- 172.21.15.5:0/1776282692 shutdown_connections 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 -- 172.21.15.5:0/1776282692 wait complete. 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.840+0000 7fa0a2845740 1 -- start start 2024-04-25T10:03:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.841+0000 7fa0a2845740 1 Processor -- start 2024-04-25T10:03:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe49080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe04df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afd17ca0 con 0x55a2afe535c0 2024-04-25T10:03:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afd0d1f0 con 0x55a2afe2e110 2024-04-25T10:03:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2afd6e620 con 0x55a2afe32cd0 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa0a2845740 10 client.0 _ll_register_callbacks cb 0x55a2afd2aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49540/0 (socket says 172.21.15.5:49540) 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 -- 172.21.15.5:0/3240303113 learned_addr learned my addr 172.21.15.5:0/3240303113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa09d000640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 -- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 msgr2=0x55a2afe49080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe49080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 -- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 msgr2=0x55a2afe04df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa09da00640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe04df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe04df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa08fe00640 1 -- 172.21.15.5:0/3240303113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2afd2cc50 con 0x55a2afe535c0 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.842+0000 7fa09d000640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe49080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa08fe00640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe048b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa0800116d0 tx=0x7fa080011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895856a4e7e1d31f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa086000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa086000640 1 -- 172.21.15.5:0/3240303113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08001f060 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa086000640 1 -- 172.21.15.5:0/3240303113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa080004030 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa09007a220 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa086000640 1 -- 172.21.15.5:0/3240303113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa080025050 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa0a2845740 10 client.5496 fetch_fsmap learned FSMap version 17 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.843+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a2afa832d0 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa086000640 1 -- 172.21.15.5:0/3240303113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09007a220 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa086000640 1 -- 172.21.15.5:0/3240303113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa080028020 con 0x55a2afe535c0 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa0a2845740 10 client.5496 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa0a2845740 10 client.5496 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa0a2845740 20 client.5496 populate_metadata read hostname 'smithi005' 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa0a2845740 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe680f0 0x55a2afe684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa0a2845740 4 client.5496 mds_command: new command op to 5005 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 --> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2afd7bb60 con 0x55a2afe680f0 2024-04-25T10:03:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.844+0000 7fa09d000640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe680f0 0x55a2afe684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.845+0000 7fa09d000640 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe680f0 0x55a2afe684f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32cb3a0c2d73a2de server_cookie=1f27fa41f69e7eb0 in_seq=0 out_seq=0 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.845+0000 7fa086000640 10 client.5496 ms_handle_connect on v2:172.21.15.80:6835/3920492243 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa086000640 1 -- 172.21.15.5:0/3240303113 <== mds.0 v2:172.21.15.80:6835/3920492243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2afd7bb60 con 0x55a2afe680f0 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa086000640 10 client.5496 handle_command_reply: tid=0 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 client.5496 shutdown 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 msgr2=0x55a2afe048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe048b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa0800116d0 tx=0x7fa080011b90 comp rx=0 tx=0).stop 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 shutdown_connections 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe680f0 0x55a2afe684f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe04df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe048b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 --2- 172.21.15.5:0/3240303113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe49080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 >> 172.21.15.5:0/3240303113 conn(0x55a2afe0a950 msgr2=0x55a2afe319b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa086000640 0 client.5496 ms_handle_reset on v2:172.21.15.80:6835/3920492243 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 shutdown_connections 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 1 -- 172.21.15.5:0/3240303113 wait complete. 2024-04-25T10:03:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.848+0000 7fa0a2845740 20 client.5496 trim_cache size 0 max 16384 2024-04-25T10:03:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.866+0000 7fa0a2845740 1 Processor -- start 2024-04-25T10:03:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.866+0000 7fa0a2845740 1 -- start start 2024-04-25T10:03:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe50350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe48670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe67a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa0a2845740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0981fb290 con 0x55a2afe535c0 2024-04-25T10:03:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09813b660 con 0x55a2afe2e110 2024-04-25T10:03:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0981facf0 con 0x55a2afe32cd0 2024-04-25T10:03:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.867+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe67a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe67a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49552/0 (socket says 172.21.15.5:49552) 2024-04-25T10:03:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 -- 172.21.15.5:0/2998479064 learned_addr learned my addr 172.21.15.5:0/2998479064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa08fe00640 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe48670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09d000640 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe50350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 -- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 msgr2=0x55a2afe48670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe48670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 -- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 msgr2=0x55a2afe50350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe50350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 -- 172.21.15.5:0/2998479064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2afb4d6b0 con 0x55a2afe535c0 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.868+0000 7fa09da00640 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe67a40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa090076680 tx=0x7fa090047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8df41402f2e3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa08cc00640 1 -- 172.21.15.5:0/2998479064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09004a030 con 0x55a2afe535c0 2024-04-25T10:03:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa08cc00640 1 -- 172.21.15.5:0/2998479064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09007a220 con 0x55a2afe535c0 2024-04-25T10:03:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa08cc00640 1 -- 172.21.15.5:0/2998479064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09007e370 con 0x55a2afe535c0 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 -- 172.21.15.5:0/2998479064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 msgr2=0x55a2afe67a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe67a40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa090076680 tx=0x7fa090047000 comp rx=0 tx=0).stop 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 -- 172.21.15.5:0/2998479064 shutdown_connections 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe67a40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe32cd0 0x55a2afe48670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 --2- 172.21.15.5:0/2998479064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe2e110 0x55a2afe50350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.869+0000 7fa0a2845740 1 -- 172.21.15.5:0/2998479064 >> 172.21.15.5:0/2998479064 conn(0x55a2afe0a950 msgr2=0x55a2afe4f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.870+0000 7fa0a2845740 1 -- 172.21.15.5:0/2998479064 shutdown_connections 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.870+0000 7fa0a2845740 1 -- 172.21.15.5:0/2998479064 wait complete. 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.870+0000 7fa0a2845740 1 -- start start 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.870+0000 7fa0a2845740 1 Processor -- start 2024-04-25T10:03:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe2e110 0x55a2afebee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe31b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afe65f40 con 0x55a2afe535c0 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2afe47580 con 0x55a2afe32cd0 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2afe65be0 con 0x55a2afe2e110 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.871+0000 7fa0a2845740 10 client.0 _ll_register_callbacks cb 0x7fa08000b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe273f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49556/0 (socket says 172.21.15.5:49556) 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 -- 172.21.15.5:0/4094748006 learned_addr learned my addr 172.21.15.5:0/4094748006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa08fe00640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe31b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09d000640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe2e110 0x55a2afebee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 -- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe2e110 msgr2=0x55a2afebee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe2e110 0x55a2afebee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 -- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 msgr2=0x55a2afe31b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe31b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.872+0000 7fa09da00640 1 -- 172.21.15.5:0/4094748006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2afe45f80 con 0x55a2afe535c0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa09da00640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe273f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa0900a9eb0 tx=0x7fa090007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1559523ce11ef9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa086000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa086000640 1 -- 172.21.15.5:0/4094748006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090082930 con 0x55a2afe535c0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa086000640 1 -- 172.21.15.5:0/4094748006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa090007840 con 0x55a2afe535c0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa09007a220 con 0x55a2afe535c0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa086000640 1 -- 172.21.15.5:0/4094748006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa09007e370 con 0x55a2afe535c0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa0a2845740 10 client.5502 fetch_fsmap learned FSMap version 17 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55a2afe320a0 con 0x55a2afe535c0 2024-04-25T10:03:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa086000640 1 -- 172.21.15.5:0/4094748006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09007a220 con 0x55a2afe535c0 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.873+0000 7fa086000640 1 -- 172.21.15.5:0/4094748006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa090076890 con 0x55a2afe535c0 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa0a2845740 10 client.5502 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa0a2845740 10 client.5502 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa0a2845740 20 client.5502 populate_metadata read hostname 'smithi005' 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa0a2845740 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe64570 0x55a2afe64950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa0a2845740 4 client.5502 mds_command: new command op to 5005 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 --> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2afe082d0 con 0x55a2afe64570 2024-04-25T10:03:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.874+0000 7fa09d000640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe64570 0x55a2afe64950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.875+0000 7fa09d000640 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe64570 0x55a2afe64950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3f7707ae455638 server_cookie=321c014950c1b272 in_seq=0 out_seq=0 2024-04-25T10:03:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.875+0000 7fa086000640 10 client.5502 ms_handle_connect on v2:172.21.15.80:6835/3920492243 2024-04-25T10:03:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa086000640 1 -- 172.21.15.5:0/4094748006 <== mds.0 v2:172.21.15.80:6835/3920492243 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55a2afe082d0 con 0x55a2afe64570 2024-04-25T10:03:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa086000640 10 client.5502 handle_command_reply: tid=0 2024-04-25T10:03:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 client.5502 shutdown 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 msgr2=0x55a2afe273f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe273f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa0900a9eb0 tx=0x7fa090007fc0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 shutdown_connections 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:6835/3920492243,v1:172.21.15.80:6837/3920492243] conn(0x55a2afe64570 0x55a2afe64950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2afe535c0 0x55a2afe273f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a2afe32cd0 0x55a2afe31b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 --2- 172.21.15.5:0/4094748006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a2afe2e110 0x55a2afebee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 >> 172.21.15.5:0/4094748006 conn(0x55a2afe0a950 msgr2=0x55a2afe493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa086000640 0 client.5502 ms_handle_reset on v2:172.21.15.80:6835/3920492243 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 shutdown_connections 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.876+0000 7fa0a2845740 1 -- 172.21.15.5:0/4094748006 wait complete. 2024-04-25T10:03:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa0a2845740 20 client.5502 trim_cache size 0 max 16384 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5005, 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 11.763251226, 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 17, 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 40, 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 40, 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 20.634524937999998 2024-04-25T10:03:48.885 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:03:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa07406b500 msgr2=0x7fa07406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa07406b500 0x7fa07406d9c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa088007fd0 tx=0x7fa088009040 comp rx=0 tx=0).stop 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 msgr2=0x7fa09813d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 0x7fa09813d580 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa09002ac50 tx=0x7fa090004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 shutdown_connections 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa098144100 0x7fa09813d580 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0980ced20 0x7fa09813d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa07406b500 0x7fa07406d9c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 --2- 172.21.15.5:0/1944961546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa09806e970 0x7fa098139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 >> 172.21.15.5:0/1944961546 conn(0x7fa098071150 msgr2=0x7fa09806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 shutdown_connections 2024-04-25T10:03:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:48.877+0000 7fa09fa00640 1 -- 172.21.15.5:0/1944961546 wait complete. 2024-04-25T10:03:48.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.096+0000 7f0bc3200640 1 Processor -- start 2024-04-25T10:03:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.096+0000 7f0bc3200640 1 -- start start 2024-04-25T10:03:49.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.098+0000 7f0bc3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0bc415ce00 0x7f0bc415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.098+0000 7f0bc3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 0x7f0bc40cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.098+0000 7f0bc3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bc4071d30 con 0x7f0bc4141da0 2024-04-25T10:03:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.098+0000 7f0bc3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc40cdcd0 con 0x7f0bc415ce00 2024-04-25T10:03:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.098+0000 7f0bc3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4142180 con 0x7f0bc40cd300 2024-04-25T10:03:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 0x7f0bc40cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 0x7f0bc40cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:43946/0 (socket says 172.21.15.5:43946) 2024-04-25T10:03:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 -- 172.21.15.5:0/4226562464 learned_addr learned my addr 172.21.15.5:0/4226562464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:49.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc1400640 1 --2- 172.21.15.5:0/4226562464 >> v2:172.21.15.80:3300/0 conn(0x7f0bc415ce00 0x7f0bc415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 -- 172.21.15.5:0/4226562464 >> v2:172.21.15.80:3300/0 conn(0x7f0bc415ce00 msgr2=0x7f0bc415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:49.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 --2- 172.21.15.5:0/4226562464 >> v2:172.21.15.80:3300/0 conn(0x7f0bc415ce00 0x7f0bc415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 -- 172.21.15.5:0/4226562464 >> v1:172.21.15.80:6789/0 conn(0x7f0bc4141da0 legacy=0x7f0bc406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.099+0000 7f0bc2800640 1 -- 172.21.15.5:0/4226562464 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc406d620 con 0x7f0bc40cd300 2024-04-25T10:03:49.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.101+0000 7f0bc2800640 1 --2- 172.21.15.5:0/4226562464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 0x7f0bc40cd790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009e50 tx=0x7f0bbc02a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f566a82277ba0f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.101+0000 7f0bc0a00640 1 -- 172.21.15.5:0/4226562464 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc036070 con 0x7f0bc40cd300 2024-04-25T10:03:49.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.101+0000 7f0bc0a00640 1 -- 172.21.15.5:0/4226562464 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bbc031600 con 0x7f0bc40cd300 2024-04-25T10:03:49.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.101+0000 7f0bc0a00640 1 -- 172.21.15.5:0/4226562464 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0319d0 con 0x7f0bc40cd300 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 -- 172.21.15.5:0/4226562464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 msgr2=0x7f0bc40cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 --2- 172.21.15.5:0/4226562464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 0x7f0bc40cd790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009e50 tx=0x7f0bbc02a730 comp rx=0 tx=0).stop 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 -- 172.21.15.5:0/4226562464 shutdown_connections 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 --2- 172.21.15.5:0/4226562464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc40cd300 0x7f0bc40cd790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 --2- 172.21.15.5:0/4226562464 >> v2:172.21.15.80:3300/0 conn(0x7f0bc415ce00 0x7f0bc415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 -- 172.21.15.5:0/4226562464 >> 172.21.15.5:0/4226562464 conn(0x7f0bc4066f20 msgr2=0x7f0bc406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 -- 172.21.15.5:0/4226562464 shutdown_connections 2024-04-25T10:03:49.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.102+0000 7f0bc3200640 1 -- 172.21.15.5:0/4226562464 wait complete. 2024-04-25T10:03:49.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.105+0000 7f0bc3200640 1 Processor -- start 2024-04-25T10:03:49.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.105+0000 7f0bc3200640 1 -- start start 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 0x7f0bc4138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bc4141da0 0x7f0bc40b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc415ce00 0x7f0bc4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc406e810 con 0x7f0bc40cd300 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc406dbe0 con 0x7f0bc4141da0 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc4071000 con 0x7f0bc415ce00 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 0x7f0bc4138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 0x7f0bc4138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49566/0 (socket says 172.21.15.5:49566) 2024-04-25T10:03:49.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.106+0000 7f0bc1e00640 1 -- 172.21.15.5:0/737254660 learned_addr learned my addr 172.21.15.5:0/737254660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:49.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc2800640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc415ce00 0x7f0bc4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1400640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bc4141da0 0x7f0bc40b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1e00640 1 -- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc415ce00 msgr2=0x7f0bc4159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1e00640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc415ce00 0x7f0bc4159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1e00640 1 -- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bc4141da0 msgr2=0x7f0bc40b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1e00640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bc4141da0 0x7f0bc40b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1e00640 1 -- 172.21.15.5:0/737254660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc40b2700 con 0x7f0bc40cd300 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.107+0000 7f0bc1e00640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 0x7f0bc4138f10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400c430 tx=0x7f0bb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.108+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40052d0 con 0x7f0bc40cd300 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.108+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb4005490 con 0x7f0bc40cd300 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.108+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4007260 con 0x7f0bc40cd300 2024-04-25T10:03:49.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.108+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc40b29e0 con 0x7f0bc40cd300 2024-04-25T10:03:49.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.108+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc40b2fa0 con 0x7f0bc40cd300 2024-04-25T10:03:49.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.109+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc4068f60 con 0x7f0bc40cd300 2024-04-25T10:03:49.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.111+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bb4007400 con 0x7f0bc40cd300 2024-04-25T10:03:49.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.112+0000 7f0bba000640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ba406b520 0x7f0ba406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.112+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0bb40898a0 con 0x7f0bc40cd300 2024-04-25T10:03:49.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.113+0000 7f0bc1400640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ba406b520 0x7f0ba406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.114+0000 7f0bc1400640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ba406b520 0x7f0ba406d9e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4140610 tx=0x7f0bb0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.117+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bb4058570 con 0x7f0bc40cd300 2024-04-25T10:03:49.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.355+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0bc406d360 con 0x7f0bc40cd300 2024-04-25T10:03:49.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.355+0000 7f0bba000640 1 -- 172.21.15.5:0/737254660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0bb4057e70 con 0x7f0bc40cd300 2024-04-25T10:03:49.360 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ba406b520 msgr2=0x7f0ba406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ba406b520 0x7f0ba406d9e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4140610 tx=0x7f0bb0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 msgr2=0x7f0bc4138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 0x7f0bc4138f10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400c430 tx=0x7f0bb400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 shutdown_connections 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bc415ce00 0x7f0bc4159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0ba406b520 0x7f0ba406d9e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bc4141da0 0x7f0bc40b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 --2- 172.21.15.5:0/737254660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc40cd300 0x7f0bc4138f10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 >> 172.21.15.5:0/737254660 conn(0x7f0bc4066f20 msgr2=0x7f0bc4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 shutdown_connections 2024-04-25T10:03:49.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:49.358+0000 7f0bc3200640 1 -- 172.21.15.5:0/737254660 wait complete. 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:03:49.382 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:49.383 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:49.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.383 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:03:49.437 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:03:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:03:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-04-25T10:03:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:03:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.439 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.440 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:03:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:49.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.442 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:49.519 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:03:49.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.524 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:03:49.581 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:03:49.581 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:49.581 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:03:49.581 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:03:49.581 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:49.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.582 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:03:49.582 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:03:49.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:03:49.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:03:49.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:03:49.582 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:03:49.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.736 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:03:49.787 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:49.787 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:49.788 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:49.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:03:49.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:49.872 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:49.926 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:49.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:49.927 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:49.981 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:03:49.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:50.119 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:50.119 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:03:50.119 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:03:50.143 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:50.143 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:03:50.143 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:03:50.143 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:03:50.143 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:03:50.172 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:50.172 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:03:50.173 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:03:50.173 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:03:50.173 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:03:50.173 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:03:50.173 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:03:50.173 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:03:50.176 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:03:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:03:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:03:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:03:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:03:50.182 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:03:50.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:50.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.271 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:03:50.358 INFO:teuthology.orchestra.run.smithi005.stdout:client4844 2024-04-25T10:03:50.359 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:50.359 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:50.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.418 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T10:03:50.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.492 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T10:03:50.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.552 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T10:03:50.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.613 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T10:03:50.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.680 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T10:03:50.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.740 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:03:50.808 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-04-25T10:03:50.808 INFO:teuthology.orchestra.run.smithi005.stdout:./rfile 2024-04-25T10:03:50.809 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir 2024-04-25T10:03:50.809 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir/pfile 2024-04-25T10:03:50.809 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir/childdir 2024-04-25T10:03:50.809 INFO:teuthology.orchestra.run.smithi005.stdout:./parentdir/childdir/cfile 2024-04-25T10:03:50.810 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:03:50.810 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:03:50.810 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:03:50.810 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:03:50.810 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:03:50.810 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:03:50.958 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:03:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:50.972 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:03:50.972 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:03:50.972 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:03:50.972 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:03:50.972 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T10:03:50.972 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:03:51.156 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-04-25T10:03:51.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:51.170 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:03:51.170 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:03:51.170 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:03:51.170 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:03:51.170 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T10:03:51.171 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:03:51.363 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:03:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:51.378 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:03:51.378 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:03:51.378 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:03:51.379 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:03:51.379 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T10:03:51.379 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:03:51.571 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628279 2024-04-25T10:03:51.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:51.585 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:03:51.586 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:03:51.586 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:03:51.586 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:03:51.586 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T10:03:51.586 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:03:51.785 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:03:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:51.799 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:03:51.799 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:03:51.800 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:03:51.800 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:03:51.800 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T10:03:51.800 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:03:51.996 INFO:teuthology.orchestra.run.smithi005.stdout:1099511628280 2024-04-25T10:03:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:52.010 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:52.010 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:52.114 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:03:52.115 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:52.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.382+0000 7f979d800640 1 Processor -- start 2024-04-25T10:03:52.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.383+0000 7f979d800640 1 -- start start 2024-04-25T10:03:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.384+0000 7f979d800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 0x7f97980b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.384+0000 7f979d800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f97980d07d0 0x7f97980b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.384+0000 7f979d800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97980cf410 con 0x7f97980d2380 2024-04-25T10:03:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.384+0000 7f979d800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97980d27c0 con 0x7f97980d07d0 2024-04-25T10:03:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.384+0000 7f979d800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980b28b0 con 0x7f9798161200 2024-04-25T10:03:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.385+0000 7f9797e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 0x7f97980b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.385+0000 7f9797e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 0x7f97980b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:43960/0 (socket says 172.21.15.5:43960) 2024-04-25T10:03:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.385+0000 7f9797e00640 1 -- 172.21.15.5:0/1467262020 learned_addr learned my addr 172.21.15.5:0/1467262020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.385+0000 7f979ce00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f97980d07d0 0x7f97980b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.386+0000 7f9797e00640 1 -- 172.21.15.5:0/1467262020 >> v2:172.21.15.80:3300/0 conn(0x7f97980d07d0 msgr2=0x7f97980b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.386+0000 7f9797e00640 1 --2- 172.21.15.5:0/1467262020 >> v2:172.21.15.80:3300/0 conn(0x7f97980d07d0 0x7f97980b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.386+0000 7f9797e00640 1 -- 172.21.15.5:0/1467262020 >> v1:172.21.15.80:6789/0 conn(0x7f97980d2380 legacy=0x7f97980d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.386+0000 7f9797e00640 1 -- 172.21.15.5:0/1467262020 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980cf6f0 con 0x7f9798161200 2024-04-25T10:03:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.387+0000 7f9797e00640 1 --2- 172.21.15.5:0/1467262020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 0x7f97980b4bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9788009e50 tx=0x7f978802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f28f0b955306b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.388+0000 7f9796a00640 1 -- 172.21.15.5:0/1467262020 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788036070 con 0x7f9798161200 2024-04-25T10:03:52.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.388+0000 7f9796a00640 1 -- 172.21.15.5:0/1467262020 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9788031600 con 0x7f9798161200 2024-04-25T10:03:52.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.388+0000 7f9796a00640 1 -- 172.21.15.5:0/1467262020 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880319d0 con 0x7f9798161200 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 -- 172.21.15.5:0/1467262020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 msgr2=0x7f97980b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 --2- 172.21.15.5:0/1467262020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 0x7f97980b4bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9788009e50 tx=0x7f978802a710 comp rx=0 tx=0).stop 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 -- 172.21.15.5:0/1467262020 shutdown_connections 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 --2- 172.21.15.5:0/1467262020 >> v2:172.21.15.80:3300/0 conn(0x7f97980d07d0 0x7f97980b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 --2- 172.21.15.5:0/1467262020 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9798161200 0x7f97980b4bb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 -- 172.21.15.5:0/1467262020 >> 172.21.15.5:0/1467262020 conn(0x7f979808af80 msgr2=0x7f97980d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 -- 172.21.15.5:0/1467262020 shutdown_connections 2024-04-25T10:03:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.389+0000 7f979d800640 1 -- 172.21.15.5:0/1467262020 wait complete. 2024-04-25T10:03:52.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.391+0000 7f979d800640 1 Processor -- start 2024-04-25T10:03:52.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.391+0000 7f979d800640 1 -- start start 2024-04-25T10:03:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979d800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97980d07d0 0x7f979806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979d800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97980d2380 0x7f97980b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 0x7f97981f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980b34c0 con 0x7f9798161200 2024-04-25T10:03:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979d800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980d42b0 con 0x7f97980d07d0 2024-04-25T10:03:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979d800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97980b3640 con 0x7f97980d2380 2024-04-25T10:03:52.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 0x7f97981f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.392+0000 7f979ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 0x7f97981f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49574/0 (socket says 172.21.15.5:49574) 2024-04-25T10:03:52.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 -- 172.21.15.5:0/118061658 learned_addr learned my addr 172.21.15.5:0/118061658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:52.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f9797e00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97980d07d0 0x7f979806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f9797400640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97980d2380 0x7f97980b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 -- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97980d2380 msgr2=0x7f97980b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97980d2380 0x7f97980b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 -- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97980d07d0 msgr2=0x7f979806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97980d07d0 0x7f979806c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 -- 172.21.15.5:0/118061658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980b6960 con 0x7f9798161200 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.393+0000 7f979ce00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 0x7f97981f3080 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f979000e930 tx=0x7f979000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.394+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790017070 con 0x7f9798161200 2024-04-25T10:03:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.394+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979000c980 con 0x7f9798161200 2024-04-25T10:03:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.394+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000cd70 con 0x7f9798161200 2024-04-25T10:03:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.394+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97980b6c40 con 0x7f9798161200 2024-04-25T10:03:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.394+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97980b7200 con 0x7f9798161200 2024-04-25T10:03:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.395+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9798068f60 con 0x7f9798161200 2024-04-25T10:03:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.396+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f979001a070 con 0x7f9798161200 2024-04-25T10:03:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.397+0000 7f978fe00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f977806b4d0 0x7f977806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.397+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f979008a7f0 con 0x7f9798161200 2024-04-25T10:03:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.398+0000 7f9797e00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f977806b4d0 0x7f977806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.399+0000 7f9797e00640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f977806b4d0 0x7f977806d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9788009e50 tx=0x7f9788009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.402+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9790059500 con 0x7f9798161200 2024-04-25T10:03:52.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.604+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f979806acf0 con 0x7f9798161200 2024-04-25T10:03:52.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.607+0000 7f978fe00640 1 -- 172.21.15.5:0/118061658 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f9790058e00 con 0x7f9798161200 2024-04-25T10:03:52.612 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:52.612 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:38.104579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.609+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f977806b4d0 msgr2=0x7f977806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.609+0000 7f979d800640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f977806b4d0 0x7f977806d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9788009e50 tx=0x7f9788009000 comp rx=0 tx=0).stop 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.609+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 msgr2=0x7f97981f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.609+0000 7f979d800640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 0x7f97981f3080 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f979000e930 tx=0x7f979000edf0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 shutdown_connections 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9798161200 0x7f97981f3080 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97980d2380 0x7f97980b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f977806b4d0 0x7f977806d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 --2- 172.21.15.5:0/118061658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97980d07d0 0x7f979806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 >> 172.21.15.5:0/118061658 conn(0x7f979808af80 msgr2=0x7f97980d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 shutdown_connections 2024-04-25T10:03:52.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.610+0000 7f979d800640 1 -- 172.21.15.5:0/118061658 wait complete. 2024-04-25T10:03:52.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:34.491131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:35.568287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:52.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.843+0000 7fda95400640 1 Processor -- start 2024-04-25T10:03:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.844+0000 7fda95400640 1 -- start start 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.846+0000 7fda95400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda9006e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.846+0000 7fda95400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda900ce570 con 0x7fda9006e670 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.846+0000 7fda95400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda900ce850 con 0x7fda9015b500 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.846+0000 7fda95400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9006ead0 con 0x7fda900cc840 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.846+0000 7fda8f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fda9006e670 0x7fda9015adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43924/0 (socket says 172.21.15.5:43924) 2024-04-25T10:03:52.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.846+0000 7fda8f400640 1 -- 172.21.15.5:0/3493486690 learned_addr learned my addr 172.21.15.5:0/3493486690 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8fe00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda9006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914619863 0 0) 0x7fda900ce570 con 0x7fda9006e670 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda74003520 con 0x7fda9006e670 2024-04-25T10:03:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907987174 0 0) 0x7fda74003520 con 0x7fda9006e670 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 msgr2=0x7fda9006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda9006e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 >> v1:172.21.15.80:6789/0 conn(0x7fda9015b500 legacy=0x7fda90071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900ceb30 con 0x7fda9006e670 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fda78001670 con 0x7fda9006e670 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 >> v1:172.21.15.5:6789/0 conn(0x7fda9006e670 legacy=0x7fda9015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda74004800 0x7fda74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7400bac0 con 0x7fda740039e0 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7400bc70 con 0x7fda74004800 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.847+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7400e080 con 0x7fda900cc840 2024-04-25T10:03:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8fe00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8f400640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda74004800 0x7fda74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8fe00640 1 -- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 msgr2=0x7fda74005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda94a00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8fe00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda74005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8fe00640 1 -- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda74004800 msgr2=0x7fda74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8fe00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda74004800 0x7fda74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8fe00640 1 -- 172.21.15.5:0/3493486690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74003520 con 0x7fda740039e0 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda94a00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900cc840 0x7fda74005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.848+0000 7fda8f400640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda74004800 0x7fda74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.849+0000 7fda8fe00640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda74003e00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda80007360 tx=0x7fda74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfab69d01eb51991 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.849+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80009be0 con 0x7fda740039e0 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.849+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda80036600 con 0x7fda740039e0 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.849+0000 7fda8ea00640 1 -- 172.21.15.5:0/3493486690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800369d0 con 0x7fda740039e0 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda95400640 1 -- 172.21.15.5:0/3493486690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 msgr2=0x7fda74003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda95400640 1 --2- 172.21.15.5:0/3493486690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda74003e00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fda80007360 tx=0x7fda74005830 comp rx=0 tx=0).stop 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda94a00640 1 -- 172.21.15.5:0/3493486690 reap_dead start 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda95400640 1 -- 172.21.15.5:0/3493486690 shutdown_connections 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda95400640 1 -- 172.21.15.5:0/3493486690 >> 172.21.15.5:0/3493486690 conn(0x7fda90066f20 msgr2=0x7fda9006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda95400640 1 -- 172.21.15.5:0/3493486690 shutdown_connections 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.850+0000 7fda95400640 1 -- 172.21.15.5:0/3493486690 wait complete. 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.852+0000 7fda95400640 1 Processor -- start 2024-04-25T10:03:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.852+0000 7fda95400640 1 -- start start 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda95400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda901f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda95400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda901570a0 0x7fda9014aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda95400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900b01f0 0x7fda900b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda95400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900ce1d0 con 0x7fda740039e0 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda95400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900b2d20 con 0x7fda901570a0 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda95400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda900ce350 con 0x7fda900b01f0 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda901f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.853+0000 7fda8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda901f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49596/0 (socket says 172.21.15.5:49596) 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 -- 172.21.15.5:0/2011478330 learned_addr learned my addr 172.21.15.5:0/2011478330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8f400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda901570a0 0x7fda9014aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 -- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900b01f0 msgr2=0x7fda900b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900b01f0 0x7fda900b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 -- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda901570a0 msgr2=0x7fda9014aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda901570a0 0x7fda9014aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 -- 172.21.15.5:0/2011478330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9014be10 con 0x7fda740039e0 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8f400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda901570a0 0x7fda9014aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:52.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.854+0000 7fda8fe00640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda901f6b40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda8002b070 tx=0x7fda80004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.855+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80045070 con 0x7fda740039e0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.855+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8002e9b0 con 0x7fda740039e0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.855+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80009be0 con 0x7fda740039e0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.855+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda901f7080 con 0x7fda740039e0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.855+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda901f7600 con 0x7fda740039e0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.856+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda900cd7f0 con 0x7fda740039e0 2024-04-25T10:03:52.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.857+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda80048070 con 0x7fda740039e0 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.858+0000 7fda87e00640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fda6c06b4d0 0x7fda6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:52.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.858+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda800aad30 con 0x7fda740039e0 2024-04-25T10:03:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.859+0000 7fda8f400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fda6c06b4d0 0x7fda6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.860+0000 7fda8f400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fda6c06b4d0 0x7fda6c06d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda9006e530 tx=0x7fda78009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:52.863+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda900cd7f0 con 0x7fda740039e0 2024-04-25T10:03:53.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.062+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda90068f60 con 0x7fda740039e0 2024-04-25T10:03:53.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.062+0000 7fda87e00640 1 -- 172.21.15.5:0/2011478330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda9014be10 con 0x7fda740039e0 2024-04-25T10:03:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fda6c06b4d0 msgr2=0x7fda6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fda6c06b4d0 0x7fda6c06d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda9006e530 tx=0x7fda78009040 comp rx=0 tx=0).stop 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 msgr2=0x7fda901f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda901f6b40 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fda8002b070 tx=0x7fda80004ab0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 shutdown_connections 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda900b01f0 0x7fda900b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fda6c06b4d0 0x7fda6c06d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda901570a0 0x7fda9014aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 --2- 172.21.15.5:0/2011478330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda740039e0 0x7fda901f6b40 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 >> 172.21.15.5:0/2011478330 conn(0x7fda90066f20 msgr2=0x7fda9015d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 shutdown_connections 2024-04-25T10:03:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.065+0000 7fda95400640 1 -- 172.21.15.5:0/2011478330 wait complete. 2024-04-25T10:03:53.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.297+0000 7faed2800640 1 Processor -- start 2024-04-25T10:03:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.297+0000 7faed2800640 1 -- start start 2024-04-25T10:03:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faed2800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faecc06d130 0x7faecc071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faed2800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc1453c0 con 0x7faecc071670 2024-04-25T10:03:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faed2800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc06d6a0 con 0x7faecc144fe0 2024-04-25T10:03:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faed2800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0a8560 con 0x7faecc06d130 2024-04-25T10:03:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faecbe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faecc071670 0x7faecc15c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43936/0 (socket says 172.21.15.5:43936) 2024-04-25T10:03:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faecbe00640 1 -- 172.21.15.5:0/1267306817 learned_addr learned my addr 172.21.15.5:0/1267306817 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.299+0000 7faec2a00640 1 --2- 172.21.15.5:0/1267306817 >> v2:172.21.15.80:3300/0 conn(0x7faecc06d130 0x7faecc071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456861265 0 0) 0x7faecc1453c0 con 0x7faecc071670 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb0003520 con 0x7faecc071670 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1208817538 0 0) 0x7faeb0003520 con 0x7faecc071670 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 >> v2:172.21.15.80:3300/0 conn(0x7faecc06d130 msgr2=0x7faecc071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 --2- 172.21.15.5:0/1267306817 >> v2:172.21.15.80:3300/0 conn(0x7faecc06d130 0x7faecc071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 >> v1:172.21.15.80:6789/0 conn(0x7faecc144fe0 legacy=0x7faecc06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc06d980 con 0x7faecc071670 2024-04-25T10:03:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7faec40019f0 con 0x7faecc071670 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 >> v1:172.21.15.5:6789/0 conn(0x7faecc071670 legacy=0x7faecc15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb00039b0 0x7faeb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeb0004800 0x7faeb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeb00051c0 0x7faeb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb000c030 con 0x7faeb0004800 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb000c1e0 con 0x7faeb00039b0 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.300+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb000c3e0 con 0x7faeb00051c0 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faec2a00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeb0004800 0x7faeb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faecb400640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb00039b0 0x7faeb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faec2a00640 1 -- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeb00051c0 msgr2=0x7faeb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faec2a00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeb00051c0 0x7faeb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faec2a00640 1 -- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb00039b0 msgr2=0x7faeb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faec2a00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb00039b0 0x7faeb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faec2a00640 1 -- 172.21.15.5:0/1267306817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0003520 con 0x7faeb0004800 2024-04-25T10:03:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faecb400640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb00039b0 0x7faeb0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:53.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.301+0000 7faecbe00640 1 -- 172.21.15.5:0/1267306817 reap_dead start 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.302+0000 7faec2a00640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeb0004800 0x7faeb0004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009ac0 tx=0x7faeb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb5c13dec6f67c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.302+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4036070 con 0x7faeb0004800 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.302+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb4031400 con 0x7faeb0004800 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.302+0000 7faecaa00640 1 -- 172.21.15.5:0/1267306817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4009690 con 0x7faeb0004800 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 -- 172.21.15.5:0/1267306817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeb0004800 msgr2=0x7faeb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeb0004800 0x7faeb0004c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009ac0 tx=0x7faeb0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 -- 172.21.15.5:0/1267306817 shutdown_connections 2024-04-25T10:03:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 --2- 172.21.15.5:0/1267306817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeb0004800 0x7faeb0004c80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 -- 172.21.15.5:0/1267306817 >> 172.21.15.5:0/1267306817 conn(0x7faecc06ab60 msgr2=0x7faecc06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 -- 172.21.15.5:0/1267306817 shutdown_connections 2024-04-25T10:03:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.303+0000 7faed2800640 1 -- 172.21.15.5:0/1267306817 wait complete. 2024-04-25T10:03:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.305+0000 7faed2800640 1 Processor -- start 2024-04-25T10:03:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.305+0000 7faed2800640 1 -- start start 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.306+0000 7faed2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb0004800 0x7faecc130710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.306+0000 7faed2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 0x7faecc140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.306+0000 7faed2800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faecc144460 0x7faecc1f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.306+0000 7faed2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc15be50 con 0x7faecc142e90 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.306+0000 7faed2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc06df40 con 0x7faeb0004800 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.306+0000 7faed2800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc071130 con 0x7faecc144460 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 0x7faecc140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 0x7faecc140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49622/0 (socket says 172.21.15.5:49622) 2024-04-25T10:03:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 -- 172.21.15.5:0/575603723 learned_addr learned my addr 172.21.15.5:0/575603723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faecb400640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb0004800 0x7faecc130710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 -- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faecc144460 msgr2=0x7faecc1f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faecbe00640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faecc144460 0x7faecc1f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faecc144460 0x7faecc1f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 -- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb0004800 msgr2=0x7faecc130710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb0004800 0x7faecc130710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.307+0000 7faec2a00640 1 -- 172.21.15.5:0/575603723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc130c50 con 0x7faecc142e90 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.308+0000 7faec2a00640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 0x7faecc140ba0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faeb4034040 tx=0x7faeb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.308+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb403f020 con 0x7faecc142e90 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.308+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb4037040 con 0x7faecc142e90 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.308+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4036070 con 0x7faecc142e90 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.308+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc1f7dd0 con 0x7faecc142e90 2024-04-25T10:03:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.308+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc1436a0 con 0x7faecc142e90 2024-04-25T10:03:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.309+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc068f60 con 0x7faecc142e90 2024-04-25T10:03:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.310+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faeb4042020 con 0x7faecc142e90 2024-04-25T10:03:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.311+0000 7faec3e00640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea806b500 0x7faea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.311+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faeb40aa180 con 0x7faecc142e90 2024-04-25T10:03:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.312+0000 7faecb400640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea806b500 0x7faea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.313+0000 7faecb400640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea806b500 0x7faea806d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.315+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faeb4049070 con 0x7faecc142e90 2024-04-25T10:03:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.513+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faecc071ed0 con 0x7faecc142e90 2024-04-25T10:03:53.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.514+0000 7faec3e00640 1 -- 172.21.15.5:0/575603723 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7faecc130c50 con 0x7faecc142e90 2024-04-25T10:03:53.519 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea806b500 msgr2=0x7faea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea806b500 0x7faea806d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 msgr2=0x7faecc140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 0x7faecc140ba0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faeb4034040 tx=0x7faeb4004730 comp rx=0 tx=0).stop 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 shutdown_connections 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faecc144460 0x7faecc1f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faecc142e90 0x7faecc140ba0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea806b500 0x7faea806d9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 --2- 172.21.15.5:0/575603723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb0004800 0x7faecc130710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 >> 172.21.15.5:0/575603723 conn(0x7faecc06ab60 msgr2=0x7faecc0a8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 shutdown_connections 2024-04-25T10:03:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.517+0000 7faed2800640 1 -- 172.21.15.5:0/575603723 wait complete. 2024-04-25T10:03:53.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:53.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:53.651 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.738+0000 7f6b35800640 1 Processor -- start 2024-04-25T10:03:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.738+0000 7f6b35800640 1 -- start start 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b35800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6b3006d130 0x7f6b30071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b35800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6b30071670 0x7f6b3015c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b35800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b301453c0 con 0x7f6b30144fe0 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b35800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300a8560 con 0x7f6b3006d130 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b35800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b3006d6a0 con 0x7f6b30071670 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b27e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6b3006d130 0x7f6b30071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b2ea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6b30144fe0 0x7f6b3006e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43952/0 (socket says 172.21.15.5:43952) 2024-04-25T10:03:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.739+0000 7f6b2ea00640 1 -- 172.21.15.5:0/506706257 learned_addr learned my addr 172.21.15.5:0/506706257 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.740+0000 7f6b2f400640 1 --2- 172.21.15.5:0/506706257 >> v2:172.21.15.80:3300/0 conn(0x7f6b30071670 0x7f6b3015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.740+0000 7f6b27e00640 1 -- 172.21.15.5:0/506706257 >> v2:172.21.15.80:3300/0 conn(0x7f6b30071670 msgr2=0x7f6b3015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.740+0000 7f6b27e00640 1 --2- 172.21.15.5:0/506706257 >> v2:172.21.15.80:3300/0 conn(0x7f6b30071670 0x7f6b3015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.740+0000 7f6b27e00640 1 -- 172.21.15.5:0/506706257 >> v1:172.21.15.5:6789/0 conn(0x7f6b30144fe0 legacy=0x7f6b3006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.740+0000 7f6b27e00640 1 -- 172.21.15.5:0/506706257 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3006d980 con 0x7f6b3006d130 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.741+0000 7f6b27e00640 1 --2- 172.21.15.5:0/506706257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b3006d130 0x7f6b30071130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6b3015bd10 tx=0x7f6b1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db31e657e568305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b2e000640 1 -- 172.21.15.5:0/506706257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18037070 con 0x7f6b3006d130 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b2e000640 1 -- 172.21.15.5:0/506706257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b18032440 con 0x7f6b3006d130 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b2e000640 1 -- 172.21.15.5:0/506706257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18032810 con 0x7f6b3006d130 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b35800640 1 -- 172.21.15.5:0/506706257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b3006d130 msgr2=0x7f6b30071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b35800640 1 --2- 172.21.15.5:0/506706257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b3006d130 0x7f6b30071130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6b3015bd10 tx=0x7f6b1802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b35800640 1 -- 172.21.15.5:0/506706257 shutdown_connections 2024-04-25T10:03:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b35800640 1 --2- 172.21.15.5:0/506706257 >> v2:172.21.15.80:3300/0 conn(0x7f6b30071670 0x7f6b3015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b35800640 1 --2- 172.21.15.5:0/506706257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b3006d130 0x7f6b30071130 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.742+0000 7f6b35800640 1 -- 172.21.15.5:0/506706257 >> 172.21.15.5:0/506706257 conn(0x7f6b3006ab60 msgr2=0x7f6b3006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.743+0000 7f6b35800640 1 -- 172.21.15.5:0/506706257 shutdown_connections 2024-04-25T10:03:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.743+0000 7f6b35800640 1 -- 172.21.15.5:0/506706257 wait complete. 2024-04-25T10:03:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.744+0000 7f6b35800640 1 Processor -- start 2024-04-25T10:03:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.744+0000 7f6b35800640 1 -- start start 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.745+0000 7f6b35800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b3006d130 0x7f6b301f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.745+0000 7f6b35800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b30071670 0x7f6b3015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.745+0000 7f6b35800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 0x7f6b300b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.745+0000 7f6b35800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3015be50 con 0x7f6b30144fe0 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.745+0000 7f6b35800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3006df40 con 0x7f6b3006d130 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.745+0000 7f6b35800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3015c8b0 con 0x7f6b30071670 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 0x7f6b300b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 0x7f6b300b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49638/0 (socket says 172.21.15.5:49638) 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 -- 172.21.15.5:0/2721003017 learned_addr learned my addr 172.21.15.5:0/2721003017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2ea00640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b3006d130 0x7f6b301f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b27e00640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b30071670 0x7f6b3015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 -- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b30071670 msgr2=0x7f6b3015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b30071670 0x7f6b3015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 -- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b3006d130 msgr2=0x7f6b301f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b3006d130 0x7f6b301f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.746+0000 7f6b2f400640 1 -- 172.21.15.5:0/2721003017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b301f7620 con 0x7f6b30144fe0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.747+0000 7f6b2f400640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 0x7f6b300b2420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6b28007e40 tx=0x7f6b2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.747+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28014620 con 0x7f6b30144fe0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.747+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28019070 con 0x7f6b30144fe0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.747+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28014a20 con 0x7f6b30144fe0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.747+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b301f7900 con 0x7f6b30144fe0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.747+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b301f7ec0 con 0x7f6b30144fe0 2024-04-25T10:03:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.748+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b30144840 con 0x7f6b30144fe0 2024-04-25T10:03:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.749+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b28014bc0 con 0x7f6b30144fe0 2024-04-25T10:03:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.751+0000 7f6b26a00640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b0c06b500 0x7f6b0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.751+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b2808ac70 con 0x7f6b30144fe0 2024-04-25T10:03:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.752+0000 7f6b2ea00640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b0c06b500 0x7f6b0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.753+0000 7f6b2ea00640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b0c06b500 0x7f6b0c06d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000f60 tx=0x7f6b20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.756+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b30144840 con 0x7f6b30144fe0 2024-04-25T10:03:53.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.952+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b30068f60 con 0x7f6b30144fe0 2024-04-25T10:03:53.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.952+0000 7f6b26a00640 1 -- 172.21.15.5:0/2721003017 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b28059950 con 0x7f6b30144fe0 2024-04-25T10:03:53.957 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b0c06b500 msgr2=0x7f6b0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b0c06b500 0x7f6b0c06d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000f60 tx=0x7f6b20009040 comp rx=0 tx=0).stop 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 msgr2=0x7f6b300b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 0x7f6b300b2420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6b28007e40 tx=0x7f6b2800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 shutdown_connections 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b30144fe0 0x7f6b300b2420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b30071670 0x7f6b3015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6b0c06b500 0x7f6b0c06d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 --2- 172.21.15.5:0/2721003017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b3006d130 0x7f6b301f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 >> 172.21.15.5:0/2721003017 conn(0x7f6b3006ab60 msgr2=0x7f6b300cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 shutdown_connections 2024-04-25T10:03:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:53.955+0000 7f6b35800640 1 -- 172.21.15.5:0/2721003017 wait complete. 2024-04-25T10:03:53.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.189+0000 7f647ca00640 1 Processor -- start 2024-04-25T10:03:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.189+0000 7f647ca00640 1 -- start start 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.191+0000 7f647ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f647815ce00 0x7f647815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.191+0000 7f647ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64780cd300 0x7f64780cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.191+0000 7f647ca00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6478071d30 con 0x7f6478141da0 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.191+0000 7f647ca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64780cdcd0 con 0x7f647815ce00 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.191+0000 7f647ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6478142180 con 0x7f64780cd300 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.192+0000 7f6476800640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6478141da0 0x7f647806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:36796/0 (socket says 172.21.15.5:36796) 2024-04-25T10:03:54.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.192+0000 7f6476800640 1 -- 172.21.15.5:0/3782292561 learned_addr learned my addr 172.21.15.5:0/3782292561 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.192+0000 7f6477200640 1 --2- 172.21.15.5:0/3782292561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64780cd300 0x7f64780cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.192+0000 7f646fe00640 1 --2- 172.21.15.5:0/3782292561 >> v2:172.21.15.80:3300/0 conn(0x7f647815ce00 0x7f647815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.193+0000 7f6477200640 1 -- 172.21.15.5:0/3782292561 >> v2:172.21.15.80:3300/0 conn(0x7f647815ce00 msgr2=0x7f647815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.193+0000 7f6477200640 1 --2- 172.21.15.5:0/3782292561 >> v2:172.21.15.80:3300/0 conn(0x7f647815ce00 0x7f647815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.193+0000 7f6477200640 1 -- 172.21.15.5:0/3782292561 >> v1:172.21.15.80:6789/0 conn(0x7f6478141da0 legacy=0x7f647806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.193+0000 7f6477200640 1 -- 172.21.15.5:0/3782292561 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647806d620 con 0x7f64780cd300 2024-04-25T10:03:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.193+0000 7f6475e00640 1 -- 172.21.15.5:0/3782292561 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261010166 0 0) 0x7f6478071d30 con 0x7f6478141da0 2024-04-25T10:03:54.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.194+0000 7f6477200640 1 --2- 172.21.15.5:0/3782292561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64780cd300 0x7f64780cd790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ad860d9bc45d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f6475e00640 1 -- 172.21.15.5:0/3782292561 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470007740 con 0x7f64780cd300 2024-04-25T10:03:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f6475e00640 1 -- 172.21.15.5:0/3782292561 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64700078e0 con 0x7f64780cd300 2024-04-25T10:03:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f6475e00640 1 -- 172.21.15.5:0/3782292561 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470007cb0 con 0x7f64780cd300 2024-04-25T10:03:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 -- 172.21.15.5:0/3782292561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64780cd300 msgr2=0x7f64780cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 --2- 172.21.15.5:0/3782292561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64780cd300 0x7f64780cd790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6470009d90 tx=0x7f647002a660 comp rx=0 tx=0).stop 2024-04-25T10:03:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 -- 172.21.15.5:0/3782292561 shutdown_connections 2024-04-25T10:03:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 --2- 172.21.15.5:0/3782292561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64780cd300 0x7f64780cd790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 --2- 172.21.15.5:0/3782292561 >> v2:172.21.15.80:3300/0 conn(0x7f647815ce00 0x7f647815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 -- 172.21.15.5:0/3782292561 >> 172.21.15.5:0/3782292561 conn(0x7f6478066f20 msgr2=0x7f647806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 -- 172.21.15.5:0/3782292561 shutdown_connections 2024-04-25T10:03:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.195+0000 7f647ca00640 1 -- 172.21.15.5:0/3782292561 wait complete. 2024-04-25T10:03:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.197+0000 7f647ca00640 1 Processor -- start 2024-04-25T10:03:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.197+0000 7f647ca00640 1 -- start start 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.198+0000 7f647ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 0x7f64780ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.198+0000 7f647ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6478141da0 0x7f647813e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.198+0000 7f647ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647815ce00 0x7f647813ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.198+0000 7f647ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647806e810 con 0x7f64780cd300 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.198+0000 7f647ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647806dbe0 con 0x7f647815ce00 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.198+0000 7f647ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6478071000 con 0x7f6478141da0 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 0x7f64780ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 0x7f64780ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49654/0 (socket says 172.21.15.5:49654) 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 -- 172.21.15.5:0/1886830296 learned_addr learned my addr 172.21.15.5:0/1886830296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f646fe00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6478141da0 0x7f647813e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6477200640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647815ce00 0x7f647813ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 -- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6478141da0 msgr2=0x7f647813e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6478141da0 0x7f647813e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 -- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647815ce00 msgr2=0x7f647813ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647815ce00 0x7f647813ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6476800640 1 -- 172.21.15.5:0/1886830296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64780c5f50 con 0x7f64780cd300 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f646fe00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6478141da0 0x7f647813e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.199+0000 7f6477200640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647815ce00 0x7f647813ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.200+0000 7f6476800640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 0x7f64780ca210 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f646800c4a0 tx=0x7f646800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.202+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468018070 con 0x7f64780cd300 2024-04-25T10:03:54.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.202+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64680074e0 con 0x7f64780cd300 2024-04-25T10:03:54.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.202+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468004a20 con 0x7f64780cd300 2024-04-25T10:03:54.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.202+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64780c6230 con 0x7f64780cd300 2024-04-25T10:03:54.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.202+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780c67f0 con 0x7f64780cd300 2024-04-25T10:03:54.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.203+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6478068f60 con 0x7f64780cd300 2024-04-25T10:03:54.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.204+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f646801b070 con 0x7f64780cd300 2024-04-25T10:03:54.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.205+0000 7f646ea00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f645406b520 0x7f645406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.205+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6468089820 con 0x7f64780cd300 2024-04-25T10:03:54.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.206+0000 7f646fe00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f645406b520 0x7f645406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.207+0000 7f646fe00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f645406b520 0x7f645406d9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f647815f7c0 tx=0x7f6460008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.209+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6468058580 con 0x7f64780cd300 2024-04-25T10:03:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.422+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64780ccae0 con 0x7f64780cd300 2024-04-25T10:03:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.422+0000 7f646ea00640 1 -- 172.21.15.5:0/1886830296 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6468057e80 con 0x7f64780cd300 2024-04-25T10:03:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:03:54.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f645406b520 msgr2=0x7f645406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f645406b520 0x7f645406d9e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f647815f7c0 tx=0x7f6460008040 comp rx=0 tx=0).stop 2024-04-25T10:03:54.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 msgr2=0x7f64780ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 0x7f64780ca210 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f646800c4a0 tx=0x7f646800c960 comp rx=0 tx=0).stop 2024-04-25T10:03:54.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 shutdown_connections 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f645406b520 0x7f645406d9e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647815ce00 0x7f647813ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6478141da0 0x7f647813e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 --2- 172.21.15.5:0/1886830296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64780cd300 0x7f64780ca210 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 >> 172.21.15.5:0/1886830296 conn(0x7f6478066f20 msgr2=0x7f647815b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.425+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 shutdown_connections 2024-04-25T10:03:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.426+0000 7f647ca00640 1 -- 172.21.15.5:0/1886830296 wait complete. 2024-04-25T10:03:54.451 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:54.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:54.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:54.594 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:03:54.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:54.595 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:54.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:54.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.865+0000 7feaa1600640 1 Processor -- start 2024-04-25T10:03:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.866+0000 7feaa1600640 1 -- start start 2024-04-25T10:03:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.867+0000 7feaa1600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fea9c0d07d0 0x7fea9c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.867+0000 7feaa1600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c0cf410 con 0x7fea9c161200 2024-04-25T10:03:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.867+0000 7feaa1600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c0cf6f0 con 0x7fea9c0d2380 2024-04-25T10:03:54.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.867+0000 7feaa1600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c0b28b0 con 0x7fea9c0d07d0 2024-04-25T10:03:54.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.868+0000 7fea9be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fea9c161200 0x7fea9c0d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43966/0 (socket says 172.21.15.5:43966) 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.868+0000 7fea9be00640 1 -- 172.21.15.5:0/1775218171 learned_addr learned my addr 172.21.15.5:0/1775218171 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.868+0000 7fea9b400640 1 --2- 172.21.15.5:0/1775218171 >> v2:172.21.15.80:3300/0 conn(0x7fea9c0d07d0 0x7fea9c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.868+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803847114 0 0) 0x7fea9c0cf410 con 0x7fea9c161200 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.868+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea80003520 con 0x7fea9c161200 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1136622943 0 0) 0x7fea80003520 con 0x7fea9c161200 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 >> v2:172.21.15.80:3300/0 conn(0x7fea9c0d07d0 msgr2=0x7fea9c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 --2- 172.21.15.5:0/1775218171 >> v2:172.21.15.80:3300/0 conn(0x7fea9c0d07d0 0x7fea9c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 >> v1:172.21.15.80:6789/0 conn(0x7fea9c0d2380 legacy=0x7fea9c0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0d3fd0 con 0x7fea9c161200 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fea8c0019f0 con 0x7fea9c161200 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 >> v1:172.21.15.5:6789/0 conn(0x7fea9c161200 legacy=0x7fea9c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea800039b0 0x7fea80003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea800051c0 0x7fea8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0b28b0 con 0x7fea80004800 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8000c040 con 0x7fea800039b0 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8000c240 con 0x7fea800051c0 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.869+0000 7fea9b400640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7feaa0c00640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea800051c0 0x7fea8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9be00640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea800039b0 0x7fea80003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9b400640 1 -- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea800051c0 msgr2=0x7fea8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9b400640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea800051c0 0x7fea8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9b400640 1 -- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea800039b0 msgr2=0x7fea80003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9b400640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea800039b0 0x7fea80003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9b400640 1 -- 172.21.15.5:0/1775218171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80003520 con 0x7fea80004800 2024-04-25T10:03:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7feaa0c00640 1 -- 172.21.15.5:0/1775218171 reap_dead start 2024-04-25T10:03:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.870+0000 7fea9be00640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea800039b0 0x7fea80003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.871+0000 7fea9b400640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea80004c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fea88009ac0 tx=0x7fea80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ce55bb01adc1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.871+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88036070 con 0x7fea80004800 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.871+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88031400 con 0x7fea80004800 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.871+0000 7fea9aa00640 1 -- 172.21.15.5:0/1775218171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88009690 con 0x7fea80004800 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 -- 172.21.15.5:0/1775218171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 msgr2=0x7fea80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea80004c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fea88009ac0 tx=0x7fea80005830 comp rx=0 tx=0).stop 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 -- 172.21.15.5:0/1775218171 shutdown_connections 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 --2- 172.21.15.5:0/1775218171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea80004c80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 -- 172.21.15.5:0/1775218171 >> 172.21.15.5:0/1775218171 conn(0x7fea9c08af80 msgr2=0x7fea9c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 -- 172.21.15.5:0/1775218171 shutdown_connections 2024-04-25T10:03:54.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.872+0000 7feaa1600640 1 -- 172.21.15.5:0/1775218171 wait complete. 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.874+0000 7feaa1600640 1 Processor -- start 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.874+0000 7feaa1600640 1 -- start start 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7feaa1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea9c1f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7feaa1600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9c0ca780 0x7fea9c0b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7feaa1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c0cac00 0x7fea9c0b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7feaa1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0b4d30 con 0x7fea80004800 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7feaa1600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0b4bb0 con 0x7fea9c0cac00 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7feaa1600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0b4eb0 con 0x7fea9c0ca780 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7fea9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea9c1f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7fea9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea9c1f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49678/0 (socket says 172.21.15.5:49678) 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.875+0000 7fea9be00640 1 -- 172.21.15.5:0/3192144433 learned_addr learned my addr 172.21.15.5:0/3192144433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9b400640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9c0ca780 0x7fea9c0b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7feaa0c00640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c0cac00 0x7fea9c0b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9be00640 1 -- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9c0ca780 msgr2=0x7fea9c0b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9be00640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9c0ca780 0x7fea9c0b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9be00640 1 -- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c0cac00 msgr2=0x7fea9c0b8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9be00640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c0cac00 0x7fea9c0b8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9be00640 1 -- 172.21.15.5:0/3192144433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0b7ac0 con 0x7fea80004800 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea9be00640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea9c1f8160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fea8c001a70 tx=0x7fea8c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c019070 con 0x7fea80004800 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c004d60 con 0x7fea80004800 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0072d0 con 0x7fea80004800 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.876+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c0b7da0 con 0x7fea80004800 2024-04-25T10:03:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.877+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c0b8360 con 0x7fea80004800 2024-04-25T10:03:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.878+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c068f60 con 0x7fea80004800 2024-04-25T10:03:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.879+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea8c01c070 con 0x7fea80004800 2024-04-25T10:03:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.879+0000 7fea93e00640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea7c06b550 0x7fea7c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.880+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fea8c08a490 con 0x7fea80004800 2024-04-25T10:03:54.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.880+0000 7fea9b400640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea7c06b550 0x7fea7c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:54.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.882+0000 7fea9b400640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea7c06b550 0x7fea7c06da10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea88033000 tx=0x7fea88008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:54.884+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea8c059160 con 0x7fea80004800 2024-04-25T10:03:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.119+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea9c0d04b0 con 0x7fea80004800 2024-04-25T10:03:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.121+0000 7fea93e00640 1 -- 172.21.15.5:0/3192144433 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7fea8c058a60 con 0x7fea80004800 2024-04-25T10:03:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:55.126 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:03:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea7c06b550 msgr2=0x7fea7c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea7c06b550 0x7fea7c06da10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea88033000 tx=0x7fea88008030 comp rx=0 tx=0).stop 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 msgr2=0x7fea9c1f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea9c1f8160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fea8c001a70 tx=0x7fea8c00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 shutdown_connections 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fea7c06b550 0x7fea7c06da10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.123+0000 7feaa1600640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c0cac00 0x7fea9c0b8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.124+0000 7feaa1600640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea9c0ca780 0x7fea9c0b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.124+0000 7feaa1600640 1 --2- 172.21.15.5:0/3192144433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea80004800 0x7fea9c1f8160 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.124+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 >> 172.21.15.5:0/3192144433 conn(0x7fea9c08af80 msgr2=0x7fea9c06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.124+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 shutdown_connections 2024-04-25T10:03:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.124+0000 7feaa1600640 1 -- 172.21.15.5:0/3192144433 wait complete. 2024-04-25T10:03:55.147 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-25T10:03:55.393 INFO:teuthology.orchestra.run.smithi080.stdout:{"message":"","return_code":0} 2024-04-25T10:03:55.393 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:03:55.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.610+0000 7f09c9e00640 1 Processor -- start 2024-04-25T10:03:55.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.610+0000 7f09c9e00640 1 -- start start 2024-04-25T10:03:55.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09c4161200 0x7f09c40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c40d2380 0x7f09c40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09c40d0600 0x7f09c40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c40d27c0 con 0x7f09c40d0600 2024-04-25T10:03:55.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c40b28b0 con 0x7f09c4161200 2024-04-25T10:03:55.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40cf410 con 0x7f09c40d2380 2024-04-25T10:03:55.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.612+0000 7f09c9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09c40d0600 0x7f09c40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09c40d0600 0x7f09c40ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49682/0 (socket says 172.21.15.5:49682) 2024-04-25T10:03:55.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 -- 172.21.15.5:0/49531481 learned_addr learned my addr 172.21.15.5:0/49531481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:55.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c3e00640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c40d2380 0x7f09c40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c8a00640 1 --2- 172.21.15.5:0/49531481 >> v2:172.21.15.80:3300/0 conn(0x7f09c4161200 0x7f09c40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 -- 172.21.15.5:0/49531481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c40d2380 msgr2=0x7f09c40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:55.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c40d2380 0x7f09c40d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 -- 172.21.15.5:0/49531481 >> v2:172.21.15.80:3300/0 conn(0x7f09c4161200 msgr2=0x7f09c40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:55.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 --2- 172.21.15.5:0/49531481 >> v2:172.21.15.80:3300/0 conn(0x7f09c4161200 0x7f09c40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c9400640 1 -- 172.21.15.5:0/49531481 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40cf6f0 con 0x7f09c40d0600 2024-04-25T10:03:55.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c3e00640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c40d2380 0x7f09c40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:55.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.613+0000 7f09c8a00640 1 --2- 172.21.15.5:0/49531481 >> v2:172.21.15.80:3300/0 conn(0x7f09c4161200 0x7f09c40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:55.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.615+0000 7f09c9400640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d0600 0x7f09c40ce280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09c408bed0 tx=0x7f09bc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a865748c9bc25f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.615+0000 7f09c3400640 1 -- 172.21.15.5:0/49531481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc036070 con 0x7f09c40d0600 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.615+0000 7f09c3400640 1 -- 172.21.15.5:0/49531481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc031600 con 0x7f09c40d0600 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.615+0000 7f09c3400640 1 -- 172.21.15.5:0/49531481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc0319d0 con 0x7f09c40d0600 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 -- 172.21.15.5:0/49531481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d0600 msgr2=0x7f09c40ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d0600 0x7f09c40ce280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f09c408bed0 tx=0x7f09bc02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 -- 172.21.15.5:0/49531481 shutdown_connections 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d0600 0x7f09c40ce280 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 --2- 172.21.15.5:0/49531481 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c40d2380 0x7f09c40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 --2- 172.21.15.5:0/49531481 >> v2:172.21.15.80:3300/0 conn(0x7f09c4161200 0x7f09c40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 -- 172.21.15.5:0/49531481 >> 172.21.15.5:0/49531481 conn(0x7f09c408af80 msgr2=0x7f09c40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:55.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 -- 172.21.15.5:0/49531481 shutdown_connections 2024-04-25T10:03:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.616+0000 7f09c9e00640 1 -- 172.21.15.5:0/49531481 wait complete. 2024-04-25T10:03:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.618+0000 7f09c9e00640 1 Processor -- start 2024-04-25T10:03:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.618+0000 7f09c9e00640 1 -- start start 2024-04-25T10:03:55.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c40d0600 0x7f09c414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 0x7f09c40b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c9e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c4161200 0x7f09c4150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c9e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40ce7c0 con 0x7f09c40d2380 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c9e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40d42b0 con 0x7f09c40d0600 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c9e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40ce940 con 0x7f09c4161200 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 0x7f09c40b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 0x7f09c40b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49696/0 (socket says 172.21.15.5:49696) 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.619+0000 7f09c3e00640 1 -- 172.21.15.5:0/497719881 learned_addr learned my addr 172.21.15.5:0/497719881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c8a00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c40d0600 0x7f09c414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c9400640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c4161200 0x7f09c4150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c3e00640 1 -- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c4161200 msgr2=0x7f09c4150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c3e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c4161200 0x7f09c4150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c3e00640 1 -- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c40d0600 msgr2=0x7f09c414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c3e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c40d0600 0x7f09c414faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c3e00640 1 -- 172.21.15.5:0/497719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40b6450 con 0x7f09c40d2380 2024-04-25T10:03:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c3e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 0x7f09c40b74f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f09b000e810 tx=0x7f09b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:55.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.620+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000f040 con 0x7f09c40d2380 2024-04-25T10:03:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.621+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b0004030 con 0x7f09c40d2380 2024-04-25T10:03:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.621+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c40b6730 con 0x7f09c40d2380 2024-04-25T10:03:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.621+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0004400 con 0x7f09c40d2380 2024-04-25T10:03:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.621+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c40b6cf0 con 0x7f09c40d2380 2024-04-25T10:03:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.622+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c4068f60 con 0x7f09c40d2380 2024-04-25T10:03:55.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.623+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09b000c6d0 con 0x7f09c40d2380 2024-04-25T10:03:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.624+0000 7f09c0c00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f09a406b520 0x7f09a406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.624+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09b0089ff0 con 0x7f09c40d2380 2024-04-25T10:03:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.624+0000 7f09c8a00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f09a406b520 0x7f09a406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:55.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.626+0000 7f09c8a00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f09a406b520 0x7f09a406d9e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f09b8006fd0 tx=0x7f09b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:55.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.628+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09b0017020 con 0x7f09c40d2380 2024-04-25T10:03:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.857+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c40b2e80 con 0x7f09c40d2380 2024-04-25T10:03:55.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.858+0000 7f09c0c00640 1 -- 172.21.15.5:0/497719881 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f09b0058ce0 con 0x7f09c40d2380 2024-04-25T10:03:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:03:55.863 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.860+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f09a406b520 msgr2=0x7f09a406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.860+0000 7f09c9e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f09a406b520 0x7f09a406d9e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f09b8006fd0 tx=0x7f09b8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.860+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 msgr2=0x7f09c40b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.860+0000 7f09c9e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 0x7f09c40b74f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f09b000e810 tx=0x7f09b000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 shutdown_connections 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09c4161200 0x7f09c4150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c40d2380 0x7f09c40b74f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f09a406b520 0x7f09a406d9e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 --2- 172.21.15.5:0/497719881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c40d0600 0x7f09c414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 >> 172.21.15.5:0/497719881 conn(0x7f09c408af80 msgr2=0x7f09c415dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 shutdown_connections 2024-04-25T10:03:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:03:55.861+0000 7f09c9e00640 1 -- 172.21.15.5:0/497719881 wait complete. 2024-04-25T10:03:55.883 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json tag path /parentdir mytag 2024-04-25T10:03:56.111 INFO:teuthology.orchestra.run.smithi080.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T10:03:56.111 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T10:04:06.113 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T10:04:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.227+0000 7fe06f60cf40 1 Processor -- start 2024-04-25T10:04:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.227+0000 7fe06f60cf40 1 -- start start 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.228+0000 7fe06f60cf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fb874f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.228+0000 7fe06f60cf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 0x560a1fc11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.228+0000 7fe06f60cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc2c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.228+0000 7fe06f60cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1faffdd0 con 0x560a1fc15b20 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.228+0000 7fe06f60cf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1fba31c0 con 0x560a1fba2db0 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.228+0000 7fe06f60cf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a1fba26d0 con 0x560a1fba4bd0 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc2c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc2c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59452/0 (socket says 172.21.15.5:59452) 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 -- 172.21.15.5:0/916148685 learned_addr learned my addr 172.21.15.5:0/916148685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06bc00640 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fb874f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06b200640 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 0x560a1fc11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 -- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 msgr2=0x560a1fc11fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 0x560a1fc11fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 -- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 msgr2=0x560a1fb874f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fb874f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06c600640 1 -- 172.21.15.5:0/916148685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a1fb57740 con 0x560a1fc15b20 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.229+0000 7fe06bc00640 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fb874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.230+0000 7fe06c600640 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc2c5f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe064001660 tx=0x7fe064001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80abe652288ed93b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.230+0000 7fe069e00640 1 -- 172.21.15.5:0/916148685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064003650 con 0x560a1fc15b20 2024-04-25T10:04:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.230+0000 7fe069e00640 1 -- 172.21.15.5:0/916148685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe064003830 con 0x560a1fc15b20 2024-04-25T10:04:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.230+0000 7fe069e00640 1 -- 172.21.15.5:0/916148685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064003ca0 con 0x560a1fc15b20 2024-04-25T10:04:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.230+0000 7fe06f60cf40 1 -- 172.21.15.5:0/916148685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 msgr2=0x560a1fc2c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.230+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc2c5f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe064001660 tx=0x7fe064001b20 comp rx=0 tx=0).stop 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 -- 172.21.15.5:0/916148685 shutdown_connections 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc2c5f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 0x560a1fc11fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/916148685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fb874f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 -- 172.21.15.5:0/916148685 >> 172.21.15.5:0/916148685 conn(0x560a1fb57c60 msgr2=0x560a1fba1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 -- 172.21.15.5:0/916148685 shutdown_connections 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 -- 172.21.15.5:0/916148685 wait complete. 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.231+0000 7fe06f60cf40 1 Processor -- start 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.232+0000 7fe06f60cf40 1 -- start start 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06f60cf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fc2b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06f60cf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 0x560a1fc31230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06f60cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc31770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06f60cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1fc2ceb0 con 0x560a1fc15b20 2024-04-25T10:04:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06f60cf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a1fb574a0 con 0x560a1fba2db0 2024-04-25T10:04:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06f60cf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a1fc32e90 con 0x560a1fba4bd0 2024-04-25T10:04:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc31770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59462/0 (socket says 172.21.15.5:59462) 2024-04-25T10:04:06.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 -- 172.21.15.5:0/2187735557 learned_addr learned my addr 172.21.15.5:0/2187735557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 -- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 msgr2=0x560a1fc31230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560a1fba4bd0 0x560a1fc31230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 -- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 msgr2=0x560a1fc2b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560a1fba2db0 0x560a1fc2b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.233+0000 7fe06c600640 1 -- 172.21.15.5:0/2187735557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a1fc2b920 con 0x560a1fc15b20 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.234+0000 7fe06c600640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc31770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x560a1fcc13d0 tx=0x7fe064001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.234+0000 7fe068a00640 1 -- 172.21.15.5:0/2187735557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064024070 con 0x560a1fc15b20 2024-04-25T10:04:06.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.234+0000 7fe068a00640 1 -- 172.21.15.5:0/2187735557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe064019040 con 0x560a1fc15b20 2024-04-25T10:04:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.234+0000 7fe068a00640 1 -- 172.21.15.5:0/2187735557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064013a70 con 0x560a1fc15b20 2024-04-25T10:04:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.234+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560a1fc10b20 con 0x560a1fc15b20 2024-04-25T10:04:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.234+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a1fc10de0 con 0x560a1fc15b20 2024-04-25T10:04:06.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.236+0000 7fe068a00640 1 -- 172.21.15.5:0/2187735557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe064027070 con 0x560a1fc15b20 2024-04-25T10:04:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.237+0000 7fe068a00640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe04c069ee0 0x7fe04c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.237+0000 7fe068a00640 1 -- 172.21.15.5:0/2187735557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe06408a220 con 0x560a1fc15b20 2024-04-25T10:04:06.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.238+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x560a1fc18ae0 0x560a1fb9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.238+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x560a1fb57040 con 0x560a1fc18ae0 2024-04-25T10:04:06.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.238+0000 7fe06bc00640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe04c069ee0 0x7fe04c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.238+0000 7fe06b200640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x560a1fc18ae0 0x560a1fb9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06b200640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x560a1fc18ae0 0x560a1fb9ae10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06b200640 1 -- 172.21.15.5:0/2187735557 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fe05800b6d0 con 0x560a1fc18ae0 2024-04-25T10:04:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06bc00640 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe04c069ee0 0x7fe04c06c3a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe05c056000 tx=0x7fe05c04e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x560a1fc18ae0 msgr2=0x560a1fb9ae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x560a1fc18ae0 0x560a1fb9ae10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe04c069ee0 msgr2=0x7fe04c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe04c069ee0 0x7fe04c06c3a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe05c056000 tx=0x7fe05c04e000 comp rx=0 tx=0).stop 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 msgr2=0x560a1fc31770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 --2- 172.21.15.5:0/2187735557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1fc15b20 0x560a1fc31770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x560a1fcc13d0 tx=0x7fe064001250 comp rx=0 tx=0).stop 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06c600640 1 -- 172.21.15.5:0/2187735557 reap_dead start 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 shutdown_connections 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.239+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 >> 172.21.15.5:0/2187735557 conn(0x560a1fb57c60 msgr2=0x560a1fc2c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.240+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 shutdown_connections 2024-04-25T10:04:06.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.240+0000 7fe06f60cf40 1 -- 172.21.15.5:0/2187735557 wait complete. 2024-04-25T10:04:06.249 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-04-25T10:04:06.250 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T10:04:06.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.337+0000 7f2d703fef40 1 Processor -- start 2024-04-25T10:04:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.337+0000 7f2d703fef40 1 -- start start 2024-04-25T10:04:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d703fef40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 0x55f8dbce6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d703fef40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f8dbcc97e0 0x55f8dbc42530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d703fef40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbc42a70 0x55f8dbce4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d703fef40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dbbb7dd0 con 0x7f2d540022a0 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d703fef40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dbc5d380 con 0x55f8dbcc97e0 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d703fef40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8dbc5a1b0 con 0x55f8dbc42a70 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 0x55f8dbce6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 0x55f8dbce6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59474/0 (socket says 172.21.15.5:59474) 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 -- 172.21.15.5:0/2189597162 learned_addr learned my addr 172.21.15.5:0/2189597162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 -- 172.21.15.5:0/2189597162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbc42a70 msgr2=0x55f8dbce4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbc42a70 0x55f8dbce4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 -- 172.21.15.5:0/2189597162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f8dbcc97e0 msgr2=0x55f8dbc42530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f8dbcc97e0 0x55f8dbc42530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.338+0000 7f2d6d600640 1 -- 172.21.15.5:0/2189597162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dbcc9410 con 0x7f2d540022a0 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d6d600640 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 0x55f8dbce6fb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2d60045c70 tx=0x7f2d60051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7ffab8644d3f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d67400640 1 -- 172.21.15.5:0/2189597162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6004f860 con 0x7f2d540022a0 2024-04-25T10:04:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d67400640 1 -- 172.21.15.5:0/2189597162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6004fa00 con 0x7f2d540022a0 2024-04-25T10:04:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d67400640 1 -- 172.21.15.5:0/2189597162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6004fdf0 con 0x7f2d540022a0 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 -- 172.21.15.5:0/2189597162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 msgr2=0x55f8dbce6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 0x55f8dbce6fb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2d60045c70 tx=0x7f2d60051000 comp rx=0 tx=0).stop 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 -- 172.21.15.5:0/2189597162 shutdown_connections 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbc42a70 0x55f8dbce4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f8dbcc97e0 0x55f8dbc42530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 --2- 172.21.15.5:0/2189597162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d540022a0 0x55f8dbce6fb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.339+0000 7f2d703fef40 1 -- 172.21.15.5:0/2189597162 >> 172.21.15.5:0/2189597162 conn(0x55f8dbc0fc60 msgr2=0x55f8dbc0f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.340+0000 7f2d703fef40 1 -- 172.21.15.5:0/2189597162 shutdown_connections 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.340+0000 7f2d703fef40 1 -- 172.21.15.5:0/2189597162 wait complete. 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.340+0000 7f2d703fef40 1 Processor -- start 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 -- start start 2024-04-25T10:04:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8dbc42a70 0x55f8dbce1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbcc97e0 0x55f8dbce2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d540022a0 0x55f8dbccbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dbce6a90 con 0x55f8dbc42a70 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8dbce7750 con 0x7f2d540022a0 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.341+0000 7f2d703fef40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8dbce4bc0 con 0x55f8dbcc97e0 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8dbc42a70 0x55f8dbce1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8dbc42a70 0x55f8dbce1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59484/0 (socket says 172.21.15.5:59484) 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 -- 172.21.15.5:0/3776740453 learned_addr learned my addr 172.21.15.5:0/3776740453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbcc97e0 0x55f8dbce2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6e000640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d540022a0 0x55f8dbccbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 -- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbcc97e0 msgr2=0x55f8dbce2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f8dbcc97e0 0x55f8dbce2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 -- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d540022a0 msgr2=0x55f8dbccbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d540022a0 0x55f8dbccbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 -- 172.21.15.5:0/3776740453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8dbce2990 con 0x55f8dbc42a70 2024-04-25T10:04:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6e000640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d540022a0 0x55f8dbccbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.342+0000 7f2d6d600640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8dbc42a70 0x55f8dbce1b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009810 tx=0x7f2d60003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.343+0000 7f2d66000640 1 -- 172.21.15.5:0/3776740453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60064520 con 0x55f8dbc42a70 2024-04-25T10:04:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.343+0000 7f2d66000640 1 -- 172.21.15.5:0/3776740453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d600646c0 con 0x55f8dbc42a70 2024-04-25T10:04:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.343+0000 7f2d66000640 1 -- 172.21.15.5:0/3776740453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60064ab0 con 0x55f8dbc42a70 2024-04-25T10:04:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.343+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f8dbccc7c0 con 0x55f8dbc42a70 2024-04-25T10:04:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.343+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8dbce8310 con 0x55f8dbc42a70 2024-04-25T10:04:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.345+0000 7f2d66000640 1 -- 172.21.15.5:0/3776740453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d60064c50 con 0x55f8dbc42a70 2024-04-25T10:04:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.346+0000 7f2d66000640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d50069eb0 0x7f2d5006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.346+0000 7f2d66000640 1 -- 172.21.15.5:0/3776740453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d600cde70 con 0x55f8dbc42a70 2024-04-25T10:04:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.347+0000 7f2d703fef40 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f8dbcc6270 0x55f8dbcc6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.347+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x55f8dbcc6bc0 con 0x55f8dbcc6270 2024-04-25T10:04:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.347+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d50069eb0 0x7f2d5006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.347+0000 7f2d6e000640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f8dbcc6270 0x55f8dbcc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d6e000640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f8dbcc6270 0x55f8dbcc6680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d50069eb0 0x7f2d5006c370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c005630 tx=0x7f2d5c0055c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d6e000640 1 -- 172.21.15.5:0/3776740453 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f2d68003450 con 0x55f8dbcc6270 2024-04-25T10:04:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f8dbcc6270 msgr2=0x55f8dbcc6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d703fef40 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f8dbcc6270 0x55f8dbcc6680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d50069eb0 msgr2=0x7f2d5006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d703fef40 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d50069eb0 0x7f2d5006c370 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c005630 tx=0x7f2d5c0055c0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8dbc42a70 msgr2=0x55f8dbce1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d703fef40 1 --2- 172.21.15.5:0/3776740453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8dbc42a70 0x55f8dbce1b40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009810 tx=0x7f2d60003020 comp rx=0 tx=0).stop 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.348+0000 7f2d6e000640 1 -- 172.21.15.5:0/3776740453 reap_dead start 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.349+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 shutdown_connections 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.349+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 >> 172.21.15.5:0/3776740453 conn(0x55f8dbc0fc60 msgr2=0x55f8dbc42330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.349+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 shutdown_connections 2024-04-25T10:04:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.349+0000 7f2d703fef40 1 -- 172.21.15.5:0/3776740453 wait complete. 2024-04-25T10:04:06.358 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-04-25T10:04:06.359 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:04:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.457+0000 7f35aca3bf40 1 Processor -- start 2024-04-25T10:04:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 -- start start 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084e5c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b970 0x557084ee2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 0x557084ee2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557084dd1dd0 con 0x557084e5b3d0 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557084dd5d30 con 0x7f35900030f0 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.458+0000 7f35aca3bf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557084ee33a0 con 0x557084e5b970 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 0x557084ee2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 0x557084ee2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59508/0 (socket says 172.21.15.5:59508) 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 -- 172.21.15.5:0/3760336050 learned_addr learned my addr 172.21.15.5:0/3760336050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35a9c00640 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084e5c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 -- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b970 msgr2=0x557084ee2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b970 0x557084ee2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 -- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 msgr2=0x557084e5c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084e5c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35aa600640 1 -- 172.21.15.5:0/3760336050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557084ee30a0 con 0x557084e5b3d0 2024-04-25T10:04:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.459+0000 7f35a9c00640 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084e5c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35aa600640 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 0x557084ee2960 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f35a40025f0 tx=0x7f35a4002ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e1bb5ba423c42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35a3400640 1 -- 172.21.15.5:0/3760336050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4016020 con 0x557084e5b3d0 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35a3400640 1 -- 172.21.15.5:0/3760336050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4012740 con 0x557084e5b3d0 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35a3400640 1 -- 172.21.15.5:0/3760336050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4012b30 con 0x557084e5b3d0 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3760336050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 msgr2=0x557084ee2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 0x557084ee2960 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f35a40025f0 tx=0x7f35a4002ab0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.460+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3760336050 shutdown_connections 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084e5b3d0 0x557084ee2960 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b970 0x557084ee2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3760336050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084e5c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3760336050 >> 172.21.15.5:0/3760336050 conn(0x557084e29c60 msgr2=0x557084f05cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3760336050 shutdown_connections 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3760336050 wait complete. 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.461+0000 7f35aca3bf40 1 Processor -- start 2024-04-25T10:04:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 -- start start 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b3d0 0x557084f9bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084f9c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084ee1140 0x557084f9c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557084ee3670 con 0x557084ee1140 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557084e741a0 con 0x7f35900030f0 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.462+0000 7f35aca3bf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557084e76f10 con 0x557084e5b3d0 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084ee1140 0x557084f9c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084ee1140 0x557084f9c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59514/0 (socket says 172.21.15.5:59514) 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 -- 172.21.15.5:0/3386594597 learned_addr learned my addr 172.21.15.5:0/3386594597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35a9c00640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b3d0 0x557084f9bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 -- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b3d0 msgr2=0x557084f9bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557084e5b3d0 0x557084f9bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 -- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 msgr2=0x557084f9c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35900030f0 0x557084f9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.463+0000 7f35aa600640 1 -- 172.21.15.5:0/3386594597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557084f03980 con 0x557084ee1140 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.464+0000 7f35aa600640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084ee1140 0x557084f9c840 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f35a4015040 tx=0x7f35a40033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.464+0000 7f35a2000640 1 -- 172.21.15.5:0/3386594597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4026480 con 0x557084ee1140 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.464+0000 7f35a2000640 1 -- 172.21.15.5:0/3386594597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a4026620 con 0x557084ee1140 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.464+0000 7f35a2000640 1 -- 172.21.15.5:0/3386594597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a40269f0 con 0x557084ee1140 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.464+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557084e5cc60 con 0x557084ee1140 2024-04-25T10:04:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.464+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557084f03c20 con 0x557084ee1140 2024-04-25T10:04:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.466+0000 7f35a2000640 1 -- 172.21.15.5:0/3386594597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35a4019020 con 0x557084ee1140 2024-04-25T10:04:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.467+0000 7f35a2000640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358c069eb0 0x7f358c06c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.467+0000 7f35a2000640 1 -- 172.21.15.5:0/3386594597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35a408b280 con 0x557084ee1140 2024-04-25T10:04:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.467+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557084efbf20 0x557084efc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.467+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x557084efc8e0 con 0x557084efbf20 2024-04-25T10:04:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.468+0000 7f35a9c00640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358c069eb0 0x7f358c06c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.468+0000 7f35a9200640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557084efbf20 0x557084efc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.468+0000 7f35a9200640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557084efbf20 0x557084efc3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35a9c00640 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358c069eb0 0x7f358c06c370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x557084e58540 tx=0x7f359c001490 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35a9200640 1 -- 172.21.15.5:0/3386594597 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f3598001470 con 0x557084efbf20 2024-04-25T10:04:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557084efbf20 msgr2=0x557084efc3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557084efbf20 0x557084efc3a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358c069eb0 msgr2=0x7f358c06c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358c069eb0 0x7f358c06c370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x557084e58540 tx=0x7f359c001490 comp rx=0 tx=0).stop 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084ee1140 msgr2=0x557084f9c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 --2- 172.21.15.5:0/3386594597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557084ee1140 0x557084f9c840 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f35a4015040 tx=0x7f35a40033a0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aa600640 1 -- 172.21.15.5:0/3386594597 reap_dead start 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 shutdown_connections 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 >> 172.21.15.5:0/3386594597 conn(0x557084e29c60 msgr2=0x557084e57a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 shutdown_connections 2024-04-25T10:04:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.469+0000 7f35aca3bf40 1 -- 172.21.15.5:0/3386594597 wait complete. 2024-04-25T10:04:06.479 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-04-25T10:04:06.479 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:04:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.580+0000 7f8595a20f40 1 Processor -- start 2024-04-25T10:04:06.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.580+0000 7f8595a20f40 1 -- start start 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8595a20f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x5647105444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8595a20f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105ebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8595a20f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105ec080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8595a20f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647104bddd0 con 0x5647105cfa60 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8595a20f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647104c1d30 con 0x564710516070 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8595a20f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564710543ef0 con 0x7f85780022d0 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8593600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105ec080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8593600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105ec080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59522/0 (socket says 172.21.15.5:59522) 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8593600640 1 -- 172.21.15.5:0/432912615 learned_addr learned my addr 172.21.15.5:0/432912615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8592c00640 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x5647105444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8593600640 1 -- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 msgr2=0x5647105444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8593600640 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x5647105444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f8593600640 1 -- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 msgr2=0x5647105ebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.581+0000 7f858a200640 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105ebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8593600640 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105ebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8593600640 1 -- 172.21.15.5:0/432912615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647105cf480 con 0x5647105cfa60 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8592c00640 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x5647105444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8593600640 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105ec080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x564710564f20 tx=0x7f858c006850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6034fb0cebc9a20a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8591800640 1 -- 172.21.15.5:0/432912615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c003650 con 0x5647105cfa60 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8591800640 1 -- 172.21.15.5:0/432912615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c0037f0 con 0x5647105cfa60 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.582+0000 7f8591800640 1 -- 172.21.15.5:0/432912615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c003ba0 con 0x5647105cfa60 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 -- 172.21.15.5:0/432912615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 msgr2=0x5647105ec080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105ec080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x564710564f20 tx=0x7f858c006850 comp rx=0 tx=0).stop 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 -- 172.21.15.5:0/432912615 shutdown_connections 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105ec080 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105ebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 --2- 172.21.15.5:0/432912615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x5647105444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 -- 172.21.15.5:0/432912615 >> 172.21.15.5:0/432912615 conn(0x564710515c60 msgr2=0x564710547030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 -- 172.21.15.5:0/432912615 shutdown_connections 2024-04-25T10:04:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.583+0000 7f8595a20f40 1 -- 172.21.15.5:0/432912615 wait complete. 2024-04-25T10:04:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.584+0000 7f8595a20f40 1 Processor -- start 2024-04-25T10:04:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.584+0000 7f8595a20f40 1 -- start start 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f8595a20f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f8595a20f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105eff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f8595a20f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x564710546fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f8595a20f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564710548920 con 0x5647105cfa60 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f8595a20f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564710543c70 con 0x564710516070 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f8595a20f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564710548bc0 con 0x7f85780022d0 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f858a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f858a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105eff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59536/0 (socket says 172.21.15.5:59536) 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.585+0000 7f858a200640 1 -- 172.21.15.5:0/3010043591 learned_addr learned my addr 172.21.15.5:0/3010043591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f8593600640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x564710546fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f8592c00640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858a200640 1 -- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 msgr2=0x564710546fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858a200640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f85780022d0 0x564710546fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858a200640 1 -- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 msgr2=0x5647105e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858a200640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858a200640 1 -- 172.21.15.5:0/3010043591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647105ce620 con 0x5647105cfa60 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f8592c00640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564710516070 0x5647105e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858a200640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105eff60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f857c009e50 tx=0x7f857c009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858be00640 1 -- 172.21.15.5:0/3010043591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c015400 con 0x5647105cfa60 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858be00640 1 -- 172.21.15.5:0/3010043591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c0155a0 con 0x5647105cfa60 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f858be00640 1 -- 172.21.15.5:0/3010043591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c015970 con 0x5647105cfa60 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.586+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56471067cfa0 con 0x5647105cfa60 2024-04-25T10:04:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.587+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647105d0080 con 0x5647105cfa60 2024-04-25T10:04:06.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.589+0000 7f858be00640 1 -- 172.21.15.5:0/3010043591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f857c015b10 con 0x5647105cfa60 2024-04-25T10:04:06.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.589+0000 7f858be00640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8570069eb0 0x7f857006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.590+0000 7f858be00640 1 -- 172.21.15.5:0/3010043591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f857c088750 con 0x5647105cfa60 2024-04-25T10:04:06.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.590+0000 7f8595a20f40 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x564710547500 0x5647105d6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.590+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x5647105d65b0 con 0x564710547500 2024-04-25T10:04:06.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.590+0000 7f8593600640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x564710547500 0x5647105d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.590+0000 7f8592c00640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8570069eb0 0x7f857006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8593600640 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x564710547500 0x5647105d6070 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8593600640 1 -- 172.21.15.5:0/3010043591 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f858c01c040 con 0x564710547500 2024-04-25T10:04:06.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x564710547500 msgr2=0x5647105d6070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8595a20f40 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x564710547500 0x5647105d6070 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8570069eb0 msgr2=0x7f857006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8595a20f40 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8570069eb0 0x7f857006c370 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f8584054000 tx=0x7f8584050000 comp rx=0 tx=0).stop 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 msgr2=0x5647105eff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8595a20f40 1 --2- 172.21.15.5:0/3010043591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647105cfa60 0x5647105eff60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f857c009e50 tx=0x7f857c009f40 comp rx=0 tx=0).stop 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.591+0000 7f8593600640 1 -- 172.21.15.5:0/3010043591 reap_dead start 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.592+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 shutdown_connections 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.592+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 >> 172.21.15.5:0/3010043591 conn(0x564710515c60 msgr2=0x564710547a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.592+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 shutdown_connections 2024-04-25T10:04:06.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.592+0000 7f8595a20f40 1 -- 172.21.15.5:0/3010043591 wait complete. 2024-04-25T10:04:06.601 INFO:teuthology.orchestra.run.smithi005.stdout:mytag 2024-04-25T10:04:06.601 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:04:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.701+0000 7f0494b8cf40 1 Processor -- start 2024-04-25T10:04:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.701+0000 7f0494b8cf40 1 -- start start 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0494b8cf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04780022d0 0x56296ad59580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0494b8cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 0x56296ad73290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0494b8cf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad764d0 0x56296ad737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0494b8cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296ac44dd0 con 0x56296ad59ac0 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0494b8cf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296ac48d30 con 0x56296ad764d0 2024-04-25T10:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0494b8cf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56296ace70b0 con 0x7f04780022d0 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 0x56296ad73290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 0x56296ad73290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59554/0 (socket says 172.21.15.5:59554) 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 -- 172.21.15.5:0/1902016715 learned_addr learned my addr 172.21.15.5:0/1902016715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 -- 172.21.15.5:0/1902016715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04780022d0 msgr2=0x56296ad59580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04780022d0 0x56296ad59580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 -- 172.21.15.5:0/1902016715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad764d0 msgr2=0x56296ad737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad764d0 0x56296ad737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.702+0000 7f0491400640 1 -- 172.21.15.5:0/1902016715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296ad5a2b0 con 0x56296ad59ac0 2024-04-25T10:04:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.703+0000 7f0491400640 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 0x56296ad73290 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f048000d6f0 tx=0x7f048000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9911f5807013bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.703+0000 7f048be00640 1 -- 172.21.15.5:0/1902016715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480013070 con 0x56296ad59ac0 2024-04-25T10:04:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.703+0000 7f048be00640 1 -- 172.21.15.5:0/1902016715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048000b8d0 con 0x56296ad59ac0 2024-04-25T10:04:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.703+0000 7f048be00640 1 -- 172.21.15.5:0/1902016715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000bca0 con 0x56296ad59ac0 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 -- 172.21.15.5:0/1902016715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 msgr2=0x56296ad73290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 0x56296ad73290 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f048000d6f0 tx=0x7f048000dbb0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 -- 172.21.15.5:0/1902016715 shutdown_connections 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad764d0 0x56296ad737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296ad59ac0 0x56296ad73290 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/1902016715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04780022d0 0x56296ad59580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 -- 172.21.15.5:0/1902016715 >> 172.21.15.5:0/1902016715 conn(0x56296ac9cc60 msgr2=0x56296ad74d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 -- 172.21.15.5:0/1902016715 shutdown_connections 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.704+0000 7f0494b8cf40 1 -- 172.21.15.5:0/1902016715 wait complete. 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.705+0000 7f0494b8cf40 1 Processor -- start 2024-04-25T10:04:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.705+0000 7f0494b8cf40 1 -- start start 2024-04-25T10:04:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0494b8cf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad59ac0 0x56296ad77590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0494b8cf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56296ad764d0 0x56296ad74d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0494b8cf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04780022d0 0x56296ad4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0494b8cf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296ad59fd0 con 0x7f04780022d0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0494b8cf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296accc3b0 con 0x56296ad59ac0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0494b8cf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56296ad74480 con 0x56296ad764d0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0492800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04780022d0 0x56296ad4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0492800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04780022d0 0x56296ad4dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59562/0 (socket says 172.21.15.5:59562) 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0492800640 1 -- 172.21.15.5:0/385342704 learned_addr learned my addr 172.21.15.5:0/385342704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0491400640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56296ad764d0 0x56296ad74d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.706+0000 7f0491e00640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad59ac0 0x56296ad77590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0492800640 1 -- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56296ad764d0 msgr2=0x56296ad74d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0492800640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56296ad764d0 0x56296ad74d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0492800640 1 -- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad59ac0 msgr2=0x56296ad77590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0492800640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56296ad59ac0 0x56296ad77590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0492800640 1 -- 172.21.15.5:0/385342704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296ad725d0 con 0x7f04780022d0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0492800640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04780022d0 0x56296ad4dad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f048c002ff0 tx=0x7f048c009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f048aa00640 1 -- 172.21.15.5:0/385342704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c019070 con 0x7f04780022d0 2024-04-25T10:04:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56296ad5b1b0 con 0x7f04780022d0 2024-04-25T10:04:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f048aa00640 1 -- 172.21.15.5:0/385342704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c008da0 con 0x7f04780022d0 2024-04-25T10:04:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f048aa00640 1 -- 172.21.15.5:0/385342704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c014630 con 0x7f04780022d0 2024-04-25T10:04:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.707+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56296ad5b770 con 0x7f04780022d0 2024-04-25T10:04:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.710+0000 7f048aa00640 1 -- 172.21.15.5:0/385342704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f048c0147d0 con 0x7f04780022d0 2024-04-25T10:04:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.710+0000 7f048aa00640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0474069eb0 0x7f047406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.711+0000 7f048aa00640 1 -- 172.21.15.5:0/385342704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f048c088210 con 0x7f04780022d0 2024-04-25T10:04:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.711+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56296ad5bc90 0x56296ad5c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.711+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x56296ad5c5b0 con 0x56296ad5bc90 2024-04-25T10:04:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.711+0000 7f0491e00640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0474069eb0 0x7f047406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.711+0000 7f0491400640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56296ad5bc90 0x56296ad5c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.712+0000 7f0491400640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56296ad5bc90 0x56296ad5c070 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.712+0000 7f0491e00640 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0474069eb0 0x7f047406c370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0484055000 tx=0x7f0484051000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.712+0000 7f0491400640 1 -- 172.21.15.5:0/385342704 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0480015030 con 0x56296ad5bc90 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56296ad5bc90 msgr2=0x56296ad5c070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56296ad5bc90 0x56296ad5c070 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0474069eb0 msgr2=0x7f047406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0474069eb0 0x7f047406c370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0484055000 tx=0x7f0484051000 comp rx=0 tx=0).stop 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04780022d0 msgr2=0x56296ad4dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 --2- 172.21.15.5:0/385342704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04780022d0 0x56296ad4dad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f048c002ff0 tx=0x7f048c009de0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0492800640 1 -- 172.21.15.5:0/385342704 reap_dead start 2024-04-25T10:04:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 shutdown_connections 2024-04-25T10:04:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 >> 172.21.15.5:0/385342704 conn(0x56296ac9cc60 msgr2=0x56296ad57ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 shutdown_connections 2024-04-25T10:04:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:06.713+0000 7f0494b8cf40 1 -- 172.21.15.5:0/385342704 wait complete. 2024-04-25T10:04:06.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:06.723 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:06.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:06.810 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:06.919 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:06.920 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:04:06.920 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:06.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:06.981 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:04:06.981 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:07.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:07.041 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:04:07.041 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:07.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:07.104 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:04:07.104 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:07.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:07.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.379+0000 7ff645400640 1 Processor -- start 2024-04-25T10:04:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.379+0000 7ff645400640 1 -- start start 2024-04-25T10:04:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.380+0000 7ff645400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff64006e970 0x7ff6400cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.380+0000 7ff645400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 0x7ff64015b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.380+0000 7ff645400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6400cdbf0 con 0x7ff640144100 2024-04-25T10:04:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.380+0000 7ff645400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6401444e0 con 0x7ff64006e970 2024-04-25T10:04:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.380+0000 7ff645400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64015ac70 con 0x7ff6400ced20 2024-04-25T10:04:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.381+0000 7ff63f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 0x7ff64015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.381+0000 7ff63de00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff64006e970 0x7ff6400cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.381+0000 7ff63de00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff64006e970 0x7ff6400cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:36796/0 (socket says 172.21.15.5:36796) 2024-04-25T10:04:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.381+0000 7ff63f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 0x7ff64015b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:50082/0 (socket says 172.21.15.5:50082) 2024-04-25T10:04:07.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.381+0000 7ff63de00640 1 -- 172.21.15.5:0/2962290933 learned_addr learned my addr 172.21.15.5:0/2962290933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.382+0000 7ff63de00640 1 -- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 msgr2=0x7ff64015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.382+0000 7ff63de00640 1 --2- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 0x7ff64015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.382+0000 7ff63de00640 1 -- 172.21.15.5:0/2962290933 >> v1:172.21.15.80:6789/0 conn(0x7ff640144100 legacy=0x7ff64006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.382+0000 7ff63de00640 1 -- 172.21.15.5:0/2962290933 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400cded0 con 0x7ff64006e970 2024-04-25T10:04:07.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.382+0000 7ff63f200640 1 --2- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 0x7ff64015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.383+0000 7ff63de00640 1 --2- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cd610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009870 tx=0x7ff62c02c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b376365d5e7480 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.384+0000 7ff63d400640 1 -- 172.21.15.5:0/2962290933 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c038070 con 0x7ff64006e970 2024-04-25T10:04:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.384+0000 7ff63d400640 1 -- 172.21.15.5:0/2962290933 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c031710 con 0x7ff64006e970 2024-04-25T10:04:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.384+0000 7ff63d400640 1 -- 172.21.15.5:0/2962290933 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c031ae0 con 0x7ff64006e970 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 -- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 msgr2=0x7ff6400cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 --2- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cd610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009870 tx=0x7ff62c02c460 comp rx=0 tx=0).stop 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 -- 172.21.15.5:0/2962290933 shutdown_connections 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 --2- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6400ced20 0x7ff64015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 --2- 172.21.15.5:0/2962290933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cd610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 -- 172.21.15.5:0/2962290933 >> 172.21.15.5:0/2962290933 conn(0x7ff640071150 msgr2=0x7ff640071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 -- 172.21.15.5:0/2962290933 shutdown_connections 2024-04-25T10:04:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.385+0000 7ff645400640 1 -- 172.21.15.5:0/2962290933 wait complete. 2024-04-25T10:04:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.387+0000 7ff645400640 1 Processor -- start 2024-04-25T10:04:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.387+0000 7ff645400640 1 -- start start 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff645400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff645400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 0x7ff64013afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff645400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff640144100 0x7ff64014bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff645400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400cea00 con 0x7ff6400ced20 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff645400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6400ce490 con 0x7ff64006e970 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff645400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64015b670 con 0x7ff640144100 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff63de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 0x7ff64013afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff63de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 0x7ff64013afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48064/0 (socket says 172.21.15.5:48064) 2024-04-25T10:04:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.388+0000 7ff63de00640 1 -- 172.21.15.5:0/3908523346 learned_addr learned my addr 172.21.15.5:0/3908523346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63e800640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63f200640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff640144100 0x7ff64014bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63de00640 1 -- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff640144100 msgr2=0x7ff64014bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63de00640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff640144100 0x7ff64014bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63de00640 1 -- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 msgr2=0x7ff6400cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63de00640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63de00640 1 -- 172.21.15.5:0/3908523346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400ca040 con 0x7ff6400ced20 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.389+0000 7ff63de00640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 0x7ff64013afe0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff62c000c00 tx=0x7ff62c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.390+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c042020 con 0x7ff6400ced20 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.390+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c03d870 con 0x7ff6400ced20 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.390+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c038040 con 0x7ff6400ced20 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.390+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401f3300 con 0x7ff6400ced20 2024-04-25T10:04:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.390+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401f38c0 con 0x7ff6400ced20 2024-04-25T10:04:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.391+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff640068fd0 con 0x7ff6400ced20 2024-04-25T10:04:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.392+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff62c045020 con 0x7ff6400ced20 2024-04-25T10:04:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.393+0000 7ff636a00640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff62006b4d0 0x7ff62006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.393+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff62c0acd60 con 0x7ff6400ced20 2024-04-25T10:04:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.394+0000 7ff63e800640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff62006b4d0 0x7ff62006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.395+0000 7ff63e800640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff62006b4d0 0x7ff62006d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f60 tx=0x7ff630009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.398+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff62c04c030 con 0x7ff6400ced20 2024-04-25T10:04:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.635+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6400b3af0 con 0x7ff6400ced20 2024-04-25T10:04:07.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.636+0000 7ff636a00640 1 -- 172.21.15.5:0/3908523346 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7ff62c07ba40 con 0x7ff6400ced20 2024-04-25T10:04:07.641 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:07.641 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:07.641 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:04:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff62006b4d0 msgr2=0x7ff62006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff62006b4d0 0x7ff62006d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f60 tx=0x7ff630009040 comp rx=0 tx=0).stop 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 msgr2=0x7ff64013afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 0x7ff64013afe0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff62c000c00 tx=0x7ff62c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 shutdown_connections 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff640144100 0x7ff64014bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6400ced20 0x7ff64013afe0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff62006b4d0 0x7ff62006d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 --2- 172.21.15.5:0/3908523346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64006e970 0x7ff6400cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.638+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 >> 172.21.15.5:0/3908523346 conn(0x7ff640071150 msgr2=0x7ff640140c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.639+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 shutdown_connections 2024-04-25T10:04:07.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.639+0000 7ff645400640 1 -- 172.21.15.5:0/3908523346 wait complete. 2024-04-25T10:04:07.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.867+0000 7f79fa200640 1 Processor -- start 2024-04-25T10:04:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.867+0000 7f79fa200640 1 -- start start 2024-04-25T10:04:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.869+0000 7f79fa200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79f4143d30 0x7f79f415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.869+0000 7f79fa200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f40ceea0 0x7f79f40b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.869+0000 7f79fa200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f40b4880 con 0x7f79f40b3ae0 2024-04-25T10:04:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.869+0000 7f79fa200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40b45a0 con 0x7f79f4143d30 2024-04-25T10:04:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.869+0000 7f79fa200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415e2c0 con 0x7f79f40ceea0 2024-04-25T10:04:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.870+0000 7f79f8e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79f4143d30 0x7f79f415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.870+0000 7f79f8e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79f4143d30 0x7f79f415eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:36818/0 (socket says 172.21.15.5:36818) 2024-04-25T10:04:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.870+0000 7f79f8e00640 1 -- 172.21.15.5:0/2366352800 learned_addr learned my addr 172.21.15.5:0/2366352800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.870+0000 7f79f9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f40ceea0 0x7f79f40b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.871+0000 7f79f8e00640 1 -- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f40ceea0 msgr2=0x7f79f40b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.871+0000 7f79f8e00640 1 --2- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f40ceea0 0x7f79f40b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.871+0000 7f79f8e00640 1 -- 172.21.15.5:0/2366352800 >> v1:172.21.15.80:6789/0 conn(0x7f79f40b3ae0 legacy=0x7f79f40ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.871+0000 7f79f8e00640 1 -- 172.21.15.5:0/2366352800 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f415e5a0 con 0x7f79f4143d30 2024-04-25T10:04:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.872+0000 7f79f8e00640 1 --2- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f4143d30 0x7f79f415eda0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009870 tx=0x7f79e802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d40397ab2133a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.872+0000 7f79f3400640 1 -- 172.21.15.5:0/2366352800 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8037070 con 0x7f79f4143d30 2024-04-25T10:04:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.872+0000 7f79f3400640 1 -- 172.21.15.5:0/2366352800 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8032440 con 0x7f79f4143d30 2024-04-25T10:04:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.872+0000 7f79f3400640 1 -- 172.21.15.5:0/2366352800 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8032810 con 0x7f79f4143d30 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 -- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f4143d30 msgr2=0x7f79f415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 --2- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f4143d30 0x7f79f415eda0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009870 tx=0x7f79e802a250 comp rx=0 tx=0).stop 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 -- 172.21.15.5:0/2366352800 shutdown_connections 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 --2- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f40ceea0 0x7f79f40b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 --2- 172.21.15.5:0/2366352800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f4143d30 0x7f79f415eda0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 -- 172.21.15.5:0/2366352800 >> 172.21.15.5:0/2366352800 conn(0x7f79f4066f20 msgr2=0x7f79f406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 -- 172.21.15.5:0/2366352800 shutdown_connections 2024-04-25T10:04:07.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.873+0000 7f79fa200640 1 -- 172.21.15.5:0/2366352800 wait complete. 2024-04-25T10:04:07.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.875+0000 7f79fa200640 1 Processor -- start 2024-04-25T10:04:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.875+0000 7f79fa200640 1 -- start start 2024-04-25T10:04:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79fa200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 0x7f79f41e4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79fa200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f40ceea0 0x7f79f41e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79fa200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f4143d30 0x7f79f415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79fa200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4144290 con 0x7f79f40b3ae0 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79fa200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f4144110 con 0x7f79f40ceea0 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79fa200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f406e430 con 0x7f79f4143d30 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79f8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 0x7f79f41e4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79f8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 0x7f79f41e4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48076/0 (socket says 172.21.15.5:48076) 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.876+0000 7f79f8e00640 1 -- 172.21.15.5:0/3445214591 learned_addr learned my addr 172.21.15.5:0/3445214591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f3e00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f40ceea0 0x7f79f41e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f8e00640 1 -- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f4143d30 msgr2=0x7f79f415c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f8e00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f4143d30 0x7f79f415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f8e00640 1 -- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f40ceea0 msgr2=0x7f79f41e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f8e00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f40ceea0 0x7f79f41e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f8e00640 1 -- 172.21.15.5:0/3445214591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f41e5100 con 0x7f79f40b3ae0 2024-04-25T10:04:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.877+0000 7f79f8e00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 0x7f79f41e4a70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000c00 tx=0x7f79e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.878+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8037040 con 0x7f79f40b3ae0 2024-04-25T10:04:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.878+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8042070 con 0x7f79f40b3ae0 2024-04-25T10:04:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.878+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f415b690 con 0x7f79f40b3ae0 2024-04-25T10:04:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.878+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e803b870 con 0x7f79f40b3ae0 2024-04-25T10:04:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.878+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f415bc50 con 0x7f79f40b3ae0 2024-04-25T10:04:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.879+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f4068f60 con 0x7f79f40b3ae0 2024-04-25T10:04:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.880+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79e803ba10 con 0x7f79f40b3ae0 2024-04-25T10:04:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.881+0000 7f79f0c00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79d806b500 0x7f79d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.881+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79e80aab90 con 0x7f79f40b3ae0 2024-04-25T10:04:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.882+0000 7f79f3e00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79d806b500 0x7f79d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.883+0000 7f79f3e00640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79d806b500 0x7f79d806d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f79f413d650 tx=0x7f79e4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:07.885+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79e8079870 con 0x7f79f40b3ae0 2024-04-25T10:04:08.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.088+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f415d700 con 0x7f79f40b3ae0 2024-04-25T10:04:08.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.091+0000 7f79f0c00640 1 -- 172.21.15.5:0/3445214591 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f79e8079170 con 0x7f79f40b3ae0 2024-04-25T10:04:08.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:08.096 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:38.104579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:08.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.094+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79d806b500 msgr2=0x7f79d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.094+0000 7f79fa200640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79d806b500 0x7f79d806d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f79f413d650 tx=0x7f79e4009040 comp rx=0 tx=0).stop 2024-04-25T10:04:08.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.094+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 msgr2=0x7f79f41e4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.094+0000 7f79fa200640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 0x7f79f41e4a70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000c00 tx=0x7f79e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.094+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 shutdown_connections 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.094+0000 7f79fa200640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79f4143d30 0x7f79f415c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.095+0000 7f79fa200640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f79d806b500 0x7f79d806d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.095+0000 7f79fa200640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79f40ceea0 0x7f79f41e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.095+0000 7f79fa200640 1 --2- 172.21.15.5:0/3445214591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79f40b3ae0 0x7f79f41e4a70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.095+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 >> 172.21.15.5:0/3445214591 conn(0x7f79f4066f20 msgr2=0x7f79f40c4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.095+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 shutdown_connections 2024-04-25T10:04:08.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.095+0000 7f79fa200640 1 -- 172.21.15.5:0/3445214591 wait complete. 2024-04-25T10:04:08.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:34.491131+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:35.568287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:08.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:08.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.325+0000 7fc94e000640 1 Processor -- start 2024-04-25T10:04:08.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.326+0000 7fc94e000640 1 -- start start 2024-04-25T10:04:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.327+0000 7fc94e000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc948071000 0x7fc948071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.327+0000 7fc94e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9480b3840 0x7fc9480b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.327+0000 7fc94e000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94806d020 con 0x7fc94815f800 2024-04-25T10:04:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.327+0000 7fc94e000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480719a0 con 0x7fc948071000 2024-04-25T10:04:08.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.327+0000 7fc94e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94806ded0 con 0x7fc9480b3840 2024-04-25T10:04:08.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc94cc00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc94815f800 0x7fc948162ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44586/0 (socket says 172.21.15.5:44586) 2024-04-25T10:04:08.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc94cc00640 1 -- 172.21.15.5:0/200282080 learned_addr learned my addr 172.21.15.5:0/200282080 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:08.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947e00640 1 --2- 172.21.15.5:0/200282080 >> v2:172.21.15.5:3300/0 conn(0x7fc948071000 0x7fc948071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc94d600640 1 --2- 172.21.15.5:0/200282080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9480b3840 0x7fc9480b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947400640 1 -- 172.21.15.5:0/200282080 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193804107 0 0) 0x7fc94806d020 con 0x7fc94815f800 2024-04-25T10:04:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947400640 1 -- 172.21.15.5:0/200282080 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc930003520 con 0x7fc94815f800 2024-04-25T10:04:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947e00640 1 -- 172.21.15.5:0/200282080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9480b3840 msgr2=0x7fc9480b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947e00640 1 --2- 172.21.15.5:0/200282080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9480b3840 0x7fc9480b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947e00640 1 -- 172.21.15.5:0/200282080 >> v1:172.21.15.5:6789/0 conn(0x7fc94815f800 legacy=0x7fc948162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.328+0000 7fc947e00640 1 -- 172.21.15.5:0/200282080 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94806d2c0 con 0x7fc948071000 2024-04-25T10:04:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.330+0000 7fc947e00640 1 --2- 172.21.15.5:0/200282080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc948071460 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc94806e050 tx=0x7fc93802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3af8d254605818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.330+0000 7fc947400640 1 -- 172.21.15.5:0/200282080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938037070 con 0x7fc948071000 2024-04-25T10:04:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.330+0000 7fc947400640 1 -- 172.21.15.5:0/200282080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938032440 con 0x7fc948071000 2024-04-25T10:04:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.330+0000 7fc947400640 1 -- 172.21.15.5:0/200282080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938032810 con 0x7fc948071000 2024-04-25T10:04:08.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 -- 172.21.15.5:0/200282080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 msgr2=0x7fc948071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 --2- 172.21.15.5:0/200282080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc948071460 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc94806e050 tx=0x7fc93802a180 comp rx=0 tx=0).stop 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 -- 172.21.15.5:0/200282080 shutdown_connections 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 --2- 172.21.15.5:0/200282080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9480b3840 0x7fc9480b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 --2- 172.21.15.5:0/200282080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc948071460 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 -- 172.21.15.5:0/200282080 >> 172.21.15.5:0/200282080 conn(0x7fc948066f20 msgr2=0x7fc94806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 -- 172.21.15.5:0/200282080 shutdown_connections 2024-04-25T10:04:08.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.331+0000 7fc94e000640 1 -- 172.21.15.5:0/200282080 wait complete. 2024-04-25T10:04:08.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.333+0000 7fc94e000640 1 Processor -- start 2024-04-25T10:04:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.333+0000 7fc94e000640 1 -- start start 2024-04-25T10:04:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.334+0000 7fc94e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc9480afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.334+0000 7fc94e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9480b3840 0x7fc9480aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.334+0000 7fc94e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc94815f800 0x7fc9481faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.334+0000 7fc94e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94806db60 con 0x7fc948071000 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.334+0000 7fc94e000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94806d880 con 0x7fc9480b3840 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.334+0000 7fc94e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94806e960 con 0x7fc94815f800 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc9480afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc9480afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48098/0 (socket says 172.21.15.5:48098) 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 -- 172.21.15.5:0/2168565306 learned_addr learned my addr 172.21.15.5:0/2168565306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc947e00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9480b3840 0x7fc9480aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94d600640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc94815f800 0x7fc9481faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 -- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc94815f800 msgr2=0x7fc9481faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc94815f800 0x7fc9481faf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 -- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9480b3840 msgr2=0x7fc9480aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9480b3840 0x7fc9480aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.335+0000 7fc94cc00640 1 -- 172.21.15.5:0/2168565306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94815cf20 con 0x7fc948071000 2024-04-25T10:04:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.336+0000 7fc94cc00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc9480afa50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0010e0 tx=0x7fc93c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.336+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c017070 con 0x7fc948071000 2024-04-25T10:04:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.336+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c005d00 con 0x7fc948071000 2024-04-25T10:04:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.336+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0072b0 con 0x7fc948071000 2024-04-25T10:04:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.336+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94815c160 con 0x7fc948071000 2024-04-25T10:04:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.336+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94815c720 con 0x7fc948071000 2024-04-25T10:04:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.338+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc948068f60 con 0x7fc948071000 2024-04-25T10:04:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.339+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc93c01a070 con 0x7fc948071000 2024-04-25T10:04:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.340+0000 7fc944c00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc92c06b4d0 0x7fc92c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.340+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc93c0897a0 con 0x7fc948071000 2024-04-25T10:04:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.341+0000 7fc947e00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc92c06b4d0 0x7fc92c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.342+0000 7fc947e00640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc92c06b4d0 0x7fc92c06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc938034000 tx=0x7fc93802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.346+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc93c058520 con 0x7fc948071000 2024-04-25T10:04:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.544+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc94815ebd0 con 0x7fc948071000 2024-04-25T10:04:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.545+0000 7fc944c00640 1 -- 172.21.15.5:0/2168565306 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc93c057e20 con 0x7fc948071000 2024-04-25T10:04:08.550 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:04:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.547+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc92c06b4d0 msgr2=0x7fc92c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.547+0000 7fc94e000640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc92c06b4d0 0x7fc92c06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc938034000 tx=0x7fc93802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.547+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 msgr2=0x7fc9480afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.547+0000 7fc94e000640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc9480afa50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0010e0 tx=0x7fc93c00c960 comp rx=0 tx=0).stop 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 shutdown_connections 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc94815f800 0x7fc9481faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc92c06b4d0 0x7fc92c06d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9480b3840 0x7fc9480aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 --2- 172.21.15.5:0/2168565306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc948071000 0x7fc9480afa50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 >> 172.21.15.5:0/2168565306 conn(0x7fc948066f20 msgr2=0x7fc94815fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 shutdown_connections 2024-04-25T10:04:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.548+0000 7fc94e000640 1 -- 172.21.15.5:0/2168565306 wait complete. 2024-04-25T10:04:08.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:04:08.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.787+0000 7f1f83200640 1 Processor -- start 2024-04-25T10:04:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.787+0000 7f1f83200640 1 -- start start 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f83200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1f7c06b270 0x7f1f7c0b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f83200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f7c0d0f40 0x7f1f7c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f83200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0b5780 con 0x7f1f7c15d990 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f83200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c15dd70 con 0x7f1f7c0d0f40 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f83200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c06b9b0 con 0x7f1f7c06b270 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f81e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1f7c15d990 0x7f1f7c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2024-04-25T10:04:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f81e00640 1 -- 172.21.15.5:0/1253981089 learned_addr learned my addr 172.21.15.5:0/1253981089 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f82800640 1 --2- 172.21.15.5:0/1253981089 >> v2:172.21.15.5:3300/0 conn(0x7f1f7c0d0f40 0x7f1f7c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.789+0000 7f1f81400640 1 --2- 172.21.15.5:0/1253981089 >> v2:172.21.15.80:3300/0 conn(0x7f1f7c06b270 0x7f1f7c0b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.790+0000 7f1f80a00640 1 -- 172.21.15.5:0/1253981089 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065868499 0 0) 0x7f1f7c0b5780 con 0x7f1f7c15d990 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.790+0000 7f1f80a00640 1 -- 172.21.15.5:0/1253981089 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f64003520 con 0x7f1f7c15d990 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.790+0000 7f1f82800640 1 -- 172.21.15.5:0/1253981089 >> v2:172.21.15.80:3300/0 conn(0x7f1f7c06b270 msgr2=0x7f1f7c0b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.790+0000 7f1f82800640 1 --2- 172.21.15.5:0/1253981089 >> v2:172.21.15.80:3300/0 conn(0x7f1f7c06b270 0x7f1f7c0b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.790+0000 7f1f82800640 1 -- 172.21.15.5:0/1253981089 >> v1:172.21.15.5:6789/0 conn(0x7f1f7c15d990 legacy=0x7f1f7c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.790+0000 7f1f82800640 1 -- 172.21.15.5:0/1253981089 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0d0180 con 0x7f1f7c0d0f40 2024-04-25T10:04:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.791+0000 7f1f82800640 1 --2- 172.21.15.5:0/1253981089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c0d0f40 0x7f1f7c0b50c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c06d860 tx=0x7f1f7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d718b6a2529d5f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.791+0000 7f1f80a00640 1 -- 172.21.15.5:0/1253981089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007740 con 0x7f1f7c0d0f40 2024-04-25T10:04:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.791+0000 7f1f80a00640 1 -- 172.21.15.5:0/1253981089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f780078e0 con 0x7f1f7c0d0f40 2024-04-25T10:04:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.791+0000 7f1f80a00640 1 -- 172.21.15.5:0/1253981089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007cb0 con 0x7f1f7c0d0f40 2024-04-25T10:04:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 -- 172.21.15.5:0/1253981089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c0d0f40 msgr2=0x7f1f7c0b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 --2- 172.21.15.5:0/1253981089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c0d0f40 0x7f1f7c0b50c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c06d860 tx=0x7f1f7802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 -- 172.21.15.5:0/1253981089 shutdown_connections 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 --2- 172.21.15.5:0/1253981089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c0d0f40 0x7f1f7c0b50c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 --2- 172.21.15.5:0/1253981089 >> v2:172.21.15.80:3300/0 conn(0x7f1f7c06b270 0x7f1f7c0b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 -- 172.21.15.5:0/1253981089 >> 172.21.15.5:0/1253981089 conn(0x7f1f7c0b5ab0 msgr2=0x7f1f7c0999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 -- 172.21.15.5:0/1253981089 shutdown_connections 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.792+0000 7f1f83200640 1 -- 172.21.15.5:0/1253981089 wait complete. 2024-04-25T10:04:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.794+0000 7f1f83200640 1 Processor -- start 2024-04-25T10:04:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.794+0000 7f1f83200640 1 -- start start 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f83200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 0x7f1f7c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f83200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f7c0d0f40 0x7f1f7c149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f83200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f7c15d990 0x7f1f7c1f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f83200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0d0a20 con 0x7f1f7c06b270 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f83200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0d0740 con 0x7f1f7c15d990 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f83200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0af980 con 0x7f1f7c0d0f40 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f81e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 0x7f1f7c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f81e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 0x7f1f7c1570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48116/0 (socket says 172.21.15.5:48116) 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f81e00640 1 -- 172.21.15.5:0/4251750780 learned_addr learned my addr 172.21.15.5:0/4251750780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.795+0000 7f1f81400640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f7c0d0f40 0x7f1f7c149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f82800640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f7c15d990 0x7f1f7c1f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f81e00640 1 -- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f7c0d0f40 msgr2=0x7f1f7c149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f81e00640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f7c0d0f40 0x7f1f7c149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f81e00640 1 -- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f7c15d990 msgr2=0x7f1f7c1f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f81e00640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f7c15d990 0x7f1f7c1f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f81e00640 1 -- 172.21.15.5:0/4251750780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c155310 con 0x7f1f7c06b270 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f81e00640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 0x7f1f7c1570c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c400 tx=0x7f1f7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70017070 con 0x7f1f7c06b270 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70005d00 con 0x7f1f7c06b270 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70007310 con 0x7f1f7c06b270 2024-04-25T10:04:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.796+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c1555f0 con 0x7f1f7c06b270 2024-04-25T10:04:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.797+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c155bb0 con 0x7f1f7c06b270 2024-04-25T10:04:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.798+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c068f60 con 0x7f1f7c06b270 2024-04-25T10:04:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.799+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f7001a070 con 0x7f1f7c06b270 2024-04-25T10:04:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.800+0000 7f1f76000640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f6006b4d0 0x7f1f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.800+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f700896d0 con 0x7f1f7c06b270 2024-04-25T10:04:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.800+0000 7f1f81400640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f6006b4d0 0x7f1f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.802+0000 7f1f81400640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f6006b4d0 0x7f1f6006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c1f69e0 tx=0x7f1f6c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:08.804+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f70058450 con 0x7f1f7c06b270 2024-04-25T10:04:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.001+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f7c155d80 con 0x7f1f7c06b270 2024-04-25T10:04:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.002+0000 7f1f76000640 1 -- 172.21.15.5:0/4251750780 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f70057d50 con 0x7f1f7c06b270 2024-04-25T10:04:09.007 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.004+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f6006b4d0 msgr2=0x7f1f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.004+0000 7f1f83200640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f6006b4d0 0x7f1f6006d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c1f69e0 tx=0x7f1f6c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.004+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 msgr2=0x7f1f7c1570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.004+0000 7f1f83200640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 0x7f1f7c1570c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c400 tx=0x7f1f7000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 shutdown_connections 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f6006b4d0 0x7f1f6006d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f7c15d990 0x7f1f7c1f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f7c0d0f40 0x7f1f7c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 --2- 172.21.15.5:0/4251750780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c06b270 0x7f1f7c1570c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 >> 172.21.15.5:0/4251750780 conn(0x7f1f7c0b5ab0 msgr2=0x7f1f7c0b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 shutdown_connections 2024-04-25T10:04:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.005+0000 7f1f83200640 1 -- 172.21.15.5:0/4251750780 wait complete. 2024-04-25T10:04:09.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:04:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.241+0000 7f96f7200640 1 Processor -- start 2024-04-25T10:04:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.242+0000 7f96f7200640 1 -- start start 2024-04-25T10:04:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.243+0000 7f96f7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f96f006e840 0x7f96f015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.243+0000 7f96f7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96f015b330 0x7f96f0071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.243+0000 7f96f7200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f00ce570 con 0x7f96f00cc840 2024-04-25T10:04:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.243+0000 7f96f7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00ce850 con 0x7f96f015b330 2024-04-25T10:04:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.243+0000 7f96f7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00ceb30 con 0x7f96f006e840 2024-04-25T10:04:09.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96f015b330 0x7f96f0071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96f015b330 0x7f96f0071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48118/0 (socket says 172.21.15.5:48118) 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 -- 172.21.15.5:0/494309839 learned_addr learned my addr 172.21.15.5:0/494309839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f5400640 1 --2- 172.21.15.5:0/494309839 >> v2:172.21.15.80:3300/0 conn(0x7f96f006e840 0x7f96f015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 -- 172.21.15.5:0/494309839 >> v2:172.21.15.80:3300/0 conn(0x7f96f006e840 msgr2=0x7f96f015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 --2- 172.21.15.5:0/494309839 >> v2:172.21.15.80:3300/0 conn(0x7f96f006e840 0x7f96f015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 -- 172.21.15.5:0/494309839 >> v1:172.21.15.80:6789/0 conn(0x7f96f00cc840 legacy=0x7f96f006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.244+0000 7f96f6800640 1 -- 172.21.15.5:0/494309839 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00cee10 con 0x7f96f015b330 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.246+0000 7f96f6800640 1 --2- 172.21.15.5:0/494309839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f015b330 0x7f96f0071000 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f96f006ba20 tx=0x7f96ec02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a97dcbec5fced7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.246+0000 7f96f4a00640 1 -- 172.21.15.5:0/494309839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec009600 con 0x7f96f015b330 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.246+0000 7f96f4a00640 1 -- 172.21.15.5:0/494309839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec0097a0 con 0x7f96f015b330 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.246+0000 7f96f4a00640 1 -- 172.21.15.5:0/494309839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec009b70 con 0x7f96f015b330 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 -- 172.21.15.5:0/494309839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f015b330 msgr2=0x7f96f0071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 --2- 172.21.15.5:0/494309839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f015b330 0x7f96f0071000 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f96f006ba20 tx=0x7f96ec02c900 comp rx=0 tx=0).stop 2024-04-25T10:04:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 -- 172.21.15.5:0/494309839 shutdown_connections 2024-04-25T10:04:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 --2- 172.21.15.5:0/494309839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f015b330 0x7f96f0071000 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 --2- 172.21.15.5:0/494309839 >> v2:172.21.15.80:3300/0 conn(0x7f96f006e840 0x7f96f015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 -- 172.21.15.5:0/494309839 >> 172.21.15.5:0/494309839 conn(0x7f96f0066f20 msgr2=0x7f96f006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 -- 172.21.15.5:0/494309839 shutdown_connections 2024-04-25T10:04:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.247+0000 7f96f7200640 1 -- 172.21.15.5:0/494309839 wait complete. 2024-04-25T10:04:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.249+0000 7f96f7200640 1 Processor -- start 2024-04-25T10:04:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.249+0000 7f96f7200640 1 -- start start 2024-04-25T10:04:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 0x7f96f006d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96f00cc840 0x7f96f006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96f015b330 0x7f96f01ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00ce350 con 0x7f96f006e840 2024-04-25T10:04:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00ce1d0 con 0x7f96f015b330 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00ccc20 con 0x7f96f00cc840 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 0x7f96f006d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 0x7f96f006d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48126/0 (socket says 172.21.15.5:48126) 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.250+0000 7f96f5e00640 1 -- 172.21.15.5:0/2971566317 learned_addr learned my addr 172.21.15.5:0/2971566317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f6800640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96f015b330 0x7f96f01ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5400640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96f00cc840 0x7f96f006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5e00640 1 -- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96f00cc840 msgr2=0x7f96f006dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5e00640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96f00cc840 0x7f96f006dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5e00640 1 -- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96f015b330 msgr2=0x7f96f01ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5e00640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96f015b330 0x7f96f01ef2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5e00640 1 -- 172.21.15.5:0/2971566317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f01f2810 con 0x7f96f006e840 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f5e00640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 0x7f96f006d7e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c430 tx=0x7f96e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4011790 con 0x7f96f006e840 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4017070 con 0x7f96f006e840 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4011bb0 con 0x7f96f006e840 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.251+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f01f2af0 con 0x7f96f006e840 2024-04-25T10:04:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.252+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f01f30b0 con 0x7f96f006e840 2024-04-25T10:04:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.253+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f0068f60 con 0x7f96f006e840 2024-04-25T10:04:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.254+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96e401c460 con 0x7f96f006e840 2024-04-25T10:04:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.255+0000 7f96ea000640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f96d406b550 0x7f96d406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.255+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96e408a1d0 con 0x7f96f006e840 2024-04-25T10:04:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.255+0000 7f96f5400640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f96d406b550 0x7f96d406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.256+0000 7f96f5400640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f96d406b550 0x7f96d406da10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f96f0153400 tx=0x7f96e0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.259+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96e4058f30 con 0x7f96f006e840 2024-04-25T10:04:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.463+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96f006d430 con 0x7f96f006e840 2024-04-25T10:04:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.464+0000 7f96ea000640 1 -- 172.21.15.5:0/2971566317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96e4058830 con 0x7f96f006e840 2024-04-25T10:04:09.469 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:04:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f96d406b550 msgr2=0x7f96d406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f96d406b550 0x7f96d406da10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f96f0153400 tx=0x7f96e0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 msgr2=0x7f96f006d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 0x7f96f006d7e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c430 tx=0x7f96e400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 shutdown_connections 2024-04-25T10:04:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f96d406b550 0x7f96d406da10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96f015b330 0x7f96f01ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96f00cc840 0x7f96f006dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 --2- 172.21.15.5:0/2971566317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96f006e840 0x7f96f006d7e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.467+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 >> 172.21.15.5:0/2971566317 conn(0x7f96f0066f20 msgr2=0x7f96f015cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.468+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 shutdown_connections 2024-04-25T10:04:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.468+0000 7f96f7200640 1 -- 172.21.15.5:0/2971566317 wait complete. 2024-04-25T10:04:09.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.699+0000 7f024c800640 1 Processor -- start 2024-04-25T10:04:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.699+0000 7f024c800640 1 -- start start 2024-04-25T10:04:09.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.700+0000 7f024c800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0244071ed0 0x7f02440ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f024c800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440ce890 0x7f024406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f024c800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024415bdc0 con 0x7f02441529e0 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f024c800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02440b2340 con 0x7f02440ce890 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f024c800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02440cecc0 con 0x7f0244071ed0 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f0249e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f02441529e0 0x7f02440c5ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44596/0 (socket says 172.21.15.5:44596) 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f0249e00640 1 -- 172.21.15.5:0/3030147595 learned_addr learned my addr 172.21.15.5:0/3030147595 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f024a800640 1 --2- 172.21.15.5:0/3030147595 >> v2:172.21.15.5:3300/0 conn(0x7f02440ce890 0x7f024406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.701+0000 7f0249400640 1 --2- 172.21.15.5:0/3030147595 >> v2:172.21.15.80:3300/0 conn(0x7f0244071ed0 0x7f02440ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.702+0000 7f024a800640 1 -- 172.21.15.5:0/3030147595 >> v2:172.21.15.80:3300/0 conn(0x7f0244071ed0 msgr2=0x7f02440ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.702+0000 7f024a800640 1 --2- 172.21.15.5:0/3030147595 >> v2:172.21.15.80:3300/0 conn(0x7f0244071ed0 0x7f02440ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.702+0000 7f024a800640 1 -- 172.21.15.5:0/3030147595 >> v1:172.21.15.5:6789/0 conn(0x7f02441529e0 legacy=0x7f02440c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.702+0000 7f024a800640 1 -- 172.21.15.5:0/3030147595 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024415c060 con 0x7f02440ce890 2024-04-25T10:04:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.702+0000 7f0248a00640 1 -- 172.21.15.5:0/3030147595 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135929305 0 0) 0x7f024415bdc0 con 0x7f02441529e0 2024-04-25T10:04:09.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.703+0000 7f024a800640 1 --2- 172.21.15.5:0/3030147595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f024406d150 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f02440b5810 tx=0x7f024002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408b9ed91aa63e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.703+0000 7f0248a00640 1 -- 172.21.15.5:0/3030147595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240007740 con 0x7f02440ce890 2024-04-25T10:04:09.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.703+0000 7f0248a00640 1 -- 172.21.15.5:0/3030147595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400078e0 con 0x7f02440ce890 2024-04-25T10:04:09.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.703+0000 7f0248a00640 1 -- 172.21.15.5:0/3030147595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240007cb0 con 0x7f02440ce890 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 -- 172.21.15.5:0/3030147595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 msgr2=0x7f024406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 --2- 172.21.15.5:0/3030147595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f024406d150 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f02440b5810 tx=0x7f024002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 -- 172.21.15.5:0/3030147595 shutdown_connections 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 --2- 172.21.15.5:0/3030147595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f024406d150 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 --2- 172.21.15.5:0/3030147595 >> v2:172.21.15.80:3300/0 conn(0x7f0244071ed0 0x7f02440ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 -- 172.21.15.5:0/3030147595 >> 172.21.15.5:0/3030147595 conn(0x7f0244071150 msgr2=0x7f0244071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 -- 172.21.15.5:0/3030147595 shutdown_connections 2024-04-25T10:04:09.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.704+0000 7f024c800640 1 -- 172.21.15.5:0/3030147595 wait complete. 2024-04-25T10:04:09.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.706+0000 7f024c800640 1 Processor -- start 2024-04-25T10:04:09.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.706+0000 7f024c800640 1 -- start start 2024-04-25T10:04:09.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f024c800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244071ed0 0x7f0244143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f024c800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f0244157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f024c800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244157d50 0x7f02441f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f024c800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024415c900 con 0x7f02440ce890 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f024c800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024415c620 con 0x7f0244071ed0 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f024c800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02440c65e0 con 0x7f0244157d50 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f0249400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f0244157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f0249400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f0244157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48138/0 (socket says 172.21.15.5:48138) 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.707+0000 7f0249400640 1 -- 172.21.15.5:0/1742911074 learned_addr learned my addr 172.21.15.5:0/1742911074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249e00640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244071ed0 0x7f0244143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f024a800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244157d50 0x7f02441f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249400640 1 -- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244157d50 msgr2=0x7f02441f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249400640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244157d50 0x7f02441f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249400640 1 -- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244071ed0 msgr2=0x7f0244143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249400640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244071ed0 0x7f0244143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249400640 1 -- 172.21.15.5:0/1742911074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02441e20a0 con 0x7f02440ce890 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.708+0000 7f0249400640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f0244157810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f023400d5c0 tx=0x7f023400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.709+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02340125f0 con 0x7f02440ce890 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.709+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0234019070 con 0x7f02440ce890 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.709+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02340129d0 con 0x7f02440ce890 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.709+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02440b28e0 con 0x7f02440ce890 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.709+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02440b2ea0 con 0x7f02440ce890 2024-04-25T10:04:09.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.710+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0244158160 con 0x7f02440ce890 2024-04-25T10:04:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.711+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0234012b70 con 0x7f02440ce890 2024-04-25T10:04:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.712+0000 7f023e000640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f022806b4d0 0x7f022806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.712+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f023408a740 con 0x7f02440ce890 2024-04-25T10:04:09.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.713+0000 7f0249e00640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f022806b4d0 0x7f022806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.714+0000 7f0249e00640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f022806b4d0 0x7f022806d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0238000f60 tx=0x7f0238009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.716+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02441e20a0 con 0x7f02440ce890 2024-04-25T10:04:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.953+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0244068fd0 con 0x7f02440ce890 2024-04-25T10:04:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.954+0000 7f023e000640 1 -- 172.21.15.5:0/1742911074 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f0234059420 con 0x7f02440ce890 2024-04-25T10:04:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:09.959 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:09.959 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:04:09.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f022806b4d0 msgr2=0x7f022806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f022806b4d0 0x7f022806d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0238000f60 tx=0x7f0238009040 comp rx=0 tx=0).stop 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 msgr2=0x7f0244157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f0244157810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f023400d5c0 tx=0x7f023400da80 comp rx=0 tx=0).stop 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 shutdown_connections 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244157d50 0x7f02441f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440ce890 0x7f0244157810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f022806b4d0 0x7f022806d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 --2- 172.21.15.5:0/1742911074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244071ed0 0x7f0244143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 >> 172.21.15.5:0/1742911074 conn(0x7f0244071150 msgr2=0x7f0244158560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 shutdown_connections 2024-04-25T10:04:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:09.957+0000 7f024c800640 1 -- 172.21.15.5:0/1742911074 wait complete. 2024-04-25T10:04:09.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.177+0000 7f2731e00640 1 Processor -- start 2024-04-25T10:04:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.177+0000 7f2731e00640 1 -- start start 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f2731e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c141da0 0x7f272c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f2731e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f272c06e640 0x7f272c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f2731e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f272c15d340 0x7f272c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f2731e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c142180 con 0x7f272c06e640 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f2731e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c071d30 con 0x7f272c15d340 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f2731e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c06d620 con 0x7f272c141da0 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f272be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f272c06e640 0x7f272c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f272be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f272c06e640 0x7f272c15ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48140/0 (socket says 172.21.15.5:48140) 2024-04-25T10:04:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.179+0000 7f272be00640 1 -- 172.21.15.5:0/2247732732 learned_addr learned my addr 172.21.15.5:0/2247732732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f2731400640 1 --2- 172.21.15.5:0/2247732732 >> v2:172.21.15.80:3300/0 conn(0x7f272c15d340 0x7f272c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f2730a00640 1 --2- 172.21.15.5:0/2247732732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c141da0 0x7f272c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f272be00640 1 -- 172.21.15.5:0/2247732732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c141da0 msgr2=0x7f272c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f272be00640 1 --2- 172.21.15.5:0/2247732732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c141da0 0x7f272c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f272be00640 1 -- 172.21.15.5:0/2247732732 >> v2:172.21.15.80:3300/0 conn(0x7f272c15d340 msgr2=0x7f272c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f272be00640 1 --2- 172.21.15.5:0/2247732732 >> v2:172.21.15.80:3300/0 conn(0x7f272c15d340 0x7f272c0cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.180+0000 7f272be00640 1 -- 172.21.15.5:0/2247732732 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c06d900 con 0x7f272c06e640 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.181+0000 7f272be00640 1 --2- 172.21.15.5:0/2247732732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c06e640 0x7f272c15ce00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f272c0b2460 tx=0x7f271802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c35cd5c4fb614b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f272b400640 1 -- 172.21.15.5:0/2247732732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718037070 con 0x7f272c06e640 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f272b400640 1 -- 172.21.15.5:0/2247732732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2718032440 con 0x7f272c06e640 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f272b400640 1 -- 172.21.15.5:0/2247732732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718032810 con 0x7f272c06e640 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 -- 172.21.15.5:0/2247732732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c06e640 msgr2=0x7f272c15ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 --2- 172.21.15.5:0/2247732732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c06e640 0x7f272c15ce00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f272c0b2460 tx=0x7f271802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 -- 172.21.15.5:0/2247732732 shutdown_connections 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 --2- 172.21.15.5:0/2247732732 >> v2:172.21.15.80:3300/0 conn(0x7f272c15d340 0x7f272c0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 --2- 172.21.15.5:0/2247732732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c06e640 0x7f272c15ce00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 --2- 172.21.15.5:0/2247732732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c141da0 0x7f272c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 -- 172.21.15.5:0/2247732732 >> 172.21.15.5:0/2247732732 conn(0x7f272c066f20 msgr2=0x7f272c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 -- 172.21.15.5:0/2247732732 shutdown_connections 2024-04-25T10:04:10.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.182+0000 7f2731e00640 1 -- 172.21.15.5:0/2247732732 wait complete. 2024-04-25T10:04:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.184+0000 7f2731e00640 1 Processor -- start 2024-04-25T10:04:10.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.185+0000 7f2731e00640 1 -- start start 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c06e640 0x7f272c13c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 0x7f272c1e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f272c15d340 0x7f272c1e7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0cd9c0 con 0x7f272c141da0 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0cd840 con 0x7f272c15d340 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0cdb40 con 0x7f272c06e640 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 0x7f272c1e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 0x7f272c1e72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48150/0 (socket says 172.21.15.5:48150) 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 -- 172.21.15.5:0/4089595507 learned_addr learned my addr 172.21.15.5:0/4089595507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:10.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2730a00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c06e640 0x7f272c13c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f2731400640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f272c15d340 0x7f272c1e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 -- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c06e640 msgr2=0x7f272c13c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c06e640 0x7f272c13c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 -- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f272c15d340 msgr2=0x7f272c1e7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f272c15d340 0x7f272c1e7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.186+0000 7f272be00640 1 -- 172.21.15.5:0/4089595507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c13d030 con 0x7f272c141da0 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.187+0000 7f272be00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 0x7f272c1e72d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f271802a220 tx=0x7f27180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.187+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718041020 con 0x7f272c141da0 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.187+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27180306c0 con 0x7f272c141da0 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.187+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718037040 con 0x7f272c141da0 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.187+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c13f810 con 0x7f272c141da0 2024-04-25T10:04:10.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.187+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c13fdd0 con 0x7f272c141da0 2024-04-25T10:04:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.188+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c0af580 con 0x7f272c141da0 2024-04-25T10:04:10.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.190+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2718044020 con 0x7f272c141da0 2024-04-25T10:04:10.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.191+0000 7f2728c00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f270c06b520 0x7f270c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.191+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27180aabe0 con 0x7f272c141da0 2024-04-25T10:04:10.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.191+0000 7f2730a00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f270c06b520 0x7f270c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.193+0000 7f2730a00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f270c06b520 0x7f270c06d9e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2720006fd0 tx=0x7f2720008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.195+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f271804b030 con 0x7f272c141da0 2024-04-25T10:04:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.395+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f272c0cca80 con 0x7f272c141da0 2024-04-25T10:04:10.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.397+0000 7f2728c00640 1 -- 172.21.15.5:0/4089595507 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f272c13d030 con 0x7f272c141da0 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:10.403 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:38.104579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f270c06b520 msgr2=0x7f270c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f270c06b520 0x7f270c06d9e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2720006fd0 tx=0x7f2720008040 comp rx=0 tx=0).stop 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 msgr2=0x7f272c1e72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 0x7f272c1e72d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f271802a220 tx=0x7f27180046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 shutdown_connections 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f270c06b520 0x7f270c06d9e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f272c15d340 0x7f272c1e7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f272c141da0 0x7f272c1e72d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 --2- 172.21.15.5:0/4089595507 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f272c06e640 0x7f272c13c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 >> 172.21.15.5:0/4089595507 conn(0x7f272c066f20 msgr2=0x7f272c0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.400+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 shutdown_connections 2024-04-25T10:04:10.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.401+0000 7f2731e00640 1 -- 172.21.15.5:0/4089595507 wait complete. 2024-04-25T10:04:10.424 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:04:10.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:10.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.623+0000 7f469e600640 1 Processor -- start 2024-04-25T10:04:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.623+0000 7f469e600640 1 -- start start 2024-04-25T10:04:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.624+0000 7f469e600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 0x7f469815e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.624+0000 7f469e600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f469815e5b0 0x7f46980b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f469e600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980b4720 con 0x7f46980b4290 2024-04-25T10:04:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f469e600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469815ea10 con 0x7f469815e5b0 2024-04-25T10:04:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f469e600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980ccd40 con 0x7f469815a430 2024-04-25T10:04:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f469dc00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f46980b4290 0x7f46980d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2024-04-25T10:04:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f469dc00640 1 -- 172.21.15.5:0/3266199367 learned_addr learned my addr 172.21.15.5:0/3266199367 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f469d200640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 0x7f469815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f4697e00640 1 --2- 172.21.15.5:0/3266199367 >> v2:172.21.15.80:3300/0 conn(0x7f469815e5b0 0x7f46980b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257403513 0 0) 0x7f46980b4720 con 0x7f46980b4290 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.625+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f467c003520 con 0x7f46980b4290 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1116326422 0 0) 0x7f467c003520 con 0x7f46980b4290 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 msgr2=0x7f469815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 0x7f469815e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 >> v2:172.21.15.80:3300/0 conn(0x7f469815e5b0 msgr2=0x7f46980b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 --2- 172.21.15.5:0/3266199367 >> v2:172.21.15.80:3300/0 conn(0x7f469815e5b0 0x7f46980b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980b49b0 con 0x7f46980b4290 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4690001670 con 0x7f46980b4290 2024-04-25T10:04:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 >> v1:172.21.15.5:6789/0 conn(0x7f46980b4290 legacy=0x7f46980d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f467c0039b0 0x7f467c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 0x7f467c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f467c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980ccd40 con 0x7f467c004d40 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c00b9c0 con 0x7f467c0039b0 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c00bb40 con 0x7f469815a430 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.626+0000 7f469dc00640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f467c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469d200640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f467c0039b0 0x7f467c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469dc00640 1 -- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 msgr2=0x7f467c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f4697e00640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 0x7f467c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469dc00640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f469815a430 0x7f467c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469dc00640 1 -- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f467c0039b0 msgr2=0x7f467c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469dc00640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f467c0039b0 0x7f467c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469dc00640 1 -- 172.21.15.5:0/3266199367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c003520 con 0x7f467c004d40 2024-04-25T10:04:10.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.627+0000 7f469d200640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f467c0039b0 0x7f467c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.628+0000 7f469dc00640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f467c0051c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f469000c290 tx=0x7f467c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e011304d90cc787 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.628+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690007d50 con 0x7f467c004d40 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.628+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4690005af0 con 0x7f467c004d40 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.628+0000 7f4697400640 1 -- 172.21.15.5:0/3266199367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690007070 con 0x7f467c004d40 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469e600640 1 -- 172.21.15.5:0/3266199367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 msgr2=0x7f467c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469e600640 1 --2- 172.21.15.5:0/3266199367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f467c0051c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f469000c290 tx=0x7f467c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469dc00640 1 -- 172.21.15.5:0/3266199367 reap_dead start 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469e600640 1 -- 172.21.15.5:0/3266199367 shutdown_connections 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469e600640 1 -- 172.21.15.5:0/3266199367 >> 172.21.15.5:0/3266199367 conn(0x7f46980999f0 msgr2=0x7f46980b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469e600640 1 -- 172.21.15.5:0/3266199367 shutdown_connections 2024-04-25T10:04:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.629+0000 7f469e600640 1 -- 172.21.15.5:0/3266199367 wait complete. 2024-04-25T10:04:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.631+0000 7f469e600640 1 Processor -- start 2024-04-25T10:04:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.631+0000 7f469e600640 1 -- start start 2024-04-25T10:04:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f469815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46981488f0 0x7f469814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469e600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4698148d50 0x7f46980c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980cd5a0 con 0x7f467c004d40 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980cd2c0 con 0x7f46981488f0 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469e600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469815a810 con 0x7f4698148d50 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f469815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f469815d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48180/0 (socket says 172.21.15.5:48180) 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.632+0000 7f469d200640 1 -- 172.21.15.5:0/3219497517 learned_addr learned my addr 172.21.15.5:0/3219497517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f4697e00640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46981488f0 0x7f469814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469dc00640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4698148d50 0x7f46980c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469d200640 1 -- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4698148d50 msgr2=0x7f46980c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469d200640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4698148d50 0x7f46980c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469d200640 1 -- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46981488f0 msgr2=0x7f469814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469d200640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46981488f0 0x7f469814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469d200640 1 -- 172.21.15.5:0/3219497517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469815c4e0 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469d200640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f469815d580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4680010890 tx=0x7f4680010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000f040 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000eb00 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468001c400 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469815c7c0 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.633+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469815cd80 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.635+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4698068f60 con 0x7f467c004d40 2024-04-25T10:04:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.636+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f468001c5a0 con 0x7f467c004d40 2024-04-25T10:04:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.637+0000 7f4694c00640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f467406b4d0 0x7f467406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.637+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f468008aa10 con 0x7f467c004d40 2024-04-25T10:04:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.638+0000 7f4697e00640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f467406b4d0 0x7f467406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.639+0000 7f4697e00640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f467406b4d0 0x7f467406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f469814e2b0 tx=0x7f468800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.642+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4680018030 con 0x7f467c004d40 2024-04-25T10:04:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.879+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4698072c90 con 0x7f467c004d40 2024-04-25T10:04:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.880+0000 7f4694c00640 1 -- 172.21.15.5:0/3219497517 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f46800590a0 con 0x7f467c004d40 2024-04-25T10:04:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:10.885 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:04:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f467406b4d0 msgr2=0x7f467406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f467406b4d0 0x7f467406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f469814e2b0 tx=0x7f468800b040 comp rx=0 tx=0).stop 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 msgr2=0x7f469815d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f469815d580 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4680010890 tx=0x7f4680010d50 comp rx=0 tx=0).stop 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 shutdown_connections 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4698148d50 0x7f46980c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f467406b4d0 0x7f467406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46981488f0 0x7f469814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 --2- 172.21.15.5:0/3219497517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f467c004d40 0x7f469815d580 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.883+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 >> 172.21.15.5:0/3219497517 conn(0x7f46980999f0 msgr2=0x7f46980cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.884+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 shutdown_connections 2024-04-25T10:04:10.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:10.884+0000 7f469e600640 1 -- 172.21.15.5:0/3219497517 wait complete. 2024-04-25T10:04:10.906 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:11.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.105+0000 7f12da600640 1 Processor -- start 2024-04-25T10:04:11.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.105+0000 7f12da600640 1 -- start start 2024-04-25T10:04:11.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.106+0000 7f12da600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d07d0 0x7f12d40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.106+0000 7f12da600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12d40d2380 0x7f12d40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.106+0000 7f12da600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40cf410 con 0x7f12d4161200 2024-04-25T10:04:11.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.106+0000 7f12da600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d40b28b0 con 0x7f12d40d2380 2024-04-25T10:04:11.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.106+0000 7f12da600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40cf6f0 con 0x7f12d40d07d0 2024-04-25T10:04:11.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.107+0000 7f12d9200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f12d4161200 0x7f12d40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44610/0 (socket says 172.21.15.5:44610) 2024-04-25T10:04:11.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.107+0000 7f12d9200640 1 -- 172.21.15.5:0/2826527509 learned_addr learned my addr 172.21.15.5:0/2826527509 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.107+0000 7f12d9c00640 1 --2- 172.21.15.5:0/2826527509 >> v2:172.21.15.5:3300/0 conn(0x7f12d40d2380 0x7f12d40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.107+0000 7f12d3e00640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d07d0 0x7f12d40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.107+0000 7f12d3400640 1 -- 172.21.15.5:0/2826527509 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56861473 0 0) 0x7f12d40cf410 con 0x7f12d4161200 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.107+0000 7f12d3400640 1 -- 172.21.15.5:0/2826527509 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b8003520 con 0x7f12d4161200 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.108+0000 7f12d9c00640 1 -- 172.21.15.5:0/2826527509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d07d0 msgr2=0x7f12d40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.108+0000 7f12d9c00640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d07d0 0x7f12d40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.108+0000 7f12d9c00640 1 -- 172.21.15.5:0/2826527509 >> v1:172.21.15.5:6789/0 conn(0x7f12d4161200 legacy=0x7f12d40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.108+0000 7f12d9c00640 1 -- 172.21.15.5:0/2826527509 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40d3fd0 con 0x7f12d40d2380 2024-04-25T10:04:11.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.108+0000 7f12d3e00640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d07d0 0x7f12d40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.109+0000 7f12d9c00640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d2380 0x7f12d40b2f80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009d90 tx=0x7f12cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43cafae077387f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.109+0000 7f12d3400640 1 -- 172.21.15.5:0/2826527509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007740 con 0x7f12d40d2380 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.109+0000 7f12d3400640 1 -- 172.21.15.5:0/2826527509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc0078e0 con 0x7f12d40d2380 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.109+0000 7f12d3400640 1 -- 172.21.15.5:0/2826527509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007cb0 con 0x7f12d40d2380 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 -- 172.21.15.5:0/2826527509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d2380 msgr2=0x7f12d40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d2380 0x7f12d40b2f80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009d90 tx=0x7f12cc02a660 comp rx=0 tx=0).stop 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 -- 172.21.15.5:0/2826527509 shutdown_connections 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d2380 0x7f12d40b2f80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 --2- 172.21.15.5:0/2826527509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d07d0 0x7f12d40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 -- 172.21.15.5:0/2826527509 >> 172.21.15.5:0/2826527509 conn(0x7f12d408af80 msgr2=0x7f12d40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 -- 172.21.15.5:0/2826527509 shutdown_connections 2024-04-25T10:04:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.110+0000 7f12da600640 1 -- 172.21.15.5:0/2826527509 wait complete. 2024-04-25T10:04:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.112+0000 7f12da600640 1 Processor -- start 2024-04-25T10:04:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.113+0000 7f12da600640 1 -- start start 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12da600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 0x7f12d41f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12da600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d2380 0x7f12d406c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12da600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12d4161200 0x7f12d406c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12da600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40b3640 con 0x7f12d40d07d0 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12da600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40b34c0 con 0x7f12d4161200 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12da600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40b37c0 con 0x7f12d40d2380 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12d9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 0x7f12d41f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12d9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 0x7f12d41f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48200/0 (socket says 172.21.15.5:48200) 2024-04-25T10:04:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.114+0000 7f12d9200640 1 -- 172.21.15.5:0/1711522668 learned_addr learned my addr 172.21.15.5:0/1711522668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d9200640 1 -- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d2380 msgr2=0x7f12d406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d9200640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d2380 0x7f12d406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d9200640 1 -- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12d4161200 msgr2=0x7f12d406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d9200640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12d4161200 0x7f12d406c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d9200640 1 -- 172.21.15.5:0/1711522668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40c9560 con 0x7f12d40d07d0 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d9200640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 0x7f12d41f2a70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f12c400c3c0 tx=0x7f12c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4017070 con 0x7f12d40d07d0 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4005ce0 con 0x7f12d40d07d0 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40072f0 con 0x7f12d40d07d0 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40c89e0 con 0x7f12d40d07d0 2024-04-25T10:04:11.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.115+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d40c8fa0 con 0x7f12d40d07d0 2024-04-25T10:04:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.117+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d4068f60 con 0x7f12d40d07d0 2024-04-25T10:04:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.118+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12c401a070 con 0x7f12d40d07d0 2024-04-25T10:04:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.118+0000 7f12d0c00640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.119+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f12c4089790 con 0x7f12d40d07d0 2024-04-25T10:04:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.119+0000 7f12d3e00640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.120+0000 7f12d3e00640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006f00 tx=0x7f12bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.128+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12c4058530 con 0x7f12d40d07d0 2024-04-25T10:04:11.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.366+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12d41f3010 con 0x7f12d40d07d0 2024-04-25T10:04:11.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.367+0000 7f12d0c00640 1 -- 172.21.15.5:0/1711522668 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4932 (secure 0 0 0) 0x7f12c4057e30 con 0x7f12d40d07d0 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:11.373 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:36.556737+0000","modified":"2024-04-25T10:03:39.640872+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5005},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5005":{"gid":5005,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6837/3920492243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":3920492243},{"type":"v1","addr":"172.21.15.80:6837","nonce":3920492243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:11.374 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-04-25T10:04:11.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12b006b4d0 msgr2=0x7f12b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12b006b4d0 0x7f12b006d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006f00 tx=0x7f12bc008040 comp rx=0 tx=0).stop 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 msgr2=0x7f12d41f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 0x7f12d41f2a70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f12c400c3c0 tx=0x7f12c400c880 comp rx=0 tx=0).stop 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 shutdown_connections 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12b006b4d0 0x7f12b006d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12d4161200 0x7f12d406c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12d40d2380 0x7f12d406c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 --2- 172.21.15.5:0/1711522668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12d40d07d0 0x7f12d41f2a70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 >> 172.21.15.5:0/1711522668 conn(0x7f12d408af80 msgr2=0x7f12d40d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 shutdown_connections 2024-04-25T10:04:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.370+0000 7f12da600640 1 -- 172.21.15.5:0/1711522668 wait complete. 2024-04-25T10:04:11.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.600+0000 7f16f1200640 1 Processor -- start 2024-04-25T10:04:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.600+0000 7f16f1200640 1 -- start start 2024-04-25T10:04:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.601+0000 7f16f1200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f16ec0b2f30 0x7f16ec0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.601+0000 7f16f1200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16ec1614e0 0x7f16ec161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.601+0000 7f16f1200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec15f410 con 0x7f16ec15d8a0 2024-04-25T10:04:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.601+0000 7f16f1200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec15ffe0 con 0x7f16ec1614e0 2024-04-25T10:04:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.601+0000 7f16f1200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec15f6b0 con 0x7f16ec0b2f30 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16ec1614e0 0x7f16ec161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f16ec1614e0 0x7f16ec161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48214/0 (socket says 172.21.15.5:48214) 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 -- 172.21.15.5:0/2351567488 learned_addr learned my addr 172.21.15.5:0/2351567488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16eaa00640 1 --2- 172.21.15.5:0/2351567488 >> v2:172.21.15.80:3300/0 conn(0x7f16ec0b2f30 0x7f16ec0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 -- 172.21.15.5:0/2351567488 >> v2:172.21.15.80:3300/0 conn(0x7f16ec0b2f30 msgr2=0x7f16ec0d2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 --2- 172.21.15.5:0/2351567488 >> v2:172.21.15.80:3300/0 conn(0x7f16ec0b2f30 0x7f16ec0d2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 -- 172.21.15.5:0/2351567488 >> v1:172.21.15.80:6789/0 conn(0x7f16ec15d8a0 legacy=0x7f16ec0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.602+0000 7f16ebe00640 1 -- 172.21.15.5:0/2351567488 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec15f990 con 0x7f16ec1614e0 2024-04-25T10:04:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.603+0000 7f16ebe00640 1 --2- 172.21.15.5:0/2351567488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec161970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f16ec08bb70 tx=0x7f16e402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1d4504f8c8ea9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.603+0000 7f16ea000640 1 -- 172.21.15.5:0/2351567488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4037070 con 0x7f16ec1614e0 2024-04-25T10:04:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.603+0000 7f16ea000640 1 -- 172.21.15.5:0/2351567488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e402ade0 con 0x7f16ec1614e0 2024-04-25T10:04:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16ea000640 1 -- 172.21.15.5:0/2351567488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4032660 con 0x7f16ec1614e0 2024-04-25T10:04:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 -- 172.21.15.5:0/2351567488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 msgr2=0x7f16ec161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 --2- 172.21.15.5:0/2351567488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec161970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f16ec08bb70 tx=0x7f16e402a0d0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 -- 172.21.15.5:0/2351567488 shutdown_connections 2024-04-25T10:04:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 --2- 172.21.15.5:0/2351567488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec161970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 --2- 172.21.15.5:0/2351567488 >> v2:172.21.15.80:3300/0 conn(0x7f16ec0b2f30 0x7f16ec0d2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 -- 172.21.15.5:0/2351567488 >> 172.21.15.5:0/2351567488 conn(0x7f16ec08af80 msgr2=0x7f16ec0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 -- 172.21.15.5:0/2351567488 shutdown_connections 2024-04-25T10:04:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.604+0000 7f16f1200640 1 -- 172.21.15.5:0/2351567488 wait complete. 2024-04-25T10:04:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.606+0000 7f16f1200640 1 Processor -- start 2024-04-25T10:04:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.606+0000 7f16f1200640 1 -- start start 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.607+0000 7f16f1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16ec0b2f30 0x7f16ec0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.607+0000 7f16f1200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16ec15d8a0 0x7f16ec0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.607+0000 7f16f1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.607+0000 7f16f1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec15ee10 con 0x7f16ec1614e0 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.607+0000 7f16f1200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec15ec90 con 0x7f16ec15d8a0 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.607+0000 7f16f1200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0b2150 con 0x7f16ec0b2f30 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48218/0 (socket says 172.21.15.5:48218) 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 -- 172.21.15.5:0/1980826652 learned_addr learned my addr 172.21.15.5:0/1980826652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16eaa00640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16ec15d8a0 0x7f16ec0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16eb400640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16ec0b2f30 0x7f16ec0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 -- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16ec0b2f30 msgr2=0x7f16ec0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16ec0b2f30 0x7f16ec0cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 -- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16ec15d8a0 msgr2=0x7f16ec0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16ec15d8a0 0x7f16ec0cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16ebe00640 1 -- 172.21.15.5:0/1980826652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec0c7260 con 0x7f16ec1614e0 2024-04-25T10:04:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.608+0000 7f16eb400640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16ec0b2f30 0x7f16ec0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.609+0000 7f16ebe00640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec0ce620 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f16e40090b0 tx=0x7f16e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.609+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4041020 con 0x7f16ec1614e0 2024-04-25T10:04:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.609+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e403d420 con 0x7f16ec1614e0 2024-04-25T10:04:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.609+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4037040 con 0x7f16ec1614e0 2024-04-25T10:04:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.609+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec0c7540 con 0x7f16ec1614e0 2024-04-25T10:04:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.609+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec156ca0 con 0x7f16ec1614e0 2024-04-25T10:04:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.610+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ec068f60 con 0x7f16ec1614e0 2024-04-25T10:04:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.611+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16e4044020 con 0x7f16ec1614e0 2024-04-25T10:04:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.612+0000 7f16e3400640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16c806b4d0 0x7f16c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.612+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16e40aa830 con 0x7f16ec1614e0 2024-04-25T10:04:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.613+0000 7f16eb400640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16c806b4d0 0x7f16c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.614+0000 7f16eb400640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16c806b4d0 0x7f16c806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16dc003e60 tx=0x7f16dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.616+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16e404b030 con 0x7f16ec1614e0 2024-04-25T10:04:11.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.814+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16ec0d0e50 con 0x7f16ec1614e0 2024-04-25T10:04:11.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.817+0000 7f16e3400640 1 -- 172.21.15.5:0/1980826652 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16834 (secure 0 0 0) 0x7f16e4079480 con 0x7f16ec1614e0 2024-04-25T10:04:11.823 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:11.823 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:03:38.104579+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:34.491131+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:35.568287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16c806b4d0 msgr2=0x7f16c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16c806b4d0 0x7f16c806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f16dc003e60 tx=0x7f16dc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 msgr2=0x7f16ec0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec0ce620 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f16e40090b0 tx=0x7f16e4004730 comp rx=0 tx=0).stop 2024-04-25T10:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 shutdown_connections 2024-04-25T10:04:11.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec1614e0 0x7f16ec0ce620 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16c806b4d0 0x7f16c806d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16ec15d8a0 0x7f16ec0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 --2- 172.21.15.5:0/1980826652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f16ec0b2f30 0x7f16ec0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 >> 172.21.15.5:0/1980826652 conn(0x7f16ec08af80 msgr2=0x7f16ec0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 shutdown_connections 2024-04-25T10:04:11.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:11.821+0000 7f16f1200640 1 -- 172.21.15.5:0/1980826652 wait complete. 2024-04-25T10:04:11.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:04:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.055+0000 7f1804200640 1 Processor -- start 2024-04-25T10:04:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.055+0000 7f1804200640 1 -- start start 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1804200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f17fc06c250 0x7f17fc09a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1804200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 0x7f17fc0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1804200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc099b20 con 0x7f17fc06bce0 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1804200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc09ae20 con 0x7f17fc06c250 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1804200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc09a450 con 0x7f17fc0d0360 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1800e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f17fc06bce0 0x7f17fc06c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44622/0 (socket says 172.21.15.5:44622) 2024-04-25T10:04:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.057+0000 7f1800e00640 1 -- 172.21.15.5:0/2244959033 learned_addr learned my addr 172.21.15.5:0/2244959033 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:12.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f1802200640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 0x7f17fc0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f1801800640 1 --2- 172.21.15.5:0/2244959033 >> v2:172.21.15.80:3300/0 conn(0x7f17fc06c250 0x7f17fc09a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806074495 0 0) 0x7f17fc099b20 con 0x7f17fc06bce0 2024-04-25T10:04:12.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17e4003520 con 0x7f17fc06bce0 2024-04-25T10:04:12.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2323653365 0 0) 0x7f17e4003520 con 0x7f17fc06bce0 2024-04-25T10:04:12.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 msgr2=0x7f17fc0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 0x7f17fc0cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 >> v2:172.21.15.80:3300/0 conn(0x7f17fc06c250 msgr2=0x7f17fc09a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 --2- 172.21.15.5:0/2244959033 >> v2:172.21.15.80:3300/0 conn(0x7f17fc06c250 0x7f17fc09a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc099e00 con 0x7f17fc06bce0 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f17e8001670 con 0x7f17fc06bce0 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 >> v1:172.21.15.5:6789/0 conn(0x7f17fc06bce0 legacy=0x7f17fc06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17e40039b0 0x7f17e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 0x7f17e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e4004d40 0x7f17e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc09ae20 con 0x7f17e40039b0 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc09a450 con 0x7f17e4004d40 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.058+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e400ba40 con 0x7f17fc0d0360 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1801800640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17e40039b0 0x7f17e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1802200640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e4004d40 0x7f17e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1801800640 1 -- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 msgr2=0x7f17e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1800e00640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 0x7f17e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1801800640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc0d0360 0x7f17e4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1801800640 1 -- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e4004d40 msgr2=0x7f17e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1801800640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e4004d40 0x7f17e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1801800640 1 -- 172.21.15.5:0/2244959033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4003520 con 0x7f17e40039b0 2024-04-25T10:04:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.059+0000 7f1802200640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e4004d40 0x7f17e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.060+0000 7f1801800640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17e40039b0 0x7f17e4003dd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009ac0 tx=0x7f17e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7199bc6a81dcb30e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0036070 con 0x7f17e40039b0 2024-04-25T10:04:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0031400 con 0x7f17e40039b0 2024-04-25T10:04:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f17fbe00640 1 -- 172.21.15.5:0/2244959033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0009690 con 0x7f17e40039b0 2024-04-25T10:04:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f1804200640 1 -- 172.21.15.5:0/2244959033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17e40039b0 msgr2=0x7f17e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f1804200640 1 --2- 172.21.15.5:0/2244959033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17e40039b0 0x7f17e4003dd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009ac0 tx=0x7f17e4005830 comp rx=0 tx=0).stop 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f1802200640 1 -- 172.21.15.5:0/2244959033 reap_dead start 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f1804200640 1 -- 172.21.15.5:0/2244959033 shutdown_connections 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f1804200640 1 -- 172.21.15.5:0/2244959033 >> 172.21.15.5:0/2244959033 conn(0x7f17fc0d16e0 msgr2=0x7f17fc0d1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.061+0000 7f1804200640 1 -- 172.21.15.5:0/2244959033 shutdown_connections 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.062+0000 7f1804200640 1 -- 172.21.15.5:0/2244959033 wait complete. 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.063+0000 7f1804200640 1 Processor -- start 2024-04-25T10:04:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.063+0000 7f1804200640 1 -- start start 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.064+0000 7f1804200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e40039b0 0x7f17fc0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.064+0000 7f1804200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 0x7f17fc1e7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.064+0000 7f1804200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc1e7aa0 0x7f17fc1e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.064+0000 7f1804200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc15f000 con 0x7f17fc0cac40 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.064+0000 7f1804200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0d0c60 con 0x7f17e40039b0 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.064+0000 7f1804200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc15f180 con 0x7f17fc1e7aa0 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 0x7f17fc1e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 0x7f17fc1e7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48244/0 (socket says 172.21.15.5:48244) 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 -- 172.21.15.5:0/3148434946 learned_addr learned my addr 172.21.15.5:0/3148434946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1801800640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e40039b0 0x7f17fc0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 -- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc1e7aa0 msgr2=0x7f17fc1e7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc1e7aa0 0x7f17fc1e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 -- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e40039b0 msgr2=0x7f17fc0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e40039b0 0x7f17fc0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.065+0000 7f1800e00640 1 -- 172.21.15.5:0/3148434946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0cb460 con 0x7f17fc0cac40 2024-04-25T10:04:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.066+0000 7f1800e00640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 0x7f17fc1e7560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d770 tx=0x7f17e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.066+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8019070 con 0x7f17fc0cac40 2024-04-25T10:04:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.066+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e80074e0 con 0x7f17fc0cac40 2024-04-25T10:04:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.066+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc0cb740 con 0x7f17fc0cac40 2024-04-25T10:04:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.066+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8004a20 con 0x7f17fc0cac40 2024-04-25T10:04:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.066+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc14b160 con 0x7f17fc0cac40 2024-04-25T10:04:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.067+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc068fd0 con 0x7f17fc0cac40 2024-04-25T10:04:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.068+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17e801c070 con 0x7f17fc0cac40 2024-04-25T10:04:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.069+0000 7f17f9600640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f17dc06b500 0x7f17dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.069+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17e8089e20 con 0x7f17fc0cac40 2024-04-25T10:04:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.070+0000 7f1801800640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f17dc06b500 0x7f17dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.071+0000 7f1801800640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f17dc06b500 0x7f17dc06d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f17f0033000 tx=0x7f17f0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.073+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17e8058a70 con 0x7f17fc0cac40 2024-04-25T10:04:12.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.316+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f17fc0d1150 con 0x7f17fc0cac40 2024-04-25T10:04:12.321 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:04:12.317+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:04:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.321+0000 7f17f9600640 1 -- 172.21.15.5:0/3148434946 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f17e8058370 con 0x7f17fc0cac40 2024-04-25T10:04:12.325 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:04:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.323+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f17dc06b500 msgr2=0x7f17dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.323+0000 7f1804200640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f17dc06b500 0x7f17dc06d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f17f0033000 tx=0x7f17f0008030 comp rx=0 tx=0).stop 2024-04-25T10:04:12.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.323+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 msgr2=0x7f17fc1e7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 0x7f17fc1e7560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f17e800d770 tx=0x7f17e800dc30 comp rx=0 tx=0).stop 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 shutdown_connections 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17fc1e7aa0 0x7f17fc1e7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17fc0cac40 0x7f17fc1e7560 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f17dc06b500 0x7f17dc06d9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 --2- 172.21.15.5:0/3148434946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17e40039b0 0x7f17fc0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.324+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 >> 172.21.15.5:0/3148434946 conn(0x7f17fc0d16e0 msgr2=0x7f17fc06f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.325+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 shutdown_connections 2024-04-25T10:04:12.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.325+0000 7f1804200640 1 -- 172.21.15.5:0/3148434946 wait complete. 2024-04-25T10:04:12.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:04:12.366 INFO:tasks.ceph.mds.b.smithi080.stderr: -6793> 2024-04-25T10:03:40.043+0000 7f1f2d400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:12.436 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:04:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.556+0000 7efea2800640 1 Processor -- start 2024-04-25T10:04:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.556+0000 7efea2800640 1 -- start start 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea2800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe9c0b48c0 0x7efe9c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea2800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe9c161d90 0x7efe9c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea2800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c1606e0 con 0x7efe9c0d3f90 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea2800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c1609c0 con 0x7efe9c0b48c0 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea2800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c160ca0 con 0x7efe9c161d90 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea0a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efe9c0d3f90 0x7efe9c161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44632/0 (socket says 172.21.15.5:44632) 2024-04-25T10:04:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.557+0000 7efea0a00640 1 -- 172.21.15.5:0/1204948381 learned_addr learned my addr 172.21.15.5:0/1204948381 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efea1400640 1 --2- 172.21.15.5:0/1204948381 >> v2:172.21.15.5:3300/0 conn(0x7efe9c0b48c0 0x7efe9c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efea1e00640 1 --2- 172.21.15.5:0/1204948381 >> v2:172.21.15.80:3300/0 conn(0x7efe9c161d90 0x7efe9c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efe9be00640 1 -- 172.21.15.5:0/1204948381 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706568966 0 0) 0x7efe9c1606e0 con 0x7efe9c0d3f90 2024-04-25T10:04:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efe9be00640 1 -- 172.21.15.5:0/1204948381 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe84003520 con 0x7efe9c0d3f90 2024-04-25T10:04:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efea1400640 1 -- 172.21.15.5:0/1204948381 >> v2:172.21.15.80:3300/0 conn(0x7efe9c161d90 msgr2=0x7efe9c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efea1400640 1 --2- 172.21.15.5:0/1204948381 >> v2:172.21.15.80:3300/0 conn(0x7efe9c161d90 0x7efe9c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efea1400640 1 -- 172.21.15.5:0/1204948381 >> v1:172.21.15.5:6789/0 conn(0x7efe9c0d3f90 legacy=0x7efe9c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.558+0000 7efea1400640 1 -- 172.21.15.5:0/1204948381 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c160f80 con 0x7efe9c0b48c0 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.559+0000 7efea1400640 1 --2- 172.21.15.5:0/1204948381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c15fe60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efe9c08d2e0 tx=0x7efe9002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8af8d5e9853294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efe9be00640 1 -- 172.21.15.5:0/1204948381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90037070 con 0x7efe9c0b48c0 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efe9be00640 1 -- 172.21.15.5:0/1204948381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe90032440 con 0x7efe9c0b48c0 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efe9be00640 1 -- 172.21.15.5:0/1204948381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9002ae60 con 0x7efe9c0b48c0 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 -- 172.21.15.5:0/1204948381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 msgr2=0x7efe9c15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 --2- 172.21.15.5:0/1204948381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c15fe60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efe9c08d2e0 tx=0x7efe9002a110 comp rx=0 tx=0).stop 2024-04-25T10:04:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 -- 172.21.15.5:0/1204948381 shutdown_connections 2024-04-25T10:04:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 --2- 172.21.15.5:0/1204948381 >> v2:172.21.15.80:3300/0 conn(0x7efe9c161d90 0x7efe9c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 --2- 172.21.15.5:0/1204948381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c15fe60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 -- 172.21.15.5:0/1204948381 >> 172.21.15.5:0/1204948381 conn(0x7efe9c08af80 msgr2=0x7efe9c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 -- 172.21.15.5:0/1204948381 shutdown_connections 2024-04-25T10:04:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.560+0000 7efea2800640 1 -- 172.21.15.5:0/1204948381 wait complete. 2024-04-25T10:04:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.562+0000 7efea2800640 1 Processor -- start 2024-04-25T10:04:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.562+0000 7efea2800640 1 -- start start 2024-04-25T10:04:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.563+0000 7efea2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c0b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.563+0000 7efea2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe9c0d3f90 0x7efe9c148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.563+0000 7efea2800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efe9c161d90 0x7efe9c0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.563+0000 7efea2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c1603a0 con 0x7efe9c0b48c0 2024-04-25T10:04:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.563+0000 7efea2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b4ca0 con 0x7efe9c0d3f90 2024-04-25T10:04:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.563+0000 7efea2800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b30b0 con 0x7efe9c161d90 2024-04-25T10:04:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c0b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c0b8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48270/0 (socket says 172.21.15.5:48270) 2024-04-25T10:04:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 -- 172.21.15.5:0/210100004 learned_addr learned my addr 172.21.15.5:0/210100004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 -- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efe9c161d90 msgr2=0x7efe9c0b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efe9c161d90 0x7efe9c0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 -- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe9c0d3f90 msgr2=0x7efe9c148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe9c0d3f90 0x7efe9c148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 -- 172.21.15.5:0/210100004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0b0380 con 0x7efe9c0b48c0 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.564+0000 7efea1400640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c0b8bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efe90000c00 tx=0x7efe900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.565+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90041030 con 0x7efe9c0b48c0 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.565+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9003d420 con 0x7efe9c0b48c0 2024-04-25T10:04:12.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.565+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90037040 con 0x7efe9c0b48c0 2024-04-25T10:04:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.565+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c0b0660 con 0x7efe9c0b48c0 2024-04-25T10:04:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.565+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c148740 con 0x7efe9c0b48c0 2024-04-25T10:04:12.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.566+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c068f60 con 0x7efe9c0b48c0 2024-04-25T10:04:12.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.567+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe90044070 con 0x7efe9c0b48c0 2024-04-25T10:04:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.568+0000 7efe99600640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efe7c06b4d0 0x7efe7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.568+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efe900aaa60 con 0x7efe9c0b48c0 2024-04-25T10:04:12.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.569+0000 7efea0a00640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efe7c06b4d0 0x7efe7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.570+0000 7efea0a00640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efe7c06b4d0 0x7efe7c06d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe88007fd0 tx=0x7efe880077c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.572+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe90079670 con 0x7efe9c0b48c0 2024-04-25T10:04:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:12.812+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efe9c0d0dd0 con 0x7efe9c0b48c0 2024-04-25T10:04:13.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.330+0000 7efe99600640 1 -- 172.21.15.5:0/210100004 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7efe90078f70 con 0x7efe9c0b48c0 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efe7c06b4d0 msgr2=0x7efe7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efe7c06b4d0 0x7efe7c06d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efe88007fd0 tx=0x7efe880077c0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 msgr2=0x7efe9c0b8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c0b8bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efe90000c00 tx=0x7efe900046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 shutdown_connections 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efe9c161d90 0x7efe9c0b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efe7c06b4d0 0x7efe7c06d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe9c0d3f90 0x7efe9c148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 --2- 172.21.15.5:0/210100004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe9c0b48c0 0x7efe9c0b8bd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.334+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 >> 172.21.15.5:0/210100004 conn(0x7efe9c08af80 msgr2=0x7efe9c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.335+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 shutdown_connections 2024-04-25T10:04:13.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.335+0000 7efea2800640 1 -- 172.21.15.5:0/210100004 wait complete. 2024-04-25T10:04:13.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:04:13.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.576+0000 7fd7c7200640 1 Processor -- start 2024-04-25T10:04:13.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.576+0000 7fd7c7200640 1 -- start start 2024-04-25T10:04:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.577+0000 7fd7c7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7c00d0aa0 0x7fd7c00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.577+0000 7fd7c7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00c6620 con 0x7fd7c00cfad0 2024-04-25T10:04:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.577+0000 7fd7c7200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00c6900 con 0x7fd7c015b8c0 2024-04-25T10:04:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.577+0000 7fd7c7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c00c63f0 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c6800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd7c00cfad0 0x7fd7c00c55c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44644/0 (socket says 172.21.15.5:44644) 2024-04-25T10:04:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c6800640 1 -- 172.21.15.5:0/1587958785 learned_addr learned my addr 172.21.15.5:0/1587958785 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c5400640 1 --2- 172.21.15.5:0/1587958785 >> v2:172.21.15.5:3300/0 conn(0x7fd7c00d0aa0 0x7fd7c00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c4a00640 1 -- 172.21.15.5:0/1587958785 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007667828 0 0) 0x7fd7c00c6620 con 0x7fd7c00cfad0 2024-04-25T10:04:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c4a00640 1 -- 172.21.15.5:0/1587958785 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a8003520 con 0x7fd7c00cfad0 2024-04-25T10:04:13.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c5400640 1 -- 172.21.15.5:0/1587958785 >> v1:172.21.15.80:6789/0 conn(0x7fd7c015b8c0 legacy=0x7fd7c00ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.578+0000 7fd7c4a00640 1 -- 172.21.15.5:0/1587958785 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506719384 0 0) 0x7fd7a8003520 con 0x7fd7c00cfad0 2024-04-25T10:04:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.579+0000 7fd7c5400640 1 -- 172.21.15.5:0/1587958785 >> v1:172.21.15.5:6789/0 conn(0x7fd7c00cfad0 legacy=0x7fd7c00c55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.579+0000 7fd7c5400640 1 -- 172.21.15.5:0/1587958785 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00c6be0 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.580+0000 7fd7c5400640 1 --2- 172.21.15.5:0/1587958785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00cf590 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40097a0 tx=0x7fd7b402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3a1c9e8372671e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.580+0000 7fd7c4a00640 1 -- 172.21.15.5:0/1587958785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4037070 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.580+0000 7fd7c4a00640 1 -- 172.21.15.5:0/1587958785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b402abc0 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.580+0000 7fd7c4a00640 1 -- 172.21.15.5:0/1587958785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0019f0 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.580+0000 7fd7c7200640 1 -- 172.21.15.5:0/1587958785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 msgr2=0x7fd7c00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.580+0000 7fd7c7200640 1 --2- 172.21.15.5:0/1587958785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00cf590 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40097a0 tx=0x7fd7b402a130 comp rx=0 tx=0).stop 2024-04-25T10:04:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.581+0000 7fd7c7200640 1 -- 172.21.15.5:0/1587958785 shutdown_connections 2024-04-25T10:04:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.581+0000 7fd7c7200640 1 --2- 172.21.15.5:0/1587958785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00cf590 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.581+0000 7fd7c7200640 1 -- 172.21.15.5:0/1587958785 >> 172.21.15.5:0/1587958785 conn(0x7fd7c00999f0 msgr2=0x7fd7c00d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.581+0000 7fd7c7200640 1 -- 172.21.15.5:0/1587958785 shutdown_connections 2024-04-25T10:04:13.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.581+0000 7fd7c7200640 1 -- 172.21.15.5:0/1587958785 wait complete. 2024-04-25T10:04:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.582+0000 7fd7c7200640 1 Processor -- start 2024-04-25T10:04:13.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 -- start start 2024-04-25T10:04:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7c00cfad0 0x7fd7c00c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7c015b8c0 0x7fd7c00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00d0f20 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cff00 con 0x7fd7c00cfad0 2024-04-25T10:04:13.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.583+0000 7fd7c7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00d10a0 con 0x7fd7c015b8c0 2024-04-25T10:04:13.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48290/0 (socket says 172.21.15.5:48290) 2024-04-25T10:04:13.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 -- 172.21.15.5:0/3891737206 learned_addr learned my addr 172.21.15.5:0/3891737206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5e00640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7c00cfad0 0x7fd7c00c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c6800640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7c015b8c0 0x7fd7c00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 -- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7c015b8c0 msgr2=0x7fd7c00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7c015b8c0 0x7fd7c00cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 -- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7c00cfad0 msgr2=0x7fd7c00c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7c00cfad0 0x7fd7c00c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5400640 1 -- 172.21.15.5:0/3891737206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00c8360 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c5e00640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7c00cfad0 0x7fd7c00c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.584+0000 7fd7c6800640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7c015b8c0 0x7fd7c00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.585+0000 7fd7c5400640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00ca930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4035040 tx=0x7fd7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.585+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4041070 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.585+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b403c660 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.585+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4037040 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.585+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c00c8640 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.585+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c00c8c00 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.586+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0068f60 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.587+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7b4044070 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.588+0000 7fd7ba000640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd7a006b500 0x7fd7a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.589+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd7b40aacf0 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.589+0000 7fd7c5e00640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd7a006b500 0x7fd7a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.590+0000 7fd7c5e00640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd7a006b500 0x7fd7a006d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0015e0 tx=0x7fd7ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.593+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7b40798f0 con 0x7fd7c00d0aa0 2024-04-25T10:04:13.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:13.790+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7c00ca400 con 0x7fd7c00d0aa0 2024-04-25T10:04:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.334+0000 7fd7ba000640 1 -- 172.21.15.5:0/3891737206 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd7b40791f0 con 0x7fd7c00d0aa0 2024-04-25T10:04:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:04:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.337+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd7a006b500 msgr2=0x7fd7a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.337+0000 7fd7c7200640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd7a006b500 0x7fd7a006d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0015e0 tx=0x7fd7ac009040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.337+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 msgr2=0x7fd7c00ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.337+0000 7fd7c7200640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00ca930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4035040 tx=0x7fd7b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 shutdown_connections 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7c015b8c0 0x7fd7c00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7c00d0aa0 0x7fd7c00ca930 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd7a006b500 0x7fd7a006d9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 --2- 172.21.15.5:0/3891737206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7c00cfad0 0x7fd7c00c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 >> 172.21.15.5:0/3891737206 conn(0x7fd7c00999f0 msgr2=0x7fd7c00b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 shutdown_connections 2024-04-25T10:04:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.338+0000 7fd7c7200640 1 -- 172.21.15.5:0/3891737206 wait complete. 2024-04-25T10:04:14.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:04:14.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.567+0000 7fc2dbe00640 1 Processor -- start 2024-04-25T10:04:14.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.568+0000 7fc2dbe00640 1 -- start start 2024-04-25T10:04:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.569+0000 7fc2dbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2dc0d1c70 0x7fc2dc0d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.569+0000 7fc2dbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 0x7fc2dc06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.569+0000 7fc2dbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc06bf50 con 0x7fc2dc15da30 2024-04-25T10:04:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.569+0000 7fc2dbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc0d2610 con 0x7fc2dc0d1c70 2024-04-25T10:04:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.569+0000 7fc2dbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc06c230 con 0x7fc2dc06cd10 2024-04-25T10:04:14.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.570+0000 7fc2db400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 0x7fc2dc06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.570+0000 7fc2da000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2dc0d1c70 0x7fc2dc0d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.570+0000 7fc2db400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 0x7fc2dc06d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:50284/0 (socket says 172.21.15.5:50284) 2024-04-25T10:04:14.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.570+0000 7fc2db400640 1 -- 172.21.15.5:0/3305008976 learned_addr learned my addr 172.21.15.5:0/3305008976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.571+0000 7fc2db400640 1 -- 172.21.15.5:0/3305008976 >> v2:172.21.15.80:3300/0 conn(0x7fc2dc0d1c70 msgr2=0x7fc2dc0d20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.571+0000 7fc2db400640 1 --2- 172.21.15.5:0/3305008976 >> v2:172.21.15.80:3300/0 conn(0x7fc2dc0d1c70 0x7fc2dc0d20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.571+0000 7fc2db400640 1 -- 172.21.15.5:0/3305008976 >> v1:172.21.15.80:6789/0 conn(0x7fc2dc15da30 legacy=0x7fc2dc0cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.571+0000 7fc2db400640 1 -- 172.21.15.5:0/3305008976 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc06c510 con 0x7fc2dc06cd10 2024-04-25T10:04:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.572+0000 7fc2db400640 1 --2- 172.21.15.5:0/3305008976 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 0x7fc2dc06d1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c1aa8a5c408fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.573+0000 7fc2d9600640 1 -- 172.21.15.5:0/3305008976 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4037070 con 0x7fc2dc06cd10 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.573+0000 7fc2d9600640 1 -- 172.21.15.5:0/3305008976 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4032440 con 0x7fc2dc06cd10 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.573+0000 7fc2d9600640 1 -- 172.21.15.5:0/3305008976 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4032810 con 0x7fc2dc06cd10 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 -- 172.21.15.5:0/3305008976 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 msgr2=0x7fc2dc06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/3305008976 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 0x7fc2dc06d1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4009870 tx=0x7fc2d402a250 comp rx=0 tx=0).stop 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 -- 172.21.15.5:0/3305008976 shutdown_connections 2024-04-25T10:04:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/3305008976 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc06cd10 0x7fc2dc06d1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/3305008976 >> v2:172.21.15.80:3300/0 conn(0x7fc2dc0d1c70 0x7fc2dc0d20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 -- 172.21.15.5:0/3305008976 >> 172.21.15.5:0/3305008976 conn(0x7fc2dc08af80 msgr2=0x7fc2dc0d3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.574+0000 7fc2dbe00640 1 -- 172.21.15.5:0/3305008976 shutdown_connections 2024-04-25T10:04:14.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.575+0000 7fc2dbe00640 1 -- 172.21.15.5:0/3305008976 wait complete. 2024-04-25T10:04:14.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.577+0000 7fc2dbe00640 1 Processor -- start 2024-04-25T10:04:14.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.577+0000 7fc2dbe00640 1 -- start start 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.578+0000 7fc2dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 0x7fc2dc1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.578+0000 7fc2dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2dc0d1c70 0x7fc2dc14fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2dbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc15da30 0x7fc2dc0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2dbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0cf570 con 0x7fc2dc06cd10 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2dbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc15de10 con 0x7fc2dc0d1c70 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2dbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0d4290 con 0x7fc2dc15da30 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2daa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 0x7fc2dc1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2daa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 0x7fc2dc1f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48300/0 (socket says 172.21.15.5:48300) 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2daa00640 1 -- 172.21.15.5:0/1887035950 learned_addr learned my addr 172.21.15.5:0/1887035950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2da000640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2dc0d1c70 0x7fc2dc14fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2db400640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc15da30 0x7fc2dc0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2daa00640 1 -- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc15da30 msgr2=0x7fc2dc0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.579+0000 7fc2daa00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc15da30 0x7fc2dc0cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2daa00640 1 -- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2dc0d1c70 msgr2=0x7fc2dc14fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2daa00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2dc0d1c70 0x7fc2dc14fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2daa00640 1 -- 172.21.15.5:0/1887035950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0ca900 con 0x7fc2dc06cd10 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2daa00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 0x7fc2dc1f44d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c730 tx=0x7fc2cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc016020 con 0x7fc2dc06cd10 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc006d50 con 0x7fc2dc06cd10 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc003bb0 con 0x7fc2dc06cd10 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0cabe0 con 0x7fc2dc06cd10 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.580+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc0cb1a0 con 0x7fc2dc06cd10 2024-04-25T10:04:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.582+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc068f60 con 0x7fc2dc06cd10 2024-04-25T10:04:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.583+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2cc00a850 con 0x7fc2dc06cd10 2024-04-25T10:04:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.584+0000 7fc2d2a00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2b806b4d0 0x7fc2b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.584+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fc2cc089ef0 con 0x7fc2dc06cd10 2024-04-25T10:04:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.585+0000 7fc2da000640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2b806b4d0 0x7fc2b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.586+0000 7fc2da000640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2b806b4d0 0x7fc2b806d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0cf480 tx=0x7fc2c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.589+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2cc020020 con 0x7fc2dc06cd10 2024-04-25T10:04:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:14.790+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc2dc0d2200 con 0x7fc2dc06cd10 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.344+0000 7fc2d2a00640 1 -- 172.21.15.5:0/1887035950 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc2cc019310 con 0x7fc2dc06cd10 2024-04-25T10:04:15.350 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:04:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.348+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2b806b4d0 msgr2=0x7fc2b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.348+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2b806b4d0 0x7fc2b806d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0cf480 tx=0x7fc2c4008040 comp rx=0 tx=0).stop 2024-04-25T10:04:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.348+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 msgr2=0x7fc2dc1f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.348+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 0x7fc2dc1f44d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c730 tx=0x7fc2cc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.349+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 shutdown_connections 2024-04-25T10:04:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.349+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2dc15da30 0x7fc2dc0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.349+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2b806b4d0 0x7fc2b806d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.349+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2dc0d1c70 0x7fc2dc14fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.349+0000 7fc2dbe00640 1 --2- 172.21.15.5:0/1887035950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2dc06cd10 0x7fc2dc1f44d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.349+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 >> 172.21.15.5:0/1887035950 conn(0x7fc2dc08af80 msgr2=0x7fc2dc0b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.351+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 shutdown_connections 2024-04-25T10:04:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.351+0000 7fc2dbe00640 1 -- 172.21.15.5:0/1887035950 wait complete. 2024-04-25T10:04:15.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:04:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.571+0000 7f6022800640 1 Processor -- start 2024-04-25T10:04:15.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.571+0000 7f6022800640 1 -- start start 2024-04-25T10:04:15.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.573+0000 7f6022800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f601c144fe0 0x7f601c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.573+0000 7f6022800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c071840 0x7f601c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.573+0000 7f6022800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601c1453c0 con 0x7f601c06d130 2024-04-25T10:04:15.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.573+0000 7f6022800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c0a8560 con 0x7f601c144fe0 2024-04-25T10:04:15.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.573+0000 7f6022800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c06d6a0 con 0x7f601c071840 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.573+0000 7f601b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f601c144fe0 0x7f601c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f601c144fe0 0x7f601c06e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48316/0 (socket says 172.21.15.5:48316) 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601b400640 1 -- 172.21.15.5:0/3479328588 learned_addr learned my addr 172.21.15.5:0/3479328588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601be00640 1 --2- 172.21.15.5:0/3479328588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c071840 0x7f601c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601b400640 1 -- 172.21.15.5:0/3479328588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c071840 msgr2=0x7f601c15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601b400640 1 --2- 172.21.15.5:0/3479328588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c071840 0x7f601c15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601b400640 1 -- 172.21.15.5:0/3479328588 >> v1:172.21.15.80:6789/0 conn(0x7f601c06d130 legacy=0x7f601c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.574+0000 7f601b400640 1 -- 172.21.15.5:0/3479328588 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c06d980 con 0x7f601c144fe0 2024-04-25T10:04:15.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.576+0000 7f601b400640 1 --2- 172.21.15.5:0/3479328588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c144fe0 0x7f601c06e560 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f600c009870 tx=0x7f600c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed10c6c41ca32a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.576+0000 7f601a000640 1 -- 172.21.15.5:0/3479328588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c037070 con 0x7f601c144fe0 2024-04-25T10:04:15.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.576+0000 7f601a000640 1 -- 172.21.15.5:0/3479328588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c032440 con 0x7f601c144fe0 2024-04-25T10:04:15.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.576+0000 7f601a000640 1 -- 172.21.15.5:0/3479328588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c032810 con 0x7f601c144fe0 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 -- 172.21.15.5:0/3479328588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c144fe0 msgr2=0x7f601c06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 --2- 172.21.15.5:0/3479328588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c144fe0 0x7f601c06e560 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f600c009870 tx=0x7f600c02a270 comp rx=0 tx=0).stop 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 -- 172.21.15.5:0/3479328588 shutdown_connections 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 --2- 172.21.15.5:0/3479328588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c071840 0x7f601c15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 --2- 172.21.15.5:0/3479328588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c144fe0 0x7f601c06e560 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 -- 172.21.15.5:0/3479328588 >> 172.21.15.5:0/3479328588 conn(0x7f601c06ab60 msgr2=0x7f601c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 -- 172.21.15.5:0/3479328588 shutdown_connections 2024-04-25T10:04:15.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.577+0000 7f6022800640 1 -- 172.21.15.5:0/3479328588 wait complete. 2024-04-25T10:04:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.579+0000 7f6022800640 1 Processor -- start 2024-04-25T10:04:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.579+0000 7f6022800640 1 -- start start 2024-04-25T10:04:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.580+0000 7f6022800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 0x7f601c1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.580+0000 7f6022800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601c071840 0x7f601c1e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.580+0000 7f6022800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c144fe0 0x7f601c0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.580+0000 7f6022800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c15be50 con 0x7f601c06d130 2024-04-25T10:04:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.580+0000 7f6022800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c06df40 con 0x7f601c071840 2024-04-25T10:04:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.580+0000 7f6022800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c06eaa0 con 0x7f601c144fe0 2024-04-25T10:04:15.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 0x7f601c1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 0x7f601c1f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48322/0 (socket says 172.21.15.5:48322) 2024-04-25T10:04:15.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 -- 172.21.15.5:0/3582958072 learned_addr learned my addr 172.21.15.5:0/3582958072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 -- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c144fe0 msgr2=0x7f601c0c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c144fe0 0x7f601c0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 -- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601c071840 msgr2=0x7f601c1e5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601c071840 0x7f601c1e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 -- 172.21.15.5:0/3582958072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0c46f0 con 0x7f601c06d130 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.581+0000 7f601b400640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 0x7f601c1f44b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f600c02a2b0 tx=0x7f600c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.582+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c037040 con 0x7f601c06d130 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.582+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c042070 con 0x7f601c06d130 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.582+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c030880 con 0x7f601c06d130 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.582+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601c0c49d0 con 0x7f601c06d130 2024-04-25T10:04:15.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.582+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c1e5fc0 con 0x7f601c06d130 2024-04-25T10:04:15.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.583+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c068f60 con 0x7f601c06d130 2024-04-25T10:04:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.584+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f600c030a20 con 0x7f601c06d130 2024-04-25T10:04:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.585+0000 7f6013400640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.585+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f600c0aa870 con 0x7f601c06d130 2024-04-25T10:04:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.586+0000 7f601aa00640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.587+0000 7f601aa00640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f60080079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.590+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f600c045360 con 0x7f601c06d130 2024-04-25T10:04:15.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.791+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f601c0ae9b0 con 0x7f601c06d130 2024-04-25T10:04:15.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.798+0000 7f6013400640 1 -- 172.21.15.5:0/3582958072 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f600c0798f0 con 0x7f601c06d130 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ffc06b4d0 msgr2=0x7f5ffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6008000f60 tx=0x7f60080079d0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 msgr2=0x7f601c1f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 0x7f601c1f44b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f600c02a2b0 tx=0x7f600c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 shutdown_connections 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f601c144fe0 0x7f601c0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ffc06b4d0 0x7f5ffc06d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601c071840 0x7f601c1e5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 --2- 172.21.15.5:0/3582958072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f601c06d130 0x7f601c1f44b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 >> 172.21.15.5:0/3582958072 conn(0x7f601c06ab60 msgr2=0x7f601c0a8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 shutdown_connections 2024-04-25T10:04:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:15.801+0000 7f6022800640 1 -- 172.21.15.5:0/3582958072 wait complete. 2024-04-25T10:04:15.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:04:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.024+0000 7f04bf200640 1 Processor -- start 2024-04-25T10:04:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.024+0000 7f04bf200640 1 -- start start 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.026+0000 7f04bf200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04c015eda0 0x7f04c00ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.026+0000 7f04bf200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00b4880 con 0x7f04c00cecd0 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.026+0000 7f04bf200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00b45a0 con 0x7f04c0143d30 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.026+0000 7f04bf200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c015e2c0 con 0x7f04c015eda0 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04be800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f04c00cecd0 0x7f04c015abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44648/0 (socket says 172.21.15.5:44648) 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04be800640 1 -- 172.21.15.5:0/2743237325 learned_addr learned my addr 172.21.15.5:0/2743237325 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:16.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04b7e00640 1 --2- 172.21.15.5:0/2743237325 >> v2:172.21.15.5:3300/0 conn(0x7f04c015eda0 0x7f04c00ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04bd400640 1 -- 172.21.15.5:0/2743237325 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893268973 0 0) 0x7f04c00b4880 con 0x7f04c00cecd0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04bd400640 1 -- 172.21.15.5:0/2743237325 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a4003520 con 0x7f04c00cecd0 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04b7e00640 1 -- 172.21.15.5:0/2743237325 >> v1:172.21.15.80:6789/0 conn(0x7f04c0143d30 legacy=0x7f04c00b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04b7e00640 1 -- 172.21.15.5:0/2743237325 >> v1:172.21.15.5:6789/0 conn(0x7f04c00cecd0 legacy=0x7f04c015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.027+0000 7f04b7e00640 1 -- 172.21.15.5:0/2743237325 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c015e5a0 con 0x7f04c015eda0 2024-04-25T10:04:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.029+0000 7f04b7e00640 1 --2- 172.21.15.5:0/2743237325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c015eda0 0x7f04c00ce790 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f04a80097a0 tx=0x7f04a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29edfbf4e6f24cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.029+0000 7f04bd400640 1 -- 172.21.15.5:0/2743237325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8037070 con 0x7f04c015eda0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.029+0000 7f04bd400640 1 -- 172.21.15.5:0/2743237325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a8032440 con 0x7f04c015eda0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.029+0000 7f04bd400640 1 -- 172.21.15.5:0/2743237325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a8032810 con 0x7f04c015eda0 2024-04-25T10:04:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 -- 172.21.15.5:0/2743237325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c015eda0 msgr2=0x7f04c00ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 --2- 172.21.15.5:0/2743237325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c015eda0 0x7f04c00ce790 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f04a80097a0 tx=0x7f04a802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 -- 172.21.15.5:0/2743237325 shutdown_connections 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 --2- 172.21.15.5:0/2743237325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c015eda0 0x7f04c00ce790 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 -- 172.21.15.5:0/2743237325 >> 172.21.15.5:0/2743237325 conn(0x7f04c0066f20 msgr2=0x7f04c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 -- 172.21.15.5:0/2743237325 shutdown_connections 2024-04-25T10:04:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.030+0000 7f04bf200640 1 -- 172.21.15.5:0/2743237325 wait complete. 2024-04-25T10:04:16.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.031+0000 7f04bf200640 1 Processor -- start 2024-04-25T10:04:16.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.032+0000 7f04bf200640 1 -- start start 2024-04-25T10:04:16.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.032+0000 7f04bf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 0x7f04c015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04c0143d30 0x7f04c00aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04c015eda0 0x7f04c00ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00b41f0 con 0x7f04c00cecd0 2024-04-25T10:04:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0144110 con 0x7f04c0143d30 2024-04-25T10:04:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00a94b0 con 0x7f04c015eda0 2024-04-25T10:04:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 0x7f04c015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 0x7f04c015a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48328/0 (socket says 172.21.15.5:48328) 2024-04-25T10:04:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 -- 172.21.15.5:0/3274544972 learned_addr learned my addr 172.21.15.5:0/3274544972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04b7e00640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04c0143d30 0x7f04c00aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04be800640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04c015eda0 0x7f04c00ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 -- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04c015eda0 msgr2=0x7f04c00ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04c015eda0 0x7f04c00ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 -- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04c0143d30 msgr2=0x7f04c00aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04c0143d30 0x7f04c00aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.033+0000 7f04bde00640 1 -- 172.21.15.5:0/3274544972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00a9790 con 0x7f04c00cecd0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.034+0000 7f04bde00640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 0x7f04c015a010 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c400 tx=0x7f04b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.034+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b00052d0 con 0x7f04c00cecd0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.034+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0005490 con 0x7f04c00cecd0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.034+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0007260 con 0x7f04c00cecd0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.034+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c00a9a70 con 0x7f04c00cecd0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.034+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c00aa550 con 0x7f04c00cecd0 2024-04-25T10:04:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.036+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c0068f60 con 0x7f04c00cecd0 2024-04-25T10:04:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.037+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04b0007400 con 0x7f04c00cecd0 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.037+0000 7f04b6000640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.038+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f04b0089550 con 0x7f04c00cecd0 2024-04-25T10:04:16.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.038+0000 7f04b7e00640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.039+0000 7f04b7e00640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f049c06b4d0 0x7f049c06d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.043+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04b00585d0 con 0x7f04c00cecd0 2024-04-25T10:04:16.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.319+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f04c00ce230 con 0x7f04c00cecd0 2024-04-25T10:04:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.347+0000 7f04b6000640 1 -- 172.21.15.5:0/3274544972 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v150) v1 ==== 142+0+0 (secure 0 0 0) 0x7f04b0057ed0 con 0x7f04c00cecd0 2024-04-25T10:04:16.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.350+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f049c06b4d0 msgr2=0x7f049c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.350+0000 7f04bf200640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f049c06b4d0 0x7f049c06d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.350+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 msgr2=0x7f04c015a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.350+0000 7f04bf200640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 0x7f04c015a010 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c400 tx=0x7f04b000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.350+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 shutdown_connections 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.350+0000 7f04bf200640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04c015eda0 0x7f04c00ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.351+0000 7f04bf200640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f049c06b4d0 0x7f049c06d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.351+0000 7f04bf200640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04c0143d30 0x7f04c00aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.351+0000 7f04bf200640 1 --2- 172.21.15.5:0/3274544972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04c00cecd0 0x7f04c015a010 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.351+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 >> 172.21.15.5:0/3274544972 conn(0x7f04c0066f20 msgr2=0x7f04c00b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.351+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 shutdown_connections 2024-04-25T10:04:16.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.351+0000 7f04bf200640 1 -- 172.21.15.5:0/3274544972 wait complete. 2024-04-25T10:04:16.375 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:04:16.375 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T10:04:16.375 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:04:16.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.574+0000 7f00ffe00640 1 Processor -- start 2024-04-25T10:04:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.575+0000 7f00ffe00640 1 -- start start 2024-04-25T10:04:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00ffe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0100161200 0x7f01000b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00ffe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01000d07d0 0x7f01000b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00ffe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01000cf410 con 0x7f01000d2380 2024-04-25T10:04:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00ffe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01000d27c0 con 0x7f01000d07d0 2024-04-25T10:04:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00ffe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01000b28b0 con 0x7f0100161200 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00fe000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f01000d2380 0x7f01000d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44658/0 (socket says 172.21.15.5:44658) 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00fe000640 1 -- 172.21.15.5:0/2164999069 learned_addr learned my addr 172.21.15.5:0/2164999069 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.576+0000 7f00ff400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01000d07d0 0x7f01000b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.577+0000 7f00fea00640 1 --2- 172.21.15.5:0/2164999069 >> v2:172.21.15.80:3300/0 conn(0x7f0100161200 0x7f01000b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.577+0000 7f00ff400640 1 -- 172.21.15.5:0/2164999069 >> v2:172.21.15.80:3300/0 conn(0x7f0100161200 msgr2=0x7f01000b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.577+0000 7f00ff400640 1 --2- 172.21.15.5:0/2164999069 >> v2:172.21.15.80:3300/0 conn(0x7f0100161200 0x7f01000b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.577+0000 7f00ff400640 1 -- 172.21.15.5:0/2164999069 >> v1:172.21.15.5:6789/0 conn(0x7f01000d2380 legacy=0x7f01000d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.577+0000 7f00ff400640 1 -- 172.21.15.5:0/2164999069 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cf6f0 con 0x7f01000d07d0 2024-04-25T10:04:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.578+0000 7f00ff400640 1 --2- 172.21.15.5:0/2164999069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d07d0 0x7f01000b2f80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f010008bed0 tx=0x7f00f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7efd882e74f079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.578+0000 7f00fd600640 1 -- 172.21.15.5:0/2164999069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8037070 con 0x7f01000d07d0 2024-04-25T10:04:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00fd600640 1 -- 172.21.15.5:0/2164999069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f8032440 con 0x7f01000d07d0 2024-04-25T10:04:16.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00fd600640 1 -- 172.21.15.5:0/2164999069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8032810 con 0x7f01000d07d0 2024-04-25T10:04:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 -- 172.21.15.5:0/2164999069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d07d0 msgr2=0x7f01000b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 --2- 172.21.15.5:0/2164999069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d07d0 0x7f01000b2f80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f010008bed0 tx=0x7f00f802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 -- 172.21.15.5:0/2164999069 shutdown_connections 2024-04-25T10:04:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 --2- 172.21.15.5:0/2164999069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d07d0 0x7f01000b2f80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 --2- 172.21.15.5:0/2164999069 >> v2:172.21.15.80:3300/0 conn(0x7f0100161200 0x7f01000b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 -- 172.21.15.5:0/2164999069 >> 172.21.15.5:0/2164999069 conn(0x7f010008af80 msgr2=0x7f01000d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 -- 172.21.15.5:0/2164999069 shutdown_connections 2024-04-25T10:04:16.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.579+0000 7f00ffe00640 1 -- 172.21.15.5:0/2164999069 wait complete. 2024-04-25T10:04:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.581+0000 7f00ffe00640 1 Processor -- start 2024-04-25T10:04:16.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.581+0000 7f00ffe00640 1 -- start start 2024-04-25T10:04:16.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01000d07d0 0x7f01000ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 0x7f010014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00ffe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0100161200 0x7f01000cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00ffe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000b34c0 con 0x7f01000d2380 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00ffe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000d42b0 con 0x7f01000d07d0 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00ffe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000b3640 con 0x7f0100161200 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.582+0000 7f00fe000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 0x7f010014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 0x7f010014d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48342/0 (socket says 172.21.15.5:48342) 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 -- 172.21.15.5:0/3376877945 learned_addr learned my addr 172.21.15.5:0/3376877945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fea00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01000d07d0 0x7f01000ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00ff400640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0100161200 0x7f01000cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 -- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0100161200 msgr2=0x7f01000cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0100161200 0x7f01000cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 -- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01000d07d0 msgr2=0x7f01000ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01000d07d0 0x7f01000ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 -- 172.21.15.5:0/3376877945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cc0b0 con 0x7f01000d2380 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.583+0000 7f00fe000640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 0x7f010014d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00c430 tx=0x7f00ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.584+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec017070 con 0x7f01000d2380 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.584+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec005ce0 con 0x7f01000d2380 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.584+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0072f0 con 0x7f01000d2380 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.584+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01000cb010 con 0x7f01000d2380 2024-04-25T10:04:16.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.584+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01000cb5d0 con 0x7f01000d2380 2024-04-25T10:04:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.585+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0100068f60 con 0x7f01000d2380 2024-04-25T10:04:16.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.586+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00ec01a070 con 0x7f01000d2380 2024-04-25T10:04:16.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.587+0000 7f00f6a00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f00e006b520 0x7f00e006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.587+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f00ec089390 con 0x7f01000d2380 2024-04-25T10:04:16.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.588+0000 7f00fea00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f00e006b520 0x7f00e006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.589+0000 7f00fea00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f00e006b520 0x7f00e006d9e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f00f00073f0 tx=0x7f00f000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.592+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00ec0584f0 con 0x7f01000d2380 2024-04-25T10:04:16.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.796+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0100160830 con 0x7f01000d2380 2024-04-25T10:04:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.798+0000 7f00f6a00640 1 -- 172.21.15.5:0/3376877945 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13185 (secure 0 0 0) 0x7f00ec057df0 con 0x7f01000d2380 2024-04-25T10:04:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:16.803 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":43,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:15.335617+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6837/3920492243":"2024-04-26T10:04:12.317478+0000","172.21.15.80:6835/3920492243":"2024-04-26T10:04:12.317478+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f00e006b520 msgr2=0x7f00e006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f00e006b520 0x7f00e006d9e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f00f00073f0 tx=0x7f00f000b040 comp rx=0 tx=0).stop 2024-04-25T10:04:16.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 msgr2=0x7f010014d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 0x7f010014d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00c430 tx=0x7f00ec00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 shutdown_connections 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0100161200 0x7f01000cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01000d2380 0x7f010014d9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f00e006b520 0x7f00e006d9e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 --2- 172.21.15.5:0/3376877945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01000d07d0 0x7f01000ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.800+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 >> 172.21.15.5:0/3376877945 conn(0x7f010008af80 msgr2=0x7f01000b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.801+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 shutdown_connections 2024-04-25T10:04:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:16.801+0000 7f00ffe00640 1 -- 172.21.15.5:0/3376877945 wait complete. 2024-04-25T10:04:16.825 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:16.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.037+0000 7f3b00c00640 1 Processor -- start 2024-04-25T10:04:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.038+0000 7f3b00c00640 1 -- start start 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.039+0000 7f3b00c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3afc0d3f90 0x7f3afc161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.039+0000 7f3b00c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3afc1606e0 con 0x7f3afc0b48c0 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.039+0000 7f3b00c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3afc1609c0 con 0x7f3afc161bc0 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.039+0000 7f3b00c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3afc161ff0 con 0x7f3afc0d3f90 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.040+0000 7f3afb400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3afc0b48c0 0x7f3afc15fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44712/0 (socket says 172.21.15.5:44712) 2024-04-25T10:04:17.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.040+0000 7f3afb400640 1 -- 172.21.15.5:0/3823023502 learned_addr learned my addr 172.21.15.5:0/3823023502 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.040+0000 7f3af2a00640 1 --2- 172.21.15.5:0/3823023502 >> v2:172.21.15.80:3300/0 conn(0x7f3afc0d3f90 0x7f3afc161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.040+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330350886 0 0) 0x7f3afc1606e0 con 0x7f3afc0b48c0 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.040+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae0003520 con 0x7f3afc0b48c0 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711619032 0 0) 0x7f3ae0003520 con 0x7f3afc0b48c0 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 >> v2:172.21.15.80:3300/0 conn(0x7f3afc0d3f90 msgr2=0x7f3afc161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 --2- 172.21.15.5:0/3823023502 >> v2:172.21.15.80:3300/0 conn(0x7f3afc0d3f90 0x7f3afc161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 >> v1:172.21.15.80:6789/0 conn(0x7f3afc161bc0 legacy=0x7f3afc0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc160ca0 con 0x7f3afc0b48c0 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f3aec001670 con 0x7f3afc0b48c0 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 >> v1:172.21.15.5:6789/0 conn(0x7f3afc0b48c0 legacy=0x7f3afc15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00039b0 0x7f3ae0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ae0004800 0x7f3ae0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae00051c0 0x7f3ae000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc161ff0 con 0x7f3ae00051c0 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae000c160 con 0x7f3ae0004800 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae000c360 con 0x7f3ae00039b0 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.041+0000 7f3afbe00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae00051c0 0x7f3ae000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3af2a00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ae0004800 0x7f3ae0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3afb400640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00039b0 0x7f3ae0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3afbe00640 1 -- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00039b0 msgr2=0x7f3ae0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3afbe00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00039b0 0x7f3ae0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3afbe00640 1 -- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ae0004800 msgr2=0x7f3ae0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3afbe00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ae0004800 0x7f3ae0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.042+0000 7f3afbe00640 1 -- 172.21.15.5:0/3823023502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0003520 con 0x7f3ae00051c0 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.043+0000 7f3afbe00640 1 -- 172.21.15.5:0/3823023502 reap_dead start 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.043+0000 7f3afbe00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae00051c0 0x7f3ae000b9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001e50 tx=0x7f3ae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1aa0502bbef8fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.043+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4001a70 con 0x7f3ae00051c0 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.043+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af4003970 con 0x7f3ae00051c0 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.043+0000 7f3afaa00640 1 -- 172.21.15.5:0/3823023502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af4003d60 con 0x7f3ae00051c0 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 -- 172.21.15.5:0/3823023502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae00051c0 msgr2=0x7f3ae000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae00051c0 0x7f3ae000b9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001e50 tx=0x7f3ae0005830 comp rx=0 tx=0).stop 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 -- 172.21.15.5:0/3823023502 shutdown_connections 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 --2- 172.21.15.5:0/3823023502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae00051c0 0x7f3ae000b9c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 -- 172.21.15.5:0/3823023502 >> 172.21.15.5:0/3823023502 conn(0x7f3afc08af80 msgr2=0x7f3afc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 -- 172.21.15.5:0/3823023502 shutdown_connections 2024-04-25T10:04:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.044+0000 7f3b00c00640 1 -- 172.21.15.5:0/3823023502 wait complete. 2024-04-25T10:04:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.046+0000 7f3b00c00640 1 Processor -- start 2024-04-25T10:04:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.046+0000 7f3b00c00640 1 -- start start 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3b00c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00051c0 0x7f3afc0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3b00c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 0x7f3afc0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3b00c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3afc0d13e0 0x7f3afc0d1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3b00c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc0b4ca0 con 0x7f3afc0ca740 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3b00c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc161260 con 0x7f3afc0d13e0 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3b00c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3afc161680 con 0x7f3ae00051c0 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3af2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 0x7f3afc0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3af2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 0x7f3afc0cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46854/0 (socket says 172.21.15.5:46854) 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.047+0000 7f3af2a00640 1 -- 172.21.15.5:0/1269737597 learned_addr learned my addr 172.21.15.5:0/1269737597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3afb400640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00051c0 0x7f3afc0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3afbe00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3afc0d13e0 0x7f3afc0d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af2a00640 1 -- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00051c0 msgr2=0x7f3afc0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af2a00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00051c0 0x7f3afc0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af2a00640 1 -- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3afc0d13e0 msgr2=0x7f3afc0d1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af2a00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3afc0d13e0 0x7f3afc0d1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af2a00640 1 -- 172.21.15.5:0/1269737597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc0c9f80 con 0x7f3afc0ca740 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3afbe00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3afc0d13e0 0x7f3afc0d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af2a00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 0x7f3afc0cb5c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4011570 tx=0x7f3ae4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4015870 con 0x7f3afc0ca740 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae4015a10 con 0x7f3afc0ca740 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4015dc0 con 0x7f3afc0ca740 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.048+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3afc0cbef0 con 0x7f3afc0ca740 2024-04-25T10:04:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.049+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3afc0cc4b0 con 0x7f3afc0ca740 2024-04-25T10:04:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.050+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3afc068f60 con 0x7f3afc0ca740 2024-04-25T10:04:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.051+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ae400f460 con 0x7f3afc0ca740 2024-04-25T10:04:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.052+0000 7f3af3e00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ad806b500 0x7f3ad806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.052+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3ae408a9b0 con 0x7f3afc0ca740 2024-04-25T10:04:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.053+0000 7f3afb400640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ad806b500 0x7f3ad806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.054+0000 7f3afb400640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ad806b500 0x7f3ad806d9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3aec001eb0 tx=0x7f3aec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.057+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ae4019030 con 0x7f3afc0ca740 2024-04-25T10:04:17.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.257+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3afc0d0dd0 con 0x7f3afc0ca740 2024-04-25T10:04:17.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.258+0000 7f3af3e00640 1 -- 172.21.15.5:0/1269737597 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ae4059990 con 0x7f3afc0ca740 2024-04-25T10:04:17.263 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:04:17.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ad806b500 msgr2=0x7f3ad806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ad806b500 0x7f3ad806d9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3aec001eb0 tx=0x7f3aec009040 comp rx=0 tx=0).stop 2024-04-25T10:04:17.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 msgr2=0x7f3afc0cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 0x7f3afc0cb5c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4011570 tx=0x7f3ae4011a30 comp rx=0 tx=0).stop 2024-04-25T10:04:17.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 shutdown_connections 2024-04-25T10:04:17.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ad806b500 0x7f3ad806d9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3afc0d13e0 0x7f3afc0d1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3afc0ca740 0x7f3afc0cb5c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 --2- 172.21.15.5:0/1269737597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ae00051c0 0x7f3afc0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 >> 172.21.15.5:0/1269737597 conn(0x7f3afc08af80 msgr2=0x7f3afc06d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 shutdown_connections 2024-04-25T10:04:17.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.261+0000 7f3b00c00640 1 -- 172.21.15.5:0/1269737597 wait complete. 2024-04-25T10:04:17.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:04:17.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.499+0000 7fc70c000640 1 Processor -- start 2024-04-25T10:04:17.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.499+0000 7fc70c000640 1 -- start start 2024-04-25T10:04:17.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.500+0000 7fc70c000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc7041529e0 0x7fc7040c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.500+0000 7fc70c000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc7040c6410 0x7fc7040ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.500+0000 7fc70c000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70415bdc0 con 0x7fc704071ed0 2024-04-25T10:04:17.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.500+0000 7fc70c000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040b2340 con 0x7fc7040c6410 2024-04-25T10:04:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.500+0000 7fc70c000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70415c060 con 0x7fc7041529e0 2024-04-25T10:04:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc7040c6410 0x7fc7040ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc7040c6410 0x7fc7040ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46864/0 (socket says 172.21.15.5:46864) 2024-04-25T10:04:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 -- 172.21.15.5:0/120983468 learned_addr learned my addr 172.21.15.5:0/120983468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc709600640 1 --2- 172.21.15.5:0/120983468 >> v2:172.21.15.80:3300/0 conn(0x7fc7041529e0 0x7fc7040c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 -- 172.21.15.5:0/120983468 >> v2:172.21.15.80:3300/0 conn(0x7fc7041529e0 msgr2=0x7fc7040c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 --2- 172.21.15.5:0/120983468 >> v2:172.21.15.80:3300/0 conn(0x7fc7041529e0 0x7fc7040c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 -- 172.21.15.5:0/120983468 >> v1:172.21.15.80:6789/0 conn(0x7fc704071ed0 legacy=0x7fc7040c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc708c00640 1 -- 172.21.15.5:0/120983468 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70415c340 con 0x7fc7040c6410 2024-04-25T10:04:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.501+0000 7fc709600640 1 --2- 172.21.15.5:0/120983468 >> v2:172.21.15.80:3300/0 conn(0x7fc7041529e0 0x7fc7040c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.502+0000 7fc708c00640 1 --2- 172.21.15.5:0/120983468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc7040ce350 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc704071d60 tx=0x7fc6f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9db74f3a6880bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.503+0000 7fc703e00640 1 -- 172.21.15.5:0/120983468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8037070 con 0x7fc7040c6410 2024-04-25T10:04:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.503+0000 7fc703e00640 1 -- 172.21.15.5:0/120983468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8032440 con 0x7fc7040c6410 2024-04-25T10:04:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.503+0000 7fc703e00640 1 -- 172.21.15.5:0/120983468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8032810 con 0x7fc7040c6410 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.503+0000 7fc70c000640 1 -- 172.21.15.5:0/120983468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 msgr2=0x7fc7040ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.503+0000 7fc70c000640 1 --2- 172.21.15.5:0/120983468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc7040ce350 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc704071d60 tx=0x7fc6f802a230 comp rx=0 tx=0).stop 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.504+0000 7fc70c000640 1 -- 172.21.15.5:0/120983468 shutdown_connections 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.504+0000 7fc70c000640 1 --2- 172.21.15.5:0/120983468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc7040ce350 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.504+0000 7fc70c000640 1 --2- 172.21.15.5:0/120983468 >> v2:172.21.15.80:3300/0 conn(0x7fc7041529e0 0x7fc7040c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.504+0000 7fc70c000640 1 -- 172.21.15.5:0/120983468 >> 172.21.15.5:0/120983468 conn(0x7fc704071150 msgr2=0x7fc704071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.504+0000 7fc70c000640 1 -- 172.21.15.5:0/120983468 shutdown_connections 2024-04-25T10:04:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.504+0000 7fc70c000640 1 -- 172.21.15.5:0/120983468 wait complete. 2024-04-25T10:04:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.506+0000 7fc70c000640 1 Processor -- start 2024-04-25T10:04:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.506+0000 7fc70c000640 1 -- start start 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc70c000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc704071ed0 0x7fc70414d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc70c000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc704155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc70c000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc70414da40 0x7fc7041557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc70c000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70406e160 con 0x7fc7040c6410 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc70c000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70415c900 con 0x7fc704071ed0 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc70c000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70406e2e0 con 0x7fc70414da40 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc704155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc704155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46876/0 (socket says 172.21.15.5:46876) 2024-04-25T10:04:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 -- 172.21.15.5:0/3307953505 learned_addr learned my addr 172.21.15.5:0/3307953505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc709600640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc704071ed0 0x7fc70414d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 -- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc70414da40 msgr2=0x7fc7041557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc70414da40 0x7fc7041557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 -- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc704071ed0 msgr2=0x7fc70414d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc704071ed0 0x7fc70414d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.507+0000 7fc708c00640 1 -- 172.21.15.5:0/3307953505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc704153a30 con 0x7fc7040c6410 2024-04-25T10:04:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.508+0000 7fc708c00640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc704155290 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc6f802a270 tx=0x7fc6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.508+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8037040 con 0x7fc7040c6410 2024-04-25T10:04:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.508+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8042070 con 0x7fc7040c6410 2024-04-25T10:04:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.508+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc704153d10 con 0x7fc7040c6410 2024-04-25T10:04:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.508+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8030bb0 con 0x7fc7040c6410 2024-04-25T10:04:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.508+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7041542d0 con 0x7fc7040c6410 2024-04-25T10:04:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.510+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7041546b0 con 0x7fc7040c6410 2024-04-25T10:04:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.511+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6f8047460 con 0x7fc7040c6410 2024-04-25T10:04:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.513+0000 7fc701600640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6e806b4d0 0x7fc6e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.513+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc6f80aad20 con 0x7fc7040c6410 2024-04-25T10:04:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.513+0000 7fc709600640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6e806b4d0 0x7fc6e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.515+0000 7fc709600640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6e806b4d0 0x7fc6e806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400a990 tx=0x7fc6f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.517+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7041546b0 con 0x7fc7040c6410 2024-04-25T10:04:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:17.790+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fc704068fd0 con 0x7fc7040c6410 2024-04-25T10:04:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.360+0000 7fc701600640 1 -- 172.21.15.5:0/3307953505 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v152) v1 ==== 152+0+0 (secure 0 0 0) 0x7fc6f80af070 con 0x7fc7040c6410 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6e806b4d0 msgr2=0x7fc6e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6e806b4d0 0x7fc6e806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400a990 tx=0x7fc6f4008040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 msgr2=0x7fc704155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc704155290 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc6f802a270 tx=0x7fc6f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 shutdown_connections 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc70414da40 0x7fc7041557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7040c6410 0x7fc704155290 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc6e806b4d0 0x7fc6e806d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 --2- 172.21.15.5:0/3307953505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc704071ed0 0x7fc70414d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.364+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 >> 172.21.15.5:0/3307953505 conn(0x7fc704071150 msgr2=0x7fc7040cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.365+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 shutdown_connections 2024-04-25T10:04:18.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.365+0000 7fc70c000640 1 -- 172.21.15.5:0/3307953505 wait complete. 2024-04-25T10:04:18.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:04:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.601+0000 7fb0dfe00640 1 Processor -- start 2024-04-25T10:04:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.601+0000 7fb0dfe00640 1 -- start start 2024-04-25T10:04:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.602+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 0x7fb0e00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.602+0000 7fb0dfe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0e015ce20 0x7fb0e015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.602+0000 7fb0dfe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e0144cd0 con 0x7fb0e015bc30 2024-04-25T10:04:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0dfe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e015f460 con 0x7fb0e015ce20 2024-04-25T10:04:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015c440 con 0x7fb0e01448f0 2024-04-25T10:04:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0df400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0e015ce20 0x7fb0e015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 0x7fb0e00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 0x7fb0e00cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:50340/0 (socket says 172.21.15.5:50340) 2024-04-25T10:04:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0df400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0e015ce20 0x7fb0e015d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:37286/0 (socket says 172.21.15.5:37286) 2024-04-25T10:04:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0dea00640 1 -- 172.21.15.5:0/1360503550 learned_addr learned my addr 172.21.15.5:0/1360503550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.603+0000 7fb0de000640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb0e015bc30 0x7fb0e015f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:36492/0 (socket says 172.21.15.5:36492) 2024-04-25T10:04:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.604+0000 7fb0df400640 1 -- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 msgr2=0x7fb0e00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.604+0000 7fb0df400640 1 --2- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 0x7fb0e00cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.604+0000 7fb0dd600640 1 -- 172.21.15.5:0/1360503550 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950442524 0 0) 0x7fb0e0144cd0 con 0x7fb0e015bc30 2024-04-25T10:04:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.604+0000 7fb0df400640 1 -- 172.21.15.5:0/1360503550 >> v1:172.21.15.80:6789/0 conn(0x7fb0e015bc30 legacy=0x7fb0e015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.604+0000 7fb0df400640 1 -- 172.21.15.5:0/1360503550 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e015c720 con 0x7fb0e015ce20 2024-04-25T10:04:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.604+0000 7fb0dea00640 1 --2- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 0x7fb0e00cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.605+0000 7fb0df400640 1 --2- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015ce20 0x7fb0e015d2b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80097a0 tx=0x7fb0d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c4f4fdac4e46b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.606+0000 7fb0dd600640 1 -- 172.21.15.5:0/1360503550 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8037070 con 0x7fb0e015ce20 2024-04-25T10:04:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.606+0000 7fb0dd600640 1 -- 172.21.15.5:0/1360503550 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d8032440 con 0x7fb0e015ce20 2024-04-25T10:04:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.606+0000 7fb0dd600640 1 -- 172.21.15.5:0/1360503550 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8032810 con 0x7fb0e015ce20 2024-04-25T10:04:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.606+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015ce20 msgr2=0x7fb0e015d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.606+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015ce20 0x7fb0e015d2b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80097a0 tx=0x7fb0d802a180 comp rx=0 tx=0).stop 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.607+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1360503550 shutdown_connections 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.607+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015ce20 0x7fb0e015d2b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.607+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1360503550 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e01448f0 0x7fb0e00cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.607+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1360503550 >> 172.21.15.5:0/1360503550 conn(0x7fb0e0066f20 msgr2=0x7fb0e006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.607+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1360503550 shutdown_connections 2024-04-25T10:04:18.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.607+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1360503550 wait complete. 2024-04-25T10:04:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.608+0000 7fb0dfe00640 1 Processor -- start 2024-04-25T10:04:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.609+0000 7fb0dfe00640 1 -- start start 2024-04-25T10:04:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.609+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 0x7fb0e01e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.609+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015bc30 0x7fb0e01e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e015ce20 0x7fb0e01f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015c090 con 0x7fb0e01448f0 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0160000 con 0x7fb0e015bc30 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dfe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e01e6490 con 0x7fb0e015ce20 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 0x7fb0e01e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 0x7fb0e01e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46886/0 (socket says 172.21.15.5:46886) 2024-04-25T10:04:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 -- 172.21.15.5:0/1576308530 learned_addr learned my addr 172.21.15.5:0/1576308530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 -- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e015ce20 msgr2=0x7fb0e01f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0de000640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015bc30 0x7fb0e01e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0df400640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e015ce20 0x7fb0e01f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e015ce20 0x7fb0e01f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 -- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015bc30 msgr2=0x7fb0e01e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015bc30 0x7fb0e01e5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.610+0000 7fb0dea00640 1 -- 172.21.15.5:0/1576308530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e01f33f0 con 0x7fb0e01448f0 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.611+0000 7fb0dea00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 0x7fb0e01e5a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000efc0 tx=0x7fb0d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.611+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0014480 con 0x7fb0e01448f0 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.611+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d0019070 con 0x7fb0e01448f0 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.611+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e01e6730 con 0x7fb0e01448f0 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.611+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0014820 con 0x7fb0e01448f0 2024-04-25T10:04:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.611+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e01f3990 con 0x7fb0e01448f0 2024-04-25T10:04:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.612+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0068f60 con 0x7fb0e01448f0 2024-04-25T10:04:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.613+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0d0005d10 con 0x7fb0e01448f0 2024-04-25T10:04:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.614+0000 7fb0d6a00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0c006b4d0 0x7fb0c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.614+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb0d008a6f0 con 0x7fb0e01448f0 2024-04-25T10:04:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.615+0000 7fb0de000640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0c006b4d0 0x7fb0c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.616+0000 7fb0de000640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0c006b4d0 0x7fb0c006d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007fd0 tx=0x7fb0cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.619+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0e01e6730 con 0x7fb0e01448f0 2024-04-25T10:04:18.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.812+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb0e015b880 con 0x7fb0e01448f0 2024-04-25T10:04:18.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.819+0000 7fb0d6a00640 1 -- 172.21.15.5:0/1576308530 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb0d0010040 con 0x7fb0e01448f0 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0c006b4d0 msgr2=0x7fb0c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0c006b4d0 0x7fb0c006d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc007fd0 tx=0x7fb0cc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 msgr2=0x7fb0e01e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 0x7fb0e01e5a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000efc0 tx=0x7fb0d000c490 comp rx=0 tx=0).stop 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 shutdown_connections 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e015ce20 0x7fb0e01f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0c006b4d0 0x7fb0c006d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e015bc30 0x7fb0e01e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 --2- 172.21.15.5:0/1576308530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e01448f0 0x7fb0e01e5a10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.822+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 >> 172.21.15.5:0/1576308530 conn(0x7fb0e0066f20 msgr2=0x7fb0e00b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.823+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 shutdown_connections 2024-04-25T10:04:18.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:18.823+0000 7fb0dfe00640 1 -- 172.21.15.5:0/1576308530 wait complete. 2024-04-25T10:04:18.846 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:18.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:18.953 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:19.082 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:19.083 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:04:19.083 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:19.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:19.144 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:04:19.144 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:19.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:19.200 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:04:19.200 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:19.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:19.259 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:04:19.259 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:19.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:19.316 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:04:19.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:19.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.534+0000 7fbe9f000640 1 Processor -- start 2024-04-25T10:04:19.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.534+0000 7fbe9f000640 1 -- start start 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.536+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 0x7fbe9806e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.536+0000 7fbe9f000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe980b16e0 0x7fbe980ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.536+0000 7fbe9f000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe98165af0 con 0x7fbe980cd240 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.536+0000 7fbe9f000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9815c760 con 0x7fbe980b16e0 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.536+0000 7fbe9f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981658a0 con 0x7fbe9815c380 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.537+0000 7fbe9d000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbe980cd240 0x7fbe9806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44724/0 (socket says 172.21.15.5:44724) 2024-04-25T10:04:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.537+0000 7fbe9d000640 1 -- 172.21.15.5:0/1557219942 learned_addr learned my addr 172.21.15.5:0/1557219942 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.537+0000 7fbe97e00640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 0x7fbe9806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.537+0000 7fbe8fe00640 1 --2- 172.21.15.5:0/1557219942 >> v2:172.21.15.80:3300/0 conn(0x7fbe980b16e0 0x7fbe980ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.537+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589857474 0 0) 0x7fbe98165af0 con 0x7fbe980cd240 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.537+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe7c003520 con 0x7fbe980cd240 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3913696398 0 0) 0x7fbe7c003520 con 0x7fbe980cd240 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 msgr2=0x7fbe9806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 0x7fbe9806e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 >> v2:172.21.15.80:3300/0 conn(0x7fbe980b16e0 msgr2=0x7fbe980ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 --2- 172.21.15.5:0/1557219942 >> v2:172.21.15.80:3300/0 conn(0x7fbe980b16e0 0x7fbe980ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98165dd0 con 0x7fbe980cd240 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fbe900019f0 con 0x7fbe980cd240 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 >> v1:172.21.15.5:6789/0 conn(0x7fbe980cd240 legacy=0x7fbe9806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 0x7fbe7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c003f30 0x7fbe7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe7c004d40 0x7fbe7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9815c760 con 0x7fbe7c004d40 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981658a0 con 0x7fbe7c003f30 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c00baa0 con 0x7fbe9815c380 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe9d000640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe7c004d40 0x7fbe7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe8fe00640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c003f30 0x7fbe7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.538+0000 7fbe97e00640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 0x7fbe7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.539+0000 7fbe9d000640 1 -- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 msgr2=0x7fbe7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.539+0000 7fbe9d000640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe9815c380 0x7fbe7c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.539+0000 7fbe9d000640 1 -- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c003f30 msgr2=0x7fbe7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.539+0000 7fbe9d000640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c003f30 0x7fbe7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.539+0000 7fbe9d000640 1 -- 172.21.15.5:0/1557219942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c003520 con 0x7fbe7c004d40 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.540+0000 7fbe9d000640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe7c004d40 0x7fbe7c0051c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe900013d0 tx=0x7fbe7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858ca720143003c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.540+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000a4e0 con 0x7fbe7c004d40 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.540+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9002f070 con 0x7fbe7c004d40 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.540+0000 7fbe97400640 1 -- 172.21.15.5:0/1557219942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9002bce0 con 0x7fbe7c004d40 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9f000640 1 -- 172.21.15.5:0/1557219942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe7c004d40 msgr2=0x7fbe7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9f000640 1 --2- 172.21.15.5:0/1557219942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe7c004d40 0x7fbe7c0051c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe900013d0 tx=0x7fbe7c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9d000640 1 -- 172.21.15.5:0/1557219942 reap_dead start 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9f000640 1 -- 172.21.15.5:0/1557219942 shutdown_connections 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9f000640 1 -- 172.21.15.5:0/1557219942 >> 172.21.15.5:0/1557219942 conn(0x7fbe98071150 msgr2=0x7fbe98071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9f000640 1 -- 172.21.15.5:0/1557219942 shutdown_connections 2024-04-25T10:04:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.541+0000 7fbe9f000640 1 -- 172.21.15.5:0/1557219942 wait complete. 2024-04-25T10:04:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.543+0000 7fbe9f000640 1 Processor -- start 2024-04-25T10:04:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.543+0000 7fbe9f000640 1 -- start start 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c004d40 0x7fbe9815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe98150e10 0x7fbe981f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 0x7fbe98153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981632c0 con 0x7fbe98152dc0 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9f000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98166390 con 0x7fbe7c004d40 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98163440 con 0x7fbe98150e10 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 0x7fbe98153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 0x7fbe98153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46906/0 (socket says 172.21.15.5:46906) 2024-04-25T10:04:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.544+0000 7fbe9d000640 1 -- 172.21.15.5:0/532222974 learned_addr learned my addr 172.21.15.5:0/532222974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe97e00640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c004d40 0x7fbe9815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe8fe00640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe98150e10 0x7fbe981f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe9d000640 1 -- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe98150e10 msgr2=0x7fbe981f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe9d000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe98150e10 0x7fbe981f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe9d000640 1 -- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c004d40 msgr2=0x7fbe9815aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe9d000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c004d40 0x7fbe9815aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe9d000640 1 -- 172.21.15.5:0/532222974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe98152470 con 0x7fbe98152dc0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe9d000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 0x7fbe98153240 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe90001cf0 tx=0x7fbe90001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90002040 con 0x7fbe98152dc0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9000a4c0 con 0x7fbe98152dc0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.545+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004d60 con 0x7fbe98152dc0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.546+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe981f8940 con 0x7fbe98152dc0 2024-04-25T10:04:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.546+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe981516b0 con 0x7fbe98152dc0 2024-04-25T10:04:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.547+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe98068fd0 con 0x7fbe98152dc0 2024-04-25T10:04:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.548+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe90034470 con 0x7fbe98152dc0 2024-04-25T10:04:19.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.549+0000 7fbe94c00640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe7406b4d0 0x7fbe7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.549+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbe900a9f80 con 0x7fbe98152dc0 2024-04-25T10:04:19.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.549+0000 7fbe97e00640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe7406b4d0 0x7fbe7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.551+0000 7fbe97e00640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe7406b4d0 0x7fbe7406d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009a10 tx=0x7fbe88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.553+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe90047030 con 0x7fbe98152dc0 2024-04-25T10:04:19.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.798+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe980cdd50 con 0x7fbe98152dc0 2024-04-25T10:04:19.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.799+0000 7fbe94c00640 1 -- 172.21.15.5:0/532222974 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3693 (secure 0 0 0) 0x7fbe90078ed0 con 0x7fbe98152dc0 2024-04-25T10:04:19.804 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:19.804 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4739,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2740282915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2740282915},{"type":"v1","addr":"172.21.15.5:6833","nonce":2740282915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4032305346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4032305346},{"type":"v1","addr":"172.21.15.5:6835","nonce":4032305346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5002,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/2664405407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2664405407},{"type":"v1","addr":"172.21.15.80:6836","nonce":2664405407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2399786211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2399786211},{"type":"v1","addr":"172.21.15.80:6837","nonce":2399786211}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T10:04:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 19 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.802+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe7406b4d0 msgr2=0x7fbe7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.802+0000 7fbe9f000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe7406b4d0 0x7fbe7406d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009a10 tx=0x7fbe88008040 comp rx=0 tx=0).stop 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.802+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 msgr2=0x7fbe98153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.802+0000 7fbe9f000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 0x7fbe98153240 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe90001cf0 tx=0x7fbe90001dc0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 shutdown_connections 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe98152dc0 0x7fbe98153240 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe98150e10 0x7fbe981f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe7406b4d0 0x7fbe7406d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 --2- 172.21.15.5:0/532222974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe7c004d40 0x7fbe9815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 >> 172.21.15.5:0/532222974 conn(0x7fbe98071150 msgr2=0x7fbe980b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 shutdown_connections 2024-04-25T10:04:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:19.803+0000 7fbe9f000640 1 -- 172.21.15.5:0/532222974 wait complete. 2024-04-25T10:04:19.827 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:04:19.827 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:04:19.827 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:04:19.827 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:19.827 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:04:19.828 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:04:19.828 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:04:19.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:19.828 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:04:19.828 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:04:19.828 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:04:19.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:19.829 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:04:19.829 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:04:19.829 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:04:19.829 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:19.845 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:04:19.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:04:19.847 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:04:19.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:04:19.851 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:04:19.851 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:04:19.861 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:04:19.861 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:04:19.894 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:04:19.894 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:19.900 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:04:19.900 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:19.908 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:04:19.909 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:19.913 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:04:19.914 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:19.914 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:04:20.049 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:04:20.049 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:04:20.062 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:04:20.063 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:04:20.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.096+0000 7f140c200640 1 Processor -- start 2024-04-25T10:04:20.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.096+0000 7f140c200640 1 -- start start 2024-04-25T10:04:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.097+0000 7f140c200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1404071ed0 0x7f14040ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.097+0000 7f140c200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140415bdc0 con 0x7f14041529e0 2024-04-25T10:04:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.097+0000 7f140c200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140415c060 con 0x7f14040ce890 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.097+0000 7f140c200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14040b2340 con 0x7f1404071ed0 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1409800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f14041529e0 0x7f14040c5ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44756/0 (socket says 172.21.15.5:44756) 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1409800640 1 -- 172.21.15.5:0/2465831745 learned_addr learned my addr 172.21.15.5:0/2465831745 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1408e00640 1 --2- 172.21.15.5:0/2465831745 >> v2:172.21.15.80:3300/0 conn(0x7f1404071ed0 0x7f14040ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1408e00640 1 -- 172.21.15.5:0/2465831745 >> v1:172.21.15.80:6789/0 conn(0x7f14040ce890 legacy=0x7f140406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1408e00640 1 -- 172.21.15.5:0/2465831745 >> v1:172.21.15.5:6789/0 conn(0x7f14041529e0 legacy=0x7f14040c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1403e00640 1 -- 172.21.15.5:0/2465831745 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982573570 0 0) 0x7f140415c060 con 0x7f14040ce890 2024-04-25T10:04:20.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.098+0000 7f1408e00640 1 -- 172.21.15.5:0/2465831745 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140415c340 con 0x7f1404071ed0 2024-04-25T10:04:20.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.100+0000 7f1408e00640 1 --2- 172.21.15.5:0/2465831745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1404071ed0 0x7f14040ce350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f13f80097a0 tx=0x7f13f802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f9714d5d157e6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.100+0000 7f1403e00640 1 -- 172.21.15.5:0/2465831745 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8037070 con 0x7f1404071ed0 2024-04-25T10:04:20.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.100+0000 7f1403e00640 1 -- 172.21.15.5:0/2465831745 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f802adc0 con 0x7f1404071ed0 2024-04-25T10:04:20.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.100+0000 7f1403e00640 1 -- 172.21.15.5:0/2465831745 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8032660 con 0x7f1404071ed0 2024-04-25T10:04:20.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.100+0000 7f140c200640 1 -- 172.21.15.5:0/2465831745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1404071ed0 msgr2=0x7f14040ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.100+0000 7f140c200640 1 --2- 172.21.15.5:0/2465831745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1404071ed0 0x7f14040ce350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f13f80097a0 tx=0x7f13f802a130 comp rx=0 tx=0).stop 2024-04-25T10:04:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.101+0000 7f140c200640 1 -- 172.21.15.5:0/2465831745 shutdown_connections 2024-04-25T10:04:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.101+0000 7f140c200640 1 --2- 172.21.15.5:0/2465831745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1404071ed0 0x7f14040ce350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.101+0000 7f140c200640 1 -- 172.21.15.5:0/2465831745 >> 172.21.15.5:0/2465831745 conn(0x7f1404071150 msgr2=0x7f1404071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.101+0000 7f140c200640 1 -- 172.21.15.5:0/2465831745 shutdown_connections 2024-04-25T10:04:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.101+0000 7f140c200640 1 -- 172.21.15.5:0/2465831745 wait complete. 2024-04-25T10:04:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.102+0000 7f140c200640 1 Processor -- start 2024-04-25T10:04:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.103+0000 7f140c200640 1 -- start start 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140c200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 0x7f140413ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140c200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14040ce890 0x7f1404152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14040ca0c0 0x7f14041598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140415c900 con 0x7f14040ca0c0 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140c200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14040cecc0 con 0x7f14040ce890 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140c200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14040c65e0 con 0x7f1404071ed0 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f140a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14040ca0c0 0x7f14041598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f1409800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 0x7f140413ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f1409800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 0x7f140413ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:50420/0 (socket says 172.21.15.5:50420) 2024-04-25T10:04:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f1409800640 1 -- 172.21.15.5:0/3395153411 learned_addr learned my addr 172.21.15.5:0/3395153411 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:20.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.104+0000 7f1408e00640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14040ce890 0x7f1404152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1409800640 1 -- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14040ce890 msgr2=0x7f1404152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1409800640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14040ce890 0x7f1404152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1409800640 1 -- 172.21.15.5:0/3395153411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14040ca0c0 msgr2=0x7f14041598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1409800640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14040ca0c0 0x7f14041598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1409800640 1 -- 172.21.15.5:0/3395153411 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404140470 con 0x7f1404071ed0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1409800640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 0x7f140413ff30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c750 tx=0x7f13f400cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017070 con 0x7f1404071ed0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f4005a70 con 0x7f1404071ed0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4005e40 con 0x7f1404071ed0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14040cc5b0 con 0x7f1404071ed0 2024-04-25T10:04:20.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.105+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14040ccb70 con 0x7f1404071ed0 2024-04-25T10:04:20.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.106+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1404068fd0 con 0x7f1404071ed0 2024-04-25T10:04:20.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.108+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13f401a070 con 0x7f1404071ed0 2024-04-25T10:04:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.109+0000 7f1401600640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13e806b500 0x7f13e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.109+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f13f4089500 con 0x7f1404071ed0 2024-04-25T10:04:20.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.109+0000 7f1408e00640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13e806b500 0x7f13e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.110+0000 7f1408e00640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13e806b500 0x7f13e806d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f13f8034000 tx=0x7f13f802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.116+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1404140470 con 0x7f1404071ed0 2024-04-25T10:04:20.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.327+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1404159180 con 0x7f1404071ed0 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.396+0000 7f1401600640 1 -- 172.21.15.5:0/3395153411 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f13f4057e90 con 0x7f1404071ed0 2024-04-25T10:04:20.402 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:04:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13e806b500 msgr2=0x7f13e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13e806b500 0x7f13e806d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f13f8034000 tx=0x7f13f802e000 comp rx=0 tx=0).stop 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 msgr2=0x7f140413ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 0x7f140413ff30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c750 tx=0x7f13f400cc10 comp rx=0 tx=0).stop 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 shutdown_connections 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14040ca0c0 0x7f14041598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f13e806b500 0x7f13e806d9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14040ce890 0x7f1404152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 --2- 172.21.15.5:0/3395153411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1404071ed0 0x7f140413ff30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.399+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 >> 172.21.15.5:0/3395153411 conn(0x7f1404071150 msgr2=0x7f14040c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.400+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 shutdown_connections 2024-04-25T10:04:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.400+0000 7f140c200640 1 -- 172.21.15.5:0/3395153411 wait complete. 2024-04-25T10:04:20.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:04:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.616+0000 7f44f7200640 1 Processor -- start 2024-04-25T10:04:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.616+0000 7f44f7200640 1 -- start start 2024-04-25T10:04:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.618+0000 7f44f7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44f80cc0d0 0x7f44f80ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.618+0000 7f44f7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc560 0x7f44f80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.618+0000 7f44f7200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f815aee0 con 0x7f44f8143e50 2024-04-25T10:04:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.618+0000 7f44f7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f8144230 con 0x7f44f80cc0d0 2024-04-25T10:04:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.618+0000 7f44f7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815b1c0 con 0x7f44f80cc560 2024-04-25T10:04:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.619+0000 7f44ed400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44f80cc0d0 0x7f44f80ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.619+0000 7f44ed400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44f80cc0d0 0x7f44f80ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:37344/0 (socket says 172.21.15.5:37344) 2024-04-25T10:04:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.619+0000 7f44f5e00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f44f8143e50 0x7f44f815bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:36530/0 (socket says 172.21.15.5:36530) 2024-04-25T10:04:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.619+0000 7f44ed400640 1 -- 172.21.15.5:0/1547862899 learned_addr learned my addr 172.21.15.5:0/1547862899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.619+0000 7f44f6800640 1 --2- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc560 0x7f44f80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.620+0000 7f44ed400640 1 -- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc560 msgr2=0x7f44f80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.620+0000 7f44ed400640 1 --2- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc560 0x7f44f80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.620+0000 7f44ed400640 1 -- 172.21.15.5:0/1547862899 >> v1:172.21.15.80:6789/0 conn(0x7f44f8143e50 legacy=0x7f44f815bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.620+0000 7f44ed400640 1 -- 172.21.15.5:0/1547862899 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f815b4a0 con 0x7f44f80cc0d0 2024-04-25T10:04:20.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.621+0000 7f44ed400640 1 --2- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc0d0 0x7f44f80ce730 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44e000bb10 tx=0x7f44e002c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=568c9bf8f876e05a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.622+0000 7f44f5400640 1 -- 172.21.15.5:0/1547862899 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0032790 con 0x7f44f80cc0d0 2024-04-25T10:04:20.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.622+0000 7f44f5400640 1 -- 172.21.15.5:0/1547862899 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0038070 con 0x7f44f80cc0d0 2024-04-25T10:04:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.622+0000 7f44f5400640 1 -- 172.21.15.5:0/1547862899 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0032b50 con 0x7f44f80cc0d0 2024-04-25T10:04:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 -- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc0d0 msgr2=0x7f44f80ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 --2- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc0d0 0x7f44f80ce730 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f44e000bb10 tx=0x7f44e002c890 comp rx=0 tx=0).stop 2024-04-25T10:04:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 -- 172.21.15.5:0/1547862899 shutdown_connections 2024-04-25T10:04:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 --2- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc560 0x7f44f80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 --2- 172.21.15.5:0/1547862899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc0d0 0x7f44f80ce730 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 -- 172.21.15.5:0/1547862899 >> 172.21.15.5:0/1547862899 conn(0x7f44f80b4f00 msgr2=0x7f44f806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 -- 172.21.15.5:0/1547862899 shutdown_connections 2024-04-25T10:04:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.623+0000 7f44f7200640 1 -- 172.21.15.5:0/1547862899 wait complete. 2024-04-25T10:04:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.625+0000 7f44f7200640 1 Processor -- start 2024-04-25T10:04:20.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.625+0000 7f44f7200640 1 -- start start 2024-04-25T10:04:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc0d0 0x7f44f81f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc560 0x7f44f80b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 0x7f44f8139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815ac30 con 0x7f44f8143e50 2024-04-25T10:04:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815aab0 con 0x7f44f80cc560 2024-04-25T10:04:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f815e3d0 con 0x7f44f80cc0d0 2024-04-25T10:04:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 0x7f44f8139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 0x7f44f8139b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46954/0 (socket says 172.21.15.5:46954) 2024-04-25T10:04:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f6800640 1 -- 172.21.15.5:0/527888444 learned_addr learned my addr 172.21.15.5:0/527888444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.627+0000 7f44f5e00640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc0d0 0x7f44f81f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44ed400640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc560 0x7f44f80b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44f6800640 1 -- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc0d0 msgr2=0x7f44f81f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44f6800640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc0d0 0x7f44f81f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44f6800640 1 -- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc560 msgr2=0x7f44f80b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44f6800640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc560 0x7f44f80b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44f6800640 1 -- 172.21.15.5:0/527888444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f8138ac0 con 0x7f44f8143e50 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44f6800640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 0x7f44f8139b60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f44f000efc0 tx=0x7f44f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.628+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0018020 con 0x7f44f8143e50 2024-04-25T10:04:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.629+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f0014480 con 0x7f44f8143e50 2024-04-25T10:04:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.629+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0014870 con 0x7f44f8143e50 2024-04-25T10:04:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.629+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f8138da0 con 0x7f44f8143e50 2024-04-25T10:04:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.629+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f8139360 con 0x7f44f8143e50 2024-04-25T10:04:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.630+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f8068f60 con 0x7f44f8143e50 2024-04-25T10:04:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.631+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44f0005d10 con 0x7f44f8143e50 2024-04-25T10:04:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.632+0000 7f44eea00640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44d406b4d0 0x7f44d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.632+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f44f008aae0 con 0x7f44f8143e50 2024-04-25T10:04:20.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.633+0000 7f44f5e00640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44d406b4d0 0x7f44d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.634+0000 7f44f5e00640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44d406b4d0 0x7f44d406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f44e8000f60 tx=0x7f44e8007c80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.636+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44f0022070 con 0x7f44f8143e50 2024-04-25T10:04:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.897+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f44f80ce2e0 con 0x7f44f8143e50 2024-04-25T10:04:20.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.898+0000 7f44eea00640 1 -- 172.21.15.5:0/527888444 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f44f0010040 con 0x7f44f8143e50 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:04:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:04:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:04:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:04:20.917 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44d406b4d0 msgr2=0x7f44d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44d406b4d0 0x7f44d406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f44e8000f60 tx=0x7f44e8007c80 comp rx=0 tx=0).stop 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 msgr2=0x7f44f8139b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 0x7f44f8139b60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f44f000efc0 tx=0x7f44f000c490 comp rx=0 tx=0).stop 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 shutdown_connections 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44f8143e50 0x7f44f8139b60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44d406b4d0 0x7f44d406d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80cc560 0x7f44f80b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 --2- 172.21.15.5:0/527888444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44f80cc0d0 0x7f44f81f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.900+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 >> 172.21.15.5:0/527888444 conn(0x7f44f80b4f00 msgr2=0x7f44f815d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.901+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 shutdown_connections 2024-04-25T10:04:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:20.901+0000 7f44f7200640 1 -- 172.21.15.5:0/527888444 wait complete. 2024-04-25T10:04:20.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:21.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.122+0000 7fedcb200640 1 Processor -- start 2024-04-25T10:04:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.122+0000 7fedcb200640 1 -- start start 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedcb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedcb200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fedcc0b3ae0 0x7fedcc0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedcb200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedcc0ce790 0x7fedcc0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedcb200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0b4880 con 0x7fedcc0ce790 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedcb200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc0b45a0 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedcb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15e2c0 con 0x7fedcc143d30 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedcc0ce790 0x7fedcc0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedcc0ce790 0x7fedcc0cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46966/0 (socket says 172.21.15.5:46966) 2024-04-25T10:04:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.123+0000 7fedca800640 1 -- 172.21.15.5:0/2354121334 learned_addr learned my addr 172.21.15.5:0/2354121334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:21.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedc9e00640 1 --2- 172.21.15.5:0/2354121334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedc3e00640 1 --2- 172.21.15.5:0/2354121334 >> v2:172.21.15.80:3300/0 conn(0x7fedcc0b3ae0 0x7fedcc0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedca800640 1 -- 172.21.15.5:0/2354121334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 msgr2=0x7fedcc15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedca800640 1 --2- 172.21.15.5:0/2354121334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedca800640 1 -- 172.21.15.5:0/2354121334 >> v2:172.21.15.80:3300/0 conn(0x7fedcc0b3ae0 msgr2=0x7fedcc0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedca800640 1 --2- 172.21.15.5:0/2354121334 >> v2:172.21.15.80:3300/0 conn(0x7fedcc0b3ae0 0x7fedcc0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.124+0000 7fedca800640 1 -- 172.21.15.5:0/2354121334 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc15e5a0 con 0x7fedcc0ce790 2024-04-25T10:04:21.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.125+0000 7fedca800640 1 --2- 172.21.15.5:0/2354121334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0ce790 0x7fedcc0cec20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fedcc072180 tx=0x7fedc402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5b529183a9b3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.125+0000 7fedc9400640 1 -- 172.21.15.5:0/2354121334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4036070 con 0x7fedcc0ce790 2024-04-25T10:04:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.125+0000 7fedc9400640 1 -- 172.21.15.5:0/2354121334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4031600 con 0x7fedcc0ce790 2024-04-25T10:04:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.125+0000 7fedc9400640 1 -- 172.21.15.5:0/2354121334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc40319d0 con 0x7fedcc0ce790 2024-04-25T10:04:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 -- 172.21.15.5:0/2354121334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0ce790 msgr2=0x7fedcc0cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 --2- 172.21.15.5:0/2354121334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0ce790 0x7fedcc0cec20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fedcc072180 tx=0x7fedc402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 -- 172.21.15.5:0/2354121334 shutdown_connections 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 --2- 172.21.15.5:0/2354121334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0ce790 0x7fedcc0cec20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 --2- 172.21.15.5:0/2354121334 >> v2:172.21.15.80:3300/0 conn(0x7fedcc0b3ae0 0x7fedcc0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 --2- 172.21.15.5:0/2354121334 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 -- 172.21.15.5:0/2354121334 >> 172.21.15.5:0/2354121334 conn(0x7fedcc066f20 msgr2=0x7fedcc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 -- 172.21.15.5:0/2354121334 shutdown_connections 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.126+0000 7fedcb200640 1 -- 172.21.15.5:0/2354121334 wait complete. 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.128+0000 7fedcb200640 1 Processor -- start 2024-04-25T10:04:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.128+0000 7fedcb200640 1 -- start start 2024-04-25T10:04:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedcb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 0x7fedcc157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedcb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedcc0ce790 0x7fedcc15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedcb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedcb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc144290 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedcb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc144110 con 0x7fedcc0ce790 2024-04-25T10:04:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedcb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc06e430 con 0x7fedcc143d30 2024-04-25T10:04:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedc9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 0x7fedcc157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.129+0000 7fedc9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 0x7fedcc157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46968/0 (socket says 172.21.15.5:46968) 2024-04-25T10:04:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 -- 172.21.15.5:0/1185541521 learned_addr learned my addr 172.21.15.5:0/1185541521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc3e00640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedcc0ce790 0x7fedcc15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedca800640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 -- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 msgr2=0x7fedcc15d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 -- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedcc0ce790 msgr2=0x7fedcc15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedcc0ce790 0x7fedcc15cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 -- 172.21.15.5:0/1185541521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0caf50 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.130+0000 7fedc9e00640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 0x7fedcc157e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00c7f0 tx=0x7fedbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.131+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc015070 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.131+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc004240 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.131+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc007620 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.131+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc0cb230 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.131+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0cb7f0 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.132+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc068f60 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.133+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedbc0079d0 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.134+0000 7fedc2000640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feda806b4d0 0x7feda806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.134+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fedbc08a560 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.135+0000 7fedc3e00640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feda806b4d0 0x7feda806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.136+0000 7fedc3e00640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feda806b4d0 0x7feda806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fedb4006fd0 tx=0x7fedb4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.138+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedbc059500 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.337+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedcc068670 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.339+0000 7fedc2000640 1 -- 172.21.15.5:0/1185541521 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+13056 (secure 0 0 0) 0x7fedbc058e00 con 0x7fedcc0b3ae0 2024-04-25T10:04:21.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:21.345 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":47,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:20.403974+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feda806b4d0 msgr2=0x7feda806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feda806b4d0 0x7feda806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fedb4006fd0 tx=0x7fedb4008040 comp rx=0 tx=0).stop 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 msgr2=0x7fedcc157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 0x7fedcc157e10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00c7f0 tx=0x7fedbc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 shutdown_connections 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedcc143d30 0x7fedcc15d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feda806b4d0 0x7feda806d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedcc0ce790 0x7fedcc15cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 --2- 172.21.15.5:0/1185541521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedcc0b3ae0 0x7fedcc157e10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.341+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 >> 172.21.15.5:0/1185541521 conn(0x7fedcc066f20 msgr2=0x7fedcc159790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.342+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 shutdown_connections 2024-04-25T10:04:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.342+0000 7fedcb200640 1 -- 172.21.15.5:0/1185541521 wait complete. 2024-04-25T10:04:21.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:21.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:21.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.573+0000 7fe5e6c00640 1 Processor -- start 2024-04-25T10:04:21.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.573+0000 7fe5e6c00640 1 -- start start 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5e6c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e0143fb0 0x7fe5e00b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5e6c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5e006e240 0x7fe5e006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5e6c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e00b2ee0 con 0x7fe5e00ce610 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5e6c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e006ebe0 con 0x7fe5e006e240 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5e6c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00b3150 con 0x7fe5e0143fb0 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5df400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5e006e240 0x7fe5e006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5df400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5e006e240 0x7fe5e006e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46974/0 (socket says 172.21.15.5:46974) 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.574+0000 7fe5df400640 1 -- 172.21.15.5:0/2275114524 learned_addr learned my addr 172.21.15.5:0/2275114524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.575+0000 7fe5dfe00640 1 --2- 172.21.15.5:0/2275114524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e0143fb0 0x7fe5e00b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.575+0000 7fe5df400640 1 -- 172.21.15.5:0/2275114524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e0143fb0 msgr2=0x7fe5e00b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.575+0000 7fe5df400640 1 --2- 172.21.15.5:0/2275114524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e0143fb0 0x7fe5e00b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.575+0000 7fe5df400640 1 -- 172.21.15.5:0/2275114524 >> v1:172.21.15.80:6789/0 conn(0x7fe5e00ce610 legacy=0x7fe5e00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.575+0000 7fe5df400640 1 -- 172.21.15.5:0/2275114524 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e00b3430 con 0x7fe5e006e240 2024-04-25T10:04:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.576+0000 7fe5df400640 1 --2- 172.21.15.5:0/2275114524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e006e240 0x7fe5e006e6a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800d150 tx=0x7fe5c802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b79c0279e881c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.576+0000 7fe5dea00640 1 -- 172.21.15.5:0/2275114524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800b2e0 con 0x7fe5e006e240 2024-04-25T10:04:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5dea00640 1 -- 172.21.15.5:0/2275114524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8009600 con 0x7fe5e006e240 2024-04-25T10:04:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5dea00640 1 -- 172.21.15.5:0/2275114524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800b480 con 0x7fe5e006e240 2024-04-25T10:04:21.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5e6c00640 1 -- 172.21.15.5:0/2275114524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e006e240 msgr2=0x7fe5e006e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/2275114524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e006e240 0x7fe5e006e6a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800d150 tx=0x7fe5c802c8e0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5e6c00640 1 -- 172.21.15.5:0/2275114524 shutdown_connections 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/2275114524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e006e240 0x7fe5e006e6a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/2275114524 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e0143fb0 0x7fe5e00b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.577+0000 7fe5e6c00640 1 -- 172.21.15.5:0/2275114524 >> 172.21.15.5:0/2275114524 conn(0x7fe5e006ab60 msgr2=0x7fe5e006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.578+0000 7fe5e6c00640 1 -- 172.21.15.5:0/2275114524 shutdown_connections 2024-04-25T10:04:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.578+0000 7fe5e6c00640 1 -- 172.21.15.5:0/2275114524 wait complete. 2024-04-25T10:04:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.579+0000 7fe5e6c00640 1 Processor -- start 2024-04-25T10:04:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.579+0000 7fe5e6c00640 1 -- start start 2024-04-25T10:04:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.580+0000 7fe5e6c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e006e240 0x7fe5e013d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.580+0000 7fe5e6c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 0x7fe5e01f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.580+0000 7fe5e6c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5e0143fb0 0x7fe5e00b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.580+0000 7fe5e6c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e00cf2f0 con 0x7fe5e00ce610 2024-04-25T10:04:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.580+0000 7fe5e6c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e0144390 con 0x7fe5e0143fb0 2024-04-25T10:04:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.580+0000 7fe5e6c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e015ac70 con 0x7fe5e006e240 2024-04-25T10:04:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 0x7fe5e01f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 0x7fe5e01f7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46978/0 (socket says 172.21.15.5:46978) 2024-04-25T10:04:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 -- 172.21.15.5:0/3063190028 learned_addr learned my addr 172.21.15.5:0/3063190028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 -- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e006e240 msgr2=0x7fe5e013d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5dfe00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e006e240 0x7fe5e013d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e006e240 0x7fe5e013d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 -- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5e0143fb0 msgr2=0x7fe5e00b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5e0143fb0 0x7fe5e00b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.581+0000 7fe5df400640 1 -- 172.21.15.5:0/3063190028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e015b660 con 0x7fe5e00ce610 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.582+0000 7fe5df400640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 0x7fe5e01f7d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe5c802c920 tx=0x7fe5c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.582+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8040030 con 0x7fe5e00ce610 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.582+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c803c640 con 0x7fe5e00ce610 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.582+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c803ca30 con 0x7fe5e00ce610 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.582+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e013dcb0 con 0x7fe5e00ce610 2024-04-25T10:04:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.582+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e013e270 con 0x7fe5e00ce610 2024-04-25T10:04:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.583+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e0068f60 con 0x7fe5e00ce610 2024-04-25T10:04:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.584+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5c8043030 con 0x7fe5e00ce610 2024-04-25T10:04:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.585+0000 7fe5d7e00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.585+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe5c8030020 con 0x7fe5e00ce610 2024-04-25T10:04:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.586+0000 7fe5dfe00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.587+0000 7fe5dfe00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006fd0 tx=0x7fe5d0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.590+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5c804a030 con 0x7fe5e00ce610 2024-04-25T10:04:21.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.791+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5e00b3f50 con 0x7fe5e00ce610 2024-04-25T10:04:21.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.792+0000 7fe5d7e00640 1 -- 172.21.15.5:0/3063190028 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5c8079e00 con 0x7fe5e00ce610 2024-04-25T10:04:21.798 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:04:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5bc06b4d0 msgr2=0x7fe5bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0006fd0 tx=0x7fe5d0008040 comp rx=0 tx=0).stop 2024-04-25T10:04:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 msgr2=0x7fe5e01f7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 0x7fe5e01f7d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe5c802c920 tx=0x7fe5c8004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:21.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 shutdown_connections 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5bc06b4d0 0x7fe5bc06d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5e0143fb0 0x7fe5e00b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5e00ce610 0x7fe5e01f7d30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 --2- 172.21.15.5:0/3063190028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5e006e240 0x7fe5e013d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 >> 172.21.15.5:0/3063190028 conn(0x7fe5e006ab60 msgr2=0x7fe5e00ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 shutdown_connections 2024-04-25T10:04:21.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:21.795+0000 7fe5e6c00640 1 -- 172.21.15.5:0/3063190028 wait complete. 2024-04-25T10:04:21.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:04:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.023+0000 7f7837e00640 1 Processor -- start 2024-04-25T10:04:22.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.024+0000 7f7837e00640 1 -- start start 2024-04-25T10:04:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.025+0000 7f7837e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f783806d930 0x7f783815c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.025+0000 7f7837e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783806d2b0 con 0x7f78380cc460 2024-04-25T10:04:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.025+0000 7f7837e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783815f310 con 0x7f7838141da0 2024-04-25T10:04:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.025+0000 7f7837e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7838142180 con 0x7f783806d930 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.025+0000 7f782fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f78380cc460 0x7f78380cd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44768/0 (socket says 172.21.15.5:44768) 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.025+0000 7f782fe00640 1 -- 172.21.15.5:0/1292598123 learned_addr learned my addr 172.21.15.5:0/1292598123 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7837400640 1 --2- 172.21.15.5:0/1292598123 >> v2:172.21.15.80:3300/0 conn(0x7f783806d930 0x7f783815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345541690 0 0) 0x7f7820001110 con 0x7f78380cc460 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003520 con 0x7f78380cc460 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 407798848 0 0) 0x7f781c003520 con 0x7f78380cc460 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 >> v2:172.21.15.80:3300/0 conn(0x7f783806d930 msgr2=0x7f783815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 --2- 172.21.15.5:0/1292598123 >> v2:172.21.15.80:3300/0 conn(0x7f783806d930 0x7f783815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 >> v1:172.21.15.80:6789/0 conn(0x7f7838141da0 legacy=0x7f78380cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783815f5f0 con 0x7f78380cc460 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f7820001840 con 0x7f78380cc460 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 >> v1:172.21.15.5:6789/0 conn(0x7f78380cc460 legacy=0x7f78380cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0039b0 0x7f781c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c004800 0x7f781c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.026+0000 7f7836000640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0051c0 0x7f781c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838142180 con 0x7f781c004800 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c00c090 con 0x7f781c0051c0 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c00c290 con 0x7f781c0039b0 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f782fe00640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c004800 0x7f781c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f782fe00640 1 -- 172.21.15.5:0/1292598123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0039b0 msgr2=0x7f781c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f782fe00640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f781c0039b0 0x7f781c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f782fe00640 1 -- 172.21.15.5:0/1292598123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0051c0 msgr2=0x7f781c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f782fe00640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c0051c0 0x7f781c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f782fe00640 1 -- 172.21.15.5:0/1292598123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c003520 con 0x7f781c004800 2024-04-25T10:04:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.027+0000 7f7837400640 1 -- 172.21.15.5:0/1292598123 reap_dead start 2024-04-25T10:04:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.028+0000 7f782fe00640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c004800 0x7f781c004c80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f782000c4c0 tx=0x7f781c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa18f179eef7191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.028+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000c0e0 con 0x7f781c004800 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.028+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820034070 con 0x7f781c004800 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7836000640 1 -- 172.21.15.5:0/1292598123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820005d30 con 0x7f781c004800 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 -- 172.21.15.5:0/1292598123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c004800 msgr2=0x7f781c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c004800 0x7f781c004c80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f782000c4c0 tx=0x7f781c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 -- 172.21.15.5:0/1292598123 shutdown_connections 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 --2- 172.21.15.5:0/1292598123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f781c004800 0x7f781c004c80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 -- 172.21.15.5:0/1292598123 >> 172.21.15.5:0/1292598123 conn(0x7f78380b4f00 msgr2=0x7f783806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 -- 172.21.15.5:0/1292598123 shutdown_connections 2024-04-25T10:04:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.029+0000 7f7837e00640 1 -- 172.21.15.5:0/1292598123 wait complete. 2024-04-25T10:04:22.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.031+0000 7f7837e00640 1 Processor -- start 2024-04-25T10:04:22.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.031+0000 7f7837e00640 1 -- start start 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.032+0000 7f7837e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c004800 0x7f78380b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.032+0000 7f7837e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78380c4490 0x7f7838152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.032+0000 7f7837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 0x7f78380c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.032+0000 7f7837e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783815fe90 con 0x7f78380c4cb0 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.032+0000 7f7837e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783815fbb0 con 0x7f781c004800 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.032+0000 7f7837e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783815c720 con 0x7f78380c4490 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 0x7f78380c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 0x7f78380c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46994/0 (socket says 172.21.15.5:46994) 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 -- 172.21.15.5:0/1886053052 learned_addr learned my addr 172.21.15.5:0/1886053052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f782fe00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78380c4490 0x7f7838152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7836a00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c004800 0x7f78380b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 -- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78380c4490 msgr2=0x7f7838152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78380c4490 0x7f7838152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 -- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c004800 msgr2=0x7f78380b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c004800 0x7f78380b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7837400640 1 -- 172.21.15.5:0/1886053052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380c4910 con 0x7f78380c4cb0 2024-04-25T10:04:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.033+0000 7f7836a00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c004800 0x7f78380b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.034+0000 7f7837400640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 0x7f78380c5130 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f783806d6b0 tx=0x7f783000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.034+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783000bb10 con 0x7f78380c4cb0 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.034+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783001a070 con 0x7f78380c4cb0 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.034+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78380b35c0 con 0x7f78380c4cb0 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.034+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830014400 con 0x7f78380c4cb0 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.034+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78380b3b80 con 0x7f78380c4cb0 2024-04-25T10:04:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.035+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7838068f60 con 0x7f78380c4cb0 2024-04-25T10:04:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.036+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7830013070 con 0x7f78380c4cb0 2024-04-25T10:04:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.038+0000 7f782ea00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f781406b500 0x7f781406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.038+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f783008a310 con 0x7f78380c4cb0 2024-04-25T10:04:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.039+0000 7f7836a00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f781406b500 0x7f781406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.040+0000 7f7836a00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f781406b500 0x7f781406d9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f78380734b0 tx=0x7f7828009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.042+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78300592b0 con 0x7f78380c4cb0 2024-04-25T10:04:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.245+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f783815c100 con 0x7f78380c4cb0 2024-04-25T10:04:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.245+0000 7f782ea00640 1 -- 172.21.15.5:0/1886053052 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7f783008a0d0 con 0x7f78380c4cb0 2024-04-25T10:04:22.251 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:22.251 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:04:22.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:22.251 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:04:22.252 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:04:22.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:22.252 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:04:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f781406b500 msgr2=0x7f781406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f781406b500 0x7f781406d9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f78380734b0 tx=0x7f7828009040 comp rx=0 tx=0).stop 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 msgr2=0x7f78380c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 0x7f78380c5130 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f783806d6b0 tx=0x7f783000ef50 comp rx=0 tx=0).stop 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 shutdown_connections 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78380c4cb0 0x7f78380c5130 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78380c4490 0x7f7838152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f781406b500 0x7f781406d9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.248+0000 7f7837e00640 1 --2- 172.21.15.5:0/1886053052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f781c004800 0x7f78380b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.249+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 >> 172.21.15.5:0/1886053052 conn(0x7f78380b4f00 msgr2=0x7f7838140410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.249+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 shutdown_connections 2024-04-25T10:04:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.249+0000 7f7837e00640 1 -- 172.21.15.5:0/1886053052 wait complete. 2024-04-25T10:04:22.273 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:04:22.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:04:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.486+0000 7ff9bdc00640 1 Processor -- start 2024-04-25T10:04:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.487+0000 7ff9bdc00640 1 -- start start 2024-04-25T10:04:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.488+0000 7ff9bdc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff9b815db10 0x7ff9b80b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.488+0000 7ff9bdc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b806bf40 con 0x7ff9b80d0820 2024-04-25T10:04:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.488+0000 7ff9bdc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b806c1f0 con 0x7ff9b806bb00 2024-04-25T10:04:22.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.488+0000 7ff9bdc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b806c4d0 con 0x7ff9b815db10 2024-04-25T10:04:22.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9bd200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff9b80d0820 0x7ff9b80d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44772/0 (socket says 172.21.15.5:44772) 2024-04-25T10:04:22.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9bd200640 1 -- 172.21.15.5:0/3305957518 learned_addr learned my addr 172.21.15.5:0/3305957518 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:22.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7e00640 1 --2- 172.21.15.5:0/3305957518 >> v2:172.21.15.80:3300/0 conn(0x7ff9b815db10 0x7ff9b80b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217370459 0 0) 0x7ff9b806bf40 con 0x7ff9b80d0820 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99c003520 con 0x7ff9b80d0820 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925810012 0 0) 0x7ff99c003520 con 0x7ff9b80d0820 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 >> v2:172.21.15.80:3300/0 conn(0x7ff9b815db10 msgr2=0x7ff9b80b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 --2- 172.21.15.5:0/3305957518 >> v2:172.21.15.80:3300/0 conn(0x7ff9b815db10 0x7ff9b80b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 >> v1:172.21.15.80:6789/0 conn(0x7ff9b806bb00 legacy=0x7ff9b80d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.489+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b806c7b0 con 0x7ff9b80d0820 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7ff9b00019f0 con 0x7ff9b80d0820 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 >> v1:172.21.15.5:6789/0 conn(0x7ff9b80d0820 legacy=0x7ff9b80d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff99c0039b0 0x7ff99c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff99c004800 0x7ff99c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c0051c0 0x7ff99c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b806c4d0 con 0x7ff99c004800 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c00c040 con 0x7ff99c0039b0 2024-04-25T10:04:22.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c00c240 con 0x7ff99c0051c0 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9afe00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff99c004800 0x7ff99c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9bd200640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c0051c0 0x7ff99c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.490+0000 7ff9afe00640 1 -- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c0051c0 msgr2=0x7ff99c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9b7e00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff99c0039b0 0x7ff99c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9afe00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c0051c0 0x7ff99c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9afe00640 1 -- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff99c0039b0 msgr2=0x7ff99c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9afe00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff99c0039b0 0x7ff99c003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9afe00640 1 -- 172.21.15.5:0/3305957518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c003520 con 0x7ff99c004800 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9bd200640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c0051c0 0x7ff99c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:22.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.491+0000 7ff9bd200640 1 -- 172.21.15.5:0/3305957518 reap_dead start 2024-04-25T10:04:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.492+0000 7ff9afe00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff99c004800 0x7ff99c004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0003db0 tx=0x7ff99c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bef3260f9ec9158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.492+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0037070 con 0x7ff99c004800 2024-04-25T10:04:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.492+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a002a9f0 con 0x7ff99c004800 2024-04-25T10:04:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.492+0000 7ff9b7400640 1 -- 172.21.15.5:0/3305957518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0003980 con 0x7ff99c004800 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 -- 172.21.15.5:0/3305957518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff99c004800 msgr2=0x7ff99c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff99c004800 0x7ff99c004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0003db0 tx=0x7ff99c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 -- 172.21.15.5:0/3305957518 shutdown_connections 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/3305957518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff99c004800 0x7ff99c004c80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 -- 172.21.15.5:0/3305957518 >> 172.21.15.5:0/3305957518 conn(0x7ff9b80b5210 msgr2=0x7ff9b80b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 -- 172.21.15.5:0/3305957518 shutdown_connections 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.493+0000 7ff9bdc00640 1 -- 172.21.15.5:0/3305957518 wait complete. 2024-04-25T10:04:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.495+0000 7ff9bdc00640 1 Processor -- start 2024-04-25T10:04:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.495+0000 7ff9bdc00640 1 -- start start 2024-04-25T10:04:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bdc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c004800 0x7ff9b81f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bdc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b8156950 0x7ff9b8156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 0x7ff9b80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bdc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815e070 con 0x7ff9b80c7230 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bdc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815def0 con 0x7ff9b8156950 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bdc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80b6ba0 con 0x7ff99c004800 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 0x7ff9b80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 0x7ff9b80c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47006/0 (socket says 172.21.15.5:47006) 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9bd200640 1 -- 172.21.15.5:0/86482851 learned_addr learned my addr 172.21.15.5:0/86482851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9afe00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b8156950 0x7ff9b8156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.496+0000 7ff9b7e00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c004800 0x7ff9b81f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bd200640 1 -- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c004800 msgr2=0x7ff9b81f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bd200640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c004800 0x7ff9b81f08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bd200640 1 -- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b8156950 msgr2=0x7ff9b8156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bd200640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b8156950 0x7ff9b8156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bd200640 1 -- 172.21.15.5:0/86482851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b81573d0 con 0x7ff9b80c7230 2024-04-25T10:04:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9afe00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b8156950 0x7ff9b8156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bd200640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 0x7ff9b80c76b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff99c004780 tx=0x7ff9b000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0004d60 con 0x7ff9b80c7230 2024-04-25T10:04:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b000bb20 con 0x7ff9b80c7230 2024-04-25T10:04:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0007110 con 0x7ff9b80c7230 2024-04-25T10:04:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.497+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b81576b0 con 0x7ff9b80c7230 2024-04-25T10:04:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.498+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b8157c70 con 0x7ff9b80c7230 2024-04-25T10:04:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.499+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b80677d0 con 0x7ff9b80c7230 2024-04-25T10:04:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.500+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9b001b020 con 0x7ff9b80c7230 2024-04-25T10:04:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.501+0000 7ff9b4c00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff99406b520 0x7ff99406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.501+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff9b0089fe0 con 0x7ff9b80c7230 2024-04-25T10:04:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.502+0000 7ff9b7e00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff99406b520 0x7ff99406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.503+0000 7ff9b7e00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff99406b520 0x7ff99406d9e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80b9810 tx=0x7ff9a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.505+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9b00590d0 con 0x7ff9b80c7230 2024-04-25T10:04:22.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:22.713+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff9b8068f60 con 0x7ff9b80c7230 2024-04-25T10:04:23.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.409+0000 7ff9b4c00640 1 -- 172.21.15.5:0/86482851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff9b00589d0 con 0x7ff9b80c7230 2024-04-25T10:04:23.415 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:04:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff99406b520 msgr2=0x7ff99406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff99406b520 0x7ff99406d9e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80b9810 tx=0x7ff9a8008040 comp rx=0 tx=0).stop 2024-04-25T10:04:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 msgr2=0x7ff9b80c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 0x7ff9b80c76b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff99c004780 tx=0x7ff9b000b4a0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 shutdown_connections 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b80c7230 0x7ff9b80c76b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff99406b520 0x7ff99406d9e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9b8156950 0x7ff9b8156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 --2- 172.21.15.5:0/86482851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff99c004800 0x7ff9b81f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 >> 172.21.15.5:0/86482851 conn(0x7ff9b80b5210 msgr2=0x7ff9b80d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 shutdown_connections 2024-04-25T10:04:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.413+0000 7ff9bdc00640 1 -- 172.21.15.5:0/86482851 wait complete. 2024-04-25T10:04:23.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:04:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.660+0000 7fe0bda00640 1 Processor -- start 2024-04-25T10:04:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.660+0000 7fe0bda00640 1 -- start start 2024-04-25T10:04:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.662+0000 7fe0bda00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0b806d130 0x7fe0b8071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.662+0000 7fe0bda00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b81453c0 con 0x7fe0b8071670 2024-04-25T10:04:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.662+0000 7fe0bda00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b806d6a0 con 0x7fe0b8144fe0 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.662+0000 7fe0bda00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b80a8560 con 0x7fe0b806d130 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.662+0000 7fe0b7400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe0b8071670 0x7fe0b815c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44788/0 (socket says 172.21.15.5:44788) 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.662+0000 7fe0b7400640 1 -- 172.21.15.5:0/3747205635 learned_addr learned my addr 172.21.15.5:0/3747205635 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.663+0000 7fe0b6000640 1 --2- 172.21.15.5:0/3747205635 >> v2:172.21.15.80:3300/0 conn(0x7fe0b806d130 0x7fe0b8071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.663+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938648311 0 0) 0x7fe0b81453c0 con 0x7fe0b8071670 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.663+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe09c003520 con 0x7fe0b8071670 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.663+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2891721060 0 0) 0x7fe09c003520 con 0x7fe0b8071670 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.663+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 >> v2:172.21.15.80:3300/0 conn(0x7fe0b806d130 msgr2=0x7fe0b8071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.663+0000 7fe0b5600640 1 --2- 172.21.15.5:0/3747205635 >> v2:172.21.15.80:3300/0 conn(0x7fe0b806d130 0x7fe0b8071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 >> v1:172.21.15.80:6789/0 conn(0x7fe0b8144fe0 legacy=0x7fe0b806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b806d980 con 0x7fe0b8071670 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe0b00019f0 con 0x7fe0b8071670 2024-04-25T10:04:23.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 >> v1:172.21.15.5:6789/0 conn(0x7fe0b8071670 legacy=0x7fe0b815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c0039b0 0x7fe09c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c004800 0x7fe09c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c0051c0 0x7fe09c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c030 con 0x7fe09c004800 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c1e0 con 0x7fe09c0039b0 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c00c3e0 con 0x7fe09c0051c0 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b6000640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c004800 0x7fe09c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b6a00640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c0039b0 0x7fe09c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.664+0000 7fe0b7400640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c0051c0 0x7fe09c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.665+0000 7fe0b6000640 1 -- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c0051c0 msgr2=0x7fe09c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.665+0000 7fe0b6000640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c0051c0 0x7fe09c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.665+0000 7fe0b6000640 1 -- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c0039b0 msgr2=0x7fe09c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.665+0000 7fe0b6000640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe09c0039b0 0x7fe09c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.665+0000 7fe0b6000640 1 -- 172.21.15.5:0/3747205635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c003520 con 0x7fe09c004800 2024-04-25T10:04:23.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.665+0000 7fe0b7400640 1 -- 172.21.15.5:0/3747205635 reap_dead start 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.666+0000 7fe0b6000640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c004800 0x7fe09c004c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009ac0 tx=0x7fe09c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1bfe2a1d29526d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.666+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4036070 con 0x7fe09c004800 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.666+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4031400 con 0x7fe09c004800 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.666+0000 7fe0b5600640 1 -- 172.21.15.5:0/3747205635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4009690 con 0x7fe09c004800 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 -- 172.21.15.5:0/3747205635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c004800 msgr2=0x7fe09c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c004800 0x7fe09c004c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009ac0 tx=0x7fe09c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 -- 172.21.15.5:0/3747205635 shutdown_connections 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 --2- 172.21.15.5:0/3747205635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09c004800 0x7fe09c004c80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 -- 172.21.15.5:0/3747205635 >> 172.21.15.5:0/3747205635 conn(0x7fe0b806ab60 msgr2=0x7fe0b806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 -- 172.21.15.5:0/3747205635 shutdown_connections 2024-04-25T10:04:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.667+0000 7fe0bda00640 1 -- 172.21.15.5:0/3747205635 wait complete. 2024-04-25T10:04:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.669+0000 7fe0bda00640 1 Processor -- start 2024-04-25T10:04:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.669+0000 7fe0bda00640 1 -- start start 2024-04-25T10:04:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0bda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c004800 0x7fe0b8140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0bda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b813d200 0x7fe0b81f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0bda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 0x7fe0b8142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0bda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b815be50 con 0x7fe0b81420a0 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0bda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b806df40 con 0x7fe0b813d200 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0bda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b8071130 con 0x7fe09c004800 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 0x7fe0b8142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 0x7fe0b8142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47026/0 (socket says 172.21.15.5:47026) 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0b7400640 1 -- 172.21.15.5:0/2405990217 learned_addr learned my addr 172.21.15.5:0/2405990217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.670+0000 7fe0b6000640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b813d200 0x7fe0b81f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b6a00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c004800 0x7fe0b8140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b7400640 1 -- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c004800 msgr2=0x7fe0b8140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b7400640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c004800 0x7fe0b8140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b7400640 1 -- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b813d200 msgr2=0x7fe0b81f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b7400640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b813d200 0x7fe0b81f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b7400640 1 -- 172.21.15.5:0/2405990217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b813d970 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0b7400640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 0x7fe0b8142fe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0000f80 tx=0x7fe0b000e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000ec30 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0018070 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.672+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b81415c0 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.671+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0013420 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.672+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b8141b80 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.673+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b8068f60 con 0x7fe0b81420a0 2024-04-25T10:04:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.674+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0b00135e0 con 0x7fe0b81420a0 2024-04-25T10:04:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.675+0000 7fe0aea00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe09806b4d0 0x7fe09806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.676+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe0b008a440 con 0x7fe0b81420a0 2024-04-25T10:04:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.676+0000 7fe0b6a00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe09806b4d0 0x7fe09806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.677+0000 7fe0b6a00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe09806b4d0 0x7fe09806d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8050700 tx=0x7fe0a8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.681+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0b0059300 con 0x7fe0b81420a0 2024-04-25T10:04:23.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:23.777 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:23.886+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe0b81e1470 con 0x7fe0b81420a0 2024-04-25T10:04:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.425+0000 7fe0aea00640 1 -- 172.21.15.5:0/2405990217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe0b0058c00 con 0x7fe0b81420a0 2024-04-25T10:04:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:04:24.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.430+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe09806b4d0 msgr2=0x7fe09806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.430+0000 7fe0bda00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe09806b4d0 0x7fe09806d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8050700 tx=0x7fe0a8009040 comp rx=0 tx=0).stop 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.430+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 msgr2=0x7fe0b8142fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.430+0000 7fe0bda00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 0x7fe0b8142fe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0000f80 tx=0x7fe0b000e160 comp rx=0 tx=0).stop 2024-04-25T10:04:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.432+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 shutdown_connections 2024-04-25T10:04:24.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.432+0000 7fe0bda00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b81420a0 0x7fe0b8142fe0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.432+0000 7fe0bda00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe09806b4d0 0x7fe09806d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.432+0000 7fe0bda00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0b813d200 0x7fe0b81f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.432+0000 7fe0bda00640 1 --2- 172.21.15.5:0/2405990217 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09c004800 0x7fe0b8140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.432+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 >> 172.21.15.5:0/2405990217 conn(0x7fe0b806ab60 msgr2=0x7fe0b80b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.433+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 shutdown_connections 2024-04-25T10:04:24.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.435+0000 7fe0bda00640 1 -- 172.21.15.5:0/2405990217 wait complete. 2024-04-25T10:04:24.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:04:24.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.650+0000 7f7185a00640 1 Processor -- start 2024-04-25T10:04:24.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.650+0000 7f7185a00640 1 -- start start 2024-04-25T10:04:24.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.652+0000 7f7185a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800b48c0 0x7f718015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.652+0000 7f7185a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7180161d90 0x7f71800b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.652+0000 7f7185a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71801606e0 con 0x7f71800d3f90 2024-04-25T10:04:24.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.652+0000 7f7185a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71801609c0 con 0x7f7180161d90 2024-04-25T10:04:24.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.652+0000 7f7185a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180160ca0 con 0x7f71800b48c0 2024-04-25T10:04:24.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.652+0000 7f7185000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7180161d90 0x7f71800b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f717fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800b48c0 0x7f718015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f717f400640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f71800d3f90 0x7f7180161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:36592/0 (socket says 172.21.15.5:36592) 2024-04-25T10:04:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f717f400640 1 -- 172.21.15.5:0/1199716740 learned_addr learned my addr 172.21.15.5:0/1199716740 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f7185000640 1 --2- 172.21.15.5:0/1199716740 >> v2:172.21.15.80:3300/0 conn(0x7f7180161d90 0x7f71800b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:37458/0 (socket says 172.21.15.5:37458) 2024-04-25T10:04:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f7185000640 1 -- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800b48c0 msgr2=0x7f718015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:24.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f7185000640 1 --2- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800b48c0 0x7f718015fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f7185000640 1 -- 172.21.15.5:0/1199716740 >> v1:172.21.15.80:6789/0 conn(0x7f71800d3f90 legacy=0x7f7180161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.653+0000 7f7185000640 1 -- 172.21.15.5:0/1199716740 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7180160f80 con 0x7f7180161d90 2024-04-25T10:04:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.655+0000 7f7185000640 1 --2- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7180161d90 0x7f71800b2b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7178009d90 tx=0x7f717802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc44ac090fe5658e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.655+0000 7f717ea00640 1 -- 172.21.15.5:0/1199716740 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178031600 con 0x7f7180161d90 2024-04-25T10:04:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.656+0000 7f717ea00640 1 -- 172.21.15.5:0/1199716740 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71780317a0 con 0x7f7180161d90 2024-04-25T10:04:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.656+0000 7f717ea00640 1 -- 172.21.15.5:0/1199716740 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178031b70 con 0x7f7180161d90 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.656+0000 7f7185a00640 1 -- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7180161d90 msgr2=0x7f71800b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.656+0000 7f7185a00640 1 --2- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7180161d90 0x7f71800b2b70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7178009d90 tx=0x7f717802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.657+0000 7f7185a00640 1 -- 172.21.15.5:0/1199716740 shutdown_connections 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.657+0000 7f7185a00640 1 --2- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7180161d90 0x7f71800b2b70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.657+0000 7f7185a00640 1 --2- 172.21.15.5:0/1199716740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800b48c0 0x7f718015fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.657+0000 7f7185a00640 1 -- 172.21.15.5:0/1199716740 >> 172.21.15.5:0/1199716740 conn(0x7f718008af80 msgr2=0x7f71800d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.657+0000 7f7185a00640 1 -- 172.21.15.5:0/1199716740 shutdown_connections 2024-04-25T10:04:24.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.657+0000 7f7185a00640 1 -- 172.21.15.5:0/1199716740 wait complete. 2024-04-25T10:04:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.659+0000 7f7185a00640 1 Processor -- start 2024-04-25T10:04:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.659+0000 7f7185a00640 1 -- start start 2024-04-25T10:04:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.660+0000 7f7185a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71800b48c0 0x7f718015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.660+0000 7f7185a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800d3f90 0x7f71800c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.660+0000 7f7185a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 0x7f71801f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.660+0000 7f7185a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71801603a0 con 0x7f7180161d90 2024-04-25T10:04:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.660+0000 7f7185a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b4ca0 con 0x7f71800b48c0 2024-04-25T10:04:24.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.660+0000 7f7185a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800b30b0 con 0x7f71800d3f90 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 0x7f71801f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 0x7f71801f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47032/0 (socket says 172.21.15.5:47032) 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 -- 172.21.15.5:0/3399676014 learned_addr learned my addr 172.21.15.5:0/3399676014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f717f400640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800d3f90 0x7f71800c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 -- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800d3f90 msgr2=0x7f71800c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800d3f90 0x7f71800c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 -- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71800b48c0 msgr2=0x7f718015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.661+0000 7f7185000640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71800b48c0 0x7f718015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.662+0000 7f7185000640 1 -- 172.21.15.5:0/3399676014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71801f4e00 con 0x7f7180161d90 2024-04-25T10:04:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.662+0000 7f7185000640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 0x7f71801f7190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7178009eb0 tx=0x7f7178004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.663+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717803f070 con 0x7f7180161d90 2024-04-25T10:04:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.663+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7178030b60 con 0x7f7180161d90 2024-04-25T10:04:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.663+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178009450 con 0x7f7180161d90 2024-04-25T10:04:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.663+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71801f50e0 con 0x7f7180161d90 2024-04-25T10:04:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.663+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71801f56a0 con 0x7f7180161d90 2024-04-25T10:04:24.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.664+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7180068f60 con 0x7f7180161d90 2024-04-25T10:04:24.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.666+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f717803b410 con 0x7f7180161d90 2024-04-25T10:04:24.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.667+0000 7f7177e00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f716006b4d0 0x7f716006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:24.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.667+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f71780ab950 con 0x7f7180161d90 2024-04-25T10:04:24.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.667+0000 7f717fe00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f716006b4d0 0x7f716006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.669+0000 7f717fe00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f716006b4d0 0x7f716006d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7170006a90 tx=0x7f7170006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.675+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71780433c0 con 0x7f7180161d90 2024-04-25T10:04:24.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:24.925+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f71800b2d90 con 0x7f7180161d90 2024-04-25T10:04:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.430+0000 7f7177e00640 1 -- 172.21.15.5:0/3399676014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f717807a600 con 0x7f7180161d90 2024-04-25T10:04:25.436 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T10:04:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.433+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f716006b4d0 msgr2=0x7f716006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.433+0000 7f7185a00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f716006b4d0 0x7f716006d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7170006a90 tx=0x7f7170006a20 comp rx=0 tx=0).stop 2024-04-25T10:04:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.433+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 msgr2=0x7f71801f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.433+0000 7f7185a00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 0x7f71801f7190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7178009eb0 tx=0x7f7178004a40 comp rx=0 tx=0).stop 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.434+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 shutdown_connections 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.434+0000 7f7185a00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7180161d90 0x7f71801f7190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.434+0000 7f7185a00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f71800d3f90 0x7f71800c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.434+0000 7f7185a00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f716006b4d0 0x7f716006d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.434+0000 7f7185a00640 1 --2- 172.21.15.5:0/3399676014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71800b48c0 0x7f718015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.434+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 >> 172.21.15.5:0/3399676014 conn(0x7f718008af80 msgr2=0x7f71800b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.435+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 shutdown_connections 2024-04-25T10:04:25.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.435+0000 7f7185a00640 1 -- 172.21.15.5:0/3399676014 wait complete. 2024-04-25T10:04:25.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.650+0000 7fce62e00640 1 Processor -- start 2024-04-25T10:04:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.650+0000 7fce62e00640 1 -- start start 2024-04-25T10:04:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.651+0000 7fce62e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fce5c071ed0 0x7fce5c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.651+0000 7fce62e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fce5c0ce890 0x7fce5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.651+0000 7fce62e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c15bdc0 con 0x7fce5c1529e0 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.651+0000 7fce62e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c0b2340 con 0x7fce5c071ed0 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.651+0000 7fce62e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c0cecc0 con 0x7fce5c0ce890 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce5be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fce5c1529e0 0x7fce5c0c5ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44794/0 (socket says 172.21.15.5:44794) 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce5be00640 1 -- 172.21.15.5:0/2177016309 learned_addr learned my addr 172.21.15.5:0/2177016309 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce53400640 1 --2- 172.21.15.5:0/2177016309 >> v2:172.21.15.5:3300/0 conn(0x7fce5c071ed0 0x7fce5c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce5b400640 1 -- 172.21.15.5:0/2177016309 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715581194 0 0) 0x7fce5c15bdc0 con 0x7fce5c1529e0 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce5b400640 1 -- 172.21.15.5:0/2177016309 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce3c003520 con 0x7fce5c1529e0 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce60e00640 1 --2- 172.21.15.5:0/2177016309 >> v2:172.21.15.80:3300/0 conn(0x7fce5c0ce890 0x7fce5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce53400640 1 -- 172.21.15.5:0/2177016309 >> v2:172.21.15.80:3300/0 conn(0x7fce5c0ce890 msgr2=0x7fce5c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce53400640 1 --2- 172.21.15.5:0/2177016309 >> v2:172.21.15.80:3300/0 conn(0x7fce5c0ce890 0x7fce5c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.652+0000 7fce53400640 1 -- 172.21.15.5:0/2177016309 >> v1:172.21.15.5:6789/0 conn(0x7fce5c1529e0 legacy=0x7fce5c0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.653+0000 7fce53400640 1 -- 172.21.15.5:0/2177016309 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c15c060 con 0x7fce5c071ed0 2024-04-25T10:04:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.653+0000 7fce60e00640 1 --2- 172.21.15.5:0/2177016309 >> v2:172.21.15.80:3300/0 conn(0x7fce5c0ce890 0x7fce5c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.654+0000 7fce53400640 1 --2- 172.21.15.5:0/2177016309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c0ce350 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0c78c0 tx=0x7fce4402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11151a48bcc9076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.654+0000 7fce5b400640 1 -- 172.21.15.5:0/2177016309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44037070 con 0x7fce5c071ed0 2024-04-25T10:04:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.654+0000 7fce5b400640 1 -- 172.21.15.5:0/2177016309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce44032440 con 0x7fce5c071ed0 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.654+0000 7fce5b400640 1 -- 172.21.15.5:0/2177016309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44032810 con 0x7fce5c071ed0 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.654+0000 7fce62e00640 1 -- 172.21.15.5:0/2177016309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 msgr2=0x7fce5c0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.654+0000 7fce62e00640 1 --2- 172.21.15.5:0/2177016309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c0ce350 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0c78c0 tx=0x7fce4402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.655+0000 7fce62e00640 1 -- 172.21.15.5:0/2177016309 shutdown_connections 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.655+0000 7fce62e00640 1 --2- 172.21.15.5:0/2177016309 >> v2:172.21.15.80:3300/0 conn(0x7fce5c0ce890 0x7fce5c06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.655+0000 7fce62e00640 1 --2- 172.21.15.5:0/2177016309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c0ce350 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.655+0000 7fce62e00640 1 -- 172.21.15.5:0/2177016309 >> 172.21.15.5:0/2177016309 conn(0x7fce5c071150 msgr2=0x7fce5c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.655+0000 7fce62e00640 1 -- 172.21.15.5:0/2177016309 shutdown_connections 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.655+0000 7fce62e00640 1 -- 172.21.15.5:0/2177016309 wait complete. 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.657+0000 7fce62e00640 1 Processor -- start 2024-04-25T10:04:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.657+0000 7fce62e00640 1 -- start start 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.658+0000 7fce62e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.658+0000 7fce62e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce5c0ce890 0x7fce5c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.658+0000 7fce62e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce5c0b3d60 0x7fce5c151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.658+0000 7fce62e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c15c900 con 0x7fce5c071ed0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.658+0000 7fce62e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c15c620 con 0x7fce5c0ce890 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.658+0000 7fce62e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0c65e0 con 0x7fce5c0b3d60 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c15cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47060/0 (socket says 172.21.15.5:47060) 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 -- 172.21.15.5:0/2163187139 learned_addr learned my addr 172.21.15.5:0/2163187139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce53400640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce5c0ce890 0x7fce5c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 -- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce5c0b3d60 msgr2=0x7fce5c151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce60e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce5c0b3d60 0x7fce5c151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce5c0b3d60 0x7fce5c151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 -- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce5c0ce890 msgr2=0x7fce5c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce5c0ce890 0x7fce5c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce5be00640 1 -- 172.21.15.5:0/2163187139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c0b4390 con 0x7fce5c071ed0 2024-04-25T10:04:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce60e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce5c0b3d60 0x7fce5c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.659+0000 7fce53400640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce5c0ce890 0x7fce5c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.660+0000 7fce5be00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c15cf90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0010e0 tx=0x7fce4c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.660+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c017070 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.660+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c005d00 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.660+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c0b4670 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.660+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0072b0 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.660+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c0b4c30 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.661+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c068fd0 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.663+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce4c01a070 con 0x7fce5c071ed0 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.663+0000 7fce58c00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce3806b500 0x7fce3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.664+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce4c0897a0 con 0x7fce5c071ed0 2024-04-25T10:04:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.664+0000 7fce53400640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce3806b500 0x7fce3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.666+0000 7fce53400640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce3806b500 0x7fce3806d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce44034000 tx=0x7fce4402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.668+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce4c058480 con 0x7fce5c071ed0 2024-04-25T10:04:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.866+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce5c0cda60 con 0x7fce5c071ed0 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.869+0000 7fce58c00640 1 -- 172.21.15.5:0/2163187139 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fce4c057d80 con 0x7fce5c071ed0 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:24.928022+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce3806b500 msgr2=0x7fce3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce3806b500 0x7fce3806d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce44034000 tx=0x7fce4402e000 comp rx=0 tx=0).stop 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 msgr2=0x7fce5c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c15cf90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0010e0 tx=0x7fce4c00c960 comp rx=0 tx=0).stop 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 shutdown_connections 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce5c0b3d60 0x7fce5c151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce3806b500 0x7fce3806d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce5c0ce890 0x7fce5c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 --2- 172.21.15.5:0/2163187139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c071ed0 0x7fce5c15cf90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.871+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 >> 172.21.15.5:0/2163187139 conn(0x7fce5c071150 msgr2=0x7fce5c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.872+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 shutdown_connections 2024-04-25T10:04:25.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:25.872+0000 7fce62e00640 1 -- 172.21.15.5:0/2163187139 wait complete. 2024-04-25T10:04:25.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:26.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.095+0000 7f6713200640 1 Processor -- start 2024-04-25T10:04:26.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.096+0000 7f6713200640 1 -- start start 2024-04-25T10:04:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.097+0000 7f6713200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6714161200 0x7f67140b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.097+0000 7f6713200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67140d2380 0x7f67140d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.097+0000 7f6713200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140cf410 con 0x7f67140d0600 2024-04-25T10:04:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.097+0000 7f6713200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140d27c0 con 0x7f67140d2380 2024-04-25T10:04:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.097+0000 7f6713200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140b28b0 con 0x7f6714161200 2024-04-25T10:04:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.098+0000 7f6711400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67140d2380 0x7f67140d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.098+0000 7f6711400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67140d2380 0x7f67140d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:37504/0 (socket says 172.21.15.5:37504) 2024-04-25T10:04:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.098+0000 7f6711400640 1 -- 172.21.15.5:0/2810655256 learned_addr learned my addr 172.21.15.5:0/2810655256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.098+0000 7f6711e00640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6714161200 0x7f67140b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.099+0000 7f6711400640 1 -- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6714161200 msgr2=0x7f67140b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.099+0000 7f6711400640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6714161200 0x7f67140b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.099+0000 7f6711400640 1 -- 172.21.15.5:0/2810655256 >> v1:172.21.15.80:6789/0 conn(0x7f67140d0600 legacy=0x7f67140ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.099+0000 7f6711400640 1 -- 172.21.15.5:0/2810655256 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140cf6f0 con 0x7f67140d2380 2024-04-25T10:04:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.099+0000 7f6711e00640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6714161200 0x7f67140b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.100+0000 7f6711400640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140d00c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098a0 tx=0x7f66fc02a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e11cc1c0c9858b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.101+0000 7f6710a00640 1 -- 172.21.15.5:0/2810655256 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc037070 con 0x7f67140d2380 2024-04-25T10:04:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.101+0000 7f6710a00640 1 -- 172.21.15.5:0/2810655256 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc032440 con 0x7f67140d2380 2024-04-25T10:04:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.101+0000 7f6710a00640 1 -- 172.21.15.5:0/2810655256 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc032810 con 0x7f67140d2380 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 -- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 msgr2=0x7f67140d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140d00c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0098a0 tx=0x7f66fc02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 -- 172.21.15.5:0/2810655256 shutdown_connections 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140d00c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 --2- 172.21.15.5:0/2810655256 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6714161200 0x7f67140b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 -- 172.21.15.5:0/2810655256 >> 172.21.15.5:0/2810655256 conn(0x7f671408af80 msgr2=0x7f67140d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 -- 172.21.15.5:0/2810655256 shutdown_connections 2024-04-25T10:04:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.102+0000 7f6713200640 1 -- 172.21.15.5:0/2810655256 wait complete. 2024-04-25T10:04:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.104+0000 7f6713200640 1 Processor -- start 2024-04-25T10:04:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.104+0000 7f6713200640 1 -- start start 2024-04-25T10:04:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.105+0000 7f6713200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67140d0600 0x7f67140cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.105+0000 7f6713200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.105+0000 7f6713200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 0x7f67141fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.105+0000 7f6713200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140b3f90 con 0x7f6714161200 2024-04-25T10:04:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.105+0000 7f6713200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140d42b0 con 0x7f67140d2380 2024-04-25T10:04:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.105+0000 7f6713200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67140b4110 con 0x7f67140d0600 2024-04-25T10:04:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 0x7f67141fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 0x7f67141fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47062/0 (socket says 172.21.15.5:47062) 2024-04-25T10:04:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 -- 172.21.15.5:0/1622234196 learned_addr learned my addr 172.21.15.5:0/1622234196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6711e00640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67140d0600 0x7f67140cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6711400640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 -- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67140d0600 msgr2=0x7f67140cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67140d0600 0x7f67140cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 -- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 msgr2=0x7f67140b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140b8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6712800640 1 -- 172.21.15.5:0/1622234196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140a1020 con 0x7f6714161200 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.106+0000 7f6711e00640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67140d0600 0x7f67140cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.107+0000 7f6712800640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 0x7f67141fbac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f670c00c390 tx=0x7f670c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.107+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c017070 con 0x7f6714161200 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.107+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c0052d0 con 0x7f6714161200 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.107+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c003810 con 0x7f6714161200 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.107+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67140a1300 con 0x7f6714161200 2024-04-25T10:04:26.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.107+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67140ccc80 con 0x7f6714161200 2024-04-25T10:04:26.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.108+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714068f60 con 0x7f6714161200 2024-04-25T10:04:26.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.109+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f670c01a070 con 0x7f6714161200 2024-04-25T10:04:26.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.110+0000 7f670a000640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66f006b4d0 0x7f66f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.110+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f670c0896e0 con 0x7f6714161200 2024-04-25T10:04:26.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.111+0000 7f6711e00640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66f006b4d0 0x7f66f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.112+0000 7f6711e00640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66f006b4d0 0x7f66f006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6704007440 tx=0x7f670400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.114+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f670c0583c0 con 0x7f6714161200 2024-04-25T10:04:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.316+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67140d2f00 con 0x7f6714161200 2024-04-25T10:04:26.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.319+0000 7f670a000640 1 -- 172.21.15.5:0/1622234196 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f670c057cc0 con 0x7f6714161200 2024-04-25T10:04:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:26.325 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:24.928022+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.321+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66f006b4d0 msgr2=0x7f66f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.321+0000 7f6713200640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66f006b4d0 0x7f66f006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6704007440 tx=0x7f670400b040 comp rx=0 tx=0).stop 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 msgr2=0x7f67141fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 0x7f67141fbac0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f670c00c390 tx=0x7f670c00c850 comp rx=0 tx=0).stop 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 shutdown_connections 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6714161200 0x7f67141fbac0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66f006b4d0 0x7f66f006d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67140d2380 0x7f67140b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 --2- 172.21.15.5:0/1622234196 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67140d0600 0x7f67140cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 >> 172.21.15.5:0/1622234196 conn(0x7f671408af80 msgr2=0x7f67140b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 shutdown_connections 2024-04-25T10:04:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.322+0000 7f6713200640 1 -- 172.21.15.5:0/1622234196 wait complete. 2024-04-25T10:04:26.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:04:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.556+0000 7fa6fce00640 1 Processor -- start 2024-04-25T10:04:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.556+0000 7fa6fce00640 1 -- start start 2024-04-25T10:04:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.557+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6f815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.557+0000 7fa6fce00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f81606e0 con 0x7fa6f80d3f90 2024-04-25T10:04:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6fce00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f81609c0 con 0x7fa6f8161d90 2024-04-25T10:04:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6fce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8160ca0 con 0x7fa6f80b48c0 2024-04-25T10:04:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6eea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa6f80d3f90 0x7fa6f8161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44802/0 (socket says 172.21.15.5:44802) 2024-04-25T10:04:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6eea00640 1 -- 172.21.15.5:0/1176653078 learned_addr learned my addr 172.21.15.5:0/1176653078 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6f7400640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6f815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686777694 0 0) 0x7fa6f81606e0 con 0x7fa6f80d3f90 2024-04-25T10:04:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.558+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6dc003520 con 0x7fa6f80d3f90 2024-04-25T10:04:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491815001 0 0) 0x7fa6dc003520 con 0x7fa6f80d3f90 2024-04-25T10:04:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 msgr2=0x7fa6f815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6f815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 >> v1:172.21.15.80:6789/0 conn(0x7fa6f8161d90 legacy=0x7fa6f80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8160f80 con 0x7fa6f80d3f90 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa6e0001b70 con 0x7fa6f80d3f90 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 >> v1:172.21.15.5:6789/0 conn(0x7fa6f80d3f90 legacy=0x7fa6f8161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6dc0039e0 0x7fa6dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6dc005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8160ca0 con 0x7fa6dc004800 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc00bb50 con 0x7fa6dc0039e0 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc00bd50 con 0x7fa6f80b48c0 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.559+0000 7fa6eea00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6f7400640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6dc0039e0 0x7fa6dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6eea00640 1 -- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 msgr2=0x7fa6dc005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6f7e00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6dc005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6eea00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6dc005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6eea00640 1 -- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6dc0039e0 msgr2=0x7fa6dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6eea00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6dc0039e0 0x7fa6dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6eea00640 1 -- 172.21.15.5:0/1176653078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc003520 con 0x7fa6dc004800 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6f7e00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b48c0 0x7fa6dc005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.560+0000 7fa6f7400640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6dc0039e0 0x7fa6dc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.561+0000 7fa6eea00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6dc004c80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c540 tx=0x7fa6dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf6a2bb2093779c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.561+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0032020 con 0x7fa6dc004800 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.561+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e0005d40 con 0x7fa6dc004800 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.561+0000 7fa6f6a00640 1 -- 172.21.15.5:0/1176653078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e000c0e0 con 0x7fa6dc004800 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6fce00640 1 -- 172.21.15.5:0/1176653078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 msgr2=0x7fa6dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6fce00640 1 --2- 172.21.15.5:0/1176653078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6dc004c80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c540 tx=0x7fa6dc005830 comp rx=0 tx=0).stop 2024-04-25T10:04:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6f7e00640 1 -- 172.21.15.5:0/1176653078 reap_dead start 2024-04-25T10:04:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6fce00640 1 -- 172.21.15.5:0/1176653078 shutdown_connections 2024-04-25T10:04:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6fce00640 1 -- 172.21.15.5:0/1176653078 >> 172.21.15.5:0/1176653078 conn(0x7fa6f808af80 msgr2=0x7fa6f80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6fce00640 1 -- 172.21.15.5:0/1176653078 shutdown_connections 2024-04-25T10:04:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.562+0000 7fa6fce00640 1 -- 172.21.15.5:0/1176653078 wait complete. 2024-04-25T10:04:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.564+0000 7fa6fce00640 1 Processor -- start 2024-04-25T10:04:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.564+0000 7fa6fce00640 1 -- start start 2024-04-25T10:04:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6f80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f80b6830 0x7fa6f80b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6fce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b6cb0 0x7fa6f80b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6fce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815fe60 con 0x7fa6dc004800 2024-04-25T10:04:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6fce00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b4ca0 con 0x7fa6f80b6830 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6fce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f815ffe0 con 0x7fa6f80b6cb0 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6f7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6f80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6f7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6f80cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47066/0 (socket says 172.21.15.5:47066) 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.565+0000 7fa6f7400640 1 -- 172.21.15.5:0/3072862906 learned_addr learned my addr 172.21.15.5:0/3072862906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6eea00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f80b6830 0x7fa6f80b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6f7400640 1 -- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b6cb0 msgr2=0x7fa6f80b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6f7400640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b6cb0 0x7fa6f80b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6f7400640 1 -- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f80b6830 msgr2=0x7fa6f80b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6f7400640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f80b6830 0x7fa6f80b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6f7400640 1 -- 172.21.15.5:0/3072862906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80cc2a0 con 0x7fa6dc004800 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6f7400640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6f80cd050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80108e0 tx=0x7fa6e8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.566+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800f040 con 0x7fa6dc004800 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.567+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e800ec20 con 0x7fa6dc004800 2024-04-25T10:04:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.567+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e801c400 con 0x7fa6dc004800 2024-04-25T10:04:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.567+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f80cc580 con 0x7fa6dc004800 2024-04-25T10:04:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.567+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f815c650 con 0x7fa6dc004800 2024-04-25T10:04:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.568+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8068f60 con 0x7fa6dc004800 2024-04-25T10:04:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.569+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6e801c5a0 con 0x7fa6dc004800 2024-04-25T10:04:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.570+0000 7fa6efe00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6d406b4d0 0x7fa6d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.570+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6e808ab60 con 0x7fa6dc004800 2024-04-25T10:04:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.570+0000 7fa6eea00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6d406b4d0 0x7fa6d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.572+0000 7fa6eea00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6d406b4d0 0x7fa6d406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c540 tx=0x7fa6e0009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.574+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6e8018030 con 0x7fa6dc004800 2024-04-25T10:04:26.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:26.811+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa6f808d730 con 0x7fa6dc004800 2024-04-25T10:04:27.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.483+0000 7fa6efe00640 1 -- 172.21.15.5:0/3072862906 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa6e805f030 con 0x7fa6dc004800 2024-04-25T10:04:27.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.486+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6d406b4d0 msgr2=0x7fa6d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6d406b4d0 0x7fa6d406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa6e000c540 tx=0x7fa6e0009030 comp rx=0 tx=0).stop 2024-04-25T10:04:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 msgr2=0x7fa6f80cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6f80cd050 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80108e0 tx=0x7fa6e8010da0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 shutdown_connections 2024-04-25T10:04:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f80b6cb0 0x7fa6f80b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa6d406b4d0 0x7fa6d406d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f80b6830 0x7fa6f80b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 --2- 172.21.15.5:0/3072862906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6dc004800 0x7fa6f80cd050 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 >> 172.21.15.5:0/3072862906 conn(0x7fa6f808af80 msgr2=0x7fa6f80b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 shutdown_connections 2024-04-25T10:04:27.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.487+0000 7fa6fce00640 1 -- 172.21.15.5:0/3072862906 wait complete. 2024-04-25T10:04:27.518 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:04:27.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:04:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.729+0000 7f9ff4400640 1 Processor -- start 2024-04-25T10:04:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.729+0000 7f9ff4400640 1 -- start start 2024-04-25T10:04:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff4400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff4400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9fec071840 0x7f9fec15c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff4400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fec1453c0 con 0x7f9fec06d130 2024-04-25T10:04:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff4400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec0a8560 con 0x7f9fec071840 2024-04-25T10:04:27.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff4400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec06d6a0 con 0x7f9fec144fe0 2024-04-25T10:04:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff1000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9fec06d130 0x7f9fec071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60530/0 (socket says 172.21.15.5:60530) 2024-04-25T10:04:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff1000640 1 -- 172.21.15.5:0/778138577 learned_addr learned my addr 172.21.15.5:0/778138577 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.731+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/778138577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9ff2400640 1 --2- 172.21.15.5:0/778138577 >> v2:172.21.15.80:3300/0 conn(0x7f9fec071840 0x7f9fec15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9febe00640 1 -- 172.21.15.5:0/778138577 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556886129 0 0) 0x7f9fec1453c0 con 0x7f9fec06d130 2024-04-25T10:04:27.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9febe00640 1 -- 172.21.15.5:0/778138577 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd4003520 con 0x7f9fec06d130 2024-04-25T10:04:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9febe00640 1 -- 172.21.15.5:0/778138577 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3300272491 0 0) 0x7f9fd4003520 con 0x7f9fec06d130 2024-04-25T10:04:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9ff1a00640 1 -- 172.21.15.5:0/778138577 >> v2:172.21.15.80:3300/0 conn(0x7f9fec071840 msgr2=0x7f9fec15c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/778138577 >> v2:172.21.15.80:3300/0 conn(0x7f9fec071840 0x7f9fec15c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9ff1a00640 1 -- 172.21.15.5:0/778138577 >> v1:172.21.15.5:6789/0 conn(0x7f9fec06d130 legacy=0x7f9fec071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.732+0000 7f9ff1a00640 1 -- 172.21.15.5:0/778138577 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec06d980 con 0x7f9fec144fe0 2024-04-25T10:04:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.734+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/778138577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec06e560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00097a0 tx=0x7f9fe002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64942803e34f0aec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.734+0000 7f9febe00640 1 -- 172.21.15.5:0/778138577 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0037070 con 0x7f9fec144fe0 2024-04-25T10:04:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.734+0000 7f9febe00640 1 -- 172.21.15.5:0/778138577 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe002aba0 con 0x7f9fec144fe0 2024-04-25T10:04:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.734+0000 7f9febe00640 1 -- 172.21.15.5:0/778138577 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8001670 con 0x7f9fec144fe0 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 -- 172.21.15.5:0/778138577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 msgr2=0x7f9fec06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 --2- 172.21.15.5:0/778138577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec06e560 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00097a0 tx=0x7f9fe002a110 comp rx=0 tx=0).stop 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 -- 172.21.15.5:0/778138577 shutdown_connections 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 --2- 172.21.15.5:0/778138577 >> v2:172.21.15.80:3300/0 conn(0x7f9fec071840 0x7f9fec15c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 --2- 172.21.15.5:0/778138577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec06e560 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 -- 172.21.15.5:0/778138577 >> 172.21.15.5:0/778138577 conn(0x7f9fec06ab60 msgr2=0x7f9fec06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 -- 172.21.15.5:0/778138577 shutdown_connections 2024-04-25T10:04:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.735+0000 7f9ff4400640 1 -- 172.21.15.5:0/778138577 wait complete. 2024-04-25T10:04:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.737+0000 7f9ff4400640 1 Processor -- start 2024-04-25T10:04:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.737+0000 7f9ff4400640 1 -- start start 2024-04-25T10:04:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff4400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fec06d130 0x7f9fec15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff4400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 0x7f9fec0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff4400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff4400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec15be50 con 0x7f9fec071840 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff4400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec06df40 con 0x7f9fec06d130 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff4400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec06eaa0 con 0x7f9fec144fe0 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 0x7f9fec0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 0x7f9fec0b0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57634/0 (socket says 172.21.15.5:57634) 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff1000640 1 -- 172.21.15.5:0/3597591374 learned_addr learned my addr 172.21.15.5:0/3597591374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fec06d130 0x7f9fec15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.738+0000 7f9ff2400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1000640 1 -- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 msgr2=0x7f9fec0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1000640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec0b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1000640 1 -- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fec06d130 msgr2=0x7f9fec15b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1000640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fec06d130 0x7f9fec15b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1000640 1 -- 172.21.15.5:0/3597591374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec13fd40 con 0x7f9fec071840 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fec06d130 0x7f9fec15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff1000640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 0x7f9fec0b0890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c7b0 tx=0x7f9fd800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8017070 con 0x7f9fec071840 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd8005ad0 con 0x7f9fec071840 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec140020 con 0x7f9fec071840 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007070 con 0x7f9fec071840 2024-04-25T10:04:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.739+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1405e0 con 0x7f9fec071840 2024-04-25T10:04:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.741+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec068f60 con 0x7f9fec071840 2024-04-25T10:04:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.741+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fd801a070 con 0x7f9fec071840 2024-04-25T10:04:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.742+0000 7f9fe9600640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.742+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9fd80898e0 con 0x7f9fec071840 2024-04-25T10:04:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.743+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.744+0000 7f9ff1a00640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0034000 tx=0x7f9fe002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.747+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fd80585d0 con 0x7f9fec071840 2024-04-25T10:04:27.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:27.990+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9fec0b2750 con 0x7f9fec071840 2024-04-25T10:04:28.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.494+0000 7f9fe9600640 1 -- 172.21.15.5:0/3597591374 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9fd8057ed0 con 0x7f9fec071840 2024-04-25T10:04:28.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.497+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9fcc06b4d0 msgr2=0x7f9fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.497+0000 7f9ff4400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0034000 tx=0x7f9fe002e000 comp rx=0 tx=0).stop 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.497+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 msgr2=0x7f9fec0b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.497+0000 7f9ff4400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 0x7f9fec0b0890 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800c7b0 tx=0x7f9fd800cc70 comp rx=0 tx=0).stop 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 shutdown_connections 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fec144fe0 0x7f9fec0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fec071840 0x7f9fec0b0890 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9fcc06b4d0 0x7f9fcc06d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 --2- 172.21.15.5:0/3597591374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fec06d130 0x7f9fec15b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 >> 172.21.15.5:0/3597591374 conn(0x7f9fec06ab60 msgr2=0x7f9fec0adf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 shutdown_connections 2024-04-25T10:04:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.498+0000 7f9ff4400640 1 -- 172.21.15.5:0/3597591374 wait complete. 2024-04-25T10:04:28.525 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:04:28.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.724+0000 7f18e2a00640 1 Processor -- start 2024-04-25T10:04:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.725+0000 7f18e2a00640 1 -- start start 2024-04-25T10:04:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.726+0000 7f18e2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18dc161200 0x7f18dc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.726+0000 7f18e2a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0cf410 con 0x7f18dc0d07d0 2024-04-25T10:04:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.727+0000 7f18e2a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0cf6f0 con 0x7f18dc0d2380 2024-04-25T10:04:28.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.727+0000 7f18e2a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0d27c0 con 0x7f18dc161200 2024-04-25T10:04:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.727+0000 7f18e2000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f18dc0d07d0 0x7f18dc0b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60540/0 (socket says 172.21.15.5:60540) 2024-04-25T10:04:28.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.727+0000 7f18e2000640 1 -- 172.21.15.5:0/2526782601 learned_addr learned my addr 172.21.15.5:0/2526782601 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.727+0000 7f18e1600640 1 --2- 172.21.15.5:0/2526782601 >> v2:172.21.15.5:3300/0 conn(0x7f18dc161200 0x7f18dc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.728+0000 7f18dbe00640 1 -- 172.21.15.5:0/2526782601 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389823580 0 0) 0x7f18dc0cf410 con 0x7f18dc0d07d0 2024-04-25T10:04:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.728+0000 7f18dbe00640 1 -- 172.21.15.5:0/2526782601 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c4003520 con 0x7f18dc0d07d0 2024-04-25T10:04:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.728+0000 7f18e1600640 1 -- 172.21.15.5:0/2526782601 >> v1:172.21.15.80:6789/0 conn(0x7f18dc0d2380 legacy=0x7f18dc0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.728+0000 7f18e1600640 1 -- 172.21.15.5:0/2526782601 >> v1:172.21.15.5:6789/0 conn(0x7f18dc0d07d0 legacy=0x7f18dc0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.728+0000 7f18e1600640 1 -- 172.21.15.5:0/2526782601 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0d3fd0 con 0x7f18dc161200 2024-04-25T10:04:28.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.729+0000 7f18e1600640 1 --2- 172.21.15.5:0/2526782601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc161200 0x7f18dc0b4bb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f18d00097a0 tx=0x7f18d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1692d7b7ab1118d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.730+0000 7f18dbe00640 1 -- 172.21.15.5:0/2526782601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0037070 con 0x7f18dc161200 2024-04-25T10:04:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.730+0000 7f18dbe00640 1 -- 172.21.15.5:0/2526782601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0032440 con 0x7f18dc161200 2024-04-25T10:04:28.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.730+0000 7f18dbe00640 1 -- 172.21.15.5:0/2526782601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0032810 con 0x7f18dc161200 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 -- 172.21.15.5:0/2526782601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc161200 msgr2=0x7f18dc0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 --2- 172.21.15.5:0/2526782601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc161200 0x7f18dc0b4bb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f18d00097a0 tx=0x7f18d002a230 comp rx=0 tx=0).stop 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 -- 172.21.15.5:0/2526782601 shutdown_connections 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 --2- 172.21.15.5:0/2526782601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc161200 0x7f18dc0b4bb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 -- 172.21.15.5:0/2526782601 >> 172.21.15.5:0/2526782601 conn(0x7f18dc08af80 msgr2=0x7f18dc0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 -- 172.21.15.5:0/2526782601 shutdown_connections 2024-04-25T10:04:28.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.731+0000 7f18e2a00640 1 -- 172.21.15.5:0/2526782601 wait complete. 2024-04-25T10:04:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.732+0000 7f18e2a00640 1 Processor -- start 2024-04-25T10:04:28.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.733+0000 7f18e2a00640 1 -- start start 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 0x7f18dc14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e2a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc0d2380 0x7f18dc14c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18dc161200 0x7f18dc14c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e2a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b3690 con 0x7f18dc0d07d0 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e2a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b28b0 con 0x7f18dc0d2380 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e2a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b3810 con 0x7f18dc161200 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 0x7f18dc14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 0x7f18dc14fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57654/0 (socket says 172.21.15.5:57654) 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e1600640 1 -- 172.21.15.5:0/1646979467 learned_addr learned my addr 172.21.15.5:0/1646979467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e0c00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc0d2380 0x7f18dc14c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.734+0000 7f18e1600640 1 -- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18dc161200 msgr2=0x7f18dc14c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e1600640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18dc161200 0x7f18dc14c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e1600640 1 -- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc0d2380 msgr2=0x7f18dc14c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e1600640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc0d2380 0x7f18dc14c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e1600640 1 -- 172.21.15.5:0/1646979467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0cc0b0 con 0x7f18dc0d07d0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e1600640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 0x7f18dc14fa10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f18d00098c0 tx=0x7f18d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0037040 con 0x7f18dc0d07d0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0030500 con 0x7f18dc0d07d0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00308d0 con 0x7f18dc0d07d0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc0cb530 con 0x7f18dc0d07d0 2024-04-25T10:04:28.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.735+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc0cbaf0 con 0x7f18dc0d07d0 2024-04-25T10:04:28.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.737+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc0cfe60 con 0x7f18dc0d07d0 2024-04-25T10:04:28.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.738+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18d0044020 con 0x7f18dc0d07d0 2024-04-25T10:04:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.740+0000 7f18d9600640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f18c006b4d0 0x7f18c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.740+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18d00aa730 con 0x7f18dc0d07d0 2024-04-25T10:04:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.741+0000 7f18e0c00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f18c006b4d0 0x7f18c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.742+0000 7f18e0c00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f18c006b4d0 0x7f18c006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18cc007fd0 tx=0x7f18cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.745+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18dc0cfe60 con 0x7f18dc0d07d0 2024-04-25T10:04:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.984+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18dc160a20 con 0x7f18dc0d07d0 2024-04-25T10:04:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.985+0000 7f18d9600640 1 -- 172.21.15.5:0/1646979467 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f18d0079380 con 0x7f18dc0d07d0 2024-04-25T10:04:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:28.992 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:04:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f18c006b4d0 msgr2=0x7f18c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f18c006b4d0 0x7f18c006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18cc007fd0 tx=0x7f18cc009040 comp rx=0 tx=0).stop 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 msgr2=0x7f18dc14fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 0x7f18dc14fa10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f18d00098c0 tx=0x7f18d00046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 shutdown_connections 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18dc161200 0x7f18dc14c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f18c006b4d0 0x7f18c006d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18dc0d2380 0x7f18dc14c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 --2- 172.21.15.5:0/1646979467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18dc0d07d0 0x7f18dc14fa10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 >> 172.21.15.5:0/1646979467 conn(0x7f18dc08af80 msgr2=0x7f18dc0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 shutdown_connections 2024-04-25T10:04:28.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:28.989+0000 7f18e2a00640 1 -- 172.21.15.5:0/1646979467 wait complete. 2024-04-25T10:04:29.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.222+0000 7f5cc5600640 1 Processor -- start 2024-04-25T10:04:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.222+0000 7f5cc5600640 1 -- start start 2024-04-25T10:04:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.223+0000 7f5cc5600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5cc0161200 0x7f5cc00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.223+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d2380 0x7f5cc00d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.223+0000 7f5cc5600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5cc00d0600 0x7f5cc00ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.223+0000 7f5cc5600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00d27c0 con 0x7f5cc0161200 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.223+0000 7f5cc5600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00b28b0 con 0x7f5cc00d0600 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.223+0000 7f5cc5600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00cf410 con 0x7f5cc00d2380 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5cc0161200 0x7f5cc00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5cc0161200 0x7f5cc00b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57668/0 (socket says 172.21.15.5:57668) 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 -- 172.21.15.5:0/1602140390 learned_addr learned my addr 172.21.15.5:0/1602140390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbf400640 1 --2- 172.21.15.5:0/1602140390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d2380 0x7f5cc00d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cc4c00640 1 --2- 172.21.15.5:0/1602140390 >> v2:172.21.15.80:3300/0 conn(0x7f5cc00d0600 0x7f5cc00ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 -- 172.21.15.5:0/1602140390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d2380 msgr2=0x7f5cc00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 --2- 172.21.15.5:0/1602140390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d2380 0x7f5cc00d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 -- 172.21.15.5:0/1602140390 >> v2:172.21.15.80:3300/0 conn(0x7f5cc00d0600 msgr2=0x7f5cc00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 --2- 172.21.15.5:0/1602140390 >> v2:172.21.15.80:3300/0 conn(0x7f5cc00d0600 0x7f5cc00ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.224+0000 7f5cbfe00640 1 -- 172.21.15.5:0/1602140390 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc00cf6f0 con 0x7f5cc0161200 2024-04-25T10:04:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.226+0000 7f5cbfe00640 1 --2- 172.21.15.5:0/1602140390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc00b4bb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00b4a80 tx=0x7f5cb002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bbf280bf6fcac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.226+0000 7f5cbea00640 1 -- 172.21.15.5:0/1602140390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0037070 con 0x7f5cc0161200 2024-04-25T10:04:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.226+0000 7f5cbea00640 1 -- 172.21.15.5:0/1602140390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb0032440 con 0x7f5cc0161200 2024-04-25T10:04:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.226+0000 7f5cbea00640 1 -- 172.21.15.5:0/1602140390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0032810 con 0x7f5cc0161200 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 -- 172.21.15.5:0/1602140390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 msgr2=0x7f5cc00b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 --2- 172.21.15.5:0/1602140390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc00b4bb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00b4a80 tx=0x7f5cb002a210 comp rx=0 tx=0).stop 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 -- 172.21.15.5:0/1602140390 shutdown_connections 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 --2- 172.21.15.5:0/1602140390 >> v2:172.21.15.80:3300/0 conn(0x7f5cc00d0600 0x7f5cc00ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 --2- 172.21.15.5:0/1602140390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d2380 0x7f5cc00d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 --2- 172.21.15.5:0/1602140390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc00b4bb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 -- 172.21.15.5:0/1602140390 >> 172.21.15.5:0/1602140390 conn(0x7f5cc008af80 msgr2=0x7f5cc00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 -- 172.21.15.5:0/1602140390 shutdown_connections 2024-04-25T10:04:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.227+0000 7f5cc5600640 1 -- 172.21.15.5:0/1602140390 wait complete. 2024-04-25T10:04:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.230+0000 7f5cc5600640 1 Processor -- start 2024-04-25T10:04:29.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.230+0000 7f5cc5600640 1 -- start start 2024-04-25T10:04:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d0600 0x7f5cc009f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc00d2380 0x7f5cc00a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc009d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc5600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00ce7c0 con 0x7f5cc0161200 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc5600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00d42b0 con 0x7f5cc00d2380 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc5600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00ce940 con 0x7f5cc00d0600 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.231+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc009d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc009d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57682/0 (socket says 172.21.15.5:57682) 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 -- 172.21.15.5:0/2740119659 learned_addr learned my addr 172.21.15.5:0/2740119659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cbfe00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d0600 0x7f5cc009f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cbf400640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc00d2380 0x7f5cc00a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 -- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d0600 msgr2=0x7f5cc009f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d0600 0x7f5cc009f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 -- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc00d2380 msgr2=0x7f5cc00a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc00d2380 0x7f5cc00a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 -- 172.21.15.5:0/2740119659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc009d910 con 0x7f5cc0161200 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.232+0000 7f5cc4c00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc009d3d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800c5a0 tx=0x7f5cb800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.233+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8015070 con 0x7f5cc0161200 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.233+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8004220 con 0x7f5cc0161200 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.233+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc009ff70 con 0x7f5cc0161200 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.233+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8003610 con 0x7f5cc0161200 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.233+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc00a0530 con 0x7f5cc0161200 2024-04-25T10:04:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.234+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc0068f60 con 0x7f5cc0161200 2024-04-25T10:04:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.235+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cb80037b0 con 0x7f5cc0161200 2024-04-25T10:04:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.236+0000 7f5cb7e00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.237+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5cb808a230 con 0x7f5cc0161200 2024-04-25T10:04:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.237+0000 7f5cbfe00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.239+0000 7f5cbfe00640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0034000 tx=0x7f5cb002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.241+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cb8058e90 con 0x7f5cc0161200 2024-04-25T10:04:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.447+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cc006d4d0 con 0x7f5cc0161200 2024-04-25T10:04:29.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.449+0000 7f5cb7e00640 1 -- 172.21.15.5:0/2740119659 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f5cb8058790 con 0x7f5cc0161200 2024-04-25T10:04:29.456 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:29.456 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:26.455238+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.452+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c9c06b4d0 msgr2=0x7f5c9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.452+0000 7f5cc5600640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0034000 tx=0x7f5cb002e000 comp rx=0 tx=0).stop 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.452+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 msgr2=0x7f5cc009d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.452+0000 7f5cc5600640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc009d3d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800c5a0 tx=0x7f5cb800ca60 comp rx=0 tx=0).stop 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 shutdown_connections 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc0161200 0x7f5cc009d3d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cc00d2380 0x7f5cc00a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 --2- 172.21.15.5:0/2740119659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cc00d0600 0x7f5cc009f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 >> 172.21.15.5:0/2740119659 conn(0x7f5cc008af80 msgr2=0x7f5cc01604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 shutdown_connections 2024-04-25T10:04:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.453+0000 7f5cc5600640 1 -- 172.21.15.5:0/2740119659 wait complete. 2024-04-25T10:04:29.477 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:04:29.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.676+0000 7fa0bbe00640 1 Processor -- start 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.677+0000 7fa0bbe00640 1 -- start start 2024-04-25T10:04:29.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.678+0000 7fa0bbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa0bc0d07d0 0x7fa0bc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.678+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0bc0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.678+0000 7fa0bbe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc0cf410 con 0x7fa0bc161200 2024-04-25T10:04:29.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.678+0000 7fa0bbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b28b0 con 0x7fa0bc0d07d0 2024-04-25T10:04:29.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.678+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0cf6f0 con 0x7fa0bc0d2380 2024-04-25T10:04:29.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.679+0000 7fa0baa00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa0bc161200 0x7fa0bc0d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60550/0 (socket says 172.21.15.5:60550) 2024-04-25T10:04:29.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.679+0000 7fa0baa00640 1 -- 172.21.15.5:0/2327768624 learned_addr learned my addr 172.21.15.5:0/2327768624 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.679+0000 7fa0bb400640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0bc0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.679+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352177543 0 0) 0x7fa0bc0cf410 con 0x7fa0bc161200 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.679+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2327768624 >> v2:172.21.15.80:3300/0 conn(0x7fa0bc0d07d0 0x7fa0bc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.679+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a0003520 con 0x7fa0bc161200 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250027961 0 0) 0x7fa0a0003520 con 0x7fa0bc161200 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 msgr2=0x7fa0bc0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0bc0b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 >> v2:172.21.15.80:3300/0 conn(0x7fa0bc0d07d0 msgr2=0x7fa0bc0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 --2- 172.21.15.5:0/2327768624 >> v2:172.21.15.80:3300/0 conn(0x7fa0bc0d07d0 0x7fa0bc0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0d3fd0 con 0x7fa0bc161200 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa0ac001670 con 0x7fa0bc161200 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 >> v1:172.21.15.5:6789/0 conn(0x7fa0bc161200 legacy=0x7fa0bc0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b28b0 con 0x7fa0a0004800 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0cf6f0 con 0x7fa0a00039b0 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a000b9c0 con 0x7fa0bc0d2380 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.680+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0bb400640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0baa00640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0ba000640 1 -- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 msgr2=0x7fa0a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0ba000640 1 -- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0a00039b0 msgr2=0x7fa0a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0ba000640 1 -- 172.21.15.5:0/2327768624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a0003520 con 0x7fa0a0004800 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0bb400640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc0d2380 0x7fa0a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.681+0000 7fa0baa00640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0a00039b0 0x7fa0a0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.682+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009ac0 tx=0x7fa0a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c007a855ed5dbead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.682+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8036070 con 0x7fa0a0004800 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.682+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8031400 con 0x7fa0a0004800 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.682+0000 7fa0b9600640 1 -- 172.21.15.5:0/2327768624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8009690 con 0x7fa0a0004800 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2327768624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 msgr2=0x7fa0a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2327768624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0a0004c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009ac0 tx=0x7fa0a0005830 comp rx=0 tx=0).stop 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bb400640 1 -- 172.21.15.5:0/2327768624 reap_dead start 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2327768624 shutdown_connections 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2327768624 >> 172.21.15.5:0/2327768624 conn(0x7fa0bc08af80 msgr2=0x7fa0bc0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2327768624 shutdown_connections 2024-04-25T10:04:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.683+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2327768624 wait complete. 2024-04-25T10:04:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.685+0000 7fa0bbe00640 1 Processor -- start 2024-04-25T10:04:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.685+0000 7fa0bbe00640 1 -- start start 2024-04-25T10:04:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0bc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc15eec0 0x7fa0bc15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0bbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0bc0ca7f0 0x7fa0bc0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b3100 con 0x7fa0a0004800 2024-04-25T10:04:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b2f80 con 0x7fa0bc0ca7f0 2024-04-25T10:04:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b3280 con 0x7fa0bc15eec0 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0bc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0bc15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57702/0 (socket says 172.21.15.5:57702) 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.686+0000 7fa0baa00640 1 -- 172.21.15.5:0/2626136603 learned_addr learned my addr 172.21.15.5:0/2626136603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc15eec0 0x7fa0bc15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0bb400640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0bc0ca7f0 0x7fa0bc0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0baa00640 1 -- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc15eec0 msgr2=0x7fa0bc15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0baa00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc15eec0 0x7fa0bc15f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0baa00640 1 -- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0bc0ca7f0 msgr2=0x7fa0bc0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0baa00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0bc0ca7f0 0x7fa0bc0cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0baa00640 1 -- 172.21.15.5:0/2626136603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0cb520 con 0x7fa0a0004800 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0bb400640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0bc0ca7f0 0x7fa0bc0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0baa00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0bc15ac50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00d890 tx=0x7fa0ac00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.687+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac019070 con 0x7fa0a0004800 2024-04-25T10:04:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.688+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac0074e0 con 0x7fa0a0004800 2024-04-25T10:04:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.688+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc0b6520 con 0x7fa0a0004800 2024-04-25T10:04:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.688+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac0049e0 con 0x7fa0a0004800 2024-04-25T10:04:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.688+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc0b6ae0 con 0x7fa0a0004800 2024-04-25T10:04:29.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.689+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc068f60 con 0x7fa0a0004800 2024-04-25T10:04:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.690+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0ac01c070 con 0x7fa0a0004800 2024-04-25T10:04:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.691+0000 7fa0b2a00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09c06b4d0 0x7fa09c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.691+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0ac089ea0 con 0x7fa0a0004800 2024-04-25T10:04:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.692+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09c06b4d0 0x7fa09c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.693+0000 7fa0ba000640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09c06b4d0 0x7fa09c06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8033000 tx=0x7fa0a8008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.695+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0ac058b80 con 0x7fa0a0004800 2024-04-25T10:04:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.895+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0bc15b190 con 0x7fa09c06b4d0 2024-04-25T10:04:29.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.914+0000 7fa0b2a00640 1 -- 172.21.15.5:0/2626136603 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251073 (secure 0 0 0) 0x7fa0a8037040 con 0x7fa09c06b4d0 2024-04-25T10:04:29.924 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:29.925 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:04:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09c06b4d0 msgr2=0x7fa09c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09c06b4d0 0x7fa09c06d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8033000 tx=0x7fa0a8008030 comp rx=0 tx=0).stop 2024-04-25T10:04:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 msgr2=0x7fa0bc15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0bc15ac50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00d890 tx=0x7fa0ac00dd50 comp rx=0 tx=0).stop 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 shutdown_connections 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09c06b4d0 0x7fa09c06d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0bc0ca7f0 0x7fa0bc0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0bc15eec0 0x7fa0bc15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 --2- 172.21.15.5:0/2626136603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0004800 0x7fa0bc15ac50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.919+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 >> 172.21.15.5:0/2626136603 conn(0x7fa0bc08af80 msgr2=0x7fa0bc0b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.920+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 shutdown_connections 2024-04-25T10:04:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:29.920+0000 7fa0bbe00640 1 -- 172.21.15.5:0/2626136603 wait complete. 2024-04-25T10:04:29.946 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-04-25T10:04:28.665343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238456,"kb_used_data":5628,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736264,"statfs":{"total":773094113280,"available":772849934336,"internally_reserved":0,"allocated":5763072,"data_stored":3159309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252602"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471647+0000","last_change":"2024-04-25T10:04:25.461883+0000","last_active":"2024-04-25T10:04:25.471647+0000","last_peered":"2024-04-25T10:04:25.471647+0000","last_clean":"2024-04-25T10:04:25.471647+0000","last_became_active":"2024-04-25T10:04:25.460987+0000","last_became_peered":"2024-04-25T10:04:25.460987+0000","last_unstale":"2024-04-25T10:04:25.471647+0000","last_undegraded":"2024-04-25T10:04:25.471647+0000","last_fullsized":"2024-04-25T10:04:25.471647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479215+0000","last_change":"2024-04-25T10:04:25.474565+0000","last_active":"2024-04-25T10:04:25.479215+0000","last_peered":"2024-04-25T10:04:25.479215+0000","last_clean":"2024-04-25T10:04:25.479215+0000","last_became_active":"2024-04-25T10:04:25.474282+0000","last_became_peered":"2024-04-25T10:04:25.474282+0000","last_unstale":"2024-04-25T10:04:25.479215+0000","last_undegraded":"2024-04-25T10:04:25.479215+0000","last_fullsized":"2024-04-25T10:04:25.479215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.461950+0000","last_change":"2024-04-25T10:04:24.461950+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.461950+0000","last_undegraded":"2024-04-25T10:04:24.461950+0000","last_fullsized":"2024-04-25T10:04:24.461950+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.459057+0000","last_change":"2024-04-25T10:04:25.454040+0000","last_active":"2024-04-25T10:04:25.459057+0000","last_peered":"2024-04-25T10:04:25.459057+0000","last_clean":"2024-04-25T10:04:25.459057+0000","last_became_active":"2024-04-25T10:04:25.453680+0000","last_became_peered":"2024-04-25T10:04:25.453680+0000","last_unstale":"2024-04-25T10:04:25.459057+0000","last_undegraded":"2024-04-25T10:04:25.459057+0000","last_fullsized":"2024-04-25T10:04:25.459057+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.470460+0000","last_change":"2024-04-25T10:04:25.468526+0000","last_active":"2024-04-25T10:04:25.470460+0000","last_peered":"2024-04-25T10:04:25.470460+0000","last_clean":"2024-04-25T10:04:25.470460+0000","last_became_active":"2024-04-25T10:04:25.468181+0000","last_became_peered":"2024-04-25T10:04:25.468181+0000","last_unstale":"2024-04-25T10:04:25.470460+0000","last_undegraded":"2024-04-25T10:04:25.470460+0000","last_fullsized":"2024-04-25T10:04:25.470460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.458948+0000","last_change":"2024-04-25T10:04:25.448768+0000","last_active":"2024-04-25T10:04:25.458948+0000","last_peered":"2024-04-25T10:04:25.458948+0000","last_clean":"2024-04-25T10:04:25.458948+0000","last_became_active":"2024-04-25T10:04:25.448208+0000","last_became_peered":"2024-04-25T10:04:25.448208+0000","last_unstale":"2024-04-25T10:04:25.458948+0000","last_undegraded":"2024-04-25T10:04:25.458948+0000","last_fullsized":"2024-04-25T10:04:25.458948+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.918327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479890+0000","last_change":"2024-04-25T10:04:25.475883+0000","last_active":"2024-04-25T10:04:25.479890+0000","last_peered":"2024-04-25T10:04:25.479890+0000","last_clean":"2024-04-25T10:04:25.479890+0000","last_became_active":"2024-04-25T10:04:25.475594+0000","last_became_peered":"2024-04-25T10:04:25.475594+0000","last_unstale":"2024-04-25T10:04:25.479890+0000","last_undegraded":"2024-04-25T10:04:25.479890+0000","last_fullsized":"2024-04-25T10:04:25.479890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475353+0000","last_change":"2024-04-25T10:04:25.461616+0000","last_active":"2024-04-25T10:04:25.475353+0000","last_peered":"2024-04-25T10:04:25.475353+0000","last_clean":"2024-04-25T10:04:25.475353+0000","last_became_active":"2024-04-25T10:04:25.461353+0000","last_became_peered":"2024-04-25T10:04:25.461353+0000","last_unstale":"2024-04-25T10:04:25.475353+0000","last_undegraded":"2024-04-25T10:04:25.475353+0000","last_fullsized":"2024-04-25T10:04:25.475353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:59.983038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.453769+0000","last_change":"2024-04-25T10:04:24.453769+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.453769+0000","last_undegraded":"2024-04-25T10:04:24.453769+0000","last_fullsized":"2024-04-25T10:04:24.453769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475605+0000","last_change":"2024-04-25T10:04:25.469094+0000","last_active":"2024-04-25T10:04:25.475605+0000","last_peered":"2024-04-25T10:04:25.475605+0000","last_clean":"2024-04-25T10:04:25.475605+0000","last_became_active":"2024-04-25T10:04:25.468584+0000","last_became_peered":"2024-04-25T10:04:25.468584+0000","last_unstale":"2024-04-25T10:04:25.475605+0000","last_undegraded":"2024-04-25T10:04:25.475605+0000","last_fullsized":"2024-04-25T10:04:25.475605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.463317+0000","last_change":"2024-04-25T10:04:25.456279+0000","last_active":"2024-04-25T10:04:25.463317+0000","last_peered":"2024-04-25T10:04:25.463317+0000","last_clean":"2024-04-25T10:04:25.463317+0000","last_became_active":"2024-04-25T10:04:25.456002+0000","last_became_peered":"2024-04-25T10:04:25.456002+0000","last_unstale":"2024-04-25T10:04:25.463317+0000","last_undegraded":"2024-04-25T10:04:25.463317+0000","last_fullsized":"2024-04-25T10:04:25.463317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477920+0000","last_change":"2024-04-25T10:04:25.467692+0000","last_active":"2024-04-25T10:04:25.477920+0000","last_peered":"2024-04-25T10:04:25.477920+0000","last_clean":"2024-04-25T10:04:25.477920+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.477920+0000","last_undegraded":"2024-04-25T10:04:25.477920+0000","last_fullsized":"2024-04-25T10:04:25.477920+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477172+0000","last_change":"2024-04-25T10:04:25.469711+0000","last_active":"2024-04-25T10:04:25.477172+0000","last_peered":"2024-04-25T10:04:25.477172+0000","last_clean":"2024-04-25T10:04:25.477172+0000","last_became_active":"2024-04-25T10:04:25.469239+0000","last_became_peered":"2024-04-25T10:04:25.469239+0000","last_unstale":"2024-04-25T10:04:25.477172+0000","last_undegraded":"2024-04-25T10:04:25.477172+0000","last_fullsized":"2024-04-25T10:04:25.477172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483001+0000","last_change":"2024-04-25T10:04:25.479586+0000","last_active":"2024-04-25T10:04:25.483001+0000","last_peered":"2024-04-25T10:04:25.483001+0000","last_clean":"2024-04-25T10:04:25.483001+0000","last_became_active":"2024-04-25T10:04:25.478560+0000","last_became_peered":"2024-04-25T10:04:25.478560+0000","last_unstale":"2024-04-25T10:04:25.483001+0000","last_undegraded":"2024-04-25T10:04:25.483001+0000","last_fullsized":"2024-04-25T10:04:25.483001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471478+0000","last_change":"2024-04-25T10:04:25.467693+0000","last_active":"2024-04-25T10:04:25.471478+0000","last_peered":"2024-04-25T10:04:25.471478+0000","last_clean":"2024-04-25T10:04:25.471478+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.471478+0000","last_undegraded":"2024-04-25T10:04:25.471478+0000","last_fullsized":"2024-04-25T10:04:25.471478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.462788+0000","last_change":"2024-04-25T10:04:25.450560+0000","last_active":"2024-04-25T10:04:25.462788+0000","last_peered":"2024-04-25T10:04:25.462788+0000","last_clean":"2024-04-25T10:04:25.462788+0000","last_became_active":"2024-04-25T10:04:25.450242+0000","last_became_peered":"2024-04-25T10:04:25.450242+0000","last_unstale":"2024-04-25T10:04:25.462788+0000","last_undegraded":"2024-04-25T10:04:25.462788+0000","last_fullsized":"2024-04-25T10:04:25.462788+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:07.537002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471792+0000","last_change":"2024-04-25T10:04:25.463042+0000","last_active":"2024-04-25T10:04:25.471792+0000","last_peered":"2024-04-25T10:04:25.471792+0000","last_clean":"2024-04-25T10:04:25.471792+0000","last_became_active":"2024-04-25T10:04:25.460640+0000","last_became_peered":"2024-04-25T10:04:25.460640+0000","last_unstale":"2024-04-25T10:04:25.471792+0000","last_undegraded":"2024-04-25T10:04:25.471792+0000","last_fullsized":"2024-04-25T10:04:25.471792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460519+0000","last_change":"2024-04-25T10:04:25.455252+0000","last_active":"2024-04-25T10:04:25.460519+0000","last_peered":"2024-04-25T10:04:25.460519+0000","last_clean":"2024-04-25T10:04:25.460519+0000","last_became_active":"2024-04-25T10:04:25.454978+0000","last_became_peered":"2024-04-25T10:04:25.454978+0000","last_unstale":"2024-04-25T10:04:25.460519+0000","last_undegraded":"2024-04-25T10:04:25.460519+0000","last_fullsized":"2024-04-25T10:04:25.460519+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.447927+0000","last_change":"2024-04-25T10:04:24.447927+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.447927+0000","last_undegraded":"2024-04-25T10:04:24.447927+0000","last_fullsized":"2024-04-25T10:04:24.447927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480500+0000","last_change":"2024-04-25T10:04:25.477500+0000","last_active":"2024-04-25T10:04:25.480500+0000","last_peered":"2024-04-25T10:04:25.480500+0000","last_clean":"2024-04-25T10:04:25.480500+0000","last_became_active":"2024-04-25T10:04:25.475152+0000","last_became_peered":"2024-04-25T10:04:25.475152+0000","last_unstale":"2024-04-25T10:04:25.480500+0000","last_undegraded":"2024-04-25T10:04:25.480500+0000","last_fullsized":"2024-04-25T10:04:25.480500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472951+0000","last_change":"2024-04-25T10:04:25.466973+0000","last_active":"2024-04-25T10:04:25.472951+0000","last_peered":"2024-04-25T10:04:25.472951+0000","last_clean":"2024-04-25T10:04:25.472951+0000","last_became_active":"2024-04-25T10:04:25.466685+0000","last_became_peered":"2024-04-25T10:04:25.466685+0000","last_unstale":"2024-04-25T10:04:25.472951+0000","last_undegraded":"2024-04-25T10:04:25.472951+0000","last_fullsized":"2024-04-25T10:04:25.472951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.454102+0000","last_change":"2024-04-25T10:04:24.454102+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.454102+0000","last_undegraded":"2024-04-25T10:04:24.454102+0000","last_fullsized":"2024-04-25T10:04:24.454102+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.452442+0000","last_change":"2024-04-25T10:04:24.452442+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.452442+0000","last_undegraded":"2024-04-25T10:04:24.452442+0000","last_fullsized":"2024-04-25T10:04:24.452442+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.448730+0000","last_change":"2024-04-25T10:04:24.448730+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.448730+0000","last_undegraded":"2024-04-25T10:04:24.448730+0000","last_fullsized":"2024-04-25T10:04:24.448730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472713+0000","last_change":"2024-04-25T10:04:25.465113+0000","last_active":"2024-04-25T10:04:25.472713+0000","last_peered":"2024-04-25T10:04:25.472713+0000","last_clean":"2024-04-25T10:04:25.472713+0000","last_became_active":"2024-04-25T10:04:25.464842+0000","last_became_peered":"2024-04-25T10:04:25.464842+0000","last_unstale":"2024-04-25T10:04:25.472713+0000","last_undegraded":"2024-04-25T10:04:25.472713+0000","last_fullsized":"2024-04-25T10:04:25.472713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.446134+0000","last_change":"2024-04-25T10:04:24.446134+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.446134+0000","last_undegraded":"2024-04-25T10:04:24.446134+0000","last_fullsized":"2024-04-25T10:04:24.446134+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474420+0000","last_change":"2024-04-25T10:04:25.465925+0000","last_active":"2024-04-25T10:04:25.474420+0000","last_peered":"2024-04-25T10:04:25.474420+0000","last_clean":"2024-04-25T10:04:25.474420+0000","last_became_active":"2024-04-25T10:04:25.465652+0000","last_became_peered":"2024-04-25T10:04:25.465652+0000","last_unstale":"2024-04-25T10:04:25.474420+0000","last_undegraded":"2024-04-25T10:04:25.474420+0000","last_fullsized":"2024-04-25T10:04:25.474420+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480477+0000","last_change":"2024-04-25T10:04:25.476769+0000","last_active":"2024-04-25T10:04:25.480477+0000","last_peered":"2024-04-25T10:04:25.480477+0000","last_clean":"2024-04-25T10:04:25.480477+0000","last_became_active":"2024-04-25T10:04:25.476451+0000","last_became_peered":"2024-04-25T10:04:25.476451+0000","last_unstale":"2024-04-25T10:04:25.480477+0000","last_undegraded":"2024-04-25T10:04:25.480477+0000","last_fullsized":"2024-04-25T10:04:25.480477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472747+0000","last_change":"2024-04-25T10:04:25.469665+0000","last_active":"2024-04-25T10:04:25.472747+0000","last_peered":"2024-04-25T10:04:25.472747+0000","last_clean":"2024-04-25T10:04:25.472747+0000","last_became_active":"2024-04-25T10:04:25.469374+0000","last_became_peered":"2024-04-25T10:04:25.469374+0000","last_unstale":"2024-04-25T10:04:25.472747+0000","last_undegraded":"2024-04-25T10:04:25.472747+0000","last_fullsized":"2024-04-25T10:04:25.472747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475056+0000","last_change":"2024-04-25T10:04:25.470230+0000","last_active":"2024-04-25T10:04:25.475056+0000","last_peered":"2024-04-25T10:04:25.475056+0000","last_clean":"2024-04-25T10:04:25.475056+0000","last_became_active":"2024-04-25T10:04:25.469402+0000","last_became_peered":"2024-04-25T10:04:25.469402+0000","last_unstale":"2024-04-25T10:04:25.475056+0000","last_undegraded":"2024-04-25T10:04:25.475056+0000","last_fullsized":"2024-04-25T10:04:25.475056+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:30.617703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.451885+0000","last_change":"2024-04-25T10:04:24.451885+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.451885+0000","last_undegraded":"2024-04-25T10:04:24.451885+0000","last_fullsized":"2024-04-25T10:04:24.451885+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465537+0000","last_change":"2024-04-25T10:04:25.464113+0000","last_active":"2024-04-25T10:04:25.465537+0000","last_peered":"2024-04-25T10:04:25.465537+0000","last_clean":"2024-04-25T10:04:25.465537+0000","last_became_active":"2024-04-25T10:04:25.463493+0000","last_became_peered":"2024-04-25T10:04:25.463493+0000","last_unstale":"2024-04-25T10:04:25.465537+0000","last_undegraded":"2024-04-25T10:04:25.465537+0000","last_fullsized":"2024-04-25T10:04:25.465537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474879+0000","last_change":"2024-04-25T10:04:25.469216+0000","last_active":"2024-04-25T10:04:25.474879+0000","last_peered":"2024-04-25T10:04:25.474879+0000","last_clean":"2024-04-25T10:04:25.474879+0000","last_became_active":"2024-04-25T10:04:25.466664+0000","last_became_peered":"2024-04-25T10:04:25.466664+0000","last_unstale":"2024-04-25T10:04:25.474879+0000","last_undegraded":"2024-04-25T10:04:25.474879+0000","last_fullsized":"2024-04-25T10:04:25.474879+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:35.597962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474510+0000","last_change":"2024-04-25T10:04:24.469487+0000","last_active":"2024-04-25T10:04:25.474510+0000","last_peered":"2024-04-25T10:04:25.474510+0000","last_clean":"2024-04-25T10:04:25.474510+0000","last_became_active":"2024-04-25T10:04:24.469185+0000","last_became_peered":"2024-04-25T10:04:24.469185+0000","last_unstale":"2024-04-25T10:04:25.474510+0000","last_undegraded":"2024-04-25T10:04:25.474510+0000","last_fullsized":"2024-04-25T10:04:25.474510+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479518+0000","last_change":"2024-04-25T10:04:25.473033+0000","last_active":"2024-04-25T10:04:25.479518+0000","last_peered":"2024-04-25T10:04:25.479518+0000","last_clean":"2024-04-25T10:04:25.479518+0000","last_became_active":"2024-04-25T10:04:25.471870+0000","last_became_peered":"2024-04-25T10:04:25.471870+0000","last_unstale":"2024-04-25T10:04:25.479518+0000","last_undegraded":"2024-04-25T10:04:25.479518+0000","last_fullsized":"2024-04-25T10:04:25.479518+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:32.751396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481374+0000","last_change":"2024-04-25T10:04:24.468749+0000","last_active":"2024-04-25T10:04:25.481374+0000","last_peered":"2024-04-25T10:04:25.481374+0000","last_clean":"2024-04-25T10:04:25.481374+0000","last_became_active":"2024-04-25T10:04:24.468458+0000","last_became_peered":"2024-04-25T10:04:24.468458+0000","last_unstale":"2024-04-25T10:04:25.481374+0000","last_undegraded":"2024-04-25T10:04:25.481374+0000","last_fullsized":"2024-04-25T10:04:25.481374+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.077398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480376+0000","last_change":"2024-04-25T10:04:25.478215+0000","last_active":"2024-04-25T10:04:25.480376+0000","last_peered":"2024-04-25T10:04:25.480376+0000","last_clean":"2024-04-25T10:04:25.480376+0000","last_became_active":"2024-04-25T10:04:25.475504+0000","last_became_peered":"2024-04-25T10:04:25.475504+0000","last_unstale":"2024-04-25T10:04:25.480376+0000","last_undegraded":"2024-04-25T10:04:25.480376+0000","last_fullsized":"2024-04-25T10:04:25.480376+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:36.035844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471375+0000","last_change":"2024-04-25T10:04:24.459064+0000","last_active":"2024-04-25T10:04:25.471375+0000","last_peered":"2024-04-25T10:04:25.471375+0000","last_clean":"2024-04-25T10:04:25.471375+0000","last_became_active":"2024-04-25T10:04:24.458599+0000","last_became_peered":"2024-04-25T10:04:24.458599+0000","last_unstale":"2024-04-25T10:04:25.471375+0000","last_undegraded":"2024-04-25T10:04:25.471375+0000","last_fullsized":"2024-04-25T10:04:25.471375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483933+0000","last_change":"2024-04-25T10:04:25.473270+0000","last_active":"2024-04-25T10:04:25.483933+0000","last_peered":"2024-04-25T10:04:25.483933+0000","last_clean":"2024-04-25T10:04:25.483933+0000","last_became_active":"2024-04-25T10:04:25.472978+0000","last_became_peered":"2024-04-25T10:04:25.472978+0000","last_unstale":"2024-04-25T10:04:25.483933+0000","last_undegraded":"2024-04-25T10:04:25.483933+0000","last_fullsized":"2024-04-25T10:04:25.483933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465518+0000","last_change":"2024-04-25T10:04:24.468225+0000","last_active":"2024-04-25T10:04:25.465518+0000","last_peered":"2024-04-25T10:04:25.465518+0000","last_clean":"2024-04-25T10:04:25.465518+0000","last_became_active":"2024-04-25T10:04:24.467670+0000","last_became_peered":"2024-04-25T10:04:24.467670+0000","last_unstale":"2024-04-25T10:04:25.465518+0000","last_undegraded":"2024-04-25T10:04:25.465518+0000","last_fullsized":"2024-04-25T10:04:25.465518+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472793+0000","last_change":"2024-04-25T10:04:25.465227+0000","last_active":"2024-04-25T10:04:25.472793+0000","last_peered":"2024-04-25T10:04:25.472793+0000","last_clean":"2024-04-25T10:04:25.472793+0000","last_became_active":"2024-04-25T10:04:25.464968+0000","last_became_peered":"2024-04-25T10:04:25.464968+0000","last_unstale":"2024-04-25T10:04:25.472793+0000","last_undegraded":"2024-04-25T10:04:25.472793+0000","last_fullsized":"2024-04-25T10:04:25.472793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.473107+0000","last_change":"2024-04-25T10:04:24.467424+0000","last_active":"2024-04-25T10:04:24.473107+0000","last_peered":"2024-04-25T10:04:24.473107+0000","last_clean":"2024-04-25T10:04:24.473107+0000","last_became_active":"2024-04-25T10:04:24.467101+0000","last_became_peered":"2024-04-25T10:04:24.467101+0000","last_unstale":"2024-04-25T10:04:24.473107+0000","last_undegraded":"2024-04-25T10:04:24.473107+0000","last_fullsized":"2024-04-25T10:04:24.473107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:26.852084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479583+0000","last_change":"2024-04-25T10:04:25.471910+0000","last_active":"2024-04-25T10:04:25.479583+0000","last_peered":"2024-04-25T10:04:25.479583+0000","last_clean":"2024-04-25T10:04:25.479583+0000","last_became_active":"2024-04-25T10:04:25.471628+0000","last_became_peered":"2024-04-25T10:04:25.471628+0000","last_unstale":"2024-04-25T10:04:25.479583+0000","last_undegraded":"2024-04-25T10:04:25.479583+0000","last_fullsized":"2024-04-25T10:04:25.479583+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:15.400296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472460+0000","last_change":"2024-04-25T10:04:24.449636+0000","last_active":"2024-04-25T10:04:25.472460+0000","last_peered":"2024-04-25T10:04:25.472460+0000","last_clean":"2024-04-25T10:04:25.472460+0000","last_became_active":"2024-04-25T10:04:24.449303+0000","last_became_peered":"2024-04-25T10:04:24.449303+0000","last_unstale":"2024-04-25T10:04:25.472460+0000","last_undegraded":"2024-04-25T10:04:25.472460+0000","last_fullsized":"2024-04-25T10:04:25.472460+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:58.775423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475584+0000","last_change":"2024-04-25T10:04:25.468855+0000","last_active":"2024-04-25T10:04:25.475584+0000","last_peered":"2024-04-25T10:04:25.475584+0000","last_clean":"2024-04-25T10:04:25.475584+0000","last_became_active":"2024-04-25T10:04:25.467011+0000","last_became_peered":"2024-04-25T10:04:25.467011+0000","last_unstale":"2024-04-25T10:04:25.475584+0000","last_undegraded":"2024-04-25T10:04:25.475584+0000","last_fullsized":"2024-04-25T10:04:25.475584+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.458937+0000","last_change":"2024-04-25T10:04:24.469032+0000","last_active":"2024-04-25T10:04:25.458937+0000","last_peered":"2024-04-25T10:04:25.458937+0000","last_clean":"2024-04-25T10:04:25.458937+0000","last_became_active":"2024-04-25T10:04:24.468452+0000","last_became_peered":"2024-04-25T10:04:24.468452+0000","last_unstale":"2024-04-25T10:04:25.458937+0000","last_undegraded":"2024-04-25T10:04:25.458937+0000","last_fullsized":"2024-04-25T10:04:25.458937+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:49.522317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460571+0000","last_change":"2024-04-25T10:04:25.456211+0000","last_active":"2024-04-25T10:04:25.460571+0000","last_peered":"2024-04-25T10:04:25.460571+0000","last_clean":"2024-04-25T10:04:25.460571+0000","last_became_active":"2024-04-25T10:04:25.455945+0000","last_became_peered":"2024-04-25T10:04:25.455945+0000","last_unstale":"2024-04-25T10:04:25.460571+0000","last_undegraded":"2024-04-25T10:04:25.460571+0000","last_fullsized":"2024-04-25T10:04:25.460571+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.473210+0000","last_change":"2024-04-25T10:04:24.468238+0000","last_active":"2024-04-25T10:04:24.473210+0000","last_peered":"2024-04-25T10:04:24.473210+0000","last_clean":"2024-04-25T10:04:24.473210+0000","last_became_active":"2024-04-25T10:04:24.467961+0000","last_became_peered":"2024-04-25T10:04:24.467961+0000","last_unstale":"2024-04-25T10:04:24.473210+0000","last_undegraded":"2024-04-25T10:04:24.473210+0000","last_fullsized":"2024-04-25T10:04:24.473210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.084728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472666+0000","last_change":"2024-04-25T10:04:25.462850+0000","last_active":"2024-04-25T10:04:25.472666+0000","last_peered":"2024-04-25T10:04:25.472666+0000","last_clean":"2024-04-25T10:04:25.472666+0000","last_became_active":"2024-04-25T10:04:25.462560+0000","last_became_peered":"2024-04-25T10:04:25.462560+0000","last_unstale":"2024-04-25T10:04:25.472666+0000","last_undegraded":"2024-04-25T10:04:25.472666+0000","last_fullsized":"2024-04-25T10:04:25.472666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.444069+0000","last_change":"2024-04-25T10:04:24.447649+0000","last_active":"2024-04-25T10:04:25.444069+0000","last_peered":"2024-04-25T10:04:25.444069+0000","last_clean":"2024-04-25T10:04:25.444069+0000","last_became_active":"2024-04-25T10:04:24.446781+0000","last_became_peered":"2024-04-25T10:04:24.446781+0000","last_unstale":"2024-04-25T10:04:25.444069+0000","last_undegraded":"2024-04-25T10:04:25.444069+0000","last_fullsized":"2024-04-25T10:04:25.444069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.029749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.457360+0000","last_change":"2024-04-25T10:04:25.452425+0000","last_active":"2024-04-25T10:04:25.457360+0000","last_peered":"2024-04-25T10:04:25.457360+0000","last_clean":"2024-04-25T10:04:25.457360+0000","last_became_active":"2024-04-25T10:04:25.451175+0000","last_became_peered":"2024-04-25T10:04:25.451175+0000","last_unstale":"2024-04-25T10:04:25.457360+0000","last_undegraded":"2024-04-25T10:04:25.457360+0000","last_fullsized":"2024-04-25T10:04:25.457360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.443720+0000","last_change":"2024-04-25T10:04:24.451547+0000","last_active":"2024-04-25T10:04:25.443720+0000","last_peered":"2024-04-25T10:04:25.443720+0000","last_clean":"2024-04-25T10:04:25.443720+0000","last_became_active":"2024-04-25T10:04:24.451221+0000","last_became_peered":"2024-04-25T10:04:24.451221+0000","last_unstale":"2024-04-25T10:04:25.443720+0000","last_undegraded":"2024-04-25T10:04:25.443720+0000","last_fullsized":"2024-04-25T10:04:25.443720+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472556+0000","last_change":"2024-04-25T10:04:25.466717+0000","last_active":"2024-04-25T10:04:25.472556+0000","last_peered":"2024-04-25T10:04:25.472556+0000","last_clean":"2024-04-25T10:04:25.472556+0000","last_became_active":"2024-04-25T10:04:25.466392+0000","last_became_peered":"2024-04-25T10:04:25.466392+0000","last_unstale":"2024-04-25T10:04:25.472556+0000","last_undegraded":"2024-04-25T10:04:25.472556+0000","last_fullsized":"2024-04-25T10:04:25.472556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480231+0000","last_change":"2024-04-25T10:04:24.462694+0000","last_active":"2024-04-25T10:04:25.480231+0000","last_peered":"2024-04-25T10:04:25.480231+0000","last_clean":"2024-04-25T10:04:25.480231+0000","last_became_active":"2024-04-25T10:04:24.462301+0000","last_became_peered":"2024-04-25T10:04:24.462301+0000","last_unstale":"2024-04-25T10:04:25.480231+0000","last_undegraded":"2024-04-25T10:04:25.480231+0000","last_fullsized":"2024-04-25T10:04:25.480231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.011021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469006+0000","last_change":"2024-04-25T10:04:25.466716+0000","last_active":"2024-04-25T10:04:25.469006+0000","last_peered":"2024-04-25T10:04:25.469006+0000","last_clean":"2024-04-25T10:04:25.469006+0000","last_became_active":"2024-04-25T10:04:25.466438+0000","last_became_peered":"2024-04-25T10:04:25.466438+0000","last_unstale":"2024-04-25T10:04:25.469006+0000","last_undegraded":"2024-04-25T10:04:25.469006+0000","last_fullsized":"2024-04-25T10:04:25.469006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.486863+0000","last_change":"2024-04-25T10:04:24.469012+0000","last_active":"2024-04-25T10:04:25.486863+0000","last_peered":"2024-04-25T10:04:25.486863+0000","last_clean":"2024-04-25T10:04:25.486863+0000","last_became_active":"2024-04-25T10:04:24.468729+0000","last_became_peered":"2024-04-25T10:04:24.468729+0000","last_unstale":"2024-04-25T10:04:25.486863+0000","last_undegraded":"2024-04-25T10:04:25.486863+0000","last_fullsized":"2024-04-25T10:04:25.486863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474650+0000","last_change":"2024-04-25T10:04:25.468212+0000","last_active":"2024-04-25T10:04:25.474650+0000","last_peered":"2024-04-25T10:04:25.474650+0000","last_clean":"2024-04-25T10:04:25.474650+0000","last_became_active":"2024-04-25T10:04:25.467940+0000","last_became_peered":"2024-04-25T10:04:25.467940+0000","last_unstale":"2024-04-25T10:04:25.474650+0000","last_undegraded":"2024-04-25T10:04:25.474650+0000","last_fullsized":"2024-04-25T10:04:25.474650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:04.217886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482369+0000","last_change":"2024-04-25T10:04:24.456904+0000","last_active":"2024-04-25T10:04:25.482369+0000","last_peered":"2024-04-25T10:04:25.482369+0000","last_clean":"2024-04-25T10:04:25.482369+0000","last_became_active":"2024-04-25T10:04:24.456095+0000","last_became_peered":"2024-04-25T10:04:24.456095+0000","last_unstale":"2024-04-25T10:04:25.482369+0000","last_undegraded":"2024-04-25T10:04:25.482369+0000","last_fullsized":"2024-04-25T10:04:25.482369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.185120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482977+0000","last_change":"2024-04-25T10:04:25.479170+0000","last_active":"2024-04-25T10:04:25.482977+0000","last_peered":"2024-04-25T10:04:25.482977+0000","last_clean":"2024-04-25T10:04:25.482977+0000","last_became_active":"2024-04-25T10:04:25.478980+0000","last_became_peered":"2024-04-25T10:04:25.478980+0000","last_unstale":"2024-04-25T10:04:25.482977+0000","last_undegraded":"2024-04-25T10:04:25.482977+0000","last_fullsized":"2024-04-25T10:04:25.482977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.442376+0000","last_change":"2024-04-25T10:04:24.458424+0000","last_active":"2024-04-25T10:04:25.442376+0000","last_peered":"2024-04-25T10:04:25.442376+0000","last_clean":"2024-04-25T10:04:25.442376+0000","last_became_active":"2024-04-25T10:04:24.458098+0000","last_became_peered":"2024-04-25T10:04:24.458098+0000","last_unstale":"2024-04-25T10:04:25.442376+0000","last_undegraded":"2024-04-25T10:04:25.442376+0000","last_fullsized":"2024-04-25T10:04:25.442376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469999+0000","last_change":"2024-04-25T10:04:25.455450+0000","last_active":"2024-04-25T10:04:25.469999+0000","last_peered":"2024-04-25T10:04:25.469999+0000","last_clean":"2024-04-25T10:04:25.469999+0000","last_became_active":"2024-04-25T10:04:25.454137+0000","last_became_peered":"2024-04-25T10:04:25.454137+0000","last_unstale":"2024-04-25T10:04:25.469999+0000","last_undegraded":"2024-04-25T10:04:25.469999+0000","last_fullsized":"2024-04-25T10:04:25.469999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:11.030611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446148+0000","last_change":"2024-04-25T10:04:24.445036+0000","last_active":"2024-04-25T10:04:25.446148+0000","last_peered":"2024-04-25T10:04:25.446148+0000","last_clean":"2024-04-25T10:04:25.446148+0000","last_became_active":"2024-04-25T10:04:24.444715+0000","last_became_peered":"2024-04-25T10:04:24.444715+0000","last_unstale":"2024-04-25T10:04:25.446148+0000","last_undegraded":"2024-04-25T10:04:25.446148+0000","last_fullsized":"2024-04-25T10:04:25.446148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.905326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.453241+0000","last_change":"2024-04-25T10:04:24.453241+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.453241+0000","last_undegraded":"2024-04-25T10:04:24.453241+0000","last_fullsized":"2024-04-25T10:04:24.453241+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482536+0000","last_change":"2024-04-25T10:04:24.465436+0000","last_active":"2024-04-25T10:04:25.482536+0000","last_peered":"2024-04-25T10:04:25.482536+0000","last_clean":"2024-04-25T10:04:25.482536+0000","last_became_active":"2024-04-25T10:04:24.465132+0000","last_became_peered":"2024-04-25T10:04:24.465132+0000","last_unstale":"2024-04-25T10:04:25.482536+0000","last_undegraded":"2024-04-25T10:04:25.482536+0000","last_fullsized":"2024-04-25T10:04:25.482536+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.884017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479931+0000","last_change":"2024-04-25T10:04:25.472528+0000","last_active":"2024-04-25T10:04:25.479931+0000","last_peered":"2024-04-25T10:04:25.479931+0000","last_clean":"2024-04-25T10:04:25.479931+0000","last_became_active":"2024-04-25T10:04:25.472245+0000","last_became_peered":"2024-04-25T10:04:25.472245+0000","last_unstale":"2024-04-25T10:04:25.479931+0000","last_undegraded":"2024-04-25T10:04:25.479931+0000","last_fullsized":"2024-04-25T10:04:25.479931+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481219+0000","last_change":"2024-04-25T10:04:24.468526+0000","last_active":"2024-04-25T10:04:25.481219+0000","last_peered":"2024-04-25T10:04:25.481219+0000","last_clean":"2024-04-25T10:04:25.481219+0000","last_became_active":"2024-04-25T10:04:24.468213+0000","last_became_peered":"2024-04-25T10:04:24.468213+0000","last_unstale":"2024-04-25T10:04:25.481219+0000","last_undegraded":"2024-04-25T10:04:25.481219+0000","last_fullsized":"2024-04-25T10:04:25.481219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:40.655704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472621+0000","last_change":"2024-04-25T10:04:25.463721+0000","last_active":"2024-04-25T10:04:25.472621+0000","last_peered":"2024-04-25T10:04:25.472621+0000","last_clean":"2024-04-25T10:04:25.472621+0000","last_became_active":"2024-04-25T10:04:25.463355+0000","last_became_peered":"2024-04-25T10:04:25.463355+0000","last_unstale":"2024-04-25T10:04:25.472621+0000","last_undegraded":"2024-04-25T10:04:25.472621+0000","last_fullsized":"2024-04-25T10:04:25.472621+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481630+0000","last_change":"2024-04-25T10:04:24.461901+0000","last_active":"2024-04-25T10:04:25.481630+0000","last_peered":"2024-04-25T10:04:25.481630+0000","last_clean":"2024-04-25T10:04:25.481630+0000","last_became_active":"2024-04-25T10:04:24.461619+0000","last_became_peered":"2024-04-25T10:04:24.461619+0000","last_unstale":"2024-04-25T10:04:25.481630+0000","last_undegraded":"2024-04-25T10:04:25.481630+0000","last_fullsized":"2024-04-25T10:04:25.481630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:32.072513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480737+0000","last_change":"2024-04-25T10:04:25.477724+0000","last_active":"2024-04-25T10:04:25.480737+0000","last_peered":"2024-04-25T10:04:25.480737+0000","last_clean":"2024-04-25T10:04:25.480737+0000","last_became_active":"2024-04-25T10:04:25.476438+0000","last_became_peered":"2024-04-25T10:04:25.476438+0000","last_unstale":"2024-04-25T10:04:25.480737+0000","last_undegraded":"2024-04-25T10:04:25.480737+0000","last_fullsized":"2024-04-25T10:04:25.480737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475376+0000","last_change":"2024-04-25T10:04:24.465620+0000","last_active":"2024-04-25T10:04:25.475376+0000","last_peered":"2024-04-25T10:04:25.475376+0000","last_clean":"2024-04-25T10:04:25.475376+0000","last_became_active":"2024-04-25T10:04:24.465402+0000","last_became_peered":"2024-04-25T10:04:24.465402+0000","last_unstale":"2024-04-25T10:04:25.475376+0000","last_undegraded":"2024-04-25T10:04:25.475376+0000","last_fullsized":"2024-04-25T10:04:25.475376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474444+0000","last_change":"2024-04-25T10:04:25.463940+0000","last_active":"2024-04-25T10:04:25.474444+0000","last_peered":"2024-04-25T10:04:25.474444+0000","last_clean":"2024-04-25T10:04:25.474444+0000","last_became_active":"2024-04-25T10:04:25.463675+0000","last_became_peered":"2024-04-25T10:04:25.463675+0000","last_unstale":"2024-04-25T10:04:25.474444+0000","last_undegraded":"2024-04-25T10:04:25.474444+0000","last_fullsized":"2024-04-25T10:04:25.474444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.456328+0000","last_change":"2024-04-25T10:04:24.466373+0000","last_active":"2024-04-25T10:04:25.456328+0000","last_peered":"2024-04-25T10:04:25.456328+0000","last_clean":"2024-04-25T10:04:25.456328+0000","last_became_active":"2024-04-25T10:04:24.465965+0000","last_became_peered":"2024-04-25T10:04:24.465965+0000","last_unstale":"2024-04-25T10:04:25.456328+0000","last_undegraded":"2024-04-25T10:04:25.456328+0000","last_fullsized":"2024-04-25T10:04:25.456328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.478776+0000","last_change":"2024-04-25T10:04:25.471532+0000","last_active":"2024-04-25T10:04:25.478776+0000","last_peered":"2024-04-25T10:04:25.478776+0000","last_clean":"2024-04-25T10:04:25.478776+0000","last_became_active":"2024-04-25T10:04:25.471276+0000","last_became_peered":"2024-04-25T10:04:25.471276+0000","last_unstale":"2024-04-25T10:04:25.478776+0000","last_undegraded":"2024-04-25T10:04:25.478776+0000","last_fullsized":"2024-04-25T10:04:25.478776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.468521+0000","last_change":"2024-04-25T10:04:24.461754+0000","last_active":"2024-04-25T10:04:25.468521+0000","last_peered":"2024-04-25T10:04:25.468521+0000","last_clean":"2024-04-25T10:04:25.468521+0000","last_became_active":"2024-04-25T10:04:24.459617+0000","last_became_peered":"2024-04-25T10:04:24.459617+0000","last_unstale":"2024-04-25T10:04:25.468521+0000","last_undegraded":"2024-04-25T10:04:25.468521+0000","last_fullsized":"2024-04-25T10:04:25.468521+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.473141+0000","last_change":"2024-04-25T10:04:25.468223+0000","last_active":"2024-04-25T10:04:25.473141+0000","last_peered":"2024-04-25T10:04:25.473141+0000","last_clean":"2024-04-25T10:04:25.473141+0000","last_became_active":"2024-04-25T10:04:25.467369+0000","last_became_peered":"2024-04-25T10:04:25.467369+0000","last_unstale":"2024-04-25T10:04:25.473141+0000","last_undegraded":"2024-04-25T10:04:25.473141+0000","last_fullsized":"2024-04-25T10:04:25.473141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.475694+0000","last_change":"2024-04-25T10:04:24.471952+0000","last_active":"2024-04-25T10:04:24.475694+0000","last_peered":"2024-04-25T10:04:24.475694+0000","last_clean":"2024-04-25T10:04:24.475694+0000","last_became_active":"2024-04-25T10:04:24.471642+0000","last_became_peered":"2024-04-25T10:04:24.471642+0000","last_unstale":"2024-04-25T10:04:24.475694+0000","last_undegraded":"2024-04-25T10:04:24.475694+0000","last_fullsized":"2024-04-25T10:04:24.475694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.562510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472693+0000","last_change":"2024-04-25T10:04:25.468105+0000","last_active":"2024-04-25T10:04:25.472693+0000","last_peered":"2024-04-25T10:04:25.472693+0000","last_clean":"2024-04-25T10:04:25.472693+0000","last_became_active":"2024-04-25T10:04:25.467841+0000","last_became_peered":"2024-04-25T10:04:25.467841+0000","last_unstale":"2024-04-25T10:04:25.472693+0000","last_undegraded":"2024-04-25T10:04:25.472693+0000","last_fullsized":"2024-04-25T10:04:25.472693+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.451244+0000","last_change":"2024-04-25T10:04:24.469768+0000","last_active":"2024-04-25T10:04:25.451244+0000","last_peered":"2024-04-25T10:04:25.451244+0000","last_clean":"2024-04-25T10:04:25.451244+0000","last_became_active":"2024-04-25T10:04:24.469488+0000","last_became_peered":"2024-04-25T10:04:24.469488+0000","last_unstale":"2024-04-25T10:04:25.451244+0000","last_undegraded":"2024-04-25T10:04:25.451244+0000","last_fullsized":"2024-04-25T10:04:25.451244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:29.785579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480787+0000","last_change":"2024-04-25T10:04:25.475514+0000","last_active":"2024-04-25T10:04:25.480787+0000","last_peered":"2024-04-25T10:04:25.480787+0000","last_clean":"2024-04-25T10:04:25.480787+0000","last_became_active":"2024-04-25T10:04:25.474364+0000","last_became_peered":"2024-04-25T10:04:25.474364+0000","last_unstale":"2024-04-25T10:04:25.480787+0000","last_undegraded":"2024-04-25T10:04:25.480787+0000","last_fullsized":"2024-04-25T10:04:25.480787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.453420+0000","last_change":"2024-04-25T10:04:24.460278+0000","last_active":"2024-04-25T10:04:25.453420+0000","last_peered":"2024-04-25T10:04:25.453420+0000","last_clean":"2024-04-25T10:04:25.453420+0000","last_became_active":"2024-04-25T10:04:24.459936+0000","last_became_peered":"2024-04-25T10:04:24.459936+0000","last_unstale":"2024-04-25T10:04:25.453420+0000","last_undegraded":"2024-04-25T10:04:25.453420+0000","last_fullsized":"2024-04-25T10:04:25.453420+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:19.332266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465336+0000","last_change":"2024-04-25T10:04:25.456950+0000","last_active":"2024-04-25T10:04:25.465336+0000","last_peered":"2024-04-25T10:04:25.465336+0000","last_clean":"2024-04-25T10:04:25.465336+0000","last_became_active":"2024-04-25T10:04:25.456664+0000","last_became_peered":"2024-04-25T10:04:25.456664+0000","last_unstale":"2024-04-25T10:04:25.465336+0000","last_undegraded":"2024-04-25T10:04:25.465336+0000","last_fullsized":"2024-04-25T10:04:25.465336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474794+0000","last_change":"2024-04-25T10:04:24.466995+0000","last_active":"2024-04-25T10:04:25.474794+0000","last_peered":"2024-04-25T10:04:25.474794+0000","last_clean":"2024-04-25T10:04:25.474794+0000","last_became_active":"2024-04-25T10:04:24.466667+0000","last_became_peered":"2024-04-25T10:04:24.466667+0000","last_unstale":"2024-04-25T10:04:25.474794+0000","last_undegraded":"2024-04-25T10:04:25.474794+0000","last_fullsized":"2024-04-25T10:04:25.474794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:07.560341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475023+0000","last_change":"2024-04-25T10:04:25.467788+0000","last_active":"2024-04-25T10:04:25.475023+0000","last_peered":"2024-04-25T10:04:25.475023+0000","last_clean":"2024-04-25T10:04:25.475023+0000","last_became_active":"2024-04-25T10:04:25.465574+0000","last_became_peered":"2024-04-25T10:04:25.465574+0000","last_unstale":"2024-04-25T10:04:25.475023+0000","last_undegraded":"2024-04-25T10:04:25.475023+0000","last_fullsized":"2024-04-25T10:04:25.475023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.459728+0000","last_change":"2024-04-25T10:04:24.463577+0000","last_active":"2024-04-25T10:04:25.459728+0000","last_peered":"2024-04-25T10:04:25.459728+0000","last_clean":"2024-04-25T10:04:25.459728+0000","last_became_active":"2024-04-25T10:04:24.462318+0000","last_became_peered":"2024-04-25T10:04:24.462318+0000","last_unstale":"2024-04-25T10:04:25.459728+0000","last_undegraded":"2024-04-25T10:04:25.459728+0000","last_fullsized":"2024-04-25T10:04:25.459728+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:01.690130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.444965+0000","last_change":"2024-04-25T10:04:24.444965+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.444965+0000","last_undegraded":"2024-04-25T10:04:24.444965+0000","last_fullsized":"2024-04-25T10:04:24.444965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.472868+0000","last_change":"2024-04-25T10:04:24.462828+0000","last_active":"2024-04-25T10:04:24.472868+0000","last_peered":"2024-04-25T10:04:24.472868+0000","last_clean":"2024-04-25T10:04:24.472868+0000","last_became_active":"2024-04-25T10:04:24.462443+0000","last_became_peered":"2024-04-25T10:04:24.462443+0000","last_unstale":"2024-04-25T10:04:24.472868+0000","last_undegraded":"2024-04-25T10:04:24.472868+0000","last_fullsized":"2024-04-25T10:04:24.472868+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:06.233856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475078+0000","last_change":"2024-04-25T10:04:25.466945+0000","last_active":"2024-04-25T10:04:25.475078+0000","last_peered":"2024-04-25T10:04:25.475078+0000","last_clean":"2024-04-25T10:04:25.475078+0000","last_became_active":"2024-04-25T10:04:25.466672+0000","last_became_peered":"2024-04-25T10:04:25.466672+0000","last_unstale":"2024-04-25T10:04:25.475078+0000","last_undegraded":"2024-04-25T10:04:25.475078+0000","last_fullsized":"2024-04-25T10:04:25.475078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.450703+0000","last_change":"2024-04-25T10:04:24.461824+0000","last_active":"2024-04-25T10:04:25.450703+0000","last_peered":"2024-04-25T10:04:25.450703+0000","last_clean":"2024-04-25T10:04:25.450703+0000","last_became_active":"2024-04-25T10:04:24.460800+0000","last_became_peered":"2024-04-25T10:04:24.460800+0000","last_unstale":"2024-04-25T10:04:25.450703+0000","last_undegraded":"2024-04-25T10:04:25.450703+0000","last_fullsized":"2024-04-25T10:04:25.450703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:25.199121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480645+0000","last_change":"2024-04-25T10:04:25.477935+0000","last_active":"2024-04-25T10:04:25.480645+0000","last_peered":"2024-04-25T10:04:25.480645+0000","last_clean":"2024-04-25T10:04:25.480645+0000","last_became_active":"2024-04-25T10:04:25.477605+0000","last_became_peered":"2024-04-25T10:04:25.477605+0000","last_unstale":"2024-04-25T10:04:25.480645+0000","last_undegraded":"2024-04-25T10:04:25.480645+0000","last_fullsized":"2024-04-25T10:04:25.480645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.473344+0000","last_change":"2024-04-25T10:04:24.471290+0000","last_active":"2024-04-25T10:04:24.473344+0000","last_peered":"2024-04-25T10:04:24.473344+0000","last_clean":"2024-04-25T10:04:24.473344+0000","last_became_active":"2024-04-25T10:04:24.470984+0000","last_became_peered":"2024-04-25T10:04:24.470984+0000","last_unstale":"2024-04-25T10:04:24.473344+0000","last_undegraded":"2024-04-25T10:04:24.473344+0000","last_fullsized":"2024-04-25T10:04:24.473344+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:22.705602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475556+0000","last_change":"2024-04-25T10:04:25.462505+0000","last_active":"2024-04-25T10:04:25.475556+0000","last_peered":"2024-04-25T10:04:25.475556+0000","last_clean":"2024-04-25T10:04:25.475556+0000","last_became_active":"2024-04-25T10:04:25.461056+0000","last_became_peered":"2024-04-25T10:04:25.461056+0000","last_unstale":"2024-04-25T10:04:25.475556+0000","last_undegraded":"2024-04-25T10:04:25.475556+0000","last_fullsized":"2024-04-25T10:04:25.475556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446296+0000","last_change":"2024-04-25T10:04:24.443846+0000","last_active":"2024-04-25T10:04:25.446296+0000","last_peered":"2024-04-25T10:04:25.446296+0000","last_clean":"2024-04-25T10:04:25.446296+0000","last_became_active":"2024-04-25T10:04:24.443511+0000","last_became_peered":"2024-04-25T10:04:24.443511+0000","last_unstale":"2024-04-25T10:04:25.446296+0000","last_undegraded":"2024-04-25T10:04:25.446296+0000","last_fullsized":"2024-04-25T10:04:25.446296+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475649+0000","last_change":"2024-04-25T10:04:25.466904+0000","last_active":"2024-04-25T10:04:25.475649+0000","last_peered":"2024-04-25T10:04:25.475649+0000","last_clean":"2024-04-25T10:04:25.475649+0000","last_became_active":"2024-04-25T10:04:25.466434+0000","last_became_peered":"2024-04-25T10:04:25.466434+0000","last_unstale":"2024-04-25T10:04:25.475649+0000","last_undegraded":"2024-04-25T10:04:25.475649+0000","last_fullsized":"2024-04-25T10:04:25.475649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441933+0000","last_change":"2024-04-25T10:04:24.467188+0000","last_active":"2024-04-25T10:04:25.441933+0000","last_peered":"2024-04-25T10:04:25.441933+0000","last_clean":"2024-04-25T10:04:25.441933+0000","last_became_active":"2024-04-25T10:04:24.466850+0000","last_became_peered":"2024-04-25T10:04:24.466850+0000","last_unstale":"2024-04-25T10:04:25.441933+0000","last_undegraded":"2024-04-25T10:04:25.441933+0000","last_fullsized":"2024-04-25T10:04:25.441933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472620+0000","last_change":"2024-04-25T10:04:25.461224+0000","last_active":"2024-04-25T10:04:25.472620+0000","last_peered":"2024-04-25T10:04:25.472620+0000","last_clean":"2024-04-25T10:04:25.472620+0000","last_became_active":"2024-04-25T10:04:25.460727+0000","last_became_peered":"2024-04-25T10:04:25.460727+0000","last_unstale":"2024-04-25T10:04:25.472620+0000","last_undegraded":"2024-04-25T10:04:25.472620+0000","last_fullsized":"2024-04-25T10:04:25.472620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.452980+0000","last_change":"2024-04-25T10:04:24.461091+0000","last_active":"2024-04-25T10:04:25.452980+0000","last_peered":"2024-04-25T10:04:25.452980+0000","last_clean":"2024-04-25T10:04:25.452980+0000","last_became_active":"2024-04-25T10:04:24.460322+0000","last_became_peered":"2024-04-25T10:04:24.460322+0000","last_unstale":"2024-04-25T10:04:25.452980+0000","last_undegraded":"2024-04-25T10:04:25.452980+0000","last_fullsized":"2024-04-25T10:04:25.452980+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:17.560293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441995+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:04:25.441995+0000","last_peered":"2024-04-25T10:04:25.441995+0000","last_clean":"2024-04-25T10:04:25.441995+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:04:25.441995+0000","last_undegraded":"2024-04-25T10:04:25.441995+0000","last_fullsized":"2024-04-25T10:04:25.441995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":241488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:29.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.193+0000 7fa0b9200640 1 Processor -- start 2024-04-25T10:04:30.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.193+0000 7fa0b9200640 1 -- start start 2024-04-25T10:04:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.194+0000 7fa0b9200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa0b406ab60 0x7fa0b406af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.194+0000 7fa0b9200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0b40afae0 0x7fa0b40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.194+0000 7fa0b9200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b40b4f80 con 0x7fa0b406ee10 2024-04-25T10:04:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.194+0000 7fa0b9200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b40b5590 con 0x7fa0b40afae0 2024-04-25T10:04:30.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.194+0000 7fa0b9200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b406b480 con 0x7fa0b406ab60 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0b40afae0 0x7fa0b40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0b40afae0 0x7fa0b40b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57710/0 (socket says 172.21.15.5:57710) 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 -- 172.21.15.5:0/2475970263 learned_addr learned my addr 172.21.15.5:0/2475970263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1e00640 1 --2- 172.21.15.5:0/2475970263 >> v2:172.21.15.80:3300/0 conn(0x7fa0b406ab60 0x7fa0b406af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 -- 172.21.15.5:0/2475970263 >> v2:172.21.15.80:3300/0 conn(0x7fa0b406ab60 msgr2=0x7fa0b406af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 --2- 172.21.15.5:0/2475970263 >> v2:172.21.15.80:3300/0 conn(0x7fa0b406ab60 0x7fa0b406af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 -- 172.21.15.5:0/2475970263 >> v1:172.21.15.80:6789/0 conn(0x7fa0b406ee10 legacy=0x7fa0b40ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.195+0000 7fa0b1400640 1 -- 172.21.15.5:0/2475970263 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b406fec0 con 0x7fa0b40afae0 2024-04-25T10:04:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.196+0000 7fa0b1400640 1 --2- 172.21.15.5:0/2475970263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b40afae0 0x7fa0b40b34d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa0b406bc40 tx=0x7fa0a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26938e7aa89089c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.197+0000 7fa0b0a00640 1 -- 172.21.15.5:0/2475970263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0037070 con 0x7fa0b40afae0 2024-04-25T10:04:30.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.197+0000 7fa0b0a00640 1 -- 172.21.15.5:0/2475970263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a0032440 con 0x7fa0b40afae0 2024-04-25T10:04:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.197+0000 7fa0b0a00640 1 -- 172.21.15.5:0/2475970263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0032810 con 0x7fa0b40afae0 2024-04-25T10:04:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.197+0000 7fa0b9200640 1 -- 172.21.15.5:0/2475970263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b40afae0 msgr2=0x7fa0b40b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.197+0000 7fa0b9200640 1 --2- 172.21.15.5:0/2475970263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b40afae0 0x7fa0b40b34d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa0b406bc40 tx=0x7fa0a002a230 comp rx=0 tx=0).stop 2024-04-25T10:04:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.198+0000 7fa0b9200640 1 -- 172.21.15.5:0/2475970263 shutdown_connections 2024-04-25T10:04:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.198+0000 7fa0b9200640 1 --2- 172.21.15.5:0/2475970263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b40afae0 0x7fa0b40b34d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.198+0000 7fa0b9200640 1 --2- 172.21.15.5:0/2475970263 >> v2:172.21.15.80:3300/0 conn(0x7fa0b406ab60 0x7fa0b406af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.198+0000 7fa0b9200640 1 -- 172.21.15.5:0/2475970263 >> 172.21.15.5:0/2475970263 conn(0x7fa0b40b59f0 msgr2=0x7fa0b40b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.198+0000 7fa0b9200640 1 -- 172.21.15.5:0/2475970263 shutdown_connections 2024-04-25T10:04:30.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.198+0000 7fa0b9200640 1 -- 172.21.15.5:0/2475970263 wait complete. 2024-04-25T10:04:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.199+0000 7fa0b9200640 1 Processor -- start 2024-04-25T10:04:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.199+0000 7fa0b9200640 1 -- start start 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.200+0000 7fa0b9200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0b406ab60 0x7fa0b40ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.200+0000 7fa0b9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 0x7fa0b40cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.200+0000 7fa0b9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0b40afae0 0x7fa0b40cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.200+0000 7fa0b9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b3a10 con 0x7fa0b406ee10 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.200+0000 7fa0b9200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40b5220 con 0x7fa0b40afae0 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.200+0000 7fa0b9200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40adbc0 con 0x7fa0b406ab60 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 0x7fa0b40cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 0x7fa0b40cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57714/0 (socket says 172.21.15.5:57714) 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 -- 172.21.15.5:0/3834985375 learned_addr learned my addr 172.21.15.5:0/3834985375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b2800640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0b40afae0 0x7fa0b40cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 -- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0b406ab60 msgr2=0x7fa0b40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0b406ab60 0x7fa0b40ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 -- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0b40afae0 msgr2=0x7fa0b40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0b40afae0 0x7fa0b40cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.201+0000 7fa0b1400640 1 -- 172.21.15.5:0/3834985375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40cdc10 con 0x7fa0b406ee10 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.202+0000 7fa0b1400640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 0x7fa0b40cd510 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa0a002a270 tx=0x7fa0a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.202+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0041020 con 0x7fa0b406ee10 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.202+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a00306c0 con 0x7fa0b406ee10 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.202+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0037040 con 0x7fa0b406ee10 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.202+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b40cdef0 con 0x7fa0b406ee10 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.202+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b40ce4b0 con 0x7fa0b406ee10 2024-04-25T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.204+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b4068fd0 con 0x7fa0b406ee10 2024-04-25T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.205+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0a0044020 con 0x7fa0b406ee10 2024-04-25T10:04:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.206+0000 7fa0aa000640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09406b4d0 0x7fa09406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.206+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0a00aac00 con 0x7fa0b406ee10 2024-04-25T10:04:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.207+0000 7fa0b1e00640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09406b4d0 0x7fa09406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.208+0000 7fa0b1e00640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09406b4d0 0x7fa09406d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006fd0 tx=0x7fa0a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.210+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0a00b1080 con 0x7fa0b406ee10 2024-04-25T10:04:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.405+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0b40ad630 con 0x7fa09406b4d0 2024-04-25T10:04:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.423+0000 7fa0aa000640 1 -- 172.21.15.5:0/3834985375 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251073 (secure 0 0 0) 0x7fa0b40ad630 con 0x7fa09406b4d0 2024-04-25T10:04:30.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.429+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09406b4d0 msgr2=0x7fa09406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.429+0000 7fa0b9200640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09406b4d0 0x7fa09406d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4006fd0 tx=0x7fa0a4008040 comp rx=0 tx=0).stop 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.429+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 msgr2=0x7fa0b40cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.429+0000 7fa0b9200640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 0x7fa0b40cd510 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa0a002a270 tx=0x7fa0a00046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 shutdown_connections 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa09406b4d0 0x7fa09406d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0b40afae0 0x7fa0b40cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0b406ee10 0x7fa0b40cd510 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 --2- 172.21.15.5:0/3834985375 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0b406ab60 0x7fa0b40ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 >> 172.21.15.5:0/3834985375 conn(0x7fa0b40b59f0 msgr2=0x7fa0b40b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 shutdown_connections 2024-04-25T10:04:30.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.430+0000 7fa0b9200640 1 -- 172.21.15.5:0/3834985375 wait complete. 2024-04-25T10:04:30.458 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-04-25T10:04:28.665343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238456,"kb_used_data":5628,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736264,"statfs":{"total":773094113280,"available":772849934336,"internally_reserved":0,"allocated":5763072,"data_stored":3159309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252602"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471647+0000","last_change":"2024-04-25T10:04:25.461883+0000","last_active":"2024-04-25T10:04:25.471647+0000","last_peered":"2024-04-25T10:04:25.471647+0000","last_clean":"2024-04-25T10:04:25.471647+0000","last_became_active":"2024-04-25T10:04:25.460987+0000","last_became_peered":"2024-04-25T10:04:25.460987+0000","last_unstale":"2024-04-25T10:04:25.471647+0000","last_undegraded":"2024-04-25T10:04:25.471647+0000","last_fullsized":"2024-04-25T10:04:25.471647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479215+0000","last_change":"2024-04-25T10:04:25.474565+0000","last_active":"2024-04-25T10:04:25.479215+0000","last_peered":"2024-04-25T10:04:25.479215+0000","last_clean":"2024-04-25T10:04:25.479215+0000","last_became_active":"2024-04-25T10:04:25.474282+0000","last_became_peered":"2024-04-25T10:04:25.474282+0000","last_unstale":"2024-04-25T10:04:25.479215+0000","last_undegraded":"2024-04-25T10:04:25.479215+0000","last_fullsized":"2024-04-25T10:04:25.479215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.461950+0000","last_change":"2024-04-25T10:04:24.461950+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.461950+0000","last_undegraded":"2024-04-25T10:04:24.461950+0000","last_fullsized":"2024-04-25T10:04:24.461950+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.459057+0000","last_change":"2024-04-25T10:04:25.454040+0000","last_active":"2024-04-25T10:04:25.459057+0000","last_peered":"2024-04-25T10:04:25.459057+0000","last_clean":"2024-04-25T10:04:25.459057+0000","last_became_active":"2024-04-25T10:04:25.453680+0000","last_became_peered":"2024-04-25T10:04:25.453680+0000","last_unstale":"2024-04-25T10:04:25.459057+0000","last_undegraded":"2024-04-25T10:04:25.459057+0000","last_fullsized":"2024-04-25T10:04:25.459057+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.470460+0000","last_change":"2024-04-25T10:04:25.468526+0000","last_active":"2024-04-25T10:04:25.470460+0000","last_peered":"2024-04-25T10:04:25.470460+0000","last_clean":"2024-04-25T10:04:25.470460+0000","last_became_active":"2024-04-25T10:04:25.468181+0000","last_became_peered":"2024-04-25T10:04:25.468181+0000","last_unstale":"2024-04-25T10:04:25.470460+0000","last_undegraded":"2024-04-25T10:04:25.470460+0000","last_fullsized":"2024-04-25T10:04:25.470460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.458948+0000","last_change":"2024-04-25T10:04:25.448768+0000","last_active":"2024-04-25T10:04:25.458948+0000","last_peered":"2024-04-25T10:04:25.458948+0000","last_clean":"2024-04-25T10:04:25.458948+0000","last_became_active":"2024-04-25T10:04:25.448208+0000","last_became_peered":"2024-04-25T10:04:25.448208+0000","last_unstale":"2024-04-25T10:04:25.458948+0000","last_undegraded":"2024-04-25T10:04:25.458948+0000","last_fullsized":"2024-04-25T10:04:25.458948+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.918327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479890+0000","last_change":"2024-04-25T10:04:25.475883+0000","last_active":"2024-04-25T10:04:25.479890+0000","last_peered":"2024-04-25T10:04:25.479890+0000","last_clean":"2024-04-25T10:04:25.479890+0000","last_became_active":"2024-04-25T10:04:25.475594+0000","last_became_peered":"2024-04-25T10:04:25.475594+0000","last_unstale":"2024-04-25T10:04:25.479890+0000","last_undegraded":"2024-04-25T10:04:25.479890+0000","last_fullsized":"2024-04-25T10:04:25.479890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475353+0000","last_change":"2024-04-25T10:04:25.461616+0000","last_active":"2024-04-25T10:04:25.475353+0000","last_peered":"2024-04-25T10:04:25.475353+0000","last_clean":"2024-04-25T10:04:25.475353+0000","last_became_active":"2024-04-25T10:04:25.461353+0000","last_became_peered":"2024-04-25T10:04:25.461353+0000","last_unstale":"2024-04-25T10:04:25.475353+0000","last_undegraded":"2024-04-25T10:04:25.475353+0000","last_fullsized":"2024-04-25T10:04:25.475353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:59.983038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.453769+0000","last_change":"2024-04-25T10:04:24.453769+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.453769+0000","last_undegraded":"2024-04-25T10:04:24.453769+0000","last_fullsized":"2024-04-25T10:04:24.453769+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475605+0000","last_change":"2024-04-25T10:04:25.469094+0000","last_active":"2024-04-25T10:04:25.475605+0000","last_peered":"2024-04-25T10:04:25.475605+0000","last_clean":"2024-04-25T10:04:25.475605+0000","last_became_active":"2024-04-25T10:04:25.468584+0000","last_became_peered":"2024-04-25T10:04:25.468584+0000","last_unstale":"2024-04-25T10:04:25.475605+0000","last_undegraded":"2024-04-25T10:04:25.475605+0000","last_fullsized":"2024-04-25T10:04:25.475605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.463317+0000","last_change":"2024-04-25T10:04:25.456279+0000","last_active":"2024-04-25T10:04:25.463317+0000","last_peered":"2024-04-25T10:04:25.463317+0000","last_clean":"2024-04-25T10:04:25.463317+0000","last_became_active":"2024-04-25T10:04:25.456002+0000","last_became_peered":"2024-04-25T10:04:25.456002+0000","last_unstale":"2024-04-25T10:04:25.463317+0000","last_undegraded":"2024-04-25T10:04:25.463317+0000","last_fullsized":"2024-04-25T10:04:25.463317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477920+0000","last_change":"2024-04-25T10:04:25.467692+0000","last_active":"2024-04-25T10:04:25.477920+0000","last_peered":"2024-04-25T10:04:25.477920+0000","last_clean":"2024-04-25T10:04:25.477920+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.477920+0000","last_undegraded":"2024-04-25T10:04:25.477920+0000","last_fullsized":"2024-04-25T10:04:25.477920+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477172+0000","last_change":"2024-04-25T10:04:25.469711+0000","last_active":"2024-04-25T10:04:25.477172+0000","last_peered":"2024-04-25T10:04:25.477172+0000","last_clean":"2024-04-25T10:04:25.477172+0000","last_became_active":"2024-04-25T10:04:25.469239+0000","last_became_peered":"2024-04-25T10:04:25.469239+0000","last_unstale":"2024-04-25T10:04:25.477172+0000","last_undegraded":"2024-04-25T10:04:25.477172+0000","last_fullsized":"2024-04-25T10:04:25.477172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483001+0000","last_change":"2024-04-25T10:04:25.479586+0000","last_active":"2024-04-25T10:04:25.483001+0000","last_peered":"2024-04-25T10:04:25.483001+0000","last_clean":"2024-04-25T10:04:25.483001+0000","last_became_active":"2024-04-25T10:04:25.478560+0000","last_became_peered":"2024-04-25T10:04:25.478560+0000","last_unstale":"2024-04-25T10:04:25.483001+0000","last_undegraded":"2024-04-25T10:04:25.483001+0000","last_fullsized":"2024-04-25T10:04:25.483001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471478+0000","last_change":"2024-04-25T10:04:25.467693+0000","last_active":"2024-04-25T10:04:25.471478+0000","last_peered":"2024-04-25T10:04:25.471478+0000","last_clean":"2024-04-25T10:04:25.471478+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.471478+0000","last_undegraded":"2024-04-25T10:04:25.471478+0000","last_fullsized":"2024-04-25T10:04:25.471478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.462788+0000","last_change":"2024-04-25T10:04:25.450560+0000","last_active":"2024-04-25T10:04:25.462788+0000","last_peered":"2024-04-25T10:04:25.462788+0000","last_clean":"2024-04-25T10:04:25.462788+0000","last_became_active":"2024-04-25T10:04:25.450242+0000","last_became_peered":"2024-04-25T10:04:25.450242+0000","last_unstale":"2024-04-25T10:04:25.462788+0000","last_undegraded":"2024-04-25T10:04:25.462788+0000","last_fullsized":"2024-04-25T10:04:25.462788+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:07.537002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471792+0000","last_change":"2024-04-25T10:04:25.463042+0000","last_active":"2024-04-25T10:04:25.471792+0000","last_peered":"2024-04-25T10:04:25.471792+0000","last_clean":"2024-04-25T10:04:25.471792+0000","last_became_active":"2024-04-25T10:04:25.460640+0000","last_became_peered":"2024-04-25T10:04:25.460640+0000","last_unstale":"2024-04-25T10:04:25.471792+0000","last_undegraded":"2024-04-25T10:04:25.471792+0000","last_fullsized":"2024-04-25T10:04:25.471792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460519+0000","last_change":"2024-04-25T10:04:25.455252+0000","last_active":"2024-04-25T10:04:25.460519+0000","last_peered":"2024-04-25T10:04:25.460519+0000","last_clean":"2024-04-25T10:04:25.460519+0000","last_became_active":"2024-04-25T10:04:25.454978+0000","last_became_peered":"2024-04-25T10:04:25.454978+0000","last_unstale":"2024-04-25T10:04:25.460519+0000","last_undegraded":"2024-04-25T10:04:25.460519+0000","last_fullsized":"2024-04-25T10:04:25.460519+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.447927+0000","last_change":"2024-04-25T10:04:24.447927+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.447927+0000","last_undegraded":"2024-04-25T10:04:24.447927+0000","last_fullsized":"2024-04-25T10:04:24.447927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480500+0000","last_change":"2024-04-25T10:04:25.477500+0000","last_active":"2024-04-25T10:04:25.480500+0000","last_peered":"2024-04-25T10:04:25.480500+0000","last_clean":"2024-04-25T10:04:25.480500+0000","last_became_active":"2024-04-25T10:04:25.475152+0000","last_became_peered":"2024-04-25T10:04:25.475152+0000","last_unstale":"2024-04-25T10:04:25.480500+0000","last_undegraded":"2024-04-25T10:04:25.480500+0000","last_fullsized":"2024-04-25T10:04:25.480500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472951+0000","last_change":"2024-04-25T10:04:25.466973+0000","last_active":"2024-04-25T10:04:25.472951+0000","last_peered":"2024-04-25T10:04:25.472951+0000","last_clean":"2024-04-25T10:04:25.472951+0000","last_became_active":"2024-04-25T10:04:25.466685+0000","last_became_peered":"2024-04-25T10:04:25.466685+0000","last_unstale":"2024-04-25T10:04:25.472951+0000","last_undegraded":"2024-04-25T10:04:25.472951+0000","last_fullsized":"2024-04-25T10:04:25.472951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.454102+0000","last_change":"2024-04-25T10:04:24.454102+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.454102+0000","last_undegraded":"2024-04-25T10:04:24.454102+0000","last_fullsized":"2024-04-25T10:04:24.454102+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.452442+0000","last_change":"2024-04-25T10:04:24.452442+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.452442+0000","last_undegraded":"2024-04-25T10:04:24.452442+0000","last_fullsized":"2024-04-25T10:04:24.452442+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.448730+0000","last_change":"2024-04-25T10:04:24.448730+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.448730+0000","last_undegraded":"2024-04-25T10:04:24.448730+0000","last_fullsized":"2024-04-25T10:04:24.448730+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472713+0000","last_change":"2024-04-25T10:04:25.465113+0000","last_active":"2024-04-25T10:04:25.472713+0000","last_peered":"2024-04-25T10:04:25.472713+0000","last_clean":"2024-04-25T10:04:25.472713+0000","last_became_active":"2024-04-25T10:04:25.464842+0000","last_became_peered":"2024-04-25T10:04:25.464842+0000","last_unstale":"2024-04-25T10:04:25.472713+0000","last_undegraded":"2024-04-25T10:04:25.472713+0000","last_fullsized":"2024-04-25T10:04:25.472713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.446134+0000","last_change":"2024-04-25T10:04:24.446134+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.446134+0000","last_undegraded":"2024-04-25T10:04:24.446134+0000","last_fullsized":"2024-04-25T10:04:24.446134+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474420+0000","last_change":"2024-04-25T10:04:25.465925+0000","last_active":"2024-04-25T10:04:25.474420+0000","last_peered":"2024-04-25T10:04:25.474420+0000","last_clean":"2024-04-25T10:04:25.474420+0000","last_became_active":"2024-04-25T10:04:25.465652+0000","last_became_peered":"2024-04-25T10:04:25.465652+0000","last_unstale":"2024-04-25T10:04:25.474420+0000","last_undegraded":"2024-04-25T10:04:25.474420+0000","last_fullsized":"2024-04-25T10:04:25.474420+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480477+0000","last_change":"2024-04-25T10:04:25.476769+0000","last_active":"2024-04-25T10:04:25.480477+0000","last_peered":"2024-04-25T10:04:25.480477+0000","last_clean":"2024-04-25T10:04:25.480477+0000","last_became_active":"2024-04-25T10:04:25.476451+0000","last_became_peered":"2024-04-25T10:04:25.476451+0000","last_unstale":"2024-04-25T10:04:25.480477+0000","last_undegraded":"2024-04-25T10:04:25.480477+0000","last_fullsized":"2024-04-25T10:04:25.480477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472747+0000","last_change":"2024-04-25T10:04:25.469665+0000","last_active":"2024-04-25T10:04:25.472747+0000","last_peered":"2024-04-25T10:04:25.472747+0000","last_clean":"2024-04-25T10:04:25.472747+0000","last_became_active":"2024-04-25T10:04:25.469374+0000","last_became_peered":"2024-04-25T10:04:25.469374+0000","last_unstale":"2024-04-25T10:04:25.472747+0000","last_undegraded":"2024-04-25T10:04:25.472747+0000","last_fullsized":"2024-04-25T10:04:25.472747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475056+0000","last_change":"2024-04-25T10:04:25.470230+0000","last_active":"2024-04-25T10:04:25.475056+0000","last_peered":"2024-04-25T10:04:25.475056+0000","last_clean":"2024-04-25T10:04:25.475056+0000","last_became_active":"2024-04-25T10:04:25.469402+0000","last_became_peered":"2024-04-25T10:04:25.469402+0000","last_unstale":"2024-04-25T10:04:25.475056+0000","last_undegraded":"2024-04-25T10:04:25.475056+0000","last_fullsized":"2024-04-25T10:04:25.475056+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:30.617703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.451885+0000","last_change":"2024-04-25T10:04:24.451885+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.451885+0000","last_undegraded":"2024-04-25T10:04:24.451885+0000","last_fullsized":"2024-04-25T10:04:24.451885+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465537+0000","last_change":"2024-04-25T10:04:25.464113+0000","last_active":"2024-04-25T10:04:25.465537+0000","last_peered":"2024-04-25T10:04:25.465537+0000","last_clean":"2024-04-25T10:04:25.465537+0000","last_became_active":"2024-04-25T10:04:25.463493+0000","last_became_peered":"2024-04-25T10:04:25.463493+0000","last_unstale":"2024-04-25T10:04:25.465537+0000","last_undegraded":"2024-04-25T10:04:25.465537+0000","last_fullsized":"2024-04-25T10:04:25.465537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474879+0000","last_change":"2024-04-25T10:04:25.469216+0000","last_active":"2024-04-25T10:04:25.474879+0000","last_peered":"2024-04-25T10:04:25.474879+0000","last_clean":"2024-04-25T10:04:25.474879+0000","last_became_active":"2024-04-25T10:04:25.466664+0000","last_became_peered":"2024-04-25T10:04:25.466664+0000","last_unstale":"2024-04-25T10:04:25.474879+0000","last_undegraded":"2024-04-25T10:04:25.474879+0000","last_fullsized":"2024-04-25T10:04:25.474879+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:35.597962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474510+0000","last_change":"2024-04-25T10:04:24.469487+0000","last_active":"2024-04-25T10:04:25.474510+0000","last_peered":"2024-04-25T10:04:25.474510+0000","last_clean":"2024-04-25T10:04:25.474510+0000","last_became_active":"2024-04-25T10:04:24.469185+0000","last_became_peered":"2024-04-25T10:04:24.469185+0000","last_unstale":"2024-04-25T10:04:25.474510+0000","last_undegraded":"2024-04-25T10:04:25.474510+0000","last_fullsized":"2024-04-25T10:04:25.474510+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479518+0000","last_change":"2024-04-25T10:04:25.473033+0000","last_active":"2024-04-25T10:04:25.479518+0000","last_peered":"2024-04-25T10:04:25.479518+0000","last_clean":"2024-04-25T10:04:25.479518+0000","last_became_active":"2024-04-25T10:04:25.471870+0000","last_became_peered":"2024-04-25T10:04:25.471870+0000","last_unstale":"2024-04-25T10:04:25.479518+0000","last_undegraded":"2024-04-25T10:04:25.479518+0000","last_fullsized":"2024-04-25T10:04:25.479518+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:32.751396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481374+0000","last_change":"2024-04-25T10:04:24.468749+0000","last_active":"2024-04-25T10:04:25.481374+0000","last_peered":"2024-04-25T10:04:25.481374+0000","last_clean":"2024-04-25T10:04:25.481374+0000","last_became_active":"2024-04-25T10:04:24.468458+0000","last_became_peered":"2024-04-25T10:04:24.468458+0000","last_unstale":"2024-04-25T10:04:25.481374+0000","last_undegraded":"2024-04-25T10:04:25.481374+0000","last_fullsized":"2024-04-25T10:04:25.481374+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.077398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480376+0000","last_change":"2024-04-25T10:04:25.478215+0000","last_active":"2024-04-25T10:04:25.480376+0000","last_peered":"2024-04-25T10:04:25.480376+0000","last_clean":"2024-04-25T10:04:25.480376+0000","last_became_active":"2024-04-25T10:04:25.475504+0000","last_became_peered":"2024-04-25T10:04:25.475504+0000","last_unstale":"2024-04-25T10:04:25.480376+0000","last_undegraded":"2024-04-25T10:04:25.480376+0000","last_fullsized":"2024-04-25T10:04:25.480376+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:36.035844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471375+0000","last_change":"2024-04-25T10:04:24.459064+0000","last_active":"2024-04-25T10:04:25.471375+0000","last_peered":"2024-04-25T10:04:25.471375+0000","last_clean":"2024-04-25T10:04:25.471375+0000","last_became_active":"2024-04-25T10:04:24.458599+0000","last_became_peered":"2024-04-25T10:04:24.458599+0000","last_unstale":"2024-04-25T10:04:25.471375+0000","last_undegraded":"2024-04-25T10:04:25.471375+0000","last_fullsized":"2024-04-25T10:04:25.471375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483933+0000","last_change":"2024-04-25T10:04:25.473270+0000","last_active":"2024-04-25T10:04:25.483933+0000","last_peered":"2024-04-25T10:04:25.483933+0000","last_clean":"2024-04-25T10:04:25.483933+0000","last_became_active":"2024-04-25T10:04:25.472978+0000","last_became_peered":"2024-04-25T10:04:25.472978+0000","last_unstale":"2024-04-25T10:04:25.483933+0000","last_undegraded":"2024-04-25T10:04:25.483933+0000","last_fullsized":"2024-04-25T10:04:25.483933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465518+0000","last_change":"2024-04-25T10:04:24.468225+0000","last_active":"2024-04-25T10:04:25.465518+0000","last_peered":"2024-04-25T10:04:25.465518+0000","last_clean":"2024-04-25T10:04:25.465518+0000","last_became_active":"2024-04-25T10:04:24.467670+0000","last_became_peered":"2024-04-25T10:04:24.467670+0000","last_unstale":"2024-04-25T10:04:25.465518+0000","last_undegraded":"2024-04-25T10:04:25.465518+0000","last_fullsized":"2024-04-25T10:04:25.465518+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472793+0000","last_change":"2024-04-25T10:04:25.465227+0000","last_active":"2024-04-25T10:04:25.472793+0000","last_peered":"2024-04-25T10:04:25.472793+0000","last_clean":"2024-04-25T10:04:25.472793+0000","last_became_active":"2024-04-25T10:04:25.464968+0000","last_became_peered":"2024-04-25T10:04:25.464968+0000","last_unstale":"2024-04-25T10:04:25.472793+0000","last_undegraded":"2024-04-25T10:04:25.472793+0000","last_fullsized":"2024-04-25T10:04:25.472793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.473107+0000","last_change":"2024-04-25T10:04:24.467424+0000","last_active":"2024-04-25T10:04:24.473107+0000","last_peered":"2024-04-25T10:04:24.473107+0000","last_clean":"2024-04-25T10:04:24.473107+0000","last_became_active":"2024-04-25T10:04:24.467101+0000","last_became_peered":"2024-04-25T10:04:24.467101+0000","last_unstale":"2024-04-25T10:04:24.473107+0000","last_undegraded":"2024-04-25T10:04:24.473107+0000","last_fullsized":"2024-04-25T10:04:24.473107+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:26.852084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479583+0000","last_change":"2024-04-25T10:04:25.471910+0000","last_active":"2024-04-25T10:04:25.479583+0000","last_peered":"2024-04-25T10:04:25.479583+0000","last_clean":"2024-04-25T10:04:25.479583+0000","last_became_active":"2024-04-25T10:04:25.471628+0000","last_became_peered":"2024-04-25T10:04:25.471628+0000","last_unstale":"2024-04-25T10:04:25.479583+0000","last_undegraded":"2024-04-25T10:04:25.479583+0000","last_fullsized":"2024-04-25T10:04:25.479583+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:15.400296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472460+0000","last_change":"2024-04-25T10:04:24.449636+0000","last_active":"2024-04-25T10:04:25.472460+0000","last_peered":"2024-04-25T10:04:25.472460+0000","last_clean":"2024-04-25T10:04:25.472460+0000","last_became_active":"2024-04-25T10:04:24.449303+0000","last_became_peered":"2024-04-25T10:04:24.449303+0000","last_unstale":"2024-04-25T10:04:25.472460+0000","last_undegraded":"2024-04-25T10:04:25.472460+0000","last_fullsized":"2024-04-25T10:04:25.472460+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:58.775423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475584+0000","last_change":"2024-04-25T10:04:25.468855+0000","last_active":"2024-04-25T10:04:25.475584+0000","last_peered":"2024-04-25T10:04:25.475584+0000","last_clean":"2024-04-25T10:04:25.475584+0000","last_became_active":"2024-04-25T10:04:25.467011+0000","last_became_peered":"2024-04-25T10:04:25.467011+0000","last_unstale":"2024-04-25T10:04:25.475584+0000","last_undegraded":"2024-04-25T10:04:25.475584+0000","last_fullsized":"2024-04-25T10:04:25.475584+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.458937+0000","last_change":"2024-04-25T10:04:24.469032+0000","last_active":"2024-04-25T10:04:25.458937+0000","last_peered":"2024-04-25T10:04:25.458937+0000","last_clean":"2024-04-25T10:04:25.458937+0000","last_became_active":"2024-04-25T10:04:24.468452+0000","last_became_peered":"2024-04-25T10:04:24.468452+0000","last_unstale":"2024-04-25T10:04:25.458937+0000","last_undegraded":"2024-04-25T10:04:25.458937+0000","last_fullsized":"2024-04-25T10:04:25.458937+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:49.522317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460571+0000","last_change":"2024-04-25T10:04:25.456211+0000","last_active":"2024-04-25T10:04:25.460571+0000","last_peered":"2024-04-25T10:04:25.460571+0000","last_clean":"2024-04-25T10:04:25.460571+0000","last_became_active":"2024-04-25T10:04:25.455945+0000","last_became_peered":"2024-04-25T10:04:25.455945+0000","last_unstale":"2024-04-25T10:04:25.460571+0000","last_undegraded":"2024-04-25T10:04:25.460571+0000","last_fullsized":"2024-04-25T10:04:25.460571+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.473210+0000","last_change":"2024-04-25T10:04:24.468238+0000","last_active":"2024-04-25T10:04:24.473210+0000","last_peered":"2024-04-25T10:04:24.473210+0000","last_clean":"2024-04-25T10:04:24.473210+0000","last_became_active":"2024-04-25T10:04:24.467961+0000","last_became_peered":"2024-04-25T10:04:24.467961+0000","last_unstale":"2024-04-25T10:04:24.473210+0000","last_undegraded":"2024-04-25T10:04:24.473210+0000","last_fullsized":"2024-04-25T10:04:24.473210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.084728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472666+0000","last_change":"2024-04-25T10:04:25.462850+0000","last_active":"2024-04-25T10:04:25.472666+0000","last_peered":"2024-04-25T10:04:25.472666+0000","last_clean":"2024-04-25T10:04:25.472666+0000","last_became_active":"2024-04-25T10:04:25.462560+0000","last_became_peered":"2024-04-25T10:04:25.462560+0000","last_unstale":"2024-04-25T10:04:25.472666+0000","last_undegraded":"2024-04-25T10:04:25.472666+0000","last_fullsized":"2024-04-25T10:04:25.472666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.444069+0000","last_change":"2024-04-25T10:04:24.447649+0000","last_active":"2024-04-25T10:04:25.444069+0000","last_peered":"2024-04-25T10:04:25.444069+0000","last_clean":"2024-04-25T10:04:25.444069+0000","last_became_active":"2024-04-25T10:04:24.446781+0000","last_became_peered":"2024-04-25T10:04:24.446781+0000","last_unstale":"2024-04-25T10:04:25.444069+0000","last_undegraded":"2024-04-25T10:04:25.444069+0000","last_fullsized":"2024-04-25T10:04:25.444069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.029749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.457360+0000","last_change":"2024-04-25T10:04:25.452425+0000","last_active":"2024-04-25T10:04:25.457360+0000","last_peered":"2024-04-25T10:04:25.457360+0000","last_clean":"2024-04-25T10:04:25.457360+0000","last_became_active":"2024-04-25T10:04:25.451175+0000","last_became_peered":"2024-04-25T10:04:25.451175+0000","last_unstale":"2024-04-25T10:04:25.457360+0000","last_undegraded":"2024-04-25T10:04:25.457360+0000","last_fullsized":"2024-04-25T10:04:25.457360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.443720+0000","last_change":"2024-04-25T10:04:24.451547+0000","last_active":"2024-04-25T10:04:25.443720+0000","last_peered":"2024-04-25T10:04:25.443720+0000","last_clean":"2024-04-25T10:04:25.443720+0000","last_became_active":"2024-04-25T10:04:24.451221+0000","last_became_peered":"2024-04-25T10:04:24.451221+0000","last_unstale":"2024-04-25T10:04:25.443720+0000","last_undegraded":"2024-04-25T10:04:25.443720+0000","last_fullsized":"2024-04-25T10:04:25.443720+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472556+0000","last_change":"2024-04-25T10:04:25.466717+0000","last_active":"2024-04-25T10:04:25.472556+0000","last_peered":"2024-04-25T10:04:25.472556+0000","last_clean":"2024-04-25T10:04:25.472556+0000","last_became_active":"2024-04-25T10:04:25.466392+0000","last_became_peered":"2024-04-25T10:04:25.466392+0000","last_unstale":"2024-04-25T10:04:25.472556+0000","last_undegraded":"2024-04-25T10:04:25.472556+0000","last_fullsized":"2024-04-25T10:04:25.472556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480231+0000","last_change":"2024-04-25T10:04:24.462694+0000","last_active":"2024-04-25T10:04:25.480231+0000","last_peered":"2024-04-25T10:04:25.480231+0000","last_clean":"2024-04-25T10:04:25.480231+0000","last_became_active":"2024-04-25T10:04:24.462301+0000","last_became_peered":"2024-04-25T10:04:24.462301+0000","last_unstale":"2024-04-25T10:04:25.480231+0000","last_undegraded":"2024-04-25T10:04:25.480231+0000","last_fullsized":"2024-04-25T10:04:25.480231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.011021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469006+0000","last_change":"2024-04-25T10:04:25.466716+0000","last_active":"2024-04-25T10:04:25.469006+0000","last_peered":"2024-04-25T10:04:25.469006+0000","last_clean":"2024-04-25T10:04:25.469006+0000","last_became_active":"2024-04-25T10:04:25.466438+0000","last_became_peered":"2024-04-25T10:04:25.466438+0000","last_unstale":"2024-04-25T10:04:25.469006+0000","last_undegraded":"2024-04-25T10:04:25.469006+0000","last_fullsized":"2024-04-25T10:04:25.469006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.486863+0000","last_change":"2024-04-25T10:04:24.469012+0000","last_active":"2024-04-25T10:04:25.486863+0000","last_peered":"2024-04-25T10:04:25.486863+0000","last_clean":"2024-04-25T10:04:25.486863+0000","last_became_active":"2024-04-25T10:04:24.468729+0000","last_became_peered":"2024-04-25T10:04:24.468729+0000","last_unstale":"2024-04-25T10:04:25.486863+0000","last_undegraded":"2024-04-25T10:04:25.486863+0000","last_fullsized":"2024-04-25T10:04:25.486863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474650+0000","last_change":"2024-04-25T10:04:25.468212+0000","last_active":"2024-04-25T10:04:25.474650+0000","last_peered":"2024-04-25T10:04:25.474650+0000","last_clean":"2024-04-25T10:04:25.474650+0000","last_became_active":"2024-04-25T10:04:25.467940+0000","last_became_peered":"2024-04-25T10:04:25.467940+0000","last_unstale":"2024-04-25T10:04:25.474650+0000","last_undegraded":"2024-04-25T10:04:25.474650+0000","last_fullsized":"2024-04-25T10:04:25.474650+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:04.217886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482369+0000","last_change":"2024-04-25T10:04:24.456904+0000","last_active":"2024-04-25T10:04:25.482369+0000","last_peered":"2024-04-25T10:04:25.482369+0000","last_clean":"2024-04-25T10:04:25.482369+0000","last_became_active":"2024-04-25T10:04:24.456095+0000","last_became_peered":"2024-04-25T10:04:24.456095+0000","last_unstale":"2024-04-25T10:04:25.482369+0000","last_undegraded":"2024-04-25T10:04:25.482369+0000","last_fullsized":"2024-04-25T10:04:25.482369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.185120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482977+0000","last_change":"2024-04-25T10:04:25.479170+0000","last_active":"2024-04-25T10:04:25.482977+0000","last_peered":"2024-04-25T10:04:25.482977+0000","last_clean":"2024-04-25T10:04:25.482977+0000","last_became_active":"2024-04-25T10:04:25.478980+0000","last_became_peered":"2024-04-25T10:04:25.478980+0000","last_unstale":"2024-04-25T10:04:25.482977+0000","last_undegraded":"2024-04-25T10:04:25.482977+0000","last_fullsized":"2024-04-25T10:04:25.482977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.442376+0000","last_change":"2024-04-25T10:04:24.458424+0000","last_active":"2024-04-25T10:04:25.442376+0000","last_peered":"2024-04-25T10:04:25.442376+0000","last_clean":"2024-04-25T10:04:25.442376+0000","last_became_active":"2024-04-25T10:04:24.458098+0000","last_became_peered":"2024-04-25T10:04:24.458098+0000","last_unstale":"2024-04-25T10:04:25.442376+0000","last_undegraded":"2024-04-25T10:04:25.442376+0000","last_fullsized":"2024-04-25T10:04:25.442376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469999+0000","last_change":"2024-04-25T10:04:25.455450+0000","last_active":"2024-04-25T10:04:25.469999+0000","last_peered":"2024-04-25T10:04:25.469999+0000","last_clean":"2024-04-25T10:04:25.469999+0000","last_became_active":"2024-04-25T10:04:25.454137+0000","last_became_peered":"2024-04-25T10:04:25.454137+0000","last_unstale":"2024-04-25T10:04:25.469999+0000","last_undegraded":"2024-04-25T10:04:25.469999+0000","last_fullsized":"2024-04-25T10:04:25.469999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:11.030611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446148+0000","last_change":"2024-04-25T10:04:24.445036+0000","last_active":"2024-04-25T10:04:25.446148+0000","last_peered":"2024-04-25T10:04:25.446148+0000","last_clean":"2024-04-25T10:04:25.446148+0000","last_became_active":"2024-04-25T10:04:24.444715+0000","last_became_peered":"2024-04-25T10:04:24.444715+0000","last_unstale":"2024-04-25T10:04:25.446148+0000","last_undegraded":"2024-04-25T10:04:25.446148+0000","last_fullsized":"2024-04-25T10:04:25.446148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.905326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.453241+0000","last_change":"2024-04-25T10:04:24.453241+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.453241+0000","last_undegraded":"2024-04-25T10:04:24.453241+0000","last_fullsized":"2024-04-25T10:04:24.453241+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482536+0000","last_change":"2024-04-25T10:04:24.465436+0000","last_active":"2024-04-25T10:04:25.482536+0000","last_peered":"2024-04-25T10:04:25.482536+0000","last_clean":"2024-04-25T10:04:25.482536+0000","last_became_active":"2024-04-25T10:04:24.465132+0000","last_became_peered":"2024-04-25T10:04:24.465132+0000","last_unstale":"2024-04-25T10:04:25.482536+0000","last_undegraded":"2024-04-25T10:04:25.482536+0000","last_fullsized":"2024-04-25T10:04:25.482536+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.884017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479931+0000","last_change":"2024-04-25T10:04:25.472528+0000","last_active":"2024-04-25T10:04:25.479931+0000","last_peered":"2024-04-25T10:04:25.479931+0000","last_clean":"2024-04-25T10:04:25.479931+0000","last_became_active":"2024-04-25T10:04:25.472245+0000","last_became_peered":"2024-04-25T10:04:25.472245+0000","last_unstale":"2024-04-25T10:04:25.479931+0000","last_undegraded":"2024-04-25T10:04:25.479931+0000","last_fullsized":"2024-04-25T10:04:25.479931+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481219+0000","last_change":"2024-04-25T10:04:24.468526+0000","last_active":"2024-04-25T10:04:25.481219+0000","last_peered":"2024-04-25T10:04:25.481219+0000","last_clean":"2024-04-25T10:04:25.481219+0000","last_became_active":"2024-04-25T10:04:24.468213+0000","last_became_peered":"2024-04-25T10:04:24.468213+0000","last_unstale":"2024-04-25T10:04:25.481219+0000","last_undegraded":"2024-04-25T10:04:25.481219+0000","last_fullsized":"2024-04-25T10:04:25.481219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:40.655704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472621+0000","last_change":"2024-04-25T10:04:25.463721+0000","last_active":"2024-04-25T10:04:25.472621+0000","last_peered":"2024-04-25T10:04:25.472621+0000","last_clean":"2024-04-25T10:04:25.472621+0000","last_became_active":"2024-04-25T10:04:25.463355+0000","last_became_peered":"2024-04-25T10:04:25.463355+0000","last_unstale":"2024-04-25T10:04:25.472621+0000","last_undegraded":"2024-04-25T10:04:25.472621+0000","last_fullsized":"2024-04-25T10:04:25.472621+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481630+0000","last_change":"2024-04-25T10:04:24.461901+0000","last_active":"2024-04-25T10:04:25.481630+0000","last_peered":"2024-04-25T10:04:25.481630+0000","last_clean":"2024-04-25T10:04:25.481630+0000","last_became_active":"2024-04-25T10:04:24.461619+0000","last_became_peered":"2024-04-25T10:04:24.461619+0000","last_unstale":"2024-04-25T10:04:25.481630+0000","last_undegraded":"2024-04-25T10:04:25.481630+0000","last_fullsized":"2024-04-25T10:04:25.481630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:32.072513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480737+0000","last_change":"2024-04-25T10:04:25.477724+0000","last_active":"2024-04-25T10:04:25.480737+0000","last_peered":"2024-04-25T10:04:25.480737+0000","last_clean":"2024-04-25T10:04:25.480737+0000","last_became_active":"2024-04-25T10:04:25.476438+0000","last_became_peered":"2024-04-25T10:04:25.476438+0000","last_unstale":"2024-04-25T10:04:25.480737+0000","last_undegraded":"2024-04-25T10:04:25.480737+0000","last_fullsized":"2024-04-25T10:04:25.480737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475376+0000","last_change":"2024-04-25T10:04:24.465620+0000","last_active":"2024-04-25T10:04:25.475376+0000","last_peered":"2024-04-25T10:04:25.475376+0000","last_clean":"2024-04-25T10:04:25.475376+0000","last_became_active":"2024-04-25T10:04:24.465402+0000","last_became_peered":"2024-04-25T10:04:24.465402+0000","last_unstale":"2024-04-25T10:04:25.475376+0000","last_undegraded":"2024-04-25T10:04:25.475376+0000","last_fullsized":"2024-04-25T10:04:25.475376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474444+0000","last_change":"2024-04-25T10:04:25.463940+0000","last_active":"2024-04-25T10:04:25.474444+0000","last_peered":"2024-04-25T10:04:25.474444+0000","last_clean":"2024-04-25T10:04:25.474444+0000","last_became_active":"2024-04-25T10:04:25.463675+0000","last_became_peered":"2024-04-25T10:04:25.463675+0000","last_unstale":"2024-04-25T10:04:25.474444+0000","last_undegraded":"2024-04-25T10:04:25.474444+0000","last_fullsized":"2024-04-25T10:04:25.474444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.456328+0000","last_change":"2024-04-25T10:04:24.466373+0000","last_active":"2024-04-25T10:04:25.456328+0000","last_peered":"2024-04-25T10:04:25.456328+0000","last_clean":"2024-04-25T10:04:25.456328+0000","last_became_active":"2024-04-25T10:04:24.465965+0000","last_became_peered":"2024-04-25T10:04:24.465965+0000","last_unstale":"2024-04-25T10:04:25.456328+0000","last_undegraded":"2024-04-25T10:04:25.456328+0000","last_fullsized":"2024-04-25T10:04:25.456328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.478776+0000","last_change":"2024-04-25T10:04:25.471532+0000","last_active":"2024-04-25T10:04:25.478776+0000","last_peered":"2024-04-25T10:04:25.478776+0000","last_clean":"2024-04-25T10:04:25.478776+0000","last_became_active":"2024-04-25T10:04:25.471276+0000","last_became_peered":"2024-04-25T10:04:25.471276+0000","last_unstale":"2024-04-25T10:04:25.478776+0000","last_undegraded":"2024-04-25T10:04:25.478776+0000","last_fullsized":"2024-04-25T10:04:25.478776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.468521+0000","last_change":"2024-04-25T10:04:24.461754+0000","last_active":"2024-04-25T10:04:25.468521+0000","last_peered":"2024-04-25T10:04:25.468521+0000","last_clean":"2024-04-25T10:04:25.468521+0000","last_became_active":"2024-04-25T10:04:24.459617+0000","last_became_peered":"2024-04-25T10:04:24.459617+0000","last_unstale":"2024-04-25T10:04:25.468521+0000","last_undegraded":"2024-04-25T10:04:25.468521+0000","last_fullsized":"2024-04-25T10:04:25.468521+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.473141+0000","last_change":"2024-04-25T10:04:25.468223+0000","last_active":"2024-04-25T10:04:25.473141+0000","last_peered":"2024-04-25T10:04:25.473141+0000","last_clean":"2024-04-25T10:04:25.473141+0000","last_became_active":"2024-04-25T10:04:25.467369+0000","last_became_peered":"2024-04-25T10:04:25.467369+0000","last_unstale":"2024-04-25T10:04:25.473141+0000","last_undegraded":"2024-04-25T10:04:25.473141+0000","last_fullsized":"2024-04-25T10:04:25.473141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.475694+0000","last_change":"2024-04-25T10:04:24.471952+0000","last_active":"2024-04-25T10:04:24.475694+0000","last_peered":"2024-04-25T10:04:24.475694+0000","last_clean":"2024-04-25T10:04:24.475694+0000","last_became_active":"2024-04-25T10:04:24.471642+0000","last_became_peered":"2024-04-25T10:04:24.471642+0000","last_unstale":"2024-04-25T10:04:24.475694+0000","last_undegraded":"2024-04-25T10:04:24.475694+0000","last_fullsized":"2024-04-25T10:04:24.475694+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:48.562510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472693+0000","last_change":"2024-04-25T10:04:25.468105+0000","last_active":"2024-04-25T10:04:25.472693+0000","last_peered":"2024-04-25T10:04:25.472693+0000","last_clean":"2024-04-25T10:04:25.472693+0000","last_became_active":"2024-04-25T10:04:25.467841+0000","last_became_peered":"2024-04-25T10:04:25.467841+0000","last_unstale":"2024-04-25T10:04:25.472693+0000","last_undegraded":"2024-04-25T10:04:25.472693+0000","last_fullsized":"2024-04-25T10:04:25.472693+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.451244+0000","last_change":"2024-04-25T10:04:24.469768+0000","last_active":"2024-04-25T10:04:25.451244+0000","last_peered":"2024-04-25T10:04:25.451244+0000","last_clean":"2024-04-25T10:04:25.451244+0000","last_became_active":"2024-04-25T10:04:24.469488+0000","last_became_peered":"2024-04-25T10:04:24.469488+0000","last_unstale":"2024-04-25T10:04:25.451244+0000","last_undegraded":"2024-04-25T10:04:25.451244+0000","last_fullsized":"2024-04-25T10:04:25.451244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:29.785579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480787+0000","last_change":"2024-04-25T10:04:25.475514+0000","last_active":"2024-04-25T10:04:25.480787+0000","last_peered":"2024-04-25T10:04:25.480787+0000","last_clean":"2024-04-25T10:04:25.480787+0000","last_became_active":"2024-04-25T10:04:25.474364+0000","last_became_peered":"2024-04-25T10:04:25.474364+0000","last_unstale":"2024-04-25T10:04:25.480787+0000","last_undegraded":"2024-04-25T10:04:25.480787+0000","last_fullsized":"2024-04-25T10:04:25.480787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.453420+0000","last_change":"2024-04-25T10:04:24.460278+0000","last_active":"2024-04-25T10:04:25.453420+0000","last_peered":"2024-04-25T10:04:25.453420+0000","last_clean":"2024-04-25T10:04:25.453420+0000","last_became_active":"2024-04-25T10:04:24.459936+0000","last_became_peered":"2024-04-25T10:04:24.459936+0000","last_unstale":"2024-04-25T10:04:25.453420+0000","last_undegraded":"2024-04-25T10:04:25.453420+0000","last_fullsized":"2024-04-25T10:04:25.453420+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:19.332266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465336+0000","last_change":"2024-04-25T10:04:25.456950+0000","last_active":"2024-04-25T10:04:25.465336+0000","last_peered":"2024-04-25T10:04:25.465336+0000","last_clean":"2024-04-25T10:04:25.465336+0000","last_became_active":"2024-04-25T10:04:25.456664+0000","last_became_peered":"2024-04-25T10:04:25.456664+0000","last_unstale":"2024-04-25T10:04:25.465336+0000","last_undegraded":"2024-04-25T10:04:25.465336+0000","last_fullsized":"2024-04-25T10:04:25.465336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474794+0000","last_change":"2024-04-25T10:04:24.466995+0000","last_active":"2024-04-25T10:04:25.474794+0000","last_peered":"2024-04-25T10:04:25.474794+0000","last_clean":"2024-04-25T10:04:25.474794+0000","last_became_active":"2024-04-25T10:04:24.466667+0000","last_became_peered":"2024-04-25T10:04:24.466667+0000","last_unstale":"2024-04-25T10:04:25.474794+0000","last_undegraded":"2024-04-25T10:04:25.474794+0000","last_fullsized":"2024-04-25T10:04:25.474794+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:07.560341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475023+0000","last_change":"2024-04-25T10:04:25.467788+0000","last_active":"2024-04-25T10:04:25.475023+0000","last_peered":"2024-04-25T10:04:25.475023+0000","last_clean":"2024-04-25T10:04:25.475023+0000","last_became_active":"2024-04-25T10:04:25.465574+0000","last_became_peered":"2024-04-25T10:04:25.465574+0000","last_unstale":"2024-04-25T10:04:25.475023+0000","last_undegraded":"2024-04-25T10:04:25.475023+0000","last_fullsized":"2024-04-25T10:04:25.475023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.459728+0000","last_change":"2024-04-25T10:04:24.463577+0000","last_active":"2024-04-25T10:04:25.459728+0000","last_peered":"2024-04-25T10:04:25.459728+0000","last_clean":"2024-04-25T10:04:25.459728+0000","last_became_active":"2024-04-25T10:04:24.462318+0000","last_became_peered":"2024-04-25T10:04:24.462318+0000","last_unstale":"2024-04-25T10:04:25.459728+0000","last_undegraded":"2024-04-25T10:04:25.459728+0000","last_fullsized":"2024-04-25T10:04:25.459728+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:01.690130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-25T10:04:24.444965+0000","last_change":"2024-04-25T10:04:24.444965+0000","last_active":"2024-04-25T10:04:24.411381+0000","last_peered":"2024-04-25T10:04:24.411381+0000","last_clean":"2024-04-25T10:04:24.411381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:24.444965+0000","last_undegraded":"2024-04-25T10:04:24.444965+0000","last_fullsized":"2024-04-25T10:04:24.444965+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.472868+0000","last_change":"2024-04-25T10:04:24.462828+0000","last_active":"2024-04-25T10:04:24.472868+0000","last_peered":"2024-04-25T10:04:24.472868+0000","last_clean":"2024-04-25T10:04:24.472868+0000","last_became_active":"2024-04-25T10:04:24.462443+0000","last_became_peered":"2024-04-25T10:04:24.462443+0000","last_unstale":"2024-04-25T10:04:24.472868+0000","last_undegraded":"2024-04-25T10:04:24.472868+0000","last_fullsized":"2024-04-25T10:04:24.472868+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:06.233856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475078+0000","last_change":"2024-04-25T10:04:25.466945+0000","last_active":"2024-04-25T10:04:25.475078+0000","last_peered":"2024-04-25T10:04:25.475078+0000","last_clean":"2024-04-25T10:04:25.475078+0000","last_became_active":"2024-04-25T10:04:25.466672+0000","last_became_peered":"2024-04-25T10:04:25.466672+0000","last_unstale":"2024-04-25T10:04:25.475078+0000","last_undegraded":"2024-04-25T10:04:25.475078+0000","last_fullsized":"2024-04-25T10:04:25.475078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.450703+0000","last_change":"2024-04-25T10:04:24.461824+0000","last_active":"2024-04-25T10:04:25.450703+0000","last_peered":"2024-04-25T10:04:25.450703+0000","last_clean":"2024-04-25T10:04:25.450703+0000","last_became_active":"2024-04-25T10:04:24.460800+0000","last_became_peered":"2024-04-25T10:04:24.460800+0000","last_unstale":"2024-04-25T10:04:25.450703+0000","last_undegraded":"2024-04-25T10:04:25.450703+0000","last_fullsized":"2024-04-25T10:04:25.450703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:25.199121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480645+0000","last_change":"2024-04-25T10:04:25.477935+0000","last_active":"2024-04-25T10:04:25.480645+0000","last_peered":"2024-04-25T10:04:25.480645+0000","last_clean":"2024-04-25T10:04:25.480645+0000","last_became_active":"2024-04-25T10:04:25.477605+0000","last_became_peered":"2024-04-25T10:04:25.477605+0000","last_unstale":"2024-04-25T10:04:25.480645+0000","last_undegraded":"2024-04-25T10:04:25.480645+0000","last_fullsized":"2024-04-25T10:04:25.480645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-25T10:04:24.473344+0000","last_change":"2024-04-25T10:04:24.471290+0000","last_active":"2024-04-25T10:04:24.473344+0000","last_peered":"2024-04-25T10:04:24.473344+0000","last_clean":"2024-04-25T10:04:24.473344+0000","last_became_active":"2024-04-25T10:04:24.470984+0000","last_became_peered":"2024-04-25T10:04:24.470984+0000","last_unstale":"2024-04-25T10:04:24.473344+0000","last_undegraded":"2024-04-25T10:04:24.473344+0000","last_fullsized":"2024-04-25T10:04:24.473344+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:22.705602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475556+0000","last_change":"2024-04-25T10:04:25.462505+0000","last_active":"2024-04-25T10:04:25.475556+0000","last_peered":"2024-04-25T10:04:25.475556+0000","last_clean":"2024-04-25T10:04:25.475556+0000","last_became_active":"2024-04-25T10:04:25.461056+0000","last_became_peered":"2024-04-25T10:04:25.461056+0000","last_unstale":"2024-04-25T10:04:25.475556+0000","last_undegraded":"2024-04-25T10:04:25.475556+0000","last_fullsized":"2024-04-25T10:04:25.475556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446296+0000","last_change":"2024-04-25T10:04:24.443846+0000","last_active":"2024-04-25T10:04:25.446296+0000","last_peered":"2024-04-25T10:04:25.446296+0000","last_clean":"2024-04-25T10:04:25.446296+0000","last_became_active":"2024-04-25T10:04:24.443511+0000","last_became_peered":"2024-04-25T10:04:24.443511+0000","last_unstale":"2024-04-25T10:04:25.446296+0000","last_undegraded":"2024-04-25T10:04:25.446296+0000","last_fullsized":"2024-04-25T10:04:25.446296+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475649+0000","last_change":"2024-04-25T10:04:25.466904+0000","last_active":"2024-04-25T10:04:25.475649+0000","last_peered":"2024-04-25T10:04:25.475649+0000","last_clean":"2024-04-25T10:04:25.475649+0000","last_became_active":"2024-04-25T10:04:25.466434+0000","last_became_peered":"2024-04-25T10:04:25.466434+0000","last_unstale":"2024-04-25T10:04:25.475649+0000","last_undegraded":"2024-04-25T10:04:25.475649+0000","last_fullsized":"2024-04-25T10:04:25.475649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441933+0000","last_change":"2024-04-25T10:04:24.467188+0000","last_active":"2024-04-25T10:04:25.441933+0000","last_peered":"2024-04-25T10:04:25.441933+0000","last_clean":"2024-04-25T10:04:25.441933+0000","last_became_active":"2024-04-25T10:04:24.466850+0000","last_became_peered":"2024-04-25T10:04:24.466850+0000","last_unstale":"2024-04-25T10:04:25.441933+0000","last_undegraded":"2024-04-25T10:04:25.441933+0000","last_fullsized":"2024-04-25T10:04:25.441933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472620+0000","last_change":"2024-04-25T10:04:25.461224+0000","last_active":"2024-04-25T10:04:25.472620+0000","last_peered":"2024-04-25T10:04:25.472620+0000","last_clean":"2024-04-25T10:04:25.472620+0000","last_became_active":"2024-04-25T10:04:25.460727+0000","last_became_peered":"2024-04-25T10:04:25.460727+0000","last_unstale":"2024-04-25T10:04:25.472620+0000","last_undegraded":"2024-04-25T10:04:25.472620+0000","last_fullsized":"2024-04-25T10:04:25.472620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.452980+0000","last_change":"2024-04-25T10:04:24.461091+0000","last_active":"2024-04-25T10:04:25.452980+0000","last_peered":"2024-04-25T10:04:25.452980+0000","last_clean":"2024-04-25T10:04:25.452980+0000","last_became_active":"2024-04-25T10:04:24.460322+0000","last_became_peered":"2024-04-25T10:04:24.460322+0000","last_unstale":"2024-04-25T10:04:25.452980+0000","last_undegraded":"2024-04-25T10:04:25.452980+0000","last_fullsized":"2024-04-25T10:04:25.452980+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:17.560293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441995+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:04:25.441995+0000","last_peered":"2024-04-25T10:04:25.441995+0000","last_clean":"2024-04-25T10:04:25.441995+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:04:25.441995+0000","last_undegraded":"2024-04-25T10:04:25.441995+0000","last_fullsized":"2024-04-25T10:04:25.441995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":241488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:30.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:30.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.711+0000 7f856be00640 1 Processor -- start 2024-04-25T10:04:30.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.711+0000 7f856be00640 1 -- start start 2024-04-25T10:04:30.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.712+0000 7f856be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f856415db00 0x7f85640d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.712+0000 7f856be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85640ce2b0 0x7f85640d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f856be00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85640d1e80 con 0x7f85640b53b0 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f856be00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85640b5930 con 0x7f85640ce2b0 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f856be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85640b5c10 con 0x7f856415db00 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8568a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f85640b53b0 0x7f8564161670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60564/0 (socket says 172.21.15.5:60564) 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8568a00640 1 -- 172.21.15.5:0/3831333375 learned_addr learned my addr 172.21.15.5:0/3831333375 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8569e00640 1 --2- 172.21.15.5:0/3831333375 >> v2:172.21.15.5:3300/0 conn(0x7f85640ce2b0 0x7f85640d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8569400640 1 --2- 172.21.15.5:0/3831333375 >> v2:172.21.15.80:3300/0 conn(0x7f856415db00 0x7f85640d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8563e00640 1 -- 172.21.15.5:0/3831333375 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901613130 0 0) 0x7f85640d1e80 con 0x7f85640b53b0 2024-04-25T10:04:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8569e00640 1 -- 172.21.15.5:0/3831333375 >> v2:172.21.15.80:3300/0 conn(0x7f856415db00 msgr2=0x7f85640d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8569e00640 1 --2- 172.21.15.5:0/3831333375 >> v2:172.21.15.80:3300/0 conn(0x7f856415db00 0x7f85640d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8569e00640 1 -- 172.21.15.5:0/3831333375 >> v1:172.21.15.5:6789/0 conn(0x7f85640b53b0 legacy=0x7f8564161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.713+0000 7f8569e00640 1 -- 172.21.15.5:0/3831333375 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640b5ef0 con 0x7f85640ce2b0 2024-04-25T10:04:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.714+0000 7f8569e00640 1 --2- 172.21.15.5:0/3831333375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640ce2b0 0x7f85640d0880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f85640d4940 tx=0x7f855c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9bc2ed79c64026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.715+0000 7f8563e00640 1 -- 172.21.15.5:0/3831333375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c007740 con 0x7f85640ce2b0 2024-04-25T10:04:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.715+0000 7f8563e00640 1 -- 172.21.15.5:0/3831333375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c0078e0 con 0x7f85640ce2b0 2024-04-25T10:04:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.715+0000 7f8563e00640 1 -- 172.21.15.5:0/3831333375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c007cb0 con 0x7f85640ce2b0 2024-04-25T10:04:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.715+0000 7f856be00640 1 -- 172.21.15.5:0/3831333375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640ce2b0 msgr2=0x7f85640d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.715+0000 7f856be00640 1 --2- 172.21.15.5:0/3831333375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640ce2b0 0x7f85640d0880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f85640d4940 tx=0x7f855c02a680 comp rx=0 tx=0).stop 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.716+0000 7f856be00640 1 -- 172.21.15.5:0/3831333375 shutdown_connections 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.716+0000 7f856be00640 1 --2- 172.21.15.5:0/3831333375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640ce2b0 0x7f85640d0880 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.716+0000 7f856be00640 1 --2- 172.21.15.5:0/3831333375 >> v2:172.21.15.80:3300/0 conn(0x7f856415db00 0x7f85640d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.716+0000 7f856be00640 1 -- 172.21.15.5:0/3831333375 >> 172.21.15.5:0/3831333375 conn(0x7f8564072290 msgr2=0x7f85640726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.716+0000 7f856be00640 1 -- 172.21.15.5:0/3831333375 shutdown_connections 2024-04-25T10:04:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.716+0000 7f856be00640 1 -- 172.21.15.5:0/3831333375 wait complete. 2024-04-25T10:04:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.717+0000 7f856be00640 1 Processor -- start 2024-04-25T10:04:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.718+0000 7f856be00640 1 -- start start 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f856be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 0x7f856414a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f856be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85640ce2b0 0x7f85641545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f856be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f856415db00 0x7f85640cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f856be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640d3660 con 0x7f85640b53b0 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f856be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640b61d0 con 0x7f85640ce2b0 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f856be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85640b6c10 con 0x7f856415db00 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 0x7f856414a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 0x7f856414a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57740/0 (socket says 172.21.15.5:57740) 2024-04-25T10:04:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 -- 172.21.15.5:0/2118799924 learned_addr learned my addr 172.21.15.5:0/2118799924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 -- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f856415db00 msgr2=0x7f85640cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f856415db00 0x7f85640cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 -- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85640ce2b0 msgr2=0x7f85641545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85640ce2b0 0x7f85641545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.719+0000 7f8569400640 1 -- 172.21.15.5:0/2118799924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640caaa0 con 0x7f85640b53b0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.720+0000 7f8569400640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 0x7f856414a280 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f855800d620 tx=0x7f855800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.720+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85580125f0 con 0x7f85640b53b0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.720+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8558019070 con 0x7f85640b53b0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.720+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85580129b0 con 0x7f85640b53b0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.720+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85640cad80 con 0x7f85640b53b0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.720+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85640c86c0 con 0x7f85640b53b0 2024-04-25T10:04:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.721+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8564068fd0 con 0x7f85640b53b0 2024-04-25T10:04:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.723+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8558012b50 con 0x7f85640b53b0 2024-04-25T10:04:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.724+0000 7f8561600640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f854806b550 0x7f854806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.724+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f855808a700 con 0x7f85640b53b0 2024-04-25T10:04:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.724+0000 7f8568a00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f854806b550 0x7f854806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.726+0000 7f8568a00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f854806b550 0x7f854806da10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8554007fd0 tx=0x7f8554007920 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.729+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85580593d0 con 0x7f85640b53b0 2024-04-25T10:04:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.923+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85640cfc00 con 0x7f854806b550 2024-04-25T10:04:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.942+0000 7f8561600640 1 -- 172.21.15.5:0/2118799924 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251904 (secure 0 0 0) 0x7f8554053810 con 0x7f854806b550 2024-04-25T10:04:30.949 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.947+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f854806b550 msgr2=0x7f854806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.947+0000 7f856be00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f854806b550 0x7f854806da10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8554007fd0 tx=0x7f8554007920 comp rx=0 tx=0).stop 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 msgr2=0x7f856414a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 0x7f856414a280 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f855800d620 tx=0x7f855800dae0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 shutdown_connections 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f856415db00 0x7f85640cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f854806b550 0x7f854806da10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85640ce2b0 0x7f85641545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 --2- 172.21.15.5:0/2118799924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85640b53b0 0x7f856414a280 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 >> 172.21.15.5:0/2118799924 conn(0x7f8564072290 msgr2=0x7f856415c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 shutdown_connections 2024-04-25T10:04:30.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:30.948+0000 7f856be00640 1 -- 172.21.15.5:0/2118799924 wait complete. 2024-04-25T10:04:30.975 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-25T10:04:30.665911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238540,"kb_used_data":5712,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736180,"statfs":{"total":773094113280,"available":772849848320,"internally_reserved":0,"allocated":5849088,"data_stored":3195488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.235888"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471647+0000","last_change":"2024-04-25T10:04:25.461883+0000","last_active":"2024-04-25T10:04:25.471647+0000","last_peered":"2024-04-25T10:04:25.471647+0000","last_clean":"2024-04-25T10:04:25.471647+0000","last_became_active":"2024-04-25T10:04:25.460987+0000","last_became_peered":"2024-04-25T10:04:25.460987+0000","last_unstale":"2024-04-25T10:04:25.471647+0000","last_undegraded":"2024-04-25T10:04:25.471647+0000","last_fullsized":"2024-04-25T10:04:25.471647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479215+0000","last_change":"2024-04-25T10:04:25.474565+0000","last_active":"2024-04-25T10:04:25.479215+0000","last_peered":"2024-04-25T10:04:25.479215+0000","last_clean":"2024-04-25T10:04:25.479215+0000","last_became_active":"2024-04-25T10:04:25.474282+0000","last_became_peered":"2024-04-25T10:04:25.474282+0000","last_unstale":"2024-04-25T10:04:25.479215+0000","last_undegraded":"2024-04-25T10:04:25.479215+0000","last_fullsized":"2024-04-25T10:04:25.479215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479735+0000","last_change":"2024-04-25T10:04:25.460733+0000","last_active":"2024-04-25T10:04:26.479735+0000","last_peered":"2024-04-25T10:04:26.479735+0000","last_clean":"2024-04-25T10:04:26.479735+0000","last_became_active":"2024-04-25T10:04:25.460475+0000","last_became_peered":"2024-04-25T10:04:25.460475+0000","last_unstale":"2024-04-25T10:04:26.479735+0000","last_undegraded":"2024-04-25T10:04:26.479735+0000","last_fullsized":"2024-04-25T10:04:26.479735+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:20.427097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.459057+0000","last_change":"2024-04-25T10:04:25.454040+0000","last_active":"2024-04-25T10:04:25.459057+0000","last_peered":"2024-04-25T10:04:25.459057+0000","last_clean":"2024-04-25T10:04:25.459057+0000","last_became_active":"2024-04-25T10:04:25.453680+0000","last_became_peered":"2024-04-25T10:04:25.453680+0000","last_unstale":"2024-04-25T10:04:25.459057+0000","last_undegraded":"2024-04-25T10:04:25.459057+0000","last_fullsized":"2024-04-25T10:04:25.459057+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.470460+0000","last_change":"2024-04-25T10:04:25.468526+0000","last_active":"2024-04-25T10:04:25.470460+0000","last_peered":"2024-04-25T10:04:25.470460+0000","last_clean":"2024-04-25T10:04:25.470460+0000","last_became_active":"2024-04-25T10:04:25.468181+0000","last_became_peered":"2024-04-25T10:04:25.468181+0000","last_unstale":"2024-04-25T10:04:25.470460+0000","last_undegraded":"2024-04-25T10:04:25.470460+0000","last_fullsized":"2024-04-25T10:04:25.470460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.458948+0000","last_change":"2024-04-25T10:04:25.448768+0000","last_active":"2024-04-25T10:04:25.458948+0000","last_peered":"2024-04-25T10:04:25.458948+0000","last_clean":"2024-04-25T10:04:25.458948+0000","last_became_active":"2024-04-25T10:04:25.448208+0000","last_became_peered":"2024-04-25T10:04:25.448208+0000","last_unstale":"2024-04-25T10:04:25.458948+0000","last_undegraded":"2024-04-25T10:04:25.458948+0000","last_fullsized":"2024-04-25T10:04:25.458948+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.918327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479890+0000","last_change":"2024-04-25T10:04:25.475883+0000","last_active":"2024-04-25T10:04:25.479890+0000","last_peered":"2024-04-25T10:04:25.479890+0000","last_clean":"2024-04-25T10:04:25.479890+0000","last_became_active":"2024-04-25T10:04:25.475594+0000","last_became_peered":"2024-04-25T10:04:25.475594+0000","last_unstale":"2024-04-25T10:04:25.479890+0000","last_undegraded":"2024-04-25T10:04:25.479890+0000","last_fullsized":"2024-04-25T10:04:25.479890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475353+0000","last_change":"2024-04-25T10:04:25.461616+0000","last_active":"2024-04-25T10:04:25.475353+0000","last_peered":"2024-04-25T10:04:25.475353+0000","last_clean":"2024-04-25T10:04:25.475353+0000","last_became_active":"2024-04-25T10:04:25.461353+0000","last_became_peered":"2024-04-25T10:04:25.461353+0000","last_unstale":"2024-04-25T10:04:25.475353+0000","last_undegraded":"2024-04-25T10:04:25.475353+0000","last_fullsized":"2024-04-25T10:04:25.475353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:59.983038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480232+0000","last_change":"2024-04-25T10:04:25.462804+0000","last_active":"2024-04-25T10:04:26.480232+0000","last_peered":"2024-04-25T10:04:26.480232+0000","last_clean":"2024-04-25T10:04:26.480232+0000","last_became_active":"2024-04-25T10:04:25.458840+0000","last_became_peered":"2024-04-25T10:04:25.458840+0000","last_unstale":"2024-04-25T10:04:26.480232+0000","last_undegraded":"2024-04-25T10:04:26.480232+0000","last_fullsized":"2024-04-25T10:04:26.480232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:33.800771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475605+0000","last_change":"2024-04-25T10:04:25.469094+0000","last_active":"2024-04-25T10:04:25.475605+0000","last_peered":"2024-04-25T10:04:25.475605+0000","last_clean":"2024-04-25T10:04:25.475605+0000","last_became_active":"2024-04-25T10:04:25.468584+0000","last_became_peered":"2024-04-25T10:04:25.468584+0000","last_unstale":"2024-04-25T10:04:25.475605+0000","last_undegraded":"2024-04-25T10:04:25.475605+0000","last_fullsized":"2024-04-25T10:04:25.475605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.463317+0000","last_change":"2024-04-25T10:04:25.456279+0000","last_active":"2024-04-25T10:04:25.463317+0000","last_peered":"2024-04-25T10:04:25.463317+0000","last_clean":"2024-04-25T10:04:25.463317+0000","last_became_active":"2024-04-25T10:04:25.456002+0000","last_became_peered":"2024-04-25T10:04:25.456002+0000","last_unstale":"2024-04-25T10:04:25.463317+0000","last_undegraded":"2024-04-25T10:04:25.463317+0000","last_fullsized":"2024-04-25T10:04:25.463317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477920+0000","last_change":"2024-04-25T10:04:25.467692+0000","last_active":"2024-04-25T10:04:25.477920+0000","last_peered":"2024-04-25T10:04:25.477920+0000","last_clean":"2024-04-25T10:04:25.477920+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.477920+0000","last_undegraded":"2024-04-25T10:04:25.477920+0000","last_fullsized":"2024-04-25T10:04:25.477920+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477172+0000","last_change":"2024-04-25T10:04:25.469711+0000","last_active":"2024-04-25T10:04:25.477172+0000","last_peered":"2024-04-25T10:04:25.477172+0000","last_clean":"2024-04-25T10:04:25.477172+0000","last_became_active":"2024-04-25T10:04:25.469239+0000","last_became_peered":"2024-04-25T10:04:25.469239+0000","last_unstale":"2024-04-25T10:04:25.477172+0000","last_undegraded":"2024-04-25T10:04:25.477172+0000","last_fullsized":"2024-04-25T10:04:25.477172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483001+0000","last_change":"2024-04-25T10:04:25.479586+0000","last_active":"2024-04-25T10:04:25.483001+0000","last_peered":"2024-04-25T10:04:25.483001+0000","last_clean":"2024-04-25T10:04:25.483001+0000","last_became_active":"2024-04-25T10:04:25.478560+0000","last_became_peered":"2024-04-25T10:04:25.478560+0000","last_unstale":"2024-04-25T10:04:25.483001+0000","last_undegraded":"2024-04-25T10:04:25.483001+0000","last_fullsized":"2024-04-25T10:04:25.483001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471478+0000","last_change":"2024-04-25T10:04:25.467693+0000","last_active":"2024-04-25T10:04:25.471478+0000","last_peered":"2024-04-25T10:04:25.471478+0000","last_clean":"2024-04-25T10:04:25.471478+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.471478+0000","last_undegraded":"2024-04-25T10:04:25.471478+0000","last_fullsized":"2024-04-25T10:04:25.471478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.462788+0000","last_change":"2024-04-25T10:04:25.450560+0000","last_active":"2024-04-25T10:04:25.462788+0000","last_peered":"2024-04-25T10:04:25.462788+0000","last_clean":"2024-04-25T10:04:25.462788+0000","last_became_active":"2024-04-25T10:04:25.450242+0000","last_became_peered":"2024-04-25T10:04:25.450242+0000","last_unstale":"2024-04-25T10:04:25.462788+0000","last_undegraded":"2024-04-25T10:04:25.462788+0000","last_fullsized":"2024-04-25T10:04:25.462788+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:07.537002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471792+0000","last_change":"2024-04-25T10:04:25.463042+0000","last_active":"2024-04-25T10:04:25.471792+0000","last_peered":"2024-04-25T10:04:25.471792+0000","last_clean":"2024-04-25T10:04:25.471792+0000","last_became_active":"2024-04-25T10:04:25.460640+0000","last_became_peered":"2024-04-25T10:04:25.460640+0000","last_unstale":"2024-04-25T10:04:25.471792+0000","last_undegraded":"2024-04-25T10:04:25.471792+0000","last_fullsized":"2024-04-25T10:04:25.471792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460519+0000","last_change":"2024-04-25T10:04:25.455252+0000","last_active":"2024-04-25T10:04:25.460519+0000","last_peered":"2024-04-25T10:04:25.460519+0000","last_clean":"2024-04-25T10:04:25.460519+0000","last_became_active":"2024-04-25T10:04:25.454978+0000","last_became_peered":"2024-04-25T10:04:25.454978+0000","last_unstale":"2024-04-25T10:04:25.460519+0000","last_undegraded":"2024-04-25T10:04:25.460519+0000","last_fullsized":"2024-04-25T10:04:25.460519+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479803+0000","last_change":"2024-04-25T10:04:25.466442+0000","last_active":"2024-04-25T10:04:26.479803+0000","last_peered":"2024-04-25T10:04:26.479803+0000","last_clean":"2024-04-25T10:04:26.479803+0000","last_became_active":"2024-04-25T10:04:25.466178+0000","last_became_peered":"2024-04-25T10:04:25.466178+0000","last_unstale":"2024-04-25T10:04:26.479803+0000","last_undegraded":"2024-04-25T10:04:26.479803+0000","last_fullsized":"2024-04-25T10:04:26.479803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:32.660382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480500+0000","last_change":"2024-04-25T10:04:25.477500+0000","last_active":"2024-04-25T10:04:25.480500+0000","last_peered":"2024-04-25T10:04:25.480500+0000","last_clean":"2024-04-25T10:04:25.480500+0000","last_became_active":"2024-04-25T10:04:25.475152+0000","last_became_peered":"2024-04-25T10:04:25.475152+0000","last_unstale":"2024-04-25T10:04:25.480500+0000","last_undegraded":"2024-04-25T10:04:25.480500+0000","last_fullsized":"2024-04-25T10:04:25.480500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472951+0000","last_change":"2024-04-25T10:04:25.466973+0000","last_active":"2024-04-25T10:04:25.472951+0000","last_peered":"2024-04-25T10:04:25.472951+0000","last_clean":"2024-04-25T10:04:25.472951+0000","last_became_active":"2024-04-25T10:04:25.466685+0000","last_became_peered":"2024-04-25T10:04:25.466685+0000","last_unstale":"2024-04-25T10:04:25.472951+0000","last_undegraded":"2024-04-25T10:04:25.472951+0000","last_fullsized":"2024-04-25T10:04:25.472951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479182+0000","last_change":"2024-04-25T10:04:25.466682+0000","last_active":"2024-04-25T10:04:26.479182+0000","last_peered":"2024-04-25T10:04:26.479182+0000","last_clean":"2024-04-25T10:04:26.479182+0000","last_became_active":"2024-04-25T10:04:25.465981+0000","last_became_peered":"2024-04-25T10:04:25.465981+0000","last_unstale":"2024-04-25T10:04:26.479182+0000","last_undegraded":"2024-04-25T10:04:26.479182+0000","last_fullsized":"2024-04-25T10:04:26.479182+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:08.084202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479180+0000","last_change":"2024-04-25T10:04:25.473782+0000","last_active":"2024-04-25T10:04:26.479180+0000","last_peered":"2024-04-25T10:04:26.479180+0000","last_clean":"2024-04-25T10:04:26.479180+0000","last_became_active":"2024-04-25T10:04:25.470089+0000","last_became_peered":"2024-04-25T10:04:25.470089+0000","last_unstale":"2024-04-25T10:04:26.479180+0000","last_undegraded":"2024-04-25T10:04:26.479180+0000","last_fullsized":"2024-04-25T10:04:26.479180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.049551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478435+0000","last_change":"2024-04-25T10:04:25.462903+0000","last_active":"2024-04-25T10:04:26.478435+0000","last_peered":"2024-04-25T10:04:26.478435+0000","last_clean":"2024-04-25T10:04:26.478435+0000","last_became_active":"2024-04-25T10:04:25.462619+0000","last_became_peered":"2024-04-25T10:04:25.462619+0000","last_unstale":"2024-04-25T10:04:26.478435+0000","last_undegraded":"2024-04-25T10:04:26.478435+0000","last_fullsized":"2024-04-25T10:04:26.478435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:45.933238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472713+0000","last_change":"2024-04-25T10:04:25.465113+0000","last_active":"2024-04-25T10:04:25.472713+0000","last_peered":"2024-04-25T10:04:25.472713+0000","last_clean":"2024-04-25T10:04:25.472713+0000","last_became_active":"2024-04-25T10:04:25.464842+0000","last_became_peered":"2024-04-25T10:04:25.464842+0000","last_unstale":"2024-04-25T10:04:25.472713+0000","last_undegraded":"2024-04-25T10:04:25.472713+0000","last_fullsized":"2024-04-25T10:04:25.472713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479918+0000","last_change":"2024-04-25T10:04:25.477772+0000","last_active":"2024-04-25T10:04:26.479918+0000","last_peered":"2024-04-25T10:04:26.479918+0000","last_clean":"2024-04-25T10:04:26.479918+0000","last_became_active":"2024-04-25T10:04:25.476288+0000","last_became_peered":"2024-04-25T10:04:25.476288+0000","last_unstale":"2024-04-25T10:04:26.479918+0000","last_undegraded":"2024-04-25T10:04:26.479918+0000","last_fullsized":"2024-04-25T10:04:26.479918+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.333518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474420+0000","last_change":"2024-04-25T10:04:25.465925+0000","last_active":"2024-04-25T10:04:25.474420+0000","last_peered":"2024-04-25T10:04:25.474420+0000","last_clean":"2024-04-25T10:04:25.474420+0000","last_became_active":"2024-04-25T10:04:25.465652+0000","last_became_peered":"2024-04-25T10:04:25.465652+0000","last_unstale":"2024-04-25T10:04:25.474420+0000","last_undegraded":"2024-04-25T10:04:25.474420+0000","last_fullsized":"2024-04-25T10:04:25.474420+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480477+0000","last_change":"2024-04-25T10:04:25.476769+0000","last_active":"2024-04-25T10:04:25.480477+0000","last_peered":"2024-04-25T10:04:25.480477+0000","last_clean":"2024-04-25T10:04:25.480477+0000","last_became_active":"2024-04-25T10:04:25.476451+0000","last_became_peered":"2024-04-25T10:04:25.476451+0000","last_unstale":"2024-04-25T10:04:25.480477+0000","last_undegraded":"2024-04-25T10:04:25.480477+0000","last_fullsized":"2024-04-25T10:04:25.480477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472747+0000","last_change":"2024-04-25T10:04:25.469665+0000","last_active":"2024-04-25T10:04:25.472747+0000","last_peered":"2024-04-25T10:04:25.472747+0000","last_clean":"2024-04-25T10:04:25.472747+0000","last_became_active":"2024-04-25T10:04:25.469374+0000","last_became_peered":"2024-04-25T10:04:25.469374+0000","last_unstale":"2024-04-25T10:04:25.472747+0000","last_undegraded":"2024-04-25T10:04:25.472747+0000","last_fullsized":"2024-04-25T10:04:25.472747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472340+0000","last_change":"2024-04-25T10:04:25.470230+0000","last_active":"2024-04-25T10:04:26.472340+0000","last_peered":"2024-04-25T10:04:26.472340+0000","last_clean":"2024-04-25T10:04:26.472340+0000","last_became_active":"2024-04-25T10:04:25.469402+0000","last_became_peered":"2024-04-25T10:04:25.469402+0000","last_unstale":"2024-04-25T10:04:26.472340+0000","last_undegraded":"2024-04-25T10:04:26.472340+0000","last_fullsized":"2024-04-25T10:04:26.472340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.544638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479366+0000","last_change":"2024-04-25T10:04:25.460768+0000","last_active":"2024-04-25T10:04:26.479366+0000","last_peered":"2024-04-25T10:04:26.479366+0000","last_clean":"2024-04-25T10:04:26.479366+0000","last_became_active":"2024-04-25T10:04:25.460525+0000","last_became_peered":"2024-04-25T10:04:25.460525+0000","last_unstale":"2024-04-25T10:04:26.479366+0000","last_undegraded":"2024-04-25T10:04:26.479366+0000","last_fullsized":"2024-04-25T10:04:26.479366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:57.909617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465537+0000","last_change":"2024-04-25T10:04:25.464113+0000","last_active":"2024-04-25T10:04:25.465537+0000","last_peered":"2024-04-25T10:04:25.465537+0000","last_clean":"2024-04-25T10:04:25.465537+0000","last_became_active":"2024-04-25T10:04:25.463493+0000","last_became_peered":"2024-04-25T10:04:25.463493+0000","last_unstale":"2024-04-25T10:04:25.465537+0000","last_undegraded":"2024-04-25T10:04:25.465537+0000","last_fullsized":"2024-04-25T10:04:25.465537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472176+0000","last_change":"2024-04-25T10:04:25.469216+0000","last_active":"2024-04-25T10:04:26.472176+0000","last_peered":"2024-04-25T10:04:26.472176+0000","last_clean":"2024-04-25T10:04:26.472176+0000","last_became_active":"2024-04-25T10:04:25.466664+0000","last_became_peered":"2024-04-25T10:04:25.466664+0000","last_unstale":"2024-04-25T10:04:26.472176+0000","last_undegraded":"2024-04-25T10:04:26.472176+0000","last_fullsized":"2024-04-25T10:04:26.472176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:32.821688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474510+0000","last_change":"2024-04-25T10:04:24.469487+0000","last_active":"2024-04-25T10:04:25.474510+0000","last_peered":"2024-04-25T10:04:25.474510+0000","last_clean":"2024-04-25T10:04:25.474510+0000","last_became_active":"2024-04-25T10:04:24.469185+0000","last_became_peered":"2024-04-25T10:04:24.469185+0000","last_unstale":"2024-04-25T10:04:25.474510+0000","last_undegraded":"2024-04-25T10:04:25.474510+0000","last_fullsized":"2024-04-25T10:04:25.474510+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472240+0000","last_change":"2024-04-25T10:04:25.473033+0000","last_active":"2024-04-25T10:04:26.472240+0000","last_peered":"2024-04-25T10:04:26.472240+0000","last_clean":"2024-04-25T10:04:26.472240+0000","last_became_active":"2024-04-25T10:04:25.471870+0000","last_became_peered":"2024-04-25T10:04:25.471870+0000","last_unstale":"2024-04-25T10:04:26.472240+0000","last_undegraded":"2024-04-25T10:04:26.472240+0000","last_fullsized":"2024-04-25T10:04:26.472240+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:55.812057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481374+0000","last_change":"2024-04-25T10:04:24.468749+0000","last_active":"2024-04-25T10:04:25.481374+0000","last_peered":"2024-04-25T10:04:25.481374+0000","last_clean":"2024-04-25T10:04:25.481374+0000","last_became_active":"2024-04-25T10:04:24.468458+0000","last_became_peered":"2024-04-25T10:04:24.468458+0000","last_unstale":"2024-04-25T10:04:25.481374+0000","last_undegraded":"2024-04-25T10:04:25.481374+0000","last_fullsized":"2024-04-25T10:04:25.481374+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.077398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472493+0000","last_change":"2024-04-25T10:04:25.478215+0000","last_active":"2024-04-25T10:04:26.472493+0000","last_peered":"2024-04-25T10:04:26.472493+0000","last_clean":"2024-04-25T10:04:26.472493+0000","last_became_active":"2024-04-25T10:04:25.475504+0000","last_became_peered":"2024-04-25T10:04:25.475504+0000","last_unstale":"2024-04-25T10:04:26.472493+0000","last_undegraded":"2024-04-25T10:04:26.472493+0000","last_fullsized":"2024-04-25T10:04:26.472493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:41.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471375+0000","last_change":"2024-04-25T10:04:24.459064+0000","last_active":"2024-04-25T10:04:25.471375+0000","last_peered":"2024-04-25T10:04:25.471375+0000","last_clean":"2024-04-25T10:04:25.471375+0000","last_became_active":"2024-04-25T10:04:24.458599+0000","last_became_peered":"2024-04-25T10:04:24.458599+0000","last_unstale":"2024-04-25T10:04:25.471375+0000","last_undegraded":"2024-04-25T10:04:25.471375+0000","last_fullsized":"2024-04-25T10:04:25.471375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483933+0000","last_change":"2024-04-25T10:04:25.473270+0000","last_active":"2024-04-25T10:04:25.483933+0000","last_peered":"2024-04-25T10:04:25.483933+0000","last_clean":"2024-04-25T10:04:25.483933+0000","last_became_active":"2024-04-25T10:04:25.472978+0000","last_became_peered":"2024-04-25T10:04:25.472978+0000","last_unstale":"2024-04-25T10:04:25.483933+0000","last_undegraded":"2024-04-25T10:04:25.483933+0000","last_fullsized":"2024-04-25T10:04:25.483933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465518+0000","last_change":"2024-04-25T10:04:24.468225+0000","last_active":"2024-04-25T10:04:25.465518+0000","last_peered":"2024-04-25T10:04:25.465518+0000","last_clean":"2024-04-25T10:04:25.465518+0000","last_became_active":"2024-04-25T10:04:24.467670+0000","last_became_peered":"2024-04-25T10:04:24.467670+0000","last_unstale":"2024-04-25T10:04:25.465518+0000","last_undegraded":"2024-04-25T10:04:25.465518+0000","last_fullsized":"2024-04-25T10:04:25.465518+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472793+0000","last_change":"2024-04-25T10:04:25.465227+0000","last_active":"2024-04-25T10:04:25.472793+0000","last_peered":"2024-04-25T10:04:25.472793+0000","last_clean":"2024-04-25T10:04:25.472793+0000","last_became_active":"2024-04-25T10:04:25.464968+0000","last_became_peered":"2024-04-25T10:04:25.464968+0000","last_unstale":"2024-04-25T10:04:25.472793+0000","last_undegraded":"2024-04-25T10:04:25.472793+0000","last_fullsized":"2024-04-25T10:04:25.472793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479731+0000","last_change":"2024-04-25T10:04:24.467424+0000","last_active":"2024-04-25T10:04:26.479731+0000","last_peered":"2024-04-25T10:04:26.479731+0000","last_clean":"2024-04-25T10:04:26.479731+0000","last_became_active":"2024-04-25T10:04:24.467101+0000","last_became_peered":"2024-04-25T10:04:24.467101+0000","last_unstale":"2024-04-25T10:04:26.479731+0000","last_undegraded":"2024-04-25T10:04:26.479731+0000","last_fullsized":"2024-04-25T10:04:26.479731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:52.095406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473097+0000","last_change":"2024-04-25T10:04:25.471910+0000","last_active":"2024-04-25T10:04:26.473097+0000","last_peered":"2024-04-25T10:04:26.473097+0000","last_clean":"2024-04-25T10:04:26.473097+0000","last_became_active":"2024-04-25T10:04:25.471628+0000","last_became_peered":"2024-04-25T10:04:25.471628+0000","last_unstale":"2024-04-25T10:04:26.473097+0000","last_undegraded":"2024-04-25T10:04:26.473097+0000","last_fullsized":"2024-04-25T10:04:26.473097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:01.732449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472494+0000","last_change":"2024-04-25T10:04:24.449636+0000","last_active":"2024-04-25T10:04:26.472494+0000","last_peered":"2024-04-25T10:04:26.472494+0000","last_clean":"2024-04-25T10:04:26.472494+0000","last_became_active":"2024-04-25T10:04:24.449303+0000","last_became_peered":"2024-04-25T10:04:24.449303+0000","last_unstale":"2024-04-25T10:04:26.472494+0000","last_undegraded":"2024-04-25T10:04:26.472494+0000","last_fullsized":"2024-04-25T10:04:26.472494+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:58.775423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475584+0000","last_change":"2024-04-25T10:04:25.468855+0000","last_active":"2024-04-25T10:04:25.475584+0000","last_peered":"2024-04-25T10:04:25.475584+0000","last_clean":"2024-04-25T10:04:25.475584+0000","last_became_active":"2024-04-25T10:04:25.467011+0000","last_became_peered":"2024-04-25T10:04:25.467011+0000","last_unstale":"2024-04-25T10:04:25.475584+0000","last_undegraded":"2024-04-25T10:04:25.475584+0000","last_fullsized":"2024-04-25T10:04:25.475584+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472322+0000","last_change":"2024-04-25T10:04:24.469032+0000","last_active":"2024-04-25T10:04:26.472322+0000","last_peered":"2024-04-25T10:04:26.472322+0000","last_clean":"2024-04-25T10:04:26.472322+0000","last_became_active":"2024-04-25T10:04:24.468452+0000","last_became_peered":"2024-04-25T10:04:24.468452+0000","last_unstale":"2024-04-25T10:04:26.472322+0000","last_undegraded":"2024-04-25T10:04:26.472322+0000","last_fullsized":"2024-04-25T10:04:26.472322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:49.522317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460571+0000","last_change":"2024-04-25T10:04:25.456211+0000","last_active":"2024-04-25T10:04:25.460571+0000","last_peered":"2024-04-25T10:04:25.460571+0000","last_clean":"2024-04-25T10:04:25.460571+0000","last_became_active":"2024-04-25T10:04:25.455945+0000","last_became_peered":"2024-04-25T10:04:25.455945+0000","last_unstale":"2024-04-25T10:04:25.460571+0000","last_undegraded":"2024-04-25T10:04:25.460571+0000","last_fullsized":"2024-04-25T10:04:25.460571+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478559+0000","last_change":"2024-04-25T10:04:24.468238+0000","last_active":"2024-04-25T10:04:26.478559+0000","last_peered":"2024-04-25T10:04:26.478559+0000","last_clean":"2024-04-25T10:04:26.478559+0000","last_became_active":"2024-04-25T10:04:24.467961+0000","last_became_peered":"2024-04-25T10:04:24.467961+0000","last_unstale":"2024-04-25T10:04:26.478559+0000","last_undegraded":"2024-04-25T10:04:26.478559+0000","last_fullsized":"2024-04-25T10:04:26.478559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:25.485389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472666+0000","last_change":"2024-04-25T10:04:25.462850+0000","last_active":"2024-04-25T10:04:25.472666+0000","last_peered":"2024-04-25T10:04:25.472666+0000","last_clean":"2024-04-25T10:04:25.472666+0000","last_became_active":"2024-04-25T10:04:25.462560+0000","last_became_peered":"2024-04-25T10:04:25.462560+0000","last_unstale":"2024-04-25T10:04:25.472666+0000","last_undegraded":"2024-04-25T10:04:25.472666+0000","last_fullsized":"2024-04-25T10:04:25.472666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.444069+0000","last_change":"2024-04-25T10:04:24.447649+0000","last_active":"2024-04-25T10:04:25.444069+0000","last_peered":"2024-04-25T10:04:25.444069+0000","last_clean":"2024-04-25T10:04:25.444069+0000","last_became_active":"2024-04-25T10:04:24.446781+0000","last_became_peered":"2024-04-25T10:04:24.446781+0000","last_unstale":"2024-04-25T10:04:25.444069+0000","last_undegraded":"2024-04-25T10:04:25.444069+0000","last_fullsized":"2024-04-25T10:04:25.444069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.029749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.457360+0000","last_change":"2024-04-25T10:04:25.452425+0000","last_active":"2024-04-25T10:04:25.457360+0000","last_peered":"2024-04-25T10:04:25.457360+0000","last_clean":"2024-04-25T10:04:25.457360+0000","last_became_active":"2024-04-25T10:04:25.451175+0000","last_became_peered":"2024-04-25T10:04:25.451175+0000","last_unstale":"2024-04-25T10:04:25.457360+0000","last_undegraded":"2024-04-25T10:04:25.457360+0000","last_fullsized":"2024-04-25T10:04:25.457360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.443720+0000","last_change":"2024-04-25T10:04:24.451547+0000","last_active":"2024-04-25T10:04:25.443720+0000","last_peered":"2024-04-25T10:04:25.443720+0000","last_clean":"2024-04-25T10:04:25.443720+0000","last_became_active":"2024-04-25T10:04:24.451221+0000","last_became_peered":"2024-04-25T10:04:24.451221+0000","last_unstale":"2024-04-25T10:04:25.443720+0000","last_undegraded":"2024-04-25T10:04:25.443720+0000","last_fullsized":"2024-04-25T10:04:25.443720+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472556+0000","last_change":"2024-04-25T10:04:25.466717+0000","last_active":"2024-04-25T10:04:25.472556+0000","last_peered":"2024-04-25T10:04:25.472556+0000","last_clean":"2024-04-25T10:04:25.472556+0000","last_became_active":"2024-04-25T10:04:25.466392+0000","last_became_peered":"2024-04-25T10:04:25.466392+0000","last_unstale":"2024-04-25T10:04:25.472556+0000","last_undegraded":"2024-04-25T10:04:25.472556+0000","last_fullsized":"2024-04-25T10:04:25.472556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473094+0000","last_change":"2024-04-25T10:04:24.462694+0000","last_active":"2024-04-25T10:04:26.473094+0000","last_peered":"2024-04-25T10:04:26.473094+0000","last_clean":"2024-04-25T10:04:26.473094+0000","last_became_active":"2024-04-25T10:04:24.462301+0000","last_became_peered":"2024-04-25T10:04:24.462301+0000","last_unstale":"2024-04-25T10:04:26.473094+0000","last_undegraded":"2024-04-25T10:04:26.473094+0000","last_fullsized":"2024-04-25T10:04:26.473094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.011021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469006+0000","last_change":"2024-04-25T10:04:25.466716+0000","last_active":"2024-04-25T10:04:25.469006+0000","last_peered":"2024-04-25T10:04:25.469006+0000","last_clean":"2024-04-25T10:04:25.469006+0000","last_became_active":"2024-04-25T10:04:25.466438+0000","last_became_peered":"2024-04-25T10:04:25.466438+0000","last_unstale":"2024-04-25T10:04:25.469006+0000","last_undegraded":"2024-04-25T10:04:25.469006+0000","last_fullsized":"2024-04-25T10:04:25.469006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.486863+0000","last_change":"2024-04-25T10:04:24.469012+0000","last_active":"2024-04-25T10:04:25.486863+0000","last_peered":"2024-04-25T10:04:25.486863+0000","last_clean":"2024-04-25T10:04:25.486863+0000","last_became_active":"2024-04-25T10:04:24.468729+0000","last_became_peered":"2024-04-25T10:04:24.468729+0000","last_unstale":"2024-04-25T10:04:25.486863+0000","last_undegraded":"2024-04-25T10:04:25.486863+0000","last_fullsized":"2024-04-25T10:04:25.486863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472871+0000","last_change":"2024-04-25T10:04:25.468212+0000","last_active":"2024-04-25T10:04:26.472871+0000","last_peered":"2024-04-25T10:04:26.472871+0000","last_clean":"2024-04-25T10:04:26.472871+0000","last_became_active":"2024-04-25T10:04:25.467940+0000","last_became_peered":"2024-04-25T10:04:25.467940+0000","last_unstale":"2024-04-25T10:04:26.472871+0000","last_undegraded":"2024-04-25T10:04:26.472871+0000","last_fullsized":"2024-04-25T10:04:26.472871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:04.368539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482369+0000","last_change":"2024-04-25T10:04:24.456904+0000","last_active":"2024-04-25T10:04:25.482369+0000","last_peered":"2024-04-25T10:04:25.482369+0000","last_clean":"2024-04-25T10:04:25.482369+0000","last_became_active":"2024-04-25T10:04:24.456095+0000","last_became_peered":"2024-04-25T10:04:24.456095+0000","last_unstale":"2024-04-25T10:04:25.482369+0000","last_undegraded":"2024-04-25T10:04:25.482369+0000","last_fullsized":"2024-04-25T10:04:25.482369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.185120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482977+0000","last_change":"2024-04-25T10:04:25.479170+0000","last_active":"2024-04-25T10:04:25.482977+0000","last_peered":"2024-04-25T10:04:25.482977+0000","last_clean":"2024-04-25T10:04:25.482977+0000","last_became_active":"2024-04-25T10:04:25.478980+0000","last_became_peered":"2024-04-25T10:04:25.478980+0000","last_unstale":"2024-04-25T10:04:25.482977+0000","last_undegraded":"2024-04-25T10:04:25.482977+0000","last_fullsized":"2024-04-25T10:04:25.482977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.442376+0000","last_change":"2024-04-25T10:04:24.458424+0000","last_active":"2024-04-25T10:04:25.442376+0000","last_peered":"2024-04-25T10:04:25.442376+0000","last_clean":"2024-04-25T10:04:25.442376+0000","last_became_active":"2024-04-25T10:04:24.458098+0000","last_became_peered":"2024-04-25T10:04:24.458098+0000","last_unstale":"2024-04-25T10:04:25.442376+0000","last_undegraded":"2024-04-25T10:04:25.442376+0000","last_fullsized":"2024-04-25T10:04:25.442376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469999+0000","last_change":"2024-04-25T10:04:25.455450+0000","last_active":"2024-04-25T10:04:25.469999+0000","last_peered":"2024-04-25T10:04:25.469999+0000","last_clean":"2024-04-25T10:04:25.469999+0000","last_became_active":"2024-04-25T10:04:25.454137+0000","last_became_peered":"2024-04-25T10:04:25.454137+0000","last_unstale":"2024-04-25T10:04:25.469999+0000","last_undegraded":"2024-04-25T10:04:25.469999+0000","last_fullsized":"2024-04-25T10:04:25.469999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:11.030611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446148+0000","last_change":"2024-04-25T10:04:24.445036+0000","last_active":"2024-04-25T10:04:25.446148+0000","last_peered":"2024-04-25T10:04:25.446148+0000","last_clean":"2024-04-25T10:04:25.446148+0000","last_became_active":"2024-04-25T10:04:24.444715+0000","last_became_peered":"2024-04-25T10:04:24.444715+0000","last_unstale":"2024-04-25T10:04:25.446148+0000","last_undegraded":"2024-04-25T10:04:25.446148+0000","last_fullsized":"2024-04-25T10:04:25.446148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.905326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480231+0000","last_change":"2024-04-25T10:04:25.460045+0000","last_active":"2024-04-25T10:04:26.480231+0000","last_peered":"2024-04-25T10:04:26.480231+0000","last_clean":"2024-04-25T10:04:26.480231+0000","last_became_active":"2024-04-25T10:04:25.458524+0000","last_became_peered":"2024-04-25T10:04:25.458524+0000","last_unstale":"2024-04-25T10:04:26.480231+0000","last_undegraded":"2024-04-25T10:04:26.480231+0000","last_fullsized":"2024-04-25T10:04:26.480231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:40.245623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482536+0000","last_change":"2024-04-25T10:04:24.465436+0000","last_active":"2024-04-25T10:04:25.482536+0000","last_peered":"2024-04-25T10:04:25.482536+0000","last_clean":"2024-04-25T10:04:25.482536+0000","last_became_active":"2024-04-25T10:04:24.465132+0000","last_became_peered":"2024-04-25T10:04:24.465132+0000","last_unstale":"2024-04-25T10:04:25.482536+0000","last_undegraded":"2024-04-25T10:04:25.482536+0000","last_fullsized":"2024-04-25T10:04:25.482536+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.884017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479931+0000","last_change":"2024-04-25T10:04:25.472528+0000","last_active":"2024-04-25T10:04:25.479931+0000","last_peered":"2024-04-25T10:04:25.479931+0000","last_clean":"2024-04-25T10:04:25.479931+0000","last_became_active":"2024-04-25T10:04:25.472245+0000","last_became_peered":"2024-04-25T10:04:25.472245+0000","last_unstale":"2024-04-25T10:04:25.479931+0000","last_undegraded":"2024-04-25T10:04:25.479931+0000","last_fullsized":"2024-04-25T10:04:25.479931+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481219+0000","last_change":"2024-04-25T10:04:24.468526+0000","last_active":"2024-04-25T10:04:25.481219+0000","last_peered":"2024-04-25T10:04:25.481219+0000","last_clean":"2024-04-25T10:04:25.481219+0000","last_became_active":"2024-04-25T10:04:24.468213+0000","last_became_peered":"2024-04-25T10:04:24.468213+0000","last_unstale":"2024-04-25T10:04:25.481219+0000","last_undegraded":"2024-04-25T10:04:25.481219+0000","last_fullsized":"2024-04-25T10:04:25.481219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:40.655704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472621+0000","last_change":"2024-04-25T10:04:25.463721+0000","last_active":"2024-04-25T10:04:25.472621+0000","last_peered":"2024-04-25T10:04:25.472621+0000","last_clean":"2024-04-25T10:04:25.472621+0000","last_became_active":"2024-04-25T10:04:25.463355+0000","last_became_peered":"2024-04-25T10:04:25.463355+0000","last_unstale":"2024-04-25T10:04:25.472621+0000","last_undegraded":"2024-04-25T10:04:25.472621+0000","last_fullsized":"2024-04-25T10:04:25.472621+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481630+0000","last_change":"2024-04-25T10:04:24.461901+0000","last_active":"2024-04-25T10:04:25.481630+0000","last_peered":"2024-04-25T10:04:25.481630+0000","last_clean":"2024-04-25T10:04:25.481630+0000","last_became_active":"2024-04-25T10:04:24.461619+0000","last_became_peered":"2024-04-25T10:04:24.461619+0000","last_unstale":"2024-04-25T10:04:25.481630+0000","last_undegraded":"2024-04-25T10:04:25.481630+0000","last_fullsized":"2024-04-25T10:04:25.481630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:32.072513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480737+0000","last_change":"2024-04-25T10:04:25.477724+0000","last_active":"2024-04-25T10:04:25.480737+0000","last_peered":"2024-04-25T10:04:25.480737+0000","last_clean":"2024-04-25T10:04:25.480737+0000","last_became_active":"2024-04-25T10:04:25.476438+0000","last_became_peered":"2024-04-25T10:04:25.476438+0000","last_unstale":"2024-04-25T10:04:25.480737+0000","last_undegraded":"2024-04-25T10:04:25.480737+0000","last_fullsized":"2024-04-25T10:04:25.480737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475376+0000","last_change":"2024-04-25T10:04:24.465620+0000","last_active":"2024-04-25T10:04:25.475376+0000","last_peered":"2024-04-25T10:04:25.475376+0000","last_clean":"2024-04-25T10:04:25.475376+0000","last_became_active":"2024-04-25T10:04:24.465402+0000","last_became_peered":"2024-04-25T10:04:24.465402+0000","last_unstale":"2024-04-25T10:04:25.475376+0000","last_undegraded":"2024-04-25T10:04:25.475376+0000","last_fullsized":"2024-04-25T10:04:25.475376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474444+0000","last_change":"2024-04-25T10:04:25.463940+0000","last_active":"2024-04-25T10:04:25.474444+0000","last_peered":"2024-04-25T10:04:25.474444+0000","last_clean":"2024-04-25T10:04:25.474444+0000","last_became_active":"2024-04-25T10:04:25.463675+0000","last_became_peered":"2024-04-25T10:04:25.463675+0000","last_unstale":"2024-04-25T10:04:25.474444+0000","last_undegraded":"2024-04-25T10:04:25.474444+0000","last_fullsized":"2024-04-25T10:04:25.474444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.456328+0000","last_change":"2024-04-25T10:04:24.466373+0000","last_active":"2024-04-25T10:04:25.456328+0000","last_peered":"2024-04-25T10:04:25.456328+0000","last_clean":"2024-04-25T10:04:25.456328+0000","last_became_active":"2024-04-25T10:04:24.465965+0000","last_became_peered":"2024-04-25T10:04:24.465965+0000","last_unstale":"2024-04-25T10:04:25.456328+0000","last_undegraded":"2024-04-25T10:04:25.456328+0000","last_fullsized":"2024-04-25T10:04:25.456328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.478776+0000","last_change":"2024-04-25T10:04:25.471532+0000","last_active":"2024-04-25T10:04:25.478776+0000","last_peered":"2024-04-25T10:04:25.478776+0000","last_clean":"2024-04-25T10:04:25.478776+0000","last_became_active":"2024-04-25T10:04:25.471276+0000","last_became_peered":"2024-04-25T10:04:25.471276+0000","last_unstale":"2024-04-25T10:04:25.478776+0000","last_undegraded":"2024-04-25T10:04:25.478776+0000","last_fullsized":"2024-04-25T10:04:25.478776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.468521+0000","last_change":"2024-04-25T10:04:24.461754+0000","last_active":"2024-04-25T10:04:25.468521+0000","last_peered":"2024-04-25T10:04:25.468521+0000","last_clean":"2024-04-25T10:04:25.468521+0000","last_became_active":"2024-04-25T10:04:24.459617+0000","last_became_peered":"2024-04-25T10:04:24.459617+0000","last_unstale":"2024-04-25T10:04:25.468521+0000","last_undegraded":"2024-04-25T10:04:25.468521+0000","last_fullsized":"2024-04-25T10:04:25.468521+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.473141+0000","last_change":"2024-04-25T10:04:25.468223+0000","last_active":"2024-04-25T10:04:25.473141+0000","last_peered":"2024-04-25T10:04:25.473141+0000","last_clean":"2024-04-25T10:04:25.473141+0000","last_became_active":"2024-04-25T10:04:25.467369+0000","last_became_peered":"2024-04-25T10:04:25.467369+0000","last_unstale":"2024-04-25T10:04:25.473141+0000","last_undegraded":"2024-04-25T10:04:25.473141+0000","last_fullsized":"2024-04-25T10:04:25.473141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479325+0000","last_change":"2024-04-25T10:04:24.471952+0000","last_active":"2024-04-25T10:04:26.479325+0000","last_peered":"2024-04-25T10:04:26.479325+0000","last_clean":"2024-04-25T10:04:26.479325+0000","last_became_active":"2024-04-25T10:04:24.471642+0000","last_became_peered":"2024-04-25T10:04:24.471642+0000","last_unstale":"2024-04-25T10:04:26.479325+0000","last_undegraded":"2024-04-25T10:04:26.479325+0000","last_fullsized":"2024-04-25T10:04:26.479325+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:00.609015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472693+0000","last_change":"2024-04-25T10:04:25.468105+0000","last_active":"2024-04-25T10:04:25.472693+0000","last_peered":"2024-04-25T10:04:25.472693+0000","last_clean":"2024-04-25T10:04:25.472693+0000","last_became_active":"2024-04-25T10:04:25.467841+0000","last_became_peered":"2024-04-25T10:04:25.467841+0000","last_unstale":"2024-04-25T10:04:25.472693+0000","last_undegraded":"2024-04-25T10:04:25.472693+0000","last_fullsized":"2024-04-25T10:04:25.472693+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.451244+0000","last_change":"2024-04-25T10:04:24.469768+0000","last_active":"2024-04-25T10:04:25.451244+0000","last_peered":"2024-04-25T10:04:25.451244+0000","last_clean":"2024-04-25T10:04:25.451244+0000","last_became_active":"2024-04-25T10:04:24.469488+0000","last_became_peered":"2024-04-25T10:04:24.469488+0000","last_unstale":"2024-04-25T10:04:25.451244+0000","last_undegraded":"2024-04-25T10:04:25.451244+0000","last_fullsized":"2024-04-25T10:04:25.451244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:29.785579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480787+0000","last_change":"2024-04-25T10:04:25.475514+0000","last_active":"2024-04-25T10:04:25.480787+0000","last_peered":"2024-04-25T10:04:25.480787+0000","last_clean":"2024-04-25T10:04:25.480787+0000","last_became_active":"2024-04-25T10:04:25.474364+0000","last_became_peered":"2024-04-25T10:04:25.474364+0000","last_unstale":"2024-04-25T10:04:25.480787+0000","last_undegraded":"2024-04-25T10:04:25.480787+0000","last_fullsized":"2024-04-25T10:04:25.480787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.453420+0000","last_change":"2024-04-25T10:04:24.460278+0000","last_active":"2024-04-25T10:04:25.453420+0000","last_peered":"2024-04-25T10:04:25.453420+0000","last_clean":"2024-04-25T10:04:25.453420+0000","last_became_active":"2024-04-25T10:04:24.459936+0000","last_became_peered":"2024-04-25T10:04:24.459936+0000","last_unstale":"2024-04-25T10:04:25.453420+0000","last_undegraded":"2024-04-25T10:04:25.453420+0000","last_fullsized":"2024-04-25T10:04:25.453420+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:19.332266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465336+0000","last_change":"2024-04-25T10:04:25.456950+0000","last_active":"2024-04-25T10:04:25.465336+0000","last_peered":"2024-04-25T10:04:25.465336+0000","last_clean":"2024-04-25T10:04:25.465336+0000","last_became_active":"2024-04-25T10:04:25.456664+0000","last_became_peered":"2024-04-25T10:04:25.456664+0000","last_unstale":"2024-04-25T10:04:25.465336+0000","last_undegraded":"2024-04-25T10:04:25.465336+0000","last_fullsized":"2024-04-25T10:04:25.465336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473040+0000","last_change":"2024-04-25T10:04:24.466995+0000","last_active":"2024-04-25T10:04:26.473040+0000","last_peered":"2024-04-25T10:04:26.473040+0000","last_clean":"2024-04-25T10:04:26.473040+0000","last_became_active":"2024-04-25T10:04:24.466667+0000","last_became_peered":"2024-04-25T10:04:24.466667+0000","last_unstale":"2024-04-25T10:04:26.473040+0000","last_undegraded":"2024-04-25T10:04:26.473040+0000","last_fullsized":"2024-04-25T10:04:26.473040+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:07.560341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475023+0000","last_change":"2024-04-25T10:04:25.467788+0000","last_active":"2024-04-25T10:04:25.475023+0000","last_peered":"2024-04-25T10:04:25.475023+0000","last_clean":"2024-04-25T10:04:25.475023+0000","last_became_active":"2024-04-25T10:04:25.465574+0000","last_became_peered":"2024-04-25T10:04:25.465574+0000","last_unstale":"2024-04-25T10:04:25.475023+0000","last_undegraded":"2024-04-25T10:04:25.475023+0000","last_fullsized":"2024-04-25T10:04:25.475023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473253+0000","last_change":"2024-04-25T10:04:24.463577+0000","last_active":"2024-04-25T10:04:26.473253+0000","last_peered":"2024-04-25T10:04:26.473253+0000","last_clean":"2024-04-25T10:04:26.473253+0000","last_became_active":"2024-04-25T10:04:24.462318+0000","last_became_peered":"2024-04-25T10:04:24.462318+0000","last_unstale":"2024-04-25T10:04:26.473253+0000","last_undegraded":"2024-04-25T10:04:26.473253+0000","last_fullsized":"2024-04-25T10:04:26.473253+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:01.690130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479939+0000","last_change":"2024-04-25T10:04:25.461326+0000","last_active":"2024-04-25T10:04:26.479939+0000","last_peered":"2024-04-25T10:04:26.479939+0000","last_clean":"2024-04-25T10:04:26.479939+0000","last_became_active":"2024-04-25T10:04:25.459940+0000","last_became_peered":"2024-04-25T10:04:25.459940+0000","last_unstale":"2024-04-25T10:04:26.479939+0000","last_undegraded":"2024-04-25T10:04:26.479939+0000","last_fullsized":"2024-04-25T10:04:26.479939+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:48.804816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479691+0000","last_change":"2024-04-25T10:04:24.462828+0000","last_active":"2024-04-25T10:04:26.479691+0000","last_peered":"2024-04-25T10:04:26.479691+0000","last_clean":"2024-04-25T10:04:26.479691+0000","last_became_active":"2024-04-25T10:04:24.462443+0000","last_became_peered":"2024-04-25T10:04:24.462443+0000","last_unstale":"2024-04-25T10:04:26.479691+0000","last_undegraded":"2024-04-25T10:04:26.479691+0000","last_fullsized":"2024-04-25T10:04:26.479691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:20.157546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475078+0000","last_change":"2024-04-25T10:04:25.466945+0000","last_active":"2024-04-25T10:04:25.475078+0000","last_peered":"2024-04-25T10:04:25.475078+0000","last_clean":"2024-04-25T10:04:25.475078+0000","last_became_active":"2024-04-25T10:04:25.466672+0000","last_became_peered":"2024-04-25T10:04:25.466672+0000","last_unstale":"2024-04-25T10:04:25.475078+0000","last_undegraded":"2024-04-25T10:04:25.475078+0000","last_fullsized":"2024-04-25T10:04:25.475078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.450703+0000","last_change":"2024-04-25T10:04:24.461824+0000","last_active":"2024-04-25T10:04:25.450703+0000","last_peered":"2024-04-25T10:04:25.450703+0000","last_clean":"2024-04-25T10:04:25.450703+0000","last_became_active":"2024-04-25T10:04:24.460800+0000","last_became_peered":"2024-04-25T10:04:24.460800+0000","last_unstale":"2024-04-25T10:04:25.450703+0000","last_undegraded":"2024-04-25T10:04:25.450703+0000","last_fullsized":"2024-04-25T10:04:25.450703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:25.199121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480645+0000","last_change":"2024-04-25T10:04:25.477935+0000","last_active":"2024-04-25T10:04:25.480645+0000","last_peered":"2024-04-25T10:04:25.480645+0000","last_clean":"2024-04-25T10:04:25.480645+0000","last_became_active":"2024-04-25T10:04:25.477605+0000","last_became_peered":"2024-04-25T10:04:25.477605+0000","last_unstale":"2024-04-25T10:04:25.480645+0000","last_undegraded":"2024-04-25T10:04:25.480645+0000","last_fullsized":"2024-04-25T10:04:25.480645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479437+0000","last_change":"2024-04-25T10:04:24.471290+0000","last_active":"2024-04-25T10:04:26.479437+0000","last_peered":"2024-04-25T10:04:26.479437+0000","last_clean":"2024-04-25T10:04:26.479437+0000","last_became_active":"2024-04-25T10:04:24.470984+0000","last_became_peered":"2024-04-25T10:04:24.470984+0000","last_unstale":"2024-04-25T10:04:26.479437+0000","last_undegraded":"2024-04-25T10:04:26.479437+0000","last_fullsized":"2024-04-25T10:04:26.479437+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:50.982046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475556+0000","last_change":"2024-04-25T10:04:25.462505+0000","last_active":"2024-04-25T10:04:25.475556+0000","last_peered":"2024-04-25T10:04:25.475556+0000","last_clean":"2024-04-25T10:04:25.475556+0000","last_became_active":"2024-04-25T10:04:25.461056+0000","last_became_peered":"2024-04-25T10:04:25.461056+0000","last_unstale":"2024-04-25T10:04:25.475556+0000","last_undegraded":"2024-04-25T10:04:25.475556+0000","last_fullsized":"2024-04-25T10:04:25.475556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446296+0000","last_change":"2024-04-25T10:04:24.443846+0000","last_active":"2024-04-25T10:04:25.446296+0000","last_peered":"2024-04-25T10:04:25.446296+0000","last_clean":"2024-04-25T10:04:25.446296+0000","last_became_active":"2024-04-25T10:04:24.443511+0000","last_became_peered":"2024-04-25T10:04:24.443511+0000","last_unstale":"2024-04-25T10:04:25.446296+0000","last_undegraded":"2024-04-25T10:04:25.446296+0000","last_fullsized":"2024-04-25T10:04:25.446296+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475649+0000","last_change":"2024-04-25T10:04:25.466904+0000","last_active":"2024-04-25T10:04:25.475649+0000","last_peered":"2024-04-25T10:04:25.475649+0000","last_clean":"2024-04-25T10:04:25.475649+0000","last_became_active":"2024-04-25T10:04:25.466434+0000","last_became_peered":"2024-04-25T10:04:25.466434+0000","last_unstale":"2024-04-25T10:04:25.475649+0000","last_undegraded":"2024-04-25T10:04:25.475649+0000","last_fullsized":"2024-04-25T10:04:25.475649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441933+0000","last_change":"2024-04-25T10:04:24.467188+0000","last_active":"2024-04-25T10:04:25.441933+0000","last_peered":"2024-04-25T10:04:25.441933+0000","last_clean":"2024-04-25T10:04:25.441933+0000","last_became_active":"2024-04-25T10:04:24.466850+0000","last_became_peered":"2024-04-25T10:04:24.466850+0000","last_unstale":"2024-04-25T10:04:25.441933+0000","last_undegraded":"2024-04-25T10:04:25.441933+0000","last_fullsized":"2024-04-25T10:04:25.441933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472620+0000","last_change":"2024-04-25T10:04:25.461224+0000","last_active":"2024-04-25T10:04:25.472620+0000","last_peered":"2024-04-25T10:04:25.472620+0000","last_clean":"2024-04-25T10:04:25.472620+0000","last_became_active":"2024-04-25T10:04:25.460727+0000","last_became_peered":"2024-04-25T10:04:25.460727+0000","last_unstale":"2024-04-25T10:04:25.472620+0000","last_undegraded":"2024-04-25T10:04:25.472620+0000","last_fullsized":"2024-04-25T10:04:25.472620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.452980+0000","last_change":"2024-04-25T10:04:24.461091+0000","last_active":"2024-04-25T10:04:25.452980+0000","last_peered":"2024-04-25T10:04:25.452980+0000","last_clean":"2024-04-25T10:04:25.452980+0000","last_became_active":"2024-04-25T10:04:24.460322+0000","last_became_peered":"2024-04-25T10:04:24.460322+0000","last_unstale":"2024-04-25T10:04:25.452980+0000","last_undegraded":"2024-04-25T10:04:25.452980+0000","last_fullsized":"2024-04-25T10:04:25.452980+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:17.560293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441995+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:04:25.441995+0000","last_peered":"2024-04-25T10:04:25.441995+0000","last_clean":"2024-04-25T10:04:25.441995+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:04:25.441995+0000","last_undegraded":"2024-04-25T10:04:25.441995+0000","last_fullsized":"2024-04-25T10:04:25.441995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":602112,"data_stored":262104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":618496,"data_stored":263718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:30.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:04:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.230+0000 7fa909600640 1 Processor -- start 2024-04-25T10:04:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.231+0000 7fa909600640 1 -- start start 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa909600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa909600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa904070f90 0x7fa9040713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa909600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90406cfb0 con 0x7fa904165890 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa909600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa904071930 con 0x7fa904070f90 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa909600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406d290 con 0x7fa904161c70 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa908c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa904165890 0x7fa904165d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60566/0 (socket says 172.21.15.5:60566) 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa908c00640 1 -- 172.21.15.5:0/3712421909 learned_addr learned my addr 172.21.15.5:0/3712421909 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.233+0000 7fa8fbe00640 1 --2- 172.21.15.5:0/3712421909 >> v2:172.21.15.5:3300/0 conn(0x7fa904070f90 0x7fa9040713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa903e00640 1 --2- 172.21.15.5:0/3712421909 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa903400640 1 -- 172.21.15.5:0/3712421909 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645067569 0 0) 0x7fa90406cfb0 con 0x7fa904165890 2024-04-25T10:04:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa903400640 1 -- 172.21.15.5:0/3712421909 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e8003520 con 0x7fa904165890 2024-04-25T10:04:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa8fbe00640 1 -- 172.21.15.5:0/3712421909 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 msgr2=0x7fa9040cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa8fbe00640 1 --2- 172.21.15.5:0/3712421909 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa8fbe00640 1 -- 172.21.15.5:0/3712421909 >> v1:172.21.15.5:6789/0 conn(0x7fa904165890 legacy=0x7fa904165d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.234+0000 7fa8fbe00640 1 -- 172.21.15.5:0/3712421909 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90406d570 con 0x7fa904070f90 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.235+0000 7fa8fbe00640 1 --2- 172.21.15.5:0/3712421909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa9040713f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0097a0 tx=0x7fa8ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6907ce76daea05bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.235+0000 7fa903400640 1 -- 172.21.15.5:0/3712421909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec037070 con 0x7fa904070f90 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.235+0000 7fa903400640 1 -- 172.21.15.5:0/3712421909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec032440 con 0x7fa904070f90 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.235+0000 7fa903400640 1 -- 172.21.15.5:0/3712421909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0327f0 con 0x7fa904070f90 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 -- 172.21.15.5:0/3712421909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 msgr2=0x7fa9040713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 --2- 172.21.15.5:0/3712421909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa9040713f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0097a0 tx=0x7fa8ec02a180 comp rx=0 tx=0).stop 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 -- 172.21.15.5:0/3712421909 shutdown_connections 2024-04-25T10:04:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 --2- 172.21.15.5:0/3712421909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa9040713f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 --2- 172.21.15.5:0/3712421909 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 -- 172.21.15.5:0/3712421909 >> 172.21.15.5:0/3712421909 conn(0x7fa904066f20 msgr2=0x7fa90406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 -- 172.21.15.5:0/3712421909 shutdown_connections 2024-04-25T10:04:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.236+0000 7fa909600640 1 -- 172.21.15.5:0/3712421909 wait complete. 2024-04-25T10:04:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.238+0000 7fa909600640 1 Processor -- start 2024-04-25T10:04:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.238+0000 7fa909600640 1 -- start start 2024-04-25T10:04:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa909600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa90415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa909600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa909600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa904165890 0x7fa9041622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa909600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040cc960 con 0x7fa904070f90 2024-04-25T10:04:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa909600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90406db30 con 0x7fa904165890 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa909600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904162050 con 0x7fa904161c70 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa90415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa90415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57752/0 (socket says 172.21.15.5:57752) 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.239+0000 7fa903e00640 1 -- 172.21.15.5:0/1797381147 learned_addr learned my addr 172.21.15.5:0/1797381147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa8fbe00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa908c00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa904165890 0x7fa9041622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa903e00640 1 -- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 msgr2=0x7fa9040aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa903e00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa903e00640 1 -- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa904165890 msgr2=0x7fa9041622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa903e00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa904165890 0x7fa9041622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa903e00640 1 -- 172.21.15.5:0/1797381147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040ae450 con 0x7fa904070f90 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa908c00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa904165890 0x7fa9041622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa903e00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa90415d530 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400d830 tx=0x7fa8f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400b6c0 con 0x7fa904070f90 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.240+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f4019070 con 0x7fa904070f90 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.241+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400ba80 con 0x7fa904070f90 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.241+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9040ae730 con 0x7fa904070f90 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.241+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040a9eb0 con 0x7fa904070f90 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.242+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9040b30a0 con 0x7fa904070f90 2024-04-25T10:04:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.244+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8f401e460 con 0x7fa904070f90 2024-04-25T10:04:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.244+0000 7fa900c00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.244+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa8f408aae0 con 0x7fa904070f90 2024-04-25T10:04:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.246+0000 7fa8fbe00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.246+0000 7fa8fbe00640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec034000 tx=0x7fa8ec02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.249+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9040b30a0 con 0x7fa904070f90 2024-04-25T10:04:31.410 INFO:tasks.ceph.mds.d.smithi080.stderr:2024-04-25T10:04:31.406+0000 7f7ed5400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:31.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.518+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa904068f60 con 0x7fa904070f90 2024-04-25T10:04:31.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.523+0000 7fa900c00640 1 -- 172.21.15.5:0/1797381147 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1432 (secure 0 0 0) 0x7fa8f4059730 con 0x7fa904070f90 2024-04-25T10:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":178,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039305,"num_in_osds":8,"osd_in_since":1714039296,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":244264960,"bytes_avail":772849848320,"bytes_total":773094113280,"write_bytes_sec":2298,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":5365}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T10:04:12.660785+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:04:31.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 msgr2=0x7fa8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec034000 tx=0x7fa8ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 msgr2=0x7fa90415d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa90415d530 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa8f400d830 tx=0x7fa8f400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 shutdown_connections 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa904165890 0x7fa9041622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa904161c70 0x7fa9040aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 --2- 172.21.15.5:0/1797381147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904070f90 0x7fa90415d530 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 >> 172.21.15.5:0/1797381147 conn(0x7fa904066f20 msgr2=0x7fa9040b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 shutdown_connections 2024-04-25T10:04:31.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.525+0000 7fa909600640 1 -- 172.21.15.5:0/1797381147 wait complete. 2024-04-25T10:04:31.550 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:04:31.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:31.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.741+0000 7fadffa00640 1 Processor -- start 2024-04-25T10:04:31.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.741+0000 7fadffa00640 1 -- start start 2024-04-25T10:04:31.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadffa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf8144100 0x7fadf80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadffa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadf80ced20 0x7fadf806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadffa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fadf806e7a0 0x7fadf815e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadffa00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf81444e0 con 0x7fadf80ced20 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadffa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf815ac70 con 0x7fadf806e7a0 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadffa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80cdbf0 con 0x7fadf8144100 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadf7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadf80ced20 0x7fadf806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadf7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadf80ced20 0x7fadf806e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57768/0 (socket says 172.21.15.5:57768) 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.743+0000 7fadf7e00640 1 -- 172.21.15.5:0/2956130638 learned_addr learned my addr 172.21.15.5:0/2956130638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadfda00640 1 --2- 172.21.15.5:0/2956130638 >> v2:172.21.15.80:3300/0 conn(0x7fadf806e7a0 0x7fadf815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadfd000640 1 --2- 172.21.15.5:0/2956130638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf8144100 0x7fadf80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadf7e00640 1 -- 172.21.15.5:0/2956130638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf8144100 msgr2=0x7fadf80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadf7e00640 1 --2- 172.21.15.5:0/2956130638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf8144100 0x7fadf80cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadf7e00640 1 -- 172.21.15.5:0/2956130638 >> v2:172.21.15.80:3300/0 conn(0x7fadf806e7a0 msgr2=0x7fadf815e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadf7e00640 1 --2- 172.21.15.5:0/2956130638 >> v2:172.21.15.80:3300/0 conn(0x7fadf806e7a0 0x7fadf815e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.744+0000 7fadf7e00640 1 -- 172.21.15.5:0/2956130638 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80cded0 con 0x7fadf80ced20 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.745+0000 7fadf7e00640 1 --2- 172.21.15.5:0/2956130638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf80ced20 0x7fadf806e260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fadf80b1e10 tx=0x7fade402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae782629ea45464b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.745+0000 7fadf7400640 1 -- 172.21.15.5:0/2956130638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4037070 con 0x7fadf80ced20 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.745+0000 7fadf7400640 1 -- 172.21.15.5:0/2956130638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade4032440 con 0x7fadf80ced20 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.745+0000 7fadf7400640 1 -- 172.21.15.5:0/2956130638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4032810 con 0x7fadf80ced20 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 -- 172.21.15.5:0/2956130638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf80ced20 msgr2=0x7fadf806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 --2- 172.21.15.5:0/2956130638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf80ced20 0x7fadf806e260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fadf80b1e10 tx=0x7fade402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 -- 172.21.15.5:0/2956130638 shutdown_connections 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 --2- 172.21.15.5:0/2956130638 >> v2:172.21.15.80:3300/0 conn(0x7fadf806e7a0 0x7fadf815e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 --2- 172.21.15.5:0/2956130638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf80ced20 0x7fadf806e260 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 --2- 172.21.15.5:0/2956130638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf8144100 0x7fadf80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 -- 172.21.15.5:0/2956130638 >> 172.21.15.5:0/2956130638 conn(0x7fadf8071150 msgr2=0x7fadf8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 -- 172.21.15.5:0/2956130638 shutdown_connections 2024-04-25T10:04:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.746+0000 7fadffa00640 1 -- 172.21.15.5:0/2956130638 wait complete. 2024-04-25T10:04:31.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.748+0000 7fadffa00640 1 Processor -- start 2024-04-25T10:04:31.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.748+0000 7fadffa00640 1 -- start start 2024-04-25T10:04:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadffa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf806e7a0 0x7fadf81f3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadffa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadf80ced20 0x7fadf806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadffa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 0x7fadf806d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadffa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80cea00 con 0x7fadf8144100 2024-04-25T10:04:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadffa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80ce490 con 0x7fadf80ced20 2024-04-25T10:04:31.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadffa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf815e7a0 con 0x7fadf806e7a0 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadfda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 0x7fadf806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadfda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 0x7fadf806d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57774/0 (socket says 172.21.15.5:57774) 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.749+0000 7fadfda00640 1 -- 172.21.15.5:0/1857445787 learned_addr learned my addr 172.21.15.5:0/1857445787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadf7e00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadf80ced20 0x7fadf806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfd000640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf806e7a0 0x7fadf81f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfda00640 1 -- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf806e7a0 msgr2=0x7fadf81f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfda00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf806e7a0 0x7fadf81f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfda00640 1 -- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadf80ced20 msgr2=0x7fadf806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfda00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadf80ced20 0x7fadf806d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfda00640 1 -- 172.21.15.5:0/1857445787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8139a10 con 0x7fadf8144100 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfd000640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf806e7a0 0x7fadf81f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadf7e00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadf80ced20 0x7fadf806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadfda00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 0x7fadf806d690 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fadf0007e40 tx=0x7fadf000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0014620 con 0x7fadf8144100 2024-04-25T10:04:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0019070 con 0x7fadf8144100 2024-04-25T10:04:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0014a20 con 0x7fadf8144100 2024-04-25T10:04:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.750+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf8139cf0 con 0x7fadf8144100 2024-04-25T10:04:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.751+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf8150f40 con 0x7fadf8144100 2024-04-25T10:04:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.752+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf8068fd0 con 0x7fadf8144100 2024-04-25T10:04:31.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.753+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadf0014bc0 con 0x7fadf8144100 2024-04-25T10:04:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.754+0000 7fadf4c00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:31.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.754+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadf008ad70 con 0x7fadf8144100 2024-04-25T10:04:31.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.754+0000 7fadfd000640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:31.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.755+0000 7fadfd000640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fadec006fd0 tx=0x7fadec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:31.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.758+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadf00599c0 con 0x7fadf8144100 2024-04-25T10:04:31.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.957+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadf8142580 con 0x7fadd806b4d0 2024-04-25T10:04:31.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.975+0000 7fadf4c00640 1 -- 172.21.15.5:0/1857445787 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251904 (secure 0 0 0) 0x7fadf8142580 con 0x7fadd806b4d0 2024-04-25T10:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:31.985 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 msgr2=0x7fadd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fadec006fd0 tx=0x7fadec008040 comp rx=0 tx=0).stop 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 msgr2=0x7fadf806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 0x7fadf806d690 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fadf0007e40 tx=0x7fadf000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 shutdown_connections 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadf8144100 0x7fadf806d690 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadf80ced20 0x7fadf806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 --2- 172.21.15.5:0/1857445787 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadf806e7a0 0x7fadf81f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 >> 172.21.15.5:0/1857445787 conn(0x7fadf8071150 msgr2=0x7fadf8142d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 shutdown_connections 2024-04-25T10:04:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:31.980+0000 7fadffa00640 1 -- 172.21.15.5:0/1857445787 wait complete. 2024-04-25T10:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-25T10:04:30.665911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238540,"kb_used_data":5712,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736180,"statfs":{"total":773094113280,"available":772849848320,"internally_reserved":0,"allocated":5849088,"data_stored":3195488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.235888"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471647+0000","last_change":"2024-04-25T10:04:25.461883+0000","last_active":"2024-04-25T10:04:25.471647+0000","last_peered":"2024-04-25T10:04:25.471647+0000","last_clean":"2024-04-25T10:04:25.471647+0000","last_became_active":"2024-04-25T10:04:25.460987+0000","last_became_peered":"2024-04-25T10:04:25.460987+0000","last_unstale":"2024-04-25T10:04:25.471647+0000","last_undegraded":"2024-04-25T10:04:25.471647+0000","last_fullsized":"2024-04-25T10:04:25.471647+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479215+0000","last_change":"2024-04-25T10:04:25.474565+0000","last_active":"2024-04-25T10:04:25.479215+0000","last_peered":"2024-04-25T10:04:25.479215+0000","last_clean":"2024-04-25T10:04:25.479215+0000","last_became_active":"2024-04-25T10:04:25.474282+0000","last_became_peered":"2024-04-25T10:04:25.474282+0000","last_unstale":"2024-04-25T10:04:25.479215+0000","last_undegraded":"2024-04-25T10:04:25.479215+0000","last_fullsized":"2024-04-25T10:04:25.479215+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479735+0000","last_change":"2024-04-25T10:04:25.460733+0000","last_active":"2024-04-25T10:04:26.479735+0000","last_peered":"2024-04-25T10:04:26.479735+0000","last_clean":"2024-04-25T10:04:26.479735+0000","last_became_active":"2024-04-25T10:04:25.460475+0000","last_became_peered":"2024-04-25T10:04:25.460475+0000","last_unstale":"2024-04-25T10:04:26.479735+0000","last_undegraded":"2024-04-25T10:04:26.479735+0000","last_fullsized":"2024-04-25T10:04:26.479735+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:20.427097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.459057+0000","last_change":"2024-04-25T10:04:25.454040+0000","last_active":"2024-04-25T10:04:25.459057+0000","last_peered":"2024-04-25T10:04:25.459057+0000","last_clean":"2024-04-25T10:04:25.459057+0000","last_became_active":"2024-04-25T10:04:25.453680+0000","last_became_peered":"2024-04-25T10:04:25.453680+0000","last_unstale":"2024-04-25T10:04:25.459057+0000","last_undegraded":"2024-04-25T10:04:25.459057+0000","last_fullsized":"2024-04-25T10:04:25.459057+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.470460+0000","last_change":"2024-04-25T10:04:25.468526+0000","last_active":"2024-04-25T10:04:25.470460+0000","last_peered":"2024-04-25T10:04:25.470460+0000","last_clean":"2024-04-25T10:04:25.470460+0000","last_became_active":"2024-04-25T10:04:25.468181+0000","last_became_peered":"2024-04-25T10:04:25.468181+0000","last_unstale":"2024-04-25T10:04:25.470460+0000","last_undegraded":"2024-04-25T10:04:25.470460+0000","last_fullsized":"2024-04-25T10:04:25.470460+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.458948+0000","last_change":"2024-04-25T10:04:25.448768+0000","last_active":"2024-04-25T10:04:25.458948+0000","last_peered":"2024-04-25T10:04:25.458948+0000","last_clean":"2024-04-25T10:04:25.458948+0000","last_became_active":"2024-04-25T10:04:25.448208+0000","last_became_peered":"2024-04-25T10:04:25.448208+0000","last_unstale":"2024-04-25T10:04:25.458948+0000","last_undegraded":"2024-04-25T10:04:25.458948+0000","last_fullsized":"2024-04-25T10:04:25.458948+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:26.918327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479890+0000","last_change":"2024-04-25T10:04:25.475883+0000","last_active":"2024-04-25T10:04:25.479890+0000","last_peered":"2024-04-25T10:04:25.479890+0000","last_clean":"2024-04-25T10:04:25.479890+0000","last_became_active":"2024-04-25T10:04:25.475594+0000","last_became_peered":"2024-04-25T10:04:25.475594+0000","last_unstale":"2024-04-25T10:04:25.479890+0000","last_undegraded":"2024-04-25T10:04:25.479890+0000","last_fullsized":"2024-04-25T10:04:25.479890+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475353+0000","last_change":"2024-04-25T10:04:25.461616+0000","last_active":"2024-04-25T10:04:25.475353+0000","last_peered":"2024-04-25T10:04:25.475353+0000","last_clean":"2024-04-25T10:04:25.475353+0000","last_became_active":"2024-04-25T10:04:25.461353+0000","last_became_peered":"2024-04-25T10:04:25.461353+0000","last_unstale":"2024-04-25T10:04:25.475353+0000","last_undegraded":"2024-04-25T10:04:25.475353+0000","last_fullsized":"2024-04-25T10:04:25.475353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:59.983038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480232+0000","last_change":"2024-04-25T10:04:25.462804+0000","last_active":"2024-04-25T10:04:26.480232+0000","last_peered":"2024-04-25T10:04:26.480232+0000","last_clean":"2024-04-25T10:04:26.480232+0000","last_became_active":"2024-04-25T10:04:25.458840+0000","last_became_peered":"2024-04-25T10:04:25.458840+0000","last_unstale":"2024-04-25T10:04:26.480232+0000","last_undegraded":"2024-04-25T10:04:26.480232+0000","last_fullsized":"2024-04-25T10:04:26.480232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:33.800771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475605+0000","last_change":"2024-04-25T10:04:25.469094+0000","last_active":"2024-04-25T10:04:25.475605+0000","last_peered":"2024-04-25T10:04:25.475605+0000","last_clean":"2024-04-25T10:04:25.475605+0000","last_became_active":"2024-04-25T10:04:25.468584+0000","last_became_peered":"2024-04-25T10:04:25.468584+0000","last_unstale":"2024-04-25T10:04:25.475605+0000","last_undegraded":"2024-04-25T10:04:25.475605+0000","last_fullsized":"2024-04-25T10:04:25.475605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.463317+0000","last_change":"2024-04-25T10:04:25.456279+0000","last_active":"2024-04-25T10:04:25.463317+0000","last_peered":"2024-04-25T10:04:25.463317+0000","last_clean":"2024-04-25T10:04:25.463317+0000","last_became_active":"2024-04-25T10:04:25.456002+0000","last_became_peered":"2024-04-25T10:04:25.456002+0000","last_unstale":"2024-04-25T10:04:25.463317+0000","last_undegraded":"2024-04-25T10:04:25.463317+0000","last_fullsized":"2024-04-25T10:04:25.463317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477920+0000","last_change":"2024-04-25T10:04:25.467692+0000","last_active":"2024-04-25T10:04:25.477920+0000","last_peered":"2024-04-25T10:04:25.477920+0000","last_clean":"2024-04-25T10:04:25.477920+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.477920+0000","last_undegraded":"2024-04-25T10:04:25.477920+0000","last_fullsized":"2024-04-25T10:04:25.477920+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.477172+0000","last_change":"2024-04-25T10:04:25.469711+0000","last_active":"2024-04-25T10:04:25.477172+0000","last_peered":"2024-04-25T10:04:25.477172+0000","last_clean":"2024-04-25T10:04:25.477172+0000","last_became_active":"2024-04-25T10:04:25.469239+0000","last_became_peered":"2024-04-25T10:04:25.469239+0000","last_unstale":"2024-04-25T10:04:25.477172+0000","last_undegraded":"2024-04-25T10:04:25.477172+0000","last_fullsized":"2024-04-25T10:04:25.477172+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483001+0000","last_change":"2024-04-25T10:04:25.479586+0000","last_active":"2024-04-25T10:04:25.483001+0000","last_peered":"2024-04-25T10:04:25.483001+0000","last_clean":"2024-04-25T10:04:25.483001+0000","last_became_active":"2024-04-25T10:04:25.478560+0000","last_became_peered":"2024-04-25T10:04:25.478560+0000","last_unstale":"2024-04-25T10:04:25.483001+0000","last_undegraded":"2024-04-25T10:04:25.483001+0000","last_fullsized":"2024-04-25T10:04:25.483001+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471478+0000","last_change":"2024-04-25T10:04:25.467693+0000","last_active":"2024-04-25T10:04:25.471478+0000","last_peered":"2024-04-25T10:04:25.471478+0000","last_clean":"2024-04-25T10:04:25.471478+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:25.471478+0000","last_undegraded":"2024-04-25T10:04:25.471478+0000","last_fullsized":"2024-04-25T10:04:25.471478+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.462788+0000","last_change":"2024-04-25T10:04:25.450560+0000","last_active":"2024-04-25T10:04:25.462788+0000","last_peered":"2024-04-25T10:04:25.462788+0000","last_clean":"2024-04-25T10:04:25.462788+0000","last_became_active":"2024-04-25T10:04:25.450242+0000","last_became_peered":"2024-04-25T10:04:25.450242+0000","last_unstale":"2024-04-25T10:04:25.462788+0000","last_undegraded":"2024-04-25T10:04:25.462788+0000","last_fullsized":"2024-04-25T10:04:25.462788+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:07.537002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471792+0000","last_change":"2024-04-25T10:04:25.463042+0000","last_active":"2024-04-25T10:04:25.471792+0000","last_peered":"2024-04-25T10:04:25.471792+0000","last_clean":"2024-04-25T10:04:25.471792+0000","last_became_active":"2024-04-25T10:04:25.460640+0000","last_became_peered":"2024-04-25T10:04:25.460640+0000","last_unstale":"2024-04-25T10:04:25.471792+0000","last_undegraded":"2024-04-25T10:04:25.471792+0000","last_fullsized":"2024-04-25T10:04:25.471792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460519+0000","last_change":"2024-04-25T10:04:25.455252+0000","last_active":"2024-04-25T10:04:25.460519+0000","last_peered":"2024-04-25T10:04:25.460519+0000","last_clean":"2024-04-25T10:04:25.460519+0000","last_became_active":"2024-04-25T10:04:25.454978+0000","last_became_peered":"2024-04-25T10:04:25.454978+0000","last_unstale":"2024-04-25T10:04:25.460519+0000","last_undegraded":"2024-04-25T10:04:25.460519+0000","last_fullsized":"2024-04-25T10:04:25.460519+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:30.798579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479803+0000","last_change":"2024-04-25T10:04:25.466442+0000","last_active":"2024-04-25T10:04:26.479803+0000","last_peered":"2024-04-25T10:04:26.479803+0000","last_clean":"2024-04-25T10:04:26.479803+0000","last_became_active":"2024-04-25T10:04:25.466178+0000","last_became_peered":"2024-04-25T10:04:25.466178+0000","last_unstale":"2024-04-25T10:04:26.479803+0000","last_undegraded":"2024-04-25T10:04:26.479803+0000","last_fullsized":"2024-04-25T10:04:26.479803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:32.660382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480500+0000","last_change":"2024-04-25T10:04:25.477500+0000","last_active":"2024-04-25T10:04:25.480500+0000","last_peered":"2024-04-25T10:04:25.480500+0000","last_clean":"2024-04-25T10:04:25.480500+0000","last_became_active":"2024-04-25T10:04:25.475152+0000","last_became_peered":"2024-04-25T10:04:25.475152+0000","last_unstale":"2024-04-25T10:04:25.480500+0000","last_undegraded":"2024-04-25T10:04:25.480500+0000","last_fullsized":"2024-04-25T10:04:25.480500+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472951+0000","last_change":"2024-04-25T10:04:25.466973+0000","last_active":"2024-04-25T10:04:25.472951+0000","last_peered":"2024-04-25T10:04:25.472951+0000","last_clean":"2024-04-25T10:04:25.472951+0000","last_became_active":"2024-04-25T10:04:25.466685+0000","last_became_peered":"2024-04-25T10:04:25.466685+0000","last_unstale":"2024-04-25T10:04:25.472951+0000","last_undegraded":"2024-04-25T10:04:25.472951+0000","last_fullsized":"2024-04-25T10:04:25.472951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479182+0000","last_change":"2024-04-25T10:04:25.466682+0000","last_active":"2024-04-25T10:04:26.479182+0000","last_peered":"2024-04-25T10:04:26.479182+0000","last_clean":"2024-04-25T10:04:26.479182+0000","last_became_active":"2024-04-25T10:04:25.465981+0000","last_became_peered":"2024-04-25T10:04:25.465981+0000","last_unstale":"2024-04-25T10:04:26.479182+0000","last_undegraded":"2024-04-25T10:04:26.479182+0000","last_fullsized":"2024-04-25T10:04:26.479182+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:08.084202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479180+0000","last_change":"2024-04-25T10:04:25.473782+0000","last_active":"2024-04-25T10:04:26.479180+0000","last_peered":"2024-04-25T10:04:26.479180+0000","last_clean":"2024-04-25T10:04:26.479180+0000","last_became_active":"2024-04-25T10:04:25.470089+0000","last_became_peered":"2024-04-25T10:04:25.470089+0000","last_unstale":"2024-04-25T10:04:26.479180+0000","last_undegraded":"2024-04-25T10:04:26.479180+0000","last_fullsized":"2024-04-25T10:04:26.479180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.049551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478435+0000","last_change":"2024-04-25T10:04:25.462903+0000","last_active":"2024-04-25T10:04:26.478435+0000","last_peered":"2024-04-25T10:04:26.478435+0000","last_clean":"2024-04-25T10:04:26.478435+0000","last_became_active":"2024-04-25T10:04:25.462619+0000","last_became_peered":"2024-04-25T10:04:25.462619+0000","last_unstale":"2024-04-25T10:04:26.478435+0000","last_undegraded":"2024-04-25T10:04:26.478435+0000","last_fullsized":"2024-04-25T10:04:26.478435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:45.933238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472713+0000","last_change":"2024-04-25T10:04:25.465113+0000","last_active":"2024-04-25T10:04:25.472713+0000","last_peered":"2024-04-25T10:04:25.472713+0000","last_clean":"2024-04-25T10:04:25.472713+0000","last_became_active":"2024-04-25T10:04:25.464842+0000","last_became_peered":"2024-04-25T10:04:25.464842+0000","last_unstale":"2024-04-25T10:04:25.472713+0000","last_undegraded":"2024-04-25T10:04:25.472713+0000","last_fullsized":"2024-04-25T10:04:25.472713+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479918+0000","last_change":"2024-04-25T10:04:25.477772+0000","last_active":"2024-04-25T10:04:26.479918+0000","last_peered":"2024-04-25T10:04:26.479918+0000","last_clean":"2024-04-25T10:04:26.479918+0000","last_became_active":"2024-04-25T10:04:25.476288+0000","last_became_peered":"2024-04-25T10:04:25.476288+0000","last_unstale":"2024-04-25T10:04:26.479918+0000","last_undegraded":"2024-04-25T10:04:26.479918+0000","last_fullsized":"2024-04-25T10:04:26.479918+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.333518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474420+0000","last_change":"2024-04-25T10:04:25.465925+0000","last_active":"2024-04-25T10:04:25.474420+0000","last_peered":"2024-04-25T10:04:25.474420+0000","last_clean":"2024-04-25T10:04:25.474420+0000","last_became_active":"2024-04-25T10:04:25.465652+0000","last_became_peered":"2024-04-25T10:04:25.465652+0000","last_unstale":"2024-04-25T10:04:25.474420+0000","last_undegraded":"2024-04-25T10:04:25.474420+0000","last_fullsized":"2024-04-25T10:04:25.474420+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480477+0000","last_change":"2024-04-25T10:04:25.476769+0000","last_active":"2024-04-25T10:04:25.480477+0000","last_peered":"2024-04-25T10:04:25.480477+0000","last_clean":"2024-04-25T10:04:25.480477+0000","last_became_active":"2024-04-25T10:04:25.476451+0000","last_became_peered":"2024-04-25T10:04:25.476451+0000","last_unstale":"2024-04-25T10:04:25.480477+0000","last_undegraded":"2024-04-25T10:04:25.480477+0000","last_fullsized":"2024-04-25T10:04:25.480477+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472747+0000","last_change":"2024-04-25T10:04:25.469665+0000","last_active":"2024-04-25T10:04:25.472747+0000","last_peered":"2024-04-25T10:04:25.472747+0000","last_clean":"2024-04-25T10:04:25.472747+0000","last_became_active":"2024-04-25T10:04:25.469374+0000","last_became_peered":"2024-04-25T10:04:25.469374+0000","last_unstale":"2024-04-25T10:04:25.472747+0000","last_undegraded":"2024-04-25T10:04:25.472747+0000","last_fullsized":"2024-04-25T10:04:25.472747+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472340+0000","last_change":"2024-04-25T10:04:25.470230+0000","last_active":"2024-04-25T10:04:26.472340+0000","last_peered":"2024-04-25T10:04:26.472340+0000","last_clean":"2024-04-25T10:04:26.472340+0000","last_became_active":"2024-04-25T10:04:25.469402+0000","last_became_peered":"2024-04-25T10:04:25.469402+0000","last_unstale":"2024-04-25T10:04:26.472340+0000","last_undegraded":"2024-04-25T10:04:26.472340+0000","last_fullsized":"2024-04-25T10:04:26.472340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.544638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479366+0000","last_change":"2024-04-25T10:04:25.460768+0000","last_active":"2024-04-25T10:04:26.479366+0000","last_peered":"2024-04-25T10:04:26.479366+0000","last_clean":"2024-04-25T10:04:26.479366+0000","last_became_active":"2024-04-25T10:04:25.460525+0000","last_became_peered":"2024-04-25T10:04:25.460525+0000","last_unstale":"2024-04-25T10:04:26.479366+0000","last_undegraded":"2024-04-25T10:04:26.479366+0000","last_fullsized":"2024-04-25T10:04:26.479366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:57.909617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465537+0000","last_change":"2024-04-25T10:04:25.464113+0000","last_active":"2024-04-25T10:04:25.465537+0000","last_peered":"2024-04-25T10:04:25.465537+0000","last_clean":"2024-04-25T10:04:25.465537+0000","last_became_active":"2024-04-25T10:04:25.463493+0000","last_became_peered":"2024-04-25T10:04:25.463493+0000","last_unstale":"2024-04-25T10:04:25.465537+0000","last_undegraded":"2024-04-25T10:04:25.465537+0000","last_fullsized":"2024-04-25T10:04:25.465537+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472176+0000","last_change":"2024-04-25T10:04:25.469216+0000","last_active":"2024-04-25T10:04:26.472176+0000","last_peered":"2024-04-25T10:04:26.472176+0000","last_clean":"2024-04-25T10:04:26.472176+0000","last_became_active":"2024-04-25T10:04:25.466664+0000","last_became_peered":"2024-04-25T10:04:25.466664+0000","last_unstale":"2024-04-25T10:04:26.472176+0000","last_undegraded":"2024-04-25T10:04:26.472176+0000","last_fullsized":"2024-04-25T10:04:26.472176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:32.821688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474510+0000","last_change":"2024-04-25T10:04:24.469487+0000","last_active":"2024-04-25T10:04:25.474510+0000","last_peered":"2024-04-25T10:04:25.474510+0000","last_clean":"2024-04-25T10:04:25.474510+0000","last_became_active":"2024-04-25T10:04:24.469185+0000","last_became_peered":"2024-04-25T10:04:24.469185+0000","last_unstale":"2024-04-25T10:04:25.474510+0000","last_undegraded":"2024-04-25T10:04:25.474510+0000","last_fullsized":"2024-04-25T10:04:25.474510+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472240+0000","last_change":"2024-04-25T10:04:25.473033+0000","last_active":"2024-04-25T10:04:26.472240+0000","last_peered":"2024-04-25T10:04:26.472240+0000","last_clean":"2024-04-25T10:04:26.472240+0000","last_became_active":"2024-04-25T10:04:25.471870+0000","last_became_peered":"2024-04-25T10:04:25.471870+0000","last_unstale":"2024-04-25T10:04:26.472240+0000","last_undegraded":"2024-04-25T10:04:26.472240+0000","last_fullsized":"2024-04-25T10:04:26.472240+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:55.812057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481374+0000","last_change":"2024-04-25T10:04:24.468749+0000","last_active":"2024-04-25T10:04:25.481374+0000","last_peered":"2024-04-25T10:04:25.481374+0000","last_clean":"2024-04-25T10:04:25.481374+0000","last_became_active":"2024-04-25T10:04:24.468458+0000","last_became_peered":"2024-04-25T10:04:24.468458+0000","last_unstale":"2024-04-25T10:04:25.481374+0000","last_undegraded":"2024-04-25T10:04:25.481374+0000","last_fullsized":"2024-04-25T10:04:25.481374+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.077398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472493+0000","last_change":"2024-04-25T10:04:25.478215+0000","last_active":"2024-04-25T10:04:26.472493+0000","last_peered":"2024-04-25T10:04:26.472493+0000","last_clean":"2024-04-25T10:04:26.472493+0000","last_became_active":"2024-04-25T10:04:25.475504+0000","last_became_peered":"2024-04-25T10:04:25.475504+0000","last_unstale":"2024-04-25T10:04:26.472493+0000","last_undegraded":"2024-04-25T10:04:26.472493+0000","last_fullsized":"2024-04-25T10:04:26.472493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:41.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.471375+0000","last_change":"2024-04-25T10:04:24.459064+0000","last_active":"2024-04-25T10:04:25.471375+0000","last_peered":"2024-04-25T10:04:25.471375+0000","last_clean":"2024-04-25T10:04:25.471375+0000","last_became_active":"2024-04-25T10:04:24.458599+0000","last_became_peered":"2024-04-25T10:04:24.458599+0000","last_unstale":"2024-04-25T10:04:25.471375+0000","last_undegraded":"2024-04-25T10:04:25.471375+0000","last_fullsized":"2024-04-25T10:04:25.471375+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.483933+0000","last_change":"2024-04-25T10:04:25.473270+0000","last_active":"2024-04-25T10:04:25.483933+0000","last_peered":"2024-04-25T10:04:25.483933+0000","last_clean":"2024-04-25T10:04:25.483933+0000","last_became_active":"2024-04-25T10:04:25.472978+0000","last_became_peered":"2024-04-25T10:04:25.472978+0000","last_unstale":"2024-04-25T10:04:25.483933+0000","last_undegraded":"2024-04-25T10:04:25.483933+0000","last_fullsized":"2024-04-25T10:04:25.483933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465518+0000","last_change":"2024-04-25T10:04:24.468225+0000","last_active":"2024-04-25T10:04:25.465518+0000","last_peered":"2024-04-25T10:04:25.465518+0000","last_clean":"2024-04-25T10:04:25.465518+0000","last_became_active":"2024-04-25T10:04:24.467670+0000","last_became_peered":"2024-04-25T10:04:24.467670+0000","last_unstale":"2024-04-25T10:04:25.465518+0000","last_undegraded":"2024-04-25T10:04:25.465518+0000","last_fullsized":"2024-04-25T10:04:25.465518+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472793+0000","last_change":"2024-04-25T10:04:25.465227+0000","last_active":"2024-04-25T10:04:25.472793+0000","last_peered":"2024-04-25T10:04:25.472793+0000","last_clean":"2024-04-25T10:04:25.472793+0000","last_became_active":"2024-04-25T10:04:25.464968+0000","last_became_peered":"2024-04-25T10:04:25.464968+0000","last_unstale":"2024-04-25T10:04:25.472793+0000","last_undegraded":"2024-04-25T10:04:25.472793+0000","last_fullsized":"2024-04-25T10:04:25.472793+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479731+0000","last_change":"2024-04-25T10:04:24.467424+0000","last_active":"2024-04-25T10:04:26.479731+0000","last_peered":"2024-04-25T10:04:26.479731+0000","last_clean":"2024-04-25T10:04:26.479731+0000","last_became_active":"2024-04-25T10:04:24.467101+0000","last_became_peered":"2024-04-25T10:04:24.467101+0000","last_unstale":"2024-04-25T10:04:26.479731+0000","last_undegraded":"2024-04-25T10:04:26.479731+0000","last_fullsized":"2024-04-25T10:04:26.479731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:52.095406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473097+0000","last_change":"2024-04-25T10:04:25.471910+0000","last_active":"2024-04-25T10:04:26.473097+0000","last_peered":"2024-04-25T10:04:26.473097+0000","last_clean":"2024-04-25T10:04:26.473097+0000","last_became_active":"2024-04-25T10:04:25.471628+0000","last_became_peered":"2024-04-25T10:04:25.471628+0000","last_unstale":"2024-04-25T10:04:26.473097+0000","last_undegraded":"2024-04-25T10:04:26.473097+0000","last_fullsized":"2024-04-25T10:04:26.473097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:01.732449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472494+0000","last_change":"2024-04-25T10:04:24.449636+0000","last_active":"2024-04-25T10:04:26.472494+0000","last_peered":"2024-04-25T10:04:26.472494+0000","last_clean":"2024-04-25T10:04:26.472494+0000","last_became_active":"2024-04-25T10:04:24.449303+0000","last_became_peered":"2024-04-25T10:04:24.449303+0000","last_unstale":"2024-04-25T10:04:26.472494+0000","last_undegraded":"2024-04-25T10:04:26.472494+0000","last_fullsized":"2024-04-25T10:04:26.472494+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:58.775423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475584+0000","last_change":"2024-04-25T10:04:25.468855+0000","last_active":"2024-04-25T10:04:25.475584+0000","last_peered":"2024-04-25T10:04:25.475584+0000","last_clean":"2024-04-25T10:04:25.475584+0000","last_became_active":"2024-04-25T10:04:25.467011+0000","last_became_peered":"2024-04-25T10:04:25.467011+0000","last_unstale":"2024-04-25T10:04:25.475584+0000","last_undegraded":"2024-04-25T10:04:25.475584+0000","last_fullsized":"2024-04-25T10:04:25.475584+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472322+0000","last_change":"2024-04-25T10:04:24.469032+0000","last_active":"2024-04-25T10:04:26.472322+0000","last_peered":"2024-04-25T10:04:26.472322+0000","last_clean":"2024-04-25T10:04:26.472322+0000","last_became_active":"2024-04-25T10:04:24.468452+0000","last_became_peered":"2024-04-25T10:04:24.468452+0000","last_unstale":"2024-04-25T10:04:26.472322+0000","last_undegraded":"2024-04-25T10:04:26.472322+0000","last_fullsized":"2024-04-25T10:04:26.472322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:49.522317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.460571+0000","last_change":"2024-04-25T10:04:25.456211+0000","last_active":"2024-04-25T10:04:25.460571+0000","last_peered":"2024-04-25T10:04:25.460571+0000","last_clean":"2024-04-25T10:04:25.460571+0000","last_became_active":"2024-04-25T10:04:25.455945+0000","last_became_peered":"2024-04-25T10:04:25.455945+0000","last_unstale":"2024-04-25T10:04:25.460571+0000","last_undegraded":"2024-04-25T10:04:25.460571+0000","last_fullsized":"2024-04-25T10:04:25.460571+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478559+0000","last_change":"2024-04-25T10:04:24.468238+0000","last_active":"2024-04-25T10:04:26.478559+0000","last_peered":"2024-04-25T10:04:26.478559+0000","last_clean":"2024-04-25T10:04:26.478559+0000","last_became_active":"2024-04-25T10:04:24.467961+0000","last_became_peered":"2024-04-25T10:04:24.467961+0000","last_unstale":"2024-04-25T10:04:26.478559+0000","last_undegraded":"2024-04-25T10:04:26.478559+0000","last_fullsized":"2024-04-25T10:04:26.478559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:25.485389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472666+0000","last_change":"2024-04-25T10:04:25.462850+0000","last_active":"2024-04-25T10:04:25.472666+0000","last_peered":"2024-04-25T10:04:25.472666+0000","last_clean":"2024-04-25T10:04:25.472666+0000","last_became_active":"2024-04-25T10:04:25.462560+0000","last_became_peered":"2024-04-25T10:04:25.462560+0000","last_unstale":"2024-04-25T10:04:25.472666+0000","last_undegraded":"2024-04-25T10:04:25.472666+0000","last_fullsized":"2024-04-25T10:04:25.472666+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.444069+0000","last_change":"2024-04-25T10:04:24.447649+0000","last_active":"2024-04-25T10:04:25.444069+0000","last_peered":"2024-04-25T10:04:25.444069+0000","last_clean":"2024-04-25T10:04:25.444069+0000","last_became_active":"2024-04-25T10:04:24.446781+0000","last_became_peered":"2024-04-25T10:04:24.446781+0000","last_unstale":"2024-04-25T10:04:25.444069+0000","last_undegraded":"2024-04-25T10:04:25.444069+0000","last_fullsized":"2024-04-25T10:04:25.444069+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.029749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.457360+0000","last_change":"2024-04-25T10:04:25.452425+0000","last_active":"2024-04-25T10:04:25.457360+0000","last_peered":"2024-04-25T10:04:25.457360+0000","last_clean":"2024-04-25T10:04:25.457360+0000","last_became_active":"2024-04-25T10:04:25.451175+0000","last_became_peered":"2024-04-25T10:04:25.451175+0000","last_unstale":"2024-04-25T10:04:25.457360+0000","last_undegraded":"2024-04-25T10:04:25.457360+0000","last_fullsized":"2024-04-25T10:04:25.457360+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.443720+0000","last_change":"2024-04-25T10:04:24.451547+0000","last_active":"2024-04-25T10:04:25.443720+0000","last_peered":"2024-04-25T10:04:25.443720+0000","last_clean":"2024-04-25T10:04:25.443720+0000","last_became_active":"2024-04-25T10:04:24.451221+0000","last_became_peered":"2024-04-25T10:04:24.451221+0000","last_unstale":"2024-04-25T10:04:25.443720+0000","last_undegraded":"2024-04-25T10:04:25.443720+0000","last_fullsized":"2024-04-25T10:04:25.443720+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472556+0000","last_change":"2024-04-25T10:04:25.466717+0000","last_active":"2024-04-25T10:04:25.472556+0000","last_peered":"2024-04-25T10:04:25.472556+0000","last_clean":"2024-04-25T10:04:25.472556+0000","last_became_active":"2024-04-25T10:04:25.466392+0000","last_became_peered":"2024-04-25T10:04:25.466392+0000","last_unstale":"2024-04-25T10:04:25.472556+0000","last_undegraded":"2024-04-25T10:04:25.472556+0000","last_fullsized":"2024-04-25T10:04:25.472556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473094+0000","last_change":"2024-04-25T10:04:24.462694+0000","last_active":"2024-04-25T10:04:26.473094+0000","last_peered":"2024-04-25T10:04:26.473094+0000","last_clean":"2024-04-25T10:04:26.473094+0000","last_became_active":"2024-04-25T10:04:24.462301+0000","last_became_peered":"2024-04-25T10:04:24.462301+0000","last_unstale":"2024-04-25T10:04:26.473094+0000","last_undegraded":"2024-04-25T10:04:26.473094+0000","last_fullsized":"2024-04-25T10:04:26.473094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.011021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469006+0000","last_change":"2024-04-25T10:04:25.466716+0000","last_active":"2024-04-25T10:04:25.469006+0000","last_peered":"2024-04-25T10:04:25.469006+0000","last_clean":"2024-04-25T10:04:25.469006+0000","last_became_active":"2024-04-25T10:04:25.466438+0000","last_became_peered":"2024-04-25T10:04:25.466438+0000","last_unstale":"2024-04-25T10:04:25.469006+0000","last_undegraded":"2024-04-25T10:04:25.469006+0000","last_fullsized":"2024-04-25T10:04:25.469006+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.914161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.486863+0000","last_change":"2024-04-25T10:04:24.469012+0000","last_active":"2024-04-25T10:04:25.486863+0000","last_peered":"2024-04-25T10:04:25.486863+0000","last_clean":"2024-04-25T10:04:25.486863+0000","last_became_active":"2024-04-25T10:04:24.468729+0000","last_became_peered":"2024-04-25T10:04:24.468729+0000","last_unstale":"2024-04-25T10:04:25.486863+0000","last_undegraded":"2024-04-25T10:04:25.486863+0000","last_fullsized":"2024-04-25T10:04:25.486863+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472871+0000","last_change":"2024-04-25T10:04:25.468212+0000","last_active":"2024-04-25T10:04:26.472871+0000","last_peered":"2024-04-25T10:04:26.472871+0000","last_clean":"2024-04-25T10:04:26.472871+0000","last_became_active":"2024-04-25T10:04:25.467940+0000","last_became_peered":"2024-04-25T10:04:25.467940+0000","last_unstale":"2024-04-25T10:04:26.472871+0000","last_undegraded":"2024-04-25T10:04:26.472871+0000","last_fullsized":"2024-04-25T10:04:26.472871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:04.368539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482369+0000","last_change":"2024-04-25T10:04:24.456904+0000","last_active":"2024-04-25T10:04:25.482369+0000","last_peered":"2024-04-25T10:04:25.482369+0000","last_clean":"2024-04-25T10:04:25.482369+0000","last_became_active":"2024-04-25T10:04:24.456095+0000","last_became_peered":"2024-04-25T10:04:24.456095+0000","last_unstale":"2024-04-25T10:04:25.482369+0000","last_undegraded":"2024-04-25T10:04:25.482369+0000","last_fullsized":"2024-04-25T10:04:25.482369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.185120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482977+0000","last_change":"2024-04-25T10:04:25.479170+0000","last_active":"2024-04-25T10:04:25.482977+0000","last_peered":"2024-04-25T10:04:25.482977+0000","last_clean":"2024-04-25T10:04:25.482977+0000","last_became_active":"2024-04-25T10:04:25.478980+0000","last_became_peered":"2024-04-25T10:04:25.478980+0000","last_unstale":"2024-04-25T10:04:25.482977+0000","last_undegraded":"2024-04-25T10:04:25.482977+0000","last_fullsized":"2024-04-25T10:04:25.482977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.442376+0000","last_change":"2024-04-25T10:04:24.458424+0000","last_active":"2024-04-25T10:04:25.442376+0000","last_peered":"2024-04-25T10:04:25.442376+0000","last_clean":"2024-04-25T10:04:25.442376+0000","last_became_active":"2024-04-25T10:04:24.458098+0000","last_became_peered":"2024-04-25T10:04:24.458098+0000","last_unstale":"2024-04-25T10:04:25.442376+0000","last_undegraded":"2024-04-25T10:04:25.442376+0000","last_fullsized":"2024-04-25T10:04:25.442376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.469999+0000","last_change":"2024-04-25T10:04:25.455450+0000","last_active":"2024-04-25T10:04:25.469999+0000","last_peered":"2024-04-25T10:04:25.469999+0000","last_clean":"2024-04-25T10:04:25.469999+0000","last_became_active":"2024-04-25T10:04:25.454137+0000","last_became_peered":"2024-04-25T10:04:25.454137+0000","last_unstale":"2024-04-25T10:04:25.469999+0000","last_undegraded":"2024-04-25T10:04:25.469999+0000","last_fullsized":"2024-04-25T10:04:25.469999+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:11.030611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446148+0000","last_change":"2024-04-25T10:04:24.445036+0000","last_active":"2024-04-25T10:04:25.446148+0000","last_peered":"2024-04-25T10:04:25.446148+0000","last_clean":"2024-04-25T10:04:25.446148+0000","last_became_active":"2024-04-25T10:04:24.444715+0000","last_became_peered":"2024-04-25T10:04:24.444715+0000","last_unstale":"2024-04-25T10:04:25.446148+0000","last_undegraded":"2024-04-25T10:04:25.446148+0000","last_fullsized":"2024-04-25T10:04:25.446148+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.905326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480231+0000","last_change":"2024-04-25T10:04:25.460045+0000","last_active":"2024-04-25T10:04:26.480231+0000","last_peered":"2024-04-25T10:04:26.480231+0000","last_clean":"2024-04-25T10:04:26.480231+0000","last_became_active":"2024-04-25T10:04:25.458524+0000","last_became_peered":"2024-04-25T10:04:25.458524+0000","last_unstale":"2024-04-25T10:04:26.480231+0000","last_undegraded":"2024-04-25T10:04:26.480231+0000","last_fullsized":"2024-04-25T10:04:26.480231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:40.245623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":20,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.482536+0000","last_change":"2024-04-25T10:04:24.465436+0000","last_active":"2024-04-25T10:04:25.482536+0000","last_peered":"2024-04-25T10:04:25.482536+0000","last_clean":"2024-04-25T10:04:25.482536+0000","last_became_active":"2024-04-25T10:04:24.465132+0000","last_became_peered":"2024-04-25T10:04:24.465132+0000","last_unstale":"2024-04-25T10:04:25.482536+0000","last_undegraded":"2024-04-25T10:04:25.482536+0000","last_fullsized":"2024-04-25T10:04:25.482536+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.884017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.479931+0000","last_change":"2024-04-25T10:04:25.472528+0000","last_active":"2024-04-25T10:04:25.479931+0000","last_peered":"2024-04-25T10:04:25.479931+0000","last_clean":"2024-04-25T10:04:25.479931+0000","last_became_active":"2024-04-25T10:04:25.472245+0000","last_became_peered":"2024-04-25T10:04:25.472245+0000","last_unstale":"2024-04-25T10:04:25.479931+0000","last_undegraded":"2024-04-25T10:04:25.479931+0000","last_fullsized":"2024-04-25T10:04:25.479931+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481219+0000","last_change":"2024-04-25T10:04:24.468526+0000","last_active":"2024-04-25T10:04:25.481219+0000","last_peered":"2024-04-25T10:04:25.481219+0000","last_clean":"2024-04-25T10:04:25.481219+0000","last_became_active":"2024-04-25T10:04:24.468213+0000","last_became_peered":"2024-04-25T10:04:24.468213+0000","last_unstale":"2024-04-25T10:04:25.481219+0000","last_undegraded":"2024-04-25T10:04:25.481219+0000","last_fullsized":"2024-04-25T10:04:25.481219+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:40.655704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472621+0000","last_change":"2024-04-25T10:04:25.463721+0000","last_active":"2024-04-25T10:04:25.472621+0000","last_peered":"2024-04-25T10:04:25.472621+0000","last_clean":"2024-04-25T10:04:25.472621+0000","last_became_active":"2024-04-25T10:04:25.463355+0000","last_became_peered":"2024-04-25T10:04:25.463355+0000","last_unstale":"2024-04-25T10:04:25.472621+0000","last_undegraded":"2024-04-25T10:04:25.472621+0000","last_fullsized":"2024-04-25T10:04:25.472621+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.481630+0000","last_change":"2024-04-25T10:04:24.461901+0000","last_active":"2024-04-25T10:04:25.481630+0000","last_peered":"2024-04-25T10:04:25.481630+0000","last_clean":"2024-04-25T10:04:25.481630+0000","last_became_active":"2024-04-25T10:04:24.461619+0000","last_became_peered":"2024-04-25T10:04:24.461619+0000","last_unstale":"2024-04-25T10:04:25.481630+0000","last_undegraded":"2024-04-25T10:04:25.481630+0000","last_fullsized":"2024-04-25T10:04:25.481630+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:32.072513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480737+0000","last_change":"2024-04-25T10:04:25.477724+0000","last_active":"2024-04-25T10:04:25.480737+0000","last_peered":"2024-04-25T10:04:25.480737+0000","last_clean":"2024-04-25T10:04:25.480737+0000","last_became_active":"2024-04-25T10:04:25.476438+0000","last_became_peered":"2024-04-25T10:04:25.476438+0000","last_unstale":"2024-04-25T10:04:25.480737+0000","last_undegraded":"2024-04-25T10:04:25.480737+0000","last_fullsized":"2024-04-25T10:04:25.480737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475376+0000","last_change":"2024-04-25T10:04:24.465620+0000","last_active":"2024-04-25T10:04:25.475376+0000","last_peered":"2024-04-25T10:04:25.475376+0000","last_clean":"2024-04-25T10:04:25.475376+0000","last_became_active":"2024-04-25T10:04:24.465402+0000","last_became_peered":"2024-04-25T10:04:24.465402+0000","last_unstale":"2024-04-25T10:04:25.475376+0000","last_undegraded":"2024-04-25T10:04:25.475376+0000","last_fullsized":"2024-04-25T10:04:25.475376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.474444+0000","last_change":"2024-04-25T10:04:25.463940+0000","last_active":"2024-04-25T10:04:25.474444+0000","last_peered":"2024-04-25T10:04:25.474444+0000","last_clean":"2024-04-25T10:04:25.474444+0000","last_became_active":"2024-04-25T10:04:25.463675+0000","last_became_peered":"2024-04-25T10:04:25.463675+0000","last_unstale":"2024-04-25T10:04:25.474444+0000","last_undegraded":"2024-04-25T10:04:25.474444+0000","last_fullsized":"2024-04-25T10:04:25.474444+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:41.042749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.456328+0000","last_change":"2024-04-25T10:04:24.466373+0000","last_active":"2024-04-25T10:04:25.456328+0000","last_peered":"2024-04-25T10:04:25.456328+0000","last_clean":"2024-04-25T10:04:25.456328+0000","last_became_active":"2024-04-25T10:04:24.465965+0000","last_became_peered":"2024-04-25T10:04:24.465965+0000","last_unstale":"2024-04-25T10:04:25.456328+0000","last_undegraded":"2024-04-25T10:04:25.456328+0000","last_fullsized":"2024-04-25T10:04:25.456328+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.478776+0000","last_change":"2024-04-25T10:04:25.471532+0000","last_active":"2024-04-25T10:04:25.478776+0000","last_peered":"2024-04-25T10:04:25.478776+0000","last_clean":"2024-04-25T10:04:25.478776+0000","last_became_active":"2024-04-25T10:04:25.471276+0000","last_became_peered":"2024-04-25T10:04:25.471276+0000","last_unstale":"2024-04-25T10:04:25.478776+0000","last_undegraded":"2024-04-25T10:04:25.478776+0000","last_fullsized":"2024-04-25T10:04:25.478776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:17.980993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.468521+0000","last_change":"2024-04-25T10:04:24.461754+0000","last_active":"2024-04-25T10:04:25.468521+0000","last_peered":"2024-04-25T10:04:25.468521+0000","last_clean":"2024-04-25T10:04:25.468521+0000","last_became_active":"2024-04-25T10:04:24.459617+0000","last_became_peered":"2024-04-25T10:04:24.459617+0000","last_unstale":"2024-04-25T10:04:25.468521+0000","last_undegraded":"2024-04-25T10:04:25.468521+0000","last_fullsized":"2024-04-25T10:04:25.468521+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.473141+0000","last_change":"2024-04-25T10:04:25.468223+0000","last_active":"2024-04-25T10:04:25.473141+0000","last_peered":"2024-04-25T10:04:25.473141+0000","last_clean":"2024-04-25T10:04:25.473141+0000","last_became_active":"2024-04-25T10:04:25.467369+0000","last_became_peered":"2024-04-25T10:04:25.467369+0000","last_unstale":"2024-04-25T10:04:25.473141+0000","last_undegraded":"2024-04-25T10:04:25.473141+0000","last_fullsized":"2024-04-25T10:04:25.473141+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479325+0000","last_change":"2024-04-25T10:04:24.471952+0000","last_active":"2024-04-25T10:04:26.479325+0000","last_peered":"2024-04-25T10:04:26.479325+0000","last_clean":"2024-04-25T10:04:26.479325+0000","last_became_active":"2024-04-25T10:04:24.471642+0000","last_became_peered":"2024-04-25T10:04:24.471642+0000","last_unstale":"2024-04-25T10:04:26.479325+0000","last_undegraded":"2024-04-25T10:04:26.479325+0000","last_fullsized":"2024-04-25T10:04:26.479325+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:00.609015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472693+0000","last_change":"2024-04-25T10:04:25.468105+0000","last_active":"2024-04-25T10:04:25.472693+0000","last_peered":"2024-04-25T10:04:25.472693+0000","last_clean":"2024-04-25T10:04:25.472693+0000","last_became_active":"2024-04-25T10:04:25.467841+0000","last_became_peered":"2024-04-25T10:04:25.467841+0000","last_unstale":"2024-04-25T10:04:25.472693+0000","last_undegraded":"2024-04-25T10:04:25.472693+0000","last_fullsized":"2024-04-25T10:04:25.472693+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.451244+0000","last_change":"2024-04-25T10:04:24.469768+0000","last_active":"2024-04-25T10:04:25.451244+0000","last_peered":"2024-04-25T10:04:25.451244+0000","last_clean":"2024-04-25T10:04:25.451244+0000","last_became_active":"2024-04-25T10:04:24.469488+0000","last_became_peered":"2024-04-25T10:04:24.469488+0000","last_unstale":"2024-04-25T10:04:25.451244+0000","last_undegraded":"2024-04-25T10:04:25.451244+0000","last_fullsized":"2024-04-25T10:04:25.451244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:29.785579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480787+0000","last_change":"2024-04-25T10:04:25.475514+0000","last_active":"2024-04-25T10:04:25.480787+0000","last_peered":"2024-04-25T10:04:25.480787+0000","last_clean":"2024-04-25T10:04:25.480787+0000","last_became_active":"2024-04-25T10:04:25.474364+0000","last_became_peered":"2024-04-25T10:04:25.474364+0000","last_unstale":"2024-04-25T10:04:25.480787+0000","last_undegraded":"2024-04-25T10:04:25.480787+0000","last_fullsized":"2024-04-25T10:04:25.480787+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:12.606739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.453420+0000","last_change":"2024-04-25T10:04:24.460278+0000","last_active":"2024-04-25T10:04:25.453420+0000","last_peered":"2024-04-25T10:04:25.453420+0000","last_clean":"2024-04-25T10:04:25.453420+0000","last_became_active":"2024-04-25T10:04:24.459936+0000","last_became_peered":"2024-04-25T10:04:24.459936+0000","last_unstale":"2024-04-25T10:04:25.453420+0000","last_undegraded":"2024-04-25T10:04:25.453420+0000","last_fullsized":"2024-04-25T10:04:25.453420+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:19.332266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.465336+0000","last_change":"2024-04-25T10:04:25.456950+0000","last_active":"2024-04-25T10:04:25.465336+0000","last_peered":"2024-04-25T10:04:25.465336+0000","last_clean":"2024-04-25T10:04:25.465336+0000","last_became_active":"2024-04-25T10:04:25.456664+0000","last_became_peered":"2024-04-25T10:04:25.456664+0000","last_unstale":"2024-04-25T10:04:25.465336+0000","last_undegraded":"2024-04-25T10:04:25.465336+0000","last_fullsized":"2024-04-25T10:04:25.465336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473040+0000","last_change":"2024-04-25T10:04:24.466995+0000","last_active":"2024-04-25T10:04:26.473040+0000","last_peered":"2024-04-25T10:04:26.473040+0000","last_clean":"2024-04-25T10:04:26.473040+0000","last_became_active":"2024-04-25T10:04:24.466667+0000","last_became_peered":"2024-04-25T10:04:24.466667+0000","last_unstale":"2024-04-25T10:04:26.473040+0000","last_undegraded":"2024-04-25T10:04:26.473040+0000","last_fullsized":"2024-04-25T10:04:26.473040+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:07.560341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475023+0000","last_change":"2024-04-25T10:04:25.467788+0000","last_active":"2024-04-25T10:04:25.475023+0000","last_peered":"2024-04-25T10:04:25.475023+0000","last_clean":"2024-04-25T10:04:25.475023+0000","last_became_active":"2024-04-25T10:04:25.465574+0000","last_became_peered":"2024-04-25T10:04:25.465574+0000","last_unstale":"2024-04-25T10:04:25.475023+0000","last_undegraded":"2024-04-25T10:04:25.475023+0000","last_fullsized":"2024-04-25T10:04:25.475023+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473253+0000","last_change":"2024-04-25T10:04:24.463577+0000","last_active":"2024-04-25T10:04:26.473253+0000","last_peered":"2024-04-25T10:04:26.473253+0000","last_clean":"2024-04-25T10:04:26.473253+0000","last_became_active":"2024-04-25T10:04:24.462318+0000","last_became_peered":"2024-04-25T10:04:24.462318+0000","last_unstale":"2024-04-25T10:04:26.473253+0000","last_undegraded":"2024-04-25T10:04:26.473253+0000","last_fullsized":"2024-04-25T10:04:26.473253+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:01.690130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479939+0000","last_change":"2024-04-25T10:04:25.461326+0000","last_active":"2024-04-25T10:04:26.479939+0000","last_peered":"2024-04-25T10:04:26.479939+0000","last_clean":"2024-04-25T10:04:26.479939+0000","last_became_active":"2024-04-25T10:04:25.459940+0000","last_became_peered":"2024-04-25T10:04:25.459940+0000","last_unstale":"2024-04-25T10:04:26.479939+0000","last_undegraded":"2024-04-25T10:04:26.479939+0000","last_fullsized":"2024-04-25T10:04:26.479939+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:48.804816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479691+0000","last_change":"2024-04-25T10:04:24.462828+0000","last_active":"2024-04-25T10:04:26.479691+0000","last_peered":"2024-04-25T10:04:26.479691+0000","last_clean":"2024-04-25T10:04:26.479691+0000","last_became_active":"2024-04-25T10:04:24.462443+0000","last_became_peered":"2024-04-25T10:04:24.462443+0000","last_unstale":"2024-04-25T10:04:26.479691+0000","last_undegraded":"2024-04-25T10:04:26.479691+0000","last_fullsized":"2024-04-25T10:04:26.479691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:20.157546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475078+0000","last_change":"2024-04-25T10:04:25.466945+0000","last_active":"2024-04-25T10:04:25.475078+0000","last_peered":"2024-04-25T10:04:25.475078+0000","last_clean":"2024-04-25T10:04:25.475078+0000","last_became_active":"2024-04-25T10:04:25.466672+0000","last_became_peered":"2024-04-25T10:04:25.466672+0000","last_unstale":"2024-04-25T10:04:25.475078+0000","last_undegraded":"2024-04-25T10:04:25.475078+0000","last_fullsized":"2024-04-25T10:04:25.475078+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.450703+0000","last_change":"2024-04-25T10:04:24.461824+0000","last_active":"2024-04-25T10:04:25.450703+0000","last_peered":"2024-04-25T10:04:25.450703+0000","last_clean":"2024-04-25T10:04:25.450703+0000","last_became_active":"2024-04-25T10:04:24.460800+0000","last_became_peered":"2024-04-25T10:04:24.460800+0000","last_unstale":"2024-04-25T10:04:25.450703+0000","last_undegraded":"2024-04-25T10:04:25.450703+0000","last_fullsized":"2024-04-25T10:04:25.450703+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:25.199121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.480645+0000","last_change":"2024-04-25T10:04:25.477935+0000","last_active":"2024-04-25T10:04:25.480645+0000","last_peered":"2024-04-25T10:04:25.480645+0000","last_clean":"2024-04-25T10:04:25.480645+0000","last_became_active":"2024-04-25T10:04:25.477605+0000","last_became_peered":"2024-04-25T10:04:25.477605+0000","last_unstale":"2024-04-25T10:04:25.480645+0000","last_undegraded":"2024-04-25T10:04:25.480645+0000","last_fullsized":"2024-04-25T10:04:25.480645+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479437+0000","last_change":"2024-04-25T10:04:24.471290+0000","last_active":"2024-04-25T10:04:26.479437+0000","last_peered":"2024-04-25T10:04:26.479437+0000","last_clean":"2024-04-25T10:04:26.479437+0000","last_became_active":"2024-04-25T10:04:24.470984+0000","last_became_peered":"2024-04-25T10:04:24.470984+0000","last_unstale":"2024-04-25T10:04:26.479437+0000","last_undegraded":"2024-04-25T10:04:26.479437+0000","last_fullsized":"2024-04-25T10:04:26.479437+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:50.982046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475556+0000","last_change":"2024-04-25T10:04:25.462505+0000","last_active":"2024-04-25T10:04:25.475556+0000","last_peered":"2024-04-25T10:04:25.475556+0000","last_clean":"2024-04-25T10:04:25.475556+0000","last_became_active":"2024-04-25T10:04:25.461056+0000","last_became_peered":"2024-04-25T10:04:25.461056+0000","last_unstale":"2024-04-25T10:04:25.475556+0000","last_undegraded":"2024-04-25T10:04:25.475556+0000","last_fullsized":"2024-04-25T10:04:25.475556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.446296+0000","last_change":"2024-04-25T10:04:24.443846+0000","last_active":"2024-04-25T10:04:25.446296+0000","last_peered":"2024-04-25T10:04:25.446296+0000","last_clean":"2024-04-25T10:04:25.446296+0000","last_became_active":"2024-04-25T10:04:24.443511+0000","last_became_peered":"2024-04-25T10:04:24.443511+0000","last_unstale":"2024-04-25T10:04:25.446296+0000","last_undegraded":"2024-04-25T10:04:25.446296+0000","last_fullsized":"2024-04-25T10:04:25.446296+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.475649+0000","last_change":"2024-04-25T10:04:25.466904+0000","last_active":"2024-04-25T10:04:25.475649+0000","last_peered":"2024-04-25T10:04:25.475649+0000","last_clean":"2024-04-25T10:04:25.475649+0000","last_became_active":"2024-04-25T10:04:25.466434+0000","last_became_peered":"2024-04-25T10:04:25.466434+0000","last_unstale":"2024-04-25T10:04:25.475649+0000","last_undegraded":"2024-04-25T10:04:25.475649+0000","last_fullsized":"2024-04-25T10:04:25.475649+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:26.212122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441933+0000","last_change":"2024-04-25T10:04:24.467188+0000","last_active":"2024-04-25T10:04:25.441933+0000","last_peered":"2024-04-25T10:04:25.441933+0000","last_clean":"2024-04-25T10:04:25.441933+0000","last_became_active":"2024-04-25T10:04:24.466850+0000","last_became_peered":"2024-04-25T10:04:24.466850+0000","last_unstale":"2024-04-25T10:04:25.441933+0000","last_undegraded":"2024-04-25T10:04:25.441933+0000","last_fullsized":"2024-04-25T10:04:25.441933+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.472620+0000","last_change":"2024-04-25T10:04:25.461224+0000","last_active":"2024-04-25T10:04:25.472620+0000","last_peered":"2024-04-25T10:04:25.472620+0000","last_clean":"2024-04-25T10:04:25.472620+0000","last_became_active":"2024-04-25T10:04:25.460727+0000","last_became_peered":"2024-04-25T10:04:25.460727+0000","last_unstale":"2024-04-25T10:04:25.472620+0000","last_undegraded":"2024-04-25T10:04:25.472620+0000","last_fullsized":"2024-04-25T10:04:25.472620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.452980+0000","last_change":"2024-04-25T10:04:24.461091+0000","last_active":"2024-04-25T10:04:25.452980+0000","last_peered":"2024-04-25T10:04:25.452980+0000","last_clean":"2024-04-25T10:04:25.452980+0000","last_became_active":"2024-04-25T10:04:24.460322+0000","last_became_peered":"2024-04-25T10:04:24.460322+0000","last_unstale":"2024-04-25T10:04:25.452980+0000","last_undegraded":"2024-04-25T10:04:25.452980+0000","last_fullsized":"2024-04-25T10:04:25.452980+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:17.560293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":223,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T10:04:25.441995+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:04:25.441995+0000","last_peered":"2024-04-25T10:04:25.441995+0000","last_clean":"2024-04-25T10:04:25.441995+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:04:25.441995+0000","last_undegraded":"2024-04-25T10:04:25.441995+0000","last_fullsized":"2024-04-25T10:04:25.441995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":602112,"data_stored":262104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":838523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":573440,"data_stored":248155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":618496,"data_stored":263718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:35.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.225+0000 7fdf56400640 1 Processor -- start 2024-04-25T10:04:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.225+0000 7fdf56400640 1 -- start start 2024-04-25T10:04:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.226+0000 7fdf56400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 0x7fdf50161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.226+0000 7fdf56400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdf501622b0 0x7fdf500ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.226+0000 7fdf56400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5015e8a0 con 0x7fdf5015da60 2024-04-25T10:04:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.226+0000 7fdf56400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5015eb80 con 0x7fdf501622b0 2024-04-25T10:04:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.226+0000 7fdf56400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5015ee60 con 0x7fdf500d0aa0 2024-04-25T10:04:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.227+0000 7fdf4f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdf5015da60 0x7fdf5015de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60574/0 (socket says 172.21.15.5:60574) 2024-04-25T10:04:35.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.227+0000 7fdf4f400640 1 -- 172.21.15.5:0/735362232 learned_addr learned my addr 172.21.15.5:0/735362232 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.227+0000 7fdf46a00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 0x7fdf50161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.227+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/735362232 >> v2:172.21.15.80:3300/0 conn(0x7fdf501622b0 0x7fdf500ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.227+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211594570 0 0) 0x7fdf5015e8a0 con 0x7fdf5015da60 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.227+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf34003520 con 0x7fdf5015da60 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 226784549 0 0) 0x7fdf34003520 con 0x7fdf5015da60 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 msgr2=0x7fdf50161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 0x7fdf50161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 >> v2:172.21.15.80:3300/0 conn(0x7fdf501622b0 msgr2=0x7fdf500ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 --2- 172.21.15.5:0/735362232 >> v2:172.21.15.80:3300/0 conn(0x7fdf501622b0 0x7fdf500ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50161850 con 0x7fdf5015da60 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fdf400019f0 con 0x7fdf5015da60 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/735362232 >> v2:172.21.15.80:3300/0 conn(0x7fdf501622b0 0x7fdf500ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 >> v1:172.21.15.5:6789/0 conn(0x7fdf5015da60 legacy=0x7fdf5015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 0x7fdf340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf34003f80 0x7fdf34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34004d40 0x7fdf340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5015eb80 con 0x7fdf34004d40 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5015ee60 con 0x7fdf34003f80 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3400bb20 con 0x7fdf500d0aa0 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34004d40 0x7fdf340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf4f400640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 0x7fdf340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.228+0000 7fdf46a00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf34003f80 0x7fdf34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.229+0000 7fdf4fe00640 1 -- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 msgr2=0x7fdf340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.229+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf500d0aa0 0x7fdf340039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.229+0000 7fdf4fe00640 1 -- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf34003f80 msgr2=0x7fdf34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.229+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf34003f80 0x7fdf34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.229+0000 7fdf4fe00640 1 -- 172.21.15.5:0/735362232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34003520 con 0x7fdf34004d40 2024-04-25T10:04:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.229+0000 7fdf46a00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf34003f80 0x7fdf34004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34004d40 0x7fdf340051c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098a0 tx=0x7fdf34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526309e67eb32b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800f040 con 0x7fdf34004d40 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4802ad20 con 0x7fdf34004d40 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf4ea00640 1 -- 172.21.15.5:0/735362232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48004250 con 0x7fdf34004d40 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf56400640 1 -- 172.21.15.5:0/735362232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34004d40 msgr2=0x7fdf340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf56400640 1 --2- 172.21.15.5:0/735362232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34004d40 0x7fdf340051c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdf480098a0 tx=0x7fdf34005830 comp rx=0 tx=0).stop 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.230+0000 7fdf4fe00640 1 -- 172.21.15.5:0/735362232 reap_dead start 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.231+0000 7fdf56400640 1 -- 172.21.15.5:0/735362232 shutdown_connections 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.231+0000 7fdf56400640 1 -- 172.21.15.5:0/735362232 >> 172.21.15.5:0/735362232 conn(0x7fdf5006bfe0 msgr2=0x7fdf5006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.231+0000 7fdf56400640 1 -- 172.21.15.5:0/735362232 shutdown_connections 2024-04-25T10:04:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.231+0000 7fdf56400640 1 -- 172.21.15.5:0/735362232 wait complete. 2024-04-25T10:04:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.233+0000 7fdf56400640 1 Processor -- start 2024-04-25T10:04:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.233+0000 7fdf56400640 1 -- start start 2024-04-25T10:04:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf56400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf34004d40 0x7fdf5015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf56400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf500cb2a0 0x7fdf501f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf56400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 0x7fdf500aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf56400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50161ef0 con 0x7fdf500aa220 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf56400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50161d70 con 0x7fdf500cb2a0 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf56400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50162070 con 0x7fdf34004d40 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 0x7fdf500aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 0x7fdf500aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57790/0 (socket says 172.21.15.5:57790) 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf4fe00640 1 -- 172.21.15.5:0/944830270 learned_addr learned my addr 172.21.15.5:0/944830270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf4f400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf34004d40 0x7fdf5015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.234+0000 7fdf46a00640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf500cb2a0 0x7fdf501f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf4fe00640 1 -- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf34004d40 msgr2=0x7fdf5015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf34004d40 0x7fdf5015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf4fe00640 1 -- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf500cb2a0 msgr2=0x7fdf501f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf500cb2a0 0x7fdf501f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf4fe00640 1 -- 172.21.15.5:0/944830270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500cbdb0 con 0x7fdf500aa220 2024-04-25T10:04:35.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf46a00640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf500cb2a0 0x7fdf501f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf4fe00640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 0x7fdf500aa6b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdf480099c0 tx=0x7fdf48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800f000 con 0x7fdf500aa220 2024-04-25T10:04:35.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48044070 con 0x7fdf500aa220 2024-04-25T10:04:35.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48003b50 con 0x7fdf500aa220 2024-04-25T10:04:35.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.235+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf500cc090 con 0x7fdf500aa220 2024-04-25T10:04:35.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.236+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf500cc650 con 0x7fdf500aa220 2024-04-25T10:04:35.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.237+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf500cbb20 con 0x7fdf500aa220 2024-04-25T10:04:35.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.237+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf4803b040 con 0x7fdf500aa220 2024-04-25T10:04:35.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.238+0000 7fdf47e00640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf2c06b4d0 0x7fdf2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.239+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdf480aa200 con 0x7fdf500aa220 2024-04-25T10:04:35.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.239+0000 7fdf4f400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf2c06b4d0 0x7fdf2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.240+0000 7fdf4f400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf2c06b4d0 0x7fdf2c06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdf500cd540 tx=0x7fdf40009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.243+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf48078e50 con 0x7fdf500aa220 2024-04-25T10:04:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.445+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf500680c0 con 0x7fdf2c06b4d0 2024-04-25T10:04:35.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.463+0000 7fdf47e00640 1 -- 172.21.15.5:0/944830270 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251958 (secure 0 0 0) 0x7fdf40002010 con 0x7fdf2c06b4d0 2024-04-25T10:04:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf2c06b4d0 msgr2=0x7fdf2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf2c06b4d0 0x7fdf2c06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdf500cd540 tx=0x7fdf40009040 comp rx=0 tx=0).stop 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 msgr2=0x7fdf500aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 0x7fdf500aa6b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdf480099c0 tx=0x7fdf48004730 comp rx=0 tx=0).stop 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 shutdown_connections 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf500aa220 0x7fdf500aa6b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf2c06b4d0 0x7fdf2c06d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf500cb2a0 0x7fdf501f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 --2- 172.21.15.5:0/944830270 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf34004d40 0x7fdf5015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 >> 172.21.15.5:0/944830270 conn(0x7fdf5006bfe0 msgr2=0x7fdf501606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 shutdown_connections 2024-04-25T10:04:35.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.470+0000 7fdf56400640 1 -- 172.21.15.5:0/944830270 wait complete. 2024-04-25T10:04:35.495 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-04-25T10:04:34.667469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238708,"kb_used_data":5896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736012,"statfs":{"total":773094113280,"available":772849676288,"internally_reserved":0,"allocated":6037504,"data_stored":3281668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.232692"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.518340+0000","last_change":"2024-04-25T10:04:25.461883+0000","last_active":"2024-04-25T10:04:26.518340+0000","last_peered":"2024-04-25T10:04:26.518340+0000","last_clean":"2024-04-25T10:04:26.518340+0000","last_became_active":"2024-04-25T10:04:25.460987+0000","last_became_peered":"2024-04-25T10:04:25.460987+0000","last_unstale":"2024-04-25T10:04:26.518340+0000","last_undegraded":"2024-04-25T10:04:26.518340+0000","last_fullsized":"2024-04-25T10:04:26.518340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:26.920195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517954+0000","last_change":"2024-04-25T10:04:25.474565+0000","last_active":"2024-04-25T10:04:26.517954+0000","last_peered":"2024-04-25T10:04:26.517954+0000","last_clean":"2024-04-25T10:04:26.517954+0000","last_became_active":"2024-04-25T10:04:25.474282+0000","last_became_peered":"2024-04-25T10:04:25.474282+0000","last_unstale":"2024-04-25T10:04:26.517954+0000","last_undegraded":"2024-04-25T10:04:26.517954+0000","last_fullsized":"2024-04-25T10:04:26.517954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:24.481673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479735+0000","last_change":"2024-04-25T10:04:25.460733+0000","last_active":"2024-04-25T10:04:26.479735+0000","last_peered":"2024-04-25T10:04:26.479735+0000","last_clean":"2024-04-25T10:04:26.479735+0000","last_became_active":"2024-04-25T10:04:25.460475+0000","last_became_peered":"2024-04-25T10:04:25.460475+0000","last_unstale":"2024-04-25T10:04:26.479735+0000","last_undegraded":"2024-04-25T10:04:26.479735+0000","last_fullsized":"2024-04-25T10:04:26.479735+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:20.427097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515930+0000","last_change":"2024-04-25T10:04:25.454040+0000","last_active":"2024-04-25T10:04:26.515930+0000","last_peered":"2024-04-25T10:04:26.515930+0000","last_clean":"2024-04-25T10:04:26.515930+0000","last_became_active":"2024-04-25T10:04:25.453680+0000","last_became_peered":"2024-04-25T10:04:25.453680+0000","last_unstale":"2024-04-25T10:04:26.515930+0000","last_undegraded":"2024-04-25T10:04:26.515930+0000","last_fullsized":"2024-04-25T10:04:26.515930+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516567+0000","last_change":"2024-04-25T10:04:25.468526+0000","last_active":"2024-04-25T10:04:26.516567+0000","last_peered":"2024-04-25T10:04:26.516567+0000","last_clean":"2024-04-25T10:04:26.516567+0000","last_became_active":"2024-04-25T10:04:25.468181+0000","last_became_peered":"2024-04-25T10:04:25.468181+0000","last_unstale":"2024-04-25T10:04:26.516567+0000","last_undegraded":"2024-04-25T10:04:26.516567+0000","last_fullsized":"2024-04-25T10:04:26.516567+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478570+0000","last_change":"2024-04-25T10:04:25.448768+0000","last_active":"2024-04-25T10:04:26.478570+0000","last_peered":"2024-04-25T10:04:26.478570+0000","last_clean":"2024-04-25T10:04:26.478570+0000","last_became_active":"2024-04-25T10:04:25.448208+0000","last_became_peered":"2024-04-25T10:04:25.448208+0000","last_unstale":"2024-04-25T10:04:26.478570+0000","last_undegraded":"2024-04-25T10:04:26.478570+0000","last_fullsized":"2024-04-25T10:04:26.478570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515697+0000","last_change":"2024-04-25T10:04:25.475883+0000","last_active":"2024-04-25T10:04:26.515697+0000","last_peered":"2024-04-25T10:04:26.515697+0000","last_clean":"2024-04-25T10:04:26.515697+0000","last_became_active":"2024-04-25T10:04:25.475594+0000","last_became_peered":"2024-04-25T10:04:25.475594+0000","last_unstale":"2024-04-25T10:04:26.515697+0000","last_undegraded":"2024-04-25T10:04:26.515697+0000","last_fullsized":"2024-04-25T10:04:26.515697+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.823142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478527+0000","last_change":"2024-04-25T10:04:25.461616+0000","last_active":"2024-04-25T10:04:26.478527+0000","last_peered":"2024-04-25T10:04:26.478527+0000","last_clean":"2024-04-25T10:04:26.478527+0000","last_became_active":"2024-04-25T10:04:25.461353+0000","last_became_peered":"2024-04-25T10:04:25.461353+0000","last_unstale":"2024-04-25T10:04:26.478527+0000","last_undegraded":"2024-04-25T10:04:26.478527+0000","last_fullsized":"2024-04-25T10:04:26.478527+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.483109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480232+0000","last_change":"2024-04-25T10:04:25.462804+0000","last_active":"2024-04-25T10:04:26.480232+0000","last_peered":"2024-04-25T10:04:26.480232+0000","last_clean":"2024-04-25T10:04:26.480232+0000","last_became_active":"2024-04-25T10:04:25.458840+0000","last_became_peered":"2024-04-25T10:04:25.458840+0000","last_unstale":"2024-04-25T10:04:26.480232+0000","last_undegraded":"2024-04-25T10:04:26.480232+0000","last_fullsized":"2024-04-25T10:04:26.480232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:33.800771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516538+0000","last_change":"2024-04-25T10:04:25.469094+0000","last_active":"2024-04-25T10:04:26.516538+0000","last_peered":"2024-04-25T10:04:26.516538+0000","last_clean":"2024-04-25T10:04:26.516538+0000","last_became_active":"2024-04-25T10:04:25.468584+0000","last_became_peered":"2024-04-25T10:04:25.468584+0000","last_unstale":"2024-04-25T10:04:26.516538+0000","last_undegraded":"2024-04-25T10:04:26.516538+0000","last_fullsized":"2024-04-25T10:04:26.516538+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:32.967882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517979+0000","last_change":"2024-04-25T10:04:25.456279+0000","last_active":"2024-04-25T10:04:26.517979+0000","last_peered":"2024-04-25T10:04:26.517979+0000","last_clean":"2024-04-25T10:04:26.517979+0000","last_became_active":"2024-04-25T10:04:25.456002+0000","last_became_peered":"2024-04-25T10:04:25.456002+0000","last_unstale":"2024-04-25T10:04:26.517979+0000","last_undegraded":"2024-04-25T10:04:26.517979+0000","last_fullsized":"2024-04-25T10:04:26.517979+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:24.481673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479098+0000","last_change":"2024-04-25T10:04:25.467692+0000","last_active":"2024-04-25T10:04:26.479098+0000","last_peered":"2024-04-25T10:04:26.479098+0000","last_clean":"2024-04-25T10:04:26.479098+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:26.479098+0000","last_undegraded":"2024-04-25T10:04:26.479098+0000","last_fullsized":"2024-04-25T10:04:26.479098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478384+0000","last_change":"2024-04-25T10:04:25.469711+0000","last_active":"2024-04-25T10:04:26.478384+0000","last_peered":"2024-04-25T10:04:26.478384+0000","last_clean":"2024-04-25T10:04:26.478384+0000","last_became_active":"2024-04-25T10:04:25.469239+0000","last_became_peered":"2024-04-25T10:04:25.469239+0000","last_unstale":"2024-04-25T10:04:26.478384+0000","last_undegraded":"2024-04-25T10:04:26.478384+0000","last_fullsized":"2024-04-25T10:04:26.478384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.345266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516493+0000","last_change":"2024-04-25T10:04:25.479586+0000","last_active":"2024-04-25T10:04:26.516493+0000","last_peered":"2024-04-25T10:04:26.516493+0000","last_clean":"2024-04-25T10:04:26.516493+0000","last_became_active":"2024-04-25T10:04:25.478560+0000","last_became_peered":"2024-04-25T10:04:25.478560+0000","last_unstale":"2024-04-25T10:04:26.516493+0000","last_undegraded":"2024-04-25T10:04:26.516493+0000","last_fullsized":"2024-04-25T10:04:26.516493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515130+0000","last_change":"2024-04-25T10:04:25.467693+0000","last_active":"2024-04-25T10:04:26.515130+0000","last_peered":"2024-04-25T10:04:26.515130+0000","last_clean":"2024-04-25T10:04:26.515130+0000","last_became_active":"2024-04-25T10:04:25.467414+0000","last_became_peered":"2024-04-25T10:04:25.467414+0000","last_unstale":"2024-04-25T10:04:26.515130+0000","last_undegraded":"2024-04-25T10:04:26.515130+0000","last_fullsized":"2024-04-25T10:04:26.515130+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478272+0000","last_change":"2024-04-25T10:04:25.450560+0000","last_active":"2024-04-25T10:04:26.478272+0000","last_peered":"2024-04-25T10:04:26.478272+0000","last_clean":"2024-04-25T10:04:26.478272+0000","last_became_active":"2024-04-25T10:04:25.450242+0000","last_became_peered":"2024-04-25T10:04:25.450242+0000","last_unstale":"2024-04-25T10:04:26.478272+0000","last_undegraded":"2024-04-25T10:04:26.478272+0000","last_fullsized":"2024-04-25T10:04:26.478272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.518559+0000","last_change":"2024-04-25T10:04:25.463042+0000","last_active":"2024-04-25T10:04:26.518559+0000","last_peered":"2024-04-25T10:04:26.518559+0000","last_clean":"2024-04-25T10:04:26.518559+0000","last_became_active":"2024-04-25T10:04:25.460640+0000","last_became_peered":"2024-04-25T10:04:25.460640+0000","last_unstale":"2024-04-25T10:04:26.518559+0000","last_undegraded":"2024-04-25T10:04:26.518559+0000","last_fullsized":"2024-04-25T10:04:26.518559+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:32.967882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517619+0000","last_change":"2024-04-25T10:04:25.455252+0000","last_active":"2024-04-25T10:04:26.517619+0000","last_peered":"2024-04-25T10:04:26.517619+0000","last_clean":"2024-04-25T10:04:26.517619+0000","last_became_active":"2024-04-25T10:04:25.454978+0000","last_became_peered":"2024-04-25T10:04:25.454978+0000","last_unstale":"2024-04-25T10:04:26.517619+0000","last_undegraded":"2024-04-25T10:04:26.517619+0000","last_fullsized":"2024-04-25T10:04:26.517619+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:24.481673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479803+0000","last_change":"2024-04-25T10:04:25.466442+0000","last_active":"2024-04-25T10:04:26.479803+0000","last_peered":"2024-04-25T10:04:26.479803+0000","last_clean":"2024-04-25T10:04:26.479803+0000","last_became_active":"2024-04-25T10:04:25.466178+0000","last_became_peered":"2024-04-25T10:04:25.466178+0000","last_unstale":"2024-04-25T10:04:26.479803+0000","last_undegraded":"2024-04-25T10:04:26.479803+0000","last_fullsized":"2024-04-25T10:04:26.479803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:32.660382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516398+0000","last_change":"2024-04-25T10:04:25.477500+0000","last_active":"2024-04-25T10:04:26.516398+0000","last_peered":"2024-04-25T10:04:26.516398+0000","last_clean":"2024-04-25T10:04:26.516398+0000","last_became_active":"2024-04-25T10:04:25.475152+0000","last_became_peered":"2024-04-25T10:04:25.475152+0000","last_unstale":"2024-04-25T10:04:26.516398+0000","last_undegraded":"2024-04-25T10:04:26.516398+0000","last_fullsized":"2024-04-25T10:04:26.516398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515587+0000","last_change":"2024-04-25T10:04:25.466973+0000","last_active":"2024-04-25T10:04:26.515587+0000","last_peered":"2024-04-25T10:04:26.515587+0000","last_clean":"2024-04-25T10:04:26.515587+0000","last_became_active":"2024-04-25T10:04:25.466685+0000","last_became_peered":"2024-04-25T10:04:25.466685+0000","last_unstale":"2024-04-25T10:04:26.515587+0000","last_undegraded":"2024-04-25T10:04:26.515587+0000","last_fullsized":"2024-04-25T10:04:26.515587+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479182+0000","last_change":"2024-04-25T10:04:25.466682+0000","last_active":"2024-04-25T10:04:26.479182+0000","last_peered":"2024-04-25T10:04:26.479182+0000","last_clean":"2024-04-25T10:04:26.479182+0000","last_became_active":"2024-04-25T10:04:25.465981+0000","last_became_peered":"2024-04-25T10:04:25.465981+0000","last_unstale":"2024-04-25T10:04:26.479182+0000","last_undegraded":"2024-04-25T10:04:26.479182+0000","last_fullsized":"2024-04-25T10:04:26.479182+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:08.084202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479180+0000","last_change":"2024-04-25T10:04:25.473782+0000","last_active":"2024-04-25T10:04:26.479180+0000","last_peered":"2024-04-25T10:04:26.479180+0000","last_clean":"2024-04-25T10:04:26.479180+0000","last_became_active":"2024-04-25T10:04:25.470089+0000","last_became_peered":"2024-04-25T10:04:25.470089+0000","last_unstale":"2024-04-25T10:04:26.479180+0000","last_undegraded":"2024-04-25T10:04:26.479180+0000","last_fullsized":"2024-04-25T10:04:26.479180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.049551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478435+0000","last_change":"2024-04-25T10:04:25.462903+0000","last_active":"2024-04-25T10:04:26.478435+0000","last_peered":"2024-04-25T10:04:26.478435+0000","last_clean":"2024-04-25T10:04:26.478435+0000","last_became_active":"2024-04-25T10:04:25.462619+0000","last_became_peered":"2024-04-25T10:04:25.462619+0000","last_unstale":"2024-04-25T10:04:26.478435+0000","last_undegraded":"2024-04-25T10:04:26.478435+0000","last_fullsized":"2024-04-25T10:04:26.478435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:45.933238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.518453+0000","last_change":"2024-04-25T10:04:25.465113+0000","last_active":"2024-04-25T10:04:26.518453+0000","last_peered":"2024-04-25T10:04:26.518453+0000","last_clean":"2024-04-25T10:04:26.518453+0000","last_became_active":"2024-04-25T10:04:25.464842+0000","last_became_peered":"2024-04-25T10:04:25.464842+0000","last_unstale":"2024-04-25T10:04:26.518453+0000","last_undegraded":"2024-04-25T10:04:26.518453+0000","last_fullsized":"2024-04-25T10:04:26.518453+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:33.048161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479918+0000","last_change":"2024-04-25T10:04:25.477772+0000","last_active":"2024-04-25T10:04:26.479918+0000","last_peered":"2024-04-25T10:04:26.479918+0000","last_clean":"2024-04-25T10:04:26.479918+0000","last_became_active":"2024-04-25T10:04:25.476288+0000","last_became_peered":"2024-04-25T10:04:25.476288+0000","last_unstale":"2024-04-25T10:04:26.479918+0000","last_undegraded":"2024-04-25T10:04:26.479918+0000","last_fullsized":"2024-04-25T10:04:26.479918+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.333518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478692+0000","last_change":"2024-04-25T10:04:25.465925+0000","last_active":"2024-04-25T10:04:26.478692+0000","last_peered":"2024-04-25T10:04:26.478692+0000","last_clean":"2024-04-25T10:04:26.478692+0000","last_became_active":"2024-04-25T10:04:25.465652+0000","last_became_peered":"2024-04-25T10:04:25.465652+0000","last_unstale":"2024-04-25T10:04:26.478692+0000","last_undegraded":"2024-04-25T10:04:26.478692+0000","last_fullsized":"2024-04-25T10:04:26.478692+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516657+0000","last_change":"2024-04-25T10:04:25.476769+0000","last_active":"2024-04-25T10:04:26.516657+0000","last_peered":"2024-04-25T10:04:26.516657+0000","last_clean":"2024-04-25T10:04:26.516657+0000","last_became_active":"2024-04-25T10:04:25.476451+0000","last_became_peered":"2024-04-25T10:04:25.476451+0000","last_unstale":"2024-04-25T10:04:26.516657+0000","last_undegraded":"2024-04-25T10:04:26.516657+0000","last_fullsized":"2024-04-25T10:04:26.516657+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517347+0000","last_change":"2024-04-25T10:04:25.469665+0000","last_active":"2024-04-25T10:04:26.517347+0000","last_peered":"2024-04-25T10:04:26.517347+0000","last_clean":"2024-04-25T10:04:26.517347+0000","last_became_active":"2024-04-25T10:04:25.469374+0000","last_became_peered":"2024-04-25T10:04:25.469374+0000","last_unstale":"2024-04-25T10:04:26.517347+0000","last_undegraded":"2024-04-25T10:04:26.517347+0000","last_fullsized":"2024-04-25T10:04:26.517347+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472340+0000","last_change":"2024-04-25T10:04:25.470230+0000","last_active":"2024-04-25T10:04:26.472340+0000","last_peered":"2024-04-25T10:04:26.472340+0000","last_clean":"2024-04-25T10:04:26.472340+0000","last_became_active":"2024-04-25T10:04:25.469402+0000","last_became_peered":"2024-04-25T10:04:25.469402+0000","last_unstale":"2024-04-25T10:04:26.472340+0000","last_undegraded":"2024-04-25T10:04:26.472340+0000","last_fullsized":"2024-04-25T10:04:26.472340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:38.544638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479366+0000","last_change":"2024-04-25T10:04:25.460768+0000","last_active":"2024-04-25T10:04:26.479366+0000","last_peered":"2024-04-25T10:04:26.479366+0000","last_clean":"2024-04-25T10:04:26.479366+0000","last_became_active":"2024-04-25T10:04:25.460525+0000","last_became_peered":"2024-04-25T10:04:25.460525+0000","last_unstale":"2024-04-25T10:04:26.479366+0000","last_undegraded":"2024-04-25T10:04:26.479366+0000","last_fullsized":"2024-04-25T10:04:26.479366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:57.909617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515631+0000","last_change":"2024-04-25T10:04:25.464113+0000","last_active":"2024-04-25T10:04:26.515631+0000","last_peered":"2024-04-25T10:04:26.515631+0000","last_clean":"2024-04-25T10:04:26.515631+0000","last_became_active":"2024-04-25T10:04:25.463493+0000","last_became_peered":"2024-04-25T10:04:25.463493+0000","last_unstale":"2024-04-25T10:04:26.515631+0000","last_undegraded":"2024-04-25T10:04:26.515631+0000","last_fullsized":"2024-04-25T10:04:26.515631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472176+0000","last_change":"2024-04-25T10:04:25.469216+0000","last_active":"2024-04-25T10:04:26.472176+0000","last_peered":"2024-04-25T10:04:26.472176+0000","last_clean":"2024-04-25T10:04:26.472176+0000","last_became_active":"2024-04-25T10:04:25.466664+0000","last_became_peered":"2024-04-25T10:04:25.466664+0000","last_unstale":"2024-04-25T10:04:26.472176+0000","last_undegraded":"2024-04-25T10:04:26.472176+0000","last_fullsized":"2024-04-25T10:04:26.472176+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:32.821688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":18,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.519899+0000","last_change":"2024-04-25T10:04:24.469487+0000","last_active":"2024-04-25T10:04:26.519899+0000","last_peered":"2024-04-25T10:04:26.519899+0000","last_clean":"2024-04-25T10:04:26.519899+0000","last_became_active":"2024-04-25T10:04:24.469185+0000","last_became_peered":"2024-04-25T10:04:24.469185+0000","last_unstale":"2024-04-25T10:04:26.519899+0000","last_undegraded":"2024-04-25T10:04:26.519899+0000","last_fullsized":"2024-04-25T10:04:26.519899+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472240+0000","last_change":"2024-04-25T10:04:25.473033+0000","last_active":"2024-04-25T10:04:26.472240+0000","last_peered":"2024-04-25T10:04:26.472240+0000","last_clean":"2024-04-25T10:04:26.472240+0000","last_became_active":"2024-04-25T10:04:25.471870+0000","last_became_peered":"2024-04-25T10:04:25.471870+0000","last_unstale":"2024-04-25T10:04:26.472240+0000","last_undegraded":"2024-04-25T10:04:26.472240+0000","last_fullsized":"2024-04-25T10:04:26.472240+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:55.812057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478545+0000","last_change":"2024-04-25T10:04:24.468749+0000","last_active":"2024-04-25T10:04:26.478545+0000","last_peered":"2024-04-25T10:04:26.478545+0000","last_clean":"2024-04-25T10:04:26.478545+0000","last_became_active":"2024-04-25T10:04:24.468458+0000","last_became_peered":"2024-04-25T10:04:24.468458+0000","last_unstale":"2024-04-25T10:04:26.478545+0000","last_undegraded":"2024-04-25T10:04:26.478545+0000","last_fullsized":"2024-04-25T10:04:26.478545+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.077398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472493+0000","last_change":"2024-04-25T10:04:25.478215+0000","last_active":"2024-04-25T10:04:26.472493+0000","last_peered":"2024-04-25T10:04:26.472493+0000","last_clean":"2024-04-25T10:04:26.472493+0000","last_became_active":"2024-04-25T10:04:25.475504+0000","last_became_peered":"2024-04-25T10:04:25.475504+0000","last_unstale":"2024-04-25T10:04:26.472493+0000","last_undegraded":"2024-04-25T10:04:26.472493+0000","last_fullsized":"2024-04-25T10:04:26.472493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:41.458468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516667+0000","last_change":"2024-04-25T10:04:24.459064+0000","last_active":"2024-04-25T10:04:26.516667+0000","last_peered":"2024-04-25T10:04:26.516667+0000","last_clean":"2024-04-25T10:04:26.516667+0000","last_became_active":"2024-04-25T10:04:24.458599+0000","last_became_peered":"2024-04-25T10:04:24.458599+0000","last_unstale":"2024-04-25T10:04:26.516667+0000","last_undegraded":"2024-04-25T10:04:26.516667+0000","last_fullsized":"2024-04-25T10:04:26.516667+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:11.593739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516317+0000","last_change":"2024-04-25T10:04:25.473270+0000","last_active":"2024-04-25T10:04:26.516317+0000","last_peered":"2024-04-25T10:04:26.516317+0000","last_clean":"2024-04-25T10:04:26.516317+0000","last_became_active":"2024-04-25T10:04:25.472978+0000","last_became_peered":"2024-04-25T10:04:25.472978+0000","last_unstale":"2024-04-25T10:04:26.516317+0000","last_undegraded":"2024-04-25T10:04:26.516317+0000","last_fullsized":"2024-04-25T10:04:26.516317+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:26.920195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516673+0000","last_change":"2024-04-25T10:04:24.468225+0000","last_active":"2024-04-25T10:04:26.516673+0000","last_peered":"2024-04-25T10:04:26.516673+0000","last_clean":"2024-04-25T10:04:26.516673+0000","last_became_active":"2024-04-25T10:04:24.467670+0000","last_became_peered":"2024-04-25T10:04:24.467670+0000","last_unstale":"2024-04-25T10:04:26.516673+0000","last_undegraded":"2024-04-25T10:04:26.516673+0000","last_fullsized":"2024-04-25T10:04:26.516673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.520053+0000","last_change":"2024-04-25T10:04:25.465227+0000","last_active":"2024-04-25T10:04:26.520053+0000","last_peered":"2024-04-25T10:04:26.520053+0000","last_clean":"2024-04-25T10:04:26.520053+0000","last_became_active":"2024-04-25T10:04:25.464968+0000","last_became_peered":"2024-04-25T10:04:25.464968+0000","last_unstale":"2024-04-25T10:04:26.520053+0000","last_undegraded":"2024-04-25T10:04:26.520053+0000","last_fullsized":"2024-04-25T10:04:26.520053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479731+0000","last_change":"2024-04-25T10:04:24.467424+0000","last_active":"2024-04-25T10:04:26.479731+0000","last_peered":"2024-04-25T10:04:26.479731+0000","last_clean":"2024-04-25T10:04:26.479731+0000","last_became_active":"2024-04-25T10:04:24.467101+0000","last_became_peered":"2024-04-25T10:04:24.467101+0000","last_unstale":"2024-04-25T10:04:26.479731+0000","last_undegraded":"2024-04-25T10:04:26.479731+0000","last_fullsized":"2024-04-25T10:04:26.479731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:52.095406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473097+0000","last_change":"2024-04-25T10:04:25.471910+0000","last_active":"2024-04-25T10:04:26.473097+0000","last_peered":"2024-04-25T10:04:26.473097+0000","last_clean":"2024-04-25T10:04:26.473097+0000","last_became_active":"2024-04-25T10:04:25.471628+0000","last_became_peered":"2024-04-25T10:04:25.471628+0000","last_unstale":"2024-04-25T10:04:26.473097+0000","last_undegraded":"2024-04-25T10:04:26.473097+0000","last_fullsized":"2024-04-25T10:04:26.473097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:01.732449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472494+0000","last_change":"2024-04-25T10:04:24.449636+0000","last_active":"2024-04-25T10:04:26.472494+0000","last_peered":"2024-04-25T10:04:26.472494+0000","last_clean":"2024-04-25T10:04:26.472494+0000","last_became_active":"2024-04-25T10:04:24.449303+0000","last_became_peered":"2024-04-25T10:04:24.449303+0000","last_unstale":"2024-04-25T10:04:26.472494+0000","last_undegraded":"2024-04-25T10:04:26.472494+0000","last_fullsized":"2024-04-25T10:04:26.472494+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:58.775423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517250+0000","last_change":"2024-04-25T10:04:25.468855+0000","last_active":"2024-04-25T10:04:26.517250+0000","last_peered":"2024-04-25T10:04:26.517250+0000","last_clean":"2024-04-25T10:04:26.517250+0000","last_became_active":"2024-04-25T10:04:25.467011+0000","last_became_peered":"2024-04-25T10:04:25.467011+0000","last_unstale":"2024-04-25T10:04:26.517250+0000","last_undegraded":"2024-04-25T10:04:26.517250+0000","last_fullsized":"2024-04-25T10:04:26.517250+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:47.599192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472322+0000","last_change":"2024-04-25T10:04:24.469032+0000","last_active":"2024-04-25T10:04:26.472322+0000","last_peered":"2024-04-25T10:04:26.472322+0000","last_clean":"2024-04-25T10:04:26.472322+0000","last_became_active":"2024-04-25T10:04:24.468452+0000","last_became_peered":"2024-04-25T10:04:24.468452+0000","last_unstale":"2024-04-25T10:04:26.472322+0000","last_undegraded":"2024-04-25T10:04:26.472322+0000","last_fullsized":"2024-04-25T10:04:26.472322+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:49.522317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516745+0000","last_change":"2024-04-25T10:04:25.456211+0000","last_active":"2024-04-25T10:04:26.516745+0000","last_peered":"2024-04-25T10:04:26.516745+0000","last_clean":"2024-04-25T10:04:26.516745+0000","last_became_active":"2024-04-25T10:04:25.455945+0000","last_became_peered":"2024-04-25T10:04:25.455945+0000","last_unstale":"2024-04-25T10:04:26.516745+0000","last_undegraded":"2024-04-25T10:04:26.516745+0000","last_fullsized":"2024-04-25T10:04:26.516745+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478559+0000","last_change":"2024-04-25T10:04:24.468238+0000","last_active":"2024-04-25T10:04:26.478559+0000","last_peered":"2024-04-25T10:04:26.478559+0000","last_clean":"2024-04-25T10:04:26.478559+0000","last_became_active":"2024-04-25T10:04:24.467961+0000","last_became_peered":"2024-04-25T10:04:24.467961+0000","last_unstale":"2024-04-25T10:04:26.478559+0000","last_undegraded":"2024-04-25T10:04:26.478559+0000","last_fullsized":"2024-04-25T10:04:26.478559+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:25.485389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.518962+0000","last_change":"2024-04-25T10:04:25.462850+0000","last_active":"2024-04-25T10:04:26.518962+0000","last_peered":"2024-04-25T10:04:26.518962+0000","last_clean":"2024-04-25T10:04:26.518962+0000","last_became_active":"2024-04-25T10:04:25.462560+0000","last_became_peered":"2024-04-25T10:04:25.462560+0000","last_unstale":"2024-04-25T10:04:26.518962+0000","last_undegraded":"2024-04-25T10:04:26.518962+0000","last_fullsized":"2024-04-25T10:04:26.518962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:35.862061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517210+0000","last_change":"2024-04-25T10:04:24.447649+0000","last_active":"2024-04-25T10:04:26.517210+0000","last_peered":"2024-04-25T10:04:26.517210+0000","last_clean":"2024-04-25T10:04:26.517210+0000","last_became_active":"2024-04-25T10:04:24.446781+0000","last_became_peered":"2024-04-25T10:04:24.446781+0000","last_unstale":"2024-04-25T10:04:26.517210+0000","last_undegraded":"2024-04-25T10:04:26.517210+0000","last_fullsized":"2024-04-25T10:04:26.517210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:40.029749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515545+0000","last_change":"2024-04-25T10:04:25.452425+0000","last_active":"2024-04-25T10:04:26.515545+0000","last_peered":"2024-04-25T10:04:26.515545+0000","last_clean":"2024-04-25T10:04:26.515545+0000","last_became_active":"2024-04-25T10:04:25.451175+0000","last_became_peered":"2024-04-25T10:04:25.451175+0000","last_unstale":"2024-04-25T10:04:26.515545+0000","last_undegraded":"2024-04-25T10:04:26.515545+0000","last_fullsized":"2024-04-25T10:04:26.515545+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:28.840843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517603+0000","last_change":"2024-04-25T10:04:24.451547+0000","last_active":"2024-04-25T10:04:26.517603+0000","last_peered":"2024-04-25T10:04:26.517603+0000","last_clean":"2024-04-25T10:04:26.517603+0000","last_became_active":"2024-04-25T10:04:24.451221+0000","last_became_peered":"2024-04-25T10:04:24.451221+0000","last_unstale":"2024-04-25T10:04:26.517603+0000","last_undegraded":"2024-04-25T10:04:26.517603+0000","last_fullsized":"2024-04-25T10:04:26.517603+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517035+0000","last_change":"2024-04-25T10:04:25.466717+0000","last_active":"2024-04-25T10:04:26.517035+0000","last_peered":"2024-04-25T10:04:26.517035+0000","last_clean":"2024-04-25T10:04:26.517035+0000","last_became_active":"2024-04-25T10:04:25.466392+0000","last_became_peered":"2024-04-25T10:04:25.466392+0000","last_unstale":"2024-04-25T10:04:26.517035+0000","last_undegraded":"2024-04-25T10:04:26.517035+0000","last_fullsized":"2024-04-25T10:04:26.517035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":17,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473094+0000","last_change":"2024-04-25T10:04:24.462694+0000","last_active":"2024-04-25T10:04:26.473094+0000","last_peered":"2024-04-25T10:04:26.473094+0000","last_clean":"2024-04-25T10:04:26.473094+0000","last_became_active":"2024-04-25T10:04:24.462301+0000","last_became_peered":"2024-04-25T10:04:24.462301+0000","last_unstale":"2024-04-25T10:04:26.473094+0000","last_undegraded":"2024-04-25T10:04:26.473094+0000","last_fullsized":"2024-04-25T10:04:26.473094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:19.011021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517038+0000","last_change":"2024-04-25T10:04:25.466716+0000","last_active":"2024-04-25T10:04:26.517038+0000","last_peered":"2024-04-25T10:04:26.517038+0000","last_clean":"2024-04-25T10:04:26.517038+0000","last_became_active":"2024-04-25T10:04:25.466438+0000","last_became_peered":"2024-04-25T10:04:25.466438+0000","last_unstale":"2024-04-25T10:04:26.517038+0000","last_undegraded":"2024-04-25T10:04:26.517038+0000","last_fullsized":"2024-04-25T10:04:26.517038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":20,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516799+0000","last_change":"2024-04-25T10:04:24.469012+0000","last_active":"2024-04-25T10:04:26.516799+0000","last_peered":"2024-04-25T10:04:26.516799+0000","last_clean":"2024-04-25T10:04:26.516799+0000","last_became_active":"2024-04-25T10:04:24.468729+0000","last_became_peered":"2024-04-25T10:04:24.468729+0000","last_unstale":"2024-04-25T10:04:26.516799+0000","last_undegraded":"2024-04-25T10:04:26.516799+0000","last_fullsized":"2024-04-25T10:04:26.516799+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:16.967993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.472871+0000","last_change":"2024-04-25T10:04:25.468212+0000","last_active":"2024-04-25T10:04:26.472871+0000","last_peered":"2024-04-25T10:04:26.472871+0000","last_clean":"2024-04-25T10:04:26.472871+0000","last_became_active":"2024-04-25T10:04:25.467940+0000","last_became_peered":"2024-04-25T10:04:25.467940+0000","last_unstale":"2024-04-25T10:04:26.472871+0000","last_undegraded":"2024-04-25T10:04:26.472871+0000","last_fullsized":"2024-04-25T10:04:26.472871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:04.368539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478568+0000","last_change":"2024-04-25T10:04:24.456904+0000","last_active":"2024-04-25T10:04:26.478568+0000","last_peered":"2024-04-25T10:04:26.478568+0000","last_clean":"2024-04-25T10:04:26.478568+0000","last_became_active":"2024-04-25T10:04:24.456095+0000","last_became_peered":"2024-04-25T10:04:24.456095+0000","last_unstale":"2024-04-25T10:04:26.478568+0000","last_undegraded":"2024-04-25T10:04:26.478568+0000","last_fullsized":"2024-04-25T10:04:26.478568+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:11.185120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.518112+0000","last_change":"2024-04-25T10:04:25.479170+0000","last_active":"2024-04-25T10:04:26.518112+0000","last_peered":"2024-04-25T10:04:26.518112+0000","last_clean":"2024-04-25T10:04:26.518112+0000","last_became_active":"2024-04-25T10:04:25.478980+0000","last_became_peered":"2024-04-25T10:04:25.478980+0000","last_unstale":"2024-04-25T10:04:26.518112+0000","last_undegraded":"2024-04-25T10:04:26.518112+0000","last_fullsized":"2024-04-25T10:04:26.518112+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:04.823142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515124+0000","last_change":"2024-04-25T10:04:24.458424+0000","last_active":"2024-04-25T10:04:26.515124+0000","last_peered":"2024-04-25T10:04:26.515124+0000","last_clean":"2024-04-25T10:04:26.515124+0000","last_became_active":"2024-04-25T10:04:24.458098+0000","last_became_peered":"2024-04-25T10:04:24.458098+0000","last_unstale":"2024-04-25T10:04:26.515124+0000","last_undegraded":"2024-04-25T10:04:26.515124+0000","last_fullsized":"2024-04-25T10:04:26.515124+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:10.017610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480187+0000","last_change":"2024-04-25T10:04:25.455450+0000","last_active":"2024-04-25T10:04:26.480187+0000","last_peered":"2024-04-25T10:04:26.480187+0000","last_clean":"2024-04-25T10:04:26.480187+0000","last_became_active":"2024-04-25T10:04:25.454137+0000","last_became_peered":"2024-04-25T10:04:25.454137+0000","last_unstale":"2024-04-25T10:04:26.480187+0000","last_undegraded":"2024-04-25T10:04:26.480187+0000","last_fullsized":"2024-04-25T10:04:26.480187+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:52.325953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515970+0000","last_change":"2024-04-25T10:04:24.445036+0000","last_active":"2024-04-25T10:04:26.515970+0000","last_peered":"2024-04-25T10:04:26.515970+0000","last_clean":"2024-04-25T10:04:26.515970+0000","last_became_active":"2024-04-25T10:04:24.444715+0000","last_became_peered":"2024-04-25T10:04:24.444715+0000","last_unstale":"2024-04-25T10:04:26.515970+0000","last_undegraded":"2024-04-25T10:04:26.515970+0000","last_fullsized":"2024-04-25T10:04:26.515970+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:25.905326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480231+0000","last_change":"2024-04-25T10:04:25.460045+0000","last_active":"2024-04-25T10:04:26.480231+0000","last_peered":"2024-04-25T10:04:26.480231+0000","last_clean":"2024-04-25T10:04:26.480231+0000","last_became_active":"2024-04-25T10:04:25.458524+0000","last_became_peered":"2024-04-25T10:04:25.458524+0000","last_unstale":"2024-04-25T10:04:26.480231+0000","last_undegraded":"2024-04-25T10:04:26.480231+0000","last_fullsized":"2024-04-25T10:04:26.480231+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:40.245623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":22,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478806+0000","last_change":"2024-04-25T10:04:24.465436+0000","last_active":"2024-04-25T10:04:26.478806+0000","last_peered":"2024-04-25T10:04:26.478806+0000","last_clean":"2024-04-25T10:04:26.478806+0000","last_became_active":"2024-04-25T10:04:24.465132+0000","last_became_peered":"2024-04-25T10:04:24.465132+0000","last_unstale":"2024-04-25T10:04:26.478806+0000","last_undegraded":"2024-04-25T10:04:26.478806+0000","last_fullsized":"2024-04-25T10:04:26.478806+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:56.884017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515556+0000","last_change":"2024-04-25T10:04:25.472528+0000","last_active":"2024-04-25T10:04:26.515556+0000","last_peered":"2024-04-25T10:04:26.515556+0000","last_clean":"2024-04-25T10:04:26.515556+0000","last_became_active":"2024-04-25T10:04:25.472245+0000","last_became_peered":"2024-04-25T10:04:25.472245+0000","last_unstale":"2024-04-25T10:04:26.515556+0000","last_undegraded":"2024-04-25T10:04:26.515556+0000","last_fullsized":"2024-04-25T10:04:26.515556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:24.481673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480435+0000","last_change":"2024-04-25T10:04:24.468526+0000","last_active":"2024-04-25T10:04:26.480435+0000","last_peered":"2024-04-25T10:04:26.480435+0000","last_clean":"2024-04-25T10:04:26.480435+0000","last_became_active":"2024-04-25T10:04:24.468213+0000","last_became_peered":"2024-04-25T10:04:24.468213+0000","last_unstale":"2024-04-25T10:04:26.480435+0000","last_undegraded":"2024-04-25T10:04:26.480435+0000","last_fullsized":"2024-04-25T10:04:26.480435+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:40.655704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517471+0000","last_change":"2024-04-25T10:04:25.463721+0000","last_active":"2024-04-25T10:04:26.517471+0000","last_peered":"2024-04-25T10:04:26.517471+0000","last_clean":"2024-04-25T10:04:26.517471+0000","last_became_active":"2024-04-25T10:04:25.463355+0000","last_became_peered":"2024-04-25T10:04:25.463355+0000","last_unstale":"2024-04-25T10:04:26.517471+0000","last_undegraded":"2024-04-25T10:04:26.517471+0000","last_fullsized":"2024-04-25T10:04:26.517471+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478799+0000","last_change":"2024-04-25T10:04:24.461901+0000","last_active":"2024-04-25T10:04:26.478799+0000","last_peered":"2024-04-25T10:04:26.478799+0000","last_clean":"2024-04-25T10:04:26.478799+0000","last_became_active":"2024-04-25T10:04:24.461619+0000","last_became_peered":"2024-04-25T10:04:24.461619+0000","last_unstale":"2024-04-25T10:04:26.478799+0000","last_undegraded":"2024-04-25T10:04:26.478799+0000","last_fullsized":"2024-04-25T10:04:26.478799+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:32.072513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515887+0000","last_change":"2024-04-25T10:04:25.477724+0000","last_active":"2024-04-25T10:04:26.515887+0000","last_peered":"2024-04-25T10:04:26.515887+0000","last_clean":"2024-04-25T10:04:26.515887+0000","last_became_active":"2024-04-25T10:04:25.476438+0000","last_became_peered":"2024-04-25T10:04:25.476438+0000","last_unstale":"2024-04-25T10:04:26.515887+0000","last_undegraded":"2024-04-25T10:04:26.515887+0000","last_fullsized":"2024-04-25T10:04:26.515887+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517137+0000","last_change":"2024-04-25T10:04:24.465620+0000","last_active":"2024-04-25T10:04:26.517137+0000","last_peered":"2024-04-25T10:04:26.517137+0000","last_clean":"2024-04-25T10:04:26.517137+0000","last_became_active":"2024-04-25T10:04:24.465402+0000","last_became_peered":"2024-04-25T10:04:24.465402+0000","last_unstale":"2024-04-25T10:04:26.517137+0000","last_undegraded":"2024-04-25T10:04:26.517137+0000","last_fullsized":"2024-04-25T10:04:26.517137+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.478114+0000","last_change":"2024-04-25T10:04:25.463940+0000","last_active":"2024-04-25T10:04:26.478114+0000","last_peered":"2024-04-25T10:04:26.478114+0000","last_clean":"2024-04-25T10:04:26.478114+0000","last_became_active":"2024-04-25T10:04:25.463675+0000","last_became_peered":"2024-04-25T10:04:25.463675+0000","last_unstale":"2024-04-25T10:04:26.478114+0000","last_undegraded":"2024-04-25T10:04:26.478114+0000","last_fullsized":"2024-04-25T10:04:26.478114+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:18.573293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516814+0000","last_change":"2024-04-25T10:04:24.466373+0000","last_active":"2024-04-25T10:04:26.516814+0000","last_peered":"2024-04-25T10:04:26.516814+0000","last_clean":"2024-04-25T10:04:26.516814+0000","last_became_active":"2024-04-25T10:04:24.465965+0000","last_became_peered":"2024-04-25T10:04:24.465965+0000","last_unstale":"2024-04-25T10:04:26.516814+0000","last_undegraded":"2024-04-25T10:04:26.516814+0000","last_fullsized":"2024-04-25T10:04:26.516814+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:58.970037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516564+0000","last_change":"2024-04-25T10:04:25.471532+0000","last_active":"2024-04-25T10:04:26.516564+0000","last_peered":"2024-04-25T10:04:26.516564+0000","last_clean":"2024-04-25T10:04:26.516564+0000","last_became_active":"2024-04-25T10:04:25.471276+0000","last_became_peered":"2024-04-25T10:04:25.471276+0000","last_unstale":"2024-04-25T10:04:26.516564+0000","last_undegraded":"2024-04-25T10:04:26.516564+0000","last_fullsized":"2024-04-25T10:04:26.516564+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:39.985342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516898+0000","last_change":"2024-04-25T10:04:24.461754+0000","last_active":"2024-04-25T10:04:26.516898+0000","last_peered":"2024-04-25T10:04:26.516898+0000","last_clean":"2024-04-25T10:04:26.516898+0000","last_became_active":"2024-04-25T10:04:24.459617+0000","last_became_peered":"2024-04-25T10:04:24.459617+0000","last_unstale":"2024-04-25T10:04:26.516898+0000","last_undegraded":"2024-04-25T10:04:26.516898+0000","last_fullsized":"2024-04-25T10:04:26.516898+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.901161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516873+0000","last_change":"2024-04-25T10:04:25.468223+0000","last_active":"2024-04-25T10:04:26.516873+0000","last_peered":"2024-04-25T10:04:26.516873+0000","last_clean":"2024-04-25T10:04:26.516873+0000","last_became_active":"2024-04-25T10:04:25.467369+0000","last_became_peered":"2024-04-25T10:04:25.467369+0000","last_unstale":"2024-04-25T10:04:26.516873+0000","last_undegraded":"2024-04-25T10:04:26.516873+0000","last_fullsized":"2024-04-25T10:04:26.516873+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.829542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479325+0000","last_change":"2024-04-25T10:04:24.471952+0000","last_active":"2024-04-25T10:04:26.479325+0000","last_peered":"2024-04-25T10:04:26.479325+0000","last_clean":"2024-04-25T10:04:26.479325+0000","last_became_active":"2024-04-25T10:04:24.471642+0000","last_became_peered":"2024-04-25T10:04:24.471642+0000","last_unstale":"2024-04-25T10:04:26.479325+0000","last_undegraded":"2024-04-25T10:04:26.479325+0000","last_fullsized":"2024-04-25T10:04:26.479325+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:00.609015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516951+0000","last_change":"2024-04-25T10:04:25.468105+0000","last_active":"2024-04-25T10:04:26.516951+0000","last_peered":"2024-04-25T10:04:26.516951+0000","last_clean":"2024-04-25T10:04:26.516951+0000","last_became_active":"2024-04-25T10:04:25.467841+0000","last_became_peered":"2024-04-25T10:04:25.467841+0000","last_unstale":"2024-04-25T10:04:26.516951+0000","last_undegraded":"2024-04-25T10:04:26.516951+0000","last_fullsized":"2024-04-25T10:04:26.516951+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.829542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515680+0000","last_change":"2024-04-25T10:04:24.469768+0000","last_active":"2024-04-25T10:04:26.515680+0000","last_peered":"2024-04-25T10:04:26.515680+0000","last_clean":"2024-04-25T10:04:26.515680+0000","last_became_active":"2024-04-25T10:04:24.469488+0000","last_became_peered":"2024-04-25T10:04:24.469488+0000","last_unstale":"2024-04-25T10:04:26.515680+0000","last_undegraded":"2024-04-25T10:04:26.515680+0000","last_fullsized":"2024-04-25T10:04:26.515680+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:29.785579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480406+0000","last_change":"2024-04-25T10:04:25.475514+0000","last_active":"2024-04-25T10:04:26.480406+0000","last_peered":"2024-04-25T10:04:26.480406+0000","last_clean":"2024-04-25T10:04:26.480406+0000","last_became_active":"2024-04-25T10:04:25.474364+0000","last_became_peered":"2024-04-25T10:04:25.474364+0000","last_unstale":"2024-04-25T10:04:26.480406+0000","last_undegraded":"2024-04-25T10:04:26.480406+0000","last_fullsized":"2024-04-25T10:04:26.480406+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:04.213695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515760+0000","last_change":"2024-04-25T10:04:24.460278+0000","last_active":"2024-04-25T10:04:26.515760+0000","last_peered":"2024-04-25T10:04:26.515760+0000","last_clean":"2024-04-25T10:04:26.515760+0000","last_became_active":"2024-04-25T10:04:24.459936+0000","last_became_peered":"2024-04-25T10:04:24.459936+0000","last_unstale":"2024-04-25T10:04:26.515760+0000","last_undegraded":"2024-04-25T10:04:26.515760+0000","last_fullsized":"2024-04-25T10:04:26.515760+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:19.332266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.514843+0000","last_change":"2024-04-25T10:04:25.456950+0000","last_active":"2024-04-25T10:04:26.514843+0000","last_peered":"2024-04-25T10:04:26.514843+0000","last_clean":"2024-04-25T10:04:26.514843+0000","last_became_active":"2024-04-25T10:04:25.456664+0000","last_became_peered":"2024-04-25T10:04:25.456664+0000","last_unstale":"2024-04-25T10:04:26.514843+0000","last_undegraded":"2024-04-25T10:04:26.514843+0000","last_fullsized":"2024-04-25T10:04:26.514843+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":16,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473040+0000","last_change":"2024-04-25T10:04:24.466995+0000","last_active":"2024-04-25T10:04:26.473040+0000","last_peered":"2024-04-25T10:04:26.473040+0000","last_clean":"2024-04-25T10:04:26.473040+0000","last_became_active":"2024-04-25T10:04:24.466667+0000","last_became_peered":"2024-04-25T10:04:24.466667+0000","last_unstale":"2024-04-25T10:04:26.473040+0000","last_undegraded":"2024-04-25T10:04:26.473040+0000","last_fullsized":"2024-04-25T10:04:26.473040+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:07.560341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517292+0000","last_change":"2024-04-25T10:04:25.467788+0000","last_active":"2024-04-25T10:04:26.517292+0000","last_peered":"2024-04-25T10:04:26.517292+0000","last_clean":"2024-04-25T10:04:26.517292+0000","last_became_active":"2024-04-25T10:04:25.465574+0000","last_became_peered":"2024-04-25T10:04:25.465574+0000","last_unstale":"2024-04-25T10:04:26.517292+0000","last_undegraded":"2024-04-25T10:04:26.517292+0000","last_fullsized":"2024-04-25T10:04:26.517292+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:50.305504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.473253+0000","last_change":"2024-04-25T10:04:24.463577+0000","last_active":"2024-04-25T10:04:26.473253+0000","last_peered":"2024-04-25T10:04:26.473253+0000","last_clean":"2024-04-25T10:04:26.473253+0000","last_became_active":"2024-04-25T10:04:24.462318+0000","last_became_peered":"2024-04-25T10:04:24.462318+0000","last_unstale":"2024-04-25T10:04:26.473253+0000","last_undegraded":"2024-04-25T10:04:26.473253+0000","last_fullsized":"2024-04-25T10:04:26.473253+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:01.690130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479939+0000","last_change":"2024-04-25T10:04:25.461326+0000","last_active":"2024-04-25T10:04:26.479939+0000","last_peered":"2024-04-25T10:04:26.479939+0000","last_clean":"2024-04-25T10:04:26.479939+0000","last_became_active":"2024-04-25T10:04:25.459940+0000","last_became_peered":"2024-04-25T10:04:25.459940+0000","last_unstale":"2024-04-25T10:04:26.479939+0000","last_undegraded":"2024-04-25T10:04:26.479939+0000","last_fullsized":"2024-04-25T10:04:26.479939+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:48.804816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479691+0000","last_change":"2024-04-25T10:04:24.462828+0000","last_active":"2024-04-25T10:04:26.479691+0000","last_peered":"2024-04-25T10:04:26.479691+0000","last_clean":"2024-04-25T10:04:26.479691+0000","last_became_active":"2024-04-25T10:04:24.462443+0000","last_became_peered":"2024-04-25T10:04:24.462443+0000","last_unstale":"2024-04-25T10:04:26.479691+0000","last_undegraded":"2024-04-25T10:04:26.479691+0000","last_fullsized":"2024-04-25T10:04:26.479691+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:20.157546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516366+0000","last_change":"2024-04-25T10:04:25.466945+0000","last_active":"2024-04-25T10:04:26.516366+0000","last_peered":"2024-04-25T10:04:26.516366+0000","last_clean":"2024-04-25T10:04:26.516366+0000","last_became_active":"2024-04-25T10:04:25.466672+0000","last_became_peered":"2024-04-25T10:04:25.466672+0000","last_unstale":"2024-04-25T10:04:26.516366+0000","last_undegraded":"2024-04-25T10:04:26.516366+0000","last_fullsized":"2024-04-25T10:04:26.516366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:33.048161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.516518+0000","last_change":"2024-04-25T10:04:24.461824+0000","last_active":"2024-04-25T10:04:26.516518+0000","last_peered":"2024-04-25T10:04:26.516518+0000","last_clean":"2024-04-25T10:04:26.516518+0000","last_became_active":"2024-04-25T10:04:24.460800+0000","last_became_peered":"2024-04-25T10:04:24.460800+0000","last_unstale":"2024-04-25T10:04:26.516518+0000","last_undegraded":"2024-04-25T10:04:26.516518+0000","last_fullsized":"2024-04-25T10:04:26.516518+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:25.199121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517513+0000","last_change":"2024-04-25T10:04:25.477935+0000","last_active":"2024-04-25T10:04:26.517513+0000","last_peered":"2024-04-25T10:04:26.517513+0000","last_clean":"2024-04-25T10:04:26.517513+0000","last_became_active":"2024-04-25T10:04:25.477605+0000","last_became_peered":"2024-04-25T10:04:25.477605+0000","last_unstale":"2024-04-25T10:04:26.517513+0000","last_undegraded":"2024-04-25T10:04:26.517513+0000","last_fullsized":"2024-04-25T10:04:26.517513+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:29.829542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.479437+0000","last_change":"2024-04-25T10:04:24.471290+0000","last_active":"2024-04-25T10:04:26.479437+0000","last_peered":"2024-04-25T10:04:26.479437+0000","last_clean":"2024-04-25T10:04:26.479437+0000","last_became_active":"2024-04-25T10:04:24.470984+0000","last_became_peered":"2024-04-25T10:04:24.470984+0000","last_unstale":"2024-04-25T10:04:26.479437+0000","last_undegraded":"2024-04-25T10:04:26.479437+0000","last_fullsized":"2024-04-25T10:04:26.479437+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:50.982046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517555+0000","last_change":"2024-04-25T10:04:25.462505+0000","last_active":"2024-04-25T10:04:26.517555+0000","last_peered":"2024-04-25T10:04:26.517555+0000","last_clean":"2024-04-25T10:04:26.517555+0000","last_became_active":"2024-04-25T10:04:25.461056+0000","last_became_peered":"2024-04-25T10:04:25.461056+0000","last_unstale":"2024-04-25T10:04:26.517555+0000","last_undegraded":"2024-04-25T10:04:26.517555+0000","last_fullsized":"2024-04-25T10:04:26.517555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:41.641994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515749+0000","last_change":"2024-04-25T10:04:24.443846+0000","last_active":"2024-04-25T10:04:26.515749+0000","last_peered":"2024-04-25T10:04:26.515749+0000","last_clean":"2024-04-25T10:04:26.515749+0000","last_became_active":"2024-04-25T10:04:24.443511+0000","last_became_peered":"2024-04-25T10:04:24.443511+0000","last_unstale":"2024-04-25T10:04:26.515749+0000","last_undegraded":"2024-04-25T10:04:26.515749+0000","last_fullsized":"2024-04-25T10:04:26.515749+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.480306+0000","last_change":"2024-04-25T10:04:25.466904+0000","last_active":"2024-04-25T10:04:26.480306+0000","last_peered":"2024-04-25T10:04:26.480306+0000","last_clean":"2024-04-25T10:04:26.480306+0000","last_became_active":"2024-04-25T10:04:25.466434+0000","last_became_peered":"2024-04-25T10:04:25.466434+0000","last_unstale":"2024-04-25T10:04:26.480306+0000","last_undegraded":"2024-04-25T10:04:26.480306+0000","last_fullsized":"2024-04-25T10:04:26.480306+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.944891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.514716+0000","last_change":"2024-04-25T10:04:24.467188+0000","last_active":"2024-04-25T10:04:26.514716+0000","last_peered":"2024-04-25T10:04:26.514716+0000","last_clean":"2024-04-25T10:04:26.514716+0000","last_became_active":"2024-04-25T10:04:24.466850+0000","last_became_peered":"2024-04-25T10:04:24.466850+0000","last_unstale":"2024-04-25T10:04:26.514716+0000","last_undegraded":"2024-04-25T10:04:26.514716+0000","last_fullsized":"2024-04-25T10:04:26.514716+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:06.524001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.519031+0000","last_change":"2024-04-25T10:04:25.461224+0000","last_active":"2024-04-25T10:04:26.519031+0000","last_peered":"2024-04-25T10:04:26.519031+0000","last_clean":"2024-04-25T10:04:26.519031+0000","last_became_active":"2024-04-25T10:04:25.460727+0000","last_became_peered":"2024-04-25T10:04:25.460727+0000","last_unstale":"2024-04-25T10:04:26.519031+0000","last_undegraded":"2024-04-25T10:04:26.519031+0000","last_fullsized":"2024-04-25T10:04:26.519031+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:24.411381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:24.411381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:36.579214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"51'1","reported_seq":15,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.517831+0000","last_change":"2024-04-25T10:04:24.461091+0000","last_active":"2024-04-25T10:04:26.517831+0000","last_peered":"2024-04-25T10:04:26.517831+0000","last_clean":"2024-04-25T10:04:26.517831+0000","last_became_active":"2024-04-25T10:04:24.460322+0000","last_became_peered":"2024-04-25T10:04:24.460322+0000","last_unstale":"2024-04-25T10:04:26.517831+0000","last_undegraded":"2024-04-25T10:04:26.517831+0000","last_fullsized":"2024-04-25T10:04:26.517831+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:23.398381+0000","last_clean_scrub_stamp":"2024-04-25T10:04:23.398381+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:17.560293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":225,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:04:26.515021+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:04:26.515021+0000","last_peered":"2024-04-25T10:04:26.515021+0000","last_clean":"2024-04-25T10:04:26.515021+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:04:26.515021+0000","last_undegraded":"2024-04-25T10:04:26.515021+0000","last_fullsized":"2024-04-25T10:04:26.515021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":606208,"data_stored":262070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":602112,"data_stored":262596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1208320,"data_stored":854084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":602112,"data_stored":262104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":602112,"data_stored":262048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1196032,"data_stored":852452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509476,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":602112,"data_stored":262596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":618496,"data_stored":263718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:35.499 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:04:35.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:04:35.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.758+0000 7f7c31200640 1 Processor -- start 2024-04-25T10:04:35.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.758+0000 7f7c31200640 1 -- start start 2024-04-25T10:04:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.760+0000 7f7c31200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c2c0ae860 0x7f7c2c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.760+0000 7f7c31200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c06f080 con 0x7f7c2c0aece0 2024-04-25T10:04:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.760+0000 7f7c31200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c06f360 con 0x7f7c2c1465d0 2024-04-25T10:04:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.760+0000 7f7c31200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2c0ae060 con 0x7f7c2c0ae860 2024-04-25T10:04:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c2a800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c2c0aece0 0x7f7c2c15f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60588/0 (socket says 172.21.15.5:60588) 2024-04-25T10:04:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c2a800640 1 -- 172.21.15.5:0/2412453527 learned_addr learned my addr 172.21.15.5:0/2412453527 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c23e00640 1 --2- 172.21.15.5:0/2412453527 >> v2:172.21.15.80:3300/0 conn(0x7f7c2c0ae860 0x7f7c2c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356072904 0 0) 0x7f7c2c06f080 con 0x7f7c2c0aece0 2024-04-25T10:04:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c10003520 con 0x7f7c2c0aece0 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3167411973 0 0) 0x7f7c10003520 con 0x7f7c2c0aece0 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 >> v2:172.21.15.80:3300/0 conn(0x7f7c2c0ae860 msgr2=0x7f7c2c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.761+0000 7f7c29400640 1 --2- 172.21.15.5:0/2412453527 >> v2:172.21.15.80:3300/0 conn(0x7f7c2c0ae860 0x7f7c2c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 >> v1:172.21.15.80:6789/0 conn(0x7f7c2c1465d0 legacy=0x7f7c2c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0ae340 con 0x7f7c2c0aece0 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f7c240019f0 con 0x7f7c2c0aece0 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 >> v1:172.21.15.5:6789/0 conn(0x7f7c2c0aece0 legacy=0x7f7c2c15f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c10003a00 0x7f7c10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c10004800 0x7f7c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c100051c0 0x7f7c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0ae060 con 0x7f7c100051c0 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000c0c0 con 0x7f7c10004800 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000c2c0 con 0x7f7c10003a00 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c2a800640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c100051c0 0x7f7c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c29e00640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c10003a00 0x7f7c10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c2a800640 1 -- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c10003a00 msgr2=0x7f7c10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c2a800640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c10003a00 0x7f7c10003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c2a800640 1 -- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c10004800 msgr2=0x7f7c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:35.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c2a800640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c10004800 0x7f7c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.762+0000 7f7c2a800640 1 -- 172.21.15.5:0/2412453527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10003520 con 0x7f7c100051c0 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.763+0000 7f7c2a800640 1 -- 172.21.15.5:0/2412453527 reap_dead start 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.764+0000 7f7c2a800640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c100051c0 0x7f7c1000b9c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400a770 tx=0x7f7c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51696d1e7b51806c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.764+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400a390 con 0x7f7c100051c0 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.764+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2402f070 con 0x7f7c100051c0 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.764+0000 7f7c29400640 1 -- 172.21.15.5:0/2412453527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2402bbe0 con 0x7f7c100051c0 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.764+0000 7f7c31200640 1 -- 172.21.15.5:0/2412453527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c100051c0 msgr2=0x7f7c1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.764+0000 7f7c31200640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c100051c0 0x7f7c1000b9c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400a770 tx=0x7f7c10005830 comp rx=0 tx=0).stop 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.765+0000 7f7c31200640 1 -- 172.21.15.5:0/2412453527 shutdown_connections 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.765+0000 7f7c31200640 1 --2- 172.21.15.5:0/2412453527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c100051c0 0x7f7c1000b9c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.765+0000 7f7c31200640 1 -- 172.21.15.5:0/2412453527 >> 172.21.15.5:0/2412453527 conn(0x7f7c2c06ab60 msgr2=0x7f7c2c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.765+0000 7f7c31200640 1 -- 172.21.15.5:0/2412453527 shutdown_connections 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.765+0000 7f7c31200640 1 -- 172.21.15.5:0/2412453527 wait complete. 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.766+0000 7f7c31200640 1 Processor -- start 2024-04-25T10:04:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.767+0000 7f7c31200640 1 -- start start 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.767+0000 7f7c31200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c100051c0 0x7f7c2c13fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c31200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 0x7f7c2c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c31200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c0b1640 0x7f7c2c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c31200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c06ece0 con 0x7f7c2c15fa00 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c31200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c06e7d0 con 0x7f7c2c0b1640 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c31200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c06ee60 con 0x7f7c100051c0 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 0x7f7c2c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 0x7f7c2c15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57812/0 (socket says 172.21.15.5:57812) 2024-04-25T10:04:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 -- 172.21.15.5:0/606800120 learned_addr learned my addr 172.21.15.5:0/606800120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 -- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c100051c0 msgr2=0x7f7c2c13fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c29e00640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c100051c0 0x7f7c2c13fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c2a800640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c0b1640 0x7f7c2c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c100051c0 0x7f7c2c13fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 -- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c0b1640 msgr2=0x7f7c2c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c0b1640 0x7f7c2c0b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.768+0000 7f7c23e00640 1 -- 172.21.15.5:0/606800120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0cce10 con 0x7f7c2c15fa00 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.769+0000 7f7c23e00640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 0x7f7c2c15fe60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400e7d0 tx=0x7f7c1400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.769+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c14017070 con 0x7f7c2c15fa00 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.769+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1400c940 con 0x7f7c2c15fa00 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.769+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c1403d0 con 0x7f7c2c15fa00 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.769+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400cd10 con 0x7f7c2c15fa00 2024-04-25T10:04:35.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.769+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c0cd390 con 0x7f7c2c15fa00 2024-04-25T10:04:35.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.770+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c1603a0 con 0x7f7c2c15fa00 2024-04-25T10:04:35.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.772+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c1401a070 con 0x7f7c2c15fa00 2024-04-25T10:04:35.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.773+0000 7f7c22000640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c0806b4d0 0x7f7c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.773+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c1408a2b0 con 0x7f7c2c15fa00 2024-04-25T10:04:35.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.774+0000 7f7c29e00640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c0806b4d0 0x7f7c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:35.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.775+0000 7f7c29e00640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c0806b4d0 0x7f7c0806d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001590 tx=0x7f7c1c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:35.778+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c14058f00 con 0x7f7c2c15fa00 2024-04-25T10:04:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.049+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7c2c0b2220 con 0x7f7c2c15fa00 2024-04-25T10:04:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.055+0000 7f7c22000640 1 -- 172.21.15.5:0/606800120 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7c14058800 con 0x7f7c2c15fa00 2024-04-25T10:04:36.060 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:04:36.060 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:04:36.060 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:04:36.060 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:04:36.060 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:04:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.057+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c0806b4d0 msgr2=0x7f7c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.057+0000 7f7c31200640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c0806b4d0 0x7f7c0806d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001590 tx=0x7f7c1c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.057+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 msgr2=0x7f7c2c15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.057+0000 7f7c31200640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 0x7f7c2c15fe60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400e7d0 tx=0x7f7c1400ec90 comp rx=0 tx=0).stop 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 shutdown_connections 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c0806b4d0 0x7f7c0806d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c0b1640 0x7f7c2c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c2c15fa00 0x7f7c2c15fe60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 --2- 172.21.15.5:0/606800120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c100051c0 0x7f7c2c13fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 >> 172.21.15.5:0/606800120 conn(0x7f7c2c06ab60 msgr2=0x7f7c2c0b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 shutdown_connections 2024-04-25T10:04:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.058+0000 7f7c31200640 1 -- 172.21.15.5:0/606800120 wait complete. 2024-04-25T10:04:36.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.281+0000 7f1d6e200640 1 Processor -- start 2024-04-25T10:04:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.282+0000 7f1d6e200640 1 -- start start 2024-04-25T10:04:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.283+0000 7f1d6e200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d680716a0 0x7f1d6815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.283+0000 7f1d6e200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680cd200 con 0x7f1d68154a10 2024-04-25T10:04:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.283+0000 7f1d6e200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6806e6f0 con 0x7f1d680cdca0 2024-04-25T10:04:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.283+0000 7f1d6e200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6806e9d0 con 0x7f1d680716a0 2024-04-25T10:04:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d6ce00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1d68154a10 0x7f1d680b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60604/0 (socket says 172.21.15.5:60604) 2024-04-25T10:04:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d6ce00640 1 -- 172.21.15.5:0/1564506934 learned_addr learned my addr 172.21.15.5:0/1564506934 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d6d800640 1 --2- 172.21.15.5:0/1564506934 >> v2:172.21.15.5:3300/0 conn(0x7f1d680716a0 0x7f1d6815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d67e00640 1 -- 172.21.15.5:0/1564506934 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932490706 0 0) 0x7f1d680cd200 con 0x7f1d68154a10 2024-04-25T10:04:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d67e00640 1 -- 172.21.15.5:0/1564506934 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d4c003520 con 0x7f1d68154a10 2024-04-25T10:04:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d6d800640 1 -- 172.21.15.5:0/1564506934 >> v1:172.21.15.80:6789/0 conn(0x7f1d680cdca0 legacy=0x7f1d68070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d6d800640 1 -- 172.21.15.5:0/1564506934 >> v1:172.21.15.5:6789/0 conn(0x7f1d68154a10 legacy=0x7f1d680b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.284+0000 7f1d6d800640 1 -- 172.21.15.5:0/1564506934 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6806ecb0 con 0x7f1d680716a0 2024-04-25T10:04:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.286+0000 7f1d6d800640 1 --2- 172.21.15.5:0/1564506934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d6815e8c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009d90 tx=0x7f1d6002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d7c75e5967c79c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.286+0000 7f1d67e00640 1 -- 172.21.15.5:0/1564506934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60007740 con 0x7f1d680716a0 2024-04-25T10:04:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.286+0000 7f1d67e00640 1 -- 172.21.15.5:0/1564506934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d600078e0 con 0x7f1d680716a0 2024-04-25T10:04:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.286+0000 7f1d67e00640 1 -- 172.21.15.5:0/1564506934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60007cb0 con 0x7f1d680716a0 2024-04-25T10:04:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.286+0000 7f1d6e200640 1 -- 172.21.15.5:0/1564506934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 msgr2=0x7f1d6815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.286+0000 7f1d6e200640 1 --2- 172.21.15.5:0/1564506934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d6815e8c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009d90 tx=0x7f1d6002a660 comp rx=0 tx=0).stop 2024-04-25T10:04:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.287+0000 7f1d6e200640 1 -- 172.21.15.5:0/1564506934 shutdown_connections 2024-04-25T10:04:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.287+0000 7f1d6e200640 1 --2- 172.21.15.5:0/1564506934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d6815e8c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.287+0000 7f1d6e200640 1 -- 172.21.15.5:0/1564506934 >> 172.21.15.5:0/1564506934 conn(0x7f1d68066f20 msgr2=0x7f1d6806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.287+0000 7f1d6e200640 1 -- 172.21.15.5:0/1564506934 shutdown_connections 2024-04-25T10:04:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.287+0000 7f1d6e200640 1 -- 172.21.15.5:0/1564506934 wait complete. 2024-04-25T10:04:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.288+0000 7f1d6e200640 1 Processor -- start 2024-04-25T10:04:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.288+0000 7f1d6e200640 1 -- start start 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.289+0000 7f1d6e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d68158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.289+0000 7f1d6e200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d680cdca0 0x7f1d68150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.289+0000 7f1d6e200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d68154a10 0x7f1d681597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.289+0000 7f1d6e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6806cff0 con 0x7f1d680716a0 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.289+0000 7f1d6e200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d680cd780 con 0x7f1d68154a10 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.289+0000 7f1d6e200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6806d170 con 0x7f1d680cdca0 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d68158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d68158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57824/0 (socket says 172.21.15.5:57824) 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 -- 172.21.15.5:0/2498397452 learned_addr learned my addr 172.21.15.5:0/2498397452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 -- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d680cdca0 msgr2=0x7f1d68150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d680cdca0 0x7f1d68150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 -- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d68154a10 msgr2=0x7f1d681597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d68154a10 0x7f1d681597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 -- 172.21.15.5:0/2498397452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68158200 con 0x7f1d680716a0 2024-04-25T10:04:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.290+0000 7f1d6ce00640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d68158e80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c400 tx=0x7f1d5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.291+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58017070 con 0x7f1d680716a0 2024-04-25T10:04:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.291+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58005ce0 con 0x7f1d680716a0 2024-04-25T10:04:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.291+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580072d0 con 0x7f1d680716a0 2024-04-25T10:04:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.291+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68159510 con 0x7f1d680716a0 2024-04-25T10:04:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.291+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d680ad3b0 con 0x7f1d680716a0 2024-04-25T10:04:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.292+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d68068f60 con 0x7f1d680716a0 2024-04-25T10:04:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.293+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d5801a070 con 0x7f1d680716a0 2024-04-25T10:04:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.294+0000 7f1d65600640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d4406b4d0 0x7f1d4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.294+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d58089730 con 0x7f1d680716a0 2024-04-25T10:04:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.295+0000 7f1d5fe00640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d4406b4d0 0x7f1d4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.296+0000 7f1d5fe00640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d4406b4d0 0x7f1d4406d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000f60 tx=0x7f1d500079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.298+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d58057d30 con 0x7f1d680716a0 2024-04-25T10:04:36.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.537+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d680b3ba0 con 0x7f1d680716a0 2024-04-25T10:04:36.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.538+0000 7f1d65600640 1 -- 172.21.15.5:0/2498397452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f1d5805bdd0 con 0x7f1d680716a0 2024-04-25T10:04:36.543 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:36.543 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:04:36.544 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:04:36.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d4406b4d0 msgr2=0x7f1d4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d4406b4d0 0x7f1d4406d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000f60 tx=0x7f1d500079b0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 msgr2=0x7f1d68158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d68158e80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c400 tx=0x7f1d5800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 shutdown_connections 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d4406b4d0 0x7f1d4406d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d68154a10 0x7f1d681597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d680cdca0 0x7f1d68150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 --2- 172.21.15.5:0/2498397452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d680716a0 0x7f1d68158e80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 >> 172.21.15.5:0/2498397452 conn(0x7f1d68066f20 msgr2=0x7f1d68153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 shutdown_connections 2024-04-25T10:04:36.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:36.541+0000 7f1d6e200640 1 -- 172.21.15.5:0/2498397452 wait complete. 2024-04-25T10:04:36.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:04:24.928361+0000', 'modified': '2024-04-25T10:04:27.993527+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5365}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5365': {'gid': 5365, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6836/2115515655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 2115515655}, {'type': 'v1', 'addr': '172.21.15.80:6836', 'nonce': 2115515655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:04:36.967 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:04:36.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:04:37.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.183+0000 7f89e3200640 1 Processor -- start 2024-04-25T10:04:37.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.184+0000 7f89e3200640 1 -- start start 2024-04-25T10:04:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.185+0000 7f89e3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89e40cdca0 0x7f89e415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.185+0000 7f89e3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e40cd200 con 0x7f89e4154a10 2024-04-25T10:04:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.185+0000 7f89e3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e406e6f0 con 0x7f89e4070f90 2024-04-25T10:04:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.185+0000 7f89e3200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e406e9d0 con 0x7f89e40cdca0 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.186+0000 7f89dbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89e40cdca0 0x7f89e415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.186+0000 7f89e1e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f89e4154a10 0x7f89e40b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56000/0 (socket says 172.21.15.5:56000) 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.186+0000 7f89e1e00640 1 -- 172.21.15.5:0/1550929646 learned_addr learned my addr 172.21.15.5:0/1550929646 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.186+0000 7f89dbe00640 1 -- 172.21.15.5:0/1550929646 >> v1:172.21.15.80:6789/0 conn(0x7f89e4070f90 legacy=0x7f89e406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.186+0000 7f89dbe00640 1 -- 172.21.15.5:0/1550929646 >> v1:172.21.15.5:6789/0 conn(0x7f89e4154a10 legacy=0x7f89e40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.186+0000 7f89dbe00640 1 -- 172.21.15.5:0/1550929646 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e406ecb0 con 0x7f89e40cdca0 2024-04-25T10:04:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.187+0000 7f89dbe00640 1 --2- 172.21.15.5:0/1550929646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e415e8c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0097a0 tx=0x7f89cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fbe86e18c329a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.188+0000 7f89e1400640 1 -- 172.21.15.5:0/1550929646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc037070 con 0x7f89e40cdca0 2024-04-25T10:04:37.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.188+0000 7f89e1400640 1 -- 172.21.15.5:0/1550929646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc02ae50 con 0x7f89e40cdca0 2024-04-25T10:04:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.188+0000 7f89e1400640 1 -- 172.21.15.5:0/1550929646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc032660 con 0x7f89e40cdca0 2024-04-25T10:04:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.188+0000 7f89e3200640 1 -- 172.21.15.5:0/1550929646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 msgr2=0x7f89e415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.188+0000 7f89e3200640 1 --2- 172.21.15.5:0/1550929646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e415e8c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0097a0 tx=0x7f89cc02a140 comp rx=0 tx=0).stop 2024-04-25T10:04:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.189+0000 7f89e3200640 1 -- 172.21.15.5:0/1550929646 shutdown_connections 2024-04-25T10:04:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.189+0000 7f89e3200640 1 --2- 172.21.15.5:0/1550929646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e415e8c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.189+0000 7f89e3200640 1 -- 172.21.15.5:0/1550929646 >> 172.21.15.5:0/1550929646 conn(0x7f89e4066f20 msgr2=0x7f89e406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.189+0000 7f89e3200640 1 -- 172.21.15.5:0/1550929646 shutdown_connections 2024-04-25T10:04:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.189+0000 7f89e3200640 1 -- 172.21.15.5:0/1550929646 wait complete. 2024-04-25T10:04:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.190+0000 7f89e3200640 1 Processor -- start 2024-04-25T10:04:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.190+0000 7f89e3200640 1 -- start start 2024-04-25T10:04:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.191+0000 7f89e3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4070f90 0x7f89e415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.191+0000 7f89e3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e4142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.191+0000 7f89e3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4154a10 0x7f89e4154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.191+0000 7f89e3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e406d700 con 0x7f89e40cdca0 2024-04-25T10:04:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.191+0000 7f89e3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40cd780 con 0x7f89e4154a10 2024-04-25T10:04:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.191+0000 7f89e3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e406d880 con 0x7f89e4070f90 2024-04-25T10:04:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e4142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e4142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43046/0 (socket says 172.21.15.5:43046) 2024-04-25T10:04:37.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 -- 172.21.15.5:0/3341485167 learned_addr learned my addr 172.21.15.5:0/3341485167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89e2800640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4154a10 0x7f89e4154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89e1e00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4070f90 0x7f89e415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 -- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4070f90 msgr2=0x7f89e415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4070f90 0x7f89e415e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 -- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4154a10 msgr2=0x7f89e4154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4154a10 0x7f89e4154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89dbe00640 1 -- 172.21.15.5:0/3341485167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e41504a0 con 0x7f89e40cdca0 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89e2800640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4154a10 0x7f89e4154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.192+0000 7f89e1e00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4070f90 0x7f89e415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.193+0000 7f89dbe00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e4142d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f89cc000c00 tx=0x7f89cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.193+0000 7f89da000640 1 -- 172.21.15.5:0/3341485167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc042070 con 0x7f89e40cdca0 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.193+0000 7f89da000640 1 -- 172.21.15.5:0/3341485167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc03e670 con 0x7f89e40cdca0 2024-04-25T10:04:37.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.193+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e4156750 con 0x7f89e40cdca0 2024-04-25T10:04:37.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.193+0000 7f89da000640 1 -- 172.21.15.5:0/3341485167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc037040 con 0x7f89e40cdca0 2024-04-25T10:04:37.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.193+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e41f4a10 con 0x7f89e40cdca0 2024-04-25T10:04:37.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.195+0000 7f89da000640 1 -- 172.21.15.5:0/3341485167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89cc045070 con 0x7f89e40cdca0 2024-04-25T10:04:37.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.196+0000 7f89da000640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c006b4d0 0x7f89c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.196+0000 7f89da000640 1 -- 172.21.15.5:0/3341485167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89cc0aad90 con 0x7f89e40cdca0 2024-04-25T10:04:37.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.197+0000 7f89ebfaf740 1 Processor -- start 2024-04-25T10:04:37.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.197+0000 7f89e1e00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c006b4d0 0x7f89c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.197+0000 7f89ebfaf740 1 -- start start 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12bfab30 0x559a12c22970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c22eb0 0x559a12bfaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12bf92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a12b5e620 con 0x559a12c437a0 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a12b07ca0 con 0x559a12c22eb0 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a12b5a6c0 con 0x559a12bfab30 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89e1e00640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c006b4d0 0x7f89c006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89d4000f60 tx=0x7f89d4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12bf92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12bf92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43058/0 (socket says 172.21.15.5:43058) 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.198+0000 7f89e2800640 1 -- 172.21.15.5:0/689471308 learned_addr learned my addr 172.21.15.5:0/689471308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e1e00640 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12bfab30 0x559a12c22970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89dbe00640 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c22eb0 0x559a12bfaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e2800640 1 -- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12bfab30 msgr2=0x559a12c22970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e2800640 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12bfab30 0x559a12c22970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e2800640 1 -- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c22eb0 msgr2=0x559a12bfaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e2800640 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c22eb0 0x559a12bfaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e2800640 1 -- 172.21.15.5:0/689471308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a12b0ef40 con 0x559a12c437a0 2024-04-25T10:04:37.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89e2800640 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12bf92f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f89dc001e40 tx=0x7f89dc00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600c9ed6b8ef5845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89d2a00640 1 -- 172.21.15.5:0/689471308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc018070 con 0x559a12c437a0 2024-04-25T10:04:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89d2a00640 1 -- 172.21.15.5:0/689471308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00de50 con 0x559a12c437a0 2024-04-25T10:04:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.199+0000 7f89d2a00640 1 -- 172.21.15.5:0/689471308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc003bb0 con 0x559a12c437a0 2024-04-25T10:04:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 -- 172.21.15.5:0/689471308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 msgr2=0x559a12bf92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12bf92f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f89dc001e40 tx=0x7f89dc00d520 comp rx=0 tx=0).stop 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 -- 172.21.15.5:0/689471308 shutdown_connections 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12bf92f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c22eb0 0x559a12bfaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/689471308 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12bfab30 0x559a12c22970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 -- 172.21.15.5:0/689471308 >> 172.21.15.5:0/689471308 conn(0x559a12bf7d50 msgr2=0x559a12c40730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 -- 172.21.15.5:0/689471308 shutdown_connections 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.200+0000 7f89ebfaf740 1 -- 172.21.15.5:0/689471308 wait complete. 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.201+0000 7f89ebfaf740 1 -- start start 2024-04-25T10:04:37.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.201+0000 7f89ebfaf740 1 Processor -- start 2024-04-25T10:04:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 0x559a12c53940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c437a0 0x559a12c548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a12c224e0 con 0x559a12c22eb0 2024-04-25T10:04:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a12afd1f0 con 0x559a12c20c70 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a12c21ed0 con 0x559a12c437a0 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.202+0000 7f89ebfaf740 10 client.0 _ll_register_callbacks cb 0x559a12b1aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 0x559a12c53940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 0x559a12c53940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43064/0 (socket says 172.21.15.5:43064) 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 -- 172.21.15.5:0/626934357 learned_addr learned my addr 172.21.15.5:0/626934357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e2800640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 -- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c437a0 msgr2=0x559a12c548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89dbe00640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c437a0 0x559a12c548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c437a0 0x559a12c548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 -- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 msgr2=0x559a12c210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 -- 172.21.15.5:0/626934357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a12b1cc50 con 0x559a12c22eb0 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e2800640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89e1e00640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 0x559a12c53940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f89d4007680 tx=0x7f89d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff607973b00d4f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.203+0000 7f89c7400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89c7400640 1 -- 172.21.15.5:0/626934357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4026060 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89c7400640 1 -- 172.21.15.5:0/626934357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89d4021c20 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89dc00de50 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89c7400640 1 -- 172.21.15.5:0/626934357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d4005e40 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 10 client.5916 fetch_fsmap learned FSMap version 26 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x559a128732d0 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89c7400640 1 -- 172.21.15.5:0/626934357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00de50 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89c7400640 1 -- 172.21.15.5:0/626934357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89d4012410 con 0x559a12c22eb0 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 10 client.5916 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 10 client.5916 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 20 client.5916 populate_metadata read hostname 'smithi005' 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12c50aa0 0x559a12c50ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 4 client.5916 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.204+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a12b6bb60 con 0x559a12c50aa0 2024-04-25T10:04:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.205+0000 7f89dbe00640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12c50aa0 0x559a12c50ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.206+0000 7f89dbe00640 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12c50aa0 0x559a12c50ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7ce81ec4e6ce2a server_cookie=d70e327c6df3ba92 in_seq=0 out_seq=0 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.206+0000 7f89c7400640 10 client.5916 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89c7400640 1 -- 172.21.15.5:0/626934357 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a12b6bb60 con 0x559a12c50aa0 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89c7400640 10 client.5916 handle_command_reply: tid=0 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 client.5916 shutdown 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 msgr2=0x559a12c53940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 0x559a12c53940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f89d4007680 tx=0x7f89d4000bc0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 shutdown_connections 2024-04-25T10:04:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12c50aa0 0x559a12c50ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89c7400640 0 client.5916 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c437a0 0x559a12c548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/626934357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c22eb0 0x559a12c53940 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 >> 172.21.15.5:0/626934357 conn(0x559a12bfab30 msgr2=0x559a12c40530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 shutdown_connections 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.208+0000 7f89ebfaf740 1 -- 172.21.15.5:0/626934357 wait complete. 2024-04-25T10:04:37.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.209+0000 7f89ebfaf740 20 client.5916 trim_cache size 0 max 16384 2024-04-25T10:04:37.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.226+0000 7f89ebfaf740 1 Processor -- start 2024-04-25T10:04:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.226+0000 7f89ebfaf740 1 -- start start 2024-04-25T10:04:37.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12ca2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41ff560 con 0x559a12c437a0 2024-04-25T10:04:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4157c10 con 0x559a12c20c70 2024-04-25T10:04:37.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41513f0 con 0x559a12c22eb0 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43070/0 (socket says 172.21.15.5:43070) 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.227+0000 7f89e2800640 1 -- 172.21.15.5:0/2754124273 learned_addr learned my addr 172.21.15.5:0/2754124273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e1e00640 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12ca2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89dbe00640 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e2800640 1 -- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 msgr2=0x559a12c50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e2800640 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e2800640 1 -- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 msgr2=0x559a12ca2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e2800640 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12ca2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e2800640 1 -- 172.21.15.5:0/2754124273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a1293d6b0 con 0x559a12c437a0 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e1e00640 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12ca2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89e2800640 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x559a12bf70c0 tx=0x7f89dc00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133ecb0a320d2448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.228+0000 7f89d2000640 1 -- 172.21.15.5:0/2754124273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc024070 con 0x559a12c437a0 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89d2000640 1 -- 172.21.15.5:0/2754124273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00de50 con 0x559a12c437a0 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89d2000640 1 -- 172.21.15.5:0/2754124273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc018070 con 0x559a12c437a0 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 -- 172.21.15.5:0/2754124273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 msgr2=0x559a12c513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x559a12bf70c0 tx=0x7f89dc00b000 comp rx=0 tx=0).stop 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 -- 172.21.15.5:0/2754124273 shutdown_connections 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/2754124273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12ca2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 -- 172.21.15.5:0/2754124273 >> 172.21.15.5:0/2754124273 conn(0x559a12bfab30 msgr2=0x559a12c633b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 -- 172.21.15.5:0/2754124273 shutdown_connections 2024-04-25T10:04:37.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.229+0000 7f89ebfaf740 1 -- 172.21.15.5:0/2754124273 wait complete. 2024-04-25T10:04:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.230+0000 7f89ebfaf740 1 -- start start 2024-04-25T10:04:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.230+0000 7f89ebfaf740 1 Processor -- start 2024-04-25T10:04:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c9f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a12c52eb0 con 0x559a12c437a0 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a12c24630 con 0x559a12c20c70 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559a12c52330 con 0x559a12c22eb0 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.231+0000 7f89ebfaf740 10 client.0 _ll_register_callbacks cb 0x559a12c22e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43084/0 (socket says 172.21.15.5:43084) 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 -- 172.21.15.5:0/807645833 learned_addr learned my addr 172.21.15.5:0/807645833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e1e00640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c9f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89dbe00640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 -- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 msgr2=0x559a12c50e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 -- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 msgr2=0x559a12c9f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c9f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 -- 172.21.15.5:0/807645833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a12c43f60 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89e2800640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f89dc014000 tx=0x7f89dc001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5f32db7e8d740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.232+0000 7f89c7400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89c7400640 1 -- 172.21.15.5:0/807645833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc018070 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89c7400640 1 -- 172.21.15.5:0/807645833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89dc006d10 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89dc00de50 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89c7400640 1 -- 172.21.15.5:0/807645833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89dc033050 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 10 client.5922 fetch_fsmap learned FSMap version 26 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x559a12c42640 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89c7400640 1 -- 172.21.15.5:0/807645833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00de50 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89c7400640 1 -- 172.21.15.5:0/807645833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89dc036020 con 0x559a12c437a0 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 10 client.5922 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 10 client.5922 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 20 client.5922 populate_metadata read hostname 'smithi005' 2024-04-25T10:04:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12bf9650 0x559a12bf9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 4 client.5922 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:04:37.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.233+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a12b65eb0 con 0x559a12bf9650 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.234+0000 7f89e1e00640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12bf9650 0x559a12bf9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.235+0000 7f89e1e00640 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12bf9650 0x559a12bf9a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3135e9e0d04473d1 server_cookie=62099058bba2afe3 in_seq=0 out_seq=0 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.235+0000 7f89c7400640 10 client.5922 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89c7400640 1 -- 172.21.15.5:0/807645833 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559a12b65eb0 con 0x559a12bf9650 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89c7400640 10 client.5922 handle_command_reply: tid=0 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 client.5922 shutdown 2024-04-25T10:04:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 msgr2=0x559a12c513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f89dc014000 tx=0x7f89dc001330 comp rx=0 tx=0).stop 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 shutdown_connections 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x559a12bf9650 0x559a12bf9a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a12c437a0 0x559a12c513c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559a12c22eb0 0x559a12c50e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89c7400640 0 client.5922 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 --2- 172.21.15.5:0/807645833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559a12c20c70 0x559a12c9f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 >> 172.21.15.5:0/807645833 conn(0x559a12bfab30 msgr2=0x559a12bf0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 shutdown_connections 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 1 -- 172.21.15.5:0/807645833 wait complete. 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.236+0000 7f89ebfaf740 20 client.5922 trim_cache size 0 max 16384 2024-04-25T10:04:37.243 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5365, 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 11.801351487, 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 51, 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 51, 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 17.174152634999999 2024-04-25T10:04:37.244 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:04:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c006b4d0 msgr2=0x7f89c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c006b4d0 0x7f89c006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89d4000f60 tx=0x7f89d4009040 comp rx=0 tx=0).stop 2024-04-25T10:04:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 msgr2=0x7f89e4142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e4142d20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f89cc000c00 tx=0x7f89cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 shutdown_connections 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c006b4d0 0x7f89c006d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4154a10 0x7f89e4154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e40cdca0 0x7f89e4142d20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 --2- 172.21.15.5:0/3341485167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4070f90 0x7f89e415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 >> 172.21.15.5:0/3341485167 conn(0x7f89e4066f20 msgr2=0x7f89e40b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 shutdown_connections 2024-04-25T10:04:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.237+0000 7f89e3200640 1 -- 172.21.15.5:0/3341485167 wait complete. 2024-04-25T10:04:37.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:04:37.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.459+0000 7f9c3be00640 1 Processor -- start 2024-04-25T10:04:37.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.460+0000 7f9c3be00640 1 -- start start 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.461+0000 7f9c3be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9c3c161200 0x7f9c3c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.461+0000 7f9c3be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c3c0d2380 0x7f9c3c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.461+0000 7f9c3be00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0cf410 con 0x7f9c3c0d0600 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.461+0000 7f9c3be00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d27c0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.461+0000 7f9c3be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d0a70 con 0x7f9c3c161200 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c3c0d2380 0x7f9c3c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9c3c0d0600 0x7f9c3c0ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56014/0 (socket says 172.21.15.5:56014) 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3b400640 1 -- 172.21.15.5:0/2111312886 learned_addr learned my addr 172.21.15.5:0/2111312886 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3aa00640 1 --2- 172.21.15.5:0/2111312886 >> v2:172.21.15.80:3300/0 conn(0x7f9c3c161200 0x7f9c3c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3a000640 1 -- 172.21.15.5:0/2111312886 >> v2:172.21.15.80:3300/0 conn(0x7f9c3c161200 msgr2=0x7f9c3c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:37.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3a000640 1 --2- 172.21.15.5:0/2111312886 >> v2:172.21.15.80:3300/0 conn(0x7f9c3c161200 0x7f9c3c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3a000640 1 -- 172.21.15.5:0/2111312886 >> v1:172.21.15.5:6789/0 conn(0x7f9c3c0d0600 legacy=0x7f9c3c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.462+0000 7f9c3a000640 1 -- 172.21.15.5:0/2111312886 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0cf6f0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.463+0000 7f9c3a000640 1 --2- 172.21.15.5:0/2111312886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c0d00c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c08bed0 tx=0x7f9c2402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dff2196d9f0ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c39600640 1 -- 172.21.15.5:0/2111312886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24037070 con 0x7f9c3c0d2380 2024-04-25T10:04:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c39600640 1 -- 172.21.15.5:0/2111312886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24032440 con 0x7f9c3c0d2380 2024-04-25T10:04:37.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c39600640 1 -- 172.21.15.5:0/2111312886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24032810 con 0x7f9c3c0d2380 2024-04-25T10:04:37.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 -- 172.21.15.5:0/2111312886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 msgr2=0x7f9c3c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2111312886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c0d00c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c08bed0 tx=0x7f9c2402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 -- 172.21.15.5:0/2111312886 shutdown_connections 2024-04-25T10:04:37.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2111312886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c0d00c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2111312886 >> v2:172.21.15.80:3300/0 conn(0x7f9c3c161200 0x7f9c3c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 -- 172.21.15.5:0/2111312886 >> 172.21.15.5:0/2111312886 conn(0x7f9c3c08af80 msgr2=0x7f9c3c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 -- 172.21.15.5:0/2111312886 shutdown_connections 2024-04-25T10:04:37.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.464+0000 7f9c3be00640 1 -- 172.21.15.5:0/2111312886 wait complete. 2024-04-25T10:04:37.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.466+0000 7f9c3be00640 1 Processor -- start 2024-04-25T10:04:37.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.466+0000 7f9c3be00640 1 -- start start 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.467+0000 7f9c3be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c3c0d0600 0x7f9c3c1f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.467+0000 7f9c3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.467+0000 7f9c3be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c3c161200 0x7f9c3c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.467+0000 7f9c3be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d42b0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.467+0000 7f9c3be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b28b0 con 0x7f9c3c0d0600 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.467+0000 7f9c3be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b3f90 con 0x7f9c3c161200 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43106/0 (socket says 172.21.15.5:43106) 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 -- 172.21.15.5:0/2606485715 learned_addr learned my addr 172.21.15.5:0/2606485715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3aa00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c3c0d0600 0x7f9c3c1f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3b400640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c3c161200 0x7f9c3c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 -- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c3c161200 msgr2=0x7f9c3c0c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c3c161200 0x7f9c3c0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 -- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c3c0d0600 msgr2=0x7f9c3c1f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c3c0d0600 0x7f9c3c1f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 -- 172.21.15.5:0/2606485715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0c9de0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3aa00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c3c0d0600 0x7f9c3c1f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.468+0000 7f9c3a000640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c153920 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9c24000c00 tx=0x7f9c240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.469+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24037040 con 0x7f9c3c0d2380 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.469+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24042070 con 0x7f9c3c0d2380 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.469+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24030bd0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.469+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0ca0c0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.469+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c0b7810 con 0x7f9c3c0d2380 2024-04-25T10:04:37.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.470+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c068f60 con 0x7f9c3c0d2380 2024-04-25T10:04:37.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.471+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c24047460 con 0x7f9c3c0d2380 2024-04-25T10:04:37.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.472+0000 7f9c32a00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c1806b4d0 0x7f9c1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:37.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.472+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c240ab0c0 con 0x7f9c3c0d2380 2024-04-25T10:04:37.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.473+0000 7f9c3aa00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c1806b4d0 0x7f9c1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:37.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.474+0000 7f9c3aa00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c1806b4d0 0x7f9c1806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006fd0 tx=0x7f9c2c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:37.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.476+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c24032e20 con 0x7f9c3c0d2380 2024-04-25T10:04:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.716+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c3c0b7a50 con 0x7f9c3c0d2380 2024-04-25T10:04:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.717+0000 7f9c32a00640 1 -- 172.21.15.5:0/2606485715 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9c24079c80 con 0x7f9c3c0d2380 2024-04-25T10:04:37.722 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.719+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c1806b4d0 msgr2=0x7f9c1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.719+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c1806b4d0 0x7f9c1806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006fd0 tx=0x7f9c2c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 msgr2=0x7f9c3c153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c153920 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9c24000c00 tx=0x7f9c240046c0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 shutdown_connections 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c3c161200 0x7f9c3c0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c3c0d2380 0x7f9c3c153920 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c1806b4d0 0x7f9c1806d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 --2- 172.21.15.5:0/2606485715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c3c0d0600 0x7f9c3c1f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 >> 172.21.15.5:0/2606485715 conn(0x7f9c3c08af80 msgr2=0x7f9c3c0b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 shutdown_connections 2024-04-25T10:04:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:37.720+0000 7f9c3be00640 1 -- 172.21.15.5:0/2606485715 wait complete. 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:04:37.745 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:04:37.746 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:37.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:37.746 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:04:37.802 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:04:37.802 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:04:37.803 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42448sec preferred_lft 42448sec 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:37.804 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:37.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:37.805 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:04:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:37.893 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:04:37.947 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:04:37.947 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:04:37.947 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:04:37.947 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:04:37.947 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:04:37.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:37.948 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:04:37.948 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:04:37.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:04:37.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:04:37.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:04:37.948 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:04:38.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.091 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:04:38.141 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:04:38.141 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:04:38.142 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:38.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:04:38.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:38.233 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:38.292 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:38.292 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:38.292 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:38.348 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:04:38.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.348 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:38.486 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:38.487 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:38.487 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:04:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:04:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:04:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:04:38.538 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:04:38.539 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:04:38.542 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:04:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:04:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:04:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:04:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:04:38.545 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:04:38.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:38.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.628 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:04:38.710 INFO:teuthology.orchestra.run.smithi005.stdout:client5934 2024-04-25T10:04:38.711 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:04:38.711 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:38.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.770 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T10:04:38.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:38.841 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T10:04:38.905 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:04:38.905 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:04:38.905 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:04:38.905 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:04:38.905 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T10:04:38.905 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:04:39.055 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:04:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:04:39.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.321+0000 7f0c5be00640 1 Processor -- start 2024-04-25T10:04:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.321+0000 7f0c5be00640 1 -- start start 2024-04-25T10:04:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.322+0000 7f0c5be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c5c161200 0x7f0c5c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.322+0000 7f0c5be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0c5c0d07d0 0x7f0c5c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.322+0000 7f0c5be00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c0cf410 con 0x7f0c5c0d2380 2024-04-25T10:04:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.322+0000 7f0c5be00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c0d27c0 con 0x7f0c5c0d07d0 2024-04-25T10:04:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.322+0000 7f0c5be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b28b0 con 0x7f0c5c161200 2024-04-25T10:04:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5a000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0c5c0d2380 0x7f0c5c0d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56026/0 (socket says 172.21.15.5:56026) 2024-04-25T10:04:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5a000640 1 -- 172.21.15.5:0/3659423267 learned_addr learned my addr 172.21.15.5:0/3659423267 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5b400640 1 --2- 172.21.15.5:0/3659423267 >> v2:172.21.15.5:3300/0 conn(0x7f0c5c0d07d0 0x7f0c5c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5aa00640 1 --2- 172.21.15.5:0/3659423267 >> v2:172.21.15.80:3300/0 conn(0x7f0c5c161200 0x7f0c5c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c59600640 1 -- 172.21.15.5:0/3659423267 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992375666 0 0) 0x7f0c5c0cf410 con 0x7f0c5c0d2380 2024-04-25T10:04:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c59600640 1 -- 172.21.15.5:0/3659423267 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c40003520 con 0x7f0c5c0d2380 2024-04-25T10:04:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5b400640 1 -- 172.21.15.5:0/3659423267 >> v2:172.21.15.80:3300/0 conn(0x7f0c5c161200 msgr2=0x7f0c5c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5b400640 1 --2- 172.21.15.5:0/3659423267 >> v2:172.21.15.80:3300/0 conn(0x7f0c5c161200 0x7f0c5c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5b400640 1 -- 172.21.15.5:0/3659423267 >> v1:172.21.15.5:6789/0 conn(0x7f0c5c0d2380 legacy=0x7f0c5c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.323+0000 7f0c5b400640 1 -- 172.21.15.5:0/3659423267 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0cf6f0 con 0x7f0c5c0d07d0 2024-04-25T10:04:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.324+0000 7f0c5b400640 1 --2- 172.21.15.5:0/3659423267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b2f80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c08bed0 tx=0x7f0c5402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29778cc629cf91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.325+0000 7f0c59600640 1 -- 172.21.15.5:0/3659423267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54037070 con 0x7f0c5c0d07d0 2024-04-25T10:04:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.325+0000 7f0c59600640 1 -- 172.21.15.5:0/3659423267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54032440 con 0x7f0c5c0d07d0 2024-04-25T10:04:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.325+0000 7f0c59600640 1 -- 172.21.15.5:0/3659423267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540327f0 con 0x7f0c5c0d07d0 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.325+0000 7f0c5be00640 1 -- 172.21.15.5:0/3659423267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d07d0 msgr2=0x7f0c5c0b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.325+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3659423267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b2f80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c08bed0 tx=0x7f0c5402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.326+0000 7f0c5be00640 1 -- 172.21.15.5:0/3659423267 shutdown_connections 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.326+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3659423267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b2f80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.326+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3659423267 >> v2:172.21.15.80:3300/0 conn(0x7f0c5c161200 0x7f0c5c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.326+0000 7f0c5be00640 1 -- 172.21.15.5:0/3659423267 >> 172.21.15.5:0/3659423267 conn(0x7f0c5c08af80 msgr2=0x7f0c5c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.326+0000 7f0c5be00640 1 -- 172.21.15.5:0/3659423267 shutdown_connections 2024-04-25T10:04:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.326+0000 7f0c5be00640 1 -- 172.21.15.5:0/3659423267 wait complete. 2024-04-25T10:04:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.327+0000 7f0c5be00640 1 Processor -- start 2024-04-25T10:04:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.328+0000 7f0c5be00640 1 -- start start 2024-04-25T10:04:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 0x7f0c5c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c5c161200 0x7f0c5c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b34c0 con 0x7f0c5c0d2380 2024-04-25T10:04:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0d42b0 con 0x7f0c5c161200 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b3640 con 0x7f0c5c0d07d0 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 0x7f0c5c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 0x7f0c5c0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43124/0 (socket says 172.21.15.5:43124) 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5a000640 1 -- 172.21.15.5:0/3098217913 learned_addr learned my addr 172.21.15.5:0/3098217913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5b400640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c5c161200 0x7f0c5c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5aa00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5a000640 1 -- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c5c0d07d0 msgr2=0x7f0c5c0b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.329+0000 7f0c5a000640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c5a000640 1 -- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c5c161200 msgr2=0x7f0c5c15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c5a000640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c5c161200 0x7f0c5c15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c5a000640 1 -- 172.21.15.5:0/3098217913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0b7370 con 0x7f0c5c0d2380 2024-04-25T10:04:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c5a000640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 0x7f0c5c0cc850 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00cb70 tx=0x7f0c4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c017070 con 0x7f0c5c0d2380 2024-04-25T10:04:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c00ad40 con 0x7f0c5c0d2380 2024-04-25T10:04:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c012620 con 0x7f0c5c0d2380 2024-04-25T10:04:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c0b7650 con 0x7f0c5c0d2380 2024-04-25T10:04:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.330+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c0b7c10 con 0x7f0c5c0d2380 2024-04-25T10:04:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.332+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c068f60 con 0x7f0c5c0d2380 2024-04-25T10:04:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.333+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c4c01a070 con 0x7f0c5c0d2380 2024-04-25T10:04:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.334+0000 7f0c52a00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.334+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c4c08a1f0 con 0x7f0c5c0d2380 2024-04-25T10:04:39.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.334+0000 7f0c5aa00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.336+0000 7f0c5aa00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c3806b4d0 0x7f0c3806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0c44006fd0 tx=0x7f0c44008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.338+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c4c058ee0 con 0x7f0c5c0d2380 2024-04-25T10:04:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.577+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c5c15ff60 con 0x7f0c5c0d2380 2024-04-25T10:04:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.578+0000 7f0c52a00640 1 -- 172.21.15.5:0/3098217913 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f0c4c0587e0 con 0x7f0c5c0d2380 2024-04-25T10:04:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:39.584 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:04:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.581+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c3806b4d0 msgr2=0x7f0c3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.581+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c3806b4d0 0x7f0c3806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0c44006fd0 tx=0x7f0c44008040 comp rx=0 tx=0).stop 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.581+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 msgr2=0x7f0c5c0cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.581+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 0x7f0c5c0cc850 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00cb70 tx=0x7f0c4c00a430 comp rx=0 tx=0).stop 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 shutdown_connections 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c3806b4d0 0x7f0c3806d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c5c161200 0x7f0c5c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c5c0d2380 0x7f0c5c0cc850 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 --2- 172.21.15.5:0/3098217913 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c5c0d07d0 0x7f0c5c0b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 >> 172.21.15.5:0/3098217913 conn(0x7f0c5c08af80 msgr2=0x7f0c5c160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 shutdown_connections 2024-04-25T10:04:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.582+0000 7f0c5be00640 1 -- 172.21.15.5:0/3098217913 wait complete. 2024-04-25T10:04:39.607 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:04:39.852 INFO:teuthology.orchestra.run.smithi080.stdout:{"message":"","return_code":0} 2024-04-25T10:04:39.852 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:04:39.853 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:04:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.967+0000 7f0864d78f40 1 Processor -- start 2024-04-25T10:04:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.968+0000 7f0864d78f40 1 -- start start 2024-04-25T10:04:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0864d78f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 0x562a5666d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0864d78f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b9da0 0x562a5669f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0864d78f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56743b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0864d78f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a56614dd0 con 0x562a566b7db0 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0864d78f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a566ba380 con 0x562a566b9da0 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0864d78f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a5669fae0 con 0x562a5672ab20 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 0x562a5666d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 0x562a5666d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43146/0 (socket says 172.21.15.5:43146) 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 -- 172.21.15.5:0/1645532625 learned_addr learned my addr 172.21.15.5:0/1645532625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862a00640 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56743b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 -- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 msgr2=0x562a56743b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56743b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 -- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b9da0 msgr2=0x562a5669f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b9da0 0x562a5669f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862000640 1 -- 172.21.15.5:0/1645532625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a56726d80 con 0x562a566b7db0 2024-04-25T10:04:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.969+0000 7f0862a00640 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56743b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.970+0000 7f0862000640 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 0x562a5666d1f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f085400a7a0 tx=0x7f085400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d402cbbcb7140598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.970+0000 7f085be00640 1 -- 172.21.15.5:0/1645532625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854019070 con 0x562a566b7db0 2024-04-25T10:04:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.970+0000 7f085be00640 1 -- 172.21.15.5:0/1645532625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854007600 con 0x562a566b7db0 2024-04-25T10:04:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.970+0000 7f085be00640 1 -- 172.21.15.5:0/1645532625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08540079d0 con 0x562a566b7db0 2024-04-25T10:04:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 -- 172.21.15.5:0/1645532625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 msgr2=0x562a5666d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 0x562a5666d1f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f085400a7a0 tx=0x7f085400a890 comp rx=0 tx=0).stop 2024-04-25T10:04:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 -- 172.21.15.5:0/1645532625 shutdown_connections 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56743b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b9da0 0x562a5669f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 --2- 172.21.15.5:0/1645532625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b7db0 0x562a5666d1f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 -- 172.21.15.5:0/1645532625 >> 172.21.15.5:0/1645532625 conn(0x562a5666cc60 msgr2=0x562a5669dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 -- 172.21.15.5:0/1645532625 shutdown_connections 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.971+0000 7f0864d78f40 1 -- 172.21.15.5:0/1645532625 wait complete. 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.972+0000 7f0864d78f40 1 Processor -- start 2024-04-25T10:04:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.972+0000 7f0864d78f40 1 -- start start 2024-04-25T10:04:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0864d78f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b7db0 0x562a5669bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0864d78f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b9da0 0x562a56740cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0864d78f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56741200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0864d78f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a567262d0 con 0x562a566b9da0 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0864d78f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a56726070 con 0x562a566b7db0 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0864d78f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a56726730 con 0x562a5672ab20 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0861600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b9da0 0x562a56740cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.973+0000 7f0861600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b9da0 0x562a56740cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43148/0 (socket says 172.21.15.5:43148) 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 -- 172.21.15.5:0/2209219130 learned_addr learned my addr 172.21.15.5:0/2209219130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0862000640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b7db0 0x562a5669bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0862a00640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56741200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 -- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 msgr2=0x562a56741200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562a5672ab20 0x562a56741200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 -- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b7db0 msgr2=0x562a5669bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562a566b7db0 0x562a5669bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 -- 172.21.15.5:0/2209219130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a56725960 con 0x562a566b9da0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.974+0000 7f0861600640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b9da0 0x562a56740cc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f085000bb50 tx=0x7f0850009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.975+0000 7f085aa00640 1 -- 172.21.15.5:0/2209219130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850014030 con 0x562a566b9da0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.975+0000 7f085aa00640 1 -- 172.21.15.5:0/2209219130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850009c40 con 0x562a566b9da0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.975+0000 7f085aa00640 1 -- 172.21.15.5:0/2209219130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085000f400 con 0x562a566b9da0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.975+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562a56725540 con 0x562a566b9da0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.975+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a56725260 con 0x562a566b9da0 2024-04-25T10:04:39.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.977+0000 7f085aa00640 1 -- 172.21.15.5:0/2209219130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0850017070 con 0x562a566b9da0 2024-04-25T10:04:39.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.978+0000 7f085aa00640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0844069eb0 0x7f084406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.978+0000 7f085aa00640 1 -- 172.21.15.5:0/2209219130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0850088810 con 0x562a566b9da0 2024-04-25T10:04:39.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.979+0000 7f0864d78f40 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x562a5669c310 0x562a567d4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.979+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) v9 -- 0x562a5669fc60 con 0x562a5669c310 2024-04-25T10:04:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.979+0000 7f0862000640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0844069eb0 0x7f084406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.980+0000 7f0862a00640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x562a5669c310 0x562a567d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.980+0000 7f0862000640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0844069eb0 0x7f084406c370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0854000c00 tx=0x7f085400c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.981+0000 7f0862a00640 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x562a5669c310 0x562a567d4600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0862a00640 1 -- 172.21.15.5:0/2209219130 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f085c0518f0 con 0x562a5669c310 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x562a5669c310 msgr2=0x562a567d4600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x562a5669c310 0x562a567d4600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0844069eb0 msgr2=0x7f084406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0844069eb0 0x7f084406c370 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0854000c00 tx=0x7f085400c000 comp rx=0 tx=0).stop 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b9da0 msgr2=0x562a56740cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:39.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 --2- 172.21.15.5:0/2209219130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a566b9da0 0x562a56740cc0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f085000bb50 tx=0x7f0850009530 comp rx=0 tx=0).stop 2024-04-25T10:04:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0862a00640 1 -- 172.21.15.5:0/2209219130 reap_dead start 2024-04-25T10:04:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 shutdown_connections 2024-04-25T10:04:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 >> 172.21.15.5:0/2209219130 conn(0x562a5666cc60 msgr2=0x562a5669ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 shutdown_connections 2024-04-25T10:04:39.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:39.982+0000 7f0864d78f40 1 -- 172.21.15.5:0/2209219130 wait complete. 2024-04-25T10:04:39.992 INFO:teuthology.orchestra.run.smithi005.stdout:Yalphaparent_a 2024-04-25T10:04:39.992 DEBUG:teuthology.orchestra.run.smithi005:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:04:40.147 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:04:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "ancestors": [ 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1099511627776, 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "alpha", 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:04:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1, 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "parent_a", 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 8, 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "old_pools": [] 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:04:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:40.166 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T10:04:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.281+0000 7f50e802af40 1 Processor -- start 2024-04-25T10:04:40.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.281+0000 7f50e802af40 1 -- start start 2024-04-25T10:04:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.282+0000 7f50e802af40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f723db0 0x56393f797290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.282+0000 7f50e802af40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 0x56393f7977d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.282+0000 7f50e802af40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f796b20 0x56393f7b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.282+0000 7f50e802af40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393f680dd0 con 0x56393f725da0 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.282+0000 7f50e802af40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393f797110 con 0x56393f723db0 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.282+0000 7f50e802af40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56393f7b2450 con 0x56393f796b20 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 0x56393f7977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 0x56393f7977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43174/0 (socket says 172.21.15.5:43174) 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 -- 172.21.15.5:0/2822087077 learned_addr learned my addr 172.21.15.5:0/2822087077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e5200640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f723db0 0x56393f797290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e5c00640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f796b20 0x56393f7b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 -- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f796b20 msgr2=0x56393f7b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f796b20 0x56393f7b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 -- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f723db0 msgr2=0x56393f797290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f723db0 0x56393f797290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e4800640 1 -- 172.21.15.5:0/2822087077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393f798140 con 0x56393f725da0 2024-04-25T10:04:40.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e5c00640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f796b20 0x56393f7b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.283+0000 7f50e5200640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f723db0 0x56393f797290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.284+0000 7f50e4800640 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 0x56393f7977d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001e20 tx=0x7f50d000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b91f1270697fe63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.284+0000 7f50e3400640 1 -- 172.21.15.5:0/2822087077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d000e5c0 con 0x56393f725da0 2024-04-25T10:04:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.284+0000 7f50e3400640 1 -- 172.21.15.5:0/2822087077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d0006070 con 0x56393f725da0 2024-04-25T10:04:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.284+0000 7f50e3400640 1 -- 172.21.15.5:0/2822087077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d000e9a0 con 0x56393f725da0 2024-04-25T10:04:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 -- 172.21.15.5:0/2822087077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 msgr2=0x56393f7977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 0x56393f7977d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f50d0001e20 tx=0x7f50d000b6c0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 -- 172.21.15.5:0/2822087077 shutdown_connections 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f796b20 0x56393f7b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f725da0 0x56393f7977d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 --2- 172.21.15.5:0/2822087077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f723db0 0x56393f797290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 -- 172.21.15.5:0/2822087077 >> 172.21.15.5:0/2822087077 conn(0x56393f6d8c60 msgr2=0x56393f7b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 -- 172.21.15.5:0/2822087077 shutdown_connections 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.285+0000 7f50e802af40 1 -- 172.21.15.5:0/2822087077 wait complete. 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.286+0000 7f50e802af40 1 Processor -- start 2024-04-25T10:04:40.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.286+0000 7f50e802af40 1 -- start start 2024-04-25T10:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e802af40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f723db0 0x56393f7b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e802af40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f725da0 0x56393f7b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e802af40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f796b20 0x56393f7ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e802af40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393f792b90 con 0x56393f796b20 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e802af40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393f797e60 con 0x56393f725da0 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e802af40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56393f7b2680 con 0x56393f723db0 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f796b20 0x56393f7ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f796b20 0x56393f7ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43182/0 (socket says 172.21.15.5:43182) 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.287+0000 7f50e5c00640 1 -- 172.21.15.5:0/1696055298 learned_addr learned my addr 172.21.15.5:0/1696055298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5200640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f723db0 0x56393f7b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e4800640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f725da0 0x56393f7b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5c00640 1 -- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f723db0 msgr2=0x56393f7b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5c00640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f723db0 0x56393f7b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5c00640 1 -- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f725da0 msgr2=0x56393f7b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5c00640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f725da0 0x56393f7b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5c00640 1 -- 172.21.15.5:0/1696055298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393f6dc960 con 0x56393f796b20 2024-04-25T10:04:40.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e4800640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56393f725da0 0x56393f7b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5200640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56393f723db0 0x56393f7b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e5c00640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f796b20 0x56393f7ae1a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00a470 tx=0x7f50dc00a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e2000640 1 -- 172.21.15.5:0/1696055298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc0107f0 con 0x56393f796b20 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e2000640 1 -- 172.21.15.5:0/1696055298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc016070 con 0x56393f796b20 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.288+0000 7f50e2000640 1 -- 172.21.15.5:0/1696055298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc010bd0 con 0x56393f796b20 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.289+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56393f6dcc40 con 0x56393f796b20 2024-04-25T10:04:40.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.289+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56393f6dd200 con 0x56393f796b20 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.291+0000 7f50e2000640 1 -- 172.21.15.5:0/1696055298 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50dc011480 con 0x56393f796b20 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.292+0000 7f50e2000640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50c4069eb0 0x7f50c406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.292+0000 7f50e2000640 1 -- 172.21.15.5:0/1696055298 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f50dc089740 con 0x56393f796b20 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.292+0000 7f50e802af40 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x56393f846640 0x56393f793fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.292+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51) v9 -- 0x56393f7aebf0 con 0x56393f846640 2024-04-25T10:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.292+0000 7f50e5200640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50c4069eb0 0x7f50c406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.293+0000 7f50e4800640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x56393f846640 0x56393f793fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.294+0000 7f50e5200640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50c4069eb0 0x7f50c406c370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f50d8057000 tx=0x7f50d8052000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.294+0000 7f50e4800640 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x56393f846640 0x56393f793fb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e4800640 1 -- 172.21.15.5:0/1696055298 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v51'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f50d000e5c0 con 0x56393f846640 2024-04-25T10:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x56393f846640 msgr2=0x56393f793fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x56393f846640 0x56393f793fb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50c4069eb0 msgr2=0x7f50c406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50c4069eb0 0x7f50c406c370 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f50d8057000 tx=0x7f50d8052000 comp rx=0 tx=0).stop 2024-04-25T10:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f796b20 msgr2=0x56393f7ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 --2- 172.21.15.5:0/1696055298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56393f796b20 0x56393f7ae1a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00a470 tx=0x7f50dc00a930 comp rx=0 tx=0).stop 2024-04-25T10:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e5c00640 1 -- 172.21.15.5:0/1696055298 reap_dead start 2024-04-25T10:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 shutdown_connections 2024-04-25T10:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 >> 172.21.15.5:0/1696055298 conn(0x56393f6d8c60 msgr2=0x56393f7b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 shutdown_connections 2024-04-25T10:04:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.298+0000 7f50e802af40 1 -- 172.21.15.5:0/1696055298 wait complete. 2024-04-25T10:04:40.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:04:40.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T10:04:40.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.496+0000 7f0a9ba00640 1 Processor -- start 2024-04-25T10:04:40.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.496+0000 7f0a9ba00640 1 -- start start 2024-04-25T10:04:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.498+0000 7f0a9ba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a94168830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.498+0000 7f0a9ba00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940d2290 con 0x7f0a940d3330 2024-04-25T10:04:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.498+0000 7f0a9ba00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a940d2570 con 0x7f0a9415de70 2024-04-25T10:04:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.498+0000 7f0a9ba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940d2030 con 0x7f0a940d3f10 2024-04-25T10:04:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.499+0000 7f0a93e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0a940d3330 0x7f0a940d3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56044/0 (socket says 172.21.15.5:56044) 2024-04-25T10:04:40.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.499+0000 7f0a93e00640 1 -- 172.21.15.5:0/1911503003 learned_addr learned my addr 172.21.15.5:0/1911503003 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:04:40.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.499+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a94168830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.499+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768893277 0 0) 0x7f0a940d2290 con 0x7f0a940d3330 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.499+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a7c003520 con 0x7f0a940d3330 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.499+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2772001296 0 0) 0x7f0a7c003520 con 0x7f0a940d3330 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 msgr2=0x7f0a94168830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a94168830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 >> v1:172.21.15.80:6789/0 conn(0x7f0a9415de70 legacy=0x7f0a940d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a940d2850 con 0x7f0a940d3330 2024-04-25T10:04:40.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0a80001670 con 0x7f0a940d3330 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 >> v1:172.21.15.5:6789/0 conn(0x7f0a940d3330 legacy=0x7f0a940d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c0039e0 0x7f0a7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a7c004d40 0x7f0a7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940d2030 con 0x7f0a7c004d40 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00bad0 con 0x7f0a7c0039e0 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00bcd0 con 0x7f0a940d3f10 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a7c004d40 0x7f0a7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a99000640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c0039e0 0x7f0a7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a99a00640 1 -- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 msgr2=0x7f0a7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.500+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.501+0000 7f0a99a00640 1 -- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c0039e0 msgr2=0x7f0a7c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.501+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c0039e0 0x7f0a7c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.501+0000 7f0a99a00640 1 -- 172.21.15.5:0/1911503003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c003520 con 0x7f0a7c004d40 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.501+0000 7f0a99000640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c0039e0 0x7f0a7c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.501+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a940d3f10 0x7f0a7c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.502+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a7c004d40 0x7f0a7c0051c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009f90 tx=0x7f0a7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80afa76d15b1bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.502+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c009c10 con 0x7f0a7c004d40 2024-04-25T10:04:40.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.502+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c036070 con 0x7f0a7c004d40 2024-04-25T10:04:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.502+0000 7f0a93400640 1 -- 172.21.15.5:0/1911503003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c031850 con 0x7f0a7c004d40 2024-04-25T10:04:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1911503003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a7c004d40 msgr2=0x7f0a7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1911503003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a7c004d40 0x7f0a7c0051c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009f90 tx=0x7f0a7c005830 comp rx=0 tx=0).stop 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a99a00640 1 -- 172.21.15.5:0/1911503003 reap_dead start 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1911503003 shutdown_connections 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1911503003 >> 172.21.15.5:0/1911503003 conn(0x7f0a940726b0 msgr2=0x7f0a94072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1911503003 shutdown_connections 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.503+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1911503003 wait complete. 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.505+0000 7f0a9ba00640 1 Processor -- start 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.505+0000 7f0a9ba00640 1 -- start start 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a9ba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c004d40 0x7f0a94155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a9ba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a94152210 0x7f0a94157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a9ba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 0x7f0a940c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a9ba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940b9290 con 0x7f0a940c9050 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a9ba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940d2e10 con 0x7f0a7c004d40 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a9ba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940b9410 con 0x7f0a94152210 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a99a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 0x7f0a940c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a99a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 0x7f0a940c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43192/0 (socket says 172.21.15.5:43192) 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.506+0000 7f0a99a00640 1 -- 172.21.15.5:0/1890030555 learned_addr learned my addr 172.21.15.5:0/1890030555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a94152210 0x7f0a94157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99000640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c004d40 0x7f0a94155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99a00640 1 -- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a94152210 msgr2=0x7f0a94157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a94152210 0x7f0a94157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99a00640 1 -- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c004d40 msgr2=0x7f0a94155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c004d40 0x7f0a94155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99a00640 1 -- 172.21.15.5:0/1890030555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94156250 con 0x7f0a940c9050 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 0x7f0a940c94b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c033040 tx=0x7f0a8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a90c00640 1 -- 172.21.15.5:0/1890030555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c009c10 con 0x7f0a940c9050 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.507+0000 7f0a90c00640 1 -- 172.21.15.5:0/1890030555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c003990 con 0x7f0a940c9050 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.508+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a94156530 con 0x7f0a940c9050 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.508+0000 7f0a90c00640 1 -- 172.21.15.5:0/1890030555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003d80 con 0x7f0a940c9050 2024-04-25T10:04:40.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.508+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a94156af0 con 0x7f0a940c9050 2024-04-25T10:04:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.510+0000 7f0a90c00640 1 -- 172.21.15.5:0/1890030555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a8c005d10 con 0x7f0a940c9050 2024-04-25T10:04:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.512+0000 7f0a90c00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a7406b500 0x7f0a7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.512+0000 7f0a90c00640 1 -- 172.21.15.5:0/1890030555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a8c0aba00 con 0x7f0a940c9050 2024-04-25T10:04:40.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.512+0000 7f0a9e7a5740 1 Processor -- start 2024-04-25T10:04:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.512+0000 7f0a9e7a5740 1 -- start start 2024-04-25T10:04:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.512+0000 7f0a99000640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a7406b500 0x7f0a7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afde8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afdea8a40 0x560afdea8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde60e80 0x560afdea7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560afddc3980 con 0x560afde8b1a0 2024-04-25T10:04:40.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560afdd6cde0 con 0x560afde60e80 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560afddbfa20 con 0x560afdea8a40 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a99000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afde8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a99000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afde8b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43202/0 (socket says 172.21.15.5:43202) 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.513+0000 7f0a99000640 1 -- 172.21.15.5:0/1079770394 learned_addr learned my addr 172.21.15.5:0/1079770394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde60e80 0x560afdea7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afdea8a40 0x560afdea8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a7406b500 0x7f0a7406d9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001780 tx=0x7f0a88001b80 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 -- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afdea8a40 msgr2=0x560afdea8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afdea8a40 0x560afdea8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 -- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde60e80 msgr2=0x560afdea7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde60e80 0x560afdea7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 -- 172.21.15.5:0/1079770394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560afdd8ae30 con 0x560afde8b1a0 2024-04-25T10:04:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde60e80 0x560afdea7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a99000640 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afde8b580 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0a88005410 tx=0x7f0a8801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b481834a73c3716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.514+0000 7f0a85600640 1 -- 172.21.15.5:0/1079770394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88025070 con 0x560afde8b1a0 2024-04-25T10:04:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a85600640 1 -- 172.21.15.5:0/1079770394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8800fe90 con 0x560afde8b1a0 2024-04-25T10:04:40.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a85600640 1 -- 172.21.15.5:0/1079770394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88003d30 con 0x560afde8b1a0 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1079770394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 msgr2=0x560afde8b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afde8b580 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0a88005410 tx=0x7f0a8801a000 comp rx=0 tx=0).stop 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1079770394 shutdown_connections 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde60e80 0x560afdea7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afdea8a40 0x560afdea8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1079770394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afde8b580 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.515+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1079770394 >> 172.21.15.5:0/1079770394 conn(0x560afde5d7b0 msgr2=0x560afdea5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.516+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1079770394 shutdown_connections 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.516+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1079770394 wait complete. 2024-04-25T10:04:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.516+0000 7f0a9e7a5740 1 -- start start 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.516+0000 7f0a9e7a5740 1 Processor -- start 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.517+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde8b1a0 0x560afde86e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.517+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 0x560afde9e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afde5a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560afde61c20 con 0x560afdea8a40 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560afdd623c0 con 0x560afde8b1a0 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560afde5b6e0 con 0x560afde5a2a0 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a9e7a5740 10 client.0 _ll_register_callbacks cb 0x560afdad7c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 0x560afde9e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 0x560afde9e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43208/0 (socket says 172.21.15.5:43208) 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 -- 172.21.15.5:0/4078017231 learned_addr learned my addr 172.21.15.5:0/4078017231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a99000640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde8b1a0 0x560afde86e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a99a00640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afde5a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 -- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 msgr2=0x560afde5a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afde5a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 -- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde8b1a0 msgr2=0x560afde86e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde8b1a0 0x560afde86e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a93e00640 1 -- 172.21.15.5:0/4078017231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560afdba07e0 con 0x560afdea8a40 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.518+0000 7f0a99000640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde8b1a0 0x560afde86e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a93e00640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 0x560afde9e3f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0a80011530 tx=0x7f0a800119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a791f6442138b4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a7a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a7a000640 1 -- 172.21.15.5:0/4078017231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80019020 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a8800fe90 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a7a000640 1 -- 172.21.15.5:0/4078017231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a80015680 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a7a000640 1 -- 172.21.15.5:0/4078017231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a80023050 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a9e7a5740 10 client.5979 fetch_fsmap learned FSMap version 26 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.519+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x560afdc00f70 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a7a000640 1 -- 172.21.15.5:0/4078017231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8800fe90 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a7a000640 1 -- 172.21.15.5:0/4078017231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a80026020 con 0x560afdea8a40 2024-04-25T10:04:40.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a9e7a5740 10 client.5979 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:04:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a9e7a5740 10 client.5979 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:04:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a9e7a5740 20 client.5979 populate_metadata read hostname 'smithi005' 2024-04-25T10:04:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afde8e460 0x560afde9d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a9e7a5740 4 client.5979 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:40.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.520+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560afddd0ec0 con 0x560afde8e460 2024-04-25T10:04:40.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.521+0000 7f0a99000640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afde8e460 0x560afde9d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.521+0000 7f0a99000640 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afde8e460 0x560afde9d060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a816c8ae402f8b server_cookie=bbd3c805aa95b197 in_seq=0 out_seq=0 2024-04-25T10:04:40.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.521+0000 7f0a7a000640 10 client.5979 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a7a000640 1 -- 172.21.15.5:0/4078017231 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560afddd0ec0 con 0x560afde8e460 2024-04-25T10:04:40.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a7a000640 10 client.5979 handle_command_reply: tid=0 2024-04-25T10:04:40.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a9e7a5740 1 client.5979 shutdown 2024-04-25T10:04:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7fde39c00640 1 Processor -- start 2024-04-25T10:04:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 msgr2=0x560afde9e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 0x560afde9e3f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0a80011530 tx=0x7f0a800119f0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7fde39c00640 1 -- start start 2024-04-25T10:04:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 shutdown_connections 2024-04-25T10:04:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.523+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afde8e460 0x560afde9d060 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afde5a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afdea8a40 0x560afde9e3f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/4078017231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afde8b1a0 0x560afde86e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 >> 172.21.15.5:0/4078017231 conn(0x560afde60e80 msgr2=0x560afdea9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a7a000640 0 client.5979 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 shutdown_connections 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/4078017231 wait complete. 2024-04-25T10:04:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.524+0000 7f0a9e7a5740 20 client.5979 trim_cache size 0 max 16384 2024-04-25T10:04:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde39c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde341610d0 0x7fde340ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde39c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde3406ab60 0x7fde3406b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde39c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde34161ac0 con 0x7fde34161620 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde39c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde340b0100 con 0x7fde341610d0 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde39c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde340b3620 con 0x7fde3406ab60 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde32a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde341610d0 0x7fde340ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde32a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde341610d0 0x7fde340ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43216/0 (socket says 172.21.15.5:43216) 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.525+0000 7fde32a00640 1 -- 172.21.15.5:0/2387461639 learned_addr learned my addr 172.21.15.5:0/2387461639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.526+0000 7fde33400640 1 --2- 172.21.15.5:0/2387461639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde3406ab60 0x7fde3406b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.526+0000 7fde32a00640 1 -- 172.21.15.5:0/2387461639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde3406ab60 msgr2=0x7fde3406b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.526+0000 7fde32a00640 1 --2- 172.21.15.5:0/2387461639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde3406ab60 0x7fde3406b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.526+0000 7fde32a00640 1 -- 172.21.15.5:0/2387461639 >> v1:172.21.15.80:6789/0 conn(0x7fde34161620 legacy=0x7fde340af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.526+0000 7fde32a00640 1 -- 172.21.15.5:0/2387461639 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34161da0 con 0x7fde341610d0 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.527+0000 7fde32a00640 1 --2- 172.21.15.5:0/2387461639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde341610d0 0x7fde340ce5b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b844aa8a6726dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.527+0000 7fde31600640 1 -- 172.21.15.5:0/2387461639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24037070 con 0x7fde341610d0 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.527+0000 7fde31600640 1 -- 172.21.15.5:0/2387461639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde24032440 con 0x7fde341610d0 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.527+0000 7fde31600640 1 -- 172.21.15.5:0/2387461639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24032810 con 0x7fde341610d0 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 -- 172.21.15.5:0/2387461639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde341610d0 msgr2=0x7fde340ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 --2- 172.21.15.5:0/2387461639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde341610d0 0x7fde340ce5b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2402a200 comp rx=0 tx=0).stop 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 -- 172.21.15.5:0/2387461639 shutdown_connections 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 --2- 172.21.15.5:0/2387461639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde3406ab60 0x7fde3406b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 --2- 172.21.15.5:0/2387461639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde341610d0 0x7fde340ce5b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 -- 172.21.15.5:0/2387461639 >> 172.21.15.5:0/2387461639 conn(0x7fde340b2320 msgr2=0x7fde340b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 -- 172.21.15.5:0/2387461639 shutdown_connections 2024-04-25T10:04:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.528+0000 7fde39c00640 1 -- 172.21.15.5:0/2387461639 wait complete. 2024-04-25T10:04:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.530+0000 7fde39c00640 1 Processor -- start 2024-04-25T10:04:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.530+0000 7fde39c00640 1 -- start start 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.531+0000 7fde39c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde3406ab60 0x7fde340cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.531+0000 7fde39c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde341610d0 0x7fde3414e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.531+0000 7fde39c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34161620 0x7fde341f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.531+0000 7fde39c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34162360 con 0x7fde34161620 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.531+0000 7fde39c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde34162080 con 0x7fde3406ab60 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.531+0000 7fde39c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde340b3ab0 con 0x7fde341610d0 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34161620 0x7fde341f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34161620 0x7fde341f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43228/0 (socket says 172.21.15.5:43228) 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 -- 172.21.15.5:0/3550259842 learned_addr learned my addr 172.21.15.5:0/3550259842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde32a00640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde3406ab60 0x7fde340cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde32000640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde341610d0 0x7fde3414e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 -- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde341610d0 msgr2=0x7fde3414e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde341610d0 0x7fde3414e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 -- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde3406ab60 msgr2=0x7fde340cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde3406ab60 0x7fde340cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 -- 172.21.15.5:0/3550259842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3414de90 con 0x7fde34161620 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.532+0000 7fde33400640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde34161620 0x7fde341f3850 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fde2c007e40 tx=0x7fde2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.533+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c014620 con 0x7fde34161620 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.533+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c019070 con 0x7fde34161620 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.533+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c014a20 con 0x7fde34161620 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.533+0000 7fde39c00640 1 -- 172.21.15.5:0/3550259842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde341f4290 con 0x7fde34161620 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.533+0000 7fde39c00640 1 -- 172.21.15.5:0/3550259842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3414e130 con 0x7fde34161620 2024-04-25T10:04:40.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.534+0000 7fde39c00640 1 -- 172.21.15.5:0/3550259842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fde34068f60 con 0x7fde34161620 2024-04-25T10:04:40.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.535+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde2c014bc0 con 0x7fde34161620 2024-04-25T10:04:40.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.536+0000 7fde2aa00640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde1406b4d0 0x7fde1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.536+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde2c08bae0 con 0x7fde34161620 2024-04-25T10:04:40.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.536+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+497 (secure 0 0 0) 0x7fde2c08c2b0 con 0x7fde34161620 2024-04-25T10:04:40.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.537+0000 7fde39c00640 1 -- 172.21.15.5:0/3550259842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fde341f47b0 con 0x7fde34161620 2024-04-25T10:04:40.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.537+0000 7fde32a00640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde1406b4d0 0x7fde1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.538+0000 7fde32a00640 1 --2- 172.21.15.5:0/3550259842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde1406b4d0 0x7fde1406d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fde24034000 tx=0x7fde2402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.542+0000 7f0a9e7a5740 1 Processor -- start 2024-04-25T10:04:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.542+0000 7f0a9e7a5740 1 -- start start 2024-04-25T10:04:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 0x560afde860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde8b1a0 0x560afdea53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afde60230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a942009d0 con 0x560afde5a2a0 2024-04-25T10:04:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941ffb60 con 0x560afdea8a40 2024-04-25T10:04:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a940cc120 con 0x560afde8b1a0 2024-04-25T10:04:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a99000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 0x560afde860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a99000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 0x560afde860b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43238/0 (socket says 172.21.15.5:43238) 2024-04-25T10:04:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.543+0000 7f0a99000640 1 -- 172.21.15.5:0/3654608434 learned_addr learned my addr 172.21.15.5:0/3654608434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99a00640 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afde60230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a93e00640 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde8b1a0 0x560afdea53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99000640 1 -- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde8b1a0 msgr2=0x560afdea53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99000640 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde8b1a0 0x560afdea53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99000640 1 -- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 msgr2=0x560afde60230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99000640 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afde60230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99000640 1 -- 172.21.15.5:0/3654608434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560afda98170 con 0x560afde5a2a0 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99a00640 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afde60230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a99000640 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 0x560afde860b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a88023040 tx=0x7f0a88009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c15ad36b5e89bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a84c00640 1 -- 172.21.15.5:0/3654608434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88025040 con 0x560afde5a2a0 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a84c00640 1 -- 172.21.15.5:0/3654608434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88007230 con 0x560afde5a2a0 2024-04-25T10:04:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.544+0000 7f0a84c00640 1 -- 172.21.15.5:0/3654608434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88007600 con 0x560afde5a2a0 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/3654608434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 msgr2=0x560afde860b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 0x560afde860b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0a88023040 tx=0x7f0a88009040 comp rx=0 tx=0).stop 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/3654608434 shutdown_connections 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afde60230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde8b1a0 0x560afdea53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/3654608434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde5a2a0 0x560afde860b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/3654608434 >> 172.21.15.5:0/3654608434 conn(0x560afde60e80 msgr2=0x560afdea9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/3654608434 shutdown_connections 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.545+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/3654608434 wait complete. 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.546+0000 7f0a9e7a5740 1 -- start start 2024-04-25T10:04:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.546+0000 7f0a9e7a5740 1 Processor -- start 2024-04-25T10:04:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afdea53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afdea5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afdf09830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560afde5ad60 con 0x560afde8b1a0 2024-04-25T10:04:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941ff5c0 con 0x560afdea8a40 2024-04-25T10:04:40.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560afdf0a730 con 0x560afde5a2a0 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.547+0000 7f0a9e7a5740 10 client.0 _ll_register_callbacks cb 0x560afde5bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afdea5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afdea5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43248/0 (socket says 172.21.15.5:43248) 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 -- 172.21.15.5:0/1022252275 learned_addr learned my addr 172.21.15.5:0/1022252275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a99000640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afdea53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afdf09830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 -- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 msgr2=0x560afdea53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afdea53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 -- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 msgr2=0x560afdf09830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afdf09830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 -- 172.21.15.5:0/1022252275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560afde9de90 con 0x560afde8b1a0 2024-04-25T10:04:40.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a99a00640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afdf09830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a99000640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afdea53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a93e00640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afdea5920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0a80011530 tx=0x7f0a80014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229f61c1913879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.548+0000 7f0a7a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a7a000640 1 -- 172.21.15.5:0/1022252275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002c060 con 0x560afde8b1a0 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a7a000640 1 -- 172.21.15.5:0/1022252275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a80013d00 con 0x560afde8b1a0 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a88007230 con 0x560afde8b1a0 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a7a000640 1 -- 172.21.15.5:0/1022252275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a80019040 con 0x560afde8b1a0 2024-04-25T10:04:40.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a9e7a5740 10 client.5991 fetch_fsmap learned FSMap version 26 2024-04-25T10:04:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x560afde5d490 con 0x560afde8b1a0 2024-04-25T10:04:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a7a000640 1 -- 172.21.15.5:0/1022252275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88007230 con 0x560afde8b1a0 2024-04-25T10:04:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.549+0000 7f0a7a000640 1 -- 172.21.15.5:0/1022252275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a80026040 con 0x560afde8b1a0 2024-04-25T10:04:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a9e7a5740 10 client.5991 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:04:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a9e7a5740 10 client.5991 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:04:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a9e7a5740 20 client.5991 populate_metadata read hostname 'smithi005' 2024-04-25T10:04:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afdef4360 0x560afdf09d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a9e7a5740 4 client.5991 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:04:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x560afddcb210 con 0x560afdef4360 2024-04-25T10:04:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.550+0000 7f0a99000640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afdef4360 0x560afdf09d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.551+0000 7f0a99000640 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afdef4360 0x560afdf09d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181586927a163834 server_cookie=1b8631b320399646 in_seq=0 out_seq=0 2024-04-25T10:04:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.551+0000 7f0a7a000640 10 client.5991 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.561 INFO:tasks.ceph.mds.d.smithi080.stderr:2024-04-25T10:04:40.554+0000 7f7ed0400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5934=0-4194304@1} caps={5934=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5934 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x555902e54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.553+0000 7f0a7a000640 1 -- 172.21.15.5:0/1022252275 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560afddcb210 con 0x560afdef4360 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.553+0000 7f0a7a000640 10 client.5991 handle_command_reply: tid=0 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.553+0000 7f0a9e7a5740 1 client.5991 shutdown 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 msgr2=0x560afdea5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afdea5920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0a80011530 tx=0x7f0a80014000 comp rx=0 tx=0).stop 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 shutdown_connections 2024-04-25T10:04:40.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x560afdef4360 0x560afdf09d70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560afdea8a40 0x560afdf09830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560afde8b1a0 0x560afdea5920 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 --2- 172.21.15.5:0/1022252275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560afde5a2a0 0x560afdea53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 >> 172.21.15.5:0/1022252275 conn(0x560afde60e80 msgr2=0x560afde60230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a7a000640 0 client.5991 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 shutdown_connections 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 1 -- 172.21.15.5:0/1022252275 wait complete. 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.554+0000 7f0a9e7a5740 20 client.5991 trim_cache size 0 max 16384 2024-04-25T10:04:40.563 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:04:40.564 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:04:40.564 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "45c9e64c-0357-4fec-a1a8-e9c45e73ae72", 2024-04-25T10:04:40.564 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:04:40.564 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:04:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a7406b500 msgr2=0x7f0a7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a7406b500 0x7f0a7406d9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001780 tx=0x7f0a88001b80 comp rx=0 tx=0).stop 2024-04-25T10:04:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 msgr2=0x7f0a940c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 0x7f0a940c94b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c033040 tx=0x7f0a8c009750 comp rx=0 tx=0).stop 2024-04-25T10:04:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 shutdown_connections 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a940c9050 0x7f0a940c94b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a94152210 0x7f0a94157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a7406b500 0x7f0a7406d9c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 --2- 172.21.15.5:0/1890030555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a7c004d40 0x7f0a94155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 >> 172.21.15.5:0/1890030555 conn(0x7f0a940726b0 msgr2=0x7f0a94158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 shutdown_connections 2024-04-25T10:04:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.555+0000 7f0a9ba00640 1 -- 172.21.15.5:0/1890030555 wait complete. 2024-04-25T10:04:40.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:40.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:04:40.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.573+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 7 ==== log(2 entries from seq 125 at 2024-04-25T10:04:39.618443+0000) v1 ==== 579+0+0 (secure 0 0 0) 0x7fde2c08c000 con 0x7fde34161620 2024-04-25T10:04:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.765+0000 7feb3a000640 1 Processor -- start 2024-04-25T10:04:40.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.766+0000 7feb3a000640 1 -- start start 2024-04-25T10:04:40.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.767+0000 7feb3a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb340b0bf0 0x7feb3415f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.767+0000 7feb3a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.767+0000 7feb3a000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb340b2d40 con 0x7feb3415f890 2024-04-25T10:04:40.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.767+0000 7feb3a000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb341601b0 con 0x7feb340b0bf0 2024-04-25T10:04:40.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.767+0000 7feb3a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340cc640 con 0x7feb340b2540 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb340b0bf0 0x7feb3415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb340b0bf0 0x7feb3415f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43254/0 (socket says 172.21.15.5:43254) 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 -- 172.21.15.5:0/1008392597 learned_addr learned my addr 172.21.15.5:0/1008392597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb39600640 1 --2- 172.21.15.5:0/1008392597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 -- 172.21.15.5:0/1008392597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 msgr2=0x7feb340ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 --2- 172.21.15.5:0/1008392597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 -- 172.21.15.5:0/1008392597 >> v1:172.21.15.80:6789/0 conn(0x7feb3415f890 legacy=0x7feb340b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.768+0000 7feb38c00640 1 -- 172.21.15.5:0/1008392597 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340b0fd0 con 0x7feb340b0bf0 2024-04-25T10:04:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.770+0000 7feb38c00640 1 --2- 172.21.15.5:0/1008392597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb340b0bf0 0x7feb3415f350 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feb24009e50 tx=0x7feb2402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164b48daf80b78d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.770+0000 7feb33400640 1 -- 172.21.15.5:0/1008392597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24036070 con 0x7feb340b0bf0 2024-04-25T10:04:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.770+0000 7feb33400640 1 -- 172.21.15.5:0/1008392597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24031600 con 0x7feb340b0bf0 2024-04-25T10:04:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.770+0000 7feb33400640 1 -- 172.21.15.5:0/1008392597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240319d0 con 0x7feb340b0bf0 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 -- 172.21.15.5:0/1008392597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb340b0bf0 msgr2=0x7feb3415f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 --2- 172.21.15.5:0/1008392597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb340b0bf0 0x7feb3415f350 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feb24009e50 tx=0x7feb2402a710 comp rx=0 tx=0).stop 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 -- 172.21.15.5:0/1008392597 shutdown_connections 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 --2- 172.21.15.5:0/1008392597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 --2- 172.21.15.5:0/1008392597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb340b0bf0 0x7feb3415f350 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 -- 172.21.15.5:0/1008392597 >> 172.21.15.5:0/1008392597 conn(0x7feb340d1470 msgr2=0x7feb340d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 -- 172.21.15.5:0/1008392597 shutdown_connections 2024-04-25T10:04:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.771+0000 7feb3a000640 1 -- 172.21.15.5:0/1008392597 wait complete. 2024-04-25T10:04:40.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.773+0000 7feb3a000640 1 Processor -- start 2024-04-25T10:04:40.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.773+0000 7feb3a000640 1 -- start start 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.774+0000 7feb3a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb340b0bf0 0x7feb340caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.774+0000 7feb3a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.774+0000 7feb3a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 0x7feb340b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.774+0000 7feb3a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb340cc2d0 con 0x7feb3415f890 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.774+0000 7feb3a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb340cbff0 con 0x7feb340b0bf0 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.774+0000 7feb3a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340cd010 con 0x7feb340b2540 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 0x7feb340b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 0x7feb340b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43264/0 (socket says 172.21.15.5:43264) 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 -- 172.21.15.5:0/858276147 learned_addr learned my addr 172.21.15.5:0/858276147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb38c00640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb340b0bf0 0x7feb340caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb33e00640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 -- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 msgr2=0x7feb340b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 -- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb340b0bf0 msgr2=0x7feb340caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb340b0bf0 0x7feb340caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.775+0000 7feb39600640 1 -- 172.21.15.5:0/858276147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340b4c80 con 0x7feb3415f890 2024-04-25T10:04:40.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.776+0000 7feb39600640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 0x7feb340b4570 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb2c004650 tx=0x7feb2c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.776+0000 7feb30c00640 1 -- 172.21.15.5:0/858276147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c017070 con 0x7feb3415f890 2024-04-25T10:04:40.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.776+0000 7feb30c00640 1 -- 172.21.15.5:0/858276147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c00c940 con 0x7feb3415f890 2024-04-25T10:04:40.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.776+0000 7feb30c00640 1 -- 172.21.15.5:0/858276147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00cd30 con 0x7feb3415f890 2024-04-25T10:04:40.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.776+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb340b4f60 con 0x7feb3415f890 2024-04-25T10:04:40.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.776+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb340b5520 con 0x7feb3415f890 2024-04-25T10:04:40.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.779+0000 7feb30c00640 1 -- 172.21.15.5:0/858276147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb2c01a070 con 0x7feb3415f890 2024-04-25T10:04:40.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.780+0000 7feb30c00640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feb1406b4d0 0x7feb1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.780+0000 7feb30c00640 1 -- 172.21.15.5:0/858276147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6130+0+0 (secure 0 0 0) 0x7feb2c08a7a0 con 0x7feb3415f890 2024-04-25T10:04:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.780+0000 7feb3e40f740 1 Processor -- start 2024-04-25T10:04:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.780+0000 7feb3e40f740 1 -- start start 2024-04-25T10:04:40.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.780+0000 7feb38c00640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feb1406b4d0 0x7feb1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9bbd70 0x56324f9bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9bc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9e6830 0x56324f9e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb3e40f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324f91fc70 con 0x56324f9f8440 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324f8c8cf0 con 0x56324f9e6830 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324f91bd10 con 0x56324f9bbd70 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.781+0000 7feb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9bc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9bc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43278/0 (socket says 172.21.15.5:43278) 2024-04-25T10:04:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 -- 172.21.15.5:0/3126261965 learned_addr learned my addr 172.21.15.5:0/3126261965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb38c00640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feb1406b4d0 0x7feb1406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb24033000 tx=0x7feb24009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb39600640 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9e6830 0x56324f9e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 -- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9bbd70 msgr2=0x56324f9bded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb38c00640 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9bbd70 0x56324f9bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9bbd70 0x56324f9bded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 -- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9e6830 msgr2=0x56324f9e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9e6830 0x56324f9e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 -- 172.21.15.5:0/3126261965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324f6fe7d0 con 0x56324f9f8440 2024-04-25T10:04:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.782+0000 7feb33e00640 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9bc150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7feb200014b0 tx=0x7feb20001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176a098807b38ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb29200640 1 -- 172.21.15.5:0/3126261965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20019070 con 0x56324f9f8440 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb29200640 1 -- 172.21.15.5:0/3126261965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200052e0 con 0x56324f9f8440 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb29200640 1 -- 172.21.15.5:0/3126261965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20003810 con 0x56324f9f8440 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 -- 172.21.15.5:0/3126261965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 msgr2=0x56324f9bc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9bc150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7feb200014b0 tx=0x7feb20001970 comp rx=0 tx=0).stop 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 -- 172.21.15.5:0/3126261965 shutdown_connections 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9e6830 0x56324f9e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9bc150 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 --2- 172.21.15.5:0/3126261965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9bbd70 0x56324f9bded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 -- 172.21.15.5:0/3126261965 >> 172.21.15.5:0/3126261965 conn(0x56324f983a30 msgr2=0x56324fa01830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.783+0000 7feb3e40f740 1 -- 172.21.15.5:0/3126261965 shutdown_connections 2024-04-25T10:04:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.784+0000 7feb3e40f740 1 -- 172.21.15.5:0/3126261965 wait complete. 2024-04-25T10:04:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.784+0000 7feb3e40f740 1 -- start start 2024-04-25T10:04:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.784+0000 7feb3e40f740 1 Processor -- start 2024-04-25T10:04:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.785+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9d5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.785+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 0x56324f9e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.785+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9f8440 0x56324f9e3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.785+0000 7feb3e40f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324f8be3a0 con 0x56324f9e6830 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.785+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324f9e4380 con 0x56324f9bbd70 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.785+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324f9f9e20 con 0x56324f9f8440 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb3e40f740 10 client.0 _ll_register_callbacks cb 0x56324f8dbb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 0x56324f9e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 0x56324f9e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43284/0 (socket says 172.21.15.5:43284) 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 -- 172.21.15.5:0/2282961360 learned_addr learned my addr 172.21.15.5:0/2282961360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb38c00640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9d5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb39600640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9f8440 0x56324f9e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 -- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9f8440 msgr2=0x56324f9e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9f8440 0x56324f9e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 -- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 msgr2=0x56324f9d5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9d5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb33e00640 1 -- 172.21.15.5:0/2282961360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324f6343f0 con 0x56324f9e6830 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.786+0000 7feb39600640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9f8440 0x56324f9e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb33e00640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 0x56324f9e76c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7feb20001480 tx=0x7feb20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f93345f7476163 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb1d600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb1d600640 1 -- 172.21.15.5:0/2282961360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb20022020 con 0x56324f9e6830 2024-04-25T10:04:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb1d600640 1 -- 172.21.15.5:0/2282961360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6130+0+0 (secure 0 0 0) 0x7feb20003ba0 con 0x56324f9e6830 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb200052e0 con 0x56324f9e6830 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb1d600640 1 -- 172.21.15.5:0/2282961360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb2002c050 con 0x56324f9e6830 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb3e40f740 10 client.6003 fetch_fsmap learned FSMap version 26 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.787+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56324f5f40f0 con 0x56324f9e6830 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb1d600640 1 -- 172.21.15.5:0/2282961360 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb200052e0 con 0x56324f9e6830 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb1d600640 1 -- 172.21.15.5:0/2282961360 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb2002f020 con 0x56324f9e6830 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb3e40f740 10 client.6003 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb3e40f740 10 client.6003 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb3e40f740 20 client.6003 populate_metadata read hostname 'smithi005' 2024-04-25T10:04:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb3e40f740 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa14de0 0x56324fa151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb3e40f740 4 client.6003 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.788+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56324f92d1b0 con 0x56324fa14de0 2024-04-25T10:04:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.789+0000 7feb38c00640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa14de0 0x56324fa151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.789+0000 7feb38c00640 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa14de0 0x56324fa151c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f085306a2e4a057d server_cookie=9a9d35dc2403d161 in_seq=0 out_seq=0 2024-04-25T10:04:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.789+0000 7feb1d600640 10 client.6003 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb1d600640 1 -- 172.21.15.5:0/2282961360 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56324f92d1b0 con 0x56324fa14de0 2024-04-25T10:04:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb1d600640 10 client.6003 handle_command_reply: tid=0 2024-04-25T10:04:40.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 client.6003 shutdown 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 msgr2=0x56324f9e76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 0x56324f9e76c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7feb20001480 tx=0x7feb20000bc0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 shutdown_connections 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa14de0 0x56324fa151c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9f8440 0x56324f9e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9e6830 0x56324f9e76c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 --2- 172.21.15.5:0/2282961360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9d5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 >> 172.21.15.5:0/2282961360 conn(0x56324f983a30 msgr2=0x56324f9e4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.791+0000 7feb1d600640 0 client.6003 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.792+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 shutdown_connections 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.792+0000 7feb3e40f740 1 -- 172.21.15.5:0/2282961360 wait complete. 2024-04-25T10:04:40.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.792+0000 7feb3e40f740 20 client.6003 trim_cache size 0 max 16384 2024-04-25T10:04:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.808+0000 7feb3e40f740 1 Processor -- start 2024-04-25T10:04:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.808+0000 7feb3e40f740 1 -- start start 2024-04-25T10:04:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.809+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.809+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324f9e7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.809+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.809+0000 7feb3e40f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb342002a0 con 0x56324f9f8440 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.809+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb341ff520 con 0x56324f9bbd70 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.809+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb340c92c0 con 0x56324f9e6830 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43296/0 (socket says 172.21.15.5:43296) 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 -- 172.21.15.5:0/962028250 learned_addr learned my addr 172.21.15.5:0/962028250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb38c00640 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 -- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 msgr2=0x56324f9e7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb33e00640 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324f9e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324f9e7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 -- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 msgr2=0x56324f9e3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 -- 172.21.15.5:0/962028250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56324f9e7e60 con 0x56324f9f8440 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb33e00640 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324f9e7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.810+0000 7feb39600640 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e7570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7feb2c088a80 tx=0x7feb2c05f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc3c4a3883a5832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb1fe00640 1 -- 172.21.15.5:0/962028250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c070020 con 0x56324f9f8440 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb1fe00640 1 -- 172.21.15.5:0/962028250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c0613e0 con 0x56324f9f8440 2024-04-25T10:04:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb1fe00640 1 -- 172.21.15.5:0/962028250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c021040 con 0x56324f9f8440 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 -- 172.21.15.5:0/962028250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 msgr2=0x56324f9e7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e7570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7feb2c088a80 tx=0x7feb2c05f040 comp rx=0 tx=0).stop 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 -- 172.21.15.5:0/962028250 shutdown_connections 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e7570 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324f9e7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 --2- 172.21.15.5:0/962028250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 -- 172.21.15.5:0/962028250 >> 172.21.15.5:0/962028250 conn(0x56324f983a30 msgr2=0x56324f9b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 -- 172.21.15.5:0/962028250 shutdown_connections 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.811+0000 7feb3e40f740 1 -- 172.21.15.5:0/962028250 wait complete. 2024-04-25T10:04:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.812+0000 7feb3e40f740 1 -- start start 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.812+0000 7feb3e40f740 1 Processor -- start 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324fa138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56324f8dd020 con 0x56324f9f8440 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb340c98c0 con 0x56324f9bbd70 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56324f9e4520 con 0x56324f9e6830 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.813+0000 7feb3e40f740 10 client.0 _ll_register_callbacks cb 0x7feb20017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:40.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43300/0 (socket says 172.21.15.5:43300) 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 -- 172.21.15.5:0/1052944381 learned_addr learned my addr 172.21.15.5:0/1052944381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb38c00640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb33e00640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324fa138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 -- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 msgr2=0x56324fa138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324fa138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 -- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 msgr2=0x56324f9e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 -- 172.21.15.5:0/1052944381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56324f9e7970 con 0x56324f9f8440 2024-04-25T10:04:40.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb33e00640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324fa138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb39600640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e21f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7feb2c004770 tx=0x7feb2c01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85eff7f40341dd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb1d600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.814+0000 7feb1d600640 1 -- 172.21.15.5:0/1052944381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c021050 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb1d600640 1 -- 172.21.15.5:0/1052944381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6130+0+0 (secure 0 0 0) 0x7feb2c058320 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb2c0613e0 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb1d600640 1 -- 172.21.15.5:0/1052944381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb2c070040 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 10 client.6009 fetch_fsmap learned FSMap version 26 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x56324f9e1d30 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb1d600640 1 -- 172.21.15.5:0/1052944381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c0613e0 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb1d600640 1 -- 172.21.15.5:0/1052944381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feb2c077ba0 con 0x56324f9f8440 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 10 client.6009 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 10 client.6009 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:04:40.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 20 client.6009 populate_metadata read hostname 'smithi005' 2024-04-25T10:04:40.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa65430 0x56324fa648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:40.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 4 client.6009 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:04:40.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.815+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56324f927500 con 0x56324fa65430 2024-04-25T10:04:40.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.816+0000 7feb38c00640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa65430 0x56324fa648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:40.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb38c00640 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa65430 0x56324fa648c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e280083d0072b4 server_cookie=2768f0d41792c501 in_seq=0 out_seq=0 2024-04-25T10:04:40.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb1d600640 10 client.6009 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb1d600640 1 -- 172.21.15.5:0/1052944381 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x56324f927500 con 0x56324fa65430 2024-04-25T10:04:40.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb1d600640 10 client.6009 handle_command_reply: tid=0 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb3e40f740 1 client.6009 shutdown 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 msgr2=0x56324f9e21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.817+0000 7feb3e40f740 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e21f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7feb2c004770 tx=0x7feb2c01e070 comp rx=0 tx=0).stop 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 shutdown_connections 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x56324fa65430 0x56324fa648c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56324f9f8440 0x56324f9e21f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56324f9e6830 0x56324fa138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 --2- 172.21.15.5:0/1052944381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56324f9bbd70 0x56324f9e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb1d600640 0 client.6009 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 >> 172.21.15.5:0/1052944381 conn(0x56324f983a30 msgr2=0x56324f9b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 shutdown_connections 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 1 -- 172.21.15.5:0/1052944381 wait complete. 2024-04-25T10:04:40.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.818+0000 7feb3e40f740 20 client.6009 trim_cache size 0 max 16384 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "45c9e64c-0357-4fec-a1a8-e9c45e73ae72": { 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "45c9e64c-0357-4fec-a1a8-e9c45e73ae72", 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:04:40.828 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:04:40.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feb1406b4d0 msgr2=0x7feb1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feb1406b4d0 0x7feb1406d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb24033000 tx=0x7feb24009000 comp rx=0 tx=0).stop 2024-04-25T10:04:40.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 msgr2=0x7feb340b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:40.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 0x7feb340b4570 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb2c004650 tx=0x7feb2c00ece0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 shutdown_connections 2024-04-25T10:04:40.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3415f890 0x7feb340b4570 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb340b2540 0x7feb340b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feb1406b4d0 0x7feb1406d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 --2- 172.21.15.5:0/858276147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb340b0bf0 0x7feb340caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:40.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 >> 172.21.15.5:0/858276147 conn(0x7feb340d1470 msgr2=0x7feb340ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:40.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 shutdown_connections 2024-04-25T10:04:40.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:40.819+0000 7feb3a000640 1 -- 172.21.15.5:0/858276147 wait complete. 2024-04-25T10:04:40.841 INFO:tasks.cephfs.filesystem:scrub status for tag:45c9e64c-0357-4fec-a1a8-e9c45e73ae72 - {'path': '/', 'tag': '45c9e64c-0357-4fec-a1a8-e9c45e73ae72', 'options': 'recursive,repair'} 2024-04-25T10:04:41.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:41.585+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 8 ==== log(7 entries from seq 1 at 2024-04-25T10:04:40.552934+0000) v1 ==== 1338+0+0 (secure 0 0 0) 0x7fde2c058160 con 0x7fde34161620 2024-04-25T10:04:41.591 INFO:teuthology.orchestra.run.smithi005.stdout: cluster: 2024-04-25T10:04:41.591 INFO:teuthology.orchestra.run.smithi005.stdout: id: 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:04:41.591 INFO:teuthology.orchestra.run.smithi005.stdout: health: HEALTH_OK 2024-04-25T10:04:41.591 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:41.591 INFO:teuthology.orchestra.run.smithi005.stdout: services: 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: mon: 3 daemons, quorum a,b,c (age 3m) 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: mgr: x(active, since 3m), standbys: y 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: data: 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: volumes: 1/1 healthy 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: objects: 24 objects, 579 KiB 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:04:41.592 INFO:teuthology.orchestra.run.smithi005.stdout: pgs: 97 active+clean 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout: io: 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout: client: 0 B/s wr, 0 op/s rd, 0 op/s wr 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:40.552934+0000 mds.d [INF] scrub queued for path: / 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:40.552949+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:40.552966+0000 mds.d [INF] scrub summary: active paths [/] 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:40.555657+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:40.555674+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T10:04:41.593 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:40.556469+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:04:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:04:45.731+0000 7fde2aa00640 1 -- 172.21.15.5:0/3550259842 <== mon.0 v2:172.21.15.5:3300/0 9 ==== log(1 entries from seq 7 at 2024-04-25T10:04:45.065974+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fde2c0583c0 con 0x7fde34161620 2024-04-25T10:04:45.737 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:04:45.065974+0000 mds.d [INF] scrub summary: idle 2024-04-25T10:04:53.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:53.940 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:10.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:10.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:05:11.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.064+0000 7f47c3200640 1 Processor -- start 2024-04-25T10:05:11.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.064+0000 7f47c3200640 1 -- start start 2024-04-25T10:05:11.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.066+0000 7f47c3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47c415ed20 0x7f47c4144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.066+0000 7f47c3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c415f3e0 con 0x7f47c4144e90 2024-04-25T10:05:11.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.066+0000 7f47c3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c415f680 con 0x7f47c4071710 2024-04-25T10:05:11.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.066+0000 7f47c3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c4144780 con 0x7f47c415ed20 2024-04-25T10:05:11.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.066+0000 7f47c1400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f47c4144e90 0x7f47c4071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34444/0 (socket says 172.21.15.5:34444) 2024-04-25T10:05:11.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.066+0000 7f47c1400640 1 -- 172.21.15.5:0/296268374 learned_addr learned my addr 172.21.15.5:0/296268374 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c1e00640 1 --2- 172.21.15.5:0/296268374 >> v2:172.21.15.80:3300/0 conn(0x7f47c415ed20 0x7f47c4144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913818330 0 0) 0x7f47c415f3e0 con 0x7f47c4144e90 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47a8003520 con 0x7f47c4144e90 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665523541 0 0) 0x7f47a8003520 con 0x7f47c4144e90 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 >> v2:172.21.15.80:3300/0 conn(0x7f47c415ed20 msgr2=0x7f47c4144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 --2- 172.21.15.5:0/296268374 >> v2:172.21.15.80:3300/0 conn(0x7f47c415ed20 0x7f47c4144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 >> v1:172.21.15.80:6789/0 conn(0x7f47c4071710 legacy=0x7f47c415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c415f960 con 0x7f47c4144e90 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f47b0001670 con 0x7f47c4144e90 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 >> v1:172.21.15.5:6789/0 conn(0x7f47c4144e90 legacy=0x7f47c4071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a8003a00 0x7f47a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47a8004800 0x7f47a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a80053c0 0x7f47a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4144780 con 0x7f47a8003a00 2024-04-25T10:05:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800c280 con 0x7f47a80053c0 2024-04-25T10:05:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.067+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a800c480 con 0x7f47a8004800 2024-04-25T10:05:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1e00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a8003a00 0x7f47a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1400640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47a8004800 0x7f47a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c2800640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a80053c0 0x7f47a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1e00640 1 -- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47a8004800 msgr2=0x7f47a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1e00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47a8004800 0x7f47a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1e00640 1 -- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a80053c0 msgr2=0x7f47a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1e00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a80053c0 0x7f47a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c1e00640 1 -- 172.21.15.5:0/296268374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8003520 con 0x7f47a8003a00 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c2800640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a80053c0 0x7f47a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.068+0000 7f47c2800640 1 -- 172.21.15.5:0/296268374 reap_dead start 2024-04-25T10:05:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.070+0000 7f47c1e00640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a8003a00 0x7f47a8003e00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f47b400ba30 tx=0x7f47a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d179e0c5eb67162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.070+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4030070 con 0x7f47a8003a00 2024-04-25T10:05:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.070+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b402ade0 con 0x7f47a8003a00 2024-04-25T10:05:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.070+0000 7f47c0a00640 1 -- 172.21.15.5:0/296268374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400b600 con 0x7f47a8003a00 2024-04-25T10:05:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 -- 172.21.15.5:0/296268374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a8003a00 msgr2=0x7f47a8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a8003a00 0x7f47a8003e00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f47b400ba30 tx=0x7f47a8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 -- 172.21.15.5:0/296268374 shutdown_connections 2024-04-25T10:05:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 --2- 172.21.15.5:0/296268374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a8003a00 0x7f47a8003e00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 -- 172.21.15.5:0/296268374 >> 172.21.15.5:0/296268374 conn(0x7f47c4066f20 msgr2=0x7f47c406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 -- 172.21.15.5:0/296268374 shutdown_connections 2024-04-25T10:05:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.071+0000 7f47c3200640 1 -- 172.21.15.5:0/296268374 wait complete. 2024-04-25T10:05:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.073+0000 7f47c3200640 1 Processor -- start 2024-04-25T10:05:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.073+0000 7f47c3200640 1 -- start start 2024-04-25T10:05:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a8003a00 0x7f47c40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 0x7f47c40c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47c413fc60 0x7f47c41400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4144950 con 0x7f47c40c9810 2024-04-25T10:05:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40cdd50 con 0x7f47a8003a00 2024-04-25T10:05:11.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4144ad0 con 0x7f47c413fc60 2024-04-25T10:05:11.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 0x7f47c40c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 0x7f47c40c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39668/0 (socket says 172.21.15.5:39668) 2024-04-25T10:05:11.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.074+0000 7f47c1400640 1 -- 172.21.15.5:0/3562324641 learned_addr learned my addr 172.21.15.5:0/3562324641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1e00640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a8003a00 0x7f47c40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c2800640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47c413fc60 0x7f47c41400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1400640 1 -- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47c413fc60 msgr2=0x7f47c41400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1400640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47c413fc60 0x7f47c41400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1400640 1 -- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a8003a00 msgr2=0x7f47c40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1400640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a8003a00 0x7f47c40cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1400640 1 -- 172.21.15.5:0/3562324641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40c52d0 con 0x7f47c40c9810 2024-04-25T10:05:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c2800640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47c413fc60 0x7f47c41400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1e00640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a8003a00 0x7f47c40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.075+0000 7f47c1400640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 0x7f47c40c9c90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f47b00010e0 tx=0x7f47b000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.076+0000 7f47ba000640 1 -- 172.21.15.5:0/3562324641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ee60 con 0x7f47c40c9810 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.076+0000 7f47ba000640 1 -- 172.21.15.5:0/3562324641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b0018070 con 0x7f47c40c9810 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.076+0000 7f47ba000640 1 -- 172.21.15.5:0/3562324641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000caf0 con 0x7f47c40c9810 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.076+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c40c55b0 con 0x7f47c40c9810 2024-04-25T10:05:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.076+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c40c5b70 con 0x7f47c40c9810 2024-04-25T10:05:11.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.079+0000 7f47ba000640 1 -- 172.21.15.5:0/3562324641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47b000cc90 con 0x7f47c40c9810 2024-04-25T10:05:11.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.080+0000 7f47ba000640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47a406b4d0 0x7f47a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.080+0000 7f47ba000640 1 -- 172.21.15.5:0/3562324641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f47b008a230 con 0x7f47c40c9810 2024-04-25T10:05:11.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.080+0000 7f47cb7ff740 1 Processor -- start 2024-04-25T10:05:11.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.081+0000 7f47c1e00640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47a406b4d0 0x7f47a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.081+0000 7f47cb7ff740 1 -- start start 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cea4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 0x557b5cea3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5cecdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ce06c70 con 0x557b5cea2d70 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cdafcf0 con 0x557b5cecd830 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b5ce02d10 con 0x557b5cedf440 2024-04-25T10:05:11.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47a406b4d0 0x7f47a406d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47b400ba00 tx=0x7f47b4030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cea4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cea4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39680/0 (socket says 172.21.15.5:39680) 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 -- 172.21.15.5:0/421854666 learned_addr learned my addr 172.21.15.5:0/421854666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c2800640 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5cecdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 -- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 msgr2=0x557b5cea3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 0x557b5cea3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 -- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 msgr2=0x557b5cecdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5cecdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.082+0000 7f47c1e00640 1 -- 172.21.15.5:0/421854666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b5cbe57d0 con 0x557b5cea2d70 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.083+0000 7f47c1e00640 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cea4ed0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f47b400bb20 tx=0x7f47b402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a51d5549bd39d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.083+0000 7f47ae600640 1 -- 172.21.15.5:0/421854666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b403fcc0 con 0x557b5cea2d70 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.083+0000 7f47ae600640 1 -- 172.21.15.5:0/421854666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b404e070 con 0x557b5cea2d70 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.083+0000 7f47ae600640 1 -- 172.21.15.5:0/421854666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b403fe60 con 0x557b5cea2d70 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 -- 172.21.15.5:0/421854666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 msgr2=0x557b5cea4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cea4ed0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f47b400bb20 tx=0x7f47b402c000 comp rx=0 tx=0).stop 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 -- 172.21.15.5:0/421854666 shutdown_connections 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5cecdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 0x557b5cea3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/421854666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cea4ed0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 -- 172.21.15.5:0/421854666 >> 172.21.15.5:0/421854666 conn(0x557b5ce6aa30 msgr2=0x557b5cee8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 -- 172.21.15.5:0/421854666 shutdown_connections 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.084+0000 7f47cb7ff740 1 -- 172.21.15.5:0/421854666 wait complete. 2024-04-25T10:05:11.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.085+0000 7f47cb7ff740 1 -- start start 2024-04-25T10:05:11.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.085+0000 7f47cb7ff740 1 Processor -- start 2024-04-25T10:05:11.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5cebfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cda53a0 con 0x557b5cea2d70 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cecb380 con 0x557b5cedf440 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b5cec09c0 con 0x557b5cecd830 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47cb7ff740 10 client.0 _ll_register_callbacks cb 0x557b5cdc2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39688/0 (socket says 172.21.15.5:39688) 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.086+0000 7f47c1e00640 1 -- 172.21.15.5:0/433009172 learned_addr learned my addr 172.21.15.5:0/433009172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c2800640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5cebfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1400640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1e00640 1 -- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 msgr2=0x557b5ceec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1e00640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1e00640 1 -- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 msgr2=0x557b5cebfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1e00640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5cebfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1e00640 1 -- 172.21.15.5:0/433009172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b5cb1b3f0 con 0x557b5cea2d70 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47c1e00640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceea5d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f47b400bb50 tx=0x7f47b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b1f559dfffbc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.087+0000 7f47a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47a2a00640 1 -- 172.21.15.5:0/433009172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b403dc80 con 0x557b5cea2d70 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47a2a00640 1 -- 172.21.15.5:0/433009172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f47b403de20 con 0x557b5cea2d70 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47b404e070 con 0x557b5cea2d70 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47a2a00640 1 -- 172.21.15.5:0/433009172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b4064050 con 0x557b5cea2d70 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 10 client.6021 fetch_fsmap learned FSMap version 26 2024-04-25T10:05:11.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x557b5cadb0f0 con 0x557b5cea2d70 2024-04-25T10:05:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47a2a00640 1 -- 172.21.15.5:0/433009172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b404e070 con 0x557b5cea2d70 2024-04-25T10:05:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47a2a00640 1 -- 172.21.15.5:0/433009172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47b4067020 con 0x557b5cea2d70 2024-04-25T10:05:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 10 client.6021 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:05:11.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 20 client.6021 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cee8e50 0x557b5cee9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 4 client.6021 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.088+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b5ce141b0 con 0x557b5cee8e50 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.089+0000 7f47c1400640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cee8e50 0x557b5cee9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.090+0000 7f47c1400640 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cee8e50 0x557b5cee9250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ac5e7b422de1b4 server_cookie=fcd826cbb258ab9 in_seq=0 out_seq=0 2024-04-25T10:05:11.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.090+0000 7f47a2a00640 10 client.6021 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:05:11.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47a2a00640 1 -- 172.21.15.5:0/433009172 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557b5ce141b0 con 0x557b5cee8e50 2024-04-25T10:05:11.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47a2a00640 10 client.6021 handle_command_reply: tid=0 2024-04-25T10:05:11.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 client.6021 shutdown 2024-04-25T10:05:11.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 msgr2=0x557b5ceea5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceea5d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f47b400bb50 tx=0x7f47b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 shutdown_connections 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cee8e50 0x557b5cee9250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5cebfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/433009172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceea5d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47a2a00640 0 client.6021 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 >> 172.21.15.5:0/433009172 conn(0x557b5ce6aa30 msgr2=0x557b5cecb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 shutdown_connections 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.092+0000 7f47cb7ff740 1 -- 172.21.15.5:0/433009172 wait complete. 2024-04-25T10:05:11.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.093+0000 7f47cb7ff740 20 client.6021 trim_cache size 0 max 16384 2024-04-25T10:05:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.109+0000 7f47cb7ff740 1 Processor -- start 2024-04-25T10:05:11.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.109+0000 7f47cb7ff740 1 -- start start 2024-04-25T10:05:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.110+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.110+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5ceeb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.110+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 0x557b5cee9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.110+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c41fc450 con 0x557b5cea2d70 2024-04-25T10:05:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.110+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b41b0 con 0x557b5cecd830 2024-04-25T10:05:11.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.110+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c40b3bb0 con 0x557b5cedf440 2024-04-25T10:05:11.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39692/0 (socket says 172.21.15.5:39692) 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 -- 172.21.15.5:0/2886196664 learned_addr learned my addr 172.21.15.5:0/2886196664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1400640 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5ceeb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 -- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 msgr2=0x557b5cee9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 0x557b5cee9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 -- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 msgr2=0x557b5ceeb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5ceeb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.111+0000 7f47c1e00640 1 -- 172.21.15.5:0/2886196664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b5cee2880 con 0x557b5cea2d70 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47c1e00640 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceead00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f47b402ae70 tx=0x7f47b4053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e20be26b6c5af4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47adc00640 1 -- 172.21.15.5:0/2886196664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b403dc80 con 0x557b5cea2d70 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47adc00640 1 -- 172.21.15.5:0/2886196664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b406d070 con 0x557b5cea2d70 2024-04-25T10:05:11.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47adc00640 1 -- 172.21.15.5:0/2886196664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400e000 con 0x557b5cea2d70 2024-04-25T10:05:11.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47cb7ff740 1 -- 172.21.15.5:0/2886196664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 msgr2=0x557b5ceead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceead00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f47b402ae70 tx=0x7f47b4053000 comp rx=0 tx=0).stop 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.112+0000 7f47cb7ff740 1 -- 172.21.15.5:0/2886196664 shutdown_connections 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cedf440 0x557b5cee9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cecd830 0x557b5ceeb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/2886196664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5ceead00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 -- 172.21.15.5:0/2886196664 >> 172.21.15.5:0/2886196664 conn(0x557b5ce6aa30 msgr2=0x557b5cea0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 -- 172.21.15.5:0/2886196664 shutdown_connections 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 -- 172.21.15.5:0/2886196664 wait complete. 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 -- start start 2024-04-25T10:05:11.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.113+0000 7f47cb7ff740 1 Processor -- start 2024-04-25T10:05:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.114+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cf55f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.114+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceeaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.114+0000 7f47cb7ff740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5ceeaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5ce9e2b0 con 0x557b5cea2d70 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b5cea4280 con 0x557b5cedf440 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47cb7ff740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b5cf0efe0 con 0x557b5cecd830 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47cb7ff740 10 client.0 _ll_register_callbacks cb 0x7f47a800c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cf55f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cf55f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39708/0 (socket says 172.21.15.5:39708) 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 -- 172.21.15.5:0/1950541183 learned_addr learned my addr 172.21.15.5:0/1950541183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 -- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 msgr2=0x557b5ceeaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceeaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 -- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 msgr2=0x557b5ceeaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5ceeaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 -- 172.21.15.5:0/1950541183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b5ceec7a0 con 0x557b5cea2d70 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47c1e00640 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cf55f40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47b402a970 tx=0x7f47b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0d1c8e4569484c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.115+0000 7f47a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:11.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47a2a00640 1 -- 172.21.15.5:0/1950541183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b405ca30 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47a2a00640 1 -- 172.21.15.5:0/1950541183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f47b4074cc0 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47b406d070 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47a2a00640 1 -- 172.21.15.5:0/1950541183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b4074ec0 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 10 client.6027 fetch_fsmap learned FSMap version 26 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x557b5cecac40 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47a2a00640 1 -- 172.21.15.5:0/1950541183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b406d070 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47a2a00640 1 -- 172.21.15.5:0/1950541183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47b4067040 con 0x557b5cea2d70 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 10 client.6027 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 10 client.6027 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 20 client.6027 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cecb7e0 0x557b5cee9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 4 client.6027 mds_command: new command op to 5365 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.116+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 --> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b5ce0e500 con 0x557b5cecb7e0 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.117+0000 7f47c1400640 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cecb7e0 0x557b5cee9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.118+0000 7f47c1400640 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cecb7e0 0x557b5cee9230 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cc90b8d130e7e0 server_cookie=20b0995d518790b6 in_seq=0 out_seq=0 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.118+0000 7f47a2a00640 10 client.6027 ms_handle_connect on v2:172.21.15.80:6834/2115515655 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47a2a00640 1 -- 172.21.15.5:0/1950541183 <== mds.0 v2:172.21.15.80:6834/2115515655 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557b5ce0e500 con 0x557b5cecb7e0 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47a2a00640 10 client.6027 handle_command_reply: tid=0 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 client.6027 shutdown 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 msgr2=0x557b5cf55f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cf55f40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47b402a970 tx=0x7f47b4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:11.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 shutdown_connections 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:6834/2115515655,v1:172.21.15.80:6836/2115515655] conn(0x557b5cecb7e0 0x557b5cee9230 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557b5cedf440 0x557b5ceeaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557b5cecd830 0x557b5ceeaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 --2- 172.21.15.5:0/1950541183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b5cea2d70 0x557b5cf55f40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47a2a00640 0 client.6027 ms_handle_reset on v2:172.21.15.80:6834/2115515655 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 >> 172.21.15.5:0/1950541183 conn(0x557b5ce6aa30 msgr2=0x557b5cec9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 shutdown_connections 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 1 -- 172.21.15.5:0/1950541183 wait complete. 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.119+0000 7f47cb7ff740 20 client.6027 trim_cache size 0 max 16384 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:05:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:05:11.128 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47a406b4d0 msgr2=0x7f47a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47a406b4d0 0x7f47a406d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47b400ba00 tx=0x7f47b4030000 comp rx=0 tx=0).stop 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 msgr2=0x7f47c40c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 0x7f47c40c9c90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f47b00010e0 tx=0x7f47b000ec10 comp rx=0 tx=0).stop 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 shutdown_connections 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47c413fc60 0x7f47c41400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47c40c9810 0x7f47c40c9c90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47a406b4d0 0x7f47a406d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 --2- 172.21.15.5:0/3562324641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47a8003a00 0x7f47c40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 >> 172.21.15.5:0/3562324641 conn(0x7f47c4066f20 msgr2=0x7f47c4071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 shutdown_connections 2024-04-25T10:05:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.120+0000 7f47c3200640 1 -- 172.21.15.5:0/3562324641 wait complete. 2024-04-25T10:05:11.145 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:05:11.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.459+0000 7f0f76600640 1 Processor -- start 2024-04-25T10:05:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.459+0000 7f0f76600640 1 -- start start 2024-04-25T10:05:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.460+0000 7f0f76600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f701529e0 0x7f0f700c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.460+0000 7f0f76600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f700c6410 0x7f0f700ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.460+0000 7f0f76600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7015bdc0 con 0x7f0f70071ed0 2024-04-25T10:05:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.460+0000 7f0f76600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f700b2340 con 0x7f0f701529e0 2024-04-25T10:05:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.460+0000 7f0f76600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015c060 con 0x7f0f700c6410 2024-04-25T10:05:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f701529e0 0x7f0f700c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f701529e0 0x7f0f700c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39710/0 (socket says 172.21.15.5:39710) 2024-04-25T10:05:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 -- 172.21.15.5:0/4282956740 learned_addr learned my addr 172.21.15.5:0/4282956740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 -- 172.21.15.5:0/4282956740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f700c6410 msgr2=0x7f0f700ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f66a00640 1 --2- 172.21.15.5:0/4282956740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f700c6410 0x7f0f700ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 --2- 172.21.15.5:0/4282956740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f700c6410 0x7f0f700ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 -- 172.21.15.5:0/4282956740 >> v1:172.21.15.80:6789/0 conn(0x7f0f70071ed0 legacy=0x7f0f700c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.461+0000 7f0f6f400640 1 -- 172.21.15.5:0/4282956740 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7015c340 con 0x7f0f701529e0 2024-04-25T10:05:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.462+0000 7f0f6f400640 1 --2- 172.21.15.5:0/4282956740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f701529e0 0x7f0f700c5ed0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009870 tx=0x7f0f6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc7b8127aef7aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f6ea00640 1 -- 172.21.15.5:0/4282956740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60037070 con 0x7f0f701529e0 2024-04-25T10:05:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f6ea00640 1 -- 172.21.15.5:0/4282956740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f60032440 con 0x7f0f701529e0 2024-04-25T10:05:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f6ea00640 1 -- 172.21.15.5:0/4282956740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60032810 con 0x7f0f701529e0 2024-04-25T10:05:11.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f76600640 1 -- 172.21.15.5:0/4282956740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f701529e0 msgr2=0x7f0f700c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f76600640 1 --2- 172.21.15.5:0/4282956740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f701529e0 0x7f0f700c5ed0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009870 tx=0x7f0f6002a210 comp rx=0 tx=0).stop 2024-04-25T10:05:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f76600640 1 -- 172.21.15.5:0/4282956740 shutdown_connections 2024-04-25T10:05:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f76600640 1 --2- 172.21.15.5:0/4282956740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f700c6410 0x7f0f700ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f76600640 1 --2- 172.21.15.5:0/4282956740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f701529e0 0x7f0f700c5ed0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.463+0000 7f0f76600640 1 -- 172.21.15.5:0/4282956740 >> 172.21.15.5:0/4282956740 conn(0x7f0f70071150 msgr2=0x7f0f70071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.464+0000 7f0f76600640 1 -- 172.21.15.5:0/4282956740 shutdown_connections 2024-04-25T10:05:11.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.464+0000 7f0f76600640 1 -- 172.21.15.5:0/4282956740 wait complete. 2024-04-25T10:05:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.465+0000 7f0f76600640 1 Processor -- start 2024-04-25T10:05:11.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.466+0000 7f0f76600640 1 -- start start 2024-04-25T10:05:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.466+0000 7f0f76600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f70071ed0 0x7f0f70158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f76600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 0x7f0f70153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f76600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f701544d0 0x7f0f70156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f76600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7006e160 con 0x7f0f700c6410 2024-04-25T10:05:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f76600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015c900 con 0x7f0f701544d0 2024-04-25T10:05:11.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f76600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7006e2e0 con 0x7f0f70071ed0 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 0x7f0f70153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 0x7f0f70153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39724/0 (socket says 172.21.15.5:39724) 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 -- 172.21.15.5:0/350964577 learned_addr learned my addr 172.21.15.5:0/350964577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f6fe00640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f701544d0 0x7f0f70156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f6f400640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f70071ed0 0x7f0f70158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 -- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f70071ed0 msgr2=0x7f0f70158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f70071ed0 0x7f0f70158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 -- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f701544d0 msgr2=0x7f0f70156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f701544d0 0x7f0f70156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f66a00640 1 -- 172.21.15.5:0/350964577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700b2970 con 0x7f0f700c6410 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.467+0000 7f0f6fe00640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f701544d0 0x7f0f70156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.468+0000 7f0f66a00640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 0x7f0f70153f90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800c7f0 tx=0x7f0f5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.468+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58015070 con 0x7f0f700c6410 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.468+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f58004220 con 0x7f0f700c6410 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.468+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58003610 con 0x7f0f700c6410 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.468+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f700b2c50 con 0x7f0f700c6410 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.468+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f700b3210 con 0x7f0f700c6410 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.469+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f70068fd0 con 0x7f0f700c6410 2024-04-25T10:05:11.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.470+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f580039c0 con 0x7f0f700c6410 2024-04-25T10:05:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.471+0000 7f0f67e00640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f4c06b4d0 0x7f0f4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.471+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0f5808a790 con 0x7f0f700c6410 2024-04-25T10:05:11.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.472+0000 7f0f6f400640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f4c06b4d0 0x7f0f4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.473+0000 7f0f6f400640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f4c06b4d0 0x7f0f4c06d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0f60034000 tx=0x7f0f6002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.476+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f580593c0 con 0x7f0f700c6410 2024-04-25T10:05:11.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.713+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f700b1810 con 0x7f0f700c6410 2024-04-25T10:05:11.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.714+0000 7f0f67e00640 1 -- 172.21.15.5:0/350964577 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f0f58058cc0 con 0x7f0f700c6410 2024-04-25T10:05:11.721 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:11.721 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:11.721 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:05:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f4c06b4d0 msgr2=0x7f0f4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f4c06b4d0 0x7f0f4c06d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0f60034000 tx=0x7f0f6002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 msgr2=0x7f0f70153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 0x7f0f70153f90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800c7f0 tx=0x7f0f5800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 shutdown_connections 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f4c06b4d0 0x7f0f4c06d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f701544d0 0x7f0f70156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f700c6410 0x7f0f70153f90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 --2- 172.21.15.5:0/350964577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f70071ed0 0x7f0f70158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 >> 172.21.15.5:0/350964577 conn(0x7f0f70071150 msgr2=0x7f0f701598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 shutdown_connections 2024-04-25T10:05:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:11.717+0000 7f0f76600640 1 -- 172.21.15.5:0/350964577 wait complete. 2024-04-25T10:05:11.742 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:05:11.980 INFO:teuthology.orchestra.run.smithi080.stdout:{"message":"","return_code":0} 2024-04-25T10:05:11.980 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:05:11.980 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:05:12.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.095+0000 7f0b3367af40 1 Processor -- start 2024-04-25T10:05:12.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.096+0000 7f0b3367af40 1 -- start start 2024-04-25T10:05:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.096+0000 7f0b3367af40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b2697817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.096+0000 7f0b3367af40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b269763a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b3367af40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697ed3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b3367af40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2696dedd0 con 0x7f0b14002870 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b3367af40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2696e2d30 con 0x55b269763f50 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b3367af40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2697672e0 con 0x55b26980aa40 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b2697817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b2697817b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39742/0 (socket says 172.21.15.5:39742) 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 -- 172.21.15.5:0/3176096632 learned_addr learned my addr 172.21.15.5:0/3176096632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 -- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 msgr2=0x55b269763a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b31200640 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697ed3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b27e00640 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b269763a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b269763a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 -- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 msgr2=0x55b2697ed3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697ed3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.097+0000 7f0b30800640 1 -- 172.21.15.5:0/3176096632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b26980a640 con 0x7f0b14002870 2024-04-25T10:05:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.098+0000 7f0b30800640 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b2697817b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d290 tx=0x7f0b2000d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1423bbc59b6aade6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.098+0000 7f0b2f400640 1 -- 172.21.15.5:0/3176096632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20003110 con 0x7f0b14002870 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.098+0000 7f0b2f400640 1 -- 172.21.15.5:0/3176096632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b200032b0 con 0x7f0b14002870 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.098+0000 7f0b2f400640 1 -- 172.21.15.5:0/3176096632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20001600 con 0x7f0b14002870 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 -- 172.21.15.5:0/3176096632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 msgr2=0x55b2697817b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b2697817b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d290 tx=0x7f0b2000d750 comp rx=0 tx=0).stop 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 -- 172.21.15.5:0/3176096632 shutdown_connections 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697ed3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b269763a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 --2- 172.21.15.5:0/3176096632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b2697817b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 -- 172.21.15.5:0/3176096632 >> 172.21.15.5:0/3176096632 conn(0x55b269764ed0 msgr2=0x55b269782210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 -- 172.21.15.5:0/3176096632 shutdown_connections 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.099+0000 7f0b3367af40 1 -- 172.21.15.5:0/3176096632 wait complete. 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.100+0000 7f0b3367af40 1 Processor -- start 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.100+0000 7f0b3367af40 1 -- start start 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b3367af40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b3367af40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b26980ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b3367af40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b269810c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b3367af40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b269780b00 con 0x7f0b14002870 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b3367af40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b269764390 con 0x55b269763f50 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b3367af40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b2697ec9f0 con 0x55b26980aa40 2024-04-25T10:05:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b31200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b269810c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b31200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b269810c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39754/0 (socket says 172.21.15.5:39754) 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b31200640 1 -- 172.21.15.5:0/1932901299 learned_addr learned my addr 172.21.15.5:0/1932901299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b30800640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.101+0000 7f0b27e00640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b26980ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b31200640 1 -- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 msgr2=0x55b26980ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b31200640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b26980aa40 0x55b26980ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b31200640 1 -- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 msgr2=0x55b2697eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b31200640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b269763f50 0x55b2697eb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b31200640 1 -- 172.21.15.5:0/1932901299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b26980f7c0 con 0x7f0b14002870 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b31200640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b269810c70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0b14002fd0 tx=0x7f0b2800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b2e000640 1 -- 172.21.15.5:0/1932901299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280062d0 con 0x7f0b14002870 2024-04-25T10:05:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b2e000640 1 -- 172.21.15.5:0/1932901299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28003450 con 0x7f0b14002870 2024-04-25T10:05:12.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b2e000640 1 -- 172.21.15.5:0/1932901299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003860 con 0x7f0b14002870 2024-04-25T10:05:12.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b26980faa0 con 0x7f0b14002870 2024-04-25T10:05:12.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.102+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b269810060 con 0x7f0b14002870 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.105+0000 7f0b2e000640 1 -- 172.21.15.5:0/1932901299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b28003a00 con 0x7f0b14002870 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.106+0000 7f0b2e000640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b0c06a050 0x7f0b0c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.106+0000 7f0b2e000640 1 -- 172.21.15.5:0/1932901299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0b280884c0 con 0x7f0b14002870 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.106+0000 7f0b3367af40 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55b26980f320 0x55b2697ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.106+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53) v9 -- 0x55b2697ee840 con 0x55b26980f320 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.107+0000 7f0b30800640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b0c06a050 0x7f0b0c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.107+0000 7f0b27e00640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55b26980f320 0x55b2697ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.108+0000 7f0b30800640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b0c06a050 0x7f0b0c06c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d790 tx=0x7f0b20002000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.108+0000 7f0b27e00640 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55b26980f320 0x55b2697ee300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b27e00640 1 -- 172.21.15.5:0/1932901299 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f0b180508a0 con 0x55b26980f320 2024-04-25T10:05:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55b26980f320 msgr2=0x55b2697ee300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55b26980f320 0x55b2697ee300 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b0c06a050 msgr2=0x7f0b0c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b0c06a050 0x7f0b0c06c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0b2000d790 tx=0x7f0b20002000 comp rx=0 tx=0).stop 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 msgr2=0x55b269810c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 --2- 172.21.15.5:0/1932901299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b14002870 0x55b269810c70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0b14002fd0 tx=0x7f0b2800c5e0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b31200640 1 -- 172.21.15.5:0/1932901299 reap_dead start 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 shutdown_connections 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 >> 172.21.15.5:0/1932901299 conn(0x55b269764ed0 msgr2=0x55b269782060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 shutdown_connections 2024-04-25T10:05:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.109+0000 7f0b3367af40 1 -- 172.21.15.5:0/1932901299 wait complete. 2024-04-25T10:05:12.120 INFO:teuthology.orchestra.run.smithi005.stdout:Yalphaparent_a 2024-04-25T10:05:12.120 DEBUG:teuthology.orchestra.run.smithi005:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: "ancestors": [ 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1099511627776, 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "alpha", 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4 2024-04-25T10:05:12.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1, 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "parent_a", 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 7 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 8, 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: "old_pools": [] 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:12.219 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:12.225 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:12.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:12.323 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:12.430 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:12.430 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:05:12.430 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.491 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:05:12.491 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:12.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.737+0000 7fcfd7200640 1 Processor -- start 2024-04-25T10:05:12.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.737+0000 7fcfd7200640 1 -- start start 2024-04-25T10:05:12.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.738+0000 7fcfd7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcfd8161200 0x7fcfd80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.738+0000 7fcfd7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcfd80d07d0 0x7fcfd80b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.738+0000 7fcfd7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd80cf410 con 0x7fcfd80d2380 2024-04-25T10:05:12.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.738+0000 7fcfd7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80d27c0 con 0x7fcfd80d07d0 2024-04-25T10:05:12.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.738+0000 7fcfd7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd80b28b0 con 0x7fcfd8161200 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd6800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcfd80d07d0 0x7fcfd80b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcfd80d2380 0x7fcfd80d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34452/0 (socket says 172.21.15.5:34452) 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd5400640 1 -- 172.21.15.5:0/3531928761 learned_addr learned my addr 172.21.15.5:0/3531928761 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd5e00640 1 --2- 172.21.15.5:0/3531928761 >> v2:172.21.15.80:3300/0 conn(0x7fcfd8161200 0x7fcfd80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd6800640 1 -- 172.21.15.5:0/3531928761 >> v2:172.21.15.80:3300/0 conn(0x7fcfd8161200 msgr2=0x7fcfd80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd6800640 1 --2- 172.21.15.5:0/3531928761 >> v2:172.21.15.80:3300/0 conn(0x7fcfd8161200 0x7fcfd80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.739+0000 7fcfd6800640 1 -- 172.21.15.5:0/3531928761 >> v1:172.21.15.5:6789/0 conn(0x7fcfd80d2380 legacy=0x7fcfd80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.740+0000 7fcfd6800640 1 -- 172.21.15.5:0/3531928761 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80cf6f0 con 0x7fcfd80d07d0 2024-04-25T10:05:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.740+0000 7fcfd4a00640 1 -- 172.21.15.5:0/3531928761 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398114448 0 0) 0x7fcfd80cf410 con 0x7fcfd80d2380 2024-04-25T10:05:12.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.741+0000 7fcfd6800640 1 --2- 172.21.15.5:0/3531928761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80b2f80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcfd808bed0 tx=0x7fcfd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0f1e3d4edb0358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.741+0000 7fcfd4a00640 1 -- 172.21.15.5:0/3531928761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0037070 con 0x7fcfd80d07d0 2024-04-25T10:05:12.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.741+0000 7fcfd4a00640 1 -- 172.21.15.5:0/3531928761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0032440 con 0x7fcfd80d07d0 2024-04-25T10:05:12.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.741+0000 7fcfd4a00640 1 -- 172.21.15.5:0/3531928761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0032810 con 0x7fcfd80d07d0 2024-04-25T10:05:12.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.741+0000 7fcfd7200640 1 -- 172.21.15.5:0/3531928761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d07d0 msgr2=0x7fcfd80b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.741+0000 7fcfd7200640 1 --2- 172.21.15.5:0/3531928761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80b2f80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcfd808bed0 tx=0x7fcfd002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.742+0000 7fcfd7200640 1 -- 172.21.15.5:0/3531928761 shutdown_connections 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.742+0000 7fcfd7200640 1 --2- 172.21.15.5:0/3531928761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80b2f80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.742+0000 7fcfd7200640 1 --2- 172.21.15.5:0/3531928761 >> v2:172.21.15.80:3300/0 conn(0x7fcfd8161200 0x7fcfd80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.742+0000 7fcfd7200640 1 -- 172.21.15.5:0/3531928761 >> 172.21.15.5:0/3531928761 conn(0x7fcfd808af80 msgr2=0x7fcfd80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.742+0000 7fcfd7200640 1 -- 172.21.15.5:0/3531928761 shutdown_connections 2024-04-25T10:05:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.742+0000 7fcfd7200640 1 -- 172.21.15.5:0/3531928761 wait complete. 2024-04-25T10:05:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.744+0000 7fcfd7200640 1 Processor -- start 2024-04-25T10:05:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.744+0000 7fcfd7200640 1 -- start start 2024-04-25T10:05:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 0x7fcfd80cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfd8161200 0x7fcfd8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80b34c0 con 0x7fcfd80d2380 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80d42b0 con 0x7fcfd80d07d0 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd80b3640 con 0x7fcfd8161200 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 0x7fcfd80cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 0x7fcfd80cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39780/0 (socket says 172.21.15.5:39780) 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 -- 172.21.15.5:0/11134746 learned_addr learned my addr 172.21.15.5:0/11134746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5e00640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd6800640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfd8161200 0x7fcfd8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 -- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfd8161200 msgr2=0x7fcfd8156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfd8161200 0x7fcfd8156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 -- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfd80d07d0 msgr2=0x7fcfd80c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5400640 1 -- 172.21.15.5:0/11134746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8157460 con 0x7fcfd80d2380 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.745+0000 7fcfd5e00640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.746+0000 7fcfd5400640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 0x7fcfd80cc920 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c4d0 tx=0x7fcfc000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.746+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0018070 con 0x7fcfd80d2380 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.746+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc00074e0 con 0x7fcfd80d2380 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.746+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd80cd090 con 0x7fcfd80d2380 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.746+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0004a20 con 0x7fcfd80d2380 2024-04-25T10:05:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.746+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd80cab90 con 0x7fcfd80d2380 2024-04-25T10:05:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.747+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd8068f60 con 0x7fcfd80d2380 2024-04-25T10:05:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.748+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfc001b070 con 0x7fcfd80d2380 2024-04-25T10:05:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.749+0000 7fcfce000640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfb406b4d0 0x7fcfb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.749+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcfc0089720 con 0x7fcfd80d2380 2024-04-25T10:05:12.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.750+0000 7fcfd5e00640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfb406b4d0 0x7fcfb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.751+0000 7fcfd5e00640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfb406b4d0 0x7fcfb406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009f50 tx=0x7fcfc800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.754+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfc00583e0 con 0x7fcfd80d2380 2024-04-25T10:05:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.961+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfd80b2470 con 0x7fcfd80d2380 2024-04-25T10:05:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.962+0000 7fcfce000640 1 -- 172.21.15.5:0/11134746 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16823 (secure 0 0 0) 0x7fcfc0087e20 con 0x7fcfd80d2380 2024-04-25T10:05:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:12.969 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:41.582074+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2b","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfb406b4d0 msgr2=0x7fcfb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfb406b4d0 0x7fcfb406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009f50 tx=0x7fcfc800b040 comp rx=0 tx=0).stop 2024-04-25T10:05:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 msgr2=0x7fcfd80cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 0x7fcfd80cc920 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c4d0 tx=0x7fcfc000c990 comp rx=0 tx=0).stop 2024-04-25T10:05:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 shutdown_connections 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfd8161200 0x7fcfd8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd80d2380 0x7fcfd80cc920 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfb406b4d0 0x7fcfb406d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 --2- 172.21.15.5:0/11134746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfd80d07d0 0x7fcfd80c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 >> 172.21.15.5:0/11134746 conn(0x7fcfd808af80 msgr2=0x7fcfd8160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 shutdown_connections 2024-04-25T10:05:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:12.965+0000 7fcfd7200640 1 -- 172.21.15.5:0/11134746 wait complete. 2024-04-25T10:05:12.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:22.770291+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:23.937149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:12.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.205+0000 7f8b52800640 1 Processor -- start 2024-04-25T10:05:13.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.205+0000 7f8b52800640 1 -- start start 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.206+0000 7f8b52800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b4c0b48c0 0x7f8b4c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b52800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8b4c161d90 0x7f8b4c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b52800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c1606e0 con 0x7f8b4c0d3f90 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b52800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c1609c0 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b52800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c160ca0 con 0x7f8b4c161d90 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b51400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b4c0b48c0 0x7f8b4c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b51400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b4c0b48c0 0x7f8b4c15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39784/0 (socket says 172.21.15.5:39784) 2024-04-25T10:05:13.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b51400640 1 -- 172.21.15.5:0/657051024 learned_addr learned my addr 172.21.15.5:0/657051024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.207+0000 7f8b51e00640 1 --2- 172.21.15.5:0/657051024 >> v2:172.21.15.80:3300/0 conn(0x7f8b4c161d90 0x7f8b4c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.208+0000 7f8b51400640 1 -- 172.21.15.5:0/657051024 >> v2:172.21.15.80:3300/0 conn(0x7f8b4c161d90 msgr2=0x7f8b4c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.208+0000 7f8b51400640 1 --2- 172.21.15.5:0/657051024 >> v2:172.21.15.80:3300/0 conn(0x7f8b4c161d90 0x7f8b4c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.208+0000 7f8b51400640 1 -- 172.21.15.5:0/657051024 >> v1:172.21.15.80:6789/0 conn(0x7f8b4c0d3f90 legacy=0x7f8b4c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.208+0000 7f8b51400640 1 -- 172.21.15.5:0/657051024 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c160f80 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.209+0000 7f8b51400640 1 --2- 172.21.15.5:0/657051024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c15fe60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c08d2e0 tx=0x7f8b4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abaa1387a7449f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.209+0000 7f8b4be00640 1 -- 172.21.15.5:0/657051024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40036070 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.209+0000 7f8b4be00640 1 -- 172.21.15.5:0/657051024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40031600 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.209+0000 7f8b4be00640 1 -- 172.21.15.5:0/657051024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400319d0 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 -- 172.21.15.5:0/657051024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 msgr2=0x7f8b4c15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 --2- 172.21.15.5:0/657051024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c15fe60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c08d2e0 tx=0x7f8b4002a660 comp rx=0 tx=0).stop 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 -- 172.21.15.5:0/657051024 shutdown_connections 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 --2- 172.21.15.5:0/657051024 >> v2:172.21.15.80:3300/0 conn(0x7f8b4c161d90 0x7f8b4c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 --2- 172.21.15.5:0/657051024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c15fe60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 -- 172.21.15.5:0/657051024 >> 172.21.15.5:0/657051024 conn(0x7f8b4c08af80 msgr2=0x7f8b4c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 -- 172.21.15.5:0/657051024 shutdown_connections 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.210+0000 7f8b52800640 1 -- 172.21.15.5:0/657051024 wait complete. 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.212+0000 7f8b52800640 1 Processor -- start 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.212+0000 7f8b52800640 1 -- start start 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b52800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b4c0d3f90 0x7f8b4c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b52800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b4c161d90 0x7f8b4c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b52800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c1603a0 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b52800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b4ca0 con 0x7f8b4c161d90 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b52800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b30b0 con 0x7f8b4c0d3f90 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b51400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b51400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c14ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39786/0 (socket says 172.21.15.5:39786) 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.213+0000 7f8b51400640 1 -- 172.21.15.5:0/3929427843 learned_addr learned my addr 172.21.15.5:0/3929427843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b50a00640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b4c0d3f90 0x7f8b4c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51400640 1 -- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b4c0d3f90 msgr2=0x7f8b4c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51e00640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b4c161d90 0x7f8b4c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51400640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b4c0d3f90 0x7f8b4c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51400640 1 -- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b4c161d90 msgr2=0x7f8b4c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51400640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b4c161d90 0x7f8b4c15caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51400640 1 -- 172.21.15.5:0/3929427843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c14f660 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b50a00640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b4c0d3f90 0x7f8b4c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51e00640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b4c161d90 0x7f8b4c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b51400640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c14ebe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8b40034040 tx=0x7f8b40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.214+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40036050 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.215+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40042070 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.215+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40009510 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.215+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c14f940 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.215+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c14ff00 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.216+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c068f60 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.217+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b40007c50 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.218+0000 7f8b49600640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b2c06b4d0 0x7f8b2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.218+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f8b400ab300 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.219+0000 7f8b50a00640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b2c06b4d0 0x7f8b2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.220+0000 7f8b50a00640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b2c06b4d0 0x7f8b2c06d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c15c370 tx=0x7f8b38009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.222+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b4007a0e0 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.420+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b4c0d0dd0 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.421+0000 7f8b49600640 1 -- 172.21.15.5:0/3929427843 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b400799e0 con 0x7f8b4c0b48c0 2024-04-25T10:05:13.427 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:05:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b2c06b4d0 msgr2=0x7f8b2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b2c06b4d0 0x7f8b2c06d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c15c370 tx=0x7f8b38009040 comp rx=0 tx=0).stop 2024-04-25T10:05:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 msgr2=0x7f8b4c14ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c14ebe0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8b40034040 tx=0x7f8b40004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:13.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 shutdown_connections 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b2c06b4d0 0x7f8b2c06d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b4c161d90 0x7f8b4c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b4c0d3f90 0x7f8b4c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 --2- 172.21.15.5:0/3929427843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4c0b48c0 0x7f8b4c14ebe0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 >> 172.21.15.5:0/3929427843 conn(0x7f8b4c08af80 msgr2=0x7f8b4c15dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 shutdown_connections 2024-04-25T10:05:13.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.424+0000 7f8b52800640 1 -- 172.21.15.5:0/3929427843 wait complete. 2024-04-25T10:05:13.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.660+0000 7fcb3b000640 1 Processor -- start 2024-04-25T10:05:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.660+0000 7fcb3b000640 1 -- start start 2024-04-25T10:05:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.661+0000 7fcb3b000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb3415deb0 0x7fcb340b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.661+0000 7fcb3b000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcb340ce030 0x7fcb340b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.661+0000 7fcb3b000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb340ce4c0 con 0x7fcb340b6f20 2024-04-25T10:05:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.661+0000 7fcb3b000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb340d1a40 con 0x7fcb3415deb0 2024-04-25T10:05:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.661+0000 7fcb3b000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb340d1d20 con 0x7fcb340ce030 2024-04-25T10:05:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.661+0000 7fcb33e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb3415deb0 0x7fcb340b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcb3415deb0 0x7fcb340b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39788/0 (socket says 172.21.15.5:39788) 2024-04-25T10:05:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33e00640 1 -- 172.21.15.5:0/2982610876 learned_addr learned my addr 172.21.15.5:0/2982610876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33400640 1 --2- 172.21.15.5:0/2982610876 >> v2:172.21.15.80:3300/0 conn(0x7fcb340ce030 0x7fcb340b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33e00640 1 -- 172.21.15.5:0/2982610876 >> v2:172.21.15.80:3300/0 conn(0x7fcb340ce030 msgr2=0x7fcb340b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33e00640 1 --2- 172.21.15.5:0/2982610876 >> v2:172.21.15.80:3300/0 conn(0x7fcb340ce030 0x7fcb340b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33e00640 1 -- 172.21.15.5:0/2982610876 >> v1:172.21.15.80:6789/0 conn(0x7fcb340b6f20 legacy=0x7fcb340d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.662+0000 7fcb33e00640 1 -- 172.21.15.5:0/2982610876 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb340d2000 con 0x7fcb3415deb0 2024-04-25T10:05:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.663+0000 7fcb33e00640 1 --2- 172.21.15.5:0/2982610876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb3415deb0 0x7fcb340b2c30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcb340d48d0 tx=0x7fcb2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc9800eeeca244c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb32a00640 1 -- 172.21.15.5:0/2982610876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24037070 con 0x7fcb3415deb0 2024-04-25T10:05:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb32a00640 1 -- 172.21.15.5:0/2982610876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24032440 con 0x7fcb3415deb0 2024-04-25T10:05:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb32a00640 1 -- 172.21.15.5:0/2982610876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24032810 con 0x7fcb3415deb0 2024-04-25T10:05:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb3b000640 1 -- 172.21.15.5:0/2982610876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb3415deb0 msgr2=0x7fcb340b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb3b000640 1 --2- 172.21.15.5:0/2982610876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb3415deb0 0x7fcb340b2c30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcb340d48d0 tx=0x7fcb2402a230 comp rx=0 tx=0).stop 2024-04-25T10:05:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb3b000640 1 -- 172.21.15.5:0/2982610876 shutdown_connections 2024-04-25T10:05:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb3b000640 1 --2- 172.21.15.5:0/2982610876 >> v2:172.21.15.80:3300/0 conn(0x7fcb340ce030 0x7fcb340b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb3b000640 1 --2- 172.21.15.5:0/2982610876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb3415deb0 0x7fcb340b2c30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.664+0000 7fcb3b000640 1 -- 172.21.15.5:0/2982610876 >> 172.21.15.5:0/2982610876 conn(0x7fcb340726b0 msgr2=0x7fcb34072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.665+0000 7fcb3b000640 1 -- 172.21.15.5:0/2982610876 shutdown_connections 2024-04-25T10:05:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.665+0000 7fcb3b000640 1 -- 172.21.15.5:0/2982610876 wait complete. 2024-04-25T10:05:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.666+0000 7fcb3b000640 1 Processor -- start 2024-04-25T10:05:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.666+0000 7fcb3b000640 1 -- start start 2024-04-25T10:05:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.667+0000 7fcb3b000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 0x7fcb34159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.667+0000 7fcb3b000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb340ce030 0x7fcb340cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.667+0000 7fcb3b000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb3415deb0 0x7fcb340cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.667+0000 7fcb3b000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340d3e70 con 0x7fcb340b6f20 2024-04-25T10:05:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.667+0000 7fcb3b000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340d25c0 con 0x7fcb340ce030 2024-04-25T10:05:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.667+0000 7fcb3b000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340d3ff0 con 0x7fcb3415deb0 2024-04-25T10:05:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 0x7fcb34159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 0x7fcb34159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39802/0 (socket says 172.21.15.5:39802) 2024-04-25T10:05:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 -- 172.21.15.5:0/1306336788 learned_addr learned my addr 172.21.15.5:0/1306336788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33400640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb340ce030 0x7fcb340cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 -- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb3415deb0 msgr2=0x7fcb340cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb3415deb0 0x7fcb340cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 -- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb340ce030 msgr2=0x7fcb340cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb340ce030 0x7fcb340cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.668+0000 7fcb33e00640 1 -- 172.21.15.5:0/1306336788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb340cd700 con 0x7fcb340b6f20 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.669+0000 7fcb33e00640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 0x7fcb34159e30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcb24000c00 tx=0x7fcb240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.669+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24041020 con 0x7fcb340b6f20 2024-04-25T10:05:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.669+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240306c0 con 0x7fcb340b6f20 2024-04-25T10:05:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.669+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24037040 con 0x7fcb340b6f20 2024-04-25T10:05:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.669+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb340cd9e0 con 0x7fcb340b6f20 2024-04-25T10:05:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.669+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb341f4c70 con 0x7fcb340b6f20 2024-04-25T10:05:13.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.670+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34068f60 con 0x7fcb340b6f20 2024-04-25T10:05:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.671+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb24044020 con 0x7fcb340b6f20 2024-04-25T10:05:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.672+0000 7fcb2be00640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb1006b4d0 0x7fcb1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.672+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcb240aac80 con 0x7fcb340b6f20 2024-04-25T10:05:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.673+0000 7fcb33400640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb1006b4d0 0x7fcb1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.674+0000 7fcb33400640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb1006b4d0 0x7fcb1006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b5c00 tx=0x7fcb1c00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.678+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb2404b030 con 0x7fcb340b6f20 2024-04-25T10:05:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.879+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb340cdbb0 con 0x7fcb340b6f20 2024-04-25T10:05:13.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.879+0000 7fcb2be00640 1 -- 172.21.15.5:0/1306336788 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb240798b0 con 0x7fcb340b6f20 2024-04-25T10:05:13.885 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:05:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb1006b4d0 msgr2=0x7fcb1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb1006b4d0 0x7fcb1006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fcb340b5c00 tx=0x7fcb1c00b040 comp rx=0 tx=0).stop 2024-04-25T10:05:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 msgr2=0x7fcb34159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 0x7fcb34159e30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcb24000c00 tx=0x7fcb240046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 shutdown_connections 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcb3415deb0 0x7fcb340cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb1006b4d0 0x7fcb1006d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb340ce030 0x7fcb340cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 --2- 172.21.15.5:0/1306336788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb340b6f20 0x7fcb34159e30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.882+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 >> 172.21.15.5:0/1306336788 conn(0x7fcb340726b0 msgr2=0x7fcb340c7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.883+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 shutdown_connections 2024-04-25T10:05:13.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:13.883+0000 7fcb3b000640 1 -- 172.21.15.5:0/1306336788 wait complete. 2024-04-25T10:05:13.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.122+0000 7f153aa00640 1 Processor -- start 2024-04-25T10:05:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.123+0000 7f153aa00640 1 -- start start 2024-04-25T10:05:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.124+0000 7f153aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 0x7f153415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.124+0000 7f153aa00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15341606e0 con 0x7f1534161d90 2024-04-25T10:05:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.124+0000 7f153aa00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15341609c0 con 0x7f15340d3f90 2024-04-25T10:05:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.124+0000 7f153aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1534160ca0 con 0x7f15340b48c0 2024-04-25T10:05:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.125+0000 7f153a000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1534161d90 0x7f15340b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34458/0 (socket says 172.21.15.5:34458) 2024-04-25T10:05:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.125+0000 7f153a000640 1 -- 172.21.15.5:0/3338558769 learned_addr learned my addr 172.21.15.5:0/3338558769 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.125+0000 7f1539600640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 0x7f153415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.125+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983386054 0 0) 0x7f15341606e0 con 0x7f1534161d90 2024-04-25T10:05:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.125+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1518003520 con 0x7f1534161d90 2024-04-25T10:05:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100532247 0 0) 0x7f1518003520 con 0x7f1534161d90 2024-04-25T10:05:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 msgr2=0x7f153415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 0x7f153415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 >> v1:172.21.15.80:6789/0 conn(0x7f15340d3f90 legacy=0x7f1534161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534160f80 con 0x7f1534161d90 2024-04-25T10:05:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f152c0019f0 con 0x7f1534161d90 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 >> v1:172.21.15.5:6789/0 conn(0x7f1534161d90 legacy=0x7f15340b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15180039e0 0x7f1518003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 0x7f1518004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1518004d40 0x7f15180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1534160ca0 con 0x7f1518004d40 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151800ba70 con 0x7f15180039e0 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151800bc70 con 0x7f15340b48c0 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f153a000640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1518004d40 0x7f15180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.126+0000 7f1539600640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15180039e0 0x7f1518003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f1538c00640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 0x7f1518004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f153a000640 1 -- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 msgr2=0x7f1518004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f153a000640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b48c0 0x7f1518004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f153a000640 1 -- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15180039e0 msgr2=0x7f1518003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f153a000640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15180039e0 0x7f1518003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f153a000640 1 -- 172.21.15.5:0/3338558769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518003520 con 0x7f1518004d40 2024-04-25T10:05:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.127+0000 7f1539600640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15180039e0 0x7f1518003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.128+0000 7f153a000640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1518004d40 0x7f15180051c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f153408bb10 tx=0x7f1518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7374d4a4a89763b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.128+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00a480 con 0x7f1518004d40 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.128+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c02f070 con 0x7f1518004d40 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.128+0000 7f1533e00640 1 -- 172.21.15.5:0/3338558769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c02bc50 con 0x7f1518004d40 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153aa00640 1 -- 172.21.15.5:0/3338558769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1518004d40 msgr2=0x7f15180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153aa00640 1 --2- 172.21.15.5:0/3338558769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1518004d40 0x7f15180051c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f153408bb10 tx=0x7f1518005830 comp rx=0 tx=0).stop 2024-04-25T10:05:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153a000640 1 -- 172.21.15.5:0/3338558769 reap_dead start 2024-04-25T10:05:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153aa00640 1 -- 172.21.15.5:0/3338558769 shutdown_connections 2024-04-25T10:05:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153aa00640 1 -- 172.21.15.5:0/3338558769 >> 172.21.15.5:0/3338558769 conn(0x7f153408af80 msgr2=0x7f15340d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153aa00640 1 -- 172.21.15.5:0/3338558769 shutdown_connections 2024-04-25T10:05:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.129+0000 7f153aa00640 1 -- 172.21.15.5:0/3338558769 wait complete. 2024-04-25T10:05:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.131+0000 7f153aa00640 1 Processor -- start 2024-04-25T10:05:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.131+0000 7f153aa00640 1 -- start start 2024-04-25T10:05:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1518004d40 0x7f15340cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b8dc0 0x7f15340c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 0x7f153406cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153415fe60 con 0x7f153406c8b0 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340b4ca0 con 0x7f1518004d40 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153415ffe0 con 0x7f15340b8dc0 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 0x7f153406cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 0x7f153406cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39810/0 (socket says 172.21.15.5:39810) 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.132+0000 7f153a000640 1 -- 172.21.15.5:0/585094663 learned_addr learned my addr 172.21.15.5:0/585094663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f1539600640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1518004d40 0x7f15340cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f1538c00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b8dc0 0x7f15340c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f153a000640 1 -- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b8dc0 msgr2=0x7f15340c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f153a000640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b8dc0 0x7f15340c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f153a000640 1 -- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1518004d40 msgr2=0x7f15340cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f153a000640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1518004d40 0x7f15340cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f153a000640 1 -- 172.21.15.5:0/585094663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340b9240 con 0x7f153406c8b0 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f1538c00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b8dc0 0x7f15340c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f1539600640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1518004d40 0x7f15340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f153a000640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 0x7f153406cd30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f152c001cf0 tx=0x7f1534161150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.133+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00a480 con 0x7f153406c8b0 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.134+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c0019e0 con 0x7f153406c8b0 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.134+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15340b94c0 con 0x7f153406c8b0 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.134+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c001fd0 con 0x7f153406c8b0 2024-04-25T10:05:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.134+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15340c99d0 con 0x7f153406c8b0 2024-04-25T10:05:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.135+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153406d270 con 0x7f153406c8b0 2024-04-25T10:05:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.136+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f152c046060 con 0x7f153406c8b0 2024-04-25T10:05:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.137+0000 7f1531600640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151406b4d0 0x7f151406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.137+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f152c0aa1d0 con 0x7f153406c8b0 2024-04-25T10:05:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.137+0000 7f1539600640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151406b4d0 0x7f151406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.139+0000 7f1539600640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151406b4d0 0x7f151406d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1524009ad0 tx=0x7f1524008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.141+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15340b9240 con 0x7f153406c8b0 2024-04-25T10:05:14.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.343+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1534068f60 con 0x7f153406c8b0 2024-04-25T10:05:14.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.344+0000 7f1531600640 1 -- 172.21.15.5:0/585094663 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f152c0333d0 con 0x7f153406c8b0 2024-04-25T10:05:14.350 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:05:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151406b4d0 msgr2=0x7f151406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151406b4d0 0x7f151406d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1524009ad0 tx=0x7f1524008040 comp rx=0 tx=0).stop 2024-04-25T10:05:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 msgr2=0x7f153406cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 0x7f153406cd30 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f152c001cf0 tx=0x7f1534161150 comp rx=0 tx=0).stop 2024-04-25T10:05:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 shutdown_connections 2024-04-25T10:05:14.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153406c8b0 0x7f153406cd30 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15340b8dc0 0x7f15340c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f151406b4d0 0x7f151406d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 --2- 172.21.15.5:0/585094663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1518004d40 0x7f15340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 >> 172.21.15.5:0/585094663 conn(0x7f153408af80 msgr2=0x7f15340b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 shutdown_connections 2024-04-25T10:05:14.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.347+0000 7f153aa00640 1 -- 172.21.15.5:0/585094663 wait complete. 2024-04-25T10:05:14.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.578+0000 7f5149200640 1 Processor -- start 2024-04-25T10:05:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.578+0000 7f5149200640 1 -- start start 2024-04-25T10:05:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.579+0000 7f5149200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f514415b000 0x7f514406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.580+0000 7f5149200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5144154810 con 0x7f51440cdaf0 2024-04-25T10:05:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.580+0000 7f5149200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440ce4d0 con 0x7f51440716a0 2024-04-25T10:05:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.580+0000 7f5149200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440b3780 con 0x7f514415b000 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.580+0000 7f513be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f51440cdaf0 0x7f5144070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34464/0 (socket says 172.21.15.5:34464) 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.580+0000 7f513be00640 1 -- 172.21.15.5:0/548470150 learned_addr learned my addr 172.21.15.5:0/548470150 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.580+0000 7f5143400640 1 --2- 172.21.15.5:0/548470150 >> v2:172.21.15.80:3300/0 conn(0x7f514415b000 0x7f514406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994004255 0 0) 0x7f512c001110 con 0x7f51440cdaf0 2024-04-25T10:05:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128003520 con 0x7f51440cdaf0 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2912997092 0 0) 0x7f5128003520 con 0x7f51440cdaf0 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 >> v2:172.21.15.80:3300/0 conn(0x7f514415b000 msgr2=0x7f514406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 --2- 172.21.15.5:0/548470150 >> v2:172.21.15.80:3300/0 conn(0x7f514415b000 0x7f514406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 >> v1:172.21.15.80:6789/0 conn(0x7f51440716a0 legacy=0x7f51440b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440ce7b0 con 0x7f51440cdaf0 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f512c001840 con 0x7f51440cdaf0 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 >> v1:172.21.15.5:6789/0 conn(0x7f51440cdaf0 legacy=0x7f5144070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51280039b0 0x7f5128003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5128004800 0x7f5128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f512800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440b3780 con 0x7f51280051c0 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512800e270 con 0x7f5128004800 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.581+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512800e470 con 0x7f51280039b0 2024-04-25T10:05:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143e00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f512800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f513be00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5128004800 0x7f5128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143400640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51280039b0 0x7f5128003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143e00640 1 -- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51280039b0 msgr2=0x7f5128003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143e00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51280039b0 0x7f5128003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143e00640 1 -- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5128004800 msgr2=0x7f5128004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143e00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5128004800 0x7f5128004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.582+0000 7f5143e00640 1 -- 172.21.15.5:0/548470150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128003520 con 0x7f51280051c0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.583+0000 7f5143e00640 1 -- 172.21.15.5:0/548470150 reap_dead start 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.583+0000 7f5143e00640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f512800dbd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f513c001e50 tx=0x7f5128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610b7e4aeef44cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.583+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c001a70 con 0x7f51280051c0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.583+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c003970 con 0x7f51280051c0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.583+0000 7f5142a00640 1 -- 172.21.15.5:0/548470150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c003d60 con 0x7f51280051c0 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 -- 172.21.15.5:0/548470150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 msgr2=0x7f512800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f512800dbd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f513c001e50 tx=0x7f5128005830 comp rx=0 tx=0).stop 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 -- 172.21.15.5:0/548470150 shutdown_connections 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 --2- 172.21.15.5:0/548470150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f512800dbd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 -- 172.21.15.5:0/548470150 >> 172.21.15.5:0/548470150 conn(0x7f5144066f20 msgr2=0x7f514406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 -- 172.21.15.5:0/548470150 shutdown_connections 2024-04-25T10:05:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.584+0000 7f5149200640 1 -- 172.21.15.5:0/548470150 wait complete. 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.586+0000 7f5149200640 1 Processor -- start 2024-04-25T10:05:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.586+0000 7f5149200640 1 -- start start 2024-04-25T10:05:14.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5149200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f51441e4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5149200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51441e8600 0x7f51441f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5149200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51441e5be0 0x7f51441e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5149200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440b3490 con 0x7f51280051c0 2024-04-25T10:05:14.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5149200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440b3310 con 0x7f51441e8600 2024-04-25T10:05:14.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5149200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514415a530 con 0x7f51441e5be0 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.587+0000 7f5143400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f51441e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f51441e4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39826/0 (socket says 172.21.15.5:39826) 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 -- 172.21.15.5:0/343160896 learned_addr learned my addr 172.21.15.5:0/343160896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f513be00640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51441e8600 0x7f51441f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143e00640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51441e5be0 0x7f51441e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 -- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51441e5be0 msgr2=0x7f51441e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51441e5be0 0x7f51441e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 -- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51441e8600 msgr2=0x7f51441f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51441e8600 0x7f51441f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.588+0000 7f5143400640 1 -- 172.21.15.5:0/343160896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51441e53d0 con 0x7f51280051c0 2024-04-25T10:05:14.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.589+0000 7f5143400640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f51441e4170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5134010890 tx=0x7f5134010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.589+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400f040 con 0x7f51280051c0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.589+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513400eb60 con 0x7f51280051c0 2024-04-25T10:05:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.589+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513401c400 con 0x7f51280051c0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.589+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51441e4850 con 0x7f51280051c0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.589+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441e4e10 con 0x7f51280051c0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.590+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5144068f60 con 0x7f51280051c0 2024-04-25T10:05:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.591+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f513401c5a0 con 0x7f51280051c0 2024-04-25T10:05:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.593+0000 7f513b400640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512006b4d0 0x7f512006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.593+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f513408aa70 con 0x7f51280051c0 2024-04-25T10:05:14.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.593+0000 7f513be00640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512006b4d0 0x7f512006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.594+0000 7f513be00640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512006b4d0 0x7f512006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f512c001eb0 tx=0x7f512c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.597+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5134018030 con 0x7f51280051c0 2024-04-25T10:05:14.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.807+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51440cc720 con 0x7f51280051c0 2024-04-25T10:05:14.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.808+0000 7f513b400640 1 -- 172.21.15.5:0/343160896 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5134059050 con 0x7f51280051c0 2024-04-25T10:05:14.814 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.810+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512006b4d0 msgr2=0x7f512006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.810+0000 7f5149200640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512006b4d0 0x7f512006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f512c001eb0 tx=0x7f512c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.810+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 msgr2=0x7f51441e4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.810+0000 7f5149200640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f51441e4170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5134010890 tx=0x7f5134010d50 comp rx=0 tx=0).stop 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 shutdown_connections 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51441e5be0 0x7f51441e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512006b4d0 0x7f512006d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51441e8600 0x7f51441f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 --2- 172.21.15.5:0/343160896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51280051c0 0x7f51441e4170 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 >> 172.21.15.5:0/343160896 conn(0x7f5144066f20 msgr2=0x7f51440b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 shutdown_connections 2024-04-25T10:05:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:14.811+0000 7f5149200640 1 -- 172.21.15.5:0/343160896 wait complete. 2024-04-25T10:05:14.833 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:05:14.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:14.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:14.957 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:05:14.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:14.957 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:15.002 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:05:15.002 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:15.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:15.041 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:05:15.041 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:15.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:15.081 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:05:15.081 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:15.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:15.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.316+0000 7f269a400640 1 Processor -- start 2024-04-25T10:05:15.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.317+0000 7f269a400640 1 -- start start 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f269a400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f269415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f269a400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26940cd300 0x7f26940cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f269a400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2694071d30 con 0x7f2694141da0 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f269a400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26940cdcd0 con 0x7f26940cd300 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f269a400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694142180 con 0x7f269415ce00 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f2699000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2694141da0 0x7f269406e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34466/0 (socket says 172.21.15.5:34466) 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f2699000640 1 -- 172.21.15.5:0/2823156714 learned_addr learned my addr 172.21.15.5:0/2823156714 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.318+0000 7f2699a00640 1 --2- 172.21.15.5:0/2823156714 >> v2:172.21.15.5:3300/0 conn(0x7f26940cd300 0x7f26940cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2693400640 1 -- 172.21.15.5:0/2823156714 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117633045 0 0) 0x7f2694071d30 con 0x7f2694141da0 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2693400640 1 -- 172.21.15.5:0/2823156714 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f267c003520 con 0x7f2694141da0 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2693e00640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f269415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2699a00640 1 -- 172.21.15.5:0/2823156714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 msgr2=0x7f269415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2699a00640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f269415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2699a00640 1 -- 172.21.15.5:0/2823156714 >> v1:172.21.15.5:6789/0 conn(0x7f2694141da0 legacy=0x7f269406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2699a00640 1 -- 172.21.15.5:0/2823156714 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269406d620 con 0x7f26940cd300 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.319+0000 7f2693e00640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f269415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.320+0000 7f2699a00640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f26940cd790 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f268c009d90 tx=0x7f268c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea42b0d1d8031bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.320+0000 7f2693400640 1 -- 172.21.15.5:0/2823156714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c007740 con 0x7f26940cd300 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.320+0000 7f2693400640 1 -- 172.21.15.5:0/2823156714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c0078e0 con 0x7f26940cd300 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.320+0000 7f2693400640 1 -- 172.21.15.5:0/2823156714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c007cb0 con 0x7f26940cd300 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 -- 172.21.15.5:0/2823156714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 msgr2=0x7f26940cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f26940cd790 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f268c009d90 tx=0x7f268c02a660 comp rx=0 tx=0).stop 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 -- 172.21.15.5:0/2823156714 shutdown_connections 2024-04-25T10:05:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f26940cd790 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 --2- 172.21.15.5:0/2823156714 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f269415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 -- 172.21.15.5:0/2823156714 >> 172.21.15.5:0/2823156714 conn(0x7f2694066f20 msgr2=0x7f269406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 -- 172.21.15.5:0/2823156714 shutdown_connections 2024-04-25T10:05:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.321+0000 7f269a400640 1 -- 172.21.15.5:0/2823156714 wait complete. 2024-04-25T10:05:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.323+0000 7f269a400640 1 Processor -- start 2024-04-25T10:05:15.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.323+0000 7f269a400640 1 -- start start 2024-04-25T10:05:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f269a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f269413b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f269a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2694141da0 0x7f269413fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f269a400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f2694157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f269a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269406e810 con 0x7f26940cd300 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f269a400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269406dbe0 con 0x7f2694141da0 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f269a400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694071000 con 0x7f269415ce00 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f2699000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f269413b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f2699000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f269413b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39848/0 (socket says 172.21.15.5:39848) 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.324+0000 7f2699000640 1 -- 172.21.15.5:0/2576559509 learned_addr learned my addr 172.21.15.5:0/2576559509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2693e00640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2694141da0 0x7f269413fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699a00640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f2694157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699000640 1 -- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 msgr2=0x7f2694157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699000640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f2694157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699000640 1 -- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2694141da0 msgr2=0x7f269413fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699000640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2694141da0 0x7f269413fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699000640 1 -- 172.21.15.5:0/2576559509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269413c280 con 0x7f26940cd300 2024-04-25T10:05:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2699000640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f269413b930 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f268800c430 tx=0x7f268800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688017070 con 0x7f26940cd300 2024-04-25T10:05:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.325+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688005ce0 con 0x7f26940cd300 2024-04-25T10:05:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.326+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880072f0 con 0x7f26940cd300 2024-04-25T10:05:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.326+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269413c560 con 0x7f26940cd300 2024-04-25T10:05:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.326+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269413cb20 con 0x7f26940cd300 2024-04-25T10:05:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.327+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26940cc860 con 0x7f26940cd300 2024-04-25T10:05:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.328+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f268801a070 con 0x7f26940cd300 2024-04-25T10:05:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.329+0000 7f2690c00640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267406b4d0 0x7f267406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.329+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2688089900 con 0x7f26940cd300 2024-04-25T10:05:15.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.329+0000 7f2693e00640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267406b4d0 0x7f267406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.331+0000 7f2693e00640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267406b4d0 0x7f267406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2694144860 tx=0x7f2680008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.334+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2688058530 con 0x7f26940cd300 2024-04-25T10:05:15.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.574+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f269406d3e0 con 0x7f26940cd300 2024-04-25T10:05:15.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.575+0000 7f2690c00640 1 -- 172.21.15.5:0/2576559509 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f2688057e30 con 0x7f26940cd300 2024-04-25T10:05:15.581 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:15.581 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:15.581 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:05:15.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267406b4d0 msgr2=0x7f267406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267406b4d0 0x7f267406d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2694144860 tx=0x7f2680008040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 msgr2=0x7f269413b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f269413b930 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f268800c430 tx=0x7f268800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 shutdown_connections 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f269415ce00 0x7f2694157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f267406b4d0 0x7f267406d990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2694141da0 0x7f269413fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 --2- 172.21.15.5:0/2576559509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26940cd300 0x7f269413b930 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.577+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 >> 172.21.15.5:0/2576559509 conn(0x7f2694066f20 msgr2=0x7f269406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.578+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 shutdown_connections 2024-04-25T10:05:15.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.578+0000 7f269a400640 1 -- 172.21.15.5:0/2576559509 wait complete. 2024-04-25T10:05:15.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.802+0000 7f5bcb200640 1 Processor -- start 2024-04-25T10:05:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.803+0000 7f5bcb200640 1 -- start start 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.804+0000 7f5bcb200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5bcc141da0 0x7f5bcc06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.804+0000 7f5bcb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc06e640 0x7f5bcc15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.804+0000 7f5bcb200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5bcc15d340 0x7f5bcc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.804+0000 7f5bcb200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bcc142180 con 0x7f5bcc15d340 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.804+0000 7f5bcb200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bcc071d30 con 0x7f5bcc141da0 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.804+0000 7f5bcb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc06d620 con 0x7f5bcc06e640 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5bcc15d340 0x7f5bcc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5bcc15d340 0x7f5bcc0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39862/0 (socket says 172.21.15.5:39862) 2024-04-25T10:05:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 -- 172.21.15.5:0/2574802776 learned_addr learned my addr 172.21.15.5:0/2574802776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bc3e00640 1 --2- 172.21.15.5:0/2574802776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc06e640 0x7f5bcc15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bc9e00640 1 --2- 172.21.15.5:0/2574802776 >> v2:172.21.15.80:3300/0 conn(0x7f5bcc141da0 0x7f5bcc06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 -- 172.21.15.5:0/2574802776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc06e640 msgr2=0x7f5bcc15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 --2- 172.21.15.5:0/2574802776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc06e640 0x7f5bcc15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 -- 172.21.15.5:0/2574802776 >> v2:172.21.15.80:3300/0 conn(0x7f5bcc141da0 msgr2=0x7f5bcc06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 --2- 172.21.15.5:0/2574802776 >> v2:172.21.15.80:3300/0 conn(0x7f5bcc141da0 0x7f5bcc06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.805+0000 7f5bca800640 1 -- 172.21.15.5:0/2574802776 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc06d900 con 0x7f5bcc15d340 2024-04-25T10:05:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.807+0000 7f5bca800640 1 --2- 172.21.15.5:0/2574802776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc15d340 0x7f5bcc0cd300 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc06bbb0 tx=0x7f5bc402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f953629c3d94689f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.807+0000 7f5bc9400640 1 -- 172.21.15.5:0/2574802776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4036070 con 0x7f5bcc15d340 2024-04-25T10:05:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.807+0000 7f5bc9400640 1 -- 172.21.15.5:0/2574802776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4031600 con 0x7f5bcc15d340 2024-04-25T10:05:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.807+0000 7f5bc9400640 1 -- 172.21.15.5:0/2574802776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40319d0 con 0x7f5bcc15d340 2024-04-25T10:05:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 -- 172.21.15.5:0/2574802776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc15d340 msgr2=0x7f5bcc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 --2- 172.21.15.5:0/2574802776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc15d340 0x7f5bcc0cd300 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc06bbb0 tx=0x7f5bc402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 -- 172.21.15.5:0/2574802776 shutdown_connections 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 --2- 172.21.15.5:0/2574802776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc15d340 0x7f5bcc0cd300 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 --2- 172.21.15.5:0/2574802776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc06e640 0x7f5bcc15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 --2- 172.21.15.5:0/2574802776 >> v2:172.21.15.80:3300/0 conn(0x7f5bcc141da0 0x7f5bcc06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 -- 172.21.15.5:0/2574802776 >> 172.21.15.5:0/2574802776 conn(0x7f5bcc066f20 msgr2=0x7f5bcc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 -- 172.21.15.5:0/2574802776 shutdown_connections 2024-04-25T10:05:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.808+0000 7f5bcb200640 1 -- 172.21.15.5:0/2574802776 wait complete. 2024-04-25T10:05:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.810+0000 7f5bcb200640 1 Processor -- start 2024-04-25T10:05:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.811+0000 7f5bcb200640 1 -- start start 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bcb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 0x7f5bcc14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bcb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bcc141da0 0x7f5bcc13e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bcb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc15d340 0x7f5bcc1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bcb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0cd9c0 con 0x7f5bcc06e640 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bcb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0cd840 con 0x7f5bcc141da0 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bcb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0cdb40 con 0x7f5bcc15d340 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bc9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 0x7f5bcc14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bc9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 0x7f5bcc14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39866/0 (socket says 172.21.15.5:39866) 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.812+0000 7f5bc9e00640 1 -- 172.21.15.5:0/3515291851 learned_addr learned my addr 172.21.15.5:0/3515291851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc3e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bcc141da0 0x7f5bcc13e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bca800640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc15d340 0x7f5bcc1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc9e00640 1 -- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc15d340 msgr2=0x7f5bcc1e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc9e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc15d340 0x7f5bcc1e5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc9e00640 1 -- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bcc141da0 msgr2=0x7f5bcc13e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc9e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bcc141da0 0x7f5bcc13e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc9e00640 1 -- 172.21.15.5:0/3515291851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc1e5a70 con 0x7f5bcc06e640 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc3e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bcc141da0 0x7f5bcc13e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bca800640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc15d340 0x7f5bcc1e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.813+0000 7f5bc9e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 0x7f5bcc14a6b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00d800 tx=0x7f5bbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.814+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00b6c0 con 0x7f5bcc06e640 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.814+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc019070 con 0x7f5bcc06e640 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.814+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00ba80 con 0x7f5bcc06e640 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.814+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc1e5d50 con 0x7f5bcc06e640 2024-04-25T10:05:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.814+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc1e6310 con 0x7f5bcc06e640 2024-04-25T10:05:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.815+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc1418e0 con 0x7f5bcc06e640 2024-04-25T10:05:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.816+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bbc00bc20 con 0x7f5bcc06e640 2024-04-25T10:05:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.817+0000 7f5bc2000640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ba806b4d0 0x7f5ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.817+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5bbc08a800 con 0x7f5bcc06e640 2024-04-25T10:05:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.817+0000 7f5bc3e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ba806b4d0 0x7f5ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.819+0000 7f5bc3e00640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ba806b4d0 0x7f5ba806d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4006fd0 tx=0x7f5bb4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:15.821+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bbc0594c0 con 0x7f5bcc06e640 2024-04-25T10:05:16.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.021+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bcc0ccae0 con 0x7f5bcc06e640 2024-04-25T10:05:16.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.024+0000 7f5bc2000640 1 -- 172.21.15.5:0/3515291851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16823 (secure 0 0 0) 0x7f5bbc058dc0 con 0x7f5bcc06e640 2024-04-25T10:05:16.031 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:16.031 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:41.582074+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2b","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ba806b4d0 msgr2=0x7f5ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ba806b4d0 0x7f5ba806d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4006fd0 tx=0x7f5bb4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 msgr2=0x7f5bcc14a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 0x7f5bcc14a6b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00d800 tx=0x7f5bbc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 shutdown_connections 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5bcc15d340 0x7f5bcc1e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5ba806b4d0 0x7f5ba806d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5bcc141da0 0x7f5bcc13e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 --2- 172.21.15.5:0/3515291851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc06e640 0x7f5bcc14a6b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.027+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 >> 172.21.15.5:0/3515291851 conn(0x7f5bcc066f20 msgr2=0x7f5bcc0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.028+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 shutdown_connections 2024-04-25T10:05:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.028+0000 7f5bcb200640 1 -- 172.21.15.5:0/3515291851 wait complete. 2024-04-25T10:05:16.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:22.770291+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:23.937149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:16.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:16.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.269+0000 7fec4fe00640 1 Processor -- start 2024-04-25T10:05:16.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.269+0000 7fec4fe00640 1 -- start start 2024-04-25T10:05:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.270+0000 7fec4fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec500b48c0 0x7fec5015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.270+0000 7fec4fe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec501606e0 con 0x7fec500d3f90 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.270+0000 7fec4fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec501609c0 con 0x7fec50161d90 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.270+0000 7fec4fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec50160ca0 con 0x7fec500b48c0 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.271+0000 7fec4ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fec500b48c0 0x7fec5015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.271+0000 7fec4e000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fec500d3f90 0x7fec50161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34468/0 (socket says 172.21.15.5:34468) 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.271+0000 7fec4e000640 1 -- 172.21.15.5:0/3369181016 learned_addr learned my addr 172.21.15.5:0/3369181016 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.272+0000 7fec4ea00640 1 -- 172.21.15.5:0/3369181016 >> v1:172.21.15.80:6789/0 conn(0x7fec50161d90 legacy=0x7fec500b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.272+0000 7fec4d600640 1 -- 172.21.15.5:0/3369181016 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428570858 0 0) 0x7fec501606e0 con 0x7fec500d3f90 2024-04-25T10:05:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.272+0000 7fec4ea00640 1 -- 172.21.15.5:0/3369181016 >> v1:172.21.15.5:6789/0 conn(0x7fec500d3f90 legacy=0x7fec50161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.272+0000 7fec4ea00640 1 -- 172.21.15.5:0/3369181016 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec50160f80 con 0x7fec500b48c0 2024-04-25T10:05:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.273+0000 7fec4ea00640 1 --2- 172.21.15.5:0/3369181016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500b48c0 0x7fec5015fe60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fec40009d90 tx=0x7fec4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d126ae48c0612321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.273+0000 7fec4d600640 1 -- 172.21.15.5:0/3369181016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40007740 con 0x7fec500b48c0 2024-04-25T10:05:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.273+0000 7fec4d600640 1 -- 172.21.15.5:0/3369181016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec400078e0 con 0x7fec500b48c0 2024-04-25T10:05:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.273+0000 7fec4d600640 1 -- 172.21.15.5:0/3369181016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40007cb0 con 0x7fec500b48c0 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 -- 172.21.15.5:0/3369181016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500b48c0 msgr2=0x7fec5015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 --2- 172.21.15.5:0/3369181016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500b48c0 0x7fec5015fe60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fec40009d90 tx=0x7fec4002a660 comp rx=0 tx=0).stop 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 -- 172.21.15.5:0/3369181016 shutdown_connections 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 --2- 172.21.15.5:0/3369181016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500b48c0 0x7fec5015fe60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 -- 172.21.15.5:0/3369181016 >> 172.21.15.5:0/3369181016 conn(0x7fec5008af80 msgr2=0x7fec500d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 -- 172.21.15.5:0/3369181016 shutdown_connections 2024-04-25T10:05:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.274+0000 7fec4fe00640 1 -- 172.21.15.5:0/3369181016 wait complete. 2024-04-25T10:05:16.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.276+0000 7fec4fe00640 1 Processor -- start 2024-04-25T10:05:16.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.276+0000 7fec4fe00640 1 -- start start 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec500b48c0 0x7fec500c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 0x7fec500b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec50161d90 0x7fec500c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501603a0 con 0x7fec500d3f90 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec500b4ca0 con 0x7fec500b48c0 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec500b3280 con 0x7fec50161d90 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 0x7fec500b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 0x7fec500b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39880/0 (socket says 172.21.15.5:39880) 2024-04-25T10:05:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.277+0000 7fec4e000640 1 -- 172.21.15.5:0/1801378324 learned_addr learned my addr 172.21.15.5:0/1801378324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4e000640 1 -- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec50161d90 msgr2=0x7fec500c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4ea00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec500b48c0 0x7fec500c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4f400640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec50161d90 0x7fec500c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4e000640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec50161d90 0x7fec500c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4e000640 1 -- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec500b48c0 msgr2=0x7fec500c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4e000640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec500b48c0 0x7fec500c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4e000640 1 -- 172.21.15.5:0/1801378324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec500c9f50 con 0x7fec500d3f90 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4e000640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 0x7fec500b1670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c730 tx=0x7fec3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38007270 con 0x7fec500d3f90 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38007430 con 0x7fec500d3f90 2024-04-25T10:05:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380059e0 con 0x7fec500d3f90 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.278+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec500cb4f0 con 0x7fec500d3f90 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.279+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec500cbab0 con 0x7fec500d3f90 2024-04-25T10:05:16.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.280+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec50068f60 con 0x7fec500d3f90 2024-04-25T10:05:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.281+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec38005b80 con 0x7fec500d3f90 2024-04-25T10:05:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.282+0000 7fec47400640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2c06b4d0 0x7fec2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.282+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fec38089ff0 con 0x7fec500d3f90 2024-04-25T10:05:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.283+0000 7fec4ea00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2c06b4d0 0x7fec2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.284+0000 7fec4ea00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2c06b4d0 0x7fec2c06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec40034000 tx=0x7fec4002f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.286+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec38010300 con 0x7fec500d3f90 2024-04-25T10:05:16.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.486+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec5008d730 con 0x7fec500d3f90 2024-04-25T10:05:16.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.487+0000 7fec47400640 1 -- 172.21.15.5:0/1801378324 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec380075e0 con 0x7fec500d3f90 2024-04-25T10:05:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:05:16.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2c06b4d0 msgr2=0x7fec2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2c06b4d0 0x7fec2c06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec40034000 tx=0x7fec4002f000 comp rx=0 tx=0).stop 2024-04-25T10:05:16.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 msgr2=0x7fec500b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 0x7fec500b1670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c730 tx=0x7fec3800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 shutdown_connections 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fec50161d90 0x7fec500c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec500d3f90 0x7fec500b1670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fec2c06b4d0 0x7fec2c06d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 --2- 172.21.15.5:0/1801378324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec500b48c0 0x7fec500c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 >> 172.21.15.5:0/1801378324 conn(0x7fec5008af80 msgr2=0x7fec5015f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.489+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 shutdown_connections 2024-04-25T10:05:16.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.490+0000 7fec4fe00640 1 -- 172.21.15.5:0/1801378324 wait complete. 2024-04-25T10:05:16.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.719+0000 7f40b4a00640 1 Processor -- start 2024-04-25T10:05:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.719+0000 7f40b4a00640 1 -- start start 2024-04-25T10:05:16.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.720+0000 7f40b4a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f40b006e840 0x7f40b015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.720+0000 7f40b4a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00ce570 con 0x7f40b015b330 2024-04-25T10:05:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40b4a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00ce850 con 0x7f40b00cc840 2024-04-25T10:05:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40b4a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b00ceb30 con 0x7f40b006e840 2024-04-25T10:05:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40ade00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f40b006e840 0x7f40b015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40af200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f40b015b330 0x7f40b0071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34482/0 (socket says 172.21.15.5:34482) 2024-04-25T10:05:16.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40af200640 1 -- 172.21.15.5:0/2719786026 learned_addr learned my addr 172.21.15.5:0/2719786026 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40ade00640 1 -- 172.21.15.5:0/2719786026 >> v1:172.21.15.80:6789/0 conn(0x7f40b00cc840 legacy=0x7f40b006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40ade00640 1 -- 172.21.15.5:0/2719786026 >> v1:172.21.15.5:6789/0 conn(0x7f40b015b330 legacy=0x7f40b0071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40ade00640 1 -- 172.21.15.5:0/2719786026 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00cee10 con 0x7f40b006e840 2024-04-25T10:05:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.721+0000 7f40ad400640 1 -- 172.21.15.5:0/2719786026 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029856069 0 0) 0x7f40b00ce570 con 0x7f40b015b330 2024-04-25T10:05:16.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.722+0000 7f40ade00640 1 --2- 172.21.15.5:0/2719786026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b015adf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f409c0097a0 tx=0x7f409c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2db20ac351229f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40ad400640 1 -- 172.21.15.5:0/2719786026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c037070 con 0x7f40b006e840 2024-04-25T10:05:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40ad400640 1 -- 172.21.15.5:0/2719786026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c032440 con 0x7f40b006e840 2024-04-25T10:05:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40ad400640 1 -- 172.21.15.5:0/2719786026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c032810 con 0x7f40b006e840 2024-04-25T10:05:16.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 -- 172.21.15.5:0/2719786026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 msgr2=0x7f40b015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 --2- 172.21.15.5:0/2719786026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b015adf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f409c0097a0 tx=0x7f409c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 -- 172.21.15.5:0/2719786026 shutdown_connections 2024-04-25T10:05:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 --2- 172.21.15.5:0/2719786026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b015adf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 -- 172.21.15.5:0/2719786026 >> 172.21.15.5:0/2719786026 conn(0x7f40b0066f20 msgr2=0x7f40b006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 -- 172.21.15.5:0/2719786026 shutdown_connections 2024-04-25T10:05:16.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.723+0000 7f40b4a00640 1 -- 172.21.15.5:0/2719786026 wait complete. 2024-04-25T10:05:16.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.725+0000 7f40b4a00640 1 Processor -- start 2024-04-25T10:05:16.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.725+0000 7f40b4a00640 1 -- start start 2024-04-25T10:05:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40b4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b00b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40b4a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f40b00cc840 0x7f40b00b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40b4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40b015b330 0x7f40b014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40b4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00ce350 con 0x7f40b006e840 2024-04-25T10:05:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40b4a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00ce1d0 con 0x7f40b015b330 2024-04-25T10:05:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40b4a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00ccc20 con 0x7f40b00cc840 2024-04-25T10:05:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.726+0000 7f40ae800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b00b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b00b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39902/0 (socket says 172.21.15.5:39902) 2024-04-25T10:05:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 -- 172.21.15.5:0/451965963 learned_addr learned my addr 172.21.15.5:0/451965963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 -- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f40b00cc840 msgr2=0x7f40b00b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f40b00cc840 0x7f40b00b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 -- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40b015b330 msgr2=0x7f40b014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40b015b330 0x7f40b014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 -- 172.21.15.5:0/451965963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00b00b0 con 0x7f40b006e840 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.727+0000 7f40ae800640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b00b0e70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f40a000c460 tx=0x7f40a000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.728+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0016020 con 0x7f40b006e840 2024-04-25T10:05:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.728+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a0012870 con 0x7f40b006e840 2024-04-25T10:05:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.728+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0012c60 con 0x7f40b006e840 2024-04-25T10:05:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.728+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b00b0390 con 0x7f40b006e840 2024-04-25T10:05:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.728+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b00b0950 con 0x7f40b006e840 2024-04-25T10:05:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.729+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b004efc0 con 0x7f40b006e840 2024-04-25T10:05:16.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.730+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40a0019020 con 0x7f40b006e840 2024-04-25T10:05:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.731+0000 7f40a6a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f409006b4d0 0x7f409006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.731+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f40a008a080 con 0x7f40b006e840 2024-04-25T10:05:16.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.731+0000 7f40ade00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f409006b4d0 0x7f409006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.733+0000 7f40ade00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f409006b4d0 0x7f409006d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f409c034000 tx=0x7f409c02c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.735+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f40b004efc0 con 0x7f40b006e840 2024-04-25T10:05:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.924+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40b0068f60 con 0x7f40b006e840 2024-04-25T10:05:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.925+0000 7f40a6a00640 1 -- 172.21.15.5:0/451965963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40a0020030 con 0x7f40b006e840 2024-04-25T10:05:16.931 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:05:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f409006b4d0 msgr2=0x7f409006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f409006b4d0 0x7f409006d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f409c034000 tx=0x7f409c02c000 comp rx=0 tx=0).stop 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 msgr2=0x7f40b00b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b00b0e70 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f40a000c460 tx=0x7f40a000c920 comp rx=0 tx=0).stop 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 shutdown_connections 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f409006b4d0 0x7f409006d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40b015b330 0x7f40b014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f40b00cc840 0x7f40b00b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 --2- 172.21.15.5:0/451965963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40b006e840 0x7f40b00b0e70 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.927+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 >> 172.21.15.5:0/451965963 conn(0x7f40b0066f20 msgr2=0x7f40b00b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.928+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 shutdown_connections 2024-04-25T10:05:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:16.928+0000 7f40b4a00640 1 -- 172.21.15.5:0/451965963 wait complete. 2024-04-25T10:05:16.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.159+0000 7fb669800640 1 Processor -- start 2024-04-25T10:05:17.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.159+0000 7fb669800640 1 -- start start 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb669800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 0x7fb66415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb669800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6640ae9e0 0x7fb6640cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb669800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66415d9f0 con 0x7fb664154590 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb669800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66415e5e0 con 0x7fb6640ae9e0 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb669800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640b4b60 con 0x7fb6640720d0 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb662a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb664154590 0x7fb6640719c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43044/0 (socket says 172.21.15.5:43044) 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.161+0000 7fb662a00640 1 -- 172.21.15.5:0/1211437649 learned_addr learned my addr 172.21.15.5:0/1211437649 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb662000640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 0x7fb66415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb663400640 1 --2- 172.21.15.5:0/1211437649 >> v2:172.21.15.80:3300/0 conn(0x7fb6640ae9e0 0x7fb6640cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853197014 0 0) 0x7fb66415d9f0 con 0x7fb664154590 2024-04-25T10:05:17.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb648003520 con 0x7fb664154590 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615491556 0 0) 0x7fb648003520 con 0x7fb664154590 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 msgr2=0x7fb66415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 0x7fb66415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 >> v2:172.21.15.80:3300/0 conn(0x7fb6640ae9e0 msgr2=0x7fb6640cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 --2- 172.21.15.5:0/1211437649 >> v2:172.21.15.80:3300/0 conn(0x7fb6640ae9e0 0x7fb6640cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66415dcb0 con 0x7fb664154590 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb654001670 con 0x7fb664154590 2024-04-25T10:05:17.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 >> v1:172.21.15.5:6789/0 conn(0x7fb664154590 legacy=0x7fb6640719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.162+0000 7fb661600640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6480039b0 0x7fb648003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb661600640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 0x7fb648004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb661600640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb6480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66415e5e0 con 0x7fb648004d40 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6640b4b60 con 0x7fb6480039b0 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64800b9c0 con 0x7fb6640720d0 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb663400640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb6480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb662a00640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6480039b0 0x7fb648003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb662000640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 0x7fb648004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb663400640 1 -- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 msgr2=0x7fb648004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb663400640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640720d0 0x7fb648004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb663400640 1 -- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6480039b0 msgr2=0x7fb648003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb663400640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6480039b0 0x7fb648003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.163+0000 7fb663400640 1 -- 172.21.15.5:0/1211437649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb648003520 con 0x7fb648004d40 2024-04-25T10:05:17.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.164+0000 7fb663400640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb6480051c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009f90 tx=0x7fb648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262936371ea87b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c009c10 con 0x7fb648004d40 2024-04-25T10:05:17.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c036070 con 0x7fb648004d40 2024-04-25T10:05:17.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb661600640 1 -- 172.21.15.5:0/1211437649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c031840 con 0x7fb648004d40 2024-04-25T10:05:17.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb669800640 1 -- 172.21.15.5:0/1211437649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 msgr2=0x7fb6480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb669800640 1 --2- 172.21.15.5:0/1211437649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb6480051c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009f90 tx=0x7fb648005830 comp rx=0 tx=0).stop 2024-04-25T10:05:17.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb663400640 1 -- 172.21.15.5:0/1211437649 reap_dead start 2024-04-25T10:05:17.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb669800640 1 -- 172.21.15.5:0/1211437649 shutdown_connections 2024-04-25T10:05:17.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb669800640 1 -- 172.21.15.5:0/1211437649 >> 172.21.15.5:0/1211437649 conn(0x7fb66406ab60 msgr2=0x7fb66406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb669800640 1 -- 172.21.15.5:0/1211437649 shutdown_connections 2024-04-25T10:05:17.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.165+0000 7fb669800640 1 -- 172.21.15.5:0/1211437649 wait complete. 2024-04-25T10:05:17.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.167+0000 7fb669800640 1 Processor -- start 2024-04-25T10:05:17.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.167+0000 7fb669800640 1 -- start start 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.168+0000 7fb669800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb664140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.168+0000 7fb669800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66414baf0 0x7fb664149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.168+0000 7fb669800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640b3630 0x7fb6640b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.168+0000 7fb669800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66415ae30 con 0x7fb648004d40 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.168+0000 7fb669800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66415e270 con 0x7fb66414baf0 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.168+0000 7fb669800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66415afb0 con 0x7fb6640b3630 2024-04-25T10:05:17.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb664140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb664140e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40854/0 (socket says 172.21.15.5:40854) 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 -- 172.21.15.5:0/1723573205 learned_addr learned my addr 172.21.15.5:0/1723573205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 -- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640b3630 msgr2=0x7fb6640b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640b3630 0x7fb6640b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 -- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66414baf0 msgr2=0x7fb664149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66414baf0 0x7fb664149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 -- 172.21.15.5:0/1723573205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6641413c0 con 0x7fb648004d40 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.169+0000 7fb662a00640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb664140e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb65400e7e0 tx=0x7fb65400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.170+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65400ee90 con 0x7fb648004d40 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.170+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb654018070 con 0x7fb648004d40 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.170+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65400ca10 con 0x7fb648004d40 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.170+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66414a250 con 0x7fb648004d40 2024-04-25T10:05:17.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.170+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6641584a0 con 0x7fb648004d40 2024-04-25T10:05:17.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.171+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6640b3ff0 con 0x7fb648004d40 2024-04-25T10:05:17.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.172+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb65400cbb0 con 0x7fb648004d40 2024-04-25T10:05:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.173+0000 7fb65aa00640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb64406b5e0 0x7fb64406daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.173+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb654089f20 con 0x7fb648004d40 2024-04-25T10:05:17.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.174+0000 7fb662000640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb64406b5e0 0x7fb64406daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.175+0000 7fb662000640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb64406b5e0 0x7fb64406daa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb664149900 tx=0x7fb650008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.178+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb654058b50 con 0x7fb648004d40 2024-04-25T10:05:17.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.368+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb664068f60 con 0x7fb648004d40 2024-04-25T10:05:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.369+0000 7fb65aa00640 1 -- 172.21.15.5:0/1723573205 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb654058450 con 0x7fb648004d40 2024-04-25T10:05:17.375 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:05:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.371+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb64406b5e0 msgr2=0x7fb64406daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.371+0000 7fb669800640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb64406b5e0 0x7fb64406daa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb664149900 tx=0x7fb650008040 comp rx=0 tx=0).stop 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 msgr2=0x7fb664140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb664140e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb65400e7e0 tx=0x7fb65400eca0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 shutdown_connections 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6640b3630 0x7fb6640b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb64406b5e0 0x7fb64406daa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb66414baf0 0x7fb664149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 --2- 172.21.15.5:0/1723573205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb648004d40 0x7fb664140e80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 >> 172.21.15.5:0/1723573205 conn(0x7fb66406ab60 msgr2=0x7fb6640af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 shutdown_connections 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.372+0000 7fb669800640 1 -- 172.21.15.5:0/1723573205 wait complete. 2024-04-25T10:05:17.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:17.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.596+0000 7fd3cb200640 1 Processor -- start 2024-04-25T10:05:17.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.597+0000 7fd3cb200640 1 -- start start 2024-04-25T10:05:17.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.598+0000 7fd3cb200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3cc0b4bb0 0x7fd3cc0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.598+0000 7fd3cb200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0cf410 con 0x7fd3cc0d07d0 2024-04-25T10:05:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.598+0000 7fd3cb200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0cf6f0 con 0x7fd3cc161200 2024-04-25T10:05:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.598+0000 7fd3cb200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b28b0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3c1400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd3cc0d07d0 0x7fd3cc0b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43046/0 (socket says 172.21.15.5:43046) 2024-04-25T10:05:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3c1400640 1 -- 172.21.15.5:0/3042876244 learned_addr learned my addr 172.21.15.5:0/3042876244 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3ca800640 1 --2- 172.21.15.5:0/3042876244 >> v2:172.21.15.5:3300/0 conn(0x7fd3cc0b4bb0 0x7fd3cc0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3c9400640 1 -- 172.21.15.5:0/3042876244 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484976280 0 0) 0x7fd3cc0cf410 con 0x7fd3cc0d07d0 2024-04-25T10:05:17.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3ca800640 1 -- 172.21.15.5:0/3042876244 >> v1:172.21.15.80:6789/0 conn(0x7fd3cc161200 legacy=0x7fd3cc0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3ca800640 1 -- 172.21.15.5:0/3042876244 >> v1:172.21.15.5:6789/0 conn(0x7fd3cc0d07d0 legacy=0x7fd3cc0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.599+0000 7fd3ca800640 1 -- 172.21.15.5:0/3042876244 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0d3fd0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.600+0000 7fd3ca800640 1 --2- 172.21.15.5:0/3042876244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc0d2380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009d90 tx=0x7fd3c402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e834e429300eaf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.600+0000 7fd3c9400640 1 -- 172.21.15.5:0/3042876244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4007740 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3c9400640 1 -- 172.21.15.5:0/3042876244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c40078e0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3c9400640 1 -- 172.21.15.5:0/3042876244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4007cb0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 -- 172.21.15.5:0/3042876244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 msgr2=0x7fd3cc0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3042876244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc0d2380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4009d90 tx=0x7fd3c402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 -- 172.21.15.5:0/3042876244 shutdown_connections 2024-04-25T10:05:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3042876244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc0d2380 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 -- 172.21.15.5:0/3042876244 >> 172.21.15.5:0/3042876244 conn(0x7fd3cc08af80 msgr2=0x7fd3cc0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 -- 172.21.15.5:0/3042876244 shutdown_connections 2024-04-25T10:05:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.601+0000 7fd3cb200640 1 -- 172.21.15.5:0/3042876244 wait complete. 2024-04-25T10:05:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.603+0000 7fd3cb200640 1 Processor -- start 2024-04-25T10:05:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.603+0000 7fd3cb200640 1 -- start start 2024-04-25T10:05:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3cb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3cb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3cc0d07d0 0x7fd3cc0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3cb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3cc161200 0x7fd3cc0c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3cb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b3810 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3cb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b3690 con 0x7fd3cc0d07d0 2024-04-25T10:05:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3cb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b3990 con 0x7fd3cc161200 2024-04-25T10:05:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.604+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40870/0 (socket says 172.21.15.5:40870) 2024-04-25T10:05:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 -- 172.21.15.5:0/3607705268 learned_addr learned my addr 172.21.15.5:0/3607705268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 -- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3cc161200 msgr2=0x7fd3cc0c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3cc161200 0x7fd3cc0c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 -- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3cc0d07d0 msgr2=0x7fd3cc0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3cc0d07d0 0x7fd3cc0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 -- 172.21.15.5:0/3607705268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0cd0d0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c9e00640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc15bf70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c430 tx=0x7fd3bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc017070 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0052d0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.605+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc003830 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.606+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc0cd3b0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.606+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc0c71c0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.607+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc068f60 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.608+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3bc01a070 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.609+0000 7fd3c2a00640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd3a806b500 0x7fd3a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.609+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fd3bc0897c0 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.609+0000 7fd3c1400640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd3a806b500 0x7fd3a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.611+0000 7fd3c1400640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd3a806b500 0x7fd3a806d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4007fd0 tx=0x7fd3b4007900 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.615+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3bc058400 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.840+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3cc0b5a70 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.842+0000 7fd3c2a00640 1 -- 172.21.15.5:0/3607705268 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7fd3bc057d00 con 0x7fd3cc0b4bb0 2024-04-25T10:05:17.848 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:17.848 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:05:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.844+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd3a806b500 msgr2=0x7fd3a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.844+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd3a806b500 0x7fd3a806d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4007fd0 tx=0x7fd3b4007900 comp rx=0 tx=0).stop 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.844+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 msgr2=0x7fd3cc15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.844+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc15bf70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00c430 tx=0x7fd3bc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 shutdown_connections 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3cc161200 0x7fd3cc0c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd3a806b500 0x7fd3a806d9c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3cc0d07d0 0x7fd3cc0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 --2- 172.21.15.5:0/3607705268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3cc0b4bb0 0x7fd3cc15bf70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 >> 172.21.15.5:0/3607705268 conn(0x7fd3cc08af80 msgr2=0x7fd3cc15fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 shutdown_connections 2024-04-25T10:05:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:17.845+0000 7fd3cb200640 1 -- 172.21.15.5:0/3607705268 wait complete. 2024-04-25T10:05:17.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:18.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.071+0000 7f2ae7200640 1 Processor -- start 2024-04-25T10:05:18.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.072+0000 7f2ae7200640 1 -- start start 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ae80d07d0 0x7f2ae80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae80cf410 con 0x7f2ae8161200 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae7200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae80cf6f0 con 0x7f2ae80d2380 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae80b28b0 con 0x7f2ae80d07d0 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae5e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2ae8161200 0x7f2ae80d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43058/0 (socket says 172.21.15.5:43058) 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae5e00640 1 -- 172.21.15.5:0/2910947267 learned_addr learned my addr 172.21.15.5:0/2910947267 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.073+0000 7f2ae5400640 1 --2- 172.21.15.5:0/2910947267 >> v2:172.21.15.5:3300/0 conn(0x7f2ae80d07d0 0x7f2ae80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.074+0000 7f2ae4a00640 1 -- 172.21.15.5:0/2910947267 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497225546 0 0) 0x7f2ae80cf410 con 0x7f2ae8161200 2024-04-25T10:05:18.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.074+0000 7f2ae4a00640 1 -- 172.21.15.5:0/2910947267 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ac8003520 con 0x7f2ae8161200 2024-04-25T10:05:18.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.074+0000 7f2ae5400640 1 -- 172.21.15.5:0/2910947267 >> v1:172.21.15.80:6789/0 conn(0x7f2ae80d2380 legacy=0x7f2ae80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.074+0000 7f2ae5400640 1 -- 172.21.15.5:0/2910947267 >> v1:172.21.15.5:6789/0 conn(0x7f2ae8161200 legacy=0x7f2ae80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.074+0000 7f2ae5400640 1 -- 172.21.15.5:0/2910947267 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae80d3fd0 con 0x7f2ae80d07d0 2024-04-25T10:05:18.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.075+0000 7f2ae5400640 1 --2- 172.21.15.5:0/2910947267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80b4bb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00097a0 tx=0x7f2ad002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992248a94f1a261e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.075+0000 7f2ae4a00640 1 -- 172.21.15.5:0/2910947267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0037070 con 0x7f2ae80d07d0 2024-04-25T10:05:18.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.075+0000 7f2ae4a00640 1 -- 172.21.15.5:0/2910947267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad0032440 con 0x7f2ae80d07d0 2024-04-25T10:05:18.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.075+0000 7f2ae4a00640 1 -- 172.21.15.5:0/2910947267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad00327f0 con 0x7f2ae80d07d0 2024-04-25T10:05:18.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 -- 172.21.15.5:0/2910947267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae80d07d0 msgr2=0x7f2ae80b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2910947267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80b4bb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00097a0 tx=0x7f2ad002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 -- 172.21.15.5:0/2910947267 shutdown_connections 2024-04-25T10:05:18.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2910947267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80b4bb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 -- 172.21.15.5:0/2910947267 >> 172.21.15.5:0/2910947267 conn(0x7f2ae808af80 msgr2=0x7f2ae80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 -- 172.21.15.5:0/2910947267 shutdown_connections 2024-04-25T10:05:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.076+0000 7f2ae7200640 1 -- 172.21.15.5:0/2910947267 wait complete. 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.078+0000 7f2ae7200640 1 Processor -- start 2024-04-25T10:05:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.078+0000 7f2ae7200640 1 -- start start 2024-04-25T10:05:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.079+0000 7f2ae7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.079+0000 7f2ae7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ae80d2380 0x7f2ae81f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.079+0000 7f2ae7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 0x7f2ae80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.079+0000 7f2ae7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae80b3810 con 0x7f2ae8161200 2024-04-25T10:05:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.079+0000 7f2ae7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae80b3690 con 0x7f2ae80d07d0 2024-04-25T10:05:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.079+0000 7f2ae7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae80b3990 con 0x7f2ae80d2380 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 0x7f2ae80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 0x7f2ae80cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40886/0 (socket says 172.21.15.5:40886) 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 -- 172.21.15.5:0/2282937836 learned_addr learned my addr 172.21.15.5:0/2282937836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae5400640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ae80d2380 0x7f2ae81f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae5e00640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 -- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ae80d2380 msgr2=0x7f2ae81f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ae80d2380 0x7f2ae81f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 -- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ae80d07d0 msgr2=0x7f2ae80c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 -- 172.21.15.5:0/2282937836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae81f3d00 con 0x7f2ae8161200 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae5e00640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.080+0000 7f2ae6800640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 0x7f2ae80cc980 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000c780 tx=0x7f2ae000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.081+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0004b60 con 0x7f2ae8161200 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.081+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae0004d00 con 0x7f2ae8161200 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.081+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000aa90 con 0x7f2ae8161200 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.081+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae81f3fe0 con 0x7f2ae8161200 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.081+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae80c83b0 con 0x7f2ae8161200 2024-04-25T10:05:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.082+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8068f60 con 0x7f2ae8161200 2024-04-25T10:05:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.083+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ae000ac30 con 0x7f2ae8161200 2024-04-25T10:05:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.085+0000 7f2ade000640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ac406b4d0 0x7f2ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.085+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2ae008a710 con 0x7f2ae8161200 2024-04-25T10:05:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.086+0000 7f2ae5e00640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ac406b4d0 0x7f2ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.087+0000 7f2ae5e00640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ac406b4d0 0x7f2ac406d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8005880 tx=0x7f2ad8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.089+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ae0020070 con 0x7f2ae8161200 2024-04-25T10:05:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.287+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ae80d3d10 con 0x7f2ae8161200 2024-04-25T10:05:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.288+0000 7f2ade000640 1 -- 172.21.15.5:0/2282937836 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16823 (secure 0 0 0) 0x7f2ae0059340 con 0x7f2ae8161200 2024-04-25T10:05:18.295 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:18.295 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:41.582074+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2b","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ac406b4d0 msgr2=0x7f2ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ac406b4d0 0x7f2ac406d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8005880 tx=0x7f2ad8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 msgr2=0x7f2ae80cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 0x7f2ae80cc980 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000c780 tx=0x7f2ae000cc40 comp rx=0 tx=0).stop 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 shutdown_connections 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ae8161200 0x7f2ae80cc980 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ae80d2380 0x7f2ae81f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ac406b4d0 0x7f2ac406d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 --2- 172.21.15.5:0/2282937836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ae80d07d0 0x7f2ae80c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 >> 172.21.15.5:0/2282937836 conn(0x7f2ae808af80 msgr2=0x7f2ae80d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 shutdown_connections 2024-04-25T10:05:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.291+0000 7f2ae7200640 1 -- 172.21.15.5:0/2282937836 wait complete. 2024-04-25T10:05:18.316 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:18.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:18.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.500+0000 7f3eabe00640 1 Processor -- start 2024-04-25T10:05:18.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.500+0000 7f3eabe00640 1 -- start start 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eabe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3eac141da0 0x7f3eac06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eabe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac071d30 con 0x7f3eac15d510 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eabe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac06d620 con 0x7f3eac06e640 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eabe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac142180 con 0x7f3eac141da0 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eab400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3eac15d510 0x7f3eac0cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43068/0 (socket says 172.21.15.5:43068) 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eaaa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3eac141da0 0x7f3eac06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eab400640 1 -- 172.21.15.5:0/1711345804 learned_addr learned my addr 172.21.15.5:0/1711345804 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3ea9600640 1 -- 172.21.15.5:0/1711345804 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694131558 0 0) 0x7f3eac071d30 con 0x7f3eac15d510 2024-04-25T10:05:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3ea9600640 1 -- 172.21.15.5:0/1711345804 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e90003520 con 0x7f3eac15d510 2024-04-25T10:05:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eaaa00640 1 -- 172.21.15.5:0/1711345804 >> v1:172.21.15.80:6789/0 conn(0x7f3eac06e640 legacy=0x7f3eac15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eaaa00640 1 -- 172.21.15.5:0/1711345804 >> v1:172.21.15.5:6789/0 conn(0x7f3eac15d510 legacy=0x7f3eac0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.503+0000 7f3eaaa00640 1 -- 172.21.15.5:0/1711345804 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac06d900 con 0x7f3eac141da0 2024-04-25T10:05:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.504+0000 7f3eaaa00640 1 --2- 172.21.15.5:0/1711345804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac141da0 0x7f3eac06e100 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0097a0 tx=0x7f3e9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde728d82ee42e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3ea9600640 1 -- 172.21.15.5:0/1711345804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c037070 con 0x7f3eac141da0 2024-04-25T10:05:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3ea9600640 1 -- 172.21.15.5:0/1711345804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c032440 con 0x7f3eac141da0 2024-04-25T10:05:18.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3ea9600640 1 -- 172.21.15.5:0/1711345804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c032810 con 0x7f3eac141da0 2024-04-25T10:05:18.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3eabe00640 1 -- 172.21.15.5:0/1711345804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac141da0 msgr2=0x7f3eac06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3eabe00640 1 --2- 172.21.15.5:0/1711345804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac141da0 0x7f3eac06e100 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0097a0 tx=0x7f3e9c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3eabe00640 1 -- 172.21.15.5:0/1711345804 shutdown_connections 2024-04-25T10:05:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3eabe00640 1 --2- 172.21.15.5:0/1711345804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac141da0 0x7f3eac06e100 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.505+0000 7f3eabe00640 1 -- 172.21.15.5:0/1711345804 >> 172.21.15.5:0/1711345804 conn(0x7f3eac066f20 msgr2=0x7f3eac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.506+0000 7f3eabe00640 1 -- 172.21.15.5:0/1711345804 shutdown_connections 2024-04-25T10:05:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.506+0000 7f3eabe00640 1 -- 172.21.15.5:0/1711345804 wait complete. 2024-04-25T10:05:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.507+0000 7f3eabe00640 1 Processor -- start 2024-04-25T10:05:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.507+0000 7f3eabe00640 1 -- start start 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eabe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eac06e640 0x7f3eac1446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eabe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eac141da0 0x7f3eac0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eabe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 0x7f3eac1f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eabe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0cdb90 con 0x7f3eac1375a0 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eabe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0cda10 con 0x7f3eac06e640 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eabe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac071000 con 0x7f3eac141da0 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.508+0000 7f3eab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 0x7f3eac1f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 0x7f3eac1f40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40896/0 (socket says 172.21.15.5:40896) 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 -- 172.21.15.5:0/2270715258 learned_addr learned my addr 172.21.15.5:0/2270715258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eaa000640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eac141da0 0x7f3eac0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 -- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eac141da0 msgr2=0x7f3eac0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eac141da0 0x7f3eac0b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 -- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eac06e640 msgr2=0x7f3eac1446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eaaa00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eac06e640 0x7f3eac1446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eac06e640 0x7f3eac1446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 -- 172.21.15.5:0/2270715258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac137b10 con 0x7f3eac1375a0 2024-04-25T10:05:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3eab400640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 0x7f3eac1f40b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400caa0 tx=0x7f3ea400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.509+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4017070 con 0x7f3eac1375a0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.510+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea400aba0 con 0x7f3eac1375a0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.510+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4012400 con 0x7f3eac1375a0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.510+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac137df0 con 0x7f3eac1375a0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.510+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1383b0 con 0x7f3eac1375a0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.511+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac068f60 con 0x7f3eac1375a0 2024-04-25T10:05:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.512+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ea401a070 con 0x7f3eac1375a0 2024-04-25T10:05:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.513+0000 7f3ea2a00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3e8c06b4d0 0x7f3e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.513+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3ea4089fe0 con 0x7f3eac1375a0 2024-04-25T10:05:18.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.514+0000 7f3eaaa00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3e8c06b4d0 0x7f3e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.515+0000 7f3eaaa00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3e8c06b4d0 0x7f3e8c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c034000 tx=0x7f3e9c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.518+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ea4058600 con 0x7f3eac1375a0 2024-04-25T10:05:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.752+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eac06d220 con 0x7f3eac1375a0 2024-04-25T10:05:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.754+0000 7f3ea2a00640 1 -- 172.21.15.5:0/2270715258 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f3ea405c6a0 con 0x7f3eac1375a0 2024-04-25T10:05:18.760 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:05:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.756+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3e8c06b4d0 msgr2=0x7f3e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.756+0000 7f3eabe00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3e8c06b4d0 0x7f3e8c06d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c034000 tx=0x7f3e9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.756+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 msgr2=0x7f3eac1f40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.756+0000 7f3eabe00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 0x7f3eac1f40b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400caa0 tx=0x7f3ea400cf60 comp rx=0 tx=0).stop 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 shutdown_connections 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eac1375a0 0x7f3eac1f40b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eac141da0 0x7f3eac0b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3e8c06b4d0 0x7f3e8c06d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 --2- 172.21.15.5:0/2270715258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eac06e640 0x7f3eac1446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 >> 172.21.15.5:0/2270715258 conn(0x7f3eac066f20 msgr2=0x7f3eac0b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 shutdown_connections 2024-04-25T10:05:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.757+0000 7f3eabe00640 1 -- 172.21.15.5:0/2270715258 wait complete. 2024-04-25T10:05:18.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.983+0000 7fb887200640 1 Processor -- start 2024-04-25T10:05:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.983+0000 7fb887200640 1 -- start start 2024-04-25T10:05:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb887200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb887200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb8880d07d0 0x7fb8880b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb887200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8880cf410 con 0x7fb8880d2380 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb887200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880d27c0 con 0x7fb8880d07d0 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb887200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880b28b0 con 0x7fb888161200 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb87d400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb8880d2380 0x7fb8880d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43078/0 (socket says 172.21.15.5:43078) 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb87d400640 1 -- 172.21.15.5:0/3593193844 learned_addr learned my addr 172.21.15.5:0/3593193844 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb886800640 1 --2- 172.21.15.5:0/3593193844 >> v2:172.21.15.5:3300/0 conn(0x7fb8880d07d0 0x7fb8880b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb885e00640 1 --2- 172.21.15.5:0/3593193844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb886800640 1 -- 172.21.15.5:0/3593193844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 msgr2=0x7fb8880b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb886800640 1 --2- 172.21.15.5:0/3593193844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb885400640 1 -- 172.21.15.5:0/3593193844 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2021073488 0 0) 0x7fb8880cf410 con 0x7fb8880d2380 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb886800640 1 -- 172.21.15.5:0/3593193844 >> v1:172.21.15.5:6789/0 conn(0x7fb8880d2380 legacy=0x7fb8880d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.985+0000 7fb886800640 1 -- 172.21.15.5:0/3593193844 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880cf6f0 con 0x7fb8880d07d0 2024-04-25T10:05:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.986+0000 7fb886800640 1 --2- 172.21.15.5:0/3593193844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb8880b2f80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb8800097d0 tx=0x7fb88002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf1b1e4abe81823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb885400640 1 -- 172.21.15.5:0/3593193844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880037070 con 0x7fb8880d07d0 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb885400640 1 -- 172.21.15.5:0/3593193844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880032440 con 0x7fb8880d07d0 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb885400640 1 -- 172.21.15.5:0/3593193844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880032810 con 0x7fb8880d07d0 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb887200640 1 -- 172.21.15.5:0/3593193844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 msgr2=0x7fb8880b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb887200640 1 --2- 172.21.15.5:0/3593193844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb8880b2f80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb8800097d0 tx=0x7fb88002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb887200640 1 -- 172.21.15.5:0/3593193844 shutdown_connections 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb887200640 1 --2- 172.21.15.5:0/3593193844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb8880b2f80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb887200640 1 --2- 172.21.15.5:0/3593193844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.987+0000 7fb887200640 1 -- 172.21.15.5:0/3593193844 >> 172.21.15.5:0/3593193844 conn(0x7fb88808af80 msgr2=0x7fb8880d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.988+0000 7fb887200640 1 -- 172.21.15.5:0/3593193844 shutdown_connections 2024-04-25T10:05:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.988+0000 7fb887200640 1 -- 172.21.15.5:0/3593193844 wait complete. 2024-04-25T10:05:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.990+0000 7fb887200640 1 Processor -- start 2024-04-25T10:05:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.990+0000 7fb887200640 1 -- start start 2024-04-25T10:05:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.990+0000 7fb887200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb88815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.990+0000 7fb887200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8880d2380 0x7fb888152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb887200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb887200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880b34c0 con 0x7fb8880d07d0 2024-04-25T10:05:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb887200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880d42b0 con 0x7fb8880d2380 2024-04-25T10:05:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb887200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880b3640 con 0x7fb888161200 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb88815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb88815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40920/0 (socket says 172.21.15.5:40920) 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 -- 172.21.15.5:0/2014369713 learned_addr learned my addr 172.21.15.5:0/2014369713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 -- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 msgr2=0x7fb8880b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb886800640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb87d400640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8880d2380 0x7fb888152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 -- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8880d2380 msgr2=0x7fb888152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8880d2380 0x7fb888152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.991+0000 7fb885e00640 1 -- 172.21.15.5:0/2014369713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880b7bc0 con 0x7fb8880d07d0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.992+0000 7fb885e00640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb88815c1a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb878007e40 tx=0x7fb87800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.992+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878014620 con 0x7fb8880d07d0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.992+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb878019070 con 0x7fb8880d07d0 2024-04-25T10:05:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.992+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb878014a00 con 0x7fb8880d07d0 2024-04-25T10:05:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.992+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8880b6b20 con 0x7fb8880d07d0 2024-04-25T10:05:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.992+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8880b70e0 con 0x7fb8880d07d0 2024-04-25T10:05:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.993+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb888068f60 con 0x7fb8880d07d0 2024-04-25T10:05:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.996+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb878014ba0 con 0x7fb8880d07d0 2024-04-25T10:05:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.996+0000 7fb87ea00640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb86406b4d0 0x7fb86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.996+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb87808acf0 con 0x7fb8880d07d0 2024-04-25T10:05:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.997+0000 7fb87d400640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb86406b4d0 0x7fb86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:18.998+0000 7fb87d400640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb86406b4d0 0x7fb86406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb870007fd0 tx=0x7fb870009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.001+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb878059920 con 0x7fb8880d07d0 2024-04-25T10:05:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.241+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8880d2c70 con 0x7fb8880d07d0 2024-04-25T10:05:19.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.242+0000 7fb87ea00640 1 -- 172.21.15.5:0/2014369713 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4929 (secure 0 0 0) 0x7fb878059220 con 0x7fb8880d07d0 2024-04-25T10:05:19.248 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:19.249 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:24.928361+0000","modified":"2024-04-25T10:04:27.993527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5365},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5365":{"gid":5365,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6836/2115515655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2115515655},{"type":"v1","addr":"172.21.15.80:6836","nonce":2115515655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:19.249 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb86406b4d0 msgr2=0x7fb86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb86406b4d0 0x7fb86406d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb870007fd0 tx=0x7fb870009040 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 msgr2=0x7fb88815c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb88815c1a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb878007e40 tx=0x7fb87800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 shutdown_connections 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb888161200 0x7fb8880b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb86406b4d0 0x7fb86406d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8880d2380 0x7fb888152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 --2- 172.21.15.5:0/2014369713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8880d07d0 0x7fb88815c1a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.245+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 >> 172.21.15.5:0/2014369713 conn(0x7fb88808af80 msgr2=0x7fb88806d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.246+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 shutdown_connections 2024-04-25T10:05:19.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.246+0000 7fb887200640 1 -- 172.21.15.5:0/2014369713 wait complete. 2024-04-25T10:05:19.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.474+0000 7fd2c4c00640 1 Processor -- start 2024-04-25T10:05:19.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.474+0000 7fd2c4c00640 1 -- start start 2024-04-25T10:05:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2c4c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2c00cdca0 0x7fd2c015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0070f90 0x7fd2c0071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2c4c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c00cd200 con 0x7fd2c0154a10 2024-04-25T10:05:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2c4c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c0071940 con 0x7fd2c00cdca0 2024-04-25T10:05:19.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2c4c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c006e6f0 con 0x7fd2c0070f90 2024-04-25T10:05:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2bea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2c00cdca0 0x7fd2c015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2bea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2c00cdca0 0x7fd2c015e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40930/0 (socket says 172.21.15.5:40930) 2024-04-25T10:05:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.476+0000 7fd2bea00640 1 -- 172.21.15.5:0/85101037 learned_addr learned my addr 172.21.15.5:0/85101037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:19.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.477+0000 7fd2bfe00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0070f90 0x7fd2c0071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.477+0000 7fd2bea00640 1 -- 172.21.15.5:0/85101037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0070f90 msgr2=0x7fd2c0071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.477+0000 7fd2bea00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0070f90 0x7fd2c0071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.477+0000 7fd2bea00640 1 -- 172.21.15.5:0/85101037 >> v1:172.21.15.80:6789/0 conn(0x7fd2c0154a10 legacy=0x7fd2c00b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.477+0000 7fd2bea00640 1 -- 172.21.15.5:0/85101037 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c006e9d0 con 0x7fd2c00cdca0 2024-04-25T10:05:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.477+0000 7fd2bfe00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0070f90 0x7fd2c0071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:19.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.478+0000 7fd2bea00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00cdca0 0x7fd2c015e8c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009870 tx=0x7fd2ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3bcf86e27fe9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.478+0000 7fd2be000640 1 -- 172.21.15.5:0/85101037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac037070 con 0x7fd2c00cdca0 2024-04-25T10:05:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.478+0000 7fd2be000640 1 -- 172.21.15.5:0/85101037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac032440 con 0x7fd2c00cdca0 2024-04-25T10:05:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.478+0000 7fd2be000640 1 -- 172.21.15.5:0/85101037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac032810 con 0x7fd2c00cdca0 2024-04-25T10:05:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 -- 172.21.15.5:0/85101037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00cdca0 msgr2=0x7fd2c015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00cdca0 0x7fd2c015e8c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009870 tx=0x7fd2ac02a180 comp rx=0 tx=0).stop 2024-04-25T10:05:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 -- 172.21.15.5:0/85101037 shutdown_connections 2024-04-25T10:05:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0070f90 0x7fd2c0071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/85101037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00cdca0 0x7fd2c015e8c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 -- 172.21.15.5:0/85101037 >> 172.21.15.5:0/85101037 conn(0x7fd2c0066f20 msgr2=0x7fd2c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 -- 172.21.15.5:0/85101037 shutdown_connections 2024-04-25T10:05:19.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.479+0000 7fd2c4c00640 1 -- 172.21.15.5:0/85101037 wait complete. 2024-04-25T10:05:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.481+0000 7fd2c4c00640 1 Processor -- start 2024-04-25T10:05:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.481+0000 7fd2c4c00640 1 -- start start 2024-04-25T10:05:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 0x7fd2c013fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2c00cdca0 0x7fd2c00b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0154a10 0x7fd2c00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2c4c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00cd780 con 0x7fd2c0070f90 2024-04-25T10:05:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2c4c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00cd4a0 con 0x7fd2c00cdca0 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2c4c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c006cff0 con 0x7fd2c0154a10 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 0x7fd2c013fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 0x7fd2c013fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40932/0 (socket says 172.21.15.5:40932) 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.482+0000 7fd2bf400640 1 -- 172.21.15.5:0/3692736896 learned_addr learned my addr 172.21.15.5:0/3692736896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bea00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2c00cdca0 0x7fd2c00b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bfe00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0154a10 0x7fd2c00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bf400640 1 -- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0154a10 msgr2=0x7fd2c00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bf400640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0154a10 0x7fd2c00b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bf400640 1 -- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2c00cdca0 msgr2=0x7fd2c00b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bf400640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2c00cdca0 0x7fd2c00b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bf400640 1 -- 172.21.15.5:0/3692736896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c01433d0 con 0x7fd2c0070f90 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bea00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2c00cdca0 0x7fd2c00b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2bf400640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 0x7fd2c013fcf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000b3f0 tx=0x7fd2b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0015020 con 0x7fd2c0070f90 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b0011830 con 0x7fd2c0070f90 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0011c20 con 0x7fd2c0070f90 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.483+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01e91f0 con 0x7fd2c0070f90 2024-04-25T10:05:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.484+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01e97b0 con 0x7fd2c0070f90 2024-04-25T10:05:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.485+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c0068f60 con 0x7fd2c0070f90 2024-04-25T10:05:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.486+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2b0018020 con 0x7fd2c0070f90 2024-04-25T10:05:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.487+0000 7fd2b7400640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2a006b4d0 0x7fd2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.487+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fd2b008a070 con 0x7fd2c0070f90 2024-04-25T10:05:19.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.487+0000 7fd2bea00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2a006b4d0 0x7fd2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.489+0000 7fd2bea00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2a006b4d0 0x7fd2a006d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac034000 tx=0x7fd2ac02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.491+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2b001f030 con 0x7fd2c0070f90 2024-04-25T10:05:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.684+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c00cce40 con 0x7fd2c0070f90 2024-04-25T10:05:19.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.688+0000 7fd2b7400640 1 -- 172.21.15.5:0/3692736896 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16823 (secure 0 0 0) 0x7fd2b00172c0 con 0x7fd2c0070f90 2024-04-25T10:05:19.693 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:19.693 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:04:41.582074+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:22.770291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:23.937149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2b","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:19.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2a006b4d0 msgr2=0x7fd2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2a006b4d0 0x7fd2a006d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac034000 tx=0x7fd2ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:19.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 msgr2=0x7fd2c013fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 0x7fd2c013fcf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000b3f0 tx=0x7fd2b000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 shutdown_connections 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2c0154a10 0x7fd2c00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2a006b4d0 0x7fd2a006d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2c00cdca0 0x7fd2c00b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 --2- 172.21.15.5:0/3692736896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c0070f90 0x7fd2c013fcf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.690+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 >> 172.21.15.5:0/3692736896 conn(0x7fd2c0066f20 msgr2=0x7fd2c00af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.691+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 shutdown_connections 2024-04-25T10:05:19.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.691+0000 7fd2c4c00640 1 -- 172.21.15.5:0/3692736896 wait complete. 2024-04-25T10:05:19.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.917+0000 7ff79da00640 1 Processor -- start 2024-04-25T10:05:19.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.917+0000 7ff79da00640 1 -- start start 2024-04-25T10:05:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff79da00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff798141da0 0x7ff79806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff79da00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff79815d510 0x7ff7980cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff79da00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff798071d30 con 0x7ff79806e640 2024-04-25T10:05:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff79da00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff798142180 con 0x7ff79815d510 2024-04-25T10:05:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff79da00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79806d620 con 0x7ff798141da0 2024-04-25T10:05:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff78f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff79806e640 0x7ff79815ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43080/0 (socket says 172.21.15.5:43080) 2024-04-25T10:05:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff78f400640 1 -- 172.21.15.5:0/3120256995 learned_addr learned my addr 172.21.15.5:0/3120256995 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.919+0000 7ff79d000640 1 --2- 172.21.15.5:0/3120256995 >> v2:172.21.15.5:3300/0 conn(0x7ff79815d510 0x7ff7980cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff797e00640 1 --2- 172.21.15.5:0/3120256995 >> v2:172.21.15.80:3300/0 conn(0x7ff798141da0 0x7ff79806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff797400640 1 -- 172.21.15.5:0/3120256995 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439388097 0 0) 0x7ff798071d30 con 0x7ff79806e640 2024-04-25T10:05:19.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff797400640 1 -- 172.21.15.5:0/3120256995 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff77c003520 con 0x7ff79806e640 2024-04-25T10:05:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff79d000640 1 -- 172.21.15.5:0/3120256995 >> v2:172.21.15.80:3300/0 conn(0x7ff798141da0 msgr2=0x7ff79806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff79d000640 1 --2- 172.21.15.5:0/3120256995 >> v2:172.21.15.80:3300/0 conn(0x7ff798141da0 0x7ff79806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff79d000640 1 -- 172.21.15.5:0/3120256995 >> v1:172.21.15.5:6789/0 conn(0x7ff79806e640 legacy=0x7ff79815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.920+0000 7ff79d000640 1 -- 172.21.15.5:0/3120256995 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79806d900 con 0x7ff79815d510 2024-04-25T10:05:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.921+0000 7ff79d000640 1 --2- 172.21.15.5:0/3120256995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79815d510 0x7ff7980cd300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff79806bbb0 tx=0x7ff79002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfff61bb6ed8a8f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.921+0000 7ff797400640 1 -- 172.21.15.5:0/3120256995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007740 con 0x7ff79815d510 2024-04-25T10:05:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.921+0000 7ff797400640 1 -- 172.21.15.5:0/3120256995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7900078e0 con 0x7ff79815d510 2024-04-25T10:05:19.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.921+0000 7ff797400640 1 -- 172.21.15.5:0/3120256995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790007cb0 con 0x7ff79815d510 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 -- 172.21.15.5:0/3120256995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79815d510 msgr2=0x7ff7980cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 --2- 172.21.15.5:0/3120256995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79815d510 0x7ff7980cd300 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff79806bbb0 tx=0x7ff79002a680 comp rx=0 tx=0).stop 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 -- 172.21.15.5:0/3120256995 shutdown_connections 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 --2- 172.21.15.5:0/3120256995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79815d510 0x7ff7980cd300 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 --2- 172.21.15.5:0/3120256995 >> v2:172.21.15.80:3300/0 conn(0x7ff798141da0 0x7ff79806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 -- 172.21.15.5:0/3120256995 >> 172.21.15.5:0/3120256995 conn(0x7ff798066f20 msgr2=0x7ff79806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 -- 172.21.15.5:0/3120256995 shutdown_connections 2024-04-25T10:05:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.922+0000 7ff79da00640 1 -- 172.21.15.5:0/3120256995 wait complete. 2024-04-25T10:05:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.925+0000 7ff79da00640 1 Processor -- start 2024-04-25T10:05:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.925+0000 7ff79da00640 1 -- start start 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff79da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 0x7ff79814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff79da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff798141da0 0x7ff79814b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff79da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff79814eda0 0x7ff79814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff79da00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cd9c0 con 0x7ff79806e640 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff79da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cd840 con 0x7ff798141da0 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff79da00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cdb40 con 0x7ff79814eda0 2024-04-25T10:05:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff797e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 0x7ff79814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff797e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 0x7ff79814da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40956/0 (socket says 172.21.15.5:40956) 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.926+0000 7ff797e00640 1 -- 172.21.15.5:0/3279359021 learned_addr learned my addr 172.21.15.5:0/3279359021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff78f400640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff798141da0 0x7ff79814b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff79d000640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff79814eda0 0x7ff79814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff797e00640 1 -- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff79814eda0 msgr2=0x7ff79814c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff797e00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff79814eda0 0x7ff79814c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff797e00640 1 -- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff798141da0 msgr2=0x7ff79814b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff797e00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff798141da0 0x7ff79814b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff797e00640 1 -- 172.21.15.5:0/3279359021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79814c980 con 0x7ff79806e640 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff79d000640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff79814eda0 0x7ff79814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.927+0000 7ff797e00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 0x7ff79814da20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9b0 tx=0x7ff78800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.928+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788017070 con 0x7ff79806e640 2024-04-25T10:05:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.928+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800c9a0 con 0x7ff79806e640 2024-04-25T10:05:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.928+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800cd90 con 0x7ff79806e640 2024-04-25T10:05:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.928+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79814cc60 con 0x7ff79806e640 2024-04-25T10:05:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.928+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79814d220 con 0x7ff79806e640 2024-04-25T10:05:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.929+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff798068f60 con 0x7ff79806e640 2024-04-25T10:05:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.930+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff78801a070 con 0x7ff79806e640 2024-04-25T10:05:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.931+0000 7ff794c00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff77406b4d0 0x7ff77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.931+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff78808a800 con 0x7ff79806e640 2024-04-25T10:05:19.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.932+0000 7ff78f400640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff77406b4d0 0x7ff77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.933+0000 7ff78f400640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff77406b4d0 0x7ff77406d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b2910 tx=0x7ff780009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:19.936+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff788059430 con 0x7ff79806e640 2024-04-25T10:05:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.175+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff79815b750 con 0x7ff79806e640 2024-04-25T10:05:20.181 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:05:20.177+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.181+0000 7ff794c00640 1 -- 172.21.15.5:0/3279359021 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff788058d30 con 0x7ff79806e640 2024-04-25T10:05:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff77406b4d0 msgr2=0x7ff77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff77406b4d0 0x7ff77406d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff7980b2910 tx=0x7ff780009040 comp rx=0 tx=0).stop 2024-04-25T10:05:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 msgr2=0x7ff79814da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 0x7ff79814da20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9b0 tx=0x7ff78800ee70 comp rx=0 tx=0).stop 2024-04-25T10:05:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 shutdown_connections 2024-04-25T10:05:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff79814eda0 0x7ff79814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff77406b4d0 0x7ff77406d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff798141da0 0x7ff79814b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.183+0000 7ff79da00640 1 --2- 172.21.15.5:0/3279359021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff79806e640 0x7ff79814da20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.184+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 >> 172.21.15.5:0/3279359021 conn(0x7ff798066f20 msgr2=0x7ff79806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.184+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 shutdown_connections 2024-04-25T10:05:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.184+0000 7ff79da00640 1 -- 172.21.15.5:0/3279359021 wait complete. 2024-04-25T10:05:20.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:20.226 INFO:tasks.ceph.mds.d.smithi080.stderr: -6105> 2024-04-25T10:04:31.406+0000 7f7ed5400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:20.236 INFO:tasks.ceph.mds.d.smithi080.stderr: -3307> 2024-04-25T10:04:40.554+0000 7f7ed0400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5934=0-4194304@1} caps={5934=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5934 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x555902e54100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:20.299 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:05:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.409+0000 7fa903200640 1 Processor -- start 2024-04-25T10:05:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.409+0000 7fa903200640 1 -- start start 2024-04-25T10:05:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.410+0000 7fa903200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040b48c0 0x7fa90415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.410+0000 7fa903200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9040d3f90 0x7fa904161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa903200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9041606e0 con 0x7fa904161bc0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa903200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9041609c0 con 0x7fa9040d3f90 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa903200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa904160ca0 con 0x7fa9040b48c0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa8f9400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9040d3f90 0x7fa904161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa901e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040b48c0 0x7fa90415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa8f9400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9040d3f90 0x7fa904161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:42240/0 (socket says 172.21.15.5:42240) 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.411+0000 7fa8f9400640 1 -- 172.21.15.5:0/3446139862 learned_addr learned my addr 172.21.15.5:0/3446139862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:20.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.412+0000 7fa8f9400640 1 -- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040b48c0 msgr2=0x7fa90415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.412+0000 7fa8f9400640 1 --2- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040b48c0 0x7fa90415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.412+0000 7fa8f9400640 1 -- 172.21.15.5:0/3446139862 >> v1:172.21.15.80:6789/0 conn(0x7fa904161bc0 legacy=0x7fa9040d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.412+0000 7fa8f9400640 1 -- 172.21.15.5:0/3446139862 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa904160f80 con 0x7fa9040d3f90 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.413+0000 7fa8f9400640 1 --2- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040d3f90 0x7fa904161680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009850 tx=0x7fa8ec02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8c87d6e825ae88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.414+0000 7fa901400640 1 -- 172.21.15.5:0/3446139862 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec004030 con 0x7fa9040d3f90 2024-04-25T10:05:20.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.414+0000 7fa901400640 1 -- 172.21.15.5:0/3446139862 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec0041d0 con 0x7fa9040d3f90 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.414+0000 7fa901400640 1 -- 172.21.15.5:0/3446139862 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec003610 con 0x7fa9040d3f90 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.414+0000 7fa903200640 1 -- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040d3f90 msgr2=0x7fa904161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.414+0000 7fa903200640 1 --2- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040d3f90 0x7fa904161680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009850 tx=0x7fa8ec02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.415+0000 7fa903200640 1 -- 172.21.15.5:0/3446139862 shutdown_connections 2024-04-25T10:05:20.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.415+0000 7fa903200640 1 --2- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040d3f90 0x7fa904161680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.415+0000 7fa903200640 1 --2- 172.21.15.5:0/3446139862 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040b48c0 0x7fa90415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.415+0000 7fa903200640 1 -- 172.21.15.5:0/3446139862 >> 172.21.15.5:0/3446139862 conn(0x7fa90408af80 msgr2=0x7fa9040d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.415+0000 7fa903200640 1 -- 172.21.15.5:0/3446139862 shutdown_connections 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.415+0000 7fa903200640 1 -- 172.21.15.5:0/3446139862 wait complete. 2024-04-25T10:05:20.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.416+0000 7fa903200640 1 Processor -- start 2024-04-25T10:05:20.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.417+0000 7fa903200640 1 -- start start 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.417+0000 7fa903200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040b48c0 0x7fa904159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.417+0000 7fa903200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040d3f90 0x7fa9040cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.417+0000 7fa903200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 0x7fa9040b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa903200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9041603a0 con 0x7fa904161bc0 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa903200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040b4ca0 con 0x7fa9040b48c0 2024-04-25T10:05:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa903200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9040b3b80 con 0x7fa9040d3f90 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 0x7fa9040b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 0x7fa9040b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40966/0 (socket says 172.21.15.5:40966) 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 -- 172.21.15.5:0/476795942 learned_addr learned my addr 172.21.15.5:0/476795942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa901e00640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040b48c0 0x7fa904159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa8f9400640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040d3f90 0x7fa9040cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 -- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040d3f90 msgr2=0x7fa9040cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040d3f90 0x7fa9040cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 -- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040b48c0 msgr2=0x7fa904159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.418+0000 7fa902800640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040b48c0 0x7fa904159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa902800640 1 -- 172.21.15.5:0/476795942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040b7f40 con 0x7fa904161bc0 2024-04-25T10:05:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa901e00640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040b48c0 0x7fa904159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa902800640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 0x7fa9040b1dd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c6b0 tx=0x7fa8fc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc003b70 con 0x7fa904161bc0 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc003d30 con 0x7fa904161bc0 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc005310 con 0x7fa904161bc0 2024-04-25T10:05:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9040b8220 con 0x7fa904161bc0 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.419+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9040ca770 con 0x7fa904161bc0 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.421+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9041619e0 con 0x7fa904161bc0 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.421+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8fc0054b0 con 0x7fa904161bc0 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.422+0000 7fa8faa00640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.422+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fa8fc08a300 con 0x7fa904161bc0 2024-04-25T10:05:20.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.423+0000 7fa901e00640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.424+0000 7fa901e00640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009b90 tx=0x7fa8f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.426+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9041619e0 con 0x7fa904161bc0 2024-04-25T10:05:20.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:20.670+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa904068f60 con 0x7fa904161bc0 2024-04-25T10:05:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.191+0000 7fa8faa00640 1 -- 172.21.15.5:0/476795942 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8fc058ee0 con 0x7fa904161bc0 2024-04-25T10:05:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 msgr2=0x7fa8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009b90 tx=0x7fa8f4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 msgr2=0x7fa9040b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 0x7fa9040b1dd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00c6b0 tx=0x7fa8fc00cb70 comp rx=0 tx=0).stop 2024-04-25T10:05:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 shutdown_connections 2024-04-25T10:05:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa904161bc0 0x7fa9040b1dd0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9040d3f90 0x7fa9040cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa8e006b4d0 0x7fa8e006d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 --2- 172.21.15.5:0/476795942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9040b48c0 0x7fa904159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 >> 172.21.15.5:0/476795942 conn(0x7fa90408af80 msgr2=0x7fa90406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 shutdown_connections 2024-04-25T10:05:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.195+0000 7fa903200640 1 -- 172.21.15.5:0/476795942 wait complete. 2024-04-25T10:05:21.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.434+0000 7fad4dc00640 1 Processor -- start 2024-04-25T10:05:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.435+0000 7fad4dc00640 1 -- start start 2024-04-25T10:05:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.436+0000 7fad4dc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad480b6ba0 0x7fad4806bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.436+0000 7fad4dc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4806c1f0 con 0x7fad480d0820 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.436+0000 7fad4dc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4806c4d0 con 0x7fad4815db10 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.436+0000 7fad4dc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4806c7b0 con 0x7fad480b6ba0 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.436+0000 7fad3fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fad480d0820 0x7fad480d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43094/0 (socket says 172.21.15.5:43094) 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.437+0000 7fad3fe00640 1 -- 172.21.15.5:0/3142570388 learned_addr learned my addr 172.21.15.5:0/3142570388 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.437+0000 7fad4d200640 1 --2- 172.21.15.5:0/3142570388 >> v2:172.21.15.5:3300/0 conn(0x7fad480b6ba0 0x7fad4806bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.437+0000 7fad4d200640 1 -- 172.21.15.5:0/3142570388 >> v1:172.21.15.80:6789/0 conn(0x7fad4815db10 legacy=0x7fad480d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.437+0000 7fad4d200640 1 -- 172.21.15.5:0/3142570388 >> v1:172.21.15.5:6789/0 conn(0x7fad480d0820 legacy=0x7fad480d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.437+0000 7fad47400640 1 -- 172.21.15.5:0/3142570388 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940468721 0 0) 0x7fad4806c1f0 con 0x7fad480d0820 2024-04-25T10:05:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.437+0000 7fad4d200640 1 -- 172.21.15.5:0/3142570388 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4806ca90 con 0x7fad480b6ba0 2024-04-25T10:05:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.438+0000 7fad4d200640 1 --2- 172.21.15.5:0/3142570388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad4806bb00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fad40009d90 tx=0x7fad4002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb207b0136582012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.439+0000 7fad47400640 1 -- 172.21.15.5:0/3142570388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40007740 con 0x7fad480b6ba0 2024-04-25T10:05:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.439+0000 7fad47400640 1 -- 172.21.15.5:0/3142570388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad400078e0 con 0x7fad480b6ba0 2024-04-25T10:05:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.439+0000 7fad47400640 1 -- 172.21.15.5:0/3142570388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40007cb0 con 0x7fad480b6ba0 2024-04-25T10:05:21.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.439+0000 7fad4dc00640 1 -- 172.21.15.5:0/3142570388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 msgr2=0x7fad4806bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.439+0000 7fad4dc00640 1 --2- 172.21.15.5:0/3142570388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad4806bb00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fad40009d90 tx=0x7fad4002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.440+0000 7fad4dc00640 1 -- 172.21.15.5:0/3142570388 shutdown_connections 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.440+0000 7fad4dc00640 1 --2- 172.21.15.5:0/3142570388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad4806bb00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.440+0000 7fad4dc00640 1 -- 172.21.15.5:0/3142570388 >> 172.21.15.5:0/3142570388 conn(0x7fad480b5210 msgr2=0x7fad480b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.440+0000 7fad4dc00640 1 -- 172.21.15.5:0/3142570388 shutdown_connections 2024-04-25T10:05:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.440+0000 7fad4dc00640 1 -- 172.21.15.5:0/3142570388 wait complete. 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.442+0000 7fad4dc00640 1 Processor -- start 2024-04-25T10:05:21.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.442+0000 7fad4dc00640 1 -- start start 2024-04-25T10:05:21.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad4dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad48160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad4dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad480d0820 0x7fad4815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad4dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad4815db10 0x7fad4814a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad4dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad48162280 con 0x7fad480b6ba0 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad4dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815e100 con 0x7fad480d0820 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad4dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad48162400 con 0x7fad4815db10 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad47e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad48160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad47e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad48160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40978/0 (socket says 172.21.15.5:40978) 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.443+0000 7fad47e00640 1 -- 172.21.15.5:0/1074746863 learned_addr learned my addr 172.21.15.5:0/1074746863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad3fe00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad480d0820 0x7fad4815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad4d200640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad4815db10 0x7fad4814a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad47e00640 1 -- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad4815db10 msgr2=0x7fad4814a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad47e00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad4815db10 0x7fad4814a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad47e00640 1 -- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad480d0820 msgr2=0x7fad4815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad47e00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad480d0820 0x7fad4815a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad47e00640 1 -- 172.21.15.5:0/1074746863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4815e280 con 0x7fad480b6ba0 2024-04-25T10:05:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad47e00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad48160200 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c400 tx=0x7fad3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38017070 con 0x7fad480b6ba0 2024-04-25T10:05:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad380052d0 con 0x7fad480b6ba0 2024-04-25T10:05:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38003830 con 0x7fad480b6ba0 2024-04-25T10:05:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.444+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4815e520 con 0x7fad480b6ba0 2024-04-25T10:05:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.445+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4815eae0 con 0x7fad480b6ba0 2024-04-25T10:05:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.446+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad480677d0 con 0x7fad480b6ba0 2024-04-25T10:05:21.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.447+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad3801a070 con 0x7fad480b6ba0 2024-04-25T10:05:21.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.448+0000 7fad44c00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad2406b4d0 0x7fad2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.448+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fad38089890 con 0x7fad480b6ba0 2024-04-25T10:05:21.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.449+0000 7fad3fe00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad2406b4d0 0x7fad2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.450+0000 7fad3fe00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad2406b4d0 0x7fad2406d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad480cb8f0 tx=0x7fad30007bb0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.454+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad380583e0 con 0x7fad480b6ba0 2024-04-25T10:05:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:21.662+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad480ce140 con 0x7fad480b6ba0 2024-04-25T10:05:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.195+0000 7fad44c00640 1 -- 172.21.15.5:0/1074746863 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad38057ce0 con 0x7fad480b6ba0 2024-04-25T10:05:22.200 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad2406b4d0 msgr2=0x7fad2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad2406b4d0 0x7fad2406d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fad480cb8f0 tx=0x7fad30007bb0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 msgr2=0x7fad48160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad48160200 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fad3800c400 tx=0x7fad3800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 shutdown_connections 2024-04-25T10:05:22.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad4815db10 0x7fad4814a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad2406b4d0 0x7fad2406d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad480d0820 0x7fad4815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 --2- 172.21.15.5:0/1074746863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad480b6ba0 0x7fad48160200 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.199+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 >> 172.21.15.5:0/1074746863 conn(0x7fad480b5210 msgr2=0x7fad480ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.200+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 shutdown_connections 2024-04-25T10:05:22.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.200+0000 7fad4dc00640 1 -- 172.21.15.5:0/1074746863 wait complete. 2024-04-25T10:05:22.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:22.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.440+0000 7f84aec00640 1 Processor -- start 2024-04-25T10:05:22.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.440+0000 7f84aec00640 1 -- start start 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.442+0000 7f84aec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a8071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.442+0000 7f84aec00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84a8071540 0x7f84a80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.442+0000 7f84aec00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a806e850 con 0x7f84a8144e20 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.442+0000 7f84aec00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a8071970 con 0x7f84a8071540 2024-04-25T10:05:22.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.442+0000 7f84aec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a815fdf0 con 0x7f84a815c870 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.442+0000 7f84ae200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84a8071540 0x7f84a80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ae200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84a8071540 0x7f84a80cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40982/0 (socket says 172.21.15.5:40982) 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ae200640 1 -- 172.21.15.5:0/2252604660 learned_addr learned my addr 172.21.15.5:0/2252604660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ace00640 1 --2- 172.21.15.5:0/2252604660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ae200640 1 -- 172.21.15.5:0/2252604660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 msgr2=0x7f84a8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ae200640 1 --2- 172.21.15.5:0/2252604660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a8071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ae200640 1 -- 172.21.15.5:0/2252604660 >> v1:172.21.15.80:6789/0 conn(0x7f84a8144e20 legacy=0x7f84a815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.443+0000 7f84ae200640 1 -- 172.21.15.5:0/2252604660 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a81600d0 con 0x7f84a8071540 2024-04-25T10:05:22.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.445+0000 7f84ae200640 1 --2- 172.21.15.5:0/2252604660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80cbd60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009e50 tx=0x7f84a002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f1cae05cb8f35d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.445+0000 7f84a7e00640 1 -- 172.21.15.5:0/2252604660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a0036070 con 0x7f84a8071540 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.445+0000 7f84a7e00640 1 -- 172.21.15.5:0/2252604660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a0031600 con 0x7f84a8071540 2024-04-25T10:05:22.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.445+0000 7f84a7e00640 1 -- 172.21.15.5:0/2252604660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a00319d0 con 0x7f84a8071540 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 -- 172.21.15.5:0/2252604660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 msgr2=0x7f84a80cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 --2- 172.21.15.5:0/2252604660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80cbd60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009e50 tx=0x7f84a002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 -- 172.21.15.5:0/2252604660 shutdown_connections 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 --2- 172.21.15.5:0/2252604660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80cbd60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 --2- 172.21.15.5:0/2252604660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a8071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 -- 172.21.15.5:0/2252604660 >> 172.21.15.5:0/2252604660 conn(0x7f84a8066f20 msgr2=0x7f84a806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 -- 172.21.15.5:0/2252604660 shutdown_connections 2024-04-25T10:05:22.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.446+0000 7f84aec00640 1 -- 172.21.15.5:0/2252604660 wait complete. 2024-04-25T10:05:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.448+0000 7f84aec00640 1 Processor -- start 2024-04-25T10:05:22.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.448+0000 7f84aec00640 1 -- start start 2024-04-25T10:05:22.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.449+0000 7f84aec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.449+0000 7f84aec00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84a8144e20 0x7f84a80ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.449+0000 7f84aec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a813f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.449+0000 7f84aec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a8145200 con 0x7f84a8071540 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.449+0000 7f84aec00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a815c070 con 0x7f84a8144e20 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.449+0000 7f84aec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a8145380 con 0x7f84a815c870 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40996/0 (socket says 172.21.15.5:40996) 2024-04-25T10:05:22.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 -- 172.21.15.5:0/732893350 learned_addr learned my addr 172.21.15.5:0/732893350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ae200640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a813f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ace00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84a8144e20 0x7f84a80ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 -- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 msgr2=0x7f84a813f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a813f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 -- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84a8144e20 msgr2=0x7f84a80ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84a8144e20 0x7f84a80ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ad800640 1 -- 172.21.15.5:0/732893350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a81f47f0 con 0x7f84a8071540 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ae200640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a813f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.450+0000 7f84ace00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84a8144e20 0x7f84a80ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.451+0000 7f84ad800640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80ad950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c480 tx=0x7f849c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.451+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0052d0 con 0x7f84a8071540 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.451+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c005470 con 0x7f84a8071540 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.451+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c012870 con 0x7f84a8071540 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.451+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a813c7c0 con 0x7f84a8071540 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.451+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a813ed60 con 0x7f84a8071540 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.452+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a8068f60 con 0x7f84a8071540 2024-04-25T10:05:22.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.453+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f849c012a10 con 0x7f84a8071540 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.454+0000 7f84a5600640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f848806b4d0 0x7f848806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.454+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f849c089ef0 con 0x7f84a8071540 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.455+0000 7f84ace00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f848806b4d0 0x7f848806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.456+0000 7f84ace00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f848806b4d0 0x7f848806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84a815ea40 tx=0x7f8494008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.458+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f849c058d10 con 0x7f84a8071540 2024-04-25T10:05:22.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:22.657+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84a81405e0 con 0x7f84a8071540 2024-04-25T10:05:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.206+0000 7f84a5600640 1 -- 172.21.15.5:0/732893350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f849c058610 con 0x7f84a8071540 2024-04-25T10:05:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.210+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f848806b4d0 msgr2=0x7f848806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.210+0000 7f84aec00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f848806b4d0 0x7f848806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84a815ea40 tx=0x7f8494008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.210+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 msgr2=0x7f84a80ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.210+0000 7f84aec00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80ad950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f849c00c480 tx=0x7f849c00c940 comp rx=0 tx=0).stop 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 shutdown_connections 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84a815c870 0x7f84a813f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f848806b4d0 0x7f848806d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84a8144e20 0x7f84a80ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 --2- 172.21.15.5:0/732893350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84a8071540 0x7f84a80ad950 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 >> 172.21.15.5:0/732893350 conn(0x7f84a8066f20 msgr2=0x7f84a80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.211+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 shutdown_connections 2024-04-25T10:05:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.212+0000 7f84aec00640 1 -- 172.21.15.5:0/732893350 wait complete. 2024-04-25T10:05:23.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:05:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.424+0000 7f3de9200640 1 Processor -- start 2024-04-25T10:05:23.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.424+0000 7f3de9200640 1 -- start start 2024-04-25T10:05:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3de9200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3de9200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3de40d3f90 0x7f3de4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3de9200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de41606e0 con 0x7f3de4161bc0 2024-04-25T10:05:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3de9200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de41609c0 con 0x7f3de40d3f90 2024-04-25T10:05:23.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3de9200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de4160ca0 con 0x7f3de40b48c0 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3ddbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3de40d3f90 0x7f3de4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3ddbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3de40d3f90 0x7f3de4161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41008/0 (socket says 172.21.15.5:41008) 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.426+0000 7f3ddbe00640 1 -- 172.21.15.5:0/3069392338 learned_addr learned my addr 172.21.15.5:0/3069392338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.427+0000 7f3de3400640 1 --2- 172.21.15.5:0/3069392338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.427+0000 7f3ddbe00640 1 -- 172.21.15.5:0/3069392338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 msgr2=0x7f3de415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.427+0000 7f3ddbe00640 1 --2- 172.21.15.5:0/3069392338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.427+0000 7f3ddbe00640 1 -- 172.21.15.5:0/3069392338 >> v1:172.21.15.80:6789/0 conn(0x7f3de4161bc0 legacy=0x7f3de40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.427+0000 7f3ddbe00640 1 -- 172.21.15.5:0/3069392338 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4160f80 con 0x7f3de40d3f90 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.428+0000 7f3ddbe00640 1 --2- 172.21.15.5:0/3069392338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de40d3f90 0x7f3de4161680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009850 tx=0x7f3dcc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840583dd7c14c97e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.428+0000 7f3de2a00640 1 -- 172.21.15.5:0/3069392338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc02ae60 con 0x7f3de40d3f90 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.428+0000 7f3de2a00640 1 -- 172.21.15.5:0/3069392338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc004030 con 0x7f3de40d3f90 2024-04-25T10:05:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.428+0000 7f3de2a00640 1 -- 172.21.15.5:0/3069392338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc004460 con 0x7f3de40d3f90 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 -- 172.21.15.5:0/3069392338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de40d3f90 msgr2=0x7f3de4161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 --2- 172.21.15.5:0/3069392338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de40d3f90 0x7f3de4161680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009850 tx=0x7f3dcc02a150 comp rx=0 tx=0).stop 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 -- 172.21.15.5:0/3069392338 shutdown_connections 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 --2- 172.21.15.5:0/3069392338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de40d3f90 0x7f3de4161680 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 --2- 172.21.15.5:0/3069392338 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 -- 172.21.15.5:0/3069392338 >> 172.21.15.5:0/3069392338 conn(0x7f3de408af80 msgr2=0x7f3de40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 -- 172.21.15.5:0/3069392338 shutdown_connections 2024-04-25T10:05:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.429+0000 7f3de9200640 1 -- 172.21.15.5:0/3069392338 wait complete. 2024-04-25T10:05:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.431+0000 7f3de9200640 1 Processor -- start 2024-04-25T10:05:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.431+0000 7f3de9200640 1 -- start start 2024-04-25T10:05:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.432+0000 7f3de9200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.432+0000 7f3de9200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40d3f90 0x7f3de40b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.432+0000 7f3de9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 0x7f3de40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.432+0000 7f3de9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de41603a0 con 0x7f3de4161bc0 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.432+0000 7f3de9200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40b4ca0 con 0x7f3de40d3f90 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.432+0000 7f3de9200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40b3b80 con 0x7f3de40b48c0 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 0x7f3de40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 0x7f3de40ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41012/0 (socket says 172.21.15.5:41012) 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 -- 172.21.15.5:0/2410908562 learned_addr learned my addr 172.21.15.5:0/2410908562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3ddbe00640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40d3f90 0x7f3de40b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3400640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 -- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 msgr2=0x7f3de415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 -- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40d3f90 msgr2=0x7f3de40b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40d3f90 0x7f3de40b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3e00640 1 -- 172.21.15.5:0/2410908562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40cbc50 con 0x7f3de4161bc0 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3ddbe00640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40d3f90 0x7f3de40b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.433+0000 7f3de3400640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.434+0000 7f3de3e00640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 0x7f3de40ccc80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b970 tx=0x7f3ddc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.434+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc019070 con 0x7f3de4161bc0 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.434+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc00cb40 con 0x7f3de4161bc0 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.434+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc014440 con 0x7f3de4161bc0 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.434+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de415c310 con 0x7f3de4161bc0 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.434+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de40cc1a0 con 0x7f3de4161bc0 2024-04-25T10:05:23.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.435+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de4068f60 con 0x7f3de4161bc0 2024-04-25T10:05:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.436+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ddc01c070 con 0x7f3de4161bc0 2024-04-25T10:05:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.437+0000 7f3ddb400640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.437+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3ddc089c70 con 0x7f3de4161bc0 2024-04-25T10:05:23.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.438+0000 7f3de3400640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.439+0000 7f3de3400640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3dc006b500 0x7f3dc006d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4006fd0 tx=0x7f3dd4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.443+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ddc058c50 con 0x7f3de4161bc0 2024-04-25T10:05:23.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.642+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3de40d0dd0 con 0x7f3de4161bc0 2024-04-25T10:05:23.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.649+0000 7f3ddb400640 1 -- 172.21.15.5:0/2410908562 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ddc058550 con 0x7f3de4161bc0 2024-04-25T10:05:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3dc006b500 msgr2=0x7f3dc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3dc006b500 0x7f3dc006d9c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4006fd0 tx=0x7f3dd4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 msgr2=0x7f3de40ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 0x7f3de40ccc80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b970 tx=0x7f3ddc00be30 comp rx=0 tx=0).stop 2024-04-25T10:05:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 shutdown_connections 2024-04-25T10:05:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de4161bc0 0x7f3de40ccc80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3dc006b500 0x7f3dc006d9c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40d3f90 0x7f3de40b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 --2- 172.21.15.5:0/2410908562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3de40b48c0 0x7f3de415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.652+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 >> 172.21.15.5:0/2410908562 conn(0x7f3de408af80 msgr2=0x7f3de40b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.653+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 shutdown_connections 2024-04-25T10:05:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.653+0000 7f3de9200640 1 -- 172.21.15.5:0/2410908562 wait complete. 2024-04-25T10:05:23.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:05:23.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.877+0000 7f2588a00640 1 Processor -- start 2024-04-25T10:05:23.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.878+0000 7f2588a00640 1 -- start start 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.879+0000 7f2588a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f258415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.879+0000 7f2588a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25840cd300 0x7f25840cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.879+0000 7f2588a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2584071d30 con 0x7f2584141da0 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.880+0000 7f2588a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840cdcd0 con 0x7f25840cd300 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.880+0000 7f2588a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584142180 con 0x7f258415ce00 2024-04-25T10:05:23.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.880+0000 7f2582a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f258415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.880+0000 7f2583e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25840cd300 0x7f25840cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.880+0000 7f2582a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f258415d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:48298/0 (socket says 172.21.15.5:48298) 2024-04-25T10:05:23.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.880+0000 7f2582a00640 1 -- 172.21.15.5:0/781768215 learned_addr learned my addr 172.21.15.5:0/781768215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.881+0000 7f2582a00640 1 -- 172.21.15.5:0/781768215 >> v2:172.21.15.80:3300/0 conn(0x7f25840cd300 msgr2=0x7f25840cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.881+0000 7f2582a00640 1 --2- 172.21.15.5:0/781768215 >> v2:172.21.15.80:3300/0 conn(0x7f25840cd300 0x7f25840cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.881+0000 7f2582a00640 1 -- 172.21.15.5:0/781768215 >> v1:172.21.15.80:6789/0 conn(0x7f2584141da0 legacy=0x7f258406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.881+0000 7f2582a00640 1 -- 172.21.15.5:0/781768215 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258406d620 con 0x7f258415ce00 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.881+0000 7f2583e00640 1 --2- 172.21.15.5:0/781768215 >> v2:172.21.15.80:3300/0 conn(0x7f25840cd300 0x7f25840cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.883+0000 7f2582a00640 1 --2- 172.21.15.5:0/781768215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f258415d280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5e8b97bac32d9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.883+0000 7f2582000640 1 -- 172.21.15.5:0/781768215 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c037070 con 0x7f258415ce00 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.883+0000 7f2582000640 1 -- 172.21.15.5:0/781768215 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c032440 con 0x7f258415ce00 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.883+0000 7f2582000640 1 -- 172.21.15.5:0/781768215 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c032810 con 0x7f258415ce00 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 -- 172.21.15.5:0/781768215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 msgr2=0x7f258415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 --2- 172.21.15.5:0/781768215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f258415d280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f256c009870 tx=0x7f256c02a250 comp rx=0 tx=0).stop 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 -- 172.21.15.5:0/781768215 shutdown_connections 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 --2- 172.21.15.5:0/781768215 >> v2:172.21.15.80:3300/0 conn(0x7f25840cd300 0x7f25840cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 --2- 172.21.15.5:0/781768215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f258415d280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 -- 172.21.15.5:0/781768215 >> 172.21.15.5:0/781768215 conn(0x7f2584066f20 msgr2=0x7f258406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 -- 172.21.15.5:0/781768215 shutdown_connections 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.884+0000 7f2588a00640 1 -- 172.21.15.5:0/781768215 wait complete. 2024-04-25T10:05:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.887+0000 7f2588a00640 1 Processor -- start 2024-04-25T10:05:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.887+0000 7f2588a00640 1 -- start start 2024-04-25T10:05:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.888+0000 7f2588a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25840cd300 0x7f2584144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.888+0000 7f2588a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 0x7f258415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.888+0000 7f2588a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f2584142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.888+0000 7f2588a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258406e810 con 0x7f2584141da0 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.888+0000 7f2588a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258406dbe0 con 0x7f25840cd300 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.888+0000 7f2588a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584071000 con 0x7f258415ce00 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 0x7f258415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 0x7f258415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41022/0 (socket says 172.21.15.5:41022) 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 -- 172.21.15.5:0/427100991 learned_addr learned my addr 172.21.15.5:0/427100991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2583e00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f2584142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2583400640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25840cd300 0x7f2584144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 -- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 msgr2=0x7f2584142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f2584142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 -- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25840cd300 msgr2=0x7f2584144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25840cd300 0x7f2584144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 -- 172.21.15.5:0/427100991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258413dcc0 con 0x7f2584141da0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2583400640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25840cd300 0x7f2584144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.889+0000 7f2582a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 0x7f258415a450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f256c009990 tx=0x7f256c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.890+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c037040 con 0x7f2584141da0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.890+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c042070 con 0x7f2584141da0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.890+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c030940 con 0x7f2584141da0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.890+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258413dfa0 con 0x7f2584141da0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.890+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258413e560 con 0x7f2584141da0 2024-04-25T10:05:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.891+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584068f60 con 0x7f2584141da0 2024-04-25T10:05:23.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.892+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f256c030ae0 con 0x7f2584141da0 2024-04-25T10:05:23.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.893+0000 7f257b400640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f256006b4d0 0x7f256006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.893+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f256c0aa980 con 0x7f2584141da0 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.894+0000 7f2583400640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f256006b4d0 0x7f256006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.895+0000 7f2583400640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f256006b4d0 0x7f256006d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2574001680 tx=0x7f2574009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:23.898+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f256c0798d0 con 0x7f2584141da0 2024-04-25T10:05:24.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:24.090 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.173+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f25840b3340 con 0x7f2584141da0 2024-04-25T10:05:24.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.221+0000 7f257b400640 1 -- 172.21.15.5:0/427100991 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v202) v1 ==== 149+0+0 (secure 0 0 0) 0x7f256c0791d0 con 0x7f2584141da0 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.223+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f256006b4d0 msgr2=0x7f256006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.223+0000 7f2588a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f256006b4d0 0x7f256006d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2574001680 tx=0x7f2574009040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.223+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 msgr2=0x7f258415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.223+0000 7f2588a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 0x7f258415a450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f256c009990 tx=0x7f256c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 shutdown_connections 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f258415ce00 0x7f2584142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2584141da0 0x7f258415a450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f256006b4d0 0x7f256006d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 --2- 172.21.15.5:0/427100991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25840cd300 0x7f2584144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 >> 172.21.15.5:0/427100991 conn(0x7f2584066f20 msgr2=0x7f2584071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 shutdown_connections 2024-04-25T10:05:24.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.224+0000 7f2588a00640 1 -- 172.21.15.5:0/427100991 wait complete. 2024-04-25T10:05:24.246 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T10:05:24.246 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:05:24.246 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:05:24.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:24.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.491+0000 7f06b3e00640 1 Processor -- start 2024-04-25T10:05:24.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.491+0000 7f06b3e00640 1 -- start start 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06b4141da0 0x7f06b406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06b406e640 0x7f06b415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b40cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b4142180 con 0x7f06b406e640 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b4071d30 con 0x7f06b4141da0 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b406d620 con 0x7f06b415d340 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06abe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06b406e640 0x7f06b415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06abe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06b406e640 0x7f06b415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41034/0 (socket says 172.21.15.5:41034) 2024-04-25T10:05:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06abe00640 1 -- 172.21.15.5:0/2048908434 learned_addr learned my addr 172.21.15.5:0/2048908434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:24.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b2a00640 1 --2- 172.21.15.5:0/2048908434 >> v2:172.21.15.80:3300/0 conn(0x7f06b4141da0 0x7f06b406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.493+0000 7f06b3400640 1 --2- 172.21.15.5:0/2048908434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b40cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.494+0000 7f06abe00640 1 -- 172.21.15.5:0/2048908434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 msgr2=0x7f06b40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.494+0000 7f06abe00640 1 --2- 172.21.15.5:0/2048908434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b40cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.494+0000 7f06abe00640 1 -- 172.21.15.5:0/2048908434 >> v2:172.21.15.80:3300/0 conn(0x7f06b4141da0 msgr2=0x7f06b406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.494+0000 7f06abe00640 1 --2- 172.21.15.5:0/2048908434 >> v2:172.21.15.80:3300/0 conn(0x7f06b4141da0 0x7f06b406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.494+0000 7f06abe00640 1 -- 172.21.15.5:0/2048908434 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b406d900 con 0x7f06b406e640 2024-04-25T10:05:24.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.495+0000 7f06abe00640 1 --2- 172.21.15.5:0/2048908434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b406e640 0x7f06b415ce00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f06b406bbb0 tx=0x7f069c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b80a18b6612f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.495+0000 7f06b2000640 1 -- 172.21.15.5:0/2048908434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c037070 con 0x7f06b406e640 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.495+0000 7f06b2000640 1 -- 172.21.15.5:0/2048908434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c032440 con 0x7f06b406e640 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.495+0000 7f06b2000640 1 -- 172.21.15.5:0/2048908434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c032810 con 0x7f06b406e640 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 -- 172.21.15.5:0/2048908434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b406e640 msgr2=0x7f06b415ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 --2- 172.21.15.5:0/2048908434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b406e640 0x7f06b415ce00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f06b406bbb0 tx=0x7f069c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 -- 172.21.15.5:0/2048908434 shutdown_connections 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 --2- 172.21.15.5:0/2048908434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b40cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 --2- 172.21.15.5:0/2048908434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b406e640 0x7f06b415ce00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 --2- 172.21.15.5:0/2048908434 >> v2:172.21.15.80:3300/0 conn(0x7f06b4141da0 0x7f06b406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 -- 172.21.15.5:0/2048908434 >> 172.21.15.5:0/2048908434 conn(0x7f06b4066f20 msgr2=0x7f06b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 -- 172.21.15.5:0/2048908434 shutdown_connections 2024-04-25T10:05:24.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.496+0000 7f06b3e00640 1 -- 172.21.15.5:0/2048908434 wait complete. 2024-04-25T10:05:24.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.498+0000 7f06b3e00640 1 Processor -- start 2024-04-25T10:05:24.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.498+0000 7f06b3e00640 1 -- start start 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b406e640 0x7f06b41e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 0x7f06b41e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06b3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b414a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06b3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40cd9c0 con 0x7f06b4141da0 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06b3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40cd840 con 0x7f06b406e640 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06b3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40cdb40 con 0x7f06b415d340 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06abe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 0x7f06b41e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06abe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 0x7f06b41e7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41050/0 (socket says 172.21.15.5:41050) 2024-04-25T10:05:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.499+0000 7f06abe00640 1 -- 172.21.15.5:0/185996007 learned_addr learned my addr 172.21.15.5:0/185996007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06b3400640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b414a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06b2a00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b406e640 0x7f06b41e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06abe00640 1 -- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 msgr2=0x7f06b414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06abe00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b414a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06abe00640 1 -- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b406e640 msgr2=0x7f06b41e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06abe00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b406e640 0x7f06b41e5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06abe00640 1 -- 172.21.15.5:0/185996007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b41e59e0 con 0x7f06b4141da0 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06b2a00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b406e640 0x7f06b41e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06abe00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 0x7f06b41e7840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f069c02a220 tx=0x7f069c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c041020 con 0x7f06b4141da0 2024-04-25T10:05:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c0306c0 con 0x7f06b4141da0 2024-04-25T10:05:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c037040 con 0x7f06b4141da0 2024-04-25T10:05:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.500+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b41e5cc0 con 0x7f06b4141da0 2024-04-25T10:05:24.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.501+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b41e6280 con 0x7f06b4141da0 2024-04-25T10:05:24.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.502+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b4068f60 con 0x7f06b4141da0 2024-04-25T10:05:24.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.503+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f069c044020 con 0x7f06b4141da0 2024-04-25T10:05:24.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.503+0000 7f06ab400640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f069006b4d0 0x7f069006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.504+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f069c0aa880 con 0x7f06b4141da0 2024-04-25T10:05:24.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.504+0000 7f06b2a00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f069006b4d0 0x7f069006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.505+0000 7f06b2a00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f069006b4d0 0x7f069006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f06a4006fd0 tx=0x7f06a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.508+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f069c04b030 con 0x7f06b4141da0 2024-04-25T10:05:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.708+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06b4068670 con 0x7f06b4141da0 2024-04-25T10:05:24.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.711+0000 7f06ab400640 1 -- 172.21.15.5:0/185996007 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+13185 (secure 0 0 0) 0x7f069c079860 con 0x7f06b4141da0 2024-04-25T10:05:24.718 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:24.719 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:23.195574+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6836/2115515655":"2024-04-26T10:05:20.177133+0000","172.21.15.80:6834/2115515655":"2024-04-26T10:05:20.177133+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f069006b4d0 msgr2=0x7f069006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f069006b4d0 0x7f069006d990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f06a4006fd0 tx=0x7f06a4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 msgr2=0x7f06b41e7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 0x7f06b41e7840 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f069c02a220 tx=0x7f069c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 shutdown_connections 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06b415d340 0x7f06b414a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b4141da0 0x7f06b41e7840 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f069006b4d0 0x7f069006d990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 --2- 172.21.15.5:0/185996007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06b406e640 0x7f06b41e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 >> 172.21.15.5:0/185996007 conn(0x7f06b4066f20 msgr2=0x7f06b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 shutdown_connections 2024-04-25T10:05:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.714+0000 7f06b3e00640 1 -- 172.21.15.5:0/185996007 wait complete. 2024-04-25T10:05:24.739 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:24.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:24.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.951+0000 7f7fb7e00640 1 Processor -- start 2024-04-25T10:05:24.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.951+0000 7f7fb7e00640 1 -- start start 2024-04-25T10:05:24.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.953+0000 7f7fb7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7fb0144fe0 0x7f7fb006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.953+0000 7f7fb7e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7fb0071840 0x7f7fb015c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.953+0000 7f7fb7e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb01453c0 con 0x7f7fb006d130 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.953+0000 7f7fb7e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00a8560 con 0x7f7fb0144fe0 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.953+0000 7f7fb7e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb006d6a0 con 0x7f7fb0071840 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb4a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7fb006d130 0x7f7fb0071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43110/0 (socket says 172.21.15.5:43110) 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb4a00640 1 -- 172.21.15.5:0/2054491176 learned_addr learned my addr 172.21.15.5:0/2054491176 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb5400640 1 --2- 172.21.15.5:0/2054491176 >> v2:172.21.15.5:3300/0 conn(0x7f7fb0144fe0 0x7f7fb006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb5e00640 1 --2- 172.21.15.5:0/2054491176 >> v2:172.21.15.80:3300/0 conn(0x7f7fb0071840 0x7f7fb015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb5400640 1 -- 172.21.15.5:0/2054491176 >> v2:172.21.15.80:3300/0 conn(0x7f7fb0071840 msgr2=0x7f7fb015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb5400640 1 --2- 172.21.15.5:0/2054491176 >> v2:172.21.15.80:3300/0 conn(0x7f7fb0071840 0x7f7fb015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fb5400640 1 -- 172.21.15.5:0/2054491176 >> v1:172.21.15.5:6789/0 conn(0x7f7fb006d130 legacy=0x7f7fb0071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.955+0000 7f7fb5400640 1 -- 172.21.15.5:0/2054491176 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb006d980 con 0x7f7fb0144fe0 2024-04-25T10:05:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.954+0000 7f7fafe00640 1 -- 172.21.15.5:0/2054491176 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535049401 0 0) 0x7f7fb01453c0 con 0x7f7fb006d130 2024-04-25T10:05:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb5400640 1 --2- 172.21.15.5:0/2054491176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb0144fe0 0x7f7fb006e560 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7fb015bd10 tx=0x7f7fa402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39724aa017700ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fafe00640 1 -- 172.21.15.5:0/2054491176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4037070 con 0x7f7fb0144fe0 2024-04-25T10:05:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fafe00640 1 -- 172.21.15.5:0/2054491176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4032440 con 0x7f7fb0144fe0 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fafe00640 1 -- 172.21.15.5:0/2054491176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4032810 con 0x7f7fb0144fe0 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb7e00640 1 -- 172.21.15.5:0/2054491176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb0144fe0 msgr2=0x7f7fb006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/2054491176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb0144fe0 0x7f7fb006e560 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7fb015bd10 tx=0x7f7fa402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb7e00640 1 -- 172.21.15.5:0/2054491176 shutdown_connections 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/2054491176 >> v2:172.21.15.80:3300/0 conn(0x7f7fb0071840 0x7f7fb015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/2054491176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb0144fe0 0x7f7fb006e560 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.956+0000 7f7fb7e00640 1 -- 172.21.15.5:0/2054491176 >> 172.21.15.5:0/2054491176 conn(0x7f7fb006ab60 msgr2=0x7f7fb006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.957+0000 7f7fb7e00640 1 -- 172.21.15.5:0/2054491176 shutdown_connections 2024-04-25T10:05:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.957+0000 7f7fb7e00640 1 -- 172.21.15.5:0/2054491176 wait complete. 2024-04-25T10:05:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.958+0000 7f7fb7e00640 1 Processor -- start 2024-04-25T10:05:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.958+0000 7f7fb7e00640 1 -- start start 2024-04-25T10:05:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.959+0000 7f7fb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 0x7f7fb00c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.959+0000 7f7fb7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fb0071840 0x7f7fb01f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.959+0000 7f7fb7e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fb0144fe0 0x7f7fb01f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.959+0000 7f7fb7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb015be50 con 0x7f7fb006d130 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.959+0000 7f7fb7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb006df40 con 0x7f7fb0071840 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.959+0000 7f7fb7e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb006eaa0 con 0x7f7fb0144fe0 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 0x7f7fb00c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 0x7f7fb00c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41076/0 (socket says 172.21.15.5:41076) 2024-04-25T10:05:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 -- 172.21.15.5:0/976462498 learned_addr learned my addr 172.21.15.5:0/976462498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 -- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fb0144fe0 msgr2=0x7f7fb01f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fb0144fe0 0x7f7fb01f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 -- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fb0071840 msgr2=0x7f7fb01f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fb0071840 0x7f7fb01f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 -- 172.21.15.5:0/976462498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb01f42d0 con 0x7f7fb006d130 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.960+0000 7f7fb5400640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 0x7f7fb00c4670 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.961+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4037040 con 0x7f7fb006d130 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.961+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa4042070 con 0x7f7fb006d130 2024-04-25T10:05:24.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.961+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4030bb0 con 0x7f7fb006d130 2024-04-25T10:05:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.961+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb01e53b0 con 0x7f7fb006d130 2024-04-25T10:05:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.961+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb01e31a0 con 0x7f7fb006d130 2024-04-25T10:05:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.962+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb0068f60 con 0x7f7fb006d130 2024-04-25T10:05:24.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.963+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fa4047460 con 0x7f7fb006d130 2024-04-25T10:05:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.964+0000 7f7fad600640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7f9006b4d0 0x7f7f9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.964+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7fa40aad90 con 0x7f7fb006d130 2024-04-25T10:05:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.964+0000 7f7fb4a00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7f9006b4d0 0x7f7f9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.966+0000 7f7fb4a00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7f9006b4d0 0x7f7f9006d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0011e0 tx=0x7f7f9c007950 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:24.968+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fb01f42d0 con 0x7f7fb006d130 2024-04-25T10:05:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.166+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fb0073d00 con 0x7f7fb006d130 2024-04-25T10:05:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.167+0000 7f7fad600640 1 -- 172.21.15.5:0/976462498 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fa4032e20 con 0x7f7fb006d130 2024-04-25T10:05:25.172 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7f9006b4d0 msgr2=0x7f7f9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7f9006b4d0 0x7f7f9006d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0011e0 tx=0x7f7f9c007950 comp rx=0 tx=0).stop 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 msgr2=0x7f7fb00c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 0x7f7fb00c4670 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 shutdown_connections 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7fb0144fe0 0x7f7fb01f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7f9006b4d0 0x7f7f9006d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7fb0071840 0x7f7fb01f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 --2- 172.21.15.5:0/976462498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fb006d130 0x7f7fb00c4670 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 >> 172.21.15.5:0/976462498 conn(0x7f7fb006ab60 msgr2=0x7f7fb0144230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 shutdown_connections 2024-04-25T10:05:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.170+0000 7f7fb7e00640 1 -- 172.21.15.5:0/976462498 wait complete. 2024-04-25T10:05:25.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:05:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.402+0000 7f325e000640 1 Processor -- start 2024-04-25T10:05:25.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.402+0000 7f325e000640 1 -- start start 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325e000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f32580b0d70 0x7f32580b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325e000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32580b6010 0x7f325806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f325806b0a0 0x7f3258161350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325e000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580b4f70 con 0x7f32580b0d70 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325e000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32580b17d0 con 0x7f32580b6010 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580b1ab0 con 0x7f325806b0a0 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325cc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f32580b0d70 0x7f32580b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325cc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f32580b0d70 0x7f32580b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41084/0 (socket says 172.21.15.5:41084) 2024-04-25T10:05:25.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325cc00640 1 -- 172.21.15.5:0/1099564506 learned_addr learned my addr 172.21.15.5:0/1099564506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f325d600640 1 --2- 172.21.15.5:0/1099564506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f325806b0a0 0x7f3258161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.404+0000 7f3257e00640 1 --2- 172.21.15.5:0/1099564506 >> v2:172.21.15.80:3300/0 conn(0x7f32580b6010 0x7f325806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.405+0000 7f325cc00640 1 -- 172.21.15.5:0/1099564506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f325806b0a0 msgr2=0x7f3258161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.405+0000 7f325cc00640 1 --2- 172.21.15.5:0/1099564506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f325806b0a0 0x7f3258161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.405+0000 7f325cc00640 1 -- 172.21.15.5:0/1099564506 >> v2:172.21.15.80:3300/0 conn(0x7f32580b6010 msgr2=0x7f325806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.405+0000 7f325cc00640 1 --2- 172.21.15.5:0/1099564506 >> v2:172.21.15.80:3300/0 conn(0x7f32580b6010 0x7f325806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.405+0000 7f325cc00640 1 -- 172.21.15.5:0/1099564506 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580b51e0 con 0x7f32580b0d70 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.406+0000 7f325cc00640 1 --2- 172.21.15.5:0/1099564506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32580b0d70 0x7f32580b3370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f32580d1460 tx=0x7f324802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1d4e8b93ac7424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.406+0000 7f3257400640 1 -- 172.21.15.5:0/1099564506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248037070 con 0x7f32580b0d70 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.406+0000 7f3257400640 1 -- 172.21.15.5:0/1099564506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3248032440 con 0x7f32580b0d70 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.406+0000 7f3257400640 1 -- 172.21.15.5:0/1099564506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248032810 con 0x7f32580b0d70 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 -- 172.21.15.5:0/1099564506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32580b0d70 msgr2=0x7f32580b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 --2- 172.21.15.5:0/1099564506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32580b0d70 0x7f32580b3370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f32580d1460 tx=0x7f324802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 -- 172.21.15.5:0/1099564506 shutdown_connections 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 --2- 172.21.15.5:0/1099564506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f325806b0a0 0x7f3258161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 --2- 172.21.15.5:0/1099564506 >> v2:172.21.15.80:3300/0 conn(0x7f32580b6010 0x7f325806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 --2- 172.21.15.5:0/1099564506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32580b0d70 0x7f32580b3370 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 -- 172.21.15.5:0/1099564506 >> 172.21.15.5:0/1099564506 conn(0x7f32580999f0 msgr2=0x7f32580b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 -- 172.21.15.5:0/1099564506 shutdown_connections 2024-04-25T10:05:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.407+0000 7f325e000640 1 -- 172.21.15.5:0/1099564506 wait complete. 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.409+0000 7f325e000640 1 Processor -- start 2024-04-25T10:05:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.409+0000 7f325e000640 1 -- start start 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 0x7f325809aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32580b0d70 0x7f325815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32580b6010 0x7f325809d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3258161890 con 0x7f325806b0a0 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325e000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580b1d90 con 0x7f32580b6010 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3258161a10 con 0x7f32580b0d70 2024-04-25T10:05:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 0x7f325809aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 0x7f325809aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41096/0 (socket says 172.21.15.5:41096) 2024-04-25T10:05:25.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.410+0000 7f325cc00640 1 -- 172.21.15.5:0/2211703393 learned_addr learned my addr 172.21.15.5:0/2211703393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f3257e00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32580b0d70 0x7f325815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325d600640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32580b6010 0x7f325809d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325cc00640 1 -- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32580b0d70 msgr2=0x7f325815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325cc00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32580b0d70 0x7f325815cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325cc00640 1 -- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32580b6010 msgr2=0x7f325809d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325cc00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32580b6010 0x7f325809d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325cc00640 1 -- 172.21.15.5:0/2211703393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32581e8a90 con 0x7f325806b0a0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f3257e00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32580b0d70 0x7f325815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f325cc00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 0x7f325809aaf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f324802a220 tx=0x7f32480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.411+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248041020 con 0x7f325806b0a0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.412+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32480306c0 con 0x7f325806b0a0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.412+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3248037040 con 0x7f325806b0a0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.412+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32581e8d70 con 0x7f325806b0a0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.412+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32581e9330 con 0x7f325806b0a0 2024-04-25T10:05:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.413+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258068f60 con 0x7f325806b0a0 2024-04-25T10:05:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.414+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3248044020 con 0x7f325806b0a0 2024-04-25T10:05:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.415+0000 7f3254c00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f323406b4d0 0x7f323406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.415+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f32480aa8d0 con 0x7f325806b0a0 2024-04-25T10:05:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.416+0000 7f3257e00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f323406b4d0 0x7f323406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.417+0000 7f3257e00640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f323406b4d0 0x7f323406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f32580b1420 tx=0x7f324000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:25.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.420+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f324804b030 con 0x7f325806b0a0 2024-04-25T10:05:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:25.695+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f325806d080 con 0x7f325806b0a0 2024-04-25T10:05:26.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.234+0000 7f3254c00640 1 -- 172.21.15.5:0/2211703393 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v204) v1 ==== 172+0+0 (secure 0 0 0) 0x7f32480798b0 con 0x7f325806b0a0 2024-04-25T10:05:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f323406b4d0 msgr2=0x7f323406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f323406b4d0 0x7f323406d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f32580b1420 tx=0x7f324000b040 comp rx=0 tx=0).stop 2024-04-25T10:05:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 msgr2=0x7f325809aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 0x7f325809aaf0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f324802a220 tx=0x7f32480046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 shutdown_connections 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f323406b4d0 0x7f323406d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32580b6010 0x7f325809d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f32580b0d70 0x7f325815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 --2- 172.21.15.5:0/2211703393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325806b0a0 0x7f325809aaf0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 >> 172.21.15.5:0/2211703393 conn(0x7f32580999f0 msgr2=0x7f3258160130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.238+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 shutdown_connections 2024-04-25T10:05:26.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.239+0000 7f325e000640 1 -- 172.21.15.5:0/2211703393 wait complete. 2024-04-25T10:05:26.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:26.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.474+0000 7faa5be00640 1 Processor -- start 2024-04-25T10:05:26.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.474+0000 7faa5be00640 1 -- start start 2024-04-25T10:05:26.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faa5c071000 0x7faa5c071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faa5c0b3840 0x7faa5c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5be00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c06d020 con 0x7faa5c15f800 2024-04-25T10:05:26.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5be00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c0719a0 con 0x7faa5c0b3840 2024-04-25T10:05:26.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c06ded0 con 0x7faa5c071000 2024-04-25T10:05:26.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faa5c0b3840 0x7faa5c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faa5c0b3840 0x7faa5c0b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41100/0 (socket says 172.21.15.5:41100) 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5b400640 1 -- 172.21.15.5:0/1349830189 learned_addr learned my addr 172.21.15.5:0/1349830189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.476+0000 7faa5a000640 1 --2- 172.21.15.5:0/1349830189 >> v2:172.21.15.80:3300/0 conn(0x7faa5c071000 0x7faa5c071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.477+0000 7faa5b400640 1 -- 172.21.15.5:0/1349830189 >> v2:172.21.15.80:3300/0 conn(0x7faa5c071000 msgr2=0x7faa5c071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.477+0000 7faa5b400640 1 --2- 172.21.15.5:0/1349830189 >> v2:172.21.15.80:3300/0 conn(0x7faa5c071000 0x7faa5c071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.477+0000 7faa5b400640 1 -- 172.21.15.5:0/1349830189 >> v1:172.21.15.80:6789/0 conn(0x7faa5c15f800 legacy=0x7faa5c162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.477+0000 7faa5b400640 1 -- 172.21.15.5:0/1349830189 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c06d2c0 con 0x7faa5c0b3840 2024-04-25T10:05:26.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.478+0000 7faa5b400640 1 --2- 172.21.15.5:0/1349830189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0b3c20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faa5c06e050 tx=0x7faa5402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660747bfb7d36cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.478+0000 7faa59600640 1 -- 172.21.15.5:0/1349830189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54036070 con 0x7faa5c0b3840 2024-04-25T10:05:26.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.478+0000 7faa59600640 1 -- 172.21.15.5:0/1349830189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54031600 con 0x7faa5c0b3840 2024-04-25T10:05:26.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.478+0000 7faa59600640 1 -- 172.21.15.5:0/1349830189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa540319d0 con 0x7faa5c0b3840 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 -- 172.21.15.5:0/1349830189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 msgr2=0x7faa5c0b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 --2- 172.21.15.5:0/1349830189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0b3c20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7faa5c06e050 tx=0x7faa5402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 -- 172.21.15.5:0/1349830189 shutdown_connections 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 --2- 172.21.15.5:0/1349830189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0b3c20 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 --2- 172.21.15.5:0/1349830189 >> v2:172.21.15.80:3300/0 conn(0x7faa5c071000 0x7faa5c071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 -- 172.21.15.5:0/1349830189 >> 172.21.15.5:0/1349830189 conn(0x7faa5c066f20 msgr2=0x7faa5c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 -- 172.21.15.5:0/1349830189 shutdown_connections 2024-04-25T10:05:26.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.479+0000 7faa5be00640 1 -- 172.21.15.5:0/1349830189 wait complete. 2024-04-25T10:05:26.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.481+0000 7faa5be00640 1 Processor -- start 2024-04-25T10:05:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.481+0000 7faa5be00640 1 -- start start 2024-04-25T10:05:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faa5c071000 0x7faa5c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa5c15f800 0x7faa5c0a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c06db60 con 0x7faa5c0b3840 2024-04-25T10:05:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c06d880 con 0x7faa5c15f800 2024-04-25T10:05:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c06e960 con 0x7faa5c071000 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41106/0 (socket says 172.21.15.5:41106) 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 -- 172.21.15.5:0/621595772 learned_addr learned my addr 172.21.15.5:0/621595772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 -- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faa5c071000 msgr2=0x7faa5c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faa5c071000 0x7faa5c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 -- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa5c15f800 msgr2=0x7faa5c0a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa5c15f800 0x7faa5c0a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.482+0000 7faa5a000640 1 -- 172.21.15.5:0/621595772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c0a9f30 con 0x7faa5c0b3840 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.483+0000 7faa5a000640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0ce760 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7faa4800d800 tx=0x7faa4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.483+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800b6c0 con 0x7faa5c0b3840 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.483+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa48019070 con 0x7faa5c0b3840 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.483+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800ba80 con 0x7faa5c0b3840 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.483+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c0cd9a0 con 0x7faa5c0b3840 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.483+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c0cdf60 con 0x7faa5c0b3840 2024-04-25T10:05:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.484+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c068f60 con 0x7faa5c0b3840 2024-04-25T10:05:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.485+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa4800bc20 con 0x7faa5c0b3840 2024-04-25T10:05:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.486+0000 7faa53400640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faa3c06b4d0 0x7faa3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.486+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7faa4808a5e0 con 0x7faa5c0b3840 2024-04-25T10:05:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.487+0000 7faa5aa00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faa3c06b4d0 0x7faa3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.488+0000 7faa5aa00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faa3c06b4d0 0x7faa3c06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.490+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa5c0a9f30 con 0x7faa5c0b3840 2024-04-25T10:05:26.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.691+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa5c161910 con 0x7faa5c0b3840 2024-04-25T10:05:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.697+0000 7faa53400640 1 -- 172.21.15.5:0/621595772 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa48059530 con 0x7faa5c0b3840 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faa3c06b4d0 msgr2=0x7faa3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faa3c06b4d0 0x7faa3c06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa4c006fd0 tx=0x7faa4c006bc0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 msgr2=0x7faa5c0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0ce760 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7faa4800d800 tx=0x7faa4800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 shutdown_connections 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faa3c06b4d0 0x7faa3c06d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa5c15f800 0x7faa5c0a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa5c0b3840 0x7faa5c0ce760 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 --2- 172.21.15.5:0/621595772 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faa5c071000 0x7faa5c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 >> 172.21.15.5:0/621595772 conn(0x7faa5c066f20 msgr2=0x7faa5c0af650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 shutdown_connections 2024-04-25T10:05:26.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:26.700+0000 7faa5be00640 1 -- 172.21.15.5:0/621595772 wait complete. 2024-04-25T10:05:26.723 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:26.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:26.837 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:26.951 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:26.951 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:05:26.951 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:27.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:27.013 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:05:27.013 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:27.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:27.072 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:05:27.072 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:27.133 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:27.133 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:05:27.133 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:27.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:27.193 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:27.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.419+0000 7f0e1d000640 1 Processor -- start 2024-04-25T10:05:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.420+0000 7f0e1d000640 1 -- start start 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.421+0000 7f0e1d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e181529e0 0x7f0e180c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.421+0000 7f0e1d000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1815bdc0 con 0x7f0e180cea60 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.421+0000 7f0e1d000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1815c060 con 0x7f0e180c6410 2024-04-25T10:05:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.421+0000 7f0e1d000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180b2340 con 0x7f0e181529e0 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.422+0000 7f0e16800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0e180cea60 0x7f0e1806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58450/0 (socket says 172.21.15.5:58450) 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.422+0000 7f0e16800640 1 -- 172.21.15.5:0/71737653 learned_addr learned my addr 172.21.15.5:0/71737653 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.422+0000 7f0e15e00640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e181529e0 0x7f0e180c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.422+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696806441 0 0) 0x7f0e1815bdc0 con 0x7f0e180cea60 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.422+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dfc003520 con 0x7f0e180cea60 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897248417 0 0) 0x7f0dfc003520 con 0x7f0e180cea60 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e181529e0 msgr2=0x7f0e180c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e181529e0 0x7f0e180c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 >> v1:172.21.15.80:6789/0 conn(0x7f0e180c6410 legacy=0x7f0e180ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1815c340 con 0x7f0e180cea60 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0e10001690 con 0x7f0e180cea60 2024-04-25T10:05:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 >> v1:172.21.15.5:6789/0 conn(0x7f0e180cea60 legacy=0x7f0e1806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc0039e0 0x7f0dfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dfc0051c0 0x7f0dfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e180b2340 con 0x7f0dfc004800 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc00bf00 con 0x7f0dfc0039e0 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc00c080 con 0x7f0dfc0051c0 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.423+0000 7f0e15400640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.424+0000 7f0e15e00640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc0039e0 0x7f0dfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.424+0000 7f0e15400640 1 -- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dfc0051c0 msgr2=0x7f0dfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.424+0000 7f0e15400640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dfc0051c0 0x7f0dfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.424+0000 7f0e15400640 1 -- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc0039e0 msgr2=0x7f0dfc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.424+0000 7f0e15400640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc0039e0 0x7f0dfc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.424+0000 7f0e15400640 1 -- 172.21.15.5:0/71737653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc003520 con 0x7f0dfc004800 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.425+0000 7f0e15400640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001750 tx=0x7f0dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5f8c7204b846d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.425+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e00001370 con 0x7f0dfc004800 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.425+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000052d0 con 0x7f0dfc004800 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.425+0000 7f0e14a00640 1 -- 172.21.15.5:0/71737653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0002abe0 con 0x7f0dfc004800 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e1d000640 1 -- 172.21.15.5:0/71737653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 msgr2=0x7f0dfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e1d000640 1 --2- 172.21.15.5:0/71737653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001750 tx=0x7f0dfc005830 comp rx=0 tx=0).stop 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e16800640 1 -- 172.21.15.5:0/71737653 reap_dead start 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e1d000640 1 -- 172.21.15.5:0/71737653 shutdown_connections 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e1d000640 1 -- 172.21.15.5:0/71737653 >> 172.21.15.5:0/71737653 conn(0x7f0e18071150 msgr2=0x7f0e18071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e1d000640 1 -- 172.21.15.5:0/71737653 shutdown_connections 2024-04-25T10:05:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.426+0000 7f0e1d000640 1 -- 172.21.15.5:0/71737653 wait complete. 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.428+0000 7f0e1d000640 1 Processor -- start 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.428+0000 7f0e1d000640 1 -- start start 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e1d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0e180c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e1d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e18157a30 0x7f0e18157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e1d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e18154690 0x7f0e18154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e1d000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e18071ed0 con 0x7f0dfc004800 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e1d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1815c900 con 0x7f0e18154690 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e1d000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e18072050 con 0x7f0e18157a30 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0e180c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0e180c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34128/0 (socket says 172.21.15.5:34128) 2024-04-25T10:05:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.429+0000 7f0e15e00640 1 -- 172.21.15.5:0/3405289097 learned_addr learned my addr 172.21.15.5:0/3405289097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15400640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e18157a30 0x7f0e18157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e16800640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e18154690 0x7f0e18154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15e00640 1 -- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e18157a30 msgr2=0x7f0e18157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15e00640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e18157a30 0x7f0e18157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15e00640 1 -- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e18154690 msgr2=0x7f0e18154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15e00640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e18154690 0x7f0e18154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15e00640 1 -- 172.21.15.5:0/3405289097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18158640 con 0x7f0dfc004800 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e16800640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e18154690 0x7f0e18154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.430+0000 7f0e15e00640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0e180c7af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007f60 tx=0x7f0e08007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.431+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0801a020 con 0x7f0dfc004800 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.431+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08016470 con 0x7f0dfc004800 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.431+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08016860 con 0x7f0dfc004800 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.431+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e18153bb0 con 0x7f0dfc004800 2024-04-25T10:05:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.431+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e180ca910 con 0x7f0dfc004800 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.432+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e180c6b80 con 0x7f0dfc004800 2024-04-25T10:05:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.433+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e08005d10 con 0x7f0dfc004800 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.434+0000 7f0e0e000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0df406b4d0 0x7f0df406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.434+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0e0808b210 con 0x7f0dfc004800 2024-04-25T10:05:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.435+0000 7f0e15400640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0df406b4d0 0x7f0df406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.436+0000 7f0e15400640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0df406b4d0 0x7f0df406d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0e00032000 tx=0x7f0e0002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.438+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e180c6b80 con 0x7f0dfc004800 2024-04-25T10:05:27.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.682+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e18068fd0 con 0x7f0dfc004800 2024-04-25T10:05:27.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.683+0000 7f0e0e000640 1 -- 172.21.15.5:0/3405289097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3690 (secure 0 0 0) 0x7f0e08024030 con 0x7f0dfc004800 2024-04-25T10:05:27.688 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:27.688 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/521639401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":521639401},{"type":"v1","addr":"172.21.15.5:6835","nonce":521639401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4955,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1953787855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1953787855},{"type":"v1","addr":"172.21.15.80:6837","nonce":1953787855}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1346836601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1346836601},{"type":"v1","addr":"172.21.15.80:6836","nonce":1346836601}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5347,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/1447984972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1447984972},{"type":"v1","addr":"172.21.15.5:6834","nonce":1447984972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-04-25T10:05:27.689 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0df406b4d0 msgr2=0x7f0df406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0df406b4d0 0x7f0df406d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0e00032000 tx=0x7f0e0002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 msgr2=0x7f0e180c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0e180c7af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0e08007f60 tx=0x7f0e08007f90 comp rx=0 tx=0).stop 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 shutdown_connections 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0df406b4d0 0x7f0df406d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e18154690 0x7f0e18154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0e18157a30 0x7f0e18157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 --2- 172.21.15.5:0/3405289097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dfc004800 0x7f0e180c7af0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 >> 172.21.15.5:0/3405289097 conn(0x7f0e18071150 msgr2=0x7f0e180cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 shutdown_connections 2024-04-25T10:05:27.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:27.686+0000 7f0e1d000640 1 -- 172.21.15.5:0/3405289097 wait complete. 2024-04-25T10:05:27.713 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:27.744 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:27.744 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:27.744 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:27.745 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:27.745 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:27.745 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:27.745 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:27.745 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:27.745 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:27.745 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:27.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:27.746 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:27.746 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:27.746 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:27.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:27.766 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:05:27.766 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:27.768 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:27.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:27.769 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:27.769 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:27.774 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:27.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:27.816 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:27.816 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:27.821 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:27.821 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:27.822 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:27.822 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:27.828 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:27.828 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:27.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:05:27.971 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:05:27.972 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:05:27.976 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:05:27.981 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:05:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.006+0000 7f7384000640 1 Processor -- start 2024-04-25T10:05:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.006+0000 7f7384000640 1 -- start start 2024-04-25T10:05:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.007+0000 7f7384000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f737c15da60 0x7f737c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.007+0000 7f7384000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15de40 0x7f737c15e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.007+0000 7f7384000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737c15e8a0 con 0x7f737c161d70 2024-04-25T10:05:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.007+0000 7f7384000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c15eb80 con 0x7f737c15da60 2024-04-25T10:05:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.007+0000 7f7384000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c15ee60 con 0x7f737c15de40 2024-04-25T10:05:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7381600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f737c15da60 0x7f737c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7382000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f737c161d70 0x7f737c0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58488/0 (socket says 172.21.15.5:58488) 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7382000640 1 -- 172.21.15.5:0/1577590914 learned_addr learned my addr 172.21.15.5:0/1577590914 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7380c00640 1 --2- 172.21.15.5:0/1577590914 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15de40 0x7f737c15e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f737be00640 1 -- 172.21.15.5:0/1577590914 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784949842 0 0) 0x7f737c15e8a0 con 0x7f737c161d70 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7381600640 1 -- 172.21.15.5:0/1577590914 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15de40 msgr2=0x7f737c15e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7381600640 1 --2- 172.21.15.5:0/1577590914 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15de40 0x7f737c15e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7381600640 1 -- 172.21.15.5:0/1577590914 >> v1:172.21.15.5:6789/0 conn(0x7f737c161d70 legacy=0x7f737c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.008+0000 7f7381600640 1 -- 172.21.15.5:0/1577590914 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c161850 con 0x7f737c15da60 2024-04-25T10:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.009+0000 7f7381600640 1 --2- 172.21.15.5:0/1577590914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c15da60 0x7f737c0d0aa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f73700097a0 tx=0x7f737002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e091fdd3a24636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f737be00640 1 -- 172.21.15.5:0/1577590914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370037070 con 0x7f737c15da60 2024-04-25T10:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f737be00640 1 -- 172.21.15.5:0/1577590914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737002ae50 con 0x7f737c15da60 2024-04-25T10:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f737be00640 1 -- 172.21.15.5:0/1577590914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370032660 con 0x7f737c15da60 2024-04-25T10:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f7384000640 1 -- 172.21.15.5:0/1577590914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c15da60 msgr2=0x7f737c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f7384000640 1 --2- 172.21.15.5:0/1577590914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c15da60 0x7f737c0d0aa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f73700097a0 tx=0x7f737002a100 comp rx=0 tx=0).stop 2024-04-25T10:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f7384000640 1 -- 172.21.15.5:0/1577590914 shutdown_connections 2024-04-25T10:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f7384000640 1 --2- 172.21.15.5:0/1577590914 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15de40 0x7f737c15e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f7384000640 1 --2- 172.21.15.5:0/1577590914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c15da60 0x7f737c0d0aa0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.010+0000 7f7384000640 1 -- 172.21.15.5:0/1577590914 >> 172.21.15.5:0/1577590914 conn(0x7f737c06bfe0 msgr2=0x7f737c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.011+0000 7f7384000640 1 -- 172.21.15.5:0/1577590914 shutdown_connections 2024-04-25T10:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.011+0000 7f7384000640 1 -- 172.21.15.5:0/1577590914 wait complete. 2024-04-25T10:05:28.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.012+0000 7f7384000640 1 Processor -- start 2024-04-25T10:05:28.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.013+0000 7f7384000640 1 -- start start 2024-04-25T10:05:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.013+0000 7f7384000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15da60 0x7f737c0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7384000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f737c15de40 0x7f737c0aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7384000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 0x7f737c1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7384000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1621e0 con 0x7f737c161d70 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7384000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0cee80 con 0x7f737c15de40 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7384000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c162360 con 0x7f737c15da60 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 0x7f737c1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 0x7f737c1596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34174/0 (socket says 172.21.15.5:34174) 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 -- 172.21.15.5:0/901980343 learned_addr learned my addr 172.21.15.5:0/901980343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7381600640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15da60 0x7f737c0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7380c00640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f737c15de40 0x7f737c0aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 -- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15da60 msgr2=0x7f737c0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15da60 0x7f737c0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 -- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f737c15de40 msgr2=0x7f737c0aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f737c15de40 0x7f737c0aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.014+0000 7f7382000640 1 -- 172.21.15.5:0/901980343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c1f3890 con 0x7f737c161d70 2024-04-25T10:05:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.015+0000 7f7382000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 0x7f737c1596b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f737400c700 tx=0x7f737400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.015+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374018070 con 0x7f737c161d70 2024-04-25T10:05:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.015+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737400ce10 con 0x7f737c161d70 2024-04-25T10:05:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.015+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400a880 con 0x7f737c161d70 2024-04-25T10:05:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.015+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c0b6f70 con 0x7f737c161d70 2024-04-25T10:05:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.015+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c0b74f0 con 0x7f737c161d70 2024-04-25T10:05:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.016+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c068fd0 con 0x7f737c161d70 2024-04-25T10:05:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.017+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f737400aa20 con 0x7f737c161d70 2024-04-25T10:05:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.018+0000 7f7379600640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f735c06b4d0 0x7f735c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.018+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f73740895d0 con 0x7f737c161d70 2024-04-25T10:05:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.019+0000 7f7381600640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f735c06b4d0 0x7f735c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.020+0000 7f7381600640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f735c06b4d0 0x7f735c06d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7370034000 tx=0x7f737002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.022+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f737c1f3890 con 0x7f737c161d70 2024-04-25T10:05:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.231+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f737c0d1560 con 0x7f737c161d70 2024-04-25T10:05:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.268+0000 7f7379600640 1 -- 172.21.15.5:0/901980343 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7374058650 con 0x7f737c161d70 2024-04-25T10:05:28.273 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:05:28.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f735c06b4d0 msgr2=0x7f735c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f735c06b4d0 0x7f735c06d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7370034000 tx=0x7f737002e000 comp rx=0 tx=0).stop 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 msgr2=0x7f737c1596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 0x7f737c1596b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f737400c700 tx=0x7f737400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 shutdown_connections 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f737c161d70 0x7f737c1596b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f735c06b4d0 0x7f735c06d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f737c15de40 0x7f737c0aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 --2- 172.21.15.5:0/901980343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f737c15da60 0x7f737c0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.270+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 >> 172.21.15.5:0/901980343 conn(0x7f737c06bfe0 msgr2=0x7f737c0d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.271+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 shutdown_connections 2024-04-25T10:05:28.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.271+0000 7f7384000640 1 -- 172.21.15.5:0/901980343 wait complete. 2024-04-25T10:05:28.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:05:28.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.491+0000 7f4438000640 1 Processor -- start 2024-04-25T10:05:28.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.491+0000 7f4438000640 1 -- start start 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4438000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44301451d0 0x7f443015cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4438000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f443015ca50 0x7f443015e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4438000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443015d7f0 con 0x7f443015edd0 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4438000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44300a85f0 con 0x7f443015ca50 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4438000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443015e400 con 0x7f44301451d0 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4436000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f443015edd0 0x7f44300cbf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58490/0 (socket says 172.21.15.5:58490) 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4434c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f443015ca50 0x7f443015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4436000640 1 -- 172.21.15.5:0/1734652544 learned_addr learned my addr 172.21.15.5:0/1734652544 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:28.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4435600640 1 --2- 172.21.15.5:0/1734652544 >> v2:172.21.15.80:3300/0 conn(0x7f44301451d0 0x7f443015cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4434c00640 1 -- 172.21.15.5:0/1734652544 >> v2:172.21.15.80:3300/0 conn(0x7f44301451d0 msgr2=0x7f443015cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:28.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.493+0000 7f4434c00640 1 --2- 172.21.15.5:0/1734652544 >> v2:172.21.15.80:3300/0 conn(0x7f44301451d0 0x7f443015cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.494+0000 7f4434c00640 1 -- 172.21.15.5:0/1734652544 >> v1:172.21.15.5:6789/0 conn(0x7f443015edd0 legacy=0x7f44300cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.494+0000 7f4434c00640 1 -- 172.21.15.5:0/1734652544 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443015dad0 con 0x7f443015ca50 2024-04-25T10:05:28.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.495+0000 7f4434c00640 1 --2- 172.21.15.5:0/1734652544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443015ca50 0x7f443015e890 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f443006b840 tx=0x7f441c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509992de628da917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.495+0000 7f442fe00640 1 -- 172.21.15.5:0/1734652544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c037070 con 0x7f443015ca50 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.495+0000 7f442fe00640 1 -- 172.21.15.5:0/1734652544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c032440 con 0x7f443015ca50 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.495+0000 7f442fe00640 1 -- 172.21.15.5:0/1734652544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c032810 con 0x7f443015ca50 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.495+0000 7f4438000640 1 -- 172.21.15.5:0/1734652544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443015ca50 msgr2=0x7f443015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.495+0000 7f4438000640 1 --2- 172.21.15.5:0/1734652544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443015ca50 0x7f443015e890 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f443006b840 tx=0x7f441c02a250 comp rx=0 tx=0).stop 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.496+0000 7f4438000640 1 -- 172.21.15.5:0/1734652544 shutdown_connections 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.496+0000 7f4438000640 1 --2- 172.21.15.5:0/1734652544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443015ca50 0x7f443015e890 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.496+0000 7f4438000640 1 --2- 172.21.15.5:0/1734652544 >> v2:172.21.15.80:3300/0 conn(0x7f44301451d0 0x7f443015cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.496+0000 7f4438000640 1 -- 172.21.15.5:0/1734652544 >> 172.21.15.5:0/1734652544 conn(0x7f4430066fd0 msgr2=0x7f443006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.496+0000 7f4438000640 1 -- 172.21.15.5:0/1734652544 shutdown_connections 2024-04-25T10:05:28.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.496+0000 7f4438000640 1 -- 172.21.15.5:0/1734652544 wait complete. 2024-04-25T10:05:28.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.497+0000 7f4438000640 1 Processor -- start 2024-04-25T10:05:28.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.498+0000 7f4438000640 1 -- start start 2024-04-25T10:05:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.498+0000 7f4438000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 0x7f44301e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.498+0000 7f4438000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f443015edd0 0x7f44301f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4438000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f443013e730 0x7f443013de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4438000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443015d430 con 0x7f44301451d0 2024-04-25T10:05:28.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4438000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443015e090 con 0x7f443015edd0 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4438000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443015d5b0 con 0x7f443013e730 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 0x7f44301e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 0x7f44301e5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34194/0 (socket says 172.21.15.5:34194) 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 -- 172.21.15.5:0/2397272076 learned_addr learned my addr 172.21.15.5:0/2397272076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 -- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f443013e730 msgr2=0x7f443013de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f443013e730 0x7f443013de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 -- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f443015edd0 msgr2=0x7f44301f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f443015edd0 0x7f44301f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.499+0000 7f4435600640 1 -- 172.21.15.5:0/2397272076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443013fbb0 con 0x7f44301451d0 2024-04-25T10:05:28.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.500+0000 7f4435600640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 0x7f44301e5030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4424007f20 tx=0x7f442400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.500+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424014620 con 0x7f44301451d0 2024-04-25T10:05:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.500+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424019070 con 0x7f44301451d0 2024-04-25T10:05:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.500+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424014a00 con 0x7f44301451d0 2024-04-25T10:05:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.500+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443013fe90 con 0x7f44301451d0 2024-04-25T10:05:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.500+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4430140450 con 0x7f44301451d0 2024-04-25T10:05:28.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.501+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4430074bb0 con 0x7f44301451d0 2024-04-25T10:05:28.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.502+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4424014ba0 con 0x7f44301451d0 2024-04-25T10:05:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.503+0000 7f442d600640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f441006b500 0x7f441006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.503+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f442408a9d0 con 0x7f44301451d0 2024-04-25T10:05:28.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.504+0000 7f4434c00640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f441006b500 0x7f441006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.505+0000 7f4434c00640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f441006b500 0x7f441006d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f441c034000 tx=0x7f441c02c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.508+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4424059970 con 0x7f44301451d0 2024-04-25T10:05:28.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.778+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4430140830 con 0x7f44301451d0 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.779+0000 7f442d600640 1 -- 172.21.15.5:0/2397272076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4424059270 con 0x7f44301451d0 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.784 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.786 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:05:28.787 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.790 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:05:28.790 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:05:28.790 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.792 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.793 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:05:28.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.796 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:05:28.798 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f441006b500 msgr2=0x7f441006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f441006b500 0x7f441006d9c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f441c034000 tx=0x7f441c02c000 comp rx=0 tx=0).stop 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 msgr2=0x7f44301e5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 0x7f44301e5030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4424007f20 tx=0x7f442400e570 comp rx=0 tx=0).stop 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 shutdown_connections 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f443013e730 0x7f443013de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f441006b500 0x7f441006d9c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f443015edd0 0x7f44301f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 --2- 172.21.15.5:0/2397272076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44301451d0 0x7f44301e5030 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.781+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 >> 172.21.15.5:0/2397272076 conn(0x7f4430066fd0 msgr2=0x7f44300b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.782+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 shutdown_connections 2024-04-25T10:05:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.782+0000 7f4438000640 1 -- 172.21.15.5:0/2397272076 wait complete. 2024-04-25T10:05:28.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.998+0000 7ff1b6c00640 1 Processor -- start 2024-04-25T10:05:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.998+0000 7ff1b6c00640 1 -- start start 2024-04-25T10:05:29.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.999+0000 7ff1b6c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1b0141da0 0x7ff1b006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.999+0000 7ff1b6c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff1b006e640 0x7ff1b015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.999+0000 7ff1b6c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b015d340 0x7ff1b00cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.999+0000 7ff1b6c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b0142180 con 0x7ff1b006e640 2024-04-25T10:05:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.999+0000 7ff1b6c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b0071d30 con 0x7ff1b0141da0 2024-04-25T10:05:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:28.999+0000 7ff1b6c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b006d620 con 0x7ff1b015d340 2024-04-25T10:05:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff1b006e640 0x7ff1b015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff1b006e640 0x7ff1b015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34196/0 (socket says 172.21.15.5:34196) 2024-04-25T10:05:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 -- 172.21.15.5:0/935500622 learned_addr learned my addr 172.21.15.5:0/935500622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:29.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b5800640 1 --2- 172.21.15.5:0/935500622 >> v2:172.21.15.80:3300/0 conn(0x7ff1b0141da0 0x7ff1b006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 -- 172.21.15.5:0/935500622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b015d340 msgr2=0x7ff1b00cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 --2- 172.21.15.5:0/935500622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b015d340 0x7ff1b00cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 -- 172.21.15.5:0/935500622 >> v2:172.21.15.80:3300/0 conn(0x7ff1b0141da0 msgr2=0x7ff1b006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 --2- 172.21.15.5:0/935500622 >> v2:172.21.15.80:3300/0 conn(0x7ff1b0141da0 0x7ff1b006e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.000+0000 7ff1b4e00640 1 -- 172.21.15.5:0/935500622 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b006d900 con 0x7ff1b006e640 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.001+0000 7ff1b4e00640 1 --2- 172.21.15.5:0/935500622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b006e640 0x7ff1b015ce00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006bbb0 tx=0x7ff19c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b0da33c2cb2e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1afe00640 1 -- 172.21.15.5:0/935500622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c037070 con 0x7ff1b006e640 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1afe00640 1 -- 172.21.15.5:0/935500622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c032440 con 0x7ff1b006e640 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1afe00640 1 -- 172.21.15.5:0/935500622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c032810 con 0x7ff1b006e640 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 -- 172.21.15.5:0/935500622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b006e640 msgr2=0x7ff1b015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/935500622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b006e640 0x7ff1b015ce00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff1b006bbb0 tx=0x7ff19c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 -- 172.21.15.5:0/935500622 shutdown_connections 2024-04-25T10:05:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/935500622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b015d340 0x7ff1b00cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/935500622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b006e640 0x7ff1b015ce00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/935500622 >> v2:172.21.15.80:3300/0 conn(0x7ff1b0141da0 0x7ff1b006e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.002+0000 7ff1b6c00640 1 -- 172.21.15.5:0/935500622 >> 172.21.15.5:0/935500622 conn(0x7ff1b0066f20 msgr2=0x7ff1b006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.003+0000 7ff1b6c00640 1 -- 172.21.15.5:0/935500622 shutdown_connections 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.003+0000 7ff1b6c00640 1 -- 172.21.15.5:0/935500622 wait complete. 2024-04-25T10:05:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.005+0000 7ff1b6c00640 1 Processor -- start 2024-04-25T10:05:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.005+0000 7ff1b6c00640 1 -- start start 2024-04-25T10:05:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.006+0000 7ff1b6c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b006e640 0x7ff1b01389f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.006+0000 7ff1b6c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1b0141da0 0x7ff1b01e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.006+0000 7ff1b6c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 0x7ff1b01e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.006+0000 7ff1b6c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00cd9c0 con 0x7ff1b015d340 2024-04-25T10:05:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.006+0000 7ff1b6c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00cd840 con 0x7ff1b0141da0 2024-04-25T10:05:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.006+0000 7ff1b6c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00cdb40 con 0x7ff1b006e640 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 0x7ff1b01e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 0x7ff1b01e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34210/0 (socket says 172.21.15.5:34210) 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b6200640 1 -- 172.21.15.5:0/834841453 learned_addr learned my addr 172.21.15.5:0/834841453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b4e00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1b0141da0 0x7ff1b01e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b5800640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b006e640 0x7ff1b01389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b6200640 1 -- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b006e640 msgr2=0x7ff1b01389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b6200640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b006e640 0x7ff1b01389f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.007+0000 7ff1b6200640 1 -- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1b0141da0 msgr2=0x7ff1b01e68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b6200640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1b0141da0 0x7ff1b01e68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b6200640 1 -- 172.21.15.5:0/834841453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b01e71b0 con 0x7ff1b015d340 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b5800640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b006e640 0x7ff1b01389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b4e00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1b0141da0 0x7ff1b01e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b6200640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 0x7ff1b01e5100 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800b450 tx=0x7ff1a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8015020 con 0x7ff1b015d340 2024-04-25T10:05:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a8011890 con 0x7ff1b015d340 2024-04-25T10:05:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b01e7490 con 0x7ff1b015d340 2024-04-25T10:05:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8011c60 con 0x7ff1b015d340 2024-04-25T10:05:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.008+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b01e7a50 con 0x7ff1b015d340 2024-04-25T10:05:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.010+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b0068f60 con 0x7ff1b015d340 2024-04-25T10:05:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.011+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1a8018020 con 0x7ff1b015d340 2024-04-25T10:05:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.011+0000 7ff1ad600640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff19006b4d0 0x7ff19006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.012+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff1a808a290 con 0x7ff1b015d340 2024-04-25T10:05:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.012+0000 7ff1b5800640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff19006b4d0 0x7ff19006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.013+0000 7ff1b5800640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff19006b4d0 0x7ff19006d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00b41e0 tx=0x7ff1a40074c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.017+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1a801f020 con 0x7ff1b015d340 2024-04-25T10:05:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.208+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1b00ccae0 con 0x7ff1b015d340 2024-04-25T10:05:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.210+0000 7ff1ad600640 1 -- 172.21.15.5:0/834841453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13056 (secure 0 0 0) 0x7ff1a800d040 con 0x7ff1b015d340 2024-04-25T10:05:29.215 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:29.215 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":60,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:28.279098+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff19006b4d0 msgr2=0x7ff19006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff19006b4d0 0x7ff19006d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00b41e0 tx=0x7ff1a40074c0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 msgr2=0x7ff1b01e5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 0x7ff1b01e5100 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800b450 tx=0x7ff1a800b910 comp rx=0 tx=0).stop 2024-04-25T10:05:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 shutdown_connections 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1b015d340 0x7ff1b01e5100 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff19006b4d0 0x7ff19006d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1b0141da0 0x7ff1b01e68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 --2- 172.21.15.5:0/834841453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1b006e640 0x7ff1b01389f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.212+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 >> 172.21.15.5:0/834841453 conn(0x7ff1b0066f20 msgr2=0x7ff1b006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.213+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 shutdown_connections 2024-04-25T10:05:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.213+0000 7ff1b6c00640 1 -- 172.21.15.5:0/834841453 wait complete. 2024-04-25T10:05:29.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:29.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.429+0000 7f893fe00640 1 Processor -- start 2024-04-25T10:05:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.429+0000 7f893fe00640 1 -- start start 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89400b48c0 0x7f894015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8940161d90 0x7f89400b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89401606e0 con 0x7f89400d3f90 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89401609c0 con 0x7f89400b48c0 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893fe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8940160ca0 con 0x7f8940161d90 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89400b48c0 0x7f894015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89400b48c0 0x7f894015fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34222/0 (socket says 172.21.15.5:34222) 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.431+0000 7f893ea00640 1 -- 172.21.15.5:0/783980670 learned_addr learned my addr 172.21.15.5:0/783980670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.432+0000 7f893f400640 1 --2- 172.21.15.5:0/783980670 >> v2:172.21.15.80:3300/0 conn(0x7f8940161d90 0x7f89400b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.432+0000 7f893ea00640 1 -- 172.21.15.5:0/783980670 >> v2:172.21.15.80:3300/0 conn(0x7f8940161d90 msgr2=0x7f89400b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.432+0000 7f893ea00640 1 --2- 172.21.15.5:0/783980670 >> v2:172.21.15.80:3300/0 conn(0x7f8940161d90 0x7f89400b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.432+0000 7f893ea00640 1 -- 172.21.15.5:0/783980670 >> v1:172.21.15.80:6789/0 conn(0x7f89400d3f90 legacy=0x7f8940161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.432+0000 7f893ea00640 1 -- 172.21.15.5:0/783980670 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940160f80 con 0x7f89400b48c0 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.433+0000 7f893ea00640 1 --2- 172.21.15.5:0/783980670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894015fe60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f894008d2e0 tx=0x7f893002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc3d2432f3673eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.433+0000 7f893e000640 1 -- 172.21.15.5:0/783980670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930037070 con 0x7f89400b48c0 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.433+0000 7f893e000640 1 -- 172.21.15.5:0/783980670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8930032440 con 0x7f89400b48c0 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.433+0000 7f893e000640 1 -- 172.21.15.5:0/783980670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930032810 con 0x7f89400b48c0 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 -- 172.21.15.5:0/783980670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 msgr2=0x7f894015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 --2- 172.21.15.5:0/783980670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894015fe60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f894008d2e0 tx=0x7f893002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 -- 172.21.15.5:0/783980670 shutdown_connections 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 --2- 172.21.15.5:0/783980670 >> v2:172.21.15.80:3300/0 conn(0x7f8940161d90 0x7f89400b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 --2- 172.21.15.5:0/783980670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894015fe60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 -- 172.21.15.5:0/783980670 >> 172.21.15.5:0/783980670 conn(0x7f894008af80 msgr2=0x7f89400d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 -- 172.21.15.5:0/783980670 shutdown_connections 2024-04-25T10:05:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.434+0000 7f893fe00640 1 -- 172.21.15.5:0/783980670 wait complete. 2024-04-25T10:05:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.436+0000 7f893fe00640 1 Processor -- start 2024-04-25T10:05:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.436+0000 7f893fe00640 1 -- start start 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89400d3f90 0x7f89400c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8940161d90 0x7f894015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401603a0 con 0x7f89400b48c0 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b4ca0 con 0x7f89400d3f90 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b30b0 con 0x7f8940161d90 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894014ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34234/0 (socket says 172.21.15.5:34234) 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 -- 172.21.15.5:0/2135506497 learned_addr learned my addr 172.21.15.5:0/2135506497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 -- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8940161d90 msgr2=0x7f894015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8940161d90 0x7f894015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 -- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89400d3f90 msgr2=0x7f89400c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89400d3f90 0x7f89400c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.437+0000 7f893ea00640 1 -- 172.21.15.5:0/2135506497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894014f660 con 0x7f89400b48c0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.438+0000 7f893ea00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894014ebe0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8930000c00 tx=0x7f89300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.438+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930041020 con 0x7f89400b48c0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.438+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300306c0 con 0x7f89400b48c0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.438+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930037040 con 0x7f89400b48c0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.438+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894014f940 con 0x7f89400b48c0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.438+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894014ff00 con 0x7f89400b48c0 2024-04-25T10:05:29.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.439+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8940068f60 con 0x7f89400b48c0 2024-04-25T10:05:29.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.441+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8930044020 con 0x7f89400b48c0 2024-04-25T10:05:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.442+0000 7f8936a00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.442+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f89300aa8d0 con 0x7f89400b48c0 2024-04-25T10:05:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.442+0000 7f8937e00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.444+0000 7f8937e00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891c06b4d0 0x7f891c06d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f894015c370 tx=0x7f8928007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.446+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f893004b030 con 0x7f89400b48c0 2024-04-25T10:05:29.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.643+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89400d0dd0 con 0x7f89400b48c0 2024-04-25T10:05:29.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.644+0000 7f8936a00640 1 -- 172.21.15.5:0/2135506497 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8930079900 con 0x7f89400b48c0 2024-04-25T10:05:29.649 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:05:29.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.646+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891c06b4d0 msgr2=0x7f891c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.646+0000 7f893fe00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891c06b4d0 0x7f891c06d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f894015c370 tx=0x7f8928007a00 comp rx=0 tx=0).stop 2024-04-25T10:05:29.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.646+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 msgr2=0x7f894014ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.646+0000 7f893fe00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894014ebe0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8930000c00 tx=0x7f89300046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 shutdown_connections 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8940161d90 0x7f894015caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f891c06b4d0 0x7f891c06d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89400d3f90 0x7f89400c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 --2- 172.21.15.5:0/2135506497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89400b48c0 0x7f894014ebe0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 >> 172.21.15.5:0/2135506497 conn(0x7f894008af80 msgr2=0x7f894015dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 shutdown_connections 2024-04-25T10:05:29.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.647+0000 7f893fe00640 1 -- 172.21.15.5:0/2135506497 wait complete. 2024-04-25T10:05:29.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.870+0000 7f4e56c00640 1 Processor -- start 2024-04-25T10:05:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.870+0000 7f4e56c00640 1 -- start start 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e56c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4e500b34d0 0x7f4e50162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e56c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 0x7f4e500cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e56c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e50166690 con 0x7f4e50161ae0 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e56c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e501631b0 con 0x7f4e500b34d0 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e56c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501657a0 con 0x7f4e500cd700 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e55800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4e50161ae0 0x7f4e500b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58502/0 (socket says 172.21.15.5:58502) 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.872+0000 7f4e55800640 1 -- 172.21.15.5:0/1043767753 learned_addr learned my addr 172.21.15.5:0/1043767753 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e54e00640 1 --2- 172.21.15.5:0/1043767753 >> v2:172.21.15.80:3300/0 conn(0x7f4e500b34d0 0x7f4e50162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e56200640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 0x7f4e500cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561994502 0 0) 0x7f4e50166690 con 0x7f4e50161ae0 2024-04-25T10:05:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e38003520 con 0x7f4e50161ae0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3738562770 0 0) 0x7f4e38003520 con 0x7f4e50161ae0 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 msgr2=0x7f4e500cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 0x7f4e500cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 >> v2:172.21.15.80:3300/0 conn(0x7f4e500b34d0 msgr2=0x7f4e50162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 --2- 172.21.15.5:0/1043767753 >> v2:172.21.15.80:3300/0 conn(0x7f4e500b34d0 0x7f4e50162c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e50165a80 con 0x7f4e50161ae0 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4e44001690 con 0x7f4e50161ae0 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 >> v1:172.21.15.5:6789/0 conn(0x7f4e50161ae0 legacy=0x7f4e500b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.873+0000 7f4e4fe00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 0x7f4e380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e4fe00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e38003f30 0x7f4e38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e4fe00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38004d40 0x7f4e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501631b0 con 0x7f4e38003f30 2024-04-25T10:05:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e501657a0 con 0x7f4e38004d40 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800b9c0 con 0x7f4e500cd700 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e54e00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e38003f30 0x7f4e38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e54e00640 1 -- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 msgr2=0x7f4e380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e55800640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 0x7f4e380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e56200640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38004d40 0x7f4e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e54e00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e500cd700 0x7f4e380039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e54e00640 1 -- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38004d40 msgr2=0x7f4e380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e54e00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38004d40 0x7f4e380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e54e00640 1 -- 172.21.15.5:0/1043767753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38003520 con 0x7f4e38003f30 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.874+0000 7f4e56200640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38004d40 0x7f4e380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.875+0000 7f4e54e00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e38003f30 0x7f4e38004800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009ac0 tx=0x7f4e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfceb8e63a623811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c036070 con 0x7f4e38003f30 2024-04-25T10:05:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c031400 con 0x7f4e38003f30 2024-04-25T10:05:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e4fe00640 1 -- 172.21.15.5:0/1043767753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c009690 con 0x7f4e38003f30 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56c00640 1 -- 172.21.15.5:0/1043767753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e38003f30 msgr2=0x7f4e38004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56c00640 1 --2- 172.21.15.5:0/1043767753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e38003f30 0x7f4e38004800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009ac0 tx=0x7f4e38005830 comp rx=0 tx=0).stop 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56200640 1 -- 172.21.15.5:0/1043767753 reap_dead start 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56c00640 1 -- 172.21.15.5:0/1043767753 shutdown_connections 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56c00640 1 -- 172.21.15.5:0/1043767753 >> 172.21.15.5:0/1043767753 conn(0x7f4e500b4f00 msgr2=0x7f4e5006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56c00640 1 -- 172.21.15.5:0/1043767753 shutdown_connections 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.876+0000 7f4e56c00640 1 -- 172.21.15.5:0/1043767753 wait complete. 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.878+0000 7f4e56c00640 1 Processor -- start 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.878+0000 7f4e56c00640 1 -- start start 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38003f30 0x7f4e50153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e50151ea0 0x7f4e5015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 0x7f4e501600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50166320 con 0x7f4e50152320 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50166040 con 0x7f4e38003f30 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e50161ec0 con 0x7f4e50151ea0 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.879+0000 7f4e56200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 0x7f4e501600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 0x7f4e501600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34260/0 (socket says 172.21.15.5:34260) 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 -- 172.21.15.5:0/3701996282 learned_addr learned my addr 172.21.15.5:0/3701996282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e55800640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38003f30 0x7f4e50153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e54e00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e50151ea0 0x7f4e5015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 -- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e50151ea0 msgr2=0x7f4e5015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e50151ea0 0x7f4e5015fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 -- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38003f30 msgr2=0x7f4e50153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38003f30 0x7f4e50153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 -- 172.21.15.5:0/3701996282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5013f160 con 0x7f4e50152320 2024-04-25T10:05:29.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.880+0000 7f4e56200640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 0x7f4e501600b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009f10 tx=0x7f4e4800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.881+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4801c070 con 0x7f4e50152320 2024-04-25T10:05:29.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.881+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e48017400 con 0x7f4e50152320 2024-04-25T10:05:29.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.881+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480177f0 con 0x7f4e50152320 2024-04-25T10:05:29.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.881+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5013f440 con 0x7f4e50152320 2024-04-25T10:05:29.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.881+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5013fa00 con 0x7f4e50152320 2024-04-25T10:05:29.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.882+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e500cd200 con 0x7f4e50152320 2024-04-25T10:05:29.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.883+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e4801f070 con 0x7f4e50152320 2024-04-25T10:05:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.884+0000 7f4e4d600640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e3006b500 0x7f4e3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.884+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4e48089f70 con 0x7f4e50152320 2024-04-25T10:05:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.885+0000 7f4e55800640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e3006b500 0x7f4e3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.886+0000 7f4e55800640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e3006b500 0x7f4e3006d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e44001880 tx=0x7f4e44009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:29.888+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e48012070 con 0x7f4e50152320 2024-04-25T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.090+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4e50068670 con 0x7f4e50152320 2024-04-25T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.091+0000 7f4e4d600640 1 -- 172.21.15.5:0/3701996282 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4e48058fa0 con 0x7f4e50152320 2024-04-25T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:05:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:05:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:05:30.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:30.097 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:05:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e3006b500 msgr2=0x7f4e3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e3006b500 0x7f4e3006d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4e44001880 tx=0x7f4e44009040 comp rx=0 tx=0).stop 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 msgr2=0x7f4e501600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 0x7f4e501600b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009f10 tx=0x7f4e4800f4a0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 shutdown_connections 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e50152320 0x7f4e501600b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e50151ea0 0x7f4e5015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e3006b500 0x7f4e3006d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 --2- 172.21.15.5:0/3701996282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e38003f30 0x7f4e50153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 >> 172.21.15.5:0/3701996282 conn(0x7f4e500b4f00 msgr2=0x7f4e5006d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 shutdown_connections 2024-04-25T10:05:30.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.094+0000 7f4e56c00640 1 -- 172.21.15.5:0/3701996282 wait complete. 2024-04-25T10:05:30.117 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:30.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:30.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.320+0000 7ff82b200640 1 Processor -- start 2024-04-25T10:05:30.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.321+0000 7ff82b200640 1 -- start start 2024-04-25T10:05:30.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.322+0000 7ff82b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff82c144e20 0x7ff82c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.322+0000 7ff82b200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff82c15f270 0x7ff82c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.322+0000 7ff82b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c071000 0x7ff82c071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.322+0000 7ff82b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0719b0 con 0x7ff82c144e20 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.322+0000 7ff82b200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c06e850 con 0x7ff82c15f270 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.322+0000 7ff82b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c15fdf0 con 0x7ff82c071000 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff82c144e20 0x7ff82c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff82c144e20 0x7ff82c15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34272/0 (socket says 172.21.15.5:34272) 2024-04-25T10:05:30.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 -- 172.21.15.5:0/723644535 learned_addr learned my addr 172.21.15.5:0/723644535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff823e00640 1 --2- 172.21.15.5:0/723644535 >> v2:172.21.15.80:3300/0 conn(0x7ff82c15f270 0x7ff82c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff82a800640 1 --2- 172.21.15.5:0/723644535 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c071000 0x7ff82c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 -- 172.21.15.5:0/723644535 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c071000 msgr2=0x7ff82c071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 --2- 172.21.15.5:0/723644535 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c071000 0x7ff82c071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 -- 172.21.15.5:0/723644535 >> v2:172.21.15.80:3300/0 conn(0x7ff82c15f270 msgr2=0x7ff82c15f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 --2- 172.21.15.5:0/723644535 >> v2:172.21.15.80:3300/0 conn(0x7ff82c15f270 0x7ff82c15f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff829e00640 1 -- 172.21.15.5:0/723644535 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c1600d0 con 0x7ff82c144e20 2024-04-25T10:05:30.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.323+0000 7ff823e00640 1 --2- 172.21.15.5:0/723644535 >> v2:172.21.15.80:3300/0 conn(0x7ff82c15f270 0x7ff82c15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.324+0000 7ff829e00640 1 --2- 172.21.15.5:0/723644535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c144e20 0x7ff82c15c870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff82c06e420 tx=0x7ff81c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85956dd81493ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.325+0000 7ff829400640 1 -- 172.21.15.5:0/723644535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c037070 con 0x7ff82c144e20 2024-04-25T10:05:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.325+0000 7ff829400640 1 -- 172.21.15.5:0/723644535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c032440 con 0x7ff82c144e20 2024-04-25T10:05:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.325+0000 7ff829400640 1 -- 172.21.15.5:0/723644535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c032810 con 0x7ff82c144e20 2024-04-25T10:05:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.325+0000 7ff82b200640 1 -- 172.21.15.5:0/723644535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c144e20 msgr2=0x7ff82c15c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.325+0000 7ff82b200640 1 --2- 172.21.15.5:0/723644535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c144e20 0x7ff82c15c870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff82c06e420 tx=0x7ff81c02a180 comp rx=0 tx=0).stop 2024-04-25T10:05:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 -- 172.21.15.5:0/723644535 shutdown_connections 2024-04-25T10:05:30.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 --2- 172.21.15.5:0/723644535 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c071000 0x7ff82c071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 --2- 172.21.15.5:0/723644535 >> v2:172.21.15.80:3300/0 conn(0x7ff82c15f270 0x7ff82c15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 --2- 172.21.15.5:0/723644535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c144e20 0x7ff82c15c870 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 -- 172.21.15.5:0/723644535 >> 172.21.15.5:0/723644535 conn(0x7ff82c066f20 msgr2=0x7ff82c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 -- 172.21.15.5:0/723644535 shutdown_connections 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.326+0000 7ff82b200640 1 -- 172.21.15.5:0/723644535 wait complete. 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.327+0000 7ff82b200640 1 Processor -- start 2024-04-25T10:05:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.327+0000 7ff82b200640 1 -- start start 2024-04-25T10:05:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.328+0000 7ff82b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 0x7ff82c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.328+0000 7ff82b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c144e20 0x7ff82c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.328+0000 7ff82b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff82c15f270 0x7ff82c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.328+0000 7ff82b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c145200 con 0x7ff82c071000 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.328+0000 7ff82b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c15c070 con 0x7ff82c15f270 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.328+0000 7ff82b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c145380 con 0x7ff82c144e20 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 0x7ff82c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 0x7ff82c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34278/0 (socket says 172.21.15.5:34278) 2024-04-25T10:05:30.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 -- 172.21.15.5:0/526827969 learned_addr learned my addr 172.21.15.5:0/526827969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:30.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 -- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c144e20 msgr2=0x7ff82c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff823e00640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c144e20 0x7ff82c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c144e20 0x7ff82c159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 -- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff82c15f270 msgr2=0x7ff82c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff82c15f270 0x7ff82c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.329+0000 7ff829e00640 1 -- 172.21.15.5:0/526827969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c143d70 con 0x7ff82c071000 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.330+0000 7ff829e00640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 0x7ff82c155dc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff81c000c00 tx=0x7ff81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.330+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c037040 con 0x7ff82c071000 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.330+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c042070 con 0x7ff82c071000 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.330+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c144050 con 0x7ff82c071000 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.330+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c030bd0 con 0x7ff82c071000 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.330+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c139890 con 0x7ff82c071000 2024-04-25T10:05:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.331+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c068f60 con 0x7ff82c071000 2024-04-25T10:05:30.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.332+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff81c047460 con 0x7ff82c071000 2024-04-25T10:05:30.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.333+0000 7ff822000640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff80806b4d0 0x7ff80806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:30.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.333+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff81c0aac70 con 0x7ff82c071000 2024-04-25T10:05:30.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.334+0000 7ff823e00640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff80806b4d0 0x7ff80806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.335+0000 7ff823e00640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff80806b4d0 0x7ff80806d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0ccc10 tx=0x7ff814008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:30.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.338+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff81c045360 con 0x7ff82c071000 2024-04-25T10:05:30.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:30.534+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff82c072180 con 0x7ff82c071000 2024-04-25T10:05:31.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.291+0000 7ff822000640 1 -- 172.21.15.5:0/526827969 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff81c079c10 con 0x7ff82c071000 2024-04-25T10:05:31.297 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff80806b4d0 msgr2=0x7ff80806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff80806b4d0 0x7ff80806d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0ccc10 tx=0x7ff814008040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 msgr2=0x7ff82c155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 0x7ff82c155dc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff81c000c00 tx=0x7ff81c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 shutdown_connections 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff80806b4d0 0x7ff80806d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff82c15f270 0x7ff82c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff82c144e20 0x7ff82c159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 --2- 172.21.15.5:0/526827969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82c071000 0x7ff82c155dc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 >> 172.21.15.5:0/526827969 conn(0x7ff82c066f20 msgr2=0x7ff82c15c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 shutdown_connections 2024-04-25T10:05:31.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.295+0000 7ff82b200640 1 -- 172.21.15.5:0/526827969 wait complete. 2024-04-25T10:05:31.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.531+0000 7f0b92200640 1 Processor -- start 2024-04-25T10:05:31.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.531+0000 7f0b92200640 1 -- start start 2024-04-25T10:05:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.532+0000 7f0b92200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0b8c161d90 0x7f0b8c1689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.532+0000 7f0b92200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c1606e0 con 0x7f0b8c0d3f90 2024-04-25T10:05:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.532+0000 7f0b92200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c1609c0 con 0x7f0b8c0b48c0 2024-04-25T10:05:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.532+0000 7f0b92200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c160ca0 con 0x7f0b8c161d90 2024-04-25T10:05:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.533+0000 7f0b8be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0b8c0d3f90 0x7f0b8c161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58512/0 (socket says 172.21.15.5:58512) 2024-04-25T10:05:31.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.533+0000 7f0b8be00640 1 -- 172.21.15.5:0/3588399280 learned_addr learned my addr 172.21.15.5:0/3588399280 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:31.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.533+0000 7f0b91800640 1 --2- 172.21.15.5:0/3588399280 >> v2:172.21.15.80:3300/0 conn(0x7f0b8c161d90 0x7f0b8c1689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.533+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362059964 0 0) 0x7f0b8c1606e0 con 0x7f0b8c0d3f90 2024-04-25T10:05:31.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.533+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b74003520 con 0x7f0b8c0d3f90 2024-04-25T10:05:31.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557498985 0 0) 0x7f0b74003520 con 0x7f0b8c0d3f90 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 >> v2:172.21.15.80:3300/0 conn(0x7f0b8c161d90 msgr2=0x7f0b8c1689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 --2- 172.21.15.5:0/3588399280 >> v2:172.21.15.80:3300/0 conn(0x7f0b8c161d90 0x7f0b8c1689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 >> v1:172.21.15.80:6789/0 conn(0x7f0b8c0b48c0 legacy=0x7f0b8c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c160f80 con 0x7f0b8c0d3f90 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0b780017f0 con 0x7f0b8c0d3f90 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 >> v1:172.21.15.5:6789/0 conn(0x7f0b8c0d3f90 legacy=0x7f0b8c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740039b0 0x7f0b74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c160ca0 con 0x7f0b740051c0 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7400c160 con 0x7f0b74004800 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7400c360 con 0x7f0b740039b0 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.534+0000 7f0b91800640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b91800640 1 -- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740039b0 msgr2=0x7f0b74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b91800640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740039b0 0x7f0b74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b91800640 1 -- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b74004800 msgr2=0x7f0b74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b8be00640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b91800640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b91800640 1 -- 172.21.15.5:0/3588399280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74003520 con 0x7f0b740051c0 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b8be00640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b74004800 0x7f0b74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.535+0000 7f0b91800640 1 -- 172.21.15.5:0/3588399280 reap_dead start 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.536+0000 7f0b91800640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009f90 tx=0x7f0b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5204f284bd89842d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.536+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84009c10 con 0x7f0b740051c0 2024-04-25T10:05:31.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.536+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b84031440 con 0x7f0b740051c0 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.536+0000 7f0b8b400640 1 -- 172.21.15.5:0/3588399280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b840317f0 con 0x7f0b740051c0 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.536+0000 7f0b92200640 1 -- 172.21.15.5:0/3588399280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b740051c0 msgr2=0x7f0b7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.536+0000 7f0b92200640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009f90 tx=0x7f0b74005830 comp rx=0 tx=0).stop 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.537+0000 7f0b92200640 1 -- 172.21.15.5:0/3588399280 shutdown_connections 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.537+0000 7f0b92200640 1 --2- 172.21.15.5:0/3588399280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b740051c0 0x7f0b7400b9c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.537+0000 7f0b92200640 1 -- 172.21.15.5:0/3588399280 >> 172.21.15.5:0/3588399280 conn(0x7f0b8c08af80 msgr2=0x7f0b8c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.537+0000 7f0b92200640 1 -- 172.21.15.5:0/3588399280 shutdown_connections 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.537+0000 7f0b92200640 1 -- 172.21.15.5:0/3588399280 wait complete. 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.538+0000 7f0b92200640 1 Processor -- start 2024-04-25T10:05:31.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 -- start start 2024-04-25T10:05:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740051c0 0x7f0b8c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c0c7ff0 0x7f0b8c0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 0x7f0b8c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c161680 con 0x7f0b8c0b6720 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0b4ca0 con 0x7f0b8c0c7ff0 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.539+0000 7f0b92200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c161800 con 0x7f0b740051c0 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 0x7f0b8c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 0x7f0b8c0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34302/0 (socket says 172.21.15.5:34302) 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 -- 172.21.15.5:0/3902335801 learned_addr learned my addr 172.21.15.5:0/3902335801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b90e00640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740051c0 0x7f0b8c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b8be00640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c0c7ff0 0x7f0b8c0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 -- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740051c0 msgr2=0x7f0b8c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740051c0 0x7f0b8c154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 -- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c0c7ff0 msgr2=0x7f0b8c0b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c0c7ff0 0x7f0b8c0b61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.540+0000 7f0b91800640 1 -- 172.21.15.5:0/3902335801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0cb420 con 0x7f0b8c0b6720 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.541+0000 7f0b91800640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 0x7f0b8c0c9700 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0b84033040 tx=0x7f0b84009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.541+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84009c10 con 0x7f0b8c0b6720 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.541+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b84043070 con 0x7f0b8c0b6720 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.541+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8403c680 con 0x7f0b8c0b6720 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.541+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c0cb6c0 con 0x7f0b8c0b6720 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.541+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c0cbc80 con 0x7f0b8c0b6720 2024-04-25T10:05:31.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.542+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c15ef20 con 0x7f0b8c0b6720 2024-04-25T10:05:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.543+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b84005d10 con 0x7f0b8c0b6720 2024-04-25T10:05:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.544+0000 7f0b88c00640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b6c06b520 0x7f0b6c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.544+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0b840aab50 con 0x7f0b8c0b6720 2024-04-25T10:05:31.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.545+0000 7f0b90e00640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b6c06b520 0x7f0b6c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.546+0000 7f0b90e00640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b6c06b520 0x7f0b6c06d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0cd690 tx=0x7f0b80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.549+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b8c15ef20 con 0x7f0b8c0b6720 2024-04-25T10:05:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:31.746+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0b8c0d0dd0 con 0x7f0b8c0b6720 2024-04-25T10:05:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.297+0000 7f0b88c00640 1 -- 172.21.15.5:0/3902335801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0b84079b20 con 0x7f0b8c0b6720 2024-04-25T10:05:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:05:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b6c06b520 msgr2=0x7f0b6c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b6c06b520 0x7f0b6c06d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0cd690 tx=0x7f0b80009040 comp rx=0 tx=0).stop 2024-04-25T10:05:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 msgr2=0x7f0b8c0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 0x7f0b8c0c9700 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0b84033040 tx=0x7f0b84009750 comp rx=0 tx=0).stop 2024-04-25T10:05:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 shutdown_connections 2024-04-25T10:05:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0b6720 0x7f0b8c0c9700 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b6c06b520 0x7f0b6c06d9e0 secure :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0cd690 tx=0x7f0b80009040 comp rx=0 tx=0).stop 2024-04-25T10:05:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c0c7ff0 0x7f0b8c0b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.305+0000 7f0b92200640 1 --2- 172.21.15.5:0/3902335801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b740051c0 0x7f0b8c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.306+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 >> 172.21.15.5:0/3902335801 conn(0x7f0b8c08af80 msgr2=0x7f0b8c06d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.313+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 shutdown_connections 2024-04-25T10:05:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.314+0000 7f0b92200640 1 -- 172.21.15.5:0/3902335801 wait complete. 2024-04-25T10:05:32.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:32.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.525+0000 7f7be8e00640 1 Processor -- start 2024-04-25T10:05:32.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.525+0000 7f7be8e00640 1 -- start start 2024-04-25T10:05:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.526+0000 7f7be8e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 0x7f7be40b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.526+0000 7f7be8e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7be41448d0 0x7f7be40b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.526+0000 7f7be8e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be406ba70 con 0x7f7be406b4e0 2024-04-25T10:05:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.526+0000 7f7be8e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be406bd50 con 0x7f7be41448d0 2024-04-25T10:05:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.526+0000 7f7be8e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be406c030 con 0x7f7be406b100 2024-04-25T10:05:32.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.527+0000 7f7be2a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7be41448d0 0x7f7be40b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.527+0000 7f7be3400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 0x7f7be40b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.527+0000 7f7be3400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 0x7f7be40b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:46648/0 (socket says 172.21.15.5:46648) 2024-04-25T10:05:32.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.527+0000 7f7be3400640 1 -- 172.21.15.5:0/2069325861 learned_addr learned my addr 172.21.15.5:0/2069325861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:32.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.528+0000 7f7be3400640 1 -- 172.21.15.5:0/2069325861 >> v2:172.21.15.80:3300/0 conn(0x7f7be41448d0 msgr2=0x7f7be40b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.528+0000 7f7be3400640 1 --2- 172.21.15.5:0/2069325861 >> v2:172.21.15.80:3300/0 conn(0x7f7be41448d0 0x7f7be40b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.528+0000 7f7be3400640 1 -- 172.21.15.5:0/2069325861 >> v1:172.21.15.80:6789/0 conn(0x7f7be406b4e0 legacy=0x7f7be415f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.528+0000 7f7be3400640 1 -- 172.21.15.5:0/2069325861 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be406c310 con 0x7f7be406b100 2024-04-25T10:05:32.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.529+0000 7f7be3400640 1 --2- 172.21.15.5:0/2069325861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 0x7f7be40b2f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009870 tx=0x7f7bd402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e589e5b1ea906eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be2000640 1 -- 172.21.15.5:0/2069325861 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4037070 con 0x7f7be406b100 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be2000640 1 -- 172.21.15.5:0/2069325861 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4032440 con 0x7f7be406b100 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be2000640 1 -- 172.21.15.5:0/2069325861 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4032810 con 0x7f7be406b100 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 -- 172.21.15.5:0/2069325861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 msgr2=0x7f7be40b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 --2- 172.21.15.5:0/2069325861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 0x7f7be40b2f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009870 tx=0x7f7bd402a250 comp rx=0 tx=0).stop 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 -- 172.21.15.5:0/2069325861 shutdown_connections 2024-04-25T10:05:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 --2- 172.21.15.5:0/2069325861 >> v2:172.21.15.80:3300/0 conn(0x7f7be41448d0 0x7f7be40b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 --2- 172.21.15.5:0/2069325861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b100 0x7f7be40b2f60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 -- 172.21.15.5:0/2069325861 >> 172.21.15.5:0/2069325861 conn(0x7f7be40cc6a0 msgr2=0x7f7be40ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.530+0000 7f7be8e00640 1 -- 172.21.15.5:0/2069325861 shutdown_connections 2024-04-25T10:05:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.531+0000 7f7be8e00640 1 -- 172.21.15.5:0/2069325861 wait complete. 2024-04-25T10:05:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.532+0000 7f7be8e00640 1 Processor -- start 2024-04-25T10:05:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.532+0000 7f7be8e00640 1 -- start start 2024-04-25T10:05:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.533+0000 7f7be8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 0x7f7be415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.533+0000 7f7be8e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b4e0 0x7f7be413ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.533+0000 7f7be8e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be41448d0 0x7f7be413f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.533+0000 7f7be8e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40cf080 con 0x7f7be406b100 2024-04-25T10:05:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.533+0000 7f7be8e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40cef00 con 0x7f7be41448d0 2024-04-25T10:05:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.533+0000 7f7be8e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40cf200 con 0x7f7be406b4e0 2024-04-25T10:05:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 0x7f7be415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 0x7f7be415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34308/0 (socket says 172.21.15.5:34308) 2024-04-25T10:05:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 -- 172.21.15.5:0/1162127583 learned_addr learned my addr 172.21.15.5:0/1162127583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be41448d0 0x7f7be413f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be2a00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b4e0 0x7f7be413ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 -- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b4e0 msgr2=0x7f7be413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b4e0 0x7f7be413ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 -- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be41448d0 msgr2=0x7f7be413f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be41448d0 0x7f7be413f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.534+0000 7f7be3400640 1 -- 172.21.15.5:0/1162127583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40b0690 con 0x7f7be406b100 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.535+0000 7f7be3400640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 0x7f7be415ab60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.535+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4037040 con 0x7f7be406b100 2024-04-25T10:05:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.535+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4042070 con 0x7f7be406b100 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.535+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be415dd40 con 0x7f7be406b100 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.535+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4030880 con 0x7f7be406b100 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.535+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be415e300 con 0x7f7be406b100 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.536+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be4068f60 con 0x7f7be406b100 2024-04-25T10:05:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.538+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bd4030a20 con 0x7f7be406b100 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.538+0000 7f7bdb400640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7bc406b4d0 0x7f7bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.539+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f7bd40ab680 con 0x7f7be406b100 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.539+0000 7f7be2a00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7bc406b4d0 0x7f7bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.540+0000 7f7be2a00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7bc406b4d0 0x7f7bc406d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7be40b10c0 tx=0x7f7bd0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:32.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.542+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7be40b0690 con 0x7f7be406b100 2024-04-25T10:05:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:32.780+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7be4072320 con 0x7f7be406b100 2024-04-25T10:05:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.346+0000 7f7bdb400640 1 -- 172.21.15.5:0/1162127583 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7bd407a4d0 con 0x7f7be406b100 2024-04-25T10:05:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T10:05:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.353+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7bc406b4d0 msgr2=0x7f7bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.353+0000 7f7be8e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7bc406b4d0 0x7f7bc406d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7be40b10c0 tx=0x7f7bd0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.353+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 msgr2=0x7f7be415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.353+0000 7f7be8e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 0x7f7be415ab60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd40046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.354+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 shutdown_connections 2024-04-25T10:05:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.354+0000 7f7be8e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7bc406b4d0 0x7f7bc406d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.354+0000 7f7be8e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7be41448d0 0x7f7be413f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.354+0000 7f7be8e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7be406b4e0 0x7f7be413ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.354+0000 7f7be8e00640 1 --2- 172.21.15.5:0/1162127583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7be406b100 0x7f7be415ab60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.354+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 >> 172.21.15.5:0/1162127583 conn(0x7f7be40cc6a0 msgr2=0x7f7be40737d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.355+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 shutdown_connections 2024-04-25T10:05:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.356+0000 7f7be8e00640 1 -- 172.21.15.5:0/1162127583 wait complete. 2024-04-25T10:05:33.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:33.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.559+0000 7fb986e00640 1 Processor -- start 2024-04-25T10:05:33.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.559+0000 7fb986e00640 1 -- start start 2024-04-25T10:05:33.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.560+0000 7fb986e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb980161200 0x7fb9800b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.560+0000 7fb986e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb9800d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.560+0000 7fb986e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9800d0600 0x7fb9800ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.560+0000 7fb986e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9800d27c0 con 0x7fb980161200 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.560+0000 7fb986e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9800b28b0 con 0x7fb9800d0600 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.560+0000 7fb986e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800cf410 con 0x7fb9800d2380 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb980161200 0x7fb9800b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb980161200 0x7fb9800b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34322/0 (socket says 172.21.15.5:34322) 2024-04-25T10:05:33.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 -- 172.21.15.5:0/3657709135 learned_addr learned my addr 172.21.15.5:0/3657709135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 -- 172.21.15.5:0/3657709135 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 msgr2=0x7fb9800d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb986400640 1 --2- 172.21.15.5:0/3657709135 >> v2:172.21.15.80:3300/0 conn(0x7fb9800d0600 0x7fb9800ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985000640 1 --2- 172.21.15.5:0/3657709135 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb9800d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 --2- 172.21.15.5:0/3657709135 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb9800d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 -- 172.21.15.5:0/3657709135 >> v2:172.21.15.80:3300/0 conn(0x7fb9800d0600 msgr2=0x7fb9800ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 --2- 172.21.15.5:0/3657709135 >> v2:172.21.15.80:3300/0 conn(0x7fb9800d0600 0x7fb9800ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb985a00640 1 -- 172.21.15.5:0/3657709135 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800cf6f0 con 0x7fb980161200 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.561+0000 7fb986400640 1 --2- 172.21.15.5:0/3657709135 >> v2:172.21.15.80:3300/0 conn(0x7fb9800d0600 0x7fb9800ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:33.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.562+0000 7fb985a00640 1 --2- 172.21.15.5:0/3657709135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800b4bb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb9800b4a80 tx=0x7fb97402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cf8e4c4fd35f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb97fe00640 1 -- 172.21.15.5:0/3657709135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974037070 con 0x7fb980161200 2024-04-25T10:05:33.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb97fe00640 1 -- 172.21.15.5:0/3657709135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb974032440 con 0x7fb980161200 2024-04-25T10:05:33.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb97fe00640 1 -- 172.21.15.5:0/3657709135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974032810 con 0x7fb980161200 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 -- 172.21.15.5:0/3657709135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 msgr2=0x7fb9800b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 --2- 172.21.15.5:0/3657709135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800b4bb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb9800b4a80 tx=0x7fb97402a200 comp rx=0 tx=0).stop 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 -- 172.21.15.5:0/3657709135 shutdown_connections 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 --2- 172.21.15.5:0/3657709135 >> v2:172.21.15.80:3300/0 conn(0x7fb9800d0600 0x7fb9800ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 --2- 172.21.15.5:0/3657709135 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb9800d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 --2- 172.21.15.5:0/3657709135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800b4bb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.563+0000 7fb986e00640 1 -- 172.21.15.5:0/3657709135 >> 172.21.15.5:0/3657709135 conn(0x7fb98008af80 msgr2=0x7fb9800d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.564+0000 7fb986e00640 1 -- 172.21.15.5:0/3657709135 shutdown_connections 2024-04-25T10:05:33.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.564+0000 7fb986e00640 1 -- 172.21.15.5:0/3657709135 wait complete. 2024-04-25T10:05:33.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.565+0000 7fb986e00640 1 Processor -- start 2024-04-25T10:05:33.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.565+0000 7fb986e00640 1 -- start start 2024-04-25T10:05:33.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.566+0000 7fb986e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800d0600 0x7fb98009f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.566+0000 7fb986e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb98009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.566+0000 7fb986e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.566+0000 7fb986e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800ce7c0 con 0x7fb980161200 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.566+0000 7fb986e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800d42b0 con 0x7fb9800d0600 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.566+0000 7fb986e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800ce940 con 0x7fb9800d2380 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34328/0 (socket says 172.21.15.5:34328) 2024-04-25T10:05:33.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 -- 172.21.15.5:0/2535266208 learned_addr learned my addr 172.21.15.5:0/2535266208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb985000640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb98009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb985a00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800d0600 0x7fb98009f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 -- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 msgr2=0x7fb98009d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb98009d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 -- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800d0600 msgr2=0x7fb98009f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800d0600 0x7fb98009f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 -- 172.21.15.5:0/2535266208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98009f940 con 0x7fb980161200 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.567+0000 7fb986400640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800cb5e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb97800cf20 tx=0x7fb97800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.568+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97801a070 con 0x7fb980161200 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.568+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97800d840 con 0x7fb980161200 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.568+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800dc30 con 0x7fb980161200 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.568+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98009fc20 con 0x7fb980161200 2024-04-25T10:05:33.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.568+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9800a01e0 con 0x7fb980161200 2024-04-25T10:05:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.569+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb980068f60 con 0x7fb980161200 2024-04-25T10:05:33.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.570+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb97801d070 con 0x7fb980161200 2024-04-25T10:05:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.571+0000 7fb97d600640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.571+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb97808acc0 con 0x7fb980161200 2024-04-25T10:05:33.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.572+0000 7fb985a00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.573+0000 7fb985a00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b4d0 0x7fb96006d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb974034000 tx=0x7fb97402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.576+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb978091080 con 0x7fb980161200 2024-04-25T10:05:33.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.767+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb98015e0f0 con 0x7fb980161200 2024-04-25T10:05:33.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.770+0000 7fb97d600640 1 -- 172.21.15.5:0/2535266208 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16837 (secure 0 0 0) 0x7fb978059910 con 0x7fb980161200 2024-04-25T10:05:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:33.775 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":63,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:32.782087+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b4d0 msgr2=0x7fb96006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b4d0 0x7fb96006d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb974034000 tx=0x7fb97402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 msgr2=0x7fb9800cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800cb5e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb97800cf20 tx=0x7fb97800a710 comp rx=0 tx=0).stop 2024-04-25T10:05:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 shutdown_connections 2024-04-25T10:05:33.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980161200 0x7fb9800cb5e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9800d2380 0x7fb98009d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b4d0 0x7fb96006d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.772+0000 7fb986e00640 1 --2- 172.21.15.5:0/2535266208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800d0600 0x7fb98009f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.773+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 >> 172.21.15.5:0/2535266208 conn(0x7fb98008af80 msgr2=0x7fb9800d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.773+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 shutdown_connections 2024-04-25T10:05:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:33.773+0000 7fb986e00640 1 -- 172.21.15.5:0/2535266208 wait complete. 2024-04-25T10:05:33.797 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:34.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.000+0000 7f35a9000640 1 Processor -- start 2024-04-25T10:05:34.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.000+0000 7f35a9000640 1 -- start start 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.001+0000 7f35a9000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35a4161200 0x7f35a40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.001+0000 7f35a9000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40cf410 con 0x7f35a40d2380 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.001+0000 7f35a9000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40cf6f0 con 0x7f35a40d07d0 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.001+0000 7f35a9000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40d27c0 con 0x7f35a4161200 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a2a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f35a40d2380 0x7f35a40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58514/0 (socket says 172.21.15.5:58514) 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a2a00640 1 -- 172.21.15.5:0/3701603251 learned_addr learned my addr 172.21.15.5:0/3701603251 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a3400640 1 --2- 172.21.15.5:0/3701603251 >> v2:172.21.15.5:3300/0 conn(0x7f35a4161200 0x7f35a40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a2000640 1 -- 172.21.15.5:0/3701603251 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208418425 0 0) 0x7f35a40cf410 con 0x7f35a40d2380 2024-04-25T10:05:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a2000640 1 -- 172.21.15.5:0/3701603251 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3584003520 con 0x7f35a40d2380 2024-04-25T10:05:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a3400640 1 -- 172.21.15.5:0/3701603251 >> v1:172.21.15.80:6789/0 conn(0x7f35a40d07d0 legacy=0x7f35a40b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a3400640 1 -- 172.21.15.5:0/3701603251 >> v1:172.21.15.5:6789/0 conn(0x7f35a40d2380 legacy=0x7f35a40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.002+0000 7f35a3400640 1 -- 172.21.15.5:0/3701603251 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40d3fd0 con 0x7f35a4161200 2024-04-25T10:05:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.004+0000 7f35a3400640 1 --2- 172.21.15.5:0/3701603251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a4161200 0x7f35a40b4bb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f35940097a0 tx=0x7f359402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2deaac985e78bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.004+0000 7f35a2000640 1 -- 172.21.15.5:0/3701603251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594037070 con 0x7f35a4161200 2024-04-25T10:05:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.004+0000 7f35a2000640 1 -- 172.21.15.5:0/3701603251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3594032440 con 0x7f35a4161200 2024-04-25T10:05:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.004+0000 7f35a2000640 1 -- 172.21.15.5:0/3701603251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594032810 con 0x7f35a4161200 2024-04-25T10:05:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 -- 172.21.15.5:0/3701603251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a4161200 msgr2=0x7f35a40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 --2- 172.21.15.5:0/3701603251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a4161200 0x7f35a40b4bb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f35940097a0 tx=0x7f359402a230 comp rx=0 tx=0).stop 2024-04-25T10:05:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 -- 172.21.15.5:0/3701603251 shutdown_connections 2024-04-25T10:05:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 --2- 172.21.15.5:0/3701603251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a4161200 0x7f35a40b4bb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 -- 172.21.15.5:0/3701603251 >> 172.21.15.5:0/3701603251 conn(0x7f35a408af80 msgr2=0x7f35a40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 -- 172.21.15.5:0/3701603251 shutdown_connections 2024-04-25T10:05:34.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.005+0000 7f35a9000640 1 -- 172.21.15.5:0/3701603251 wait complete. 2024-04-25T10:05:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.007+0000 7f35a9000640 1 Processor -- start 2024-04-25T10:05:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.007+0000 7f35a9000640 1 -- start start 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a9000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35a40d07d0 0x7f35a40cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 0x7f35a40b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a9000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35a4161200 0x7f35a41f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a9000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b3690 con 0x7f35a40d2380 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a9000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b28b0 con 0x7f35a40d07d0 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a9000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40b3810 con 0x7f35a4161200 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 0x7f35a40b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 0x7f35a40b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34346/0 (socket says 172.21.15.5:34346) 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.008+0000 7f35a2a00640 1 -- 172.21.15.5:0/3861989305 learned_addr learned my addr 172.21.15.5:0/3861989305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a3400640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35a40d07d0 0x7f35a40cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a3e00640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35a4161200 0x7f35a41f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a2a00640 1 -- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35a4161200 msgr2=0x7f35a41f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a2a00640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35a4161200 0x7f35a41f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a2a00640 1 -- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35a40d07d0 msgr2=0x7f35a40cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a2a00640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35a40d07d0 0x7f35a40cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a2a00640 1 -- 172.21.15.5:0/3861989305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40b6890 con 0x7f35a40d2380 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.009+0000 7f35a2a00640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 0x7f35a40b7930 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f358c00cbf0 tx=0x7f358c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.010+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c017070 con 0x7f35a40d2380 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.010+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00ad50 con 0x7f35a40d2380 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.010+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c012620 con 0x7f35a40d2380 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.010+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a40b6b70 con 0x7f35a40d2380 2024-04-25T10:05:34.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.010+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a40b7130 con 0x7f35a40d2380 2024-04-25T10:05:34.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.011+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a4068f60 con 0x7f35a40d2380 2024-04-25T10:05:34.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.012+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f358c01a070 con 0x7f35a40d2380 2024-04-25T10:05:34.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.013+0000 7f359b400640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358006b4d0 0x7f358006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.013+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f358c08a2e0 con 0x7f35a40d2380 2024-04-25T10:05:34.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.013+0000 7f35a3400640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358006b4d0 0x7f358006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.014+0000 7f35a3400640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358006b4d0 0x7f358006d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3594034000 tx=0x7f359402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.017+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f358c058fc0 con 0x7f35a40d2380 2024-04-25T10:05:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.220+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35a40ccb70 con 0x7f35a40d2380 2024-04-25T10:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.222+0000 7f359b400640 1 -- 172.21.15.5:0/3861989305 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f358c0588c0 con 0x7f35a40d2380 2024-04-25T10:05:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:34.228 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":63,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:32.782087+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358006b4d0 msgr2=0x7f358006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358006b4d0 0x7f358006d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3594034000 tx=0x7f359402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 msgr2=0x7f35a40b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 0x7f35a40b7930 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f358c00cbf0 tx=0x7f358c00a430 comp rx=0 tx=0).stop 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 shutdown_connections 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f35a4161200 0x7f35a41f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35a40d2380 0x7f35a40b7930 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f358006b4d0 0x7f358006d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 --2- 172.21.15.5:0/3861989305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35a40d07d0 0x7f35a40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 >> 172.21.15.5:0/3861989305 conn(0x7f35a408af80 msgr2=0x7f35a41603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 shutdown_connections 2024-04-25T10:05:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.225+0000 7f35a9000640 1 -- 172.21.15.5:0/3861989305 wait complete. 2024-04-25T10:05:34.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.462+0000 7fd083e00640 1 Processor -- start 2024-04-25T10:05:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.463+0000 7fd083e00640 1 -- start start 2024-04-25T10:05:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.464+0000 7fd083e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0841448f0 0x7fd0840cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.464+0000 7fd083e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd08415ce20 0x7fd08415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.464+0000 7fd083e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd084144cd0 con 0x7fd08415bc30 2024-04-25T10:05:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.464+0000 7fd083e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08415f460 con 0x7fd0841448f0 2024-04-25T10:05:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.464+0000 7fd083e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08415c440 con 0x7fd08415ce20 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0841448f0 0x7fd0840cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0841448f0 0x7fd0840cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34354/0 (socket says 172.21.15.5:34354) 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 -- 172.21.15.5:0/1506677723 learned_addr learned my addr 172.21.15.5:0/1506677723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd083400640 1 --2- 172.21.15.5:0/1506677723 >> v2:172.21.15.80:3300/0 conn(0x7fd08415ce20 0x7fd08415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 -- 172.21.15.5:0/1506677723 >> v2:172.21.15.80:3300/0 conn(0x7fd08415ce20 msgr2=0x7fd08415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 --2- 172.21.15.5:0/1506677723 >> v2:172.21.15.80:3300/0 conn(0x7fd08415ce20 0x7fd08415d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 -- 172.21.15.5:0/1506677723 >> v1:172.21.15.80:6789/0 conn(0x7fd08415bc30 legacy=0x7fd08415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.465+0000 7fd082a00640 1 -- 172.21.15.5:0/1506677723 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08415c720 con 0x7fd0841448f0 2024-04-25T10:05:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.466+0000 7fd082a00640 1 --2- 172.21.15.5:0/1506677723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0841448f0 0x7fd0840cdaa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd08406bc00 tx=0x7fd07402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00206e44a2061d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.467+0000 7fd082000640 1 -- 172.21.15.5:0/1506677723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074037070 con 0x7fd0841448f0 2024-04-25T10:05:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.467+0000 7fd082000640 1 -- 172.21.15.5:0/1506677723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074032440 con 0x7fd0841448f0 2024-04-25T10:05:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.467+0000 7fd082000640 1 -- 172.21.15.5:0/1506677723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074032810 con 0x7fd0841448f0 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 -- 172.21.15.5:0/1506677723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0841448f0 msgr2=0x7fd0840cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 --2- 172.21.15.5:0/1506677723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0841448f0 0x7fd0840cdaa0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd08406bc00 tx=0x7fd07402a230 comp rx=0 tx=0).stop 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 -- 172.21.15.5:0/1506677723 shutdown_connections 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 --2- 172.21.15.5:0/1506677723 >> v2:172.21.15.80:3300/0 conn(0x7fd08415ce20 0x7fd08415d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 --2- 172.21.15.5:0/1506677723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0841448f0 0x7fd0840cdaa0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 -- 172.21.15.5:0/1506677723 >> 172.21.15.5:0/1506677723 conn(0x7fd084066f20 msgr2=0x7fd08406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 -- 172.21.15.5:0/1506677723 shutdown_connections 2024-04-25T10:05:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.468+0000 7fd083e00640 1 -- 172.21.15.5:0/1506677723 wait complete. 2024-04-25T10:05:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.470+0000 7fd083e00640 1 Processor -- start 2024-04-25T10:05:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.470+0000 7fd083e00640 1 -- start start 2024-04-25T10:05:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd083e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0841448f0 0x7fd084140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd083e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 0x7fd08415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd083e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd08415ce20 0x7fd0841f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd083e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084160000 con 0x7fd08415bc30 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd083e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08415c0c0 con 0x7fd0841448f0 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd083e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd084160180 con 0x7fd08415ce20 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd07be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 0x7fd08415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd07be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 0x7fd08415af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34360/0 (socket says 172.21.15.5:34360) 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.471+0000 7fd07be00640 1 -- 172.21.15.5:0/3764580750 learned_addr learned my addr 172.21.15.5:0/3764580750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd082a00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0841448f0 0x7fd084140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd083400640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd08415ce20 0x7fd0841f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07be00640 1 -- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd08415ce20 msgr2=0x7fd0841f7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07be00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd08415ce20 0x7fd0841f7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07be00640 1 -- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0841448f0 msgr2=0x7fd084140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07be00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0841448f0 0x7fd084140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07be00640 1 -- 172.21.15.5:0/3764580750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0841f7b60 con 0x7fd08415bc30 2024-04-25T10:05:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07be00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 0x7fd08415af30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c430 tx=0x7fd06c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c011790 con 0x7fd08415bc30 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c017070 con 0x7fd08415bc30 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08413df30 con 0x7fd08415bc30 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.472+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c011bb0 con 0x7fd08415bc30 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.473+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08413fc70 con 0x7fd08415bc30 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.474+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd084068f60 con 0x7fd08415bc30 2024-04-25T10:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.475+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd06c01c460 con 0x7fd08415bc30 2024-04-25T10:05:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.475+0000 7fd07b400640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd06006b4d0 0x7fd06006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.476+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd06c08a270 con 0x7fd08415bc30 2024-04-25T10:05:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.476+0000 7fd082a00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd06006b4d0 0x7fd06006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.477+0000 7fd082a00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd06006b4d0 0x7fd06006d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd074034000 tx=0x7fd07402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.480+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0841f7b60 con 0x7fd08415bc30 2024-04-25T10:05:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:34.710+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd084072180 con 0x7fd08415bc30 2024-04-25T10:05:35.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.366+0000 7fd07b400640 1 -- 172.21.15.5:0/3764580750 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd06c058f60 con 0x7fd08415bc30 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd06006b4d0 msgr2=0x7fd06006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd06006b4d0 0x7fd06006d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd074034000 tx=0x7fd07402e000 comp rx=0 tx=0).stop 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 msgr2=0x7fd08415af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 0x7fd08415af30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00c430 tx=0x7fd06c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 shutdown_connections 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd08415ce20 0x7fd0841f7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd08415bc30 0x7fd08415af30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd06006b4d0 0x7fd06006d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.369+0000 7fd083e00640 1 --2- 172.21.15.5:0/3764580750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0841448f0 0x7fd084140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.370+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 >> 172.21.15.5:0/3764580750 conn(0x7fd084066f20 msgr2=0x7fd0840cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.370+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 shutdown_connections 2024-04-25T10:05:35.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.370+0000 7fd083e00640 1 -- 172.21.15.5:0/3764580750 wait complete. 2024-04-25T10:05:35.399 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:35.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.612+0000 7f43d1400640 1 Processor -- start 2024-04-25T10:05:35.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.612+0000 7f43d1400640 1 -- start start 2024-04-25T10:05:35.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.613+0000 7f43d1400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f43cc161d90 0x7f43cc1689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.613+0000 7f43d1400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc1606e0 con 0x7f43cc0b48c0 2024-04-25T10:05:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.613+0000 7f43d1400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc1609c0 con 0x7f43cc0d3f90 2024-04-25T10:05:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.613+0000 7f43d1400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc160ca0 con 0x7f43cc161d90 2024-04-25T10:05:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43cbe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f43cc0b48c0 0x7f43cc15fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58520/0 (socket says 172.21.15.5:58520) 2024-04-25T10:05:35.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43cbe00640 1 -- 172.21.15.5:0/1361900382 learned_addr learned my addr 172.21.15.5:0/1361900382 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183637592 0 0) 0x7f43cc1606e0 con 0x7f43cc0b48c0 2024-04-25T10:05:35.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43d0a00640 1 --2- 172.21.15.5:0/1361900382 >> v2:172.21.15.80:3300/0 conn(0x7f43cc161d90 0x7f43cc1689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b0003520 con 0x7f43cc0b48c0 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3064301222 0 0) 0x7f43b0003520 con 0x7f43cc0b48c0 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 >> v2:172.21.15.80:3300/0 conn(0x7f43cc161d90 msgr2=0x7f43cc1689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.614+0000 7f43caa00640 1 --2- 172.21.15.5:0/1361900382 >> v2:172.21.15.80:3300/0 conn(0x7f43cc161d90 0x7f43cc1689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 >> v1:172.21.15.80:6789/0 conn(0x7f43cc0d3f90 legacy=0x7f43cc161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc160f80 con 0x7f43cc0b48c0 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f43bc001670 con 0x7f43cc0b48c0 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 >> v1:172.21.15.5:6789/0 conn(0x7f43cc0b48c0 legacy=0x7f43cc15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43b0004800 0x7f43b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43b00052e0 0x7f43b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc160ca0 con 0x7f43b0003a00 2024-04-25T10:05:35.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b000c200 con 0x7f43b0004800 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b000c400 con 0x7f43b00052e0 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43cbe00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.615+0000 7f43cb400640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43b0004800 0x7f43b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.616+0000 7f43cbe00640 1 -- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43b00052e0 msgr2=0x7f43b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.616+0000 7f43cbe00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43b00052e0 0x7f43b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.616+0000 7f43cbe00640 1 -- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43b0004800 msgr2=0x7f43b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.616+0000 7f43cbe00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43b0004800 0x7f43b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.616+0000 7f43cbe00640 1 -- 172.21.15.5:0/1361900382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0003520 con 0x7f43b0003a00 2024-04-25T10:05:35.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.616+0000 7f43d0a00640 1 -- 172.21.15.5:0/1361900382 reap_dead start 2024-04-25T10:05:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.617+0000 7f43cbe00640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43b0003e20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c0e0 tx=0x7f43b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92af6c172b0765a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.617+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc007c20 con 0x7f43b0003a00 2024-04-25T10:05:35.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.617+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc005af0 con 0x7f43b0003a00 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.617+0000 7f43caa00640 1 -- 172.21.15.5:0/1361900382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc007070 con 0x7f43b0003a00 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.617+0000 7f43d1400640 1 -- 172.21.15.5:0/1361900382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 msgr2=0x7f43b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.617+0000 7f43d1400640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43b0003e20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c0e0 tx=0x7f43b0005830 comp rx=0 tx=0).stop 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.618+0000 7f43d1400640 1 -- 172.21.15.5:0/1361900382 shutdown_connections 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.618+0000 7f43d1400640 1 --2- 172.21.15.5:0/1361900382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43b0003e20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.618+0000 7f43d1400640 1 -- 172.21.15.5:0/1361900382 >> 172.21.15.5:0/1361900382 conn(0x7f43cc08af80 msgr2=0x7f43cc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.618+0000 7f43d1400640 1 -- 172.21.15.5:0/1361900382 shutdown_connections 2024-04-25T10:05:35.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.618+0000 7f43d1400640 1 -- 172.21.15.5:0/1361900382 wait complete. 2024-04-25T10:05:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.619+0000 7f43d1400640 1 Processor -- start 2024-04-25T10:05:35.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.620+0000 7f43d1400640 1 -- start start 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43d1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43cc0b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43d1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43cc0cbbf0 0x7f43cc0cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43d1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43cc0cf3d0 0x7f43cc0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43d1400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc161680 con 0x7f43b0003a00 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43d1400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0b4ca0 con 0x7f43cc0cf3d0 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43d1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc161800 con 0x7f43cc0cbbf0 2024-04-25T10:05:35.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43cc0b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43cc0b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34380/0 (socket says 172.21.15.5:34380) 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 -- 172.21.15.5:0/821617828 learned_addr learned my addr 172.21.15.5:0/821617828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 -- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43cc0cbbf0 msgr2=0x7f43cc0cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43cc0cbbf0 0x7f43cc0cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 -- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43cc0cf3d0 msgr2=0x7f43cc0b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43cc0cf3d0 0x7f43cc0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:35.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.621+0000 7f43cbe00640 1 -- 172.21.15.5:0/821617828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc1f3e40 con 0x7f43b0003a00 2024-04-25T10:05:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.622+0000 7f43cbe00640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43cc0b6a30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c0e0 tx=0x7f43cc161150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.622+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc037040 con 0x7f43b0003a00 2024-04-25T10:05:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.622+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43bc041070 con 0x7f43b0003a00 2024-04-25T10:05:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.622+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43bc0049e0 con 0x7f43b0003a00 2024-04-25T10:05:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.622+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc1f4120 con 0x7f43b0003a00 2024-04-25T10:05:35.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.622+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc1f46e0 con 0x7f43b0003a00 2024-04-25T10:05:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.623+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc068f60 con 0x7f43b0003a00 2024-04-25T10:05:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.624+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43bc004b80 con 0x7f43b0003a00 2024-04-25T10:05:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.626+0000 7f43c3e00640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f43a806b4d0 0x7f43a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:35.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.626+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43bc0aa380 con 0x7f43b0003a00 2024-04-25T10:05:35.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.626+0000 7f43cb400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f43a806b4d0 0x7f43a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:35.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.627+0000 7f43cb400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f43a806b4d0 0x7f43a806d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0cc010 tx=0x7f43b4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:35.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.630+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43bc079180 con 0x7f43b0003a00 2024-04-25T10:05:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:35.872+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f43cc0d0dd0 con 0x7f43b0003a00 2024-04-25T10:05:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.376+0000 7f43c3e00640 1 -- 172.21.15.5:0/821617828 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43bc078a80 con 0x7f43b0003a00 2024-04-25T10:05:36.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f43a806b4d0 msgr2=0x7f43a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f43a806b4d0 0x7f43a806d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0cc010 tx=0x7f43b4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 msgr2=0x7f43cc0b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43cc0b6a30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c0e0 tx=0x7f43cc161150 comp rx=0 tx=0).stop 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 shutdown_connections 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f43a806b4d0 0x7f43a806d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43cc0cf3d0 0x7f43cc0b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f43cc0cbbf0 0x7f43cc0cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 --2- 172.21.15.5:0/821617828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43b0003a00 0x7f43cc0b6a30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.380+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 >> 172.21.15.5:0/821617828 conn(0x7f43cc08af80 msgr2=0x7f43cc15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.381+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 shutdown_connections 2024-04-25T10:05:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.381+0000 7f43d1400640 1 -- 172.21.15.5:0/821617828 wait complete. 2024-04-25T10:05:36.411 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:36.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.613+0000 7f3a2b200640 1 Processor -- start 2024-04-25T10:05:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.613+0000 7f3a2b200640 1 -- start start 2024-04-25T10:05:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.614+0000 7f3a2b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a2406e970 0x7f3a240cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.614+0000 7f3a2b200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a240ced20 0x7f3a2415b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.614+0000 7f3a2b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a240cdbf0 con 0x7f3a24144100 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.614+0000 7f3a2b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a241444e0 con 0x7f3a2406e970 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.614+0000 7f3a2b200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2415ac70 con 0x7f3a240ced20 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a1be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a2406e970 0x7f3a240cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a23e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3a24144100 0x7f3a2406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58524/0 (socket says 172.21.15.5:58524) 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a23e00640 1 -- 172.21.15.5:0/3747116841 learned_addr learned my addr 172.21.15.5:0/3747116841 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a29200640 1 --2- 172.21.15.5:0/3747116841 >> v2:172.21.15.80:3300/0 conn(0x7f3a240ced20 0x7f3a2415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a1be00640 1 -- 172.21.15.5:0/3747116841 >> v2:172.21.15.80:3300/0 conn(0x7f3a240ced20 msgr2=0x7f3a2415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a1be00640 1 --2- 172.21.15.5:0/3747116841 >> v2:172.21.15.80:3300/0 conn(0x7f3a240ced20 0x7f3a2415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a1be00640 1 -- 172.21.15.5:0/3747116841 >> v1:172.21.15.5:6789/0 conn(0x7f3a24144100 legacy=0x7f3a2406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a23400640 1 -- 172.21.15.5:0/3747116841 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793857828 0 0) 0x7f3a240cdbf0 con 0x7f3a24144100 2024-04-25T10:05:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.615+0000 7f3a1be00640 1 -- 172.21.15.5:0/3747116841 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240cded0 con 0x7f3a2406e970 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.616+0000 7f3a1be00640 1 --2- 172.21.15.5:0/3747116841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a240cd610 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3a240710e0 tx=0x7f3a0c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c049170cff5478 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a23400640 1 -- 172.21.15.5:0/3747116841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c037070 con 0x7f3a2406e970 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a23400640 1 -- 172.21.15.5:0/3747116841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c032440 con 0x7f3a2406e970 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a23400640 1 -- 172.21.15.5:0/3747116841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c02ae60 con 0x7f3a2406e970 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a2b200640 1 -- 172.21.15.5:0/3747116841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 msgr2=0x7f3a240cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a2b200640 1 --2- 172.21.15.5:0/3747116841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a240cd610 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3a240710e0 tx=0x7f3a0c02a110 comp rx=0 tx=0).stop 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a2b200640 1 -- 172.21.15.5:0/3747116841 shutdown_connections 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a2b200640 1 --2- 172.21.15.5:0/3747116841 >> v2:172.21.15.80:3300/0 conn(0x7f3a240ced20 0x7f3a2415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a2b200640 1 --2- 172.21.15.5:0/3747116841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a240cd610 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.617+0000 7f3a2b200640 1 -- 172.21.15.5:0/3747116841 >> 172.21.15.5:0/3747116841 conn(0x7f3a24071150 msgr2=0x7f3a24071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.618+0000 7f3a2b200640 1 -- 172.21.15.5:0/3747116841 shutdown_connections 2024-04-25T10:05:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.618+0000 7f3a2b200640 1 -- 172.21.15.5:0/3747116841 wait complete. 2024-04-25T10:05:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.619+0000 7f3a2b200640 1 Processor -- start 2024-04-25T10:05:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.619+0000 7f3a2b200640 1 -- start start 2024-04-25T10:05:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.620+0000 7f3a2b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a241594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.620+0000 7f3a2b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a240ced20 0x7f3a2413c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.620+0000 7f3a2b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a24144100 0x7f3a241e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.620+0000 7f3a2b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240cea00 con 0x7f3a2406e970 2024-04-25T10:05:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.620+0000 7f3a2b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a240ce490 con 0x7f3a24144100 2024-04-25T10:05:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.620+0000 7f3a2b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2415b670 con 0x7f3a240ced20 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a241594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a241594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34402/0 (socket says 172.21.15.5:34402) 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 -- 172.21.15.5:0/1234051643 learned_addr learned my addr 172.21.15.5:0/1234051643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a29200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a24144100 0x7f3a241e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 -- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a240ced20 msgr2=0x7f3a2413c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a240ced20 0x7f3a2413c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 -- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a24144100 msgr2=0x7f3a241e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a24144100 0x7f3a241e1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 -- 172.21.15.5:0/1234051643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a24159a20 con 0x7f3a2406e970 2024-04-25T10:05:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.621+0000 7f3a23e00640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a241594e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400c430 tx=0x7f3a1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.622+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140074e0 con 0x7f3a2406e970 2024-04-25T10:05:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.622+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a140047e0 con 0x7f3a2406e970 2024-04-25T10:05:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.622+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a14004bd0 con 0x7f3a2406e970 2024-04-25T10:05:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.622+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2413df50 con 0x7f3a2406e970 2024-04-25T10:05:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.622+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2413c050 con 0x7f3a2406e970 2024-04-25T10:05:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.623+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a24068fd0 con 0x7f3a2406e970 2024-04-25T10:05:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.624+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a14011070 con 0x7f3a2406e970 2024-04-25T10:05:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.625+0000 7f3a20c00640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a0006b4d0 0x7f3a0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.625+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a1408a0a0 con 0x7f3a2406e970 2024-04-25T10:05:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.626+0000 7f3a1be00640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a0006b4d0 0x7f3a0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.627+0000 7f3a1be00640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a0006b4d0 0x7f3a0006d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c034000 tx=0x7f3a0c02c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.629+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a14058cf0 con 0x7f3a2406e970 2024-04-25T10:05:36.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.867+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a2413c430 con 0x7f3a2406e970 2024-04-25T10:05:36.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.869+0000 7f3a20c00640 1 -- 172.21.15.5:0/1234051643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f3a140585f0 con 0x7f3a2406e970 2024-04-25T10:05:36.874 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:36.874 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:05:35.874224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:36.874 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.871+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a0006b4d0 msgr2=0x7f3a0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.871+0000 7f3a2b200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a0006b4d0 0x7f3a0006d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c034000 tx=0x7f3a0c02c000 comp rx=0 tx=0).stop 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.871+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 msgr2=0x7f3a241594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.871+0000 7f3a2b200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a241594e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400c430 tx=0x7f3a1400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 shutdown_connections 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a0006b4d0 0x7f3a0006d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a24144100 0x7f3a241e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a240ced20 0x7f3a2413c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 --2- 172.21.15.5:0/1234051643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2406e970 0x7f3a241594e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 >> 172.21.15.5:0/1234051643 conn(0x7f3a24071150 msgr2=0x7f3a240c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:36.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 shutdown_connections 2024-04-25T10:05:36.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:36.872+0000 7f3a2b200640 1 -- 172.21.15.5:0/1234051643 wait complete. 2024-04-25T10:05:36.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.102+0000 7f59ca800640 1 Processor -- start 2024-04-25T10:05:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.103+0000 7f59ca800640 1 -- start start 2024-04-25T10:05:37.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.104+0000 7f59ca800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.104+0000 7f59ca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59c40d07d0 0x7f59c40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.104+0000 7f59ca800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c40cf410 con 0x7f59c40d2380 2024-04-25T10:05:37.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.104+0000 7f59ca800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c40d27c0 con 0x7f59c40d07d0 2024-04-25T10:05:37.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.104+0000 7f59ca800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40b28b0 con 0x7f59c4161200 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.104+0000 7f59c9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59c40d07d0 0x7f59c40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59c40d07d0 0x7f59c40b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39796/0 (socket says 172.21.15.5:39796) 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9e00640 1 -- 172.21.15.5:0/1325497856 learned_addr learned my addr 172.21.15.5:0/1325497856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9400640 1 --2- 172.21.15.5:0/1325497856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9e00640 1 -- 172.21.15.5:0/1325497856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 msgr2=0x7f59c40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9e00640 1 --2- 172.21.15.5:0/1325497856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9e00640 1 -- 172.21.15.5:0/1325497856 >> v1:172.21.15.80:6789/0 conn(0x7f59c40d2380 legacy=0x7f59c40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.105+0000 7f59c9e00640 1 -- 172.21.15.5:0/1325497856 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c40cf6f0 con 0x7f59c40d07d0 2024-04-25T10:05:37.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.106+0000 7f59c9e00640 1 --2- 172.21.15.5:0/1325497856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d07d0 0x7f59c40b2f80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098a0 tx=0x7f59bc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba0bd405d5279fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.106+0000 7f59c3e00640 1 -- 172.21.15.5:0/1325497856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc037070 con 0x7f59c40d07d0 2024-04-25T10:05:37.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.106+0000 7f59c3e00640 1 -- 172.21.15.5:0/1325497856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc032440 con 0x7f59c40d07d0 2024-04-25T10:05:37.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59c3e00640 1 -- 172.21.15.5:0/1325497856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc032810 con 0x7f59c40d07d0 2024-04-25T10:05:37.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 -- 172.21.15.5:0/1325497856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d07d0 msgr2=0x7f59c40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 --2- 172.21.15.5:0/1325497856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d07d0 0x7f59c40b2f80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0098a0 tx=0x7f59bc02a250 comp rx=0 tx=0).stop 2024-04-25T10:05:37.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 -- 172.21.15.5:0/1325497856 shutdown_connections 2024-04-25T10:05:37.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 --2- 172.21.15.5:0/1325497856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d07d0 0x7f59c40b2f80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 --2- 172.21.15.5:0/1325497856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 -- 172.21.15.5:0/1325497856 >> 172.21.15.5:0/1325497856 conn(0x7f59c408af80 msgr2=0x7f59c40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 -- 172.21.15.5:0/1325497856 shutdown_connections 2024-04-25T10:05:37.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.107+0000 7f59ca800640 1 -- 172.21.15.5:0/1325497856 wait complete. 2024-04-25T10:05:37.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.109+0000 7f59ca800640 1 Processor -- start 2024-04-25T10:05:37.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.109+0000 7f59ca800640 1 -- start start 2024-04-25T10:05:37.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59ca800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59c40d07d0 0x7f59c4158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 0x7f59c4158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59ca800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59ca800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40b34c0 con 0x7f59c40d2380 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59ca800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40d42b0 con 0x7f59c40d07d0 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59ca800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c40b3640 con 0x7f59c4161200 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.110+0000 7f59c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 0x7f59c4158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 0x7f59c4158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39804/0 (socket says 172.21.15.5:39804) 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 -- 172.21.15.5:0/2186296897 learned_addr learned my addr 172.21.15.5:0/2186296897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c9400640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59c40d07d0 0x7f59c4158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c9e00640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 -- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 msgr2=0x7f59c40c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 -- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59c40d07d0 msgr2=0x7f59c4158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59c40d07d0 0x7f59c4158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 -- 172.21.15.5:0/2186296897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c41f6fa0 con 0x7f59c40d2380 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.111+0000 7f59c8a00640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 0x7f59c4158520 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c430 tx=0x7f59b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.112+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011790 con 0x7f59c40d2380 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.112+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b0017070 con 0x7f59c40d2380 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.112+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011bb0 con 0x7f59c40d2380 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.112+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c41f7240 con 0x7f59c40d2380 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.112+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c41f7800 con 0x7f59c40d2380 2024-04-25T10:05:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.113+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c41f7b30 con 0x7f59c40d2380 2024-04-25T10:05:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.114+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59b001c460 con 0x7f59c40d2380 2024-04-25T10:05:37.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.115+0000 7f59c1600640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f59a406b500 0x7f59a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.115+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f59b008a2c0 con 0x7f59c40d2380 2024-04-25T10:05:37.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.116+0000 7f59c9400640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f59a406b500 0x7f59a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.117+0000 7f59c9400640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f59a406b500 0x7f59a406d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f59b40099c0 tx=0x7f59b4003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.120+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59b0058f10 con 0x7f59c40d2380 2024-04-25T10:05:37.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.310+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59c40b5d30 con 0x7f59c40d2380 2024-04-25T10:05:37.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.313+0000 7f59c1600640 1 -- 172.21.15.5:0/2186296897 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16824 (secure 0 0 0) 0x7f59c40b5d30 con 0x7f59c40d2380 2024-04-25T10:05:37.318 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:37.319 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":64,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:34.342121+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:37.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f59a406b500 msgr2=0x7f59a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f59a406b500 0x7f59a406d9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f59b40099c0 tx=0x7f59b4003990 comp rx=0 tx=0).stop 2024-04-25T10:05:37.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 msgr2=0x7f59c4158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 0x7f59c4158520 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c430 tx=0x7f59b000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 shutdown_connections 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59c4161200 0x7f59c40c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c40d2380 0x7f59c4158520 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f59a406b500 0x7f59a406d9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 --2- 172.21.15.5:0/2186296897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59c40d07d0 0x7f59c4158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 >> 172.21.15.5:0/2186296897 conn(0x7f59c408af80 msgr2=0x7f59c406bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 shutdown_connections 2024-04-25T10:05:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.316+0000 7f59ca800640 1 -- 172.21.15.5:0/2186296897 wait complete. 2024-04-25T10:05:37.341 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:37.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.548+0000 7fa82fe00640 1 Processor -- start 2024-04-25T10:05:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.548+0000 7fa82fe00640 1 -- start start 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.549+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa830161200 0x7fa8300b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.549+0000 7fa82fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa8300d07d0 0x7fa8300b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8300cf410 con 0x7fa8300d2380 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300d27c0 con 0x7fa8300d07d0 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300b28b0 con 0x7fa830161200 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa8300d07d0 0x7fa8300b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa8300d07d0 0x7fa8300b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39812/0 (socket says 172.21.15.5:39812) 2024-04-25T10:05:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 -- 172.21.15.5:0/3714071174 learned_addr learned my addr 172.21.15.5:0/3714071174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82ea00640 1 --2- 172.21.15.5:0/3714071174 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa830161200 0x7fa8300b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 -- 172.21.15.5:0/3714071174 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa830161200 msgr2=0x7fa8300b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 --2- 172.21.15.5:0/3714071174 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa830161200 0x7fa8300b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 -- 172.21.15.5:0/3714071174 >> v1:172.21.15.80:6789/0 conn(0x7fa8300d2380 legacy=0x7fa8300d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.550+0000 7fa82f400640 1 -- 172.21.15.5:0/3714071174 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300cf6f0 con 0x7fa8300d07d0 2024-04-25T10:05:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.551+0000 7fa82f400640 1 --2- 172.21.15.5:0/3714071174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d07d0 0x7fa8300b2f80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa828009e80 tx=0x7fa82802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b468c0bdc0eecb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.552+0000 7fa82d600640 1 -- 172.21.15.5:0/3714071174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828036070 con 0x7fa8300d07d0 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.552+0000 7fa82d600640 1 -- 172.21.15.5:0/3714071174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828031600 con 0x7fa8300d07d0 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.552+0000 7fa82d600640 1 -- 172.21.15.5:0/3714071174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280319d0 con 0x7fa8300d07d0 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.552+0000 7fa82fe00640 1 -- 172.21.15.5:0/3714071174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d07d0 msgr2=0x7fa8300b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.552+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3714071174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d07d0 0x7fa8300b2f80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa828009e80 tx=0x7fa82802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.553+0000 7fa82fe00640 1 -- 172.21.15.5:0/3714071174 shutdown_connections 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.553+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3714071174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d07d0 0x7fa8300b2f80 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.553+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3714071174 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa830161200 0x7fa8300b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.553+0000 7fa82fe00640 1 -- 172.21.15.5:0/3714071174 >> 172.21.15.5:0/3714071174 conn(0x7fa83008af80 msgr2=0x7fa8300d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.553+0000 7fa82fe00640 1 -- 172.21.15.5:0/3714071174 shutdown_connections 2024-04-25T10:05:37.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.553+0000 7fa82fe00640 1 -- 172.21.15.5:0/3714071174 wait complete. 2024-04-25T10:05:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.554+0000 7fa82fe00640 1 Processor -- start 2024-04-25T10:05:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.555+0000 7fa82fe00640 1 -- start start 2024-04-25T10:05:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8300d07d0 0x7fa830158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 0x7fa830158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa830161200 0x7fa8300c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300b34c0 con 0x7fa8300d2380 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300d42b0 con 0x7fa830161200 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300b3640 con 0x7fa8300d07d0 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 0x7fa830158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 0x7fa830158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39814/0 (socket says 172.21.15.5:39814) 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 -- 172.21.15.5:0/4286098546 learned_addr learned my addr 172.21.15.5:0/4286098546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82ea00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8300d07d0 0x7fa830158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82f400640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa830161200 0x7fa8300c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 -- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8300d07d0 msgr2=0x7fa830158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8300d07d0 0x7fa830158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 -- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa830161200 msgr2=0x7fa8300c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.556+0000 7fa82e000640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa830161200 0x7fa8300c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa82e000640 1 -- 172.21.15.5:0/4286098546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8301f6fa0 con 0x7fa8300d2380 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa82f400640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa830161200 0x7fa8300c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa82e000640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 0x7fa830158520 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c430 tx=0x7fa81800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818011790 con 0x7fa8300d2380 2024-04-25T10:05:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa8300d2380 2024-04-25T10:05:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818011bb0 con 0x7fa8300d2380 2024-04-25T10:05:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8301f7240 con 0x7fa8300d2380 2024-04-25T10:05:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.557+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8301f7800 con 0x7fa8300d2380 2024-04-25T10:05:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.559+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8301f7b30 con 0x7fa8300d2380 2024-04-25T10:05:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.560+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa81801c460 con 0x7fa8300d2380 2024-04-25T10:05:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.560+0000 7fa826a00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b500 0x7fa80c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.561+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa81808a2c0 con 0x7fa8300d2380 2024-04-25T10:05:37.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.561+0000 7fa82ea00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b500 0x7fa80c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:37.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.562+0000 7fa82ea00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b500 0x7fa80c06d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:37.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.565+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa818058f10 con 0x7fa8300d2380 2024-04-25T10:05:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.764+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa830068f60 con 0x7fa80c06b500 2024-04-25T10:05:37.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.783+0000 7fa826a00640 1 -- 172.21.15.5:0/4286098546 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252013 (secure 0 0 0) 0x7fa830068f60 con 0x7fa80c06b500 2024-04-25T10:05:37.795 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:05:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b500 msgr2=0x7fa80c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b500 0x7fa80c06d9c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa820006fd0 tx=0x7fa820008040 comp rx=0 tx=0).stop 2024-04-25T10:05:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 msgr2=0x7fa830158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 0x7fa830158520 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c430 tx=0x7fa81800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 shutdown_connections 2024-04-25T10:05:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa80c06b500 0x7fa80c06d9c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa830161200 0x7fa8300c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8300d2380 0x7fa830158520 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 --2- 172.21.15.5:0/4286098546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8300d07d0 0x7fa830158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 >> 172.21.15.5:0/4286098546 conn(0x7fa83008af80 msgr2=0x7fa83006bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 shutdown_connections 2024-04-25T10:05:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:37.789+0000 7fa82fe00640 1 -- 172.21.15.5:0/4286098546 wait complete. 2024-04-25T10:05:37.818 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-25T10:05:36.691007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244160,"kb_used_data":7172,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730560,"statfs":{"total":773094113280,"available":772844093440,"internally_reserved":0,"allocated":7344128,"data_stored":3903864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.397011"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294521+0000","last_change":"2024-04-25T10:05:33.358085+0000","last_active":"2024-04-25T10:05:35.294521+0000","last_peered":"2024-04-25T10:05:35.294521+0000","last_clean":"2024-04-25T10:05:35.294521+0000","last_became_active":"2024-04-25T10:05:33.357810+0000","last_became_peered":"2024-04-25T10:05:33.357810+0000","last_unstale":"2024-04-25T10:05:35.294521+0000","last_undegraded":"2024-04-25T10:05:35.294521+0000","last_fullsized":"2024-04-25T10:05:35.294521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:02.318344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354863+0000","last_change":"2024-04-25T10:05:33.366915+0000","last_active":"2024-04-25T10:05:34.354863+0000","last_peered":"2024-04-25T10:05:34.354863+0000","last_clean":"2024-04-25T10:05:34.354863+0000","last_became_active":"2024-04-25T10:05:33.363262+0000","last_became_peered":"2024-04-25T10:05:33.363262+0000","last_unstale":"2024-04-25T10:05:34.354863+0000","last_undegraded":"2024-04-25T10:05:34.354863+0000","last_fullsized":"2024-04-25T10:05:34.354863+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:33.828482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295997+0000","last_change":"2024-04-25T10:05:33.367255+0000","last_active":"2024-04-25T10:05:35.295997+0000","last_peered":"2024-04-25T10:05:35.295997+0000","last_clean":"2024-04-25T10:05:35.295997+0000","last_became_active":"2024-04-25T10:05:33.366898+0000","last_became_peered":"2024-04-25T10:05:33.366898+0000","last_unstale":"2024-04-25T10:05:35.295997+0000","last_undegraded":"2024-04-25T10:05:35.295997+0000","last_fullsized":"2024-04-25T10:05:35.295997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:13.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351309+0000","last_change":"2024-04-25T10:05:33.343798+0000","last_active":"2024-04-25T10:05:34.351309+0000","last_peered":"2024-04-25T10:05:34.351309+0000","last_clean":"2024-04-25T10:05:34.351309+0000","last_became_active":"2024-04-25T10:05:33.343340+0000","last_became_peered":"2024-04-25T10:05:33.343340+0000","last_unstale":"2024-04-25T10:05:34.351309+0000","last_undegraded":"2024-04-25T10:05:34.351309+0000","last_fullsized":"2024-04-25T10:05:34.351309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:27.622180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295670+0000","last_change":"2024-04-25T10:05:33.369441+0000","last_active":"2024-04-25T10:05:35.295670+0000","last_peered":"2024-04-25T10:05:35.295670+0000","last_clean":"2024-04-25T10:05:35.295670+0000","last_became_active":"2024-04-25T10:05:33.368432+0000","last_became_peered":"2024-04-25T10:05:33.368432+0000","last_unstale":"2024-04-25T10:05:35.295670+0000","last_undegraded":"2024-04-25T10:05:35.295670+0000","last_fullsized":"2024-04-25T10:05:35.295670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.919366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353989+0000","last_change":"2024-04-25T10:05:33.364065+0000","last_active":"2024-04-25T10:05:34.353989+0000","last_peered":"2024-04-25T10:05:34.353989+0000","last_clean":"2024-04-25T10:05:34.353989+0000","last_became_active":"2024-04-25T10:05:33.363331+0000","last_became_peered":"2024-04-25T10:05:33.363331+0000","last_unstale":"2024-04-25T10:05:34.353989+0000","last_undegraded":"2024-04-25T10:05:34.353989+0000","last_fullsized":"2024-04-25T10:05:34.353989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293731+0000","last_change":"2024-04-25T10:05:33.360268+0000","last_active":"2024-04-25T10:05:35.293731+0000","last_peered":"2024-04-25T10:05:35.293731+0000","last_clean":"2024-04-25T10:05:35.293731+0000","last_became_active":"2024-04-25T10:05:33.359954+0000","last_became_peered":"2024-04-25T10:05:33.359954+0000","last_unstale":"2024-04-25T10:05:35.293731+0000","last_undegraded":"2024-04-25T10:05:35.293731+0000","last_fullsized":"2024-04-25T10:05:35.293731+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:41.636006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294601+0000","last_change":"2024-04-25T10:05:33.352701+0000","last_active":"2024-04-25T10:05:35.294601+0000","last_peered":"2024-04-25T10:05:35.294601+0000","last_clean":"2024-04-25T10:05:35.294601+0000","last_became_active":"2024-04-25T10:05:33.352410+0000","last_became_peered":"2024-04-25T10:05:33.352410+0000","last_unstale":"2024-04-25T10:05:35.294601+0000","last_undegraded":"2024-04-25T10:05:35.294601+0000","last_fullsized":"2024-04-25T10:05:35.294601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:24.695367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350072+0000","last_change":"2024-04-25T10:05:33.382080+0000","last_active":"2024-04-25T10:05:34.350072+0000","last_peered":"2024-04-25T10:05:34.350072+0000","last_clean":"2024-04-25T10:05:34.350072+0000","last_became_active":"2024-04-25T10:05:33.381729+0000","last_became_peered":"2024-04-25T10:05:33.381729+0000","last_unstale":"2024-04-25T10:05:34.350072+0000","last_undegraded":"2024-04-25T10:05:34.350072+0000","last_fullsized":"2024-04-25T10:05:34.350072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:16.677491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707751+0000","last_change":"2024-04-25T10:05:33.364383+0000","last_active":"2024-04-25T10:05:34.707751+0000","last_peered":"2024-04-25T10:05:34.707751+0000","last_clean":"2024-04-25T10:05:34.707751+0000","last_became_active":"2024-04-25T10:05:33.364062+0000","last_became_peered":"2024-04-25T10:05:33.364062+0000","last_unstale":"2024-04-25T10:05:34.707751+0000","last_undegraded":"2024-04-25T10:05:34.707751+0000","last_fullsized":"2024-04-25T10:05:34.707751+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:46.085106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351257+0000","last_change":"2024-04-25T10:05:33.365949+0000","last_active":"2024-04-25T10:05:34.351257+0000","last_peered":"2024-04-25T10:05:34.351257+0000","last_clean":"2024-04-25T10:05:34.351257+0000","last_became_active":"2024-04-25T10:05:33.362338+0000","last_became_peered":"2024-04-25T10:05:33.362338+0000","last_unstale":"2024-04-25T10:05:34.351257+0000","last_undegraded":"2024-04-25T10:05:34.351257+0000","last_fullsized":"2024-04-25T10:05:34.351257+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.290132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351362+0000","last_change":"2024-04-25T10:05:33.356482+0000","last_active":"2024-04-25T10:05:34.351362+0000","last_peered":"2024-04-25T10:05:34.351362+0000","last_clean":"2024-04-25T10:05:34.351362+0000","last_became_active":"2024-04-25T10:05:33.356198+0000","last_became_peered":"2024-04-25T10:05:33.356198+0000","last_unstale":"2024-04-25T10:05:34.351362+0000","last_undegraded":"2024-04-25T10:05:34.351362+0000","last_fullsized":"2024-04-25T10:05:34.351362+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293199+0000","last_change":"2024-04-25T10:05:33.354098+0000","last_active":"2024-04-25T10:05:35.293199+0000","last_peered":"2024-04-25T10:05:35.293199+0000","last_clean":"2024-04-25T10:05:35.293199+0000","last_became_active":"2024-04-25T10:05:33.353582+0000","last_became_peered":"2024-04-25T10:05:33.353582+0000","last_unstale":"2024-04-25T10:05:35.293199+0000","last_undegraded":"2024-04-25T10:05:35.293199+0000","last_fullsized":"2024-04-25T10:05:35.293199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:46.424760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.292998+0000","last_change":"2024-04-25T10:05:33.366197+0000","last_active":"2024-04-25T10:05:35.292998+0000","last_peered":"2024-04-25T10:05:35.292998+0000","last_clean":"2024-04-25T10:05:35.292998+0000","last_became_active":"2024-04-25T10:05:33.363635+0000","last_became_peered":"2024-04-25T10:05:33.363635+0000","last_unstale":"2024-04-25T10:05:35.292998+0000","last_undegraded":"2024-04-25T10:05:35.292998+0000","last_fullsized":"2024-04-25T10:05:35.292998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:03.692180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707284+0000","last_change":"2024-04-25T10:05:33.364983+0000","last_active":"2024-04-25T10:05:34.707284+0000","last_peered":"2024-04-25T10:05:34.707284+0000","last_clean":"2024-04-25T10:05:34.707284+0000","last_became_active":"2024-04-25T10:05:33.364682+0000","last_became_peered":"2024-04-25T10:05:33.364682+0000","last_unstale":"2024-04-25T10:05:34.707284+0000","last_undegraded":"2024-04-25T10:05:34.707284+0000","last_fullsized":"2024-04-25T10:05:34.707284+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:29.479275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352611+0000","last_change":"2024-04-25T10:05:33.359243+0000","last_active":"2024-04-25T10:05:34.352611+0000","last_peered":"2024-04-25T10:05:34.352611+0000","last_clean":"2024-04-25T10:05:34.352611+0000","last_became_active":"2024-04-25T10:05:33.358941+0000","last_became_peered":"2024-04-25T10:05:33.358941+0000","last_unstale":"2024-04-25T10:05:34.352611+0000","last_undegraded":"2024-04-25T10:05:34.352611+0000","last_fullsized":"2024-04-25T10:05:34.352611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:23.280419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.362256+0000","last_change":"2024-04-25T10:05:33.358218+0000","last_active":"2024-04-25T10:05:33.362256+0000","last_peered":"2024-04-25T10:05:33.362256+0000","last_clean":"2024-04-25T10:05:33.362256+0000","last_became_active":"2024-04-25T10:05:33.356777+0000","last_became_peered":"2024-04-25T10:05:33.356777+0000","last_unstale":"2024-04-25T10:05:33.362256+0000","last_undegraded":"2024-04-25T10:05:33.362256+0000","last_fullsized":"2024-04-25T10:05:33.362256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:23.837823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351863+0000","last_change":"2024-04-25T10:05:33.368431+0000","last_active":"2024-04-25T10:05:34.351863+0000","last_peered":"2024-04-25T10:05:34.351863+0000","last_clean":"2024-04-25T10:05:34.351863+0000","last_became_active":"2024-04-25T10:05:33.368144+0000","last_became_peered":"2024-04-25T10:05:33.368144+0000","last_unstale":"2024-04-25T10:05:34.351863+0000","last_undegraded":"2024-04-25T10:05:34.351863+0000","last_fullsized":"2024-04-25T10:05:34.351863+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:50.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351494+0000","last_change":"2024-04-25T10:05:33.362815+0000","last_active":"2024-04-25T10:05:34.351494+0000","last_peered":"2024-04-25T10:05:34.351494+0000","last_clean":"2024-04-25T10:05:34.351494+0000","last_became_active":"2024-04-25T10:05:33.362289+0000","last_became_peered":"2024-04-25T10:05:33.362289+0000","last_unstale":"2024-04-25T10:05:34.351494+0000","last_undegraded":"2024-04-25T10:05:34.351494+0000","last_fullsized":"2024-04-25T10:05:34.351494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:23.476713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.354566+0000","last_change":"2024-04-25T10:05:33.348298+0000","last_active":"2024-04-25T10:05:33.354566+0000","last_peered":"2024-04-25T10:05:33.354566+0000","last_clean":"2024-04-25T10:05:33.354566+0000","last_became_active":"2024-04-25T10:05:33.347350+0000","last_became_peered":"2024-04-25T10:05:33.347350+0000","last_unstale":"2024-04-25T10:05:33.354566+0000","last_undegraded":"2024-04-25T10:05:33.354566+0000","last_fullsized":"2024-04-25T10:05:33.354566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:03.915686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293536+0000","last_change":"2024-04-25T10:05:33.360549+0000","last_active":"2024-04-25T10:05:35.293536+0000","last_peered":"2024-04-25T10:05:35.293536+0000","last_clean":"2024-04-25T10:05:35.293536+0000","last_became_active":"2024-04-25T10:05:33.360241+0000","last_became_peered":"2024-04-25T10:05:33.360241+0000","last_unstale":"2024-04-25T10:05:35.293536+0000","last_undegraded":"2024-04-25T10:05:35.293536+0000","last_fullsized":"2024-04-25T10:05:35.293536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:12.248661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354929+0000","last_change":"2024-04-25T10:05:33.363825+0000","last_active":"2024-04-25T10:05:34.354929+0000","last_peered":"2024-04-25T10:05:34.354929+0000","last_clean":"2024-04-25T10:05:34.354929+0000","last_became_active":"2024-04-25T10:05:33.362895+0000","last_became_peered":"2024-04-25T10:05:33.362895+0000","last_unstale":"2024-04-25T10:05:34.354929+0000","last_undegraded":"2024-04-25T10:05:34.354929+0000","last_fullsized":"2024-04-25T10:05:34.354929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:50.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352995+0000","last_change":"2024-04-25T10:05:33.366433+0000","last_active":"2024-04-25T10:05:34.352995+0000","last_peered":"2024-04-25T10:05:34.352995+0000","last_clean":"2024-04-25T10:05:34.352995+0000","last_became_active":"2024-04-25T10:05:33.366139+0000","last_became_peered":"2024-04-25T10:05:33.366139+0000","last_unstale":"2024-04-25T10:05:34.352995+0000","last_undegraded":"2024-04-25T10:05:34.352995+0000","last_fullsized":"2024-04-25T10:05:34.352995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.915967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295243+0000","last_change":"2024-04-25T10:05:33.375331+0000","last_active":"2024-04-25T10:05:35.295243+0000","last_peered":"2024-04-25T10:05:35.295243+0000","last_clean":"2024-04-25T10:05:35.295243+0000","last_became_active":"2024-04-25T10:05:33.372614+0000","last_became_peered":"2024-04-25T10:05:33.372614+0000","last_unstale":"2024-04-25T10:05:35.295243+0000","last_undegraded":"2024-04-25T10:05:35.295243+0000","last_fullsized":"2024-04-25T10:05:35.295243+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:52.407607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294197+0000","last_change":"2024-04-25T10:05:33.358404+0000","last_active":"2024-04-25T10:05:35.294197+0000","last_peered":"2024-04-25T10:05:35.294197+0000","last_clean":"2024-04-25T10:05:35.294197+0000","last_became_active":"2024-04-25T10:05:33.358078+0000","last_became_peered":"2024-04-25T10:05:33.358078+0000","last_unstale":"2024-04-25T10:05:35.294197+0000","last_undegraded":"2024-04-25T10:05:35.294197+0000","last_fullsized":"2024-04-25T10:05:35.294197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:29.479275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296245+0000","last_change":"2024-04-25T10:05:33.359621+0000","last_active":"2024-04-25T10:05:35.296245+0000","last_peered":"2024-04-25T10:05:35.296245+0000","last_clean":"2024-04-25T10:05:35.296245+0000","last_became_active":"2024-04-25T10:05:33.359275+0000","last_became_peered":"2024-04-25T10:05:33.359275+0000","last_unstale":"2024-04-25T10:05:35.296245+0000","last_undegraded":"2024-04-25T10:05:35.296245+0000","last_fullsized":"2024-04-25T10:05:35.296245+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:01.515552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293698+0000","last_change":"2024-04-25T10:05:33.362012+0000","last_active":"2024-04-25T10:05:35.293698+0000","last_peered":"2024-04-25T10:05:35.293698+0000","last_clean":"2024-04-25T10:05:35.293698+0000","last_became_active":"2024-04-25T10:05:33.361709+0000","last_became_peered":"2024-04-25T10:05:33.361709+0000","last_unstale":"2024-04-25T10:05:35.293698+0000","last_undegraded":"2024-04-25T10:05:35.293698+0000","last_fullsized":"2024-04-25T10:05:35.293698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:09.612571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294131+0000","last_change":"2024-04-25T10:05:33.374631+0000","last_active":"2024-04-25T10:05:35.294131+0000","last_peered":"2024-04-25T10:05:35.294131+0000","last_clean":"2024-04-25T10:05:35.294131+0000","last_became_active":"2024-04-25T10:05:33.372923+0000","last_became_peered":"2024-04-25T10:05:33.372923+0000","last_unstale":"2024-04-25T10:05:35.294131+0000","last_undegraded":"2024-04-25T10:05:35.294131+0000","last_fullsized":"2024-04-25T10:05:35.294131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:16.677491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350665+0000","last_change":"2024-04-25T10:05:33.375855+0000","last_active":"2024-04-25T10:05:34.350665+0000","last_peered":"2024-04-25T10:05:34.350665+0000","last_clean":"2024-04-25T10:05:34.350665+0000","last_became_active":"2024-04-25T10:05:33.374097+0000","last_became_peered":"2024-04-25T10:05:33.374097+0000","last_unstale":"2024-04-25T10:05:34.350665+0000","last_undegraded":"2024-04-25T10:05:34.350665+0000","last_fullsized":"2024-04-25T10:05:34.350665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:02.318344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350001+0000","last_change":"2024-04-25T10:05:33.382091+0000","last_active":"2024-04-25T10:05:34.350001+0000","last_peered":"2024-04-25T10:05:34.350001+0000","last_clean":"2024-04-25T10:05:34.350001+0000","last_became_active":"2024-04-25T10:05:33.379536+0000","last_became_peered":"2024-04-25T10:05:33.379536+0000","last_unstale":"2024-04-25T10:05:34.350001+0000","last_undegraded":"2024-04-25T10:05:34.350001+0000","last_fullsized":"2024-04-25T10:05:34.350001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:26.941321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295334+0000","last_change":"2024-04-25T10:05:33.367118+0000","last_active":"2024-04-25T10:05:35.295334+0000","last_peered":"2024-04-25T10:05:35.295334+0000","last_clean":"2024-04-25T10:05:35.295334+0000","last_became_active":"2024-04-25T10:05:33.366791+0000","last_became_peered":"2024-04-25T10:05:33.366791+0000","last_unstale":"2024-04-25T10:05:35.295334+0000","last_undegraded":"2024-04-25T10:05:35.295334+0000","last_fullsized":"2024-04-25T10:05:35.295334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:52.171069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293827+0000","last_change":"2024-04-25T10:05:33.360858+0000","last_active":"2024-04-25T10:05:35.293827+0000","last_peered":"2024-04-25T10:05:35.293827+0000","last_clean":"2024-04-25T10:05:35.293827+0000","last_became_active":"2024-04-25T10:05:33.360540+0000","last_became_peered":"2024-04-25T10:05:33.360540+0000","last_unstale":"2024-04-25T10:05:35.293827+0000","last_undegraded":"2024-04-25T10:05:35.293827+0000","last_fullsized":"2024-04-25T10:05:35.293827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:32.515551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352513+0000","last_change":"2024-04-25T10:05:33.362644+0000","last_active":"2024-04-25T10:05:34.352513+0000","last_peered":"2024-04-25T10:05:34.352513+0000","last_clean":"2024-04-25T10:05:34.352513+0000","last_became_active":"2024-04-25T10:05:33.362276+0000","last_became_peered":"2024-04-25T10:05:33.362276+0000","last_unstale":"2024-04-25T10:05:34.352513+0000","last_undegraded":"2024-04-25T10:05:34.352513+0000","last_fullsized":"2024-04-25T10:05:34.352513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.452945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294512+0000","last_change":"2024-04-25T10:05:32.342003+0000","last_active":"2024-04-25T10:05:35.294512+0000","last_peered":"2024-04-25T10:05:35.294512+0000","last_clean":"2024-04-25T10:05:35.294512+0000","last_became_active":"2024-04-25T10:05:32.341742+0000","last_became_peered":"2024-04-25T10:05:32.341742+0000","last_unstale":"2024-04-25T10:05:35.294512+0000","last_undegraded":"2024-04-25T10:05:35.294512+0000","last_fullsized":"2024-04-25T10:05:35.294512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:28.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296325+0000","last_change":"2024-04-25T10:05:33.356123+0000","last_active":"2024-04-25T10:05:35.296325+0000","last_peered":"2024-04-25T10:05:35.296325+0000","last_clean":"2024-04-25T10:05:35.296325+0000","last_became_active":"2024-04-25T10:05:33.355811+0000","last_became_peered":"2024-04-25T10:05:33.355811+0000","last_unstale":"2024-04-25T10:05:35.296325+0000","last_undegraded":"2024-04-25T10:05:35.296325+0000","last_fullsized":"2024-04-25T10:05:35.296325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:04.047617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294276+0000","last_change":"2024-04-25T10:05:32.330756+0000","last_active":"2024-04-25T10:05:35.294276+0000","last_peered":"2024-04-25T10:05:35.294276+0000","last_clean":"2024-04-25T10:05:35.294276+0000","last_became_active":"2024-04-25T10:05:32.330349+0000","last_became_peered":"2024-04-25T10:05:32.330349+0000","last_unstale":"2024-04-25T10:05:35.294276+0000","last_undegraded":"2024-04-25T10:05:35.294276+0000","last_fullsized":"2024-04-25T10:05:35.294276+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:11.235686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.355114+0000","last_change":"2024-04-25T10:05:33.363634+0000","last_active":"2024-04-25T10:05:34.355114+0000","last_peered":"2024-04-25T10:05:34.355114+0000","last_clean":"2024-04-25T10:05:34.355114+0000","last_became_active":"2024-04-25T10:05:33.363311+0000","last_became_peered":"2024-04-25T10:05:33.363311+0000","last_unstale":"2024-04-25T10:05:34.355114+0000","last_undegraded":"2024-04-25T10:05:34.355114+0000","last_fullsized":"2024-04-25T10:05:34.355114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:14.465628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294734+0000","last_change":"2024-04-25T10:05:32.335473+0000","last_active":"2024-04-25T10:05:35.294734+0000","last_peered":"2024-04-25T10:05:35.294734+0000","last_clean":"2024-04-25T10:05:35.294734+0000","last_became_active":"2024-04-25T10:05:32.335195+0000","last_became_peered":"2024-04-25T10:05:32.335195+0000","last_unstale":"2024-04-25T10:05:35.294734+0000","last_undegraded":"2024-04-25T10:05:35.294734+0000","last_fullsized":"2024-04-25T10:05:35.294734+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:05.273043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352124+0000","last_change":"2024-04-25T10:05:33.342494+0000","last_active":"2024-04-25T10:05:34.352124+0000","last_peered":"2024-04-25T10:05:34.352124+0000","last_clean":"2024-04-25T10:05:34.352124+0000","last_became_active":"2024-04-25T10:05:33.342075+0000","last_became_peered":"2024-04-25T10:05:33.342075+0000","last_unstale":"2024-04-25T10:05:34.352124+0000","last_undegraded":"2024-04-25T10:05:34.352124+0000","last_fullsized":"2024-04-25T10:05:34.352124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:14.465628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354107+0000","last_change":"2024-04-25T10:05:32.340386+0000","last_active":"2024-04-25T10:05:34.354107+0000","last_peered":"2024-04-25T10:05:34.354107+0000","last_clean":"2024-04-25T10:05:34.354107+0000","last_became_active":"2024-04-25T10:05:32.340089+0000","last_became_peered":"2024-04-25T10:05:32.340089+0000","last_unstale":"2024-04-25T10:05:34.354107+0000","last_undegraded":"2024-04-25T10:05:34.354107+0000","last_fullsized":"2024-04-25T10:05:34.354107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:41.502547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.348565+0000","last_change":"2024-04-25T10:05:33.341527+0000","last_active":"2024-04-25T10:05:33.348565+0000","last_peered":"2024-04-25T10:05:33.348565+0000","last_clean":"2024-04-25T10:05:33.348565+0000","last_became_active":"2024-04-25T10:05:33.340718+0000","last_became_peered":"2024-04-25T10:05:33.340718+0000","last_unstale":"2024-04-25T10:05:33.348565+0000","last_undegraded":"2024-04-25T10:05:33.348565+0000","last_fullsized":"2024-04-25T10:05:33.348565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:21.584362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.372956+0000","last_change":"2024-04-25T10:05:32.335077+0000","last_active":"2024-04-25T10:05:33.372956+0000","last_peered":"2024-04-25T10:05:33.372956+0000","last_clean":"2024-04-25T10:05:33.372956+0000","last_became_active":"2024-04-25T10:05:32.334801+0000","last_became_peered":"2024-04-25T10:05:32.334801+0000","last_unstale":"2024-04-25T10:05:33.372956+0000","last_undegraded":"2024-04-25T10:05:33.372956+0000","last_fullsized":"2024-04-25T10:05:33.372956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.404475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293826+0000","last_change":"2024-04-25T10:05:33.354848+0000","last_active":"2024-04-25T10:05:35.293826+0000","last_peered":"2024-04-25T10:05:35.293826+0000","last_clean":"2024-04-25T10:05:35.293826+0000","last_became_active":"2024-04-25T10:05:33.354478+0000","last_became_peered":"2024-04-25T10:05:33.354478+0000","last_unstale":"2024-04-25T10:05:35.293826+0000","last_undegraded":"2024-04-25T10:05:35.293826+0000","last_fullsized":"2024-04-25T10:05:35.293826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.206514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.355038+0000","last_change":"2024-04-25T10:05:32.343554+0000","last_active":"2024-04-25T10:05:34.355038+0000","last_peered":"2024-04-25T10:05:34.355038+0000","last_clean":"2024-04-25T10:05:34.355038+0000","last_became_active":"2024-04-25T10:05:32.342963+0000","last_became_peered":"2024-04-25T10:05:32.342963+0000","last_unstale":"2024-04-25T10:05:34.355038+0000","last_undegraded":"2024-04-25T10:05:34.355038+0000","last_fullsized":"2024-04-25T10:05:34.355038+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.200654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707448+0000","last_change":"2024-04-25T10:05:33.377799+0000","last_active":"2024-04-25T10:05:34.707448+0000","last_peered":"2024-04-25T10:05:34.707448+0000","last_clean":"2024-04-25T10:05:34.707448+0000","last_became_active":"2024-04-25T10:05:33.377467+0000","last_became_peered":"2024-04-25T10:05:33.377467+0000","last_unstale":"2024-04-25T10:05:34.707448+0000","last_undegraded":"2024-04-25T10:05:34.707448+0000","last_fullsized":"2024-04-25T10:05:34.707448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:28.913557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293972+0000","last_change":"2024-04-25T10:05:32.330296+0000","last_active":"2024-04-25T10:05:35.293972+0000","last_peered":"2024-04-25T10:05:35.293972+0000","last_clean":"2024-04-25T10:05:35.293972+0000","last_became_active":"2024-04-25T10:05:32.329971+0000","last_became_peered":"2024-04-25T10:05:32.329971+0000","last_unstale":"2024-04-25T10:05:35.293972+0000","last_undegraded":"2024-04-25T10:05:35.293972+0000","last_fullsized":"2024-04-25T10:05:35.293972+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:08.599596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.356251+0000","last_change":"2024-04-25T10:05:33.349092+0000","last_active":"2024-04-25T10:05:33.356251+0000","last_peered":"2024-04-25T10:05:33.356251+0000","last_clean":"2024-04-25T10:05:33.356251+0000","last_became_active":"2024-04-25T10:05:33.348812+0000","last_became_peered":"2024-04-25T10:05:33.348812+0000","last_unstale":"2024-04-25T10:05:33.356251+0000","last_undegraded":"2024-04-25T10:05:33.356251+0000","last_fullsized":"2024-04-25T10:05:33.356251+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:17.600078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.349785+0000","last_change":"2024-04-25T10:05:32.324120+0000","last_active":"2024-04-25T10:05:34.349785+0000","last_peered":"2024-04-25T10:05:34.349785+0000","last_clean":"2024-04-25T10:05:34.349785+0000","last_became_active":"2024-04-25T10:05:32.323551+0000","last_became_peered":"2024-04-25T10:05:32.323551+0000","last_unstale":"2024-04-25T10:05:34.349785+0000","last_undegraded":"2024-04-25T10:05:34.349785+0000","last_fullsized":"2024-04-25T10:05:34.349785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:39.618543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294328+0000","last_change":"2024-04-25T10:05:33.362210+0000","last_active":"2024-04-25T10:05:35.294328+0000","last_peered":"2024-04-25T10:05:35.294328+0000","last_clean":"2024-04-25T10:05:35.294328+0000","last_became_active":"2024-04-25T10:05:33.361755+0000","last_became_peered":"2024-04-25T10:05:33.361755+0000","last_unstale":"2024-04-25T10:05:35.294328+0000","last_undegraded":"2024-04-25T10:05:35.294328+0000","last_fullsized":"2024-04-25T10:05:35.294328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.330430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352121+0000","last_change":"2024-04-25T10:05:32.341935+0000","last_active":"2024-04-25T10:05:34.352121+0000","last_peered":"2024-04-25T10:05:34.352121+0000","last_clean":"2024-04-25T10:05:34.352121+0000","last_became_active":"2024-04-25T10:05:32.340956+0000","last_became_peered":"2024-04-25T10:05:32.340956+0000","last_unstale":"2024-04-25T10:05:34.352121+0000","last_undegraded":"2024-04-25T10:05:34.352121+0000","last_fullsized":"2024-04-25T10:05:34.352121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:39.688835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.362264+0000","last_change":"2024-04-25T10:05:33.357527+0000","last_active":"2024-04-25T10:05:33.362264+0000","last_peered":"2024-04-25T10:05:33.362264+0000","last_clean":"2024-04-25T10:05:33.362264+0000","last_became_active":"2024-04-25T10:05:33.357167+0000","last_became_peered":"2024-04-25T10:05:33.357167+0000","last_unstale":"2024-04-25T10:05:33.362264+0000","last_undegraded":"2024-04-25T10:05:33.362264+0000","last_fullsized":"2024-04-25T10:05:33.362264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:54.155205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294407+0000","last_change":"2024-04-25T10:05:32.340262+0000","last_active":"2024-04-25T10:05:35.294407+0000","last_peered":"2024-04-25T10:05:35.294407+0000","last_clean":"2024-04-25T10:05:35.294407+0000","last_became_active":"2024-04-25T10:05:32.339958+0000","last_became_peered":"2024-04-25T10:05:32.339958+0000","last_unstale":"2024-04-25T10:05:35.294407+0000","last_undegraded":"2024-04-25T10:05:35.294407+0000","last_fullsized":"2024-04-25T10:05:35.294407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:01.468559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294237+0000","last_change":"2024-04-25T10:05:33.355010+0000","last_active":"2024-04-25T10:05:35.294237+0000","last_peered":"2024-04-25T10:05:35.294237+0000","last_clean":"2024-04-25T10:05:35.294237+0000","last_became_active":"2024-04-25T10:05:33.354274+0000","last_became_peered":"2024-04-25T10:05:33.354274+0000","last_unstale":"2024-04-25T10:05:35.294237+0000","last_undegraded":"2024-04-25T10:05:35.294237+0000","last_fullsized":"2024-04-25T10:05:35.294237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:06.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296305+0000","last_change":"2024-04-25T10:05:32.335169+0000","last_active":"2024-04-25T10:05:35.296305+0000","last_peered":"2024-04-25T10:05:35.296305+0000","last_clean":"2024-04-25T10:05:35.296305+0000","last_became_active":"2024-04-25T10:05:32.334838+0000","last_became_peered":"2024-04-25T10:05:32.334838+0000","last_unstale":"2024-04-25T10:05:35.296305+0000","last_undegraded":"2024-04-25T10:05:35.296305+0000","last_fullsized":"2024-04-25T10:05:35.296305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:13.452653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296361+0000","last_change":"2024-04-25T10:05:33.367687+0000","last_active":"2024-04-25T10:05:35.296361+0000","last_peered":"2024-04-25T10:05:35.296361+0000","last_clean":"2024-04-25T10:05:35.296361+0000","last_became_active":"2024-04-25T10:05:33.367248+0000","last_became_peered":"2024-04-25T10:05:33.367248+0000","last_unstale":"2024-04-25T10:05:35.296361+0000","last_undegraded":"2024-04-25T10:05:35.296361+0000","last_fullsized":"2024-04-25T10:05:35.296361+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:19.529401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354430+0000","last_change":"2024-04-25T10:05:32.340050+0000","last_active":"2024-04-25T10:05:34.354430+0000","last_peered":"2024-04-25T10:05:34.354430+0000","last_clean":"2024-04-25T10:05:34.354430+0000","last_became_active":"2024-04-25T10:05:32.339705+0000","last_became_peered":"2024-04-25T10:05:32.339705+0000","last_unstale":"2024-04-25T10:05:34.354430+0000","last_undegraded":"2024-04-25T10:05:34.354430+0000","last_fullsized":"2024-04-25T10:05:34.354430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:52.003507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707019+0000","last_change":"2024-04-25T10:05:33.378310+0000","last_active":"2024-04-25T10:05:34.707019+0000","last_peered":"2024-04-25T10:05:34.707019+0000","last_clean":"2024-04-25T10:05:34.707019+0000","last_became_active":"2024-04-25T10:05:33.378027+0000","last_became_peered":"2024-04-25T10:05:33.378027+0000","last_unstale":"2024-04-25T10:05:34.707019+0000","last_undegraded":"2024-04-25T10:05:34.707019+0000","last_fullsized":"2024-04-25T10:05:34.707019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:15.232714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350821+0000","last_change":"2024-04-25T10:05:32.319461+0000","last_active":"2024-04-25T10:05:34.350821+0000","last_peered":"2024-04-25T10:05:34.350821+0000","last_clean":"2024-04-25T10:05:34.350821+0000","last_became_active":"2024-04-25T10:05:32.319129+0000","last_became_peered":"2024-04-25T10:05:32.319129+0000","last_unstale":"2024-04-25T10:05:34.350821+0000","last_undegraded":"2024-04-25T10:05:34.350821+0000","last_fullsized":"2024-04-25T10:05:34.350821+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:42.902991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707211+0000","last_change":"2024-04-25T10:05:33.364719+0000","last_active":"2024-04-25T10:05:34.707211+0000","last_peered":"2024-04-25T10:05:34.707211+0000","last_clean":"2024-04-25T10:05:34.707211+0000","last_became_active":"2024-04-25T10:05:33.364440+0000","last_became_peered":"2024-04-25T10:05:33.364440+0000","last_unstale":"2024-04-25T10:05:34.707211+0000","last_undegraded":"2024-04-25T10:05:34.707211+0000","last_fullsized":"2024-04-25T10:05:34.707211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:06.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707144+0000","last_change":"2024-04-25T10:05:32.324456+0000","last_active":"2024-04-25T10:05:34.707144+0000","last_peered":"2024-04-25T10:05:34.707144+0000","last_clean":"2024-04-25T10:05:34.707144+0000","last_became_active":"2024-04-25T10:05:32.324149+0000","last_became_peered":"2024-04-25T10:05:32.324149+0000","last_unstale":"2024-04-25T10:05:34.707144+0000","last_undegraded":"2024-04-25T10:05:34.707144+0000","last_fullsized":"2024-04-25T10:05:34.707144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.706990+0000","last_change":"2024-04-25T10:05:33.365512+0000","last_active":"2024-04-25T10:05:34.706990+0000","last_peered":"2024-04-25T10:05:34.706990+0000","last_clean":"2024-04-25T10:05:34.706990+0000","last_became_active":"2024-04-25T10:05:33.365009+0000","last_became_peered":"2024-04-25T10:05:33.365009+0000","last_unstale":"2024-04-25T10:05:34.706990+0000","last_undegraded":"2024-04-25T10:05:34.706990+0000","last_fullsized":"2024-04-25T10:05:34.706990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:09.612571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295234+0000","last_change":"2024-04-25T10:05:32.326249+0000","last_active":"2024-04-25T10:05:35.295234+0000","last_peered":"2024-04-25T10:05:35.295234+0000","last_clean":"2024-04-25T10:05:35.295234+0000","last_became_active":"2024-04-25T10:05:32.325929+0000","last_became_peered":"2024-04-25T10:05:32.325929+0000","last_unstale":"2024-04-25T10:05:35.295234+0000","last_undegraded":"2024-04-25T10:05:35.295234+0000","last_fullsized":"2024-04-25T10:05:35.295234+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:49.451888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294029+0000","last_change":"2024-04-25T10:05:33.356268+0000","last_active":"2024-04-25T10:05:35.294029+0000","last_peered":"2024-04-25T10:05:35.294029+0000","last_clean":"2024-04-25T10:05:35.294029+0000","last_became_active":"2024-04-25T10:05:33.355980+0000","last_became_peered":"2024-04-25T10:05:33.355980+0000","last_unstale":"2024-04-25T10:05:35.294029+0000","last_undegraded":"2024-04-25T10:05:35.294029+0000","last_fullsized":"2024-04-25T10:05:35.294029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:26.941321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.706921+0000","last_change":"2024-04-25T10:05:32.321450+0000","last_active":"2024-04-25T10:05:34.706921+0000","last_peered":"2024-04-25T10:05:34.706921+0000","last_clean":"2024-04-25T10:05:34.706921+0000","last_became_active":"2024-04-25T10:05:32.321104+0000","last_became_peered":"2024-04-25T10:05:32.321104+0000","last_unstale":"2024-04-25T10:05:34.706921+0000","last_undegraded":"2024-04-25T10:05:34.706921+0000","last_fullsized":"2024-04-25T10:05:34.706921+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:11.085033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351622+0000","last_change":"2024-04-25T10:05:33.361936+0000","last_active":"2024-04-25T10:05:34.351622+0000","last_peered":"2024-04-25T10:05:34.351622+0000","last_clean":"2024-04-25T10:05:34.351622+0000","last_became_active":"2024-04-25T10:05:33.355750+0000","last_became_peered":"2024-04-25T10:05:33.355750+0000","last_unstale":"2024-04-25T10:05:34.351622+0000","last_undegraded":"2024-04-25T10:05:34.351622+0000","last_fullsized":"2024-04-25T10:05:34.351622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:33.828482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352307+0000","last_change":"2024-04-25T10:05:32.342380+0000","last_active":"2024-04-25T10:05:34.352307+0000","last_peered":"2024-04-25T10:05:34.352307+0000","last_clean":"2024-04-25T10:05:34.352307+0000","last_became_active":"2024-04-25T10:05:32.340667+0000","last_became_peered":"2024-04-25T10:05:32.340667+0000","last_unstale":"2024-04-25T10:05:34.352307+0000","last_undegraded":"2024-04-25T10:05:34.352307+0000","last_fullsized":"2024-04-25T10:05:34.352307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:48.325615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294141+0000","last_change":"2024-04-25T10:05:33.355615+0000","last_active":"2024-04-25T10:05:35.294141+0000","last_peered":"2024-04-25T10:05:35.294141+0000","last_clean":"2024-04-25T10:05:35.294141+0000","last_became_active":"2024-04-25T10:05:33.355339+0000","last_became_peered":"2024-04-25T10:05:33.355339+0000","last_unstale":"2024-04-25T10:05:35.294141+0000","last_undegraded":"2024-04-25T10:05:35.294141+0000","last_fullsized":"2024-04-25T10:05:35.294141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:15.232714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351990+0000","last_change":"2024-04-25T10:05:32.338669+0000","last_active":"2024-04-25T10:05:34.351990+0000","last_peered":"2024-04-25T10:05:34.351990+0000","last_clean":"2024-04-25T10:05:34.351990+0000","last_became_active":"2024-04-25T10:05:32.338346+0000","last_became_peered":"2024-04-25T10:05:32.338346+0000","last_unstale":"2024-04-25T10:05:34.351990+0000","last_undegraded":"2024-04-25T10:05:34.351990+0000","last_fullsized":"2024-04-25T10:05:34.351990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:22.267443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294252+0000","last_change":"2024-04-25T10:05:33.369359+0000","last_active":"2024-04-25T10:05:35.294252+0000","last_peered":"2024-04-25T10:05:35.294252+0000","last_clean":"2024-04-25T10:05:35.294252+0000","last_became_active":"2024-04-25T10:05:33.365638+0000","last_became_peered":"2024-04-25T10:05:33.365638+0000","last_unstale":"2024-04-25T10:05:35.294252+0000","last_undegraded":"2024-04-25T10:05:35.294252+0000","last_fullsized":"2024-04-25T10:05:35.294252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:28.913557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296484+0000","last_change":"2024-04-25T10:05:32.333729+0000","last_active":"2024-04-25T10:05:35.296484+0000","last_peered":"2024-04-25T10:05:35.296484+0000","last_clean":"2024-04-25T10:05:35.296484+0000","last_became_active":"2024-04-25T10:05:32.333410+0000","last_became_peered":"2024-04-25T10:05:32.333410+0000","last_unstale":"2024-04-25T10:05:35.296484+0000","last_undegraded":"2024-04-25T10:05:35.296484+0000","last_fullsized":"2024-04-25T10:05:35.296484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.439969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296353+0000","last_change":"2024-04-25T10:05:33.360841+0000","last_active":"2024-04-25T10:05:35.296353+0000","last_peered":"2024-04-25T10:05:35.296353+0000","last_clean":"2024-04-25T10:05:35.296353+0000","last_became_active":"2024-04-25T10:05:33.360407+0000","last_became_peered":"2024-04-25T10:05:33.360407+0000","last_unstale":"2024-04-25T10:05:35.296353+0000","last_undegraded":"2024-04-25T10:05:35.296353+0000","last_fullsized":"2024-04-25T10:05:35.296353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:56.165349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354097+0000","last_change":"2024-04-25T10:05:32.321787+0000","last_active":"2024-04-25T10:05:34.354097+0000","last_peered":"2024-04-25T10:05:34.354097+0000","last_clean":"2024-04-25T10:05:34.354097+0000","last_became_active":"2024-04-25T10:05:32.320831+0000","last_became_peered":"2024-04-25T10:05:32.320831+0000","last_unstale":"2024-04-25T10:05:34.354097+0000","last_undegraded":"2024-04-25T10:05:34.354097+0000","last_fullsized":"2024-04-25T10:05:34.354097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:11.085033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353462+0000","last_change":"2024-04-25T10:05:33.357462+0000","last_active":"2024-04-25T10:05:34.353462+0000","last_peered":"2024-04-25T10:05:34.353462+0000","last_clean":"2024-04-25T10:05:34.353462+0000","last_became_active":"2024-04-25T10:05:33.357134+0000","last_became_peered":"2024-04-25T10:05:33.357134+0000","last_unstale":"2024-04-25T10:05:34.353462+0000","last_undegraded":"2024-04-25T10:05:34.353462+0000","last_fullsized":"2024-04-25T10:05:34.353462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:27.622180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353802+0000","last_change":"2024-04-25T10:05:32.337767+0000","last_active":"2024-04-25T10:05:34.353802+0000","last_peered":"2024-04-25T10:05:34.353802+0000","last_clean":"2024-04-25T10:05:34.353802+0000","last_became_active":"2024-04-25T10:05:32.336592+0000","last_became_peered":"2024-04-25T10:05:32.336592+0000","last_unstale":"2024-04-25T10:05:34.353802+0000","last_undegraded":"2024-04-25T10:05:34.353802+0000","last_fullsized":"2024-04-25T10:05:34.353802+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:43.513074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.292691+0000","last_change":"2024-04-25T10:05:33.360950+0000","last_active":"2024-04-25T10:05:35.292691+0000","last_peered":"2024-04-25T10:05:35.292691+0000","last_clean":"2024-04-25T10:05:35.292691+0000","last_became_active":"2024-04-25T10:05:33.356927+0000","last_became_peered":"2024-04-25T10:05:33.356927+0000","last_unstale":"2024-04-25T10:05:35.292691+0000","last_undegraded":"2024-04-25T10:05:35.292691+0000","last_fullsized":"2024-04-25T10:05:35.292691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:40.701811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707289+0000","last_change":"2024-04-25T10:05:32.340205+0000","last_active":"2024-04-25T10:05:34.707289+0000","last_peered":"2024-04-25T10:05:34.707289+0000","last_clean":"2024-04-25T10:05:34.707289+0000","last_became_active":"2024-04-25T10:05:32.339847+0000","last_became_peered":"2024-04-25T10:05:32.339847+0000","last_unstale":"2024-04-25T10:05:34.707289+0000","last_undegraded":"2024-04-25T10:05:34.707289+0000","last_fullsized":"2024-04-25T10:05:34.707289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:37.484850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350369+0000","last_change":"2024-04-25T10:05:33.344941+0000","last_active":"2024-04-25T10:05:34.350369+0000","last_peered":"2024-04-25T10:05:34.350369+0000","last_clean":"2024-04-25T10:05:34.350369+0000","last_became_active":"2024-04-25T10:05:33.344151+0000","last_became_peered":"2024-04-25T10:05:33.344151+0000","last_unstale":"2024-04-25T10:05:34.350369+0000","last_undegraded":"2024-04-25T10:05:34.350369+0000","last_fullsized":"2024-04-25T10:05:34.350369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.330430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354708+0000","last_change":"2024-04-25T10:05:32.336040+0000","last_active":"2024-04-25T10:05:34.354708+0000","last_peered":"2024-04-25T10:05:34.354708+0000","last_clean":"2024-04-25T10:05:34.354708+0000","last_became_active":"2024-04-25T10:05:32.335710+0000","last_became_peered":"2024-04-25T10:05:32.335710+0000","last_unstale":"2024-04-25T10:05:34.354708+0000","last_undegraded":"2024-04-25T10:05:34.354708+0000","last_fullsized":"2024-04-25T10:05:34.354708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:45.072130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.367194+0000","last_change":"2024-04-25T10:05:33.360142+0000","last_active":"2024-04-25T10:05:33.367194+0000","last_peered":"2024-04-25T10:05:33.367194+0000","last_clean":"2024-04-25T10:05:33.367194+0000","last_became_active":"2024-04-25T10:05:33.359852+0000","last_became_peered":"2024-04-25T10:05:33.359852+0000","last_unstale":"2024-04-25T10:05:33.367194+0000","last_undegraded":"2024-04-25T10:05:33.367194+0000","last_fullsized":"2024-04-25T10:05:33.367194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:41.993533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352306+0000","last_change":"2024-04-25T10:05:32.324691+0000","last_active":"2024-04-25T10:05:34.352306+0000","last_peered":"2024-04-25T10:05:34.352306+0000","last_clean":"2024-04-25T10:05:34.352306+0000","last_became_active":"2024-04-25T10:05:32.324414+0000","last_became_peered":"2024-04-25T10:05:32.324414+0000","last_unstale":"2024-04-25T10:05:34.352306+0000","last_undegraded":"2024-04-25T10:05:34.352306+0000","last_fullsized":"2024-04-25T10:05:34.352306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:02.679204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.369349+0000","last_change":"2024-04-25T10:05:33.363928+0000","last_active":"2024-04-25T10:05:33.369349+0000","last_peered":"2024-04-25T10:05:33.369349+0000","last_clean":"2024-04-25T10:05:33.369349+0000","last_became_active":"2024-04-25T10:05:33.362037+0000","last_became_peered":"2024-04-25T10:05:33.362037+0000","last_unstale":"2024-04-25T10:05:33.369349+0000","last_undegraded":"2024-04-25T10:05:33.369349+0000","last_fullsized":"2024-04-25T10:05:33.369349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.989447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.366919+0000","last_change":"2024-04-25T10:05:32.316530+0000","last_active":"2024-04-25T10:05:34.366919+0000","last_peered":"2024-04-25T10:05:34.366919+0000","last_clean":"2024-04-25T10:05:34.366919+0000","last_became_active":"2024-04-25T10:05:32.316249+0000","last_became_peered":"2024-04-25T10:05:32.316249+0000","last_unstale":"2024-04-25T10:05:34.366919+0000","last_undegraded":"2024-04-25T10:05:34.366919+0000","last_fullsized":"2024-04-25T10:05:34.366919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:45.411785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296317+0000","last_change":"2024-04-25T10:05:33.370056+0000","last_active":"2024-04-25T10:05:35.296317+0000","last_peered":"2024-04-25T10:05:35.296317+0000","last_clean":"2024-04-25T10:05:35.296317+0000","last_became_active":"2024-04-25T10:05:33.369775+0000","last_became_peered":"2024-04-25T10:05:33.369775+0000","last_unstale":"2024-04-25T10:05:35.296317+0000","last_undegraded":"2024-04-25T10:05:35.296317+0000","last_fullsized":"2024-04-25T10:05:35.296317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:35.143413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354257+0000","last_change":"2024-04-25T10:05:32.337930+0000","last_active":"2024-04-25T10:05:34.354257+0000","last_peered":"2024-04-25T10:05:34.354257+0000","last_clean":"2024-04-25T10:05:34.354257+0000","last_became_active":"2024-04-25T10:05:32.337585+0000","last_became_peered":"2024-04-25T10:05:32.337585+0000","last_unstale":"2024-04-25T10:05:34.354257+0000","last_undegraded":"2024-04-25T10:05:34.354257+0000","last_fullsized":"2024-04-25T10:05:34.354257+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:40.623031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296851+0000","last_change":"2024-04-25T10:05:33.359177+0000","last_active":"2024-04-25T10:05:35.296851+0000","last_peered":"2024-04-25T10:05:35.296851+0000","last_clean":"2024-04-25T10:05:35.296851+0000","last_became_active":"2024-04-25T10:05:33.358451+0000","last_became_peered":"2024-04-25T10:05:33.358451+0000","last_unstale":"2024-04-25T10:05:35.296851+0000","last_undegraded":"2024-04-25T10:05:35.296851+0000","last_fullsized":"2024-04-25T10:05:35.296851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:21.086544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354395+0000","last_change":"2024-04-25T10:05:32.330355+0000","last_active":"2024-04-25T10:05:34.354395+0000","last_peered":"2024-04-25T10:05:34.354395+0000","last_clean":"2024-04-25T10:05:34.354395+0000","last_became_active":"2024-04-25T10:05:32.330043+0000","last_became_peered":"2024-04-25T10:05:32.330043+0000","last_unstale":"2024-04-25T10:05:34.354395+0000","last_undegraded":"2024-04-25T10:05:34.354395+0000","last_fullsized":"2024-04-25T10:05:34.354395+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.349799+0000","last_change":"2024-04-25T10:05:33.361521+0000","last_active":"2024-04-25T10:05:34.349799+0000","last_peered":"2024-04-25T10:05:34.349799+0000","last_clean":"2024-04-25T10:05:34.349799+0000","last_became_active":"2024-04-25T10:05:33.361195+0000","last_became_peered":"2024-04-25T10:05:33.361195+0000","last_unstale":"2024-04-25T10:05:34.349799+0000","last_undegraded":"2024-04-25T10:05:34.349799+0000","last_fullsized":"2024-04-25T10:05:34.349799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.206514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293533+0000","last_change":"2024-04-25T10:05:32.340448+0000","last_active":"2024-04-25T10:05:35.293533+0000","last_peered":"2024-04-25T10:05:35.293533+0000","last_clean":"2024-04-25T10:05:35.293533+0000","last_became_active":"2024-04-25T10:05:32.340150+0000","last_became_peered":"2024-04-25T10:05:32.340150+0000","last_unstale":"2024-04-25T10:05:35.293533+0000","last_undegraded":"2024-04-25T10:05:35.293533+0000","last_fullsized":"2024-04-25T10:05:35.293533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.583863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293231+0000","last_change":"2024-04-25T10:05:33.368630+0000","last_active":"2024-04-25T10:05:35.293231+0000","last_peered":"2024-04-25T10:05:35.293231+0000","last_clean":"2024-04-25T10:05:35.293231+0000","last_became_active":"2024-04-25T10:05:33.363531+0000","last_became_peered":"2024-04-25T10:05:33.363531+0000","last_unstale":"2024-04-25T10:05:35.293231+0000","last_undegraded":"2024-04-25T10:05:35.293231+0000","last_fullsized":"2024-04-25T10:05:35.293231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:49.338591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293533+0000","last_change":"2024-04-25T10:05:32.340407+0000","last_active":"2024-04-25T10:05:35.293533+0000","last_peered":"2024-04-25T10:05:35.293533+0000","last_clean":"2024-04-25T10:05:35.293533+0000","last_became_active":"2024-04-25T10:05:32.340045+0000","last_became_peered":"2024-04-25T10:05:32.340045+0000","last_unstale":"2024-04-25T10:05:35.293533+0000","last_undegraded":"2024-04-25T10:05:35.293533+0000","last_fullsized":"2024-04-25T10:05:35.293533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:14.219738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354386+0000","last_change":"2024-04-25T10:05:33.382358+0000","last_active":"2024-04-25T10:05:34.354386+0000","last_peered":"2024-04-25T10:05:34.354386+0000","last_clean":"2024-04-25T10:05:34.354386+0000","last_became_active":"2024-04-25T10:05:33.382032+0000","last_became_peered":"2024-04-25T10:05:33.382032+0000","last_unstale":"2024-04-25T10:05:34.354386+0000","last_undegraded":"2024-04-25T10:05:34.354386+0000","last_fullsized":"2024-04-25T10:05:34.354386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:23.476713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353300+0000","last_change":"2024-04-25T10:05:32.333371+0000","last_active":"2024-04-25T10:05:34.353300+0000","last_peered":"2024-04-25T10:05:34.353300+0000","last_clean":"2024-04-25T10:05:34.353300+0000","last_became_active":"2024-04-25T10:05:32.333059+0000","last_became_peered":"2024-04-25T10:05:32.333059+0000","last_unstale":"2024-04-25T10:05:34.353300+0000","last_undegraded":"2024-04-25T10:05:34.353300+0000","last_fullsized":"2024-04-25T10:05:34.353300+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:09.570277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.350923+0000","last_change":"2024-04-25T10:05:33.344822+0000","last_active":"2024-04-25T10:05:33.350923+0000","last_peered":"2024-04-25T10:05:33.350923+0000","last_clean":"2024-04-25T10:05:33.350923+0000","last_became_active":"2024-04-25T10:05:33.344305+0000","last_became_peered":"2024-04-25T10:05:33.344305+0000","last_unstale":"2024-04-25T10:05:33.350923+0000","last_undegraded":"2024-04-25T10:05:33.350923+0000","last_fullsized":"2024-04-25T10:05:33.350923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:45.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354956+0000","last_change":"2024-04-25T10:05:32.338824+0000","last_active":"2024-04-25T10:05:34.354956+0000","last_peered":"2024-04-25T10:05:34.354956+0000","last_clean":"2024-04-25T10:05:34.354956+0000","last_became_active":"2024-04-25T10:05:32.337668+0000","last_became_peered":"2024-04-25T10:05:32.337668+0000","last_unstale":"2024-04-25T10:05:34.354956+0000","last_undegraded":"2024-04-25T10:05:34.354956+0000","last_fullsized":"2024-04-25T10:05:34.354956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:27.900581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707635+0000","last_change":"2024-04-25T10:05:33.372324+0000","last_active":"2024-04-25T10:05:34.707635+0000","last_peered":"2024-04-25T10:05:34.707635+0000","last_clean":"2024-04-25T10:05:34.707635+0000","last_became_active":"2024-04-25T10:05:33.368096+0000","last_became_peered":"2024-04-25T10:05:33.368096+0000","last_unstale":"2024-04-25T10:05:34.707635+0000","last_undegraded":"2024-04-25T10:05:34.707635+0000","last_fullsized":"2024-04-25T10:05:34.707635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:49.216813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354708+0000","last_change":"2024-04-25T10:05:32.333938+0000","last_active":"2024-04-25T10:05:34.354708+0000","last_peered":"2024-04-25T10:05:34.354708+0000","last_clean":"2024-04-25T10:05:34.354708+0000","last_became_active":"2024-04-25T10:05:32.333652+0000","last_became_peered":"2024-04-25T10:05:32.333652+0000","last_unstale":"2024-04-25T10:05:34.354708+0000","last_undegraded":"2024-04-25T10:05:34.354708+0000","last_fullsized":"2024-04-25T10:05:34.354708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:48.203838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707086+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:05:34.707086+0000","last_peered":"2024-04-25T10:05:34.707086+0000","last_clean":"2024-04-25T10:05:34.707086+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:05:34.707086+0000","last_undegraded":"2024-04-25T10:05:34.707086+0000","last_fullsized":"2024-04-25T10:05:34.707086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":342298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":337870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":934390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":774144,"data_stored":342458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":342354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":926008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":765952,"data_stored":342298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":757760,"data_stored":336188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:37.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.070+0000 7f1837e00640 1 Processor -- start 2024-04-25T10:05:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.070+0000 7f1837e00640 1 -- start start 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1837e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f183815ce00 0x7f183815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1837e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18380cd300 0x7f18380cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1837e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1838071d30 con 0x7f1838141da0 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1837e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380cdcd0 con 0x7f183815ce00 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1837e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1838142180 con 0x7f18380cd300 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1836a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1838141da0 0x7f183806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59176/0 (socket says 172.21.15.5:59176) 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1836a00640 1 -- 172.21.15.5:0/2396929839 learned_addr learned my addr 172.21.15.5:0/2396929839 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:38.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1836000640 1 --2- 172.21.15.5:0/2396929839 >> v2:172.21.15.5:3300/0 conn(0x7f183815ce00 0x7f183815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.072+0000 7f1837400640 1 --2- 172.21.15.5:0/2396929839 >> v2:172.21.15.80:3300/0 conn(0x7f18380cd300 0x7f18380cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.073+0000 7f1835600640 1 -- 172.21.15.5:0/2396929839 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603311328 0 0) 0x7f1838071d30 con 0x7f1838141da0 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.073+0000 7f1835600640 1 -- 172.21.15.5:0/2396929839 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f181c003520 con 0x7f1838141da0 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.073+0000 7f1836000640 1 -- 172.21.15.5:0/2396929839 >> v2:172.21.15.80:3300/0 conn(0x7f18380cd300 msgr2=0x7f18380cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.073+0000 7f1836000640 1 --2- 172.21.15.5:0/2396929839 >> v2:172.21.15.80:3300/0 conn(0x7f18380cd300 0x7f18380cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.073+0000 7f1836000640 1 -- 172.21.15.5:0/2396929839 >> v1:172.21.15.5:6789/0 conn(0x7f1838141da0 legacy=0x7f183806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.073+0000 7f1836000640 1 -- 172.21.15.5:0/2396929839 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183806d620 con 0x7f183815ce00 2024-04-25T10:05:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.074+0000 7f1836000640 1 --2- 172.21.15.5:0/2396929839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183815d280 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f183815f220 tx=0x7f182402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365f58d60205eaee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.074+0000 7f1835600640 1 -- 172.21.15.5:0/2396929839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824037070 con 0x7f183815ce00 2024-04-25T10:05:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.074+0000 7f1835600640 1 -- 172.21.15.5:0/2396929839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824032440 con 0x7f183815ce00 2024-04-25T10:05:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.074+0000 7f1835600640 1 -- 172.21.15.5:0/2396929839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824032810 con 0x7f183815ce00 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 -- 172.21.15.5:0/2396929839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 msgr2=0x7f183815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 --2- 172.21.15.5:0/2396929839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183815d280 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f183815f220 tx=0x7f182402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 -- 172.21.15.5:0/2396929839 shutdown_connections 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 --2- 172.21.15.5:0/2396929839 >> v2:172.21.15.80:3300/0 conn(0x7f18380cd300 0x7f18380cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 --2- 172.21.15.5:0/2396929839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183815d280 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 -- 172.21.15.5:0/2396929839 >> 172.21.15.5:0/2396929839 conn(0x7f1838066f20 msgr2=0x7f183806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 -- 172.21.15.5:0/2396929839 shutdown_connections 2024-04-25T10:05:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.075+0000 7f1837e00640 1 -- 172.21.15.5:0/2396929839 wait complete. 2024-04-25T10:05:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.077+0000 7f1837e00640 1 Processor -- start 2024-04-25T10:05:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.077+0000 7f1837e00640 1 -- start start 2024-04-25T10:05:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18380cd300 0x7f18380cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1838141da0 0x7f18380c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183812e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183806e810 con 0x7f183815ce00 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183806dbe0 con 0x7f1838141da0 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1838071000 con 0x7f18380cd300 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183812e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183812e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39836/0 (socket says 172.21.15.5:39836) 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.078+0000 7f1837400640 1 -- 172.21.15.5:0/20762202 learned_addr learned my addr 172.21.15.5:0/20762202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1836000640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1838141da0 0x7f18380c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1836a00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18380cd300 0x7f18380cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1837400640 1 -- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18380cd300 msgr2=0x7f18380cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1837400640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18380cd300 0x7f18380cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1837400640 1 -- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1838141da0 msgr2=0x7f18380c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1837400640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1838141da0 0x7f18380c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1837400640 1 -- 172.21.15.5:0/20762202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380c92e0 con 0x7f183815ce00 2024-04-25T10:05:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.079+0000 7f1837400640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183812e260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f183000d800 tx=0x7f183000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.080+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000b6c0 con 0x7f183815ce00 2024-04-25T10:05:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.080+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830019070 con 0x7f183815ce00 2024-04-25T10:05:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.080+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183000bac0 con 0x7f183815ce00 2024-04-25T10:05:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.080+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18380c95c0 con 0x7f183815ce00 2024-04-25T10:05:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.080+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18380c9b80 con 0x7f183815ce00 2024-04-25T10:05:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.081+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1838068f60 con 0x7f183815ce00 2024-04-25T10:05:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.082+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f183000bc60 con 0x7f183815ce00 2024-04-25T10:05:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.083+0000 7f182ea00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f181806b4d0 0x7f181806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.083+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f183008a8a0 con 0x7f183815ce00 2024-04-25T10:05:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.084+0000 7f1836a00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f181806b4d0 0x7f181806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.085+0000 7f1836a00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f181806b4d0 0x7f181806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1828005c00 tx=0x7f1828009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.087+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18300594f0 con 0x7f183815ce00 2024-04-25T10:05:38.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.281+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18380ccae0 con 0x7f181806b4d0 2024-04-25T10:05:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.299+0000 7f182ea00640 1 -- 172.21.15.5:0/20762202 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252013 (secure 0 0 0) 0x7f18380ccae0 con 0x7f181806b4d0 2024-04-25T10:05:38.308 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:38.309 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:05:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f181806b4d0 msgr2=0x7f181806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f181806b4d0 0x7f181806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1828005c00 tx=0x7f1828009040 comp rx=0 tx=0).stop 2024-04-25T10:05:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 msgr2=0x7f183812e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183812e260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f183000d800 tx=0x7f183000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 shutdown_connections 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f183815ce00 0x7f183812e260 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f181806b4d0 0x7f181806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1838141da0 0x7f18380c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 --2- 172.21.15.5:0/20762202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f18380cd300 0x7f18380cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 >> 172.21.15.5:0/20762202 conn(0x7f1838066f20 msgr2=0x7f1838073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 shutdown_connections 2024-04-25T10:05:38.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.305+0000 7f1837e00640 1 -- 172.21.15.5:0/20762202 wait complete. 2024-04-25T10:05:38.332 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-25T10:05:36.691007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244160,"kb_used_data":7172,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730560,"statfs":{"total":773094113280,"available":772844093440,"internally_reserved":0,"allocated":7344128,"data_stored":3903864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.397011"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294521+0000","last_change":"2024-04-25T10:05:33.358085+0000","last_active":"2024-04-25T10:05:35.294521+0000","last_peered":"2024-04-25T10:05:35.294521+0000","last_clean":"2024-04-25T10:05:35.294521+0000","last_became_active":"2024-04-25T10:05:33.357810+0000","last_became_peered":"2024-04-25T10:05:33.357810+0000","last_unstale":"2024-04-25T10:05:35.294521+0000","last_undegraded":"2024-04-25T10:05:35.294521+0000","last_fullsized":"2024-04-25T10:05:35.294521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:02.318344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354863+0000","last_change":"2024-04-25T10:05:33.366915+0000","last_active":"2024-04-25T10:05:34.354863+0000","last_peered":"2024-04-25T10:05:34.354863+0000","last_clean":"2024-04-25T10:05:34.354863+0000","last_became_active":"2024-04-25T10:05:33.363262+0000","last_became_peered":"2024-04-25T10:05:33.363262+0000","last_unstale":"2024-04-25T10:05:34.354863+0000","last_undegraded":"2024-04-25T10:05:34.354863+0000","last_fullsized":"2024-04-25T10:05:34.354863+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:33.828482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295997+0000","last_change":"2024-04-25T10:05:33.367255+0000","last_active":"2024-04-25T10:05:35.295997+0000","last_peered":"2024-04-25T10:05:35.295997+0000","last_clean":"2024-04-25T10:05:35.295997+0000","last_became_active":"2024-04-25T10:05:33.366898+0000","last_became_peered":"2024-04-25T10:05:33.366898+0000","last_unstale":"2024-04-25T10:05:35.295997+0000","last_undegraded":"2024-04-25T10:05:35.295997+0000","last_fullsized":"2024-04-25T10:05:35.295997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:13.132587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351309+0000","last_change":"2024-04-25T10:05:33.343798+0000","last_active":"2024-04-25T10:05:34.351309+0000","last_peered":"2024-04-25T10:05:34.351309+0000","last_clean":"2024-04-25T10:05:34.351309+0000","last_became_active":"2024-04-25T10:05:33.343340+0000","last_became_peered":"2024-04-25T10:05:33.343340+0000","last_unstale":"2024-04-25T10:05:34.351309+0000","last_undegraded":"2024-04-25T10:05:34.351309+0000","last_fullsized":"2024-04-25T10:05:34.351309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:27.622180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295670+0000","last_change":"2024-04-25T10:05:33.369441+0000","last_active":"2024-04-25T10:05:35.295670+0000","last_peered":"2024-04-25T10:05:35.295670+0000","last_clean":"2024-04-25T10:05:35.295670+0000","last_became_active":"2024-04-25T10:05:33.368432+0000","last_became_peered":"2024-04-25T10:05:33.368432+0000","last_unstale":"2024-04-25T10:05:35.295670+0000","last_undegraded":"2024-04-25T10:05:35.295670+0000","last_fullsized":"2024-04-25T10:05:35.295670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:34.919366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353989+0000","last_change":"2024-04-25T10:05:33.364065+0000","last_active":"2024-04-25T10:05:34.353989+0000","last_peered":"2024-04-25T10:05:34.353989+0000","last_clean":"2024-04-25T10:05:34.353989+0000","last_became_active":"2024-04-25T10:05:33.363331+0000","last_became_peered":"2024-04-25T10:05:33.363331+0000","last_unstale":"2024-04-25T10:05:34.353989+0000","last_undegraded":"2024-04-25T10:05:34.353989+0000","last_fullsized":"2024-04-25T10:05:34.353989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293731+0000","last_change":"2024-04-25T10:05:33.360268+0000","last_active":"2024-04-25T10:05:35.293731+0000","last_peered":"2024-04-25T10:05:35.293731+0000","last_clean":"2024-04-25T10:05:35.293731+0000","last_became_active":"2024-04-25T10:05:33.359954+0000","last_became_peered":"2024-04-25T10:05:33.359954+0000","last_unstale":"2024-04-25T10:05:35.293731+0000","last_undegraded":"2024-04-25T10:05:35.293731+0000","last_fullsized":"2024-04-25T10:05:35.293731+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:41.636006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294601+0000","last_change":"2024-04-25T10:05:33.352701+0000","last_active":"2024-04-25T10:05:35.294601+0000","last_peered":"2024-04-25T10:05:35.294601+0000","last_clean":"2024-04-25T10:05:35.294601+0000","last_became_active":"2024-04-25T10:05:33.352410+0000","last_became_peered":"2024-04-25T10:05:33.352410+0000","last_unstale":"2024-04-25T10:05:35.294601+0000","last_undegraded":"2024-04-25T10:05:35.294601+0000","last_fullsized":"2024-04-25T10:05:35.294601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:24.695367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350072+0000","last_change":"2024-04-25T10:05:33.382080+0000","last_active":"2024-04-25T10:05:34.350072+0000","last_peered":"2024-04-25T10:05:34.350072+0000","last_clean":"2024-04-25T10:05:34.350072+0000","last_became_active":"2024-04-25T10:05:33.381729+0000","last_became_peered":"2024-04-25T10:05:33.381729+0000","last_unstale":"2024-04-25T10:05:34.350072+0000","last_undegraded":"2024-04-25T10:05:34.350072+0000","last_fullsized":"2024-04-25T10:05:34.350072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:16.677491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707751+0000","last_change":"2024-04-25T10:05:33.364383+0000","last_active":"2024-04-25T10:05:34.707751+0000","last_peered":"2024-04-25T10:05:34.707751+0000","last_clean":"2024-04-25T10:05:34.707751+0000","last_became_active":"2024-04-25T10:05:33.364062+0000","last_became_peered":"2024-04-25T10:05:33.364062+0000","last_unstale":"2024-04-25T10:05:34.707751+0000","last_undegraded":"2024-04-25T10:05:34.707751+0000","last_fullsized":"2024-04-25T10:05:34.707751+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:46.085106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351257+0000","last_change":"2024-04-25T10:05:33.365949+0000","last_active":"2024-04-25T10:05:34.351257+0000","last_peered":"2024-04-25T10:05:34.351257+0000","last_clean":"2024-04-25T10:05:34.351257+0000","last_became_active":"2024-04-25T10:05:33.362338+0000","last_became_peered":"2024-04-25T10:05:33.362338+0000","last_unstale":"2024-04-25T10:05:34.351257+0000","last_undegraded":"2024-04-25T10:05:34.351257+0000","last_fullsized":"2024-04-25T10:05:34.351257+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:12.290132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351362+0000","last_change":"2024-04-25T10:05:33.356482+0000","last_active":"2024-04-25T10:05:34.351362+0000","last_peered":"2024-04-25T10:05:34.351362+0000","last_clean":"2024-04-25T10:05:34.351362+0000","last_became_active":"2024-04-25T10:05:33.356198+0000","last_became_peered":"2024-04-25T10:05:33.356198+0000","last_unstale":"2024-04-25T10:05:34.351362+0000","last_undegraded":"2024-04-25T10:05:34.351362+0000","last_fullsized":"2024-04-25T10:05:34.351362+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:42.124848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293199+0000","last_change":"2024-04-25T10:05:33.354098+0000","last_active":"2024-04-25T10:05:35.293199+0000","last_peered":"2024-04-25T10:05:35.293199+0000","last_clean":"2024-04-25T10:05:35.293199+0000","last_became_active":"2024-04-25T10:05:33.353582+0000","last_became_peered":"2024-04-25T10:05:33.353582+0000","last_unstale":"2024-04-25T10:05:35.293199+0000","last_undegraded":"2024-04-25T10:05:35.293199+0000","last_fullsized":"2024-04-25T10:05:35.293199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:46.424760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.292998+0000","last_change":"2024-04-25T10:05:33.366197+0000","last_active":"2024-04-25T10:05:35.292998+0000","last_peered":"2024-04-25T10:05:35.292998+0000","last_clean":"2024-04-25T10:05:35.292998+0000","last_became_active":"2024-04-25T10:05:33.363635+0000","last_became_peered":"2024-04-25T10:05:33.363635+0000","last_unstale":"2024-04-25T10:05:35.292998+0000","last_undegraded":"2024-04-25T10:05:35.292998+0000","last_fullsized":"2024-04-25T10:05:35.292998+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:03.692180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707284+0000","last_change":"2024-04-25T10:05:33.364983+0000","last_active":"2024-04-25T10:05:34.707284+0000","last_peered":"2024-04-25T10:05:34.707284+0000","last_clean":"2024-04-25T10:05:34.707284+0000","last_became_active":"2024-04-25T10:05:33.364682+0000","last_became_peered":"2024-04-25T10:05:33.364682+0000","last_unstale":"2024-04-25T10:05:34.707284+0000","last_undegraded":"2024-04-25T10:05:34.707284+0000","last_fullsized":"2024-04-25T10:05:34.707284+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:29.479275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352611+0000","last_change":"2024-04-25T10:05:33.359243+0000","last_active":"2024-04-25T10:05:34.352611+0000","last_peered":"2024-04-25T10:05:34.352611+0000","last_clean":"2024-04-25T10:05:34.352611+0000","last_became_active":"2024-04-25T10:05:33.358941+0000","last_became_peered":"2024-04-25T10:05:33.358941+0000","last_unstale":"2024-04-25T10:05:34.352611+0000","last_undegraded":"2024-04-25T10:05:34.352611+0000","last_fullsized":"2024-04-25T10:05:34.352611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:23.280419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.362256+0000","last_change":"2024-04-25T10:05:33.358218+0000","last_active":"2024-04-25T10:05:33.362256+0000","last_peered":"2024-04-25T10:05:33.362256+0000","last_clean":"2024-04-25T10:05:33.362256+0000","last_became_active":"2024-04-25T10:05:33.356777+0000","last_became_peered":"2024-04-25T10:05:33.356777+0000","last_unstale":"2024-04-25T10:05:33.362256+0000","last_undegraded":"2024-04-25T10:05:33.362256+0000","last_fullsized":"2024-04-25T10:05:33.362256+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:23.837823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351863+0000","last_change":"2024-04-25T10:05:33.368431+0000","last_active":"2024-04-25T10:05:34.351863+0000","last_peered":"2024-04-25T10:05:34.351863+0000","last_clean":"2024-04-25T10:05:34.351863+0000","last_became_active":"2024-04-25T10:05:33.368144+0000","last_became_peered":"2024-04-25T10:05:33.368144+0000","last_unstale":"2024-04-25T10:05:34.351863+0000","last_undegraded":"2024-04-25T10:05:34.351863+0000","last_fullsized":"2024-04-25T10:05:34.351863+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:50.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351494+0000","last_change":"2024-04-25T10:05:33.362815+0000","last_active":"2024-04-25T10:05:34.351494+0000","last_peered":"2024-04-25T10:05:34.351494+0000","last_clean":"2024-04-25T10:05:34.351494+0000","last_became_active":"2024-04-25T10:05:33.362289+0000","last_became_peered":"2024-04-25T10:05:33.362289+0000","last_unstale":"2024-04-25T10:05:34.351494+0000","last_undegraded":"2024-04-25T10:05:34.351494+0000","last_fullsized":"2024-04-25T10:05:34.351494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:23.476713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.354566+0000","last_change":"2024-04-25T10:05:33.348298+0000","last_active":"2024-04-25T10:05:33.354566+0000","last_peered":"2024-04-25T10:05:33.354566+0000","last_clean":"2024-04-25T10:05:33.354566+0000","last_became_active":"2024-04-25T10:05:33.347350+0000","last_became_peered":"2024-04-25T10:05:33.347350+0000","last_unstale":"2024-04-25T10:05:33.354566+0000","last_undegraded":"2024-04-25T10:05:33.354566+0000","last_fullsized":"2024-04-25T10:05:33.354566+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:03.915686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293536+0000","last_change":"2024-04-25T10:05:33.360549+0000","last_active":"2024-04-25T10:05:35.293536+0000","last_peered":"2024-04-25T10:05:35.293536+0000","last_clean":"2024-04-25T10:05:35.293536+0000","last_became_active":"2024-04-25T10:05:33.360241+0000","last_became_peered":"2024-04-25T10:05:33.360241+0000","last_unstale":"2024-04-25T10:05:35.293536+0000","last_undegraded":"2024-04-25T10:05:35.293536+0000","last_fullsized":"2024-04-25T10:05:35.293536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:12.248661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354929+0000","last_change":"2024-04-25T10:05:33.363825+0000","last_active":"2024-04-25T10:05:34.354929+0000","last_peered":"2024-04-25T10:05:34.354929+0000","last_clean":"2024-04-25T10:05:34.354929+0000","last_became_active":"2024-04-25T10:05:33.362895+0000","last_became_peered":"2024-04-25T10:05:33.362895+0000","last_unstale":"2024-04-25T10:05:34.354929+0000","last_undegraded":"2024-04-25T10:05:34.354929+0000","last_fullsized":"2024-04-25T10:05:34.354929+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:50.464863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352995+0000","last_change":"2024-04-25T10:05:33.366433+0000","last_active":"2024-04-25T10:05:34.352995+0000","last_peered":"2024-04-25T10:05:34.352995+0000","last_clean":"2024-04-25T10:05:34.352995+0000","last_became_active":"2024-04-25T10:05:33.366139+0000","last_became_peered":"2024-04-25T10:05:33.366139+0000","last_unstale":"2024-04-25T10:05:34.352995+0000","last_undegraded":"2024-04-25T10:05:34.352995+0000","last_fullsized":"2024-04-25T10:05:34.352995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.915967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295243+0000","last_change":"2024-04-25T10:05:33.375331+0000","last_active":"2024-04-25T10:05:35.295243+0000","last_peered":"2024-04-25T10:05:35.295243+0000","last_clean":"2024-04-25T10:05:35.295243+0000","last_became_active":"2024-04-25T10:05:33.372614+0000","last_became_peered":"2024-04-25T10:05:33.372614+0000","last_unstale":"2024-04-25T10:05:35.295243+0000","last_undegraded":"2024-04-25T10:05:35.295243+0000","last_fullsized":"2024-04-25T10:05:35.295243+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:52.407607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294197+0000","last_change":"2024-04-25T10:05:33.358404+0000","last_active":"2024-04-25T10:05:35.294197+0000","last_peered":"2024-04-25T10:05:35.294197+0000","last_clean":"2024-04-25T10:05:35.294197+0000","last_became_active":"2024-04-25T10:05:33.358078+0000","last_became_peered":"2024-04-25T10:05:33.358078+0000","last_unstale":"2024-04-25T10:05:35.294197+0000","last_undegraded":"2024-04-25T10:05:35.294197+0000","last_fullsized":"2024-04-25T10:05:35.294197+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:29.479275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296245+0000","last_change":"2024-04-25T10:05:33.359621+0000","last_active":"2024-04-25T10:05:35.296245+0000","last_peered":"2024-04-25T10:05:35.296245+0000","last_clean":"2024-04-25T10:05:35.296245+0000","last_became_active":"2024-04-25T10:05:33.359275+0000","last_became_peered":"2024-04-25T10:05:33.359275+0000","last_unstale":"2024-04-25T10:05:35.296245+0000","last_undegraded":"2024-04-25T10:05:35.296245+0000","last_fullsized":"2024-04-25T10:05:35.296245+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:01.515552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293698+0000","last_change":"2024-04-25T10:05:33.362012+0000","last_active":"2024-04-25T10:05:35.293698+0000","last_peered":"2024-04-25T10:05:35.293698+0000","last_clean":"2024-04-25T10:05:35.293698+0000","last_became_active":"2024-04-25T10:05:33.361709+0000","last_became_peered":"2024-04-25T10:05:33.361709+0000","last_unstale":"2024-04-25T10:05:35.293698+0000","last_undegraded":"2024-04-25T10:05:35.293698+0000","last_fullsized":"2024-04-25T10:05:35.293698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:09.612571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294131+0000","last_change":"2024-04-25T10:05:33.374631+0000","last_active":"2024-04-25T10:05:35.294131+0000","last_peered":"2024-04-25T10:05:35.294131+0000","last_clean":"2024-04-25T10:05:35.294131+0000","last_became_active":"2024-04-25T10:05:33.372923+0000","last_became_peered":"2024-04-25T10:05:33.372923+0000","last_unstale":"2024-04-25T10:05:35.294131+0000","last_undegraded":"2024-04-25T10:05:35.294131+0000","last_fullsized":"2024-04-25T10:05:35.294131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:16.677491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350665+0000","last_change":"2024-04-25T10:05:33.375855+0000","last_active":"2024-04-25T10:05:34.350665+0000","last_peered":"2024-04-25T10:05:34.350665+0000","last_clean":"2024-04-25T10:05:34.350665+0000","last_became_active":"2024-04-25T10:05:33.374097+0000","last_became_peered":"2024-04-25T10:05:33.374097+0000","last_unstale":"2024-04-25T10:05:34.350665+0000","last_undegraded":"2024-04-25T10:05:34.350665+0000","last_fullsized":"2024-04-25T10:05:34.350665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:02.318344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350001+0000","last_change":"2024-04-25T10:05:33.382091+0000","last_active":"2024-04-25T10:05:34.350001+0000","last_peered":"2024-04-25T10:05:34.350001+0000","last_clean":"2024-04-25T10:05:34.350001+0000","last_became_active":"2024-04-25T10:05:33.379536+0000","last_became_peered":"2024-04-25T10:05:33.379536+0000","last_unstale":"2024-04-25T10:05:34.350001+0000","last_undegraded":"2024-04-25T10:05:34.350001+0000","last_fullsized":"2024-04-25T10:05:34.350001+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:26.941321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295334+0000","last_change":"2024-04-25T10:05:33.367118+0000","last_active":"2024-04-25T10:05:35.295334+0000","last_peered":"2024-04-25T10:05:35.295334+0000","last_clean":"2024-04-25T10:05:35.295334+0000","last_became_active":"2024-04-25T10:05:33.366791+0000","last_became_peered":"2024-04-25T10:05:33.366791+0000","last_unstale":"2024-04-25T10:05:35.295334+0000","last_undegraded":"2024-04-25T10:05:35.295334+0000","last_fullsized":"2024-04-25T10:05:35.295334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:52.171069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293827+0000","last_change":"2024-04-25T10:05:33.360858+0000","last_active":"2024-04-25T10:05:35.293827+0000","last_peered":"2024-04-25T10:05:35.293827+0000","last_clean":"2024-04-25T10:05:35.293827+0000","last_became_active":"2024-04-25T10:05:33.360540+0000","last_became_peered":"2024-04-25T10:05:33.360540+0000","last_unstale":"2024-04-25T10:05:35.293827+0000","last_undegraded":"2024-04-25T10:05:35.293827+0000","last_fullsized":"2024-04-25T10:05:35.293827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:32.515551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352513+0000","last_change":"2024-04-25T10:05:33.362644+0000","last_active":"2024-04-25T10:05:34.352513+0000","last_peered":"2024-04-25T10:05:34.352513+0000","last_clean":"2024-04-25T10:05:34.352513+0000","last_became_active":"2024-04-25T10:05:33.362276+0000","last_became_peered":"2024-04-25T10:05:33.362276+0000","last_unstale":"2024-04-25T10:05:34.352513+0000","last_undegraded":"2024-04-25T10:05:34.352513+0000","last_fullsized":"2024-04-25T10:05:34.352513+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:45.452945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294512+0000","last_change":"2024-04-25T10:05:32.342003+0000","last_active":"2024-04-25T10:05:35.294512+0000","last_peered":"2024-04-25T10:05:35.294512+0000","last_clean":"2024-04-25T10:05:35.294512+0000","last_became_active":"2024-04-25T10:05:32.341742+0000","last_became_peered":"2024-04-25T10:05:32.341742+0000","last_unstale":"2024-04-25T10:05:35.294512+0000","last_undegraded":"2024-04-25T10:05:35.294512+0000","last_fullsized":"2024-04-25T10:05:35.294512+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:28.466299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296325+0000","last_change":"2024-04-25T10:05:33.356123+0000","last_active":"2024-04-25T10:05:35.296325+0000","last_peered":"2024-04-25T10:05:35.296325+0000","last_clean":"2024-04-25T10:05:35.296325+0000","last_became_active":"2024-04-25T10:05:33.355811+0000","last_became_peered":"2024-04-25T10:05:33.355811+0000","last_unstale":"2024-04-25T10:05:35.296325+0000","last_undegraded":"2024-04-25T10:05:35.296325+0000","last_fullsized":"2024-04-25T10:05:35.296325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:04.047617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294276+0000","last_change":"2024-04-25T10:05:32.330756+0000","last_active":"2024-04-25T10:05:35.294276+0000","last_peered":"2024-04-25T10:05:35.294276+0000","last_clean":"2024-04-25T10:05:35.294276+0000","last_became_active":"2024-04-25T10:05:32.330349+0000","last_became_peered":"2024-04-25T10:05:32.330349+0000","last_unstale":"2024-04-25T10:05:35.294276+0000","last_undegraded":"2024-04-25T10:05:35.294276+0000","last_fullsized":"2024-04-25T10:05:35.294276+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:11.235686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.355114+0000","last_change":"2024-04-25T10:05:33.363634+0000","last_active":"2024-04-25T10:05:34.355114+0000","last_peered":"2024-04-25T10:05:34.355114+0000","last_clean":"2024-04-25T10:05:34.355114+0000","last_became_active":"2024-04-25T10:05:33.363311+0000","last_became_peered":"2024-04-25T10:05:33.363311+0000","last_unstale":"2024-04-25T10:05:34.355114+0000","last_undegraded":"2024-04-25T10:05:34.355114+0000","last_fullsized":"2024-04-25T10:05:34.355114+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:14.465628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294734+0000","last_change":"2024-04-25T10:05:32.335473+0000","last_active":"2024-04-25T10:05:35.294734+0000","last_peered":"2024-04-25T10:05:35.294734+0000","last_clean":"2024-04-25T10:05:35.294734+0000","last_became_active":"2024-04-25T10:05:32.335195+0000","last_became_peered":"2024-04-25T10:05:32.335195+0000","last_unstale":"2024-04-25T10:05:35.294734+0000","last_undegraded":"2024-04-25T10:05:35.294734+0000","last_fullsized":"2024-04-25T10:05:35.294734+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:05.273043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352124+0000","last_change":"2024-04-25T10:05:33.342494+0000","last_active":"2024-04-25T10:05:34.352124+0000","last_peered":"2024-04-25T10:05:34.352124+0000","last_clean":"2024-04-25T10:05:34.352124+0000","last_became_active":"2024-04-25T10:05:33.342075+0000","last_became_peered":"2024-04-25T10:05:33.342075+0000","last_unstale":"2024-04-25T10:05:34.352124+0000","last_undegraded":"2024-04-25T10:05:34.352124+0000","last_fullsized":"2024-04-25T10:05:34.352124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:14.465628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354107+0000","last_change":"2024-04-25T10:05:32.340386+0000","last_active":"2024-04-25T10:05:34.354107+0000","last_peered":"2024-04-25T10:05:34.354107+0000","last_clean":"2024-04-25T10:05:34.354107+0000","last_became_active":"2024-04-25T10:05:32.340089+0000","last_became_peered":"2024-04-25T10:05:32.340089+0000","last_unstale":"2024-04-25T10:05:34.354107+0000","last_undegraded":"2024-04-25T10:05:34.354107+0000","last_fullsized":"2024-04-25T10:05:34.354107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:41.502547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.348565+0000","last_change":"2024-04-25T10:05:33.341527+0000","last_active":"2024-04-25T10:05:33.348565+0000","last_peered":"2024-04-25T10:05:33.348565+0000","last_clean":"2024-04-25T10:05:33.348565+0000","last_became_active":"2024-04-25T10:05:33.340718+0000","last_became_peered":"2024-04-25T10:05:33.340718+0000","last_unstale":"2024-04-25T10:05:33.348565+0000","last_undegraded":"2024-04-25T10:05:33.348565+0000","last_fullsized":"2024-04-25T10:05:33.348565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:21.584362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.372956+0000","last_change":"2024-04-25T10:05:32.335077+0000","last_active":"2024-04-25T10:05:33.372956+0000","last_peered":"2024-04-25T10:05:33.372956+0000","last_clean":"2024-04-25T10:05:33.372956+0000","last_became_active":"2024-04-25T10:05:32.334801+0000","last_became_peered":"2024-04-25T10:05:32.334801+0000","last_unstale":"2024-04-25T10:05:33.372956+0000","last_undegraded":"2024-04-25T10:05:33.372956+0000","last_fullsized":"2024-04-25T10:05:33.372956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.404475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293826+0000","last_change":"2024-04-25T10:05:33.354848+0000","last_active":"2024-04-25T10:05:35.293826+0000","last_peered":"2024-04-25T10:05:35.293826+0000","last_clean":"2024-04-25T10:05:35.293826+0000","last_became_active":"2024-04-25T10:05:33.354478+0000","last_became_peered":"2024-04-25T10:05:33.354478+0000","last_unstale":"2024-04-25T10:05:35.293826+0000","last_undegraded":"2024-04-25T10:05:35.293826+0000","last_fullsized":"2024-04-25T10:05:35.293826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.206514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.355038+0000","last_change":"2024-04-25T10:05:32.343554+0000","last_active":"2024-04-25T10:05:34.355038+0000","last_peered":"2024-04-25T10:05:34.355038+0000","last_clean":"2024-04-25T10:05:34.355038+0000","last_became_active":"2024-04-25T10:05:32.342963+0000","last_became_peered":"2024-04-25T10:05:32.342963+0000","last_unstale":"2024-04-25T10:05:34.355038+0000","last_undegraded":"2024-04-25T10:05:34.355038+0000","last_fullsized":"2024-04-25T10:05:34.355038+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:28.200654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707448+0000","last_change":"2024-04-25T10:05:33.377799+0000","last_active":"2024-04-25T10:05:34.707448+0000","last_peered":"2024-04-25T10:05:34.707448+0000","last_clean":"2024-04-25T10:05:34.707448+0000","last_became_active":"2024-04-25T10:05:33.377467+0000","last_became_peered":"2024-04-25T10:05:33.377467+0000","last_unstale":"2024-04-25T10:05:34.707448+0000","last_undegraded":"2024-04-25T10:05:34.707448+0000","last_fullsized":"2024-04-25T10:05:34.707448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:28.913557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293972+0000","last_change":"2024-04-25T10:05:32.330296+0000","last_active":"2024-04-25T10:05:35.293972+0000","last_peered":"2024-04-25T10:05:35.293972+0000","last_clean":"2024-04-25T10:05:35.293972+0000","last_became_active":"2024-04-25T10:05:32.329971+0000","last_became_peered":"2024-04-25T10:05:32.329971+0000","last_unstale":"2024-04-25T10:05:35.293972+0000","last_undegraded":"2024-04-25T10:05:35.293972+0000","last_fullsized":"2024-04-25T10:05:35.293972+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:08.599596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.356251+0000","last_change":"2024-04-25T10:05:33.349092+0000","last_active":"2024-04-25T10:05:33.356251+0000","last_peered":"2024-04-25T10:05:33.356251+0000","last_clean":"2024-04-25T10:05:33.356251+0000","last_became_active":"2024-04-25T10:05:33.348812+0000","last_became_peered":"2024-04-25T10:05:33.348812+0000","last_unstale":"2024-04-25T10:05:33.356251+0000","last_undegraded":"2024-04-25T10:05:33.356251+0000","last_fullsized":"2024-04-25T10:05:33.356251+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:17.600078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.349785+0000","last_change":"2024-04-25T10:05:32.324120+0000","last_active":"2024-04-25T10:05:34.349785+0000","last_peered":"2024-04-25T10:05:34.349785+0000","last_clean":"2024-04-25T10:05:34.349785+0000","last_became_active":"2024-04-25T10:05:32.323551+0000","last_became_peered":"2024-04-25T10:05:32.323551+0000","last_unstale":"2024-04-25T10:05:34.349785+0000","last_undegraded":"2024-04-25T10:05:34.349785+0000","last_fullsized":"2024-04-25T10:05:34.349785+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:39.618543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294328+0000","last_change":"2024-04-25T10:05:33.362210+0000","last_active":"2024-04-25T10:05:35.294328+0000","last_peered":"2024-04-25T10:05:35.294328+0000","last_clean":"2024-04-25T10:05:35.294328+0000","last_became_active":"2024-04-25T10:05:33.361755+0000","last_became_peered":"2024-04-25T10:05:33.361755+0000","last_unstale":"2024-04-25T10:05:35.294328+0000","last_undegraded":"2024-04-25T10:05:35.294328+0000","last_fullsized":"2024-04-25T10:05:35.294328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.330430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352121+0000","last_change":"2024-04-25T10:05:32.341935+0000","last_active":"2024-04-25T10:05:34.352121+0000","last_peered":"2024-04-25T10:05:34.352121+0000","last_clean":"2024-04-25T10:05:34.352121+0000","last_became_active":"2024-04-25T10:05:32.340956+0000","last_became_peered":"2024-04-25T10:05:32.340956+0000","last_unstale":"2024-04-25T10:05:34.352121+0000","last_undegraded":"2024-04-25T10:05:34.352121+0000","last_fullsized":"2024-04-25T10:05:34.352121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:39.688835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.362264+0000","last_change":"2024-04-25T10:05:33.357527+0000","last_active":"2024-04-25T10:05:33.362264+0000","last_peered":"2024-04-25T10:05:33.362264+0000","last_clean":"2024-04-25T10:05:33.362264+0000","last_became_active":"2024-04-25T10:05:33.357167+0000","last_became_peered":"2024-04-25T10:05:33.357167+0000","last_unstale":"2024-04-25T10:05:33.362264+0000","last_undegraded":"2024-04-25T10:05:33.362264+0000","last_fullsized":"2024-04-25T10:05:33.362264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:54.155205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294407+0000","last_change":"2024-04-25T10:05:32.340262+0000","last_active":"2024-04-25T10:05:35.294407+0000","last_peered":"2024-04-25T10:05:35.294407+0000","last_clean":"2024-04-25T10:05:35.294407+0000","last_became_active":"2024-04-25T10:05:32.339958+0000","last_became_peered":"2024-04-25T10:05:32.339958+0000","last_unstale":"2024-04-25T10:05:35.294407+0000","last_undegraded":"2024-04-25T10:05:35.294407+0000","last_fullsized":"2024-04-25T10:05:35.294407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:01.468559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294237+0000","last_change":"2024-04-25T10:05:33.355010+0000","last_active":"2024-04-25T10:05:35.294237+0000","last_peered":"2024-04-25T10:05:35.294237+0000","last_clean":"2024-04-25T10:05:35.294237+0000","last_became_active":"2024-04-25T10:05:33.354274+0000","last_became_peered":"2024-04-25T10:05:33.354274+0000","last_unstale":"2024-04-25T10:05:35.294237+0000","last_undegraded":"2024-04-25T10:05:35.294237+0000","last_fullsized":"2024-04-25T10:05:35.294237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:06.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296305+0000","last_change":"2024-04-25T10:05:32.335169+0000","last_active":"2024-04-25T10:05:35.296305+0000","last_peered":"2024-04-25T10:05:35.296305+0000","last_clean":"2024-04-25T10:05:35.296305+0000","last_became_active":"2024-04-25T10:05:32.334838+0000","last_became_peered":"2024-04-25T10:05:32.334838+0000","last_unstale":"2024-04-25T10:05:35.296305+0000","last_undegraded":"2024-04-25T10:05:35.296305+0000","last_fullsized":"2024-04-25T10:05:35.296305+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:13.452653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296361+0000","last_change":"2024-04-25T10:05:33.367687+0000","last_active":"2024-04-25T10:05:35.296361+0000","last_peered":"2024-04-25T10:05:35.296361+0000","last_clean":"2024-04-25T10:05:35.296361+0000","last_became_active":"2024-04-25T10:05:33.367248+0000","last_became_peered":"2024-04-25T10:05:33.367248+0000","last_unstale":"2024-04-25T10:05:35.296361+0000","last_undegraded":"2024-04-25T10:05:35.296361+0000","last_fullsized":"2024-04-25T10:05:35.296361+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:19.529401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":20,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354430+0000","last_change":"2024-04-25T10:05:32.340050+0000","last_active":"2024-04-25T10:05:34.354430+0000","last_peered":"2024-04-25T10:05:34.354430+0000","last_clean":"2024-04-25T10:05:34.354430+0000","last_became_active":"2024-04-25T10:05:32.339705+0000","last_became_peered":"2024-04-25T10:05:32.339705+0000","last_unstale":"2024-04-25T10:05:34.354430+0000","last_undegraded":"2024-04-25T10:05:34.354430+0000","last_fullsized":"2024-04-25T10:05:34.354430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:52.003507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707019+0000","last_change":"2024-04-25T10:05:33.378310+0000","last_active":"2024-04-25T10:05:34.707019+0000","last_peered":"2024-04-25T10:05:34.707019+0000","last_clean":"2024-04-25T10:05:34.707019+0000","last_became_active":"2024-04-25T10:05:33.378027+0000","last_became_peered":"2024-04-25T10:05:33.378027+0000","last_unstale":"2024-04-25T10:05:34.707019+0000","last_undegraded":"2024-04-25T10:05:34.707019+0000","last_fullsized":"2024-04-25T10:05:34.707019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:15.232714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350821+0000","last_change":"2024-04-25T10:05:32.319461+0000","last_active":"2024-04-25T10:05:34.350821+0000","last_peered":"2024-04-25T10:05:34.350821+0000","last_clean":"2024-04-25T10:05:34.350821+0000","last_became_active":"2024-04-25T10:05:32.319129+0000","last_became_peered":"2024-04-25T10:05:32.319129+0000","last_unstale":"2024-04-25T10:05:34.350821+0000","last_undegraded":"2024-04-25T10:05:34.350821+0000","last_fullsized":"2024-04-25T10:05:34.350821+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:42.902991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707211+0000","last_change":"2024-04-25T10:05:33.364719+0000","last_active":"2024-04-25T10:05:34.707211+0000","last_peered":"2024-04-25T10:05:34.707211+0000","last_clean":"2024-04-25T10:05:34.707211+0000","last_became_active":"2024-04-25T10:05:33.364440+0000","last_became_peered":"2024-04-25T10:05:33.364440+0000","last_unstale":"2024-04-25T10:05:34.707211+0000","last_undegraded":"2024-04-25T10:05:34.707211+0000","last_fullsized":"2024-04-25T10:05:34.707211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:06.286018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707144+0000","last_change":"2024-04-25T10:05:32.324456+0000","last_active":"2024-04-25T10:05:34.707144+0000","last_peered":"2024-04-25T10:05:34.707144+0000","last_clean":"2024-04-25T10:05:34.707144+0000","last_became_active":"2024-04-25T10:05:32.324149+0000","last_became_peered":"2024-04-25T10:05:32.324149+0000","last_unstale":"2024-04-25T10:05:34.707144+0000","last_undegraded":"2024-04-25T10:05:34.707144+0000","last_fullsized":"2024-04-25T10:05:34.707144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.706990+0000","last_change":"2024-04-25T10:05:33.365512+0000","last_active":"2024-04-25T10:05:34.706990+0000","last_peered":"2024-04-25T10:05:34.706990+0000","last_clean":"2024-04-25T10:05:34.706990+0000","last_became_active":"2024-04-25T10:05:33.365009+0000","last_became_peered":"2024-04-25T10:05:33.365009+0000","last_unstale":"2024-04-25T10:05:34.706990+0000","last_undegraded":"2024-04-25T10:05:34.706990+0000","last_fullsized":"2024-04-25T10:05:34.706990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:09.612571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.295234+0000","last_change":"2024-04-25T10:05:32.326249+0000","last_active":"2024-04-25T10:05:35.295234+0000","last_peered":"2024-04-25T10:05:35.295234+0000","last_clean":"2024-04-25T10:05:35.295234+0000","last_became_active":"2024-04-25T10:05:32.325929+0000","last_became_peered":"2024-04-25T10:05:32.325929+0000","last_unstale":"2024-04-25T10:05:35.295234+0000","last_undegraded":"2024-04-25T10:05:35.295234+0000","last_fullsized":"2024-04-25T10:05:35.295234+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:49.451888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294029+0000","last_change":"2024-04-25T10:05:33.356268+0000","last_active":"2024-04-25T10:05:35.294029+0000","last_peered":"2024-04-25T10:05:35.294029+0000","last_clean":"2024-04-25T10:05:35.294029+0000","last_became_active":"2024-04-25T10:05:33.355980+0000","last_became_peered":"2024-04-25T10:05:33.355980+0000","last_unstale":"2024-04-25T10:05:35.294029+0000","last_undegraded":"2024-04-25T10:05:35.294029+0000","last_fullsized":"2024-04-25T10:05:35.294029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:26.941321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":22,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.706921+0000","last_change":"2024-04-25T10:05:32.321450+0000","last_active":"2024-04-25T10:05:34.706921+0000","last_peered":"2024-04-25T10:05:34.706921+0000","last_clean":"2024-04-25T10:05:34.706921+0000","last_became_active":"2024-04-25T10:05:32.321104+0000","last_became_peered":"2024-04-25T10:05:32.321104+0000","last_unstale":"2024-04-25T10:05:34.706921+0000","last_undegraded":"2024-04-25T10:05:34.706921+0000","last_fullsized":"2024-04-25T10:05:34.706921+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:11.085033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351622+0000","last_change":"2024-04-25T10:05:33.361936+0000","last_active":"2024-04-25T10:05:34.351622+0000","last_peered":"2024-04-25T10:05:34.351622+0000","last_clean":"2024-04-25T10:05:34.351622+0000","last_became_active":"2024-04-25T10:05:33.355750+0000","last_became_peered":"2024-04-25T10:05:33.355750+0000","last_unstale":"2024-04-25T10:05:34.351622+0000","last_undegraded":"2024-04-25T10:05:34.351622+0000","last_fullsized":"2024-04-25T10:05:34.351622+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:33.828482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352307+0000","last_change":"2024-04-25T10:05:32.342380+0000","last_active":"2024-04-25T10:05:34.352307+0000","last_peered":"2024-04-25T10:05:34.352307+0000","last_clean":"2024-04-25T10:05:34.352307+0000","last_became_active":"2024-04-25T10:05:32.340667+0000","last_became_peered":"2024-04-25T10:05:32.340667+0000","last_unstale":"2024-04-25T10:05:34.352307+0000","last_undegraded":"2024-04-25T10:05:34.352307+0000","last_fullsized":"2024-04-25T10:05:34.352307+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:48.325615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294141+0000","last_change":"2024-04-25T10:05:33.355615+0000","last_active":"2024-04-25T10:05:35.294141+0000","last_peered":"2024-04-25T10:05:35.294141+0000","last_clean":"2024-04-25T10:05:35.294141+0000","last_became_active":"2024-04-25T10:05:33.355339+0000","last_became_peered":"2024-04-25T10:05:33.355339+0000","last_unstale":"2024-04-25T10:05:35.294141+0000","last_undegraded":"2024-04-25T10:05:35.294141+0000","last_fullsized":"2024-04-25T10:05:35.294141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:15.232714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.351990+0000","last_change":"2024-04-25T10:05:32.338669+0000","last_active":"2024-04-25T10:05:34.351990+0000","last_peered":"2024-04-25T10:05:34.351990+0000","last_clean":"2024-04-25T10:05:34.351990+0000","last_became_active":"2024-04-25T10:05:32.338346+0000","last_became_peered":"2024-04-25T10:05:32.338346+0000","last_unstale":"2024-04-25T10:05:34.351990+0000","last_undegraded":"2024-04-25T10:05:34.351990+0000","last_fullsized":"2024-04-25T10:05:34.351990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:22.267443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.294252+0000","last_change":"2024-04-25T10:05:33.369359+0000","last_active":"2024-04-25T10:05:35.294252+0000","last_peered":"2024-04-25T10:05:35.294252+0000","last_clean":"2024-04-25T10:05:35.294252+0000","last_became_active":"2024-04-25T10:05:33.365638+0000","last_became_peered":"2024-04-25T10:05:33.365638+0000","last_unstale":"2024-04-25T10:05:35.294252+0000","last_undegraded":"2024-04-25T10:05:35.294252+0000","last_fullsized":"2024-04-25T10:05:35.294252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:28.913557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296484+0000","last_change":"2024-04-25T10:05:32.333729+0000","last_active":"2024-04-25T10:05:35.296484+0000","last_peered":"2024-04-25T10:05:35.296484+0000","last_clean":"2024-04-25T10:05:35.296484+0000","last_became_active":"2024-04-25T10:05:32.333410+0000","last_became_peered":"2024-04-25T10:05:32.333410+0000","last_unstale":"2024-04-25T10:05:35.296484+0000","last_undegraded":"2024-04-25T10:05:35.296484+0000","last_fullsized":"2024-04-25T10:05:35.296484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:44.439969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296353+0000","last_change":"2024-04-25T10:05:33.360841+0000","last_active":"2024-04-25T10:05:35.296353+0000","last_peered":"2024-04-25T10:05:35.296353+0000","last_clean":"2024-04-25T10:05:35.296353+0000","last_became_active":"2024-04-25T10:05:33.360407+0000","last_became_peered":"2024-04-25T10:05:33.360407+0000","last_unstale":"2024-04-25T10:05:35.296353+0000","last_undegraded":"2024-04-25T10:05:35.296353+0000","last_fullsized":"2024-04-25T10:05:35.296353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:56.165349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354097+0000","last_change":"2024-04-25T10:05:32.321787+0000","last_active":"2024-04-25T10:05:34.354097+0000","last_peered":"2024-04-25T10:05:34.354097+0000","last_clean":"2024-04-25T10:05:34.354097+0000","last_became_active":"2024-04-25T10:05:32.320831+0000","last_became_peered":"2024-04-25T10:05:32.320831+0000","last_unstale":"2024-04-25T10:05:34.354097+0000","last_undegraded":"2024-04-25T10:05:34.354097+0000","last_fullsized":"2024-04-25T10:05:34.354097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:11.085033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353462+0000","last_change":"2024-04-25T10:05:33.357462+0000","last_active":"2024-04-25T10:05:34.353462+0000","last_peered":"2024-04-25T10:05:34.353462+0000","last_clean":"2024-04-25T10:05:34.353462+0000","last_became_active":"2024-04-25T10:05:33.357134+0000","last_became_peered":"2024-04-25T10:05:33.357134+0000","last_unstale":"2024-04-25T10:05:34.353462+0000","last_undegraded":"2024-04-25T10:05:34.353462+0000","last_fullsized":"2024-04-25T10:05:34.353462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:27.622180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353802+0000","last_change":"2024-04-25T10:05:32.337767+0000","last_active":"2024-04-25T10:05:34.353802+0000","last_peered":"2024-04-25T10:05:34.353802+0000","last_clean":"2024-04-25T10:05:34.353802+0000","last_became_active":"2024-04-25T10:05:32.336592+0000","last_became_peered":"2024-04-25T10:05:32.336592+0000","last_unstale":"2024-04-25T10:05:34.353802+0000","last_undegraded":"2024-04-25T10:05:34.353802+0000","last_fullsized":"2024-04-25T10:05:34.353802+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:43.513074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.292691+0000","last_change":"2024-04-25T10:05:33.360950+0000","last_active":"2024-04-25T10:05:35.292691+0000","last_peered":"2024-04-25T10:05:35.292691+0000","last_clean":"2024-04-25T10:05:35.292691+0000","last_became_active":"2024-04-25T10:05:33.356927+0000","last_became_peered":"2024-04-25T10:05:33.356927+0000","last_unstale":"2024-04-25T10:05:35.292691+0000","last_undegraded":"2024-04-25T10:05:35.292691+0000","last_fullsized":"2024-04-25T10:05:35.292691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:40.701811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707289+0000","last_change":"2024-04-25T10:05:32.340205+0000","last_active":"2024-04-25T10:05:34.707289+0000","last_peered":"2024-04-25T10:05:34.707289+0000","last_clean":"2024-04-25T10:05:34.707289+0000","last_became_active":"2024-04-25T10:05:32.339847+0000","last_became_peered":"2024-04-25T10:05:32.339847+0000","last_unstale":"2024-04-25T10:05:34.707289+0000","last_undegraded":"2024-04-25T10:05:34.707289+0000","last_fullsized":"2024-04-25T10:05:34.707289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:37.484850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.350369+0000","last_change":"2024-04-25T10:05:33.344941+0000","last_active":"2024-04-25T10:05:34.350369+0000","last_peered":"2024-04-25T10:05:34.350369+0000","last_clean":"2024-04-25T10:05:34.350369+0000","last_became_active":"2024-04-25T10:05:33.344151+0000","last_became_peered":"2024-04-25T10:05:33.344151+0000","last_unstale":"2024-04-25T10:05:34.350369+0000","last_undegraded":"2024-04-25T10:05:34.350369+0000","last_fullsized":"2024-04-25T10:05:34.350369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:49.330430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354708+0000","last_change":"2024-04-25T10:05:32.336040+0000","last_active":"2024-04-25T10:05:34.354708+0000","last_peered":"2024-04-25T10:05:34.354708+0000","last_clean":"2024-04-25T10:05:34.354708+0000","last_became_active":"2024-04-25T10:05:32.335710+0000","last_became_peered":"2024-04-25T10:05:32.335710+0000","last_unstale":"2024-04-25T10:05:34.354708+0000","last_undegraded":"2024-04-25T10:05:34.354708+0000","last_fullsized":"2024-04-25T10:05:34.354708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:45.072130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.367194+0000","last_change":"2024-04-25T10:05:33.360142+0000","last_active":"2024-04-25T10:05:33.367194+0000","last_peered":"2024-04-25T10:05:33.367194+0000","last_clean":"2024-04-25T10:05:33.367194+0000","last_became_active":"2024-04-25T10:05:33.359852+0000","last_became_peered":"2024-04-25T10:05:33.359852+0000","last_unstale":"2024-04-25T10:05:33.367194+0000","last_undegraded":"2024-04-25T10:05:33.367194+0000","last_fullsized":"2024-04-25T10:05:33.367194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:41.993533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.352306+0000","last_change":"2024-04-25T10:05:32.324691+0000","last_active":"2024-04-25T10:05:34.352306+0000","last_peered":"2024-04-25T10:05:34.352306+0000","last_clean":"2024-04-25T10:05:34.352306+0000","last_became_active":"2024-04-25T10:05:32.324414+0000","last_became_peered":"2024-04-25T10:05:32.324414+0000","last_unstale":"2024-04-25T10:05:34.352306+0000","last_undegraded":"2024-04-25T10:05:34.352306+0000","last_fullsized":"2024-04-25T10:05:34.352306+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:02.679204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.369349+0000","last_change":"2024-04-25T10:05:33.363928+0000","last_active":"2024-04-25T10:05:33.369349+0000","last_peered":"2024-04-25T10:05:33.369349+0000","last_clean":"2024-04-25T10:05:33.369349+0000","last_became_active":"2024-04-25T10:05:33.362037+0000","last_became_peered":"2024-04-25T10:05:33.362037+0000","last_unstale":"2024-04-25T10:05:33.369349+0000","last_undegraded":"2024-04-25T10:05:33.369349+0000","last_fullsized":"2024-04-25T10:05:33.369349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:09.989447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.366919+0000","last_change":"2024-04-25T10:05:32.316530+0000","last_active":"2024-04-25T10:05:34.366919+0000","last_peered":"2024-04-25T10:05:34.366919+0000","last_clean":"2024-04-25T10:05:34.366919+0000","last_became_active":"2024-04-25T10:05:32.316249+0000","last_became_peered":"2024-04-25T10:05:32.316249+0000","last_unstale":"2024-04-25T10:05:34.366919+0000","last_undegraded":"2024-04-25T10:05:34.366919+0000","last_fullsized":"2024-04-25T10:05:34.366919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:45.411785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296317+0000","last_change":"2024-04-25T10:05:33.370056+0000","last_active":"2024-04-25T10:05:35.296317+0000","last_peered":"2024-04-25T10:05:35.296317+0000","last_clean":"2024-04-25T10:05:35.296317+0000","last_became_active":"2024-04-25T10:05:33.369775+0000","last_became_peered":"2024-04-25T10:05:33.369775+0000","last_unstale":"2024-04-25T10:05:35.296317+0000","last_undegraded":"2024-04-25T10:05:35.296317+0000","last_fullsized":"2024-04-25T10:05:35.296317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:35.143413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354257+0000","last_change":"2024-04-25T10:05:32.337930+0000","last_active":"2024-04-25T10:05:34.354257+0000","last_peered":"2024-04-25T10:05:34.354257+0000","last_clean":"2024-04-25T10:05:34.354257+0000","last_became_active":"2024-04-25T10:05:32.337585+0000","last_became_peered":"2024-04-25T10:05:32.337585+0000","last_unstale":"2024-04-25T10:05:34.354257+0000","last_undegraded":"2024-04-25T10:05:34.354257+0000","last_fullsized":"2024-04-25T10:05:34.354257+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:40.623031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.296851+0000","last_change":"2024-04-25T10:05:33.359177+0000","last_active":"2024-04-25T10:05:35.296851+0000","last_peered":"2024-04-25T10:05:35.296851+0000","last_clean":"2024-04-25T10:05:35.296851+0000","last_became_active":"2024-04-25T10:05:33.358451+0000","last_became_peered":"2024-04-25T10:05:33.358451+0000","last_unstale":"2024-04-25T10:05:35.296851+0000","last_undegraded":"2024-04-25T10:05:35.296851+0000","last_fullsized":"2024-04-25T10:05:35.296851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:21.086544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354395+0000","last_change":"2024-04-25T10:05:32.330355+0000","last_active":"2024-04-25T10:05:34.354395+0000","last_peered":"2024-04-25T10:05:34.354395+0000","last_clean":"2024-04-25T10:05:34.354395+0000","last_became_active":"2024-04-25T10:05:32.330043+0000","last_became_peered":"2024-04-25T10:05:32.330043+0000","last_unstale":"2024-04-25T10:05:34.354395+0000","last_undegraded":"2024-04-25T10:05:34.354395+0000","last_fullsized":"2024-04-25T10:05:34.354395+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.465109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.349799+0000","last_change":"2024-04-25T10:05:33.361521+0000","last_active":"2024-04-25T10:05:34.349799+0000","last_peered":"2024-04-25T10:05:34.349799+0000","last_clean":"2024-04-25T10:05:34.349799+0000","last_became_active":"2024-04-25T10:05:33.361195+0000","last_became_peered":"2024-04-25T10:05:33.361195+0000","last_unstale":"2024-04-25T10:05:34.349799+0000","last_undegraded":"2024-04-25T10:05:34.349799+0000","last_fullsized":"2024-04-25T10:05:34.349799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:23.206514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293533+0000","last_change":"2024-04-25T10:05:32.340448+0000","last_active":"2024-04-25T10:05:35.293533+0000","last_peered":"2024-04-25T10:05:35.293533+0000","last_clean":"2024-04-25T10:05:35.293533+0000","last_became_active":"2024-04-25T10:05:32.340150+0000","last_became_peered":"2024-04-25T10:05:32.340150+0000","last_unstale":"2024-04-25T10:05:35.293533+0000","last_undegraded":"2024-04-25T10:05:35.293533+0000","last_fullsized":"2024-04-25T10:05:35.293533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:16.583863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293231+0000","last_change":"2024-04-25T10:05:33.368630+0000","last_active":"2024-04-25T10:05:35.293231+0000","last_peered":"2024-04-25T10:05:35.293231+0000","last_clean":"2024-04-25T10:05:35.293231+0000","last_became_active":"2024-04-25T10:05:33.363531+0000","last_became_peered":"2024-04-25T10:05:33.363531+0000","last_unstale":"2024-04-25T10:05:35.293231+0000","last_undegraded":"2024-04-25T10:05:35.293231+0000","last_fullsized":"2024-04-25T10:05:35.293231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:49.338591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:35.293533+0000","last_change":"2024-04-25T10:05:32.340407+0000","last_active":"2024-04-25T10:05:35.293533+0000","last_peered":"2024-04-25T10:05:35.293533+0000","last_clean":"2024-04-25T10:05:35.293533+0000","last_became_active":"2024-04-25T10:05:32.340045+0000","last_became_peered":"2024-04-25T10:05:32.340045+0000","last_unstale":"2024-04-25T10:05:35.293533+0000","last_undegraded":"2024-04-25T10:05:35.293533+0000","last_fullsized":"2024-04-25T10:05:35.293533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:14.219738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354386+0000","last_change":"2024-04-25T10:05:33.382358+0000","last_active":"2024-04-25T10:05:34.354386+0000","last_peered":"2024-04-25T10:05:34.354386+0000","last_clean":"2024-04-25T10:05:34.354386+0000","last_became_active":"2024-04-25T10:05:33.382032+0000","last_became_peered":"2024-04-25T10:05:33.382032+0000","last_unstale":"2024-04-25T10:05:34.354386+0000","last_undegraded":"2024-04-25T10:05:34.354386+0000","last_fullsized":"2024-04-25T10:05:34.354386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:23.476713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.353300+0000","last_change":"2024-04-25T10:05:32.333371+0000","last_active":"2024-04-25T10:05:34.353300+0000","last_peered":"2024-04-25T10:05:34.353300+0000","last_clean":"2024-04-25T10:05:34.353300+0000","last_became_active":"2024-04-25T10:05:32.333059+0000","last_became_peered":"2024-04-25T10:05:32.333059+0000","last_unstale":"2024-04-25T10:05:34.353300+0000","last_undegraded":"2024-04-25T10:05:34.353300+0000","last_fullsized":"2024-04-25T10:05:34.353300+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:09.570277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T10:05:33.350923+0000","last_change":"2024-04-25T10:05:33.344822+0000","last_active":"2024-04-25T10:05:33.350923+0000","last_peered":"2024-04-25T10:05:33.350923+0000","last_clean":"2024-04-25T10:05:33.350923+0000","last_became_active":"2024-04-25T10:05:33.344305+0000","last_became_peered":"2024-04-25T10:05:33.344305+0000","last_unstale":"2024-04-25T10:05:33.350923+0000","last_undegraded":"2024-04-25T10:05:33.350923+0000","last_fullsized":"2024-04-25T10:05:33.350923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:45.906184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354956+0000","last_change":"2024-04-25T10:05:32.338824+0000","last_active":"2024-04-25T10:05:34.354956+0000","last_peered":"2024-04-25T10:05:34.354956+0000","last_clean":"2024-04-25T10:05:34.354956+0000","last_became_active":"2024-04-25T10:05:32.337668+0000","last_became_peered":"2024-04-25T10:05:32.337668+0000","last_unstale":"2024-04-25T10:05:34.354956+0000","last_undegraded":"2024-04-25T10:05:34.354956+0000","last_fullsized":"2024-04-25T10:05:34.354956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:27.900581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707635+0000","last_change":"2024-04-25T10:05:33.372324+0000","last_active":"2024-04-25T10:05:34.707635+0000","last_peered":"2024-04-25T10:05:34.707635+0000","last_clean":"2024-04-25T10:05:34.707635+0000","last_became_active":"2024-04-25T10:05:33.368096+0000","last_became_peered":"2024-04-25T10:05:33.368096+0000","last_unstale":"2024-04-25T10:05:34.707635+0000","last_undegraded":"2024-04-25T10:05:34.707635+0000","last_fullsized":"2024-04-25T10:05:34.707635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:32.291504+0000","last_clean_scrub_stamp":"2024-04-25T10:05:32.291504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:49.216813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.354708+0000","last_change":"2024-04-25T10:05:32.333938+0000","last_active":"2024-04-25T10:05:34.354708+0000","last_peered":"2024-04-25T10:05:34.354708+0000","last_clean":"2024-04-25T10:05:34.354708+0000","last_became_active":"2024-04-25T10:05:32.333652+0000","last_became_peered":"2024-04-25T10:05:32.333652+0000","last_unstale":"2024-04-25T10:05:34.354708+0000","last_undegraded":"2024-04-25T10:05:34.354708+0000","last_fullsized":"2024-04-25T10:05:34.354708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:31.278528+0000","last_clean_scrub_stamp":"2024-04-25T10:05:31.278528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:48.203838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":248,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-25T10:05:34.707086+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:05:34.707086+0000","last_peered":"2024-04-25T10:05:34.707086+0000","last_clean":"2024-04-25T10:05:34.707086+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:05:34.707086+0000","last_undegraded":"2024-04-25T10:05:34.707086+0000","last_fullsized":"2024-04-25T10:05:34.707086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":765952,"data_stored":342298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509490,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":337870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":934390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":774144,"data_stored":342458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":342354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":926008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":765952,"data_stored":342298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":757760,"data_stored":336188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:38.336 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:38.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:05:38.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.586+0000 7f3cc3e00640 1 Processor -- start 2024-04-25T10:05:38.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.586+0000 7f3cc3e00640 1 -- start start 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3cc40b48c0 0x7f3cc415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3cc40d3f90 0x7f3cc4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc3e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc41606e0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc3e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc4161ff0 con 0x7f3cc40d3f90 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc41609c0 con 0x7f3cc40b48c0 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc2000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3cc40d3f90 0x7f3cc4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.588+0000 7f3cc3400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3cc4161bc0 0x7f3cc40d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59180/0 (socket says 172.21.15.5:59180) 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc3400640 1 -- 172.21.15.5:0/4130492014 learned_addr learned my addr 172.21.15.5:0/4130492014 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc2a00640 1 --2- 172.21.15.5:0/4130492014 >> v2:172.21.15.80:3300/0 conn(0x7f3cc40b48c0 0x7f3cc415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc2000640 1 -- 172.21.15.5:0/4130492014 >> v2:172.21.15.80:3300/0 conn(0x7f3cc40b48c0 msgr2=0x7f3cc415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc2000640 1 --2- 172.21.15.5:0/4130492014 >> v2:172.21.15.80:3300/0 conn(0x7f3cc40b48c0 0x7f3cc415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc2000640 1 -- 172.21.15.5:0/4130492014 >> v1:172.21.15.5:6789/0 conn(0x7f3cc4161bc0 legacy=0x7f3cc40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc2000640 1 -- 172.21.15.5:0/4130492014 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4160ca0 con 0x7f3cc40d3f90 2024-04-25T10:05:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.589+0000 7f3cc1600640 1 -- 172.21.15.5:0/4130492014 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241438631 0 0) 0x7f3cc41606e0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.590+0000 7f3cc2000640 1 --2- 172.21.15.5:0/4130492014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc40d3f90 0x7f3cc4161680 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3cc408ba40 tx=0x7f3cb002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a03f1c209ee01c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc1600640 1 -- 172.21.15.5:0/4130492014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0037070 con 0x7f3cc40d3f90 2024-04-25T10:05:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc1600640 1 -- 172.21.15.5:0/4130492014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb0032440 con 0x7f3cc40d3f90 2024-04-25T10:05:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc1600640 1 -- 172.21.15.5:0/4130492014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb002ae60 con 0x7f3cc40d3f90 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc3e00640 1 -- 172.21.15.5:0/4130492014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc40d3f90 msgr2=0x7f3cc4161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/4130492014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc40d3f90 0x7f3cc4161680 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3cc408ba40 tx=0x7f3cb002a110 comp rx=0 tx=0).stop 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc3e00640 1 -- 172.21.15.5:0/4130492014 shutdown_connections 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/4130492014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc40d3f90 0x7f3cc4161680 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/4130492014 >> v2:172.21.15.80:3300/0 conn(0x7f3cc40b48c0 0x7f3cc415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.591+0000 7f3cc3e00640 1 -- 172.21.15.5:0/4130492014 >> 172.21.15.5:0/4130492014 conn(0x7f3cc408af80 msgr2=0x7f3cc40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.592+0000 7f3cc3e00640 1 -- 172.21.15.5:0/4130492014 shutdown_connections 2024-04-25T10:05:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.592+0000 7f3cc3e00640 1 -- 172.21.15.5:0/4130492014 wait complete. 2024-04-25T10:05:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.593+0000 7f3cc3e00640 1 Processor -- start 2024-04-25T10:05:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.594+0000 7f3cc3e00640 1 -- start start 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.594+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cc40b48c0 0x7f3cc40c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.594+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cc40d3f90 0x7f3cc4155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 0x7f3cc40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40b4ca0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4161260 con 0x7f3cc40b48c0 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41603a0 con 0x7f3cc40d3f90 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 0x7f3cc40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 0x7f3cc40cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39850/0 (socket says 172.21.15.5:39850) 2024-04-25T10:05:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 -- 172.21.15.5:0/3808013452 learned_addr learned my addr 172.21.15.5:0/3808013452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc2a00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cc40b48c0 0x7f3cc40c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 -- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cc40d3f90 msgr2=0x7f3cc4155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cc40d3f90 0x7f3cc4155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 -- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cc40b48c0 msgr2=0x7f3cc40c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cc40b48c0 0x7f3cc40c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.595+0000 7f3cc3400640 1 -- 172.21.15.5:0/3808013452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4155ed0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.596+0000 7f3cc3400640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 0x7f3cc40cc2e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c7d0 tx=0x7f3cbc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.596+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc018070 con 0x7f3cc4161bc0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.596+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00a5e0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.596+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00a9d0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.596+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc40c7b40 con 0x7f3cc4161bc0 2024-04-25T10:05:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.596+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc40c8100 con 0x7f3cc4161bc0 2024-04-25T10:05:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.597+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc4068f60 con 0x7f3cc4161bc0 2024-04-25T10:05:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.598+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cbc00ab70 con 0x7f3cc4161bc0 2024-04-25T10:05:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.600+0000 7f3cbaa00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ca406b500 0x7f3ca406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.600+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cbc0898f0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.601+0000 7f3cc2a00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ca406b500 0x7f3ca406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.602+0000 7f3cc2a00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ca406b500 0x7f3ca406d9c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006fd0 tx=0x7f3cb4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.605+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cbc058670 con 0x7f3cc4161bc0 2024-04-25T10:05:38.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.876+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3cc40d0dd0 con 0x7f3cc4161bc0 2024-04-25T10:05:38.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.881+0000 7f3cbaa00640 1 -- 172.21.15.5:0/3808013452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3cbc057f70 con 0x7f3cc4161bc0 2024-04-25T10:05:38.886 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:05:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:05:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:05:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:05:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:38.887 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:05:38.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ca406b500 msgr2=0x7f3ca406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ca406b500 0x7f3ca406d9c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006fd0 tx=0x7f3cb4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:38.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 msgr2=0x7f3cc40cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 0x7f3cc40cc2e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c7d0 tx=0x7f3cbc00cc90 comp rx=0 tx=0).stop 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 shutdown_connections 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cc4161bc0 0x7f3cc40cc2e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3cc40d3f90 0x7f3cc4155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3ca406b500 0x7f3ca406d9c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 --2- 172.21.15.5:0/3808013452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3cc40b48c0 0x7f3cc40c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 >> 172.21.15.5:0/3808013452 conn(0x7f3cc408af80 msgr2=0x7f3cc406c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.884+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 shutdown_connections 2024-04-25T10:05:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:38.885+0000 7f3cc3e00640 1 -- 172.21.15.5:0/3808013452 wait complete. 2024-04-25T10:05:38.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.113+0000 7f267da00640 1 Processor -- start 2024-04-25T10:05:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.113+0000 7f267da00640 1 -- start start 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.115+0000 7f267da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.115+0000 7f267da00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26780cd300 0x7f26780cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.115+0000 7f267da00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2678071d30 con 0x7f2678141da0 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267da00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780cdcd0 con 0x7f26780cd300 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267da00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678142180 con 0x7f267815ce00 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26780cd300 0x7f26780cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26780cd300 0x7f26780cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39858/0 (socket says 172.21.15.5:39858) 2024-04-25T10:05:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 -- 172.21.15.5:0/586293350 learned_addr learned my addr 172.21.15.5:0/586293350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f266fe00640 1 --2- 172.21.15.5:0/586293350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 -- 172.21.15.5:0/586293350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 msgr2=0x7f267815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 --2- 172.21.15.5:0/586293350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 -- 172.21.15.5:0/586293350 >> v1:172.21.15.80:6789/0 conn(0x7f2678141da0 legacy=0x7f267806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.116+0000 7f267d000640 1 -- 172.21.15.5:0/586293350 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267806d620 con 0x7f26780cd300 2024-04-25T10:05:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.118+0000 7f267d000640 1 --2- 172.21.15.5:0/586293350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26780cd300 0x7f26780cd790 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2670009e50 tx=0x7f267002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba3fac1a6bb9656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.118+0000 7f2677400640 1 -- 172.21.15.5:0/586293350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670036070 con 0x7f26780cd300 2024-04-25T10:05:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.118+0000 7f2677400640 1 -- 172.21.15.5:0/586293350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670031600 con 0x7f26780cd300 2024-04-25T10:05:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.118+0000 7f2677400640 1 -- 172.21.15.5:0/586293350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700319d0 con 0x7f26780cd300 2024-04-25T10:05:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 -- 172.21.15.5:0/586293350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26780cd300 msgr2=0x7f26780cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 --2- 172.21.15.5:0/586293350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26780cd300 0x7f26780cd790 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f2670009e50 tx=0x7f267002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 -- 172.21.15.5:0/586293350 shutdown_connections 2024-04-25T10:05:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 --2- 172.21.15.5:0/586293350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26780cd300 0x7f26780cd790 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 --2- 172.21.15.5:0/586293350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 -- 172.21.15.5:0/586293350 >> 172.21.15.5:0/586293350 conn(0x7f2678066f20 msgr2=0x7f267806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 -- 172.21.15.5:0/586293350 shutdown_connections 2024-04-25T10:05:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.119+0000 7f267da00640 1 -- 172.21.15.5:0/586293350 wait complete. 2024-04-25T10:05:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.121+0000 7f267da00640 1 Processor -- start 2024-04-25T10:05:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.122+0000 7f267da00640 1 -- start start 2024-04-25T10:05:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f267da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26780cd300 0x7f26781e7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f267da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 0x7f26780b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f267da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f267da00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267806e810 con 0x7f2678141da0 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f267da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267806dbe0 con 0x7f26780cd300 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f267da00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678071000 con 0x7f267815ce00 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f266fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 0x7f26780b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f266fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 0x7f26780b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39872/0 (socket says 172.21.15.5:39872) 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f266fe00640 1 -- 172.21.15.5:0/420955335 learned_addr learned my addr 172.21.15.5:0/420955335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f2677e00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26780cd300 0x7f26781e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.123+0000 7f266fe00640 1 -- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 msgr2=0x7f267814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f266fe00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f266fe00640 1 -- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26780cd300 msgr2=0x7f26781e7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f266fe00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26780cd300 0x7f26781e7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f266fe00640 1 -- 172.21.15.5:0/420955335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678150460 con 0x7f2678141da0 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f266fe00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 0x7f26780b24b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f266000d5f0 tx=0x7f266000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600125f0 con 0x7f2678141da0 2024-04-25T10:05:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2660019070 con 0x7f2678141da0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600129d0 con 0x7f2678141da0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26780b2be0 con 0x7f2678141da0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.124+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780b31a0 con 0x7f2678141da0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.126+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2678068f60 con 0x7f2678141da0 2024-04-25T10:05:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.127+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2660012b70 con 0x7f2678141da0 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.128+0000 7f2674c00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f265406b4d0 0x7f265406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.128+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f266008a7f0 con 0x7f2678141da0 2024-04-25T10:05:39.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.129+0000 7f2677e00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f265406b4d0 0x7f265406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.130+0000 7f2677e00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f265406b4d0 0x7f265406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2668003d90 tx=0x7f2668009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.133+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2660059440 con 0x7f2678141da0 2024-04-25T10:05:39.289 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:05:39.284+0000 7fce04600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.373+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26780ccae0 con 0x7f2678141da0 2024-04-25T10:05:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.374+0000 7f2674c00640 1 -- 172.21.15.5:0/420955335 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f2660058d40 con 0x7f2678141da0 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:05:35.874224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f265406b4d0 msgr2=0x7f265406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f265406b4d0 0x7f265406d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2668003d90 tx=0x7f2668009040 comp rx=0 tx=0).stop 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 msgr2=0x7f26780b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 0x7f26780b24b0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f266000d5f0 tx=0x7f266000dab0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 shutdown_connections 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f267815ce00 0x7f267814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2678141da0 0x7f26780b24b0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f265406b4d0 0x7f265406d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 --2- 172.21.15.5:0/420955335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26780cd300 0x7f26781e7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 >> 172.21.15.5:0/420955335 conn(0x7f2678066f20 msgr2=0x7f2678073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 shutdown_connections 2024-04-25T10:05:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:39.377+0000 7f267da00640 1 -- 172.21.15.5:0/420955335 wait complete. 2024-04-25T10:05:39.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:32.782629+0000', 'modified': '2024-04-25T10:05:35.874224+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6255}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6255': {'gid': 6255, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/1822355874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 1822355874}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 1822355874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:39.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:39.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.023+0000 7f450dc00640 1 Processor -- start 2024-04-25T10:05:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.023+0000 7f450dc00640 1 -- start start 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.025+0000 7f450dc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45080b4bc0 0x7f450815b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.025+0000 7f450dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f450815b750 0x7f45080ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.025+0000 7f450dc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45080b5070 con 0x7f4508154b80 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.025+0000 7f450dc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450806be80 con 0x7f45080b4bc0 2024-04-25T10:05:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.025+0000 7f450dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450806c160 con 0x7f450815b750 2024-04-25T10:05:40.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.026+0000 7f4507400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45080b4bc0 0x7f450815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.026+0000 7f450d200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f450815b750 0x7f45080ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.026+0000 7f4507400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45080b4bc0 0x7f450815b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:43158/0 (socket says 172.21.15.5:43158) 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.026+0000 7f4507400640 1 -- 172.21.15.5:0/605290273 learned_addr learned my addr 172.21.15.5:0/605290273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.027+0000 7f4507400640 1 -- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f450815b750 msgr2=0x7f45080ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.027+0000 7f4507400640 1 --2- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f450815b750 0x7f45080ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.027+0000 7f4507400640 1 -- 172.21.15.5:0/605290273 >> v1:172.21.15.80:6789/0 conn(0x7f4508154b80 legacy=0x7f450815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.027+0000 7f4507400640 1 -- 172.21.15.5:0/605290273 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450806c440 con 0x7f45080b4bc0 2024-04-25T10:05:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.027+0000 7f450d200640 1 --2- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f450815b750 0x7f45080ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.028+0000 7f4507400640 1 --2- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45080b4bc0 0x7f450815b210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44f400bb10 tx=0x7f44f402c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f41eff726e17cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.029+0000 7f4506a00640 1 -- 172.21.15.5:0/605290273 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4032790 con 0x7f45080b4bc0 2024-04-25T10:05:40.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.029+0000 7f4506a00640 1 -- 172.21.15.5:0/605290273 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4038070 con 0x7f45080b4bc0 2024-04-25T10:05:40.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.029+0000 7f4506a00640 1 -- 172.21.15.5:0/605290273 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4032b50 con 0x7f45080b4bc0 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.029+0000 7f450dc00640 1 -- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45080b4bc0 msgr2=0x7f450815b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.029+0000 7f450dc00640 1 --2- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45080b4bc0 0x7f450815b210 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f44f400bb10 tx=0x7f44f402c890 comp rx=0 tx=0).stop 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.030+0000 7f450dc00640 1 -- 172.21.15.5:0/605290273 shutdown_connections 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.030+0000 7f450dc00640 1 --2- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f450815b750 0x7f45080ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.030+0000 7f450dc00640 1 --2- 172.21.15.5:0/605290273 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45080b4bc0 0x7f450815b210 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.030+0000 7f450dc00640 1 -- 172.21.15.5:0/605290273 >> 172.21.15.5:0/605290273 conn(0x7f45080cc6a0 msgr2=0x7f45080ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.030+0000 7f450dc00640 1 -- 172.21.15.5:0/605290273 shutdown_connections 2024-04-25T10:05:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.030+0000 7f450dc00640 1 -- 172.21.15.5:0/605290273 wait complete. 2024-04-25T10:05:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.031+0000 7f450dc00640 1 Processor -- start 2024-04-25T10:05:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.031+0000 7f450dc00640 1 -- start start 2024-04-25T10:05:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.032+0000 7f450dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45080b4bc0 0x7f45080ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.032+0000 7f450dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508154b80 0x7f45080ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.032+0000 7f450dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 0x7f45080b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.032+0000 7f450dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4508154f60 con 0x7f450815b750 2024-04-25T10:05:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.032+0000 7f450dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080cde20 con 0x7f4508154b80 2024-04-25T10:05:40.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.032+0000 7f450dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450806d580 con 0x7f45080b4bc0 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 0x7f45080b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 0x7f45080b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39882/0 (socket says 172.21.15.5:39882) 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 -- 172.21.15.5:0/468264588 learned_addr learned my addr 172.21.15.5:0/468264588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f4507400640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508154b80 0x7f45080ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f4507e00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45080b4bc0 0x7f45080ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 -- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45080b4bc0 msgr2=0x7f45080ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45080b4bc0 0x7f45080ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 -- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508154b80 msgr2=0x7f45080ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508154b80 0x7f45080ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f450d200640 1 -- 172.21.15.5:0/468264588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45080b1300 con 0x7f450815b750 2024-04-25T10:05:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.033+0000 7f4507400640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508154b80 0x7f45080ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.034+0000 7f450d200640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 0x7f45080b0920 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f450000efc0 tx=0x7f450000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.034+0000 7f44ffe00640 1 -- 172.21.15.5:0/468264588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500018030 con 0x7f450815b750 2024-04-25T10:05:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.034+0000 7f44ffe00640 1 -- 172.21.15.5:0/468264588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000cdf0 con 0x7f450815b750 2024-04-25T10:05:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.034+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45080b15e0 con 0x7f450815b750 2024-04-25T10:05:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.034+0000 7f44ffe00640 1 -- 172.21.15.5:0/468264588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500003bb0 con 0x7f450815b750 2024-04-25T10:05:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.034+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45080ad260 con 0x7f450815b750 2024-04-25T10:05:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.036+0000 7f44ffe00640 1 -- 172.21.15.5:0/468264588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4500005d10 con 0x7f450815b750 2024-04-25T10:05:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.037+0000 7f44ffe00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44e806b4d0 0x7f44e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.037+0000 7f44ffe00640 1 -- 172.21.15.5:0/468264588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f450008ad80 con 0x7f450815b750 2024-04-25T10:05:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.038+0000 7f4512176740 1 Processor -- start 2024-04-25T10:05:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.038+0000 7f4507e00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44e806b4d0 0x7f44e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.038+0000 7f4512176740 1 -- start start 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4507e00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44e806b4d0 0x7f44e806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f44f8000f60 tx=0x7f44f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348b30 0x562367370d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4512176740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 0x562367349c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367347fc0 0x56236738f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4512176740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623672ac620 con 0x562367348f10 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562367255ca0 con 0x562367347fc0 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623672a86c0 con 0x562367348b30 2024-04-25T10:05:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.039+0000 7f4507400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 0x562367349c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 0x562367349c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39886/0 (socket says 172.21.15.5:39886) 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 -- 172.21.15.5:0/3381917574 learned_addr learned my addr 172.21.15.5:0/3381917574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507e00640 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348b30 0x562367370d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f450d200640 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367347fc0 0x56236738f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 -- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348b30 msgr2=0x562367370d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348b30 0x562367370d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 -- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367347fc0 msgr2=0x56236738f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367347fc0 0x56236738f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 -- 172.21.15.5:0/3381917574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236725cf40 con 0x562367348f10 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f450d200640 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367347fc0 0x56236738f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.040+0000 7f4507400640 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 0x562367349c60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f44f402c8d0 tx=0x7f44f4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58acbe23873c8251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f44fcc00640 1 -- 172.21.15.5:0/3381917574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4032890 con 0x562367348f10 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f44fcc00640 1 -- 172.21.15.5:0/3381917574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4038040 con 0x562367348f10 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f44fcc00640 1 -- 172.21.15.5:0/3381917574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4032c70 con 0x562367348f10 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f4512176740 1 -- 172.21.15.5:0/3381917574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 msgr2=0x562367349c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f4512176740 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 0x562367349c60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f44f402c8d0 tx=0x7f44f4032000 comp rx=0 tx=0).stop 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f4512176740 1 -- 172.21.15.5:0/3381917574 shutdown_connections 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f4512176740 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367347fc0 0x56236738f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.041+0000 7f4512176740 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348f10 0x562367349c60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.042+0000 7f4512176740 1 --2- 172.21.15.5:0/3381917574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348b30 0x562367370d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.042+0000 7f4512176740 1 -- 172.21.15.5:0/3381917574 >> 172.21.15.5:0/3381917574 conn(0x562367345d50 msgr2=0x56236738e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.042+0000 7f4512176740 1 -- 172.21.15.5:0/3381917574 shutdown_connections 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.042+0000 7f4512176740 1 -- 172.21.15.5:0/3381917574 wait complete. 2024-04-25T10:05:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.042+0000 7f4512176740 1 -- start start 2024-04-25T10:05:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.043+0000 7f4512176740 1 Processor -- start 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x562367376630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562367343c60 con 0x56236736ad70 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236724b1f0 con 0x562367348b30 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236738f0f0 con 0x562367348f10 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4512176740 10 client.0 _ll_register_callbacks cb 0x562367268ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f450d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x562367376630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f450d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x562367376630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39892/0 (socket says 172.21.15.5:39892) 2024-04-25T10:05:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f450d200640 1 -- 172.21.15.5:0/3876713283 learned_addr learned my addr 172.21.15.5:0/3876713283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4507e00640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.044+0000 7f4507400640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f450d200640 1 -- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 msgr2=0x5623673760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f450d200640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673760f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f450d200640 1 -- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 msgr2=0x5623673a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f450d200640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f450d200640 1 -- 172.21.15.5:0/3876713283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236726ac50 con 0x56236736ad70 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f4507e00640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f450d200640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x562367376630 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4500061540 tx=0x7f450005d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0f965822723279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f44f1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f44f1600640 1 -- 172.21.15.5:0/3876713283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000587b0 con 0x56236736ad70 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f44f1600640 1 -- 172.21.15.5:0/3876713283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4500058950 con 0x56236736ad70 2024-04-25T10:05:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44f4038040 con 0x56236736ad70 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f44f1600640 1 -- 172.21.15.5:0/3876713283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f450005f440 con 0x56236736ad70 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f4512176740 10 client.6387 fetch_fsmap learned FSMap version 35 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.045+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x562366fc12d0 con 0x56236736ad70 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f44f1600640 1 -- 172.21.15.5:0/3876713283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500073020 con 0x56236736ad70 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f44f1600640 1 -- 172.21.15.5:0/3876713283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f450005f5e0 con 0x56236736ad70 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4512176740 10 client.6387 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4512176740 10 client.6387 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4512176740 20 client.6387 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4512176740 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5623673725e0 0x5623673729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4512176740 4 client.6387 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623672b9b60 con 0x5623673725e0 2024-04-25T10:05:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.046+0000 7f4507e00640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5623673725e0 0x5623673729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.047+0000 7f4507e00640 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5623673725e0 0x5623673729c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a77bf64adcc8ec server_cookie=1ceaa40bedc5eec8 in_seq=0 out_seq=0 2024-04-25T10:05:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.047+0000 7f44f1600640 10 client.6387 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f44f1600640 1 -- 172.21.15.5:0/3876713283 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623672b9b60 con 0x5623673725e0 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f44f1600640 10 client.6387 handle_command_reply: tid=0 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 client.6387 shutdown 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 msgr2=0x562367376630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x562367376630 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4500061540 tx=0x7f450005d4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 shutdown_connections 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5623673725e0 0x5623673729c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x562367376630 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 --2- 172.21.15.5:0/3876713283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f44f1600640 0 client.6387 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 >> 172.21.15.5:0/3876713283 conn(0x562367347fc0 msgr2=0x56236738e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 shutdown_connections 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.048+0000 7f4512176740 1 -- 172.21.15.5:0/3876713283 wait complete. 2024-04-25T10:05:40.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.049+0000 7f4512176740 20 client.6387 trim_cache size 0 max 16384 2024-04-25T10:05:40.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.066+0000 7f4512176740 1 Processor -- start 2024-04-25T10:05:40.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.066+0000 7f4512176740 1 -- start start 2024-04-25T10:05:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4512176740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 0x56236738cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x562367373f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56236736ad70 0x56236738d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4512176740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081fb0d0 con 0x562367348b30 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45080af8e0 con 0x56236736ad70 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45080afee0 con 0x562367348f10 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4507e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 0x56236738cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4507e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 0x56236738cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39898/0 (socket says 172.21.15.5:39898) 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.067+0000 7f4507e00640 1 -- 172.21.15.5:0/2034357407 learned_addr learned my addr 172.21.15.5:0/2034357407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f4507e00640 1 -- 172.21.15.5:0/2034357407 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 msgr2=0x562367373f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f4507e00640 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x562367373f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f4507e00640 1 -- 172.21.15.5:0/2034357407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56236736ad70 msgr2=0x56236738d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f4507e00640 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56236736ad70 0x56236738d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f4507e00640 1 -- 172.21.15.5:0/2034357407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236708b6b0 con 0x562367348b30 2024-04-25T10:05:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f4507e00640 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 0x56236738cb50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4508072b00 tx=0x7f44f8017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88f5b1f5514922c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f44f3e00640 1 -- 172.21.15.5:0/2034357407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f801e040 con 0x562367348b30 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f44f3e00640 1 -- 172.21.15.5:0/2034357407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8028020 con 0x562367348b30 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.068+0000 7f44f3e00640 1 -- 172.21.15.5:0/2034357407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f80247f0 con 0x562367348b30 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 -- 172.21.15.5:0/2034357407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 msgr2=0x56236738cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 0x56236738cb50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4508072b00 tx=0x7f44f8017040 comp rx=0 tx=0).stop 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 -- 172.21.15.5:0/2034357407 shutdown_connections 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56236736ad70 0x56236738d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x562367373f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 --2- 172.21.15.5:0/2034357407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562367348b30 0x56236738cb50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 -- 172.21.15.5:0/2034357407 >> 172.21.15.5:0/2034357407 conn(0x562367347fc0 msgr2=0x562367347780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 -- 172.21.15.5:0/2034357407 shutdown_connections 2024-04-25T10:05:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.069+0000 7f4512176740 1 -- 172.21.15.5:0/2034357407 wait complete. 2024-04-25T10:05:40.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.070+0000 7f4512176740 1 -- start start 2024-04-25T10:05:40.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.070+0000 7f4512176740 1 Processor -- start 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x5623673fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623673a33a0 con 0x56236736ad70 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623673a2de0 con 0x562367348b30 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562367369fa0 con 0x562367348f10 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f4512176740 10 client.0 _ll_register_callbacks cb 0x7f4500059130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f450d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x5623673fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f450d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x5623673fd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39904/0 (socket says 172.21.15.5:39904) 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.071+0000 7f450d200640 1 -- 172.21.15.5:0/3574333632 learned_addr learned my addr 172.21.15.5:0/3574333632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f4507e00640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f4507400640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f450d200640 1 -- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 msgr2=0x5623673a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f450d200640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f450d200640 1 -- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 msgr2=0x5623673ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f450d200640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f450d200640 1 -- 172.21.15.5:0/3574333632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623673fdf10 con 0x56236736ad70 2024-04-25T10:05:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f450d200640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x5623673fd9d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f45000592f0 tx=0x7f4500009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258e9e9a56635056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f44f1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f44f1600640 1 -- 172.21.15.5:0/3574333632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500080060 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f44f1600640 1 -- 172.21.15.5:0/3574333632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45000551a0 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44f8028020 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f44f1600640 1 -- 172.21.15.5:0/3574333632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4500066050 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.072+0000 7f4512176740 10 client.6393 fetch_fsmap learned FSMap version 35 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5623673ad0d0 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f44f1600640 1 -- 172.21.15.5:0/3574333632 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f8028020 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f44f1600640 1 -- 172.21.15.5:0/3574333632 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4500058220 con 0x56236736ad70 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 10 client.6393 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 10 client.6393 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 20 client.6393 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x56236738aae0 0x5623673428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 4 client.6393 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:40.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623672b3eb0 con 0x56236738aae0 2024-04-25T10:05:40.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.073+0000 7f4507e00640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x56236738aae0 0x5623673428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4507e00640 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x56236738aae0 0x5623673428b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5107c1a664578758 server_cookie=73e66e216af8b950 in_seq=0 out_seq=0 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f44f1600640 10 client.6393 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f44f1600640 1 -- 172.21.15.5:0/3574333632 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5623672b3eb0 con 0x56236738aae0 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f44f1600640 10 client.6393 handle_command_reply: tid=0 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 client.6393 shutdown 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 msgr2=0x5623673fd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x5623673fd9d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f45000592f0 tx=0x7f4500009f30 comp rx=0 tx=0).stop 2024-04-25T10:05:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 shutdown_connections 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x56236738aae0 0x5623673428b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56236736ad70 0x5623673fd9d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562367348f10 0x5623673a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 --2- 172.21.15.5:0/3574333632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562367348b30 0x5623673ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f44f1600640 0 client.6393 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.074+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 >> 172.21.15.5:0/3574333632 conn(0x562367347fc0 msgr2=0x56236738b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 shutdown_connections 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f4512176740 1 -- 172.21.15.5:0/3574333632 wait complete. 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f4512176740 20 client.6393 trim_cache size 0 max 16384 2024-04-25T10:05:40.083 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6255, 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.7271457359999998, 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 35, 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 64, 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 64, 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 12.092549536 2024-04-25T10:05:40.084 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44e806b4d0 msgr2=0x7f44e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f450dc00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44e806b4d0 0x7f44e806d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f44f8000f60 tx=0x7f44f8009040 comp rx=0 tx=0).stop 2024-04-25T10:05:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 msgr2=0x7f45080b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f450dc00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 0x7f45080b0920 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f450000efc0 tx=0x7f450000c490 comp rx=0 tx=0).stop 2024-04-25T10:05:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 shutdown_connections 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.075+0000 7f450dc00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f450815b750 0x7f45080b0920 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.076+0000 7f450dc00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f44e806b4d0 0x7f44e806d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.076+0000 7f450dc00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4508154b80 0x7f45080ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.076+0000 7f450dc00640 1 --2- 172.21.15.5:0/468264588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45080b4bc0 0x7f45080ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.076+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 >> 172.21.15.5:0/468264588 conn(0x7f45080cc6a0 msgr2=0x7f450806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.076+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 shutdown_connections 2024-04-25T10:05:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.076+0000 7f450dc00640 1 -- 172.21.15.5:0/468264588 wait complete. 2024-04-25T10:05:40.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.292+0000 7fe578200640 1 Processor -- start 2024-04-25T10:05:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.292+0000 7fe578200640 1 -- start start 2024-04-25T10:05:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.294+0000 7fe578200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5701529e0 0x7fe5700c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.294+0000 7fe578200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5700cea60 0x7fe57006d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.294+0000 7fe578200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57015bdc0 con 0x7fe5700c6410 2024-04-25T10:05:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.294+0000 7fe578200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5700b2340 con 0x7fe5700cea60 2024-04-25T10:05:40.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.294+0000 7fe578200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57015c060 con 0x7fe5701529e0 2024-04-25T10:05:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.294+0000 7fe576200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5700cea60 0x7fe57006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe576200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5700cea60 0x7fe57006d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39914/0 (socket says 172.21.15.5:39914) 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe576200640 1 -- 172.21.15.5:0/2651990483 learned_addr learned my addr 172.21.15.5:0/2651990483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe575800640 1 --2- 172.21.15.5:0/2651990483 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5701529e0 0x7fe5700c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe576200640 1 -- 172.21.15.5:0/2651990483 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5701529e0 msgr2=0x7fe5700c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe576200640 1 --2- 172.21.15.5:0/2651990483 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5701529e0 0x7fe5700c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe576200640 1 -- 172.21.15.5:0/2651990483 >> v1:172.21.15.80:6789/0 conn(0x7fe5700c6410 legacy=0x7fe5700ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.295+0000 7fe576200640 1 -- 172.21.15.5:0/2651990483 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57015c340 con 0x7fe5700cea60 2024-04-25T10:05:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.296+0000 7fe576200640 1 --2- 172.21.15.5:0/2651990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5700cea60 0x7fe57006d150 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe568009e50 tx=0x7fe56802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c036f5cdafb2645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe56fe00640 1 -- 172.21.15.5:0/2651990483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568036070 con 0x7fe5700cea60 2024-04-25T10:05:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe56fe00640 1 -- 172.21.15.5:0/2651990483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568031600 con 0x7fe5700cea60 2024-04-25T10:05:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe56fe00640 1 -- 172.21.15.5:0/2651990483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680319d0 con 0x7fe5700cea60 2024-04-25T10:05:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe578200640 1 -- 172.21.15.5:0/2651990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5700cea60 msgr2=0x7fe57006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe578200640 1 --2- 172.21.15.5:0/2651990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5700cea60 0x7fe57006d150 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe568009e50 tx=0x7fe56802a710 comp rx=0 tx=0).stop 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe578200640 1 -- 172.21.15.5:0/2651990483 shutdown_connections 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe578200640 1 --2- 172.21.15.5:0/2651990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5700cea60 0x7fe57006d150 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe578200640 1 --2- 172.21.15.5:0/2651990483 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5701529e0 0x7fe5700c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.297+0000 7fe578200640 1 -- 172.21.15.5:0/2651990483 >> 172.21.15.5:0/2651990483 conn(0x7fe570071150 msgr2=0x7fe570071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.298+0000 7fe578200640 1 -- 172.21.15.5:0/2651990483 shutdown_connections 2024-04-25T10:05:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.298+0000 7fe578200640 1 -- 172.21.15.5:0/2651990483 wait complete. 2024-04-25T10:05:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.299+0000 7fe578200640 1 Processor -- start 2024-04-25T10:05:40.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 -- start start 2024-04-25T10:05:40.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5700c6410 0x7fe570141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5700cea60 0x7fe5700b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 0x7fe57015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe570071ed0 con 0x7fe57015eef0 2024-04-25T10:05:40.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57015c900 con 0x7fe5700c6410 2024-04-25T10:05:40.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.300+0000 7fe578200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe570072050 con 0x7fe5700cea60 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 0x7fe57015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 0x7fe57015f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39922/0 (socket says 172.21.15.5:39922) 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 -- 172.21.15.5:0/1689859776 learned_addr learned my addr 172.21.15.5:0/1689859776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 -- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5700cea60 msgr2=0x7fe5700b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5700cea60 0x7fe5700b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 -- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5700c6410 msgr2=0x7fe570141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5700c6410 0x7fe570141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.301+0000 7fe576200640 1 -- 172.21.15.5:0/1689859776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700b4350 con 0x7fe57015eef0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.302+0000 7fe576200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 0x7fe57015f380 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe568009e50 tx=0x7fe568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.302+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568036050 con 0x7fe57015eef0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.302+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568042070 con 0x7fe57015eef0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.302+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568009510 con 0x7fe57015eef0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.302+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5700b4630 con 0x7fe57015eef0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.302+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5700b4bf0 con 0x7fe57015eef0 2024-04-25T10:05:40.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.303+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe570068fd0 con 0x7fe57015eef0 2024-04-25T10:05:40.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.304+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe568007bf0 con 0x7fe57015eef0 2024-04-25T10:05:40.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.305+0000 7fe56d600640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe55006b4d0 0x7fe55006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:40.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.305+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5680ab270 con 0x7fe57015eef0 2024-04-25T10:05:40.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.306+0000 7fe575800640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe55006b4d0 0x7fe55006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:40.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.307+0000 7fe575800640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe55006b4d0 0x7fe55006d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe564006a90 tx=0x7fe564006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:40.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.309+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe568079ec0 con 0x7fe57015eef0 2024-04-25T10:05:40.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.554+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5700b4040 con 0x7fe57015eef0 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.555+0000 7fe56d600640 1 -- 172.21.15.5:0/1689859776 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5680797c0 con 0x7fe57015eef0 2024-04-25T10:05:40.560 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe55006b4d0 msgr2=0x7fe55006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe55006b4d0 0x7fe55006d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe564006a90 tx=0x7fe564006a20 comp rx=0 tx=0).stop 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 msgr2=0x7fe57015f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 0x7fe57015f380 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe568009e50 tx=0x7fe568004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 shutdown_connections 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57015eef0 0x7fe57015f380 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5700cea60 0x7fe5700b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe55006b4d0 0x7fe55006d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 --2- 172.21.15.5:0/1689859776 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5700c6410 0x7fe570141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 >> 172.21.15.5:0/1689859776 conn(0x7fe570071150 msgr2=0x7fe5700c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 shutdown_connections 2024-04-25T10:05:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:40.558+0000 7fe578200640 1 -- 172.21.15.5:0/1689859776 wait complete. 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:40.583 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:40.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.584 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:40.637 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42385sec preferred_lft 42385sec 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.638 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:40.639 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:40.640 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:05:40.640 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:40.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.641 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:40.723 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:05:40.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.727 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:05:40.781 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:05:40.781 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:40.781 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:05:40.782 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:05:40.782 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:40.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.783 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:05:40.783 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:05:40.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:05:40.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:05:40.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:05:40.783 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:05:40.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.938 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:05:40.993 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:40.993 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:05:40.993 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:40.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:40.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:05:41.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:41.084 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:41.138 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:41.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:41.138 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:41.195 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:05:41.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:41.331 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:41.332 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:41.332 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:05:41.356 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:41.356 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:41.356 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:05:41.356 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:05:41.356 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:05:41.390 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:05:41.393 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:05:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:05:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:05:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:05:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:05:41.399 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:05:41.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:41.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.479 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:05:41.542 INFO:teuthology.orchestra.run.smithi005.stdout:client5186 2024-04-25T10:05:41.543 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:05:41.543 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:41.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.595 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T10:05:41.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:41.659 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T10:05:41.720 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:05:41.721 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:05:41.721 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:05:41.721 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:05:41.721 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T10:05:41.721 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:05:41.868 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:05:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:41.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.134+0000 7fad69a00640 1 Processor -- start 2024-04-25T10:05:42.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.135+0000 7fad69a00640 1 -- start start 2024-04-25T10:05:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.136+0000 7fad69a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fad6415d7b0 0x7fad6406fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.136+0000 7fad69a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad640b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.136+0000 7fad69a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad641618f0 con 0x7fad640b53c0 2024-04-25T10:05:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.136+0000 7fad69a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad64161bd0 con 0x7fad6415d7b0 2024-04-25T10:05:42.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.136+0000 7fad69a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad64161eb0 con 0x7fad6415eba0 2024-04-25T10:05:42.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.137+0000 7fad63400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad640b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.137+0000 7fad62a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fad6415d7b0 0x7fad6406fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.137+0000 7fad63400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad640b3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:35878/0 (socket says 172.21.15.5:35878) 2024-04-25T10:05:42.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.137+0000 7fad63400640 1 -- 172.21.15.5:0/3310167358 learned_addr learned my addr 172.21.15.5:0/3310167358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:42.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.138+0000 7fad63400640 1 -- 172.21.15.5:0/3310167358 >> v2:172.21.15.80:3300/0 conn(0x7fad6415d7b0 msgr2=0x7fad6406fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.138+0000 7fad63400640 1 --2- 172.21.15.5:0/3310167358 >> v2:172.21.15.80:3300/0 conn(0x7fad6415d7b0 0x7fad6406fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.138+0000 7fad63400640 1 -- 172.21.15.5:0/3310167358 >> v1:172.21.15.80:6789/0 conn(0x7fad640b53c0 legacy=0x7fad6406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.138+0000 7fad63400640 1 -- 172.21.15.5:0/3310167358 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64162190 con 0x7fad6415eba0 2024-04-25T10:05:42.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.138+0000 7fad62a00640 1 --2- 172.21.15.5:0/3310167358 >> v2:172.21.15.80:3300/0 conn(0x7fad6415d7b0 0x7fad6406fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:42.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.139+0000 7fad63400640 1 --2- 172.21.15.5:0/3310167358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad640b3a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009e50 tx=0x7fad5c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4954e685a48244e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.140+0000 7fad61600640 1 -- 172.21.15.5:0/3310167358 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c036070 con 0x7fad6415eba0 2024-04-25T10:05:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.140+0000 7fad61600640 1 -- 172.21.15.5:0/3310167358 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c031600 con 0x7fad6415eba0 2024-04-25T10:05:42.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.140+0000 7fad61600640 1 -- 172.21.15.5:0/3310167358 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0319d0 con 0x7fad6415eba0 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 -- 172.21.15.5:0/3310167358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 msgr2=0x7fad640b3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 --2- 172.21.15.5:0/3310167358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad640b3a70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009e50 tx=0x7fad5c02a710 comp rx=0 tx=0).stop 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 -- 172.21.15.5:0/3310167358 shutdown_connections 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 --2- 172.21.15.5:0/3310167358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad640b3a70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 --2- 172.21.15.5:0/3310167358 >> v2:172.21.15.80:3300/0 conn(0x7fad6415d7b0 0x7fad6406fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 -- 172.21.15.5:0/3310167358 >> 172.21.15.5:0/3310167358 conn(0x7fad640b2390 msgr2=0x7fad640b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 -- 172.21.15.5:0/3310167358 shutdown_connections 2024-04-25T10:05:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.141+0000 7fad69a00640 1 -- 172.21.15.5:0/3310167358 wait complete. 2024-04-25T10:05:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.143+0000 7fad69a00640 1 Processor -- start 2024-04-25T10:05:42.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.143+0000 7fad69a00640 1 -- start start 2024-04-25T10:05:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad69a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad640b53c0 0x7fad64149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad69a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 0x7fad641583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad69a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad641601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad69a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6406b3f0 con 0x7fad6415d7b0 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad69a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6406b270 con 0x7fad640b53c0 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad69a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad640b5850 con 0x7fad6415eba0 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad62000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 0x7fad641583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad62000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 0x7fad641583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39938/0 (socket says 172.21.15.5:39938) 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.144+0000 7fad62000640 1 -- 172.21.15.5:0/4233520800 learned_addr learned my addr 172.21.15.5:0/4233520800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad63400640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad641601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad640b53c0 0x7fad64149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62000640 1 -- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 msgr2=0x7fad641601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62000640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad641601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62000640 1 -- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad640b53c0 msgr2=0x7fad64149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62000640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad640b53c0 0x7fad64149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62000640 1 -- 172.21.15.5:0/4233520800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6415f180 con 0x7fad6415d7b0 2024-04-25T10:05:42.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad63400640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad641601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad62000640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 0x7fad641583b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c3c0 tx=0x7fad5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500052d0 con 0x7fad6415d7b0 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad50005490 con 0x7fad6415d7b0 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.145+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6415f420 con 0x7fad6415d7b0 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.146+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50007260 con 0x7fad6415d7b0 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.146+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6415f9e0 con 0x7fad6415d7b0 2024-04-25T10:05:42.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.147+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad64068fd0 con 0x7fad6415d7b0 2024-04-25T10:05:42.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.148+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad50007400 con 0x7fad6415d7b0 2024-04-25T10:05:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.149+0000 7fad5aa00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.149+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fad50089920 con 0x7fad6415d7b0 2024-04-25T10:05:42.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.150+0000 7fad62a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.151+0000 7fad62a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad4406b4d0 0x7fad4406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fad54009560 tx=0x7fad54008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.154+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad50058560 con 0x7fad6415d7b0 2024-04-25T10:05:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.391+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad6406d2b0 con 0x7fad6415d7b0 2024-04-25T10:05:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.392+0000 7fad5aa00640 1 -- 172.21.15.5:0/4233520800 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4930 (secure 0 0 0) 0x7fad50057e60 con 0x7fad6415d7b0 2024-04-25T10:05:42.398 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:42.398 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:05:35.874224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.395+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad4406b4d0 msgr2=0x7fad4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.395+0000 7fad69a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad4406b4d0 0x7fad4406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fad54009560 tx=0x7fad54008040 comp rx=0 tx=0).stop 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.395+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 msgr2=0x7fad641583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.395+0000 7fad69a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 0x7fad641583b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c3c0 tx=0x7fad5000c880 comp rx=0 tx=0).stop 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 shutdown_connections 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad6415eba0 0x7fad641601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad6415d7b0 0x7fad641583b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 --2- 172.21.15.5:0/4233520800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad640b53c0 0x7fad64149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 >> 172.21.15.5:0/4233520800 conn(0x7fad640b2390 msgr2=0x7fad6415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 shutdown_connections 2024-04-25T10:05:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.396+0000 7fad69a00640 1 -- 172.21.15.5:0/4233520800 wait complete. 2024-04-25T10:05:42.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:05:42.666 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-04-25T10:05:42.666 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:05:42.667 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:05:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.760+0000 7f6776e0df40 1 Processor -- start 2024-04-25T10:05:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.760+0000 7f6776e0df40 1 -- start start 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6776e0df40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c8481e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6776e0df40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c84821ce0 0x564c84820340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6776e0df40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c848220c0 0x564c84794bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6776e0df40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8470add0 con 0x564c847b0710 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6776e0df40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8470ed30 con 0x564c84821ce0 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6776e0df40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c84843070 con 0x564c848220c0 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c8481e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c8481e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39946/0 (socket says 172.21.15.5:39946) 2024-04-25T10:05:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 -- 172.21.15.5:0/581400390 learned_addr learned my addr 172.21.15.5:0/581400390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6772a00640 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c84821ce0 0x564c84820340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 -- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c848220c0 msgr2=0x564c84794bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c848220c0 0x564c84794bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 -- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c84821ce0 msgr2=0x564c84820340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c84821ce0 0x564c84820340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.761+0000 7f6773400640 1 -- 172.21.15.5:0/581400390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8481d360 con 0x564c847b0710 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.762+0000 7f6773400640 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c8481e190 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6764001e30 tx=0x7f676400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab2cbfab612d1c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.762+0000 7f6771600640 1 -- 172.21.15.5:0/581400390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764012020 con 0x564c847b0710 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.762+0000 7f6771600640 1 -- 172.21.15.5:0/581400390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676400e670 con 0x564c847b0710 2024-04-25T10:05:42.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.762+0000 7f6771600640 1 -- 172.21.15.5:0/581400390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400ea80 con 0x564c847b0710 2024-04-25T10:05:42.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 -- 172.21.15.5:0/581400390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 msgr2=0x564c8481e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c8481e190 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6764001e30 tx=0x7f676400b780 comp rx=0 tx=0).stop 2024-04-25T10:05:42.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 -- 172.21.15.5:0/581400390 shutdown_connections 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c848220c0 0x564c84794bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c84821ce0 0x564c84820340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 --2- 172.21.15.5:0/581400390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c8481e190 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 -- 172.21.15.5:0/581400390 >> 172.21.15.5:0/581400390 conn(0x564c84762c60 msgr2=0x564c84793d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 -- 172.21.15.5:0/581400390 shutdown_connections 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.763+0000 7f6776e0df40 1 -- 172.21.15.5:0/581400390 wait complete. 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.764+0000 7f6776e0df40 1 Processor -- start 2024-04-25T10:05:42.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.764+0000 7f6776e0df40 1 -- start start 2024-04-25T10:05:42.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6776e0df40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c84841b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6776e0df40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c84821ce0 0x564c8481c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6776e0df40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c848220c0 0x564c848d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6776e0df40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c84795c80 con 0x564c847b0710 2024-04-25T10:05:42.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6776e0df40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c847ad710 con 0x564c848220c0 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6776e0df40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c84834510 con 0x564c84821ce0 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6773400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c84841b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6773400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c84841b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39962/0 (socket says 172.21.15.5:39962) 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.765+0000 7f6773400640 1 -- 172.21.15.5:0/546883887 learned_addr learned my addr 172.21.15.5:0/546883887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773e00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c848220c0 0x564c848d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773400640 1 -- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c84821ce0 msgr2=0x564c8481c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773400640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564c84821ce0 0x564c8481c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773400640 1 -- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c848220c0 msgr2=0x564c848d52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773400640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c848220c0 0x564c848d52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773400640 1 -- 172.21.15.5:0/546883887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8481ceb0 con 0x564c847b0710 2024-04-25T10:05:42.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773e00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564c848220c0 0x564c848d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:42.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6773400640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c84841b80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f676400bc80 tx=0x7f6764000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f676be00640 1 -- 172.21.15.5:0/546883887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764027070 con 0x564c847b0710 2024-04-25T10:05:42.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f676be00640 1 -- 172.21.15.5:0/546883887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6764017630 con 0x564c847b0710 2024-04-25T10:05:42.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564c848369c0 con 0x564c847b0710 2024-04-25T10:05:42.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f676be00640 1 -- 172.21.15.5:0/546883887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764017a00 con 0x564c847b0710 2024-04-25T10:05:42.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.766+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c84834760 con 0x564c847b0710 2024-04-25T10:05:42.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.768+0000 7f676be00640 1 -- 172.21.15.5:0/546883887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f676402a070 con 0x564c847b0710 2024-04-25T10:05:42.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.769+0000 7f676be00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6750069eb0 0x7f675006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.769+0000 7f676be00640 1 -- 172.21.15.5:0/546883887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f676408a7f0 con 0x564c847b0710 2024-04-25T10:05:42.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.770+0000 7f6776e0df40 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x564c84835240 0x564c84835660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.770+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v9 -- 0x564c84820340 con 0x564c84835240 2024-04-25T10:05:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.770+0000 7f6772a00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6750069eb0 0x7f675006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.770+0000 7f6773e00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x564c84835240 0x564c84835660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6773e00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x564c84835240 0x564c84835660 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6773e00640 1 -- 172.21.15.5:0/546883887 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f676c006040 con 0x564c84835240 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x564c84835240 msgr2=0x564c84835660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x564c84835240 0x564c84835660 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6750069eb0 msgr2=0x7f675006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6772a00640 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6750069eb0 0x7f675006c370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f675c056000 tx=0x7f675c04e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6750069eb0 0x7f675006c370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f675c056000 tx=0x7f675c04e000 comp rx=0 tx=0).stop 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 msgr2=0x564c84841b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 --2- 172.21.15.5:0/546883887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c847b0710 0x564c84841b80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f676400bc80 tx=0x7f6764000bc0 comp rx=0 tx=0).stop 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6773e00640 1 -- 172.21.15.5:0/546883887 reap_dead start 2024-04-25T10:05:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 shutdown_connections 2024-04-25T10:05:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 >> 172.21.15.5:0/546883887 conn(0x564c84762c60 msgr2=0x564c84793aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 shutdown_connections 2024-04-25T10:05:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:42.771+0000 7f6776e0df40 1 -- 172.21.15.5:0/546883887 wait complete. 2024-04-25T10:05:42.778 INFO:teuthology.orchestra.run.smithi005.stdout:Xfiledir_test 2024-04-25T10:05:42.781 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:42.781 DEBUG:teuthology.orchestra.run.smithi005:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:05:42.884 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-04-25T10:05:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "ancestors": [ 2024-04-25T10:05:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1099511627776, 2024-04-25T10:05:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "file", 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "dirino": 1, 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "dir_test", 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 10, 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "old_pools": [] 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:05:42.890 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T10:05:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.001+0000 7fb42d218f40 1 Processor -- start 2024-04-25T10:05:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.001+0000 7fb42d218f40 1 -- start start 2024-04-25T10:05:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb42d218f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4ecadb0 0x5604e4f57590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb42d218f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 0x5604e4f39300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb42d218f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4f3db20 0x5604e4f58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb42d218f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e4e27dd0 con 0x5604e4eccda0 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb42d218f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e4ecd380 con 0x5604e4ecadb0 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb42d218f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604e4f5aa20 con 0x5604e4f3db20 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 0x5604e4f39300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 0x5604e4f39300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39984/0 (socket says 172.21.15.5:39984) 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 -- 172.21.15.5:0/459307450 learned_addr learned my addr 172.21.15.5:0/459307450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 -- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4f3db20 msgr2=0x5604e4f58ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4f3db20 0x5604e4f58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb429800640 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4ecadb0 0x5604e4f57590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 -- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4ecadb0 msgr2=0x5604e4f57590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4ecadb0 0x5604e4f57590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb428e00640 1 -- 172.21.15.5:0/459307450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e4f5aca0 con 0x5604e4eccda0 2024-04-25T10:05:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.002+0000 7fb429800640 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4ecadb0 0x5604e4f57590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.003+0000 7fb428e00640 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 0x5604e4f39300 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb418006f20 tx=0x7fb41800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b25eafac035c1a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.003+0000 7fb423400640 1 -- 172.21.15.5:0/459307450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418012070 con 0x5604e4eccda0 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.003+0000 7fb423400640 1 -- 172.21.15.5:0/459307450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41800e690 con 0x5604e4eccda0 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.003+0000 7fb423400640 1 -- 172.21.15.5:0/459307450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800ea80 con 0x5604e4eccda0 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 -- 172.21.15.5:0/459307450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 msgr2=0x5604e4f39300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 0x5604e4f39300 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb418006f20 tx=0x7fb41800b780 comp rx=0 tx=0).stop 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 -- 172.21.15.5:0/459307450 shutdown_connections 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4f3db20 0x5604e4f58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4eccda0 0x5604e4f39300 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 --2- 172.21.15.5:0/459307450 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4ecadb0 0x5604e4f57590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 -- 172.21.15.5:0/459307450 >> 172.21.15.5:0/459307450 conn(0x5604e4e7fc60 msgr2=0x5604e4e80040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 -- 172.21.15.5:0/459307450 shutdown_connections 2024-04-25T10:05:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.004+0000 7fb42d218f40 1 -- 172.21.15.5:0/459307450 wait complete. 2024-04-25T10:05:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.005+0000 7fb42d218f40 1 Processor -- start 2024-04-25T10:05:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.005+0000 7fb42d218f40 1 -- start start 2024-04-25T10:05:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42d218f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4ecadb0 0x5604e4f3e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42d218f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4eccda0 0x5604e4f40ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42d218f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4f3db20 0x5604e4f532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42d218f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e4f56d90 con 0x5604e4f3db20 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42d218f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e4f56ab0 con 0x5604e4eccda0 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42d218f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604e4f57070 con 0x5604e4ecadb0 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4f3db20 0x5604e4f532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4f3db20 0x5604e4f532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39988/0 (socket says 172.21.15.5:39988) 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.006+0000 7fb42a200640 1 -- 172.21.15.5:0/416168533 learned_addr learned my addr 172.21.15.5:0/416168533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb428e00640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4eccda0 0x5604e4f40ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42a200640 1 -- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4ecadb0 msgr2=0x5604e4f3e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42a200640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5604e4ecadb0 0x5604e4f3e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42a200640 1 -- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4eccda0 msgr2=0x5604e4f40ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42a200640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4eccda0 0x5604e4f40ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42a200640 1 -- 172.21.15.5:0/416168533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e4f364c0 con 0x5604e4f3db20 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb428e00640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5604e4eccda0 0x5604e4f40ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42a200640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4f3db20 0x5604e4f532b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb42400a4f0 tx=0x7fb424045d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb422000640 1 -- 172.21.15.5:0/416168533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424064030 con 0x5604e4f3db20 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb422000640 1 -- 172.21.15.5:0/416168533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4240547f0 con 0x5604e4f3db20 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb422000640 1 -- 172.21.15.5:0/416168533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424059040 con 0x5604e4f3db20 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.007+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5604e4f361e0 con 0x5604e4f3db20 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.008+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604e4f406f0 con 0x5604e4f3db20 2024-04-25T10:05:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.010+0000 7fb422000640 1 -- 172.21.15.5:0/416168533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb424067070 con 0x5604e4f3db20 2024-04-25T10:05:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.010+0000 7fb422000640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb40c069ee0 0x7fb40c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.011+0000 7fb422000640 1 -- 172.21.15.5:0/416168533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fb4240ccdf0 con 0x5604e4f3db20 2024-04-25T10:05:43.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.011+0000 7fb42d218f40 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5604e4f537f0 0x5604e4f34370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.011+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v9 -- 0x5604e4f53c80 con 0x5604e4f537f0 2024-04-25T10:05:43.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.011+0000 7fb428e00640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5604e4f537f0 0x5604e4f34370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.011+0000 7fb429800640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb40c069ee0 0x7fb40c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.012+0000 7fb428e00640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5604e4f537f0 0x5604e4f34370 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.012+0000 7fb429800640 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb40c069ee0 0x7fb40c06c3a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb41c005b70 tx=0x7fb41c005a60 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb428e00640 1 -- 172.21.15.5:0/416168533 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v66'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fb418014030 con 0x5604e4f537f0 2024-04-25T10:05:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5604e4f537f0 msgr2=0x5604e4f34370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5604e4f537f0 0x5604e4f34370 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb40c069ee0 msgr2=0x7fb40c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb40c069ee0 0x7fb40c06c3a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb41c005b70 tx=0x7fb41c005a60 comp rx=0 tx=0).stop 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4f3db20 msgr2=0x5604e4f532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 --2- 172.21.15.5:0/416168533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e4f3db20 0x5604e4f532b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb42400a4f0 tx=0x7fb424045d20 comp rx=0 tx=0).stop 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42a200640 1 -- 172.21.15.5:0/416168533 reap_dead start 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 shutdown_connections 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 >> 172.21.15.5:0/416168533 conn(0x5604e4e7fc60 msgr2=0x5604e4e80040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 shutdown_connections 2024-04-25T10:05:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.015+0000 7fb42d218f40 1 -- 172.21.15.5:0/416168533 wait complete. 2024-04-25T10:05:43.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:43.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T10:05:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.226+0000 7f86da000640 1 Processor -- start 2024-04-25T10:05:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.226+0000 7f86da000640 1 -- start start 2024-04-25T10:05:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86da000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 0x7f86d406e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86da000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f86d41426a0 0x7f86d4142af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86da000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d4143030 con 0x7f86d406ed60 2024-04-25T10:05:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86da000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d41423f0 con 0x7f86d41426a0 2024-04-25T10:05:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86da000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40cc210 con 0x7f86d4142010 2024-04-25T10:05:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86d2a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f86d406ed60 0x7f86d40cc8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59214/0 (socket says 172.21.15.5:59214) 2024-04-25T10:05:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86d2a00640 1 -- 172.21.15.5:0/120659741 learned_addr learned my addr 172.21.15.5:0/120659741 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:05:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.228+0000 7f86d3400640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 0x7f86d406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d3e00640 1 --2- 172.21.15.5:0/120659741 >> v2:172.21.15.80:3300/0 conn(0x7f86d41426a0 0x7f86d4142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792737288 0 0) 0x7f86d4143030 con 0x7f86d406ed60 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b8003520 con 0x7f86d406ed60 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215679859 0 0) 0x7f86b8003520 con 0x7f86d406ed60 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 msgr2=0x7f86d406e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 0x7f86d406e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 >> v2:172.21.15.80:3300/0 conn(0x7f86d41426a0 msgr2=0x7f86d4142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 --2- 172.21.15.5:0/120659741 >> v2:172.21.15.80:3300/0 conn(0x7f86d41426a0 0x7f86d4142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d406d780 con 0x7f86d406ed60 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f86c00019f0 con 0x7f86d406ed60 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 >> v1:172.21.15.5:6789/0 conn(0x7f86d406ed60 legacy=0x7f86d40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 0x7f86b80039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86b8003f50 0x7f86b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.229+0000 7f86d2000640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86b8004d40 0x7f86b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41423f0 con 0x7f86b8004d40 2024-04-25T10:05:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d40cc210 con 0x7f86b8003f50 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b800ba40 con 0x7f86d4142010 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3e00640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86b8004d40 0x7f86b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d2a00640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86b8003f50 0x7f86b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3400640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 0x7f86b80039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3e00640 1 -- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 msgr2=0x7f86b80039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3e00640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d4142010 0x7f86b80039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3e00640 1 -- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86b8003f50 msgr2=0x7f86b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3e00640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86b8003f50 0x7f86b8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.230+0000 7f86d3e00640 1 -- 172.21.15.5:0/120659741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b8003520 con 0x7f86b8004d40 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.231+0000 7f86d3e00640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86b8004d40 0x7f86b80051c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009ac0 tx=0x7f86b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47ea08718c2d611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc036070 con 0x7f86b8004d40 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc031400 con 0x7f86b8004d40 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86d2000640 1 -- 172.21.15.5:0/120659741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc009690 con 0x7f86b8004d40 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86da000640 1 -- 172.21.15.5:0/120659741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86b8004d40 msgr2=0x7f86b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86da000640 1 --2- 172.21.15.5:0/120659741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86b8004d40 0x7f86b80051c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009ac0 tx=0x7f86b8005830 comp rx=0 tx=0).stop 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86d3e00640 1 -- 172.21.15.5:0/120659741 reap_dead start 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86da000640 1 -- 172.21.15.5:0/120659741 shutdown_connections 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86da000640 1 -- 172.21.15.5:0/120659741 >> 172.21.15.5:0/120659741 conn(0x7f86d4066f80 msgr2=0x7f86d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86da000640 1 -- 172.21.15.5:0/120659741 shutdown_connections 2024-04-25T10:05:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.232+0000 7f86da000640 1 -- 172.21.15.5:0/120659741 wait complete. 2024-04-25T10:05:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.234+0000 7f86da000640 1 Processor -- start 2024-04-25T10:05:43.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.234+0000 7f86da000640 1 -- start start 2024-04-25T10:05:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.235+0000 7f86da000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 0x7f86d41f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.235+0000 7f86da000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d41597d0 0x7f86d4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.235+0000 7f86da000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d41582a0 0x7f86d4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86da000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d406e020 con 0x7f86d413fdf0 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86da000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d406dd40 con 0x7f86d41597d0 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86da000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d406e300 con 0x7f86d41582a0 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 0x7f86d41f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 0x7f86d41f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40014/0 (socket says 172.21.15.5:40014) 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 -- 172.21.15.5:0/4071198968 learned_addr learned my addr 172.21.15.5:0/4071198968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d2a00640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d41597d0 0x7f86d4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3e00640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d41582a0 0x7f86d4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 -- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d41582a0 msgr2=0x7f86d4158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d41582a0 0x7f86d4158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 -- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d41597d0 msgr2=0x7f86d4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d41597d0 0x7f86d4159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3400640 1 -- 172.21.15.5:0/4071198968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4158e80 con 0x7f86d413fdf0 2024-04-25T10:05:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.236+0000 7f86d3e00640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d41582a0 0x7f86d4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.237+0000 7f86d3400640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 0x7f86d41f36a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f86c4010860 tx=0x7f86c4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.237+0000 7f86cb400640 1 -- 172.21.15.5:0/4071198968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c400f040 con 0x7f86d413fdf0 2024-04-25T10:05:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.237+0000 7f86cb400640 1 -- 172.21.15.5:0/4071198968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c400eb60 con 0x7f86d413fdf0 2024-04-25T10:05:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.237+0000 7f86cb400640 1 -- 172.21.15.5:0/4071198968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c401c400 con 0x7f86d413fdf0 2024-04-25T10:05:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.237+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d415c090 con 0x7f86d413fdf0 2024-04-25T10:05:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.237+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d415c650 con 0x7f86d413fdf0 2024-04-25T10:05:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.240+0000 7f86cb400640 1 -- 172.21.15.5:0/4071198968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86c401c5a0 con 0x7f86d413fdf0 2024-04-25T10:05:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.241+0000 7f86cb400640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86b406b4d0 0x7f86b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.241+0000 7f86cb400640 1 -- 172.21.15.5:0/4071198968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f86c408aa90 con 0x7f86d413fdf0 2024-04-25T10:05:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.241+0000 7f86dce61740 1 Processor -- start 2024-04-25T10:05:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.241+0000 7f86dce61740 1 -- start start 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86d2a00640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86b406b4d0 0x7f86b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb93a70 0x55747eb46400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86dce61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 0x55747eb75f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb76450 0x55747eb4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86dce61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747eaaf920 con 0x55747eb76b90 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747ea58da0 con 0x55747eb93a70 2024-04-25T10:05:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.242+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747eaab9c0 con 0x55747eb76450 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 0x55747eb75f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 0x55747eb75f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40018/0 (socket says 172.21.15.5:40018) 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 -- 172.21.15.5:0/656231132 learned_addr learned my addr 172.21.15.5:0/656231132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86b406b4d0 0x7f86b406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f86c0002230 tx=0x7f86c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d3400640 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb93a70 0x55747eb46400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d3e00640 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb76450 0x55747eb4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 -- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb76450 msgr2=0x55747eb4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb76450 0x55747eb4b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 -- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb93a70 msgr2=0x55747eb46400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb93a70 0x55747eb46400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d2a00640 1 -- 172.21.15.5:0/656231132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55747ea65290 con 0x55747eb76b90 2024-04-25T10:05:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.243+0000 7f86d3e00640 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb76450 0x55747eb4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.244+0000 7f86d2a00640 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 0x55747eb75f10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f86c0015460 tx=0x7f86c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9871abe33bd3822d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.244+0000 7f86bf400640 1 -- 172.21.15.5:0/656231132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0026070 con 0x55747eb76b90 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.244+0000 7f86bf400640 1 -- 172.21.15.5:0/656231132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0019070 con 0x55747eb76b90 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.244+0000 7f86bf400640 1 -- 172.21.15.5:0/656231132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c001b070 con 0x55747eb76b90 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.244+0000 7f86dce61740 1 -- 172.21.15.5:0/656231132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 msgr2=0x55747eb75f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.244+0000 7f86dce61740 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 0x55747eb75f10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f86c0015460 tx=0x7f86c0012000 comp rx=0 tx=0).stop 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 -- 172.21.15.5:0/656231132 shutdown_connections 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb76450 0x55747eb4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb76b90 0x55747eb75f10 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 --2- 172.21.15.5:0/656231132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb93a70 0x55747eb46400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 -- 172.21.15.5:0/656231132 >> 172.21.15.5:0/656231132 conn(0x55747eb48670 msgr2=0x55747eb90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 -- 172.21.15.5:0/656231132 shutdown_connections 2024-04-25T10:05:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 -- 172.21.15.5:0/656231132 wait complete. 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.245+0000 7f86dce61740 1 -- start start 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.246+0000 7f86dce61740 1 Processor -- start 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eb92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 0x55747eb91450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747eb4a820 con 0x55747eb93a70 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747ea4e260 con 0x55747eb76b90 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747eb49c10 con 0x55747eb669e0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86dce61740 10 client.0 _ll_register_callbacks cb 0x55747ea6bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb90f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40032/0 (socket says 172.21.15.5:40032) 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 -- 172.21.15.5:0/502438347 learned_addr learned my addr 172.21.15.5:0/502438347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d3400640 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eb92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 -- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 msgr2=0x55747eb91450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 0x55747eb91450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 -- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 msgr2=0x55747eb92040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eb92040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.247+0000 7f86d2a00640 1 -- 172.21.15.5:0/502438347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55747e8907b0 con 0x55747eb93a70 2024-04-25T10:05:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86d2a00640 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb90f10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f86c0002200 tx=0x7f86d415bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05955093059f846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86b3e00640 1 -- 172.21.15.5:0/502438347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0026040 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86b3e00640 1 -- 172.21.15.5:0/502438347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f86c0013db0 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86c0019070 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86b3e00640 1 -- 172.21.15.5:0/502438347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c0038050 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86dce61740 10 client.6438 fetch_fsmap learned FSMap version 35 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.248+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55747e8f0f40 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86b3e00640 1 -- 172.21.15.5:0/502438347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0019070 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86b3e00640 1 -- 172.21.15.5:0/502438347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86c003b020 con 0x55747eb93a70 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86dce61740 10 client.6438 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86dce61740 10 client.6438 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86dce61740 20 client.6438 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86dce61740 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747eb89270 0x55747eb896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86dce61740 4 client.6438 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55747eabce60 con 0x55747eb89270 2024-04-25T10:05:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.249+0000 7f86d3400640 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747eb89270 0x55747eb896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.250+0000 7f86d3400640 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747eb89270 0x55747eb896e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1cde64ec261c369 server_cookie=a8112ed15ff0bef in_seq=0 out_seq=0 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.250+0000 7f86b3e00640 10 client.6438 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.250+0000 7f86b3e00640 1 -- 172.21.15.5:0/502438347 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55747eabce60 con 0x55747eb89270 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.250+0000 7f86b3e00640 10 client.6438 handle_command_reply: tid=0 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.250+0000 7f86dce61740 1 client.6438 shutdown 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 msgr2=0x55747eb90f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb90f10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f86c0002200 tx=0x7f86d415bf10 comp rx=0 tx=0).stop 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 shutdown_connections 2024-04-25T10:05:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747eb89270 0x55747eb896e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 0x55747eb91450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb90f10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 --2- 172.21.15.5:0/502438347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eb92040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86b3e00640 0 client.6438 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 >> 172.21.15.5:0/502438347 conn(0x55747eb76450 msgr2=0x55747eb73360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 shutdown_connections 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 1 -- 172.21.15.5:0/502438347 wait complete. 2024-04-25T10:05:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.251+0000 7f86dce61740 20 client.6438 trim_cache size 0 max 16384 2024-04-25T10:05:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.269+0000 7f86dce61740 1 Processor -- start 2024-04-25T10:05:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.269+0000 7f86dce61740 1 -- start start 2024-04-25T10:05:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.270+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 0x55747eba04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.270+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eba09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.270+0000 7f86dce61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.270+0000 7f86dce61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d4201460 con 0x55747eb93a70 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.270+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d4200580 con 0x55747eb76b90 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.270+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41ff960 con 0x55747eb669e0 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40034/0 (socket says 172.21.15.5:40034) 2024-04-25T10:05:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 -- 172.21.15.5:0/574724633 learned_addr learned my addr 172.21.15.5:0/574724633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 -- 172.21.15.5:0/574724633 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 msgr2=0x55747eba04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 0x55747eba04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 -- 172.21.15.5:0/574724633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 msgr2=0x55747eba09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eba09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 -- 172.21.15.5:0/574724633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55747e7840a0 con 0x55747eb93a70 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.271+0000 7f86d3e00640 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb464a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f86cc02abe0 tx=0x7f86cc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f1fc3214034860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86bea00640 1 -- 172.21.15.5:0/574724633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc036070 con 0x55747eb93a70 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86bea00640 1 -- 172.21.15.5:0/574724633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc041070 con 0x55747eb93a70 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86bea00640 1 -- 172.21.15.5:0/574724633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc004280 con 0x55747eb93a70 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 -- 172.21.15.5:0/574724633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 msgr2=0x55747eb464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb464a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f86cc02abe0 tx=0x7f86cc02c000 comp rx=0 tx=0).stop 2024-04-25T10:05:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 -- 172.21.15.5:0/574724633 shutdown_connections 2024-04-25T10:05:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb93a70 0x55747eb464a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eba09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 --2- 172.21.15.5:0/574724633 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb669e0 0x55747eba04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.272+0000 7f86dce61740 1 -- 172.21.15.5:0/574724633 >> 172.21.15.5:0/574724633 conn(0x55747eb76450 msgr2=0x55747eb49210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.273+0000 7f86dce61740 1 -- 172.21.15.5:0/574724633 shutdown_connections 2024-04-25T10:05:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.273+0000 7f86dce61740 1 -- 172.21.15.5:0/574724633 wait complete. 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.273+0000 7f86dce61740 1 -- start start 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.273+0000 7f86dce61740 1 Processor -- start 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.274+0000 7f86dce61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 0x55747eb77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.274+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eba04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.274+0000 7f86dce61740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb93a70 0x55747eba09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86dce61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55747ebb3960 con 0x55747eb669e0 2024-04-25T10:05:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d41fff60 con 0x55747eb76b90 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86dce61740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55747eb71bb0 con 0x55747eb93a70 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86dce61740 10 client.0 _ll_register_callbacks cb 0x7f86c0024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 0x55747eb77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 0x55747eb77080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40040/0 (socket says 172.21.15.5:40040) 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 -- 172.21.15.5:0/3583115216 learned_addr learned my addr 172.21.15.5:0/3583115216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 -- 172.21.15.5:0/3583115216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb93a70 msgr2=0x55747eba09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb93a70 0x55747eba09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 -- 172.21.15.5:0/3583115216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 msgr2=0x55747eba04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eba04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 -- 172.21.15.5:0/3583115216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55747eb73720 con 0x55747eb669e0 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86d3400640 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 0x55747eb77080 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f86c40554d0 tx=0x7f86c405d110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae4d4de1f152a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.275+0000 7f86b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86b3e00640 1 -- 172.21.15.5:0/3583115216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4021040 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86b3e00640 1 -- 172.21.15.5:0/3583115216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f86c40556e0 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86cc041070 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86b3e00640 1 -- 172.21.15.5:0/3583115216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c4061240 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 10 client.6444 fetch_fsmap learned FSMap version 35 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55747ec07d10 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86b3e00640 1 -- 172.21.15.5:0/3583115216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc041070 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86b3e00640 1 -- 172.21.15.5:0/3583115216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f86c405b050 con 0x55747eb669e0 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 10 client.6444 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:05:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 10 client.6444 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 20 client.6444 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747ebfaf50 0x55747ec04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 4 client.6444 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T10:05:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.276+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55747eb8a0b0 con 0x55747ebfaf50 2024-04-25T10:05:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.277+0000 7f86d2a00640 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747ebfaf50 0x55747ec04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.278+0000 7f86b3e00640 10 client.6444 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.278+0000 7f86d2a00640 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747ebfaf50 0x55747ec04760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d83b58a2e236bf server_cookie=7fd6fe38783e3135 in_seq=0 out_seq=0 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86b3e00640 1 -- 172.21.15.5:0/3583115216 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55747eb8a0b0 con 0x55747ebfaf50 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86b3e00640 10 client.6444 handle_command_reply: tid=0 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 client.6444 shutdown 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 msgr2=0x55747eb77080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 0x55747eb77080 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f86c40554d0 tx=0x7f86c405d110 comp rx=0 tx=0).stop 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 shutdown_connections 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55747ebfaf50 0x55747ec04760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55747eb93a70 0x55747eba09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55747eb76b90 0x55747eba04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 --2- 172.21.15.5:0/3583115216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747eb669e0 0x55747eb77080 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 >> 172.21.15.5:0/3583115216 conn(0x55747eb76450 msgr2=0x55747eb92b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86b3e00640 0 client.6444 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.288 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:05:43.279+0000 7fcdff600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5186=0-4194304@1} caps={5186=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5186 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x556f993c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 shutdown_connections 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 1 -- 172.21.15.5:0/3583115216 wait complete. 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.279+0000 7f86dce61740 20 client.6444 trim_cache size 0 max 16384 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "b444d278-3a8b-4cda-a6ce-ec5457f2c56b", 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86b406b4d0 msgr2=0x7f86b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86b406b4d0 0x7f86b406d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f86c0002230 tx=0x7f86c0009040 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 msgr2=0x7f86d41f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 0x7f86d41f36a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f86c4010860 tx=0x7f86c4010d20 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 shutdown_connections 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86d41582a0 0x7f86d4158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86b406b4d0 0x7f86b406d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86d41597d0 0x7f86d4159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 --2- 172.21.15.5:0/4071198968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86d413fdf0 0x7f86d41f36a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 >> 172.21.15.5:0/4071198968 conn(0x7f86d4066f80 msgr2=0x7f86d40ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 shutdown_connections 2024-04-25T10:05:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.280+0000 7f86da000640 1 -- 172.21.15.5:0/4071198968 wait complete. 2024-04-25T10:05:43.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:43.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:05:43.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.514+0000 7f512ec00640 1 Processor -- start 2024-04-25T10:05:43.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.514+0000 7f512ec00640 1 -- start start 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.516+0000 7f512ec00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5128143d10 0x7f512815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.516+0000 7f512ec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51280adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.516+0000 7f512ec00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512815f5e0 con 0x7f51280adac0 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.516+0000 7f512ec00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512806d250 con 0x7f5128143d10 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.516+0000 7f512ec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51280ae680 con 0x7f5128071710 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.516+0000 7f512d800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5128143d10 0x7f512815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512d800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5128143d10 0x7f512815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40052/0 (socket says 172.21.15.5:40052) 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512d800640 1 -- 172.21.15.5:0/3145824 learned_addr learned my addr 172.21.15.5:0/3145824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512e200640 1 --2- 172.21.15.5:0/3145824 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51280adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512d800640 1 -- 172.21.15.5:0/3145824 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 msgr2=0x7f51280adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512d800640 1 --2- 172.21.15.5:0/3145824 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51280adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512d800640 1 -- 172.21.15.5:0/3145824 >> v1:172.21.15.80:6789/0 conn(0x7f51280adac0 legacy=0x7f5128071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.517+0000 7f512d800640 1 -- 172.21.15.5:0/3145824 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280ae960 con 0x7f5128143d10 2024-04-25T10:05:43.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.519+0000 7f512d800640 1 --2- 172.21.15.5:0/3145824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815f0a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f511c009870 tx=0x7f511c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7373d507a5802956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.519+0000 7f5127e00640 1 -- 172.21.15.5:0/3145824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c037070 con 0x7f5128143d10 2024-04-25T10:05:43.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.519+0000 7f5127e00640 1 -- 172.21.15.5:0/3145824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c032440 con 0x7f5128143d10 2024-04-25T10:05:43.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.519+0000 7f5127e00640 1 -- 172.21.15.5:0/3145824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c032810 con 0x7f5128143d10 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 -- 172.21.15.5:0/3145824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 msgr2=0x7f512815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 --2- 172.21.15.5:0/3145824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815f0a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f511c009870 tx=0x7f511c02a270 comp rx=0 tx=0).stop 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 -- 172.21.15.5:0/3145824 shutdown_connections 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 --2- 172.21.15.5:0/3145824 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51280adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 --2- 172.21.15.5:0/3145824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815f0a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 -- 172.21.15.5:0/3145824 >> 172.21.15.5:0/3145824 conn(0x7f5128066f20 msgr2=0x7f512806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 -- 172.21.15.5:0/3145824 shutdown_connections 2024-04-25T10:05:43.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.520+0000 7f512ec00640 1 -- 172.21.15.5:0/3145824 wait complete. 2024-04-25T10:05:43.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.522+0000 7f512ec00640 1 Processor -- start 2024-04-25T10:05:43.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.523+0000 7f512ec00640 1 -- start start 2024-04-25T10:05:43.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512ec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51281f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512ec00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51280adac0 0x7f512813fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281440f0 con 0x7f5128143d10 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512ec00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512806ea40 con 0x7f51280adac0 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512ec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512806d6e0 con 0x7f5128071710 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40060/0 (socket says 172.21.15.5:40060) 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.524+0000 7f512e200640 1 -- 172.21.15.5:0/1291676089 learned_addr learned my addr 172.21.15.5:0/1291676089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512d800640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51281f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512ce00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51280adac0 0x7f512813fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512e200640 1 -- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 msgr2=0x7f51281f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512e200640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51281f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512e200640 1 -- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51280adac0 msgr2=0x7f512813fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512e200640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51280adac0 0x7f512813fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512e200640 1 -- 172.21.15.5:0/1291676089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128140820 con 0x7f5128143d10 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f512e200640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815e880 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5120007de0 tx=0x7f512000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.525+0000 7f5125600640 1 -- 172.21.15.5:0/1291676089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120014620 con 0x7f5128143d10 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.526+0000 7f5125600640 1 -- 172.21.15.5:0/1291676089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120019070 con 0x7f5128143d10 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.526+0000 7f5125600640 1 -- 172.21.15.5:0/1291676089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120014a20 con 0x7f5128143d10 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.526+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51281f43e0 con 0x7f5128143d10 2024-04-25T10:05:43.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.526+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5128141880 con 0x7f5128143d10 2024-04-25T10:05:43.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.528+0000 7f5125600640 1 -- 172.21.15.5:0/1291676089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5120014bc0 con 0x7f5128143d10 2024-04-25T10:05:43.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.529+0000 7f5125600640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f510806b4d0 0x7f510806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.529+0000 7f5125600640 1 -- 172.21.15.5:0/1291676089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f512008acd0 con 0x7f5128143d10 2024-04-25T10:05:43.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.530+0000 7f5132f82740 1 Processor -- start 2024-04-25T10:05:43.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.530+0000 7f512d800640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f510806b4d0 0x7f510806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.530+0000 7f5132f82740 1 -- start start 2024-04-25T10:05:43.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa3aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f5132f82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 0x55daefa39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefa63cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f5132f82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daef99cc70 con 0x55daefa75440 2024-04-25T10:05:43.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daef945cf0 con 0x55daefa63830 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55daef998d10 con 0x55daefa38d70 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f512ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 0x55daefa39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f512ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 0x55daefa39150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40068/0 (socket says 172.21.15.5:40068) 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f512ce00640 1 -- 172.21.15.5:0/505568241 learned_addr learned my addr 172.21.15.5:0/505568241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f512d800640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f510806b4d0 0x7f510806d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f511c034000 tx=0x7f511c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f512d800640 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa3aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.531+0000 7f512e200640 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefa63cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512ce00640 1 -- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 msgr2=0x55daefa3aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512ce00640 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa3aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512ce00640 1 -- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 msgr2=0x55daefa63cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512ce00640 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefa63cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512ce00640 1 -- 172.21.15.5:0/505568241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daef77b7d0 con 0x55daefa75440 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512e200640 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefa63cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f512ce00640 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 0x55daefa39150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5114001570 tx=0x7f5114001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c431515df2faf2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f511a000640 1 -- 172.21.15.5:0/505568241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114019070 con 0x55daefa75440 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f511a000640 1 -- 172.21.15.5:0/505568241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51140050d0 con 0x55daefa75440 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.532+0000 7f511a000640 1 -- 172.21.15.5:0/505568241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51140054a0 con 0x55daefa75440 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 -- 172.21.15.5:0/505568241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 msgr2=0x55daefa39150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 0x55daefa39150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5114001570 tx=0x7f5114001a30 comp rx=0 tx=0).stop 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 -- 172.21.15.5:0/505568241 shutdown_connections 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefa63cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa75440 0x55daefa39150 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 --2- 172.21.15.5:0/505568241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa3aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 -- 172.21.15.5:0/505568241 >> 172.21.15.5:0/505568241 conn(0x55daefa00a30 msgr2=0x55daefa7e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 -- 172.21.15.5:0/505568241 shutdown_connections 2024-04-25T10:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.533+0000 7f5132f82740 1 -- 172.21.15.5:0/505568241 wait complete. 2024-04-25T10:05:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.534+0000 7f5132f82740 1 -- start start 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.534+0000 7f5132f82740 1 Processor -- start 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa79750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daef93b3a0 con 0x55daefa63830 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daefa61380 con 0x55daefa75440 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55daefa8ec40 con 0x55daefa38d70 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.535+0000 7f5132f82740 10 client.0 _ll_register_callbacks cb 0x55daef958b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa79750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa79750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40072/0 (socket says 172.21.15.5:40072) 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 -- 172.21.15.5:0/1786563188 learned_addr learned my addr 172.21.15.5:0/1786563188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512d800640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512e200640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 -- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 msgr2=0x55daefa605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 -- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 msgr2=0x55daefa81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 -- 172.21.15.5:0/1786563188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daef6b13f0 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f512ce00640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa79750 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5114001540 tx=0x7f5114000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a1acda88306e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.536+0000 7f510ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f510ea00640 1 -- 172.21.15.5:0/1786563188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114024060 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f510ea00640 1 -- 172.21.15.5:0/1786563188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f51140069d0 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51140050d0 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f510ea00640 1 -- 172.21.15.5:0/1786563188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5114019040 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f5132f82740 10 client.6456 fetch_fsmap learned FSMap version 35 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55daef6710f0 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f510ea00640 1 -- 172.21.15.5:0/1786563188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51140050d0 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.537+0000 7f510ea00640 1 -- 172.21.15.5:0/1786563188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f511402e080 con 0x55daefa63830 2024-04-25T10:05:43.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f5132f82740 10 client.6456 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f5132f82740 10 client.6456 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f5132f82740 20 client.6456 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f5132f82740 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa81dd0 0x55daefa664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f5132f82740 4 client.6456 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daef9aa1b0 con 0x55daefa81dd0 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.538+0000 7f512d800640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa81dd0 0x55daefa664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.539+0000 7f512d800640 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa81dd0 0x55daefa664c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=957cdc579adadcac server_cookie=4cd2d4ae97c81b57 in_seq=0 out_seq=0 2024-04-25T10:05:43.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.539+0000 7f510ea00640 10 client.6456 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.539+0000 7f510ea00640 1 -- 172.21.15.5:0/1786563188 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55daef9aa1b0 con 0x55daefa81dd0 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.539+0000 7f510ea00640 10 client.6456 handle_command_reply: tid=0 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.539+0000 7f5132f82740 1 client.6456 shutdown 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 msgr2=0x55daefa79750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa79750 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5114001540 tx=0x7f5114000bc0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 shutdown_connections 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa81dd0 0x55daefa664c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa79750 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 --2- 172.21.15.5:0/1786563188 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f510ea00640 0 client.6456 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 >> 172.21.15.5:0/1786563188 conn(0x55daefa00a30 msgr2=0x55daefa61530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 shutdown_connections 2024-04-25T10:05:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 1 -- 172.21.15.5:0/1786563188 wait complete. 2024-04-25T10:05:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.540+0000 7f5132f82740 20 client.6456 trim_cache size 0 max 16384 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.557+0000 7f5132f82740 1 Processor -- start 2024-04-25T10:05:43.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.557+0000 7f5132f82740 1 -- start start 2024-04-25T10:05:43.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa7a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f5132f82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa7d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f5132f82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daefa33ab0 con 0x55daefa63830 2024-04-25T10:05:43.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5128200440 con 0x55daefa75440 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51281ff560 con 0x55daefa38d70 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40084/0 (socket says 172.21.15.5:40084) 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512ce00640 1 -- 172.21.15.5:0/1448040908 learned_addr learned my addr 172.21.15.5:0/1448040908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512e200640 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa7d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512d800640 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa7a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512ce00640 1 -- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 msgr2=0x55daefa7a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512ce00640 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa7a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.558+0000 7f512ce00640 1 -- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 msgr2=0x55daefa7d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.559+0000 7f512ce00640 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa7d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.559+0000 7f512ce00640 1 -- 172.21.15.5:0/1448040908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daefaa1210 con 0x55daefa63830 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.559+0000 7f512ce00640 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa605e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f511401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a743cc220b381b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.559+0000 7f5119600640 1 -- 172.21.15.5:0/1448040908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114031070 con 0x55daefa63830 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.559+0000 7f5119600640 1 -- 172.21.15.5:0/1448040908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51140050d0 con 0x55daefa63830 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.559+0000 7f5119600640 1 -- 172.21.15.5:0/1448040908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114005490 con 0x55daefa63830 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 -- 172.21.15.5:0/1448040908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 msgr2=0x55daefa605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa605e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f511401d000 comp rx=0 tx=0).stop 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 -- 172.21.15.5:0/1448040908 shutdown_connections 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa75440 0x55daefa7d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa63830 0x55daefa605e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 --2- 172.21.15.5:0/1448040908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa38d70 0x55daefa7a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 -- 172.21.15.5:0/1448040908 >> 172.21.15.5:0/1448040908 conn(0x55daefa00a30 msgr2=0x55daefa5f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 -- 172.21.15.5:0/1448040908 shutdown_connections 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.560+0000 7f5132f82740 1 -- 172.21.15.5:0/1448040908 wait complete. 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.561+0000 7f5132f82740 1 -- start start 2024-04-25T10:05:43.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.561+0000 7f5132f82740 1 Processor -- start 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 0x55daefa7e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefae7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa75440 0x55daefae81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281fef60 con 0x55daefa38d70 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281f7800 con 0x55daefa63830 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55daefa980f0 con 0x55daefa75440 2024-04-25T10:05:43.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f5132f82740 10 client.0 _ll_register_callbacks cb 0x7f5114017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 0x55daefa7e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 0x55daefa7e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40090/0 (socket says 172.21.15.5:40090) 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 -- 172.21.15.5:0/2759248588 learned_addr learned my addr 172.21.15.5:0/2759248588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512ce00640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefae7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512e200640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa75440 0x55daefae81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 -- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa75440 msgr2=0x55daefae81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa75440 0x55daefae81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 -- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 msgr2=0x55daefae7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefae7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.562+0000 7f512d800640 1 -- 172.21.15.5:0/2759248588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daefa9cfc0 con 0x55daefa38d70 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f512ce00640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefae7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f512d800640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 0x55daefa7e1e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f511c035040 tx=0x7f511c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a843661320db4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f510ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:05:43.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f510ea00640 1 -- 172.21.15.5:0/2759248588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c057070 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f510ea00640 1 -- 172.21.15.5:0/2759248588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f511c04a450 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51140050d0 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f510ea00640 1 -- 172.21.15.5:0/2759248588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f511c062050 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.563+0000 7f5132f82740 10 client.6462 fetch_fsmap learned FSMap version 35 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55daefaa08a0 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f510ea00640 1 -- 172.21.15.5:0/2759248588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51140050d0 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f510ea00640 1 -- 172.21.15.5:0/2759248588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f511c065020 con 0x55daefa38d70 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 10 client.6462 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 10 client.6462 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 20 client.6462 populate_metadata read hostname 'smithi005' 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa8cef0 0x55daefa378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 4 client.6462 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55daef9a4500 con 0x55daefa8cef0 2024-04-25T10:05:43.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.564+0000 7f512ce00640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa8cef0 0x55daefa378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f512ce00640 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa8cef0 0x55daefa378d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a824bf6f6affd5 server_cookie=d1ed13d57fb16409 in_seq=0 out_seq=0 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f510ea00640 10 client.6462 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f510ea00640 1 -- 172.21.15.5:0/2759248588 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55daef9a4500 con 0x55daefa8cef0 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f510ea00640 10 client.6462 handle_command_reply: tid=0 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f5132f82740 1 client.6462 shutdown 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 msgr2=0x55daefa7e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f5132f82740 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 0x55daefa7e1e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f511c035040 tx=0x7f511c02c070 comp rx=0 tx=0).stop 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 shutdown_connections 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.565+0000 7f5132f82740 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55daefa8cef0 0x55daefa378d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55daefa75440 0x55daefae81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55daefa63830 0x55daefae7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f510ea00640 0 client.6462 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:05:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 1 --2- 172.21.15.5:0/2759248588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daefa38d70 0x55daefa7e1e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 >> 172.21.15.5:0/2759248588 conn(0x55daefa00a30 msgr2=0x55daefa605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 shutdown_connections 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 1 -- 172.21.15.5:0/2759248588 wait complete. 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f5132f82740 20 client.6462 trim_cache size 0 max 16384 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stdout: "b444d278-3a8b-4cda-a6ce-ec5457f2c56b": { 2024-04-25T10:05:43.574 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-25T10:05:43.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "b444d278-3a8b-4cda-a6ce-ec5457f2c56b", 2024-04-25T10:05:43.575 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive" 2024-04-25T10:05:43.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:43.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:05:43.575 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:05:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f510806b4d0 msgr2=0x7f510806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f512ec00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f510806b4d0 0x7f510806d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f511c034000 tx=0x7f511c02e000 comp rx=0 tx=0).stop 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 msgr2=0x7f512815e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.566+0000 7f512ec00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815e880 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5120007de0 tx=0x7f512000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 shutdown_connections 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5128143d10 0x7f512815e880 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f510806b4d0 0x7f510806d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51280adac0 0x7f512813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 --2- 172.21.15.5:0/1291676089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5128071710 0x7f51281f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 >> 172.21.15.5:0/1291676089 conn(0x7f5128066f20 msgr2=0x7f51280b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 shutdown_connections 2024-04-25T10:05:43.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:05:43.567+0000 7f512ec00640 1 -- 172.21.15.5:0/1291676089 wait complete. 2024-04-25T10:05:43.590 INFO:tasks.cephfs.filesystem:scrub status for tag:b444d278-3a8b-4cda-a6ce-ec5457f2c56b - {'path': '/', 'tag': 'b444d278-3a8b-4cda-a6ce-ec5457f2c56b', 'options': 'recursive'} 2024-04-25T10:05:49.457 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:05:49.452+0000 7f92a8200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:05:54.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:54.222 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:13.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:13.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:06:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.803+0000 7f0a3ca00640 1 Processor -- start 2024-04-25T10:06:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.804+0000 7f0a3ca00640 1 -- start start 2024-04-25T10:06:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.805+0000 7f0a3ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0a380b48c0 0x7f0a3815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.805+0000 7f0a3ca00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a381606e0 con 0x7f0a380d3f90 2024-04-25T10:06:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.805+0000 7f0a3ca00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a381609c0 con 0x7f0a38161d90 2024-04-25T10:06:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.805+0000 7f0a3ca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a38160ca0 con 0x7f0a380b48c0 2024-04-25T10:06:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.806+0000 7f0a35e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0a380d3f90 0x7f0a38161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44126/0 (socket says 172.21.15.5:44126) 2024-04-25T10:06:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.806+0000 7f0a35e00640 1 -- 172.21.15.5:0/831994222 learned_addr learned my addr 172.21.15.5:0/831994222 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.806+0000 7f0a36800640 1 --2- 172.21.15.5:0/831994222 >> v2:172.21.15.80:3300/0 conn(0x7f0a380b48c0 0x7f0a3815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.806+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142599275 0 0) 0x7f0a381606e0 con 0x7f0a380d3f90 2024-04-25T10:06:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.806+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a1c003520 con 0x7f0a380d3f90 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 736100094 0 0) 0x7f0a1c003520 con 0x7f0a380d3f90 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 >> v2:172.21.15.80:3300/0 conn(0x7f0a380b48c0 msgr2=0x7f0a3815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 --2- 172.21.15.5:0/831994222 >> v2:172.21.15.80:3300/0 conn(0x7f0a380b48c0 0x7f0a3815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 >> v1:172.21.15.80:6789/0 conn(0x7f0a38161d90 legacy=0x7f0a380b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38160f80 con 0x7f0a380d3f90 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0a200017f0 con 0x7f0a380d3f90 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 >> v1:172.21.15.5:6789/0 conn(0x7f0a380d3f90 legacy=0x7f0a38161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c0039b0 0x7f0a1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a1c004800 0x7f0a1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a1c0051c0 0x7f0a1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38160ca0 con 0x7f0a1c004800 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c00c160 con 0x7f0a1c0051c0 2024-04-25T10:06:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c00c360 con 0x7f0a1c0039b0 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.807+0000 7f0a35e00640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a1c004800 0x7f0a1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a36800640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c0039b0 0x7f0a1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a37200640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a1c0051c0 0x7f0a1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a35e00640 1 -- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c0039b0 msgr2=0x7f0a1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a35e00640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c0039b0 0x7f0a1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a35e00640 1 -- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a1c0051c0 msgr2=0x7f0a1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a35e00640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a1c0051c0 0x7f0a1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a35e00640 1 -- 172.21.15.5:0/831994222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c003520 con 0x7f0a1c004800 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a37200640 1 -- 172.21.15.5:0/831994222 reap_dead start 2024-04-25T10:06:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.808+0000 7f0a36800640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c0039b0 0x7f0a1c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.809+0000 7f0a35e00640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a1c004800 0x7f0a1c004c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c240 tx=0x7f0a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec724406d1db03b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.809+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20007ce0 con 0x7f0a1c004800 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.809+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20007070 con 0x7f0a1c004800 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.809+0000 7f0a35400640 1 -- 172.21.15.5:0/831994222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200074c0 con 0x7f0a1c004800 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 -- 172.21.15.5:0/831994222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a1c004800 msgr2=0x7f0a1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a1c004800 0x7f0a1c004c80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c240 tx=0x7f0a1c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 -- 172.21.15.5:0/831994222 shutdown_connections 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/831994222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a1c004800 0x7f0a1c004c80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 -- 172.21.15.5:0/831994222 >> 172.21.15.5:0/831994222 conn(0x7f0a3808af80 msgr2=0x7f0a380d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 -- 172.21.15.5:0/831994222 shutdown_connections 2024-04-25T10:06:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.810+0000 7f0a3ca00640 1 -- 172.21.15.5:0/831994222 wait complete. 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.812+0000 7f0a3ca00640 1 Processor -- start 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.812+0000 7f0a3ca00640 1 -- start start 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a3ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c004800 0x7f0a381541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a3ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a380cb180 0x7f0a380c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a3ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 0x7f0a380ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a3ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38161680 con 0x7f0a380cc770 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a3ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b4ca0 con 0x7f0a380cb180 2024-04-25T10:06:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a3ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38161800 con 0x7f0a1c004800 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a37200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 0x7f0a380ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a37200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 0x7f0a380ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35368/0 (socket says 172.21.15.5:35368) 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.813+0000 7f0a37200640 1 -- 172.21.15.5:0/3402456502 learned_addr learned my addr 172.21.15.5:0/3402456502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a35e00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a380cb180 0x7f0a380c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a36800640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c004800 0x7f0a381541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a37200640 1 -- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c004800 msgr2=0x7f0a381541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a37200640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c004800 0x7f0a381541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a37200640 1 -- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a380cb180 msgr2=0x7f0a380c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a37200640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a380cb180 0x7f0a380c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a37200640 1 -- 172.21.15.5:0/3402456502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38154700 con 0x7f0a380cc770 2024-04-25T10:06:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a37200640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 0x7f0a380ccbf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0a30007fd0 tx=0x7f0a3000e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.814+0000 7f0a2ea00640 1 -- 172.21.15.5:0/3402456502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30019070 con 0x7f0a380cc770 2024-04-25T10:06:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.815+0000 7f0a2ea00640 1 -- 172.21.15.5:0/3402456502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a30005280 con 0x7f0a380cc770 2024-04-25T10:06:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.815+0000 7f0a2ea00640 1 -- 172.21.15.5:0/3402456502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3000eea0 con 0x7f0a380cc770 2024-04-25T10:06:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.815+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a380b6540 con 0x7f0a380cc770 2024-04-25T10:06:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.815+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a380cb9b0 con 0x7f0a380cc770 2024-04-25T10:06:13.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.817+0000 7f0a2ea00640 1 -- 172.21.15.5:0/3402456502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a30007250 con 0x7f0a380cc770 2024-04-25T10:06:13.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.818+0000 7f0a2ea00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a1406b4d0 0x7f0a1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.818+0000 7f0a2ea00640 1 -- 172.21.15.5:0/3402456502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f0a300897b0 con 0x7f0a380cc770 2024-04-25T10:06:13.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.819+0000 7f0a36800640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a1406b4d0 0x7f0a1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.819+0000 7f0a403af740 1 Processor -- start 2024-04-25T10:06:13.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.819+0000 7f0a403af740 1 -- start start 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b959bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a403af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 0x5582b959a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95c4830 0x5582b95c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a403af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b94fdc70 con 0x5582b95d6440 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b94a6cf0 con 0x5582b9599d70 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582b94f9d10 con 0x5582b95c4830 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a36800640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a1406b4d0 0x7f0a1406d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a38050700 tx=0x7f0a2800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 0x5582b959a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 0x5582b959a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35372/0 (socket says 172.21.15.5:35372) 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a35e00640 1 -- 172.21.15.5:0/2157786842 learned_addr learned my addr 172.21.15.5:0/2157786842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a36800640 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b959bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.820+0000 7f0a37200640 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95c4830 0x5582b95c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a35e00640 1 -- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95c4830 msgr2=0x5582b95c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a35e00640 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95c4830 0x5582b95c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a35e00640 1 -- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 msgr2=0x5582b959bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a35e00640 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b959bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a35e00640 1 -- 172.21.15.5:0/2157786842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582b92dc7d0 con 0x5582b95d6440 2024-04-25T10:06:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a35e00640 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 0x5582b959a150 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0a20000ef0 tx=0x7f0a20007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71487722d3eb1590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a27000640 1 -- 172.21.15.5:0/2157786842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200015f0 con 0x5582b95d6440 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a27000640 1 -- 172.21.15.5:0/2157786842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20001790 con 0x5582b95d6440 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.821+0000 7f0a27000640 1 -- 172.21.15.5:0/2157786842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20007070 con 0x5582b95d6440 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 -- 172.21.15.5:0/2157786842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 msgr2=0x5582b959a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 0x5582b959a150 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0a20000ef0 tx=0x7f0a20007000 comp rx=0 tx=0).stop 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 -- 172.21.15.5:0/2157786842 shutdown_connections 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95c4830 0x5582b95c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95d6440 0x5582b959a150 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 --2- 172.21.15.5:0/2157786842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b959bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 -- 172.21.15.5:0/2157786842 >> 172.21.15.5:0/2157786842 conn(0x5582b9561a30 msgr2=0x5582b95df830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 -- 172.21.15.5:0/2157786842 shutdown_connections 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.822+0000 7f0a403af740 1 -- 172.21.15.5:0/2157786842 wait complete. 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.823+0000 7f0a403af740 1 -- start start 2024-04-25T10:06:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.823+0000 7f0a403af740 1 Processor -- start 2024-04-25T10:06:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b9599d70 0x5582b95c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b95d6440 0x5582b95ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b949c3a0 con 0x5582b95c4830 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b95c2380 con 0x5582b95d6440 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582b95effc0 con 0x5582b9599d70 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.824+0000 7f0a403af740 10 client.0 _ll_register_callbacks cb 0x5582b94b9b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35378/0 (socket says 172.21.15.5:35378) 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 -- 172.21.15.5:0/15672795 learned_addr learned my addr 172.21.15.5:0/15672795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a37200640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b95d6440 0x5582b95ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a36800640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b9599d70 0x5582b95c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 -- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b9599d70 msgr2=0x5582b95c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b9599d70 0x5582b95c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 -- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b95d6440 msgr2=0x5582b95ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b95d6440 0x5582b95ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 -- 172.21.15.5:0/15672795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582b92123f0 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a35e00640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95c62c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001af0 tx=0x7f0a38160b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c1ef5eb38e8d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.825+0000 7f0a1b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a1b400640 1 -- 172.21.15.5:0/15672795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200015f0 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a1b400640 1 -- 172.21.15.5:0/15672795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f0a20038070 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a20001790 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a1b400640 1 -- 172.21.15.5:0/15672795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2003e650 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a403af740 10 client.6474 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5582b91d20f0 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a1b400640 1 -- 172.21.15.5:0/15672795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a20001790 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a1b400640 1 -- 172.21.15.5:0/15672795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a2003e7f0 con 0x5582b95c4830 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a403af740 10 client.6474 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a403af740 10 client.6474 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.826+0000 7f0a403af740 20 client.6474 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.827+0000 7f0a403af740 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b95ef9c0 0x5582b95b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.827+0000 7f0a403af740 4 client.6474 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.827+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582b950b1b0 con 0x5582b95ef9c0 2024-04-25T10:06:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.827+0000 7f0a36800640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b95ef9c0 0x5582b95b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.827+0000 7f0a36800640 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b95ef9c0 0x5582b95b4c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=788590f2afca031d server_cookie=fb1d2d8104bbfe01 in_seq=0 out_seq=0 2024-04-25T10:06:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.827+0000 7f0a1b400640 10 client.6474 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.828+0000 7f0a1b400640 1 -- 172.21.15.5:0/15672795 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582b950b1b0 con 0x5582b95ef9c0 2024-04-25T10:06:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.828+0000 7f0a1b400640 10 client.6474 handle_command_reply: tid=0 2024-04-25T10:06:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.828+0000 7f0a403af740 1 client.6474 shutdown 2024-04-25T10:06:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.828+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 msgr2=0x5582b95c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.828+0000 7f0a403af740 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95c62c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001af0 tx=0x7f0a38160b90 comp rx=0 tx=0).stop 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 shutdown_connections 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b95ef9c0 0x5582b95b4c40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b95d6440 0x5582b95ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95c62c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 --2- 172.21.15.5:0/15672795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b9599d70 0x5582b95c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a1b400640 0 client.6474 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 >> 172.21.15.5:0/15672795 conn(0x5582b9561a30 msgr2=0x5582b95e2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 shutdown_connections 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 1 -- 172.21.15.5:0/15672795 wait complete. 2024-04-25T10:06:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.829+0000 7f0a403af740 20 client.6474 trim_cache size 0 max 16384 2024-04-25T10:06:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.846+0000 7f0a403af740 1 Processor -- start 2024-04-25T10:06:13.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.846+0000 7f0a403af740 1 -- start start 2024-04-25T10:06:13.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.846+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.846+0000 7f0a403af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a403af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38200d60 con 0x5582b95c4830 2024-04-25T10:06:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381ffe80 con 0x5582b9599d70 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381ff220 con 0x5582b95d6440 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35384/0 (socket says 172.21.15.5:35384) 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 -- 172.21.15.5:0/1727566981 learned_addr learned my addr 172.21.15.5:0/1727566981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 -- 172.21.15.5:0/1727566981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 msgr2=0x5582b95eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 -- 172.21.15.5:0/1727566981 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 msgr2=0x5582b95c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.847+0000 7f0a35e00640 1 -- 172.21.15.5:0/1727566981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582b96025e0 con 0x5582b95c4830 2024-04-25T10:06:13.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.848+0000 7f0a35e00640 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001e10 tx=0x7f0a20038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fdaa769f90b098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.848+0000 7f0a26600640 1 -- 172.21.15.5:0/1727566981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2003e970 con 0x5582b95c4830 2024-04-25T10:06:13.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.848+0000 7f0a26600640 1 -- 172.21.15.5:0/1727566981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a200015f0 con 0x5582b95c4830 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.848+0000 7f0a26600640 1 -- 172.21.15.5:0/1727566981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20033420 con 0x5582b95c4830 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.848+0000 7f0a403af740 1 -- 172.21.15.5:0/1727566981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 msgr2=0x5582b95ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.848+0000 7f0a403af740 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001e10 tx=0x7f0a20038000 comp rx=0 tx=0).stop 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 -- 172.21.15.5:0/1727566981 shutdown_connections 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 --2- 172.21.15.5:0/1727566981 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 -- 172.21.15.5:0/1727566981 >> 172.21.15.5:0/1727566981 conn(0x5582b9561a30 msgr2=0x5582b959ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 -- 172.21.15.5:0/1727566981 shutdown_connections 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 -- 172.21.15.5:0/1727566981 wait complete. 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.849+0000 7f0a403af740 1 -- start start 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.850+0000 7f0a403af740 1 Processor -- start 2024-04-25T10:06:13.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582b95d7ab0 con 0x5582b95c4830 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381ff820 con 0x5582b9599d70 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5582b95c6340 con 0x5582b95d6440 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a403af740 10 client.0 _ll_register_callbacks cb 0x5582b9594b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35400/0 (socket says 172.21.15.5:35400) 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 -- 172.21.15.5:0/391874474 learned_addr learned my addr 172.21.15.5:0/391874474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a36800640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a37200640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 -- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 msgr2=0x5582b95eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 -- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 msgr2=0x5582b95995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a35e00640 1 -- 172.21.15.5:0/391874474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582b95f8550 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.851+0000 7f0a36800640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a35e00640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001020 tx=0x7f0a20009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d9af3c75995786 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a1b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a1b400640 1 -- 172.21.15.5:0/391874474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200586f0 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a1b400640 1 -- 172.21.15.5:0/391874474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f0a20058890 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a200015f0 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a1b400640 1 -- 172.21.15.5:0/391874474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a2006e050 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a403af740 10 client.6480 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.852+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5582b962f160 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a1b400640 1 -- 172.21.15.5:0/391874474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a200015f0 con 0x5582b95c4830 2024-04-25T10:06:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a1b400640 1 -- 172.21.15.5:0/391874474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a20071020 con 0x5582b95c4830 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a403af740 10 client.6480 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a403af740 10 client.6480 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a403af740 20 client.6480 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a403af740 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b9598f40 0x5582b95c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a403af740 4 client.6480 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5582b9505500 con 0x5582b9598f40 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.853+0000 7f0a36800640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b9598f40 0x5582b95c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a36800640 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b9598f40 0x5582b95c0920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712c7083f46dce7d server_cookie=af35adf39d8aa3cc in_seq=0 out_seq=0 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a1b400640 10 client.6480 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a1b400640 1 -- 172.21.15.5:0/391874474 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5582b9505500 con 0x5582b9598f40 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a1b400640 10 client.6480 handle_command_reply: tid=0 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 client.6480 shutdown 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 msgr2=0x5582b95ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001020 tx=0x7f0a20009030 comp rx=0 tx=0).stop 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 shutdown_connections 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x5582b9598f40 0x5582b95c0920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5582b95d6440 0x5582b95eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582b95c4830 0x5582b95ee9a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 --2- 172.21.15.5:0/391874474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582b9599d70 0x5582b95995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a1b400640 0 client.6480 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.854+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 >> 172.21.15.5:0/391874474 conn(0x5582b9561a30 msgr2=0x5582b95c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 shutdown_connections 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a403af740 1 -- 172.21.15.5:0/391874474 wait complete. 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a403af740 20 client.6480 trim_cache size 0 max 16384 2024-04-25T10:06:13.865 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:06:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:06:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:06:13.866 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a1406b4d0 msgr2=0x7f0a1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a1406b4d0 0x7f0a1406d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0a38050700 tx=0x7f0a2800b040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 msgr2=0x7f0a380ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 0x7f0a380ccbf0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0a30007fd0 tx=0x7f0a3000e670 comp rx=0 tx=0).stop 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 shutdown_connections 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a380cc770 0x7f0a380ccbf0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0a1406b4d0 0x7f0a1406d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a380cb180 0x7f0a380c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 --2- 172.21.15.5:0/3402456502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a1c004800 0x7f0a381541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.855+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 >> 172.21.15.5:0/3402456502 conn(0x7f0a3808af80 msgr2=0x7f0a380b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.856+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 shutdown_connections 2024-04-25T10:06:13.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:13.856+0000 7f0a3ca00640 1 -- 172.21.15.5:0/3402456502 wait complete. 2024-04-25T10:06:13.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:13.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.074+0000 7f5147e00640 1 Processor -- start 2024-04-25T10:06:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.074+0000 7f5147e00640 1 -- start start 2024-04-25T10:06:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.075+0000 7f5147e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5148143d30 0x7f514815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.075+0000 7f5147e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f51480b3ae0 0x7f51480b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.075+0000 7f5147e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51480b4880 con 0x7f51480ce790 2024-04-25T10:06:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.075+0000 7f5147e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480b45a0 con 0x7f51480b3ae0 2024-04-25T10:06:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.075+0000 7f5147e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514815e2c0 con 0x7f5148143d30 2024-04-25T10:06:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.076+0000 7f513e000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f51480b3ae0 0x7f51480b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.076+0000 7f5146a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5148143d30 0x7f514815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.076+0000 7f513e000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f51480b3ae0 0x7f51480b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:34150/0 (socket says 172.21.15.5:34150) 2024-04-25T10:06:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.076+0000 7f513e000640 1 -- 172.21.15.5:0/791077447 learned_addr learned my addr 172.21.15.5:0/791077447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.077+0000 7f5146a00640 1 -- 172.21.15.5:0/791077447 >> v2:172.21.15.80:3300/0 conn(0x7f51480b3ae0 msgr2=0x7f51480b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.077+0000 7f5146a00640 1 --2- 172.21.15.5:0/791077447 >> v2:172.21.15.80:3300/0 conn(0x7f51480b3ae0 0x7f51480b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.077+0000 7f5146a00640 1 -- 172.21.15.5:0/791077447 >> v1:172.21.15.80:6789/0 conn(0x7f51480ce790 legacy=0x7f514815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.077+0000 7f5146a00640 1 -- 172.21.15.5:0/791077447 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514815e5a0 con 0x7f5148143d30 2024-04-25T10:06:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.078+0000 7f5146a00640 1 --2- 172.21.15.5:0/791077447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5148143d30 0x7f514815eda0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51380097a0 tx=0x7f513802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719f2bfecb87f181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.079+0000 7f5146000640 1 -- 172.21.15.5:0/791077447 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138037070 con 0x7f5148143d30 2024-04-25T10:06:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.079+0000 7f5146000640 1 -- 172.21.15.5:0/791077447 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138032440 con 0x7f5148143d30 2024-04-25T10:06:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.079+0000 7f5146000640 1 -- 172.21.15.5:0/791077447 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138032810 con 0x7f5148143d30 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.079+0000 7f5147e00640 1 -- 172.21.15.5:0/791077447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5148143d30 msgr2=0x7f514815eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.079+0000 7f5147e00640 1 --2- 172.21.15.5:0/791077447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5148143d30 0x7f514815eda0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f51380097a0 tx=0x7f513802a180 comp rx=0 tx=0).stop 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.080+0000 7f5147e00640 1 -- 172.21.15.5:0/791077447 shutdown_connections 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.080+0000 7f5147e00640 1 --2- 172.21.15.5:0/791077447 >> v2:172.21.15.80:3300/0 conn(0x7f51480b3ae0 0x7f51480b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.080+0000 7f5147e00640 1 --2- 172.21.15.5:0/791077447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5148143d30 0x7f514815eda0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.080+0000 7f5147e00640 1 -- 172.21.15.5:0/791077447 >> 172.21.15.5:0/791077447 conn(0x7f5148066f20 msgr2=0x7f514806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.080+0000 7f5147e00640 1 -- 172.21.15.5:0/791077447 shutdown_connections 2024-04-25T10:06:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.080+0000 7f5147e00640 1 -- 172.21.15.5:0/791077447 wait complete. 2024-04-25T10:06:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.081+0000 7f5147e00640 1 Processor -- start 2024-04-25T10:06:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.082+0000 7f5147e00640 1 -- start start 2024-04-25T10:06:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.082+0000 7f5147e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51480b3ae0 0x7f514815c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51480ce790 0x7f51480c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 0x7f5148158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148144290 con 0x7f5148143d30 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148144110 con 0x7f51480b3ae0 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514806e430 con 0x7f51480ce790 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 0x7f5148158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 0x7f5148158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35402/0 (socket says 172.21.15.5:35402) 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 -- 172.21.15.5:0/2906459189 learned_addr learned my addr 172.21.15.5:0/2906459189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5146a00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51480b3ae0 0x7f514815c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f513e000640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51480ce790 0x7f51480c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 -- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51480ce790 msgr2=0x7f51480c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51480ce790 0x7f51480c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 -- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51480b3ae0 msgr2=0x7f514815c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51480b3ae0 0x7f514815c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.083+0000 7f5147400640 1 -- 172.21.15.5:0/2906459189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480c8e10 con 0x7f5148143d30 2024-04-25T10:06:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.084+0000 7f5147400640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 0x7f5148158050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f514000c740 tx=0x7f514000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.084+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140016020 con 0x7f5148143d30 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.084+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000ce50 con 0x7f5148143d30 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.084+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51480c90f0 con 0x7f5148143d30 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.084+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140003970 con 0x7f5148143d30 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.084+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51480c96b0 con 0x7f5148143d30 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.086+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51480cd910 con 0x7f5148143d30 2024-04-25T10:06:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.086+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5140003b10 con 0x7f5148143d30 2024-04-25T10:06:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.088+0000 7f513f400640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512406b4d0 0x7f512406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.088+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f514008a0a0 con 0x7f5148143d30 2024-04-25T10:06:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.089+0000 7f5146a00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512406b4d0 0x7f512406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.090+0000 7f5146a00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512406b4d0 0x7f512406d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5138034000 tx=0x7f513802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.093+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51480cd910 con 0x7f5148143d30 2024-04-25T10:06:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.335+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5148068f60 con 0x7f5148143d30 2024-04-25T10:06:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.336+0000 7f513f400640 1 -- 172.21.15.5:0/2906459189 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f5140020020 con 0x7f5148143d30 2024-04-25T10:06:14.342 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:14.342 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:05:49.389040+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":7,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-04-25T10:06:14.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512406b4d0 msgr2=0x7f512406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512406b4d0 0x7f512406d990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5138034000 tx=0x7f513802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 msgr2=0x7f5148158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 0x7f5148158050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f514000c740 tx=0x7f514000cc00 comp rx=0 tx=0).stop 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 shutdown_connections 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5148143d30 0x7f5148158050 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51480ce790 0x7f51480c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f512406b4d0 0x7f512406d990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 --2- 172.21.15.5:0/2906459189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51480b3ae0 0x7f514815c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 >> 172.21.15.5:0/2906459189 conn(0x7f5148066f20 msgr2=0x7f51480cd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 shutdown_connections 2024-04-25T10:06:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.339+0000 7f5147e00640 1 -- 172.21.15.5:0/2906459189 wait complete. 2024-04-25T10:06:14.364 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:06:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.569+0000 7fe6cbe00640 1 Processor -- start 2024-04-25T10:06:14.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.569+0000 7fe6cbe00640 1 -- start start 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6cbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6c4143fb0 0x7fe6c40b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6cbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6c406e240 0x7fe6c406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6cbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c40b2ee0 con 0x7fe6c40ce610 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6cbe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40b3150 con 0x7fe6c4143fb0 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6cbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40b3430 con 0x7fe6c406e240 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6c9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6c4143fb0 0x7fe6c40b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6c9400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6c4143fb0 0x7fe6c40b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35414/0 (socket says 172.21.15.5:35414) 2024-04-25T10:06:14.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6c9400640 1 -- 172.21.15.5:0/2050926295 learned_addr learned my addr 172.21.15.5:0/2050926295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.571+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/2050926295 >> v2:172.21.15.80:3300/0 conn(0x7fe6c406e240 0x7fe6c406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.572+0000 7fe6c9400640 1 -- 172.21.15.5:0/2050926295 >> v2:172.21.15.80:3300/0 conn(0x7fe6c406e240 msgr2=0x7fe6c406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.572+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2050926295 >> v2:172.21.15.80:3300/0 conn(0x7fe6c406e240 0x7fe6c406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.572+0000 7fe6c9400640 1 -- 172.21.15.5:0/2050926295 >> v1:172.21.15.80:6789/0 conn(0x7fe6c40ce610 legacy=0x7fe6c40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.572+0000 7fe6c9400640 1 -- 172.21.15.5:0/2050926295 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40b3710 con 0x7fe6c4143fb0 2024-04-25T10:06:14.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.573+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2050926295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c40b3c30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe6c406baf0 tx=0x7fe6b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29dfafcf91cc88a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.573+0000 7fe6c3e00640 1 -- 172.21.15.5:0/2050926295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8037070 con 0x7fe6c4143fb0 2024-04-25T10:06:14.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.573+0000 7fe6c3e00640 1 -- 172.21.15.5:0/2050926295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8032440 con 0x7fe6c4143fb0 2024-04-25T10:06:14.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.573+0000 7fe6c3e00640 1 -- 172.21.15.5:0/2050926295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8032810 con 0x7fe6c4143fb0 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 -- 172.21.15.5:0/2050926295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 msgr2=0x7fe6c40b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/2050926295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c40b3c30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe6c406baf0 tx=0x7fe6b802a230 comp rx=0 tx=0).stop 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 -- 172.21.15.5:0/2050926295 shutdown_connections 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/2050926295 >> v2:172.21.15.80:3300/0 conn(0x7fe6c406e240 0x7fe6c406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/2050926295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c40b3c30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 -- 172.21.15.5:0/2050926295 >> 172.21.15.5:0/2050926295 conn(0x7fe6c406ab60 msgr2=0x7fe6c406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 -- 172.21.15.5:0/2050926295 shutdown_connections 2024-04-25T10:06:14.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.574+0000 7fe6cbe00640 1 -- 172.21.15.5:0/2050926295 wait complete. 2024-04-25T10:06:14.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.576+0000 7fe6cbe00640 1 Processor -- start 2024-04-25T10:06:14.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.576+0000 7fe6cbe00640 1 -- start start 2024-04-25T10:06:14.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6cbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe6c406e240 0x7fe6c413bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6cbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6c40ce610 0x7fe6c4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c4159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6cbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415ac70 con 0x7fe6c4143fb0 2024-04-25T10:06:14.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6cbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40cf2f0 con 0x7fe6c40ce610 2024-04-25T10:06:14.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6cbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415adf0 con 0x7fe6c406e240 2024-04-25T10:06:14.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c4159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c4159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35420/0 (socket says 172.21.15.5:35420) 2024-04-25T10:06:14.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.577+0000 7fe6c9e00640 1 -- 172.21.15.5:0/854932215 learned_addr learned my addr 172.21.15.5:0/854932215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6c40ce610 0x7fe6c4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9400640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe6c406e240 0x7fe6c413bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9e00640 1 -- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe6c406e240 msgr2=0x7fe6c413bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe6c406e240 0x7fe6c413bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9e00640 1 -- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6c40ce610 msgr2=0x7fe6c4159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6c40ce610 0x7fe6c4159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9e00640 1 -- 172.21.15.5:0/854932215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40b1c90 con 0x7fe6c4143fb0 2024-04-25T10:06:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9400640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe6c406e240 0x7fe6c413bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c4159940 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c780 tx=0x7fe6bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c1600640 1 -- 172.21.15.5:0/854932215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc004b60 con 0x7fe6c4143fb0 2024-04-25T10:06:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.578+0000 7fe6c1600640 1 -- 172.21.15.5:0/854932215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6bc004d00 con 0x7fe6c4143fb0 2024-04-25T10:06:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.579+0000 7fe6c1600640 1 -- 172.21.15.5:0/854932215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc00aa90 con 0x7fe6c4143fb0 2024-04-25T10:06:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.579+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c40b13f0 con 0x7fe6c4143fb0 2024-04-25T10:06:14.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.579+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c40b19b0 con 0x7fe6c4143fb0 2024-04-25T10:06:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.581+0000 7fe6c1600640 1 -- 172.21.15.5:0/854932215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6bc00ac30 con 0x7fe6c4143fb0 2024-04-25T10:06:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.582+0000 7fe6c1600640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.582+0000 7fe6c1600640 1 -- 172.21.15.5:0/854932215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fe6bc08aa00 con 0x7fe6c4143fb0 2024-04-25T10:06:14.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.582+0000 7fe6cedb1740 1 Processor -- start 2024-04-25T10:06:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.583+0000 7fe6c9400640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.583+0000 7fe6cedb1740 1 -- start start 2024-04-25T10:06:14.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed5805750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5823160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57db050 0x55fed58212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed573dae0 con 0x55fed5822c10 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed56e67c0 con 0x55fed57db050 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fed5739b80 con 0x55fed5805370 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c9400640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe6a406b4d0 0x7fe6a406d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8034000 tx=0x7fe6b802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5823160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5823160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35434/0 (socket says 172.21.15.5:35434) 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 -- 172.21.15.5:0/703042332 learned_addr learned my addr 172.21.15.5:0/703042332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c9400640 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed5805750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57db050 0x55fed58212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 -- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 msgr2=0x55fed5805750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed5805750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 -- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57db050 msgr2=0x55fed58212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57db050 0x55fed58212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c8a00640 1 -- 172.21.15.5:0/703042332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fed551e7c0 con 0x55fed5822c10 2024-04-25T10:06:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.584+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57db050 0x55fed58212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:14.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.585+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5823160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000b8d0 tx=0x7fe6b000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1be5bc549ff5b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.585+0000 7fe6b5c00640 1 -- 172.21.15.5:0/703042332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018020 con 0x55fed5822c10 2024-04-25T10:06:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.585+0000 7fe6b5c00640 1 -- 172.21.15.5:0/703042332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b000f9c0 con 0x55fed5822c10 2024-04-25T10:06:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.585+0000 7fe6b5c00640 1 -- 172.21.15.5:0/703042332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000fd70 con 0x55fed5822c10 2024-04-25T10:06:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 -- 172.21.15.5:0/703042332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 msgr2=0x55fed5823160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5823160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000b8d0 tx=0x7fe6b000bd90 comp rx=0 tx=0).stop 2024-04-25T10:06:14.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 -- 172.21.15.5:0/703042332 shutdown_connections 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57db050 0x55fed58212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5823160 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/703042332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed5805750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 -- 172.21.15.5:0/703042332 >> 172.21.15.5:0/703042332 conn(0x55fed57d7980 msgr2=0x55fed5820180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 -- 172.21.15.5:0/703042332 shutdown_connections 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.586+0000 7fe6cedb1740 1 -- 172.21.15.5:0/703042332 wait complete. 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.587+0000 7fe6cedb1740 1 -- start start 2024-04-25T10:06:14.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.587+0000 7fe6cedb1740 1 Processor -- start 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5805370 0x55fed5806630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed57d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed57d4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed57dbdf0 con 0x55fed5822c10 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed56dbd40 con 0x55fed5805370 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fed57d58b0 con 0x55fed57f4c90 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6cedb1740 10 client.0 _ll_register_callbacks cb 0x55fed5451ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed57d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed57d46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35438/0 (socket says 172.21.15.5:35438) 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.588+0000 7fe6c8a00640 1 -- 172.21.15.5:0/2168058496 learned_addr learned my addr 172.21.15.5:0/2168058496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed57d4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5805370 0x55fed5806630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c8a00640 1 -- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 msgr2=0x55fed57d4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed57d4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c8a00640 1 -- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5805370 msgr2=0x55fed5806630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5805370 0x55fed5806630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c8a00640 1 -- 172.21.15.5:0/2168058496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fed557ef50 con 0x55fed5822c10 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5805370 0x55fed5806630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed57d46e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000f5b0 tx=0x7fe6b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e857539fd90e150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6aa000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6aa000640 1 -- 172.21.15.5:0/2168058496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018050 con 0x55fed5822c10 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.589+0000 7fe6aa000640 1 -- 172.21.15.5:0/2168058496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fe6b000f7c0 con 0x55fed5822c10 2024-04-25T10:06:14.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6b000f9c0 con 0x55fed5822c10 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6aa000640 1 -- 172.21.15.5:0/2168058496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6b002e050 con 0x55fed5822c10 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 10 client.6495 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55fed54120b0 con 0x55fed5822c10 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6aa000640 1 -- 172.21.15.5:0/2168058496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b000f9c0 con 0x55fed5822c10 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6aa000640 1 -- 172.21.15.5:0/2168058496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe6b0031020 con 0x55fed5822c10 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 10 client.6495 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 10 client.6495 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 20 client.6495 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5803430 0x55fed5803810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 4 client.6495 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.590+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fed579cb60 con 0x55fed5803430 2024-04-25T10:06:14.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.591+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5803430 0x55fed5803810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.591+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5803430 0x55fed5803810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13c3b23edaffbee server_cookie=45b2ee9bb5d3d98a in_seq=0 out_seq=0 2024-04-25T10:06:14.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.591+0000 7fe6aa000640 10 client.6495 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6aa000640 1 -- 172.21.15.5:0/2168058496 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fed579cb60 con 0x55fed5803430 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6aa000640 10 client.6495 handle_command_reply: tid=0 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 client.6495 shutdown 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 msgr2=0x55fed57d46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed57d46e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000f5b0 tx=0x7fe6b0004730 comp rx=0 tx=0).stop 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 shutdown_connections 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5803430 0x55fed5803810 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed57d4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed57d46e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2168058496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5805370 0x55fed5806630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6aa000640 0 client.6495 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.592+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 >> 172.21.15.5:0/2168058496 conn(0x55fed57db050 msgr2=0x55fed581f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.593+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 shutdown_connections 2024-04-25T10:06:14.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.593+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2168058496 wait complete. 2024-04-25T10:06:14.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.593+0000 7fe6cedb1740 20 client.6495 trim_cache size 0 max 16384 2024-04-25T10:06:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.606+0000 7fe6cedb1740 1 Processor -- start 2024-04-25T10:06:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.606+0000 7fe6cedb1740 1 -- start start 2024-04-25T10:06:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed5804b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 0x55fed5803b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5822c10 0x55fed57d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41ff8f0 con 0x55fed5805370 2024-04-25T10:06:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41fed30 con 0x55fed5822c10 2024-04-25T10:06:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c41ff2d0 con 0x55fed57f4c90 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 0x55fed5803b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 0x55fed5803b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35448/0 (socket says 172.21.15.5:35448) 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6c8a00640 1 -- 172.21.15.5:0/569725498 learned_addr learned my addr 172.21.15.5:0/569725498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5822c10 0x55fed57d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6c8a00640 1 -- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 msgr2=0x55fed5804b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.607+0000 7fe6c9400640 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed5804b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed5804b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6c8a00640 1 -- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5822c10 msgr2=0x55fed57d93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5822c10 0x55fed57d93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6c8a00640 1 -- 172.21.15.5:0/569725498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fed58069a0 con 0x55fed5805370 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5822c10 0x55fed57d93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 0x55fed5803b60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0000c00 tx=0x7fe6b0020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a45f6b485800dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6b5200640 1 -- 172.21.15.5:0/569725498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0037070 con 0x55fed5805370 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6b5200640 1 -- 172.21.15.5:0/569725498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0022030 con 0x55fed5805370 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.608+0000 7fe6b5200640 1 -- 172.21.15.5:0/569725498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018030 con 0x55fed5805370 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 -- 172.21.15.5:0/569725498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 msgr2=0x55fed5803b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 0x55fed5803b60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0000c00 tx=0x7fe6b0020070 comp rx=0 tx=0).stop 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 -- 172.21.15.5:0/569725498 shutdown_connections 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed5822c10 0x55fed57d93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5805370 0x55fed5803b60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/569725498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed57f4c90 0x55fed5804b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 -- 172.21.15.5:0/569725498 >> 172.21.15.5:0/569725498 conn(0x55fed57db050 msgr2=0x55fed57d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 -- 172.21.15.5:0/569725498 shutdown_connections 2024-04-25T10:06:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.609+0000 7fe6cedb1740 1 -- 172.21.15.5:0/569725498 wait complete. 2024-04-25T10:06:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.610+0000 7fe6cedb1740 1 -- start start 2024-04-25T10:06:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.610+0000 7fe6cedb1740 1 Processor -- start 2024-04-25T10:06:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57f4c90 0x55fed57d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed57d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5822ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed5830eb0 con 0x55fed5822c10 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fed57dbdf0 con 0x55fed57f4c90 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fed5830120 con 0x55fed5805370 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6cedb1740 10 client.0 _ll_register_callbacks cb 0x55fed57d5d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5822ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5822ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35450/0 (socket says 172.21.15.5:35450) 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.611+0000 7fe6c9e00640 1 -- 172.21.15.5:0/2078345662 learned_addr learned my addr 172.21.15.5:0/2078345662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57f4c90 0x55fed57d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c8a00640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed57d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9e00640 1 -- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 msgr2=0x55fed57d94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed57d94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9e00640 1 -- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57f4c90 msgr2=0x55fed57d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57f4c90 0x55fed57d8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9e00640 1 -- 172.21.15.5:0/2078345662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fed5821540 con 0x55fed5822c10 2024-04-25T10:06:14.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6c9e00640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5822ff0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc058bb0 tx=0x7fe6bc0596f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993d2a2e18462406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6aa000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6aa000640 1 -- 172.21.15.5:0/2078345662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc086070 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6aa000640 1 -- 172.21.15.5:0/2078345662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fe6bc022da0 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.612+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6b0022030 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6aa000640 1 -- 172.21.15.5:0/2078345662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6bc0615c0 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 10 client.6501 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55fed57da890 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6aa000640 1 -- 172.21.15.5:0/2078345662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0022030 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6aa000640 1 -- 172.21.15.5:0/2078345662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe6bc023060 con 0x55fed5822c10 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 10 client.6501 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 10 client.6501 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 20 client.6501 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5820180 0x55fed5820560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 4 client.6501 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:06:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55fed5818800 con 0x55fed5820180 2024-04-25T10:06:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.613+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5820180 0x55fed5820560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6c9400640 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5820180 0x55fed5820560 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784d1260b6b0b6af server_cookie=243ecc669cc0319b in_seq=0 out_seq=0 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6aa000640 10 client.6501 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6aa000640 1 -- 172.21.15.5:0/2078345662 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55fed5818800 con 0x55fed5820180 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6aa000640 10 client.6501 handle_command_reply: tid=0 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 client.6501 shutdown 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 msgr2=0x55fed5822ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5822ff0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc058bb0 tx=0x7fe6bc0596f0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 shutdown_connections 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55fed5820180 0x55fed5820560 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fed5822c10 0x55fed5822ff0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fed5805370 0x55fed57d94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 --2- 172.21.15.5:0/2078345662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fed57f4c90 0x55fed57d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6aa000640 0 client.6501 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.614+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 >> 172.21.15.5:0/2078345662 conn(0x55fed57db050 msgr2=0x55fed57d63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 shutdown_connections 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cedb1740 1 -- 172.21.15.5:0/2078345662 wait complete. 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cedb1740 20 client.6501 trim_cache size 0 max 16384 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:14.623 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 1651025562, 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir_test/file" 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:06:14.624 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe6a406b4d0 msgr2=0x7fe6a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe6a406b4d0 0x7fe6a406d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8034000 tx=0x7fe6b802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 msgr2=0x7fe6c4159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.615+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c4159940 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c780 tx=0x7fe6bc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 shutdown_connections 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c4143fb0 0x7fe6c4159940 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe6a406b4d0 0x7fe6a406d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6c40ce610 0x7fe6c4159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 --2- 172.21.15.5:0/854932215 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe6c406e240 0x7fe6c413bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 >> 172.21.15.5:0/854932215 conn(0x7fe6c406ab60 msgr2=0x7fe6c41420d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 shutdown_connections 2024-04-25T10:06:14.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.616+0000 7fe6cbe00640 1 -- 172.21.15.5:0/854932215 wait complete. 2024-04-25T10:06:14.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:06:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.835+0000 7fa773e00640 1 Processor -- start 2024-04-25T10:06:14.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.835+0000 7fa773e00640 1 -- start start 2024-04-25T10:06:14.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa773e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa7740ce680 0x7fa7740b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa773e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77406d870 con 0x7fa774140e00 2024-04-25T10:06:14.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa773e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77406db50 con 0x7fa7740b4040 2024-04-25T10:06:14.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa773e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77406de30 con 0x7fa7740ce680 2024-04-25T10:06:14.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa772a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa774140e00 0x7fa77406cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44140/0 (socket says 172.21.15.5:44140) 2024-04-25T10:06:14.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa772a00640 1 -- 172.21.15.5:0/569007489 learned_addr learned my addr 172.21.15.5:0/569007489 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:14.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.837+0000 7fa76a000640 1 --2- 172.21.15.5:0/569007489 >> v2:172.21.15.80:3300/0 conn(0x7fa7740ce680 0x7fa7740b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817824895 0 0) 0x7fa77406d870 con 0x7fa774140e00 2024-04-25T10:06:14.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa758003520 con 0x7fa774140e00 2024-04-25T10:06:14.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168462417 0 0) 0x7fa758003520 con 0x7fa774140e00 2024-04-25T10:06:14.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 >> v2:172.21.15.80:3300/0 conn(0x7fa7740ce680 msgr2=0x7fa7740b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 --2- 172.21.15.5:0/569007489 >> v2:172.21.15.80:3300/0 conn(0x7fa7740ce680 0x7fa7740b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 >> v1:172.21.15.80:6789/0 conn(0x7fa7740b4040 legacy=0x7fa774141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77406e110 con 0x7fa774140e00 2024-04-25T10:06:14.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa764001670 con 0x7fa774140e00 2024-04-25T10:06:14.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 >> v1:172.21.15.5:6789/0 conn(0x7fa774140e00 legacy=0x7fa77406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa758003a00 0x7fa758003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa758004800 0x7fa758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7580051c0 0x7fa75800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77406de30 con 0x7fa758003a00 2024-04-25T10:06:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75800c0c0 con 0x7fa758004800 2024-04-25T10:06:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.838+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75800c2c0 con 0x7fa7580051c0 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa772a00640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa758003a00 0x7fa758003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa76a000640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa758004800 0x7fa758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa772a00640 1 -- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7580051c0 msgr2=0x7fa75800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa772a00640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7580051c0 0x7fa75800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa772a00640 1 -- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa758004800 msgr2=0x7fa758004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa772a00640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa758004800 0x7fa758004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa772a00640 1 -- 172.21.15.5:0/569007489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa758003520 con 0x7fa758003a00 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.839+0000 7fa773400640 1 -- 172.21.15.5:0/569007489 reap_dead start 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.840+0000 7fa772a00640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa758003a00 0x7fa758003e20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c0e0 tx=0x7fa758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd0ddf935f735d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.840+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764007c20 con 0x7fa758003a00 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.840+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764005af0 con 0x7fa758003a00 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.840+0000 7fa772000640 1 -- 172.21.15.5:0/569007489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764007070 con 0x7fa758003a00 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 -- 172.21.15.5:0/569007489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa758003a00 msgr2=0x7fa758003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa758003a00 0x7fa758003e20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c0e0 tx=0x7fa758005830 comp rx=0 tx=0).stop 2024-04-25T10:06:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 -- 172.21.15.5:0/569007489 shutdown_connections 2024-04-25T10:06:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 --2- 172.21.15.5:0/569007489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa758003a00 0x7fa758003e20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 -- 172.21.15.5:0/569007489 >> 172.21.15.5:0/569007489 conn(0x7fa774066f20 msgr2=0x7fa77406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 -- 172.21.15.5:0/569007489 shutdown_connections 2024-04-25T10:06:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.841+0000 7fa773e00640 1 -- 172.21.15.5:0/569007489 wait complete. 2024-04-25T10:06:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.843+0000 7fa773e00640 1 Processor -- start 2024-04-25T10:06:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.843+0000 7fa773e00640 1 -- start start 2024-04-25T10:06:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa758003a00 0x7fa774142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7741f6870 0x7fa7740cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 0x7fa77413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b3c80 con 0x7fa77413e8c0 2024-04-25T10:06:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b3b00 con 0x7fa7741f6870 2024-04-25T10:06:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b3e00 con 0x7fa758003a00 2024-04-25T10:06:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 0x7fa77413ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 0x7fa77413ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35460/0 (socket says 172.21.15.5:35460) 2024-04-25T10:06:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.844+0000 7fa773400640 1 -- 172.21.15.5:0/1122326380 learned_addr learned my addr 172.21.15.5:0/1122326380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa772a00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa758003a00 0x7fa774142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa76a000640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7741f6870 0x7fa7740cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa773400640 1 -- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa758003a00 msgr2=0x7fa774142410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa773400640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa758003a00 0x7fa774142410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa773400640 1 -- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7741f6870 msgr2=0x7fa7740cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa773400640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7741f6870 0x7fa7740cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa773400640 1 -- 172.21.15.5:0/1122326380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7741f6d70 con 0x7fa77413e8c0 2024-04-25T10:06:14.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa773400640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 0x7fa77413ed50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b4a60 tx=0x7fa76c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.845+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c017070 con 0x7fa77413e8c0 2024-04-25T10:06:14.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.846+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c005d60 con 0x7fa77413e8c0 2024-04-25T10:06:14.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.846+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00a850 con 0x7fa77413e8c0 2024-04-25T10:06:14.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.846+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7741f7050 con 0x7fa77413e8c0 2024-04-25T10:06:14.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.846+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7741f7610 con 0x7fa77413e8c0 2024-04-25T10:06:14.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.847+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774068f60 con 0x7fa77413e8c0 2024-04-25T10:06:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.848+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa76c00a9f0 con 0x7fa77413e8c0 2024-04-25T10:06:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.849+0000 7fa76b400640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa75006b4d0 0x7fa75006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.849+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa76c089570 con 0x7fa77413e8c0 2024-04-25T10:06:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.850+0000 7fa772a00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa75006b4d0 0x7fa75006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.851+0000 7fa772a00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa75006b4d0 0x7fa75006d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa764032000 tx=0x7fa764009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:14.853+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa76c0581b0 con 0x7fa77413e8c0 2024-04-25T10:06:15.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.097+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa774068670 con 0x7fa77413e8c0 2024-04-25T10:06:15.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.097+0000 7fa76b400640 1 -- 172.21.15.5:0/1122326380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fa76c057ab0 con 0x7fa77413e8c0 2024-04-25T10:06:15.103 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:15.103 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa75006b4d0 msgr2=0x7fa75006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa75006b4d0 0x7fa75006d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa764032000 tx=0x7fa764009030 comp rx=0 tx=0).stop 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 msgr2=0x7fa77413ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 0x7fa77413ed50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b4a60 tx=0x7fa76c00cb90 comp rx=0 tx=0).stop 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 shutdown_connections 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa77413e8c0 0x7fa77413ed50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa75006b4d0 0x7fa75006d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7741f6870 0x7fa7740cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 --2- 172.21.15.5:0/1122326380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa758003a00 0x7fa774142410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 >> 172.21.15.5:0/1122326380 conn(0x7fa774066f20 msgr2=0x7fa7740afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 shutdown_connections 2024-04-25T10:06:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.100+0000 7fa773e00640 1 -- 172.21.15.5:0/1122326380 wait complete. 2024-04-25T10:06:15.126 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:06:15.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:15.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T10:06:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.342+0000 7fda15e00640 1 Processor -- start 2024-04-25T10:06:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.342+0000 7fda15e00640 1 -- start start 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.343+0000 7fda15e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 0x7fda1006e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.343+0000 7fda15e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fda1006ed60 0x7fda100cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda15e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1006d780 con 0x7fda100cce30 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda15e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda101423f0 con 0x7fda1006ed60 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda15e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda100cc210 con 0x7fda10142010 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda0f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fda100cce30 0x7fda101426a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44152/0 (socket says 172.21.15.5:44152) 2024-04-25T10:06:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda0f400640 1 -- 172.21.15.5:0/599949930 learned_addr learned my addr 172.21.15.5:0/599949930 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda0e000640 1 --2- 172.21.15.5:0/599949930 >> v2:172.21.15.80:3300/0 conn(0x7fda1006ed60 0x7fda100cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda0ea00640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 0x7fda1006e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.344+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213249677 0 0) 0x7fda1006d780 con 0x7fda100cce30 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f4003520 con 0x7fda100cce30 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1646854706 0 0) 0x7fd9f4003520 con 0x7fda100cce30 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 msgr2=0x7fda1006e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 0x7fda1006e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 >> v2:172.21.15.80:3300/0 conn(0x7fda1006ed60 msgr2=0x7fda100cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 --2- 172.21.15.5:0/599949930 >> v2:172.21.15.80:3300/0 conn(0x7fda1006ed60 0x7fda100cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1006da60 con 0x7fda100cce30 2024-04-25T10:06:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fda080019f0 con 0x7fda100cce30 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 >> v1:172.21.15.5:6789/0 conn(0x7fda100cce30 legacy=0x7fda101426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9f4004800 0x7fd9f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 0x7fd9f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101423f0 con 0x7fd9f40039b0 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100cc210 con 0x7fd9f4004800 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f400b9f0 con 0x7fda10142010 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0e000640 1 --2- 172.21.15.5:0/599949930 >> v2:172.21.15.80:3300/0 conn(0x7fda1006ed60 0x7fda100cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.345+0000 7fda0ea00640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.346+0000 7fda0f400640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 0x7fd9f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.346+0000 7fda0ea00640 1 -- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 msgr2=0x7fd9f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.346+0000 7fda0ea00640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10142010 0x7fd9f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.346+0000 7fda0ea00640 1 -- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9f4004800 msgr2=0x7fd9f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.346+0000 7fda0ea00640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9f4004800 0x7fd9f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.346+0000 7fda0ea00640 1 -- 172.21.15.5:0/599949930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4003520 con 0x7fd9f40039b0 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.347+0000 7fda0ea00640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003df0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009ac0 tx=0x7fd9f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fd3d732fa8fae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.347+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8036070 con 0x7fd9f40039b0 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031400 con 0x7fd9f40039b0 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda0d600640 1 -- 172.21.15.5:0/599949930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8009690 con 0x7fd9f40039b0 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda15e00640 1 -- 172.21.15.5:0/599949930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 msgr2=0x7fd9f4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda15e00640 1 --2- 172.21.15.5:0/599949930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fd9f4003df0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009ac0 tx=0x7fd9f4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda0f400640 1 -- 172.21.15.5:0/599949930 reap_dead start 2024-04-25T10:06:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda15e00640 1 -- 172.21.15.5:0/599949930 shutdown_connections 2024-04-25T10:06:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda15e00640 1 -- 172.21.15.5:0/599949930 >> 172.21.15.5:0/599949930 conn(0x7fda10066f80 msgr2=0x7fda1006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda15e00640 1 -- 172.21.15.5:0/599949930 shutdown_connections 2024-04-25T10:06:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.348+0000 7fda15e00640 1 -- 172.21.15.5:0/599949930 wait complete. 2024-04-25T10:06:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.350+0000 7fda15e00640 1 Processor -- start 2024-04-25T10:06:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.350+0000 7fda15e00640 1 -- start start 2024-04-25T10:06:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.351+0000 7fda15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fda1015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.351+0000 7fda15e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10130690 0x7fda10130af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.351+0000 7fda15e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda101e46b0 0x7fda101e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.351+0000 7fda15e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1006e300 con 0x7fd9f40039b0 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.351+0000 7fda15e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1006e020 con 0x7fda101e46b0 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.351+0000 7fda15e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda10142db0 con 0x7fda10130690 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fda1015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fda1015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35488/0 (socket says 172.21.15.5:35488) 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 -- 172.21.15.5:0/3012774194 learned_addr learned my addr 172.21.15.5:0/3012774194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 -- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10130690 msgr2=0x7fda10130af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10130690 0x7fda10130af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 -- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda101e46b0 msgr2=0x7fda101e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda101e46b0 0x7fda101e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.352+0000 7fda0ea00640 1 -- 172.21.15.5:0/3012774194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1015af00 con 0x7fd9f40039b0 2024-04-25T10:06:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.353+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fda1015a9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009ac0 tx=0x7fd9f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.353+0000 7fda07400640 1 -- 172.21.15.5:0/3012774194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8040020 con 0x7fd9f40039b0 2024-04-25T10:06:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.353+0000 7fda07400640 1 -- 172.21.15.5:0/3012774194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8003bc0 con 0x7fd9f40039b0 2024-04-25T10:06:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.353+0000 7fda07400640 1 -- 172.21.15.5:0/3012774194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8036040 con 0x7fd9f40039b0 2024-04-25T10:06:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.353+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda101e5400 con 0x7fd9f40039b0 2024-04-25T10:06:15.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.353+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda101e59c0 con 0x7fd9f40039b0 2024-04-25T10:06:15.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.355+0000 7fda07400640 1 -- 172.21.15.5:0/3012774194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9f8043020 con 0x7fd9f40039b0 2024-04-25T10:06:15.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.356+0000 7fda07400640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.356+0000 7fda07400640 1 -- 172.21.15.5:0/3012774194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fd9f80aa6e0 con 0x7fd9f40039b0 2024-04-25T10:06:15.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.357+0000 7fda18c56740 1 Processor -- start 2024-04-25T10:06:15.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.357+0000 7fda18c56740 1 -- start start 2024-04-25T10:06:15.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.357+0000 7fda0e000640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda18c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 0x55b3816175f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381647100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647640 0x55b38161c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda18c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3815809f0 con 0x55b381664c60 2024-04-25T10:06:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b381529eb0 con 0x55b381647d80 2024-04-25T10:06:15.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b38157ca90 con 0x55b381647640 2024-04-25T10:06:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 0x55b3816175f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 0x55b3816175f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35502/0 (socket says 172.21.15.5:35502) 2024-04-25T10:06:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda0ea00640 1 -- 172.21.15.5:0/2222331797 learned_addr learned my addr 172.21.15.5:0/2222331797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.358+0000 7fda0e000640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fda000093d0 tx=0x7fda00009300 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0e000640 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381647100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0f400640 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647640 0x55b38161c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0ea00640 1 -- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647640 msgr2=0x55b38161c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0ea00640 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647640 0x55b38161c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0ea00640 1 -- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 msgr2=0x55b381647100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0ea00640 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381647100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0ea00640 1 -- 172.21.15.5:0/2222331797 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b381361940 con 0x55b381664c60 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0e000640 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381647100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fda0ea00640 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 0x55b3816175f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8034040 tx=0x7fd9f8004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca001ea7f3bc3441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fd9ff400640 1 -- 172.21.15.5:0/2222331797 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f807cc50 con 0x55b381664c60 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fd9ff400640 1 -- 172.21.15.5:0/2222331797 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80812a0 con 0x55b381664c60 2024-04-25T10:06:15.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.359+0000 7fd9ff400640 1 -- 172.21.15.5:0/2222331797 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8075170 con 0x55b381664c60 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 -- 172.21.15.5:0/2222331797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 msgr2=0x55b3816175f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 0x55b3816175f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8034040 tx=0x7fd9f8004620 comp rx=0 tx=0).stop 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 -- 172.21.15.5:0/2222331797 shutdown_connections 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647640 0x55b38161c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381647100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 --2- 172.21.15.5:0/2222331797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381664c60 0x55b3816175f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 -- 172.21.15.5:0/2222331797 >> 172.21.15.5:0/2222331797 conn(0x55b381619860 msgr2=0x55b381662100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 -- 172.21.15.5:0/2222331797 shutdown_connections 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.360+0000 7fda18c56740 1 -- 172.21.15.5:0/2222331797 wait complete. 2024-04-25T10:06:15.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.361+0000 7fda18c56740 1 -- start start 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.361+0000 7fda18c56740 1 Processor -- start 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381648270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b381674610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381674b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38157ca90 con 0x55b381642250 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b38151f370 con 0x55b381647d80 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3815809f0 con 0x55b381664c60 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda18c56740 10 client.0 _ll_register_callbacks cb 0x55b381294d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda0f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381674b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.362+0000 7fda0f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381674b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35516/0 (socket says 172.21.15.5:35516) 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 -- 172.21.15.5:0/2067640737 learned_addr learned my addr 172.21.15.5:0/2067640737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0ea00640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381648270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0e000640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b381674610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 -- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 msgr2=0x55b381674610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b381674610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 -- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 msgr2=0x55b381648270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381648270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 -- 172.21.15.5:0/2067640737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3813c20d0 con 0x55b381642250 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fda0f400640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381674b50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fda100b1e50 tx=0x7fda08007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd2593b2e79a914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.363+0000 7fd9f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fd9f3e00640 1 -- 172.21.15.5:0/2067640737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08019060 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fd9f3e00640 1 -- 172.21.15.5:0/2067640737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fda0800f8d0 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f80812a0 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fd9f3e00640 1 -- 172.21.15.5:0/2067640737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0801f050 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 10 client.6519 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55b381255230 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fd9f3e00640 1 -- 172.21.15.5:0/2067640737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80812a0 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fd9f3e00640 1 -- 172.21.15.5:0/2067640737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda08022020 con 0x55b381642250 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 10 client.6519 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 10 client.6519 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 20 client.6519 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b381663230 0x55b381663640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 4 client.6519 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:15.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.364+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b38158df30 con 0x55b381663230 2024-04-25T10:06:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.365+0000 7fda0ea00640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b381663230 0x55b381663640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.365+0000 7fda0ea00640 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b381663230 0x55b381663640 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35033098f4cb10e7 server_cookie=fd20b99f6a761c63 in_seq=0 out_seq=0 2024-04-25T10:06:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.365+0000 7fd9f3e00640 10 client.6519 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fd9f3e00640 1 -- 172.21.15.5:0/2067640737 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b38158df30 con 0x55b381663230 2024-04-25T10:06:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fd9f3e00640 10 client.6519 handle_command_reply: tid=0 2024-04-25T10:06:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 client.6519 shutdown 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 msgr2=0x55b381674b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381674b50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fda100b1e50 tx=0x7fda08007850 comp rx=0 tx=0).stop 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 shutdown_connections 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b381663230 0x55b381663640 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381674b50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b381674610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 --2- 172.21.15.5:0/2067640737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381648270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 >> 172.21.15.5:0/2067640737 conn(0x55b381647640 msgr2=0x55b381616d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.366+0000 7fd9f3e00640 0 client.6519 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.367+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 shutdown_connections 2024-04-25T10:06:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.367+0000 7fda18c56740 1 -- 172.21.15.5:0/2067640737 wait complete. 2024-04-25T10:06:15.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.367+0000 7fda18c56740 20 client.6519 trim_cache size 0 max 16384 2024-04-25T10:06:15.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.384+0000 7fda18c56740 1 Processor -- start 2024-04-25T10:06:15.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.384+0000 7fda18c56740 1 -- start start 2024-04-25T10:06:15.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda18c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b38161a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381617930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b38163f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda18c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10201050 con 0x55b381642250 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10200170 con 0x55b381647d80 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda101e7700 con 0x55b381664c60 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b38161a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b38161a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35526/0 (socket says 172.21.15.5:35526) 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 -- 172.21.15.5:0/3668777686 learned_addr learned my addr 172.21.15.5:0/3668777686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 -- 172.21.15.5:0/3668777686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 msgr2=0x55b38163f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b38163f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 -- 172.21.15.5:0/3668777686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 msgr2=0x55b381617930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381617930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.385+0000 7fda0ea00640 1 -- 172.21.15.5:0/3668777686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b38164b5d0 con 0x55b381642250 2024-04-25T10:06:15.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.386+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b38161a5b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8078a40 tx=0x7fd9f8097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2918a5b2fdeb2c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.386+0000 7fd9fea00640 1 -- 172.21.15.5:0/3668777686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8076c20 con 0x55b381642250 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.386+0000 7fd9fea00640 1 -- 172.21.15.5:0/3668777686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8075170 con 0x55b381642250 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.386+0000 7fd9fea00640 1 -- 172.21.15.5:0/3668777686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8081010 con 0x55b381642250 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.386+0000 7fda18c56740 1 -- 172.21.15.5:0/3668777686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 msgr2=0x55b38161a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.386+0000 7fda18c56740 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b38161a5b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8078a40 tx=0x7fd9f8097040 comp rx=0 tx=0).stop 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 -- 172.21.15.5:0/3668777686 shutdown_connections 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381664c60 0x55b38163f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381647d80 0x55b381617930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 --2- 172.21.15.5:0/3668777686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b38161a5b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 -- 172.21.15.5:0/3668777686 >> 172.21.15.5:0/3668777686 conn(0x55b381647640 msgr2=0x55b38161cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 -- 172.21.15.5:0/3668777686 shutdown_connections 2024-04-25T10:06:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.387+0000 7fda18c56740 1 -- 172.21.15.5:0/3668777686 wait complete. 2024-04-25T10:06:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.388+0000 7fda18c56740 1 -- start start 2024-04-25T10:06:15.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.388+0000 7fda18c56740 1 Processor -- start 2024-04-25T10:06:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381671210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647d80 0x55b381659360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381664c60 0x55b3816598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b381529eb0 con 0x55b381642250 2024-04-25T10:06:15.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101ffb50 con 0x55b381664c60 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b381643e70 con 0x55b381647d80 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda18c56740 10 client.0 _ll_register_callbacks cb 0x7fd9f80788d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381671210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381671210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35528/0 (socket says 172.21.15.5:35528) 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 -- 172.21.15.5:0/3694230575 learned_addr learned my addr 172.21.15.5:0/3694230575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 -- 172.21.15.5:0/3694230575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647d80 msgr2=0x55b381659360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647d80 0x55b381659360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 -- 172.21.15.5:0/3694230575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381664c60 msgr2=0x55b3816598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381664c60 0x55b3816598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.389+0000 7fda0ea00640 1 -- 172.21.15.5:0/3694230575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3816130b0 con 0x55b381642250 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fda0ea00640 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381671210 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8092040 tx=0x7fd9f8006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b394543fcfffdfe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fd9f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fd9f3e00640 1 -- 172.21.15.5:0/3694230575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f809c060 con 0x55b381642250 2024-04-25T10:06:15.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fd9f3e00640 1 -- 172.21.15.5:0/3694230575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fd9f8080e10 con 0x55b381642250 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f8075170 con 0x55b381642250 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fd9f3e00640 1 -- 172.21.15.5:0/3694230575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f80a2050 con 0x55b381642250 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fda18c56740 10 client.6525 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.390+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55b38161e6d0 con 0x55b381642250 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fd9f3e00640 1 -- 172.21.15.5:0/3694230575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8075170 con 0x55b381642250 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fd9f3e00640 1 -- 172.21.15.5:0/3694230575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9f808cb50 con 0x55b381642250 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda18c56740 10 client.6525 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda18c56740 10 client.6525 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda18c56740 20 client.6525 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda18c56740 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b3816439c0 0x55b3816d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda18c56740 4 client.6525 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x55b381588280 con 0x55b3816439c0 2024-04-25T10:06:15.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.391+0000 7fda0e000640 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b3816439c0 0x55b3816d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.392+0000 7fda0e000640 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b3816439c0 0x55b3816d53c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f008735db5971b7 server_cookie=61aefeb1c8ec2040 in_seq=0 out_seq=0 2024-04-25T10:06:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.392+0000 7fd9f3e00640 10 client.6525 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.392+0000 7fd9f3e00640 1 -- 172.21.15.5:0/3694230575 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b381588280 con 0x55b3816439c0 2024-04-25T10:06:15.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.392+0000 7fd9f3e00640 10 client.6525 handle_command_reply: tid=0 2024-04-25T10:06:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.392+0000 7fda18c56740 1 client.6525 shutdown 2024-04-25T10:06:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 msgr2=0x55b381671210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381671210 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8092040 tx=0x7fd9f8006f40 comp rx=0 tx=0).stop 2024-04-25T10:06:15.400 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fcdff600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:05:41.713288+0000 1=1+0) (iversion lock) caps={5186=pAsLsXsFs/-@2},l=5186(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x556f993c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 shutdown_connections 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55b3816439c0 0x55b3816d53c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b381664c60 0x55b3816598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b381647d80 0x55b381659360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 --2- 172.21.15.5:0/3694230575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b381642250 0x55b381671210 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 >> 172.21.15.5:0/3694230575 conn(0x55b381647640 msgr2=0x55b381617930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fd9f3e00640 0 client.6525 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 shutdown_connections 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 1 -- 172.21.15.5:0/3694230575 wait complete. 2024-04-25T10:06:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.393+0000 7fda18c56740 20 client.6525 trim_cache size 0 max 16384 2024-04-25T10:06:15.402 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:06:15.402 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:06:15.402 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6a16c086-9b83-4f4c-a99d-0d9b3044051b", 2024-04-25T10:06:15.402 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:06:15.402 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd9ec06b4d0 msgr2=0x7fd9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fda000093d0 tx=0x7fda00009300 comp rx=0 tx=0).stop 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 msgr2=0x7fda1015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fda1015a9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009ac0 tx=0x7fd9f8004730 comp rx=0 tx=0).stop 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 shutdown_connections 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd9ec06b4d0 0x7fd9ec06d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fda101e46b0 0x7fda101e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fda10130690 0x7fda10130af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 --2- 172.21.15.5:0/3012774194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9f40039b0 0x7fda1015a9c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 >> 172.21.15.5:0/3012774194 conn(0x7fda10066f80 msgr2=0x7fda100ac9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 shutdown_connections 2024-04-25T10:06:15.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.394+0000 7fda15e00640 1 -- 172.21.15.5:0/3012774194 wait complete. 2024-04-25T10:06:15.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:15.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:06:15.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.615+0000 7fa9cbe00640 1 Processor -- start 2024-04-25T10:06:15.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.615+0000 7fa9cbe00640 1 -- start start 2024-04-25T10:06:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9cbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 0x7fa9cc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9cbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc071710 0x7fa9cc0adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9cbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc15f5e0 con 0x7fa9cc0adac0 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9cbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc06d250 con 0x7fa9cc143d10 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9cbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0ae680 con 0x7fa9cc071710 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9caa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 0x7fa9cc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9cb400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc071710 0x7fa9cc0adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9caa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 0x7fa9cc15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:34272/0 (socket says 172.21.15.5:34272) 2024-04-25T10:06:15.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.617+0000 7fa9caa00640 1 -- 172.21.15.5:0/1789034990 learned_addr learned my addr 172.21.15.5:0/1789034990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.618+0000 7fa9cb400640 1 -- 172.21.15.5:0/1789034990 >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 msgr2=0x7fa9cc15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.618+0000 7fa9cb400640 1 --2- 172.21.15.5:0/1789034990 >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 0x7fa9cc15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.618+0000 7fa9cb400640 1 -- 172.21.15.5:0/1789034990 >> v1:172.21.15.80:6789/0 conn(0x7fa9cc0adac0 legacy=0x7fa9cc071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.618+0000 7fa9cb400640 1 -- 172.21.15.5:0/1789034990 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0ae960 con 0x7fa9cc071710 2024-04-25T10:06:15.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.618+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1789034990 >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 0x7fa9cc15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:15.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.620+0000 7fa9cb400640 1 --2- 172.21.15.5:0/1789034990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc071710 0x7fa9cc0adf50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009d90 tx=0x7fa9c402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84fb9e2532aaf9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.620+0000 7fa9ca000640 1 -- 172.21.15.5:0/1789034990 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4007740 con 0x7fa9cc071710 2024-04-25T10:06:15.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.620+0000 7fa9ca000640 1 -- 172.21.15.5:0/1789034990 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c40078e0 con 0x7fa9cc071710 2024-04-25T10:06:15.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.620+0000 7fa9ca000640 1 -- 172.21.15.5:0/1789034990 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4007cb0 con 0x7fa9cc071710 2024-04-25T10:06:15.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1789034990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc071710 msgr2=0x7fa9cc0adf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1789034990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc071710 0x7fa9cc0adf50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009d90 tx=0x7fa9c402a660 comp rx=0 tx=0).stop 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1789034990 shutdown_connections 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1789034990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc071710 0x7fa9cc0adf50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1789034990 >> v2:172.21.15.80:3300/0 conn(0x7fa9cc143d10 0x7fa9cc15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1789034990 >> 172.21.15.5:0/1789034990 conn(0x7fa9cc066f20 msgr2=0x7fa9cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1789034990 shutdown_connections 2024-04-25T10:06:15.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.621+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1789034990 wait complete. 2024-04-25T10:06:15.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.623+0000 7fa9cbe00640 1 Processor -- start 2024-04-25T10:06:15.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.623+0000 7fa9cbe00640 1 -- start start 2024-04-25T10:06:15.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.624+0000 7fa9cbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9cc071710 0x7fa9cc13fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.624+0000 7fa9cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 0x7fa9cc146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.624+0000 7fa9cbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc143d10 0x7fa9cc141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.624+0000 7fa9cbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1440f0 con 0x7fa9cc0adac0 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.624+0000 7fa9cbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc06ea40 con 0x7fa9cc071710 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.624+0000 7fa9cbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc06d6e0 con 0x7fa9cc143d10 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 0x7fa9cc146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 0x7fa9cc146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35530/0 (socket says 172.21.15.5:35530) 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 -- 172.21.15.5:0/1557955384 learned_addr learned my addr 172.21.15.5:0/1557955384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9cc071710 0x7fa9cc13fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9cb400640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc143d10 0x7fa9cc141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 -- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc143d10 msgr2=0x7fa9cc141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc143d10 0x7fa9cc141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 -- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9cc071710 msgr2=0x7fa9cc13fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9cc071710 0x7fa9cc13fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.625+0000 7fa9c3e00640 1 -- 172.21.15.5:0/1557955384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc15d6a0 con 0x7fa9cc0adac0 2024-04-25T10:06:15.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.626+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 0x7fa9cc146490 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400c450 tx=0x7fa9b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.626+0000 7fa9c3400640 1 -- 172.21.15.5:0/1557955384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40052d0 con 0x7fa9cc0adac0 2024-04-25T10:06:15.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.626+0000 7fa9c3400640 1 -- 172.21.15.5:0/1557955384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b4005470 con 0x7fa9cc0adac0 2024-04-25T10:06:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.626+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc1453f0 con 0x7fa9cc0adac0 2024-04-25T10:06:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.626+0000 7fa9c3400640 1 -- 172.21.15.5:0/1557955384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4012870 con 0x7fa9cc0adac0 2024-04-25T10:06:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.626+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc145c90 con 0x7fa9cc0adac0 2024-04-25T10:06:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.629+0000 7fa9c3400640 1 -- 172.21.15.5:0/1557955384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9b4012a10 con 0x7fa9cc0adac0 2024-04-25T10:06:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.630+0000 7fa9c3400640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa9a806b500 0x7fa9a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.630+0000 7fa9c3400640 1 -- 172.21.15.5:0/1557955384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa9b4089f80 con 0x7fa9cc0adac0 2024-04-25T10:06:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.631+0000 7fa9d4bfc740 1 Processor -- start 2024-04-25T10:06:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.631+0000 7fa9d4bfc740 1 -- start start 2024-04-25T10:06:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.631+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa9a806b500 0x7fa9a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e35fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e35e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e388cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0e2c1c70 con 0x560e0e39a440 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0e26acf0 con 0x560e0e388830 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e0e2bdd10 con 0x560e0e35dd70 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e35e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e35e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35534/0 (socket says 172.21.15.5:35534) 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa9a806b500 0x7fa9a806d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc006ca0 tx=0x7fa9bc009240 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.632+0000 7fa9c3e00640 1 -- 172.21.15.5:0/2703686986 learned_addr learned my addr 172.21.15.5:0/2703686986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9cb400640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e388cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9caa00640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e35fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9c3e00640 1 -- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 msgr2=0x560e0e35fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e35fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9c3e00640 1 -- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 msgr2=0x560e0e388cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e388cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9c3e00640 1 -- 172.21.15.5:0/2703686986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0e0a07d0 con 0x560e0e39a440 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9cb400640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e388cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9caa00640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e35fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e35e150 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4054920 tx=0x7fa9b4058d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfa139c585633e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9bb400640 1 -- 172.21.15.5:0/2703686986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4060bd0 con 0x560e0e39a440 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9bb400640 1 -- 172.21.15.5:0/2703686986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b40567c0 con 0x560e0e39a440 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.633+0000 7fa9bb400640 1 -- 172.21.15.5:0/2703686986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4054b30 con 0x560e0e39a440 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/2703686986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 msgr2=0x560e0e35e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e35e150 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4054920 tx=0x7fa9b4058d00 comp rx=0 tx=0).stop 2024-04-25T10:06:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/2703686986 shutdown_connections 2024-04-25T10:06:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e388cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e35e150 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/2703686986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e35fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/2703686986 >> 172.21.15.5:0/2703686986 conn(0x560e0e325a30 msgr2=0x560e0e3a3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/2703686986 shutdown_connections 2024-04-25T10:06:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.634+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/2703686986 wait complete. 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.635+0000 7fa9d4bfc740 1 -- start start 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.635+0000 7fa9d4bfc740 1 Processor -- start 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e385910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e39c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e39cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0e2603a0 con 0x560e0e35dd70 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0e386380 con 0x560e0e39a440 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e0e3b3b80 con 0x560e0e388830 2024-04-25T10:06:15.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.636+0000 7fa9d4bfc740 10 client.0 _ll_register_callbacks cb 0x560e0e27db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e385910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e385910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35542/0 (socket says 172.21.15.5:35542) 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 -- 172.21.15.5:0/629040414 learned_addr learned my addr 172.21.15.5:0/629040414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9cb400640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e39cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e39c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 -- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 msgr2=0x560e0e39c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e39c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 -- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 msgr2=0x560e0e39cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e39cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 -- 172.21.15.5:0/629040414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e0dfd63f0 con 0x560e0e35dd70 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9caa00640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e385910 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0149f0 tx=0x7fa9bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f627dad68c84d422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.637+0000 7fa9afe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9afe00640 1 -- 172.21.15.5:0/629040414 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc026020 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9b40567c0 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9afe00640 1 -- 172.21.15.5:0/629040414 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa9bc022440 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9afe00640 1 -- 172.21.15.5:0/629040414 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9bc030050 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9d4bfc740 10 client.6534 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x560e0df960f0 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9afe00640 1 -- 172.21.15.5:0/629040414 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b40567c0 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.638+0000 7fa9afe00640 1 -- 172.21.15.5:0/629040414 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9bc033020 con 0x560e0e35dd70 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9d4bfc740 10 client.6534 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9d4bfc740 10 client.6534 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9d4bfc740 20 client.6534 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e3b3480 0x560e0e38b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9d4bfc740 4 client.6534 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e0e2cf1b0 con 0x560e0e3b3480 2024-04-25T10:06:15.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e3b3480 0x560e0e38b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e3b3480 0x560e0e38b840 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54964c8bdc9f21d5 server_cookie=7cf241b095a65893 in_seq=0 out_seq=0 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.639+0000 7fa9afe00640 10 client.6534 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9afe00640 1 -- 172.21.15.5:0/629040414 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e0e2cf1b0 con 0x560e0e3b3480 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9afe00640 10 client.6534 handle_command_reply: tid=0 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 client.6534 shutdown 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 msgr2=0x560e0e385910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e385910 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0149f0 tx=0x7fa9bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 shutdown_connections 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e3b3480 0x560e0e38b840 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e39cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e39c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/629040414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e385910 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.640+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 >> 172.21.15.5:0/629040414 conn(0x560e0e325a30 msgr2=0x560e0e35c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.641+0000 7fa9afe00640 0 client.6534 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.641+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 shutdown_connections 2024-04-25T10:06:15.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.641+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/629040414 wait complete. 2024-04-25T10:06:15.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.641+0000 7fa9d4bfc740 20 client.6534 trim_cache size 0 max 16384 2024-04-25T10:06:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.658+0000 7fa9d4bfc740 1 Processor -- start 2024-04-25T10:06:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.658+0000 7fa9d4bfc740 1 -- start start 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.658+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e359990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e407010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e384790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc2004f0 con 0x560e0e35dd70 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1ff610 con 0x560e0e39a440 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1fea10 con 0x560e0e388830 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e359990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e359990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35552/0 (socket says 172.21.15.5:35552) 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 -- 172.21.15.5:0/1919247361 learned_addr learned my addr 172.21.15.5:0/1919247361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 -- 172.21.15.5:0/1919247361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 msgr2=0x560e0e407010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e407010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 -- 172.21.15.5:0/1919247361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 msgr2=0x560e0e384790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e384790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.659+0000 7fa9caa00640 1 -- 172.21.15.5:0/1919247361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0e354660 con 0x560e0e35dd70 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9caa00640 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e359990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009710 tx=0x7fa9bc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66903a57f38d96e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9baa00640 1 -- 172.21.15.5:0/1919247361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc026070 con 0x560e0e35dd70 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9baa00640 1 -- 172.21.15.5:0/1919247361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc01b030 con 0x560e0e35dd70 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9baa00640 1 -- 172.21.15.5:0/1919247361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc02bc10 con 0x560e0e35dd70 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/1919247361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 msgr2=0x560e0e359990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e359990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009710 tx=0x7fa9bc022000 comp rx=0 tx=0).stop 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/1919247361 shutdown_connections 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e39a440 0x560e0e384790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e388830 0x560e0e407010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/1919247361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e35dd70 0x560e0e359990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.660+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/1919247361 >> 172.21.15.5:0/1919247361 conn(0x560e0e325a30 msgr2=0x560e0e357bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.661+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/1919247361 shutdown_connections 2024-04-25T10:06:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.661+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/1919247361 wait complete. 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.661+0000 7fa9d4bfc740 1 -- start start 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.661+0000 7fa9d4bfc740 1 Processor -- start 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.662+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e405b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.662+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e39e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.662+0000 7fa9d4bfc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e39e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.662+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1ff010 con 0x560e0e39a440 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.662+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0e354390 con 0x560e0e388830 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.662+0000 7fa9d4bfc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e0e3c1d40 con 0x560e0e35dd70 2024-04-25T10:06:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9d4bfc740 10 client.0 _ll_register_callbacks cb 0x560e0e39e3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e39e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e39e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35560/0 (socket says 172.21.15.5:35560) 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 -- 172.21.15.5:0/528908936 learned_addr learned my addr 172.21.15.5:0/528908936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e39e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9caa00640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e405b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 -- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 msgr2=0x560e0e405b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e405b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 -- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 msgr2=0x560e0e39e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e39e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9cb400640 1 -- 172.21.15.5:0/528908936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e0e35c8f0 con 0x560e0e39a440 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9caa00640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e405b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.663+0000 7fa9c3e00640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e39e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9cb400640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e39e990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009eb0 tx=0x7fa9c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b7db79c476a45d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9afe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9afe00640 1 -- 172.21.15.5:0/528908936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c404a060 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9bc01b030 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9afe00640 1 -- 172.21.15.5:0/528908936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa9c4045410 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9afe00640 1 -- 172.21.15.5:0/528908936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c4050050 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9d4bfc740 10 client.6540 fetch_fsmap learned FSMap version 36 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.664+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x560e0e38c400 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9afe00640 1 -- 172.21.15.5:0/528908936 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc01b030 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9afe00640 1 -- 172.21.15.5:0/528908936 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa9c4053020 con 0x560e0e39a440 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9d4bfc740 10 client.6540 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9d4bfc740 10 client.6540 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9d4bfc740 20 client.6540 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e4061b0 0x560e0e38b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9d4bfc740 4 client.6540 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e0e2c9500 con 0x560e0e4061b0 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.665+0000 7fa9caa00640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e4061b0 0x560e0e38b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9caa00640 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e4061b0 0x560e0e38b490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aeb48b52e38423b server_cookie=e3e9686651e81b66 in_seq=0 out_seq=0 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9afe00640 10 client.6540 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9afe00640 1 -- 172.21.15.5:0/528908936 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x560e0e2c9500 con 0x560e0e4061b0 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9afe00640 10 client.6540 handle_command_reply: tid=0 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 client.6540 shutdown 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 msgr2=0x560e0e39e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e39e990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009eb0 tx=0x7fa9c4004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 shutdown_connections 2024-04-25T10:06:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x560e0e4061b0 0x560e0e38b490 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e0e39a440 0x560e0e39e990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560e0e388830 0x560e0e39e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 --2- 172.21.15.5:0/528908936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560e0e35dd70 0x560e0e405b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 >> 172.21.15.5:0/528908936 conn(0x560e0e325a30 msgr2=0x560e0e3c5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9afe00640 0 client.6540 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.666+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 shutdown_connections 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9d4bfc740 1 -- 172.21.15.5:0/528908936 wait complete. 2024-04-25T10:06:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9d4bfc740 20 client.6540 trim_cache size 0 max 16384 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: "6a16c086-9b83-4f4c-a99d-0d9b3044051b": { 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6a16c086-9b83-4f4c-a99d-0d9b3044051b", 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair,force" 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:06:15.677 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa9a806b500 msgr2=0x7fa9a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa9a806b500 0x7fa9a806d9c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc006ca0 tx=0x7fa9bc009240 comp rx=0 tx=0).stop 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 msgr2=0x7fa9cc146490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 0x7fa9cc146490 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400c450 tx=0x7fa9b400c910 comp rx=0 tx=0).stop 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 shutdown_connections 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9cc143d10 0x7fa9cc141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9cc0adac0 0x7fa9cc146490 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa9a806b500 0x7fa9a806d9c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.667+0000 7fa9cbe00640 1 --2- 172.21.15.5:0/1557955384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9cc071710 0x7fa9cc13fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.668+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 >> 172.21.15.5:0/1557955384 conn(0x7fa9cc066f20 msgr2=0x7fa9cc06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.668+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 shutdown_connections 2024-04-25T10:06:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:15.668+0000 7fa9cbe00640 1 -- 172.21.15.5:0/1557955384 wait complete. 2024-04-25T10:06:15.689 INFO:tasks.cephfs.filesystem:scrub status for tag:6a16c086-9b83-4f4c-a99d-0d9b3044051b - {'path': '/', 'tag': '6a16c086-9b83-4f4c-a99d-0d9b3044051b', 'options': 'recursive,repair,force'} 2024-04-25T10:06:24.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:24.388 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:45.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:45.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:06:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.916+0000 7f8a6fe00640 1 Processor -- start 2024-04-25T10:06:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.916+0000 7f8a6fe00640 1 -- start start 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a7015ed20 0x7f8a70144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70144e90 0x7f8a70071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7015f3e0 con 0x7f8a70071540 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a700719b0 con 0x7f8a7015ed20 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a70144780 con 0x7f8a70144e90 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a7015ed20 0x7f8a70144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a7015ed20 0x7f8a70144950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47770/0 (socket says 172.21.15.5:47770) 2024-04-25T10:06:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3795372226 learned_addr learned my addr 172.21.15.5:0/3795372226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.918+0000 7f8a67e00640 1 --2- 172.21.15.5:0/3795372226 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70144e90 0x7f8a70071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.919+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3795372226 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70144e90 msgr2=0x7f8a70071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.919+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3795372226 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70144e90 0x7f8a70071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.919+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3795372226 >> v1:172.21.15.80:6789/0 conn(0x7f8a70071540 legacy=0x7f8a7006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.919+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3795372226 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7015f680 con 0x7f8a7015ed20 2024-04-25T10:06:45.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.921+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3795372226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7015ed20 0x7f8a70144950 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdbb32864313b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.921+0000 7f8a6e000640 1 -- 172.21.15.5:0/3795372226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60037070 con 0x7f8a7015ed20 2024-04-25T10:06:45.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.921+0000 7f8a6e000640 1 -- 172.21.15.5:0/3795372226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60032440 con 0x7f8a7015ed20 2024-04-25T10:06:45.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.921+0000 7f8a6e000640 1 -- 172.21.15.5:0/3795372226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60032810 con 0x7f8a7015ed20 2024-04-25T10:06:45.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.922+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3795372226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7015ed20 msgr2=0x7f8a70144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.922+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3795372226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7015ed20 0x7f8a70144950 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.922+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3795372226 shutdown_connections 2024-04-25T10:06:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.922+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3795372226 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70144e90 0x7f8a70071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.922+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3795372226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a7015ed20 0x7f8a70144950 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.922+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3795372226 >> 172.21.15.5:0/3795372226 conn(0x7f8a70066f20 msgr2=0x7f8a7006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.923+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3795372226 shutdown_connections 2024-04-25T10:06:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.923+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3795372226 wait complete. 2024-04-25T10:06:45.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.925+0000 7f8a6fe00640 1 Processor -- start 2024-04-25T10:06:45.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.925+0000 7f8a6fe00640 1 -- start start 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70071540 0x7f8a700ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 0x7f8a700cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a6fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7015ed20 0x7f8a7015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a700cdd50 con 0x7f8a70144e90 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015fc40 con 0x7f8a7015ed20 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a6fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7015cdd0 con 0x7f8a70071540 2024-04-25T10:06:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 0x7f8a700cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 0x7f8a700cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47784/0 (socket says 172.21.15.5:47784) 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.926+0000 7f8a67e00640 1 -- 172.21.15.5:0/3027940137 learned_addr learned my addr 172.21.15.5:0/3027940137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70071540 0x7f8a700ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a6f400640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7015ed20 0x7f8a7015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a67e00640 1 -- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70071540 msgr2=0x7f8a700ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a67e00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70071540 0x7f8a700ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a67e00640 1 -- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7015ed20 msgr2=0x7f8a7015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a67e00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7015ed20 0x7f8a7015b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a67e00640 1 -- 172.21.15.5:0/3027940137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a700cab00 con 0x7f8a70144e90 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a6f400640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7015ed20 0x7f8a7015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a67e00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 0x7f8a700cc5a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d800 tx=0x7f8a5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a66a00640 1 -- 172.21.15.5:0/3027940137 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800b6c0 con 0x7f8a70144e90 2024-04-25T10:06:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a66a00640 1 -- 172.21.15.5:0/3027940137 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58019070 con 0x7f8a70144e90 2024-04-25T10:06:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a700cade0 con 0x7f8a70144e90 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.927+0000 7f8a66a00640 1 -- 172.21.15.5:0/3027940137 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800ba80 con 0x7f8a70144e90 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.928+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a700cb3a0 con 0x7f8a70144e90 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.930+0000 7f8a66a00640 1 -- 172.21.15.5:0/3027940137 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a5800bc20 con 0x7f8a70144e90 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.931+0000 7f8a66a00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a4c06b500 0x7f8a4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.931+0000 7f8a66a00640 1 -- 172.21.15.5:0/3027940137 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8a5808a9c0 con 0x7f8a70144e90 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.932+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a4c06b500 0x7f8a4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.932+0000 7f8a78a35740 1 Processor -- start 2024-04-25T10:06:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.932+0000 7f8a78a35740 1 -- start start 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a4c06b500 0x7f8a4c06d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a60034000 tx=0x7f8a6002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f04ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f03150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a78a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5e66c70 con 0x55f9a5f3f440 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5e0fcf0 con 0x55f9a5f02d70 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5e62d10 con 0x55f9a5f2d830 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.933+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f03150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f03150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47786/0 (socket says 172.21.15.5:47786) 2024-04-25T10:06:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 -- 172.21.15.5:0/1825244276 learned_addr learned my addr 172.21.15.5:0/1825244276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f04ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a6f400640 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 -- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 msgr2=0x55f9a5f2dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 -- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 msgr2=0x55f9a5f04ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f04ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 -- 172.21.15.5:0/1825244276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9a5c457d0 con 0x55f9a5f3f440 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a6f400640 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.934+0000 7f8a67e00640 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f03150 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8a58061050 tx=0x7f8a58058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631e83b0c437106f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a5f400640 1 -- 172.21.15.5:0/1825244276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58070030 con 0x55f9a5f3f440 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a5f400640 1 -- 172.21.15.5:0/1825244276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5805cf30 con 0x55f9a5f3f440 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a5f400640 1 -- 172.21.15.5:0/1825244276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58061580 con 0x55f9a5f3f440 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 -- 172.21.15.5:0/1825244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 msgr2=0x55f9a5f03150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f03150 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8a58061050 tx=0x7f8a58058ce0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 -- 172.21.15.5:0/1825244276 shutdown_connections 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f03150 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 --2- 172.21.15.5:0/1825244276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f04ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.935+0000 7f8a78a35740 1 -- 172.21.15.5:0/1825244276 >> 172.21.15.5:0/1825244276 conn(0x55f9a5ecaa30 msgr2=0x55f9a5f48830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.936+0000 7f8a78a35740 1 -- 172.21.15.5:0/1825244276 shutdown_connections 2024-04-25T10:06:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.936+0000 7f8a78a35740 1 -- 172.21.15.5:0/1825244276 wait complete. 2024-04-25T10:06:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.936+0000 7f8a78a35740 1 -- start start 2024-04-25T10:06:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.936+0000 7f8a78a35740 1 Processor -- start 2024-04-25T10:06:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5f225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5e053a0 con 0x55f9a5f02d70 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5f2b380 con 0x55f9a5f2d830 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5f241c0 con 0x55f9a5f3f440 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a78a35740 10 client.0 _ll_register_callbacks cb 0x55f9a5e22b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5f225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5f225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47788/0 (socket says 172.21.15.5:47788) 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 -- 172.21.15.5:0/4166035843 learned_addr learned my addr 172.21.15.5:0/4166035843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a67e00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6f400640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 -- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 msgr2=0x55f9a5f27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f27130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 -- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 msgr2=0x55f9a5f30c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f30c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a6ea00640 1 -- 172.21.15.5:0/4166035843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9a5b7b3f0 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.938+0000 7f8a67e00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f30c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.939+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5f225f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210e7b2577342ce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.939+0000 7f8a53e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.939+0000 7f8a53e00640 1 -- 172.21.15.5:0/4166035843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60037040 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.939+0000 7f8a53e00640 1 -- 172.21.15.5:0/4166035843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8a6003ca30 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.939+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a5805cf30 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.939+0000 7f8a53e00640 1 -- 172.21.15.5:0/4166035843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a6003cc30 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 10 client.6552 fetch_fsmap learned FSMap version 37 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55f9a5b3b0f0 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a53e00640 1 -- 172.21.15.5:0/4166035843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5805cf30 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a53e00640 1 -- 172.21.15.5:0/4166035843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8a60058020 con 0x55f9a5f02d70 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 10 client.6552 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T10:06:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 10 client.6552 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 20 client.6552 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f2f060 0x55f9a5f2f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 4 client.6552 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9a5e741b0 con 0x55f9a5f2f060 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.940+0000 7f8a67e00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f2f060 0x55f9a5f2f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.941+0000 7f8a67e00640 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f2f060 0x55f9a5f2f4d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553404f83f95f754 server_cookie=641056e5c9323db9 in_seq=0 out_seq=0 2024-04-25T10:06:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.941+0000 7f8a53e00640 10 client.6552 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a53e00640 1 -- 172.21.15.5:0/4166035843 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9a5e741b0 con 0x55f9a5f2f060 2024-04-25T10:06:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a53e00640 10 client.6552 handle_command_reply: tid=0 2024-04-25T10:06:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 client.6552 shutdown 2024-04-25T10:06:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 msgr2=0x55f9a5f225f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5f225f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6002c070 comp rx=0 tx=0).stop 2024-04-25T10:06:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 shutdown_connections 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f2f060 0x55f9a5f2f4d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f30c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 --2- 172.21.15.5:0/4166035843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5f225f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a53e00640 0 client.6552 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 >> 172.21.15.5:0/4166035843 conn(0x55f9a5ecaa30 msgr2=0x55f9a5f4bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 shutdown_connections 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 1 -- 172.21.15.5:0/4166035843 wait complete. 2024-04-25T10:06:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.942+0000 7f8a78a35740 20 client.6552 trim_cache size 0 max 16384 2024-04-25T10:06:45.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.959+0000 7f8a78a35740 1 Processor -- start 2024-04-25T10:06:45.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.959+0000 7f8a78a35740 1 -- start start 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f266b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a78a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a702004e0 con 0x55f9a5f3f440 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701ff600 con 0x55f9a5f02d70 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701f4700 con 0x55f9a5f2d830 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.960+0000 7f8a6f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f266b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f266b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47792/0 (socket says 172.21.15.5:47792) 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 -- 172.21.15.5:0/2522751975 learned_addr learned my addr 172.21.15.5:0/2522751975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a67e00640 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 -- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 msgr2=0x55f9a5f2dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 -- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 msgr2=0x55f9a5f02620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f02620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 -- 172.21.15.5:0/2522751975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9a5f6b900 con 0x55f9a5f3f440 2024-04-25T10:06:45.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.961+0000 7f8a6f400640 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f266b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8a68001c60 tx=0x7f8a68002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19a4d77ac424497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a5ea00640 1 -- 172.21.15.5:0/2522751975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a68003da0 con 0x55f9a5f3f440 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a5ea00640 1 -- 172.21.15.5:0/2522751975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68018070 con 0x55f9a5f3f440 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a5ea00640 1 -- 172.21.15.5:0/2522751975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6800fab0 con 0x55f9a5f3f440 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 -- 172.21.15.5:0/2522751975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 msgr2=0x55f9a5f266b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f266b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8a68001c60 tx=0x7f8a68002120 comp rx=0 tx=0).stop 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 -- 172.21.15.5:0/2522751975 shutdown_connections 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f3f440 0x55f9a5f266b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f2d830 0x55f9a5f2dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 --2- 172.21.15.5:0/2522751975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f02d70 0x55f9a5f02620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 -- 172.21.15.5:0/2522751975 >> 172.21.15.5:0/2522751975 conn(0x55f9a5ecaa30 msgr2=0x55f9a5f26e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 -- 172.21.15.5:0/2522751975 shutdown_connections 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.962+0000 7f8a78a35740 1 -- 172.21.15.5:0/2522751975 wait complete. 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.963+0000 7f8a78a35740 1 -- start start 2024-04-25T10:06:45.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.963+0000 7f8a78a35740 1 Processor -- start 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5fb5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f3ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5f2aeb0 con 0x55f9a5f02d70 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701ff030 con 0x55f9a5f2d830 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5f22bf0 con 0x55f9a5f3f440 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.964+0000 7f8a78a35740 10 client.0 _ll_register_callbacks cb 0x55f9a5f2d010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5fb5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5fb5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47802/0 (socket says 172.21.15.5:47802) 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3337085605 learned_addr learned my addr 172.21.15.5:0/3337085605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3337085605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 msgr2=0x55f9a5f2b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3337085605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 msgr2=0x55f9a5f3ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f3ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 -- 172.21.15.5:0/3337085605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9a5f23e00 con 0x55f9a5f02d70 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a6ea00640 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5fb5fa0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8a6003d020 tx=0x7f8a60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acda0c0e0b35648a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.965+0000 7f8a53e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a53e00640 1 -- 172.21.15.5:0/3337085605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60037040 con 0x55f9a5f02d70 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a53e00640 1 -- 172.21.15.5:0/3337085605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8a600509f0 con 0x55f9a5f02d70 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a68018070 con 0x55f9a5f02d70 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a53e00640 1 -- 172.21.15.5:0/3337085605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a60067050 con 0x55f9a5f02d70 2024-04-25T10:06:45.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 10 client.6558 fetch_fsmap learned FSMap version 37 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55f9a5f23c00 con 0x55f9a5f02d70 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a53e00640 1 -- 172.21.15.5:0/3337085605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a68018070 con 0x55f9a5f02d70 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a53e00640 1 -- 172.21.15.5:0/3337085605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8a6006a020 con 0x55f9a5f02d70 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 10 client.6558 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 10 client.6558 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 20 client.6558 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f3e410 0x55f9a5f2b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 4 client.6558 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.966+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9a5f2e8f0 con 0x55f9a5f3e410 2024-04-25T10:06:45.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.967+0000 7f8a67e00640 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f3e410 0x55f9a5f2b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.967+0000 7f8a67e00640 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f3e410 0x55f9a5f2b850 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe34d7b005d140f server_cookie=b7b1624cde969adf in_seq=0 out_seq=0 2024-04-25T10:06:45.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.967+0000 7f8a53e00640 10 client.6558 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a53e00640 1 -- 172.21.15.5:0/3337085605 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f9a5f2e8f0 con 0x55f9a5f3e410 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a53e00640 10 client.6558 handle_command_reply: tid=0 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 client.6558 shutdown 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 msgr2=0x55f9a5fb5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5fb5fa0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8a6003d020 tx=0x7f8a60004730 comp rx=0 tx=0).stop 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 shutdown_connections 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55f9a5f3e410 0x55f9a5f2b850 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f9a5f3f440 0x55f9a5f2b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f9a5f2d830 0x55f9a5f3ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 --2- 172.21.15.5:0/3337085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9a5f02d70 0x55f9a5fb5fa0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a53e00640 0 client.6558 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 >> 172.21.15.5:0/3337085605 conn(0x55f9a5ecaa30 msgr2=0x55f9a5efcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 shutdown_connections 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 1 -- 172.21.15.5:0/3337085605 wait complete. 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.968+0000 7f8a78a35740 20 client.6558 trim_cache size 0 max 16384 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:06:45.978 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:06:45.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a4c06b500 msgr2=0x7f8a4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a4c06b500 0x7f8a4c06d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a60034000 tx=0x7f8a6002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:45.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 msgr2=0x7f8a700cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 0x7f8a700cc5a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d800 tx=0x7f8a5800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 shutdown_connections 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a4c06b500 0x7f8a4c06d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a7015ed20 0x7f8a7015b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a70144e90 0x7f8a700cc5a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 --2- 172.21.15.5:0/3027940137 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a70071540 0x7f8a700ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 >> 172.21.15.5:0/3027940137 conn(0x7f8a70066f20 msgr2=0x7f8a7015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 shutdown_connections 2024-04-25T10:06:45.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:45.969+0000 7f8a6fe00640 1 -- 172.21.15.5:0/3027940137 wait complete. 2024-04-25T10:06:45.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:45.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.191+0000 7f817e800640 1 Processor -- start 2024-04-25T10:06:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.191+0000 7f817e800640 1 -- start start 2024-04-25T10:06:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.192+0000 7f817e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 0x7f817815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.192+0000 7f817e800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8178071d30 con 0x7f817815ce00 2024-04-25T10:06:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.192+0000 7f817e800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f817806d620 con 0x7f8178141da0 2024-04-25T10:06:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.192+0000 7f817e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8178142180 con 0x7f81780cda10 2024-04-25T10:06:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.193+0000 7f817ca00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f817815ce00 0x7f81780cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38280/0 (socket says 172.21.15.5:38280) 2024-04-25T10:06:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.193+0000 7f817ca00640 1 -- 172.21.15.5:0/894687765 learned_addr learned my addr 172.21.15.5:0/894687765 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.193+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825388964 0 0) 0x7f8178071d30 con 0x7f817815ce00 2024-04-25T10:06:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.193+0000 7f817de00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 0x7f817815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.193+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8160003520 con 0x7f817815ce00 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205680244 0 0) 0x7f8160003520 con 0x7f817815ce00 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 msgr2=0x7f817815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 0x7f817815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 >> v1:172.21.15.80:6789/0 conn(0x7f8178141da0 legacy=0x7f817806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817806d900 con 0x7f817815ce00 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f816c001670 con 0x7f817815ce00 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 >> v1:172.21.15.5:6789/0 conn(0x7f817815ce00 legacy=0x7f81780cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 0x7f81600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8160003fb0 0x7f8160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8160004d40 0x7f81600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178142180 con 0x7f8160004d40 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816000bb00 con 0x7f8160003fb0 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.194+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816000bd00 con 0x7f81780cda10 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817de00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8160004d40 0x7f81600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817d400640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 0x7f81600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817ca00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8160003fb0 0x7f8160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817de00640 1 -- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 msgr2=0x7f81600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817de00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780cda10 0x7f81600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817de00640 1 -- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8160003fb0 msgr2=0x7f8160004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817de00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8160003fb0 0x7f8160004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817de00640 1 -- 172.21.15.5:0/894687765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160003520 con 0x7f8160004d40 2024-04-25T10:06:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.195+0000 7f817ca00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8160003fb0 0x7f8160004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.196+0000 7f817de00640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8160004d40 0x7f81600051c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8170009f90 tx=0x7f8160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a458fcab0b1d4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.196+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170009c10 con 0x7f8160004d40 2024-04-25T10:06:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170036070 con 0x7f8160004d40 2024-04-25T10:06:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f8177e00640 1 -- 172.21.15.5:0/894687765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170031850 con 0x7f8160004d40 2024-04-25T10:06:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817e800640 1 -- 172.21.15.5:0/894687765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8160004d40 msgr2=0x7f81600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817e800640 1 --2- 172.21.15.5:0/894687765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8160004d40 0x7f81600051c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8170009f90 tx=0x7f8160005830 comp rx=0 tx=0).stop 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817de00640 1 -- 172.21.15.5:0/894687765 reap_dead start 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817e800640 1 -- 172.21.15.5:0/894687765 shutdown_connections 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817e800640 1 -- 172.21.15.5:0/894687765 >> 172.21.15.5:0/894687765 conn(0x7f8178066f20 msgr2=0x7f817806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817e800640 1 -- 172.21.15.5:0/894687765 shutdown_connections 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.197+0000 7f817e800640 1 -- 172.21.15.5:0/894687765 wait complete. 2024-04-25T10:06:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.199+0000 7f817e800640 1 Processor -- start 2024-04-25T10:06:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.199+0000 7f817e800640 1 -- start start 2024-04-25T10:06:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.200+0000 7f817e800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817815b1f0 0x7f81780ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.200+0000 7f817e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 0x7f81780ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.200+0000 7f817e800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780aa860 0x7f81780aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.200+0000 7f817e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817815d280 con 0x7f81780ad1c0 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.200+0000 7f817e800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817806e810 con 0x7f817815b1f0 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.200+0000 7f817e800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817815d400 con 0x7f81780aa860 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 0x7f81780ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 0x7f81780ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47822/0 (socket says 172.21.15.5:47822) 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 -- 172.21.15.5:0/4032108279 learned_addr learned my addr 172.21.15.5:0/4032108279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 -- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780aa860 msgr2=0x7f81780aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780aa860 0x7f81780aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 -- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817815b1f0 msgr2=0x7f81780ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817815b1f0 0x7f81780ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.201+0000 7f817ca00640 1 -- 172.21.15.5:0/4032108279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817815b5d0 con 0x7f81780ad1c0 2024-04-25T10:06:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.202+0000 7f817ca00640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 0x7f81780ad640 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f816c00d940 tx=0x7f816c00de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.202+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c019070 con 0x7f81780ad1c0 2024-04-25T10:06:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.202+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816c0074e0 con 0x7f81780ad1c0 2024-04-25T10:06:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.202+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0049e0 con 0x7f81780ad1c0 2024-04-25T10:06:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.202+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81780ab590 con 0x7f81780ad1c0 2024-04-25T10:06:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.202+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81780abb50 con 0x7f81780ad1c0 2024-04-25T10:06:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.203+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178068f60 con 0x7f81780ad1c0 2024-04-25T10:06:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.204+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f816c01c070 con 0x7f81780ad1c0 2024-04-25T10:06:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.205+0000 7f8175600640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f815c06b4d0 0x7f815c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.206+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f816c089fd0 con 0x7f81780ad1c0 2024-04-25T10:06:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.206+0000 7f817d400640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f815c06b4d0 0x7f815c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.208+0000 7f817d400640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f815c06b4d0 0x7f815c06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f81680014f0 tx=0x7f8168009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.210+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f816c058b80 con 0x7f81780ad1c0 2024-04-25T10:06:46.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.453+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f817806e5d0 con 0x7f81780ad1c0 2024-04-25T10:06:46.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.455+0000 7f8175600640 1 -- 172.21.15.5:0/4032108279 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f816c058480 con 0x7f81780ad1c0 2024-04-25T10:06:46.461 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:46.461 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:06:21.391577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:46.461 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f815c06b4d0 msgr2=0x7f815c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f815c06b4d0 0x7f815c06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f81680014f0 tx=0x7f8168009040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 msgr2=0x7f81780ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 0x7f81780ad640 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f816c00d940 tx=0x7f816c00de00 comp rx=0 tx=0).stop 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 shutdown_connections 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f81780aa860 0x7f81780aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81780ad1c0 0x7f81780ad640 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f815c06b4d0 0x7f815c06d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 --2- 172.21.15.5:0/4032108279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817815b1f0 0x7f81780ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 >> 172.21.15.5:0/4032108279 conn(0x7f8178066f20 msgr2=0x7f81780b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 shutdown_connections 2024-04-25T10:06:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.457+0000 7f817e800640 1 -- 172.21.15.5:0/4032108279 wait complete. 2024-04-25T10:06:46.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:06:46.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.690+0000 7feeb7200640 1 Processor -- start 2024-04-25T10:06:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.690+0000 7feeb7200640 1 -- start start 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feeb815ed20 0x7feeb8144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb815f3e0 con 0x7feeb8144e90 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb7200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb815f680 con 0x7feeb8071710 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb8144780 con 0x7feeb815ed20 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feeb815ed20 0x7feeb8144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7feeb8144e90 0x7feeb8071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38282/0 (socket says 172.21.15.5:38282) 2024-04-25T10:06:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.692+0000 7feeb5400640 1 -- 172.21.15.5:0/963959140 learned_addr learned my addr 172.21.15.5:0/963959140 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.693+0000 7feeb5e00640 1 -- 172.21.15.5:0/963959140 >> v1:172.21.15.80:6789/0 conn(0x7feeb8071710 legacy=0x7feeb815bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.693+0000 7feeb5e00640 1 -- 172.21.15.5:0/963959140 >> v1:172.21.15.5:6789/0 conn(0x7feeb8144e90 legacy=0x7feeb8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.693+0000 7feeb5e00640 1 -- 172.21.15.5:0/963959140 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb815f960 con 0x7feeb815ed20 2024-04-25T10:06:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.693+0000 7feeb4a00640 1 -- 172.21.15.5:0/963959140 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163545494 0 0) 0x7feeb815f3e0 con 0x7feeb8144e90 2024-04-25T10:06:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.694+0000 7feeb5e00640 1 --2- 172.21.15.5:0/963959140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb815ed20 0x7feeb8144950 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7feea80097a0 tx=0x7feea802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b569fc1e2ca94398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.695+0000 7feeb4a00640 1 -- 172.21.15.5:0/963959140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8037070 con 0x7feeb815ed20 2024-04-25T10:06:46.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.695+0000 7feeb4a00640 1 -- 172.21.15.5:0/963959140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea802adc0 con 0x7feeb815ed20 2024-04-25T10:06:46.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.695+0000 7feeb4a00640 1 -- 172.21.15.5:0/963959140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8032660 con 0x7feeb815ed20 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.695+0000 7feeb7200640 1 -- 172.21.15.5:0/963959140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb815ed20 msgr2=0x7feeb8144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.695+0000 7feeb7200640 1 --2- 172.21.15.5:0/963959140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb815ed20 0x7feeb8144950 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7feea80097a0 tx=0x7feea802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.696+0000 7feeb7200640 1 -- 172.21.15.5:0/963959140 shutdown_connections 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.696+0000 7feeb7200640 1 --2- 172.21.15.5:0/963959140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb815ed20 0x7feeb8144950 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.696+0000 7feeb7200640 1 -- 172.21.15.5:0/963959140 >> 172.21.15.5:0/963959140 conn(0x7feeb8066f20 msgr2=0x7feeb806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.696+0000 7feeb7200640 1 -- 172.21.15.5:0/963959140 shutdown_connections 2024-04-25T10:06:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.696+0000 7feeb7200640 1 -- 172.21.15.5:0/963959140 wait complete. 2024-04-25T10:06:46.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.697+0000 7feeb7200640 1 Processor -- start 2024-04-25T10:06:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.698+0000 7feeb7200640 1 -- start start 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feeb8071710 0x7feeb80b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 0x7feeb8152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feeb815ed20 0x7feeb80ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815c4d0 con 0x7feeb8144e90 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb80cdd50 con 0x7feeb815ed20 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815c650 con 0x7feeb8071710 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 0x7feeb8152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 0x7feeb8152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47832/0 (socket says 172.21.15.5:47832) 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 -- 172.21.15.5:0/110244290 learned_addr learned my addr 172.21.15.5:0/110244290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5e00640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feeb8071710 0x7feeb80b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb6800640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feeb815ed20 0x7feeb80ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 -- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feeb8071710 msgr2=0x7feeb80b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feeb8071710 0x7feeb80b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 -- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feeb815ed20 msgr2=0x7feeb80ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feeb815ed20 0x7feeb80ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb5400640 1 -- 172.21.15.5:0/110244290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb815b450 con 0x7feeb8144e90 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.699+0000 7feeb6800640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feeb815ed20 0x7feeb80ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.700+0000 7feeb5400640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 0x7feeb8152dc0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7feea400c4d0 tx=0x7feea400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.700+0000 7feeae000640 1 -- 172.21.15.5:0/110244290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4018070 con 0x7feeb8144e90 2024-04-25T10:06:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.700+0000 7feeae000640 1 -- 172.21.15.5:0/110244290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4007500 con 0x7feeb8144e90 2024-04-25T10:06:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.700+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb80b2bb0 con 0x7feeb8144e90 2024-04-25T10:06:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.700+0000 7feeae000640 1 -- 172.21.15.5:0/110244290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4004a20 con 0x7feeb8144e90 2024-04-25T10:06:46.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.700+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb80b3170 con 0x7feeb8144e90 2024-04-25T10:06:46.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.703+0000 7feeae000640 1 -- 172.21.15.5:0/110244290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feea401b070 con 0x7feeb8144e90 2024-04-25T10:06:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.704+0000 7feeae000640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fee9806b4d0 0x7fee9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.704+0000 7feeae000640 1 -- 172.21.15.5:0/110244290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7feea4089750 con 0x7feeb8144e90 2024-04-25T10:06:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.704+0000 7feebf98d740 1 Processor -- start 2024-04-25T10:06:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.704+0000 7feebf98d740 1 -- start start 2024-04-25T10:06:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feeb5e00640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fee9806b4d0 0x7fee9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feebf98d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 0x55d9837ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ebdf0 0x55d9837ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837a4230 0x55d9837ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feebf98d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d983706ae0 con 0x55d9837ce550 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9836af7c0 con 0x55d9837a4230 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.705+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d983702b80 con 0x55d9837ebdf0 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 0x55d9837ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 0x55d9837ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47840/0 (socket says 172.21.15.5:47840) 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 -- 172.21.15.5:0/2622168048 learned_addr learned my addr 172.21.15.5:0/2622168048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb6800640 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837a4230 0x55d9837ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fee9806b4d0 0x7fee9806d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feea8034000 tx=0x7feea802c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5400640 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ebdf0 0x55d9837ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 -- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ebdf0 msgr2=0x55d9837ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ebdf0 0x55d9837ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 -- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837a4230 msgr2=0x55d9837ea4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837a4230 0x55d9837ea4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 -- 172.21.15.5:0/2622168048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9834e77c0 con 0x55d9837ce550 2024-04-25T10:06:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.706+0000 7feeb5e00640 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 0x55d9837ce930 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7feea802a0f0 tx=0x7feea803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20a4220f6d1687a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.707+0000 7feea2600640 1 -- 172.21.15.5:0/2622168048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8045070 con 0x55d9837ce550 2024-04-25T10:06:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.707+0000 7feea2600640 1 -- 172.21.15.5:0/2622168048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea804e070 con 0x55d9837ce550 2024-04-25T10:06:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.707+0000 7feea2600640 1 -- 172.21.15.5:0/2622168048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8037040 con 0x55d9837ce550 2024-04-25T10:06:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.707+0000 7feebf98d740 1 -- 172.21.15.5:0/2622168048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 msgr2=0x55d9837ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.707+0000 7feebf98d740 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 0x55d9837ce930 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7feea802a0f0 tx=0x7feea803e000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 -- 172.21.15.5:0/2622168048 shutdown_connections 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837a4230 0x55d9837ea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ebdf0 0x55d9837ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 --2- 172.21.15.5:0/2622168048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9837ce550 0x55d9837ce930 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 -- 172.21.15.5:0/2622168048 >> 172.21.15.5:0/2622168048 conn(0x55d9837a0b60 msgr2=0x55d9837e9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 -- 172.21.15.5:0/2622168048 shutdown_connections 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 -- 172.21.15.5:0/2622168048 wait complete. 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.708+0000 7feebf98d740 1 -- start start 2024-04-25T10:06:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.709+0000 7feebf98d740 1 Processor -- start 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98379dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9837a4fd0 con 0x55d98379d780 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9836a4d40 con 0x55d9837ebdf0 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d98379ea90 con 0x55d9837ce550 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feebf98d740 10 client.0 _ll_register_callbacks cb 0x55d98341aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feeb6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98379dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feeb6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98379dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47854/0 (socket says 172.21.15.5:47854) 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feeb6800640 1 -- 172.21.15.5:0/221149429 learned_addr learned my addr 172.21.15.5:0/221149429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feeb5400640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.710+0000 7feeb5e00640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb6800640 1 -- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 msgr2=0x55d9837bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb6800640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb6800640 1 -- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 msgr2=0x55d9837d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb6800640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb6800640 1 -- 172.21.15.5:0/221149429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d983547f50 con 0x55d98379d780 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb5e00640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feeb6800640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98379dc00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7feeb000f510 tx=0x7feeb000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87655bfc0198870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7fee96a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7fee96a00640 1 -- 172.21.15.5:0/221149429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0003be0 con 0x55d98379d780 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7fee96a00640 1 -- 172.21.15.5:0/221149429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7feeb0003d80 con 0x55d98379d780 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feea804e070 con 0x55d98379d780 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.711+0000 7fee96a00640 1 -- 172.21.15.5:0/221149429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feeb0022050 con 0x55d98379d780 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 10 client.6579 fetch_fsmap learned FSMap version 37 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55d9833db0b0 con 0x55d98379d780 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7fee96a00640 1 -- 172.21.15.5:0/221149429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea804e070 con 0x55d98379d780 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7fee96a00640 1 -- 172.21.15.5:0/221149429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feeb0025020 con 0x55d98379d780 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 10 client.6579 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 10 client.6579 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 20 client.6579 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837e0770 0x55d9837e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 4 client.6579 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d983765b60 con 0x55d9837e0770 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.712+0000 7feeb5e00640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837e0770 0x55d9837e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.713+0000 7feeb5e00640 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837e0770 0x55d9837e0b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5702b21d87edf4e0 server_cookie=8cba699b11ef33b3 in_seq=0 out_seq=0 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.713+0000 7fee96a00640 10 client.6579 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7fee96a00640 1 -- 172.21.15.5:0/221149429 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d983765b60 con 0x55d9837e0770 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7fee96a00640 10 client.6579 handle_command_reply: tid=0 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 client.6579 shutdown 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 msgr2=0x55d98379dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98379dc00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7feeb000f510 tx=0x7feeb000f9d0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 shutdown_connections 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837e0770 0x55d9837e0b50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98379dc00 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 --2- 172.21.15.5:0/221149429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7fee96a00640 0 client.6579 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 >> 172.21.15.5:0/221149429 conn(0x55d9837a4230 msgr2=0x55d9837ed170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 shutdown_connections 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.714+0000 7feebf98d740 1 -- 172.21.15.5:0/221149429 wait complete. 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.715+0000 7feebf98d740 20 client.6579 trim_cache size 0 max 16384 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.728+0000 7feebf98d740 1 Processor -- start 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.728+0000 7feebf98d740 1 -- start start 2024-04-25T10:06:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feebf98d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d9837a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feebf98d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb81ff540 con 0x55d98379d780 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb81f42e0 con 0x55d9837ebdf0 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb81fef70 con 0x55d9837ce550 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d9837a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d9837a3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47862/0 (socket says 172.21.15.5:47862) 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 -- 172.21.15.5:0/2188630933 learned_addr learned my addr 172.21.15.5:0/2188630933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 -- 172.21.15.5:0/2188630933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 msgr2=0x55d9837a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 -- 172.21.15.5:0/2188630933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 msgr2=0x55d9837d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.729+0000 7feeb5e00640 1 -- 172.21.15.5:0/2188630933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9837e1ad0 con 0x55d98379d780 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.730+0000 7feeb5e00640 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d9837a3730 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7feea80098c0 tx=0x7feea8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37943b6a6b416898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.730+0000 7feea1c00640 1 -- 172.21.15.5:0/2188630933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8045040 con 0x55d98379d780 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.730+0000 7feea1c00640 1 -- 172.21.15.5:0/2188630933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8058020 con 0x55d98379d780 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.730+0000 7feea1c00640 1 -- 172.21.15.5:0/2188630933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8037040 con 0x55d98379d780 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 -- 172.21.15.5:0/2188630933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 msgr2=0x55d9837a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d9837a3730 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7feea80098c0 tx=0x7feea8008030 comp rx=0 tx=0).stop 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 -- 172.21.15.5:0/2188630933 shutdown_connections 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d9837a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 --2- 172.21.15.5:0/2188630933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d9837a3730 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 -- 172.21.15.5:0/2188630933 >> 172.21.15.5:0/2188630933 conn(0x55d9837a4230 msgr2=0x55d98379db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 -- 172.21.15.5:0/2188630933 shutdown_connections 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 -- 172.21.15.5:0/2188630933 wait complete. 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.731+0000 7feebf98d740 1 -- start start 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.732+0000 7feebf98d740 1 Processor -- start 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98383bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d983799e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9837d0910 con 0x55d98379d780 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9837d17e0 con 0x55d9837ebdf0 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9837fbb80 con 0x55d9837ce550 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feebf98d740 10 client.0 _ll_register_callbacks cb 0x55d98379ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98383bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98383bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47872/0 (socket says 172.21.15.5:47872) 2024-04-25T10:06:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 -- 172.21.15.5:0/3687243747 learned_addr learned my addr 172.21.15.5:0/3687243747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 -- 172.21.15.5:0/3687243747 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 msgr2=0x55d983799e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d983799e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 -- 172.21.15.5:0/3687243747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 msgr2=0x55d9837cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.733+0000 7feeb5e00640 1 -- 172.21.15.5:0/3687243747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9837fa0c0 con 0x55d98379d780 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7feeb5e00640 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98383bb70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7feea80097a0 tx=0x7feea8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e479c44c6d486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7fee96a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7fee96a00640 1 -- 172.21.15.5:0/3687243747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8045070 con 0x55d98379d780 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7fee96a00640 1 -- 172.21.15.5:0/3687243747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7feea803e6a0 con 0x55d98379d780 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feea8058020 con 0x55d98379d780 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7fee96a00640 1 -- 172.21.15.5:0/3687243747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feea8037040 con 0x55d98379d780 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7feebf98d740 10 client.6585 fetch_fsmap learned FSMap version 37 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.734+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55d9837c9790 con 0x55d98379d780 2024-04-25T10:06:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7fee96a00640 1 -- 172.21.15.5:0/3687243747 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8058020 con 0x55d98379d780 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7fee96a00640 1 -- 172.21.15.5:0/3687243747 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feea8048070 con 0x55d98379d780 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feebf98d740 10 client.6585 fetch_fsmap finished waiting for FSMap version 37 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feebf98d740 10 client.6585 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feebf98d740 20 client.6585 populate_metadata read hostname 'smithi005' 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feebf98d740 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837fa570 0x55d9837fa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feebf98d740 4 client.6585 mds_command: new command op to 6255 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 --> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55d983714020 con 0x55d9837fa570 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.735+0000 7feeb5400640 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837fa570 0x55d9837fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feeb5400640 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837fa570 0x55d9837fa950 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c5864a2cd164b7 server_cookie=d47a2130ea4d159a in_seq=0 out_seq=0 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7fee96a00640 10 client.6585 ms_handle_connect on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7fee96a00640 1 -- 172.21.15.5:0/3687243747 <== mds.0 v2:172.21.15.5:6834/1822355874 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d983714020 con 0x55d9837fa570 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7fee96a00640 10 client.6585 handle_command_reply: tid=0 2024-04-25T10:06:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 client.6585 shutdown 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 msgr2=0x55d98383bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98383bb70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7feea80097a0 tx=0x7feea8004730 comp rx=0 tx=0).stop 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 shutdown_connections 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:6834/1822355874,v1:172.21.15.5:6835/1822355874] conn(0x55d9837fa570 0x55d9837fa950 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d9837ebdf0 0x55d9837cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d9837ce550 0x55d983799e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 --2- 172.21.15.5:0/3687243747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98379d780 0x55d98383bb70 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 >> 172.21.15.5:0/3687243747 conn(0x55d9837a4230 msgr2=0x55d9837a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7fee96a00640 0 client.6585 ms_handle_reset on v2:172.21.15.5:6834/1822355874 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 shutdown_connections 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.736+0000 7feebf98d740 1 -- 172.21.15.5:0/3687243747 wait complete. 2024-04-25T10:06:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feebf98d740 20 client.6585 trim_cache size 0 max 16384 2024-04-25T10:06:46.747 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:46.747 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-25T10:06:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fee9806b4d0 msgr2=0x7fee9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fee9806b4d0 0x7fee9806d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7feea8034000 tx=0x7feea802c000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 msgr2=0x7feeb8152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 0x7feeb8152dc0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7feea400c4d0 tx=0x7feea400c990 comp rx=0 tx=0).stop 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 shutdown_connections 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fee9806b4d0 0x7fee9806d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feeb815ed20 0x7feeb80ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feeb8144e90 0x7feeb8152dc0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 --2- 172.21.15.5:0/110244290 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feeb8071710 0x7feeb80b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 >> 172.21.15.5:0/110244290 conn(0x7feeb8066f20 msgr2=0x7feeb815c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.737+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 shutdown_connections 2024-04-25T10:06:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.738+0000 7feeb7200640 1 -- 172.21.15.5:0/110244290 wait complete. 2024-04-25T10:06:46.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:06:46.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.960+0000 7fb3b7200640 1 Processor -- start 2024-04-25T10:06:46.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.960+0000 7fb3b7200640 1 -- start start 2024-04-25T10:06:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.961+0000 7fb3b7200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb3b80d0b10 0x7fb3b80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.961+0000 7fb3b7200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3b80ce700 0x7fb3b80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.961+0000 7fb3b7200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b806cb40 con 0x7fb3b806bdc0 2024-04-25T10:06:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.961+0000 7fb3b7200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b80d2f10 con 0x7fb3b80d0b10 2024-04-25T10:06:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.961+0000 7fb3b7200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b806c8e0 con 0x7fb3b80ce700 2024-04-25T10:06:46.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb3b80d0b10 0x7fb3b80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b6800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb3b806bdc0 0x7fb3b806c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49050/0 (socket says 172.21.15.5:49050) 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b6800640 1 -- 172.21.15.5:0/2702984358 learned_addr learned my addr 172.21.15.5:0/2702984358 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5400640 1 --2- 172.21.15.5:0/2702984358 >> v2:172.21.15.80:3300/0 conn(0x7fb3b80ce700 0x7fb3b80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5e00640 1 -- 172.21.15.5:0/2702984358 >> v2:172.21.15.80:3300/0 conn(0x7fb3b80ce700 msgr2=0x7fb3b80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5e00640 1 --2- 172.21.15.5:0/2702984358 >> v2:172.21.15.80:3300/0 conn(0x7fb3b80ce700 0x7fb3b80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5e00640 1 -- 172.21.15.5:0/2702984358 >> v1:172.21.15.5:6789/0 conn(0x7fb3b806bdc0 legacy=0x7fb3b806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b4a00640 1 -- 172.21.15.5:0/2702984358 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070344779 0 0) 0x7fb3b806cb40 con 0x7fb3b806bdc0 2024-04-25T10:06:46.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5e00640 1 -- 172.21.15.5:0/2702984358 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80d1c30 con 0x7fb3b80d0b10 2024-04-25T10:06:46.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.962+0000 7fb3b5400640 1 --2- 172.21.15.5:0/2702984358 >> v2:172.21.15.80:3300/0 conn(0x7fb3b80ce700 0x7fb3b80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.964+0000 7fb3b5e00640 1 --2- 172.21.15.5:0/2702984358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b80d29d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b30f0 tx=0x7fb3a802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1c5beb2f4706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.964+0000 7fb3b4a00640 1 -- 172.21.15.5:0/2702984358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8007740 con 0x7fb3b80d0b10 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.964+0000 7fb3b4a00640 1 -- 172.21.15.5:0/2702984358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a80078e0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.964+0000 7fb3b4a00640 1 -- 172.21.15.5:0/2702984358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8007cb0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 -- 172.21.15.5:0/2702984358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 msgr2=0x7fb3b80d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 --2- 172.21.15.5:0/2702984358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b80d29d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80b30f0 tx=0x7fb3a802a680 comp rx=0 tx=0).stop 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 -- 172.21.15.5:0/2702984358 shutdown_connections 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 --2- 172.21.15.5:0/2702984358 >> v2:172.21.15.80:3300/0 conn(0x7fb3b80ce700 0x7fb3b80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 --2- 172.21.15.5:0/2702984358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b80d29d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 -- 172.21.15.5:0/2702984358 >> 172.21.15.5:0/2702984358 conn(0x7fb3b80b53d0 msgr2=0x7fb3b80b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 -- 172.21.15.5:0/2702984358 shutdown_connections 2024-04-25T10:06:46.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.965+0000 7fb3b7200640 1 -- 172.21.15.5:0/2702984358 wait complete. 2024-04-25T10:06:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.967+0000 7fb3b7200640 1 Processor -- start 2024-04-25T10:06:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.967+0000 7fb3b7200640 1 -- start start 2024-04-25T10:06:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.968+0000 7fb3b7200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3b806bdc0 0x7fb3b80ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b7200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3b80ce700 0x7fb3b80ab420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b81f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80d24d0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b7200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b80d21f0 con 0x7fb3b806bdc0 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b7200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b815f390 con 0x7fb3b80ce700 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b81f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b81f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43030/0 (socket says 172.21.15.5:43030) 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 -- 172.21.15.5:0/292595786 learned_addr learned my addr 172.21.15.5:0/292595786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b5e00640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3b806bdc0 0x7fb3b80ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b5400640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3b80ce700 0x7fb3b80ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 -- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3b80ce700 msgr2=0x7fb3b80ab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3b80ce700 0x7fb3b80ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 -- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3b806bdc0 msgr2=0x7fb3b80ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3b806bdc0 0x7fb3b80ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.969+0000 7fb3b6800640 1 -- 172.21.15.5:0/292595786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80aa1f0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.970+0000 7fb3b6800640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b81f2c70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c450 tx=0x7fb3b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.970+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0018070 con 0x7fb3b80d0b10 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.970+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b0007500 con 0x7fb3b80d0b10 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.970+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004a20 con 0x7fb3b80d0b10 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.970+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b80aa4d0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.970+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b80aaa90 con 0x7fb3b80d0b10 2024-04-25T10:06:46.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.972+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b80677d0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.973+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3b0004bc0 con 0x7fb3b80d0b10 2024-04-25T10:06:46.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.974+0000 7fb3ae000640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb39806b4d0 0x7fb39806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.974+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fb3b0089730 con 0x7fb3b80d0b10 2024-04-25T10:06:46.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.974+0000 7fb3b5e00640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb39806b4d0 0x7fb39806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.976+0000 7fb3b5e00640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb39806b4d0 0x7fb39806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8034000 tx=0x7fb3a8033040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:46.978+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3b00582e0 con 0x7fb3b80d0b10 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.222+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb3b8068f60 con 0x7fb3b80d0b10 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.223+0000 7fb3ae000640 1 -- 172.21.15.5:0/292595786 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb3b0057be0 con 0x7fb3b80d0b10 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:06:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.226+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb39806b4d0 msgr2=0x7fb39806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.226+0000 7fb3b7200640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb39806b4d0 0x7fb39806d990 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8034000 tx=0x7fb3a8033040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.226+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 msgr2=0x7fb3b81f2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.226+0000 7fb3b7200640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b81f2c70 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c450 tx=0x7fb3b000c910 comp rx=0 tx=0).stop 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 shutdown_connections 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b80d0b10 0x7fb3b81f2c70 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3b80ce700 0x7fb3b80ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb39806b4d0 0x7fb39806d990 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 --2- 172.21.15.5:0/292595786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3b806bdc0 0x7fb3b80ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 >> 172.21.15.5:0/292595786 conn(0x7fb3b80b53d0 msgr2=0x7fb3b80b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 shutdown_connections 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.227+0000 7fb3b7200640 1 -- 172.21.15.5:0/292595786 wait complete. 2024-04-25T10:06:47.253 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:06:47.253 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:47.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:47.356 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:47.471 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:47.472 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:06:47.472 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:47.532 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:06:47.532 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:47.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:47.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.806+0000 7f53c5400640 1 Processor -- start 2024-04-25T10:06:47.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.806+0000 7f53c5400640 1 -- start start 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.808+0000 7f53c5400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f53c01529e0 0x7f53c00c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.808+0000 7f53c5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c00c6410 0x7f53c00ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.808+0000 7f53c5400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c015bdc0 con 0x7f53c0071ed0 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.808+0000 7f53c5400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c00b2340 con 0x7f53c01529e0 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.808+0000 7f53c5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c015c060 con 0x7f53c00c6410 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.808+0000 7f53bf200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53c0071ed0 0x7f53c00c4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49054/0 (socket says 172.21.15.5:49054) 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53bf200640 1 -- 172.21.15.5:0/225930110 learned_addr learned my addr 172.21.15.5:0/225930110 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53be800640 1 --2- 172.21.15.5:0/225930110 >> v2:172.21.15.5:3300/0 conn(0x7f53c01529e0 0x7f53c00c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53b5e00640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c00c6410 0x7f53c00ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53bde00640 1 -- 172.21.15.5:0/225930110 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133610605 0 0) 0x7f53c015bdc0 con 0x7f53c0071ed0 2024-04-25T10:06:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53bde00640 1 -- 172.21.15.5:0/225930110 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a4003520 con 0x7f53c0071ed0 2024-04-25T10:06:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53be800640 1 -- 172.21.15.5:0/225930110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c00c6410 msgr2=0x7f53c00ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53be800640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c00c6410 0x7f53c00ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53be800640 1 -- 172.21.15.5:0/225930110 >> v1:172.21.15.5:6789/0 conn(0x7f53c0071ed0 legacy=0x7f53c00c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53be800640 1 -- 172.21.15.5:0/225930110 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c015c340 con 0x7f53c01529e0 2024-04-25T10:06:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.809+0000 7f53b5e00640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c00c6410 0x7f53c00ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:47.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.810+0000 7f53be800640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c01529e0 0x7f53c00c5ed0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f53b00097a0 tx=0x7f53b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf2d25915d7685a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53bde00640 1 -- 172.21.15.5:0/225930110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0037070 con 0x7f53c01529e0 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53bde00640 1 -- 172.21.15.5:0/225930110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b0032440 con 0x7f53c01529e0 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53bde00640 1 -- 172.21.15.5:0/225930110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0032810 con 0x7f53c01529e0 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 -- 172.21.15.5:0/225930110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c01529e0 msgr2=0x7f53c00c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c01529e0 0x7f53c00c5ed0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f53b00097a0 tx=0x7f53b002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 -- 172.21.15.5:0/225930110 shutdown_connections 2024-04-25T10:06:47.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c00c6410 0x7f53c00ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 --2- 172.21.15.5:0/225930110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c01529e0 0x7f53c00c5ed0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 -- 172.21.15.5:0/225930110 >> 172.21.15.5:0/225930110 conn(0x7f53c0071150 msgr2=0x7f53c0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.811+0000 7f53c5400640 1 -- 172.21.15.5:0/225930110 shutdown_connections 2024-04-25T10:06:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.812+0000 7f53c5400640 1 -- 172.21.15.5:0/225930110 wait complete. 2024-04-25T10:06:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.813+0000 7f53c5400640 1 Processor -- start 2024-04-25T10:06:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.813+0000 7f53c5400640 1 -- start start 2024-04-25T10:06:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.814+0000 7f53c5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c0071ed0 0x7f53c01f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.814+0000 7f53c5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53c00c6410 0x7f53c0157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.814+0000 7f53c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 0x7f53c01f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.814+0000 7f53c5400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c006e160 con 0x7f53c0156350 2024-04-25T10:06:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.814+0000 7f53c5400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c015c900 con 0x7f53c00c6410 2024-04-25T10:06:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.814+0000 7f53c5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c006e2e0 con 0x7f53c0071ed0 2024-04-25T10:06:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 0x7f53c01f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 0x7f53c01f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43042/0 (socket says 172.21.15.5:43042) 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 -- 172.21.15.5:0/3826993432 learned_addr learned my addr 172.21.15.5:0/3826993432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53be800640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c0071ed0 0x7f53c01f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53b5e00640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53c00c6410 0x7f53c0157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 -- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c0071ed0 msgr2=0x7f53c01f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c0071ed0 0x7f53c01f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 -- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53c00c6410 msgr2=0x7f53c0157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53c00c6410 0x7f53c0157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53bf200640 1 -- 172.21.15.5:0/3826993432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0157aa0 con 0x7f53c0156350 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.815+0000 7f53b5e00640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53c00c6410 0x7f53c0157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.816+0000 7f53bf200640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 0x7f53c01f4b20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f53b800ca70 tx=0x7f53b800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.816+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8017070 con 0x7f53c0156350 2024-04-25T10:06:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.816+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800abd0 con 0x7f53c0156350 2024-04-25T10:06:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.816+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8012400 con 0x7f53c0156350 2024-04-25T10:06:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.816+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c01388b0 con 0x7f53c0156350 2024-04-25T10:06:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.816+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c0138e70 con 0x7f53c0156350 2024-04-25T10:06:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.817+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c01386d0 con 0x7f53c0156350 2024-04-25T10:06:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.819+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53b801a070 con 0x7f53c0156350 2024-04-25T10:06:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.820+0000 7f53b7400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.820+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f53b808a100 con 0x7f53c0156350 2024-04-25T10:06:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.820+0000 7f53be800640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.822+0000 7f53be800640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f539c06b4d0 0x7f539c06d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53b0034000 tx=0x7f53b002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:47.824+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53c01386d0 con 0x7f53c0156350 2024-04-25T10:06:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.024+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53c015ac80 con 0x7f53c0156350 2024-04-25T10:06:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.026+0000 7f53b7400640 1 -- 172.21.15.5:0/3826993432 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17161 (secure 0 0 0) 0x7f53c015ac80 con 0x7f53c0156350 2024-04-25T10:06:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:48.034 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:42.412821+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1d","mappings":[{"from":2,"to":1}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.31","mappings":[{"from":4,"to":5}]},{"pgid":"10.33","mappings":[{"from":2,"to":1}]},{"pgid":"10.35","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f539c06b4d0 msgr2=0x7f539c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f539c06b4d0 0x7f539c06d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53b0034000 tx=0x7f53b002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 msgr2=0x7f53c01f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 0x7f53c01f4b20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f53b800ca70 tx=0x7f53b800cf30 comp rx=0 tx=0).stop 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 shutdown_connections 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53c0156350 0x7f53c01f4b20 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53c00c6410 0x7f53c0157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 --2- 172.21.15.5:0/3826993432 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53c0071ed0 0x7f53c01f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.029+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 >> 172.21.15.5:0/3826993432 conn(0x7f53c0071150 msgr2=0x7f53c00c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.030+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 shutdown_connections 2024-04-25T10:06:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.030+0000 7f53c5400640 1 -- 172.21.15.5:0/3826993432 wait complete. 2024-04-25T10:06:48.055 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:30.592093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:31.804256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:48.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.262+0000 7fbb33e00640 1 Processor -- start 2024-04-25T10:06:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.263+0000 7fbb33e00640 1 -- start start 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.264+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb341448f0 0x7fbb340cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.264+0000 7fbb33e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbb3415bc30 0x7fbb3415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb33e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb34144cd0 con 0x7fbb3415fe30 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb33e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3415f460 con 0x7fbb3415bc30 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb33e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415c440 con 0x7fbb341448f0 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb33400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbb3415fe30 0x7fbb3415ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49068/0 (socket says 172.21.15.5:49068) 2024-04-25T10:06:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb33400640 1 -- 172.21.15.5:0/4057894841 learned_addr learned my addr 172.21.15.5:0/4057894841 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb2be00640 1 --2- 172.21.15.5:0/4057894841 >> v2:172.21.15.5:3300/0 conn(0x7fbb3415bc30 0x7fbb3415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.265+0000 7fbb32a00640 1 --2- 172.21.15.5:0/4057894841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb341448f0 0x7fbb340cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.266+0000 7fbb32000640 1 -- 172.21.15.5:0/4057894841 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604716236 0 0) 0x7fbb34144cd0 con 0x7fbb3415fe30 2024-04-25T10:06:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.266+0000 7fbb2be00640 1 -- 172.21.15.5:0/4057894841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb341448f0 msgr2=0x7fbb340cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.266+0000 7fbb2be00640 1 --2- 172.21.15.5:0/4057894841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb341448f0 0x7fbb340cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.266+0000 7fbb2be00640 1 -- 172.21.15.5:0/4057894841 >> v1:172.21.15.5:6789/0 conn(0x7fbb3415fe30 legacy=0x7fbb3415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.266+0000 7fbb2be00640 1 -- 172.21.15.5:0/4057894841 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3415c720 con 0x7fbb3415bc30 2024-04-25T10:06:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.267+0000 7fbb2be00640 1 --2- 172.21.15.5:0/4057894841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415bc30 0x7fbb3415f8f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0097a0 tx=0x7fbb1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2dce524bd54ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.267+0000 7fbb32000640 1 -- 172.21.15.5:0/4057894841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c037070 con 0x7fbb3415bc30 2024-04-25T10:06:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.267+0000 7fbb32000640 1 -- 172.21.15.5:0/4057894841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c032440 con 0x7fbb3415bc30 2024-04-25T10:06:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.267+0000 7fbb32000640 1 -- 172.21.15.5:0/4057894841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c032810 con 0x7fbb3415bc30 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 -- 172.21.15.5:0/4057894841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415bc30 msgr2=0x7fbb3415f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 --2- 172.21.15.5:0/4057894841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415bc30 0x7fbb3415f8f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0097a0 tx=0x7fbb1c02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 -- 172.21.15.5:0/4057894841 shutdown_connections 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 --2- 172.21.15.5:0/4057894841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415bc30 0x7fbb3415f8f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 --2- 172.21.15.5:0/4057894841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb341448f0 0x7fbb340cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 -- 172.21.15.5:0/4057894841 >> 172.21.15.5:0/4057894841 conn(0x7fbb34066f20 msgr2=0x7fbb3406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 -- 172.21.15.5:0/4057894841 shutdown_connections 2024-04-25T10:06:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.268+0000 7fbb33e00640 1 -- 172.21.15.5:0/4057894841 wait complete. 2024-04-25T10:06:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.270+0000 7fbb33e00640 1 Processor -- start 2024-04-25T10:06:48.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.270+0000 7fbb33e00640 1 -- start start 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb341448f0 0x7fbb340b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb3415bc30 0x7fbb340ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 0x7fbb340b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415c090 con 0x7fbb3415fe30 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415d530 con 0x7fbb341448f0 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb341f39e0 con 0x7fbb3415bc30 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 0x7fbb340b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 0x7fbb340b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43050/0 (socket says 172.21.15.5:43050) 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.271+0000 7fbb33400640 1 -- 172.21.15.5:0/3222411245 learned_addr learned my addr 172.21.15.5:0/3222411245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb2be00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb3415bc30 0x7fbb340ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb32a00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb341448f0 0x7fbb340b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb33400640 1 -- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb3415bc30 msgr2=0x7fbb340ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb33400640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb3415bc30 0x7fbb340ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb33400640 1 -- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb341448f0 msgr2=0x7fbb340b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb33400640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb341448f0 0x7fbb340b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb33400640 1 -- 172.21.15.5:0/3222411245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb341f3cc0 con 0x7fbb3415fe30 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb33400640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 0x7fbb340b0210 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c700 tx=0x7fbb2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.272+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c018070 con 0x7fbb3415fe30 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.273+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00ce10 con 0x7fbb3415fe30 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.273+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb341f3fa0 con 0x7fbb3415fe30 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.273+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00a860 con 0x7fbb3415fe30 2024-04-25T10:06:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.273+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb340b2a60 con 0x7fbb3415fe30 2024-04-25T10:06:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.274+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb34068f60 con 0x7fbb3415fe30 2024-04-25T10:06:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.275+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb2c01b070 con 0x7fbb3415fe30 2024-04-25T10:06:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.276+0000 7fbb2b400640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb1006b4d0 0x7fbb1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.276+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbb2c089a00 con 0x7fbb3415fe30 2024-04-25T10:06:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.276+0000 7fbb32a00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb1006b4d0 0x7fbb1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.278+0000 7fbb32a00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb1006b4d0 0x7fbb1006d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.280+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb2c058630 con 0x7fbb3415fe30 2024-04-25T10:06:48.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.477+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb34072180 con 0x7fbb3415fe30 2024-04-25T10:06:48.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.478+0000 7fbb2b400640 1 -- 172.21.15.5:0/3222411245 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb2c057f30 con 0x7fbb3415fe30 2024-04-25T10:06:48.485 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb1006b4d0 msgr2=0x7fbb1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb1006b4d0 0x7fbb1006d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24008040 comp rx=0 tx=0).stop 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 msgr2=0x7fbb340b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 0x7fbb340b0210 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c700 tx=0x7fbb2c00cbc0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 shutdown_connections 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb3415fe30 0x7fbb340b0210 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbb3415bc30 0x7fbb340ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbb1006b4d0 0x7fbb1006d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 --2- 172.21.15.5:0/3222411245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb341448f0 0x7fbb340b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 >> 172.21.15.5:0/3222411245 conn(0x7fbb34066f20 msgr2=0x7fbb3406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 shutdown_connections 2024-04-25T10:06:48.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.481+0000 7fbb33e00640 1 -- 172.21.15.5:0/3222411245 wait complete. 2024-04-25T10:06:48.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.713+0000 7ff406800640 1 Processor -- start 2024-04-25T10:06:48.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.714+0000 7ff406800640 1 -- start start 2024-04-25T10:06:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.715+0000 7ff406800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff4000b48c0 0x7ff40015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.715+0000 7ff406800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 0x7ff400161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.715+0000 7ff406800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4001606e0 con 0x7ff400161bc0 2024-04-25T10:06:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.715+0000 7ff406800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4001609c0 con 0x7ff4000b48c0 2024-04-25T10:06:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff406800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff400160ca0 con 0x7ff4000d3f90 2024-04-25T10:06:48.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff405e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff400161bc0 0x7ff4000d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49082/0 (socket says 172.21.15.5:49082) 2024-04-25T10:06:48.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff405e00640 1 -- 172.21.15.5:0/2994499720 learned_addr learned my addr 172.21.15.5:0/2994499720 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff404a00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 0x7ff400161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff405400640 1 --2- 172.21.15.5:0/2994499720 >> v2:172.21.15.80:3300/0 conn(0x7ff4000b48c0 0x7ff40015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448131589 0 0) 0x7ff4001606e0 con 0x7ff400161bc0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.716+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e8003520 con 0x7ff400161bc0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2241682815 0 0) 0x7ff3e8003520 con 0x7ff400161bc0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 msgr2=0x7ff400161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 0x7ff400161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 >> v2:172.21.15.80:3300/0 conn(0x7ff4000b48c0 msgr2=0x7ff40015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 --2- 172.21.15.5:0/2994499720 >> v2:172.21.15.80:3300/0 conn(0x7ff4000b48c0 0x7ff40015fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400160f80 con 0x7ff400161bc0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7ff3f8001670 con 0x7ff400161bc0 2024-04-25T10:06:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 >> v1:172.21.15.5:6789/0 conn(0x7ff400161bc0 legacy=0x7ff4000d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3e80039b0 0x7ff3e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3e8004800 0x7ff3e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 0x7ff3e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4001609c0 con 0x7ff3e80039b0 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff400160ca0 con 0x7ff3e8004800 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e800b9c0 con 0x7ff4000d3f90 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.717+0000 7ff405400640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3e80039b0 0x7ff3e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff404a00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3e8004800 0x7ff3e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff405e00640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 0x7ff3e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff405400640 1 -- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 msgr2=0x7ff3e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff405400640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000d3f90 0x7ff3e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff405400640 1 -- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3e8004800 msgr2=0x7ff3e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff405400640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3e8004800 0x7ff3e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.718+0000 7ff405400640 1 -- 172.21.15.5:0/2994499720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8003520 con 0x7ff3e80039b0 2024-04-25T10:06:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.719+0000 7ff405400640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3e80039b0 0x7ff3e8003dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e170 tx=0x7ff3e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ec793bae67340f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.719+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4034020 con 0x7ff3e80039b0 2024-04-25T10:06:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.719+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4030430 con 0x7ff3e80039b0 2024-04-25T10:06:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.719+0000 7ff3ffe00640 1 -- 172.21.15.5:0/2994499720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4009c10 con 0x7ff3e80039b0 2024-04-25T10:06:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff406800640 1 -- 172.21.15.5:0/2994499720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3e80039b0 msgr2=0x7ff3e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff406800640 1 --2- 172.21.15.5:0/2994499720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3e80039b0 0x7ff3e8003dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e170 tx=0x7ff3e8005830 comp rx=0 tx=0).stop 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff405e00640 1 -- 172.21.15.5:0/2994499720 reap_dead start 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff406800640 1 -- 172.21.15.5:0/2994499720 shutdown_connections 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff406800640 1 -- 172.21.15.5:0/2994499720 >> 172.21.15.5:0/2994499720 conn(0x7ff40008af80 msgr2=0x7ff4000d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff406800640 1 -- 172.21.15.5:0/2994499720 shutdown_connections 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.720+0000 7ff406800640 1 -- 172.21.15.5:0/2994499720 wait complete. 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.722+0000 7ff406800640 1 Processor -- start 2024-04-25T10:06:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.722+0000 7ff406800640 1 -- start start 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.723+0000 7ff406800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 0x7ff4000cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.723+0000 7ff406800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4000cfb70 0x7ff4000b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.723+0000 7ff406800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000b7880 0x7ff4000b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.723+0000 7ff406800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4001603a0 con 0x7ff400154ed0 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.723+0000 7ff406800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000b4ca0 con 0x7ff4000cfb70 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.723+0000 7ff406800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4000b3b80 con 0x7ff4000b7880 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 0x7ff4000cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 0x7ff4000cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43062/0 (socket says 172.21.15.5:43062) 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 -- 172.21.15.5:0/3647918037 learned_addr learned my addr 172.21.15.5:0/3647918037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff404a00640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4000cfb70 0x7ff4000b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405e00640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000b7880 0x7ff4000b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 -- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000b7880 msgr2=0x7ff4000b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000b7880 0x7ff4000b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 -- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4000cfb70 msgr2=0x7ff4000b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4000cfb70 0x7ff4000b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff405400640 1 -- 172.21.15.5:0/3647918037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4000ca920 con 0x7ff400154ed0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.724+0000 7ff404a00640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4000cfb70 0x7ff4000b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.725+0000 7ff405400640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 0x7ff4000cf630 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009f90 tx=0x7ff3f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.725+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4034070 con 0x7ff400154ed0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.725+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4003b90 con 0x7ff400154ed0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.725+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4000cac00 con 0x7ff400154ed0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.725+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4009c10 con 0x7ff400154ed0 2024-04-25T10:06:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.725+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4000cb1c0 con 0x7ff400154ed0 2024-04-25T10:06:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.726+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4000ca6b0 con 0x7ff400154ed0 2024-04-25T10:06:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.727+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3f4037020 con 0x7ff400154ed0 2024-04-25T10:06:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.728+0000 7ff3fd600640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.728+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ff3f40ab0a0 con 0x7ff400154ed0 2024-04-25T10:06:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.729+0000 7ff404a00640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.730+0000 7ff404a00640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3e006b4d0 0x7ff3e006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff400158ce0 tx=0x7ff3ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.733+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3f4079bd0 con 0x7ff400154ed0 2024-04-25T10:06:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.936+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4000cff50 con 0x7ff400154ed0 2024-04-25T10:06:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.937+0000 7ff3fd600640 1 -- 172.21.15.5:0/3647918037 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3f40794d0 con 0x7ff400154ed0 2024-04-25T10:06:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:06:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.939+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3e006b4d0 msgr2=0x7ff3e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.939+0000 7ff406800640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3e006b4d0 0x7ff3e006d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff400158ce0 tx=0x7ff3ec008040 comp rx=0 tx=0).stop 2024-04-25T10:06:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.939+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 msgr2=0x7ff4000cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 0x7ff4000cf630 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009f90 tx=0x7ff3f4009750 comp rx=0 tx=0).stop 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 shutdown_connections 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4000b7880 0x7ff4000b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4000cfb70 0x7ff4000b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 --2- 172.21.15.5:0/3647918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff400154ed0 0x7ff4000cf630 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 >> 172.21.15.5:0/3647918037 conn(0x7ff40008af80 msgr2=0x7ff40006c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 shutdown_connections 2024-04-25T10:06:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:48.940+0000 7ff406800640 1 -- 172.21.15.5:0/3647918037 wait complete. 2024-04-25T10:06:48.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.175+0000 7f6833200640 1 Processor -- start 2024-04-25T10:06:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.176+0000 7f6833200640 1 -- start start 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.177+0000 7f6833200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f68340b48c0 0x7f683415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.177+0000 7f6833200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.177+0000 7f6833200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68341606e0 con 0x7f68340d3f90 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.177+0000 7f6833200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68341609c0 con 0x7f68340b48c0 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.177+0000 7f6833200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6834160ca0 con 0x7f6834161d90 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6831400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f68340d3f90 0x7f6834161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49094/0 (socket says 172.21.15.5:49094) 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6831400640 1 -- 172.21.15.5:0/1619892164 learned_addr learned my addr 172.21.15.5:0/1619892164 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6831e00640 1 --2- 172.21.15.5:0/1619892164 >> v2:172.21.15.5:3300/0 conn(0x7f68340b48c0 0x7f683415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6832800640 1 --2- 172.21.15.5:0/1619892164 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6830a00640 1 -- 172.21.15.5:0/1619892164 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781031064 0 0) 0x7f68341606e0 con 0x7f68340d3f90 2024-04-25T10:06:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6830a00640 1 -- 172.21.15.5:0/1619892164 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6814003520 con 0x7f68340d3f90 2024-04-25T10:06:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6831e00640 1 -- 172.21.15.5:0/1619892164 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 msgr2=0x7f68340b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6831e00640 1 --2- 172.21.15.5:0/1619892164 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.178+0000 7f6831e00640 1 -- 172.21.15.5:0/1619892164 >> v1:172.21.15.5:6789/0 conn(0x7f68340d3f90 legacy=0x7f6834161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.179+0000 7f6831e00640 1 -- 172.21.15.5:0/1619892164 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834160f80 con 0x7f68340b48c0 2024-04-25T10:06:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.180+0000 7f6831e00640 1 --2- 172.21.15.5:0/1619892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340b48c0 0x7f683415fe60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f68240097a0 tx=0x7f682402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82056670d8fd7eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.180+0000 7f6830a00640 1 -- 172.21.15.5:0/1619892164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824037070 con 0x7f68340b48c0 2024-04-25T10:06:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.180+0000 7f6830a00640 1 -- 172.21.15.5:0/1619892164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682402ae50 con 0x7f68340b48c0 2024-04-25T10:06:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.180+0000 7f6830a00640 1 -- 172.21.15.5:0/1619892164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824032660 con 0x7f68340b48c0 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.180+0000 7f6833200640 1 -- 172.21.15.5:0/1619892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340b48c0 msgr2=0x7f683415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 --2- 172.21.15.5:0/1619892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340b48c0 0x7f683415fe60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f68240097a0 tx=0x7f682402a100 comp rx=0 tx=0).stop 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 -- 172.21.15.5:0/1619892164 shutdown_connections 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 --2- 172.21.15.5:0/1619892164 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 --2- 172.21.15.5:0/1619892164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340b48c0 0x7f683415fe60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 -- 172.21.15.5:0/1619892164 >> 172.21.15.5:0/1619892164 conn(0x7f683408af80 msgr2=0x7f68340d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 -- 172.21.15.5:0/1619892164 shutdown_connections 2024-04-25T10:06:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.181+0000 7f6833200640 1 -- 172.21.15.5:0/1619892164 wait complete. 2024-04-25T10:06:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.182+0000 7f6833200640 1 Processor -- start 2024-04-25T10:06:49.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.183+0000 7f6833200640 1 -- start start 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6833200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68340b48c0 0x7f6834154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6833200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 0x7f68340cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6833200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6833200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68341603a0 con 0x7f68340d3f90 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6833200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68340b4ca0 con 0x7f68340b48c0 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6833200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68340b30b0 con 0x7f6834161d90 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 0x7f68340cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 0x7f68340cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43088/0 (socket says 172.21.15.5:43088) 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 -- 172.21.15.5:0/2028032852 learned_addr learned my addr 172.21.15.5:0/2028032852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6832800640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 -- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 msgr2=0x7f68340c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831e00640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68340b48c0 0x7f6834154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 -- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68340b48c0 msgr2=0x7f6834154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68340b48c0 0x7f6834154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6831400640 1 -- 172.21.15.5:0/2028032852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68340ccee0 con 0x7f68340d3f90 2024-04-25T10:06:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.184+0000 7f6832800640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.185+0000 7f6831400640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 0x7f68340cbf10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c840 tx=0x7f681c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.185+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c017070 con 0x7f68340d3f90 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.185+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00a640 con 0x7f68340d3f90 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.185+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00aa30 con 0x7f68340d3f90 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.185+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68340cc920 con 0x7f68340d3f90 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.185+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68340b0a20 con 0x7f68340d3f90 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.186+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6834068f60 con 0x7f68340d3f90 2024-04-25T10:06:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.187+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f681c01a070 con 0x7f68340d3f90 2024-04-25T10:06:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.188+0000 7f682a000640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f681006b4d0 0x7f681006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.188+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f681c089a70 con 0x7f68340d3f90 2024-04-25T10:06:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.189+0000 7f6831e00640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f681006b4d0 0x7f681006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.190+0000 7f6831e00640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f681006b4d0 0x7f681006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6824034000 tx=0x7f682402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.193+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f681c057fd0 con 0x7f68340d3f90 2024-04-25T10:06:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.397+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68341552b0 con 0x7f68340d3f90 2024-04-25T10:06:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.398+0000 7f682a000640 1 -- 172.21.15.5:0/2028032852 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f681c05c070 con 0x7f68340d3f90 2024-04-25T10:06:49.404 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f681006b4d0 msgr2=0x7f681006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f681006b4d0 0x7f681006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6824034000 tx=0x7f682402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 msgr2=0x7f68340cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 0x7f68340cbf10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c840 tx=0x7f681c00cd00 comp rx=0 tx=0).stop 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 shutdown_connections 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6834161d90 0x7f68340c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68340d3f90 0x7f68340cbf10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f681006b4d0 0x7f681006d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 --2- 172.21.15.5:0/2028032852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68340b48c0 0x7f6834154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.400+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 >> 172.21.15.5:0/2028032852 conn(0x7f683408af80 msgr2=0x7f68340ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.401+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 shutdown_connections 2024-04-25T10:06:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.401+0000 7f6833200640 1 -- 172.21.15.5:0/2028032852 wait complete. 2024-04-25T10:06:49.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.631+0000 7ff49b200640 1 Processor -- start 2024-04-25T10:06:49.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.631+0000 7ff49b200640 1 -- start start 2024-04-25T10:06:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.632+0000 7ff49b200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff49c141da0 0x7ff49c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.632+0000 7ff49b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c06e640 0x7ff49c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.632+0000 7ff49b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff49c15d340 0x7ff49c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.632+0000 7ff49b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c142180 con 0x7ff49c15d340 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff49b200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c071d30 con 0x7ff49c141da0 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff49b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06d620 con 0x7ff49c06e640 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff49a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff49c15d340 0x7ff49c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff49a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff49c15d340 0x7ff49c0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43094/0 (socket says 172.21.15.5:43094) 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff49a800640 1 -- 172.21.15.5:0/2172593222 learned_addr learned my addr 172.21.15.5:0/2172593222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff499e00640 1 --2- 172.21.15.5:0/2172593222 >> v2:172.21.15.80:3300/0 conn(0x7ff49c141da0 0x7ff49c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.633+0000 7ff493e00640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c06e640 0x7ff49c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff49a800640 1 -- 172.21.15.5:0/2172593222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c06e640 msgr2=0x7ff49c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff49a800640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c06e640 0x7ff49c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff49a800640 1 -- 172.21.15.5:0/2172593222 >> v2:172.21.15.80:3300/0 conn(0x7ff49c141da0 msgr2=0x7ff49c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff49a800640 1 --2- 172.21.15.5:0/2172593222 >> v2:172.21.15.80:3300/0 conn(0x7ff49c141da0 0x7ff49c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff49a800640 1 -- 172.21.15.5:0/2172593222 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c06d900 con 0x7ff49c15d340 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff493e00640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c06e640 0x7ff49c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.634+0000 7ff499e00640 1 --2- 172.21.15.5:0/2172593222 >> v2:172.21.15.80:3300/0 conn(0x7ff49c141da0 0x7ff49c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.635+0000 7ff49a800640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c15d340 0x7ff49c0cd300 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff49c06bbb0 tx=0x7ff49402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e207941736fc4da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff499400640 1 -- 172.21.15.5:0/2172593222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494036070 con 0x7ff49c15d340 2024-04-25T10:06:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff499400640 1 -- 172.21.15.5:0/2172593222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494031600 con 0x7ff49c15d340 2024-04-25T10:06:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff499400640 1 -- 172.21.15.5:0/2172593222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940319d0 con 0x7ff49c15d340 2024-04-25T10:06:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 -- 172.21.15.5:0/2172593222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c15d340 msgr2=0x7ff49c0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c15d340 0x7ff49c0cd300 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff49c06bbb0 tx=0x7ff49402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 -- 172.21.15.5:0/2172593222 shutdown_connections 2024-04-25T10:06:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c15d340 0x7ff49c0cd300 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 --2- 172.21.15.5:0/2172593222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c06e640 0x7ff49c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 --2- 172.21.15.5:0/2172593222 >> v2:172.21.15.80:3300/0 conn(0x7ff49c141da0 0x7ff49c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.636+0000 7ff49b200640 1 -- 172.21.15.5:0/2172593222 >> 172.21.15.5:0/2172593222 conn(0x7ff49c066f20 msgr2=0x7ff49c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.637+0000 7ff49b200640 1 -- 172.21.15.5:0/2172593222 shutdown_connections 2024-04-25T10:06:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.637+0000 7ff49b200640 1 -- 172.21.15.5:0/2172593222 wait complete. 2024-04-25T10:06:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.638+0000 7ff49b200640 1 Processor -- start 2024-04-25T10:06:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.638+0000 7ff49b200640 1 -- start start 2024-04-25T10:06:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.639+0000 7ff49b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff49c06e640 0x7ff49c13cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.639+0000 7ff49b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 0x7ff49c1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.639+0000 7ff49b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c15d340 0x7ff49c0ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.639+0000 7ff49b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0cd9c0 con 0x7ff49c141da0 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.639+0000 7ff49b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0cd840 con 0x7ff49c06e640 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.639+0000 7ff49b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0cdb40 con 0x7ff49c15d340 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 0x7ff49c1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 0x7ff49c1e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43098/0 (socket says 172.21.15.5:43098) 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 -- 172.21.15.5:0/2617391937 learned_addr learned my addr 172.21.15.5:0/2617391937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff49a800640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c15d340 0x7ff49c0ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff499e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff49c06e640 0x7ff49c13cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 -- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c15d340 msgr2=0x7ff49c0ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c15d340 0x7ff49c0ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 -- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff49c06e640 msgr2=0x7ff49c13cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff49c06e640 0x7ff49c13cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff493e00640 1 -- 172.21.15.5:0/2617391937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0ad4f0 con 0x7ff49c141da0 2024-04-25T10:06:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.640+0000 7ff499e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff49c06e640 0x7ff49c13cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.641+0000 7ff493e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 0x7ff49c1e5100 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff48400d800 tx=0x7ff48400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.641+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484009640 con 0x7ff49c141da0 2024-04-25T10:06:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.641+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484019070 con 0x7ff49c141da0 2024-04-25T10:06:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.641+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484009a20 con 0x7ff49c141da0 2024-04-25T10:06:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.641+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c0ad7d0 con 0x7ff49c141da0 2024-04-25T10:06:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.641+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c0add90 con 0x7ff49c141da0 2024-04-25T10:06:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.643+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c0ccae0 con 0x7ff49c141da0 2024-04-25T10:06:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.643+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff484009bc0 con 0x7ff49c141da0 2024-04-25T10:06:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.644+0000 7ff492000640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff47806b4d0 0x7ff47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.644+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7ff48408a740 con 0x7ff49c141da0 2024-04-25T10:06:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.645+0000 7ff499e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff47806b4d0 0x7ff47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.646+0000 7ff499e00640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff47806b4d0 0x7ff47806d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff48c006fd0 tx=0x7ff48c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.649+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff484059400 con 0x7ff49c141da0 2024-04-25T10:06:49.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.858+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff49c0ae170 con 0x7ff49c141da0 2024-04-25T10:06:49.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.859+0000 7ff492000640 1 -- 172.21.15.5:0/2617391937 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff484058d00 con 0x7ff49c141da0 2024-04-25T10:06:49.866 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff47806b4d0 msgr2=0x7ff47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff47806b4d0 0x7ff47806d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff48c006fd0 tx=0x7ff48c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 msgr2=0x7ff49c1e5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 0x7ff49c1e5100 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff48400d800 tx=0x7ff48400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 shutdown_connections 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff49c15d340 0x7ff49c0ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c141da0 0x7ff49c1e5100 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff47806b4d0 0x7ff47806d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 --2- 172.21.15.5:0/2617391937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff49c06e640 0x7ff49c13cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 >> 172.21.15.5:0/2617391937 conn(0x7ff49c066f20 msgr2=0x7ff49c0af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 shutdown_connections 2024-04-25T10:06:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:49.862+0000 7ff49b200640 1 -- 172.21.15.5:0/2617391937 wait complete. 2024-04-25T10:06:49.889 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:06:49.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:49.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:50.031 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:06:50.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:50.031 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:50.083 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:06:50.083 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:50.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:50.124 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:06:50.124 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:50.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:50.165 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:06:50.165 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:50.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:50.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.414+0000 7f2854c00640 1 Processor -- start 2024-04-25T10:06:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.414+0000 7f2854c00640 1 -- start start 2024-04-25T10:06:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.416+0000 7f2854c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 0x7f28500b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.416+0000 7f2854c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500cf410 con 0x7f28500d07d0 2024-04-25T10:06:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.416+0000 7f2854c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500cf6f0 con 0x7f28500d2380 2024-04-25T10:06:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.416+0000 7f2854c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500d27c0 con 0x7f2850161200 2024-04-25T10:06:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.416+0000 7f284fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f28500d07d0 0x7f28500b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49104/0 (socket says 172.21.15.5:49104) 2024-04-25T10:06:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.416+0000 7f284fe00640 1 -- 172.21.15.5:0/4291559412 learned_addr learned my addr 172.21.15.5:0/4291559412 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284f400640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 0x7f28500b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654186133 0 0) 0x7f28500cf410 con 0x7f28500d07d0 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2834003520 con 0x7f28500d07d0 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 224709627 0 0) 0x7f2834003520 con 0x7f28500d07d0 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 msgr2=0x7f28500b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 0x7f28500b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 >> v1:172.21.15.80:6789/0 conn(0x7f28500d2380 legacy=0x7f28500d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500d3fd0 con 0x7f28500d07d0 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f28480019f0 con 0x7f28500d07d0 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 >> v1:172.21.15.5:6789/0 conn(0x7f28500d07d0 legacy=0x7f28500b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28340039e0 0x7f2834003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.417+0000 7f284e000640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f2834004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284e000640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 0x7f28340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500d27c0 con 0x7f2834004800 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283400baa0 con 0x7f28340039e0 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283400bca0 con 0x7f2850161200 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284ea00640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f2834004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284f400640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28340039e0 0x7f2834003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284fe00640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 0x7f28340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284ea00640 1 -- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 msgr2=0x7f28340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284ea00640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2850161200 0x7f28340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284ea00640 1 -- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28340039e0 msgr2=0x7f2834003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284ea00640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28340039e0 0x7f2834003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.418+0000 7f284ea00640 1 -- 172.21.15.5:0/4291559412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834003520 con 0x7f2834004800 2024-04-25T10:06:50.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.419+0000 7f284ea00640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f2834004c80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f283c001720 tx=0x7f2834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201c98d06a77405c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c001370 con 0x7f2834004800 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0052d0 con 0x7f2834004800 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f284e000640 1 -- 172.21.15.5:0/4291559412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c02ab90 con 0x7f2834004800 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f2854c00640 1 -- 172.21.15.5:0/4291559412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 msgr2=0x7f2834004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f2854c00640 1 --2- 172.21.15.5:0/4291559412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f2834004c80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f283c001720 tx=0x7f2834005830 comp rx=0 tx=0).stop 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f284fe00640 1 -- 172.21.15.5:0/4291559412 reap_dead start 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f2854c00640 1 -- 172.21.15.5:0/4291559412 shutdown_connections 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.420+0000 7f2854c00640 1 -- 172.21.15.5:0/4291559412 >> 172.21.15.5:0/4291559412 conn(0x7f285008af80 msgr2=0x7f28500d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.421+0000 7f2854c00640 1 -- 172.21.15.5:0/4291559412 shutdown_connections 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.421+0000 7f2854c00640 1 -- 172.21.15.5:0/4291559412 wait complete. 2024-04-25T10:06:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.422+0000 7f2854c00640 1 Processor -- start 2024-04-25T10:06:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.423+0000 7f2854c00640 1 -- start start 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.423+0000 7f2854c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f28500b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.423+0000 7f2854c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f285015edb0 0x7f285015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.423+0000 7f2854c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f285015a600 0x7f285015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.423+0000 7f2854c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b4bb0 con 0x7f2834004800 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.423+0000 7f2854c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b28b0 con 0x7f285015edb0 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f2854c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b4d30 con 0x7f285015a600 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f28500b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f28500b8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43104/0 (socket says 172.21.15.5:43104) 2024-04-25T10:06:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 -- 172.21.15.5:0/4274274991 learned_addr learned my addr 172.21.15.5:0/4274274991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:50.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284ea00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f285015edb0 0x7f285015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284fe00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f285015a600 0x7f285015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 -- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f285015a600 msgr2=0x7f285015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f285015a600 0x7f285015aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 -- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f285015edb0 msgr2=0x7f285015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f285015edb0 0x7f285015f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.424+0000 7f284f400640 1 -- 172.21.15.5:0/4274274991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285015fa60 con 0x7f2834004800 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.425+0000 7f284f400640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f28500b8b70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2840011600 tx=0x7f2840011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.425+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284000f920 con 0x7f2834004800 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.425+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284000fac0 con 0x7f2834004800 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.425+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284000fe70 con 0x7f2834004800 2024-04-25T10:06:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.425+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285015fd40 con 0x7f2834004800 2024-04-25T10:06:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.425+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28500cb870 con 0x7f2834004800 2024-04-25T10:06:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.427+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28500cb380 con 0x7f2834004800 2024-04-25T10:06:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.427+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2840015460 con 0x7f2834004800 2024-04-25T10:06:50.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.428+0000 7f2847400640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f283006b500 0x7f283006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.429+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f284008ae80 con 0x7f2834004800 2024-04-25T10:06:50.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.429+0000 7f284ea00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f283006b500 0x7f283006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.431+0000 7f284ea00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f283006b500 0x7f283006d9c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f283c032000 tx=0x7f283c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.433+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2840019030 con 0x7f2834004800 2024-04-25T10:06:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.668+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2850068f60 con 0x7f2834004800 2024-04-25T10:06:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.670+0000 7f2847400640 1 -- 172.21.15.5:0/4274274991 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f2840059a30 con 0x7f2834004800 2024-04-25T10:06:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:50.676 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:06:21.391577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-04-25T10:06:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f283006b500 msgr2=0x7f283006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f283006b500 0x7f283006d9c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f283c032000 tx=0x7f283c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 msgr2=0x7f28500b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f28500b8b70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2840011600 tx=0x7f2840011ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 shutdown_connections 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f285015a600 0x7f285015aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f283006b500 0x7f283006d9c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f285015edb0 0x7f285015f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 --2- 172.21.15.5:0/4274274991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2834004800 0x7f28500b8b70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 >> 172.21.15.5:0/4274274991 conn(0x7f285008af80 msgr2=0x7f28500b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 shutdown_connections 2024-04-25T10:06:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.673+0000 7f2854c00640 1 -- 172.21.15.5:0/4274274991 wait complete. 2024-04-25T10:06:50.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.908+0000 7fa5c4a00640 1 Processor -- start 2024-04-25T10:06:50.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.908+0000 7fa5c4a00640 1 -- start start 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.910+0000 7fa5c4a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5c0143d30 0x7fa5c015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.910+0000 7fa5c4a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00b3ae0 0x7fa5c00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.910+0000 7fa5c4a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c00b4880 con 0x7fa5c00ce790 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.910+0000 7fa5c4a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c00b45a0 con 0x7fa5c0143d30 2024-04-25T10:06:50.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.910+0000 7fa5c4a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c015e2c0 con 0x7fa5c00b3ae0 2024-04-25T10:06:50.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.910+0000 7fa5bf400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5c0143d30 0x7fa5c015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5bf400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5c0143d30 0x7fa5c015eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43110/0 (socket says 172.21.15.5:43110) 2024-04-25T10:06:50.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5bf400640 1 -- 172.21.15.5:0/3970256544 learned_addr learned my addr 172.21.15.5:0/3970256544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5b6a00640 1 --2- 172.21.15.5:0/3970256544 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00b3ae0 0x7fa5c00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5bf400640 1 -- 172.21.15.5:0/3970256544 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00b3ae0 msgr2=0x7fa5c00b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5bf400640 1 --2- 172.21.15.5:0/3970256544 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00b3ae0 0x7fa5c00b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5bf400640 1 -- 172.21.15.5:0/3970256544 >> v1:172.21.15.80:6789/0 conn(0x7fa5c00ce790 legacy=0x7fa5c015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.911+0000 7fa5bf400640 1 -- 172.21.15.5:0/3970256544 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c015e5a0 con 0x7fa5c0143d30 2024-04-25T10:06:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.912+0000 7fa5bf400640 1 --2- 172.21.15.5:0/3970256544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015eda0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009e50 tx=0x7fa5b002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f14dfc72a96ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.912+0000 7fa5bea00640 1 -- 172.21.15.5:0/3970256544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0036070 con 0x7fa5c0143d30 2024-04-25T10:06:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5bea00640 1 -- 172.21.15.5:0/3970256544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0031600 con 0x7fa5c0143d30 2024-04-25T10:06:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5bea00640 1 -- 172.21.15.5:0/3970256544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00319d0 con 0x7fa5c0143d30 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 -- 172.21.15.5:0/3970256544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 msgr2=0x7fa5c015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/3970256544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015eda0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009e50 tx=0x7fa5b002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 -- 172.21.15.5:0/3970256544 shutdown_connections 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/3970256544 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00b3ae0 0x7fa5c00b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/3970256544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015eda0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 -- 172.21.15.5:0/3970256544 >> 172.21.15.5:0/3970256544 conn(0x7fa5c0066f20 msgr2=0x7fa5c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 -- 172.21.15.5:0/3970256544 shutdown_connections 2024-04-25T10:06:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.913+0000 7fa5c4a00640 1 -- 172.21.15.5:0/3970256544 wait complete. 2024-04-25T10:06:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.915+0000 7fa5c4a00640 1 Processor -- start 2024-04-25T10:06:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.915+0000 7fa5c4a00640 1 -- start start 2024-04-25T10:06:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5c4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5c00b3ae0 0x7fa5c013c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5c4a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00ce790 0x7fa5c01e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5c4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5c4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0144290 con 0x7fa5c0143d30 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5c4a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0144110 con 0x7fa5c00b3ae0 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5c4a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c006e430 con 0x7fa5c00ce790 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.916+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43120/0 (socket says 172.21.15.5:43120) 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 -- 172.21.15.5:0/2321894839 learned_addr learned my addr 172.21.15.5:0/2321894839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bf400640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5c00b3ae0 0x7fa5c013c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5b6a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00ce790 0x7fa5c01e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 -- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00ce790 msgr2=0x7fa5c01e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00ce790 0x7fa5c01e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 -- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5c00b3ae0 msgr2=0x7fa5c013c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5c00b3ae0 0x7fa5c013c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 -- 172.21.15.5:0/2321894839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c013ce60 con 0x7fa5c0143d30 2024-04-25T10:06:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bf400640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5c00b3ae0 0x7fa5c013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.917+0000 7fa5bfe00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015c820 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c440 tx=0x7fa5b800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.918+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8011790 con 0x7fa5c0143d30 2024-04-25T10:06:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.918+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8017070 con 0x7fa5c0143d30 2024-04-25T10:06:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.918+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8011b70 con 0x7fa5c0143d30 2024-04-25T10:06:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.918+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c01f7010 con 0x7fa5c0143d30 2024-04-25T10:06:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.918+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c01f75d0 con 0x7fa5c0143d30 2024-04-25T10:06:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.919+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c015da60 con 0x7fa5c0143d30 2024-04-25T10:06:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.920+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5b8011d10 con 0x7fa5c0143d30 2024-04-25T10:06:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.921+0000 7fa5b7e00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.921+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fa5b808a250 con 0x7fa5c0143d30 2024-04-25T10:06:50.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.922+0000 7fa5bf400640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.923+0000 7fa5bf400640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa59c06b4d0 0x7fa59c06d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0033000 tx=0x7fa5b0009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:50.925+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5c015da60 con 0x7fa5c0143d30 2024-04-25T10:06:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.124+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5c0068f60 con 0x7fa5c0143d30 2024-04-25T10:06:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.127+0000 7fa5b7e00640 1 -- 172.21.15.5:0/2321894839 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17161 (secure 0 0 0) 0x7fa5b8058e90 con 0x7fa5c0143d30 2024-04-25T10:06:51.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:51.134 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:42.412821+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1d","mappings":[{"from":2,"to":1}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.31","mappings":[{"from":4,"to":5}]},{"pgid":"10.33","mappings":[{"from":2,"to":1}]},{"pgid":"10.35","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.129+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa59c06b4d0 msgr2=0x7fa59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.129+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa59c06b4d0 0x7fa59c06d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0033000 tx=0x7fa5b0009000 comp rx=0 tx=0).stop 2024-04-25T10:06:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.129+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 msgr2=0x7fa5c015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.129+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015c820 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c440 tx=0x7fa5b800c900 comp rx=0 tx=0).stop 2024-04-25T10:06:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 shutdown_connections 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5c0143d30 0x7fa5c015c820 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5c00ce790 0x7fa5c01e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa59c06b4d0 0x7fa59c06d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 --2- 172.21.15.5:0/2321894839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5c00b3ae0 0x7fa5c013c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 >> 172.21.15.5:0/2321894839 conn(0x7fa5c0066f20 msgr2=0x7fa5c0159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 shutdown_connections 2024-04-25T10:06:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.130+0000 7fa5c4a00640 1 -- 172.21.15.5:0/2321894839 wait complete. 2024-04-25T10:06:51.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:30.592093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:31.804256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:51.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.367+0000 7fbc91000640 1 Processor -- start 2024-04-25T10:06:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.367+0000 7fbc91000640 1 -- start start 2024-04-25T10:06:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc91000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbc8c06e840 0x7fbc8c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc91000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc8c15b330 0x7fbc8c071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc91000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c0ce570 con 0x7fbc8c0cc840 2024-04-25T10:06:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc91000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c0ce850 con 0x7fbc8c15b330 2024-04-25T10:06:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc91000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c0ceb30 con 0x7fbc8c06e840 2024-04-25T10:06:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc8be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbc8c15b330 0x7fbc8c071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc8b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc8c0cc840 0x7fbc8c06e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49106/0 (socket says 172.21.15.5:49106) 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.369+0000 7fbc8b400640 1 -- 172.21.15.5:0/115496411 learned_addr learned my addr 172.21.15.5:0/115496411 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8aa00640 1 --2- 172.21.15.5:0/115496411 >> v2:172.21.15.80:3300/0 conn(0x7fbc8c06e840 0x7fbc8c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8a000640 1 -- 172.21.15.5:0/115496411 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985597813 0 0) 0x7fbc8c0ce570 con 0x7fbc8c0cc840 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8a000640 1 -- 172.21.15.5:0/115496411 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc70003520 con 0x7fbc8c0cc840 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8be00640 1 -- 172.21.15.5:0/115496411 >> v2:172.21.15.80:3300/0 conn(0x7fbc8c06e840 msgr2=0x7fbc8c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8be00640 1 --2- 172.21.15.5:0/115496411 >> v2:172.21.15.80:3300/0 conn(0x7fbc8c06e840 0x7fbc8c15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8be00640 1 -- 172.21.15.5:0/115496411 >> v1:172.21.15.5:6789/0 conn(0x7fbc8c0cc840 legacy=0x7fbc8c06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8be00640 1 -- 172.21.15.5:0/115496411 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c0cee10 con 0x7fbc8c15b330 2024-04-25T10:06:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.370+0000 7fbc8aa00640 1 --2- 172.21.15.5:0/115496411 >> v2:172.21.15.80:3300/0 conn(0x7fbc8c06e840 0x7fbc8c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.371+0000 7fbc8be00640 1 --2- 172.21.15.5:0/115496411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c15b330 0x7fbc8c071000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c06ba20 tx=0x7fbc8402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8b6109dcf39171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.372+0000 7fbc8a000640 1 -- 172.21.15.5:0/115496411 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007740 con 0x7fbc8c15b330 2024-04-25T10:06:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.372+0000 7fbc8a000640 1 -- 172.21.15.5:0/115496411 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc840078e0 con 0x7fbc8c15b330 2024-04-25T10:06:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.372+0000 7fbc8a000640 1 -- 172.21.15.5:0/115496411 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84007cb0 con 0x7fbc8c15b330 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.372+0000 7fbc91000640 1 -- 172.21.15.5:0/115496411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c15b330 msgr2=0x7fbc8c071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.372+0000 7fbc91000640 1 --2- 172.21.15.5:0/115496411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c15b330 0x7fbc8c071000 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c06ba20 tx=0x7fbc8402a680 comp rx=0 tx=0).stop 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.372+0000 7fbc91000640 1 -- 172.21.15.5:0/115496411 shutdown_connections 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.373+0000 7fbc91000640 1 --2- 172.21.15.5:0/115496411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c15b330 0x7fbc8c071000 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.373+0000 7fbc91000640 1 --2- 172.21.15.5:0/115496411 >> v2:172.21.15.80:3300/0 conn(0x7fbc8c06e840 0x7fbc8c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.373+0000 7fbc91000640 1 -- 172.21.15.5:0/115496411 >> 172.21.15.5:0/115496411 conn(0x7fbc8c066f20 msgr2=0x7fbc8c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.373+0000 7fbc91000640 1 -- 172.21.15.5:0/115496411 shutdown_connections 2024-04-25T10:06:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.373+0000 7fbc91000640 1 -- 172.21.15.5:0/115496411 wait complete. 2024-04-25T10:06:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.374+0000 7fbc91000640 1 Processor -- start 2024-04-25T10:06:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.375+0000 7fbc91000640 1 -- start start 2024-04-25T10:06:51.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc91000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 0x7fbc8c142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc91000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8c0cc840 0x7fbc8c147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc91000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8c15b330 0x7fbc8c1482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc91000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0ce350 con 0x7fbc8c06e840 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc91000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0ce1d0 con 0x7fbc8c0cc840 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc91000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0ccc20 con 0x7fbc8c15b330 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 0x7fbc8c142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 0x7fbc8c142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43144/0 (socket says 172.21.15.5:43144) 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 -- 172.21.15.5:0/49726685 learned_addr learned my addr 172.21.15.5:0/49726685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 -- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8c15b330 msgr2=0x7fbc8c1482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8be00640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8c15b330 0x7fbc8c1482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8c15b330 0x7fbc8c1482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 -- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8c0cc840 msgr2=0x7fbc8c147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8c0cc840 0x7fbc8c147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.376+0000 7fbc8b400640 1 -- 172.21.15.5:0/49726685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c158460 con 0x7fbc8c06e840 2024-04-25T10:06:51.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.377+0000 7fbc8b400640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 0x7fbc8c142d90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c430 tx=0x7fbc7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.377+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c017070 con 0x7fbc8c06e840 2024-04-25T10:06:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.377+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c005ce0 con 0x7fbc8c06e840 2024-04-25T10:06:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.377+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0072f0 con 0x7fbc8c06e840 2024-04-25T10:06:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.377+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c158740 con 0x7fbc8c06e840 2024-04-25T10:06:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.377+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c1432d0 con 0x7fbc8c06e840 2024-04-25T10:06:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.378+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c073d40 con 0x7fbc8c06e840 2024-04-25T10:06:51.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.379+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc7c01a070 con 0x7fbc8c06e840 2024-04-25T10:06:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.381+0000 7fbc83400640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.381+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbc7c0899b0 con 0x7fbc8c06e840 2024-04-25T10:06:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.382+0000 7fbc8aa00640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.383+0000 7fbc8aa00640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbc78006e80 tx=0x7fbc78008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.385+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc8c073d40 con 0x7fbc8c06e840 2024-04-25T10:06:51.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.585+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc8c0cb8d0 con 0x7fbc8c06e840 2024-04-25T10:06:51.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.586+0000 7fbc83400640 1 -- 172.21.15.5:0/49726685 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc7c058560 con 0x7fbc8c06e840 2024-04-25T10:06:51.592 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc6c06b4d0 msgr2=0x7fbc6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbc78006e80 tx=0x7fbc78008040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 msgr2=0x7fbc8c142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 0x7fbc8c142d90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c430 tx=0x7fbc7c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 shutdown_connections 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8c15b330 0x7fbc8c1482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc6c06b4d0 0x7fbc6c06d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8c0cc840 0x7fbc8c147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 --2- 172.21.15.5:0/49726685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc8c06e840 0x7fbc8c142d90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.589+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 >> 172.21.15.5:0/49726685 conn(0x7fbc8c066f20 msgr2=0x7fbc8c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.590+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 shutdown_connections 2024-04-25T10:06:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.590+0000 7fbc91000640 1 -- 172.21.15.5:0/49726685 wait complete. 2024-04-25T10:06:51.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.830+0000 7fbf43a00640 1 Processor -- start 2024-04-25T10:06:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.831+0000 7fbf43a00640 1 -- start start 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.832+0000 7fbf43a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf3c1465d0 0x7fbf3c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.832+0000 7fbf43a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbf3c0ae860 0x7fbf3c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.832+0000 7fbf43a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0aece0 0x7fbf3c071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.832+0000 7fbf43a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c15eed0 con 0x7fbf3c1465d0 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.832+0000 7fbf43a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c06f080 con 0x7fbf3c0ae860 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.832+0000 7fbf43a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c06f360 con 0x7fbf3c0aece0 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf3c1465d0 0x7fbf3c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf3c1465d0 0x7fbf3c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43148/0 (socket says 172.21.15.5:43148) 2024-04-25T10:06:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 -- 172.21.15.5:0/4101523999 learned_addr learned my addr 172.21.15.5:0/4101523999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:51.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf3be00640 1 --2- 172.21.15.5:0/4101523999 >> v2:172.21.15.80:3300/0 conn(0x7fbf3c0ae860 0x7fbf3c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41a00640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0aece0 0x7fbf3c071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 -- 172.21.15.5:0/4101523999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0aece0 msgr2=0x7fbf3c071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0aece0 0x7fbf3c071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 -- 172.21.15.5:0/4101523999 >> v2:172.21.15.80:3300/0 conn(0x7fbf3c0ae860 msgr2=0x7fbf3c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 --2- 172.21.15.5:0/4101523999 >> v2:172.21.15.80:3300/0 conn(0x7fbf3c0ae860 0x7fbf3c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41000640 1 -- 172.21.15.5:0/4101523999 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0ae060 con 0x7fbf3c1465d0 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf3be00640 1 --2- 172.21.15.5:0/4101523999 >> v2:172.21.15.80:3300/0 conn(0x7fbf3c0ae860 0x7fbf3c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.833+0000 7fbf41a00640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0aece0 0x7fbf3c071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.835+0000 7fbf41000640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c15e990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06ba00 tx=0x7fbf2c02a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225e0a6e0e85f459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.835+0000 7fbf3b400640 1 -- 172.21.15.5:0/4101523999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c036070 con 0x7fbf3c1465d0 2024-04-25T10:06:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.835+0000 7fbf3b400640 1 -- 172.21.15.5:0/4101523999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c031600 con 0x7fbf3c1465d0 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.835+0000 7fbf3b400640 1 -- 172.21.15.5:0/4101523999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0319d0 con 0x7fbf3c1465d0 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 -- 172.21.15.5:0/4101523999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c1465d0 msgr2=0x7fbf3c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c15e990 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c06ba00 tx=0x7fbf2c02a690 comp rx=0 tx=0).stop 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 -- 172.21.15.5:0/4101523999 shutdown_connections 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0aece0 0x7fbf3c071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 --2- 172.21.15.5:0/4101523999 >> v2:172.21.15.80:3300/0 conn(0x7fbf3c0ae860 0x7fbf3c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 --2- 172.21.15.5:0/4101523999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c15e990 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 -- 172.21.15.5:0/4101523999 >> 172.21.15.5:0/4101523999 conn(0x7fbf3c06ab60 msgr2=0x7fbf3c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 -- 172.21.15.5:0/4101523999 shutdown_connections 2024-04-25T10:06:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.836+0000 7fbf43a00640 1 -- 172.21.15.5:0/4101523999 wait complete. 2024-04-25T10:06:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.838+0000 7fbf43a00640 1 Processor -- start 2024-04-25T10:06:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.839+0000 7fbf43a00640 1 -- start start 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf43a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0ae860 0x7fbf3c13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf43a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 0x7fbf3c1e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf43a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c1e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf43a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c06e7d0 con 0x7fbf3c0aece0 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf43a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c1469b0 con 0x7fbf3c1465d0 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf43a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c06ece0 con 0x7fbf3c0ae860 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 0x7fbf3c1e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 0x7fbf3c1e8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43156/0 (socket says 172.21.15.5:43156) 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.840+0000 7fbf3be00640 1 -- 172.21.15.5:0/290842628 learned_addr learned my addr 172.21.15.5:0/290842628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf41000640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0ae860 0x7fbf3c13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf41a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c1e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf3be00640 1 -- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0ae860 msgr2=0x7fbf3c13eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf3be00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0ae860 0x7fbf3c13eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf3be00640 1 -- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf3c1465d0 msgr2=0x7fbf3c1e87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf3be00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c1e87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf3be00640 1 -- 172.21.15.5:0/290842628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c1f3550 con 0x7fbf3c0aece0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.841+0000 7fbf3be00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 0x7fbf3c1e8280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c7f0 tx=0x7fbf2800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.842+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28015070 con 0x7fbf3c0aece0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.842+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28007a30 con 0x7fbf3c0aece0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.842+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c0b2140 con 0x7fbf3c0aece0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.842+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28007e00 con 0x7fbf3c0aece0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.842+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c0b1e60 con 0x7fbf3c0aece0 2024-04-25T10:06:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.843+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c1e8030 con 0x7fbf3c0aece0 2024-04-25T10:06:51.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.844+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf28018070 con 0x7fbf3c0aece0 2024-04-25T10:06:51.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.845+0000 7fbf38c00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.845+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7fbf2808a3a0 con 0x7fbf3c0aece0 2024-04-25T10:06:51.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.846+0000 7fbf41000640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.847+0000 7fbf41000640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c033000 tx=0x7fbf2c009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:51.851+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf3c1e8030 con 0x7fbf3c0aece0 2024-04-25T10:06:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.051+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf3c068f60 con 0x7fbf3c0aece0 2024-04-25T10:06:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.052+0000 7fbf38c00640 1 -- 172.21.15.5:0/290842628 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf28058fd0 con 0x7fbf3c0aece0 2024-04-25T10:06:52.059 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf1c06b4d0 msgr2=0x7fbf1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c033000 tx=0x7fbf2c009000 comp rx=0 tx=0).stop 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 msgr2=0x7fbf3c1e8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 0x7fbf3c1e8280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c7f0 tx=0x7fbf2800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 shutdown_connections 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf3c1465d0 0x7fbf3c1e87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0aece0 0x7fbf3c1e8280 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 --2- 172.21.15.5:0/290842628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf3c0ae860 0x7fbf3c13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.055+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 >> 172.21.15.5:0/290842628 conn(0x7fbf3c06ab60 msgr2=0x7fbf3c1444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.056+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 shutdown_connections 2024-04-25T10:06:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.056+0000 7fbf43a00640 1 -- 172.21.15.5:0/290842628 wait complete. 2024-04-25T10:06:52.081 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.287+0000 7f6a10800640 1 Processor -- start 2024-04-25T10:06:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.288+0000 7f6a10800640 1 -- start start 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.289+0000 7f6a10800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a081465d0 0x7f6a0815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.289+0000 7f6a10800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080ae860 0x7f6a08071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a10800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0815eed0 con 0x7f6a080aece0 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a10800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0806f080 con 0x7f6a081465d0 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a10800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0806f360 con 0x7f6a080ae860 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6a081465d0 0x7f6a0815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0e800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6a080aece0 0x7f6a0815f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49108/0 (socket says 172.21.15.5:49108) 2024-04-25T10:06:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0e800640 1 -- 172.21.15.5:0/69330353 learned_addr learned my addr 172.21.15.5:0/69330353 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0de00640 1 -- 172.21.15.5:0/69330353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080ae860 msgr2=0x7f6a08071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0d400640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080ae860 0x7f6a08071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0ca00640 1 -- 172.21.15.5:0/69330353 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610855332 0 0) 0x7f6a04001490 con 0x7f6a080aece0 2024-04-25T10:06:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0de00640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080ae860 0x7f6a08071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0de00640 1 -- 172.21.15.5:0/69330353 >> v1:172.21.15.5:6789/0 conn(0x7f6a080aece0 legacy=0x7f6a0815f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.290+0000 7f6a0de00640 1 -- 172.21.15.5:0/69330353 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a080ae060 con 0x7f6a081465d0 2024-04-25T10:06:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.291+0000 7f6a0d400640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080ae860 0x7f6a08071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.292+0000 7f6a0de00640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a081465d0 0x7f6a0815e990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0097a0 tx=0x7f69fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44675c20aa44e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.292+0000 7f6a0ca00640 1 -- 172.21.15.5:0/69330353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037070 con 0x7f6a081465d0 2024-04-25T10:06:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.292+0000 7f6a0ca00640 1 -- 172.21.15.5:0/69330353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc032440 con 0x7f6a081465d0 2024-04-25T10:06:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.292+0000 7f6a0ca00640 1 -- 172.21.15.5:0/69330353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc032810 con 0x7f6a081465d0 2024-04-25T10:06:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.292+0000 7f6a10800640 1 -- 172.21.15.5:0/69330353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a081465d0 msgr2=0x7f6a0815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.292+0000 7f6a10800640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a081465d0 0x7f6a0815e990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0097a0 tx=0x7f69fc02a180 comp rx=0 tx=0).stop 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.293+0000 7f6a10800640 1 -- 172.21.15.5:0/69330353 shutdown_connections 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.293+0000 7f6a10800640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080ae860 0x7f6a08071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.293+0000 7f6a10800640 1 --2- 172.21.15.5:0/69330353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a081465d0 0x7f6a0815e990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.293+0000 7f6a10800640 1 -- 172.21.15.5:0/69330353 >> 172.21.15.5:0/69330353 conn(0x7f6a0806ab60 msgr2=0x7f6a0806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.293+0000 7f6a10800640 1 -- 172.21.15.5:0/69330353 shutdown_connections 2024-04-25T10:06:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.293+0000 7f6a10800640 1 -- 172.21.15.5:0/69330353 wait complete. 2024-04-25T10:06:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.295+0000 7f6a10800640 1 Processor -- start 2024-04-25T10:06:52.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.295+0000 7f6a10800640 1 -- start start 2024-04-25T10:06:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a10800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 0x7f6a080ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a10800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080aece0 0x7f6a080b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a10800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a081465d0 0x7f6a0813d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a10800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0806e7d0 con 0x7f6a080ae860 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a10800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a081469b0 con 0x7f6a081465d0 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a10800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0806ece0 con 0x7f6a080aece0 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a0de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 0x7f6a080ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a0de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 0x7f6a080ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43182/0 (socket says 172.21.15.5:43182) 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a0de00640 1 -- 172.21.15.5:0/4114605502 learned_addr learned my addr 172.21.15.5:0/4114605502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a0de00640 1 -- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080aece0 msgr2=0x7f6a080b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a0de00640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080aece0 0x7f6a080b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.296+0000 7f6a0de00640 1 -- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a081465d0 msgr2=0x7f6a0813d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a0e800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a081465d0 0x7f6a0813d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a0de00640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a081465d0 0x7f6a0813d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a0de00640 1 -- 172.21.15.5:0/4114605502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0813add0 con 0x7f6a080ae860 2024-04-25T10:06:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a0e800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a081465d0 0x7f6a0813d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a0de00640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 0x7f6a080ced90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69fc035040 tx=0x7f69fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc037040 con 0x7f6a080ae860 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc030770 con 0x7f6a080ae860 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc030b80 con 0x7f6a080ae860 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.297+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a080cf2d0 con 0x7f6a080ae860 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.298+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0813b040 con 0x7f6a080ae860 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.299+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a08068f60 con 0x7f6a080ae860 2024-04-25T10:06:52.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.300+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69fc044020 con 0x7f6a080ae860 2024-04-25T10:06:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.301+0000 7f6a02000640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69ec06b500 0x7f69ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.301+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f69fc0aab50 con 0x7f6a080ae860 2024-04-25T10:06:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.301+0000 7f6a0d400640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69ec06b500 0x7f69ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.303+0000 7f6a0d400640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69ec06b500 0x7f69ec06d9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006fd0 tx=0x7f69f8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.305+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69fc032e20 con 0x7f6a080ae860 2024-04-25T10:06:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.509+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6a0806af40 con 0x7f6a080ae860 2024-04-25T10:06:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.510+0000 7f6a02000640 1 -- 172.21.15.5:0/4114605502 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69fc0313a0 con 0x7f6a080ae860 2024-04-25T10:06:52.518 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69ec06b500 msgr2=0x7f69ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69ec06b500 0x7f69ec06d9c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f69f8006fd0 tx=0x7f69f8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 msgr2=0x7f6a080ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 0x7f6a080ced90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69fc035040 tx=0x7f69fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 shutdown_connections 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69ec06b500 0x7f69ec06d9c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a081465d0 0x7f6a0813d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a080aece0 0x7f6a080b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 --2- 172.21.15.5:0/4114605502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a080ae860 0x7f6a080ced90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.513+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 >> 172.21.15.5:0/4114605502 conn(0x7f6a0806ab60 msgr2=0x7f6a08145280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.514+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 shutdown_connections 2024-04-25T10:06:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.514+0000 7f6a10800640 1 -- 172.21.15.5:0/4114605502 wait complete. 2024-04-25T10:06:52.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.747+0000 7f6be5e00640 1 Processor -- start 2024-04-25T10:06:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.748+0000 7f6be5e00640 1 -- start start 2024-04-25T10:06:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.749+0000 7f6be5e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be0161200 0x7f6be00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.749+0000 7f6be5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6be00d2380 0x7f6be00d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.749+0000 7f6be5e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be00cf410 con 0x7f6be00d0600 2024-04-25T10:06:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.749+0000 7f6be5e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00d27c0 con 0x7f6be00d2380 2024-04-25T10:06:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.749+0000 7f6be5e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b28b0 con 0x7f6be0161200 2024-04-25T10:06:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6bdfe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6be00d2380 0x7f6be00d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6be5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6be00d0600 0x7f6be00ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49120/0 (socket says 172.21.15.5:49120) 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6be5400640 1 -- 172.21.15.5:0/2079315697 learned_addr learned my addr 172.21.15.5:0/2079315697 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6be4a00640 1 --2- 172.21.15.5:0/2079315697 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be0161200 0x7f6be00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6bdfe00640 1 -- 172.21.15.5:0/2079315697 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be0161200 msgr2=0x7f6be00b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6bdfe00640 1 --2- 172.21.15.5:0/2079315697 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be0161200 0x7f6be00b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6bdfe00640 1 -- 172.21.15.5:0/2079315697 >> v1:172.21.15.5:6789/0 conn(0x7f6be00d0600 legacy=0x7f6be00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6bdfe00640 1 -- 172.21.15.5:0/2079315697 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be00cf6f0 con 0x7f6be00d2380 2024-04-25T10:06:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.750+0000 7f6bdf400640 1 -- 172.21.15.5:0/2079315697 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829126177 0 0) 0x7f6be00cf410 con 0x7f6be00d0600 2024-04-25T10:06:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.751+0000 7f6bdfe00640 1 --2- 172.21.15.5:0/2079315697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be00d00c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80097d0 tx=0x7f6bc802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6e339d92464ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.752+0000 7f6bdf400640 1 -- 172.21.15.5:0/2079315697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8037070 con 0x7f6be00d2380 2024-04-25T10:06:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.752+0000 7f6bdf400640 1 -- 172.21.15.5:0/2079315697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8032440 con 0x7f6be00d2380 2024-04-25T10:06:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.752+0000 7f6bdf400640 1 -- 172.21.15.5:0/2079315697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8032810 con 0x7f6be00d2380 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.752+0000 7f6be5e00640 1 -- 172.21.15.5:0/2079315697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 msgr2=0x7f6be00d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.752+0000 7f6be5e00640 1 --2- 172.21.15.5:0/2079315697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be00d00c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80097d0 tx=0x7f6bc802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.753+0000 7f6be5e00640 1 -- 172.21.15.5:0/2079315697 shutdown_connections 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.753+0000 7f6be5e00640 1 --2- 172.21.15.5:0/2079315697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be00d00c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.753+0000 7f6be5e00640 1 --2- 172.21.15.5:0/2079315697 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be0161200 0x7f6be00b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.753+0000 7f6be5e00640 1 -- 172.21.15.5:0/2079315697 >> 172.21.15.5:0/2079315697 conn(0x7f6be008af80 msgr2=0x7f6be00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.753+0000 7f6be5e00640 1 -- 172.21.15.5:0/2079315697 shutdown_connections 2024-04-25T10:06:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.753+0000 7f6be5e00640 1 -- 172.21.15.5:0/2079315697 wait complete. 2024-04-25T10:06:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.754+0000 7f6be5e00640 1 Processor -- start 2024-04-25T10:06:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.755+0000 7f6be5e00640 1 -- start start 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be00d0600 0x7f6be00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6be0161200 0x7f6be01f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b3f90 con 0x7f6be00d2380 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00d42b0 con 0x7f6be0161200 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b4110 con 0x7f6be00d0600 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be01549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43196/0 (socket says 172.21.15.5:43196) 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 -- 172.21.15.5:0/3957372680 learned_addr learned my addr 172.21.15.5:0/3957372680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be4a00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be00d0600 0x7f6be00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be5400640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6be0161200 0x7f6be01f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 -- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be00d0600 msgr2=0x7f6be00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be00d0600 0x7f6be00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 -- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6be0161200 msgr2=0x7f6be01f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6be0161200 0x7f6be01f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6bdfe00640 1 -- 172.21.15.5:0/3957372680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be00c8670 con 0x7f6be00d2380 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.756+0000 7f6be4a00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be00d0600 0x7f6be00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.757+0000 7f6bdfe00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be01549a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8035040 tx=0x7f6bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.757+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8037050 con 0x7f6be00d2380 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.757+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8038040 con 0x7f6be00d2380 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.757+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be00c8950 con 0x7f6be00d2380 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.757+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8004470 con 0x7f6be00d2380 2024-04-25T10:06:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.757+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be00c8f10 con 0x7f6be00d2380 2024-04-25T10:06:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.758+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be0068f60 con 0x7f6be00d2380 2024-04-25T10:06:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.759+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bc8030900 con 0x7f6be00d2380 2024-04-25T10:06:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.760+0000 7f6bdcc00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6bbc06b4d0 0x7f6bbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.761+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f6bc80ab090 con 0x7f6be00d2380 2024-04-25T10:06:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.761+0000 7f6be4a00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6bbc06b4d0 0x7f6bbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.762+0000 7f6be4a00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6bbc06b4d0 0x7f6bbc06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009f50 tx=0x7f6bd000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:52.764+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bc8079df0 con 0x7f6be00d2380 2024-04-25T10:06:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.006+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be006d960 con 0x7f6be00d2380 2024-04-25T10:06:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.007+0000 7f6bdcc00640 1 -- 172.21.15.5:0/3957372680 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f6bc80796f0 con 0x7f6be00d2380 2024-04-25T10:06:53.014 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:06:21.391577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:53.014 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-04-25T10:06:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6bbc06b4d0 msgr2=0x7f6bbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6bbc06b4d0 0x7f6bbc06d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009f50 tx=0x7f6bd000b040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 msgr2=0x7f6be01549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be01549a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8035040 tx=0x7f6bc80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 shutdown_connections 2024-04-25T10:06:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6bbc06b4d0 0x7f6bbc06d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6be0161200 0x7f6be01f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6be00d2380 0x7f6be01549a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 --2- 172.21.15.5:0/3957372680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6be00d0600 0x7f6be00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 >> 172.21.15.5:0/3957372680 conn(0x7f6be008af80 msgr2=0x7f6be015f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 shutdown_connections 2024-04-25T10:06:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.010+0000 7f6be5e00640 1 -- 172.21.15.5:0/3957372680 wait complete. 2024-04-25T10:06:53.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.235+0000 7f6abda00640 1 Processor -- start 2024-04-25T10:06:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.236+0000 7f6abda00640 1 -- start start 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.237+0000 7f6abda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.237+0000 7f6abda00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6ab80d2380 0x7f6ab80d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.237+0000 7f6abda00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ab80d0600 0x7f6ab80ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.237+0000 7f6abda00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab80d27c0 con 0x7f6ab80d0600 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.237+0000 7f6abda00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab80b28b0 con 0x7f6ab80d2380 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.237+0000 7f6abda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80cf410 con 0x7f6ab8161200 2024-04-25T10:06:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ab80d0600 0x7f6ab80ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ab80d0600 0x7f6ab80ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43200/0 (socket says 172.21.15.5:43200) 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 -- 172.21.15.5:0/2123215882 learned_addr learned my addr 172.21.15.5:0/2123215882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6ab7400640 1 --2- 172.21.15.5:0/2123215882 >> v2:172.21.15.80:3300/0 conn(0x7f6ab80d2380 0x7f6ab80d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6ab7e00640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 -- 172.21.15.5:0/2123215882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 msgr2=0x7f6ab80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 -- 172.21.15.5:0/2123215882 >> v2:172.21.15.80:3300/0 conn(0x7f6ab80d2380 msgr2=0x7f6ab80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 --2- 172.21.15.5:0/2123215882 >> v2:172.21.15.80:3300/0 conn(0x7f6ab80d2380 0x7f6ab80d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6abd000640 1 -- 172.21.15.5:0/2123215882 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80cf6f0 con 0x7f6ab80d0600 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.238+0000 7f6ab7e00640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.240+0000 7f6abd000640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab80ce280 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6ab808ba40 tx=0x7f6ab002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ebf06d0a3df9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.240+0000 7f6ab6a00640 1 -- 172.21.15.5:0/2123215882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0036070 con 0x7f6ab80d0600 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.240+0000 7f6ab6a00640 1 -- 172.21.15.5:0/2123215882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab0031600 con 0x7f6ab80d0600 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.240+0000 7f6ab6a00640 1 -- 172.21.15.5:0/2123215882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00319d0 con 0x7f6ab80d0600 2024-04-25T10:06:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.240+0000 7f6abda00640 1 -- 172.21.15.5:0/2123215882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 msgr2=0x7f6ab80ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.240+0000 7f6abda00640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab80ce280 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6ab808ba40 tx=0x7f6ab002a6d0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 -- 172.21.15.5:0/2123215882 shutdown_connections 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab80ce280 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 --2- 172.21.15.5:0/2123215882 >> v2:172.21.15.80:3300/0 conn(0x7f6ab80d2380 0x7f6ab80d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 --2- 172.21.15.5:0/2123215882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 -- 172.21.15.5:0/2123215882 >> 172.21.15.5:0/2123215882 conn(0x7f6ab808af80 msgr2=0x7f6ab80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 -- 172.21.15.5:0/2123215882 shutdown_connections 2024-04-25T10:06:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.241+0000 7f6abda00640 1 -- 172.21.15.5:0/2123215882 wait complete. 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.242+0000 7f6abda00640 1 Processor -- start 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.243+0000 7f6abda00640 1 -- start start 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.243+0000 7f6abda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab814ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6abda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ab80d2380 0x7f6ab80b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6abda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6abda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80ce7c0 con 0x7f6ab80d0600 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6abda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80d42b0 con 0x7f6ab80d2380 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6abda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80ce940 con 0x7f6ab8161200 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab814ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab814ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43202/0 (socket says 172.21.15.5:43202) 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 -- 172.21.15.5:0/2832497424 learned_addr learned my addr 172.21.15.5:0/2832497424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 -- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 msgr2=0x7f6ab815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 -- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ab80d2380 msgr2=0x7f6ab80b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7400640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ab80d2380 0x7f6ab80b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ab80d2380 0x7f6ab80b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.244+0000 7f6ab7e00640 1 -- 172.21.15.5:0/2832497424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab80b8070 con 0x7f6ab80d0600 2024-04-25T10:06:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.245+0000 7f6ab7e00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab814ff10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800c7f0 tx=0x7f6aa800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.245+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8015070 con 0x7f6ab80d0600 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.245+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa8007820 con 0x7f6ab80d0600 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.245+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8007bd0 con 0x7f6ab80d0600 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.245+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab80b8350 con 0x7f6ab80d0600 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.245+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab80b8910 con 0x7f6ab80d0600 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.246+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab80ce280 con 0x7f6ab80d0600 2024-04-25T10:06:53.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.247+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6aa8018070 con 0x7f6ab80d0600 2024-04-25T10:06:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.248+0000 7f6aafe00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6a9406b4d0 0x7f6a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.248+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f6aa808a640 con 0x7f6ab80d0600 2024-04-25T10:06:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.249+0000 7f6ab7400640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6a9406b4d0 0x7f6a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.250+0000 7f6ab7400640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6a9406b4d0 0x7f6a9406d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80b1100 tx=0x7f6aa0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.253+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ab80ce280 con 0x7f6ab80d0600 2024-04-25T10:06:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.459+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ab80b03c0 con 0x7f6ab80d0600 2024-04-25T10:06:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.462+0000 7f6aafe00640 1 -- 172.21.15.5:0/2832497424 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17161 (secure 0 0 0) 0x7f6aa80591f0 con 0x7f6ab80d0600 2024-04-25T10:06:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:53.469 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:42.412821+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1d","mappings":[{"from":2,"to":1}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.31","mappings":[{"from":4,"to":5}]},{"pgid":"10.33","mappings":[{"from":2,"to":1}]},{"pgid":"10.35","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6a9406b4d0 msgr2=0x7f6a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6a9406b4d0 0x7f6a9406d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80b1100 tx=0x7f6aa0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 msgr2=0x7f6ab814ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab814ff10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800c7f0 tx=0x7f6aa800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 shutdown_connections 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ab8161200 0x7f6ab815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6a9406b4d0 0x7f6a9406d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ab80d2380 0x7f6ab80b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 --2- 172.21.15.5:0/2832497424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ab80d0600 0x7f6ab814ff10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 >> 172.21.15.5:0/2832497424 conn(0x7f6ab808af80 msgr2=0x7f6ab81604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.464+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 shutdown_connections 2024-04-25T10:06:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.465+0000 7f6abda00640 1 -- 172.21.15.5:0/2832497424 wait complete. 2024-04-25T10:06:53.489 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:53.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:53.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.696+0000 7f8b26a00640 1 Processor -- start 2024-04-25T10:06:53.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.696+0000 7f8b26a00640 1 -- start start 2024-04-25T10:06:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.696+0000 7f8b26a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8b20141da0 0x7f8b2006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.696+0000 7f8b26a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b20071d30 con 0x7f8b2006e640 2024-04-25T10:06:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b26a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2006d620 con 0x7f8b2015d510 2024-04-25T10:06:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b26a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b20142180 con 0x7f8b20141da0 2024-04-25T10:06:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b24c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8b2006e640 0x7f8b2015ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49134/0 (socket says 172.21.15.5:49134) 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b24c00640 1 -- 172.21.15.5:0/445772854 learned_addr learned my addr 172.21.15.5:0/445772854 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b25600640 1 --2- 172.21.15.5:0/445772854 >> v2:172.21.15.80:3300/0 conn(0x7f8b20141da0 0x7f8b2006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799008797 0 0) 0x7f8b20071d30 con 0x7f8b2006e640 2024-04-25T10:06:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.697+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b08003520 con 0x7f8b2006e640 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2763208058 0 0) 0x7f8b08003520 con 0x7f8b2006e640 2024-04-25T10:06:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 >> v2:172.21.15.80:3300/0 conn(0x7f8b20141da0 msgr2=0x7f8b2006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 --2- 172.21.15.5:0/445772854 >> v2:172.21.15.80:3300/0 conn(0x7f8b20141da0 0x7f8b2006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 >> v1:172.21.15.80:6789/0 conn(0x7f8b2015d510 legacy=0x7f8b200cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2006d900 con 0x7f8b2006e640 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8b0c001670 con 0x7f8b2006e640 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 >> v1:172.21.15.5:6789/0 conn(0x7f8b2006e640 legacy=0x7f8b2015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b080039b0 0x7f8b08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08004800 0x7f8b08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b080051c0 0x7f8b0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b20142180 con 0x7f8b08004800 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0800c010 con 0x7f8b080051c0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0800c210 con 0x7f8b080039b0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.698+0000 7f8b24c00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08004800 0x7f8b08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b25600640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b080039b0 0x7f8b08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b26000640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b080051c0 0x7f8b0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b24c00640 1 -- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b080039b0 msgr2=0x7f8b08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b24c00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b080039b0 0x7f8b08003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b24c00640 1 -- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b080051c0 msgr2=0x7f8b0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b24c00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b080051c0 0x7f8b0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b24c00640 1 -- 172.21.15.5:0/445772854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2006d900 con 0x7f8b08004800 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b26000640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b080051c0 0x7f8b0800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.699+0000 7f8b26000640 1 -- 172.21.15.5:0/445772854 reap_dead start 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.700+0000 7f8b24c00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08004800 0x7f8b08004c80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00c340 tx=0x7f8b08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a3a9a11f207e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.700+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c007df0 con 0x7f8b08004800 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.700+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c005ad0 con 0x7f8b08004800 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.700+0000 7f8b1fe00640 1 -- 172.21.15.5:0/445772854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c007070 con 0x7f8b08004800 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 -- 172.21.15.5:0/445772854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08004800 msgr2=0x7f8b08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08004800 0x7f8b08004c80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00c340 tx=0x7f8b08005830 comp rx=0 tx=0).stop 2024-04-25T10:06:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 -- 172.21.15.5:0/445772854 shutdown_connections 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 --2- 172.21.15.5:0/445772854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08004800 0x7f8b08004c80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 -- 172.21.15.5:0/445772854 >> 172.21.15.5:0/445772854 conn(0x7f8b20066f20 msgr2=0x7f8b2006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 -- 172.21.15.5:0/445772854 shutdown_connections 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.701+0000 7f8b26a00640 1 -- 172.21.15.5:0/445772854 wait complete. 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.703+0000 7f8b26a00640 1 Processor -- start 2024-04-25T10:06:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.703+0000 7f8b26a00640 1 -- start start 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b08004800 0x7f8b2013bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b2013cea0 0x7f8b200b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 0x7f8b201e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2006e280 con 0x7f8b201e5100 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2006e100 con 0x7f8b08004800 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2006e400 con 0x7f8b2013cea0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 0x7f8b201e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 0x7f8b201e5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43216/0 (socket says 172.21.15.5:43216) 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.704+0000 7f8b26000640 1 -- 172.21.15.5:0/2993581473 learned_addr learned my addr 172.21.15.5:0/2993581473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b25600640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b08004800 0x7f8b2013bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b24c00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b2013cea0 0x7f8b200b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b26000640 1 -- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b2013cea0 msgr2=0x7f8b200b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b26000640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b2013cea0 0x7f8b200b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b26000640 1 -- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b08004800 msgr2=0x7f8b2013bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b26000640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b08004800 0x7f8b2013bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b26000640 1 -- 172.21.15.5:0/2993581473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2013c160 con 0x7f8b201e5100 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.705+0000 7f8b26000640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 0x7f8b201e5590 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8b200cd2b0 tx=0x7f8b1800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.706+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18018070 con 0x7f8b201e5100 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.706+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b18003da0 con 0x7f8b201e5100 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.706+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b200b19b0 con 0x7f8b201e5100 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.706+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180052b0 con 0x7f8b201e5100 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.706+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b200b16d0 con 0x7f8b201e5100 2024-04-25T10:06:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.707+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b20068f60 con 0x7f8b201e5100 2024-04-25T10:06:53.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.708+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b1801b070 con 0x7f8b201e5100 2024-04-25T10:06:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.709+0000 7f8b1d600640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b0006b4d0 0x7f8b0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.709+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f8b18089a60 con 0x7f8b201e5100 2024-04-25T10:06:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.709+0000 7f8b25600640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b0006b4d0 0x7f8b0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.712+0000 7f8b25600640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b0006b4d0 0x7f8b0006d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009b40 tx=0x7f8b14008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.713+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b2013c160 con 0x7f8b201e5100 2024-04-25T10:06:53.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.950+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b2006f240 con 0x7f8b201e5100 2024-04-25T10:06:53.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.951+0000 7f8b1d600640 1 -- 172.21.15.5:0/2993581473 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f8b18058050 con 0x7f8b201e5100 2024-04-25T10:06:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:53.957 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:06:21.391577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:53.958 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-04-25T10:06:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.953+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b0006b4d0 msgr2=0x7f8b0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.953+0000 7f8b26a00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b0006b4d0 0x7f8b0006d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009b40 tx=0x7f8b14008040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.953+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 msgr2=0x7f8b201e5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.953+0000 7f8b26a00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 0x7f8b201e5590 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8b200cd2b0 tx=0x7f8b1800d9f0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 shutdown_connections 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b201e5100 0x7f8b201e5590 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8b2013cea0 0x7f8b200b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8b0006b4d0 0x7f8b0006d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 --2- 172.21.15.5:0/2993581473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b08004800 0x7f8b2013bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 >> 172.21.15.5:0/2993581473 conn(0x7f8b20066f20 msgr2=0x7f8b2015f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 shutdown_connections 2024-04-25T10:06:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:53.954+0000 7f8b26a00640 1 -- 172.21.15.5:0/2993581473 wait complete. 2024-04-25T10:06:53.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:54.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.181+0000 7f6d7a200640 1 Processor -- start 2024-04-25T10:06:54.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.181+0000 7f6d7a200640 1 -- start start 2024-04-25T10:06:54.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d7a200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d741529e0 0x7f6d740c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d7a200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6d740cea60 0x7f6d7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d7a200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7415bdc0 con 0x7f6d740c6410 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d7a200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d740b2340 con 0x7f6d741529e0 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d7a200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7415c060 con 0x7f6d740cea60 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d73400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d741529e0 0x7f6d740c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d73400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6d741529e0 0x7f6d740c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43232/0 (socket says 172.21.15.5:43232) 2024-04-25T10:06:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d73400640 1 -- 172.21.15.5:0/1414805898 learned_addr learned my addr 172.21.15.5:0/1414805898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.183+0000 7f6d73e00640 1 --2- 172.21.15.5:0/1414805898 >> v2:172.21.15.80:3300/0 conn(0x7f6d740cea60 0x7f6d7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.184+0000 7f6d73400640 1 -- 172.21.15.5:0/1414805898 >> v2:172.21.15.80:3300/0 conn(0x7f6d740cea60 msgr2=0x7f6d7406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.184+0000 7f6d73400640 1 --2- 172.21.15.5:0/1414805898 >> v2:172.21.15.80:3300/0 conn(0x7f6d740cea60 0x7f6d7406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.184+0000 7f6d73400640 1 -- 172.21.15.5:0/1414805898 >> v1:172.21.15.80:6789/0 conn(0x7f6d740c6410 legacy=0x7f6d740ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.184+0000 7f6d73400640 1 -- 172.21.15.5:0/1414805898 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7415c340 con 0x7f6d741529e0 2024-04-25T10:06:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.184+0000 7f6d73e00640 1 --2- 172.21.15.5:0/1414805898 >> v2:172.21.15.80:3300/0 conn(0x7f6d740cea60 0x7f6d7406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.185+0000 7f6d73400640 1 --2- 172.21.15.5:0/1414805898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d741529e0 0x7f6d740c5ed0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6d740c78c0 tx=0x7f6d6402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8568ab091fc95ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.185+0000 7f6d72000640 1 -- 172.21.15.5:0/1414805898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64037070 con 0x7f6d741529e0 2024-04-25T10:06:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.185+0000 7f6d72000640 1 -- 172.21.15.5:0/1414805898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d64032440 con 0x7f6d741529e0 2024-04-25T10:06:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.185+0000 7f6d72000640 1 -- 172.21.15.5:0/1414805898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64032810 con 0x7f6d741529e0 2024-04-25T10:06:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 -- 172.21.15.5:0/1414805898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d741529e0 msgr2=0x7f6d740c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 --2- 172.21.15.5:0/1414805898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d741529e0 0x7f6d740c5ed0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6d740c78c0 tx=0x7f6d6402a230 comp rx=0 tx=0).stop 2024-04-25T10:06:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 -- 172.21.15.5:0/1414805898 shutdown_connections 2024-04-25T10:06:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 --2- 172.21.15.5:0/1414805898 >> v2:172.21.15.80:3300/0 conn(0x7f6d740cea60 0x7f6d7406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 --2- 172.21.15.5:0/1414805898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d741529e0 0x7f6d740c5ed0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 -- 172.21.15.5:0/1414805898 >> 172.21.15.5:0/1414805898 conn(0x7f6d74071150 msgr2=0x7f6d74071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 -- 172.21.15.5:0/1414805898 shutdown_connections 2024-04-25T10:06:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.186+0000 7f6d7a200640 1 -- 172.21.15.5:0/1414805898 wait complete. 2024-04-25T10:06:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.188+0000 7f6d7a200640 1 Processor -- start 2024-04-25T10:06:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.188+0000 7f6d7a200640 1 -- start start 2024-04-25T10:06:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d7a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d740c6410 0x7f6d740b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d7a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 0x7f6d741f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d7a200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d740cdec0 0x7f6d740ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d7a200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74071ed0 con 0x7f6d740cea60 2024-04-25T10:06:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d7a200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7415c900 con 0x7f6d740cdec0 2024-04-25T10:06:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d7a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74072050 con 0x7f6d740c6410 2024-04-25T10:06:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 0x7f6d741f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 0x7f6d741f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43242/0 (socket says 172.21.15.5:43242) 2024-04-25T10:06:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d72a00640 1 -- 172.21.15.5:0/3963546348 learned_addr learned my addr 172.21.15.5:0/3963546348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.189+0000 7f6d73e00640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d740cdec0 0x7f6d740ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d73400640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d740c6410 0x7f6d740b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d72a00640 1 -- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d740c6410 msgr2=0x7f6d740b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d72a00640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d740c6410 0x7f6d740b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d72a00640 1 -- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d740cdec0 msgr2=0x7f6d740ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d72a00640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d740cdec0 0x7f6d740ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d72a00640 1 -- 172.21.15.5:0/3963546348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d740b33f0 con 0x7f6d740cea60 2024-04-25T10:06:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d73e00640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d740cdec0 0x7f6d740ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d72a00640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 0x7f6d741f31c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000ba50 tx=0x7f6d6000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60019070 con 0x7f6d740cea60 2024-04-25T10:06:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6000cbb0 con 0x7f6d740cea60 2024-04-25T10:06:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d740b36d0 con 0x7f6d740cea60 2024-04-25T10:06:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60014440 con 0x7f6d740cea60 2024-04-25T10:06:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.190+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d740b3c90 con 0x7f6d740cea60 2024-04-25T10:06:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.193+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d740b3e90 con 0x7f6d740cea60 2024-04-25T10:06:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.193+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d60007370 con 0x7f6d740cea60 2024-04-25T10:06:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.194+0000 7f6d6b400640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6d5406b500 0x7f6d5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.194+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f6d6008a0b0 con 0x7f6d740cea60 2024-04-25T10:06:54.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.194+0000 7f6d73400640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6d5406b500 0x7f6d5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.195+0000 7f6d73400640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6d5406b500 0x7f6d5406d9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d64034000 tx=0x7f6d6402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.198+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d60058cf0 con 0x7f6d740cea60 2024-04-25T10:06:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.437+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d74068fd0 con 0x7f6d740cea60 2024-04-25T10:06:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.439+0000 7f6d6b400640 1 -- 172.21.15.5:0/3963546348 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f6d600585f0 con 0x7f6d740cea60 2024-04-25T10:06:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:54.445 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:32.782629+0000","modified":"2024-04-25T10:06:21.391577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6255":{"gid":6255,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.5:6835/1822355874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":1822355874},{"type":"v1","addr":"172.21.15.5:6835","nonce":1822355874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-04-25T10:06:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6d5406b500 msgr2=0x7f6d5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6d5406b500 0x7f6d5406d9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6d64034000 tx=0x7f6d6402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 msgr2=0x7f6d741f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 0x7f6d741f31c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000ba50 tx=0x7f6d6000bf10 comp rx=0 tx=0).stop 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 shutdown_connections 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6d5406b500 0x7f6d5406d9c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d740cdec0 0x7f6d740ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d740cea60 0x7f6d741f31c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 --2- 172.21.15.5:0/3963546348 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6d740c6410 0x7f6d740b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 >> 172.21.15.5:0/3963546348 conn(0x7f6d74071150 msgr2=0x7f6d74156030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 shutdown_connections 2024-04-25T10:06:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.441+0000 7f6d7a200640 1 -- 172.21.15.5:0/3963546348 wait complete. 2024-04-25T10:06:54.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:54.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:54.521 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:54.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.661+0000 7f9549400640 1 Processor -- start 2024-04-25T10:06:54.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.661+0000 7f9549400640 1 -- start start 2024-04-25T10:06:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9549400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9544141da0 0x7f954406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9549400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9544071d30 con 0x7f954406e640 2024-04-25T10:06:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9549400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954406d620 con 0x7f954415d510 2024-04-25T10:06:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9549400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9544142180 con 0x7f9544141da0 2024-04-25T10:06:54.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9543400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f954406e640 0x7f954415ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49138/0 (socket says 172.21.15.5:49138) 2024-04-25T10:06:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9543400640 1 -- 172.21.15.5:0/132114511 learned_addr learned my addr 172.21.15.5:0/132114511 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.663+0000 7f9543e00640 1 --2- 172.21.15.5:0/132114511 >> v2:172.21.15.5:3300/0 conn(0x7f9544141da0 0x7f954406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.664+0000 7f9543e00640 1 -- 172.21.15.5:0/132114511 >> v1:172.21.15.80:6789/0 conn(0x7f954415d510 legacy=0x7f95440cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.664+0000 7f9543e00640 1 -- 172.21.15.5:0/132114511 >> v1:172.21.15.5:6789/0 conn(0x7f954406e640 legacy=0x7f954415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.664+0000 7f9543e00640 1 -- 172.21.15.5:0/132114511 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954406d900 con 0x7f9544141da0 2024-04-25T10:06:54.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.664+0000 7f9542a00640 1 -- 172.21.15.5:0/132114511 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734768284 0 0) 0x7f9544071d30 con 0x7f954406e640 2024-04-25T10:06:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.665+0000 7f9543e00640 1 --2- 172.21.15.5:0/132114511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9544141da0 0x7f954406e100 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f95340097a0 tx=0x7f953402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a379ae32856a3d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.665+0000 7f9542a00640 1 -- 172.21.15.5:0/132114511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534037070 con 0x7f9544141da0 2024-04-25T10:06:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.665+0000 7f9542a00640 1 -- 172.21.15.5:0/132114511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534032440 con 0x7f9544141da0 2024-04-25T10:06:54.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.665+0000 7f9542a00640 1 -- 172.21.15.5:0/132114511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534032810 con 0x7f9544141da0 2024-04-25T10:06:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 -- 172.21.15.5:0/132114511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9544141da0 msgr2=0x7f954406e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 --2- 172.21.15.5:0/132114511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9544141da0 0x7f954406e100 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f95340097a0 tx=0x7f953402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 -- 172.21.15.5:0/132114511 shutdown_connections 2024-04-25T10:06:54.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 --2- 172.21.15.5:0/132114511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9544141da0 0x7f954406e100 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 -- 172.21.15.5:0/132114511 >> 172.21.15.5:0/132114511 conn(0x7f9544066f20 msgr2=0x7f954406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 -- 172.21.15.5:0/132114511 shutdown_connections 2024-04-25T10:06:54.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.666+0000 7f9549400640 1 -- 172.21.15.5:0/132114511 wait complete. 2024-04-25T10:06:54.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.668+0000 7f9549400640 1 Processor -- start 2024-04-25T10:06:54.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.668+0000 7f9549400640 1 -- start start 2024-04-25T10:06:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.669+0000 7f9549400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f954406e640 0x7f95441560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.669+0000 7f9549400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9544141da0 0x7f95441584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.669+0000 7f9549400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 0x7f9544159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.669+0000 7f9549400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cdb90 con 0x7f95441f5540 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.669+0000 7f9549400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cda10 con 0x7f954406e640 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.669+0000 7f9549400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9544071000 con 0x7f9544141da0 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 0x7f9544159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 0x7f9544159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43248/0 (socket says 172.21.15.5:43248) 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 -- 172.21.15.5:0/680139058 learned_addr learned my addr 172.21.15.5:0/680139058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9543e00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f954406e640 0x7f95441560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9543400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9544141da0 0x7f95441584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 -- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9544141da0 msgr2=0x7f95441584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9544141da0 0x7f95441584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 -- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f954406e640 msgr2=0x7f95441560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f954406e640 0x7f95441560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.670+0000 7f9548a00640 1 -- 172.21.15.5:0/680139058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95441567b0 con 0x7f95441f5540 2024-04-25T10:06:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.671+0000 7f9548a00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 0x7f9544159e30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f953c00be30 tx=0x7f953c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.671+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c019070 con 0x7f95441f5540 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.671+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c00caf0 con 0x7f95441f5540 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.671+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9544156a90 con 0x7f95441f5540 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.671+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c014440 con 0x7f95441f5540 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.671+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95441f5020 con 0x7f95441f5540 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.672+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9544068f60 con 0x7f95441f5540 2024-04-25T10:06:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.673+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f953c01c070 con 0x7f95441f5540 2024-04-25T10:06:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.674+0000 7f953be00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f952406b4d0 0x7f952406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.674+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f953c089fe0 con 0x7f95441f5540 2024-04-25T10:06:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.675+0000 7f9543e00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f952406b4d0 0x7f952406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.676+0000 7f9543e00640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f952406b4d0 0x7f952406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9534034000 tx=0x7f953402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.679+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f953c058bb0 con 0x7f95441f5540 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.879+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95440ccae0 con 0x7f95441f5540 2024-04-25T10:06:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.882+0000 7f953be00640 1 -- 172.21.15.5:0/680139058 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17161 (secure 0 0 0) 0x7f953c0584b0 con 0x7f95441f5540 2024-04-25T10:06:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:06:54.888 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:05:42.412821+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:30.592093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:31.804256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1d","mappings":[{"from":2,"to":1}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.31","mappings":[{"from":4,"to":5}]},{"pgid":"10.33","mappings":[{"from":2,"to":1}]},{"pgid":"10.35","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f952406b4d0 msgr2=0x7f952406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f952406b4d0 0x7f952406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9534034000 tx=0x7f953402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 msgr2=0x7f9544159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 0x7f9544159e30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f953c00be30 tx=0x7f953c00bf20 comp rx=0 tx=0).stop 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 shutdown_connections 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95441f5540 0x7f9544159e30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9544141da0 0x7f95441584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.884+0000 7f9549400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f952406b4d0 0x7f952406d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.885+0000 7f9549400640 1 --2- 172.21.15.5:0/680139058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f954406e640 0x7f95441560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.885+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 >> 172.21.15.5:0/680139058 conn(0x7f9544066f20 msgr2=0x7f954406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.885+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 shutdown_connections 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:54.885+0000 7f9549400640 1 -- 172.21.15.5:0/680139058 wait complete. 2024-04-25T10:06:54.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.162+0000 7f45dd800640 1 Processor -- start 2024-04-25T10:06:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.162+0000 7f45dd800640 1 -- start start 2024-04-25T10:06:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.164+0000 7f45dd800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45d8141da0 0x7f45d806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.164+0000 7f45dd800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d806e640 0x7f45d815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.164+0000 7f45dd800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d8071d30 con 0x7f45d815d340 2024-04-25T10:06:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.164+0000 7f45dd800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d8142180 con 0x7f45d8141da0 2024-04-25T10:06:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.164+0000 7f45dd800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d806d620 con 0x7f45d806e640 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.165+0000 7f45d7e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45d8141da0 0x7f45d806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.165+0000 7f45d7400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d806e640 0x7f45d815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.165+0000 7f45d7e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45d8141da0 0x7f45d806e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:40608/0 (socket says 172.21.15.5:40608) 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.165+0000 7f45d7e00640 1 -- 172.21.15.5:0/3363140190 learned_addr learned my addr 172.21.15.5:0/3363140190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.166+0000 7f45d7e00640 1 -- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d806e640 msgr2=0x7f45d815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.166+0000 7f45d7e00640 1 --2- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d806e640 0x7f45d815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.166+0000 7f45d7e00640 1 -- 172.21.15.5:0/3363140190 >> v1:172.21.15.80:6789/0 conn(0x7f45d815d340 legacy=0x7f45d80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.166+0000 7f45d7e00640 1 -- 172.21.15.5:0/3363140190 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d806d900 con 0x7f45d8141da0 2024-04-25T10:06:55.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.166+0000 7f45d7400640 1 --2- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d806e640 0x7f45d815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.167+0000 7f45d7e00640 1 --2- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d8141da0 0x7f45d806e100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009870 tx=0x7f45c802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9fb6925e981ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45d6a00640 1 -- 172.21.15.5:0/3363140190 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8037070 con 0x7f45d8141da0 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45d6a00640 1 -- 172.21.15.5:0/3363140190 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8032440 con 0x7f45d8141da0 2024-04-25T10:06:55.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45d6a00640 1 -- 172.21.15.5:0/3363140190 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8032810 con 0x7f45d8141da0 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 -- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d8141da0 msgr2=0x7f45d806e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 --2- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d8141da0 0x7f45d806e100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009870 tx=0x7f45c802a250 comp rx=0 tx=0).stop 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 -- 172.21.15.5:0/3363140190 shutdown_connections 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 --2- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d806e640 0x7f45d815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 --2- 172.21.15.5:0/3363140190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d8141da0 0x7f45d806e100 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 -- 172.21.15.5:0/3363140190 >> 172.21.15.5:0/3363140190 conn(0x7f45d8066f20 msgr2=0x7f45d806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 -- 172.21.15.5:0/3363140190 shutdown_connections 2024-04-25T10:06:55.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.168+0000 7f45dd800640 1 -- 172.21.15.5:0/3363140190 wait complete. 2024-04-25T10:06:55.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.170+0000 7f45dd800640 1 Processor -- start 2024-04-25T10:06:55.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.170+0000 7f45dd800640 1 -- start start 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dd800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d806e640 0x7f45d815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dd800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d8141da0 0x7f45d81e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 0x7f45d81e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dd800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80cdb90 con 0x7f45d815d340 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dd800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80cda10 con 0x7f45d806e640 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dd800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8071000 con 0x7f45d8141da0 2024-04-25T10:06:55.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.171+0000 7f45dce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 0x7f45d81e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 0x7f45d81e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43258/0 (socket says 172.21.15.5:43258) 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 -- 172.21.15.5:0/3832724453 learned_addr learned my addr 172.21.15.5:0/3832724453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45d7e00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d806e640 0x7f45d815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45d7400640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d8141da0 0x7f45d81e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 -- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d8141da0 msgr2=0x7f45d81e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d8141da0 0x7f45d81e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 -- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d806e640 msgr2=0x7f45d815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d806e640 0x7f45d815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 -- 172.21.15.5:0/3832724453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d81e8a00 con 0x7f45d815d340 2024-04-25T10:06:55.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.172+0000 7f45dce00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 0x7f45d81e81b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f45d000c730 tx=0x7f45d000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.173+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0016020 con 0x7f45d815d340 2024-04-25T10:06:55.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.173+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0006d50 con 0x7f45d815d340 2024-04-25T10:06:55.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.173+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d81e8ce0 con 0x7f45d815d340 2024-04-25T10:06:55.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.173+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003bb0 con 0x7f45d815d340 2024-04-25T10:06:55.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.173+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d81f39d0 con 0x7f45d815d340 2024-04-25T10:06:55.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.174+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d80ccae0 con 0x7f45d815d340 2024-04-25T10:06:55.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.175+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45d00050d0 con 0x7f45d815d340 2024-04-25T10:06:55.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.176+0000 7f45cfe00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.176+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f45d008a030 con 0x7f45d815d340 2024-04-25T10:06:55.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.177+0000 7f45d7e00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.178+0000 7f45d7e00640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f45c8034000 tx=0x7f45c802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.180+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45d0020020 con 0x7f45d815d340 2024-04-25T10:06:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.420+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f45d8068670 con 0x7f45d815d340 2024-04-25T10:06:55.427 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:06:55.422+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.426+0000 7f45cfe00640 1 -- 172.21.15.5:0/3832724453 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7f45d00079e0 con 0x7f45d815d340 2024-04-25T10:06:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 msgr2=0x7f45b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f45c8034000 tx=0x7f45c802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 msgr2=0x7f45d81e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 0x7f45d81e81b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f45d000c730 tx=0x7f45d000cb30 comp rx=0 tx=0).stop 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 shutdown_connections 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d815d340 0x7f45d81e81b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d8141da0 0x7f45d81e7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f45c8034000 tx=0x7f45c802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 --2- 172.21.15.5:0/3832724453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d806e640 0x7f45d815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.429+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 >> 172.21.15.5:0/3832724453 conn(0x7f45d8066f20 msgr2=0x7f45d80b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.430+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 shutdown_connections 2024-04-25T10:06:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.430+0000 7f45dd800640 1 -- 172.21.15.5:0/3832724453 wait complete. 2024-04-25T10:06:55.456 INFO:tasks.ceph.mds.c.smithi005.stderr: -7645> 2024-04-25T10:05:39.284+0000 7fce04600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:55.461 INFO:tasks.ceph.mds.c.smithi005.stderr: -5006> 2024-04-25T10:05:43.279+0000 7fcdff600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5186=0-4194304@1} caps={5186=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5186 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x556f993c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:06:55.463 INFO:tasks.ceph.mds.c.smithi005.stderr: -3041> 2024-04-25T10:06:15.393+0000 7fcdff600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:05:41.713288+0000 1=1+0) (iversion lock) caps={5186=pAsLsXsFs/-@2},l=5186(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x556f993c5b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:06:55.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:55.513 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:06:55.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.656+0000 7f4473200640 1 Processor -- start 2024-04-25T10:06:55.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.656+0000 7f4473200640 1 -- start start 2024-04-25T10:06:55.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.657+0000 7f4473200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44740999f0 0x7f44740b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.657+0000 7f4473200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4474070330 0x7f447415e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.657+0000 7f4473200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4474099dd0 0x7f4474161410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.657+0000 7f4473200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447415dd90 con 0x7f4474070330 2024-04-25T10:06:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.657+0000 7f4473200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740b0c10 con 0x7f4474099dd0 2024-04-25T10:06:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.657+0000 7f4473200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447406f290 con 0x7f44740999f0 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4474070330 0x7f447415e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4472800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4474099dd0 0x7f4474161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4474070330 0x7f447415e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43288/0 (socket says 172.21.15.5:43288) 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 -- 172.21.15.5:0/3201082485 learned_addr learned my addr 172.21.15.5:0/3201082485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471e00640 1 --2- 172.21.15.5:0/3201082485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44740999f0 0x7f44740b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 -- 172.21.15.5:0/3201082485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44740999f0 msgr2=0x7f44740b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 --2- 172.21.15.5:0/3201082485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44740999f0 0x7f44740b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 -- 172.21.15.5:0/3201082485 >> v2:172.21.15.80:3300/0 conn(0x7f4474099dd0 msgr2=0x7f4474161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 --2- 172.21.15.5:0/3201082485 >> v2:172.21.15.80:3300/0 conn(0x7f4474099dd0 0x7f4474161410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.658+0000 7f4471400640 1 -- 172.21.15.5:0/3201082485 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447406f570 con 0x7f4474070330 2024-04-25T10:06:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.660+0000 7f4471400640 1 --2- 172.21.15.5:0/3201082485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f447415e8d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f44740b2f20 tx=0x7f445c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d94fef54b8bc0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.660+0000 7f4470a00640 1 -- 172.21.15.5:0/3201082485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c037070 con 0x7f4474070330 2024-04-25T10:06:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.660+0000 7f4470a00640 1 -- 172.21.15.5:0/3201082485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c032440 con 0x7f4474070330 2024-04-25T10:06:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.660+0000 7f4470a00640 1 -- 172.21.15.5:0/3201082485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c032810 con 0x7f4474070330 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 -- 172.21.15.5:0/3201082485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 msgr2=0x7f447415e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 --2- 172.21.15.5:0/3201082485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f447415e8d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f44740b2f20 tx=0x7f445c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 -- 172.21.15.5:0/3201082485 shutdown_connections 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 --2- 172.21.15.5:0/3201082485 >> v2:172.21.15.80:3300/0 conn(0x7f4474099dd0 0x7f4474161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 --2- 172.21.15.5:0/3201082485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f447415e8d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 --2- 172.21.15.5:0/3201082485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44740999f0 0x7f44740b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 -- 172.21.15.5:0/3201082485 >> 172.21.15.5:0/3201082485 conn(0x7f44740b2f90 msgr2=0x7f44740b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 -- 172.21.15.5:0/3201082485 shutdown_connections 2024-04-25T10:06:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.661+0000 7f4473200640 1 -- 172.21.15.5:0/3201082485 wait complete. 2024-04-25T10:06:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.663+0000 7f4473200640 1 Processor -- start 2024-04-25T10:06:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.664+0000 7f4473200640 1 -- start start 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4473200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f44741f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4473200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44740999f0 0x7f44741f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4473200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4474099dd0 0x7f44740c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4473200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447406fe10 con 0x7f4474070330 2024-04-25T10:06:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4473200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447406fb30 con 0x7f44740999f0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4473200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447409a240 con 0x7f4474099dd0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4471e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f44741f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4471e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f44741f39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43292/0 (socket says 172.21.15.5:43292) 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.665+0000 7f4471e00640 1 -- 172.21.15.5:0/2018832150 learned_addr learned my addr 172.21.15.5:0/2018832150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471400640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44740999f0 0x7f44741f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4472800640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4474099dd0 0x7f44740c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471e00640 1 -- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4474099dd0 msgr2=0x7f44740c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471e00640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4474099dd0 0x7f44740c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471e00640 1 -- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44740999f0 msgr2=0x7f44741f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471e00640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44740999f0 0x7f44741f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471e00640 1 -- 172.21.15.5:0/2018832150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740c6a40 con 0x7f4474070330 2024-04-25T10:06:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f4471e00640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f44741f39d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4464007e40 tx=0x7f446400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464014620 con 0x7f4474070330 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4464019070 con 0x7f4474070330 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.666+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464014a00 con 0x7f4474070330 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.667+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44740c6760 con 0x7f4474070330 2024-04-25T10:06:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.667+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447415bc00 con 0x7f4474070330 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.668+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447415d540 con 0x7f4474070330 2024-04-25T10:06:55.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.669+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4464014ba0 con 0x7f4474070330 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.670+0000 7f446a000640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f445006b4d0 0x7f445006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.670+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6390+0+0 (secure 0 0 0) 0x7f446408acd0 con 0x7f4474070330 2024-04-25T10:06:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.670+0000 7f4471400640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f445006b4d0 0x7f445006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.671+0000 7f4471400640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f445006b4d0 0x7f445006d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f445c034000 tx=0x7f445c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.674+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f447415d540 con 0x7f4474070330 2024-04-25T10:06:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:55.908+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4474068f60 con 0x7f4474070330 2024-04-25T10:06:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.430+0000 7f446a000640 1 -- 172.21.15.5:0/2018832150 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4464059890 con 0x7f4474070330 2024-04-25T10:06:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f445006b4d0 msgr2=0x7f445006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f445006b4d0 0x7f445006d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f445c034000 tx=0x7f445c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 msgr2=0x7f44741f39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f44741f39d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4464007e40 tx=0x7f446400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 shutdown_connections 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4474099dd0 0x7f44740c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f445006b4d0 0x7f445006d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44740999f0 0x7f44741f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 --2- 172.21.15.5:0/2018832150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4474070330 0x7f44741f39d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.433+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 >> 172.21.15.5:0/2018832150 conn(0x7f44740b2f90 msgr2=0x7f447406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.434+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 shutdown_connections 2024-04-25T10:06:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.434+0000 7f4473200640 1 -- 172.21.15.5:0/2018832150 wait complete. 2024-04-25T10:06:56.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.717+0000 7f1470c00640 1 Processor -- start 2024-04-25T10:06:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.718+0000 7f1470c00640 1 -- start start 2024-04-25T10:06:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.719+0000 7f1470c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f146c0d0820 0x7f146c0b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.719+0000 7f1470c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c06c1f0 con 0x7f146c06bb00 2024-04-25T10:06:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.719+0000 7f1470c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c06c4d0 con 0x7f146c15db10 2024-04-25T10:06:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.719+0000 7f1470c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c06bfb0 con 0x7f146c0d0820 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.720+0000 7f146be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f146c06bb00 0x7f146c0d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:49140/0 (socket says 172.21.15.5:49140) 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.720+0000 7f146be00640 1 -- 172.21.15.5:0/2594756024 learned_addr learned my addr 172.21.15.5:0/2594756024 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.720+0000 7f146aa00640 1 --2- 172.21.15.5:0/2594756024 >> v2:172.21.15.80:3300/0 conn(0x7f146c0d0820 0x7f146c0b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.720+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999918962 0 0) 0x7f146c06c1f0 con 0x7f146c06bb00 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.720+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003520 con 0x7f146c06bb00 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845539642 0 0) 0x7f144c003520 con 0x7f146c06bb00 2024-04-25T10:06:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 >> v2:172.21.15.80:3300/0 conn(0x7f146c0d0820 msgr2=0x7f146c0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 --2- 172.21.15.5:0/2594756024 >> v2:172.21.15.80:3300/0 conn(0x7f146c0d0820 0x7f146c0b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 >> v1:172.21.15.80:6789/0 conn(0x7f146c15db10 legacy=0x7f146c0d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c06c7b0 con 0x7f146c06bb00 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1464001670 con 0x7f146c06bb00 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 >> v1:172.21.15.5:6789/0 conn(0x7f146c06bb00 legacy=0x7f146c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c003a00 0x7f144c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144c004800 0x7f144c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f144c0051c0 0x7f144c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c06bfb0 con 0x7f144c004800 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00c0c0 con 0x7f144c003a00 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144c00c2c0 con 0x7f144c0051c0 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146aa00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144c004800 0x7f144c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146b400640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c003a00 0x7f144c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.721+0000 7f146be00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f144c0051c0 0x7f144c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.722+0000 7f146aa00640 1 -- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f144c0051c0 msgr2=0x7f144c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.722+0000 7f146aa00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f144c0051c0 0x7f144c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.722+0000 7f146aa00640 1 -- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c003a00 msgr2=0x7f144c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.722+0000 7f146aa00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c003a00 0x7f144c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.722+0000 7f146aa00640 1 -- 172.21.15.5:0/2594756024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c003520 con 0x7f144c004800 2024-04-25T10:06:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.722+0000 7f146be00640 1 -- 172.21.15.5:0/2594756024 reap_dead start 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.723+0000 7f146aa00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144c004800 0x7f144c004c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1454009fc0 tx=0x7f144c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44377ada1c22d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.723+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454009c10 con 0x7f144c004800 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.723+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454036070 con 0x7f144c004800 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.723+0000 7f146a000640 1 -- 172.21.15.5:0/2594756024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454031880 con 0x7f144c004800 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 -- 172.21.15.5:0/2594756024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144c004800 msgr2=0x7f144c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144c004800 0x7f144c004c80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f1454009fc0 tx=0x7f144c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 -- 172.21.15.5:0/2594756024 shutdown_connections 2024-04-25T10:06:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 --2- 172.21.15.5:0/2594756024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144c004800 0x7f144c004c80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 -- 172.21.15.5:0/2594756024 >> 172.21.15.5:0/2594756024 conn(0x7f146c0b5210 msgr2=0x7f146c0b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 -- 172.21.15.5:0/2594756024 shutdown_connections 2024-04-25T10:06:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.724+0000 7f1470c00640 1 -- 172.21.15.5:0/2594756024 wait complete. 2024-04-25T10:06:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.726+0000 7f1470c00640 1 Processor -- start 2024-04-25T10:06:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.726+0000 7f1470c00640 1 -- start start 2024-04-25T10:06:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f1470c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c004800 0x7f146c1e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f1470c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 0x7f146c15fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f1470c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f146c160220 0x7f146c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f1470c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15e070 con 0x7f146c15e260 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f1470c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15def0 con 0x7f144c004800 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f1470c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0b6ba0 con 0x7f146c160220 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f146aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 0x7f146c15fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f146aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 0x7f146c15fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43296/0 (socket says 172.21.15.5:43296) 2024-04-25T10:06:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.727+0000 7f146aa00640 1 -- 172.21.15.5:0/2290855121 learned_addr learned my addr 172.21.15.5:0/2290855121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146be00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f146c160220 0x7f146c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146b400640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c004800 0x7f146c1e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146aa00640 1 -- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f146c160220 msgr2=0x7f146c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146aa00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f146c160220 0x7f146c1606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146aa00640 1 -- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c004800 msgr2=0x7f146c1e7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146aa00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c004800 0x7f146c1e7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146aa00640 1 -- 172.21.15.5:0/2290855121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c160be0 con 0x7f146c15e260 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146b400640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c004800 0x7f146c1e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146be00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f146c160220 0x7f146c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f146aa00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 0x7f146c15fce0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f145400e230 tx=0x7f1454009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.728+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454009c10 con 0x7f146c15e260 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.729+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454003990 con 0x7f146c15e260 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.729+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454003d60 con 0x7f146c15e260 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.729+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c151e30 con 0x7f146c15e260 2024-04-25T10:06:56.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.729+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c1e8590 con 0x7f146c15e260 2024-04-25T10:06:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.730+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0677d0 con 0x7f146c15e260 2024-04-25T10:06:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.731+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1454005d10 con 0x7f146c15e260 2024-04-25T10:06:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.732+0000 7f1463400640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f144806b4d0 0x7f144806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.732+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6390+0+0 (secure 0 0 0) 0x7f14540ab840 con 0x7f146c15e260 2024-04-25T10:06:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.733+0000 7f146b400640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f144806b4d0 0x7f144806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.734+0000 7f146b400640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f144806b4d0 0x7f144806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f145c000c00 tx=0x7f145c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.736+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f145407a520 con 0x7f146c15e260 2024-04-25T10:06:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:56.935+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f146c161220 con 0x7f146c15e260 2024-04-25T10:06:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.931+0000 7f1463400640 1 -- 172.21.15.5:0/2290855121 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1454079e20 con 0x7f146c15e260 2024-04-25T10:06:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.935+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f144806b4d0 msgr2=0x7f144806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.935+0000 7f1470c00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f144806b4d0 0x7f144806d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f145c000c00 tx=0x7f145c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.935+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 msgr2=0x7f146c15fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.935+0000 7f1470c00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 0x7f146c15fce0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f145400e230 tx=0x7f1454009750 comp rx=0 tx=0).stop 2024-04-25T10:06:57.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.936+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 shutdown_connections 2024-04-25T10:06:57.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.936+0000 7f1470c00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f146c160220 0x7f146c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.936+0000 7f1470c00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f146c15e260 0x7f146c15fce0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.936+0000 7f1470c00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f144806b4d0 0x7f144806d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.936+0000 7f1470c00640 1 --2- 172.21.15.5:0/2290855121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f144c004800 0x7f146c1e7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.936+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 >> 172.21.15.5:0/2290855121 conn(0x7f146c0b5210 msgr2=0x7f146c0b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.938+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 shutdown_connections 2024-04-25T10:06:57.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:57.938+0000 7f1470c00640 1 -- 172.21.15.5:0/2290855121 wait complete. 2024-04-25T10:06:57.969 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.181+0000 7fc087e00640 1 Processor -- start 2024-04-25T10:06:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.181+0000 7fc087e00640 1 -- start start 2024-04-25T10:06:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc088144e20 0x7fc08815c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc08815f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc088071000 0x7fc088071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc088071970 con 0x7fc088071000 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08806e850 con 0x7fc088144e20 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815fdf0 con 0x7fc08815f270 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.183+0000 7fc087400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc088071000 0x7fc088071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc088071000 0x7fc088071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43836/0 (socket says 172.21.15.5:43836) 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 -- 172.21.15.5:0/2615478414 learned_addr learned my addr 172.21.15.5:0/2615478414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc086a00640 1 --2- 172.21.15.5:0/2615478414 >> v2:172.21.15.80:3300/0 conn(0x7fc088144e20 0x7fc08815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc086000640 1 --2- 172.21.15.5:0/2615478414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc08815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 -- 172.21.15.5:0/2615478414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 msgr2=0x7fc08815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 --2- 172.21.15.5:0/2615478414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc08815f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 -- 172.21.15.5:0/2615478414 >> v2:172.21.15.80:3300/0 conn(0x7fc088144e20 msgr2=0x7fc08815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 --2- 172.21.15.5:0/2615478414 >> v2:172.21.15.80:3300/0 conn(0x7fc088144e20 0x7fc08815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.184+0000 7fc087400640 1 -- 172.21.15.5:0/2615478414 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0881600d0 con 0x7fc088071000 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.185+0000 7fc087400640 1 --2- 172.21.15.5:0/2615478414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc088071430 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc0880b49a0 tx=0x7fc08002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc733c26b11acae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc085600640 1 -- 172.21.15.5:0/2615478414 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080036070 con 0x7fc088071000 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc085600640 1 -- 172.21.15.5:0/2615478414 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080031600 con 0x7fc088071000 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc085600640 1 -- 172.21.15.5:0/2615478414 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800319d0 con 0x7fc088071000 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 -- 172.21.15.5:0/2615478414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 msgr2=0x7fc088071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 --2- 172.21.15.5:0/2615478414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc088071430 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc0880b49a0 tx=0x7fc08002a630 comp rx=0 tx=0).stop 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 -- 172.21.15.5:0/2615478414 shutdown_connections 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 --2- 172.21.15.5:0/2615478414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc088071430 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 --2- 172.21.15.5:0/2615478414 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc08815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 --2- 172.21.15.5:0/2615478414 >> v2:172.21.15.80:3300/0 conn(0x7fc088144e20 0x7fc08815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.186+0000 7fc087e00640 1 -- 172.21.15.5:0/2615478414 >> 172.21.15.5:0/2615478414 conn(0x7fc088066f20 msgr2=0x7fc08806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.187+0000 7fc087e00640 1 -- 172.21.15.5:0/2615478414 shutdown_connections 2024-04-25T10:06:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.187+0000 7fc087e00640 1 -- 172.21.15.5:0/2615478414 wait complete. 2024-04-25T10:06:58.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.188+0000 7fc087e00640 1 Processor -- start 2024-04-25T10:06:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.189+0000 7fc087e00640 1 -- start start 2024-04-25T10:06:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.189+0000 7fc087e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc0881e7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc088144e20 0x7fc0881e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc0881e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc088145200 con 0x7fc088071000 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815c070 con 0x7fc088144e20 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc088145380 con 0x7fc08815f270 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc0881e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc0881e7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43846/0 (socket says 172.21.15.5:43846) 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 -- 172.21.15.5:0/2991371622 learned_addr learned my addr 172.21.15.5:0/2991371622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086000640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc088144e20 0x7fc0881e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087400640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc0881e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 -- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 msgr2=0x7fc0881e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc0881e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 -- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc088144e20 msgr2=0x7fc0881e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc088144e20 0x7fc0881e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc086a00640 1 -- 172.21.15.5:0/2991371622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0881e7870 con 0x7fc088071000 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.190+0000 7fc087400640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc0881e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.191+0000 7fc086a00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc0881e7330 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d830 tx=0x7fc07800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.191+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078018020 con 0x7fc088071000 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.191+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0780096a0 con 0x7fc088071000 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.191+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0881e5b20 con 0x7fc088071000 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.191+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078009a70 con 0x7fc088071000 2024-04-25T10:06:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.191+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0881e60e0 con 0x7fc088071000 2024-04-25T10:06:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.193+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc088068f60 con 0x7fc088071000 2024-04-25T10:06:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.193+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc07801b020 con 0x7fc088071000 2024-04-25T10:06:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.195+0000 7fc07ea00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc06806b4d0 0x7fc06806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.195+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fc07808a840 con 0x7fc088071000 2024-04-25T10:06:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.195+0000 7fc086000640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc06806b4d0 0x7fc06806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.197+0000 7fc086000640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc06806b4d0 0x7fc06806d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc0881427f0 tx=0x7fc074008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.199+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc078022030 con 0x7fc088071000 2024-04-25T10:06:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.398+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc0880b4160 con 0x7fc088071000 2024-04-25T10:06:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.941+0000 7fc07ea00640 1 -- 172.21.15.5:0/2991371622 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc0881e7870 con 0x7fc088071000 2024-04-25T10:06:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc06806b4d0 msgr2=0x7fc06806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc06806b4d0 0x7fc06806d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc0881427f0 tx=0x7fc074008040 comp rx=0 tx=0).stop 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 msgr2=0x7fc0881e7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc0881e7330 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d830 tx=0x7fc07800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 shutdown_connections 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc08815f270 0x7fc0881e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc06806b4d0 0x7fc06806d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc088144e20 0x7fc0881e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 --2- 172.21.15.5:0/2991371622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc088071000 0x7fc0881e7330 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 >> 172.21.15.5:0/2991371622 conn(0x7fc088066f20 msgr2=0x7fc088142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 shutdown_connections 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:58.945+0000 7fc087e00640 1 -- 172.21.15.5:0/2991371622 wait complete. 2024-04-25T10:06:58.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.180+0000 7f568f200640 1 Processor -- start 2024-04-25T10:06:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.181+0000 7f568f200640 1 -- start start 2024-04-25T10:06:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.182+0000 7f568f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56880d0780 0x7f56880d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.182+0000 7f568f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56880b9130 0x7f56880d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.182+0000 7f568f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880d10a0 con 0x7f56880b9590 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.182+0000 7f568f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880b7ef0 con 0x7f56880d0780 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.182+0000 7f568f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880b81d0 con 0x7f56880b9130 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56880d0780 0x7f56880d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56880d0780 0x7f56880d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43860/0 (socket says 172.21.15.5:43860) 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 -- 172.21.15.5:0/146915557 learned_addr learned my addr 172.21.15.5:0/146915557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f567f400640 1 --2- 172.21.15.5:0/146915557 >> v2:172.21.15.80:3300/0 conn(0x7f56880b9130 0x7f56880d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 -- 172.21.15.5:0/146915557 >> v2:172.21.15.80:3300/0 conn(0x7f56880b9130 msgr2=0x7f56880d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 --2- 172.21.15.5:0/146915557 >> v2:172.21.15.80:3300/0 conn(0x7f56880b9130 0x7f56880d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 -- 172.21.15.5:0/146915557 >> v1:172.21.15.80:6789/0 conn(0x7f56880b9590 legacy=0x7f56880d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.183+0000 7f5687e00640 1 -- 172.21.15.5:0/146915557 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880b84b0 con 0x7f56880d0780 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.184+0000 7f5687e00640 1 --2- 172.21.15.5:0/146915557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56880d0b60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f56880b61b0 tx=0x7f567802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b882e72fd2272e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f5687400640 1 -- 172.21.15.5:0/146915557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678036070 con 0x7f56880d0780 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f5687400640 1 -- 172.21.15.5:0/146915557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678031600 con 0x7f56880d0780 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f5687400640 1 -- 172.21.15.5:0/146915557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780319d0 con 0x7f56880d0780 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f568f200640 1 -- 172.21.15.5:0/146915557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 msgr2=0x7f56880d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f568f200640 1 --2- 172.21.15.5:0/146915557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56880d0b60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f56880b61b0 tx=0x7f567802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f568f200640 1 -- 172.21.15.5:0/146915557 shutdown_connections 2024-04-25T10:06:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f568f200640 1 --2- 172.21.15.5:0/146915557 >> v2:172.21.15.80:3300/0 conn(0x7f56880b9130 0x7f56880d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.185+0000 7f568f200640 1 --2- 172.21.15.5:0/146915557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56880d0b60 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.186+0000 7f568f200640 1 -- 172.21.15.5:0/146915557 >> 172.21.15.5:0/146915557 conn(0x7f56880726b0 msgr2=0x7f5688072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.186+0000 7f568f200640 1 -- 172.21.15.5:0/146915557 shutdown_connections 2024-04-25T10:06:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.186+0000 7f568f200640 1 -- 172.21.15.5:0/146915557 wait complete. 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.187+0000 7f568f200640 1 Processor -- start 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.188+0000 7f568f200640 1 -- start start 2024-04-25T10:06:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.188+0000 7f568f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56880b9130 0x7f56880c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56880b9590 0x7f568815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56881f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881611b0 con 0x7f56880d0780 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880b8a70 con 0x7f56880b9590 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5688161330 con 0x7f56880b9130 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56881f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56881f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43864/0 (socket says 172.21.15.5:43864) 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 -- 172.21.15.5:0/4208162959 learned_addr learned my addr 172.21.15.5:0/4208162959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f567f400640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56880b9590 0x7f568815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f5687e00640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56880b9130 0x7f56880c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 -- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56880b9130 msgr2=0x7f56880c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56880b9130 0x7f56880c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 -- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56880b9590 msgr2=0x7f568815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56880b9590 0x7f568815ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f568d200640 1 -- 172.21.15.5:0/4208162959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568815b2c0 con 0x7f56880d0780 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.189+0000 7f567f400640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56880b9590 0x7f568815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.190+0000 7f568d200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56881f3320 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f568000c410 tx=0x7f568000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.190+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680017070 con 0x7f56880d0780 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.190+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5680007230 con 0x7f56880d0780 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.190+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56881f3b70 con 0x7f56880d0780 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.190+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680007660 con 0x7f56880d0780 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.190+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56881f4130 con 0x7f56880d0780 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.192+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688068f60 con 0x7f56880d0780 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.192+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f568001a070 con 0x7f56880d0780 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.193+0000 7f5684c00640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f566406b4d0 0x7f566406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.193+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5680089350 con 0x7f56880d0780 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.194+0000 7f5687e00640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f566406b4d0 0x7f566406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.195+0000 7f5687e00640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f566406b4d0 0x7f566406d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5678033000 tx=0x7f5678009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.206+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56800583d0 con 0x7f56880d0780 2024-04-25T10:06:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.414+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f56881f3860 con 0x7f56880d0780 2024-04-25T10:06:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.420+0000 7f5684c00640 1 -- 172.21.15.5:0/4208162959 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5680057cd0 con 0x7f56880d0780 2024-04-25T10:06:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.422+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f566406b4d0 msgr2=0x7f566406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.422+0000 7f568f200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f566406b4d0 0x7f566406d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5678033000 tx=0x7f5678009000 comp rx=0 tx=0).stop 2024-04-25T10:06:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.422+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 msgr2=0x7f56881f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.422+0000 7f568f200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56881f3320 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f568000c410 tx=0x7f568000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 shutdown_connections 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56880d0780 0x7f56881f3320 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f566406b4d0 0x7f566406d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56880b9590 0x7f568815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 --2- 172.21.15.5:0/4208162959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56880b9130 0x7f56880c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 >> 172.21.15.5:0/4208162959 conn(0x7f56880726b0 msgr2=0x7f568815e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 shutdown_connections 2024-04-25T10:06:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.423+0000 7f568f200640 1 -- 172.21.15.5:0/4208162959 wait complete. 2024-04-25T10:06:59.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:06:59.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.654+0000 7f362d000640 1 Processor -- start 2024-04-25T10:06:59.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.654+0000 7f362d000640 1 -- start start 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.655+0000 7f362d000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f362815ce00 0x7f362815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.655+0000 7f362d000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36280cd300 0x7f36280cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.655+0000 7f362d000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3628071d30 con 0x7f3628141da0 2024-04-25T10:06:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.655+0000 7f362d000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36280cdcd0 con 0x7f36280cd300 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.655+0000 7f362d000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3628142180 con 0x7f362815ce00 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3628141da0 0x7f362806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43284/0 (socket says 172.21.15.5:43284) 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627400640 1 -- 172.21.15.5:0/1930040124 learned_addr learned my addr 172.21.15.5:0/1930040124 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627e00640 1 --2- 172.21.15.5:0/1930040124 >> v2:172.21.15.5:3300/0 conn(0x7f36280cd300 0x7f36280cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3626a00640 1 --2- 172.21.15.5:0/1930040124 >> v2:172.21.15.80:3300/0 conn(0x7f362815ce00 0x7f362815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3626000640 1 -- 172.21.15.5:0/1930040124 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783203682 0 0) 0x7f3628071d30 con 0x7f3628141da0 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627e00640 1 -- 172.21.15.5:0/1930040124 >> v2:172.21.15.80:3300/0 conn(0x7f362815ce00 msgr2=0x7f362815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627e00640 1 --2- 172.21.15.5:0/1930040124 >> v2:172.21.15.80:3300/0 conn(0x7f362815ce00 0x7f362815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627e00640 1 -- 172.21.15.5:0/1930040124 >> v1:172.21.15.5:6789/0 conn(0x7f3628141da0 legacy=0x7f362806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.656+0000 7f3627e00640 1 -- 172.21.15.5:0/1930040124 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362806d620 con 0x7f36280cd300 2024-04-25T10:06:59.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.657+0000 7f3627e00640 1 --2- 172.21.15.5:0/1930040124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36280cd300 0x7f36280cd790 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f362806c800 tx=0x7f362002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154fd28e01d1de8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f3626000640 1 -- 172.21.15.5:0/1930040124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620007740 con 0x7f36280cd300 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f3626000640 1 -- 172.21.15.5:0/1930040124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36200078e0 con 0x7f36280cd300 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f3626000640 1 -- 172.21.15.5:0/1930040124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620007cb0 con 0x7f36280cd300 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 -- 172.21.15.5:0/1930040124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36280cd300 msgr2=0x7f36280cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 --2- 172.21.15.5:0/1930040124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36280cd300 0x7f36280cd790 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f362806c800 tx=0x7f362002a680 comp rx=0 tx=0).stop 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 -- 172.21.15.5:0/1930040124 shutdown_connections 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 --2- 172.21.15.5:0/1930040124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36280cd300 0x7f36280cd790 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 --2- 172.21.15.5:0/1930040124 >> v2:172.21.15.80:3300/0 conn(0x7f362815ce00 0x7f362815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 -- 172.21.15.5:0/1930040124 >> 172.21.15.5:0/1930040124 conn(0x7f3628066f20 msgr2=0x7f362806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 -- 172.21.15.5:0/1930040124 shutdown_connections 2024-04-25T10:06:59.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.658+0000 7f362d000640 1 -- 172.21.15.5:0/1930040124 wait complete. 2024-04-25T10:06:59.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.660+0000 7f362d000640 1 Processor -- start 2024-04-25T10:06:59.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.660+0000 7f362d000640 1 -- start start 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.661+0000 7f362d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36280cd300 0x7f36281318e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.661+0000 7f362d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 0x7f36281e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.661+0000 7f362d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f362815ce00 0x7f36281403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.661+0000 7f362d000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362806e810 con 0x7f3628141da0 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.661+0000 7f362d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362806dbe0 con 0x7f362815ce00 2024-04-25T10:06:59.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.661+0000 7f362d000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3628071000 con 0x7f36280cd300 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 0x7f36281e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 0x7f36281e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43882/0 (socket says 172.21.15.5:43882) 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 -- 172.21.15.5:0/2879258953 learned_addr learned my addr 172.21.15.5:0/2879258953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3627e00640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f362815ce00 0x7f36281403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3627400640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36280cd300 0x7f36281318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 -- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36280cd300 msgr2=0x7f36281318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36280cd300 0x7f36281318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 -- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f362815ce00 msgr2=0x7f36281403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f362815ce00 0x7f36281403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 -- 172.21.15.5:0/2879258953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36281e5a10 con 0x7f3628141da0 2024-04-25T10:06:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3627e00640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f362815ce00 0x7f36281403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.662+0000 7f3626a00640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 0x7f36281e5100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f361400d620 tx=0x7f361400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.663+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400b620 con 0x7f3628141da0 2024-04-25T10:06:59.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.663+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614019070 con 0x7f3628141da0 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.663+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400ba00 con 0x7f3628141da0 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.663+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36281e5cf0 con 0x7f3628141da0 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.663+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36281e62b0 con 0x7f3628141da0 2024-04-25T10:06:59.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.664+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3628068f60 con 0x7f3628141da0 2024-04-25T10:06:59.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.666+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f361400bba0 con 0x7f3628141da0 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.666+0000 7f361f400640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f360806b4d0 0x7f360806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.667+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f361408a570 con 0x7f3628141da0 2024-04-25T10:06:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.667+0000 7f3627400640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f360806b4d0 0x7f360806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.669+0000 7f3627400640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f360806b4d0 0x7f360806d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3618007fd0 tx=0x7f3618009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.671+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36140594c0 con 0x7f3628141da0 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:06:59.941+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f36280721f0 con 0x7f3628141da0 2024-04-25T10:07:00.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.947+0000 7f361f400640 1 -- 172.21.15.5:0/2879258953 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v272) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3614058dc0 con 0x7f3628141da0 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f360806b4d0 msgr2=0x7f360806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f360806b4d0 0x7f360806d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3618007fd0 tx=0x7f3618009040 comp rx=0 tx=0).stop 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 msgr2=0x7f36281e5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 0x7f36281e5100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f361400d620 tx=0x7f361400dae0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 shutdown_connections 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f360806b4d0 0x7f360806d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f362815ce00 0x7f36281403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3628141da0 0x7f36281e5100 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 --2- 172.21.15.5:0/2879258953 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36280cd300 0x7f36281318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.952+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 >> 172.21.15.5:0/2879258953 conn(0x7f3628066f20 msgr2=0x7f362806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.953+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 shutdown_connections 2024-04-25T10:07:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:00.953+0000 7f362d000640 1 -- 172.21.15.5:0/2879258953 wait complete. 2024-04-25T10:07:00.980 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:07:00.981 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:07:00.981 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:07:00.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.187+0000 7fbff3e00640 1 Processor -- start 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.187+0000 7fbff3e00640 1 -- start start 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbfec1529e0 0x7fbfec0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfec0c6410 0x7fbfec0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec071ed0 0x7fbfec0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff3e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec0b2340 con 0x7fbfec0c6410 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfec15bdc0 con 0x7fbfec1529e0 2024-04-25T10:07:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15c060 con 0x7fbfec071ed0 2024-04-25T10:07:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff0a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfec0c6410 0x7fbfec0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff0a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfec0c6410 0x7fbfec0ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43894/0 (socket says 172.21.15.5:43894) 2024-04-25T10:07:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff0a00640 1 -- 172.21.15.5:0/2984569204 learned_addr learned my addr 172.21.15.5:0/2984569204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff1400640 1 --2- 172.21.15.5:0/2984569204 >> v2:172.21.15.80:3300/0 conn(0x7fbfec1529e0 0x7fbfec0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.189+0000 7fbff1e00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec071ed0 0x7fbfec0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff0a00640 1 -- 172.21.15.5:0/2984569204 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec071ed0 msgr2=0x7fbfec0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff0a00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec071ed0 0x7fbfec0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff0a00640 1 -- 172.21.15.5:0/2984569204 >> v2:172.21.15.80:3300/0 conn(0x7fbfec1529e0 msgr2=0x7fbfec0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff0a00640 1 --2- 172.21.15.5:0/2984569204 >> v2:172.21.15.80:3300/0 conn(0x7fbfec1529e0 0x7fbfec0c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff0a00640 1 -- 172.21.15.5:0/2984569204 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec15c340 con 0x7fbfec0c6410 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff1e00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec071ed0 0x7fbfec0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.190+0000 7fbff1400640 1 --2- 172.21.15.5:0/2984569204 >> v2:172.21.15.80:3300/0 conn(0x7fbfec1529e0 0x7fbfec0c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.191+0000 7fbff0a00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec0c6410 0x7fbfec0ce350 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbfec071d60 tx=0x7fbfdc02a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92462c3c311f5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.191+0000 7fbfebe00640 1 -- 172.21.15.5:0/2984569204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc036070 con 0x7fbfec0c6410 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.191+0000 7fbfebe00640 1 -- 172.21.15.5:0/2984569204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc031600 con 0x7fbfec0c6410 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.191+0000 7fbfebe00640 1 -- 172.21.15.5:0/2984569204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0319d0 con 0x7fbfec0c6410 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 -- 172.21.15.5:0/2984569204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec0c6410 msgr2=0x7fbfec0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec0c6410 0x7fbfec0ce350 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbfec071d60 tx=0x7fbfdc02a690 comp rx=0 tx=0).stop 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 -- 172.21.15.5:0/2984569204 shutdown_connections 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec071ed0 0x7fbfec0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 --2- 172.21.15.5:0/2984569204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec0c6410 0x7fbfec0ce350 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 --2- 172.21.15.5:0/2984569204 >> v2:172.21.15.80:3300/0 conn(0x7fbfec1529e0 0x7fbfec0c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 -- 172.21.15.5:0/2984569204 >> 172.21.15.5:0/2984569204 conn(0x7fbfec071150 msgr2=0x7fbfec071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 -- 172.21.15.5:0/2984569204 shutdown_connections 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.192+0000 7fbff3e00640 1 -- 172.21.15.5:0/2984569204 wait complete. 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.194+0000 7fbff3e00640 1 Processor -- start 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.194+0000 7fbff3e00640 1 -- start start 2024-04-25T10:07:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 0x7fbfec140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec0c6410 0x7fbfec0c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfec0c9fe0 0x7fbfec0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0c4660 con 0x7fbfec071ed0 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec15c900 con 0x7fbfec0c9fe0 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfec0c47e0 con 0x7fbfec0c6410 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 0x7fbfec140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 0x7fbfec140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43908/0 (socket says 172.21.15.5:43908) 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.195+0000 7fbff1400640 1 -- 172.21.15.5:0/1053519422 learned_addr learned my addr 172.21.15.5:0/1053519422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff0a00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec0c6410 0x7fbfec0c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfec0c9fe0 0x7fbfec0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1400640 1 -- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec0c6410 msgr2=0x7fbfec0c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1400640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec0c6410 0x7fbfec0c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1400640 1 -- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfec0c9fe0 msgr2=0x7fbfec0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1400640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfec0c9fe0 0x7fbfec0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1400640 1 -- 172.21.15.5:0/1053519422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec141480 con 0x7fbfec071ed0 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfec0c9fe0 0x7fbfec0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.196+0000 7fbff1400640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 0x7fbfec140f40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c7f0 tx=0x7fbfe000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.197+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0015070 con 0x7fbfec071ed0 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.197+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe0007820 con 0x7fbfec071ed0 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.197+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0007bd0 con 0x7fbfec071ed0 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.197+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfec0ca560 con 0x7fbfec071ed0 2024-04-25T10:07:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.197+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfec1419b0 con 0x7fbfec071ed0 2024-04-25T10:07:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.198+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec068fd0 con 0x7fbfec071ed0 2024-04-25T10:07:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.199+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfe0018070 con 0x7fbfec071ed0 2024-04-25T10:07:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.200+0000 7fbfe9600640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfd006b4d0 0x7fbfd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.200+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbfe008a110 con 0x7fbfec071ed0 2024-04-25T10:07:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.201+0000 7fbff0a00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfd006b4d0 0x7fbfd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.202+0000 7fbff0a00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfd006b4d0 0x7fbfd006d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc033000 tx=0x7fbfdc009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.204+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfec141480 con 0x7fbfec071ed0 2024-04-25T10:07:01.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.405+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfec0b27d0 con 0x7fbfec071ed0 2024-04-25T10:07:01.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.407+0000 7fbfe9600640 1 -- 172.21.15.5:0/1053519422 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+13184 (secure 0 0 0) 0x7fbfe00590f0 con 0x7fbfec071ed0 2024-04-25T10:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":69,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:06:58.932561+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/1822355874":"2024-04-26T10:06:55.422356+0000","172.21.15.5:6834/1822355874":"2024-04-26T10:06:55.422356+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfd006b4d0 msgr2=0x7fbfd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfd006b4d0 0x7fbfd006d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc033000 tx=0x7fbfdc009000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 msgr2=0x7fbfec140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 0x7fbfec140f40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c7f0 tx=0x7fbfe000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 shutdown_connections 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfd006b4d0 0x7fbfd006d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfec0c9fe0 0x7fbfec0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfec0c6410 0x7fbfec0c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 --2- 172.21.15.5:0/1053519422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec071ed0 0x7fbfec140f40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 >> 172.21.15.5:0/1053519422 conn(0x7fbfec071150 msgr2=0x7fbfec0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 shutdown_connections 2024-04-25T10:07:01.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.410+0000 7fbff3e00640 1 -- 172.21.15.5:0/1053519422 wait complete. 2024-04-25T10:07:01.434 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:01.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:01.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.644+0000 7f956d000640 1 Processor -- start 2024-04-25T10:07:01.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.644+0000 7f956d000640 1 -- start start 2024-04-25T10:07:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.646+0000 7f956d000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95681448f0 0x7f95680cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.646+0000 7f956d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f956815bc30 0x7f956815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.646+0000 7f956d000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9568144cd0 con 0x7f956815fe30 2024-04-25T10:07:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.646+0000 7f956d000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956815f460 con 0x7f95681448f0 2024-04-25T10:07:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.646+0000 7f956d000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956815c440 con 0x7f956815bc30 2024-04-25T10:07:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.647+0000 7f9567400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95681448f0 0x7f95680cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.647+0000 7f9567400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95681448f0 0x7f95680cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:37726/0 (socket says 172.21.15.5:37726) 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.647+0000 7f9567400640 1 -- 172.21.15.5:0/2402785873 learned_addr learned my addr 172.21.15.5:0/2402785873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.647+0000 7f9566a00640 1 --2- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f956815bc30 0x7f956815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.648+0000 7f9567400640 1 -- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f956815bc30 msgr2=0x7f956815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.648+0000 7f9567400640 1 --2- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f956815bc30 0x7f956815f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.648+0000 7f9567400640 1 -- 172.21.15.5:0/2402785873 >> v1:172.21.15.80:6789/0 conn(0x7f956815fe30 legacy=0x7f956815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.648+0000 7f9567400640 1 -- 172.21.15.5:0/2402785873 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956815c720 con 0x7f95681448f0 2024-04-25T10:07:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.649+0000 7f9567400640 1 --2- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95681448f0 0x7f95680cdaa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9558009870 tx=0x7f955802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=106dea267e97b522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.649+0000 7f9566000640 1 -- 172.21.15.5:0/2402785873 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558037070 con 0x7f95681448f0 2024-04-25T10:07:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.649+0000 7f9566000640 1 -- 172.21.15.5:0/2402785873 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558032440 con 0x7f95681448f0 2024-04-25T10:07:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.649+0000 7f9566000640 1 -- 172.21.15.5:0/2402785873 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558032810 con 0x7f95681448f0 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 -- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95681448f0 msgr2=0x7f95680cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 --2- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95681448f0 0x7f95680cdaa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9558009870 tx=0x7f955802a250 comp rx=0 tx=0).stop 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 -- 172.21.15.5:0/2402785873 shutdown_connections 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 --2- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f956815bc30 0x7f956815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 --2- 172.21.15.5:0/2402785873 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95681448f0 0x7f95680cdaa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 -- 172.21.15.5:0/2402785873 >> 172.21.15.5:0/2402785873 conn(0x7f9568066f20 msgr2=0x7f956806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 -- 172.21.15.5:0/2402785873 shutdown_connections 2024-04-25T10:07:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.650+0000 7f956d000640 1 -- 172.21.15.5:0/2402785873 wait complete. 2024-04-25T10:07:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.653+0000 7f956d000640 1 Processor -- start 2024-04-25T10:07:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.653+0000 7f956d000640 1 -- start start 2024-04-25T10:07:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.654+0000 7f956d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95681448f0 0x7f95680b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.654+0000 7f956d000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f956815bc30 0x7f956813d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.654+0000 7f956d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 0x7f95680aa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.654+0000 7f956d000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956815c090 con 0x7f956815fe30 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.654+0000 7f956d000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956815d530 con 0x7f956815bc30 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.654+0000 7f956d000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95680b1470 con 0x7f95681448f0 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 0x7f95680aa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 0x7f95680aa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43910/0 (socket says 172.21.15.5:43910) 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 -- 172.21.15.5:0/3102259031 learned_addr learned my addr 172.21.15.5:0/3102259031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9566a00640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f956815bc30 0x7f956813d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567400640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95681448f0 0x7f95680b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 -- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95681448f0 msgr2=0x7f95680b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95681448f0 0x7f95680b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 -- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f956815bc30 msgr2=0x7f956813d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f956815bc30 0x7f956813d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.655+0000 7f9567e00640 1 -- 172.21.15.5:0/3102259031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680b3090 con 0x7f956815fe30 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.656+0000 7f9567e00640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 0x7f95680aa2d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f956000c6e0 tx=0x7f956000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.656+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560003b70 con 0x7f956815fe30 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.656+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9560003d30 con 0x7f956815fe30 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.656+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560005310 con 0x7f956815fe30 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.656+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95680b0eb0 con 0x7f956815fe30 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.656+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95680b3d90 con 0x7f956815fe30 2024-04-25T10:07:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.658+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9568068f60 con 0x7f956815fe30 2024-04-25T10:07:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.658+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95600054b0 con 0x7f956815fe30 2024-04-25T10:07:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.659+0000 7f955f400640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f954806b500 0x7f954806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.659+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9560089e70 con 0x7f956815fe30 2024-04-25T10:07:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.660+0000 7f9567400640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f954806b500 0x7f954806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.661+0000 7f9567400640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f954806b500 0x7f954806d9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9558034000 tx=0x7f955802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.664+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9560058e50 con 0x7f956815fe30 2024-04-25T10:07:01.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.863+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9568072180 con 0x7f956815fe30 2024-04-25T10:07:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.864+0000 7f955f400640 1 -- 172.21.15.5:0/3102259031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9560058750 con 0x7f956815fe30 2024-04-25T10:07:01.870 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f954806b500 msgr2=0x7f954806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f954806b500 0x7f954806d9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9558034000 tx=0x7f955802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 msgr2=0x7f95680aa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 0x7f95680aa2d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f956000c6e0 tx=0x7f956000cba0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 shutdown_connections 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f956815fe30 0x7f95680aa2d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f954806b500 0x7f954806d9c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f956815bc30 0x7f956813d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 --2- 172.21.15.5:0/3102259031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95681448f0 0x7f95680b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 >> 172.21.15.5:0/3102259031 conn(0x7f9568066f20 msgr2=0x7f95680af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.867+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 shutdown_connections 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:01.868+0000 7f956d000640 1 -- 172.21.15.5:0/3102259031 wait complete. 2024-04-25T10:07:01.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:07:02.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.100+0000 7f890a200640 1 Processor -- start 2024-04-25T10:07:02.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.100+0000 7f890a200640 1 -- start start 2024-04-25T10:07:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.101+0000 7f890a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f890415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f890a200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8904071d30 con 0x7f8904141da0 2024-04-25T10:07:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f890a200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890406d620 con 0x7f89040cd300 2024-04-25T10:07:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f890a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8904142180 con 0x7f890415ce00 2024-04-25T10:07:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f8908e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8904141da0 0x7f890406e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43288/0 (socket says 172.21.15.5:43288) 2024-04-25T10:07:02.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f8908e00640 1 -- 172.21.15.5:0/341299592 learned_addr learned my addr 172.21.15.5:0/341299592 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:02.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664562897 0 0) 0x7f8904071d30 con 0x7f8904141da0 2024-04-25T10:07:02.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e8003520 con 0x7f8904141da0 2024-04-25T10:07:02.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.102+0000 7f8903e00640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f890415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370681694 0 0) 0x7f88e8003520 con 0x7f8904141da0 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 msgr2=0x7f890415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f890415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 >> v1:172.21.15.80:6789/0 conn(0x7f89040cd300 legacy=0x7f8904071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890406d900 con 0x7f8904141da0 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f88f4001670 con 0x7f8904141da0 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 >> v1:172.21.15.5:6789/0 conn(0x7f8904141da0 legacy=0x7f890406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e80039e0 0x7f88e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e8004800 0x7f88e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f88e8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8904142180 con 0x7f88e8004800 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e800bb50 con 0x7f88e80039e0 2024-04-25T10:07:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e800bd50 con 0x7f890415ce00 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8903e00640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e8004800 0x7f88e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.103+0000 7f8908e00640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e80039e0 0x7f88e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8909800640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f88e8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8903e00640 1 -- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 msgr2=0x7f88e8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8903e00640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f88e8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8903e00640 1 -- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e80039e0 msgr2=0x7f88e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8903e00640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e80039e0 0x7f88e8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8903e00640 1 -- 172.21.15.5:0/341299592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8003520 con 0x7f88e8004800 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.104+0000 7f8909800640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f890415ce00 0x7f88e8005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.105+0000 7f8903e00640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e8004800 0x7f88e8004c80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f88f000ba30 tx=0x7f88e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ba87e9077be44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.105+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0030070 con 0x7f88e8004800 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.105+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f002ade0 con 0x7f88e8004800 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.105+0000 7f8903400640 1 -- 172.21.15.5:0/341299592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f000b600 con 0x7f88e8004800 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f890a200640 1 -- 172.21.15.5:0/341299592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e8004800 msgr2=0x7f88e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f890a200640 1 --2- 172.21.15.5:0/341299592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e8004800 0x7f88e8004c80 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f88f000ba30 tx=0x7f88e8005830 comp rx=0 tx=0).stop 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f8909800640 1 -- 172.21.15.5:0/341299592 reap_dead start 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f890a200640 1 -- 172.21.15.5:0/341299592 shutdown_connections 2024-04-25T10:07:02.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f890a200640 1 -- 172.21.15.5:0/341299592 >> 172.21.15.5:0/341299592 conn(0x7f8904066f20 msgr2=0x7f890406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f890a200640 1 -- 172.21.15.5:0/341299592 shutdown_connections 2024-04-25T10:07:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.106+0000 7f890a200640 1 -- 172.21.15.5:0/341299592 wait complete. 2024-04-25T10:07:02.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.108+0000 7f890a200640 1 Processor -- start 2024-04-25T10:07:02.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.108+0000 7f890a200640 1 -- start start 2024-04-25T10:07:02.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f890a200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e8004800 0x7f890414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f890a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 0x7f89040ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f890a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89040add40 0x7f89040ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f890a200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8904071710 con 0x7f89040b1680 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f890a200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890406e810 con 0x7f88e8004800 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f890a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8904071890 con 0x7f89040add40 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f8903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 0x7f89040ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f8903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 0x7f89040ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43934/0 (socket says 172.21.15.5:43934) 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.109+0000 7f8903e00640 1 -- 172.21.15.5:0/1432334542 learned_addr learned my addr 172.21.15.5:0/1432334542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8908e00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e8004800 0x7f890414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8909800640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89040add40 0x7f89040ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8903e00640 1 -- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89040add40 msgr2=0x7f89040ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8903e00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89040add40 0x7f89040ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8903e00640 1 -- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e8004800 msgr2=0x7f890414d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8903e00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e8004800 0x7f890414d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8903e00640 1 -- 172.21.15.5:0/1432334542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040b1de0 con 0x7f89040b1680 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8909800640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89040add40 0x7f89040ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8903e00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 0x7f89040ad800 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f88f0036040 tx=0x7f88f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0043030 con 0x7f89040b1680 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f002abe0 con 0x7f89040b1680 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.110+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f000b600 con 0x7f89040b1680 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.111+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89040aca40 con 0x7f89040b1680 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.111+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89040ad000 con 0x7f89040b1680 2024-04-25T10:07:02.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.112+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8904068f60 con 0x7f89040b1680 2024-04-25T10:07:02.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.113+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88f0046070 con 0x7f89040b1680 2024-04-25T10:07:02.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.113+0000 7f8900c00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88e406b500 0x7f88e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.114+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f88f00aaba0 con 0x7f89040b1680 2024-04-25T10:07:02.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.114+0000 7f8908e00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88e406b500 0x7f88e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.115+0000 7f8908e00640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88e406b500 0x7f88e406d9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88f4001050 tx=0x7f88f4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.118+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88f0079a60 con 0x7f89040b1680 2024-04-25T10:07:02.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.387+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f89040b1b00 con 0x7f89040b1680 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.953+0000 7f8900c00640 1 -- 172.21.15.5:0/1432334542 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v274) v1 ==== 169+0+0 (secure 0 0 0) 0x7f88f0079360 con 0x7f89040b1680 2024-04-25T10:07:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88e406b500 msgr2=0x7f88e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88e406b500 0x7f88e406d9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88f4001050 tx=0x7f88f4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 msgr2=0x7f89040ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 0x7f89040ad800 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f88f0036040 tx=0x7f88f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 shutdown_connections 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89040add40 0x7f89040ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89040b1680 0x7f89040ad800 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f88e406b500 0x7f88e406d9c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 --2- 172.21.15.5:0/1432334542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88e8004800 0x7f890414d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 >> 172.21.15.5:0/1432334542 conn(0x7f8904066f20 msgr2=0x7f890406e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 shutdown_connections 2024-04-25T10:07:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:02.957+0000 7f890a200640 1 -- 172.21.15.5:0/1432334542 wait complete. 2024-04-25T10:07:02.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:07:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.199+0000 7f9a63e00640 1 Processor -- start 2024-04-25T10:07:03.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.199+0000 7f9a63e00640 1 -- start start 2024-04-25T10:07:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.200+0000 7f9a63e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9a640cc840 0x7f9a6406e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.200+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a6406e670 0x7f9a6415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.200+0000 7f9a63e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a64072010 con 0x7f9a6415b330 2024-04-25T10:07:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.200+0000 7f9a63e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6406ead0 con 0x7f9a640cc840 2024-04-25T10:07:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.200+0000 7f9a63e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640ce570 con 0x7f9a6406e670 2024-04-25T10:07:03.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9a640cc840 0x7f9a6406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9a640cc840 0x7f9a6406e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43946/0 (socket says 172.21.15.5:43946) 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62a00640 1 -- 172.21.15.5:0/2330735748 learned_addr learned my addr 172.21.15.5:0/2330735748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62000640 1 --2- 172.21.15.5:0/2330735748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a6406e670 0x7f9a6415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62a00640 1 -- 172.21.15.5:0/2330735748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a6406e670 msgr2=0x7f9a6415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62a00640 1 --2- 172.21.15.5:0/2330735748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a6406e670 0x7f9a6415adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.201+0000 7f9a62a00640 1 -- 172.21.15.5:0/2330735748 >> v1:172.21.15.80:6789/0 conn(0x7f9a6415b330 legacy=0x7f9a6415e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:03.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.202+0000 7f9a62a00640 1 -- 172.21.15.5:0/2330735748 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640ce850 con 0x7f9a640cc840 2024-04-25T10:07:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.203+0000 7f9a62a00640 1 --2- 172.21.15.5:0/2330735748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a640cc840 0x7f9a6406e130 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009e50 tx=0x7f9a5402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891a408c10a0a4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.203+0000 7f9a61600640 1 -- 172.21.15.5:0/2330735748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54036070 con 0x7f9a640cc840 2024-04-25T10:07:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.203+0000 7f9a61600640 1 -- 172.21.15.5:0/2330735748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a54031600 con 0x7f9a640cc840 2024-04-25T10:07:03.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.203+0000 7f9a61600640 1 -- 172.21.15.5:0/2330735748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540319d0 con 0x7f9a640cc840 2024-04-25T10:07:03.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 -- 172.21.15.5:0/2330735748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a640cc840 msgr2=0x7f9a6406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 --2- 172.21.15.5:0/2330735748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a640cc840 0x7f9a6406e130 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009e50 tx=0x7f9a5402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 -- 172.21.15.5:0/2330735748 shutdown_connections 2024-04-25T10:07:03.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 --2- 172.21.15.5:0/2330735748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a6406e670 0x7f9a6415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 --2- 172.21.15.5:0/2330735748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a640cc840 0x7f9a6406e130 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 -- 172.21.15.5:0/2330735748 >> 172.21.15.5:0/2330735748 conn(0x7f9a64066f20 msgr2=0x7f9a6406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 -- 172.21.15.5:0/2330735748 shutdown_connections 2024-04-25T10:07:03.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.204+0000 7f9a63e00640 1 -- 172.21.15.5:0/2330735748 wait complete. 2024-04-25T10:07:03.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.206+0000 7f9a63e00640 1 Processor -- start 2024-04-25T10:07:03.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.206+0000 7f9a63e00640 1 -- start start 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a6406e670 0x7f9a64152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a640cc840 0x7f9a640ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 0x7f9a640b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640b2d20 con 0x7f9a6415b330 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640cee10 con 0x7f9a6406e670 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640ce1d0 con 0x7f9a640cc840 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 0x7f9a640b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 0x7f9a640b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43962/0 (socket says 172.21.15.5:43962) 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 -- 172.21.15.5:0/1185852937 learned_addr learned my addr 172.21.15.5:0/1185852937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a62000640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a640cc840 0x7f9a640ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a62a00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a6406e670 0x7f9a64152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 -- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a640cc840 msgr2=0x7f9a640ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a640cc840 0x7f9a640ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 -- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a6406e670 msgr2=0x7f9a64152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a6406e670 0x7f9a64152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.207+0000 7f9a63400640 1 -- 172.21.15.5:0/1185852937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640acf60 con 0x7f9a6415b330 2024-04-25T10:07:03.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.208+0000 7f9a63400640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 0x7f9a640b18a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b440 tx=0x7f9a5c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.208+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c015020 con 0x7f9a6415b330 2024-04-25T10:07:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.208+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c011830 con 0x7f9a6415b330 2024-04-25T10:07:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.208+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c011c20 con 0x7f9a6415b330 2024-04-25T10:07:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.208+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a640ad240 con 0x7f9a6415b330 2024-04-25T10:07:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.208+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a640ad800 con 0x7f9a6415b330 2024-04-25T10:07:03.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.210+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a64068f60 con 0x7f9a6415b330 2024-04-25T10:07:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.211+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a5c018020 con 0x7f9a6415b330 2024-04-25T10:07:03.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.211+0000 7f9a5aa00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a4406b500 0x7f9a4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.211+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9a5c089c40 con 0x7f9a6415b330 2024-04-25T10:07:03.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.212+0000 7f9a62a00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a4406b500 0x7f9a4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:03.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.213+0000 7f9a62a00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a4406b500 0x7f9a4406d9c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a54033000 tx=0x7f9a54009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:03.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.216+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a5c01f030 con 0x7f9a6415b330 2024-04-25T10:07:03.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.408+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9a6415bc40 con 0x7f9a6415b330 2024-04-25T10:07:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.416+0000 7f9a5aa00640 1 -- 172.21.15.5:0/1185852937 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9a5c058c30 con 0x7f9a6415b330 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a4406b500 msgr2=0x7f9a4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a4406b500 0x7f9a4406d9c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9a54033000 tx=0x7f9a54009000 comp rx=0 tx=0).stop 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 msgr2=0x7f9a640b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 0x7f9a640b18a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00b440 tx=0x7f9a5c00b900 comp rx=0 tx=0).stop 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 shutdown_connections 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a6415b330 0x7f9a640b18a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a640cc840 0x7f9a640ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a4406b500 0x7f9a4406d9c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 --2- 172.21.15.5:0/1185852937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a6406e670 0x7f9a64152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 >> 172.21.15.5:0/1185852937 conn(0x7f9a64066f20 msgr2=0x7f9a6415e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 shutdown_connections 2024-04-25T10:07:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:03.418+0000 7f9a63e00640 1 -- 172.21.15.5:0/1185852937 wait complete. 2024-04-25T10:07:03.443 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:03.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:03.550 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:03.662 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:03.663 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:03.663 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:03.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:03.723 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:07:03.723 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:03.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:03.778 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:03.778 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:03.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:03.840 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:07:03.840 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:03.896 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:03.896 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:07:03.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:04.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.111+0000 7f079d600640 1 Processor -- start 2024-04-25T10:07:04.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.112+0000 7f079d600640 1 -- start start 2024-04-25T10:07:04.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.113+0000 7f079d600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0798161200 0x7f07980b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.113+0000 7f079d600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07980d2380 0x7f07980d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.113+0000 7f079d600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07980cf410 con 0x7f07980d0600 2024-04-25T10:07:04.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.113+0000 7f079d600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07980d27c0 con 0x7f0798161200 2024-04-25T10:07:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.113+0000 7f079d600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07980d0a70 con 0x7f07980d2380 2024-04-25T10:07:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0798161200 0x7f07980b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0798161200 0x7f07980b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43970/0 (socket says 172.21.15.5:43970) 2024-04-25T10:07:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 -- 172.21.15.5:0/2248667713 learned_addr learned my addr 172.21.15.5:0/2248667713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797400640 1 --2- 172.21.15.5:0/2248667713 >> v2:172.21.15.80:3300/0 conn(0x7f07980d2380 0x7f07980d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 -- 172.21.15.5:0/2248667713 >> v2:172.21.15.80:3300/0 conn(0x7f07980d2380 msgr2=0x7f07980d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 --2- 172.21.15.5:0/2248667713 >> v2:172.21.15.80:3300/0 conn(0x7f07980d2380 0x7f07980d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 -- 172.21.15.5:0/2248667713 >> v1:172.21.15.80:6789/0 conn(0x7f07980d0600 legacy=0x7f07980ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.114+0000 7f0797e00640 1 -- 172.21.15.5:0/2248667713 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980cf6f0 con 0x7f0798161200 2024-04-25T10:07:04.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.115+0000 7f0797e00640 1 --2- 172.21.15.5:0/2248667713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0798161200 0x7f07980b4bb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f079808bb30 tx=0x7f078802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42e6eb83816dec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.116+0000 7f0796a00640 1 -- 172.21.15.5:0/2248667713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788036070 con 0x7f0798161200 2024-04-25T10:07:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.116+0000 7f0796a00640 1 -- 172.21.15.5:0/2248667713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788031600 con 0x7f0798161200 2024-04-25T10:07:04.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.116+0000 7f0796a00640 1 -- 172.21.15.5:0/2248667713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880319d0 con 0x7f0798161200 2024-04-25T10:07:04.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.116+0000 7f079d600640 1 -- 172.21.15.5:0/2248667713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0798161200 msgr2=0x7f07980b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.116+0000 7f079d600640 1 --2- 172.21.15.5:0/2248667713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0798161200 0x7f07980b4bb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f079808bb30 tx=0x7f078802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.117+0000 7f079d600640 1 -- 172.21.15.5:0/2248667713 shutdown_connections 2024-04-25T10:07:04.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.117+0000 7f079d600640 1 --2- 172.21.15.5:0/2248667713 >> v2:172.21.15.80:3300/0 conn(0x7f07980d2380 0x7f07980d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.117+0000 7f079d600640 1 --2- 172.21.15.5:0/2248667713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0798161200 0x7f07980b4bb0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.117+0000 7f079d600640 1 -- 172.21.15.5:0/2248667713 >> 172.21.15.5:0/2248667713 conn(0x7f079808af80 msgr2=0x7f07980d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.117+0000 7f079d600640 1 -- 172.21.15.5:0/2248667713 shutdown_connections 2024-04-25T10:07:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.117+0000 7f079d600640 1 -- 172.21.15.5:0/2248667713 wait complete. 2024-04-25T10:07:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.119+0000 7f079d600640 1 Processor -- start 2024-04-25T10:07:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.119+0000 7f079d600640 1 -- start start 2024-04-25T10:07:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f079d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 0x7f0798158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f079d600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07980d2380 0x7f079815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f079d600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f079815b7c0 0x7f07980ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f079d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07980d42b0 con 0x7f07980d0600 2024-04-25T10:07:04.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f079d600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b28b0 con 0x7f079815b7c0 2024-04-25T10:07:04.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f079d600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07980b3f90 con 0x7f07980d2380 2024-04-25T10:07:04.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f0797e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 0x7f0798158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f0797e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 0x7f0798158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43976/0 (socket says 172.21.15.5:43976) 2024-04-25T10:07:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f0797e00640 1 -- 172.21.15.5:0/3128569451 learned_addr learned my addr 172.21.15.5:0/3128569451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.120+0000 7f0797400640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07980d2380 0x7f079815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f079cc00640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f079815b7c0 0x7f07980ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f0797e00640 1 -- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07980d2380 msgr2=0x7f079815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f0797e00640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07980d2380 0x7f079815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f0797e00640 1 -- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f079815b7c0 msgr2=0x7f07980ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f0797e00640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f079815b7c0 0x7f07980ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f0797e00640 1 -- 172.21.15.5:0/3128569451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980cd3b0 con 0x7f07980d0600 2024-04-25T10:07:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f079cc00640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f079815b7c0 0x7f07980ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f0797e00640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 0x7f0798158630 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0788034040 tx=0x7f0788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788036050 con 0x7f07980d0600 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788042070 con 0x7f07980d0600 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0788009510 con 0x7f07980d0600 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.121+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079815a7a0 con 0x7f07980d0600 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.122+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079815ad60 con 0x7f07980d0600 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.123+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079815a590 con 0x7f07980d0600 2024-04-25T10:07:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.124+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0788007c50 con 0x7f07980d0600 2024-04-25T10:07:04.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.125+0000 7f078fe00640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f077406b520 0x7f077406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.125+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f07880aaea0 con 0x7f07980d0600 2024-04-25T10:07:04.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.126+0000 7f0797400640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f077406b520 0x7f077406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.127+0000 7f0797400640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f077406b520 0x7f077406d9e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0780006fd0 tx=0x7f0780008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.129+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0788079f00 con 0x7f07980d0600 2024-04-25T10:07:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.371+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0798068f60 con 0x7f07980d0600 2024-04-25T10:07:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.373+0000 7f078fe00640 1 -- 172.21.15.5:0/3128569451 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3690 (secure 0 0 0) 0x7f0788079800 con 0x7f07980d0600 2024-04-25T10:07:04.378 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:04.378 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5626,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/1419426598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":1419426598},{"type":"v1","addr":"172.21.15.80:6836","nonce":1419426598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/415948219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":415948219},{"type":"v1","addr":"172.21.15.80:6837","nonce":415948219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6252,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/2196821361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2196821361},{"type":"v1","addr":"172.21.15.5:6833","nonce":2196821361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6714,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4204570152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4204570152},{"type":"v1","addr":"172.21.15.5:6835","nonce":4204570152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[]} 2024-04-25T10:07:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 39 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f077406b520 msgr2=0x7f077406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f077406b520 0x7f077406d9e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0780006fd0 tx=0x7f0780008040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 msgr2=0x7f0798158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 0x7f0798158630 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0788034040 tx=0x7f0788004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 shutdown_connections 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f077406b520 0x7f077406d9e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f079815b7c0 0x7f07980ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07980d2380 0x7f079815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 --2- 172.21.15.5:0/3128569451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07980d0600 0x7f0798158630 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 >> 172.21.15.5:0/3128569451 conn(0x7f079808af80 msgr2=0x7f07980d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.375+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 shutdown_connections 2024-04-25T10:07:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.376+0000 7f079d600640 1 -- 172.21.15.5:0/3128569451 wait complete. 2024-04-25T10:07:04.402 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:04.403 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:07:04.403 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:04.403 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:04.403 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:04.403 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:07:04.403 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:04.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:04.404 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:04.404 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:07:04.404 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:04.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:04.404 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:04.404 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:07:04.405 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:04.405 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:04.420 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:04.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:04.425 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:04.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:04.435 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:04.435 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:04.436 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:04.436 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:04.472 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:04.472 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:04.479 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:04.479 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:04.482 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:04.482 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:04.487 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:04.488 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:04.488 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:07:04.631 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:07:04.632 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:07:04.637 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:07:04.641 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:07:04.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.668+0000 7f72c0600640 1 Processor -- start 2024-04-25T10:07:04.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.668+0000 7f72c0600640 1 -- start start 2024-04-25T10:07:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72c0600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f72b8144d80 0x7f72b806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72c0600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72b81452d0 0x7f72b806b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72c0600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b80d1450 con 0x7f72b8144940 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72c0600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b80d08b0 con 0x7f72b81452d0 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72c0600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b80cfca0 con 0x7f72b8144d80 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72b81452d0 0x7f72b806b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72b81452d0 0x7f72b806b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44004/0 (socket says 172.21.15.5:44004) 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 -- 172.21.15.5:0/2514753875 learned_addr learned my addr 172.21.15.5:0/2514753875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72bd200640 1 --2- 172.21.15.5:0/2514753875 >> v2:172.21.15.80:3300/0 conn(0x7f72b8144d80 0x7f72b806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 -- 172.21.15.5:0/2514753875 >> v2:172.21.15.80:3300/0 conn(0x7f72b8144d80 msgr2=0x7f72b806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 --2- 172.21.15.5:0/2514753875 >> v2:172.21.15.80:3300/0 conn(0x7f72b8144d80 0x7f72b806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 -- 172.21.15.5:0/2514753875 >> v1:172.21.15.80:6789/0 conn(0x7f72b8144940 legacy=0x7f72b80d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72be600640 1 -- 172.21.15.5:0/2514753875 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80cff80 con 0x7f72b81452d0 2024-04-25T10:07:04.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.670+0000 7f72bd200640 1 --2- 172.21.15.5:0/2514753875 >> v2:172.21.15.80:3300/0 conn(0x7f72b8144d80 0x7f72b806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.672+0000 7f72be600640 1 --2- 172.21.15.5:0/2514753875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b806b0a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f72b806f3c0 tx=0x7f72b002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dd844420b857f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.672+0000 7f72b7e00640 1 -- 172.21.15.5:0/2514753875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0036070 con 0x7f72b81452d0 2024-04-25T10:07:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.672+0000 7f72b7e00640 1 -- 172.21.15.5:0/2514753875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b0031600 con 0x7f72b81452d0 2024-04-25T10:07:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.672+0000 7f72b7e00640 1 -- 172.21.15.5:0/2514753875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b00319d0 con 0x7f72b81452d0 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.672+0000 7f72c0600640 1 -- 172.21.15.5:0/2514753875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 msgr2=0x7f72b806b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.672+0000 7f72c0600640 1 --2- 172.21.15.5:0/2514753875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b806b0a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f72b806f3c0 tx=0x7f72b002a730 comp rx=0 tx=0).stop 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.673+0000 7f72c0600640 1 -- 172.21.15.5:0/2514753875 shutdown_connections 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.673+0000 7f72c0600640 1 --2- 172.21.15.5:0/2514753875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b806b0a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.673+0000 7f72c0600640 1 --2- 172.21.15.5:0/2514753875 >> v2:172.21.15.80:3300/0 conn(0x7f72b8144d80 0x7f72b806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.673+0000 7f72c0600640 1 -- 172.21.15.5:0/2514753875 >> 172.21.15.5:0/2514753875 conn(0x7f72b80b2400 msgr2=0x7f72b80b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.673+0000 7f72c0600640 1 -- 172.21.15.5:0/2514753875 shutdown_connections 2024-04-25T10:07:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.673+0000 7f72c0600640 1 -- 172.21.15.5:0/2514753875 wait complete. 2024-04-25T10:07:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.674+0000 7f72c0600640 1 Processor -- start 2024-04-25T10:07:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.675+0000 7f72c0600640 1 -- start start 2024-04-25T10:07:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.675+0000 7f72c0600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f72b8144940 0x7f72b81f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.675+0000 7f72c0600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72b8144d80 0x7f72b80b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72c0600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b81f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72c0600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80cf8c0 con 0x7f72b81452d0 2024-04-25T10:07:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72c0600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80d0540 con 0x7f72b8144d80 2024-04-25T10:07:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72c0600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80cfa40 con 0x7f72b8144940 2024-04-25T10:07:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b81f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b81f0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44018/0 (socket says 172.21.15.5:44018) 2024-04-25T10:07:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 -- 172.21.15.5:0/916871113 learned_addr learned my addr 172.21.15.5:0/916871113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72bdc00640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f72b8144940 0x7f72b81f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72bd200640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72b8144d80 0x7f72b80b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 -- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f72b8144940 msgr2=0x7f72b81f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f72b8144940 0x7f72b81f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 -- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72b8144d80 msgr2=0x7f72b80b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72b8144d80 0x7f72b80b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.676+0000 7f72be600640 1 -- 172.21.15.5:0/916871113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80af8b0 con 0x7f72b81452d0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.677+0000 7f72be600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b81f0890 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009f40 tx=0x7f72b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.677+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0041020 con 0x7f72b81452d0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.677+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b00077a0 con 0x7f72b81452d0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.677+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b0036040 con 0x7f72b81452d0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.677+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b80b3f00 con 0x7f72b81452d0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.677+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b80b44c0 con 0x7f72b81452d0 2024-04-25T10:07:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.679+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b8068fd0 con 0x7f72b81452d0 2024-04-25T10:07:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.680+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72b0044020 con 0x7f72b81452d0 2024-04-25T10:07:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.681+0000 7f72b5600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.681+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f72b00ab060 con 0x7f72b81452d0 2024-04-25T10:07:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.682+0000 7f72bdc00640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.683+0000 7f72bdc00640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f729806b4d0 0x7f729806d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001780 tx=0x7f72ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.684+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72b004b030 con 0x7f72b81452d0 2024-04-25T10:07:04.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.895+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f72b815f140 con 0x7f72b81452d0 2024-04-25T10:07:04.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.978+0000 7f72b5600640 1 -- 172.21.15.5:0/916871113 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7f72b0079b90 con 0x7f72b81452d0 2024-04-25T10:07:04.984 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.982+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f729806b4d0 msgr2=0x7f729806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.982+0000 7f72c0600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f729806b4d0 0x7f729806d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001780 tx=0x7f72ac009040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.982+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 msgr2=0x7f72b81f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.982+0000 7f72c0600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b81f0890 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009f40 tx=0x7f72b0004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:04.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 shutdown_connections 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72b81452d0 0x7f72b81f0890 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f729806b4d0 0x7f729806d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72b8144d80 0x7f72b80b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 --2- 172.21.15.5:0/916871113 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f72b8144940 0x7f72b81f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 >> 172.21.15.5:0/916871113 conn(0x7f72b80b2400 msgr2=0x7f72b80afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 shutdown_connections 2024-04-25T10:07:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:04.983+0000 7f72c0600640 1 -- 172.21.15.5:0/916871113 wait complete. 2024-04-25T10:07:05.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.203+0000 7f67a8e00640 1 Processor -- start 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.203+0000 7f67a8e00640 1 -- start start 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a8e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a4143e50 0x7f67a415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a8e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67a40cc0d0 0x7f67a40ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a8e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67a40cc530 0x7f67a40cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a8e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a4144230 con 0x7f67a40cc530 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a8e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a415c1b0 con 0x7f67a40cc0d0 2024-04-25T10:07:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a8e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a415aee0 con 0x7f67a4143e50 2024-04-25T10:07:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67a40cc530 0x7f67a40cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67a40cc530 0x7f67a40cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44030/0 (socket says 172.21.15.5:44030) 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a3e00640 1 -- 172.21.15.5:0/376015404 learned_addr learned my addr 172.21.15.5:0/376015404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a3e00640 1 -- 172.21.15.5:0/376015404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a4143e50 msgr2=0x7f67a415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.205+0000 7f67a3e00640 1 --2- 172.21.15.5:0/376015404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a4143e50 0x7f67a415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.206+0000 7f67a3e00640 1 -- 172.21.15.5:0/376015404 >> v2:172.21.15.80:3300/0 conn(0x7f67a40cc0d0 msgr2=0x7f67a40ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.206+0000 7f67a3e00640 1 --2- 172.21.15.5:0/376015404 >> v2:172.21.15.80:3300/0 conn(0x7f67a40cc0d0 0x7f67a40ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.206+0000 7f67a3e00640 1 -- 172.21.15.5:0/376015404 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a415b1c0 con 0x7f67a40cc530 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.207+0000 7f67a3e00640 1 --2- 172.21.15.5:0/376015404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a40cc530 0x7f67a40cec70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f67a4072150 tx=0x7f679c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df250ea67738e502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.207+0000 7f67a2000640 1 -- 172.21.15.5:0/376015404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c036070 con 0x7f67a40cc530 2024-04-25T10:07:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.207+0000 7f67a2000640 1 -- 172.21.15.5:0/376015404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c031600 con 0x7f67a40cc530 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.207+0000 7f67a2000640 1 -- 172.21.15.5:0/376015404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0319d0 con 0x7f67a40cc530 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 -- 172.21.15.5:0/376015404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a40cc530 msgr2=0x7f67a40cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 --2- 172.21.15.5:0/376015404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a40cc530 0x7f67a40cec70 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f67a4072150 tx=0x7f679c02a610 comp rx=0 tx=0).stop 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 -- 172.21.15.5:0/376015404 shutdown_connections 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 --2- 172.21.15.5:0/376015404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a40cc530 0x7f67a40cec70 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 --2- 172.21.15.5:0/376015404 >> v2:172.21.15.80:3300/0 conn(0x7f67a40cc0d0 0x7f67a40ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 --2- 172.21.15.5:0/376015404 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a4143e50 0x7f67a415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 -- 172.21.15.5:0/376015404 >> 172.21.15.5:0/376015404 conn(0x7f67a40b4f00 msgr2=0x7f67a406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 -- 172.21.15.5:0/376015404 shutdown_connections 2024-04-25T10:07:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.208+0000 7f67a8e00640 1 -- 172.21.15.5:0/376015404 wait complete. 2024-04-25T10:07:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.210+0000 7f67a8e00640 1 Processor -- start 2024-04-25T10:07:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.210+0000 7f67a8e00640 1 -- start start 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a8e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67a40cc0d0 0x7f67a4159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a8e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a40cc530 0x7f67a41429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 0x7f67a4142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a8e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a415aab0 con 0x7f67a4143e50 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a8e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a415b780 con 0x7f67a40cc0d0 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a8e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a415ac30 con 0x7f67a40cc530 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 0x7f67a4142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 0x7f67a4142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44032/0 (socket says 172.21.15.5:44032) 2024-04-25T10:07:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 -- 172.21.15.5:0/4104358961 learned_addr learned my addr 172.21.15.5:0/4104358961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3400640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67a40cc0d0 0x7f67a4159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a2a00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a40cc530 0x7f67a41429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 -- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a40cc530 msgr2=0x7f67a41429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a40cc530 0x7f67a41429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 -- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67a40cc0d0 msgr2=0x7f67a4159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67a40cc0d0 0x7f67a4159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.211+0000 7f67a3e00640 1 -- 172.21.15.5:0/4104358961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a413f1e0 con 0x7f67a4143e50 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f67a3400640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67a40cc0d0 0x7f67a4159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f67a3e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 0x7f67a4142ee0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f679c009f40 tx=0x7f679c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c041020 con 0x7f67a4143e50 2024-04-25T10:07:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c007780 con 0x7f67a4143e50 2024-04-25T10:07:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c036040 con 0x7f67a4143e50 2024-04-25T10:07:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a413f4c0 con 0x7f67a4143e50 2024-04-25T10:07:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.212+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a413fa80 con 0x7f67a4143e50 2024-04-25T10:07:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.214+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4068f60 con 0x7f67a4143e50 2024-04-25T10:07:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.214+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f679c044020 con 0x7f67a4143e50 2024-04-25T10:07:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.215+0000 7f679b400640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f678406b500 0x7f678406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.215+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f679c0aaed0 con 0x7f67a4143e50 2024-04-25T10:07:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.216+0000 7f67a3400640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f678406b500 0x7f678406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.217+0000 7f67a3400640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f678406b500 0x7f678406d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6794006fd0 tx=0x7f6794008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.220+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f679c04b030 con 0x7f67a4143e50 2024-04-25T10:07:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.490+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f67a413ad80 con 0x7f67a4143e50 2024-04-25T10:07:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.492+0000 7f679b400640 1 -- 172.21.15.5:0/4104358961 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f679c079de0 con 0x7f67a4143e50 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:07:05.498 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.500 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:07:05.502 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.505 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.507 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:07:05.508 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:07:05.510 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:07:05.511 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:05.512 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:07:05.513 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:07:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.494+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f678406b500 msgr2=0x7f678406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.494+0000 7f67a8e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f678406b500 0x7f678406d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6794006fd0 tx=0x7f6794008040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.494+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 msgr2=0x7f67a4142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.494+0000 7f67a8e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 0x7f67a4142ee0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f679c009f40 tx=0x7f679c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 shutdown_connections 2024-04-25T10:07:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67a4143e50 0x7f67a4142ee0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67a40cc530 0x7f67a41429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f678406b500 0x7f678406d9c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 --2- 172.21.15.5:0/4104358961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67a40cc0d0 0x7f67a4159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 >> 172.21.15.5:0/4104358961 conn(0x7f67a40b4f00 msgr2=0x7f67a415cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 shutdown_connections 2024-04-25T10:07:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.495+0000 7f67a8e00640 1 -- 172.21.15.5:0/4104358961 wait complete. 2024-04-25T10:07:05.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.716+0000 7f2fb6a00640 1 Processor -- start 2024-04-25T10:07:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.717+0000 7f2fb6a00640 1 -- start start 2024-04-25T10:07:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb00d07d0 0x7f2fb00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb6a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2fb00d2380 0x7f2fb00b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb6a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb00cf410 con 0x7f2fb0161200 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb6a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb00b28b0 con 0x7f2fb00d2380 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb6a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00cf6f0 con 0x7f2fb00d07d0 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb5600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2fb0161200 0x7f2fb00d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43310/0 (socket says 172.21.15.5:43310) 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb5600640 1 -- 172.21.15.5:0/2832235240 learned_addr learned my addr 172.21.15.5:0/2832235240 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.718+0000 7f2fb6000640 1 --2- 172.21.15.5:0/2832235240 >> v2:172.21.15.5:3300/0 conn(0x7f2fb00d2380 0x7f2fb00b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fb4c00640 1 --2- 172.21.15.5:0/2832235240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb00d07d0 0x7f2fb00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fafe00640 1 -- 172.21.15.5:0/2832235240 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203258883 0 0) 0x7f2fb00cf410 con 0x7f2fb0161200 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fafe00640 1 -- 172.21.15.5:0/2832235240 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f94003520 con 0x7f2fb0161200 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fb6000640 1 -- 172.21.15.5:0/2832235240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb00d07d0 msgr2=0x7f2fb00b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fb6000640 1 --2- 172.21.15.5:0/2832235240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb00d07d0 0x7f2fb00b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fb6000640 1 -- 172.21.15.5:0/2832235240 >> v1:172.21.15.5:6789/0 conn(0x7f2fb0161200 legacy=0x7f2fb00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.719+0000 7f2fb6000640 1 -- 172.21.15.5:0/2832235240 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00d3fd0 con 0x7f2fb00d2380 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.720+0000 7f2fb6000640 1 --2- 172.21.15.5:0/2832235240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d2380 0x7f2fb00b2f80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009d90 tx=0x7f2fa802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6304b9160a414a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.720+0000 7f2fafe00640 1 -- 172.21.15.5:0/2832235240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8007740 con 0x7f2fb00d2380 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.720+0000 7f2fafe00640 1 -- 172.21.15.5:0/2832235240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa80078e0 con 0x7f2fb00d2380 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.720+0000 7f2fafe00640 1 -- 172.21.15.5:0/2832235240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8007c90 con 0x7f2fb00d2380 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 -- 172.21.15.5:0/2832235240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d2380 msgr2=0x7f2fb00b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/2832235240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d2380 0x7f2fb00b2f80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009d90 tx=0x7f2fa802a660 comp rx=0 tx=0).stop 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 -- 172.21.15.5:0/2832235240 shutdown_connections 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/2832235240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d2380 0x7f2fb00b2f80 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/2832235240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb00d07d0 0x7f2fb00b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 -- 172.21.15.5:0/2832235240 >> 172.21.15.5:0/2832235240 conn(0x7f2fb008af80 msgr2=0x7f2fb00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 -- 172.21.15.5:0/2832235240 shutdown_connections 2024-04-25T10:07:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.721+0000 7f2fb6a00640 1 -- 172.21.15.5:0/2832235240 wait complete. 2024-04-25T10:07:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.723+0000 7f2fb6a00640 1 Processor -- start 2024-04-25T10:07:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.723+0000 7f2fb6a00640 1 -- start start 2024-04-25T10:07:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 0x7f2fb009edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fb00d2380 0x7f2fb009f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb0161200 0x7f2fb014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00b3640 con 0x7f2fb00d07d0 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb6a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00b34c0 con 0x7f2fb00d2380 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb6a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00b37c0 con 0x7f2fb0161200 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 0x7f2fb009edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 0x7f2fb009edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44056/0 (socket says 172.21.15.5:44056) 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 -- 172.21.15.5:0/1542424608 learned_addr learned my addr 172.21.15.5:0/1542424608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 -- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb0161200 msgr2=0x7f2fb014f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb0161200 0x7f2fb014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 -- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fb00d2380 msgr2=0x7f2fb009f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fb00d2380 0x7f2fb009f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.724+0000 7f2fb5600640 1 -- 172.21.15.5:0/1542424608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb014e660 con 0x7f2fb00d07d0 2024-04-25T10:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.725+0000 7f2fb5600640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 0x7f2fb009edc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c390 tx=0x7f2fa000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.725+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0017070 con 0x7f2fb00d07d0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.725+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa0005ce0 con 0x7f2fb00d07d0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.725+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00072f0 con 0x7f2fb00d07d0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.725+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb014e940 con 0x7f2fb00d07d0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.725+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb014ef00 con 0x7f2fb00d07d0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.726+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb0068f60 con 0x7f2fb00d07d0 2024-04-25T10:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.727+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fa001a070 con 0x7f2fb00d07d0 2024-04-25T10:07:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.728+0000 7f2fad600640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f9006b4d0 0x7f2f9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.728+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2fa0089480 con 0x7f2fb00d07d0 2024-04-25T10:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.729+0000 7f2fb4c00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f9006b4d0 0x7f2f9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.730+0000 7f2fb4c00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f9006b4d0 0x7f2f9006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00cdb10 tx=0x7f2f9c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.732+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fa0058420 con 0x7f2fb00d07d0 2024-04-25T10:07:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.923+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fb00b2f20 con 0x7f2fb00d07d0 2024-04-25T10:07:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.924+0000 7f2fad600640 1 -- 172.21.15.5:0/1542424608 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+13057 (secure 0 0 0) 0x7f2fa0057d20 con 0x7f2fb00d07d0 2024-04-25T10:07:05.931 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:05.931 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:04.992036+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:05.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f9006b4d0 msgr2=0x7f2f9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f9006b4d0 0x7f2f9006d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00cdb10 tx=0x7f2f9c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 msgr2=0x7f2fb009edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 0x7f2fb009edc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c390 tx=0x7f2fa000c850 comp rx=0 tx=0).stop 2024-04-25T10:07:05.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 shutdown_connections 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fb0161200 0x7f2fb014f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f9006b4d0 0x7f2f9006d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fb00d2380 0x7f2fb009f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 --2- 172.21.15.5:0/1542424608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fb00d07d0 0x7f2fb009edc0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 >> 172.21.15.5:0/1542424608 conn(0x7f2fb008af80 msgr2=0x7f2fb00b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 shutdown_connections 2024-04-25T10:07:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:05.927+0000 7f2fb6a00640 1 -- 172.21.15.5:0/1542424608 wait complete. 2024-04-25T10:07:05.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:05.954 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.159+0000 7f6feda00640 1 Processor -- start 2024-04-25T10:07:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.159+0000 7f6feda00640 1 -- start start 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.160+0000 7f6feda00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fe80b48c0 0x7f6fe815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.160+0000 7f6feda00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6fe80d3f90 0x7f6fe8161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.160+0000 7f6feda00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe81606e0 con 0x7f6fe8161bc0 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6feda00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe8161ff0 con 0x7f6fe80b48c0 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6feda00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe81609c0 con 0x7f6fe80d3f90 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fe80b48c0 0x7f6fe815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fe80b48c0 0x7f6fe815fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44058/0 (socket says 172.21.15.5:44058) 2024-04-25T10:07:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 -- 172.21.15.5:0/2947856703 learned_addr learned my addr 172.21.15.5:0/2947856703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 -- 172.21.15.5:0/2947856703 >> v2:172.21.15.80:3300/0 conn(0x7f6fe80d3f90 msgr2=0x7f6fe8161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 --2- 172.21.15.5:0/2947856703 >> v2:172.21.15.80:3300/0 conn(0x7f6fe80d3f90 0x7f6fe8161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 -- 172.21.15.5:0/2947856703 >> v1:172.21.15.80:6789/0 conn(0x7f6fe8161bc0 legacy=0x7f6fe80d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.161+0000 7f6fe7e00640 1 -- 172.21.15.5:0/2947856703 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8160ca0 con 0x7f6fe80b48c0 2024-04-25T10:07:06.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.162+0000 7f6fe7e00640 1 --2- 172.21.15.5:0/2947856703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80b48c0 0x7f6fe815fe60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80b61b0 tx=0x7f6fd802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea78db0b9f0f9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.163+0000 7f6fe6a00640 1 -- 172.21.15.5:0/2947856703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8037070 con 0x7f6fe80b48c0 2024-04-25T10:07:06.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.163+0000 7f6fe6a00640 1 -- 172.21.15.5:0/2947856703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd8032440 con 0x7f6fe80b48c0 2024-04-25T10:07:06.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.163+0000 7f6fe6a00640 1 -- 172.21.15.5:0/2947856703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8032810 con 0x7f6fe80b48c0 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.163+0000 7f6feda00640 1 -- 172.21.15.5:0/2947856703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80b48c0 msgr2=0x7f6fe815fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.163+0000 7f6feda00640 1 --2- 172.21.15.5:0/2947856703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80b48c0 0x7f6fe815fe60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80b61b0 tx=0x7f6fd802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.164+0000 7f6feda00640 1 -- 172.21.15.5:0/2947856703 shutdown_connections 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.164+0000 7f6feda00640 1 --2- 172.21.15.5:0/2947856703 >> v2:172.21.15.80:3300/0 conn(0x7f6fe80d3f90 0x7f6fe8161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.164+0000 7f6feda00640 1 --2- 172.21.15.5:0/2947856703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80b48c0 0x7f6fe815fe60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.164+0000 7f6feda00640 1 -- 172.21.15.5:0/2947856703 >> 172.21.15.5:0/2947856703 conn(0x7f6fe808af80 msgr2=0x7f6fe80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.164+0000 7f6feda00640 1 -- 172.21.15.5:0/2947856703 shutdown_connections 2024-04-25T10:07:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.164+0000 7f6feda00640 1 -- 172.21.15.5:0/2947856703 wait complete. 2024-04-25T10:07:06.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.166+0000 7f6feda00640 1 Processor -- start 2024-04-25T10:07:06.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.166+0000 7f6feda00640 1 -- start start 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6feda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fe80b48c0 0x7f6fe80b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6feda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 0x7f6fe80b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6feda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6fe8161bc0 0x7f6fe815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6feda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe80b4ca0 con 0x7f6fe80d3f90 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6feda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe8161260 con 0x7f6fe80b48c0 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6feda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe81603a0 con 0x7f6fe8161bc0 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 0x7f6fe80b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 0x7f6fe80b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44066/0 (socket says 172.21.15.5:44066) 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 -- 172.21.15.5:0/594055966 learned_addr learned my addr 172.21.15.5:0/594055966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 -- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6fe8161bc0 msgr2=0x7f6fe815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6fe8161bc0 0x7f6fe815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 -- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fe80b48c0 msgr2=0x7f6fe80b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7e00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fe80b48c0 0x7f6fe80b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fe80b48c0 0x7f6fe80b88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.168+0000 7f6fe7400640 1 -- 172.21.15.5:0/594055966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe80cbb90 con 0x7f6fe80d3f90 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.169+0000 7f6fe7400640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 0x7f6fe80b7050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000b3f0 tx=0x7f6fd000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.169+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0015030 con 0x7f6fe80d3f90 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.169+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd0005970 con 0x7f6fe80d3f90 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.169+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0005da0 con 0x7f6fe80d3f90 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.169+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe80cbe70 con 0x7f6fe80d3f90 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.169+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe80cc430 con 0x7f6fe80d3f90 2024-04-25T10:07:06.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.171+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe8068f60 con 0x7f6fe80d3f90 2024-04-25T10:07:06.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.172+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fd0018070 con 0x7f6fe80d3f90 2024-04-25T10:07:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.173+0000 7f6fdfe00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6fc406b4d0 0x7f6fc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.173+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6fd0089b70 con 0x7f6fe80d3f90 2024-04-25T10:07:06.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.173+0000 7f6fe7e00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6fc406b4d0 0x7f6fc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.175+0000 7f6fe7e00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6fc406b4d0 0x7f6fc406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8034000 tx=0x7f6fd802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.177+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fd0058b10 con 0x7f6fe80d3f90 2024-04-25T10:07:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.377+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fe80d0dd0 con 0x7f6fe80d3f90 2024-04-25T10:07:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.379+0000 7f6fdfe00640 1 -- 172.21.15.5:0/594055966 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fd0058410 con 0x7f6fe80d3f90 2024-04-25T10:07:06.384 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:07:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6fc406b4d0 msgr2=0x7f6fc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6fc406b4d0 0x7f6fc406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8034000 tx=0x7f6fd802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 msgr2=0x7f6fe80b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 0x7f6fe80b7050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000b3f0 tx=0x7f6fd000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 shutdown_connections 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6fe8161bc0 0x7f6fe815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fe80d3f90 0x7f6fe80b7050 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6fc406b4d0 0x7f6fc406d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 --2- 172.21.15.5:0/594055966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fe80b48c0 0x7f6fe80b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 >> 172.21.15.5:0/594055966 conn(0x7f6fe808af80 msgr2=0x7f6fe80d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 shutdown_connections 2024-04-25T10:07:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.382+0000 7f6feda00640 1 -- 172.21.15.5:0/594055966 wait complete. 2024-04-25T10:07:06.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:07:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.616+0000 7eff3b400640 1 Processor -- start 2024-04-25T10:07:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.617+0000 7eff3b400640 1 -- start start 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.618+0000 7eff3b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7eff3406c2d0 0x7eff340d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.618+0000 7eff3b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff340b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.618+0000 7eff3b400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340ced20 con 0x7eff340ce840 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.618+0000 7eff3b400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff340b72a0 con 0x7eff3406c2d0 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.618+0000 7eff3b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340cf000 con 0x7eff340b7730 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.618+0000 7eff2be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7eff340ce840 0x7eff3406c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43314/0 (socket says 172.21.15.5:43314) 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff2be00640 1 -- 172.21.15.5:0/1541642711 learned_addr learned my addr 172.21.15.5:0/1541642711 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff38a00640 1 --2- 172.21.15.5:0/1541642711 >> v2:172.21.15.5:3300/0 conn(0x7eff3406c2d0 0x7eff340d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff39400640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff340b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff33e00640 1 -- 172.21.15.5:0/1541642711 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356136623 0 0) 0x7eff340ced20 con 0x7eff340ce840 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff33e00640 1 -- 172.21.15.5:0/1541642711 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff18003520 con 0x7eff340ce840 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff38a00640 1 -- 172.21.15.5:0/1541642711 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 msgr2=0x7eff340b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff38a00640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff340b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff38a00640 1 -- 172.21.15.5:0/1541642711 >> v1:172.21.15.5:6789/0 conn(0x7eff340ce840 legacy=0x7eff3406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff38a00640 1 -- 172.21.15.5:0/1541642711 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340cf2e0 con 0x7eff3406c2d0 2024-04-25T10:07:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.619+0000 7eff39400640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff340b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:06.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.620+0000 7eff38a00640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff340d32c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7eff240097a0 tx=0x7eff2402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6411e3ab14adff52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.621+0000 7eff33e00640 1 -- 172.21.15.5:0/1541642711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24037070 con 0x7eff3406c2d0 2024-04-25T10:07:06.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.621+0000 7eff33e00640 1 -- 172.21.15.5:0/1541642711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24032440 con 0x7eff3406c2d0 2024-04-25T10:07:06.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.621+0000 7eff33e00640 1 -- 172.21.15.5:0/1541642711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240327f0 con 0x7eff3406c2d0 2024-04-25T10:07:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.621+0000 7eff3b400640 1 -- 172.21.15.5:0/1541642711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 msgr2=0x7eff340d32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.621+0000 7eff3b400640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff340d32c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7eff240097a0 tx=0x7eff2402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.622+0000 7eff3b400640 1 -- 172.21.15.5:0/1541642711 shutdown_connections 2024-04-25T10:07:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.622+0000 7eff3b400640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff340b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.622+0000 7eff3b400640 1 --2- 172.21.15.5:0/1541642711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff340d32c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.622+0000 7eff3b400640 1 -- 172.21.15.5:0/1541642711 >> 172.21.15.5:0/1541642711 conn(0x7eff340d0580 msgr2=0x7eff340d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.622+0000 7eff3b400640 1 -- 172.21.15.5:0/1541642711 shutdown_connections 2024-04-25T10:07:06.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.622+0000 7eff3b400640 1 -- 172.21.15.5:0/1541642711 wait complete. 2024-04-25T10:07:06.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.623+0000 7eff3b400640 1 Processor -- start 2024-04-25T10:07:06.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.624+0000 7eff3b400640 1 -- start start 2024-04-25T10:07:06.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.624+0000 7eff3b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff341f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff3b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff3414b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff3b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff340ce840 0x7eff3414ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff3b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b6fb0 con 0x7eff3406c2d0 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff3b400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b6e30 con 0x7eff340ce840 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff3b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b8120 con 0x7eff340b7730 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff341f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff341f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44086/0 (socket says 172.21.15.5:44086) 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 -- 172.21.15.5:0/1715251612 learned_addr learned my addr 172.21.15.5:0/1715251612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 -- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 msgr2=0x7eff3414b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff3414b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 -- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff340ce840 msgr2=0x7eff3414ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff340ce840 0x7eff3414ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.625+0000 7eff38a00640 1 -- 172.21.15.5:0/1715251612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff341560d0 con 0x7eff3406c2d0 2024-04-25T10:07:06.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.626+0000 7eff38a00640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff341f34d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7eff24035040 tx=0x7eff240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.626+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24041020 con 0x7eff3406c2d0 2024-04-25T10:07:06.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.626+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24030700 con 0x7eff3406c2d0 2024-04-25T10:07:06.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.626+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff340c8e60 con 0x7eff3406c2d0 2024-04-25T10:07:06.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.626+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24037040 con 0x7eff3406c2d0 2024-04-25T10:07:06.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.626+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff34155b60 con 0x7eff3406c2d0 2024-04-25T10:07:06.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.628+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff34074bb0 con 0x7eff3406c2d0 2024-04-25T10:07:06.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.628+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff24044020 con 0x7eff3406c2d0 2024-04-25T10:07:06.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.629+0000 7eff31600640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7eff1006b4d0 0x7eff1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.629+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5249+0+0 (secure 0 0 0) 0x7eff240aa7e0 con 0x7eff3406c2d0 2024-04-25T10:07:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.630+0000 7eff2be00640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7eff1006b4d0 0x7eff1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.631+0000 7eff2be00640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7eff1006b4d0 0x7eff1006d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000f60 tx=0x7eff1c0078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.634+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff2404b030 con 0x7eff3406c2d0 2024-04-25T10:07:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.839+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7eff3406c900 con 0x7eff3406c2d0 2024-04-25T10:07:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.839+0000 7eff31600640 1 -- 172.21.15.5:0/1715251612 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7eff24079810 con 0x7eff3406c2d0 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:06.845 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7eff1006b4d0 msgr2=0x7eff1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7eff1006b4d0 0x7eff1006d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7eff1c000f60 tx=0x7eff1c0078f0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 msgr2=0x7eff341f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff341f34d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7eff24035040 tx=0x7eff240046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 shutdown_connections 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7eff1006b4d0 0x7eff1006d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff340ce840 0x7eff3414ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7eff340b7730 0x7eff3414b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 --2- 172.21.15.5:0/1715251612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff3406c2d0 0x7eff341f34d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 >> 172.21.15.5:0/1715251612 conn(0x7eff340d0580 msgr2=0x7eff340b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.842+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 shutdown_connections 2024-04-25T10:07:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:06.843+0000 7eff3b400640 1 -- 172.21.15.5:0/1715251612 wait complete. 2024-04-25T10:07:06.869 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:07:06.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:07:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.085+0000 7f3a40800640 1 Processor -- start 2024-04-25T10:07:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.085+0000 7f3a40800640 1 -- start start 2024-04-25T10:07:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a40800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a3815de70 0x7f3a380d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a40800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a380d3330 0x7f3a380d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a40800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380b9290 0x7f3a380d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a40800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380d2290 con 0x7f3a3815de70 2024-04-25T10:07:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a40800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380d2570 con 0x7f3a380d3330 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a40800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380d2850 con 0x7f3a380b9290 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a3de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a3815de70 0x7f3a380d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a3de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a3815de70 0x7f3a380d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58398/0 (socket says 172.21.15.5:58398) 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a3de00640 1 -- 172.21.15.5:0/3688934727 learned_addr learned my addr 172.21.15.5:0/3688934727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a3d400640 1 --2- 172.21.15.5:0/3688934727 >> v2:172.21.15.80:3300/0 conn(0x7f3a380d3330 0x7f3a380d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.087+0000 7f3a3e800640 1 --2- 172.21.15.5:0/3688934727 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380b9290 0x7f3a380d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.088+0000 7f3a3de00640 1 -- 172.21.15.5:0/3688934727 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380b9290 msgr2=0x7f3a380d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.088+0000 7f3a3de00640 1 --2- 172.21.15.5:0/3688934727 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380b9290 0x7f3a380d3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.088+0000 7f3a3de00640 1 -- 172.21.15.5:0/3688934727 >> v2:172.21.15.80:3300/0 conn(0x7f3a380d3330 msgr2=0x7f3a380d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.088+0000 7f3a3de00640 1 --2- 172.21.15.5:0/3688934727 >> v2:172.21.15.80:3300/0 conn(0x7f3a380d3330 0x7f3a380d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.088+0000 7f3a3de00640 1 -- 172.21.15.5:0/3688934727 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380d2b30 con 0x7f3a3815de70 2024-04-25T10:07:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.089+0000 7f3a3de00640 1 --2- 172.21.15.5:0/3688934727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3815de70 0x7f3a380d1920 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3a380d0860 tx=0x7f3a3002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4a87ce9b0c9bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.089+0000 7f3a3ca00640 1 -- 172.21.15.5:0/3688934727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30036070 con 0x7f3a3815de70 2024-04-25T10:07:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.089+0000 7f3a3ca00640 1 -- 172.21.15.5:0/3688934727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30031600 con 0x7f3a3815de70 2024-04-25T10:07:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.089+0000 7f3a3ca00640 1 -- 172.21.15.5:0/3688934727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300319d0 con 0x7f3a3815de70 2024-04-25T10:07:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 -- 172.21.15.5:0/3688934727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3815de70 msgr2=0x7f3a380d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 --2- 172.21.15.5:0/3688934727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3815de70 0x7f3a380d1920 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3a380d0860 tx=0x7f3a3002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 -- 172.21.15.5:0/3688934727 shutdown_connections 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 --2- 172.21.15.5:0/3688934727 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380b9290 0x7f3a380d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 --2- 172.21.15.5:0/3688934727 >> v2:172.21.15.80:3300/0 conn(0x7f3a380d3330 0x7f3a380d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 --2- 172.21.15.5:0/3688934727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a3815de70 0x7f3a380d1920 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 -- 172.21.15.5:0/3688934727 >> 172.21.15.5:0/3688934727 conn(0x7f3a380726b0 msgr2=0x7f3a38072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 -- 172.21.15.5:0/3688934727 shutdown_connections 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.090+0000 7f3a40800640 1 -- 172.21.15.5:0/3688934727 wait complete. 2024-04-25T10:07:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.092+0000 7f3a40800640 1 Processor -- start 2024-04-25T10:07:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.092+0000 7f3a40800640 1 -- start start 2024-04-25T10:07:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.093+0000 7f3a40800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 0x7f3a380cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.093+0000 7f3a40800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380d3330 0x7f3a38143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.093+0000 7f3a40800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3815de70 0x7f3a380c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.093+0000 7f3a40800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380d1fe0 con 0x7f3a380b9290 2024-04-25T10:07:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.093+0000 7f3a40800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380d1e60 con 0x7f3a3815de70 2024-04-25T10:07:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.093+0000 7f3a40800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380b52f0 con 0x7f3a380d3330 2024-04-25T10:07:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 0x7f3a380cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 0x7f3a380cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58406/0 (socket says 172.21.15.5:58406) 2024-04-25T10:07:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 -- 172.21.15.5:0/1086833090 learned_addr learned my addr 172.21.15.5:0/1086833090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3d400640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380d3330 0x7f3a38143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3e800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3815de70 0x7f3a380c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 -- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380d3330 msgr2=0x7f3a38143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380d3330 0x7f3a38143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 -- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3815de70 msgr2=0x7f3a380c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3815de70 0x7f3a380c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3de00640 1 -- 172.21.15.5:0/1086833090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380cc090 con 0x7f3a380b9290 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.094+0000 7f3a3d400640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380d3330 0x7f3a38143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.095+0000 7f3a3de00640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 0x7f3a380cd130 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3a30034040 tx=0x7f3a30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.095+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30036050 con 0x7f3a380b9290 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.095+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a30042070 con 0x7f3a380b9290 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.095+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30009510 con 0x7f3a380b9290 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.095+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380cc370 con 0x7f3a380b9290 2024-04-25T10:07:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.095+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380cc930 con 0x7f3a380b9290 2024-04-25T10:07:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.096+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a38069040 con 0x7f3a380b9290 2024-04-25T10:07:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.097+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a30007c30 con 0x7f3a380b9290 2024-04-25T10:07:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.098+0000 7f3a2e000640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.098+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3a300aafe0 con 0x7f3a380b9290 2024-04-25T10:07:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.099+0000 7f3a3d400640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.100+0000 7f3a3d400640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3a380cc000 tx=0x7f3a28008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.103+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a30045360 con 0x7f3a380b9290 2024-04-25T10:07:07.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:07.305+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3a38068030 con 0x7f3a380b9290 2024-04-25T10:07:08.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.008+0000 7f3a2e000640 1 -- 172.21.15.5:0/1086833090 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a30079fc0 con 0x7f3a380b9290 2024-04-25T10:07:08.014 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 msgr2=0x7f3a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3a380cc000 tx=0x7f3a28008040 comp rx=0 tx=0).stop 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 msgr2=0x7f3a380cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 0x7f3a380cd130 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f3a30034040 tx=0x7f3a30004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 shutdown_connections 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a3815de70 0x7f3a380c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a380d3330 0x7f3a38143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 --2- 172.21.15.5:0/1086833090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a380b9290 0x7f3a380cd130 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.012+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 >> 172.21.15.5:0/1086833090 conn(0x7f3a380726b0 msgr2=0x7f3a380ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.013+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 shutdown_connections 2024-04-25T10:07:08.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.013+0000 7f3a40800640 1 -- 172.21.15.5:0/1086833090 wait complete. 2024-04-25T10:07:08.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:07:08.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.248+0000 7f8295c00640 1 Processor -- start 2024-04-25T10:07:08.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.249+0000 7f8295c00640 1 -- start start 2024-04-25T10:07:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.250+0000 7f8295c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8290144fe0 0x7f829006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.250+0000 7f8295c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f829006d130 0x7f8290071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.250+0000 7f8295c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82901453c0 con 0x7f8290071670 2024-04-25T10:07:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.250+0000 7f8295c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900a8560 con 0x7f8290144fe0 2024-04-25T10:07:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.250+0000 7f8295c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829006d6a0 con 0x7f829006d130 2024-04-25T10:07:08.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.251+0000 7f828ea00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8290144fe0 0x7f829006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.251+0000 7f828ea00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8290144fe0 0x7f829006e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:58514/0 (socket says 172.21.15.5:58514) 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.251+0000 7f828ea00640 1 -- 172.21.15.5:0/2482388176 learned_addr learned my addr 172.21.15.5:0/2482388176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.251+0000 7f8287e00640 1 --2- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f829006d130 0x7f8290071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.252+0000 7f828ea00640 1 -- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f829006d130 msgr2=0x7f8290071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.252+0000 7f828ea00640 1 --2- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f829006d130 0x7f8290071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.252+0000 7f828ea00640 1 -- 172.21.15.5:0/2482388176 >> v1:172.21.15.80:6789/0 conn(0x7f8290071670 legacy=0x7f82900cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.252+0000 7f828ea00640 1 -- 172.21.15.5:0/2482388176 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829006d980 con 0x7f8290144fe0 2024-04-25T10:07:08.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.253+0000 7f828ea00640 1 --2- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f829006e560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8280009870 tx=0x7f828002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7462b200a59f7545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.254+0000 7f828e000640 1 -- 172.21.15.5:0/2482388176 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280037070 con 0x7f8290144fe0 2024-04-25T10:07:08.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.254+0000 7f828e000640 1 -- 172.21.15.5:0/2482388176 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280032440 con 0x7f8290144fe0 2024-04-25T10:07:08.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.254+0000 7f828e000640 1 -- 172.21.15.5:0/2482388176 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280032810 con 0x7f8290144fe0 2024-04-25T10:07:08.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.254+0000 7f8295c00640 1 -- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 msgr2=0x7f829006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.254+0000 7f8295c00640 1 --2- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f829006e560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8280009870 tx=0x7f828002a250 comp rx=0 tx=0).stop 2024-04-25T10:07:08.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.255+0000 7f8295c00640 1 -- 172.21.15.5:0/2482388176 shutdown_connections 2024-04-25T10:07:08.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.255+0000 7f8295c00640 1 --2- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f829006d130 0x7f8290071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.255+0000 7f8295c00640 1 --2- 172.21.15.5:0/2482388176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f829006e560 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.255+0000 7f8295c00640 1 -- 172.21.15.5:0/2482388176 >> 172.21.15.5:0/2482388176 conn(0x7f829006ab60 msgr2=0x7f829006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.255+0000 7f8295c00640 1 -- 172.21.15.5:0/2482388176 shutdown_connections 2024-04-25T10:07:08.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.255+0000 7f8295c00640 1 -- 172.21.15.5:0/2482388176 wait complete. 2024-04-25T10:07:08.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.257+0000 7f8295c00640 1 Processor -- start 2024-04-25T10:07:08.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.257+0000 7f8295c00640 1 -- start start 2024-04-25T10:07:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f8295c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 0x7f8290140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f8295c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8290071670 0x7f82900c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f8295c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f82900cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f8295c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829015be50 con 0x7f829006d130 2024-04-25T10:07:08.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f8295c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829006df40 con 0x7f8290144fe0 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f8295c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829006eaa0 con 0x7f8290071670 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f828ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 0x7f8290140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f828ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 0x7f8290140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58414/0 (socket says 172.21.15.5:58414) 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.258+0000 7f828ea00640 1 -- 172.21.15.5:0/4259679045 learned_addr learned my addr 172.21.15.5:0/4259679045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828ea00640 1 -- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8290071670 msgr2=0x7f82900c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828ea00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8290071670 0x7f82900c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828ea00640 1 -- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 msgr2=0x7f82900cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828f400640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f82900cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828ea00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f82900cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828ea00640 1 -- 172.21.15.5:0/4259679045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900c9fd0 con 0x7f829006d130 2024-04-25T10:07:08.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828f400640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f82900cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:08.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.259+0000 7f828ea00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 0x7f8290140a50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8280000c00 tx=0x7f82800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.260+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280037040 con 0x7f829006d130 2024-04-25T10:07:08.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.260+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280042070 con 0x7f829006d130 2024-04-25T10:07:08.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.260+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280030880 con 0x7f829006d130 2024-04-25T10:07:08.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.260+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82900b28f0 con 0x7f829006d130 2024-04-25T10:07:08.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.260+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82900b2eb0 con 0x7f829006d130 2024-04-25T10:07:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.261+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8290068f60 con 0x7f829006d130 2024-04-25T10:07:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.262+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8280030a20 con 0x7f829006d130 2024-04-25T10:07:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.263+0000 7f8286a00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.263+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f82800aab10 con 0x7f829006d130 2024-04-25T10:07:08.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.264+0000 7f8287e00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.265+0000 7f8287e00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f826c06b4d0 0x7f826c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8278006a90 tx=0x7f8278006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.268+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8280079940 con 0x7f829006d130 2024-04-25T10:07:08.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:08.474+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8290072340 con 0x7f829006d130 2024-04-25T10:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.014+0000 7f8286a00640 1 -- 172.21.15.5:0/4259679045 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8280079240 con 0x7f829006d130 2024-04-25T10:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f826c06b4d0 msgr2=0x7f826c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f826c06b4d0 0x7f826c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8278006a90 tx=0x7f8278006a20 comp rx=0 tx=0).stop 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 msgr2=0x7f8290140a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 0x7f8290140a50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8280000c00 tx=0x7f82800046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 shutdown_connections 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f826c06b4d0 0x7f826c06d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8290144fe0 0x7f82900cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8290071670 0x7f82900c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 --2- 172.21.15.5:0/4259679045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829006d130 0x7f8290140a50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.018+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 >> 172.21.15.5:0/4259679045 conn(0x7f829006ab60 msgr2=0x7f8290143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.019+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 shutdown_connections 2024-04-25T10:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.019+0000 7f8295c00640 1 -- 172.21.15.5:0/4259679045 wait complete. 2024-04-25T10:07:09.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:07:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.233+0000 7f82e1600640 1 Processor -- start 2024-04-25T10:07:09.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.234+0000 7f82e1600640 1 -- start start 2024-04-25T10:07:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.235+0000 7f82e1600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 0x7f82dc161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.235+0000 7f82e1600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc1606e0 con 0x7f82dc161bc0 2024-04-25T10:07:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.235+0000 7f82e1600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc1609c0 con 0x7f82dc0b48c0 2024-04-25T10:07:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.235+0000 7f82e1600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc160ca0 con 0x7f82dc0d3f90 2024-04-25T10:07:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82e0c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f82dc161bc0 0x7f82dc0b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54910/0 (socket says 172.21.15.5:54910) 2024-04-25T10:07:09.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82e0c00640 1 -- 172.21.15.5:0/2580247860 learned_addr learned my addr 172.21.15.5:0/2580247860 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82db400640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 0x7f82dc161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869377246 0 0) 0x7f82dc1606e0 con 0x7f82dc161bc0 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82bc003520 con 0x7f82dc161bc0 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216612509 0 0) 0x7f82bc003520 con 0x7f82dc161bc0 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.236+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 msgr2=0x7f82dc161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 0x7f82dc161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 >> v1:172.21.15.80:6789/0 conn(0x7f82dc0b48c0 legacy=0x7f82dc15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc160f80 con 0x7f82dc161bc0 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f82d4001670 con 0x7f82dc161bc0 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 >> v1:172.21.15.5:6789/0 conn(0x7f82dc161bc0 legacy=0x7f82dc0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 0x7f82bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc160ca0 con 0x7f82bc0039e0 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc00baf0 con 0x7f82bc004800 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc00bcf0 con 0x7f82dc0d3f90 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82e0c00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 0x7f82bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82db400640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82dbe00640 1 -- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 msgr2=0x7f82bc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.237+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0d3f90 0x7f82bc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.238+0000 7f82dbe00640 1 -- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82bc004800 msgr2=0x7f82bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.238+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.238+0000 7f82dbe00640 1 -- 172.21.15.5:0/2580247860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc003520 con 0x7f82bc0039e0 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.238+0000 7f82db400640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82dbe00640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82bc003e00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f82cc001750 tx=0x7f82bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0aded7c12f0af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc001370 con 0x7f82bc0039e0 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc0052d0 con 0x7f82bc0039e0 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82daa00640 1 -- 172.21.15.5:0/2580247860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc02abe0 con 0x7f82bc0039e0 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82e1600640 1 -- 172.21.15.5:0/2580247860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 msgr2=0x7f82bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82e1600640 1 --2- 172.21.15.5:0/2580247860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82bc003e00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f82cc001750 tx=0x7f82bc005830 comp rx=0 tx=0).stop 2024-04-25T10:07:09.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.239+0000 7f82e0c00640 1 -- 172.21.15.5:0/2580247860 reap_dead start 2024-04-25T10:07:09.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.240+0000 7f82e1600640 1 -- 172.21.15.5:0/2580247860 shutdown_connections 2024-04-25T10:07:09.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.240+0000 7f82e1600640 1 -- 172.21.15.5:0/2580247860 >> 172.21.15.5:0/2580247860 conn(0x7f82dc08af80 msgr2=0x7f82dc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.240+0000 7f82e1600640 1 -- 172.21.15.5:0/2580247860 shutdown_connections 2024-04-25T10:07:09.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.240+0000 7f82e1600640 1 -- 172.21.15.5:0/2580247860 wait complete. 2024-04-25T10:07:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.241+0000 7f82e1600640 1 Processor -- start 2024-04-25T10:07:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 -- start start 2024-04-25T10:07:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82dc159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82dc0cf400 0x7f82dc0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0ca6d0 0x7f82dc0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc161680 con 0x7f82bc0039e0 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b4ca0 con 0x7f82dc0cf400 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.242+0000 7f82e1600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc161800 con 0x7f82dc0ca6d0 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82dc159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82dc159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58436/0 (socket says 172.21.15.5:58436) 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 -- 172.21.15.5:0/1188637380 learned_addr learned my addr 172.21.15.5:0/1188637380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 -- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0ca6d0 msgr2=0x7f82dc0cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0ca6d0 0x7f82dc0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 -- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82dc0cf400 msgr2=0x7f82dc0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82dc0cf400 0x7f82dc0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 -- 172.21.15.5:0/1188637380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc1599c0 con 0x7f82bc0039e0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.243+0000 7f82dbe00640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82dc159480 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f82cc008fd0 tx=0x7f82cc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.244+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc040020 con 0x7f82bc0039e0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.244+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc007680 con 0x7f82bc0039e0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.244+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc0cb2e0 con 0x7f82bc0039e0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.244+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc007a30 con 0x7f82bc0039e0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.244+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc0c7240 con 0x7f82bc0039e0 2024-04-25T10:07:09.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.245+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc15fe60 con 0x7f82bc0039e0 2024-04-25T10:07:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.246+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82cc043020 con 0x7f82bc0039e0 2024-04-25T10:07:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.247+0000 7f82d3e00640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82b806b4d0 0x7f82b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.247+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f82cc0aa320 con 0x7f82bc0039e0 2024-04-25T10:07:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.248+0000 7f82db400640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82b806b4d0 0x7f82b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.249+0000 7f82db400640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82b806b4d0 0x7f82b806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f82dc06da00 tx=0x7f82c4009750 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.252+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82dc15fe60 con 0x7f82bc0039e0 2024-04-25T10:07:09.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:09.490+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f82dc068f60 con 0x7f82bc0039e0 2024-04-25T10:07:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.061+0000 7f82d3e00640 1 -- 172.21.15.5:0/1188637380 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f82cc04a030 con 0x7f82bc0039e0 2024-04-25T10:07:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T10:07:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.069+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82b806b4d0 msgr2=0x7f82b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.069+0000 7f82e1600640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82b806b4d0 0x7f82b806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f82dc06da00 tx=0x7f82c4009750 comp rx=0 tx=0).stop 2024-04-25T10:07:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.069+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 msgr2=0x7f82dc159480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.069+0000 7f82e1600640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82dc159480 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f82cc008fd0 tx=0x7f82cc001120 comp rx=0 tx=0).stop 2024-04-25T10:07:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.070+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 shutdown_connections 2024-04-25T10:07:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.070+0000 7f82e1600640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82dc0ca6d0 0x7f82dc0cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.070+0000 7f82e1600640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82b806b4d0 0x7f82b806d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.070+0000 7f82e1600640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82dc0cf400 0x7f82dc0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.070+0000 7f82e1600640 1 --2- 172.21.15.5:0/1188637380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82bc0039e0 0x7f82dc159480 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.070+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 >> 172.21.15.5:0/1188637380 conn(0x7f82dc08af80 msgr2=0x7f82dc15e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.073+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 shutdown_connections 2024-04-25T10:07:10.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.073+0000 7f82e1600640 1 -- 172.21.15.5:0/1188637380 wait complete. 2024-04-25T10:07:10.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.275+0000 7fe0a3200640 1 Processor -- start 2024-04-25T10:07:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.276+0000 7fe0a3200640 1 -- start start 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.277+0000 7fe0a3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0a415ce00 0x7fe0a415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.277+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 0x7fe0a40cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.277+0000 7fe0a3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a4071d30 con 0x7fe0a4141da0 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.277+0000 7fe0a3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a40cdcd0 con 0x7fe0a415ce00 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.277+0000 7fe0a3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4142180 con 0x7fe0a40cd300 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.278+0000 7fe0a1e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe0a4141da0 0x7fe0a406e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54918/0 (socket says 172.21.15.5:54918) 2024-04-25T10:07:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.278+0000 7fe0a1e00640 1 -- 172.21.15.5:0/4116549258 learned_addr learned my addr 172.21.15.5:0/4116549258 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.278+0000 7fe0a2800640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 0x7fe0a40cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.278+0000 7fe099400640 1 --2- 172.21.15.5:0/4116549258 >> v2:172.21.15.80:3300/0 conn(0x7fe0a415ce00 0x7fe0a415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.278+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659313377 0 0) 0x7fe0a4071d30 con 0x7fe0a4141da0 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.278+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe088003520 con 0x7fe0a4141da0 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2018369086 0 0) 0x7fe088003520 con 0x7fe0a4141da0 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 msgr2=0x7fe0a40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 0x7fe0a40cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 >> v2:172.21.15.80:3300/0 conn(0x7fe0a415ce00 msgr2=0x7fe0a415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 --2- 172.21.15.5:0/4116549258 >> v2:172.21.15.80:3300/0 conn(0x7fe0a415ce00 0x7fe0a415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a406d620 con 0x7fe0a4141da0 2024-04-25T10:07:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe094001670 con 0x7fe0a4141da0 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 >> v1:172.21.15.5:6789/0 conn(0x7fe0a4141da0 legacy=0x7fe0a406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe088003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088004800 0x7fe088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 0x7fe0880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40cdcd0 con 0x7fe0880039b0 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4142180 con 0x7fe088004800 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08800b9c0 con 0x7fe0a40cd300 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe088003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.279+0000 7fe099400640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088004800 0x7fe088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.280+0000 7fe0a1e00640 1 -- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 msgr2=0x7fe0880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.280+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a40cd300 0x7fe0880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.280+0000 7fe0a1e00640 1 -- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088004800 msgr2=0x7fe088004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.280+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088004800 0x7fe088004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.280+0000 7fe0a1e00640 1 -- 172.21.15.5:0/4116549258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088003520 con 0x7fe0880039b0 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.280+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe088003dd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe09400c160 tx=0x7fe088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b885561d933e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094007c80 con 0x7fe0880039b0 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094005ad0 con 0x7fe0880039b0 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a1400640 1 -- 172.21.15.5:0/4116549258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094007070 con 0x7fe0880039b0 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a3200640 1 -- 172.21.15.5:0/4116549258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 msgr2=0x7fe088003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a3200640 1 --2- 172.21.15.5:0/4116549258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe088003dd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe09400c160 tx=0x7fe088005830 comp rx=0 tx=0).stop 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a2800640 1 -- 172.21.15.5:0/4116549258 reap_dead start 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a3200640 1 -- 172.21.15.5:0/4116549258 shutdown_connections 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.281+0000 7fe0a3200640 1 -- 172.21.15.5:0/4116549258 >> 172.21.15.5:0/4116549258 conn(0x7fe0a4066f20 msgr2=0x7fe0a406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.282+0000 7fe0a3200640 1 -- 172.21.15.5:0/4116549258 shutdown_connections 2024-04-25T10:07:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.282+0000 7fe0a3200640 1 -- 172.21.15.5:0/4116549258 wait complete. 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.283+0000 7fe0a3200640 1 Processor -- start 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.283+0000 7fe0a3200640 1 -- start start 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe0a413b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a413d640 0x7fe0a413dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0a41f7850 0x7fe0a41f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a406e810 con 0x7fe0880039b0 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a406dbe0 con 0x7fe0a41f7850 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40cd790 con 0x7fe0a413d640 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.284+0000 7fe0a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe0a413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe0a413b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58452/0 (socket says 172.21.15.5:58452) 2024-04-25T10:07:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 -- 172.21.15.5:0/2700403433 learned_addr learned my addr 172.21.15.5:0/2700403433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 -- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a413d640 msgr2=0x7fe0a413dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a413d640 0x7fe0a413dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 -- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0a41f7850 msgr2=0x7fe0a41f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0a41f7850 0x7fe0a41f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 -- 172.21.15.5:0/2700403433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a415a680 con 0x7fe0880039b0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe0a1e00640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe0a413b530 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe09400c280 tx=0x7fe0a4071f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094040020 con 0x7fe0880039b0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094001430 con 0x7fe0880039b0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.285+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094037040 con 0x7fe0880039b0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.286+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a415a960 con 0x7fe0880039b0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.286+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a415a330 con 0x7fe0880039b0 2024-04-25T10:07:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.287+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a415ae80 con 0x7fe0880039b0 2024-04-25T10:07:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.288+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe094043020 con 0x7fe0880039b0 2024-04-25T10:07:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.289+0000 7fe09aa00640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe08006b500 0x7fe08006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.289+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0940a9fa0 con 0x7fe0880039b0 2024-04-25T10:07:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.289+0000 7fe099400640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe08006b500 0x7fe08006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.291+0000 7fe099400640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe08006b500 0x7fe08006d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4071960 tx=0x7fe08c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.293+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe09404a030 con 0x7fe0880039b0 2024-04-25T10:07:10.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.489+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0a4068f60 con 0x7fe0880039b0 2024-04-25T10:07:10.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.491+0000 7fe09aa00640 1 -- 172.21.15.5:0/2700403433 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fe094078bf0 con 0x7fe0880039b0 2024-04-25T10:07:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:10.497 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:09.492314+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe08006b500 msgr2=0x7fe08006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe08006b500 0x7fe08006d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4071960 tx=0x7fe08c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 msgr2=0x7fe0a413b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe0a413b530 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe09400c280 tx=0x7fe0a4071f00 comp rx=0 tx=0).stop 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 shutdown_connections 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe08006b500 0x7fe08006d9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0a41f7850 0x7fe0a41f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0a413d640 0x7fe0a413dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 --2- 172.21.15.5:0/2700403433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0880039b0 0x7fe0a413b530 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 >> 172.21.15.5:0/2700403433 conn(0x7fe0a4066f20 msgr2=0x7fe0a40737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 shutdown_connections 2024-04-25T10:07:10.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.494+0000 7fe0a3200640 1 -- 172.21.15.5:0/2700403433 wait complete. 2024-04-25T10:07:10.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.721+0000 7f791a000640 1 Processor -- start 2024-04-25T10:07:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.722+0000 7f791a000640 1 -- start start 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f791a000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79140cdca0 0x7f791415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f791a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 0x7f7914071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f791a000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79140cd200 con 0x7f7914154a10 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f791a000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7914071940 con 0x7f79140cdca0 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f791a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791406e6f0 con 0x7f7914070f90 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f7918c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7914154a10 0x7f79140b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54930/0 (socket says 172.21.15.5:54930) 2024-04-25T10:07:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f7918c00640 1 -- 172.21.15.5:0/3491583659 learned_addr learned my addr 172.21.15.5:0/3491583659 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f7919600640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 0x7f7914071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f7913e00640 1 --2- 172.21.15.5:0/3491583659 >> v2:172.21.15.80:3300/0 conn(0x7f79140cdca0 0x7f791415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.724+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697203725 0 0) 0x7f79140cd200 con 0x7f7914154a10 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78fc003520 con 0x7f7914154a10 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190333100 0 0) 0x7f78fc003520 con 0x7f7914154a10 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 msgr2=0x7f7914071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 0x7f7914071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 >> v2:172.21.15.80:3300/0 conn(0x7f79140cdca0 msgr2=0x7f791415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 --2- 172.21.15.5:0/3491583659 >> v2:172.21.15.80:3300/0 conn(0x7f79140cdca0 0x7f791415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791406e9d0 con 0x7f7914154a10 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f7908001670 con 0x7f7914154a10 2024-04-25T10:07:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 >> v1:172.21.15.5:6789/0 conn(0x7f7914154a10 legacy=0x7f79140b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 0x7f78fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78fc003f80 0x7f78fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc004d40 0x7f78fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914071940 con 0x7f78fc003f80 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791406e6f0 con 0x7f78fc004d40 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc00ba70 con 0x7f7914070f90 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.725+0000 7f7913e00640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78fc003f80 0x7f78fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.726+0000 7f7918c00640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 0x7f78fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.726+0000 7f7913e00640 1 -- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 msgr2=0x7f78fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.726+0000 7f7913e00640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7914070f90 0x7f78fc0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.726+0000 7f7913e00640 1 -- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc004d40 msgr2=0x7f78fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.726+0000 7f7913e00640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc004d40 0x7f78fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.726+0000 7f7913e00640 1 -- 172.21.15.5:0/3491583659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc003520 con 0x7f78fc003f80 2024-04-25T10:07:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.727+0000 7f7913e00640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78fc003f80 0x7f78fc004800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7900009ac0 tx=0x7f78fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37347099cafcea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.727+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900036070 con 0x7f78fc003f80 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.727+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7900031400 con 0x7f78fc003f80 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.727+0000 7f7913400640 1 -- 172.21.15.5:0/3491583659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7900009690 con 0x7f78fc003f80 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f791a000640 1 -- 172.21.15.5:0/3491583659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78fc003f80 msgr2=0x7f78fc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f791a000640 1 --2- 172.21.15.5:0/3491583659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78fc003f80 0x7f78fc004800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7900009ac0 tx=0x7f78fc005830 comp rx=0 tx=0).stop 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f7919600640 1 -- 172.21.15.5:0/3491583659 reap_dead start 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f791a000640 1 -- 172.21.15.5:0/3491583659 shutdown_connections 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f791a000640 1 -- 172.21.15.5:0/3491583659 >> 172.21.15.5:0/3491583659 conn(0x7f7914066f20 msgr2=0x7f791406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f791a000640 1 -- 172.21.15.5:0/3491583659 shutdown_connections 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.728+0000 7f791a000640 1 -- 172.21.15.5:0/3491583659 wait complete. 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.730+0000 7f791a000640 1 Processor -- start 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.730+0000 7f791a000640 1 -- start start 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f791a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc003f80 0x7f79141e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f791a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f791415ef20 0x7f7914142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f791a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 0x7f7914151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f791a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79140cd780 con 0x7f791415fd70 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f791a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79140cd4a0 con 0x7f78fc003f80 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f791a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7914071400 con 0x7f791415ef20 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f7919600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 0x7f7914151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f7919600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 0x7f7914151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58476/0 (socket says 172.21.15.5:58476) 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f7919600640 1 -- 172.21.15.5:0/2878424317 learned_addr learned my addr 172.21.15.5:0/2878424317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f7913e00640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f791415ef20 0x7f7914142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.731+0000 7f7918c00640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc003f80 0x7f79141e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7919600640 1 -- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f791415ef20 msgr2=0x7f7914142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7919600640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f791415ef20 0x7f7914142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7919600640 1 -- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc003f80 msgr2=0x7f79141e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7919600640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc003f80 0x7f79141e7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7919600640 1 -- 172.21.15.5:0/2878424317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791414fda0 con 0x7f791415fd70 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7913e00640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f791415ef20 0x7f7914142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7919600640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 0x7f7914151610 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f79140b1040 tx=0x7f790c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c00bcb0 con 0x7f791415fd70 2024-04-25T10:07:10.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c01a070 con 0x7f791415fd70 2024-04-25T10:07:10.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c014640 con 0x7f791415fd70 2024-04-25T10:07:10.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791415d540 con 0x7f791415fd70 2024-04-25T10:07:10.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.732+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791415db00 con 0x7f791415fd70 2024-04-25T10:07:10.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.734+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914068f60 con 0x7f791415fd70 2024-04-25T10:07:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.735+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f790c005d10 con 0x7f791415fd70 2024-04-25T10:07:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.735+0000 7f7910c00640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f406b500 0x7f78f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.736+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f790c08a630 con 0x7f791415fd70 2024-04-25T10:07:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.736+0000 7f7918c00640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f406b500 0x7f78f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.737+0000 7f7918c00640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f406b500 0x7f78f406d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7908005a50 tx=0x7f7908009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.740+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f790c024020 con 0x7f791415fd70 2024-04-25T10:07:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.932+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79140cce40 con 0x7f791415fd70 2024-04-25T10:07:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.935+0000 7f7910c00640 1 -- 172.21.15.5:0/2878424317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f790c059280 con 0x7f791415fd70 2024-04-25T10:07:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:10.941 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:09.492314+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f406b500 msgr2=0x7f78f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f406b500 0x7f78f406d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7908005a50 tx=0x7f7908009040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 msgr2=0x7f7914151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 0x7f7914151610 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f79140b1040 tx=0x7f790c00b480 comp rx=0 tx=0).stop 2024-04-25T10:07:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 shutdown_connections 2024-04-25T10:07:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f791415fd70 0x7f7914151610 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f791415ef20 0x7f7914142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f406b500 0x7f78f406d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 --2- 172.21.15.5:0/2878424317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78fc003f80 0x7f79141e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 >> 172.21.15.5:0/2878424317 conn(0x7f7914066f20 msgr2=0x7f79140af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.937+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 shutdown_connections 2024-04-25T10:07:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:10.938+0000 7f791a000640 1 -- 172.21.15.5:0/2878424317 wait complete. 2024-04-25T10:07:10.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:07:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.172+0000 7f7b8f200640 1 Processor -- start 2024-04-25T10:07:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.173+0000 7f7b8f200640 1 -- start start 2024-04-25T10:07:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7b900d3f90 0x7f7b90161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b900b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b901606e0 con 0x7f7b900b48c0 2024-04-25T10:07:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b90162030 con 0x7f7b900d3f90 2024-04-25T10:07:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901609c0 con 0x7f7b90161bc0 2024-04-25T10:07:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8de00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7b900b48c0 0x7f7b9015fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54938/0 (socket says 172.21.15.5:54938) 2024-04-25T10:07:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.174+0000 7f7b8de00640 1 -- 172.21.15.5:0/3220738074 learned_addr learned my addr 172.21.15.5:0/3220738074 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028875045 0 0) 0x7f7b901606e0 con 0x7f7b900b48c0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b70003520 con 0x7f7b900b48c0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8d400640 1 --2- 172.21.15.5:0/3220738074 >> v2:172.21.15.80:3300/0 conn(0x7f7b900d3f90 0x7f7b90161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8e800640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b900b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024118340 0 0) 0x7f7b70003520 con 0x7f7b900b48c0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 msgr2=0x7f7b900b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b900b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 >> v2:172.21.15.80:3300/0 conn(0x7f7b900d3f90 msgr2=0x7f7b90161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 --2- 172.21.15.5:0/3220738074 >> v2:172.21.15.80:3300/0 conn(0x7f7b900d3f90 0x7f7b90161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90160ca0 con 0x7f7b900b48c0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f7b80001670 con 0x7f7b900b48c0 2024-04-25T10:07:11.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 >> v1:172.21.15.5:6789/0 conn(0x7f7b900b48c0 legacy=0x7f7b9015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b700039b0 0x7f7b70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b70004800 0x7f7b70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b700051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90162030 con 0x7f7b70004800 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901609c0 con 0x7f7b700039b0 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.175+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7000bb50 con 0x7f7b90161bc0 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8d400640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b70004800 0x7f7b70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8de00640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b700039b0 0x7f7b70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8e800640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b700051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8d400640 1 -- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 msgr2=0x7f7b700051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8d400640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b700051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8d400640 1 -- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b700039b0 msgr2=0x7f7b70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8d400640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b700039b0 0x7f7b70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8d400640 1 -- 172.21.15.5:0/3220738074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70003520 con 0x7f7b70004800 2024-04-25T10:07:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.176+0000 7f7b8e800640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b90161bc0 0x7f7b700051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.177+0000 7f7b8d400640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b70004800 0x7f7b70004c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7b780073e0 tx=0x7f7b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb68797aa29ce45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.177+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009c30 con 0x7f7b70004800 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7802e660 con 0x7f7b70004800 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8ca00640 1 -- 172.21.15.5:0/3220738074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7802ea30 con 0x7f7b70004800 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8f200640 1 -- 172.21.15.5:0/3220738074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b70004800 msgr2=0x7f7b70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8f200640 1 --2- 172.21.15.5:0/3220738074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b70004800 0x7f7b70004c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7b780073e0 tx=0x7f7b70005830 comp rx=0 tx=0).stop 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8e800640 1 -- 172.21.15.5:0/3220738074 reap_dead start 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8f200640 1 -- 172.21.15.5:0/3220738074 shutdown_connections 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8f200640 1 -- 172.21.15.5:0/3220738074 >> 172.21.15.5:0/3220738074 conn(0x7f7b9008af80 msgr2=0x7f7b900d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8f200640 1 -- 172.21.15.5:0/3220738074 shutdown_connections 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.178+0000 7f7b8f200640 1 -- 172.21.15.5:0/3220738074 wait complete. 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.180+0000 7f7b8f200640 1 Processor -- start 2024-04-25T10:07:11.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.180+0000 7f7b8f200640 1 -- start start 2024-04-25T10:07:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b70004800 0x7f7b900b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 0x7f7b900c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b900cb910 0x7f7b900cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900b4ca0 con 0x7f7b900c80c0 2024-04-25T10:07:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90161260 con 0x7f7b70004800 2024-04-25T10:07:11.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900b2b70 con 0x7f7b900cb910 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 0x7f7b900c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 0x7f7b900c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58490/0 (socket says 172.21.15.5:58490) 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.181+0000 7f7b8d400640 1 -- 172.21.15.5:0/1811971121 learned_addr learned my addr 172.21.15.5:0/1811971121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8de00640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b70004800 0x7f7b900b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8e800640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b900cb910 0x7f7b900cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8d400640 1 -- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b900cb910 msgr2=0x7f7b900cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8d400640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b900cb910 0x7f7b900cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8d400640 1 -- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b70004800 msgr2=0x7f7b900b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8d400640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b70004800 0x7f7b900b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8d400640 1 -- 172.21.15.5:0/1811971121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900cc2d0 con 0x7f7b900c80c0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.182+0000 7f7b8d400640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 0x7f7b900c8540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802b0e0 tx=0x7f7b78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.183+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78045070 con 0x7f7b900c80c0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.183+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b780369c0 con 0x7f7b900c80c0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.183+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009c30 con 0x7f7b900c80c0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.183+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b900ca870 con 0x7f7b900c80c0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.183+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b900cae30 con 0x7f7b900c80c0 2024-04-25T10:07:11.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.184+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b90068f60 con 0x7f7b900c80c0 2024-04-25T10:07:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.185+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b78048070 con 0x7f7b900c80c0 2024-04-25T10:07:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.186+0000 7f7b86000640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7b6c06b4d0 0x7f7b6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.186+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b780aae90 con 0x7f7b900c80c0 2024-04-25T10:07:11.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.186+0000 7f7b8de00640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7b6c06b4d0 0x7f7b6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.188+0000 7f7b8de00640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7b6c06b4d0 0x7f7b6c06d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b80001050 tx=0x7f7b80009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.190+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b900cc2d0 con 0x7f7b900c80c0 2024-04-25T10:07:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:11.430+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7b900d0dd0 con 0x7f7b900c80c0 2024-04-25T10:07:12.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.084+0000 7f7b86000640 1 -- 172.21.15.5:0/1811971121 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7b780ac0b0 con 0x7f7b900c80c0 2024-04-25T10:07:12.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.088+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7b6c06b4d0 msgr2=0x7f7b6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.088+0000 7f7b8f200640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7b6c06b4d0 0x7f7b6c06d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b80001050 tx=0x7f7b80009040 comp rx=0 tx=0).stop 2024-04-25T10:07:12.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.088+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 msgr2=0x7f7b900c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.088+0000 7f7b8f200640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 0x7f7b900c8540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7b7802b0e0 tx=0x7f7b78004ab0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 shutdown_connections 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7b900cb910 0x7f7b900cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b900c80c0 0x7f7b900c8540 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7b6c06b4d0 0x7f7b6c06d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 --2- 172.21.15.5:0/1811971121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b70004800 0x7f7b900b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 >> 172.21.15.5:0/1811971121 conn(0x7f7b9008af80 msgr2=0x7f7b9015dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 shutdown_connections 2024-04-25T10:07:12.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.089+0000 7f7b8f200640 1 -- 172.21.15.5:0/1811971121 wait complete. 2024-04-25T10:07:12.119 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:07:12.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:07:12.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.329+0000 7fe865200640 1 Processor -- start 2024-04-25T10:07:12.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.329+0000 7fe865200640 1 -- start start 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.331+0000 7fe865200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe8600ae860 0x7fe860071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.331+0000 7fe865200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8600aece0 0x7fe860071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.331+0000 7fe865200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86006f080 con 0x7fe8601465d0 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.331+0000 7fe865200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86006f360 con 0x7fe8600aece0 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.331+0000 7fe865200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8600ae060 con 0x7fe8600ae860 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.331+0000 7fe85e800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8600aece0 0x7fe860071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85e800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8600aece0 0x7fe860071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58498/0 (socket says 172.21.15.5:58498) 2024-04-25T10:07:12.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85e800640 1 -- 172.21.15.5:0/1894207920 learned_addr learned my addr 172.21.15.5:0/1894207920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:12.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85d400640 1 --2- 172.21.15.5:0/1894207920 >> v2:172.21.15.80:3300/0 conn(0x7fe8600ae860 0x7fe860071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85e800640 1 -- 172.21.15.5:0/1894207920 >> v2:172.21.15.80:3300/0 conn(0x7fe8600ae860 msgr2=0x7fe860071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85e800640 1 --2- 172.21.15.5:0/1894207920 >> v2:172.21.15.80:3300/0 conn(0x7fe8600ae860 0x7fe860071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85e800640 1 -- 172.21.15.5:0/1894207920 >> v1:172.21.15.80:6789/0 conn(0x7fe8601465d0 legacy=0x7fe86015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85e800640 1 -- 172.21.15.5:0/1894207920 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8600ae340 con 0x7fe8600aece0 2024-04-25T10:07:12.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.332+0000 7fe85d400640 1 --2- 172.21.15.5:0/1894207920 >> v2:172.21.15.80:3300/0 conn(0x7fe8600ae860 0x7fe860071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:12.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.333+0000 7fe85e800640 1 --2- 172.21.15.5:0/1894207920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600aece0 0x7fe860071670 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe86006ba00 tx=0x7fe85802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7df267697332fac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.334+0000 7fe85ca00640 1 -- 172.21.15.5:0/1894207920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858036070 con 0x7fe8600aece0 2024-04-25T10:07:12.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.334+0000 7fe85ca00640 1 -- 172.21.15.5:0/1894207920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858031600 con 0x7fe8600aece0 2024-04-25T10:07:12.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.334+0000 7fe85ca00640 1 -- 172.21.15.5:0/1894207920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580319d0 con 0x7fe8600aece0 2024-04-25T10:07:12.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 -- 172.21.15.5:0/1894207920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600aece0 msgr2=0x7fe860071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 --2- 172.21.15.5:0/1894207920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600aece0 0x7fe860071670 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe86006ba00 tx=0x7fe85802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 -- 172.21.15.5:0/1894207920 shutdown_connections 2024-04-25T10:07:12.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 --2- 172.21.15.5:0/1894207920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600aece0 0x7fe860071670 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 --2- 172.21.15.5:0/1894207920 >> v2:172.21.15.80:3300/0 conn(0x7fe8600ae860 0x7fe860071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 -- 172.21.15.5:0/1894207920 >> 172.21.15.5:0/1894207920 conn(0x7fe86006ab60 msgr2=0x7fe86006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 -- 172.21.15.5:0/1894207920 shutdown_connections 2024-04-25T10:07:12.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.335+0000 7fe865200640 1 -- 172.21.15.5:0/1894207920 wait complete. 2024-04-25T10:07:12.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.337+0000 7fe865200640 1 Processor -- start 2024-04-25T10:07:12.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.337+0000 7fe865200640 1 -- start start 2024-04-25T10:07:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe865200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 0x7fe860132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe865200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8600aece0 0x7fe8600b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe865200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8601465d0 0x7fe8600b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe865200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86006ece0 con 0x7fe8600ae860 2024-04-25T10:07:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe865200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86006e7d0 con 0x7fe8601465d0 2024-04-25T10:07:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe865200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86006ee60 con 0x7fe8600aece0 2024-04-25T10:07:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe85de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 0x7fe860132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe85de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 0x7fe860132b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58504/0 (socket says 172.21.15.5:58504) 2024-04-25T10:07:12.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.339+0000 7fe85de00640 1 -- 172.21.15.5:0/3069114394 learned_addr learned my addr 172.21.15.5:0/3069114394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85de00640 1 -- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8600aece0 msgr2=0x7fe8600b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85d400640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8600aece0 0x7fe8600b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85de00640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8600aece0 0x7fe8600b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85de00640 1 -- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8601465d0 msgr2=0x7fe8600b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85de00640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8601465d0 0x7fe8600b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85de00640 1 -- 172.21.15.5:0/3069114394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8600b1800 con 0x7fe8600ae860 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85d400640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8600aece0 0x7fe8600b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe85de00640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 0x7fe860132b70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe85000b3e0 tx=0x7fe85000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850015020 con 0x7fe8600ae860 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe850011850 con 0x7fe8600ae860 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8600b1ae0 con 0x7fe8600ae860 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.340+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe850011c40 con 0x7fe8600ae860 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.341+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8601330b0 con 0x7fe8600ae860 2024-04-25T10:07:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.342+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe860068f60 con 0x7fe8600ae860 2024-04-25T10:07:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.343+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe850018020 con 0x7fe8600ae860 2024-04-25T10:07:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.345+0000 7fe856000640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe84006b4d0 0x7fe84006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.345+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe85008a3d0 con 0x7fe8600ae860 2024-04-25T10:07:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.346+0000 7fe85d400640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe84006b4d0 0x7fe84006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.347+0000 7fe85d400640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe84006b4d0 0x7fe84006d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.351+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe85001f020 con 0x7fe8600ae860 2024-04-25T10:07:12.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:12.592+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe8600b2000 con 0x7fe8600ae860 2024-04-25T10:07:13.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.087+0000 7fe856000640 1 -- 172.21.15.5:0/3069114394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe8500172c0 con 0x7fe8600ae860 2024-04-25T10:07:13.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.090+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe84006b4d0 msgr2=0x7fe84006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.090+0000 7fe865200640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe84006b4d0 0x7fe84006d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 msgr2=0x7fe860132b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 0x7fe860132b70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe85000b3e0 tx=0x7fe85000b8a0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 shutdown_connections 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe84006b4d0 0x7fe84006d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8601465d0 0x7fe8600b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8600aece0 0x7fe8600b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 --2- 172.21.15.5:0/3069114394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8600ae860 0x7fe860132b70 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 >> 172.21.15.5:0/3069114394 conn(0x7fe86006ab60 msgr2=0x7fe860143210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 shutdown_connections 2024-04-25T10:07:13.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.091+0000 7fe865200640 1 -- 172.21.15.5:0/3069114394 wait complete. 2024-04-25T10:07:13.117 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:07:13.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:13.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.311+0000 7fd721200640 1 Processor -- start 2024-04-25T10:07:13.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.311+0000 7fd721200640 1 -- start start 2024-04-25T10:07:13.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd721200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd71c0cdca0 0x7fd71c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd721200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 0x7fd71c071400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd721200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c0cd200 con 0x7fd71c154a10 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd721200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71c071940 con 0x7fd71c0cdca0 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd721200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c06e6f0 con 0x7fd71c070f90 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd71b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd71c154a10 0x7fd71c0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54946/0 (socket says 172.21.15.5:54946) 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.313+0000 7fd71b400640 1 -- 172.21.15.5:0/641131005 learned_addr learned my addr 172.21.15.5:0/641131005 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084678822 0 0) 0x7fd71c0cd200 con 0x7fd71c154a10 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd700003520 con 0x7fd71c154a10 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71be00640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 0x7fd71c071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71aa00640 1 --2- 172.21.15.5:0/641131005 >> v2:172.21.15.80:3300/0 conn(0x7fd71c0cdca0 0x7fd71c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2244360909 0 0) 0x7fd700003520 con 0x7fd71c154a10 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 msgr2=0x7fd71c071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 0x7fd71c071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 >> v2:172.21.15.80:3300/0 conn(0x7fd71c0cdca0 msgr2=0x7fd71c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 --2- 172.21.15.5:0/641131005 >> v2:172.21.15.80:3300/0 conn(0x7fd71c0cdca0 0x7fd71c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c06e9d0 con 0x7fd71c154a10 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd70c001670 con 0x7fd71c154a10 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 >> v1:172.21.15.5:6789/0 conn(0x7fd71c154a10 legacy=0x7fd71c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7000039b0 0x7fd700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd700004800 0x7fd700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 0x7fd7000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c071940 con 0x7fd700004800 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c06e6f0 con 0x7fd7000039b0 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.314+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70000bb20 con 0x7fd71c070f90 2024-04-25T10:07:13.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71aa00640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd700004800 0x7fd700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71aa00640 1 -- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 msgr2=0x7fd7000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71aa00640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd71c070f90 0x7fd7000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71b400640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7000039b0 0x7fd700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71aa00640 1 -- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7000039b0 msgr2=0x7fd700003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71aa00640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7000039b0 0x7fd700003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71aa00640 1 -- 172.21.15.5:0/641131005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700003520 con 0x7fd700004800 2024-04-25T10:07:13.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.315+0000 7fd71b400640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7000039b0 0x7fd700003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.316+0000 7fd71aa00640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd700004800 0x7fd700004c80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd70800ba00 tx=0x7fd700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24415ae7ec6d3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.316+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708030070 con 0x7fd700004800 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.316+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70802add0 con 0x7fd700004800 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.316+0000 7fd71a000640 1 -- 172.21.15.5:0/641131005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70800b600 con 0x7fd700004800 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd721200640 1 -- 172.21.15.5:0/641131005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd700004800 msgr2=0x7fd700004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd721200640 1 --2- 172.21.15.5:0/641131005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd700004800 0x7fd700004c80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd70800ba00 tx=0x7fd700005830 comp rx=0 tx=0).stop 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd71be00640 1 -- 172.21.15.5:0/641131005 reap_dead start 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd721200640 1 -- 172.21.15.5:0/641131005 shutdown_connections 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd721200640 1 -- 172.21.15.5:0/641131005 >> 172.21.15.5:0/641131005 conn(0x7fd71c066f20 msgr2=0x7fd71c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd721200640 1 -- 172.21.15.5:0/641131005 shutdown_connections 2024-04-25T10:07:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.317+0000 7fd721200640 1 -- 172.21.15.5:0/641131005 wait complete. 2024-04-25T10:07:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.319+0000 7fd721200640 1 Processor -- start 2024-04-25T10:07:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.319+0000 7fd721200640 1 -- start start 2024-04-25T10:07:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd721200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd700004800 0x7fd71c14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd721200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd71c152160 0x7fd71c0b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd721200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 0x7fd71c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd721200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0cd780 con 0x7fd71c1525e0 2024-04-25T10:07:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd721200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c0cd4a0 con 0x7fd71c152160 2024-04-25T10:07:13.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd721200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c071400 con 0x7fd700004800 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd71be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 0x7fd71c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd71be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 0x7fd71c0b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58522/0 (socket says 172.21.15.5:58522) 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.320+0000 7fd71be00640 1 -- 172.21.15.5:0/2246691460 learned_addr learned my addr 172.21.15.5:0/2246691460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71b400640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd700004800 0x7fd71c14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71aa00640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd71c152160 0x7fd71c0b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71be00640 1 -- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd700004800 msgr2=0x7fd71c14fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71be00640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd700004800 0x7fd71c14fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71be00640 1 -- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd71c152160 msgr2=0x7fd71c0b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71be00640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd71c152160 0x7fd71c0b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71be00640 1 -- 172.21.15.5:0/2246691460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c15f540 con 0x7fd71c1525e0 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71aa00640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd71c152160 0x7fd71c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd71be00640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 0x7fd71c0b2ec0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0b1040 tx=0x7fd71400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400ed30 con 0x7fd71c1525e0 2024-04-25T10:07:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714019070 con 0x7fd71c1525e0 2024-04-25T10:07:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.321+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714014660 con 0x7fd71c1525e0 2024-04-25T10:07:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.322+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c15f820 con 0x7fd71c1525e0 2024-04-25T10:07:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.322+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c15fde0 con 0x7fd71c1525e0 2024-04-25T10:07:13.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.323+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c068f60 con 0x7fd71c1525e0 2024-04-25T10:07:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.324+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd714014800 con 0x7fd71c1525e0 2024-04-25T10:07:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.325+0000 7fd713400640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd6fc06b4d0 0x7fd6fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.325+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd71408a620 con 0x7fd71c1525e0 2024-04-25T10:07:13.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.326+0000 7fd71b400640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd6fc06b4d0 0x7fd6fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.327+0000 7fd71b400640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd6fc06b4d0 0x7fd6fc06d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001eb0 tx=0x7fd70c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.329+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd714059300 con 0x7fd71c1525e0 2024-04-25T10:07:13.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.566+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd71c0cce40 con 0x7fd71c1525e0 2024-04-25T10:07:13.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.567+0000 7fd713400640 1 -- 172.21.15.5:0/2246691460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fd714058c00 con 0x7fd71c1525e0 2024-04-25T10:07:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:13.572 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:12.593375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5920":{"gid":5920,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1098619647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1098619647},{"type":"v1","addr":"172.21.15.80:6835","nonce":1098619647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:13.573 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-04-25T10:07:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd6fc06b4d0 msgr2=0x7fd6fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd6fc06b4d0 0x7fd6fc06d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001eb0 tx=0x7fd70c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 msgr2=0x7fd71c0b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 0x7fd71c0b2ec0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0b1040 tx=0x7fd71400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 shutdown_connections 2024-04-25T10:07:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd71c1525e0 0x7fd71c0b2ec0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd6fc06b4d0 0x7fd6fc06d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd71c152160 0x7fd71c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 --2- 172.21.15.5:0/2246691460 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd700004800 0x7fd71c14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.569+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 >> 172.21.15.5:0/2246691460 conn(0x7fd71c066f20 msgr2=0x7fd71c0b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.570+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 shutdown_connections 2024-04-25T10:07:13.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.570+0000 7fd721200640 1 -- 172.21.15.5:0/2246691460 wait complete. 2024-04-25T10:07:13.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.798+0000 7f29dbe00640 1 Processor -- start 2024-04-25T10:07:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.798+0000 7f29dbe00640 1 -- start start 2024-04-25T10:07:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc15ce00 0x7f29dc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29dbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29dc0cd300 0x7f29dc0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29dbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc071d30 con 0x7f29dc141da0 2024-04-25T10:07:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29dbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29dc0cdcd0 con 0x7f29dc0cd300 2024-04-25T10:07:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29dbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc142180 con 0x7f29dc15ce00 2024-04-25T10:07:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29d3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc15ce00 0x7f29dc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29db400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29dc0cd300 0x7f29dc0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.800+0000 7f29daa00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f29dc141da0 0x7f29dc06e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:52524/0 (socket says 172.21.15.5:52524) 2024-04-25T10:07:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.801+0000 7f29daa00640 1 -- 172.21.15.5:0/3882241000 learned_addr learned my addr 172.21.15.5:0/3882241000 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.801+0000 7f29d3e00640 1 -- 172.21.15.5:0/3882241000 >> v2:172.21.15.80:3300/0 conn(0x7f29dc0cd300 msgr2=0x7f29dc0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.801+0000 7f29d3e00640 1 --2- 172.21.15.5:0/3882241000 >> v2:172.21.15.80:3300/0 conn(0x7f29dc0cd300 0x7f29dc0cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.801+0000 7f29d3e00640 1 -- 172.21.15.5:0/3882241000 >> v1:172.21.15.80:6789/0 conn(0x7f29dc141da0 legacy=0x7f29dc06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.801+0000 7f29d3e00640 1 -- 172.21.15.5:0/3882241000 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc06d620 con 0x7f29dc15ce00 2024-04-25T10:07:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.801+0000 7f29db400640 1 --2- 172.21.15.5:0/3882241000 >> v2:172.21.15.80:3300/0 conn(0x7f29dc0cd300 0x7f29dc0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.803+0000 7f29d3e00640 1 --2- 172.21.15.5:0/3882241000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc15ce00 0x7f29dc15d280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6b70e2e7d32865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.803+0000 7f29da000640 1 -- 172.21.15.5:0/3882241000 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4037070 con 0x7f29dc15ce00 2024-04-25T10:07:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.803+0000 7f29da000640 1 -- 172.21.15.5:0/3882241000 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4032440 con 0x7f29dc15ce00 2024-04-25T10:07:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.803+0000 7f29da000640 1 -- 172.21.15.5:0/3882241000 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4032810 con 0x7f29dc15ce00 2024-04-25T10:07:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.804+0000 7f29dbe00640 1 -- 172.21.15.5:0/3882241000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc15ce00 msgr2=0x7f29dc15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.804+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3882241000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc15ce00 0x7f29dc15d280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f29c40097a0 tx=0x7f29c402a180 comp rx=0 tx=0).stop 2024-04-25T10:07:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.804+0000 7f29dbe00640 1 -- 172.21.15.5:0/3882241000 shutdown_connections 2024-04-25T10:07:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.804+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3882241000 >> v2:172.21.15.80:3300/0 conn(0x7f29dc0cd300 0x7f29dc0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.804+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3882241000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc15ce00 0x7f29dc15d280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.804+0000 7f29dbe00640 1 -- 172.21.15.5:0/3882241000 >> 172.21.15.5:0/3882241000 conn(0x7f29dc066f20 msgr2=0x7f29dc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.805+0000 7f29dbe00640 1 -- 172.21.15.5:0/3882241000 shutdown_connections 2024-04-25T10:07:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.805+0000 7f29dbe00640 1 -- 172.21.15.5:0/3882241000 wait complete. 2024-04-25T10:07:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.807+0000 7f29dbe00640 1 Processor -- start 2024-04-25T10:07:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.807+0000 7f29dbe00640 1 -- start start 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.808+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc0cd300 0x7f29dc1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.808+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29dc141da0 0x7f29dc15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.808+0000 7f29dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 0x7f29dc12b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.808+0000 7f29dbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc06e810 con 0x7f29dc15ce00 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.808+0000 7f29dbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc06dbe0 con 0x7f29dc141da0 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.808+0000 7f29dbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc071000 con 0x7f29dc0cd300 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 0x7f29dc12b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 0x7f29dc12b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58532/0 (socket says 172.21.15.5:58532) 2024-04-25T10:07:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 -- 172.21.15.5:0/3385516834 learned_addr learned my addr 172.21.15.5:0/3385516834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29daa00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc0cd300 0x7f29dc1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29d3e00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29dc141da0 0x7f29dc15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 -- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc0cd300 msgr2=0x7f29dc1e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc0cd300 0x7f29dc1e5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 -- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29dc141da0 msgr2=0x7f29dc15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29dc141da0 0x7f29dc15b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.809+0000 7f29db400640 1 -- 172.21.15.5:0/3385516834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc1ee3a0 con 0x7f29dc15ce00 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.810+0000 7f29db400640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 0x7f29dc12b6d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f29d400efc0 tx=0x7f29d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.810+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4018030 con 0x7f29dc15ce00 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.810+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d400cde0 con 0x7f29dc15ce00 2024-04-25T10:07:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.810+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003bb0 con 0x7f29dc15ce00 2024-04-25T10:07:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.810+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc129820 con 0x7f29dc15ce00 2024-04-25T10:07:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.810+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc129de0 con 0x7f29dc15ce00 2024-04-25T10:07:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.811+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc068f60 con 0x7f29dc15ce00 2024-04-25T10:07:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.812+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29d401b070 con 0x7f29dc15ce00 2024-04-25T10:07:13.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.813+0000 7f29d2a00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29b806b500 0x7f29b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.813+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29d408ae00 con 0x7f29dc15ce00 2024-04-25T10:07:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.814+0000 7f29daa00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29b806b500 0x7f29b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.815+0000 7f29daa00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29b806b500 0x7f29b806d9c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005880 tx=0x7f29cc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:13.818+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29d4010040 con 0x7f29dc15ce00 2024-04-25T10:07:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.020+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29dc0ce320 con 0x7f29dc15ce00 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.023+0000 7f29d2a00640 1 -- 172.21.15.5:0/3385516834 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f29d4012340 con 0x7f29dc15ce00 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:14.029 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":77,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:11.039355+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29b806b500 msgr2=0x7f29b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29b806b500 0x7f29b806d9c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f29cc005880 tx=0x7f29cc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 msgr2=0x7f29dc12b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 0x7f29dc12b6d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f29d400efc0 tx=0x7f29d4009f90 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 shutdown_connections 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29dc15ce00 0x7f29dc12b6d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29b806b500 0x7f29b806d9c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29dc141da0 0x7f29dc15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 --2- 172.21.15.5:0/3385516834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29dc0cd300 0x7f29dc1e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 >> 172.21.15.5:0/3385516834 conn(0x7f29dc066f20 msgr2=0x7f29dc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.026+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 shutdown_connections 2024-04-25T10:07:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.027+0000 7f29dbe00640 1 -- 172.21.15.5:0/3385516834 wait complete. 2024-04-25T10:07:14.051 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:07:14.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.262+0000 7fbe63200640 1 Processor -- start 2024-04-25T10:07:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.262+0000 7fbe63200640 1 -- start start 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.263+0000 7fbe63200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 0x7fbe6406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.263+0000 7fbe63200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe6406e640 0x7fbe6415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.263+0000 7fbe63200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe64071d30 con 0x7fbe6415d340 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.263+0000 7fbe63200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe64142180 con 0x7fbe6406e640 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe63200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6406d620 con 0x7fbe64141da0 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe62800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbe6415d340 0x7fbe640cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54952/0 (socket says 172.21.15.5:54952) 2024-04-25T10:07:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe62800640 1 -- 172.21.15.5:0/1732099802 learned_addr learned my addr 172.21.15.5:0/1732099802 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe61e00640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 0x7fbe6406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe61400640 1 --2- 172.21.15.5:0/1732099802 >> v2:172.21.15.80:3300/0 conn(0x7fbe6406e640 0x7fbe6415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738438746 0 0) 0x7fbe64071d30 con 0x7fbe6415d340 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.264+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe48003520 con 0x7fbe6415d340 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 567548147 0 0) 0x7fbe48003520 con 0x7fbe6415d340 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 msgr2=0x7fbe6406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 0x7fbe6406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 >> v2:172.21.15.80:3300/0 conn(0x7fbe6406e640 msgr2=0x7fbe6415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 --2- 172.21.15.5:0/1732099802 >> v2:172.21.15.80:3300/0 conn(0x7fbe6406e640 0x7fbe6415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6406d900 con 0x7fbe6415d340 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fbe5c001670 con 0x7fbe6415d340 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 >> v1:172.21.15.5:6789/0 conn(0x7fbe6415d340 legacy=0x7fbe640cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 0x7fbe480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe48003f30 0x7fbe48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48004d40 0x7fbe480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64142180 con 0x7fbe48003f30 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6406d620 con 0x7fbe48004d40 2024-04-25T10:07:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4800b9c0 con 0x7fbe64141da0 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.265+0000 7fbe61400640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe48003f30 0x7fbe48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe61e00640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 0x7fbe480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe62800640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48004d40 0x7fbe480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe61400640 1 -- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 msgr2=0x7fbe480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe61400640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe64141da0 0x7fbe480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe61400640 1 -- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48004d40 msgr2=0x7fbe480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe61400640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48004d40 0x7fbe480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.266+0000 7fbe61400640 1 -- 172.21.15.5:0/1732099802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48003520 con 0x7fbe48003f30 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.267+0000 7fbe61400640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe48003f30 0x7fbe48004800 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009f90 tx=0x7fbe48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b68fbcd05350acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.267+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c009c10 con 0x7fbe48003f30 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.267+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c036070 con 0x7fbe48003f30 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.267+0000 7fbe60a00640 1 -- 172.21.15.5:0/1732099802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c031630 con 0x7fbe48003f30 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe63200640 1 -- 172.21.15.5:0/1732099802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe48003f30 msgr2=0x7fbe48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe63200640 1 --2- 172.21.15.5:0/1732099802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe48003f30 0x7fbe48004800 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009f90 tx=0x7fbe48005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe62800640 1 -- 172.21.15.5:0/1732099802 reap_dead start 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe63200640 1 -- 172.21.15.5:0/1732099802 shutdown_connections 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe63200640 1 -- 172.21.15.5:0/1732099802 >> 172.21.15.5:0/1732099802 conn(0x7fbe64066f20 msgr2=0x7fbe6406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe63200640 1 -- 172.21.15.5:0/1732099802 shutdown_connections 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.268+0000 7fbe63200640 1 -- 172.21.15.5:0/1732099802 wait complete. 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.270+0000 7fbe63200640 1 Processor -- start 2024-04-25T10:07:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.270+0000 7fbe63200640 1 -- start start 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.271+0000 7fbe63200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48003f30 0x7fbe64156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.271+0000 7fbe63200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 0x7fbe6413fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.271+0000 7fbe63200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe641400b0 0x7fbe641f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.271+0000 7fbe63200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640cdb90 con 0x7fbe6413d730 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.271+0000 7fbe63200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640cda10 con 0x7fbe48003f30 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.271+0000 7fbe63200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6406e100 con 0x7fbe641400b0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 0x7fbe6413fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 0x7fbe6413fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58556/0 (socket says 172.21.15.5:58556) 2024-04-25T10:07:14.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 -- 172.21.15.5:0/972708336 learned_addr learned my addr 172.21.15.5:0/972708336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe62800640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe641400b0 0x7fbe641f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 -- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe641400b0 msgr2=0x7fbe641f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61e00640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48003f30 0x7fbe64156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe641400b0 0x7fbe641f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 -- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48003f30 msgr2=0x7fbe64156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48003f30 0x7fbe64156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.272+0000 7fbe61400640 1 -- 172.21.15.5:0/972708336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6413e740 con 0x7fbe6413d730 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.273+0000 7fbe61400640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 0x7fbe6413fb70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c033040 tx=0x7fbe4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.273+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c009c10 con 0x7fbe6413d730 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.273+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c045630 con 0x7fbe6413d730 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.273+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6413e180 con 0x7fbe6413d730 2024-04-25T10:07:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.273+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c045a00 con 0x7fbe6413d730 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.273+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6413f370 con 0x7fbe6413d730 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.274+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe64068f60 con 0x7fbe6413d730 2024-04-25T10:07:14.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.275+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe4c005d10 con 0x7fbe6413d730 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.276+0000 7fbe5a000640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe4006b500 0x7fbe4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.276+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe4c02e030 con 0x7fbe6413d730 2024-04-25T10:07:14.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.277+0000 7fbe61e00640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe4006b500 0x7fbe4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.278+0000 7fbe61e00640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe4006b500 0x7fbe4006d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009870 tx=0x7fbe54008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.281+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe4c0393b0 con 0x7fbe6413d730 2024-04-25T10:07:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.480+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe6415f990 con 0x7fbe4006b500 2024-04-25T10:07:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.500+0000 7fbe5a000640 1 -- 172.21.15.5:0/972708336 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251434 (secure 0 0 0) 0x7fbe6415f990 con 0x7fbe4006b500 2024-04-25T10:07:14.509 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:07:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe4006b500 msgr2=0x7fbe4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe4006b500 0x7fbe4006d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009870 tx=0x7fbe54008040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 msgr2=0x7fbe6413fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 0x7fbe6413fb70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c033040 tx=0x7fbe4c009750 comp rx=0 tx=0).stop 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 shutdown_connections 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe641400b0 0x7fbe641f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe6413d730 0x7fbe6413fb70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe4006b500 0x7fbe4006d9c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 --2- 172.21.15.5:0/972708336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe48003f30 0x7fbe64156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 >> 172.21.15.5:0/972708336 conn(0x7fbe64066f20 msgr2=0x7fbe640b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 shutdown_connections 2024-04-25T10:07:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.507+0000 7fbe63200640 1 -- 172.21.15.5:0/972708336 wait complete. 2024-04-25T10:07:14.534 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-04-25T10:07:12.728506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253660,"kb_used_data":8360,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721060,"statfs":{"total":773094113280,"available":772834365440,"internally_reserved":0,"allocated":8560640,"data_stored":4497511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.715178"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.092382+0000","last_change":"2024-04-25T10:07:10.090709+0000","last_active":"2024-04-25T10:07:10.092382+0000","last_peered":"2024-04-25T10:07:10.092382+0000","last_clean":"2024-04-25T10:07:10.092382+0000","last_became_active":"2024-04-25T10:07:10.089186+0000","last_became_peered":"2024-04-25T10:07:10.089186+0000","last_unstale":"2024-04-25T10:07:10.092382+0000","last_undegraded":"2024-04-25T10:07:10.092382+0000","last_fullsized":"2024-04-25T10:07:10.092382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.803884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.098597+0000","last_change":"2024-04-25T10:07:10.095329+0000","last_active":"2024-04-25T10:07:10.098597+0000","last_peered":"2024-04-25T10:07:10.098597+0000","last_clean":"2024-04-25T10:07:10.098597+0000","last_became_active":"2024-04-25T10:07:10.095016+0000","last_became_peered":"2024-04-25T10:07:10.095016+0000","last_unstale":"2024-04-25T10:07:10.098597+0000","last_undegraded":"2024-04-25T10:07:10.098597+0000","last_fullsized":"2024-04-25T10:07:10.098597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:30.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.092235+0000","last_change":"2024-04-25T10:07:10.087385+0000","last_active":"2024-04-25T10:07:10.092235+0000","last_peered":"2024-04-25T10:07:10.092235+0000","last_clean":"2024-04-25T10:07:10.092235+0000","last_became_active":"2024-04-25T10:07:10.087033+0000","last_became_peered":"2024-04-25T10:07:10.087033+0000","last_unstale":"2024-04-25T10:07:10.092235+0000","last_undegraded":"2024-04-25T10:07:10.092235+0000","last_fullsized":"2024-04-25T10:07:10.092235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:07.702579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.073968+0000","last_change":"2024-04-25T10:07:10.069285+0000","last_active":"2024-04-25T10:07:10.073968+0000","last_peered":"2024-04-25T10:07:10.073968+0000","last_clean":"2024-04-25T10:07:10.073968+0000","last_became_active":"2024-04-25T10:07:10.068958+0000","last_became_peered":"2024-04-25T10:07:10.068958+0000","last_unstale":"2024-04-25T10:07:10.073968+0000","last_undegraded":"2024-04-25T10:07:10.073968+0000","last_fullsized":"2024-04-25T10:07:10.073968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.636706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085618+0000","last_change":"2024-04-25T10:07:10.078013+0000","last_active":"2024-04-25T10:07:10.085618+0000","last_peered":"2024-04-25T10:07:10.085618+0000","last_clean":"2024-04-25T10:07:10.085618+0000","last_became_active":"2024-04-25T10:07:10.076185+0000","last_became_peered":"2024-04-25T10:07:10.076185+0000","last_unstale":"2024-04-25T10:07:10.085618+0000","last_undegraded":"2024-04-25T10:07:10.085618+0000","last_fullsized":"2024-04-25T10:07:10.085618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:16.653884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.095988+0000","last_change":"2024-04-25T10:07:10.093710+0000","last_active":"2024-04-25T10:07:10.095988+0000","last_peered":"2024-04-25T10:07:10.095988+0000","last_clean":"2024-04-25T10:07:10.095988+0000","last_became_active":"2024-04-25T10:07:10.093372+0000","last_became_peered":"2024-04-25T10:07:10.093372+0000","last_unstale":"2024-04-25T10:07:10.095988+0000","last_undegraded":"2024-04-25T10:07:10.095988+0000","last_fullsized":"2024-04-25T10:07:10.095988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:34.955739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084999+0000","last_change":"2024-04-25T10:07:10.083499+0000","last_active":"2024-04-25T10:07:10.084999+0000","last_peered":"2024-04-25T10:07:10.084999+0000","last_clean":"2024-04-25T10:07:10.084999+0000","last_became_active":"2024-04-25T10:07:10.083202+0000","last_became_peered":"2024-04-25T10:07:10.083202+0000","last_unstale":"2024-04-25T10:07:10.084999+0000","last_undegraded":"2024-04-25T10:07:10.084999+0000","last_fullsized":"2024-04-25T10:07:10.084999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:02.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.024756+0000","last_change":"2024-04-25T10:07:09.024756+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.024756+0000","last_undegraded":"2024-04-25T10:07:09.024756+0000","last_fullsized":"2024-04-25T10:07:09.024756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.078272+0000","last_change":"2024-04-25T10:07:10.075069+0000","last_active":"2024-04-25T10:07:10.078272+0000","last_peered":"2024-04-25T10:07:10.078272+0000","last_clean":"2024-04-25T10:07:10.078272+0000","last_became_active":"2024-04-25T10:07:10.074614+0000","last_became_peered":"2024-04-25T10:07:10.074614+0000","last_unstale":"2024-04-25T10:07:10.078272+0000","last_undegraded":"2024-04-25T10:07:10.078272+0000","last_fullsized":"2024-04-25T10:07:10.078272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:40.764957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089358+0000","last_change":"2024-04-25T10:07:10.087645+0000","last_active":"2024-04-25T10:07:10.089358+0000","last_peered":"2024-04-25T10:07:10.089358+0000","last_clean":"2024-04-25T10:07:10.089358+0000","last_became_active":"2024-04-25T10:07:10.086862+0000","last_became_peered":"2024-04-25T10:07:10.086862+0000","last_unstale":"2024-04-25T10:07:10.089358+0000","last_undegraded":"2024-04-25T10:07:10.089358+0000","last_fullsized":"2024-04-25T10:07:10.089358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.246741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.098938+0000","last_change":"2024-04-25T10:07:10.096129+0000","last_active":"2024-04-25T10:07:10.098938+0000","last_peered":"2024-04-25T10:07:10.098938+0000","last_clean":"2024-04-25T10:07:10.098938+0000","last_became_active":"2024-04-25T10:07:10.095868+0000","last_became_peered":"2024-04-25T10:07:10.095868+0000","last_unstale":"2024-04-25T10:07:10.098938+0000","last_undegraded":"2024-04-25T10:07:10.098938+0000","last_fullsized":"2024-04-25T10:07:10.098938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:15.076173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.077038+0000","last_change":"2024-04-25T10:07:10.071893+0000","last_active":"2024-04-25T10:07:10.077038+0000","last_peered":"2024-04-25T10:07:10.077038+0000","last_clean":"2024-04-25T10:07:10.077038+0000","last_became_active":"2024-04-25T10:07:10.071529+0000","last_became_peered":"2024-04-25T10:07:10.071529+0000","last_unstale":"2024-04-25T10:07:10.077038+0000","last_undegraded":"2024-04-25T10:07:10.077038+0000","last_fullsized":"2024-04-25T10:07:10.077038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:41.130634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089668+0000","last_change":"2024-04-25T10:07:10.087289+0000","last_active":"2024-04-25T10:07:10.089668+0000","last_peered":"2024-04-25T10:07:10.089668+0000","last_clean":"2024-04-25T10:07:10.089668+0000","last_became_active":"2024-04-25T10:07:10.086764+0000","last_became_peered":"2024-04-25T10:07:10.086764+0000","last_unstale":"2024-04-25T10:07:10.089668+0000","last_undegraded":"2024-04-25T10:07:10.089668+0000","last_fullsized":"2024-04-25T10:07:10.089668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:16.711444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.029552+0000","last_change":"2024-04-25T10:07:09.029552+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.029552+0000","last_undegraded":"2024-04-25T10:07:09.029552+0000","last_fullsized":"2024-04-25T10:07:09.029552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085512+0000","last_change":"2024-04-25T10:07:10.079081+0000","last_active":"2024-04-25T10:07:10.085512+0000","last_peered":"2024-04-25T10:07:10.085512+0000","last_clean":"2024-04-25T10:07:10.085512+0000","last_became_active":"2024-04-25T10:07:10.078800+0000","last_became_peered":"2024-04-25T10:07:10.078800+0000","last_unstale":"2024-04-25T10:07:10.085512+0000","last_undegraded":"2024-04-25T10:07:10.085512+0000","last_fullsized":"2024-04-25T10:07:10.085512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:01.412707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084662+0000","last_change":"2024-04-25T10:07:10.076364+0000","last_active":"2024-04-25T10:07:10.084662+0000","last_peered":"2024-04-25T10:07:10.084662+0000","last_clean":"2024-04-25T10:07:10.084662+0000","last_became_active":"2024-04-25T10:07:10.075916+0000","last_became_peered":"2024-04-25T10:07:10.075916+0000","last_unstale":"2024-04-25T10:07:10.084662+0000","last_undegraded":"2024-04-25T10:07:10.084662+0000","last_fullsized":"2024-04-25T10:07:10.084662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:32.882689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.027118+0000","last_change":"2024-04-25T10:07:09.027118+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.027118+0000","last_undegraded":"2024-04-25T10:07:09.027118+0000","last_fullsized":"2024-04-25T10:07:09.027118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.090061+0000","last_change":"2024-04-25T10:07:10.088011+0000","last_active":"2024-04-25T10:07:10.090061+0000","last_peered":"2024-04-25T10:07:10.090061+0000","last_clean":"2024-04-25T10:07:10.090061+0000","last_became_active":"2024-04-25T10:07:10.087755+0000","last_became_peered":"2024-04-25T10:07:10.087755+0000","last_unstale":"2024-04-25T10:07:10.090061+0000","last_undegraded":"2024-04-25T10:07:10.090061+0000","last_fullsized":"2024-04-25T10:07:10.090061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:07.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.027017+0000","last_change":"2024-04-25T10:07:09.027017+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.027017+0000","last_undegraded":"2024-04-25T10:07:09.027017+0000","last_fullsized":"2024-04-25T10:07:09.027017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085596+0000","last_change":"2024-04-25T10:07:10.078269+0000","last_active":"2024-04-25T10:07:10.085596+0000","last_peered":"2024-04-25T10:07:10.085596+0000","last_clean":"2024-04-25T10:07:10.085596+0000","last_became_active":"2024-04-25T10:07:10.077936+0000","last_became_peered":"2024-04-25T10:07:10.077936+0000","last_unstale":"2024-04-25T10:07:10.085596+0000","last_undegraded":"2024-04-25T10:07:10.085596+0000","last_fullsized":"2024-04-25T10:07:10.085596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:15.076173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.093593+0000","last_change":"2024-04-25T10:07:10.092413+0000","last_active":"2024-04-25T10:07:10.093593+0000","last_peered":"2024-04-25T10:07:10.093593+0000","last_clean":"2024-04-25T10:07:10.093593+0000","last_became_active":"2024-04-25T10:07:10.091689+0000","last_became_peered":"2024-04-25T10:07:10.091689+0000","last_unstale":"2024-04-25T10:07:10.093593+0000","last_undegraded":"2024-04-25T10:07:10.093593+0000","last_fullsized":"2024-04-25T10:07:10.093593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:15.076173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.083448+0000","last_change":"2024-04-25T10:07:10.079875+0000","last_active":"2024-04-25T10:07:10.083448+0000","last_peered":"2024-04-25T10:07:10.083448+0000","last_clean":"2024-04-25T10:07:10.083448+0000","last_became_active":"2024-04-25T10:07:10.079372+0000","last_became_peered":"2024-04-25T10:07:10.079372+0000","last_unstale":"2024-04-25T10:07:10.083448+0000","last_undegraded":"2024-04-25T10:07:10.083448+0000","last_fullsized":"2024-04-25T10:07:10.083448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:57.499900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088985+0000","last_change":"2024-04-25T10:07:10.084608+0000","last_active":"2024-04-25T10:07:10.088985+0000","last_peered":"2024-04-25T10:07:10.088985+0000","last_clean":"2024-04-25T10:07:10.088985+0000","last_became_active":"2024-04-25T10:07:10.083990+0000","last_became_peered":"2024-04-25T10:07:10.083990+0000","last_unstale":"2024-04-25T10:07:10.088985+0000","last_undegraded":"2024-04-25T10:07:10.088985+0000","last_fullsized":"2024-04-25T10:07:10.088985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:40.764957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089702+0000","last_change":"2024-04-25T10:07:10.087376+0000","last_active":"2024-04-25T10:07:10.089702+0000","last_peered":"2024-04-25T10:07:10.089702+0000","last_clean":"2024-04-25T10:07:10.089702+0000","last_became_active":"2024-04-25T10:07:10.086656+0000","last_became_peered":"2024-04-25T10:07:10.086656+0000","last_unstale":"2024-04-25T10:07:10.089702+0000","last_undegraded":"2024-04-25T10:07:10.089702+0000","last_fullsized":"2024-04-25T10:07:10.089702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:39.743425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089363+0000","last_change":"2024-04-25T10:07:10.085420+0000","last_active":"2024-04-25T10:07:10.089363+0000","last_peered":"2024-04-25T10:07:10.089363+0000","last_clean":"2024-04-25T10:07:10.089363+0000","last_became_active":"2024-04-25T10:07:10.085112+0000","last_became_peered":"2024-04-25T10:07:10.085112+0000","last_unstale":"2024-04-25T10:07:10.089363+0000","last_undegraded":"2024-04-25T10:07:10.089363+0000","last_fullsized":"2024-04-25T10:07:10.089363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.012681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085850+0000","last_change":"2024-04-25T10:07:10.082372+0000","last_active":"2024-04-25T10:07:10.085850+0000","last_peered":"2024-04-25T10:07:10.085850+0000","last_clean":"2024-04-25T10:07:10.085850+0000","last_became_active":"2024-04-25T10:07:10.082087+0000","last_became_peered":"2024-04-25T10:07:10.082087+0000","last_unstale":"2024-04-25T10:07:10.085850+0000","last_undegraded":"2024-04-25T10:07:10.085850+0000","last_fullsized":"2024-04-25T10:07:10.085850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.803884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.080853+0000","last_change":"2024-04-25T10:07:10.074949+0000","last_active":"2024-04-25T10:07:10.080853+0000","last_peered":"2024-04-25T10:07:10.080853+0000","last_clean":"2024-04-25T10:07:10.080853+0000","last_became_active":"2024-04-25T10:07:10.074647+0000","last_became_peered":"2024-04-25T10:07:10.074647+0000","last_unstale":"2024-04-25T10:07:10.080853+0000","last_undegraded":"2024-04-25T10:07:10.080853+0000","last_fullsized":"2024-04-25T10:07:10.080853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:11.860753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084562+0000","last_change":"2024-04-25T10:07:10.073901+0000","last_active":"2024-04-25T10:07:10.084562+0000","last_peered":"2024-04-25T10:07:10.084562+0000","last_clean":"2024-04-25T10:07:10.084562+0000","last_became_active":"2024-04-25T10:07:10.073595+0000","last_became_peered":"2024-04-25T10:07:10.073595+0000","last_unstale":"2024-04-25T10:07:10.084562+0000","last_undegraded":"2024-04-25T10:07:10.084562+0000","last_fullsized":"2024-04-25T10:07:10.084562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:32.202822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085456+0000","last_change":"2024-04-25T10:07:10.073215+0000","last_active":"2024-04-25T10:07:10.085456+0000","last_peered":"2024-04-25T10:07:10.085456+0000","last_clean":"2024-04-25T10:07:10.085456+0000","last_became_active":"2024-04-25T10:07:10.069330+0000","last_became_peered":"2024-04-25T10:07:10.069330+0000","last_unstale":"2024-04-25T10:07:10.085456+0000","last_undegraded":"2024-04-25T10:07:10.085456+0000","last_fullsized":"2024-04-25T10:07:10.085456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:18.353706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089141+0000","last_change":"2024-04-25T10:07:10.083911+0000","last_active":"2024-04-25T10:07:10.089141+0000","last_peered":"2024-04-25T10:07:10.089141+0000","last_clean":"2024-04-25T10:07:10.089141+0000","last_became_active":"2024-04-25T10:07:10.083621+0000","last_became_peered":"2024-04-25T10:07:10.083621+0000","last_unstale":"2024-04-25T10:07:10.089141+0000","last_undegraded":"2024-04-25T10:07:10.089141+0000","last_fullsized":"2024-04-25T10:07:10.089141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:11.860753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084931+0000","last_change":"2024-04-25T10:07:10.082178+0000","last_active":"2024-04-25T10:07:10.084931+0000","last_peered":"2024-04-25T10:07:10.084931+0000","last_clean":"2024-04-25T10:07:10.084931+0000","last_became_active":"2024-04-25T10:07:10.081804+0000","last_became_peered":"2024-04-25T10:07:10.081804+0000","last_unstale":"2024-04-25T10:07:10.084931+0000","last_undegraded":"2024-04-25T10:07:10.084931+0000","last_fullsized":"2024-04-25T10:07:10.084931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:23.179485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.090071+0000","last_change":"2024-04-25T10:07:10.088189+0000","last_active":"2024-04-25T10:07:10.090071+0000","last_peered":"2024-04-25T10:07:10.090071+0000","last_clean":"2024-04-25T10:07:10.090071+0000","last_became_active":"2024-04-25T10:07:10.086425+0000","last_became_peered":"2024-04-25T10:07:10.086425+0000","last_unstale":"2024-04-25T10:07:10.090071+0000","last_undegraded":"2024-04-25T10:07:10.090071+0000","last_fullsized":"2024-04-25T10:07:10.090071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:53.134581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.082979+0000","last_change":"2024-04-25T10:07:10.071980+0000","last_active":"2024-04-25T10:07:10.082979+0000","last_peered":"2024-04-25T10:07:10.082979+0000","last_clean":"2024-04-25T10:07:10.082979+0000","last_became_active":"2024-04-25T10:07:10.071021+0000","last_became_peered":"2024-04-25T10:07:10.071021+0000","last_unstale":"2024-04-25T10:07:10.082979+0000","last_undegraded":"2024-04-25T10:07:10.082979+0000","last_fullsized":"2024-04-25T10:07:10.082979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:02.540803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"76'4","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100484+0000","last_change":"2024-04-25T10:07:09.057740+0000","last_active":"2024-04-25T10:07:10.100484+0000","last_peered":"2024-04-25T10:07:10.100484+0000","last_clean":"2024-04-25T10:07:10.100484+0000","last_became_active":"2024-04-25T10:07:09.056840+0000","last_became_peered":"2024-04-25T10:07:09.056840+0000","last_unstale":"2024-04-25T10:07:10.100484+0000","last_undegraded":"2024-04-25T10:07:10.100484+0000","last_fullsized":"2024-04-25T10:07:10.100484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:35.289093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.081664+0000","last_change":"2024-04-25T10:07:10.072265+0000","last_active":"2024-04-25T10:07:10.081664+0000","last_peered":"2024-04-25T10:07:10.081664+0000","last_clean":"2024-04-25T10:07:10.081664+0000","last_became_active":"2024-04-25T10:07:10.071970+0000","last_became_peered":"2024-04-25T10:07:10.071970+0000","last_unstale":"2024-04-25T10:07:10.081664+0000","last_undegraded":"2024-04-25T10:07:10.081664+0000","last_fullsized":"2024-04-25T10:07:10.081664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:13.624610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.097870+0000","last_change":"2024-04-25T10:07:09.039541+0000","last_active":"2024-04-25T10:07:10.097870+0000","last_peered":"2024-04-25T10:07:10.097870+0000","last_clean":"2024-04-25T10:07:10.097870+0000","last_became_active":"2024-04-25T10:07:09.039236+0000","last_became_peered":"2024-04-25T10:07:09.039236+0000","last_unstale":"2024-04-25T10:07:10.097870+0000","last_undegraded":"2024-04-25T10:07:10.097870+0000","last_fullsized":"2024-04-25T10:07:10.097870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.673920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.086505+0000","last_change":"2024-04-25T10:07:10.080995+0000","last_active":"2024-04-25T10:07:10.086505+0000","last_peered":"2024-04-25T10:07:10.086505+0000","last_clean":"2024-04-25T10:07:10.086505+0000","last_became_active":"2024-04-25T10:07:10.080705+0000","last_became_peered":"2024-04-25T10:07:10.080705+0000","last_unstale":"2024-04-25T10:07:10.086505+0000","last_undegraded":"2024-04-25T10:07:10.086505+0000","last_fullsized":"2024-04-25T10:07:10.086505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:07.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.101100+0000","last_change":"2024-04-25T10:07:09.046617+0000","last_active":"2024-04-25T10:07:10.101100+0000","last_peered":"2024-04-25T10:07:10.101100+0000","last_clean":"2024-04-25T10:07:10.101100+0000","last_became_active":"2024-04-25T10:07:09.046309+0000","last_became_peered":"2024-04-25T10:07:09.046309+0000","last_unstale":"2024-04-25T10:07:10.101100+0000","last_undegraded":"2024-04-25T10:07:10.101100+0000","last_fullsized":"2024-04-25T10:07:10.101100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:00.398258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.087957+0000","last_change":"2024-04-25T10:07:10.083636+0000","last_active":"2024-04-25T10:07:10.087957+0000","last_peered":"2024-04-25T10:07:10.087957+0000","last_clean":"2024-04-25T10:07:10.087957+0000","last_became_active":"2024-04-25T10:07:10.083322+0000","last_became_peered":"2024-04-25T10:07:10.083322+0000","last_unstale":"2024-04-25T10:07:10.087957+0000","last_undegraded":"2024-04-25T10:07:10.087957+0000","last_fullsized":"2024-04-25T10:07:10.087957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:36.245370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100218+0000","last_change":"2024-04-25T10:07:09.043837+0000","last_active":"2024-04-25T10:07:10.100218+0000","last_peered":"2024-04-25T10:07:10.100218+0000","last_clean":"2024-04-25T10:07:10.100218+0000","last_became_active":"2024-04-25T10:07:09.043376+0000","last_became_peered":"2024-04-25T10:07:09.043376+0000","last_unstale":"2024-04-25T10:07:10.100218+0000","last_undegraded":"2024-04-25T10:07:10.100218+0000","last_fullsized":"2024-04-25T10:07:10.100218+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:22.165036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085575+0000","last_change":"2024-04-25T10:07:10.075160+0000","last_active":"2024-04-25T10:07:10.085575+0000","last_peered":"2024-04-25T10:07:10.085575+0000","last_clean":"2024-04-25T10:07:10.085575+0000","last_became_active":"2024-04-25T10:07:10.074485+0000","last_became_peered":"2024-04-25T10:07:10.074485+0000","last_unstale":"2024-04-25T10:07:10.085575+0000","last_undegraded":"2024-04-25T10:07:10.085575+0000","last_fullsized":"2024-04-25T10:07:10.085575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:30.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.098732+0000","last_change":"2024-04-25T10:07:09.050338+0000","last_active":"2024-04-25T10:07:10.098732+0000","last_peered":"2024-04-25T10:07:10.098732+0000","last_clean":"2024-04-25T10:07:10.098732+0000","last_became_active":"2024-04-25T10:07:09.050019+0000","last_became_peered":"2024-04-25T10:07:09.050019+0000","last_unstale":"2024-04-25T10:07:10.098732+0000","last_undegraded":"2024-04-25T10:07:10.098732+0000","last_fullsized":"2024-04-25T10:07:10.098732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:56.485451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.086593+0000","last_change":"2024-04-25T10:07:10.085290+0000","last_active":"2024-04-25T10:07:10.086593+0000","last_peered":"2024-04-25T10:07:10.086593+0000","last_clean":"2024-04-25T10:07:10.086593+0000","last_became_active":"2024-04-25T10:07:10.084955+0000","last_became_peered":"2024-04-25T10:07:10.084955+0000","last_unstale":"2024-04-25T10:07:10.086593+0000","last_undegraded":"2024-04-25T10:07:10.086593+0000","last_fullsized":"2024-04-25T10:07:10.086593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:35.127679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.101063+0000","last_change":"2024-04-25T10:07:09.048891+0000","last_active":"2024-04-25T10:07:10.101063+0000","last_peered":"2024-04-25T10:07:10.101063+0000","last_clean":"2024-04-25T10:07:10.101063+0000","last_became_active":"2024-04-25T10:07:09.048041+0000","last_became_peered":"2024-04-25T10:07:09.048041+0000","last_unstale":"2024-04-25T10:07:10.101063+0000","last_undegraded":"2024-04-25T10:07:10.101063+0000","last_fullsized":"2024-04-25T10:07:10.101063+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:20.939890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088836+0000","last_change":"2024-04-25T10:07:10.083571+0000","last_active":"2024-04-25T10:07:10.088836+0000","last_peered":"2024-04-25T10:07:10.088836+0000","last_clean":"2024-04-25T10:07:10.088836+0000","last_became_active":"2024-04-25T10:07:10.083272+0000","last_became_peered":"2024-04-25T10:07:10.083272+0000","last_unstale":"2024-04-25T10:07:10.088836+0000","last_undegraded":"2024-04-25T10:07:10.088836+0000","last_fullsized":"2024-04-25T10:07:10.088836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:18.353706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.046037+0000","last_change":"2024-04-25T10:07:09.038479+0000","last_active":"2024-04-25T10:07:09.046037+0000","last_peered":"2024-04-25T10:07:09.046037+0000","last_clean":"2024-04-25T10:07:09.046037+0000","last_became_active":"2024-04-25T10:07:09.038033+0000","last_became_peered":"2024-04-25T10:07:09.038033+0000","last_unstale":"2024-04-25T10:07:09.046037+0000","last_undegraded":"2024-04-25T10:07:09.046037+0000","last_fullsized":"2024-04-25T10:07:09.046037+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:05.939640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084048+0000","last_change":"2024-04-25T10:07:10.079450+0000","last_active":"2024-04-25T10:07:10.084048+0000","last_peered":"2024-04-25T10:07:10.084048+0000","last_clean":"2024-04-25T10:07:10.084048+0000","last_became_active":"2024-04-25T10:07:10.079094+0000","last_became_peered":"2024-04-25T10:07:10.079094+0000","last_unstale":"2024-04-25T10:07:10.084048+0000","last_undegraded":"2024-04-25T10:07:10.084048+0000","last_fullsized":"2024-04-25T10:07:10.084048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:47.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.066541+0000","last_change":"2024-04-25T10:07:09.053173+0000","last_active":"2024-04-25T10:07:10.066541+0000","last_peered":"2024-04-25T10:07:10.066541+0000","last_clean":"2024-04-25T10:07:10.066541+0000","last_became_active":"2024-04-25T10:07:09.051858+0000","last_became_peered":"2024-04-25T10:07:09.051858+0000","last_unstale":"2024-04-25T10:07:10.066541+0000","last_undegraded":"2024-04-25T10:07:10.066541+0000","last_fullsized":"2024-04-25T10:07:10.066541+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:53.657090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.081960+0000","last_change":"2024-04-25T10:07:10.077351+0000","last_active":"2024-04-25T10:07:10.081960+0000","last_peered":"2024-04-25T10:07:10.081960+0000","last_clean":"2024-04-25T10:07:10.081960+0000","last_became_active":"2024-04-25T10:07:10.077041+0000","last_became_peered":"2024-04-25T10:07:10.077041+0000","last_unstale":"2024-04-25T10:07:10.081960+0000","last_undegraded":"2024-04-25T10:07:10.081960+0000","last_fullsized":"2024-04-25T10:07:10.081960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:07.833001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.061161+0000","last_change":"2024-04-25T10:07:09.047435+0000","last_active":"2024-04-25T10:07:10.061161+0000","last_peered":"2024-04-25T10:07:10.061161+0000","last_clean":"2024-04-25T10:07:10.061161+0000","last_became_active":"2024-04-25T10:07:09.045179+0000","last_became_peered":"2024-04-25T10:07:09.045179+0000","last_unstale":"2024-04-25T10:07:10.061161+0000","last_undegraded":"2024-04-25T10:07:10.061161+0000","last_fullsized":"2024-04-25T10:07:10.061161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:00.398258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.078282+0000","last_change":"2024-04-25T10:07:10.072865+0000","last_active":"2024-04-25T10:07:10.078282+0000","last_peered":"2024-04-25T10:07:10.078282+0000","last_clean":"2024-04-25T10:07:10.078282+0000","last_became_active":"2024-04-25T10:07:10.072551+0000","last_became_peered":"2024-04-25T10:07:10.072551+0000","last_unstale":"2024-04-25T10:07:10.078282+0000","last_undegraded":"2024-04-25T10:07:10.078282+0000","last_fullsized":"2024-04-25T10:07:10.078282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:39.743425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.081582+0000","last_change":"2024-04-25T10:07:09.049382+0000","last_active":"2024-04-25T10:07:10.081582+0000","last_peered":"2024-04-25T10:07:10.081582+0000","last_clean":"2024-04-25T10:07:10.081582+0000","last_became_active":"2024-04-25T10:07:09.049076+0000","last_became_peered":"2024-04-25T10:07:09.049076+0000","last_unstale":"2024-04-25T10:07:10.081582+0000","last_undegraded":"2024-04-25T10:07:10.081582+0000","last_fullsized":"2024-04-25T10:07:10.081582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.835478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.025053+0000","last_change":"2024-04-25T10:07:09.025053+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.025053+0000","last_undegraded":"2024-04-25T10:07:09.025053+0000","last_fullsized":"2024-04-25T10:07:09.025053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"76'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.102626+0000","last_change":"2024-04-25T10:07:09.058380+0000","last_active":"2024-04-25T10:07:10.102626+0000","last_peered":"2024-04-25T10:07:10.102626+0000","last_clean":"2024-04-25T10:07:10.102626+0000","last_became_active":"2024-04-25T10:07:09.057710+0000","last_became_peered":"2024-04-25T10:07:09.057710+0000","last_unstale":"2024-04-25T10:07:10.102626+0000","last_undegraded":"2024-04-25T10:07:10.102626+0000","last_fullsized":"2024-04-25T10:07:10.102626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:40.116185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.087704+0000","last_change":"2024-04-25T10:07:10.082926+0000","last_active":"2024-04-25T10:07:10.087704+0000","last_peered":"2024-04-25T10:07:10.087704+0000","last_clean":"2024-04-25T10:07:10.087704+0000","last_became_active":"2024-04-25T10:07:10.082650+0000","last_became_peered":"2024-04-25T10:07:10.082650+0000","last_unstale":"2024-04-25T10:07:10.087704+0000","last_undegraded":"2024-04-25T10:07:10.087704+0000","last_fullsized":"2024-04-25T10:07:10.087704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:10.757549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.103879+0000","last_change":"2024-04-25T10:07:09.043553+0000","last_active":"2024-04-25T10:07:10.103879+0000","last_peered":"2024-04-25T10:07:10.103879+0000","last_clean":"2024-04-25T10:07:10.103879+0000","last_became_active":"2024-04-25T10:07:09.043250+0000","last_became_peered":"2024-04-25T10:07:09.043250+0000","last_unstale":"2024-04-25T10:07:10.103879+0000","last_undegraded":"2024-04-25T10:07:10.103879+0000","last_fullsized":"2024-04-25T10:07:10.103879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.516166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.073990+0000","last_change":"2024-04-25T10:07:10.069910+0000","last_active":"2024-04-25T10:07:10.073990+0000","last_peered":"2024-04-25T10:07:10.073990+0000","last_clean":"2024-04-25T10:07:10.073990+0000","last_became_active":"2024-04-25T10:07:10.069580+0000","last_became_peered":"2024-04-25T10:07:10.069580+0000","last_unstale":"2024-04-25T10:07:10.073990+0000","last_undegraded":"2024-04-25T10:07:10.073990+0000","last_fullsized":"2024-04-25T10:07:10.073990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:49.849927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100477+0000","last_change":"2024-04-25T10:07:09.043457+0000","last_active":"2024-04-25T10:07:10.100477+0000","last_peered":"2024-04-25T10:07:10.100477+0000","last_clean":"2024-04-25T10:07:10.100477+0000","last_became_active":"2024-04-25T10:07:09.043147+0000","last_became_peered":"2024-04-25T10:07:09.043147+0000","last_unstale":"2024-04-25T10:07:10.100477+0000","last_undegraded":"2024-04-25T10:07:10.100477+0000","last_fullsized":"2024-04-25T10:07:10.100477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.622257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088534+0000","last_change":"2024-04-25T10:07:10.082950+0000","last_active":"2024-04-25T10:07:10.088534+0000","last_peered":"2024-04-25T10:07:10.088534+0000","last_clean":"2024-04-25T10:07:10.088534+0000","last_became_active":"2024-04-25T10:07:10.082473+0000","last_became_peered":"2024-04-25T10:07:10.082473+0000","last_unstale":"2024-04-25T10:07:10.088534+0000","last_undegraded":"2024-04-25T10:07:10.088534+0000","last_fullsized":"2024-04-25T10:07:10.088534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:38.232892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.062370+0000","last_change":"2024-04-25T10:07:09.044513+0000","last_active":"2024-04-25T10:07:10.062370+0000","last_peered":"2024-04-25T10:07:10.062370+0000","last_clean":"2024-04-25T10:07:10.062370+0000","last_became_active":"2024-04-25T10:07:09.044177+0000","last_became_peered":"2024-04-25T10:07:09.044177+0000","last_unstale":"2024-04-25T10:07:10.062370+0000","last_undegraded":"2024-04-25T10:07:10.062370+0000","last_fullsized":"2024-04-25T10:07:10.062370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:34.113230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.077696+0000","last_change":"2024-04-25T10:07:10.072284+0000","last_active":"2024-04-25T10:07:10.077696+0000","last_peered":"2024-04-25T10:07:10.077696+0000","last_clean":"2024-04-25T10:07:10.077696+0000","last_became_active":"2024-04-25T10:07:10.071957+0000","last_became_peered":"2024-04-25T10:07:10.071957+0000","last_unstale":"2024-04-25T10:07:10.077696+0000","last_undegraded":"2024-04-25T10:07:10.077696+0000","last_fullsized":"2024-04-25T10:07:10.077696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:16.711444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.054994+0000","last_change":"2024-04-25T10:07:09.057016+0000","last_active":"2024-04-25T10:07:10.054994+0000","last_peered":"2024-04-25T10:07:10.054994+0000","last_clean":"2024-04-25T10:07:10.054994+0000","last_became_active":"2024-04-25T10:07:09.056737+0000","last_became_peered":"2024-04-25T10:07:09.056737+0000","last_unstale":"2024-04-25T10:07:10.054994+0000","last_undegraded":"2024-04-25T10:07:10.054994+0000","last_fullsized":"2024-04-25T10:07:10.054994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:21.569196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.093671+0000","last_change":"2024-04-25T10:07:10.091116+0000","last_active":"2024-04-25T10:07:10.093671+0000","last_peered":"2024-04-25T10:07:10.093671+0000","last_clean":"2024-04-25T10:07:10.093671+0000","last_became_active":"2024-04-25T10:07:10.090279+0000","last_became_peered":"2024-04-25T10:07:10.090279+0000","last_unstale":"2024-04-25T10:07:10.093671+0000","last_undegraded":"2024-04-25T10:07:10.093671+0000","last_fullsized":"2024-04-25T10:07:10.093671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:02.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.103876+0000","last_change":"2024-04-25T10:07:09.045532+0000","last_active":"2024-04-25T10:07:10.103876+0000","last_peered":"2024-04-25T10:07:10.103876+0000","last_clean":"2024-04-25T10:07:10.103876+0000","last_became_active":"2024-04-25T10:07:09.045053+0000","last_became_peered":"2024-04-25T10:07:09.045053+0000","last_unstale":"2024-04-25T10:07:10.103876+0000","last_undegraded":"2024-04-25T10:07:10.103876+0000","last_fullsized":"2024-04-25T10:07:10.103876+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:15.696995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084883+0000","last_change":"2024-04-25T10:07:10.079838+0000","last_active":"2024-04-25T10:07:10.084883+0000","last_peered":"2024-04-25T10:07:10.084883+0000","last_clean":"2024-04-25T10:07:10.084883+0000","last_became_active":"2024-04-25T10:07:10.079580+0000","last_became_peered":"2024-04-25T10:07:10.079580+0000","last_unstale":"2024-04-25T10:07:10.084883+0000","last_undegraded":"2024-04-25T10:07:10.084883+0000","last_fullsized":"2024-04-25T10:07:10.084883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:29.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100869+0000","last_change":"2024-04-25T10:07:09.038313+0000","last_active":"2024-04-25T10:07:10.100869+0000","last_peered":"2024-04-25T10:07:10.100869+0000","last_clean":"2024-04-25T10:07:10.100869+0000","last_became_active":"2024-04-25T10:07:09.038062+0000","last_became_peered":"2024-04-25T10:07:09.038062+0000","last_unstale":"2024-04-25T10:07:10.100869+0000","last_undegraded":"2024-04-25T10:07:10.100869+0000","last_fullsized":"2024-04-25T10:07:10.100869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:27.873961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088596+0000","last_change":"2024-04-25T10:07:10.085796+0000","last_active":"2024-04-25T10:07:10.088596+0000","last_peered":"2024-04-25T10:07:10.088596+0000","last_clean":"2024-04-25T10:07:10.088596+0000","last_became_active":"2024-04-25T10:07:10.085470+0000","last_became_peered":"2024-04-25T10:07:10.085470+0000","last_unstale":"2024-04-25T10:07:10.088596+0000","last_undegraded":"2024-04-25T10:07:10.088596+0000","last_fullsized":"2024-04-25T10:07:10.088596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:41.130634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100261+0000","last_change":"2024-04-25T10:07:09.051576+0000","last_active":"2024-04-25T10:07:10.100261+0000","last_peered":"2024-04-25T10:07:10.100261+0000","last_clean":"2024-04-25T10:07:10.100261+0000","last_became_active":"2024-04-25T10:07:09.051286+0000","last_became_peered":"2024-04-25T10:07:09.051286+0000","last_unstale":"2024-04-25T10:07:10.100261+0000","last_undegraded":"2024-04-25T10:07:10.100261+0000","last_fullsized":"2024-04-25T10:07:10.100261+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:28.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.028289+0000","last_change":"2024-04-25T10:07:09.028289+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.028289+0000","last_undegraded":"2024-04-25T10:07:09.028289+0000","last_fullsized":"2024-04-25T10:07:09.028289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.094042+0000","last_change":"2024-04-25T10:07:09.048764+0000","last_active":"2024-04-25T10:07:10.094042+0000","last_peered":"2024-04-25T10:07:10.094042+0000","last_clean":"2024-04-25T10:07:10.094042+0000","last_became_active":"2024-04-25T10:07:09.048491+0000","last_became_peered":"2024-04-25T10:07:09.048491+0000","last_unstale":"2024-04-25T10:07:10.094042+0000","last_undegraded":"2024-04-25T10:07:10.094042+0000","last_fullsized":"2024-04-25T10:07:10.094042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:34.578041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.082935+0000","last_change":"2024-04-25T10:07:10.071859+0000","last_active":"2024-04-25T10:07:10.082935+0000","last_peered":"2024-04-25T10:07:10.082935+0000","last_clean":"2024-04-25T10:07:10.082935+0000","last_became_active":"2024-04-25T10:07:10.069819+0000","last_became_peered":"2024-04-25T10:07:10.069819+0000","last_unstale":"2024-04-25T10:07:10.082935+0000","last_undegraded":"2024-04-25T10:07:10.082935+0000","last_fullsized":"2024-04-25T10:07:10.082935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:52.639578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.054495+0000","last_change":"2024-04-25T10:07:09.051328+0000","last_active":"2024-04-25T10:07:10.054495+0000","last_peered":"2024-04-25T10:07:10.054495+0000","last_clean":"2024-04-25T10:07:10.054495+0000","last_became_active":"2024-04-25T10:07:09.051018+0000","last_became_peered":"2024-04-25T10:07:09.051018+0000","last_unstale":"2024-04-25T10:07:10.054495+0000","last_undegraded":"2024-04-25T10:07:10.054495+0000","last_fullsized":"2024-04-25T10:07:10.054495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:21.155836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084308+0000","last_change":"2024-04-25T10:07:10.073820+0000","last_active":"2024-04-25T10:07:10.084308+0000","last_peered":"2024-04-25T10:07:10.084308+0000","last_clean":"2024-04-25T10:07:10.084308+0000","last_became_active":"2024-04-25T10:07:10.073492+0000","last_became_peered":"2024-04-25T10:07:10.073492+0000","last_unstale":"2024-04-25T10:07:10.084308+0000","last_undegraded":"2024-04-25T10:07:10.084308+0000","last_fullsized":"2024-04-25T10:07:10.084308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:59.728574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.099553+0000","last_change":"2024-04-25T10:07:09.055663+0000","last_active":"2024-04-25T10:07:10.099553+0000","last_peered":"2024-04-25T10:07:10.099553+0000","last_clean":"2024-04-25T10:07:10.099553+0000","last_became_active":"2024-04-25T10:07:09.055287+0000","last_became_peered":"2024-04-25T10:07:09.055287+0000","last_unstale":"2024-04-25T10:07:10.099553+0000","last_undegraded":"2024-04-25T10:07:10.099553+0000","last_fullsized":"2024-04-25T10:07:10.099553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:26.416130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084564+0000","last_change":"2024-04-25T10:07:10.073873+0000","last_active":"2024-04-25T10:07:10.084564+0000","last_peered":"2024-04-25T10:07:10.084564+0000","last_clean":"2024-04-25T10:07:10.084564+0000","last_became_active":"2024-04-25T10:07:10.073054+0000","last_became_peered":"2024-04-25T10:07:10.073054+0000","last_unstale":"2024-04-25T10:07:10.084564+0000","last_undegraded":"2024-04-25T10:07:10.084564+0000","last_fullsized":"2024-04-25T10:07:10.084564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:47.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T10:07:09.052196+0000","last_change":"2024-04-25T10:07:09.048404+0000","last_active":"2024-04-25T10:07:09.052196+0000","last_peered":"2024-04-25T10:07:09.052196+0000","last_clean":"2024-04-25T10:07:09.052196+0000","last_became_active":"2024-04-25T10:07:09.047800+0000","last_became_peered":"2024-04-25T10:07:09.047800+0000","last_unstale":"2024-04-25T10:07:09.052196+0000","last_undegraded":"2024-04-25T10:07:09.052196+0000","last_fullsized":"2024-04-25T10:07:09.052196+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:34.765067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.074128+0000","last_change":"2024-04-25T10:07:10.072289+0000","last_active":"2024-04-25T10:07:10.074128+0000","last_peered":"2024-04-25T10:07:10.074128+0000","last_clean":"2024-04-25T10:07:10.074128+0000","last_became_active":"2024-04-25T10:07:10.071949+0000","last_became_peered":"2024-04-25T10:07:10.071949+0000","last_unstale":"2024-04-25T10:07:10.074128+0000","last_undegraded":"2024-04-25T10:07:10.074128+0000","last_fullsized":"2024-04-25T10:07:10.074128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:22.193310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.070440+0000","last_change":"2024-04-25T10:07:09.053836+0000","last_active":"2024-04-25T10:07:10.070440+0000","last_peered":"2024-04-25T10:07:10.070440+0000","last_clean":"2024-04-25T10:07:10.070440+0000","last_became_active":"2024-04-25T10:07:09.053519+0000","last_became_peered":"2024-04-25T10:07:09.053519+0000","last_unstale":"2024-04-25T10:07:10.070440+0000","last_undegraded":"2024-04-25T10:07:10.070440+0000","last_fullsized":"2024-04-25T10:07:10.070440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:08.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.077454+0000","last_change":"2024-04-25T10:07:10.073960+0000","last_active":"2024-04-25T10:07:10.077454+0000","last_peered":"2024-04-25T10:07:10.077454+0000","last_clean":"2024-04-25T10:07:10.077454+0000","last_became_active":"2024-04-25T10:07:10.073580+0000","last_became_peered":"2024-04-25T10:07:10.073580+0000","last_unstale":"2024-04-25T10:07:10.077454+0000","last_undegraded":"2024-04-25T10:07:10.077454+0000","last_fullsized":"2024-04-25T10:07:10.077454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:38.232892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100089+0000","last_change":"2024-04-25T10:07:09.055929+0000","last_active":"2024-04-25T10:07:10.100089+0000","last_peered":"2024-04-25T10:07:10.100089+0000","last_clean":"2024-04-25T10:07:10.100089+0000","last_became_active":"2024-04-25T10:07:09.055641+0000","last_became_peered":"2024-04-25T10:07:09.055641+0000","last_unstale":"2024-04-25T10:07:10.100089+0000","last_undegraded":"2024-04-25T10:07:10.100089+0000","last_fullsized":"2024-04-25T10:07:10.100089+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:27.873961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089041+0000","last_change":"2024-04-25T10:07:10.084922+0000","last_active":"2024-04-25T10:07:10.089041+0000","last_peered":"2024-04-25T10:07:10.089041+0000","last_clean":"2024-04-25T10:07:10.089041+0000","last_became_active":"2024-04-25T10:07:10.084350+0000","last_became_peered":"2024-04-25T10:07:10.084350+0000","last_unstale":"2024-04-25T10:07:10.089041+0000","last_undegraded":"2024-04-25T10:07:10.089041+0000","last_fullsized":"2024-04-25T10:07:10.089041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:32.882689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.049360+0000","last_change":"2024-04-25T10:07:09.056568+0000","last_active":"2024-04-25T10:07:10.049360+0000","last_peered":"2024-04-25T10:07:10.049360+0000","last_clean":"2024-04-25T10:07:10.049360+0000","last_became_active":"2024-04-25T10:07:09.056255+0000","last_became_peered":"2024-04-25T10:07:09.056255+0000","last_unstale":"2024-04-25T10:07:10.049360+0000","last_undegraded":"2024-04-25T10:07:10.049360+0000","last_fullsized":"2024-04-25T10:07:10.049360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:06.416705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084490+0000","last_change":"2024-04-25T10:07:10.079361+0000","last_active":"2024-04-25T10:07:10.084490+0000","last_peered":"2024-04-25T10:07:10.084490+0000","last_clean":"2024-04-25T10:07:10.084490+0000","last_became_active":"2024-04-25T10:07:10.078539+0000","last_became_peered":"2024-04-25T10:07:10.078539+0000","last_unstale":"2024-04-25T10:07:10.084490+0000","last_undegraded":"2024-04-25T10:07:10.084490+0000","last_fullsized":"2024-04-25T10:07:10.084490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:07.702579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.078199+0000","last_change":"2024-04-25T10:07:09.045936+0000","last_active":"2024-04-25T10:07:10.078199+0000","last_peered":"2024-04-25T10:07:10.078199+0000","last_clean":"2024-04-25T10:07:10.078199+0000","last_became_active":"2024-04-25T10:07:09.042881+0000","last_became_peered":"2024-04-25T10:07:09.042881+0000","last_unstale":"2024-04-25T10:07:10.078199+0000","last_undegraded":"2024-04-25T10:07:10.078199+0000","last_fullsized":"2024-04-25T10:07:10.078199+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:28.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":75,"state":"creating+peering","last_fresh":"2024-04-25T10:07:09.025061+0000","last_change":"2024-04-25T10:07:09.025061+0000","last_active":"2024-04-25T10:07:09.008844+0000","last_peered":"2024-04-25T10:07:09.008844+0000","last_clean":"2024-04-25T10:07:09.008844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:07:09.025061+0000","last_undegraded":"2024-04-25T10:07:09.025061+0000","last_fullsized":"2024-04-25T10:07:09.025061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.060664+0000","last_change":"2024-04-25T10:07:09.044733+0000","last_active":"2024-04-25T10:07:10.060664+0000","last_peered":"2024-04-25T10:07:10.060664+0000","last_clean":"2024-04-25T10:07:10.060664+0000","last_became_active":"2024-04-25T10:07:09.042478+0000","last_became_peered":"2024-04-25T10:07:09.042478+0000","last_unstale":"2024-04-25T10:07:10.060664+0000","last_undegraded":"2024-04-25T10:07:10.060664+0000","last_fullsized":"2024-04-25T10:07:10.060664+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:15.639435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088231+0000","last_change":"2024-04-25T10:07:10.083591+0000","last_active":"2024-04-25T10:07:10.088231+0000","last_peered":"2024-04-25T10:07:10.088231+0000","last_clean":"2024-04-25T10:07:10.088231+0000","last_became_active":"2024-04-25T10:07:10.083297+0000","last_became_peered":"2024-04-25T10:07:10.083297+0000","last_unstale":"2024-04-25T10:07:10.088231+0000","last_undegraded":"2024-04-25T10:07:10.088231+0000","last_fullsized":"2024-04-25T10:07:10.088231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:30.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.045529+0000","last_change":"2024-04-25T10:07:09.039347+0000","last_active":"2024-04-25T10:07:10.045529+0000","last_peered":"2024-04-25T10:07:10.045529+0000","last_clean":"2024-04-25T10:07:10.045529+0000","last_became_active":"2024-04-25T10:07:09.039011+0000","last_became_peered":"2024-04-25T10:07:09.039011+0000","last_unstale":"2024-04-25T10:07:10.045529+0000","last_undegraded":"2024-04-25T10:07:10.045529+0000","last_fullsized":"2024-04-25T10:07:10.045529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:29.697295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.082850+0000","last_change":"2024-04-25T10:07:10.081350+0000","last_active":"2024-04-25T10:07:10.082850+0000","last_peered":"2024-04-25T10:07:10.082850+0000","last_clean":"2024-04-25T10:07:10.082850+0000","last_became_active":"2024-04-25T10:07:10.081055+0000","last_became_peered":"2024-04-25T10:07:10.081055+0000","last_unstale":"2024-04-25T10:07:10.082850+0000","last_undegraded":"2024-04-25T10:07:10.082850+0000","last_fullsized":"2024-04-25T10:07:10.082850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.246741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.049007+0000","last_change":"2024-04-25T10:07:09.050041+0000","last_active":"2024-04-25T10:07:10.049007+0000","last_peered":"2024-04-25T10:07:10.049007+0000","last_clean":"2024-04-25T10:07:10.049007+0000","last_became_active":"2024-04-25T10:07:09.048053+0000","last_became_peered":"2024-04-25T10:07:09.048053+0000","last_unstale":"2024-04-25T10:07:10.049007+0000","last_undegraded":"2024-04-25T10:07:10.049007+0000","last_fullsized":"2024-04-25T10:07:10.049007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:06.688130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.097478+0000","last_change":"2024-04-25T10:07:10.093694+0000","last_active":"2024-04-25T10:07:10.097478+0000","last_peered":"2024-04-25T10:07:10.097478+0000","last_clean":"2024-04-25T10:07:10.097478+0000","last_became_active":"2024-04-25T10:07:10.093426+0000","last_became_peered":"2024-04-25T10:07:10.093426+0000","last_unstale":"2024-04-25T10:07:10.097478+0000","last_undegraded":"2024-04-25T10:07:10.097478+0000","last_fullsized":"2024-04-25T10:07:10.097478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:18.353706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.056204+0000","last_change":"2024-04-25T10:07:09.038288+0000","last_active":"2024-04-25T10:07:10.056204+0000","last_peered":"2024-04-25T10:07:10.056204+0000","last_clean":"2024-04-25T10:07:10.056204+0000","last_became_active":"2024-04-25T10:07:09.038001+0000","last_became_peered":"2024-04-25T10:07:09.038001+0000","last_unstale":"2024-04-25T10:07:10.056204+0000","last_undegraded":"2024-04-25T10:07:10.056204+0000","last_fullsized":"2024-04-25T10:07:10.056204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:31.188373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.083061+0000","last_change":"2024-04-25T10:07:10.073504+0000","last_active":"2024-04-25T10:07:10.083061+0000","last_peered":"2024-04-25T10:07:10.083061+0000","last_clean":"2024-04-25T10:07:10.083061+0000","last_became_active":"2024-04-25T10:07:10.072731+0000","last_became_peered":"2024-04-25T10:07:10.072731+0000","last_unstale":"2024-04-25T10:07:10.083061+0000","last_undegraded":"2024-04-25T10:07:10.083061+0000","last_fullsized":"2024-04-25T10:07:10.083061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:35.580788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.067102+0000","last_change":"2024-04-25T10:07:09.052901+0000","last_active":"2024-04-25T10:07:10.067102+0000","last_peered":"2024-04-25T10:07:10.067102+0000","last_clean":"2024-04-25T10:07:10.067102+0000","last_became_active":"2024-04-25T10:07:09.051470+0000","last_became_peered":"2024-04-25T10:07:09.051470+0000","last_unstale":"2024-04-25T10:07:10.067102+0000","last_undegraded":"2024-04-25T10:07:10.067102+0000","last_fullsized":"2024-04-25T10:07:10.067102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:29.901785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.069777+0000","last_change":"2024-04-25T10:07:10.059287+0000","last_active":"2024-04-25T10:07:10.069777+0000","last_peered":"2024-04-25T10:07:10.069777+0000","last_clean":"2024-04-25T10:07:10.069777+0000","last_became_active":"2024-04-25T10:07:10.058491+0000","last_became_peered":"2024-04-25T10:07:10.058491+0000","last_unstale":"2024-04-25T10:07:10.069777+0000","last_undegraded":"2024-04-25T10:07:10.069777+0000","last_fullsized":"2024-04-25T10:07:10.069777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:49.133479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.048625+0000","last_change":"2024-04-25T10:07:09.052332+0000","last_active":"2024-04-25T10:07:10.048625+0000","last_peered":"2024-04-25T10:07:10.048625+0000","last_clean":"2024-04-25T10:07:10.048625+0000","last_became_active":"2024-04-25T10:07:09.051372+0000","last_became_peered":"2024-04-25T10:07:09.051372+0000","last_unstale":"2024-04-25T10:07:10.048625+0000","last_undegraded":"2024-04-25T10:07:10.048625+0000","last_fullsized":"2024-04-25T10:07:10.048625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:38.728976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":271,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.075313+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:07:10.075313+0000","last_peered":"2024-04-25T10:07:10.075313+0000","last_clean":"2024-04-25T10:07:10.075313+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:07:10.075313+0000","last_undegraded":"2024-04-25T10:07:10.075313+0000","last_fullsized":"2024-04-25T10:07:10.075313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509507,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":417554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":417574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509508,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":1007360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":409735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":416902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1531904,"data_stored":1008916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":409735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":909312,"data_stored":409735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:14.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.787+0000 7f36b9c00640 1 Processor -- start 2024-04-25T10:07:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.787+0000 7f36b9c00640 1 -- start start 2024-04-25T10:07:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.789+0000 7f36b9c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36b4071ed0 0x7f36b40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.789+0000 7f36b9c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b415bdc0 con 0x7f36b41529e0 2024-04-25T10:07:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.789+0000 7f36b9c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b415c060 con 0x7f36b40ce890 2024-04-25T10:07:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.789+0000 7f36b9c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b40b2340 con 0x7f36b4071ed0 2024-04-25T10:07:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.790+0000 7f36b2a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36b41529e0 0x7f36b40c5ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54956/0 (socket says 172.21.15.5:54956) 2024-04-25T10:07:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.790+0000 7f36b2a00640 1 -- 172.21.15.5:0/2093131688 learned_addr learned my addr 172.21.15.5:0/2093131688 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.790+0000 7f36b2000640 1 --2- 172.21.15.5:0/2093131688 >> v2:172.21.15.5:3300/0 conn(0x7f36b4071ed0 0x7f36b40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.790+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749952554 0 0) 0x7f36b415bdc0 con 0x7f36b41529e0 2024-04-25T10:07:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.790+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3698003520 con 0x7f36b41529e0 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996619847 0 0) 0x7f3698003520 con 0x7f36b41529e0 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 >> v2:172.21.15.5:3300/0 conn(0x7f36b4071ed0 msgr2=0x7f36b40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 --2- 172.21.15.5:0/2093131688 >> v2:172.21.15.5:3300/0 conn(0x7f36b4071ed0 0x7f36b40ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 >> v1:172.21.15.80:6789/0 conn(0x7f36b40ce890 legacy=0x7f36b406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b415c340 con 0x7f36b41529e0 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f36a4001690 con 0x7f36b41529e0 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 >> v1:172.21.15.5:6789/0 conn(0x7f36b41529e0 legacy=0x7f36b40c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3698003a00 0x7f3698003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3698004800 0x7f3698004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36980051c0 0x7f369800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40b2340 con 0x7f3698004800 2024-04-25T10:07:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369800c0c0 con 0x7f36980051c0 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369800c2c0 con 0x7f3698003a00 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.791+0000 7f36b2000640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3698004800 0x7f3698004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b3400640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36980051c0 0x7f369800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b2a00640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3698003a00 0x7f3698003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b2000640 1 -- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3698003a00 msgr2=0x7f3698003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b2000640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3698003a00 0x7f3698003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b2000640 1 -- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36980051c0 msgr2=0x7f369800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b2000640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36980051c0 0x7f369800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b3400640 1 -- 172.21.15.5:0/2093131688 reap_dead start 2024-04-25T10:07:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.792+0000 7f36b2000640 1 -- 172.21.15.5:0/2093131688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3698003520 con 0x7f3698004800 2024-04-25T10:07:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.795+0000 7f36b2000640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3698004800 0x7f3698004c80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009a90 tx=0x7f3698005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4878a49c2f1cba07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.795+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0036070 con 0x7f3698004800 2024-04-25T10:07:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.795+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0031400 con 0x7f3698004800 2024-04-25T10:07:14.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.795+0000 7f36b1600640 1 -- 172.21.15.5:0/2093131688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0009690 con 0x7f3698004800 2024-04-25T10:07:14.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 -- 172.21.15.5:0/2093131688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3698004800 msgr2=0x7f3698004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3698004800 0x7f3698004c80 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009a90 tx=0x7f3698005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 -- 172.21.15.5:0/2093131688 shutdown_connections 2024-04-25T10:07:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 --2- 172.21.15.5:0/2093131688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3698004800 0x7f3698004c80 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 -- 172.21.15.5:0/2093131688 >> 172.21.15.5:0/2093131688 conn(0x7f36b4071150 msgr2=0x7f36b4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 -- 172.21.15.5:0/2093131688 shutdown_connections 2024-04-25T10:07:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.796+0000 7f36b9c00640 1 -- 172.21.15.5:0/2093131688 wait complete. 2024-04-25T10:07:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.798+0000 7f36b9c00640 1 Processor -- start 2024-04-25T10:07:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.798+0000 7f36b9c00640 1 -- start start 2024-04-25T10:07:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b9c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3698004800 0x7f36b4150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 0x7f36b4158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b9c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36b4144ca0 0x7f36b41f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b9c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b415c900 con 0x7f36b414f2a0 2024-04-25T10:07:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b9c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40cecc0 con 0x7f3698004800 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b9c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40ce350 con 0x7f36b4144ca0 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 0x7f36b4158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 0x7f36b4158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58598/0 (socket says 172.21.15.5:58598) 2024-04-25T10:07:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 -- 172.21.15.5:0/204420916 learned_addr learned my addr 172.21.15.5:0/204420916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2a00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3698004800 0x7f36b4150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b3400640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36b4144ca0 0x7f36b41f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 -- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36b4144ca0 msgr2=0x7f36b41f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36b4144ca0 0x7f36b41f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 -- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3698004800 msgr2=0x7f36b4150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3698004800 0x7f36b4150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.799+0000 7f36b2000640 1 -- 172.21.15.5:0/204420916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b414f720 con 0x7f36b414f2a0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36b2a00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3698004800 0x7f36b4150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36b2000640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 0x7f36b4158ca0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f36a0034040 tx=0x7f36a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a003f020 con 0x7f36b414f2a0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0037040 con 0x7f36b414f2a0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b414f9f0 con 0x7f36b414f2a0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0036070 con 0x7f36b414f2a0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.800+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b414ffb0 con 0x7f36b414f2a0 2024-04-25T10:07:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.801+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b4068fd0 con 0x7f36b414f2a0 2024-04-25T10:07:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.802+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36a0042020 con 0x7f36b414f2a0 2024-04-25T10:07:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.803+0000 7f36aaa00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369406b500 0x7f369406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.803+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36a00aa1e0 con 0x7f36b414f2a0 2024-04-25T10:07:14.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.804+0000 7f36b2a00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369406b500 0x7f369406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.805+0000 7f36b2a00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369406b500 0x7f369406d9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f36b4050700 tx=0x7f36a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.808+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36a0049070 con 0x7f36b414f2a0 2024-04-25T10:07:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:14.998+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36b415af90 con 0x7f369406b500 2024-04-25T10:07:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.017+0000 7f36aaa00640 1 -- 172.21.15.5:0/204420916 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252008 (secure 0 0 0) 0x7f36b415af90 con 0x7f369406b500 2024-04-25T10:07:15.026 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:15.027 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:07:15.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369406b500 msgr2=0x7f369406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369406b500 0x7f369406d9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f36b4050700 tx=0x7f36a4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 msgr2=0x7f36b4158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 0x7f36b4158ca0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f36a0034040 tx=0x7f36a0004730 comp rx=0 tx=0).stop 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 shutdown_connections 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36b4144ca0 0x7f36b41f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36b414f2a0 0x7f36b4158ca0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369406b500 0x7f369406d9c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 --2- 172.21.15.5:0/204420916 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3698004800 0x7f36b4150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 >> 172.21.15.5:0/204420916 conn(0x7f36b4071150 msgr2=0x7f36b406d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 shutdown_connections 2024-04-25T10:07:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.022+0000 7f36b9c00640 1 -- 172.21.15.5:0/204420916 wait complete. 2024-04-25T10:07:15.046 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-04-25T10:07:14.729113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253684,"kb_used_data":8388,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721036,"statfs":{"total":773094113280,"available":772834340864,"internally_reserved":0,"allocated":8589312,"data_stored":4511392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002157"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.092382+0000","last_change":"2024-04-25T10:07:10.090709+0000","last_active":"2024-04-25T10:07:10.092382+0000","last_peered":"2024-04-25T10:07:10.092382+0000","last_clean":"2024-04-25T10:07:10.092382+0000","last_became_active":"2024-04-25T10:07:10.089186+0000","last_became_peered":"2024-04-25T10:07:10.089186+0000","last_unstale":"2024-04-25T10:07:10.092382+0000","last_undegraded":"2024-04-25T10:07:10.092382+0000","last_fullsized":"2024-04-25T10:07:10.092382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.803884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.098597+0000","last_change":"2024-04-25T10:07:10.095329+0000","last_active":"2024-04-25T10:07:10.098597+0000","last_peered":"2024-04-25T10:07:10.098597+0000","last_clean":"2024-04-25T10:07:10.098597+0000","last_became_active":"2024-04-25T10:07:10.095016+0000","last_became_peered":"2024-04-25T10:07:10.095016+0000","last_unstale":"2024-04-25T10:07:10.098597+0000","last_undegraded":"2024-04-25T10:07:10.098597+0000","last_fullsized":"2024-04-25T10:07:10.098597+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:30.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.092235+0000","last_change":"2024-04-25T10:07:10.087385+0000","last_active":"2024-04-25T10:07:10.092235+0000","last_peered":"2024-04-25T10:07:10.092235+0000","last_clean":"2024-04-25T10:07:10.092235+0000","last_became_active":"2024-04-25T10:07:10.087033+0000","last_became_peered":"2024-04-25T10:07:10.087033+0000","last_unstale":"2024-04-25T10:07:10.092235+0000","last_undegraded":"2024-04-25T10:07:10.092235+0000","last_fullsized":"2024-04-25T10:07:10.092235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:07.702579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.073968+0000","last_change":"2024-04-25T10:07:10.069285+0000","last_active":"2024-04-25T10:07:10.073968+0000","last_peered":"2024-04-25T10:07:10.073968+0000","last_clean":"2024-04-25T10:07:10.073968+0000","last_became_active":"2024-04-25T10:07:10.068958+0000","last_became_peered":"2024-04-25T10:07:10.068958+0000","last_unstale":"2024-04-25T10:07:10.073968+0000","last_undegraded":"2024-04-25T10:07:10.073968+0000","last_fullsized":"2024-04-25T10:07:10.073968+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.636706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085618+0000","last_change":"2024-04-25T10:07:10.078013+0000","last_active":"2024-04-25T10:07:10.085618+0000","last_peered":"2024-04-25T10:07:10.085618+0000","last_clean":"2024-04-25T10:07:10.085618+0000","last_became_active":"2024-04-25T10:07:10.076185+0000","last_became_peered":"2024-04-25T10:07:10.076185+0000","last_unstale":"2024-04-25T10:07:10.085618+0000","last_undegraded":"2024-04-25T10:07:10.085618+0000","last_fullsized":"2024-04-25T10:07:10.085618+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:16.653884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.095988+0000","last_change":"2024-04-25T10:07:10.093710+0000","last_active":"2024-04-25T10:07:10.095988+0000","last_peered":"2024-04-25T10:07:10.095988+0000","last_clean":"2024-04-25T10:07:10.095988+0000","last_became_active":"2024-04-25T10:07:10.093372+0000","last_became_peered":"2024-04-25T10:07:10.093372+0000","last_unstale":"2024-04-25T10:07:10.095988+0000","last_undegraded":"2024-04-25T10:07:10.095988+0000","last_fullsized":"2024-04-25T10:07:10.095988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:34.955739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084999+0000","last_change":"2024-04-25T10:07:10.083499+0000","last_active":"2024-04-25T10:07:10.084999+0000","last_peered":"2024-04-25T10:07:10.084999+0000","last_clean":"2024-04-25T10:07:10.084999+0000","last_became_active":"2024-04-25T10:07:10.083202+0000","last_became_peered":"2024-04-25T10:07:10.083202+0000","last_unstale":"2024-04-25T10:07:10.084999+0000","last_undegraded":"2024-04-25T10:07:10.084999+0000","last_fullsized":"2024-04-25T10:07:10.084999+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:02.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.050902+0000","last_change":"2024-04-25T10:07:10.074174+0000","last_active":"2024-04-25T10:07:11.050902+0000","last_peered":"2024-04-25T10:07:11.050902+0000","last_clean":"2024-04-25T10:07:11.050902+0000","last_became_active":"2024-04-25T10:07:10.073896+0000","last_became_peered":"2024-04-25T10:07:10.073896+0000","last_unstale":"2024-04-25T10:07:11.050902+0000","last_undegraded":"2024-04-25T10:07:11.050902+0000","last_fullsized":"2024-04-25T10:07:11.050902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:31.271570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.078272+0000","last_change":"2024-04-25T10:07:10.075069+0000","last_active":"2024-04-25T10:07:10.078272+0000","last_peered":"2024-04-25T10:07:10.078272+0000","last_clean":"2024-04-25T10:07:10.078272+0000","last_became_active":"2024-04-25T10:07:10.074614+0000","last_became_peered":"2024-04-25T10:07:10.074614+0000","last_unstale":"2024-04-25T10:07:10.078272+0000","last_undegraded":"2024-04-25T10:07:10.078272+0000","last_fullsized":"2024-04-25T10:07:10.078272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:40.764957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089358+0000","last_change":"2024-04-25T10:07:10.087645+0000","last_active":"2024-04-25T10:07:10.089358+0000","last_peered":"2024-04-25T10:07:10.089358+0000","last_clean":"2024-04-25T10:07:10.089358+0000","last_became_active":"2024-04-25T10:07:10.086862+0000","last_became_peered":"2024-04-25T10:07:10.086862+0000","last_unstale":"2024-04-25T10:07:10.089358+0000","last_undegraded":"2024-04-25T10:07:10.089358+0000","last_fullsized":"2024-04-25T10:07:10.089358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.246741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.098938+0000","last_change":"2024-04-25T10:07:10.096129+0000","last_active":"2024-04-25T10:07:10.098938+0000","last_peered":"2024-04-25T10:07:10.098938+0000","last_clean":"2024-04-25T10:07:10.098938+0000","last_became_active":"2024-04-25T10:07:10.095868+0000","last_became_peered":"2024-04-25T10:07:10.095868+0000","last_unstale":"2024-04-25T10:07:10.098938+0000","last_undegraded":"2024-04-25T10:07:10.098938+0000","last_fullsized":"2024-04-25T10:07:10.098938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:15.076173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.077038+0000","last_change":"2024-04-25T10:07:10.071893+0000","last_active":"2024-04-25T10:07:10.077038+0000","last_peered":"2024-04-25T10:07:10.077038+0000","last_clean":"2024-04-25T10:07:10.077038+0000","last_became_active":"2024-04-25T10:07:10.071529+0000","last_became_peered":"2024-04-25T10:07:10.071529+0000","last_unstale":"2024-04-25T10:07:10.077038+0000","last_undegraded":"2024-04-25T10:07:10.077038+0000","last_fullsized":"2024-04-25T10:07:10.077038+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:41.130634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089668+0000","last_change":"2024-04-25T10:07:10.087289+0000","last_active":"2024-04-25T10:07:10.089668+0000","last_peered":"2024-04-25T10:07:10.089668+0000","last_clean":"2024-04-25T10:07:10.089668+0000","last_became_active":"2024-04-25T10:07:10.086764+0000","last_became_peered":"2024-04-25T10:07:10.086764+0000","last_unstale":"2024-04-25T10:07:10.089668+0000","last_undegraded":"2024-04-25T10:07:10.089668+0000","last_fullsized":"2024-04-25T10:07:10.089668+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:16.711444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.051636+0000","last_change":"2024-04-25T10:07:10.075557+0000","last_active":"2024-04-25T10:07:11.051636+0000","last_peered":"2024-04-25T10:07:11.051636+0000","last_clean":"2024-04-25T10:07:11.051636+0000","last_became_active":"2024-04-25T10:07:10.075208+0000","last_became_peered":"2024-04-25T10:07:10.075208+0000","last_unstale":"2024-04-25T10:07:11.051636+0000","last_undegraded":"2024-04-25T10:07:11.051636+0000","last_fullsized":"2024-04-25T10:07:11.051636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:18.374685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085512+0000","last_change":"2024-04-25T10:07:10.079081+0000","last_active":"2024-04-25T10:07:10.085512+0000","last_peered":"2024-04-25T10:07:10.085512+0000","last_clean":"2024-04-25T10:07:10.085512+0000","last_became_active":"2024-04-25T10:07:10.078800+0000","last_became_peered":"2024-04-25T10:07:10.078800+0000","last_unstale":"2024-04-25T10:07:10.085512+0000","last_undegraded":"2024-04-25T10:07:10.085512+0000","last_fullsized":"2024-04-25T10:07:10.085512+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:01.412707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084662+0000","last_change":"2024-04-25T10:07:10.076364+0000","last_active":"2024-04-25T10:07:10.084662+0000","last_peered":"2024-04-25T10:07:10.084662+0000","last_clean":"2024-04-25T10:07:10.084662+0000","last_became_active":"2024-04-25T10:07:10.075916+0000","last_became_peered":"2024-04-25T10:07:10.075916+0000","last_unstale":"2024-04-25T10:07:10.084662+0000","last_undegraded":"2024-04-25T10:07:10.084662+0000","last_fullsized":"2024-04-25T10:07:10.084662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:32.882689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.052671+0000","last_change":"2024-04-25T10:07:10.078838+0000","last_active":"2024-04-25T10:07:11.052671+0000","last_peered":"2024-04-25T10:07:11.052671+0000","last_clean":"2024-04-25T10:07:11.052671+0000","last_became_active":"2024-04-25T10:07:10.078521+0000","last_became_peered":"2024-04-25T10:07:10.078521+0000","last_unstale":"2024-04-25T10:07:11.052671+0000","last_undegraded":"2024-04-25T10:07:11.052671+0000","last_fullsized":"2024-04-25T10:07:11.052671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:36.082429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.090061+0000","last_change":"2024-04-25T10:07:10.088011+0000","last_active":"2024-04-25T10:07:10.090061+0000","last_peered":"2024-04-25T10:07:10.090061+0000","last_clean":"2024-04-25T10:07:10.090061+0000","last_became_active":"2024-04-25T10:07:10.087755+0000","last_became_peered":"2024-04-25T10:07:10.087755+0000","last_unstale":"2024-04-25T10:07:10.090061+0000","last_undegraded":"2024-04-25T10:07:10.090061+0000","last_fullsized":"2024-04-25T10:07:10.090061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:07.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.052902+0000","last_change":"2024-04-25T10:07:10.090696+0000","last_active":"2024-04-25T10:07:11.052902+0000","last_peered":"2024-04-25T10:07:11.052902+0000","last_clean":"2024-04-25T10:07:11.052902+0000","last_became_active":"2024-04-25T10:07:10.088683+0000","last_became_peered":"2024-04-25T10:07:10.088683+0000","last_unstale":"2024-04-25T10:07:11.052902+0000","last_undegraded":"2024-04-25T10:07:11.052902+0000","last_fullsized":"2024-04-25T10:07:11.052902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.114674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085596+0000","last_change":"2024-04-25T10:07:10.078269+0000","last_active":"2024-04-25T10:07:10.085596+0000","last_peered":"2024-04-25T10:07:10.085596+0000","last_clean":"2024-04-25T10:07:10.085596+0000","last_became_active":"2024-04-25T10:07:10.077936+0000","last_became_peered":"2024-04-25T10:07:10.077936+0000","last_unstale":"2024-04-25T10:07:10.085596+0000","last_undegraded":"2024-04-25T10:07:10.085596+0000","last_fullsized":"2024-04-25T10:07:10.085596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:15.076173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.093593+0000","last_change":"2024-04-25T10:07:10.092413+0000","last_active":"2024-04-25T10:07:10.093593+0000","last_peered":"2024-04-25T10:07:10.093593+0000","last_clean":"2024-04-25T10:07:10.093593+0000","last_became_active":"2024-04-25T10:07:10.091689+0000","last_became_peered":"2024-04-25T10:07:10.091689+0000","last_unstale":"2024-04-25T10:07:10.093593+0000","last_undegraded":"2024-04-25T10:07:10.093593+0000","last_fullsized":"2024-04-25T10:07:10.093593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:15.076173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.083448+0000","last_change":"2024-04-25T10:07:10.079875+0000","last_active":"2024-04-25T10:07:10.083448+0000","last_peered":"2024-04-25T10:07:10.083448+0000","last_clean":"2024-04-25T10:07:10.083448+0000","last_became_active":"2024-04-25T10:07:10.079372+0000","last_became_peered":"2024-04-25T10:07:10.079372+0000","last_unstale":"2024-04-25T10:07:10.083448+0000","last_undegraded":"2024-04-25T10:07:10.083448+0000","last_fullsized":"2024-04-25T10:07:10.083448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:57.499900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088985+0000","last_change":"2024-04-25T10:07:10.084608+0000","last_active":"2024-04-25T10:07:10.088985+0000","last_peered":"2024-04-25T10:07:10.088985+0000","last_clean":"2024-04-25T10:07:10.088985+0000","last_became_active":"2024-04-25T10:07:10.083990+0000","last_became_peered":"2024-04-25T10:07:10.083990+0000","last_unstale":"2024-04-25T10:07:10.088985+0000","last_undegraded":"2024-04-25T10:07:10.088985+0000","last_fullsized":"2024-04-25T10:07:10.088985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:40.764957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089702+0000","last_change":"2024-04-25T10:07:10.087376+0000","last_active":"2024-04-25T10:07:10.089702+0000","last_peered":"2024-04-25T10:07:10.089702+0000","last_clean":"2024-04-25T10:07:10.089702+0000","last_became_active":"2024-04-25T10:07:10.086656+0000","last_became_peered":"2024-04-25T10:07:10.086656+0000","last_unstale":"2024-04-25T10:07:10.089702+0000","last_undegraded":"2024-04-25T10:07:10.089702+0000","last_fullsized":"2024-04-25T10:07:10.089702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:39.743425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089363+0000","last_change":"2024-04-25T10:07:10.085420+0000","last_active":"2024-04-25T10:07:10.089363+0000","last_peered":"2024-04-25T10:07:10.089363+0000","last_clean":"2024-04-25T10:07:10.089363+0000","last_became_active":"2024-04-25T10:07:10.085112+0000","last_became_peered":"2024-04-25T10:07:10.085112+0000","last_unstale":"2024-04-25T10:07:10.089363+0000","last_undegraded":"2024-04-25T10:07:10.089363+0000","last_fullsized":"2024-04-25T10:07:10.089363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:51.012681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085850+0000","last_change":"2024-04-25T10:07:10.082372+0000","last_active":"2024-04-25T10:07:10.085850+0000","last_peered":"2024-04-25T10:07:10.085850+0000","last_clean":"2024-04-25T10:07:10.085850+0000","last_became_active":"2024-04-25T10:07:10.082087+0000","last_became_peered":"2024-04-25T10:07:10.082087+0000","last_unstale":"2024-04-25T10:07:10.085850+0000","last_undegraded":"2024-04-25T10:07:10.085850+0000","last_fullsized":"2024-04-25T10:07:10.085850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:57.803884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.080853+0000","last_change":"2024-04-25T10:07:10.074949+0000","last_active":"2024-04-25T10:07:10.080853+0000","last_peered":"2024-04-25T10:07:10.080853+0000","last_clean":"2024-04-25T10:07:10.080853+0000","last_became_active":"2024-04-25T10:07:10.074647+0000","last_became_peered":"2024-04-25T10:07:10.074647+0000","last_unstale":"2024-04-25T10:07:10.080853+0000","last_undegraded":"2024-04-25T10:07:10.080853+0000","last_fullsized":"2024-04-25T10:07:10.080853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:11.860753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084562+0000","last_change":"2024-04-25T10:07:10.073901+0000","last_active":"2024-04-25T10:07:10.084562+0000","last_peered":"2024-04-25T10:07:10.084562+0000","last_clean":"2024-04-25T10:07:10.084562+0000","last_became_active":"2024-04-25T10:07:10.073595+0000","last_became_peered":"2024-04-25T10:07:10.073595+0000","last_unstale":"2024-04-25T10:07:10.084562+0000","last_undegraded":"2024-04-25T10:07:10.084562+0000","last_fullsized":"2024-04-25T10:07:10.084562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:32.202822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085456+0000","last_change":"2024-04-25T10:07:10.073215+0000","last_active":"2024-04-25T10:07:10.085456+0000","last_peered":"2024-04-25T10:07:10.085456+0000","last_clean":"2024-04-25T10:07:10.085456+0000","last_became_active":"2024-04-25T10:07:10.069330+0000","last_became_peered":"2024-04-25T10:07:10.069330+0000","last_unstale":"2024-04-25T10:07:10.085456+0000","last_undegraded":"2024-04-25T10:07:10.085456+0000","last_fullsized":"2024-04-25T10:07:10.085456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:18.353706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089141+0000","last_change":"2024-04-25T10:07:10.083911+0000","last_active":"2024-04-25T10:07:10.089141+0000","last_peered":"2024-04-25T10:07:10.089141+0000","last_clean":"2024-04-25T10:07:10.089141+0000","last_became_active":"2024-04-25T10:07:10.083621+0000","last_became_peered":"2024-04-25T10:07:10.083621+0000","last_unstale":"2024-04-25T10:07:10.089141+0000","last_undegraded":"2024-04-25T10:07:10.089141+0000","last_fullsized":"2024-04-25T10:07:10.089141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:11.860753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084931+0000","last_change":"2024-04-25T10:07:10.082178+0000","last_active":"2024-04-25T10:07:10.084931+0000","last_peered":"2024-04-25T10:07:10.084931+0000","last_clean":"2024-04-25T10:07:10.084931+0000","last_became_active":"2024-04-25T10:07:10.081804+0000","last_became_peered":"2024-04-25T10:07:10.081804+0000","last_unstale":"2024-04-25T10:07:10.084931+0000","last_undegraded":"2024-04-25T10:07:10.084931+0000","last_fullsized":"2024-04-25T10:07:10.084931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:23.179485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.090071+0000","last_change":"2024-04-25T10:07:10.088189+0000","last_active":"2024-04-25T10:07:10.090071+0000","last_peered":"2024-04-25T10:07:10.090071+0000","last_clean":"2024-04-25T10:07:10.090071+0000","last_became_active":"2024-04-25T10:07:10.086425+0000","last_became_peered":"2024-04-25T10:07:10.086425+0000","last_unstale":"2024-04-25T10:07:10.090071+0000","last_undegraded":"2024-04-25T10:07:10.090071+0000","last_fullsized":"2024-04-25T10:07:10.090071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:53.134581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.082979+0000","last_change":"2024-04-25T10:07:10.071980+0000","last_active":"2024-04-25T10:07:10.082979+0000","last_peered":"2024-04-25T10:07:10.082979+0000","last_clean":"2024-04-25T10:07:10.082979+0000","last_became_active":"2024-04-25T10:07:10.071021+0000","last_became_peered":"2024-04-25T10:07:10.071021+0000","last_unstale":"2024-04-25T10:07:10.082979+0000","last_undegraded":"2024-04-25T10:07:10.082979+0000","last_fullsized":"2024-04-25T10:07:10.082979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:02.540803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"76'4","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100484+0000","last_change":"2024-04-25T10:07:09.057740+0000","last_active":"2024-04-25T10:07:10.100484+0000","last_peered":"2024-04-25T10:07:10.100484+0000","last_clean":"2024-04-25T10:07:10.100484+0000","last_became_active":"2024-04-25T10:07:09.056840+0000","last_became_peered":"2024-04-25T10:07:09.056840+0000","last_unstale":"2024-04-25T10:07:10.100484+0000","last_undegraded":"2024-04-25T10:07:10.100484+0000","last_fullsized":"2024-04-25T10:07:10.100484+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:35.289093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.081664+0000","last_change":"2024-04-25T10:07:10.072265+0000","last_active":"2024-04-25T10:07:10.081664+0000","last_peered":"2024-04-25T10:07:10.081664+0000","last_clean":"2024-04-25T10:07:10.081664+0000","last_became_active":"2024-04-25T10:07:10.071970+0000","last_became_peered":"2024-04-25T10:07:10.071970+0000","last_unstale":"2024-04-25T10:07:10.081664+0000","last_undegraded":"2024-04-25T10:07:10.081664+0000","last_fullsized":"2024-04-25T10:07:10.081664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:13.624610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.097870+0000","last_change":"2024-04-25T10:07:09.039541+0000","last_active":"2024-04-25T10:07:10.097870+0000","last_peered":"2024-04-25T10:07:10.097870+0000","last_clean":"2024-04-25T10:07:10.097870+0000","last_became_active":"2024-04-25T10:07:09.039236+0000","last_became_peered":"2024-04-25T10:07:09.039236+0000","last_unstale":"2024-04-25T10:07:10.097870+0000","last_undegraded":"2024-04-25T10:07:10.097870+0000","last_fullsized":"2024-04-25T10:07:10.097870+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.673920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.086505+0000","last_change":"2024-04-25T10:07:10.080995+0000","last_active":"2024-04-25T10:07:10.086505+0000","last_peered":"2024-04-25T10:07:10.086505+0000","last_clean":"2024-04-25T10:07:10.086505+0000","last_became_active":"2024-04-25T10:07:10.080705+0000","last_became_peered":"2024-04-25T10:07:10.080705+0000","last_unstale":"2024-04-25T10:07:10.086505+0000","last_undegraded":"2024-04-25T10:07:10.086505+0000","last_fullsized":"2024-04-25T10:07:10.086505+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:07.431154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.101100+0000","last_change":"2024-04-25T10:07:09.046617+0000","last_active":"2024-04-25T10:07:10.101100+0000","last_peered":"2024-04-25T10:07:10.101100+0000","last_clean":"2024-04-25T10:07:10.101100+0000","last_became_active":"2024-04-25T10:07:09.046309+0000","last_became_peered":"2024-04-25T10:07:09.046309+0000","last_unstale":"2024-04-25T10:07:10.101100+0000","last_undegraded":"2024-04-25T10:07:10.101100+0000","last_fullsized":"2024-04-25T10:07:10.101100+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:00.398258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.087957+0000","last_change":"2024-04-25T10:07:10.083636+0000","last_active":"2024-04-25T10:07:10.087957+0000","last_peered":"2024-04-25T10:07:10.087957+0000","last_clean":"2024-04-25T10:07:10.087957+0000","last_became_active":"2024-04-25T10:07:10.083322+0000","last_became_peered":"2024-04-25T10:07:10.083322+0000","last_unstale":"2024-04-25T10:07:10.087957+0000","last_undegraded":"2024-04-25T10:07:10.087957+0000","last_fullsized":"2024-04-25T10:07:10.087957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:36.245370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100218+0000","last_change":"2024-04-25T10:07:09.043837+0000","last_active":"2024-04-25T10:07:10.100218+0000","last_peered":"2024-04-25T10:07:10.100218+0000","last_clean":"2024-04-25T10:07:10.100218+0000","last_became_active":"2024-04-25T10:07:09.043376+0000","last_became_peered":"2024-04-25T10:07:09.043376+0000","last_unstale":"2024-04-25T10:07:10.100218+0000","last_undegraded":"2024-04-25T10:07:10.100218+0000","last_fullsized":"2024-04-25T10:07:10.100218+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:22.165036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.085575+0000","last_change":"2024-04-25T10:07:10.075160+0000","last_active":"2024-04-25T10:07:10.085575+0000","last_peered":"2024-04-25T10:07:10.085575+0000","last_clean":"2024-04-25T10:07:10.085575+0000","last_became_active":"2024-04-25T10:07:10.074485+0000","last_became_peered":"2024-04-25T10:07:10.074485+0000","last_unstale":"2024-04-25T10:07:10.085575+0000","last_undegraded":"2024-04-25T10:07:10.085575+0000","last_fullsized":"2024-04-25T10:07:10.085575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:30.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.098732+0000","last_change":"2024-04-25T10:07:09.050338+0000","last_active":"2024-04-25T10:07:10.098732+0000","last_peered":"2024-04-25T10:07:10.098732+0000","last_clean":"2024-04-25T10:07:10.098732+0000","last_became_active":"2024-04-25T10:07:09.050019+0000","last_became_peered":"2024-04-25T10:07:09.050019+0000","last_unstale":"2024-04-25T10:07:10.098732+0000","last_undegraded":"2024-04-25T10:07:10.098732+0000","last_fullsized":"2024-04-25T10:07:10.098732+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:56.485451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.086593+0000","last_change":"2024-04-25T10:07:10.085290+0000","last_active":"2024-04-25T10:07:10.086593+0000","last_peered":"2024-04-25T10:07:10.086593+0000","last_clean":"2024-04-25T10:07:10.086593+0000","last_became_active":"2024-04-25T10:07:10.084955+0000","last_became_peered":"2024-04-25T10:07:10.084955+0000","last_unstale":"2024-04-25T10:07:10.086593+0000","last_undegraded":"2024-04-25T10:07:10.086593+0000","last_fullsized":"2024-04-25T10:07:10.086593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:35.127679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.101063+0000","last_change":"2024-04-25T10:07:09.048891+0000","last_active":"2024-04-25T10:07:10.101063+0000","last_peered":"2024-04-25T10:07:10.101063+0000","last_clean":"2024-04-25T10:07:10.101063+0000","last_became_active":"2024-04-25T10:07:09.048041+0000","last_became_peered":"2024-04-25T10:07:09.048041+0000","last_unstale":"2024-04-25T10:07:10.101063+0000","last_undegraded":"2024-04-25T10:07:10.101063+0000","last_fullsized":"2024-04-25T10:07:10.101063+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:20.939890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088836+0000","last_change":"2024-04-25T10:07:10.083571+0000","last_active":"2024-04-25T10:07:10.088836+0000","last_peered":"2024-04-25T10:07:10.088836+0000","last_clean":"2024-04-25T10:07:10.088836+0000","last_became_active":"2024-04-25T10:07:10.083272+0000","last_became_peered":"2024-04-25T10:07:10.083272+0000","last_unstale":"2024-04-25T10:07:10.088836+0000","last_undegraded":"2024-04-25T10:07:10.088836+0000","last_fullsized":"2024-04-25T10:07:10.088836+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:18.353706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.050783+0000","last_change":"2024-04-25T10:07:09.038479+0000","last_active":"2024-04-25T10:07:11.050783+0000","last_peered":"2024-04-25T10:07:11.050783+0000","last_clean":"2024-04-25T10:07:11.050783+0000","last_became_active":"2024-04-25T10:07:09.038033+0000","last_became_peered":"2024-04-25T10:07:09.038033+0000","last_unstale":"2024-04-25T10:07:11.050783+0000","last_undegraded":"2024-04-25T10:07:11.050783+0000","last_fullsized":"2024-04-25T10:07:11.050783+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.955111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084048+0000","last_change":"2024-04-25T10:07:10.079450+0000","last_active":"2024-04-25T10:07:10.084048+0000","last_peered":"2024-04-25T10:07:10.084048+0000","last_clean":"2024-04-25T10:07:10.084048+0000","last_became_active":"2024-04-25T10:07:10.079094+0000","last_became_peered":"2024-04-25T10:07:10.079094+0000","last_unstale":"2024-04-25T10:07:10.084048+0000","last_undegraded":"2024-04-25T10:07:10.084048+0000","last_fullsized":"2024-04-25T10:07:10.084048+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:47.595780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.066541+0000","last_change":"2024-04-25T10:07:09.053173+0000","last_active":"2024-04-25T10:07:10.066541+0000","last_peered":"2024-04-25T10:07:10.066541+0000","last_clean":"2024-04-25T10:07:10.066541+0000","last_became_active":"2024-04-25T10:07:09.051858+0000","last_became_peered":"2024-04-25T10:07:09.051858+0000","last_unstale":"2024-04-25T10:07:10.066541+0000","last_undegraded":"2024-04-25T10:07:10.066541+0000","last_fullsized":"2024-04-25T10:07:10.066541+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:53.657090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.081960+0000","last_change":"2024-04-25T10:07:10.077351+0000","last_active":"2024-04-25T10:07:10.081960+0000","last_peered":"2024-04-25T10:07:10.081960+0000","last_clean":"2024-04-25T10:07:10.081960+0000","last_became_active":"2024-04-25T10:07:10.077041+0000","last_became_peered":"2024-04-25T10:07:10.077041+0000","last_unstale":"2024-04-25T10:07:10.081960+0000","last_undegraded":"2024-04-25T10:07:10.081960+0000","last_fullsized":"2024-04-25T10:07:10.081960+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:07.833001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.061161+0000","last_change":"2024-04-25T10:07:09.047435+0000","last_active":"2024-04-25T10:07:10.061161+0000","last_peered":"2024-04-25T10:07:10.061161+0000","last_clean":"2024-04-25T10:07:10.061161+0000","last_became_active":"2024-04-25T10:07:09.045179+0000","last_became_peered":"2024-04-25T10:07:09.045179+0000","last_unstale":"2024-04-25T10:07:10.061161+0000","last_undegraded":"2024-04-25T10:07:10.061161+0000","last_fullsized":"2024-04-25T10:07:10.061161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:00.398258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.078282+0000","last_change":"2024-04-25T10:07:10.072865+0000","last_active":"2024-04-25T10:07:10.078282+0000","last_peered":"2024-04-25T10:07:10.078282+0000","last_clean":"2024-04-25T10:07:10.078282+0000","last_became_active":"2024-04-25T10:07:10.072551+0000","last_became_peered":"2024-04-25T10:07:10.072551+0000","last_unstale":"2024-04-25T10:07:10.078282+0000","last_undegraded":"2024-04-25T10:07:10.078282+0000","last_fullsized":"2024-04-25T10:07:10.078282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:39.743425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.081582+0000","last_change":"2024-04-25T10:07:09.049382+0000","last_active":"2024-04-25T10:07:10.081582+0000","last_peered":"2024-04-25T10:07:10.081582+0000","last_clean":"2024-04-25T10:07:10.081582+0000","last_became_active":"2024-04-25T10:07:09.049076+0000","last_became_peered":"2024-04-25T10:07:09.049076+0000","last_unstale":"2024-04-25T10:07:10.081582+0000","last_undegraded":"2024-04-25T10:07:10.081582+0000","last_fullsized":"2024-04-25T10:07:10.081582+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:48.835478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.052204+0000","last_change":"2024-04-25T10:07:10.086967+0000","last_active":"2024-04-25T10:07:11.052204+0000","last_peered":"2024-04-25T10:07:11.052204+0000","last_clean":"2024-04-25T10:07:11.052204+0000","last_became_active":"2024-04-25T10:07:10.075595+0000","last_became_peered":"2024-04-25T10:07:10.075595+0000","last_unstale":"2024-04-25T10:07:11.052204+0000","last_undegraded":"2024-04-25T10:07:11.052204+0000","last_fullsized":"2024-04-25T10:07:11.052204+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:29.881030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"76'3","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.102626+0000","last_change":"2024-04-25T10:07:09.058380+0000","last_active":"2024-04-25T10:07:10.102626+0000","last_peered":"2024-04-25T10:07:10.102626+0000","last_clean":"2024-04-25T10:07:10.102626+0000","last_became_active":"2024-04-25T10:07:09.057710+0000","last_became_peered":"2024-04-25T10:07:09.057710+0000","last_unstale":"2024-04-25T10:07:10.102626+0000","last_undegraded":"2024-04-25T10:07:10.102626+0000","last_fullsized":"2024-04-25T10:07:10.102626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:40.116185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.087704+0000","last_change":"2024-04-25T10:07:10.082926+0000","last_active":"2024-04-25T10:07:10.087704+0000","last_peered":"2024-04-25T10:07:10.087704+0000","last_clean":"2024-04-25T10:07:10.087704+0000","last_became_active":"2024-04-25T10:07:10.082650+0000","last_became_peered":"2024-04-25T10:07:10.082650+0000","last_unstale":"2024-04-25T10:07:10.087704+0000","last_undegraded":"2024-04-25T10:07:10.087704+0000","last_fullsized":"2024-04-25T10:07:10.087704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:10.757549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"76'6","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.103879+0000","last_change":"2024-04-25T10:07:09.043553+0000","last_active":"2024-04-25T10:07:10.103879+0000","last_peered":"2024-04-25T10:07:10.103879+0000","last_clean":"2024-04-25T10:07:10.103879+0000","last_became_active":"2024-04-25T10:07:09.043250+0000","last_became_peered":"2024-04-25T10:07:09.043250+0000","last_unstale":"2024-04-25T10:07:10.103879+0000","last_undegraded":"2024-04-25T10:07:10.103879+0000","last_fullsized":"2024-04-25T10:07:10.103879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.516166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.073990+0000","last_change":"2024-04-25T10:07:10.069910+0000","last_active":"2024-04-25T10:07:10.073990+0000","last_peered":"2024-04-25T10:07:10.073990+0000","last_clean":"2024-04-25T10:07:10.073990+0000","last_became_active":"2024-04-25T10:07:10.069580+0000","last_became_peered":"2024-04-25T10:07:10.069580+0000","last_unstale":"2024-04-25T10:07:10.073990+0000","last_undegraded":"2024-04-25T10:07:10.073990+0000","last_fullsized":"2024-04-25T10:07:10.073990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:49.849927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100477+0000","last_change":"2024-04-25T10:07:09.043457+0000","last_active":"2024-04-25T10:07:10.100477+0000","last_peered":"2024-04-25T10:07:10.100477+0000","last_clean":"2024-04-25T10:07:10.100477+0000","last_became_active":"2024-04-25T10:07:09.043147+0000","last_became_peered":"2024-04-25T10:07:09.043147+0000","last_unstale":"2024-04-25T10:07:10.100477+0000","last_undegraded":"2024-04-25T10:07:10.100477+0000","last_fullsized":"2024-04-25T10:07:10.100477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.622257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088534+0000","last_change":"2024-04-25T10:07:10.082950+0000","last_active":"2024-04-25T10:07:10.088534+0000","last_peered":"2024-04-25T10:07:10.088534+0000","last_clean":"2024-04-25T10:07:10.088534+0000","last_became_active":"2024-04-25T10:07:10.082473+0000","last_became_peered":"2024-04-25T10:07:10.082473+0000","last_unstale":"2024-04-25T10:07:10.088534+0000","last_undegraded":"2024-04-25T10:07:10.088534+0000","last_fullsized":"2024-04-25T10:07:10.088534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:38.232892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.062370+0000","last_change":"2024-04-25T10:07:09.044513+0000","last_active":"2024-04-25T10:07:10.062370+0000","last_peered":"2024-04-25T10:07:10.062370+0000","last_clean":"2024-04-25T10:07:10.062370+0000","last_became_active":"2024-04-25T10:07:09.044177+0000","last_became_peered":"2024-04-25T10:07:09.044177+0000","last_unstale":"2024-04-25T10:07:10.062370+0000","last_undegraded":"2024-04-25T10:07:10.062370+0000","last_fullsized":"2024-04-25T10:07:10.062370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:34.113230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.077696+0000","last_change":"2024-04-25T10:07:10.072284+0000","last_active":"2024-04-25T10:07:10.077696+0000","last_peered":"2024-04-25T10:07:10.077696+0000","last_clean":"2024-04-25T10:07:10.077696+0000","last_became_active":"2024-04-25T10:07:10.071957+0000","last_became_peered":"2024-04-25T10:07:10.071957+0000","last_unstale":"2024-04-25T10:07:10.077696+0000","last_undegraded":"2024-04-25T10:07:10.077696+0000","last_fullsized":"2024-04-25T10:07:10.077696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:16.711444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.054994+0000","last_change":"2024-04-25T10:07:09.057016+0000","last_active":"2024-04-25T10:07:10.054994+0000","last_peered":"2024-04-25T10:07:10.054994+0000","last_clean":"2024-04-25T10:07:10.054994+0000","last_became_active":"2024-04-25T10:07:09.056737+0000","last_became_peered":"2024-04-25T10:07:09.056737+0000","last_unstale":"2024-04-25T10:07:10.054994+0000","last_undegraded":"2024-04-25T10:07:10.054994+0000","last_fullsized":"2024-04-25T10:07:10.054994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:21.569196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.093671+0000","last_change":"2024-04-25T10:07:10.091116+0000","last_active":"2024-04-25T10:07:10.093671+0000","last_peered":"2024-04-25T10:07:10.093671+0000","last_clean":"2024-04-25T10:07:10.093671+0000","last_became_active":"2024-04-25T10:07:10.090279+0000","last_became_peered":"2024-04-25T10:07:10.090279+0000","last_unstale":"2024-04-25T10:07:10.093671+0000","last_undegraded":"2024-04-25T10:07:10.093671+0000","last_fullsized":"2024-04-25T10:07:10.093671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:02.909549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"76'8","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.103876+0000","last_change":"2024-04-25T10:07:09.045532+0000","last_active":"2024-04-25T10:07:10.103876+0000","last_peered":"2024-04-25T10:07:10.103876+0000","last_clean":"2024-04-25T10:07:10.103876+0000","last_became_active":"2024-04-25T10:07:09.045053+0000","last_became_peered":"2024-04-25T10:07:09.045053+0000","last_unstale":"2024-04-25T10:07:10.103876+0000","last_undegraded":"2024-04-25T10:07:10.103876+0000","last_fullsized":"2024-04-25T10:07:10.103876+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:15.696995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084883+0000","last_change":"2024-04-25T10:07:10.079838+0000","last_active":"2024-04-25T10:07:10.084883+0000","last_peered":"2024-04-25T10:07:10.084883+0000","last_clean":"2024-04-25T10:07:10.084883+0000","last_became_active":"2024-04-25T10:07:10.079580+0000","last_became_peered":"2024-04-25T10:07:10.079580+0000","last_unstale":"2024-04-25T10:07:10.084883+0000","last_undegraded":"2024-04-25T10:07:10.084883+0000","last_fullsized":"2024-04-25T10:07:10.084883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:29.124948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100869+0000","last_change":"2024-04-25T10:07:09.038313+0000","last_active":"2024-04-25T10:07:10.100869+0000","last_peered":"2024-04-25T10:07:10.100869+0000","last_clean":"2024-04-25T10:07:10.100869+0000","last_became_active":"2024-04-25T10:07:09.038062+0000","last_became_peered":"2024-04-25T10:07:09.038062+0000","last_unstale":"2024-04-25T10:07:10.100869+0000","last_undegraded":"2024-04-25T10:07:10.100869+0000","last_fullsized":"2024-04-25T10:07:10.100869+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:27.873961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088596+0000","last_change":"2024-04-25T10:07:10.085796+0000","last_active":"2024-04-25T10:07:10.088596+0000","last_peered":"2024-04-25T10:07:10.088596+0000","last_clean":"2024-04-25T10:07:10.088596+0000","last_became_active":"2024-04-25T10:07:10.085470+0000","last_became_peered":"2024-04-25T10:07:10.085470+0000","last_unstale":"2024-04-25T10:07:10.088596+0000","last_undegraded":"2024-04-25T10:07:10.088596+0000","last_fullsized":"2024-04-25T10:07:10.088596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:41.130634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100261+0000","last_change":"2024-04-25T10:07:09.051576+0000","last_active":"2024-04-25T10:07:10.100261+0000","last_peered":"2024-04-25T10:07:10.100261+0000","last_clean":"2024-04-25T10:07:10.100261+0000","last_became_active":"2024-04-25T10:07:09.051286+0000","last_became_peered":"2024-04-25T10:07:09.051286+0000","last_unstale":"2024-04-25T10:07:10.100261+0000","last_undegraded":"2024-04-25T10:07:10.100261+0000","last_fullsized":"2024-04-25T10:07:10.100261+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:28.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.051794+0000","last_change":"2024-04-25T10:07:10.089694+0000","last_active":"2024-04-25T10:07:11.051794+0000","last_peered":"2024-04-25T10:07:11.051794+0000","last_clean":"2024-04-25T10:07:11.051794+0000","last_became_active":"2024-04-25T10:07:10.089423+0000","last_became_peered":"2024-04-25T10:07:10.089423+0000","last_unstale":"2024-04-25T10:07:11.051794+0000","last_undegraded":"2024-04-25T10:07:11.051794+0000","last_fullsized":"2024-04-25T10:07:11.051794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:17.491533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"76'1","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.094042+0000","last_change":"2024-04-25T10:07:09.048764+0000","last_active":"2024-04-25T10:07:10.094042+0000","last_peered":"2024-04-25T10:07:10.094042+0000","last_clean":"2024-04-25T10:07:10.094042+0000","last_became_active":"2024-04-25T10:07:09.048491+0000","last_became_peered":"2024-04-25T10:07:09.048491+0000","last_unstale":"2024-04-25T10:07:10.094042+0000","last_undegraded":"2024-04-25T10:07:10.094042+0000","last_fullsized":"2024-04-25T10:07:10.094042+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:34.578041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.082935+0000","last_change":"2024-04-25T10:07:10.071859+0000","last_active":"2024-04-25T10:07:10.082935+0000","last_peered":"2024-04-25T10:07:10.082935+0000","last_clean":"2024-04-25T10:07:10.082935+0000","last_became_active":"2024-04-25T10:07:10.069819+0000","last_became_peered":"2024-04-25T10:07:10.069819+0000","last_unstale":"2024-04-25T10:07:10.082935+0000","last_undegraded":"2024-04-25T10:07:10.082935+0000","last_fullsized":"2024-04-25T10:07:10.082935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:52.639578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.054495+0000","last_change":"2024-04-25T10:07:09.051328+0000","last_active":"2024-04-25T10:07:10.054495+0000","last_peered":"2024-04-25T10:07:10.054495+0000","last_clean":"2024-04-25T10:07:10.054495+0000","last_became_active":"2024-04-25T10:07:09.051018+0000","last_became_peered":"2024-04-25T10:07:09.051018+0000","last_unstale":"2024-04-25T10:07:10.054495+0000","last_undegraded":"2024-04-25T10:07:10.054495+0000","last_fullsized":"2024-04-25T10:07:10.054495+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:21.155836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084308+0000","last_change":"2024-04-25T10:07:10.073820+0000","last_active":"2024-04-25T10:07:10.084308+0000","last_peered":"2024-04-25T10:07:10.084308+0000","last_clean":"2024-04-25T10:07:10.084308+0000","last_became_active":"2024-04-25T10:07:10.073492+0000","last_became_peered":"2024-04-25T10:07:10.073492+0000","last_unstale":"2024-04-25T10:07:10.084308+0000","last_undegraded":"2024-04-25T10:07:10.084308+0000","last_fullsized":"2024-04-25T10:07:10.084308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:59.728574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.099553+0000","last_change":"2024-04-25T10:07:09.055663+0000","last_active":"2024-04-25T10:07:10.099553+0000","last_peered":"2024-04-25T10:07:10.099553+0000","last_clean":"2024-04-25T10:07:10.099553+0000","last_became_active":"2024-04-25T10:07:09.055287+0000","last_became_peered":"2024-04-25T10:07:09.055287+0000","last_unstale":"2024-04-25T10:07:10.099553+0000","last_undegraded":"2024-04-25T10:07:10.099553+0000","last_fullsized":"2024-04-25T10:07:10.099553+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:26.416130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084564+0000","last_change":"2024-04-25T10:07:10.073873+0000","last_active":"2024-04-25T10:07:10.084564+0000","last_peered":"2024-04-25T10:07:10.084564+0000","last_clean":"2024-04-25T10:07:10.084564+0000","last_became_active":"2024-04-25T10:07:10.073054+0000","last_became_peered":"2024-04-25T10:07:10.073054+0000","last_unstale":"2024-04-25T10:07:10.084564+0000","last_undegraded":"2024-04-25T10:07:10.084564+0000","last_fullsized":"2024-04-25T10:07:10.084564+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:47.619104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.051020+0000","last_change":"2024-04-25T10:07:09.048404+0000","last_active":"2024-04-25T10:07:11.051020+0000","last_peered":"2024-04-25T10:07:11.051020+0000","last_clean":"2024-04-25T10:07:11.051020+0000","last_became_active":"2024-04-25T10:07:09.047800+0000","last_became_peered":"2024-04-25T10:07:09.047800+0000","last_unstale":"2024-04-25T10:07:11.051020+0000","last_undegraded":"2024-04-25T10:07:11.051020+0000","last_fullsized":"2024-04-25T10:07:11.051020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:57.340014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.074128+0000","last_change":"2024-04-25T10:07:10.072289+0000","last_active":"2024-04-25T10:07:10.074128+0000","last_peered":"2024-04-25T10:07:10.074128+0000","last_clean":"2024-04-25T10:07:10.074128+0000","last_became_active":"2024-04-25T10:07:10.071949+0000","last_became_peered":"2024-04-25T10:07:10.071949+0000","last_unstale":"2024-04-25T10:07:10.074128+0000","last_undegraded":"2024-04-25T10:07:10.074128+0000","last_fullsized":"2024-04-25T10:07:10.074128+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:22.193310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.070440+0000","last_change":"2024-04-25T10:07:09.053836+0000","last_active":"2024-04-25T10:07:10.070440+0000","last_peered":"2024-04-25T10:07:10.070440+0000","last_clean":"2024-04-25T10:07:10.070440+0000","last_became_active":"2024-04-25T10:07:09.053519+0000","last_became_peered":"2024-04-25T10:07:09.053519+0000","last_unstale":"2024-04-25T10:07:10.070440+0000","last_undegraded":"2024-04-25T10:07:10.070440+0000","last_fullsized":"2024-04-25T10:07:10.070440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:08.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.077454+0000","last_change":"2024-04-25T10:07:10.073960+0000","last_active":"2024-04-25T10:07:10.077454+0000","last_peered":"2024-04-25T10:07:10.077454+0000","last_clean":"2024-04-25T10:07:10.077454+0000","last_became_active":"2024-04-25T10:07:10.073580+0000","last_became_peered":"2024-04-25T10:07:10.073580+0000","last_unstale":"2024-04-25T10:07:10.077454+0000","last_undegraded":"2024-04-25T10:07:10.077454+0000","last_fullsized":"2024-04-25T10:07:10.077454+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:38.232892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"76'2","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.100089+0000","last_change":"2024-04-25T10:07:09.055929+0000","last_active":"2024-04-25T10:07:10.100089+0000","last_peered":"2024-04-25T10:07:10.100089+0000","last_clean":"2024-04-25T10:07:10.100089+0000","last_became_active":"2024-04-25T10:07:09.055641+0000","last_became_peered":"2024-04-25T10:07:09.055641+0000","last_unstale":"2024-04-25T10:07:10.100089+0000","last_undegraded":"2024-04-25T10:07:10.100089+0000","last_fullsized":"2024-04-25T10:07:10.100089+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:27.873961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.089041+0000","last_change":"2024-04-25T10:07:10.084922+0000","last_active":"2024-04-25T10:07:10.089041+0000","last_peered":"2024-04-25T10:07:10.089041+0000","last_clean":"2024-04-25T10:07:10.089041+0000","last_became_active":"2024-04-25T10:07:10.084350+0000","last_became_peered":"2024-04-25T10:07:10.084350+0000","last_unstale":"2024-04-25T10:07:10.089041+0000","last_undegraded":"2024-04-25T10:07:10.089041+0000","last_fullsized":"2024-04-25T10:07:10.089041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:32.882689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.049360+0000","last_change":"2024-04-25T10:07:09.056568+0000","last_active":"2024-04-25T10:07:10.049360+0000","last_peered":"2024-04-25T10:07:10.049360+0000","last_clean":"2024-04-25T10:07:10.049360+0000","last_became_active":"2024-04-25T10:07:09.056255+0000","last_became_peered":"2024-04-25T10:07:09.056255+0000","last_unstale":"2024-04-25T10:07:10.049360+0000","last_undegraded":"2024-04-25T10:07:10.049360+0000","last_fullsized":"2024-04-25T10:07:10.049360+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:48:06.416705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.084490+0000","last_change":"2024-04-25T10:07:10.079361+0000","last_active":"2024-04-25T10:07:10.084490+0000","last_peered":"2024-04-25T10:07:10.084490+0000","last_clean":"2024-04-25T10:07:10.084490+0000","last_became_active":"2024-04-25T10:07:10.078539+0000","last_became_peered":"2024-04-25T10:07:10.078539+0000","last_unstale":"2024-04-25T10:07:10.084490+0000","last_undegraded":"2024-04-25T10:07:10.084490+0000","last_fullsized":"2024-04-25T10:07:10.084490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:07.702579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.078199+0000","last_change":"2024-04-25T10:07:09.045936+0000","last_active":"2024-04-25T10:07:10.078199+0000","last_peered":"2024-04-25T10:07:10.078199+0000","last_clean":"2024-04-25T10:07:10.078199+0000","last_became_active":"2024-04-25T10:07:09.042881+0000","last_became_peered":"2024-04-25T10:07:09.042881+0000","last_unstale":"2024-04-25T10:07:10.078199+0000","last_undegraded":"2024-04-25T10:07:10.078199+0000","last_fullsized":"2024-04-25T10:07:10.078199+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:28.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T10:07:11.052311+0000","last_change":"2024-04-25T10:07:10.084847+0000","last_active":"2024-04-25T10:07:11.052311+0000","last_peered":"2024-04-25T10:07:11.052311+0000","last_clean":"2024-04-25T10:07:11.052311+0000","last_became_active":"2024-04-25T10:07:10.084169+0000","last_became_peered":"2024-04-25T10:07:10.084169+0000","last_unstale":"2024-04-25T10:07:11.052311+0000","last_undegraded":"2024-04-25T10:07:11.052311+0000","last_fullsized":"2024-04-25T10:07:11.052311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:19.822937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.060664+0000","last_change":"2024-04-25T10:07:09.044733+0000","last_active":"2024-04-25T10:07:10.060664+0000","last_peered":"2024-04-25T10:07:10.060664+0000","last_clean":"2024-04-25T10:07:10.060664+0000","last_became_active":"2024-04-25T10:07:09.042478+0000","last_became_peered":"2024-04-25T10:07:09.042478+0000","last_unstale":"2024-04-25T10:07:10.060664+0000","last_undegraded":"2024-04-25T10:07:10.060664+0000","last_fullsized":"2024-04-25T10:07:10.060664+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:15.639435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.088231+0000","last_change":"2024-04-25T10:07:10.083591+0000","last_active":"2024-04-25T10:07:10.088231+0000","last_peered":"2024-04-25T10:07:10.088231+0000","last_clean":"2024-04-25T10:07:10.088231+0000","last_became_active":"2024-04-25T10:07:10.083297+0000","last_became_peered":"2024-04-25T10:07:10.083297+0000","last_unstale":"2024-04-25T10:07:10.088231+0000","last_undegraded":"2024-04-25T10:07:10.088231+0000","last_fullsized":"2024-04-25T10:07:10.088231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:30.896557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.045529+0000","last_change":"2024-04-25T10:07:09.039347+0000","last_active":"2024-04-25T10:07:10.045529+0000","last_peered":"2024-04-25T10:07:10.045529+0000","last_clean":"2024-04-25T10:07:10.045529+0000","last_became_active":"2024-04-25T10:07:09.039011+0000","last_became_peered":"2024-04-25T10:07:09.039011+0000","last_unstale":"2024-04-25T10:07:10.045529+0000","last_undegraded":"2024-04-25T10:07:10.045529+0000","last_fullsized":"2024-04-25T10:07:10.045529+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:29.697295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.082850+0000","last_change":"2024-04-25T10:07:10.081350+0000","last_active":"2024-04-25T10:07:10.082850+0000","last_peered":"2024-04-25T10:07:10.082850+0000","last_clean":"2024-04-25T10:07:10.082850+0000","last_became_active":"2024-04-25T10:07:10.081055+0000","last_became_peered":"2024-04-25T10:07:10.081055+0000","last_unstale":"2024-04-25T10:07:10.082850+0000","last_undegraded":"2024-04-25T10:07:10.082850+0000","last_fullsized":"2024-04-25T10:07:10.082850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.246741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.049007+0000","last_change":"2024-04-25T10:07:09.050041+0000","last_active":"2024-04-25T10:07:10.049007+0000","last_peered":"2024-04-25T10:07:10.049007+0000","last_clean":"2024-04-25T10:07:10.049007+0000","last_became_active":"2024-04-25T10:07:09.048053+0000","last_became_peered":"2024-04-25T10:07:09.048053+0000","last_unstale":"2024-04-25T10:07:10.049007+0000","last_undegraded":"2024-04-25T10:07:10.049007+0000","last_fullsized":"2024-04-25T10:07:10.049007+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:06.688130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.097478+0000","last_change":"2024-04-25T10:07:10.093694+0000","last_active":"2024-04-25T10:07:10.097478+0000","last_peered":"2024-04-25T10:07:10.097478+0000","last_clean":"2024-04-25T10:07:10.097478+0000","last_became_active":"2024-04-25T10:07:10.093426+0000","last_became_peered":"2024-04-25T10:07:10.093426+0000","last_unstale":"2024-04-25T10:07:10.097478+0000","last_undegraded":"2024-04-25T10:07:10.097478+0000","last_fullsized":"2024-04-25T10:07:10.097478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:18.353706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.056204+0000","last_change":"2024-04-25T10:07:09.038288+0000","last_active":"2024-04-25T10:07:10.056204+0000","last_peered":"2024-04-25T10:07:10.056204+0000","last_clean":"2024-04-25T10:07:10.056204+0000","last_became_active":"2024-04-25T10:07:09.038001+0000","last_became_peered":"2024-04-25T10:07:09.038001+0000","last_unstale":"2024-04-25T10:07:10.056204+0000","last_undegraded":"2024-04-25T10:07:10.056204+0000","last_fullsized":"2024-04-25T10:07:10.056204+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:31.188373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.083061+0000","last_change":"2024-04-25T10:07:10.073504+0000","last_active":"2024-04-25T10:07:10.083061+0000","last_peered":"2024-04-25T10:07:10.083061+0000","last_clean":"2024-04-25T10:07:10.083061+0000","last_became_active":"2024-04-25T10:07:10.072731+0000","last_became_peered":"2024-04-25T10:07:10.072731+0000","last_unstale":"2024-04-25T10:07:10.083061+0000","last_undegraded":"2024-04-25T10:07:10.083061+0000","last_fullsized":"2024-04-25T10:07:10.083061+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:35.580788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.067102+0000","last_change":"2024-04-25T10:07:09.052901+0000","last_active":"2024-04-25T10:07:10.067102+0000","last_peered":"2024-04-25T10:07:10.067102+0000","last_clean":"2024-04-25T10:07:10.067102+0000","last_became_active":"2024-04-25T10:07:09.051470+0000","last_became_peered":"2024-04-25T10:07:09.051470+0000","last_unstale":"2024-04-25T10:07:10.067102+0000","last_undegraded":"2024-04-25T10:07:10.067102+0000","last_fullsized":"2024-04-25T10:07:10.067102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:29.901785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.069777+0000","last_change":"2024-04-25T10:07:10.059287+0000","last_active":"2024-04-25T10:07:10.069777+0000","last_peered":"2024-04-25T10:07:10.069777+0000","last_clean":"2024-04-25T10:07:10.069777+0000","last_became_active":"2024-04-25T10:07:10.058491+0000","last_became_peered":"2024-04-25T10:07:10.058491+0000","last_unstale":"2024-04-25T10:07:10.069777+0000","last_undegraded":"2024-04-25T10:07:10.069777+0000","last_fullsized":"2024-04-25T10:07:10.069777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:09.008844+0000","last_clean_scrub_stamp":"2024-04-25T10:07:09.008844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:49.133479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.048625+0000","last_change":"2024-04-25T10:07:09.052332+0000","last_active":"2024-04-25T10:07:10.048625+0000","last_peered":"2024-04-25T10:07:10.048625+0000","last_clean":"2024-04-25T10:07:10.048625+0000","last_became_active":"2024-04-25T10:07:09.051372+0000","last_became_peered":"2024-04-25T10:07:09.051372+0000","last_unstale":"2024-04-25T10:07:10.048625+0000","last_undegraded":"2024-04-25T10:07:10.048625+0000","last_fullsized":"2024-04-25T10:07:10.048625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:07.994395+0000","last_clean_scrub_stamp":"2024-04-25T10:07:07.994395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:38.728976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":271,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-04-25T10:07:10.075313+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:07:10.075313+0000","last_peered":"2024-04-25T10:07:10.075313+0000","last_clean":"2024-04-25T10:07:10.075313+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:07:10.075313+0000","last_undegraded":"2024-04-25T10:07:10.075313+0000","last_fullsized":"2024-04-25T10:07:10.075313+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509507,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":417554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":417574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509508,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":1007360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":409735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":921600,"data_stored":416902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509507,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1531904,"data_stored":1008916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509508,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":909312,"data_stored":409735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":937984,"data_stored":423616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:15.049 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:07:15.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:07:15.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.308+0000 7f9fb0600640 1 Processor -- start 2024-04-25T10:07:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.308+0000 7f9fb0600640 1 -- start start 2024-04-25T10:07:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fb0600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8161c30 0x7f9fa80b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fb0600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9fa8162da0 0x7f9fa8071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fb0600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa80b2d60 con 0x7f9fa8162010 2024-04-25T10:07:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fb0600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa80b4bd0 con 0x7f9fa8162da0 2024-04-25T10:07:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fb0600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80cd4c0 con 0x7f9fa8161c30 2024-04-25T10:07:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fad200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9fa8162010 0x7f9fa80b2650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54968/0 (socket says 172.21.15.5:54968) 2024-04-25T10:07:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.310+0000 7f9fad200640 1 -- 172.21.15.5:0/3455359077 learned_addr learned my addr 172.21.15.5:0/3455359077 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fae600640 1 --2- 172.21.15.5:0/3455359077 >> v2:172.21.15.80:3300/0 conn(0x7f9fa8162da0 0x7f9fa8071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fadc00640 1 --2- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8161c30 0x7f9fa80b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fa7e00640 1 -- 172.21.15.5:0/3455359077 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205816367 0 0) 0x7f9fa80b2d60 con 0x7f9fa8162010 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fa7e00640 1 -- 172.21.15.5:0/3455359077 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f90003520 con 0x7f9fa8162010 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fae600640 1 -- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8161c30 msgr2=0x7f9fa80b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fae600640 1 --2- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8161c30 0x7f9fa80b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fae600640 1 -- 172.21.15.5:0/3455359077 >> v1:172.21.15.5:6789/0 conn(0x7f9fa8162010 legacy=0x7f9fa80b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.311+0000 7f9fae600640 1 -- 172.21.15.5:0/3455359077 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80cd760 con 0x7f9fa8162da0 2024-04-25T10:07:15.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.313+0000 7f9fae600640 1 --2- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8162da0 0x7f9fa8071130 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009d90 tx=0x7f9fa002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37bfa1ff2d80642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.313+0000 7f9fa7e00640 1 -- 172.21.15.5:0/3455359077 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0007740 con 0x7f9fa8162da0 2024-04-25T10:07:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fa7e00640 1 -- 172.21.15.5:0/3455359077 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa00078e0 con 0x7f9fa8162da0 2024-04-25T10:07:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fa7e00640 1 -- 172.21.15.5:0/3455359077 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0007cb0 con 0x7f9fa8162da0 2024-04-25T10:07:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 -- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8162da0 msgr2=0x7f9fa8071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 --2- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8162da0 0x7f9fa8071130 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009d90 tx=0x7f9fa002a660 comp rx=0 tx=0).stop 2024-04-25T10:07:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 -- 172.21.15.5:0/3455359077 shutdown_connections 2024-04-25T10:07:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 --2- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8162da0 0x7f9fa8071130 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 --2- 172.21.15.5:0/3455359077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8161c30 0x7f9fa80b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 -- 172.21.15.5:0/3455359077 >> 172.21.15.5:0/3455359077 conn(0x7f9fa806ab60 msgr2=0x7f9fa806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.314+0000 7f9fb0600640 1 -- 172.21.15.5:0/3455359077 shutdown_connections 2024-04-25T10:07:15.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.315+0000 7f9fb0600640 1 -- 172.21.15.5:0/3455359077 wait complete. 2024-04-25T10:07:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.316+0000 7f9fb0600640 1 Processor -- start 2024-04-25T10:07:15.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.316+0000 7f9fb0600640 1 -- start start 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.317+0000 7f9fb0600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8161c30 0x7f9fa814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.317+0000 7f9fb0600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8162010 0x7f9fa80c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.317+0000 7f9fb0600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 0x7f9fa80c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.317+0000 7f9fb0600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8163230 con 0x7f9fa8162da0 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.317+0000 7f9fb0600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80cda00 con 0x7f9fa8161c30 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.317+0000 7f9fb0600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8071670 con 0x7f9fa8162010 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 0x7f9fa80c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 0x7f9fa80c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58614/0 (socket says 172.21.15.5:58614) 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 -- 172.21.15.5:0/878968623 learned_addr learned my addr 172.21.15.5:0/878968623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fadc00640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8161c30 0x7f9fa814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fad200640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8162010 0x7f9fa80c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 -- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8162010 msgr2=0x7f9fa80c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8162010 0x7f9fa80c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 -- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8161c30 msgr2=0x7f9fa814a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8161c30 0x7f9fa814a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fae600640 1 -- 172.21.15.5:0/878968623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80c9ba0 con 0x7f9fa8162da0 2024-04-25T10:07:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.318+0000 7f9fadc00640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8161c30 0x7f9fa814a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:15.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.319+0000 7f9fae600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 0x7f9fa80c7610 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9fa002ac50 tx=0x7f9fa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.319+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa003f070 con 0x7f9fa8162da0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.319+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0031bc0 con 0x7f9fa8162da0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.319+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0009450 con 0x7f9fa8162da0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.319+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa814acb0 con 0x7f9fa8162da0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.319+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa814b270 con 0x7f9fa8162da0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.320+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa8068f60 con 0x7f9fa8162da0 2024-04-25T10:07:15.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.321+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fa003b410 con 0x7f9fa8162da0 2024-04-25T10:07:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.322+0000 7f9fa5600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.323+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9fa00aba60 con 0x7f9fa8162da0 2024-04-25T10:07:15.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.323+0000 7f9fadc00640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.324+0000 7f9fadc00640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9f8806b4d0 0x7f9f8806d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ac0 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.327+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fa007a740 con 0x7f9fa8162da0 2024-04-25T10:07:15.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.601+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9fa806b2c0 con 0x7f9fa8162da0 2024-04-25T10:07:15.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.605+0000 7f9fa5600640 1 -- 172.21.15.5:0/878968623 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9fa007a040 con 0x7f9fa8162da0 2024-04-25T10:07:15.611 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:07:15.611 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:07:15.611 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:07:15.611 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:07:15.611 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:15.611 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:07:15.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9f8806b4d0 msgr2=0x7f9f8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9f8806b4d0 0x7f9f8806d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009ac0 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 msgr2=0x7f9fa80c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 0x7f9fa80c7610 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9fa002ac50 tx=0x7f9fa0004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 shutdown_connections 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fa8162da0 0x7f9fa80c7610 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9fa8162010 0x7f9fa80c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9f8806b4d0 0x7f9f8806d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 --2- 172.21.15.5:0/878968623 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fa8161c30 0x7f9fa814a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 >> 172.21.15.5:0/878968623 conn(0x7f9fa806ab60 msgr2=0x7f9fa80cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 shutdown_connections 2024-04-25T10:07:15.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.608+0000 7f9fb0600640 1 -- 172.21.15.5:0/878968623 wait complete. 2024-04-25T10:07:15.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.841+0000 7f7487800640 1 Processor -- start 2024-04-25T10:07:15.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.842+0000 7f7487800640 1 -- start start 2024-04-25T10:07:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7487800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f748015dd00 0x7f74800b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7487800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f748006c6c0 0x7f74800b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7487800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748006cd00 con 0x7f74800b6a40 2024-04-25T10:07:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7487800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748015e0e0 con 0x7f748015dd00 2024-04-25T10:07:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7487800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748006cfe0 con 0x7f748006c6c0 2024-04-25T10:07:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7484e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f748015dd00 0x7f74800b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7484e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f748015dd00 0x7f74800b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58626/0 (socket says 172.21.15.5:58626) 2024-04-25T10:07:15.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.843+0000 7f7484e00640 1 -- 172.21.15.5:0/608689501 learned_addr learned my addr 172.21.15.5:0/608689501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.844+0000 7f7485800640 1 --2- 172.21.15.5:0/608689501 >> v2:172.21.15.80:3300/0 conn(0x7f748006c6c0 0x7f74800b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.844+0000 7f7484e00640 1 -- 172.21.15.5:0/608689501 >> v2:172.21.15.80:3300/0 conn(0x7f748006c6c0 msgr2=0x7f74800b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.844+0000 7f7484e00640 1 --2- 172.21.15.5:0/608689501 >> v2:172.21.15.80:3300/0 conn(0x7f748006c6c0 0x7f74800b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.844+0000 7f7484e00640 1 -- 172.21.15.5:0/608689501 >> v1:172.21.15.80:6789/0 conn(0x7f74800b6a40 legacy=0x7f748006bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.844+0000 7f7484e00640 1 -- 172.21.15.5:0/608689501 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800d1ca0 con 0x7f748015dd00 2024-04-25T10:07:15.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.845+0000 7f7484e00640 1 --2- 172.21.15.5:0/608689501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f748015dd00 0x7f74800b5750 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f74800d0270 tx=0x7f746802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee4d9582962a29e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.845+0000 7f747f400640 1 -- 172.21.15.5:0/608689501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468037070 con 0x7f748015dd00 2024-04-25T10:07:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.845+0000 7f747f400640 1 -- 172.21.15.5:0/608689501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7468032440 con 0x7f748015dd00 2024-04-25T10:07:15.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.845+0000 7f747f400640 1 -- 172.21.15.5:0/608689501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468032810 con 0x7f748015dd00 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 -- 172.21.15.5:0/608689501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f748015dd00 msgr2=0x7f74800b5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 --2- 172.21.15.5:0/608689501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f748015dd00 0x7f74800b5750 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f74800d0270 tx=0x7f746802a2c0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 -- 172.21.15.5:0/608689501 shutdown_connections 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 --2- 172.21.15.5:0/608689501 >> v2:172.21.15.80:3300/0 conn(0x7f748006c6c0 0x7f74800b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 --2- 172.21.15.5:0/608689501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f748015dd00 0x7f74800b5750 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 -- 172.21.15.5:0/608689501 >> 172.21.15.5:0/608689501 conn(0x7f74800d07b0 msgr2=0x7f74800d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 -- 172.21.15.5:0/608689501 shutdown_connections 2024-04-25T10:07:15.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.846+0000 7f7487800640 1 -- 172.21.15.5:0/608689501 wait complete. 2024-04-25T10:07:15.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.848+0000 7f7487800640 1 Processor -- start 2024-04-25T10:07:15.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.848+0000 7f7487800640 1 -- start start 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f7487800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f748006c6c0 0x7f74800c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f7487800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 0x7f74800cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f7487800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748015dd00 0x7f74800ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f7487800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800d2780 con 0x7f74800b6a40 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f7487800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74800d2260 con 0x7f748015dd00 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f7487800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74800d2900 con 0x7f748006c6c0 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f747fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 0x7f74800cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f747fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 0x7f74800cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58638/0 (socket says 172.21.15.5:58638) 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.849+0000 7f747fe00640 1 -- 172.21.15.5:0/2021564849 learned_addr learned my addr 172.21.15.5:0/2021564849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f7484e00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f748006c6c0 0x7f74800c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f7485800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748015dd00 0x7f74800ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747fe00640 1 -- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f748006c6c0 msgr2=0x7f74800c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747fe00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f748006c6c0 0x7f74800c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747fe00640 1 -- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748015dd00 msgr2=0x7f74800ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747fe00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748015dd00 0x7f74800ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747fe00640 1 -- 172.21.15.5:0/2021564849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748014d7e0 con 0x7f74800b6a40 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f7485800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748015dd00 0x7f74800ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747fe00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 0x7f74800cd170 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f747000c7c0 tx=0x7f747000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470016020 con 0x7f74800b6a40 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747000aa50 con 0x7f74800b6a40 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470012420 con 0x7f74800b6a40 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748014cc60 con 0x7f74800b6a40 2024-04-25T10:07:15.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.850+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748014d220 con 0x7f74800b6a40 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.852+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7480068fd0 con 0x7f74800b6a40 2024-04-25T10:07:15.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.853+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7470019020 con 0x7f74800b6a40 2024-04-25T10:07:15.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.853+0000 7f747cc00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f745c06b4d0 0x7f745c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.854+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f747008a3f0 con 0x7f74800b6a40 2024-04-25T10:07:15.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.854+0000 7f7484e00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f745c06b4d0 0x7f745c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.855+0000 7f7484e00640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f745c06b4d0 0x7f745c06d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7468034000 tx=0x7f746802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:15.858+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7470020030 con 0x7f74800b6a40 2024-04-25T10:07:15.998 INFO:tasks.ceph.mds.d.smithi080.stderr:2024-04-25T10:07:15.992+0000 7f6927400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.088+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f748015eae0 con 0x7f74800b6a40 2024-04-25T10:07:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.089+0000 7f747cc00640 1 -- 172.21.15.5:0/2021564849 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f74700590d0 con 0x7f74800b6a40 2024-04-25T10:07:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:16.095 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:12.593375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5920":{"gid":5920,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1098619647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1098619647},{"type":"v1","addr":"172.21.15.80:6835","nonce":1098619647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-04-25T10:07:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.091+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f745c06b4d0 msgr2=0x7f745c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.091+0000 7f7487800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f745c06b4d0 0x7f745c06d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7468034000 tx=0x7f746802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.091+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 msgr2=0x7f74800cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.091+0000 7f7487800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 0x7f74800cd170 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f747000c7c0 tx=0x7f747000cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 shutdown_connections 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f745c06b4d0 0x7f745c06d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f748015dd00 0x7f74800ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74800b6a40 0x7f74800cd170 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 --2- 172.21.15.5:0/2021564849 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f748006c6c0 0x7f74800c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 >> 172.21.15.5:0/2021564849 conn(0x7f74800d07b0 msgr2=0x7f74800b7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 shutdown_connections 2024-04-25T10:07:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.092+0000 7f7487800640 1 -- 172.21.15.5:0/2021564849 wait complete. 2024-04-25T10:07:16.517 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:09.493208+0000', 'modified': '2024-04-25T10:07:12.593375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5920}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5920': {'gid': 5920, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/1098619647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 1098619647}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 1098619647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:16.518 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:16.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:07:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.734+0000 7fd76aa00640 1 Processor -- start 2024-04-25T10:07:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.735+0000 7fd76aa00640 1 -- start start 2024-04-25T10:07:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76aa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd764161200 0x7fd7640b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 0x7fd7640d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7640d0600 0x7fd7640ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76aa00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7640d27c0 con 0x7fd7640d0600 2024-04-25T10:07:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76aa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7640b28b0 con 0x7fd764161200 2024-04-25T10:07:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640cf410 con 0x7fd7640d2380 2024-04-25T10:07:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7640d0600 0x7fd7640ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7640d0600 0x7fd7640ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58646/0 (socket says 172.21.15.5:58646) 2024-04-25T10:07:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76a000640 1 -- 172.21.15.5:0/2252172444 learned_addr learned my addr 172.21.15.5:0/2252172444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.736+0000 7fd76a000640 1 -- 172.21.15.5:0/2252172444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 msgr2=0x7fd7640d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.737+0000 7fd76a000640 1 --2- 172.21.15.5:0/2252172444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 0x7fd7640d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.737+0000 7fd76a000640 1 -- 172.21.15.5:0/2252172444 >> v2:172.21.15.80:3300/0 conn(0x7fd764161200 msgr2=0x7fd7640b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.737+0000 7fd769600640 1 --2- 172.21.15.5:0/2252172444 >> v2:172.21.15.80:3300/0 conn(0x7fd764161200 0x7fd7640b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.737+0000 7fd76a000640 1 --2- 172.21.15.5:0/2252172444 >> v2:172.21.15.80:3300/0 conn(0x7fd764161200 0x7fd7640b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.737+0000 7fd76a000640 1 -- 172.21.15.5:0/2252172444 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640cf6f0 con 0x7fd7640d0600 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.737+0000 7fd769600640 1 --2- 172.21.15.5:0/2252172444 >> v2:172.21.15.80:3300/0 conn(0x7fd764161200 0x7fd7640b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.738+0000 7fd76a000640 1 --2- 172.21.15.5:0/2252172444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7640d0600 0x7fd7640ce280 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd76408bed0 tx=0x7fd75c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea02f2b9271ea42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.738+0000 7fd763e00640 1 -- 172.21.15.5:0/2252172444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c036070 con 0x7fd7640d0600 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.738+0000 7fd763e00640 1 -- 172.21.15.5:0/2252172444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c031600 con 0x7fd7640d0600 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.738+0000 7fd763e00640 1 -- 172.21.15.5:0/2252172444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0319d0 con 0x7fd7640d0600 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 -- 172.21.15.5:0/2252172444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7640d0600 msgr2=0x7fd7640ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 --2- 172.21.15.5:0/2252172444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7640d0600 0x7fd7640ce280 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd76408bed0 tx=0x7fd75c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 -- 172.21.15.5:0/2252172444 shutdown_connections 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 --2- 172.21.15.5:0/2252172444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7640d0600 0x7fd7640ce280 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 --2- 172.21.15.5:0/2252172444 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 0x7fd7640d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 --2- 172.21.15.5:0/2252172444 >> v2:172.21.15.80:3300/0 conn(0x7fd764161200 0x7fd7640b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 -- 172.21.15.5:0/2252172444 >> 172.21.15.5:0/2252172444 conn(0x7fd76408af80 msgr2=0x7fd7640d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 -- 172.21.15.5:0/2252172444 shutdown_connections 2024-04-25T10:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.739+0000 7fd76aa00640 1 -- 172.21.15.5:0/2252172444 wait complete. 2024-04-25T10:07:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.740+0000 7fd76aa00640 1 Processor -- start 2024-04-25T10:07:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 -- start start 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7640d0600 0x7fd7640c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 0x7fd7640b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 0x7fd764153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640ce7c0 con 0x7fd764161200 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640d42b0 con 0x7fd7640d0600 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.741+0000 7fd76aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7640ce940 con 0x7fd7640d2380 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 0x7fd764153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 0x7fd764153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58652/0 (socket says 172.21.15.5:58652) 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 -- 172.21.15.5:0/1832481297 learned_addr learned my addr 172.21.15.5:0/1832481297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 -- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 msgr2=0x7fd7640b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 0x7fd7640b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 -- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7640d0600 msgr2=0x7fd7640c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd769600640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7640d0600 0x7fd7640c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7640d0600 0x7fd7640c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.742+0000 7fd76a000640 1 -- 172.21.15.5:0/1832481297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640b8d20 con 0x7fd764161200 2024-04-25T10:07:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.743+0000 7fd76a000640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 0x7fd764153f10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009e50 tx=0x7fd75c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.743+0000 7fd761600640 1 -- 172.21.15.5:0/1832481297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c036050 con 0x7fd764161200 2024-04-25T10:07:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.743+0000 7fd761600640 1 -- 172.21.15.5:0/1832481297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c042070 con 0x7fd764161200 2024-04-25T10:07:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.743+0000 7fd761600640 1 -- 172.21.15.5:0/1832481297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c009510 con 0x7fd764161200 2024-04-25T10:07:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.743+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7640c7630 con 0x7fd764161200 2024-04-25T10:07:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.743+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7640c7bf0 con 0x7fd764161200 2024-04-25T10:07:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.745+0000 7fd761600640 1 -- 172.21.15.5:0/1832481297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd75c007bd0 con 0x7fd764161200 2024-04-25T10:07:16.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.746+0000 7fd761600640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd74406b4d0 0x7fd74406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.746+0000 7fd761600640 1 -- 172.21.15.5:0/1832481297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd75c0ab3d0 con 0x7fd764161200 2024-04-25T10:07:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.747+0000 7fd76ee2a740 1 Processor -- start 2024-04-25T10:07:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.747+0000 7fd76ee2a740 1 -- start start 2024-04-25T10:07:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.747+0000 7fd769600640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd74406b4d0 0x7fd74406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.747+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1c8bb30 0x561bf1cb3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.747+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.747+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd47a0 0x561bf1c8a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1bef620 con 0x561bf1cb3eb0 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1b98ca0 con 0x561bf1cd47a0 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bf1beb6c0 con 0x561bf1c8bb30 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58662/0 (socket says 172.21.15.5:58662) 2024-04-25T10:07:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 -- 172.21.15.5:0/4186991415 learned_addr learned my addr 172.21.15.5:0/4186991415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 -- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1c8bb30 msgr2=0x561bf1cb3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1c8bb30 0x561bf1cb3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd76a000640 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd47a0 0x561bf1c8a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd769600640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd74406b4d0 0x7fd74406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd75400b890 tx=0x7fd754008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 -- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd47a0 msgr2=0x561bf1c8a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd47a0 0x561bf1c8a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.748+0000 7fd768c00640 1 -- 172.21.15.5:0/4186991415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf1b9ff40 con 0x561bf1cb3eb0 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.749+0000 7fd768c00640 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8bf10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd750009750 tx=0x7fd750009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce674bf385bcf318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.749+0000 7fd759c00640 1 -- 172.21.15.5:0/4186991415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750015020 con 0x561bf1cb3eb0 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.749+0000 7fd759c00640 1 -- 172.21.15.5:0/4186991415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75000bad0 con 0x561bf1cb3eb0 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.749+0000 7fd759c00640 1 -- 172.21.15.5:0/4186991415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750009e60 con 0x561bf1cb3eb0 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.749+0000 7fd76ee2a740 1 -- 172.21.15.5:0/4186991415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 msgr2=0x561bf1c8bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.749+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8bf10 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd750009750 tx=0x7fd750009c10 comp rx=0 tx=0).stop 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 -- 172.21.15.5:0/4186991415 shutdown_connections 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd47a0 0x561bf1c8a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8bf10 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/4186991415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1c8bb30 0x561bf1cb3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 -- 172.21.15.5:0/4186991415 >> 172.21.15.5:0/4186991415 conn(0x561bf1c88d50 msgr2=0x561bf1cd1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 -- 172.21.15.5:0/4186991415 shutdown_connections 2024-04-25T10:07:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 -- 172.21.15.5:0/4186991415 wait complete. 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.750+0000 7fd76ee2a740 1 -- start start 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.751+0000 7fd76ee2a740 1 Processor -- start 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 0x561bf1cb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 0x561bf1cd3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1cd3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1cb34e0 con 0x561bf1cd47a0 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1b8e1f0 con 0x561bf1caa910 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bf1cb2ed0 con 0x561bf1cb3eb0 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd76ee2a740 10 client.0 _ll_register_callbacks cb 0x561bf1babad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 0x561bf1cd3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 0x561bf1cd3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58664/0 (socket says 172.21.15.5:58664) 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 -- 172.21.15.5:0/293113894 learned_addr learned my addr 172.21.15.5:0/293113894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 -- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 msgr2=0x561bf1cb0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 0x561bf1cb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 -- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 msgr2=0x561bf1cd3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1cd3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 -- 172.21.15.5:0/293113894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bf1badc50 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.752+0000 7fd768c00640 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 0x561bf1cd3a80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd750009c50 tx=0x7fd750004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a699d5fae1207af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd74e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd74e000640 1 -- 172.21.15.5:0/293113894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750029060 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd74e000640 1 -- 172.21.15.5:0/293113894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd75001a420 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd75000bad0 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd74e000640 1 -- 172.21.15.5:0/293113894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd75002f050 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd76ee2a740 10 client.6921 fetch_fsmap learned FSMap version 46 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x561bf19042d0 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd74e000640 1 -- 172.21.15.5:0/293113894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75000bad0 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd74e000640 1 -- 172.21.15.5:0/293113894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd750032020 con 0x561bf1cd47a0 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd76ee2a740 10 client.6921 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:07:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd76ee2a740 10 client.6921 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.753+0000 7fd76ee2a740 20 client.6921 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.754+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce2020 0x561bf1ce2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.754+0000 7fd76ee2a740 4 client.6921 mds_command: new command op to 5920 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.754+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 --> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bf1bfcb60 con 0x561bf1ce2020 2024-04-25T10:07:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.754+0000 7fd769600640 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce2020 0x561bf1ce2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.755+0000 7fd769600640 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce2020 0x561bf1ce2400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b034f9cf8b28266a server_cookie=24479da2225e1f57 in_seq=0 out_seq=0 2024-04-25T10:07:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.755+0000 7fd74e000640 10 client.6921 ms_handle_connect on v2:172.21.15.80:6834/1098619647 2024-04-25T10:07:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.757+0000 7fd74e000640 1 -- 172.21.15.5:0/293113894 <== mds.0 v2:172.21.15.80:6834/1098619647 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bf1bfcb60 con 0x561bf1ce2020 2024-04-25T10:07:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.757+0000 7fd74e000640 10 client.6921 handle_command_reply: tid=0 2024-04-25T10:07:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.757+0000 7fd76ee2a740 1 client.6921 shutdown 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 msgr2=0x561bf1cd3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 0x561bf1cd3a80 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd750009c50 tx=0x7fd750004840 comp rx=0 tx=0).stop 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 shutdown_connections 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce2020 0x561bf1ce2400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1cd3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cd47a0 0x561bf1cd3a80 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/293113894 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 0x561bf1cb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 >> 172.21.15.5:0/293113894 conn(0x561bf1c8bb30 msgr2=0x561bf1c8a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd74e000640 0 client.6921 ms_handle_reset on v2:172.21.15.80:6834/1098619647 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 shutdown_connections 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 1 -- 172.21.15.5:0/293113894 wait complete. 2024-04-25T10:07:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.758+0000 7fd76ee2a740 20 client.6921 trim_cache size 0 max 16384 2024-04-25T10:07:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.775+0000 7fd76ee2a740 1 Processor -- start 2024-04-25T10:07:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.776+0000 7fd76ee2a740 1 -- start start 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.776+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 0x561bf1ce15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.776+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 0x561bf1cd39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.776+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd3540 0x561bf1cd3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641ff3d0 con 0x561bf1caa910 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641f7540 con 0x561bf1cd3540 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641fefa0 con 0x561bf1cb3eb0 2024-04-25T10:07:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 0x561bf1ce15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 0x561bf1ce15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58670/0 (socket says 172.21.15.5:58670) 2024-04-25T10:07:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 -- 172.21.15.5:0/1106518699 learned_addr learned my addr 172.21.15.5:0/1106518699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd76a000640 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd3540 0x561bf1cd3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 -- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 msgr2=0x561bf1cd39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 0x561bf1cd39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 -- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd3540 msgr2=0x561bf1cd3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd3540 0x561bf1cd3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.777+0000 7fd769600640 1 -- 172.21.15.5:0/1106518699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bf19ce6b0 con 0x561bf1caa910 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.778+0000 7fd769600640 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 0x561bf1ce15a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd75400db30 tx=0x7fd7540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f028b3b4bfbd59cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.778+0000 7fd759200640 1 -- 172.21.15.5:0/1106518699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754019040 con 0x561bf1caa910 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.778+0000 7fd759200640 1 -- 172.21.15.5:0/1106518699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd754026070 con 0x561bf1caa910 2024-04-25T10:07:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.778+0000 7fd759200640 1 -- 172.21.15.5:0/1106518699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7540138f0 con 0x561bf1caa910 2024-04-25T10:07:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1106518699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 msgr2=0x561bf1ce15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 0x561bf1ce15a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd75400db30 tx=0x7fd7540046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1106518699 shutdown_connections 2024-04-25T10:07:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1cd3540 0x561bf1cd3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cb3eb0 0x561bf1cd39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1106518699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1caa910 0x561bf1ce15a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1106518699 >> 172.21.15.5:0/1106518699 conn(0x561bf1c8bb30 msgr2=0x561bf1cf44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1106518699 shutdown_connections 2024-04-25T10:07:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.779+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1106518699 wait complete. 2024-04-25T10:07:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.780+0000 7fd76ee2a740 1 -- start start 2024-04-25T10:07:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.780+0000 7fd76ee2a740 1 Processor -- start 2024-04-25T10:07:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1ce4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cd3540 0x561bf1d3be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1cb15c0 con 0x561bf1cb3eb0 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bf1cb34e0 con 0x561bf1caa910 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x561bf1ca8e90 con 0x561bf1cd3540 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd76ee2a740 10 client.0 _ll_register_callbacks cb 0x7fd7500095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd768c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd768c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58672/0 (socket says 172.21.15.5:58672) 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.781+0000 7fd768c00640 1 -- 172.21.15.5:0/1182614954 learned_addr learned my addr 172.21.15.5:0/1182614954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd76a000640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cd3540 0x561bf1d3be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd769600640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1ce4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd768c00640 1 -- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cd3540 msgr2=0x561bf1d3be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd768c00640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cd3540 0x561bf1d3be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd768c00640 1 -- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 msgr2=0x561bf1ce4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd768c00640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1ce4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd768c00640 1 -- 172.21.15.5:0/1182614954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bf1ce3c50 con 0x561bf1cb3eb0 2024-04-25T10:07:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd769600640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1ce4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd768c00640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8c8f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd75001a930 tx=0x7fd7500047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae100a00860965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd74e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd74e000640 1 -- 172.21.15.5:0/1182614954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750015040 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd74e000640 1 -- 172.21.15.5:0/1182614954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd750025c60 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd754026070 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd74e000640 1 -- 172.21.15.5:0/1182614954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd750029030 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.782+0000 7fd76ee2a740 10 client.6927 fetch_fsmap learned FSMap version 46 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x561bf1c8ce30 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd74e000640 1 -- 172.21.15.5:0/1182614954 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd754026070 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd74e000640 1 -- 172.21.15.5:0/1182614954 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd750032030 con 0x561bf1cb3eb0 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 10 client.6927 fetch_fsmap finished waiting for FSMap version 46 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 10 client.6927 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 20 client.6927 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce35f0 0x561bf1d47c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 4 client.6927 mds_command: new command op to 5920 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.783+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 --> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561bf1bf6eb0 con 0x561bf1ce35f0 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.784+0000 7fd769600640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce35f0 0x561bf1d47c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.784+0000 7fd769600640 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce35f0 0x561bf1d47c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0a4c5ff6108752 server_cookie=898ce8c1ddd8e6b7 in_seq=0 out_seq=0 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.784+0000 7fd74e000640 10 client.6927 ms_handle_connect on v2:172.21.15.80:6834/1098619647 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.785+0000 7fd74e000640 1 -- 172.21.15.5:0/1182614954 <== mds.0 v2:172.21.15.80:6834/1098619647 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561bf1bf6eb0 con 0x561bf1ce35f0 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.785+0000 7fd74e000640 10 client.6927 handle_command_reply: tid=0 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.785+0000 7fd76ee2a740 1 client.6927 shutdown 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.785+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 msgr2=0x561bf1c8c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.785+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8c8f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd75001a930 tx=0x7fd7500047d0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 shutdown_connections 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:6834/1098619647,v1:172.21.15.80:6835/1098619647] conn(0x561bf1ce35f0 0x561bf1d47c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561bf1cd3540 0x561bf1d3be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf1cb3eb0 0x561bf1c8c8f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 --2- 172.21.15.5:0/1182614954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561bf1caa910 0x561bf1ce4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd74e000640 0 client.6927 ms_handle_reset on v2:172.21.15.80:6834/1098619647 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 >> 172.21.15.5:0/1182614954 conn(0x561bf1c8bb30 msgr2=0x561bf1c8abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 shutdown_connections 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 1 -- 172.21.15.5:0/1182614954 wait complete. 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76ee2a740 20 client.6927 trim_cache size 0 max 16384 2024-04-25T10:07:16.793 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5920, 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.7161226489999999, 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 46, 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 77, 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 77, 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 12.147429057 2024-04-25T10:07:16.794 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:07:16.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd74406b4d0 msgr2=0x7fd74406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76aa00640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd74406b4d0 0x7fd74406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd75400b890 tx=0x7fd754008040 comp rx=0 tx=0).stop 2024-04-25T10:07:16.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 msgr2=0x7fd764153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.786+0000 7fd76aa00640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 0x7fd764153f10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd75c009e50 tx=0x7fd75c004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 shutdown_connections 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd764161200 0x7fd764153f10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7640d2380 0x7fd7640b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd74406b4d0 0x7fd74406d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 --2- 172.21.15.5:0/1832481297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7640d0600 0x7fd7640c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 >> 172.21.15.5:0/1832481297 conn(0x7fd76408af80 msgr2=0x7fd7640b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 shutdown_connections 2024-04-25T10:07:16.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:16.787+0000 7fd76aa00640 1 -- 172.21.15.5:0/1832481297 wait complete. 2024-04-25T10:07:16.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.012+0000 7f1727c00640 1 Processor -- start 2024-04-25T10:07:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.012+0000 7f1727c00640 1 -- start start 2024-04-25T10:07:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f1727c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f17200cbb50 0x7f17200ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f1727c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200c9f90 con 0x7f17201472c0 2024-04-25T10:07:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f1727c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17200ca270 con 0x7f17200cad50 2024-04-25T10:07:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f1727c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200ca550 con 0x7f17200cbb50 2024-04-25T10:07:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f1725200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f17201472c0 0x7f17200cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50516/0 (socket says 172.21.15.5:50516) 2024-04-25T10:07:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f1725200640 1 -- 172.21.15.5:0/3593121850 learned_addr learned my addr 172.21.15.5:0/3593121850 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.014+0000 7f171fe00640 1 --2- 172.21.15.5:0/3593121850 >> v2:172.21.15.80:3300/0 conn(0x7f17200cbb50 0x7f17200ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444605223 0 0) 0x7f17200c9f90 con 0x7f17201472c0 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1704003520 con 0x7f17201472c0 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014209866 0 0) 0x7f1704003520 con 0x7f17201472c0 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 >> v2:172.21.15.80:3300/0 conn(0x7f17200cbb50 msgr2=0x7f17200ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 --2- 172.21.15.5:0/3593121850 >> v2:172.21.15.80:3300/0 conn(0x7f17200cbb50 0x7f17200ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 >> v1:172.21.15.80:6789/0 conn(0x7f17200cad50 legacy=0x7f17200cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200ca830 con 0x7f17201472c0 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1710001690 con 0x7f17201472c0 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 >> v1:172.21.15.5:6789/0 conn(0x7f17201472c0 legacy=0x7f17200cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040039b0 0x7f1704003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f1704004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17040052e0 0x7f170400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200ca550 con 0x7f1704004800 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170400c230 con 0x7f17040052e0 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170400c430 con 0x7f17040039b0 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171fe00640 1 --2- 172.21.15.5:0/3593121850 >> v2:172.21.15.80:3300/0 conn(0x7f17200cbb50 0x7f17200ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.015+0000 7f171fe00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f1704004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f1725c00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17040052e0 0x7f170400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f1725200640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040039b0 0x7f1704003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f171fe00640 1 -- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040039b0 msgr2=0x7f1704003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f171fe00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040039b0 0x7f1704003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f171fe00640 1 -- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17040052e0 msgr2=0x7f170400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f171fe00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17040052e0 0x7f170400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f171fe00640 1 -- 172.21.15.5:0/3593121850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1704003520 con 0x7f1704004800 2024-04-25T10:07:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f1725c00640 1 -- 172.21.15.5:0/3593121850 reap_dead start 2024-04-25T10:07:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.016+0000 7f1725200640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040039b0 0x7f1704003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.017+0000 7f171fe00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f1704004c80 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f170c006fd0 tx=0x7f1704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10810d1fcf057dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.017+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00f040 con 0x7f1704004800 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.017+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c02ac70 con 0x7f1704004800 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.017+0000 7f171f400640 1 -- 172.21.15.5:0/3593121850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c004270 con 0x7f1704004800 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 -- 172.21.15.5:0/3593121850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 msgr2=0x7f1704004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f1704004c80 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f170c006fd0 tx=0x7f1704005830 comp rx=0 tx=0).stop 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 -- 172.21.15.5:0/3593121850 shutdown_connections 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 --2- 172.21.15.5:0/3593121850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f1704004c80 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 -- 172.21.15.5:0/3593121850 >> 172.21.15.5:0/3593121850 conn(0x7f1720071150 msgr2=0x7f1720071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 -- 172.21.15.5:0/3593121850 shutdown_connections 2024-04-25T10:07:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.018+0000 7f1727c00640 1 -- 172.21.15.5:0/3593121850 wait complete. 2024-04-25T10:07:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.020+0000 7f1727c00640 1 Processor -- start 2024-04-25T10:07:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.020+0000 7f1727c00640 1 -- start start 2024-04-25T10:07:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1727c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f17200b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1727c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201e73a0 0x7f17201e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1727c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17200b4a50 0x7f17200b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1727c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200cb5c0 con 0x7f1704004800 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1727c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200cb440 con 0x7f17200b4a50 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1727c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17200cb740 con 0x7f17201e73a0 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1725200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f17200b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1725200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f17200b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55078/0 (socket says 172.21.15.5:55078) 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.021+0000 7f1725200640 1 -- 172.21.15.5:0/2384611530 learned_addr learned my addr 172.21.15.5:0/2384611530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17200b4a50 0x7f17200b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f171fe00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201e73a0 0x7f17201e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725200640 1 -- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201e73a0 msgr2=0x7f17201e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725200640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201e73a0 0x7f17201e81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725200640 1 -- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17200b4a50 msgr2=0x7f17200b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725200640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17200b4a50 0x7f17200b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725200640 1 -- 172.21.15.5:0/2384611530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17201e89f0 con 0x7f1704004800 2024-04-25T10:07:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17200b4a50 0x7f17200b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f1725200640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f17200b24e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f171000d770 tx=0x7f171000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.022+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710019070 con 0x7f1704004800 2024-04-25T10:07:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.023+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1710007500 con 0x7f1704004800 2024-04-25T10:07:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.023+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710004a20 con 0x7f1704004800 2024-04-25T10:07:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.023+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172013fb40 con 0x7f1704004800 2024-04-25T10:07:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.023+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17201e8700 con 0x7f1704004800 2024-04-25T10:07:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.024+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1720068fd0 con 0x7f1704004800 2024-04-25T10:07:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.025+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f171001c070 con 0x7f1704004800 2024-04-25T10:07:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.026+0000 7f171cc00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.026+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1710089d90 con 0x7f1704004800 2024-04-25T10:07:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.026+0000 7f171fe00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.027+0000 7f171fe00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f170c033000 tx=0x7f170c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.030+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f171000de20 con 0x7f1704004800 2024-04-25T10:07:17.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.275+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17200b55f0 con 0x7f1704004800 2024-04-25T10:07:17.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.276+0000 7f171cc00640 1 -- 172.21.15.5:0/2384611530 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1710058a70 con 0x7f1704004800 2024-04-25T10:07:17.282 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 msgr2=0x7f170006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f170c033000 tx=0x7f170c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 msgr2=0x7f17200b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f17200b24e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f171000d770 tx=0x7f171000dc30 comp rx=0 tx=0).stop 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 shutdown_connections 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f170006b4d0 0x7f170006d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17200b4a50 0x7f17200b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17201e73a0 0x7f17201e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 --2- 172.21.15.5:0/2384611530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704004800 0x7f17200b24e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 >> 172.21.15.5:0/2384611530 conn(0x7f1720071150 msgr2=0x7f172015f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 shutdown_connections 2024-04-25T10:07:17.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:17.279+0000 7f1727c00640 1 -- 172.21.15.5:0/2384611530 wait complete. 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:17.304 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:17.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.305 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42288sec preferred_lft 42288sec 2024-04-25T10:07:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:17.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.362 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:17.443 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:07:17.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.447 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:07:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:07:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:07:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:07:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:17.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:07:17.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.667 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:07:17.718 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:17.718 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:17.719 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:17.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:07:17.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:17.808 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.864 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:17.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:17.865 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.918 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:18.055 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:18.056 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:18.056 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:07:18.081 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:18.081 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:18.081 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:07:18.081 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:07:18.081 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:07:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:07:18.112 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:07:18.112 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:18.112 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:18.112 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:18.112 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:18.119 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:18.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.198 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:client5402 2024-04-25T10:07:18.279 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:18.279 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:18.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.335 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T10:07:18.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.410 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T10:07:18.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.475 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T10:07:18.527 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:07:18.527 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:07:18.527 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:07:18.527 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:07:18.527 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T10:07:18.527 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:07:18.649 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:18.662 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:18.662 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:18.757 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:18.757 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:18.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.048+0000 7f1136c00640 1 Processor -- start 2024-04-25T10:07:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.048+0000 7f1136c00640 1 -- start start 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1136c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f11300c5010 0x7f113015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1136c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11300cd510 con 0x7f113006e970 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1136c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11300cd7f0 con 0x7f1130151720 2024-04-25T10:07:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1136c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11300c53f0 con 0x7f11300c5010 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1127400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f113006e970 0x7f11300c4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50526/0 (socket says 172.21.15.5:50526) 2024-04-25T10:07:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1127400640 1 -- 172.21.15.5:0/1064944447 learned_addr learned my addr 172.21.15.5:0/1064944447 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.051+0000 7f1134c00640 1 --2- 172.21.15.5:0/1064944447 >> v2:172.21.15.80:3300/0 conn(0x7f11300c5010 0x7f113015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601772147 0 0) 0x7f11300cd510 con 0x7f113006e970 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1114005730 con 0x7f113006e970 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 697191456 0 0) 0x7f1114005730 con 0x7f113006e970 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 >> v2:172.21.15.80:3300/0 conn(0x7f11300c5010 msgr2=0x7f113015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 --2- 172.21.15.5:0/1064944447 >> v2:172.21.15.80:3300/0 conn(0x7f11300c5010 0x7f113015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 >> v1:172.21.15.80:6789/0 conn(0x7f1130151720 legacy=0x7f113006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11300cdad0 con 0x7f113006e970 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1118001670 con 0x7f113006e970 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 >> v1:172.21.15.5:6789/0 conn(0x7f113006e970 legacy=0x7f11300c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1114005c10 0x7f1114006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1114006a10 0x7f1114006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11140073d0 0x7f111400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300c53f0 con 0x7f1114006a10 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111400e2d0 con 0x7f11140073d0 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.052+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1114003520 con 0x7f1114005c10 2024-04-25T10:07:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1127400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1114006a10 0x7f1114006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1134c00640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11140073d0 0x7f111400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f112fe00640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1114005c10 0x7f1114006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1127400640 1 -- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1114005c10 msgr2=0x7f1114006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1127400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1114005c10 0x7f1114006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1127400640 1 -- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11140073d0 msgr2=0x7f111400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1127400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11140073d0 0x7f111400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1127400640 1 -- 172.21.15.5:0/1064944447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1114005730 con 0x7f1114006a10 2024-04-25T10:07:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f1134c00640 1 -- 172.21.15.5:0/1064944447 reap_dead start 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.053+0000 7f112fe00640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1114005c10 0x7f1114006030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.054+0000 7f1127400640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1114006a10 0x7f1114006e90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f111800c190 tx=0x7f1114007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee2cda89ccf6cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.055+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118007c60 con 0x7f1114006a10 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.055+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118005af0 con 0x7f1114006a10 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.055+0000 7f112f400640 1 -- 172.21.15.5:0/1064944447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118007070 con 0x7f1114006a10 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.055+0000 7f1136c00640 1 -- 172.21.15.5:0/1064944447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1114006a10 msgr2=0x7f1114006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.055+0000 7f1136c00640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1114006a10 0x7f1114006e90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f111800c190 tx=0x7f1114007a40 comp rx=0 tx=0).stop 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.056+0000 7f1136c00640 1 -- 172.21.15.5:0/1064944447 shutdown_connections 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.056+0000 7f1136c00640 1 --2- 172.21.15.5:0/1064944447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1114006a10 0x7f1114006e90 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.056+0000 7f1136c00640 1 -- 172.21.15.5:0/1064944447 >> 172.21.15.5:0/1064944447 conn(0x7f1130071150 msgr2=0x7f1130071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.056+0000 7f1136c00640 1 -- 172.21.15.5:0/1064944447 shutdown_connections 2024-04-25T10:07:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.056+0000 7f1136c00640 1 -- 172.21.15.5:0/1064944447 wait complete. 2024-04-25T10:07:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.059+0000 7f1136c00640 1 Processor -- start 2024-04-25T10:07:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.059+0000 7f1136c00640 1 -- start start 2024-04-25T10:07:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.060+0000 7f1136c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1114006a10 0x7f11300b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.060+0000 7f1136c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1130155c10 0x7f1130156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.060+0000 7f1136c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 0x7f113014b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1136c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130151b00 con 0x7f113014ac00 2024-04-25T10:07:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1136c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300ce090 con 0x7f1114006a10 2024-04-25T10:07:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1136c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113015be70 con 0x7f1130155c10 2024-04-25T10:07:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1134c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 0x7f113014b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1134c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 0x7f113014b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55090/0 (socket says 172.21.15.5:55090) 2024-04-25T10:07:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1134c00640 1 -- 172.21.15.5:0/917659610 learned_addr learned my addr 172.21.15.5:0/917659610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1127400640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1130155c10 0x7f1130156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f112fe00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1114006a10 0x7f11300b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.061+0000 7f1134c00640 1 -- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1130155c10 msgr2=0x7f1130156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f1134c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1130155c10 0x7f1130156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f1134c00640 1 -- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1114006a10 msgr2=0x7f11300b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f1134c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1114006a10 0x7f11300b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f1134c00640 1 -- 172.21.15.5:0/917659610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1130151ed0 con 0x7f113014ac00 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f112fe00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1114006a10 0x7f11300b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f1134c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 0x7f113014b0b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1130071f80 tx=0x7f1128011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128019020 con 0x7f113014ac00 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128004460 con 0x7f113014ac00 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128003850 con 0x7f113014ac00 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.062+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11301521b0 con 0x7f113014ac00 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.063+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1130152770 con 0x7f113014ac00 2024-04-25T10:07:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.064+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11300c5f60 con 0x7f113014ac00 2024-04-25T10:07:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.066+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f112801c020 con 0x7f113014ac00 2024-04-25T10:07:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.067+0000 7f112cc00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110c06b500 0x7f110c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.067+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f112808a700 con 0x7f113014ac00 2024-04-25T10:07:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.068+0000 7f112fe00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110c06b500 0x7f110c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.069+0000 7f112fe00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110c06b500 0x7f110c06d9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1120007fd0 tx=0x7f1120009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.076+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11300c5f60 con 0x7f113014ac00 2024-04-25T10:07:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.278+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1130152970 con 0x7f113014ac00 2024-04-25T10:07:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.281+0000 7f112cc00640 1 -- 172.21.15.5:0/917659610 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f1130152970 con 0x7f113014ac00 2024-04-25T10:07:19.287 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:19.287 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":77,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:11.039355+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110c06b500 msgr2=0x7f110c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110c06b500 0x7f110c06d9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1120007fd0 tx=0x7f1120009040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 msgr2=0x7f113014b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 0x7f113014b0b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1130071f80 tx=0x7f1128011920 comp rx=0 tx=0).stop 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 shutdown_connections 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f113014ac00 0x7f113014b0b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1130155c10 0x7f1130156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110c06b500 0x7f110c06d9c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 --2- 172.21.15.5:0/917659610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1114006a10 0x7f11300b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 >> 172.21.15.5:0/917659610 conn(0x7f1130071150 msgr2=0x7f1130158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 shutdown_connections 2024-04-25T10:07:19.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.284+0000 7f1136c00640 1 -- 172.21.15.5:0/917659610 wait complete. 2024-04-25T10:07:19.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:07.362188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:08.532434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:19.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:19.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.513+0000 7fe9b2e00640 1 Processor -- start 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.514+0000 7fe9b2e00640 1 -- start start 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b2e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac0d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b2e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9ac0d21b0 0x7fe9ac0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b2e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac0d29c0 con 0x7fe9ac15db70 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b2e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b6ce0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b2e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0d2f80 con 0x7fe9ac0cefb0 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b0e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9ac0d21b0 0x7fe9ac0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b0e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe9ac0d21b0 0x7fe9ac0b7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55100/0 (socket says 172.21.15.5:55100) 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.515+0000 7fe9b0e00640 1 -- 172.21.15.5:0/3641196839 learned_addr learned my addr 172.21.15.5:0/3641196839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.516+0000 7fe9a3400640 1 --2- 172.21.15.5:0/3641196839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac0d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.516+0000 7fe9b0e00640 1 -- 172.21.15.5:0/3641196839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 msgr2=0x7fe9ac0d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.516+0000 7fe9b0e00640 1 --2- 172.21.15.5:0/3641196839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac0d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.516+0000 7fe9b0e00640 1 -- 172.21.15.5:0/3641196839 >> v1:172.21.15.80:6789/0 conn(0x7fe9ac15db70 legacy=0x7fe9ac0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.516+0000 7fe9b0e00640 1 -- 172.21.15.5:0/3641196839 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0d2ca0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.517+0000 7fe9b0e00640 1 --2- 172.21.15.5:0/3641196839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0b7710 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009e50 tx=0x7fe9a402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bf3ce0b36ecd39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.517+0000 7fe9ab400640 1 -- 172.21.15.5:0/3641196839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4036070 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.517+0000 7fe9ab400640 1 -- 172.21.15.5:0/3641196839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4031600 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.517+0000 7fe9ab400640 1 -- 172.21.15.5:0/3641196839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40319d0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3641196839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 msgr2=0x7fe9ac0b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3641196839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0b7710 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009e50 tx=0x7fe9a402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3641196839 shutdown_connections 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3641196839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0b7710 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3641196839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac0d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3641196839 >> 172.21.15.5:0/3641196839 conn(0x7fe9ac0b5370 msgr2=0x7fe9ac0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3641196839 shutdown_connections 2024-04-25T10:07:19.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.518+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3641196839 wait complete. 2024-04-25T10:07:19.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.520+0000 7fe9b2e00640 1 Processor -- start 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.520+0000 7fe9b2e00640 1 -- start start 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9b2e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9b2e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9b2e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac15db70 0x7fe9ac0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9b2e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0d2640 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9b2e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0b7280 con 0x7fe9ac15db70 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9b2e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac15df50 con 0x7fe9ac0cefb0 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9a3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55114/0 (socket says 172.21.15.5:55114) 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.521+0000 7fe9a3400640 1 -- 172.21.15.5:0/3276663219 learned_addr learned my addr 172.21.15.5:0/3276663219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9b0e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac15db70 0x7fe9ac0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9abe00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9a3400640 1 -- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 msgr2=0x7fe9ac1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9a3400640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9a3400640 1 -- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac15db70 msgr2=0x7fe9ac0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9a3400640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac15db70 0x7fe9ac0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9a3400640 1 -- 172.21.15.5:0/3276663219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0c82c0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.522+0000 7fe9a3400640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0c8550 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe99400d830 tx=0x7fe99400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.523+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400b6c0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.523+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994019070 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.523+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400ba80 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.523+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac1f43e0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.523+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac1f49a0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.524+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac068f60 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.525+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe99401e460 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.527+0000 7fe9a8c00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98806b4d0 0x7fe98806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.527+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe99408aae0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.527+0000 7fe9abe00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98806b4d0 0x7fe98806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.529+0000 7fe9abe00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98806b4d0 0x7fe98806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006fd0 tx=0x7fe99c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.531+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe994059730 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.732+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9ac0d0ea0 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.732+0000 7fe9a8c00640 1 -- 172.21.15.5:0/3276663219 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe994059030 con 0x7fe9ac0d21b0 2024-04-25T10:07:19.738 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:07:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98806b4d0 msgr2=0x7fe98806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98806b4d0 0x7fe98806d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe99c006fd0 tx=0x7fe99c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 msgr2=0x7fe9ac0c8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0c8550 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe99400d830 tx=0x7fe99400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 shutdown_connections 2024-04-25T10:07:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe98806b4d0 0x7fe98806d990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac15db70 0x7fe9ac0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac0d21b0 0x7fe9ac0c8550 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 --2- 172.21.15.5:0/3276663219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe9ac0cefb0 0x7fe9ac1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.735+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 >> 172.21.15.5:0/3276663219 conn(0x7fe9ac0b5370 msgr2=0x7fe9ac0b9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.736+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 shutdown_connections 2024-04-25T10:07:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.736+0000 7fe9b2e00640 1 -- 172.21.15.5:0/3276663219 wait complete. 2024-04-25T10:07:19.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.953+0000 7fa7ea000640 1 Processor -- start 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.953+0000 7fa7ea000640 1 -- start start 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.954+0000 7fa7ea000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40cf750 0x7fa7e406c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.954+0000 7fa7ea000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa7e40d0d60 0x7fa7e406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.954+0000 7fa7ea000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e406c120 con 0x7fa7e415da00 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.954+0000 7fa7ea000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e40d08d0 con 0x7fa7e40d0d60 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.954+0000 7fa7ea000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40cfcc0 con 0x7fa7e40cf750 2024-04-25T10:07:19.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa7e40d0d60 0x7fa7e406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa7e40d0d60 0x7fa7e406cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55130/0 (socket says 172.21.15.5:55130) 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 -- 172.21.15.5:0/4113927626 learned_addr learned my addr 172.21.15.5:0/4113927626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e3e00640 1 --2- 172.21.15.5:0/4113927626 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40cf750 0x7fa7e406c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 -- 172.21.15.5:0/4113927626 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40cf750 msgr2=0x7fa7e406c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 --2- 172.21.15.5:0/4113927626 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40cf750 0x7fa7e406c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 -- 172.21.15.5:0/4113927626 >> v1:172.21.15.80:6789/0 conn(0x7fa7e415da00 legacy=0x7fa7e4161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.955+0000 7fa7e9600640 1 -- 172.21.15.5:0/4113927626 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40cffa0 con 0x7fa7e40d0d60 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.957+0000 7fa7e9600640 1 --2- 172.21.15.5:0/4113927626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e40d0d60 0x7fa7e406cb40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009e50 tx=0x7fa7dc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92253af74c2c081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.957+0000 7fa7e3400640 1 -- 172.21.15.5:0/4113927626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc036070 con 0x7fa7e40d0d60 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.957+0000 7fa7e3400640 1 -- 172.21.15.5:0/4113927626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc031600 con 0x7fa7e40d0d60 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.957+0000 7fa7e3400640 1 -- 172.21.15.5:0/4113927626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0319d0 con 0x7fa7e40d0d60 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 -- 172.21.15.5:0/4113927626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e40d0d60 msgr2=0x7fa7e406cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 --2- 172.21.15.5:0/4113927626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e40d0d60 0x7fa7e406cb40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009e50 tx=0x7fa7dc02a630 comp rx=0 tx=0).stop 2024-04-25T10:07:19.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 -- 172.21.15.5:0/4113927626 shutdown_connections 2024-04-25T10:07:19.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 --2- 172.21.15.5:0/4113927626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e40d0d60 0x7fa7e406cb40 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 --2- 172.21.15.5:0/4113927626 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40cf750 0x7fa7e406c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 -- 172.21.15.5:0/4113927626 >> 172.21.15.5:0/4113927626 conn(0x7fa7e40b3000 msgr2=0x7fa7e40b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 -- 172.21.15.5:0/4113927626 shutdown_connections 2024-04-25T10:07:19.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.958+0000 7fa7ea000640 1 -- 172.21.15.5:0/4113927626 wait complete. 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.960+0000 7fa7ea000640 1 Processor -- start 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.961+0000 7fa7ea000640 1 -- start start 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7ea000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7e40cf750 0x7fa7e41f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7ea000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40d0d60 0x7fa7e40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7ea000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 0x7fa7e40b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7ea000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e415dde0 con 0x7fa7e415da00 2024-04-25T10:07:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7ea000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40d0560 con 0x7fa7e40cf750 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7ea000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b0e70 con 0x7fa7e40d0d60 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7e9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 0x7fa7e40b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7e9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 0x7fa7e40b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55140/0 (socket says 172.21.15.5:55140) 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7e9600640 1 -- 172.21.15.5:0/485578584 learned_addr learned my addr 172.21.15.5:0/485578584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7e8c00640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7e40cf750 0x7fa7e41f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.962+0000 7fa7e3e00640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40d0d60 0x7fa7e40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e9600640 1 -- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40d0d60 msgr2=0x7fa7e40afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e9600640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40d0d60 0x7fa7e40afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e9600640 1 -- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7e40cf750 msgr2=0x7fa7e41f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e9600640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7e40cf750 0x7fa7e41f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e9600640 1 -- 172.21.15.5:0/485578584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40b65d0 con 0x7fa7e415da00 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e9600640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 0x7fa7e40b6090 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc034040 tx=0x7fa7dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc036050 con 0x7fa7e415da00 2024-04-25T10:07:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc042070 con 0x7fa7e415da00 2024-04-25T10:07:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc009510 con 0x7fa7e415da00 2024-04-25T10:07:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.963+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e41f4160 con 0x7fa7e415da00 2024-04-25T10:07:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.964+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e41f4720 con 0x7fa7e415da00 2024-04-25T10:07:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.965+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e4068f60 con 0x7fa7e415da00 2024-04-25T10:07:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.966+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7dc007bd0 con 0x7fa7e415da00 2024-04-25T10:07:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.967+0000 7fa7e0c00640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.967+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7dc0ab1d0 con 0x7fa7e415da00 2024-04-25T10:07:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.967+0000 7fa7e8c00640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.969+0000 7fa7e8c00640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa7c406b4d0 0x7fa7c406d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8006fd0 tx=0x7fa7d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:19.971+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7e40b65d0 con 0x7fa7e415da00 2024-04-25T10:07:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.169+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7e406f2f0 con 0x7fa7e415da00 2024-04-25T10:07:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.170+0000 7fa7e0c00640 1 -- 172.21.15.5:0/485578584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7dc079f50 con 0x7fa7e415da00 2024-04-25T10:07:20.175 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.172+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa7c406b4d0 msgr2=0x7fa7c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.172+0000 7fa7ea000640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa7c406b4d0 0x7fa7c406d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8006fd0 tx=0x7fa7d8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.172+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 msgr2=0x7fa7e40b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.172+0000 7fa7ea000640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 0x7fa7e40b6090 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc034040 tx=0x7fa7dc004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 shutdown_connections 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7e415da00 0x7fa7e40b6090 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa7e40d0d60 0x7fa7e40afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 --2- 172.21.15.5:0/485578584 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7e40cf750 0x7fa7e41f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 >> 172.21.15.5:0/485578584 conn(0x7fa7e40b3000 msgr2=0x7fa7e4155bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 shutdown_connections 2024-04-25T10:07:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.173+0000 7fa7ea000640 1 -- 172.21.15.5:0/485578584 wait complete. 2024-04-25T10:07:20.197 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.406+0000 7f5e2a800640 1 Processor -- start 2024-04-25T10:07:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.406+0000 7f5e2a800640 1 -- start start 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.408+0000 7f5e2a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 0x7f5e24162ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.408+0000 7f5e2a800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5e24071000 0x7f5e24071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.408+0000 7f5e2a800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e241630e0 con 0x7f5e240b2420 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.408+0000 7f5e2a800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e240719a0 con 0x7f5e24071000 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.408+0000 7f5e2a800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406ded0 con 0x7f5e2415f800 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.408+0000 7f5e29e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5e240b2420 0x7f5e240b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50532/0 (socket says 172.21.15.5:50532) 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e29e00640 1 -- 172.21.15.5:0/3705959784 learned_addr learned my addr 172.21.15.5:0/3705959784 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3705959784 >> v2:172.21.15.80:3300/0 conn(0x7f5e24071000 0x7f5e24071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e29400640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 0x7f5e24162ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3604597919 0 0) 0x7f5e1c001490 con 0x7f5e240b2420 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e0c003520 con 0x7f5e240b2420 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1364381339 0 0) 0x7f5e0c003520 con 0x7f5e240b2420 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 msgr2=0x7f5e24162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 0x7f5e24162ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 >> v2:172.21.15.80:3300/0 conn(0x7f5e24071000 msgr2=0x7f5e24071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 --2- 172.21.15.5:0/3705959784 >> v2:172.21.15.80:3300/0 conn(0x7f5e24071000 0x7f5e24071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2406d020 con 0x7f5e240b2420 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.409+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f5e1c001bc0 con 0x7f5e240b2420 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 >> v1:172.21.15.5:6789/0 conn(0x7f5e240b2420 legacy=0x7f5e240b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 0x7f5e0c0039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c003f50 0x7f5e0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e0c004d40 0x7f5e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406ded0 con 0x7f5e0c003f50 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c00b9c0 con 0x7f5e0c004d40 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c00bbc0 con 0x7f5e2415f800 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3705959784 >> v2:172.21.15.80:3300/0 conn(0x7f5e24071000 0x7f5e24071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c003f50 0x7f5e0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e29400640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 0x7f5e0c0039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e29e00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e0c004d40 0x7f5e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e28a00640 1 -- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 msgr2=0x7f5e0c0039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e2415f800 0x7f5e0c0039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e28a00640 1 -- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e0c004d40 msgr2=0x7f5e0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.410+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e0c004d40 0x7f5e0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.411+0000 7f5e28a00640 1 -- 172.21.15.5:0/3705959784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c003520 con 0x7f5e0c003f50 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.411+0000 7f5e29e00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e0c004d40 0x7f5e0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c003f50 0x7f5e0c004800 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098a0 tx=0x7f5e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b977da03f3fca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10037070 con 0x7f5e0c003f50 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1002ad20 con 0x7f5e0c003f50 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e23e00640 1 -- 172.21.15.5:0/3705959784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000f040 con 0x7f5e0c003f50 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e2a800640 1 -- 172.21.15.5:0/3705959784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c003f50 msgr2=0x7f5e0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3705959784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c003f50 0x7f5e0c004800 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5e100098a0 tx=0x7f5e0c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e29e00640 1 -- 172.21.15.5:0/3705959784 reap_dead start 2024-04-25T10:07:20.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e2a800640 1 -- 172.21.15.5:0/3705959784 shutdown_connections 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.412+0000 7f5e2a800640 1 -- 172.21.15.5:0/3705959784 >> 172.21.15.5:0/3705959784 conn(0x7f5e24066f20 msgr2=0x7f5e2406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.413+0000 7f5e2a800640 1 -- 172.21.15.5:0/3705959784 shutdown_connections 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.413+0000 7f5e2a800640 1 -- 172.21.15.5:0/3705959784 wait complete. 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.414+0000 7f5e2a800640 1 Processor -- start 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 -- start start 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e24159090 0x7f5e240ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 0x7f5e2415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e24159470 0x7f5e2415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406d5a0 con 0x7f5e240cd090 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e240b4850 con 0x7f5e24159470 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.415+0000 7f5e2a800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2406d880 con 0x7f5e24159090 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 0x7f5e2415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 0x7f5e2415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55156/0 (socket says 172.21.15.5:55156) 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 -- 172.21.15.5:0/3310282890 learned_addr learned my addr 172.21.15.5:0/3310282890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 -- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e24159090 msgr2=0x7f5e240ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e24159090 0x7f5e240ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e29e00640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e24159470 0x7f5e2415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 -- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e24159470 msgr2=0x7f5e2415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e24159470 0x7f5e2415cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e28a00640 1 -- 172.21.15.5:0/3310282890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240cd550 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.416+0000 7f5e29e00640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e24159470 0x7f5e2415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.417+0000 7f5e28a00640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 0x7f5e2415c9f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5e10035040 tx=0x7f5e10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.417+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10040020 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.417+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10038040 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.417+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e240cd830 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.417+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10037070 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.417+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e240cddf0 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.418+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2406b890 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.419+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e10043020 con 0x7f5e240cd090 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.420+0000 7f5e21600640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.420+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e100aa2f0 con 0x7f5e240cd090 2024-04-25T10:07:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.421+0000 7f5e29400640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.422+0000 7f5e29400640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e0406b4d0 0x7f5e0406d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5e241fb500 tx=0x7f5e18008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.425+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e2406b890 con 0x7f5e240cd090 2024-04-25T10:07:20.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.624+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e240cc9d0 con 0x7f5e240cd090 2024-04-25T10:07:20.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.625+0000 7f5e21600640 1 -- 172.21.15.5:0/3310282890 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e240cc9d0 con 0x7f5e240cd090 2024-04-25T10:07:20.631 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:07:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e0406b4d0 msgr2=0x7f5e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e0406b4d0 0x7f5e0406d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5e241fb500 tx=0x7f5e18008040 comp rx=0 tx=0).stop 2024-04-25T10:07:20.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 msgr2=0x7f5e2415c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 0x7f5e2415c9f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5e10035040 tx=0x7f5e10004730 comp rx=0 tx=0).stop 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 shutdown_connections 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e0406b4d0 0x7f5e0406d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e24159470 0x7f5e2415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e240cd090 0x7f5e2415c9f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 --2- 172.21.15.5:0/3310282890 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e24159090 0x7f5e240ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 >> 172.21.15.5:0/3310282890 conn(0x7f5e24066f20 msgr2=0x7f5e240afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.628+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 shutdown_connections 2024-04-25T10:07:20.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.629+0000 7f5e2a800640 1 -- 172.21.15.5:0/3310282890 wait complete. 2024-04-25T10:07:20.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.859+0000 7fe3fd400640 1 Processor -- start 2024-04-25T10:07:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.859+0000 7fe3fd400640 1 -- start start 2024-04-25T10:07:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3fd400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3f80d0a50 0x7fe3f80d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3fd400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f80d1e80 con 0x7fe3f815db00 2024-04-25T10:07:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3fd400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f80b5930 con 0x7fe3f80b53b0 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3fd400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b5c10 con 0x7fe3f80d0a50 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3f6800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe3f815db00 0x7fe3f80d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50548/0 (socket says 172.21.15.5:50548) 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3f5e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3f80d0a50 0x7fe3f80d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.861+0000 7fe3f6800640 1 -- 172.21.15.5:0/3796313107 learned_addr learned my addr 172.21.15.5:0/3796313107 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.862+0000 7fe3f5e00640 1 -- 172.21.15.5:0/3796313107 >> v1:172.21.15.80:6789/0 conn(0x7fe3f80b53b0 legacy=0x7fe3f80d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.862+0000 7fe3f5e00640 1 -- 172.21.15.5:0/3796313107 >> v1:172.21.15.5:6789/0 conn(0x7fe3f815db00 legacy=0x7fe3f80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.862+0000 7fe3f5e00640 1 -- 172.21.15.5:0/3796313107 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f80b5ef0 con 0x7fe3f80d0a50 2024-04-25T10:07:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.863+0000 7fe3f5e00640 1 --2- 172.21.15.5:0/3796313107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80d0a50 0x7fe3f80d2330 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40097a0 tx=0x7fe3e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61c13e0624d1ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.863+0000 7fe3f5400640 1 -- 172.21.15.5:0/3796313107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4037070 con 0x7fe3f80d0a50 2024-04-25T10:07:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.863+0000 7fe3f5400640 1 -- 172.21.15.5:0/3796313107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4032440 con 0x7fe3f80d0a50 2024-04-25T10:07:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.863+0000 7fe3f5400640 1 -- 172.21.15.5:0/3796313107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4032810 con 0x7fe3f80d0a50 2024-04-25T10:07:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 -- 172.21.15.5:0/3796313107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80d0a50 msgr2=0x7fe3f80d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 --2- 172.21.15.5:0/3796313107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80d0a50 0x7fe3f80d2330 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40097a0 tx=0x7fe3e402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 -- 172.21.15.5:0/3796313107 shutdown_connections 2024-04-25T10:07:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 --2- 172.21.15.5:0/3796313107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80d0a50 0x7fe3f80d2330 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 -- 172.21.15.5:0/3796313107 >> 172.21.15.5:0/3796313107 conn(0x7fe3f8072290 msgr2=0x7fe3f80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 -- 172.21.15.5:0/3796313107 shutdown_connections 2024-04-25T10:07:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.864+0000 7fe3fd400640 1 -- 172.21.15.5:0/3796313107 wait complete. 2024-04-25T10:07:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.865+0000 7fe3fd400640 1 Processor -- start 2024-04-25T10:07:20.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.865+0000 7fe3fd400640 1 -- start start 2024-04-25T10:07:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.866+0000 7fe3fd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 0x7fe3f8155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.866+0000 7fe3fd400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f80d0a50 0x7fe3f815a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.866+0000 7fe3fd400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3f815db00 0x7fe3f81509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.866+0000 7fe3fd400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80d3660 con 0x7fe3f80b53b0 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.866+0000 7fe3fd400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80b61d0 con 0x7fe3f80d0a50 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.866+0000 7fe3fd400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80d2f80 con 0x7fe3f815db00 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 0x7fe3f8155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 0x7fe3f8155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55166/0 (socket says 172.21.15.5:55166) 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 -- 172.21.15.5:0/2374818146 learned_addr learned my addr 172.21.15.5:0/2374818146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f5e00640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f80d0a50 0x7fe3f815a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f7200640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3f815db00 0x7fe3f81509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 -- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3f815db00 msgr2=0x7fe3f81509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3f815db00 0x7fe3f81509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 -- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f80d0a50 msgr2=0x7fe3f815a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f80d0a50 0x7fe3f815a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f6800640 1 -- 172.21.15.5:0/2374818146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f81564f0 con 0x7fe3f80b53b0 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.867+0000 7fe3f7200640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3f815db00 0x7fe3f81509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.868+0000 7fe3f6800640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 0x7fe3f8155bf0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800c3f0 tx=0x7fe3e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.868+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8017070 con 0x7fe3f80b53b0 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.868+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e8005ce0 con 0x7fe3f80b53b0 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.868+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80072f0 con 0x7fe3f80b53b0 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.868+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f81567d0 con 0x7fe3f80b53b0 2024-04-25T10:07:20.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.868+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8156d90 con 0x7fe3f80b53b0 2024-04-25T10:07:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.869+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f80d05b0 con 0x7fe3f80b53b0 2024-04-25T10:07:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.870+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3e801a070 con 0x7fe3f80b53b0 2024-04-25T10:07:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.871+0000 7fe3eea00640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe3d806b4d0 0x7fe3d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.871+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3e80897c0 con 0x7fe3f80b53b0 2024-04-25T10:07:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.872+0000 7fe3f5e00640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe3d806b4d0 0x7fe3d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.873+0000 7fe3f5e00640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe3d806b4d0 0x7fe3d806d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4034000 tx=0x7fe3e402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:20.875+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3e8058410 con 0x7fe3f80b53b0 2024-04-25T10:07:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.085+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3f80d16b0 con 0x7fe3f80b53b0 2024-04-25T10:07:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.086+0000 7fe3eea00640 1 -- 172.21.15.5:0/2374818146 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe3e8057d10 con 0x7fe3f80b53b0 2024-04-25T10:07:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe3d806b4d0 msgr2=0x7fe3d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe3d806b4d0 0x7fe3d806d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4034000 tx=0x7fe3e402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 msgr2=0x7fe3f8155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 0x7fe3f8155bf0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800c3f0 tx=0x7fe3e800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 shutdown_connections 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3f815db00 0x7fe3f81509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe3d806b4d0 0x7fe3d806d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f80d0a50 0x7fe3f815a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 --2- 172.21.15.5:0/2374818146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3f80b53b0 0x7fe3f8155bf0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 >> 172.21.15.5:0/2374818146 conn(0x7fe3f8072290 msgr2=0x7fe3f80b6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 shutdown_connections 2024-04-25T10:07:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.089+0000 7fe3fd400640 1 -- 172.21.15.5:0/2374818146 wait complete. 2024-04-25T10:07:21.115 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:21.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:21.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:21.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:07:21.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:21.268 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:21.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.531+0000 7ffaaae00640 1 Processor -- start 2024-04-25T10:07:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.531+0000 7ffaaae00640 1 -- start start 2024-04-25T10:07:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.533+0000 7ffaaae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.533+0000 7ffaaae00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffaa40d07d0 0x7ffaa40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.533+0000 7ffaaae00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa40cf410 con 0x7ffaa40d2380 2024-04-25T10:07:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.533+0000 7ffaaae00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40d27c0 con 0x7ffaa40d07d0 2024-04-25T10:07:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.533+0000 7ffaaae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40b28b0 con 0x7ffaa4161200 2024-04-25T10:07:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.533+0000 7ffaaa400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffaa40d07d0 0x7ffaa40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaa9a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaaa400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffaa40d07d0 0x7ffaa40b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:38080/0 (socket says 172.21.15.5:38080) 2024-04-25T10:07:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaaa400640 1 -- 172.21.15.5:0/1046630370 learned_addr learned my addr 172.21.15.5:0/1046630370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaaa400640 1 -- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 msgr2=0x7ffaa40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaaa400640 1 --2- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaaa400640 1 -- 172.21.15.5:0/1046630370 >> v1:172.21.15.80:6789/0 conn(0x7ffaa40d2380 legacy=0x7ffaa40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.534+0000 7ffaaa400640 1 -- 172.21.15.5:0/1046630370 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40cf6f0 con 0x7ffaa40d07d0 2024-04-25T10:07:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.536+0000 7ffaaa400640 1 --2- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b2f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0098a0 tx=0x7ffa9c02a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f4c5a557e6e8b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.536+0000 7ffaa3e00640 1 -- 172.21.15.5:0/1046630370 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c037070 con 0x7ffaa40d07d0 2024-04-25T10:07:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.536+0000 7ffaa3e00640 1 -- 172.21.15.5:0/1046630370 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c032440 con 0x7ffaa40d07d0 2024-04-25T10:07:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.536+0000 7ffaa3e00640 1 -- 172.21.15.5:0/1046630370 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c032810 con 0x7ffaa40d07d0 2024-04-25T10:07:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 -- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d07d0 msgr2=0x7ffaa40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 --2- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b2f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0098a0 tx=0x7ffa9c02a290 comp rx=0 tx=0).stop 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 -- 172.21.15.5:0/1046630370 shutdown_connections 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 --2- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b2f80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 --2- 172.21.15.5:0/1046630370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 -- 172.21.15.5:0/1046630370 >> 172.21.15.5:0/1046630370 conn(0x7ffaa408af80 msgr2=0x7ffaa40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 -- 172.21.15.5:0/1046630370 shutdown_connections 2024-04-25T10:07:21.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.537+0000 7ffaaae00640 1 -- 172.21.15.5:0/1046630370 wait complete. 2024-04-25T10:07:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.539+0000 7ffaaae00640 1 Processor -- start 2024-04-25T10:07:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.539+0000 7ffaaae00640 1 -- start start 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaaae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaaae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d2380 0x7ffaa40b8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaaae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaaae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40b34c0 con 0x7ffaa40d07d0 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaaae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40d42b0 con 0x7ffaa40d2380 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaaae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40b3640 con 0x7ffaa4161200 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaa9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaa9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55174/0 (socket says 172.21.15.5:55174) 2024-04-25T10:07:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.540+0000 7ffaa9a00640 1 -- 172.21.15.5:0/3074651545 learned_addr learned my addr 172.21.15.5:0/3074651545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaaa400640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9000640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d2380 0x7ffaa40b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9a00640 1 -- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 msgr2=0x7ffaa40b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9a00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9a00640 1 -- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d2380 msgr2=0x7ffaa40b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9a00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d2380 0x7ffaa40b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9a00640 1 -- 172.21.15.5:0/3074651545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40b7ba0 con 0x7ffaa40d07d0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.541+0000 7ffaa9a00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b62d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400efc0 tx=0x7ffa9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.542+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94018020 con 0x7ffaa40d07d0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.542+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa94014480 con 0x7ffaa40d07d0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.542+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94014850 con 0x7ffaa40d07d0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.542+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa40b6b00 con 0x7ffaa40d07d0 2024-04-25T10:07:21.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.542+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa40b70c0 con 0x7ffaa40d07d0 2024-04-25T10:07:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.543+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa4068f60 con 0x7ffaa40d07d0 2024-04-25T10:07:21.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.544+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa94005d10 con 0x7ffaa40d07d0 2024-04-25T10:07:21.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.545+0000 7ffaa1600640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa8406b4d0 0x7ffa8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.545+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa9408b240 con 0x7ffaa40d07d0 2024-04-25T10:07:21.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.546+0000 7ffaa9000640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa8406b4d0 0x7ffa8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.547+0000 7ffaa9000640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa8406b4d0 0x7ffa8406d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ffaa415da50 tx=0x7ffa90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.550+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa94022030 con 0x7ffaa40d07d0 2024-04-25T10:07:21.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.782+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa40b8530 con 0x7ffaa40d07d0 2024-04-25T10:07:21.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.783+0000 7ffaa1600640 1 -- 172.21.15.5:0/3074651545 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4928 (secure 0 0 0) 0x7ffa94010040 con 0x7ffaa40d07d0 2024-04-25T10:07:21.789 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:21.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:12.593375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5920},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5920":{"gid":5920,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1098619647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1098619647},{"type":"v1","addr":"172.21.15.80:6835","nonce":1098619647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:21.789 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-04-25T10:07:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.785+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa8406b4d0 msgr2=0x7ffa8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.785+0000 7ffaaae00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa8406b4d0 0x7ffa8406d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ffaa415da50 tx=0x7ffa90009040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.785+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 msgr2=0x7ffaa40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.785+0000 7ffaaae00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b62d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400efc0 tx=0x7ffa9400c490 comp rx=0 tx=0).stop 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 shutdown_connections 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaa4161200 0x7ffaa40b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa8406b4d0 0x7ffa8406d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaa40d2380 0x7ffaa40b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 --2- 172.21.15.5:0/3074651545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d07d0 0x7ffaa40b62d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 >> 172.21.15.5:0/3074651545 conn(0x7ffaa408af80 msgr2=0x7ffaa415fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 shutdown_connections 2024-04-25T10:07:21.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:21.786+0000 7ffaaae00640 1 -- 172.21.15.5:0/3074651545 wait complete. 2024-04-25T10:07:21.811 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-25T10:07:22.060 INFO:teuthology.orchestra.run.smithi080.stdout:{"message":"","return_code":0} 2024-04-25T10:07:22.060 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:22.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:22.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.270+0000 7fc098a00640 1 Processor -- start 2024-04-25T10:07:22.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.270+0000 7fc098a00640 1 -- start start 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc098a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc09406e970 0x7fc0940cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc098a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc0940ced20 0x7fc09415b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc098a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940cdbf0 con 0x7fc094144100 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc098a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0941444e0 con 0x7fc09406e970 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc098a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09415ac70 con 0x7fc0940ced20 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc091e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc094144100 0x7fc09406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50552/0 (socket says 172.21.15.5:50552) 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.271+0000 7fc091e00640 1 -- 172.21.15.5:0/91621534 learned_addr learned my addr 172.21.15.5:0/91621534 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc091400640 1 --2- 172.21.15.5:0/91621534 >> v2:172.21.15.5:3300/0 conn(0x7fc09406e970 0x7fc0940cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc092800640 1 --2- 172.21.15.5:0/91621534 >> v2:172.21.15.80:3300/0 conn(0x7fc0940ced20 0x7fc09415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc090a00640 1 -- 172.21.15.5:0/91621534 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123127510 0 0) 0x7fc0940cdbf0 con 0x7fc094144100 2024-04-25T10:07:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc090a00640 1 -- 172.21.15.5:0/91621534 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003520 con 0x7fc094144100 2024-04-25T10:07:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc091400640 1 -- 172.21.15.5:0/91621534 >> v2:172.21.15.80:3300/0 conn(0x7fc0940ced20 msgr2=0x7fc09415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc091400640 1 --2- 172.21.15.5:0/91621534 >> v2:172.21.15.80:3300/0 conn(0x7fc0940ced20 0x7fc09415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc091400640 1 -- 172.21.15.5:0/91621534 >> v1:172.21.15.5:6789/0 conn(0x7fc094144100 legacy=0x7fc09406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.272+0000 7fc091400640 1 -- 172.21.15.5:0/91621534 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940cded0 con 0x7fc09406e970 2024-04-25T10:07:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.273+0000 7fc091400640 1 --2- 172.21.15.5:0/91621534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc09406e970 0x7fc0940cd610 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc0940710e0 tx=0x7fc08002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c14ec2412eb20d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.274+0000 7fc090a00640 1 -- 172.21.15.5:0/91621534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080037070 con 0x7fc09406e970 2024-04-25T10:07:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.274+0000 7fc090a00640 1 -- 172.21.15.5:0/91621534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080032440 con 0x7fc09406e970 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.274+0000 7fc090a00640 1 -- 172.21.15.5:0/91621534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08002ae60 con 0x7fc09406e970 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.274+0000 7fc098a00640 1 -- 172.21.15.5:0/91621534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc09406e970 msgr2=0x7fc0940cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.274+0000 7fc098a00640 1 --2- 172.21.15.5:0/91621534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc09406e970 0x7fc0940cd610 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc0940710e0 tx=0x7fc08002a110 comp rx=0 tx=0).stop 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.275+0000 7fc098a00640 1 -- 172.21.15.5:0/91621534 shutdown_connections 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.275+0000 7fc098a00640 1 --2- 172.21.15.5:0/91621534 >> v2:172.21.15.80:3300/0 conn(0x7fc0940ced20 0x7fc09415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.275+0000 7fc098a00640 1 --2- 172.21.15.5:0/91621534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc09406e970 0x7fc0940cd610 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.275+0000 7fc098a00640 1 -- 172.21.15.5:0/91621534 >> 172.21.15.5:0/91621534 conn(0x7fc094071150 msgr2=0x7fc094071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.275+0000 7fc098a00640 1 -- 172.21.15.5:0/91621534 shutdown_connections 2024-04-25T10:07:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.275+0000 7fc098a00640 1 -- 172.21.15.5:0/91621534 wait complete. 2024-04-25T10:07:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.276+0000 7fc098a00640 1 Processor -- start 2024-04-25T10:07:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.277+0000 7fc098a00640 1 -- start start 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.277+0000 7fc098a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09406e970 0x7fc09413cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.277+0000 7fc098a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 0x7fc09415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.277+0000 7fc098a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc094144100 0x7fc094141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc098a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cea00 con 0x7fc0940ced20 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc098a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940ce490 con 0x7fc094144100 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc098a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09415b670 con 0x7fc09406e970 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 0x7fc09415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 0x7fc09415a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55200/0 (socket says 172.21.15.5:55200) 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 -- 172.21.15.5:0/396084587 learned_addr learned my addr 172.21.15.5:0/396084587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 -- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09406e970 msgr2=0x7fc09413cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09406e970 0x7fc09413cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 -- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc094144100 msgr2=0x7fc094141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc094144100 0x7fc094141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.278+0000 7fc091400640 1 -- 172.21.15.5:0/396084587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940b11a0 con 0x7fc0940ced20 2024-04-25T10:07:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.279+0000 7fc091400640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 0x7fc09415a040 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc080009890 tx=0x7fc0800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.279+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080037040 con 0x7fc0940ced20 2024-04-25T10:07:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.279+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080042070 con 0x7fc0940ced20 2024-04-25T10:07:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.279+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080030960 con 0x7fc0940ced20 2024-04-25T10:07:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.279+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09413d5f0 con 0x7fc0940ced20 2024-04-25T10:07:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.279+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09413dbb0 con 0x7fc0940ced20 2024-04-25T10:07:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.280+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc094068fd0 con 0x7fc0940ced20 2024-04-25T10:07:22.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.281+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc080030b00 con 0x7fc0940ced20 2024-04-25T10:07:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.282+0000 7fc08a000640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b4d0 0x7fc07406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.282+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0800aad90 con 0x7fc0940ced20 2024-04-25T10:07:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.283+0000 7fc091e00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b4d0 0x7fc07406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.284+0000 7fc091e00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b4d0 0x7fc07406d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc084000f60 tx=0x7fc0840078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.286+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc080045360 con 0x7fc0940ced20 2024-04-25T10:07:22.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.530+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc09406e4a0 con 0x7fc0940ced20 2024-04-25T10:07:22.536 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:07:22.531+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.534+0000 7fc08a000640 1 -- 172.21.15.5:0/396084587 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc0800799e0 con 0x7fc0940ced20 2024-04-25T10:07:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.538+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b4d0 msgr2=0x7fc07406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.538+0000 7fc098a00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b4d0 0x7fc07406d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc084000f60 tx=0x7fc0840078f0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.538+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 msgr2=0x7fc09415a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.538+0000 7fc098a00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 0x7fc09415a040 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc080009890 tx=0x7fc0800046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 shutdown_connections 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b4d0 0x7fc07406d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc094144100 0x7fc094141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940ced20 0x7fc09415a040 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 --2- 172.21.15.5:0/396084587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09406e970 0x7fc09413cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 >> 172.21.15.5:0/396084587 conn(0x7fc094071150 msgr2=0x7fc0940b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 shutdown_connections 2024-04-25T10:07:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.539+0000 7fc098a00640 1 -- 172.21.15.5:0/396084587 wait complete. 2024-04-25T10:07:22.563 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:07:22.574 INFO:tasks.ceph.mds.d.smithi080.stderr: -4418> 2024-04-25T10:07:15.992+0000 7f6927400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:22.655 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:07:22.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.660+0000 7f91c1830f40 1 Processor -- start 2024-04-25T10:07:22.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.660+0000 7f91c1830f40 1 -- start start 2024-04-25T10:07:22.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91c1830f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f0124c2db0 0x55f012536ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91c1830f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f0124a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91c1830f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 0x55f01254e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91c1830f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01241fdd0 con 0x55f012535b20 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91c1830f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f012536110 con 0x55f0124c2db0 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91c1830f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f012478550 con 0x55f0124c4da0 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 0x55f01254e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 0x55f01254e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55214/0 (socket says 172.21.15.5:55214) 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.661+0000 7f91bf400640 1 -- 172.21.15.5:0/1162725768 learned_addr learned my addr 172.21.15.5:0/1162725768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bea00640 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f0124c2db0 0x55f012536ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91b6000640 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f0124a9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bf400640 1 -- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 msgr2=0x55f0124a9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bf400640 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f0124a9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bf400640 1 -- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f0124c2db0 msgr2=0x55f012536ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bf400640 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f0124c2db0 0x55f012536ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bf400640 1 -- 172.21.15.5:0/1162725768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0124a76e0 con 0x55f012535b20 2024-04-25T10:07:22.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91b6000640 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f0124a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:22.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bf400640 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 0x55f01254e910 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f91b800a4f0 tx=0x7f91b804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31df16ab9318d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bd600640 1 -- 172.21.15.5:0/1162725768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8065070 con 0x55f012535b20 2024-04-25T10:07:22.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.662+0000 7f91bd600640 1 -- 172.21.15.5:0/1162725768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b80547f0 con 0x55f012535b20 2024-04-25T10:07:22.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91bd600640 1 -- 172.21.15.5:0/1162725768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8059040 con 0x55f012535b20 2024-04-25T10:07:22.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 -- 172.21.15.5:0/1162725768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 msgr2=0x55f01254e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 0x55f01254e910 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f91b800a4f0 tx=0x7f91b804e000 comp rx=0 tx=0).stop 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 -- 172.21.15.5:0/1162725768 shutdown_connections 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f012535b20 0x55f01254e910 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f0124a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 --2- 172.21.15.5:0/1162725768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f0124c2db0 0x55f012536ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 -- 172.21.15.5:0/1162725768 >> 172.21.15.5:0/1162725768 conn(0x55f012477c60 msgr2=0x55f0124a8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 -- 172.21.15.5:0/1162725768 shutdown_connections 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.663+0000 7f91c1830f40 1 -- 172.21.15.5:0/1162725768 wait complete. 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.664+0000 7f91c1830f40 1 Processor -- start 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.664+0000 7f91c1830f40 1 -- start start 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91c1830f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0124c2db0 0x55f012538630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91c1830f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f012530060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91c1830f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f012535b20 0x55f0125305a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91c1830f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0124a9f30 con 0x55f0124c2db0 2024-04-25T10:07:22.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91c1830f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0124a7220 con 0x55f012535b20 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91c1830f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0125371e0 con 0x55f0124c4da0 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91bea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0124c2db0 0x55f012538630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91bea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0124c2db0 0x55f012538630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55220/0 (socket says 172.21.15.5:55220) 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.665+0000 7f91bea00640 1 -- 172.21.15.5:0/2619989381 learned_addr learned my addr 172.21.15.5:0/2619989381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bf400640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f012535b20 0x55f0125305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91b6000640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f012530060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bea00640 1 -- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 msgr2=0x55f012530060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bea00640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f012530060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bea00640 1 -- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f012535b20 msgr2=0x55f0125305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bea00640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f012535b20 0x55f0125305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bea00640 1 -- 172.21.15.5:0/2619989381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f012531fb0 con 0x55f0124c2db0 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91b6000640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f0124c4da0 0x55f012530060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:22.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91bea00640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0124c2db0 0x55f012538630 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f91b0010680 tx=0x7f91b0010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91b7e00640 1 -- 172.21.15.5:0/2619989381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0018020 con 0x55f0124c2db0 2024-04-25T10:07:22.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91b7e00640 1 -- 172.21.15.5:0/2619989381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b000e820 con 0x55f0124c2db0 2024-04-25T10:07:22.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91b7e00640 1 -- 172.21.15.5:0/2619989381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000ebd0 con 0x55f0124c2db0 2024-04-25T10:07:22.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.666+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f012532290 con 0x55f0124c2db0 2024-04-25T10:07:22.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.667+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0125e1500 con 0x55f0124c2db0 2024-04-25T10:07:22.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.669+0000 7f91b7e00640 1 -- 172.21.15.5:0/2619989381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91b001b020 con 0x55f0124c2db0 2024-04-25T10:07:22.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.669+0000 7f91b7e00640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f919c069ee0 0x7f919c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.670+0000 7f91b7e00640 1 -- 172.21.15.5:0/2619989381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f91b0089a50 con 0x55f0124c2db0 2024-04-25T10:07:22.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.670+0000 7f91c1830f40 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f012538b70 0x55f012533280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.670+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78) v9 -- 0x55f01254c190 con 0x55f012538b70 2024-04-25T10:07:22.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.670+0000 7f91b6000640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f919c069ee0 0x7f919c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.670+0000 7f91bf400640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f012538b70 0x55f012533280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.671+0000 7f91b6000640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f919c069ee0 0x7f919c06c3a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55f012535520 tx=0x7f91a8003050 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.671+0000 7f91bf400640 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f012538b70 0x55f012533280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.673+0000 7f91bf400640 1 -- 172.21.15.5:0/2619989381 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v78'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f91b805c040 con 0x55f012538b70 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f012538b70 msgr2=0x55f012533280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f012538b70 0x55f012533280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f919c069ee0 msgr2=0x7f919c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f919c069ee0 0x7f919c06c3a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55f012535520 tx=0x7f91a8003050 comp rx=0 tx=0).stop 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0124c2db0 msgr2=0x55f012538630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 --2- 172.21.15.5:0/2619989381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0124c2db0 0x55f012538630 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f91b0010680 tx=0x7f91b0010b40 comp rx=0 tx=0).stop 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91bf400640 1 -- 172.21.15.5:0/2619989381 reap_dead start 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 shutdown_connections 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 >> 172.21.15.5:0/2619989381 conn(0x55f012477c60 msgr2=0x55f01254e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 shutdown_connections 2024-04-25T10:07:22.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.674+0000 7f91c1830f40 1 -- 172.21.15.5:0/2619989381 wait complete. 2024-04-25T10:07:22.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:07:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.877+0000 7f6c70c00640 1 Processor -- start 2024-04-25T10:07:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.877+0000 7f6c70c00640 1 -- start start 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.878+0000 7f6c70c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.878+0000 7f6c70c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c6c0d3f90 0x7f6c6c161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.878+0000 7f6c70c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c1606e0 con 0x7f6c6c161bc0 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.878+0000 7f6c70c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c1609c0 con 0x7f6c6c0d3f90 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.878+0000 7f6c70c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c160ca0 con 0x7f6c6c0b48c0 2024-04-25T10:07:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c6c0d3f90 0x7f6c6c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c6c0d3f90 0x7f6c6c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55230/0 (socket says 172.21.15.5:55230) 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 -- 172.21.15.5:0/1890611027 learned_addr learned my addr 172.21.15.5:0/1890611027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 -- 172.21.15.5:0/1890611027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 msgr2=0x7f6c6c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 --2- 172.21.15.5:0/1890611027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 -- 172.21.15.5:0/1890611027 >> v1:172.21.15.80:6789/0 conn(0x7f6c6c161bc0 legacy=0x7f6c6c0d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.879+0000 7f6c6aa00640 1 -- 172.21.15.5:0/1890611027 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c160f80 con 0x7f6c6c0d3f90 2024-04-25T10:07:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.880+0000 7f6c6aa00640 1 --2- 172.21.15.5:0/1890611027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c161680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009860 tx=0x7f6c5402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af3dcf0348d5cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.881+0000 7f6c6a000640 1 -- 172.21.15.5:0/1890611027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54037070 con 0x7f6c6c0d3f90 2024-04-25T10:07:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.881+0000 7f6c6a000640 1 -- 172.21.15.5:0/1890611027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c54032440 con 0x7f6c6c0d3f90 2024-04-25T10:07:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.881+0000 7f6c6a000640 1 -- 172.21.15.5:0/1890611027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54032810 con 0x7f6c6c0d3f90 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.881+0000 7f6c70c00640 1 -- 172.21.15.5:0/1890611027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c0d3f90 msgr2=0x7f6c6c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.881+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1890611027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c161680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009860 tx=0x7f6c5402a160 comp rx=0 tx=0).stop 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.882+0000 7f6c70c00640 1 -- 172.21.15.5:0/1890611027 shutdown_connections 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.882+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1890611027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c161680 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.882+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1890611027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c15fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.882+0000 7f6c70c00640 1 -- 172.21.15.5:0/1890611027 >> 172.21.15.5:0/1890611027 conn(0x7f6c6c08af80 msgr2=0x7f6c6c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.882+0000 7f6c70c00640 1 -- 172.21.15.5:0/1890611027 shutdown_connections 2024-04-25T10:07:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.882+0000 7f6c70c00640 1 -- 172.21.15.5:0/1890611027 wait complete. 2024-04-25T10:07:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.884+0000 7f6c70c00640 1 Processor -- start 2024-04-25T10:07:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.884+0000 7f6c70c00640 1 -- start start 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c70c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c70c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c70c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 0x7f6c6c0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c70c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1603a0 con 0x7f6c6c161bc0 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c70c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0b4ca0 con 0x7f6c6c0d3f90 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c70c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0b3b80 con 0x7f6c6c0b48c0 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 0x7f6c6c0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 0x7f6c6c0ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55246/0 (socket says 172.21.15.5:55246) 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.885+0000 7f6c6be00640 1 -- 172.21.15.5:0/1911602160 learned_addr learned my addr 172.21.15.5:0/1911602160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6aa00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6be00640 1 -- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 msgr2=0x7f6c6c0b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6b400640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6be00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c0b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6be00640 1 -- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c6c0d3f90 msgr2=0x7f6c6c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6be00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6be00640 1 -- 172.21.15.5:0/1911602160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0b9000 con 0x7f6c6c161bc0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6aa00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6b400640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c0b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c6be00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 0x7f6c6c0ca6d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400b9a0 tx=0x7f6c6400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64019070 con 0x7f6c6c161bc0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6400cba0 con 0x7f6c6c161bc0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64014440 con 0x7f6c6c161bc0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.886+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c0b7f60 con 0x7f6c6c161bc0 2024-04-25T10:07:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.887+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c0b8520 con 0x7f6c6c161bc0 2024-04-25T10:07:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.888+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c068f60 con 0x7f6c6c161bc0 2024-04-25T10:07:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.889+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c6401c070 con 0x7f6c6c161bc0 2024-04-25T10:07:22.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.889+0000 7f6c63400640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c4806b4d0 0x7f6c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.890+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c6408a0c0 con 0x7f6c6c161bc0 2024-04-25T10:07:22.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.890+0000 7f6c6b400640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c4806b4d0 0x7f6c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.891+0000 7f6c6b400640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c4806b4d0 0x7f6c4806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:22.894+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c64058cc0 con 0x7f6c6c161bc0 2024-04-25T10:07:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.131+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6c6c08d730 con 0x7f6c6c161bc0 2024-04-25T10:07:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.379+0000 7f6c63400640 1 -- 172.21.15.5:0/1911602160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6c640585c0 con 0x7f6c6c161bc0 2024-04-25T10:07:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:07:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.382+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c4806b4d0 msgr2=0x7f6c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.382+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c4806b4d0 0x7f6c4806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.382+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 msgr2=0x7f6c6c0ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.382+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 0x7f6c6c0ca6d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400b9a0 tx=0x7f6c6400be60 comp rx=0 tx=0).stop 2024-04-25T10:07:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 shutdown_connections 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c6c161bc0 0x7f6c6c0ca6d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c4806b4d0 0x7f6c4806d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c6c0d3f90 0x7f6c6c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 --2- 172.21.15.5:0/1911602160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c6c0b48c0 0x7f6c6c0b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 >> 172.21.15.5:0/1911602160 conn(0x7f6c6c08af80 msgr2=0x7f6c6c15dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 shutdown_connections 2024-04-25T10:07:23.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.383+0000 7f6c70c00640 1 -- 172.21.15.5:0/1911602160 wait complete. 2024-04-25T10:07:23.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.612+0000 7f643f000640 1 Processor -- start 2024-04-25T10:07:23.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.612+0000 7f643f000640 1 -- start start 2024-04-25T10:07:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.613+0000 7f643f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f64380d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.613+0000 7f643f000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f64380b53b0 0x7f64380d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.613+0000 7f643f000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64380d1e80 con 0x7f643815db00 2024-04-25T10:07:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.613+0000 7f643f000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64380b5930 con 0x7f64380b53b0 2024-04-25T10:07:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.613+0000 7f643f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380b5c10 con 0x7f64380d0a50 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f64380b53b0 0x7f64380d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f64380b53b0 0x7f64380d2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55248/0 (socket says 172.21.15.5:55248) 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 -- 172.21.15.5:0/1207891640 learned_addr learned my addr 172.21.15.5:0/1207891640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f6437400640 1 --2- 172.21.15.5:0/1207891640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f64380d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 -- 172.21.15.5:0/1207891640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 msgr2=0x7f64380d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 --2- 172.21.15.5:0/1207891640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f64380d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 -- 172.21.15.5:0/1207891640 >> v1:172.21.15.80:6789/0 conn(0x7f643815db00 legacy=0x7f64380d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.614+0000 7f643d000640 1 -- 172.21.15.5:0/1207891640 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380b5ef0 con 0x7f64380b53b0 2024-04-25T10:07:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.615+0000 7f643d000640 1 --2- 172.21.15.5:0/1207891640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64380b53b0 0x7f64380d2870 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6430009870 tx=0x7f643002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcb56850a0f6e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.616+0000 7f6436a00640 1 -- 172.21.15.5:0/1207891640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430037070 con 0x7f64380b53b0 2024-04-25T10:07:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.616+0000 7f6436a00640 1 -- 172.21.15.5:0/1207891640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6430032440 con 0x7f64380b53b0 2024-04-25T10:07:23.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.616+0000 7f6436a00640 1 -- 172.21.15.5:0/1207891640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430032810 con 0x7f64380b53b0 2024-04-25T10:07:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 -- 172.21.15.5:0/1207891640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64380b53b0 msgr2=0x7f64380d2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 --2- 172.21.15.5:0/1207891640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64380b53b0 0x7f64380d2870 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6430009870 tx=0x7f643002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 -- 172.21.15.5:0/1207891640 shutdown_connections 2024-04-25T10:07:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 --2- 172.21.15.5:0/1207891640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64380b53b0 0x7f64380d2870 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 --2- 172.21.15.5:0/1207891640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f64380d2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 -- 172.21.15.5:0/1207891640 >> 172.21.15.5:0/1207891640 conn(0x7f6438072290 msgr2=0x7f64380726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 -- 172.21.15.5:0/1207891640 shutdown_connections 2024-04-25T10:07:23.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.617+0000 7f643f000640 1 -- 172.21.15.5:0/1207891640 wait complete. 2024-04-25T10:07:23.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.618+0000 7f643f000640 1 Processor -- start 2024-04-25T10:07:23.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.619+0000 7f643f000640 1 -- start start 2024-04-25T10:07:23.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643f000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64380b53b0 0x7f64380b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f6438150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 0x7f643815e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380d3660 con 0x7f643815db00 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643f000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64380b61d0 con 0x7f64380b53b0 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64380ce2b0 con 0x7f64380d0a50 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 0x7f643815e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 0x7f643815e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55254/0 (socket says 172.21.15.5:55254) 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643d000640 1 -- 172.21.15.5:0/4286415799 learned_addr learned my addr 172.21.15.5:0/4286415799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f6437400640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f6438150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f6437e00640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64380b53b0 0x7f64380b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643d000640 1 -- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 msgr2=0x7f6438150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.620+0000 7f643d000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f6438150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f643d000640 1 -- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64380b53b0 msgr2=0x7f64380b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f643d000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64380b53b0 0x7f64380b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f643d000640 1 -- 172.21.15.5:0/4286415799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64380b83d0 con 0x7f643815db00 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f643d000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 0x7f643815e0f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f643002a250 tx=0x7f64300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430041020 con 0x7f643815db00 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64300306c0 con 0x7f643815db00 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430037040 con 0x7f643815db00 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64380b86b0 con 0x7f643815db00 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.621+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64380c9500 con 0x7f643815db00 2024-04-25T10:07:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.623+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6438068fd0 con 0x7f643815db00 2024-04-25T10:07:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.624+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6430044020 con 0x7f643815db00 2024-04-25T10:07:23.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.625+0000 7f642fe00640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f641406b4d0 0x7f641406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.625+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f64300aaca0 con 0x7f643815db00 2024-04-25T10:07:23.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.626+0000 7f6437e00640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f641406b4d0 0x7f641406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.627+0000 7f6437e00640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f641406b4d0 0x7f641406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6428009f20 tx=0x7f64280094d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.630+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f643004b030 con 0x7f643815db00 2024-04-25T10:07:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.866+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64380d16b0 con 0x7f643815db00 2024-04-25T10:07:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.867+0000 7f642fe00640 1 -- 172.21.15.5:0/4286415799 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f64300798a0 con 0x7f643815db00 2024-04-25T10:07:23.872 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:23.872 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:23.376096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 48 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f641406b4d0 msgr2=0x7f641406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f641406b4d0 0x7f641406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6428009f20 tx=0x7f64280094d0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 msgr2=0x7f643815e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 0x7f643815e0f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f643002a250 tx=0x7f64300046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 shutdown_connections 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f643815db00 0x7f643815e0f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f64380d0a50 0x7f6438150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f641406b4d0 0x7f641406d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 --2- 172.21.15.5:0/4286415799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64380b53b0 0x7f64380b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 >> 172.21.15.5:0/4286415799 conn(0x7f6438072290 msgr2=0x7f643815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 shutdown_connections 2024-04-25T10:07:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:23.869+0000 7f643f000640 1 -- 172.21.15.5:0/4286415799 wait complete. 2024-04-25T10:07:24.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:09.493208+0000', 'modified': '2024-04-25T10:07:23.376096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6025': {'gid': 6025, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.80:6835/3658047546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 3658047546}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 3658047546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:24.298 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6025:up:replay 2024-04-25T10:07:24.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:24.643 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:25.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.512+0000 7f9885e00640 1 Processor -- start 2024-04-25T10:07:25.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.512+0000 7f9885e00640 1 -- start start 2024-04-25T10:07:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.513+0000 7f9885e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f988015db00 0x7f98800d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.513+0000 7f9885e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 0x7f98800d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.513+0000 7f9885e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98800d1e80 con 0x7f98800ce2b0 2024-04-25T10:07:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.513+0000 7f9885e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98800b5930 con 0x7f988015db00 2024-04-25T10:07:25.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.513+0000 7f9885e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b5c10 con 0x7f98800b53b0 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.514+0000 7f9877e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 0x7f98800d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.514+0000 7f9877e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 0x7f98800d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52314/0 (socket says 172.21.15.5:52314) 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.514+0000 7f9877e00640 1 -- 172.21.15.5:0/3011625008 learned_addr learned my addr 172.21.15.5:0/3011625008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.514+0000 7f987ea00640 1 --2- 172.21.15.5:0/3011625008 >> v2:172.21.15.80:3300/0 conn(0x7f988015db00 0x7f98800d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.515+0000 7f9877e00640 1 -- 172.21.15.5:0/3011625008 >> v2:172.21.15.80:3300/0 conn(0x7f988015db00 msgr2=0x7f98800d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.515+0000 7f9877e00640 1 --2- 172.21.15.5:0/3011625008 >> v2:172.21.15.80:3300/0 conn(0x7f988015db00 0x7f98800d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.515+0000 7f9877e00640 1 -- 172.21.15.5:0/3011625008 >> v1:172.21.15.80:6789/0 conn(0x7f98800ce2b0 legacy=0x7f9880161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.515+0000 7f9877e00640 1 -- 172.21.15.5:0/3011625008 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800b5ef0 con 0x7f98800b53b0 2024-04-25T10:07:25.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.516+0000 7f9877e00640 1 --2- 172.21.15.5:0/3011625008 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 0x7f98800d0880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9868009870 tx=0x7f986802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3e4b7f7ea56836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.516+0000 7f987e000640 1 -- 172.21.15.5:0/3011625008 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868037070 con 0x7f98800b53b0 2024-04-25T10:07:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.516+0000 7f987e000640 1 -- 172.21.15.5:0/3011625008 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9868032440 con 0x7f98800b53b0 2024-04-25T10:07:25.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.516+0000 7f987e000640 1 -- 172.21.15.5:0/3011625008 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868032810 con 0x7f98800b53b0 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 -- 172.21.15.5:0/3011625008 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 msgr2=0x7f98800d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 --2- 172.21.15.5:0/3011625008 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 0x7f98800d0880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9868009870 tx=0x7f986802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 -- 172.21.15.5:0/3011625008 shutdown_connections 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 --2- 172.21.15.5:0/3011625008 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800b53b0 0x7f98800d0880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 --2- 172.21.15.5:0/3011625008 >> v2:172.21.15.80:3300/0 conn(0x7f988015db00 0x7f98800d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 -- 172.21.15.5:0/3011625008 >> 172.21.15.5:0/3011625008 conn(0x7f9880072290 msgr2=0x7f98800726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 -- 172.21.15.5:0/3011625008 shutdown_connections 2024-04-25T10:07:25.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.517+0000 7f9885e00640 1 -- 172.21.15.5:0/3011625008 wait complete. 2024-04-25T10:07:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.519+0000 7f9885e00640 1 Processor -- start 2024-04-25T10:07:25.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.519+0000 7f9885e00640 1 -- start start 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.520+0000 7f9885e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98800b53b0 0x7f98800b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.520+0000 7f9885e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800ce2b0 0x7f98800c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.520+0000 7f9885e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 0x7f9880158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.520+0000 7f9885e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800d3660 con 0x7f988015db00 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.520+0000 7f9885e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b61d0 con 0x7f98800b53b0 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.520+0000 7f9885e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b6c10 con 0x7f98800ce2b0 2024-04-25T10:07:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 0x7f9880158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 0x7f9880158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55266/0 (socket says 172.21.15.5:55266) 2024-04-25T10:07:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 -- 172.21.15.5:0/1941736175 learned_addr learned my addr 172.21.15.5:0/1941736175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987ea00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98800b53b0 0x7f98800b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f9877e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800ce2b0 0x7f98800c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 -- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800ce2b0 msgr2=0x7f98800c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800ce2b0 0x7f98800c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 -- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98800b53b0 msgr2=0x7f98800b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98800b53b0 0x7f98800b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.521+0000 7f987f400640 1 -- 172.21.15.5:0/1941736175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880159930 con 0x7f988015db00 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.522+0000 7f987f400640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 0x7f9880158bc0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f987800c7a0 tx=0x7f987800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.522+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878003b70 con 0x7f988015db00 2024-04-25T10:07:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.522+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878003d30 con 0x7f988015db00 2024-04-25T10:07:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.522+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987800aa30 con 0x7f988015db00 2024-04-25T10:07:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.522+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98800c8960 con 0x7f988015db00 2024-04-25T10:07:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.522+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98800c8f20 con 0x7f988015db00 2024-04-25T10:07:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.523+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9880068fd0 con 0x7f988015db00 2024-04-25T10:07:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.524+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f987800abd0 con 0x7f988015db00 2024-04-25T10:07:25.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.525+0000 7f9877400640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f985c06b4d0 0x7f985c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.525+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f987808a290 con 0x7f988015db00 2024-04-25T10:07:25.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.526+0000 7f987ea00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f985c06b4d0 0x7f985c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.526+0000 7f987ea00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f985c06b4d0 0x7f985c06d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9870009aa0 tx=0x7f9870009190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.530+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98780587b0 con 0x7f988015db00 2024-04-25T10:07:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.767+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98800b9640 con 0x7f988015db00 2024-04-25T10:07:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.768+0000 7f9877400640 1 -- 172.21.15.5:0/1941736175 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f987805c850 con 0x7f988015db00 2024-04-25T10:07:25.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:25.774 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:24.389312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.771+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f985c06b4d0 msgr2=0x7f985c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.771+0000 7f9885e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f985c06b4d0 0x7f985c06d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9870009aa0 tx=0x7f9870009190 comp rx=0 tx=0).stop 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.771+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 msgr2=0x7f9880158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.771+0000 7f9885e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 0x7f9880158bc0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f987800c7a0 tx=0x7f987800cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 shutdown_connections 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f988015db00 0x7f9880158bc0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f98800ce2b0 0x7f98800c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f985c06b4d0 0x7f985c06d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 --2- 172.21.15.5:0/1941736175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98800b53b0 0x7f98800b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 >> 172.21.15.5:0/1941736175 conn(0x7f9880072290 msgr2=0x7f988015c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 shutdown_connections 2024-04-25T10:07:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:25.772+0000 7f9885e00640 1 -- 172.21.15.5:0/1941736175 wait complete. 2024-04-25T10:07:26.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:09.493208+0000', 'modified': '2024-04-25T10:07:24.389312+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6025': {'gid': 6025, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.80:6835/3658047546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 3658047546}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 3658047546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:26.198 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6025:up:rejoin 2024-04-25T10:07:27.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.412+0000 7f8401800640 1 Processor -- start 2024-04-25T10:07:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.412+0000 7f8401800640 1 -- start start 2024-04-25T10:07:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f8401800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83fc15aa10 0x7f83fc0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f8401800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f83fc070240 0x7f83fc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f8401800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0b5360 con 0x7f83fc15adf0 2024-04-25T10:07:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f8401800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc0ceb60 con 0x7f83fc15aa10 2024-04-25T10:07:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f8401800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc0b5640 con 0x7f83fc070240 2024-04-25T10:07:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f83faa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83fc15aa10 0x7f83fc0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f83faa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f83fc15aa10 0x7f83fc0b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47548/0 (socket says 172.21.15.5:47548) 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.414+0000 7f83faa00640 1 -- 172.21.15.5:0/1137443959 learned_addr learned my addr 172.21.15.5:0/1137443959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.415+0000 7f83faa00640 1 -- 172.21.15.5:0/1137443959 >> v2:172.21.15.80:3300/0 conn(0x7f83fc070240 msgr2=0x7f83fc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.415+0000 7f83fb400640 1 --2- 172.21.15.5:0/1137443959 >> v2:172.21.15.80:3300/0 conn(0x7f83fc070240 0x7f83fc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.415+0000 7f83faa00640 1 --2- 172.21.15.5:0/1137443959 >> v2:172.21.15.80:3300/0 conn(0x7f83fc070240 0x7f83fc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.415+0000 7f83faa00640 1 -- 172.21.15.5:0/1137443959 >> v1:172.21.15.80:6789/0 conn(0x7f83fc15adf0 legacy=0x7f83fc06fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.415+0000 7f83faa00640 1 -- 172.21.15.5:0/1137443959 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0b5920 con 0x7f83fc15aa10 2024-04-25T10:07:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.415+0000 7f83fb400640 1 --2- 172.21.15.5:0/1137443959 >> v2:172.21.15.80:3300/0 conn(0x7f83fc070240 0x7f83fc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.416+0000 7f83faa00640 1 --2- 172.21.15.5:0/1137443959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc15aa10 0x7f83fc0b4110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f83fc067500 tx=0x7f83ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec86fdc972ede86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.416+0000 7f83f9600640 1 -- 172.21.15.5:0/1137443959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec037070 con 0x7f83fc15aa10 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.416+0000 7f83f9600640 1 -- 172.21.15.5:0/1137443959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec02ae10 con 0x7f83fc15aa10 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.416+0000 7f83f9600640 1 -- 172.21.15.5:0/1137443959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec032660 con 0x7f83fc15aa10 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 -- 172.21.15.5:0/1137443959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc15aa10 msgr2=0x7f83fc0b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 --2- 172.21.15.5:0/1137443959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc15aa10 0x7f83fc0b4110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f83fc067500 tx=0x7f83ec02a100 comp rx=0 tx=0).stop 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 -- 172.21.15.5:0/1137443959 shutdown_connections 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 --2- 172.21.15.5:0/1137443959 >> v2:172.21.15.80:3300/0 conn(0x7f83fc070240 0x7f83fc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 --2- 172.21.15.5:0/1137443959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc15aa10 0x7f83fc0b4110 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 -- 172.21.15.5:0/1137443959 >> 172.21.15.5:0/1137443959 conn(0x7f83fc0b2400 msgr2=0x7f83fc0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 -- 172.21.15.5:0/1137443959 shutdown_connections 2024-04-25T10:07:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.417+0000 7f8401800640 1 -- 172.21.15.5:0/1137443959 wait complete. 2024-04-25T10:07:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.419+0000 7f8401800640 1 Processor -- start 2024-04-25T10:07:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.419+0000 7f8401800640 1 -- start start 2024-04-25T10:07:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f8401800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 0x7f83fc145850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f8401800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83fc15aa10 0x7f83fc158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f8401800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83fc15adf0 0x7f83fc06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f8401800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0ce870 con 0x7f83fc070240 2024-04-25T10:07:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f8401800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0ce6f0 con 0x7f83fc15adf0 2024-04-25T10:07:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f8401800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0b4650 con 0x7f83fc15aa10 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 0x7f83fc145850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 0x7f83fc145850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47554/0 (socket says 172.21.15.5:47554) 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 -- 172.21.15.5:0/2927844516 learned_addr learned my addr 172.21.15.5:0/2927844516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 -- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83fc15aa10 msgr2=0x7f83fc158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83fc15aa10 0x7f83fc158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 -- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83fc15adf0 msgr2=0x7f83fc06ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83fc15adf0 0x7f83fc06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.421+0000 7f83faa00640 1 -- 172.21.15.5:0/2927844516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc166a70 con 0x7f83fc070240 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.422+0000 7f83faa00640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 0x7f83fc145850 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f83ec000c00 tx=0x7f83ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.422+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec042070 con 0x7f83fc070240 2024-04-25T10:07:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.422+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec03e670 con 0x7f83fc070240 2024-04-25T10:07:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.422+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec037040 con 0x7f83fc070240 2024-04-25T10:07:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.422+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc166d50 con 0x7f83fc070240 2024-04-25T10:07:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.422+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc146830 con 0x7f83fc070240 2024-04-25T10:07:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.423+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc068fd0 con 0x7f83fc070240 2024-04-25T10:07:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.424+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83ec045070 con 0x7f83fc070240 2024-04-25T10:07:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.425+0000 7f83f2a00640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f83dc06b4d0 0x7f83dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.425+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f83ec0aad40 con 0x7f83fc070240 2024-04-25T10:07:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.426+0000 7f83fa000640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f83dc06b4d0 0x7f83dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.427+0000 7f83fa000640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f83dc06b4d0 0x7f83dc06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f83fc14ffb0 tx=0x7f83e8006ba0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.429+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83ec079940 con 0x7f83fc070240 2024-04-25T10:07:27.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.669+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83fc146a60 con 0x7f83fc070240 2024-04-25T10:07:27.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.670+0000 7f83f2a00640 1 -- 172.21.15.5:0/2927844516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f83ec079240 con 0x7f83fc070240 2024-04-25T10:07:27.675 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:27.676 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:25.399714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":4,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:27.676 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 51 2024-04-25T10:07:27.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.672+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f83dc06b4d0 msgr2=0x7f83dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.672+0000 7f8401800640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f83dc06b4d0 0x7f83dc06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f83fc14ffb0 tx=0x7f83e8006ba0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.672+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 msgr2=0x7f83fc145850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.672+0000 7f8401800640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 0x7f83fc145850 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f83ec000c00 tx=0x7f83ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 shutdown_connections 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f83dc06b4d0 0x7f83dc06d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83fc15adf0 0x7f83fc06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83fc15aa10 0x7f83fc158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 --2- 172.21.15.5:0/2927844516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83fc070240 0x7f83fc145850 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 >> 172.21.15.5:0/2927844516 conn(0x7f83fc0b2400 msgr2=0x7f83fc0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 shutdown_connections 2024-04-25T10:07:27.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:27.673+0000 7f8401800640 1 -- 172.21.15.5:0/2927844516 wait complete. 2024-04-25T10:07:28.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:09.493208+0000', 'modified': '2024-04-25T10:07:25.399714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6025': {'gid': 6025, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.80:6835/3658047546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 3658047546}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 3658047546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:28.100 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:28.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:07:28.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.317+0000 7fb48fe00640 1 Processor -- start 2024-04-25T10:07:28.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.317+0000 7fb48fe00640 1 -- start start 2024-04-25T10:07:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb48fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb49015ce00 0x7fb49015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb48fe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb490071d30 con 0x7fb490141da0 2024-04-25T10:07:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb48fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49006d620 con 0x7fb4900cd300 2024-04-25T10:07:28.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb48fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb490142180 con 0x7fb49015ce00 2024-04-25T10:07:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb48ea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb490141da0 0x7fb49006e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59962/0 (socket says 172.21.15.5:59962) 2024-04-25T10:07:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb48ea00640 1 -- 172.21.15.5:0/403829812 learned_addr learned my addr 172.21.15.5:0/403829812 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.319+0000 7fb486000640 1 --2- 172.21.15.5:0/403829812 >> v2:172.21.15.5:3300/0 conn(0x7fb49015ce00 0x7fb49015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.320+0000 7fb48e000640 1 -- 172.21.15.5:0/403829812 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466226769 0 0) 0x7fb490071d30 con 0x7fb490141da0 2024-04-25T10:07:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.320+0000 7fb48e000640 1 -- 172.21.15.5:0/403829812 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb474003520 con 0x7fb490141da0 2024-04-25T10:07:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.320+0000 7fb486000640 1 -- 172.21.15.5:0/403829812 >> v1:172.21.15.80:6789/0 conn(0x7fb4900cd300 legacy=0x7fb490071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.320+0000 7fb486000640 1 -- 172.21.15.5:0/403829812 >> v1:172.21.15.5:6789/0 conn(0x7fb490141da0 legacy=0x7fb49006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.320+0000 7fb486000640 1 -- 172.21.15.5:0/403829812 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49006d900 con 0x7fb49015ce00 2024-04-25T10:07:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.321+0000 7fb486000640 1 --2- 172.21.15.5:0/403829812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015d280 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb4780097a0 tx=0x7fb47802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dc769e8e6ec1ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.321+0000 7fb48e000640 1 -- 172.21.15.5:0/403829812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478037070 con 0x7fb49015ce00 2024-04-25T10:07:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.321+0000 7fb48e000640 1 -- 172.21.15.5:0/403829812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb478032440 con 0x7fb49015ce00 2024-04-25T10:07:28.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.321+0000 7fb48e000640 1 -- 172.21.15.5:0/403829812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478032810 con 0x7fb49015ce00 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 -- 172.21.15.5:0/403829812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 msgr2=0x7fb49015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 --2- 172.21.15.5:0/403829812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015d280 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb4780097a0 tx=0x7fb47802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 -- 172.21.15.5:0/403829812 shutdown_connections 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 --2- 172.21.15.5:0/403829812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015d280 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 -- 172.21.15.5:0/403829812 >> 172.21.15.5:0/403829812 conn(0x7fb490066f20 msgr2=0x7fb49006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 -- 172.21.15.5:0/403829812 shutdown_connections 2024-04-25T10:07:28.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.322+0000 7fb48fe00640 1 -- 172.21.15.5:0/403829812 wait complete. 2024-04-25T10:07:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.324+0000 7fb48fe00640 1 Processor -- start 2024-04-25T10:07:28.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.324+0000 7fb48fe00640 1 -- start start 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.325+0000 7fb48fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4900cd300 0x7fb49013e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.325+0000 7fb48fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb490141da0 0x7fb490159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.325+0000 7fb48fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.325+0000 7fb48fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb490071710 con 0x7fb49015ce00 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.325+0000 7fb48fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49006e810 con 0x7fb4900cd300 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.325+0000 7fb48fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb490071890 con 0x7fb490141da0 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47572/0 (socket says 172.21.15.5:47572) 2024-04-25T10:07:28.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 -- 172.21.15.5:0/1448666191 learned_addr learned my addr 172.21.15.5:0/1448666191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 -- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb490141da0 msgr2=0x7fb490159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb486000640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb490141da0 0x7fb490159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48ea00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4900cd300 0x7fb49013e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb490141da0 0x7fb490159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 -- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4900cd300 msgr2=0x7fb49013e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4900cd300 0x7fb49013e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48f400640 1 -- 172.21.15.5:0/1448666191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900c5410 con 0x7fb49015ce00 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.326+0000 7fb48ea00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4900cd300 0x7fb49013e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.327+0000 7fb48f400640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015a2d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c7a0 tx=0x7fb48800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.327+0000 7fb487400640 1 -- 172.21.15.5:0/1448666191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488003b70 con 0x7fb49015ce00 2024-04-25T10:07:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.327+0000 7fb487400640 1 -- 172.21.15.5:0/1448666191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb488003d30 con 0x7fb49015ce00 2024-04-25T10:07:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.327+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4900c56b0 con 0x7fb49015ce00 2024-04-25T10:07:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.327+0000 7fb487400640 1 -- 172.21.15.5:0/1448666191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800aa90 con 0x7fb49015ce00 2024-04-25T10:07:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.327+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4900c5c70 con 0x7fb49015ce00 2024-04-25T10:07:28.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.329+0000 7fb487400640 1 -- 172.21.15.5:0/1448666191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb48800ac30 con 0x7fb49015ce00 2024-04-25T10:07:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.330+0000 7fb487400640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb46c06b4d0 0x7fb46c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.330+0000 7fb487400640 1 -- 172.21.15.5:0/1448666191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb48808a3c0 con 0x7fb49015ce00 2024-04-25T10:07:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.331+0000 7fb49883d740 1 Processor -- start 2024-04-25T10:07:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.331+0000 7fb48ea00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb46c06b4d0 0x7fb46c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.331+0000 7fb49883d740 1 -- start start 2024-04-25T10:07:28.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281b84b30 0x55d281bac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281b84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb49883d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281b832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb49883d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d281ae8620 con 0x55d281bcd7a0 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d281a91ca0 con 0x55d281b84b30 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d281ae46c0 con 0x55d281baceb0 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb48f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281b832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb48ea00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb46c06b4d0 0x7fb46c06d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb480007fd0 tx=0x7fb480009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb48f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281b832f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47576/0 (socket says 172.21.15.5:47576) 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.332+0000 7fb48f400640 1 -- 172.21.15.5:0/3442247936 learned_addr learned my addr 172.21.15.5:0/3442247936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48ea00640 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281b84b30 0x55d281bac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb486000640 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281b84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48f400640 1 -- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 msgr2=0x55d281b84f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48f400640 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281b84f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48f400640 1 -- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281b84b30 msgr2=0x55d281bac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48f400640 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281b84b30 0x55d281bac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48f400640 1 -- 172.21.15.5:0/3442247936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d281a98f40 con 0x55d281bcd7a0 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb48f400640 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281b832f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb488058850 tx=0x7fb488011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628268cc35545c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb47f400640 1 -- 172.21.15.5:0/3442247936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4880588c0 con 0x55d281bcd7a0 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb47f400640 1 -- 172.21.15.5:0/3442247936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48805ca10 con 0x55d281bcd7a0 2024-04-25T10:07:28.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.333+0000 7fb47f400640 1 -- 172.21.15.5:0/3442247936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488060fd0 con 0x55d281bcd7a0 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 -- 172.21.15.5:0/3442247936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 msgr2=0x55d281b832f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281b832f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb488058850 tx=0x7fb488011000 comp rx=0 tx=0).stop 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 -- 172.21.15.5:0/3442247936 shutdown_connections 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281b832f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281b84f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 --2- 172.21.15.5:0/3442247936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281b84b30 0x55d281bac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 -- 172.21.15.5:0/3442247936 >> 172.21.15.5:0/3442247936 conn(0x55d281b81d50 msgr2=0x55d281bca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 -- 172.21.15.5:0/3442247936 shutdown_connections 2024-04-25T10:07:28.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.334+0000 7fb49883d740 1 -- 172.21.15.5:0/3442247936 wait complete. 2024-04-25T10:07:28.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.335+0000 7fb49883d740 1 -- start start 2024-04-25T10:07:28.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.335+0000 7fb49883d740 1 Processor -- start 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.336+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281bb35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.336+0000 7fb49883d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281bdeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.336+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281ba7d10 0x55d281ba8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.336+0000 7fb49883d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d281bac4e0 con 0x55d281bcd7a0 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.336+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d281a871f0 con 0x55d281ba7d10 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.336+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d281babed0 con 0x55d281baceb0 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb49883d740 10 client.0 _ll_register_callbacks cb 0x55d281aa4ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281bdeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281bdeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47586/0 (socket says 172.21.15.5:47586) 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 -- 172.21.15.5:0/471141882 learned_addr learned my addr 172.21.15.5:0/471141882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 -- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 msgr2=0x55d281bb35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281bb35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 -- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281ba7d10 msgr2=0x55d281ba8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281ba7d10 0x55d281ba8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.337+0000 7fb486000640 1 -- 172.21.15.5:0/471141882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d281aa6c50 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb486000640 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281bdeae0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb478035040 tx=0x7fb478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96111fc2246e85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb473e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb473e00640 1 -- 172.21.15.5:0/471141882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb478037040 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb473e00640 1 -- 172.21.15.5:0/471141882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb478032dd0 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb48805ca10 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb473e00640 1 -- 172.21.15.5:0/471141882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb478006e10 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb49883d740 10 client.7032 fetch_fsmap learned FSMap version 51 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55d2817fd2d0 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.338+0000 7fb473e00640 1 -- 172.21.15.5:0/471141882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48805ca10 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb473e00640 1 -- 172.21.15.5:0/471141882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb478048440 con 0x55d281bcd7a0 2024-04-25T10:07:28.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb49883d740 10 client.7032 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb49883d740 10 client.7032 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb49883d740 20 client.7032 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb49883d740 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281be8410 0x55d281bec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb49883d740 4 client.7032 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d281af5b60 con 0x55d281be8410 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.339+0000 7fb48ea00640 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281be8410 0x55d281bec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.340+0000 7fb48ea00640 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281be8410 0x55d281bec630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbe3b8a27767b44 server_cookie=622ab30839f68540 in_seq=0 out_seq=0 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.340+0000 7fb473e00640 10 client.7032 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb473e00640 1 -- 172.21.15.5:0/471141882 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d281af5b60 con 0x55d281be8410 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb473e00640 10 client.7032 handle_command_reply: tid=0 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 client.7032 shutdown 2024-04-25T10:07:28.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 msgr2=0x55d281bdeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281bdeae0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb478035040 tx=0x7fb478004730 comp rx=0 tx=0).stop 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 shutdown_connections 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281be8410 0x55d281bec630 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281ba7d10 0x55d281ba8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bcd7a0 0x55d281bdeae0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 --2- 172.21.15.5:0/471141882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281baceb0 0x55d281bb35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 >> 172.21.15.5:0/471141882 conn(0x55d281b84b30 msgr2=0x55d281bca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb473e00640 0 client.7032 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 shutdown_connections 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 1 -- 172.21.15.5:0/471141882 wait complete. 2024-04-25T10:07:28.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.343+0000 7fb49883d740 20 client.7032 trim_cache size 0 max 16384 2024-04-25T10:07:28.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.361+0000 7fb49883d740 1 Processor -- start 2024-04-25T10:07:28.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.361+0000 7fb49883d740 1 -- start start 2024-04-25T10:07:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.361+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281bccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.361+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281baceb0 0x55d281bc8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.361+0000 7fb49883d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 0x55d281bb3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb49883d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900b3db0 con 0x55d281bdc100 2024-04-25T10:07:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900c7de0 con 0x55d281baceb0 2024-04-25T10:07:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900b37b0 con 0x55d281ba7d10 2024-04-25T10:07:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 0x55d281bb3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 0x55d281bb3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47598/0 (socket says 172.21.15.5:47598) 2024-04-25T10:07:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 -- 172.21.15.5:0/1605751748 learned_addr learned my addr 172.21.15.5:0/1605751748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 -- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 msgr2=0x55d281bccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb486000640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281baceb0 0x55d281bc8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48ea00640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281bccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281bccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 -- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281baceb0 msgr2=0x55d281bc8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281baceb0 0x55d281bc8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48f400640 1 -- 172.21.15.5:0/1605751748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2818c76b0 con 0x55d281bdc100 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb48ea00640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281bccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.362+0000 7fb486000640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281baceb0 0x55d281bc8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.363+0000 7fb48f400640 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 0x55d281bb3570 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb4880585c0 tx=0x7fb48801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38a87b929819b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.363+0000 7fb47ea00640 1 -- 172.21.15.5:0/1605751748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488060fd0 con 0x55d281bdc100 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.363+0000 7fb47ea00640 1 -- 172.21.15.5:0/1605751748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4880588c0 con 0x55d281bdc100 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.363+0000 7fb47ea00640 1 -- 172.21.15.5:0/1605751748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488056bc0 con 0x55d281bdc100 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 -- 172.21.15.5:0/1605751748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 msgr2=0x55d281bb3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 0x55d281bb3570 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb4880585c0 tx=0x7fb48801d040 comp rx=0 tx=0).stop 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 -- 172.21.15.5:0/1605751748 shutdown_connections 2024-04-25T10:07:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281bdc100 0x55d281bb3570 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281baceb0 0x55d281bc8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 --2- 172.21.15.5:0/1605751748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281bccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 -- 172.21.15.5:0/1605751748 >> 172.21.15.5:0/1605751748 conn(0x55d281b84b30 msgr2=0x55d281b85bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 -- 172.21.15.5:0/1605751748 shutdown_connections 2024-04-25T10:07:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 -- 172.21.15.5:0/1605751748 wait complete. 2024-04-25T10:07:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.364+0000 7fb49883d740 1 -- start start 2024-04-25T10:07:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.365+0000 7fb49883d740 1 Processor -- start 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281c22280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 0x55d281bcb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281bdc100 0x55d281bf0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d281be8290 con 0x55d281baceb0 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d281bc3630 con 0x55d281bdc100 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d281be8890 con 0x55d281ba7d10 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb49883d740 10 client.0 _ll_register_callbacks cb 0x7fb478035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 0x55d281bcb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 0x55d281bcb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47604/0 (socket says 172.21.15.5:47604) 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 -- 172.21.15.5:0/2485008978 learned_addr learned my addr 172.21.15.5:0/2485008978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 -- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 msgr2=0x55d281c22280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281c22280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 -- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281bdc100 msgr2=0x55d281bf0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281bdc100 0x55d281bf0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.366+0000 7fb486000640 1 -- 172.21.15.5:0/2485008978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d281bcbb00 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb486000640 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 0x55d281bcb5c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb478035040 tx=0x7fb478009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b83cf1f0753f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb473e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb473e00640 1 -- 172.21.15.5:0/2485008978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4780327e0 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb473e00640 1 -- 172.21.15.5:0/2485008978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb478005990 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4880588c0 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb473e00640 1 -- 172.21.15.5:0/2485008978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb478059050 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb49883d740 10 client.7038 fetch_fsmap learned FSMap version 51 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.367+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55d281baa930 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb473e00640 1 -- 172.21.15.5:0/2485008978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4880588c0 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb473e00640 1 -- 172.21.15.5:0/2485008978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb47805c020 con 0x55d281baceb0 2024-04-25T10:07:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb49883d740 10 client.7038 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:07:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb49883d740 10 client.7038 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb49883d740 20 client.7038 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb49883d740 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281bc9a80 0x55d281b832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb49883d740 4 client.7038 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d281aefeb0 con 0x55d281bc9a80 2024-04-25T10:07:28.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.368+0000 7fb48ea00640 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281bc9a80 0x55d281b832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.369+0000 7fb48ea00640 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281bc9a80 0x55d281b832f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c621f724a7efd4 server_cookie=44b27ad7b07515db in_seq=0 out_seq=0 2024-04-25T10:07:28.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.369+0000 7fb473e00640 10 client.7038 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:28.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb473e00640 1 -- 172.21.15.5:0/2485008978 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d281aefeb0 con 0x55d281bc9a80 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb473e00640 10 client.7038 handle_command_reply: tid=0 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 client.7038 shutdown 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 msgr2=0x55d281bcb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 0x55d281bcb5c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb478035040 tx=0x7fb478009190 comp rx=0 tx=0).stop 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 shutdown_connections 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55d281bc9a80 0x55d281b832f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d281bdc100 0x55d281bf0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281baceb0 0x55d281bcb5c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 --2- 172.21.15.5:0/2485008978 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d281ba7d10 0x55d281c22280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb473e00640 0 client.7038 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 >> 172.21.15.5:0/2485008978 conn(0x55d281b84b30 msgr2=0x55d281b85bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 shutdown_connections 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.370+0000 7fb49883d740 1 -- 172.21.15.5:0/2485008978 wait complete. 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb49883d740 20 client.7038 trim_cache size 0 max 16384 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6025, 2024-04-25T10:07:28.379 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.9881522299999999, 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 51, 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 78, 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 78, 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 5.7158670239999996 2024-04-25T10:07:28.380 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb46c06b4d0 msgr2=0x7fb46c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb46c06b4d0 0x7fb46c06d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb480007fd0 tx=0x7fb480009040 comp rx=0 tx=0).stop 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 msgr2=0x7fb49015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015a2d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c7a0 tx=0x7fb48800cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 shutdown_connections 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015ce00 0x7fb49015a2d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb490141da0 0x7fb490159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb46c06b4d0 0x7fb46c06d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 --2- 172.21.15.5:0/1448666191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4900cd300 0x7fb49013e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 >> 172.21.15.5:0/1448666191 conn(0x7fb490066f20 msgr2=0x7fb49006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 shutdown_connections 2024-04-25T10:07:28.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.371+0000 7fb48fe00640 1 -- 172.21.15.5:0/1448666191 wait complete. 2024-04-25T10:07:28.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:28.396 INFO:tasks.ceph.mds.d.smithi080.stderr:2024-04-25T10:07:28.391+0000 7f392ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:28.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.591+0000 7f2e4f200640 1 Processor -- start 2024-04-25T10:07:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.591+0000 7f2e4f200640 1 -- start start 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.592+0000 7f2e4f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e5015b780 0x7f2e500b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.592+0000 7f2e4f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e500cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.592+0000 7f2e4f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5015bb60 con 0x7f2e500cfe00 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.592+0000 7f2e4f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e500d0250 con 0x7f2e5015b780 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.592+0000 7f2e4f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500cc230 con 0x7f2e500b29d0 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e5015b780 0x7f2e500b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e5015b780 0x7f2e500b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47620/0 (socket says 172.21.15.5:47620) 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 -- 172.21.15.5:0/448969222 learned_addr learned my addr 172.21.15.5:0/448969222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4d400640 1 --2- 172.21.15.5:0/448969222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e500cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 -- 172.21.15.5:0/448969222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 msgr2=0x7f2e500cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 --2- 172.21.15.5:0/448969222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e500cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 -- 172.21.15.5:0/448969222 >> v1:172.21.15.80:6789/0 conn(0x7f2e500cfe00 legacy=0x7f2e500999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.593+0000 7f2e4de00640 1 -- 172.21.15.5:0/448969222 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500cc510 con 0x7f2e5015b780 2024-04-25T10:07:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.594+0000 7f2e4de00640 1 --2- 172.21.15.5:0/448969222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e5015b780 0x7f2e500b2490 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009870 tx=0x7f2e4002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa7265dbcf38e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.595+0000 7f2e4ca00640 1 -- 172.21.15.5:0/448969222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40037070 con 0x7f2e5015b780 2024-04-25T10:07:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.595+0000 7f2e4ca00640 1 -- 172.21.15.5:0/448969222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e40032440 con 0x7f2e5015b780 2024-04-25T10:07:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.595+0000 7f2e4ca00640 1 -- 172.21.15.5:0/448969222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40032810 con 0x7f2e5015b780 2024-04-25T10:07:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.595+0000 7f2e4f200640 1 -- 172.21.15.5:0/448969222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e5015b780 msgr2=0x7f2e500b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.595+0000 7f2e4f200640 1 --2- 172.21.15.5:0/448969222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e5015b780 0x7f2e500b2490 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009870 tx=0x7f2e4002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.596+0000 7f2e4f200640 1 -- 172.21.15.5:0/448969222 shutdown_connections 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.596+0000 7f2e4f200640 1 --2- 172.21.15.5:0/448969222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e500cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.596+0000 7f2e4f200640 1 --2- 172.21.15.5:0/448969222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e5015b780 0x7f2e500b2490 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.596+0000 7f2e4f200640 1 -- 172.21.15.5:0/448969222 >> 172.21.15.5:0/448969222 conn(0x7f2e500d1470 msgr2=0x7f2e500d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.596+0000 7f2e4f200640 1 -- 172.21.15.5:0/448969222 shutdown_connections 2024-04-25T10:07:28.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.596+0000 7f2e4f200640 1 -- 172.21.15.5:0/448969222 wait complete. 2024-04-25T10:07:28.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.598+0000 7f2e4f200640 1 Processor -- start 2024-04-25T10:07:28.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.598+0000 7f2e4f200640 1 -- start start 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.599+0000 7f2e4f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e501f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.599+0000 7f2e4f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 0x7f2e500c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.599+0000 7f2e4f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e5015b780 0x7f2e500ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.599+0000 7f2e4f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500ccdb0 con 0x7f2e500cfe00 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.599+0000 7f2e4f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500ccad0 con 0x7f2e5015b780 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.599+0000 7f2e4f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e500cbe50 con 0x7f2e500b29d0 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 0x7f2e500c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 0x7f2e500c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47622/0 (socket says 172.21.15.5:47622) 2024-04-25T10:07:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 -- 172.21.15.5:0/2338416808 learned_addr learned my addr 172.21.15.5:0/2338416808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4de00640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e501f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4e800640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e5015b780 0x7f2e500ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 -- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 msgr2=0x7f2e501f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e501f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 -- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e5015b780 msgr2=0x7f2e500ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e5015b780 0x7f2e500ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4d400640 1 -- 172.21.15.5:0/2338416808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500c5380 con 0x7f2e500cfe00 2024-04-25T10:07:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.600+0000 7f2e4e800640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e5015b780 0x7f2e500ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.601+0000 7f2e4d400640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 0x7f2e500c8540 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800cd00 tx=0x7f2e3800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.601+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38003990 con 0x7f2e500cfe00 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.601+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e38003b30 con 0x7f2e500cfe00 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.601+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004020 con 0x7f2e500cfe00 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.601+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e500c5660 con 0x7f2e500cfe00 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.601+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e500c8a80 con 0x7f2e500cfe00 2024-04-25T10:07:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.602+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e50068f60 con 0x7f2e500cfe00 2024-04-25T10:07:28.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.604+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e380041c0 con 0x7f2e500cfe00 2024-04-25T10:07:28.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.604+0000 7f2e46000640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e2c06b520 0x7f2e2c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.605+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2e3808a930 con 0x7f2e500cfe00 2024-04-25T10:07:28.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.605+0000 7f2e4de00640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e2c06b520 0x7f2e2c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.606+0000 7f2e4de00640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e2c06b520 0x7f2e2c06d9e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2e40034000 tx=0x7f2e4002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.610+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e38014030 con 0x7f2e500cfe00 2024-04-25T10:07:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.850+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e500701b0 con 0x7f2e500cfe00 2024-04-25T10:07:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.851+0000 7f2e46000640 1 -- 172.21.15.5:0/2338416808 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e380595b0 con 0x7f2e500cfe00 2024-04-25T10:07:28.856 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e2c06b520 msgr2=0x7f2e2c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e2c06b520 0x7f2e2c06d9e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2e40034000 tx=0x7f2e4002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 msgr2=0x7f2e500c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 0x7f2e500c8540 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800cd00 tx=0x7f2e3800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 shutdown_connections 2024-04-25T10:07:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e2c06b520 0x7f2e2c06d9e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e5015b780 0x7f2e500ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e500cfe00 0x7f2e500c8540 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 --2- 172.21.15.5:0/2338416808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e500b29d0 0x7f2e501f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.853+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 >> 172.21.15.5:0/2338416808 conn(0x7f2e500d1470 msgr2=0x7f2e50070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.854+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 shutdown_connections 2024-04-25T10:07:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:28.854+0000 7f2e4f200640 1 -- 172.21.15.5:0/2338416808 wait complete. 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:28.879 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:28.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.880 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:07:28.935 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:28.935 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:28.936 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:07:28.937 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.938 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:28.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.939 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:29.018 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:07:29.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.022 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:07:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:07:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:07:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:07:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:29.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.080 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:07:29.080 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:07:29.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:07:29.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:07:29.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:07:29.080 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:07:29.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.241 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:07:29.294 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:29.294 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:29.295 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:29.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:07:29.386 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:29.386 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:29.445 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:29.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:29.446 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:29.495 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:29.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:29.619 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:29.619 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:29.619 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:07:29.641 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:29.641 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:29.641 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:07:29.641 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:07:29.641 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:29.668 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:29.668 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:29.668 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:07:29.668 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:07:29.669 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:29.669 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:07:29.669 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:07:29.669 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:07:29.671 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:07:29.671 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:29.671 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:29.671 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:29.671 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:29.676 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:29.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:29.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.751 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:29.808 INFO:teuthology.orchestra.run.smithi005.stdout:client6049 2024-04-25T10:07:29.808 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:29.808 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:29.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:29.859 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T10:07:29.919 INFO:teuthology.orchestra.run.smithi005.stdout:file_undamaged 2024-04-25T10:07:29.921 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:07:29.921 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:07:29.921 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> import json 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> import sys 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> try: 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> except OSError as e: 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> sys.exit(e.errno) 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> print(json.dumps( 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> indent=2)) 2024-04-25T10:07:29.922 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:07:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:07:30.090 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:07:30.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:30.090 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:07:30.186 INFO:teuthology.orchestra.run.smithi005.stdout:2 2024-04-25T10:07:30.186 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:30.187 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:30.237 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:07:30.237 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:30.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:30.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.539+0000 7fa648a00640 1 Processor -- start 2024-04-25T10:07:30.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.539+0000 7fa648a00640 1 -- start start 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa648a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa6440b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa648a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6440af3d0 0x7fa6440b0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa648a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6440b50c0 con 0x7fa6440b0e70 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa648a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6440b1ed0 con 0x7fa6440af3d0 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa648a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440afee0 con 0x7fa64415d8e0 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa641e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6440af3d0 0x7fa6440b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa643200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa6440b0e70 0x7fa6440ad2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59972/0 (socket says 172.21.15.5:59972) 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa643200640 1 -- 172.21.15.5:0/2794787431 learned_addr learned my addr 172.21.15.5:0/2794787431 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:30.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.541+0000 7fa642800640 1 --2- 172.21.15.5:0/2794787431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa6440b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.542+0000 7fa641e00640 1 -- 172.21.15.5:0/2794787431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 msgr2=0x7fa6440b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.542+0000 7fa641e00640 1 --2- 172.21.15.5:0/2794787431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa6440b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.542+0000 7fa641e00640 1 -- 172.21.15.5:0/2794787431 >> v1:172.21.15.5:6789/0 conn(0x7fa6440b0e70 legacy=0x7fa6440ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.542+0000 7fa641e00640 1 -- 172.21.15.5:0/2794787431 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64415dcc0 con 0x7fa6440af3d0 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.542+0000 7fa641400640 1 -- 172.21.15.5:0/2794787431 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153181666 0 0) 0x7fa6440b50c0 con 0x7fa6440b0e70 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.543+0000 7fa641e00640 1 --2- 172.21.15.5:0/2794787431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440af3d0 0x7fa6440b0930 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa630009d90 tx=0x7fa63002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fd4b6c90d10970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.543+0000 7fa641400640 1 -- 172.21.15.5:0/2794787431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630031600 con 0x7fa6440af3d0 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.543+0000 7fa641400640 1 -- 172.21.15.5:0/2794787431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6300317a0 con 0x7fa6440af3d0 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.543+0000 7fa641400640 1 -- 172.21.15.5:0/2794787431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630031b70 con 0x7fa6440af3d0 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 -- 172.21.15.5:0/2794787431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440af3d0 msgr2=0x7fa6440b0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 --2- 172.21.15.5:0/2794787431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440af3d0 0x7fa6440b0930 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa630009d90 tx=0x7fa63002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 -- 172.21.15.5:0/2794787431 shutdown_connections 2024-04-25T10:07:30.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 --2- 172.21.15.5:0/2794787431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440af3d0 0x7fa6440b0930 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 --2- 172.21.15.5:0/2794787431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa6440b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 -- 172.21.15.5:0/2794787431 >> 172.21.15.5:0/2794787431 conn(0x7fa6440b5ab0 msgr2=0x7fa6440999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 -- 172.21.15.5:0/2794787431 shutdown_connections 2024-04-25T10:07:30.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.544+0000 7fa648a00640 1 -- 172.21.15.5:0/2794787431 wait complete. 2024-04-25T10:07:30.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.546+0000 7fa648a00640 1 Processor -- start 2024-04-25T10:07:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.546+0000 7fa648a00640 1 -- start start 2024-04-25T10:07:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa648a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6440af3d0 0x7fa644158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa648a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 0x7fa644141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa648a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa644144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa648a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b0130 con 0x7fa6440b0e70 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa648a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b5640 con 0x7fa6440af3d0 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa648a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b0410 con 0x7fa64415d8e0 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 0x7fa644141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa641e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 0x7fa644141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47634/0 (socket says 172.21.15.5:47634) 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.547+0000 7fa641e00640 1 -- 172.21.15.5:0/1220264 learned_addr learned my addr 172.21.15.5:0/1220264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa642800640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6440af3d0 0x7fa644158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa643200640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa644144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa641e00640 1 -- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 msgr2=0x7fa644144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa641e00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa644144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa641e00640 1 -- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6440af3d0 msgr2=0x7fa644158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa641e00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6440af3d0 0x7fa644158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa641e00640 1 -- 172.21.15.5:0/1220264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6441424f0 con 0x7fa6440b0e70 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.548+0000 7fa641e00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 0x7fa644141a00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa630009eb0 tx=0x7fa630004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.549+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63003f070 con 0x7fa6440b0e70 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.549+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630030be0 con 0x7fa6440b0e70 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.549+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6441427d0 con 0x7fa6440b0e70 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.549+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630009450 con 0x7fa6440b0e70 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.549+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa644142d90 con 0x7fa6440b0e70 2024-04-25T10:07:30.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.550+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440b43c0 con 0x7fa6440b0e70 2024-04-25T10:07:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.551+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa630043410 con 0x7fa6440b0e70 2024-04-25T10:07:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.552+0000 7fa63aa00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa62406b4d0 0x7fa62406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.552+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa6300ab610 con 0x7fa6440b0e70 2024-04-25T10:07:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.553+0000 7fa642800640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa62406b4d0 0x7fa62406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.554+0000 7fa642800640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa62406b4d0 0x7fa62406d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa634009ad0 tx=0x7fa634008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.556+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6440b43c0 con 0x7fa6440b0e70 2024-04-25T10:07:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.758+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa644068f60 con 0x7fa6440b0e70 2024-04-25T10:07:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.761+0000 7fa63aa00640 1 -- 172.21.15.5:0/1220264 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16906 (secure 0 0 0) 0x7fa63007a2a0 con 0x7fa6440b0e70 2024-04-25T10:07:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:07:30.768 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:22.531478+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1098619647":"2024-04-26T10:07:22.531455+0000","172.21.15.80:6834/1098619647":"2024-04-26T10:07:22.531455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa62406b4d0 msgr2=0x7fa62406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa62406b4d0 0x7fa62406d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa634009ad0 tx=0x7fa634008040 comp rx=0 tx=0).stop 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 msgr2=0x7fa644141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 0x7fa644141a00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa630009eb0 tx=0x7fa630004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 shutdown_connections 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa64415d8e0 0x7fa644144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6440b0e70 0x7fa644141a00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa62406b4d0 0x7fa62406d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 --2- 172.21.15.5:0/1220264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6440af3d0 0x7fa644158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 >> 172.21.15.5:0/1220264 conn(0x7fa6440b5ab0 msgr2=0x7fa6440ad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 shutdown_connections 2024-04-25T10:07:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:30.764+0000 7fa648a00640 1 -- 172.21.15.5:0/1220264 wait complete. 2024-04-25T10:07:30.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:07.362188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:08.532434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:30.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:31.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.005+0000 7fc136200640 1 Processor -- start 2024-04-25T10:07:31.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.005+0000 7fc136200640 1 -- start start 2024-04-25T10:07:31.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.007+0000 7fc136200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc13006ec70 0x7fc130071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.007+0000 7fc136200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1300ade10 con 0x7fc130143f30 2024-04-25T10:07:31.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.007+0000 7fc136200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1300ae0f0 con 0x7fc1300aebd0 2024-04-25T10:07:31.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.007+0000 7fc136200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1300ae3d0 con 0x7fc13006ec70 2024-04-25T10:07:31.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.007+0000 7fc134e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc130143f30 0x7fc13006da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59974/0 (socket says 172.21.15.5:59974) 2024-04-25T10:07:31.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.007+0000 7fc134e00640 1 -- 172.21.15.5:0/2986424253 learned_addr learned my addr 172.21.15.5:0/2986424253 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:31.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485249705 0 0) 0x7fc1300ade10 con 0x7fc130143f30 2024-04-25T10:07:31.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12fe00640 1 --2- 172.21.15.5:0/2986424253 >> v2:172.21.15.80:3300/0 conn(0x7fc13006ec70 0x7fc130071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc118003520 con 0x7fc130143f30 2024-04-25T10:07:31.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1135559214 0 0) 0x7fc118003520 con 0x7fc130143f30 2024-04-25T10:07:31.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 >> v2:172.21.15.80:3300/0 conn(0x7fc13006ec70 msgr2=0x7fc130071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 --2- 172.21.15.5:0/2986424253 >> v2:172.21.15.80:3300/0 conn(0x7fc13006ec70 0x7fc130071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 >> v1:172.21.15.80:6789/0 conn(0x7fc1300aebd0 legacy=0x7fc1300b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1300ae6b0 con 0x7fc130143f30 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fc124001670 con 0x7fc130143f30 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 >> v1:172.21.15.5:6789/0 conn(0x7fc130143f30 legacy=0x7fc13006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1180039b0 0x7fc118003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc118004800 0x7fc118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1180052e0 0x7fc11800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300ae3d0 con 0x7fc1180039b0 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11800c1b0 con 0x7fc118004800 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.008+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11800c3b0 con 0x7fc1180052e0 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc134e00640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1180039b0 0x7fc118003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc12fe00640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc118004800 0x7fc118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc135800640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1180052e0 0x7fc11800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc134e00640 1 -- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1180052e0 msgr2=0x7fc11800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc134e00640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1180052e0 0x7fc11800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc134e00640 1 -- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc118004800 msgr2=0x7fc118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc134e00640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc118004800 0x7fc118004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc134e00640 1 -- 172.21.15.5:0/2986424253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118003520 con 0x7fc1180039b0 2024-04-25T10:07:31.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.009+0000 7fc135800640 1 -- 172.21.15.5:0/2986424253 reap_dead start 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.010+0000 7fc134e00640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1180039b0 0x7fc118003dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c210 tx=0x7fc118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ceb53a827e9b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124007cf0 con 0x7fc1180039b0 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc124005af0 con 0x7fc1180039b0 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc12f400640 1 -- 172.21.15.5:0/2986424253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc124007070 con 0x7fc1180039b0 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 -- 172.21.15.5:0/2986424253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1180039b0 msgr2=0x7fc118003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1180039b0 0x7fc118003dd0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc12400c210 tx=0x7fc118005830 comp rx=0 tx=0).stop 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 -- 172.21.15.5:0/2986424253 shutdown_connections 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 --2- 172.21.15.5:0/2986424253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1180039b0 0x7fc118003dd0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 -- 172.21.15.5:0/2986424253 >> 172.21.15.5:0/2986424253 conn(0x7fc130066f20 msgr2=0x7fc13006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 -- 172.21.15.5:0/2986424253 shutdown_connections 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.011+0000 7fc136200640 1 -- 172.21.15.5:0/2986424253 wait complete. 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.013+0000 7fc136200640 1 Processor -- start 2024-04-25T10:07:31.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.013+0000 7fc136200640 1 -- start start 2024-04-25T10:07:31.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.014+0000 7fc136200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1180039b0 0x7fc130140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.014+0000 7fc136200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1301f3620 0x7fc130160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.014+0000 7fc136200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 0x7fc13015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.014+0000 7fc136200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13006dbb0 con 0x7fc1300b3900 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.014+0000 7fc136200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13006da30 con 0x7fc1180039b0 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc136200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13006dd30 con 0x7fc1301f3620 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 0x7fc13015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 0x7fc13015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47646/0 (socket says 172.21.15.5:47646) 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 -- 172.21.15.5:0/4030193595 learned_addr learned my addr 172.21.15.5:0/4030193595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc134e00640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1180039b0 0x7fc130140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc12fe00640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1301f3620 0x7fc130160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 -- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1301f3620 msgr2=0x7fc130160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1301f3620 0x7fc130160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 -- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1180039b0 msgr2=0x7fc130140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1180039b0 0x7fc130140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.015+0000 7fc135800640 1 -- 172.21.15.5:0/4030193595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130161280 con 0x7fc1300b3900 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.016+0000 7fc135800640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 0x7fc13015cb10 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc1300723d0 tx=0x7fc12800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.016+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128018070 con 0x7fc1300b3900 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.016+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc128003b70 con 0x7fc1300b3900 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.016+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128005070 con 0x7fc1300b3900 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.016+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc130161560 con 0x7fc1300b3900 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.016+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc130161b20 con 0x7fc1300b3900 2024-04-25T10:07:31.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.017+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1301418c0 con 0x7fc1300b3900 2024-04-25T10:07:31.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.018+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc12801b070 con 0x7fc1300b3900 2024-04-25T10:07:31.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.019+0000 7fc12cc00640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc11006b500 0x7fc11006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.019+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc128089ac0 con 0x7fc1300b3900 2024-04-25T10:07:31.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.020+0000 7fc134e00640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc11006b500 0x7fc11006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.021+0000 7fc134e00640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc11006b500 0x7fc11006d9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc118004780 tx=0x7fc124009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.024+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1301418c0 con 0x7fc1300b3900 2024-04-25T10:07:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.220+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc130068f60 con 0x7fc1300b3900 2024-04-25T10:07:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.221+0000 7fc12cc00640 1 -- 172.21.15.5:0/4030193595 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc128058070 con 0x7fc1300b3900 2024-04-25T10:07:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc11006b500 msgr2=0x7fc11006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc11006b500 0x7fc11006d9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc118004780 tx=0x7fc124009030 comp rx=0 tx=0).stop 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 msgr2=0x7fc13015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 0x7fc13015cb10 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc1300723d0 tx=0x7fc12800dc00 comp rx=0 tx=0).stop 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 shutdown_connections 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1300b3900 0x7fc13015cb10 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1301f3620 0x7fc130160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc11006b500 0x7fc11006d9c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 --2- 172.21.15.5:0/4030193595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1180039b0 0x7fc130140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.224+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 >> 172.21.15.5:0/4030193595 conn(0x7fc130066f20 msgr2=0x7fc13006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.225+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 shutdown_connections 2024-04-25T10:07:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.225+0000 7fc136200640 1 -- 172.21.15.5:0/4030193595 wait complete. 2024-04-25T10:07:31.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.452+0000 7fddbd800640 1 Processor -- start 2024-04-25T10:07:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.452+0000 7fddbd800640 1 -- start start 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddbd800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb815b7f0 0x7fddb80cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddbd800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fddb80b27c0 0x7fddb80cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddbd800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb80cc510 con 0x7fddb80cfd70 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddbd800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb815bbd0 con 0x7fddb80b27c0 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddbd800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80cc7f0 con 0x7fddb815b7f0 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddb7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fddb80b27c0 0x7fddb80cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddb7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fddb80b27c0 0x7fddb80cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47652/0 (socket says 172.21.15.5:47652) 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddb7400640 1 -- 172.21.15.5:0/1609325112 learned_addr learned my addr 172.21.15.5:0/1609325112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.454+0000 7fddb7e00640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb815b7f0 0x7fddb80cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.455+0000 7fddb7400640 1 -- 172.21.15.5:0/1609325112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb815b7f0 msgr2=0x7fddb80cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.455+0000 7fddb7400640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb815b7f0 0x7fddb80cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.455+0000 7fddb7400640 1 -- 172.21.15.5:0/1609325112 >> v1:172.21.15.80:6789/0 conn(0x7fddb80cfd70 legacy=0x7fddb80999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.455+0000 7fddb7400640 1 -- 172.21.15.5:0/1609325112 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80ccad0 con 0x7fddb80b27c0 2024-04-25T10:07:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.455+0000 7fddb7e00640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb815b7f0 0x7fddb80cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:31.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.456+0000 7fddb7400640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb80b27c0 0x7fddb80cf830 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009870 tx=0x7fdda002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd67d6db6325dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.456+0000 7fddb6a00640 1 -- 172.21.15.5:0/1609325112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0037070 con 0x7fddb80b27c0 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.456+0000 7fddb6a00640 1 -- 172.21.15.5:0/1609325112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda0032440 con 0x7fddb80b27c0 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.456+0000 7fddb6a00640 1 -- 172.21.15.5:0/1609325112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda0032810 con 0x7fddb80b27c0 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 -- 172.21.15.5:0/1609325112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb80b27c0 msgr2=0x7fddb80cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb80b27c0 0x7fddb80cf830 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009870 tx=0x7fdda002a210 comp rx=0 tx=0).stop 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 -- 172.21.15.5:0/1609325112 shutdown_connections 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb80b27c0 0x7fddb80cf830 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 --2- 172.21.15.5:0/1609325112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb815b7f0 0x7fddb80cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 -- 172.21.15.5:0/1609325112 >> 172.21.15.5:0/1609325112 conn(0x7fddb80d1470 msgr2=0x7fddb80d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 -- 172.21.15.5:0/1609325112 shutdown_connections 2024-04-25T10:07:31.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.457+0000 7fddbd800640 1 -- 172.21.15.5:0/1609325112 wait complete. 2024-04-25T10:07:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.459+0000 7fddbd800640 1 Processor -- start 2024-04-25T10:07:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.459+0000 7fddbd800640 1 -- start start 2024-04-25T10:07:31.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbd800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb80b27c0 0x7fddb814d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbd800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb80cfd70 0x7fddb814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 0x7fddb80c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbd800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80b2c20 con 0x7fddb815b7f0 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbd800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80cd090 con 0x7fddb80b27c0 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbd800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb80c7e90 con 0x7fddb80cfd70 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.460+0000 7fddbce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 0x7fddb80c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 0x7fddb80c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47664/0 (socket says 172.21.15.5:47664) 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 -- 172.21.15.5:0/2557316780 learned_addr learned my addr 172.21.15.5:0/2557316780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddb7400640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb80cfd70 0x7fddb814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 -- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb80cfd70 msgr2=0x7fddb814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb80cfd70 0x7fddb814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddb7e00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb80b27c0 0x7fddb814d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 -- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb80b27c0 msgr2=0x7fddb814d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb80b27c0 0x7fddb814d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 -- 172.21.15.5:0/2557316780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80b5260 con 0x7fddb815b7f0 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddb7400640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb80cfd70 0x7fddb814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:31.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.461+0000 7fddbce00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 0x7fddb80c4fa0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c780 tx=0x7fddb000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.462+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb0004b60 con 0x7fddb815b7f0 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.462+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb0004d00 con 0x7fddb815b7f0 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.462+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb000aa90 con 0x7fddb815b7f0 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.462+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb80b5540 con 0x7fddb815b7f0 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.462+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb80c54e0 con 0x7fddb815b7f0 2024-04-25T10:07:31.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.463+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb8068f60 con 0x7fddb815b7f0 2024-04-25T10:07:31.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.464+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddb000ac30 con 0x7fddb815b7f0 2024-04-25T10:07:31.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.465+0000 7fddafe00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd9406b4d0 0x7fdd9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.465+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fddb008a760 con 0x7fddb815b7f0 2024-04-25T10:07:31.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.466+0000 7fddb7e00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd9406b4d0 0x7fdd9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.467+0000 7fddb7e00640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd9406b4d0 0x7fdd9406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b840 tx=0x7fdda8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.469+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddb0020070 con 0x7fddb815b7f0 2024-04-25T10:07:31.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.671+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fddb8073b10 con 0x7fddb815b7f0 2024-04-25T10:07:31.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.672+0000 7fddafe00640 1 -- 172.21.15.5:0/2557316780 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddb0059360 con 0x7fddb815b7f0 2024-04-25T10:07:31.679 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:07:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd9406b4d0 msgr2=0x7fdd9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd9406b4d0 0x7fdd9406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdda800b840 tx=0x7fdda8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 msgr2=0x7fddb80c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 0x7fddb80c4fa0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fddb000c780 tx=0x7fddb000cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 shutdown_connections 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb815b7f0 0x7fddb80c4fa0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fddb80cfd70 0x7fddb814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd9406b4d0 0x7fdd9406d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 --2- 172.21.15.5:0/2557316780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fddb80b27c0 0x7fddb814d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 >> 172.21.15.5:0/2557316780 conn(0x7fddb80d1470 msgr2=0x7fddb80702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 shutdown_connections 2024-04-25T10:07:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.675+0000 7fddbd800640 1 -- 172.21.15.5:0/2557316780 wait complete. 2024-04-25T10:07:31.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:31.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.910+0000 7f80a5200640 1 Processor -- start 2024-04-25T10:07:31.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.911+0000 7f80a5200640 1 -- start start 2024-04-25T10:07:31.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.912+0000 7f80a5200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00b48c0 0x7f80a015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.912+0000 7f80a5200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f80a0161d90 0x7f80a00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.912+0000 7f80a5200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a01606e0 con 0x7f80a00d3f90 2024-04-25T10:07:31.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.912+0000 7f80a5200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a01609c0 con 0x7f80a0161d90 2024-04-25T10:07:31.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.912+0000 7f80a5200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a0160ca0 con 0x7f80a00b48c0 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809ea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f80a00d3f90 0x7f80a0161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59982/0 (socket says 172.21.15.5:59982) 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809ea00640 1 -- 172.21.15.5:0/3175974539 learned_addr learned my addr 172.21.15.5:0/3175974539 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809fe00640 1 --2- 172.21.15.5:0/3175974539 >> v2:172.21.15.5:3300/0 conn(0x7f80a0161d90 0x7f80a00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809f400640 1 --2- 172.21.15.5:0/3175974539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00b48c0 0x7f80a015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809e000640 1 -- 172.21.15.5:0/3175974539 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882947182 0 0) 0x7f80a01606e0 con 0x7f80a00d3f90 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809e000640 1 -- 172.21.15.5:0/3175974539 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8084003520 con 0x7f80a00d3f90 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809fe00640 1 -- 172.21.15.5:0/3175974539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00b48c0 msgr2=0x7f80a015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809fe00640 1 --2- 172.21.15.5:0/3175974539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00b48c0 0x7f80a015fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809fe00640 1 -- 172.21.15.5:0/3175974539 >> v1:172.21.15.5:6789/0 conn(0x7f80a00d3f90 legacy=0x7f80a0161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.913+0000 7f809fe00640 1 -- 172.21.15.5:0/3175974539 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0160f80 con 0x7f80a0161d90 2024-04-25T10:07:31.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.915+0000 7f809fe00640 1 --2- 172.21.15.5:0/3175974539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00b2b70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f80980097a0 tx=0x7f809802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5304dfab817b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.915+0000 7f809e000640 1 -- 172.21.15.5:0/3175974539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098037070 con 0x7f80a0161d90 2024-04-25T10:07:31.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.915+0000 7f809e000640 1 -- 172.21.15.5:0/3175974539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809802ae50 con 0x7f80a0161d90 2024-04-25T10:07:31.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.915+0000 7f809e000640 1 -- 172.21.15.5:0/3175974539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098032660 con 0x7f80a0161d90 2024-04-25T10:07:31.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.915+0000 7f80a5200640 1 -- 172.21.15.5:0/3175974539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 msgr2=0x7f80a00b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.915+0000 7f80a5200640 1 --2- 172.21.15.5:0/3175974539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00b2b70 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f80980097a0 tx=0x7f809802a100 comp rx=0 tx=0).stop 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.916+0000 7f80a5200640 1 -- 172.21.15.5:0/3175974539 shutdown_connections 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.916+0000 7f80a5200640 1 --2- 172.21.15.5:0/3175974539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00b2b70 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.916+0000 7f80a5200640 1 --2- 172.21.15.5:0/3175974539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00b48c0 0x7f80a015fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.916+0000 7f80a5200640 1 -- 172.21.15.5:0/3175974539 >> 172.21.15.5:0/3175974539 conn(0x7f80a008af80 msgr2=0x7f80a00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.916+0000 7f80a5200640 1 -- 172.21.15.5:0/3175974539 shutdown_connections 2024-04-25T10:07:31.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.916+0000 7f80a5200640 1 -- 172.21.15.5:0/3175974539 wait complete. 2024-04-25T10:07:31.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.918+0000 7f80a5200640 1 Processor -- start 2024-04-25T10:07:31.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.918+0000 7f80a5200640 1 -- start start 2024-04-25T10:07:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f80a5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80a00b48c0 0x7f80a00c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f80a5200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00d3f90 0x7f80a00c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f80a5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f80a5200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a01603a0 con 0x7f80a0161d90 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f80a5200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a00b4ca0 con 0x7f80a00b48c0 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f80a5200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a00b30b0 con 0x7f80a00d3f90 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47680/0 (socket says 172.21.15.5:47680) 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 -- 172.21.15.5:0/1149075277 learned_addr learned my addr 172.21.15.5:0/1149075277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 -- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00d3f90 msgr2=0x7f80a00c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00d3f90 0x7f80a00c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 -- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80a00b48c0 msgr2=0x7f80a00c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80a00b48c0 0x7f80a00c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.919+0000 7f809fe00640 1 -- 172.21.15.5:0/1149075277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00c7dd0 con 0x7f80a0161d90 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.920+0000 7f809fe00640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00cc450 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8098035040 tx=0x7f80980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.920+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098041070 con 0x7f80a0161d90 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.920+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809803c460 con 0x7f80a0161d90 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.920+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8098037040 con 0x7f80a0161d90 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.920+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a00ca870 con 0x7f80a0161d90 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.920+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a00cae30 con 0x7f80a0161d90 2024-04-25T10:07:31.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.921+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a0068f60 con 0x7f80a0161d90 2024-04-25T10:07:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.922+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8098044070 con 0x7f80a0161d90 2024-04-25T10:07:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.923+0000 7f8097400640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807c06b4d0 0x7f807c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.923+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f80980aaa50 con 0x7f80a0161d90 2024-04-25T10:07:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.924+0000 7f809f400640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807c06b4d0 0x7f807c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.925+0000 7f809f400640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807c06b4d0 0x7f807c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f80900070b0 tx=0x7f8090007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:31.927+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8098079770 con 0x7f80a0161d90 2024-04-25T10:07:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.124+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80a00d0dd0 con 0x7f80a0161d90 2024-04-25T10:07:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.125+0000 7f8097400640 1 -- 172.21.15.5:0/1149075277 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8098079070 con 0x7f80a0161d90 2024-04-25T10:07:32.131 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807c06b4d0 msgr2=0x7f807c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807c06b4d0 0x7f807c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f80900070b0 tx=0x7f8090007040 comp rx=0 tx=0).stop 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 msgr2=0x7f80a00cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00cc450 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8098035040 tx=0x7f80980046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 shutdown_connections 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80a0161d90 0x7f80a00cc450 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f80a00d3f90 0x7f80a00c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f807c06b4d0 0x7f807c06d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 --2- 172.21.15.5:0/1149075277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80a00b48c0 0x7f80a00c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 >> 172.21.15.5:0/1149075277 conn(0x7f80a008af80 msgr2=0x7f80a006d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 shutdown_connections 2024-04-25T10:07:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.128+0000 7f80a5200640 1 -- 172.21.15.5:0/1149075277 wait complete. 2024-04-25T10:07:32.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.355+0000 7f2608a00640 1 Processor -- start 2024-04-25T10:07:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.355+0000 7f2608a00640 1 -- start start 2024-04-25T10:07:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.356+0000 7f2608a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2604141da0 0x7f260406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2608a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2604071d30 con 0x7f260406e640 2024-04-25T10:07:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2608a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260406d620 con 0x7f260415d510 2024-04-25T10:07:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2608a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2604142180 con 0x7f2604141da0 2024-04-25T10:07:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2603400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2604141da0 0x7f260406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2602a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f260406e640 0x7f260415ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59996/0 (socket says 172.21.15.5:59996) 2024-04-25T10:07:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2602a00640 1 -- 172.21.15.5:0/2473743276 learned_addr learned my addr 172.21.15.5:0/2473743276 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2603400640 1 -- 172.21.15.5:0/2473743276 >> v1:172.21.15.80:6789/0 conn(0x7f260415d510 legacy=0x7f26040cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2603400640 1 -- 172.21.15.5:0/2473743276 >> v1:172.21.15.5:6789/0 conn(0x7f260406e640 legacy=0x7f260415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.357+0000 7f2603400640 1 -- 172.21.15.5:0/2473743276 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260406d900 con 0x7f2604141da0 2024-04-25T10:07:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.359+0000 7f2603400640 1 --2- 172.21.15.5:0/2473743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2604141da0 0x7f260406e100 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f25f40097a0 tx=0x7f25f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be6fe43c4fa41a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.359+0000 7f2602000640 1 -- 172.21.15.5:0/2473743276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4037070 con 0x7f2604141da0 2024-04-25T10:07:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.359+0000 7f2602000640 1 -- 172.21.15.5:0/2473743276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4032440 con 0x7f2604141da0 2024-04-25T10:07:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.359+0000 7f2602000640 1 -- 172.21.15.5:0/2473743276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4032810 con 0x7f2604141da0 2024-04-25T10:07:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.359+0000 7f2608a00640 1 -- 172.21.15.5:0/2473743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2604141da0 msgr2=0x7f260406e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.359+0000 7f2608a00640 1 --2- 172.21.15.5:0/2473743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2604141da0 0x7f260406e100 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f25f40097a0 tx=0x7f25f402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.360+0000 7f2608a00640 1 -- 172.21.15.5:0/2473743276 shutdown_connections 2024-04-25T10:07:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.360+0000 7f2608a00640 1 --2- 172.21.15.5:0/2473743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2604141da0 0x7f260406e100 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.360+0000 7f2608a00640 1 -- 172.21.15.5:0/2473743276 >> 172.21.15.5:0/2473743276 conn(0x7f2604066f20 msgr2=0x7f260406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.360+0000 7f2608a00640 1 -- 172.21.15.5:0/2473743276 shutdown_connections 2024-04-25T10:07:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.360+0000 7f2608a00640 1 -- 172.21.15.5:0/2473743276 wait complete. 2024-04-25T10:07:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.361+0000 7f2608a00640 1 Processor -- start 2024-04-25T10:07:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.361+0000 7f2608a00640 1 -- start start 2024-04-25T10:07:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.362+0000 7f2608a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f260406e640 0x7f26040ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.362+0000 7f2608a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2604141da0 0x7f2604159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.362+0000 7f2608a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 0x7f26040af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.362+0000 7f2608a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040cdb90 con 0x7f26040aef00 2024-04-25T10:07:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.362+0000 7f2608a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26040cda10 con 0x7f2604141da0 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.362+0000 7f2608a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2604071000 con 0x7f260406e640 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 0x7f26040af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 0x7f26040af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47710/0 (socket says 172.21.15.5:47710) 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 -- 172.21.15.5:0/1637902802 learned_addr learned my addr 172.21.15.5:0/1637902802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603400640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f260406e640 0x7f26040ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2602a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2604141da0 0x7f2604159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 -- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f260406e640 msgr2=0x7f26040ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f260406e640 0x7f26040ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 -- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2604141da0 msgr2=0x7f2604159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2604141da0 0x7f2604159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2603e00640 1 -- 172.21.15.5:0/1637902802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040aca40 con 0x7f26040aef00 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.363+0000 7f2602a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2604141da0 0x7f2604159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.364+0000 7f2603e00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 0x7f26040af360 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00be30 tx=0x7f25fc00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.364+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc019070 con 0x7f26040aef00 2024-04-25T10:07:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.364+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00caf0 con 0x7f26040aef00 2024-04-25T10:07:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.364+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26040acd20 con 0x7f26040aef00 2024-04-25T10:07:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.364+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc014440 con 0x7f26040aef00 2024-04-25T10:07:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.364+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041e8c50 con 0x7f26040aef00 2024-04-25T10:07:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.365+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26040ccae0 con 0x7f26040aef00 2024-04-25T10:07:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.366+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25fc01c070 con 0x7f26040aef00 2024-04-25T10:07:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.367+0000 7f25fb400640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25e406b4d0 0x7f25e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.367+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f25fc089fb0 con 0x7f26040aef00 2024-04-25T10:07:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.368+0000 7f2603400640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25e406b4d0 0x7f25e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.369+0000 7f2603400640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25e406b4d0 0x7f25e406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.371+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25fc058bb0 con 0x7f26040aef00 2024-04-25T10:07:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.591+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2604068670 con 0x7f26040aef00 2024-04-25T10:07:32.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.591+0000 7f25fb400640 1 -- 172.21.15.5:0/1637902802 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) v1 ==== 80+0+122 (secure 0 0 0) 0x7f25fc0584b0 con 0x7f26040aef00 2024-04-25T10:07:32.598 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6835/1098619647 2024-04-26T10:07:22.531455+0000 2024-04-25T10:07:32.598 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6834/1098619647 2024-04-26T10:07:22.531455+0000 2024-04-25T10:07:32.598 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-04-25T10:07:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.594+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25e406b4d0 msgr2=0x7f25e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.594+0000 7f2608a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25e406b4d0 0x7f25e406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f25f4034000 tx=0x7f25f402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.594+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 msgr2=0x7f26040af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.594+0000 7f2608a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 0x7f26040af360 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00be30 tx=0x7f25fc00bf20 comp rx=0 tx=0).stop 2024-04-25T10:07:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.594+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 shutdown_connections 2024-04-25T10:07:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26040aef00 0x7f26040af360 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25e406b4d0 0x7f25e406d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2604141da0 0x7f2604159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 --2- 172.21.15.5:0/1637902802 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f260406e640 0x7f26040ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 >> 172.21.15.5:0/1637902802 conn(0x7f2604066f20 msgr2=0x7f260413fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 shutdown_connections 2024-04-25T10:07:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.595+0000 7f2608a00640 1 -- 172.21.15.5:0/1637902802 wait complete. 2024-04-25T10:07:32.622 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:32.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:32.719 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:07:32.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.719 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:32.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:32.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T10:07:32.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.989+0000 7fbf52000640 1 Processor -- start 2024-04-25T10:07:32.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.989+0000 7fbf52000640 1 -- start start 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.990+0000 7fbf52000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf4c0d3f10 0x7fbf4c168830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.990+0000 7fbf52000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0d2290 con 0x7fbf4c15de70 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.990+0000 7fbf52000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0d2570 con 0x7fbf4c0d3330 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.990+0000 7fbf52000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c0d2030 con 0x7fbf4c0d3f10 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.990+0000 7fbf4b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf4c15de70 0x7fbf4c0d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60012/0 (socket says 172.21.15.5:60012) 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.990+0000 7fbf4b400640 1 -- 172.21.15.5:0/3746245053 learned_addr learned my addr 172.21.15.5:0/3746245053 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.991+0000 7fbf4be00640 1 --2- 172.21.15.5:0/3746245053 >> v2:172.21.15.5:3300/0 conn(0x7fbf4c0d3f10 0x7fbf4c168830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.991+0000 7fbf4a000640 1 -- 172.21.15.5:0/3746245053 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563842534 0 0) 0x7fbf4c0d2290 con 0x7fbf4c15de70 2024-04-25T10:07:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.991+0000 7fbf4a000640 1 -- 172.21.15.5:0/3746245053 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf30003520 con 0x7fbf4c15de70 2024-04-25T10:07:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.991+0000 7fbf4be00640 1 -- 172.21.15.5:0/3746245053 >> v1:172.21.15.80:6789/0 conn(0x7fbf4c0d3330 legacy=0x7fbf4c0d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.991+0000 7fbf4be00640 1 -- 172.21.15.5:0/3746245053 >> v1:172.21.15.5:6789/0 conn(0x7fbf4c15de70 legacy=0x7fbf4c0d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.991+0000 7fbf4be00640 1 -- 172.21.15.5:0/3746245053 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0d2850 con 0x7fbf4c0d3f10 2024-04-25T10:07:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.992+0000 7fbf4be00640 1 --2- 172.21.15.5:0/3746245053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c168830 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009d90 tx=0x7fbf4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252e0d89503dae09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf4a000640 1 -- 172.21.15.5:0/3746245053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44007740 con 0x7fbf4c0d3f10 2024-04-25T10:07:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf4a000640 1 -- 172.21.15.5:0/3746245053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf440078e0 con 0x7fbf4c0d3f10 2024-04-25T10:07:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf4a000640 1 -- 172.21.15.5:0/3746245053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44007cb0 con 0x7fbf4c0d3f10 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf52000640 1 -- 172.21.15.5:0/3746245053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c0d3f10 msgr2=0x7fbf4c168830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf52000640 1 --2- 172.21.15.5:0/3746245053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c168830 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009d90 tx=0x7fbf4402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf52000640 1 -- 172.21.15.5:0/3746245053 shutdown_connections 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf52000640 1 --2- 172.21.15.5:0/3746245053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c168830 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.993+0000 7fbf52000640 1 -- 172.21.15.5:0/3746245053 >> 172.21.15.5:0/3746245053 conn(0x7fbf4c0726b0 msgr2=0x7fbf4c072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.994+0000 7fbf52000640 1 -- 172.21.15.5:0/3746245053 shutdown_connections 2024-04-25T10:07:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.994+0000 7fbf52000640 1 -- 172.21.15.5:0/3746245053 wait complete. 2024-04-25T10:07:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.995+0000 7fbf52000640 1 Processor -- start 2024-04-25T10:07:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.995+0000 7fbf52000640 1 -- start start 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.996+0000 7fbf52000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf4c0d3330 0x7fbf4c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.996+0000 7fbf52000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.996+0000 7fbf52000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 0x7fbf4c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.996+0000 7fbf52000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0b9290 con 0x7fbf4c15de70 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.996+0000 7fbf52000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0d2e10 con 0x7fbf4c0d3f10 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.996+0000 7fbf52000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0b9410 con 0x7fbf4c0d3330 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 0x7fbf4c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 0x7fbf4c0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47714/0 (socket says 172.21.15.5:47714) 2024-04-25T10:07:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 -- 172.21.15.5:0/3779533421 learned_addr learned my addr 172.21.15.5:0/3779533421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4b400640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf4c0d3330 0x7fbf4c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 -- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf4c0d3330 msgr2=0x7fbf4c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf4c0d3330 0x7fbf4c0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 -- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf4c0d3f10 msgr2=0x7fbf4c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c0cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.997+0000 7fbf4be00640 1 -- 172.21.15.5:0/3779533421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0c7f30 con 0x7fbf4c15de70 2024-04-25T10:07:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.998+0000 7fbf4be00640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 0x7fbf4c0c7320 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009eb0 tx=0x7fbf44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.998+0000 7fbf43400640 1 -- 172.21.15.5:0/3779533421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44040070 con 0x7fbf4c15de70 2024-04-25T10:07:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.998+0000 7fbf43400640 1 -- 172.21.15.5:0/3779533421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf44031bc0 con 0x7fbf4c15de70 2024-04-25T10:07:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.998+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0c8210 con 0x7fbf4c15de70 2024-04-25T10:07:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.998+0000 7fbf43400640 1 -- 172.21.15.5:0/3779533421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44009450 con 0x7fbf4c15de70 2024-04-25T10:07:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:32.998+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c0c8ab0 con 0x7fbf4c15de70 2024-04-25T10:07:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.001+0000 7fbf43400640 1 -- 172.21.15.5:0/3779533421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf44039040 con 0x7fbf4c15de70 2024-04-25T10:07:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.002+0000 7fbf43400640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf2c06b4d0 0x7fbf2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.002+0000 7fbf43400640 1 -- 172.21.15.5:0/3779533421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbf440abd10 con 0x7fbf4c15de70 2024-04-25T10:07:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.003+0000 7fbf54e26740 1 Processor -- start 2024-04-25T10:07:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.003+0000 7fbf4b400640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf2c06b4d0 0x7fbf2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.003+0000 7fbf54e26740 1 -- start start 2024-04-25T10:07:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343e360 0x5653534418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf4b400640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf2c06b4d0 0x7fbf2c06d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0011e0 tx=0x7fbf3c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x56535345dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf54e26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56535337aee0 con 0x56535343ec80 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565353323cf0 con 0x56535343e360 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x565353376f80 con 0x565353441e30 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x56535345dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x56535345dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47728/0 (socket says 172.21.15.5:47728) 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.004+0000 7fbf4be00640 1 -- 172.21.15.5:0/1707291716 learned_addr learned my addr 172.21.15.5:0/1707291716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4b400640 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343e360 0x5653534418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4be00640 1 -- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 msgr2=0x56535343e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4be00640 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4be00640 1 -- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343e360 msgr2=0x5653534418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4be00640 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343e360 0x5653534418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4be00640 1 -- 172.21.15.5:0/1707291716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56535315ba20 con 0x56535343ec80 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf4be00640 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x56535345dd70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbf4407a1a0 tx=0x7fbf4407a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec9755b3d4b893f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf37400640 1 -- 172.21.15.5:0/1707291716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44082920 con 0x56535343ec80 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf37400640 1 -- 172.21.15.5:0/1707291716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4407e0d0 con 0x56535343ec80 2024-04-25T10:07:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.005+0000 7fbf37400640 1 -- 172.21.15.5:0/1707291716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44007840 con 0x56535343ec80 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 -- 172.21.15.5:0/1707291716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 msgr2=0x56535345dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x56535345dd70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fbf4407a1a0 tx=0x7fbf4407a110 comp rx=0 tx=0).stop 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 -- 172.21.15.5:0/1707291716 shutdown_connections 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x56535345dd70 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1707291716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343e360 0x5653534418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 -- 172.21.15.5:0/1707291716 >> 172.21.15.5:0/1707291716 conn(0x565353414150 msgr2=0x565353415150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 -- 172.21.15.5:0/1707291716 shutdown_connections 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.006+0000 7fbf54e26740 1 -- 172.21.15.5:0/1707291716 wait complete. 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.007+0000 7fbf54e26740 1 -- start start 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.007+0000 7fbf54e26740 1 Processor -- start 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x56535343a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 0x56535343cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565353412b90 con 0x565353459390 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x565353319240 con 0x56535343ec80 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x565353411f80 con 0x565353441e30 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.008+0000 7fbf54e26740 10 client.0 _ll_register_callbacks cb 0x56535308ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 0x56535343cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 0x56535343cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47732/0 (socket says 172.21.15.5:47732) 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 -- 172.21.15.5:0/1750223515 learned_addr learned my addr 172.21.15.5:0/1750223515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4b400640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x56535343a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 -- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 msgr2=0x56535343c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 -- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 msgr2=0x56535343a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x56535343a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 -- 172.21.15.5:0/1750223515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56535304f310 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf4be00640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 0x56535343cbe0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbf4407a3d0 tx=0x7fbf44031f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4c993ffe91d752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.009+0000 7fbf2be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf2be00640 1 -- 172.21.15.5:0/1750223515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44078510 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf2be00640 1 -- 172.21.15.5:0/1750223515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbf4407a470 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4407e0d0 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf2be00640 1 -- 172.21.15.5:0/1750223515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf440a3050 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 10 client.7098 fetch_fsmap learned FSMap version 51 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x5653531bc1b0 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf2be00640 1 -- 172.21.15.5:0/1750223515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4407e0d0 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf2be00640 1 -- 172.21.15.5:0/1750223515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbf44082490 con 0x565353459390 2024-04-25T10:07:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 10 client.7098 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:07:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 10 client.7098 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:07:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 20 client.7098 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5653534575b0 0x5653534579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 4 client.7098 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.010+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653533d90c0 con 0x5653534575b0 2024-04-25T10:07:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.011+0000 7fbf4b400640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5653534575b0 0x5653534579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.012+0000 7fbf4b400640 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5653534575b0 0x5653534579c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2262e3f88c99ad server_cookie=fb3734695dcaf6d1 in_seq=0 out_seq=0 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.012+0000 7fbf2be00640 10 client.7098 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf2be00640 1 -- 172.21.15.5:0/1750223515 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653533d90c0 con 0x5653534575b0 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf2be00640 10 client.7098 handle_command_reply: tid=0 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 client.7098 shutdown 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 msgr2=0x56535343cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 0x56535343cbe0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fbf4407a3d0 tx=0x7fbf44031f90 comp rx=0 tx=0).stop 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 shutdown_connections 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5653534575b0 0x5653534579c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353459390 0x56535343cbe0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353441e30 0x56535343c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf2be00640 0 client.7098 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 --2- 172.21.15.5:0/1750223515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x56535343a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.014+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 >> 172.21.15.5:0/1750223515 conn(0x56535343e360 msgr2=0x56535345bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.015+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 shutdown_connections 2024-04-25T10:07:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.015+0000 7fbf54e26740 1 -- 172.21.15.5:0/1750223515 wait complete. 2024-04-25T10:07:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.015+0000 7fbf54e26740 20 client.7098 trim_cache size 0 max 16384 2024-04-25T10:07:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.032+0000 7fbf54e26740 1 Processor -- start 2024-04-25T10:07:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.032+0000 7fbf54e26740 1 -- start start 2024-04-25T10:07:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.032+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x565353439f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 0x5653534124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x56535345bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf54e26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c200420 con 0x565353441e30 2024-04-25T10:07:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c1ff600 con 0x56535343ec80 2024-04-25T10:07:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0cb710 con 0x565353459390 2024-04-25T10:07:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 0x5653534124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 0x5653534124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47748/0 (socket says 172.21.15.5:47748) 2024-04-25T10:07:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 -- 172.21.15.5:0/2600593377 learned_addr learned my addr 172.21.15.5:0/2600593377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4be00640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x56535345bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4b400640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x565353439f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 -- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 msgr2=0x56535345bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x56535345bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 -- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 msgr2=0x565353439f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x565353439f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.033+0000 7fbf4aa00640 1 -- 172.21.15.5:0/2600593377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565353028490 con 0x565353441e30 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.034+0000 7fbf4b400640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x565353439f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.034+0000 7fbf4be00640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x56535345bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.034+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 0x5653534124e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fbf380054f0 tx=0x7fbf380055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4e6acc2f539e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.034+0000 7fbf36a00640 1 -- 172.21.15.5:0/2600593377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38007050 con 0x565353441e30 2024-04-25T10:07:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.034+0000 7fbf36a00640 1 -- 172.21.15.5:0/2600593377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3801b070 con 0x565353441e30 2024-04-25T10:07:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.034+0000 7fbf36a00640 1 -- 172.21.15.5:0/2600593377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf380073f0 con 0x565353441e30 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 -- 172.21.15.5:0/2600593377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 msgr2=0x5653534124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 0x5653534124e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fbf380054f0 tx=0x7fbf380055e0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 -- 172.21.15.5:0/2600593377 shutdown_connections 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x56535345bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565353441e30 0x5653534124e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2600593377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56535343ec80 0x565353439f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 -- 172.21.15.5:0/2600593377 >> 172.21.15.5:0/2600593377 conn(0x56535343e360 msgr2=0x56535343bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 -- 172.21.15.5:0/2600593377 shutdown_connections 2024-04-25T10:07:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.035+0000 7fbf54e26740 1 -- 172.21.15.5:0/2600593377 wait complete. 2024-04-25T10:07:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.036+0000 7fbf54e26740 1 -- start start 2024-04-25T10:07:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.036+0000 7fbf54e26740 1 Processor -- start 2024-04-25T10:07:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x5653534c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565353441e30 0x56535343c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x565353438ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56535343eb00 con 0x56535343ec80 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c1f7b40 con 0x565353441e30 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653534318b0 con 0x565353459390 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.037+0000 7fbf54e26740 10 client.0 _ll_register_callbacks cb 0x7fbf44009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x5653534c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x5653534c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47760/0 (socket says 172.21.15.5:47760) 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 -- 172.21.15.5:0/2608539732 learned_addr learned my addr 172.21.15.5:0/2608539732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565353441e30 0x56535343c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4be00640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x565353438ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 -- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 msgr2=0x565353438ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x565353438ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 -- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565353441e30 msgr2=0x56535343c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565353441e30 0x56535343c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.038+0000 7fbf4b400640 1 -- 172.21.15.5:0/2608539732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56535345c6e0 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf4b400640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x5653534c72f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005920 tx=0x7fbf3c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0eea52143b74f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf2be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf2be00640 1 -- 172.21.15.5:0/2608539732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0072e0 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf2be00640 1 -- 172.21.15.5:0/2608539732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbf3c010040 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf3801b070 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf2be00640 1 -- 172.21.15.5:0/2608539732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf3c036050 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf54e26740 10 client.7104 fetch_fsmap learned FSMap version 51 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x5653534389d0 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf2be00640 1 -- 172.21.15.5:0/2608539732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3801b070 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.039+0000 7fbf2be00640 1 -- 172.21.15.5:0/2608539732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbf3c039020 con 0x56535343ec80 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf54e26740 10 client.7104 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf54e26740 10 client.7104 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf54e26740 20 client.7104 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x565353413690 0x5653534178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf54e26740 4 client.7104 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x565353388420 con 0x565353413690 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.040+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x565353413690 0x5653534178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.041+0000 7fbf4aa00640 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x565353413690 0x5653534178b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f294998ca98b1313 server_cookie=e230e98de9aaa58d in_seq=0 out_seq=0 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.041+0000 7fbf2be00640 10 client.7104 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf2be00640 1 -- 172.21.15.5:0/2608539732 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x565353388420 con 0x565353413690 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf2be00640 10 client.7104 handle_command_reply: tid=0 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 client.7104 shutdown 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 msgr2=0x5653534c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x5653534c72f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c005920 tx=0x7fbf3c011000 comp rx=0 tx=0).stop 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 shutdown_connections 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x565353413690 0x5653534178b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x565353459390 0x565353438ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x565353441e30 0x56535343c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 --2- 172.21.15.5:0/2608539732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56535343ec80 0x5653534c72f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 >> 172.21.15.5:0/2608539732 conn(0x56535343e360 msgr2=0x5653534410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf2be00640 0 client.7104 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 shutdown_connections 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 1 -- 172.21.15.5:0/2608539732 wait complete. 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.043+0000 7fbf54e26740 20 client.7104 trim_cache size 0 max 16384 2024-04-25T10:07:33.054 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:07:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:07:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "91a21849-ed96-4b61-8cd8-57b47eb480df", 2024-04-25T10:07:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:07:33.055 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf2c06b4d0 msgr2=0x7fbf2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf2c06b4d0 0x7fbf2c06d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0011e0 tx=0x7fbf3c009040 comp rx=0 tx=0).stop 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 msgr2=0x7fbf4c0c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 0x7fbf4c0c7320 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009eb0 tx=0x7fbf44004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 shutdown_connections 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf4c15de70 0x7fbf4c0c7320 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbf2c06b4d0 0x7fbf2c06d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf4c0d3f10 0x7fbf4c0cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 --2- 172.21.15.5:0/3779533421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf4c0d3330 0x7fbf4c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 >> 172.21.15.5:0/3779533421 conn(0x7fbf4c0726b0 msgr2=0x7fbf4c154880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 shutdown_connections 2024-04-25T10:07:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.044+0000 7fbf52000640 1 -- 172.21.15.5:0/3779533421 wait complete. 2024-04-25T10:07:33.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:33.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:07:33.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.273+0000 7f42cba00640 1 Processor -- start 2024-04-25T10:07:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.273+0000 7f42cba00640 1 -- start start 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.274+0000 7f42cba00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f42c4144fe0 0x7f42c406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.274+0000 7f42cba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c406d130 0x7f42c4071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.274+0000 7f42cba00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42c4071670 0x7f42c40cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.274+0000 7f42cba00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40a8560 con 0x7f42c4144fe0 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.274+0000 7f42cba00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c41453c0 con 0x7f42c4071670 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.274+0000 7f42cba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c406d6a0 con 0x7f42c406d130 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f42c4144fe0 0x7f42c406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f42c4144fe0 0x7f42c406e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47772/0 (socket says 172.21.15.5:47772) 2024-04-25T10:07:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 -- 172.21.15.5:0/3750203114 learned_addr learned my addr 172.21.15.5:0/3750203114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9a00640 1 --2- 172.21.15.5:0/3750203114 >> v2:172.21.15.80:3300/0 conn(0x7f42c4071670 0x7f42c40cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c3e00640 1 --2- 172.21.15.5:0/3750203114 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c406d130 0x7f42c4071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 -- 172.21.15.5:0/3750203114 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c406d130 msgr2=0x7f42c4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 --2- 172.21.15.5:0/3750203114 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c406d130 0x7f42c4071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 -- 172.21.15.5:0/3750203114 >> v2:172.21.15.80:3300/0 conn(0x7f42c4071670 msgr2=0x7f42c40cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 --2- 172.21.15.5:0/3750203114 >> v2:172.21.15.80:3300/0 conn(0x7f42c4071670 0x7f42c40cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.275+0000 7f42c9000640 1 -- 172.21.15.5:0/3750203114 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c406d980 con 0x7f42c4144fe0 2024-04-25T10:07:33.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.277+0000 7f42c9000640 1 --2- 172.21.15.5:0/3750203114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c4144fe0 0x7f42c406e560 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f42c415bd40 tx=0x7f42b402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37908b9107699e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.277+0000 7f42c3400640 1 -- 172.21.15.5:0/3750203114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4036070 con 0x7f42c4144fe0 2024-04-25T10:07:33.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.277+0000 7f42c3400640 1 -- 172.21.15.5:0/3750203114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4031600 con 0x7f42c4144fe0 2024-04-25T10:07:33.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.277+0000 7f42c3400640 1 -- 172.21.15.5:0/3750203114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b40319d0 con 0x7f42c4144fe0 2024-04-25T10:07:33.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 -- 172.21.15.5:0/3750203114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c4144fe0 msgr2=0x7f42c406e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 --2- 172.21.15.5:0/3750203114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c4144fe0 0x7f42c406e560 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f42c415bd40 tx=0x7f42b402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 -- 172.21.15.5:0/3750203114 shutdown_connections 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 --2- 172.21.15.5:0/3750203114 >> v2:172.21.15.80:3300/0 conn(0x7f42c4071670 0x7f42c40cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 --2- 172.21.15.5:0/3750203114 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c406d130 0x7f42c4071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 --2- 172.21.15.5:0/3750203114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c4144fe0 0x7f42c406e560 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 -- 172.21.15.5:0/3750203114 >> 172.21.15.5:0/3750203114 conn(0x7f42c406ab60 msgr2=0x7f42c406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 -- 172.21.15.5:0/3750203114 shutdown_connections 2024-04-25T10:07:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.278+0000 7f42cba00640 1 -- 172.21.15.5:0/3750203114 wait complete. 2024-04-25T10:07:33.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.280+0000 7f42cba00640 1 Processor -- start 2024-04-25T10:07:33.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.280+0000 7f42cba00640 1 -- start start 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42cba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 0x7f42c41f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42cba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c4071670 0x7f42c40b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42cba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c4144fe0 0x7f42c40cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42cba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c415be50 con 0x7f42c406d130 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42cba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c406df40 con 0x7f42c4144fe0 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42cba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c406eaa0 con 0x7f42c4071670 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 0x7f42c41f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 0x7f42c41f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47774/0 (socket says 172.21.15.5:47774) 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 -- 172.21.15.5:0/2637895129 learned_addr learned my addr 172.21.15.5:0/2637895129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9a00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c4144fe0 0x7f42c40cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c4071670 0x7f42c40b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 -- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c4071670 msgr2=0x7f42c40b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c4071670 0x7f42c40b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 -- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c4144fe0 msgr2=0x7f42c40cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c4144fe0 0x7f42c40cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.281+0000 7f42c9000640 1 -- 172.21.15.5:0/2637895129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c41f7b50 con 0x7f42c406d130 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42c9a00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c4144fe0 0x7f42c40cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c4071670 0x7f42c40b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42c9000640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 0x7f42c41f7610 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009f40 tx=0x7f42b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42c0c00640 1 -- 172.21.15.5:0/2637895129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b403f020 con 0x7f42c406d130 2024-04-25T10:07:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42c0c00640 1 -- 172.21.15.5:0/2637895129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4037040 con 0x7f42c406d130 2024-04-25T10:07:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42c0c00640 1 -- 172.21.15.5:0/2637895129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4036070 con 0x7f42c406d130 2024-04-25T10:07:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c41f39b0 con 0x7f42c406d130 2024-04-25T10:07:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.282+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c41f3f70 con 0x7f42c406d130 2024-04-25T10:07:33.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.285+0000 7f42c0c00640 1 -- 172.21.15.5:0/2637895129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42b4042020 con 0x7f42c406d130 2024-04-25T10:07:33.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.286+0000 7f42c0c00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f42a006b4d0 0x7f42a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.286+0000 7f42c0c00640 1 -- 172.21.15.5:0/2637895129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f42b40aa940 con 0x7f42c406d130 2024-04-25T10:07:33.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.286+0000 7f42ce96b740 1 Processor -- start 2024-04-25T10:07:33.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.286+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f42a006b4d0 0x7f42a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.286+0000 7f42ce96b740 1 -- start start 2024-04-25T10:07:33.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.287+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adba1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.287+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adb9ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.287+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbcaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.287+0000 7f42ce96b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7adb03c70 con 0x55c7adbca670 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.287+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7adaaccf0 con 0x55c7adb9fbb0 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.287+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7adaffd10 con 0x55c7adbdc280 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbcaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbcaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47780/0 (socket says 172.21.15.5:47780) 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 -- 172.21.15.5:0/1872761321 learned_addr learned my addr 172.21.15.5:0/1872761321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f42a006b4d0 0x7f42a006d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42ac006fd0 tx=0x7f42ac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9000640 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adba1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c3e00640 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adb9ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 -- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 msgr2=0x55c7adb9ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adb9ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 -- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 msgr2=0x55c7adba1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adba1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 -- 172.21.15.5:0/1872761321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7ad8e27d0 con 0x55c7adbca670 2024-04-25T10:07:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9000640 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adba1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.288+0000 7f42c9a00640 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbcaaf0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009a10 tx=0x7f42bc009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788047a8f9c1f5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.289+0000 7f42b9600640 1 -- 172.21.15.5:0/1872761321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00c070 con 0x55c7adbca670 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.289+0000 7f42b9600640 1 -- 172.21.15.5:0/1872761321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc018070 con 0x55c7adbca670 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.289+0000 7f42b9600640 1 -- 172.21.15.5:0/1872761321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00f040 con 0x55c7adbca670 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.289+0000 7f42ce96b740 1 -- 172.21.15.5:0/1872761321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 msgr2=0x55c7adbcaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.289+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbcaaf0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009a10 tx=0x7f42bc009ed0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 -- 172.21.15.5:0/1872761321 shutdown_connections 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbcaaf0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adb9ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1872761321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adba1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 -- 172.21.15.5:0/1872761321 >> 172.21.15.5:0/1872761321 conn(0x55c7adb67a30 msgr2=0x55c7adbe5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 -- 172.21.15.5:0/1872761321 shutdown_connections 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 -- 172.21.15.5:0/1872761321 wait complete. 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.290+0000 7f42ce96b740 1 -- start start 2024-04-25T10:07:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.291+0000 7f42ce96b740 1 Processor -- start 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.291+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adb9fbb0 0x55c7adbf64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.291+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbdff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbdc280 0x55c7adbc6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42ce96b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7adaa23a0 con 0x55c7adbca670 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7adbc81c0 con 0x55c7adbdc280 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7adbcbd40 con 0x55c7adb9fbb0 2024-04-25T10:07:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42ce96b740 10 client.0 _ll_register_callbacks cb 0x55c7adabfb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbdff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbdff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47782/0 (socket says 172.21.15.5:47782) 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 -- 172.21.15.5:0/2846513974 learned_addr learned my addr 172.21.15.5:0/2846513974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 -- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adb9fbb0 msgr2=0x55c7adbf64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c9a00640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbdc280 0x55c7adbc6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c9000640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adb9fbb0 0x55c7adbf64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adb9fbb0 0x55c7adbf64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 -- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbdc280 msgr2=0x55c7adbc6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbdc280 0x55c7adbc6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.292+0000 7f42c3e00640 1 -- 172.21.15.5:0/2846513974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7ad8183f0 con 0x55c7adbca670 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42c3e00640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbdff80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ead0 tx=0x7f42ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1517c4db429b913d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42b2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42b2000640 1 -- 172.21.15.5:0/2846513974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00f040 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42b2000640 1 -- 172.21.15.5:0/2846513974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f42ac023ca0 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42bc018070 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42b2000640 1 -- 172.21.15.5:0/2846513974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ac0044e0 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42ce96b740 10 client.7116 fetch_fsmap learned FSMap version 51 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.293+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55c7ad7d80f0 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42b2000640 1 -- 172.21.15.5:0/2846513974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc018070 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42b2000640 1 -- 172.21.15.5:0/2846513974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f42ac02a050 con 0x55c7adbca670 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42ce96b740 10 client.7116 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42ce96b740 10 client.7116 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42ce96b740 20 client.7116 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42ce96b740 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbf8fc0 0x55c7adbf9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42ce96b740 4 client.7116 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7adb111b0 con 0x55c7adbf8fc0 2024-04-25T10:07:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.294+0000 7f42c9000640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbf8fc0 0x55c7adbf9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.295+0000 7f42c9000640 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbf8fc0 0x55c7adbf9440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ec1974f546f04d server_cookie=5b63042695c08c9d in_seq=0 out_seq=0 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.295+0000 7f42b2000640 10 client.7116 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.297+0000 7f42b2000640 1 -- 172.21.15.5:0/2846513974 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7adb111b0 con 0x55c7adbf8fc0 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.297+0000 7f42b2000640 10 client.7116 handle_command_reply: tid=0 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.297+0000 7f42ce96b740 1 client.7116 shutdown 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 msgr2=0x55c7adbdff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbdff80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00ead0 tx=0x7f42ac0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 shutdown_connections 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbf8fc0 0x55c7adbf9440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbdc280 0x55c7adbc6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbdff80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 --2- 172.21.15.5:0/2846513974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adb9fbb0 0x55c7adbf64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42b2000640 0 client.7116 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 >> 172.21.15.5:0/2846513974 conn(0x55c7adb67a30 msgr2=0x55c7adbc7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 shutdown_connections 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 1 -- 172.21.15.5:0/2846513974 wait complete. 2024-04-25T10:07:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.298+0000 7f42ce96b740 20 client.7116 trim_cache size 0 max 16384 2024-04-25T10:07:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.315+0000 7f42ce96b740 1 Processor -- start 2024-04-25T10:07:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.315+0000 7f42ce96b740 1 -- start start 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 0x55c7adbe5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbca670 0x55c7adbff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbf5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42ce96b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4200560 con 0x55c7adb9fbb0 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c41ff6c0 con 0x55c7adbca670 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c41e3d30 con 0x55c7adbdc280 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42c9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 0x55c7adbe5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42c9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 0x55c7adbe5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47798/0 (socket says 172.21.15.5:47798) 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.316+0000 7f42c9000640 1 -- 172.21.15.5:0/1982777026 learned_addr learned my addr 172.21.15.5:0/1982777026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42c9000640 1 -- 172.21.15.5:0/1982777026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 msgr2=0x55c7adbf5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42c9000640 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbf5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42c9000640 1 -- 172.21.15.5:0/1982777026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbca670 msgr2=0x55c7adbff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42c9000640 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbca670 0x55c7adbff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42c9000640 1 -- 172.21.15.5:0/1982777026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7adb9be90 con 0x55c7adb9fbb0 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42c9000640 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 0x55c7adbe5500 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f42b4075330 tx=0x7f42b40a8b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ea9465a1bf7cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42b8c00640 1 -- 172.21.15.5:0/1982777026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b40a7070 con 0x55c7adb9fbb0 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42b8c00640 1 -- 172.21.15.5:0/1982777026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b40795d0 con 0x55c7adb9fbb0 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.317+0000 7f42b8c00640 1 -- 172.21.15.5:0/1982777026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4090070 con 0x55c7adb9fbb0 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 -- 172.21.15.5:0/1982777026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 msgr2=0x55c7adbe5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 0x55c7adbe5500 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f42b4075330 tx=0x7f42b40a8b50 comp rx=0 tx=0).stop 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 -- 172.21.15.5:0/1982777026 shutdown_connections 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbf5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adbca670 0x55c7adbff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 --2- 172.21.15.5:0/1982777026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adb9fbb0 0x55c7adbe5500 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 -- 172.21.15.5:0/1982777026 >> 172.21.15.5:0/1982777026 conn(0x55c7adb67a30 msgr2=0x55c7adbdfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 -- 172.21.15.5:0/1982777026 shutdown_connections 2024-04-25T10:07:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.318+0000 7f42ce96b740 1 -- 172.21.15.5:0/1982777026 wait complete. 2024-04-25T10:07:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.319+0000 7f42ce96b740 1 -- start start 2024-04-25T10:07:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.319+0000 7f42ce96b740 1 Processor -- start 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adbde390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbde8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbdb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7adb9a140 con 0x55c7adbca670 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c41ff0c0 con 0x55c7adb9fbb0 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7adc510e0 con 0x55c7adbdc280 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42ce96b740 10 client.0 _ll_register_callbacks cb 0x7f42ac0095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbde8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbde8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47810/0 (socket says 172.21.15.5:47810) 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.320+0000 7f42c3e00640 1 -- 172.21.15.5:0/654604143 learned_addr learned my addr 172.21.15.5:0/654604143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c9000640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adbde390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c9a00640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbdb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c3e00640 1 -- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 msgr2=0x55c7adbdb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c3e00640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbdb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c3e00640 1 -- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 msgr2=0x55c7adbde390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c3e00640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adbde390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c3e00640 1 -- 172.21.15.5:0/654604143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7adbffcf0 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42c3e00640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbde8d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f42ac010010 tx=0x7f42ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fb7815d1267243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42b2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42b2000640 1 -- 172.21.15.5:0/654604143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac030020 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42b2000640 1 -- 172.21.15.5:0/654604143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f42ac005b90 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.321+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42b40795d0 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42b2000640 1 -- 172.21.15.5:0/654604143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ac03a050 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 10 client.7122 fetch_fsmap learned FSMap version 51 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55c7adbdcf00 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42b2000640 1 -- 172.21.15.5:0/654604143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b40795d0 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42b2000640 1 -- 172.21.15.5:0/654604143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f42ac03d020 con 0x55c7adbca670 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 10 client.7122 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T10:07:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 10 client.7122 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:07:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 20 client.7122 populate_metadata read hostname 'smithi005' 2024-04-25T10:07:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbe9580 0x55c7adbdb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 4 client.7122 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.322+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7adb0b500 con 0x55c7adbe9580 2024-04-25T10:07:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.323+0000 7f42c9000640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbe9580 0x55c7adbdb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.324+0000 7f42c9000640 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbe9580 0x55c7adbdb540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6d503805a98f81 server_cookie=5c1678d51e559346 in_seq=0 out_seq=0 2024-04-25T10:07:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.324+0000 7f42b2000640 10 client.7122 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.324+0000 7f42b2000640 1 -- 172.21.15.5:0/654604143 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c7adb0b500 con 0x55c7adbe9580 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.324+0000 7f42b2000640 10 client.7122 handle_command_reply: tid=0 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 client.7122 shutdown 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 msgr2=0x55c7adbde8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbde8d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f42ac010010 tx=0x7f42ac013000 comp rx=0 tx=0).stop 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 shutdown_connections 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55c7adbe9580 0x55c7adbdb540 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c7adbdc280 0x55c7adbdb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7adbca670 0x55c7adbde8d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 --2- 172.21.15.5:0/654604143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c7adb9fbb0 0x55c7adbde390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42b2000640 0 client.7122 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:07:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 >> 172.21.15.5:0/654604143 conn(0x55c7adb67a30 msgr2=0x55c7adbff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 shutdown_connections 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 1 -- 172.21.15.5:0/654604143 wait complete. 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.325+0000 7f42ce96b740 20 client.7122 trim_cache size 0 max 16384 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout: "91a21849-ed96-4b61-8cd8-57b47eb480df": { 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/subdir", 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "91a21849-ed96-4b61-8cd8-57b47eb480df", 2024-04-25T10:07:33.334 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive" 2024-04-25T10:07:33.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:33.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:07:33.335 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:07:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f42a006b4d0 msgr2=0x7f42a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f42a006b4d0 0x7f42a006d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42ac006fd0 tx=0x7f42ac008040 comp rx=0 tx=0).stop 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 msgr2=0x7f42c41f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 0x7f42c41f7610 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009f40 tx=0x7f42b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 shutdown_connections 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f42a006b4d0 0x7f42a006d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c4144fe0 0x7f42c40cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c4071670 0x7f42c40b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 --2- 172.21.15.5:0/2637895129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42c406d130 0x7f42c41f7610 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 >> 172.21.15.5:0/2637895129 conn(0x7f42c406ab60 msgr2=0x7f42c41442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 shutdown_connections 2024-04-25T10:07:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:07:33.326+0000 7f42cba00640 1 -- 172.21.15.5:0/2637895129 wait complete. 2024-04-25T10:07:33.349 INFO:tasks.cephfs.filesystem:scrub status for tag:91a21849-ed96-4b61-8cd8-57b47eb480df - {'path': '/subdir', 'tag': '91a21849-ed96-4b61-8cd8-57b47eb480df', 'options': 'recursive'} 2024-04-25T10:07:33.442 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:07:33.435+0000 7f92a8200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:07:54.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:54.792 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:03.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:03.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.568+0000 7f1014400640 1 Processor -- start 2024-04-25T10:08:03.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.568+0000 7f1014400640 1 -- start start 2024-04-25T10:08:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.570+0000 7f1014400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f100c06dcb0 0x7f100c06e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.570+0000 7f1014400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.570+0000 7f1014400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f100c0acd30 0x7f100c06b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.570+0000 7f1014400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c06eb10 con 0x7f100c0acd30 2024-04-25T10:08:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.570+0000 7f1014400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c0ad290 con 0x7f100c06dcb0 2024-04-25T10:08:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.570+0000 7f1014400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0b3cf0 con 0x7f100c0b37f0 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f100c0acd30 0x7f100c06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f100c0acd30 0x7f100c06b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59522/0 (socket says 172.21.15.5:59522) 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 -- 172.21.15.5:0/3687339921 learned_addr learned my addr 172.21.15.5:0/3687339921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1011000640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1011a00640 1 --2- 172.21.15.5:0/3687339921 >> v2:172.21.15.80:3300/0 conn(0x7f100c06dcb0 0x7f100c06e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 -- 172.21.15.5:0/3687339921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 msgr2=0x7f100c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 -- 172.21.15.5:0/3687339921 >> v2:172.21.15.80:3300/0 conn(0x7f100c06dcb0 msgr2=0x7f100c06e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 --2- 172.21.15.5:0/3687339921 >> v2:172.21.15.80:3300/0 conn(0x7f100c06dcb0 0x7f100c06e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1012400640 1 -- 172.21.15.5:0/3687339921 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c06ab60 con 0x7f100c0acd30 2024-04-25T10:08:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.571+0000 7f1011000640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.573+0000 7f1012400640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c06b920 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f100c072c30 tx=0x7f100402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b838cba350a251eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.573+0000 7f100be00640 1 -- 172.21.15.5:0/3687339921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004036070 con 0x7f100c0acd30 2024-04-25T10:08:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.573+0000 7f100be00640 1 -- 172.21.15.5:0/3687339921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1004031600 con 0x7f100c0acd30 2024-04-25T10:08:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.573+0000 7f100be00640 1 -- 172.21.15.5:0/3687339921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040319d0 con 0x7f100c0acd30 2024-04-25T10:08:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 -- 172.21.15.5:0/3687339921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 msgr2=0x7f100c06b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c06b920 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f100c072c30 tx=0x7f100402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 -- 172.21.15.5:0/3687339921 shutdown_connections 2024-04-25T10:08:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c06b920 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 --2- 172.21.15.5:0/3687339921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 --2- 172.21.15.5:0/3687339921 >> v2:172.21.15.80:3300/0 conn(0x7f100c06dcb0 0x7f100c06e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 -- 172.21.15.5:0/3687339921 >> 172.21.15.5:0/3687339921 conn(0x7f100c0cfdc0 msgr2=0x7f100c0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 -- 172.21.15.5:0/3687339921 shutdown_connections 2024-04-25T10:08:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.574+0000 7f1014400640 1 -- 172.21.15.5:0/3687339921 wait complete. 2024-04-25T10:08:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.576+0000 7f1014400640 1 Processor -- start 2024-04-25T10:08:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.576+0000 7f1014400640 1 -- start start 2024-04-25T10:08:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1014400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f100c06dcb0 0x7f100c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1014400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1014400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1014400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c06b400 con 0x7f100c0acd30 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1014400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c06b120 con 0x7f100c06dcb0 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1014400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c06f6d0 con 0x7f100c0b37f0 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c1449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59536/0 (socket says 172.21.15.5:59536) 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.577+0000 7f1011000640 1 -- 172.21.15.5:0/2206307688 learned_addr learned my addr 172.21.15.5:0/2206307688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011a00640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f100c06dcb0 0x7f100c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1012400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011000640 1 -- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 msgr2=0x7f100c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011000640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c1582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011000640 1 -- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f100c06dcb0 msgr2=0x7f100c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011000640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f100c06dcb0 0x7f100c15f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011000640 1 -- 172.21.15.5:0/2206307688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c15ffc0 con 0x7f100c0acd30 2024-04-25T10:08:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1011000640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c1449b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e810 tx=0x7f0ff800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1009600640 1 -- 172.21.15.5:0/2206307688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800f040 con 0x7f100c0acd30 2024-04-25T10:08:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.578+0000 7f1009600640 1 -- 172.21.15.5:0/2206307688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8004030 con 0x7f100c0acd30 2024-04-25T10:08:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.579+0000 7f1009600640 1 -- 172.21.15.5:0/2206307688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8004400 con 0x7f100c0acd30 2024-04-25T10:08:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.579+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c1602a0 con 0x7f100c0acd30 2024-04-25T10:08:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.579+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c160860 con 0x7f100c0acd30 2024-04-25T10:08:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.581+0000 7f1009600640 1 -- 172.21.15.5:0/2206307688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ff800c6d0 con 0x7f100c0acd30 2024-04-25T10:08:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.582+0000 7f1009600640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0fec06b500 0x7f0fec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.582+0000 7f1009600640 1 -- 172.21.15.5:0/2206307688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0ff808a180 con 0x7f100c0acd30 2024-04-25T10:08:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.582+0000 7f1011a00640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0fec06b500 0x7f0fec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.583+0000 7f10171bc740 1 Processor -- start 2024-04-25T10:08:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.583+0000 7f10171bc740 1 -- start start 2024-04-25T10:08:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f1011a00640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0fec06b500 0x7f0fec06d9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1000006fd0 tx=0x7f1000008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f799cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f10171bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f799af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79c5670 0x5590f79c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f10171bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f78fec70 con 0x5590f79d7280 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f78a7cf0 con 0x5590f799abb0 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590f78fad10 con 0x5590f79c5670 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f799af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f799af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59546/0 (socket says 172.21.15.5:59546) 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.584+0000 7f1011000640 1 -- 172.21.15.5:0/1978458807 learned_addr learned my addr 172.21.15.5:0/1978458807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011a00640 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f799cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1012400640 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79c5670 0x5590f79c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011000640 1 -- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79c5670 msgr2=0x5590f79c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011000640 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79c5670 0x5590f79c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011000640 1 -- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 msgr2=0x5590f799cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011000640 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f799cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011000640 1 -- 172.21.15.5:0/1978458807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f76dd7d0 con 0x5590f79d7280 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011a00640 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f799cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f1011000640 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f799af90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8088440 tx=0x7f0ff8060e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad73ef9c772317d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f0ffe000640 1 -- 172.21.15.5:0/1978458807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8020040 con 0x5590f79d7280 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f0ffe000640 1 -- 172.21.15.5:0/1978458807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8070120 con 0x5590f79d7280 2024-04-25T10:08:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.585+0000 7f0ffe000640 1 -- 172.21.15.5:0/1978458807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8057b70 con 0x5590f79d7280 2024-04-25T10:08:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 -- 172.21.15.5:0/1978458807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 msgr2=0x5590f799af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f799af90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8088440 tx=0x7f0ff8060e30 comp rx=0 tx=0).stop 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 -- 172.21.15.5:0/1978458807 shutdown_connections 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79c5670 0x5590f79c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f799af90 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 --2- 172.21.15.5:0/1978458807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f799cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 -- 172.21.15.5:0/1978458807 >> 172.21.15.5:0/1978458807 conn(0x5590f7962a30 msgr2=0x5590f79e0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 -- 172.21.15.5:0/1978458807 shutdown_connections 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.586+0000 7f10171bc740 1 -- 172.21.15.5:0/1978458807 wait complete. 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.587+0000 7f10171bc740 1 -- start start 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.587+0000 7f10171bc740 1 Processor -- start 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f79c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 0x5590f79dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f789d3a0 con 0x5590f79c5670 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f79c31c0 con 0x5590f799abb0 2024-04-25T10:08:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590f79f0670 con 0x5590f79d7280 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.588+0000 7f10171bc740 10 client.0 _ll_register_callbacks cb 0x5590f78bab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59558/0 (socket says 172.21.15.5:59558) 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 -- 172.21.15.5:0/3867701628 learned_addr learned my addr 172.21.15.5:0/3867701628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 -- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 msgr2=0x5590f79dd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 0x5590f79dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 -- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 msgr2=0x5590f79c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f79c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 -- 172.21.15.5:0/3867701628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f76133f0 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f1011000640 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79c7210 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e7e0 tx=0x7f0ff801d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0fa06c5b9ca5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f0ff2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.589+0000 7f0ff2a00640 1 -- 172.21.15.5:0/3867701628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8020040 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f0ff2a00640 1 -- 172.21.15.5:0/3867701628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0ff8071d40 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ff8070120 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f0ff2a00640 1 -- 172.21.15.5:0/3867701628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff8058720 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 10 client.7134 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5590f75d30f0 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f0ff2a00640 1 -- 172.21.15.5:0/3867701628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8070120 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f0ff2a00640 1 -- 172.21.15.5:0/3867701628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ff8012050 con 0x5590f79c5670 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 10 client.7134 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 10 client.7134 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 20 client.7134 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79f3a40 0x5590f79f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 4 client.7134 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.590+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590f790c1b0 con 0x5590f79f3a40 2024-04-25T10:08:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.591+0000 7f1011a00640 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79f3a40 0x5590f79f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.592+0000 7f1011a00640 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79f3a40 0x5590f79f3ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93bb5f1515d39e26 server_cookie=5fdeb7c5ae6cda2c in_seq=0 out_seq=0 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.592+0000 7f0ff2a00640 10 client.7134 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f0ff2a00640 1 -- 172.21.15.5:0/3867701628 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590f790c1b0 con 0x5590f79f3a40 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f0ff2a00640 10 client.7134 handle_command_reply: tid=0 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 client.7134 shutdown 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 msgr2=0x5590f79c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79c7210 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e7e0 tx=0x7f0ff801d070 comp rx=0 tx=0).stop 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 shutdown_connections 2024-04-25T10:08:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79f3a40 0x5590f79f3ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 0x5590f79dd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79c7210 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 --2- 172.21.15.5:0/3867701628 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f79c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 >> 172.21.15.5:0/3867701628 conn(0x5590f7962a30 msgr2=0x5590f79e3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f0ff2a00640 0 client.7134 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 shutdown_connections 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.594+0000 7f10171bc740 1 -- 172.21.15.5:0/3867701628 wait complete. 2024-04-25T10:08:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.595+0000 7f10171bc740 20 client.7134 trim_cache size 0 max 16384 2024-04-25T10:08:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.611+0000 7f10171bc740 1 Processor -- start 2024-04-25T10:08:03.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.611+0000 7f10171bc740 1 -- start start 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.612+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f79f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.612+0000 7f10171bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.612+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 0x5590f79d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.612+0000 7f10171bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c200500 con 0x5590f79c5670 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.612+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c1ff700 con 0x5590f799abb0 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.612+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c15bb80 con 0x5590f79d7280 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79fa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59574/0 (socket says 172.21.15.5:59574) 2024-04-25T10:08:03.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 -- 172.21.15.5:0/1004025933 learned_addr learned my addr 172.21.15.5:0/1004025933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 -- 172.21.15.5:0/1004025933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 msgr2=0x5590f79d96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 0x5590f79d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 -- 172.21.15.5:0/1004025933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 msgr2=0x5590f79f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f79f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 -- 172.21.15.5:0/1004025933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590f79dda50 con 0x5590f79c5670 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.613+0000 7f1011000640 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79fa390 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0ff805ca30 tx=0x7f0ff8083070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448a37058a1d04fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f0ffd600640 1 -- 172.21.15.5:0/1004025933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8020070 con 0x5590f79c5670 2024-04-25T10:08:03.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f0ffd600640 1 -- 172.21.15.5:0/1004025933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8085030 con 0x5590f79c5670 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f0ffd600640 1 -- 172.21.15.5:0/1004025933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8054470 con 0x5590f79c5670 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 -- 172.21.15.5:0/1004025933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 msgr2=0x5590f79fa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79fa390 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0ff805ca30 tx=0x7f0ff8083070 comp rx=0 tx=0).stop 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 -- 172.21.15.5:0/1004025933 shutdown_connections 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f79d7280 0x5590f79d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79c5670 0x5590f79fa390 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 --2- 172.21.15.5:0/1004025933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f799abb0 0x5590f79f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 -- 172.21.15.5:0/1004025933 >> 172.21.15.5:0/1004025933 conn(0x5590f7962a30 msgr2=0x5590f79e30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 -- 172.21.15.5:0/1004025933 shutdown_connections 2024-04-25T10:08:03.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.614+0000 7f10171bc740 1 -- 172.21.15.5:0/1004025933 wait complete. 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.615+0000 7f10171bc740 1 -- start start 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.615+0000 7f10171bc740 1 Processor -- start 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f799abb0 0x5590f7a300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f79c5670 0x5590f79dce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f79f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c15c180 con 0x5590f79d7280 2024-04-25T10:08:03.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590f7a30b20 con 0x5590f79c5670 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590f79e3130 con 0x5590f799abb0 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.616+0000 7f10171bc740 10 client.0 _ll_register_callbacks cb 0x7f0ff8070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f79f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f79f9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59584/0 (socket says 172.21.15.5:59584) 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 -- 172.21.15.5:0/38996881 learned_addr learned my addr 172.21.15.5:0/38996881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1011a00640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f799abb0 0x5590f7a300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1011000640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f79c5670 0x5590f79dce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 -- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f799abb0 msgr2=0x5590f7a300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f799abb0 0x5590f7a300a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 -- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f79c5670 msgr2=0x5590f79dce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f79c5670 0x5590f79dce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 -- 172.21.15.5:0/38996881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590f79e3660 con 0x5590f79d7280 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f1012400640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f79f9e50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f100402ac90 tx=0x7f10040318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f07f29969af91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.617+0000 7f0ff2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f0ff2a00640 1 -- 172.21.15.5:0/38996881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004041070 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f0ff2a00640 1 -- 172.21.15.5:0/38996881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f100403c440 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ff8085030 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f0ff2a00640 1 -- 172.21.15.5:0/38996881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f100404c050 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 10 client.7140 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5590f7a333e0 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f0ff2a00640 1 -- 172.21.15.5:0/38996881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8085030 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f0ff2a00640 1 -- 172.21.15.5:0/38996881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f100404f020 con 0x5590f79d7280 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 10 client.7140 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 10 client.7140 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 20 client.7140 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79e19e0 0x5590f79e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 4 client.7140 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.618+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590f7906500 con 0x5590f79e19e0 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.619+0000 7f1011a00640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79e19e0 0x5590f79e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.620+0000 7f1011a00640 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79e19e0 0x5590f79e1df0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7dc1672451dde62 server_cookie=8facc43a705d0754 in_seq=0 out_seq=0 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.620+0000 7f0ff2a00640 10 client.7140 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f0ff2a00640 1 -- 172.21.15.5:0/38996881 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590f7906500 con 0x5590f79e19e0 2024-04-25T10:08:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f0ff2a00640 10 client.7140 handle_command_reply: tid=0 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 client.7140 shutdown 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 msgr2=0x5590f79f9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f79f9e50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f100402ac90 tx=0x7f10040318b0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 shutdown_connections 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x5590f79e19e0 0x5590f79e1df0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5590f79d7280 0x5590f79f9e50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5590f79c5670 0x5590f79dce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 --2- 172.21.15.5:0/38996881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5590f799abb0 0x5590f7a300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f0ff2a00640 0 client.7140 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 >> 172.21.15.5:0/38996881 conn(0x5590f7962a30 msgr2=0x5590f79c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 shutdown_connections 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 1 -- 172.21.15.5:0/38996881 wait complete. 2024-04-25T10:08:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.621+0000 7f10171bc740 20 client.7140 trim_cache size 0 max 16384 2024-04-25T10:08:03.630 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:08:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:08:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:08:03.630 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0fec06b500 msgr2=0x7f0fec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0fec06b500 0x7f0fec06d9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1000006fd0 tx=0x7f1000008040 comp rx=0 tx=0).stop 2024-04-25T10:08:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 msgr2=0x7f100c1449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c1449b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e810 tx=0x7f0ff800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 shutdown_connections 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f100c0b37f0 0x7f100c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f100c0acd30 0x7f100c1449b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0fec06b500 0x7f0fec06d9c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 --2- 172.21.15.5:0/2206307688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f100c06dcb0 0x7f100c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 >> 172.21.15.5:0/2206307688 conn(0x7f100c0cfdc0 msgr2=0x7f100c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 shutdown_connections 2024-04-25T10:08:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.622+0000 7f1014400640 1 -- 172.21.15.5:0/2206307688 wait complete. 2024-04-25T10:08:03.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:08:03.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.837+0000 7f8999a00640 1 Processor -- start 2024-04-25T10:08:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.838+0000 7f8999a00640 1 -- start start 2024-04-25T10:08:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.839+0000 7f8999a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8994161200 0x7f89940b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.839+0000 7f8999a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89940d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.839+0000 7f8999a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89940cf410 con 0x7f89940d0600 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.839+0000 7f8999a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89940d27c0 con 0x7f8994161200 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.839+0000 7f8999a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89940b28b0 con 0x7f89940d2380 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.839+0000 7f8993e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8994161200 0x7f89940b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8994161200 0x7f89940b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59586/0 (socket says 172.21.15.5:59586) 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993e00640 1 -- 172.21.15.5:0/221321361 learned_addr learned my addr 172.21.15.5:0/221321361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993400640 1 --2- 172.21.15.5:0/221321361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89940d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993e00640 1 -- 172.21.15.5:0/221321361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 msgr2=0x7f89940d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993e00640 1 --2- 172.21.15.5:0/221321361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89940d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993e00640 1 -- 172.21.15.5:0/221321361 >> v1:172.21.15.5:6789/0 conn(0x7f89940d0600 legacy=0x7f89940ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.840+0000 7f8993e00640 1 -- 172.21.15.5:0/221321361 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89940cf6f0 con 0x7f8994161200 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.841+0000 7f8993e00640 1 --2- 172.21.15.5:0/221321361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89940b4bb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f89840098a0 tx=0x7f898402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb62103778a877a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8992a00640 1 -- 172.21.15.5:0/221321361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984037070 con 0x7f8994161200 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8992a00640 1 -- 172.21.15.5:0/221321361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8984032440 con 0x7f8994161200 2024-04-25T10:08:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8992a00640 1 -- 172.21.15.5:0/221321361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984032810 con 0x7f8994161200 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8999a00640 1 -- 172.21.15.5:0/221321361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 msgr2=0x7f89940b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8999a00640 1 --2- 172.21.15.5:0/221321361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89940b4bb0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f89840098a0 tx=0x7f898402a290 comp rx=0 tx=0).stop 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8999a00640 1 -- 172.21.15.5:0/221321361 shutdown_connections 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8999a00640 1 --2- 172.21.15.5:0/221321361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89940d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8999a00640 1 --2- 172.21.15.5:0/221321361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89940b4bb0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.842+0000 7f8999a00640 1 -- 172.21.15.5:0/221321361 >> 172.21.15.5:0/221321361 conn(0x7f899408af80 msgr2=0x7f89940d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.843+0000 7f8999a00640 1 -- 172.21.15.5:0/221321361 shutdown_connections 2024-04-25T10:08:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.843+0000 7f8999a00640 1 -- 172.21.15.5:0/221321361 wait complete. 2024-04-25T10:08:03.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.844+0000 7f8999a00640 1 Processor -- start 2024-04-25T10:08:03.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.844+0000 7f8999a00640 1 -- start start 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.845+0000 7f8999a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940d0600 0x7f89940b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.845+0000 7f8999a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89941f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.845+0000 7f8999a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89941f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.845+0000 7f8999a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89940b3f90 con 0x7f8994161200 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.845+0000 7f8999a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89940d42b0 con 0x7f89940d0600 2024-04-25T10:08:03.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.845+0000 7f8999a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89940b4110 con 0x7f89940d2380 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89941f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89941f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59602/0 (socket says 172.21.15.5:59602) 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 -- 172.21.15.5:0/3323239911 learned_addr learned my addr 172.21.15.5:0/3323239911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8993400640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89941f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 -- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 msgr2=0x7f89941f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89941f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8993e00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940d0600 0x7f89940b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 -- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940d0600 msgr2=0x7f89940b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940d0600 0x7f89940b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8999000640 1 -- 172.21.15.5:0/3323239911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89940a0ff0 con 0x7f8994161200 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.846+0000 7f8993e00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940d0600 0x7f89940b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.847+0000 7f8999000640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89941f72d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c6b0 tx=0x7f898c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.847+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c003b70 con 0x7f8994161200 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.847+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c003d30 con 0x7f8994161200 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.847+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8994151730 con 0x7f8994161200 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.847+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c005310 con 0x7f8994161200 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.847+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8994151cf0 con 0x7f8994161200 2024-04-25T10:08:03.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.848+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8994068f60 con 0x7f8994161200 2024-04-25T10:08:03.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.850+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f898c0054b0 con 0x7f8994161200 2024-04-25T10:08:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.851+0000 7f898be00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f897406b520 0x7f897406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.851+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f898c08a290 con 0x7f8994161200 2024-04-25T10:08:03.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.852+0000 7f8993e00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f897406b520 0x7f897406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.853+0000 7f8993e00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f897406b520 0x7f897406d9e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8984034000 tx=0x7f898402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:03.856+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f898c058f10 con 0x7f8994161200 2024-04-25T10:08:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.093+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89940d3570 con 0x7f8994161200 2024-04-25T10:08:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.094+0000 7f898be00640 1 -- 172.21.15.5:0/3323239911 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f898c058810 con 0x7f8994161200 2024-04-25T10:08:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:04.101 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:07:33.396051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:04.101 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 52 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f897406b520 msgr2=0x7f897406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f897406b520 0x7f897406d9e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8984034000 tx=0x7f898402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 msgr2=0x7f89941f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89941f72d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c6b0 tx=0x7f898c00cb70 comp rx=0 tx=0).stop 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 shutdown_connections 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8994161200 0x7f89941f72d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89940d2380 0x7f89941f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f897406b520 0x7f897406d9e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 --2- 172.21.15.5:0/3323239911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940d0600 0x7f89940b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 >> 172.21.15.5:0/3323239911 conn(0x7f899408af80 msgr2=0x7f899406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 shutdown_connections 2024-04-25T10:08:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.097+0000 7f8999a00640 1 -- 172.21.15.5:0/3323239911 wait complete. 2024-04-25T10:08:04.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-25T10:08:04.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.331+0000 7f84f5800640 1 Processor -- start 2024-04-25T10:08:04.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.332+0000 7f84f5800640 1 -- start start 2024-04-25T10:08:04.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.333+0000 7f84f5800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f00d3e70 0x7f84f00ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.333+0000 7f84f5800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00d1a40 con 0x7f84f00b2c30 2024-04-25T10:08:04.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.333+0000 7f84f5800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00d1d20 con 0x7f84f015deb0 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.333+0000 7f84f5800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00d2000 con 0x7f84f00d3e70 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ee000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f84f00b2c30 0x7f84f00d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50222/0 (socket says 172.21.15.5:50222) 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ee000640 1 -- 172.21.15.5:0/1696351287 learned_addr learned my addr 172.21.15.5:0/1696351287 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ef400640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f00d3e70 0x7f84f00ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378396466 0 0) 0x7f84f00d1a40 con 0x7f84f00b2c30 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d4003520 con 0x7f84f00b2c30 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213899250 0 0) 0x7f84d4003520 con 0x7f84f00b2c30 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f00d3e70 msgr2=0x7f84f00ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f00d3e70 0x7f84f00ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 >> v1:172.21.15.80:6789/0 conn(0x7f84f015deb0 legacy=0x7f84f00b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.334+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00d22e0 con 0x7f84f00b2c30 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f84dc001670 con 0x7f84f00b2c30 2024-04-25T10:08:04.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 >> v1:172.21.15.5:6789/0 conn(0x7f84f00b2c30 legacy=0x7f84f00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d40039e0 0x7f84d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84d4004800 0x7f84d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84d40051c0 0x7f84d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00d2000 con 0x7f84d4004800 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d400c060 con 0x7f84d40039e0 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d400c260 con 0x7f84d40051c0 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ef400640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f00d3e70 0x7f84f00ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ee000640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84d4004800 0x7f84d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ee000640 1 -- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84d40051c0 msgr2=0x7f84d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ef400640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84d40051c0 0x7f84d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ee000640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84d40051c0 0x7f84d400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ee000640 1 -- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d40039e0 msgr2=0x7f84d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ee000640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d40039e0 0x7f84d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.335+0000 7f84ee000640 1 -- 172.21.15.5:0/1696351287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d4003520 con 0x7f84d4004800 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.336+0000 7f84ee000640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84d4004800 0x7f84d4004c80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c210 tx=0x7f84d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df68cf2eb654663d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc007d30 con 0x7f84d4004800 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc005ad0 con 0x7f84d4004800 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84ed600640 1 -- 172.21.15.5:0/1696351287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc007070 con 0x7f84d4004800 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84f5800640 1 -- 172.21.15.5:0/1696351287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84d4004800 msgr2=0x7f84d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84f5800640 1 --2- 172.21.15.5:0/1696351287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84d4004800 0x7f84d4004c80 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00c210 tx=0x7f84d4005830 comp rx=0 tx=0).stop 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84ef400640 1 -- 172.21.15.5:0/1696351287 reap_dead start 2024-04-25T10:08:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84f5800640 1 -- 172.21.15.5:0/1696351287 shutdown_connections 2024-04-25T10:08:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.337+0000 7f84f5800640 1 -- 172.21.15.5:0/1696351287 >> 172.21.15.5:0/1696351287 conn(0x7f84f00726b0 msgr2=0x7f84f0072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.338+0000 7f84f5800640 1 -- 172.21.15.5:0/1696351287 shutdown_connections 2024-04-25T10:08:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.338+0000 7f84f5800640 1 -- 172.21.15.5:0/1696351287 wait complete. 2024-04-25T10:08:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.339+0000 7f84f5800640 1 Processor -- start 2024-04-25T10:08:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.339+0000 7f84f5800640 1 -- start start 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.340+0000 7f84f5800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d4004800 0x7f84f00cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.340+0000 7f84f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 0x7f84f01f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.340+0000 7f84f5800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f01f6cd0 0x7f84f01f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.340+0000 7f84f5800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00ce030 con 0x7f84f00aa3d0 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.340+0000 7f84f5800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00b70f0 con 0x7f84d4004800 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.340+0000 7f84f5800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00ce1b0 con 0x7f84f01f6cd0 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 0x7f84f01f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 0x7f84f01f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59618/0 (socket says 172.21.15.5:59618) 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 -- 172.21.15.5:0/3227191865 learned_addr learned my addr 172.21.15.5:0/3227191865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84eea00640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d4004800 0x7f84f00cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ef400640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f01f6cd0 0x7f84f01f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 -- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f01f6cd0 msgr2=0x7f84f01f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f01f6cd0 0x7f84f01f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 -- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d4004800 msgr2=0x7f84f00cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d4004800 0x7f84f00cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ee000640 1 -- 172.21.15.5:0/3227191865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00b4bf0 con 0x7f84f00aa3d0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.341+0000 7f84ef400640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f01f6cd0 0x7f84f01f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.342+0000 7f84ee000640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 0x7f84f01f0a80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f84dc034040 tx=0x7f84dc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.342+0000 7f84e6a00640 1 -- 172.21.15.5:0/3227191865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc042020 con 0x7f84f00aa3d0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.342+0000 7f84e6a00640 1 -- 172.21.15.5:0/3227191865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc004bf0 con 0x7f84f00aa3d0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.342+0000 7f84e6a00640 1 -- 172.21.15.5:0/3227191865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc007070 con 0x7f84f00aa3d0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.342+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f00b4ed0 con 0x7f84f00aa3d0 2024-04-25T10:08:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.342+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00b5490 con 0x7f84f00aa3d0 2024-04-25T10:08:04.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.344+0000 7f84e6a00640 1 -- 172.21.15.5:0/3227191865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84dc045020 con 0x7f84f00aa3d0 2024-04-25T10:08:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.345+0000 7f84e6a00640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84d006b4d0 0x7f84d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.345+0000 7f84e6a00640 1 -- 172.21.15.5:0/3227191865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f84dc0aa590 con 0x7f84f00aa3d0 2024-04-25T10:08:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.346+0000 7f84f8774740 1 Processor -- start 2024-04-25T10:08:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.346+0000 7f84eea00640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84d006b4d0 0x7f84d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.346+0000 7f84f8774740 1 -- start start 2024-04-25T10:08:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb78d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7aac10 0x559efb7ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84f8774740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 0x559efb7a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84f8774740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb6c5ae0 con 0x559efb763050 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb66e7c0 con 0x559efb78d370 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559efb6c1b80 con 0x559efb7aac10 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84eea00640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84d006b4d0 0x7f84d006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007fd0 tx=0x7f84e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 0x559efb7a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 0x559efb7a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59622/0 (socket says 172.21.15.5:59622) 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84ef400640 1 -- 172.21.15.5:0/3730677951 learned_addr learned my addr 172.21.15.5:0/3730677951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.347+0000 7f84eea00640 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb78d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ee000640 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7aac10 0x559efb7ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ef400640 1 -- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7aac10 msgr2=0x559efb7ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ef400640 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7aac10 0x559efb7ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ef400640 1 -- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 msgr2=0x559efb78d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ef400640 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb78d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ef400640 1 -- 172.21.15.5:0/3730677951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efb4a67c0 con 0x559efb763050 2024-04-25T10:08:04.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84ef400640 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 0x559efb7a92d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f84f00b93a0 tx=0x7f84e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc2d7098e5e1023 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84db400640 1 -- 172.21.15.5:0/3730677951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8018070 con 0x559efb763050 2024-04-25T10:08:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.348+0000 7f84db400640 1 -- 172.21.15.5:0/3730677951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800be50 con 0x559efb763050 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84db400640 1 -- 172.21.15.5:0/3730677951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8010c30 con 0x559efb763050 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 -- 172.21.15.5:0/3730677951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 msgr2=0x559efb7a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 0x559efb7a92d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f84f00b93a0 tx=0x7f84e8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 -- 172.21.15.5:0/3730677951 shutdown_connections 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb763050 0x559efb7a92d0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7aac10 0x559efb7ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 --2- 172.21.15.5:0/3730677951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb78d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 -- 172.21.15.5:0/3730677951 >> 172.21.15.5:0/3730677951 conn(0x559efb75f980 msgr2=0x559efb7a8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 -- 172.21.15.5:0/3730677951 shutdown_connections 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.349+0000 7f84f8774740 1 -- 172.21.15.5:0/3730677951 wait complete. 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.350+0000 7f84f8774740 1 -- start start 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.350+0000 7f84f8774740 1 Processor -- start 2024-04-25T10:08:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 0x559efb7b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7aac10 0x559efb790d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb763df0 con 0x559efb78d370 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb663d40 con 0x559efb7aac10 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559efb75d8b0 con 0x559efb7a1000 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.351+0000 7f84f8774740 10 client.0 _ll_register_callbacks cb 0x559efb3d9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 0x559efb7b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 0x559efb7b80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59634/0 (socket says 172.21.15.5:59634) 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 -- 172.21.15.5:0/1321149470 learned_addr learned my addr 172.21.15.5:0/1321149470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84ef400640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84ee000640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7aac10 0x559efb790d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 -- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 msgr2=0x559efb7ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 -- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7aac10 msgr2=0x559efb790d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7aac10 0x559efb790d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84eea00640 1 -- 172.21.15.5:0/1321149470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efb506f50 con 0x559efb78d370 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84ef400640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.352+0000 7f84ee000640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7aac10 0x559efb790d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84eea00640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 0x559efb7b80e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f84e000eb80 tx=0x7f84e0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c9adffe8570a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84cfe00640 1 -- 172.21.15.5:0/1321149470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0027060 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84cfe00640 1 -- 172.21.15.5:0/1321149470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f84e0010040 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84e800be50 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84cfe00640 1 -- 172.21.15.5:0/1321149470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e00057e0 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84f8774740 10 client.7161 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559efb39a0b0 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.353+0000 7f84cfe00640 1 -- 172.21.15.5:0/1321149470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800be50 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84cfe00640 1 -- 172.21.15.5:0/1321149470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f84e0022430 con 0x559efb78d370 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84f8774740 10 client.7161 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84f8774740 10 client.7161 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84f8774740 20 client.7161 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84f8774740 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb78f610 0x559efb79fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84f8774740 4 client.7161 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559efb724b60 con 0x559efb78f610 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.354+0000 7f84ee000640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb78f610 0x559efb79fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.355+0000 7f84ee000640 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb78f610 0x559efb79fae0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46770cb6c9ec5f21 server_cookie=922dbebd29d63b1e in_seq=0 out_seq=0 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.355+0000 7f84cfe00640 10 client.7161 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84cfe00640 1 -- 172.21.15.5:0/1321149470 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559efb724b60 con 0x559efb78f610 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84cfe00640 10 client.7161 handle_command_reply: tid=0 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 client.7161 shutdown 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 msgr2=0x559efb7b80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 0x559efb7b80e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f84e000eb80 tx=0x7f84e0001250 comp rx=0 tx=0).stop 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 shutdown_connections 2024-04-25T10:08:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb78f610 0x559efb79fae0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7aac10 0x559efb790d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 --2- 172.21.15.5:0/1321149470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb78d370 0x559efb7b80e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 >> 172.21.15.5:0/1321149470 conn(0x559efb763050 msgr2=0x559efb75c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.357+0000 7f84cfe00640 0 client.7161 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.358+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 shutdown_connections 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.358+0000 7f84f8774740 1 -- 172.21.15.5:0/1321149470 wait complete. 2024-04-25T10:08:04.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.358+0000 7f84f8774740 20 client.7161 trim_cache size 0 max 16384 2024-04-25T10:08:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.371+0000 7f84f8774740 1 Processor -- start 2024-04-25T10:08:04.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.371+0000 7f84f8774740 1 -- start start 2024-04-25T10:08:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb78d370 0x559efb7aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7a1000 0x559efb7ab530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84f8774740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb75c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84f8774740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01ffd50 con 0x559efb7aac10 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01ff150 con 0x559efb7a1000 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f01ff750 con 0x559efb78d370 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb75c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb75c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59640/0 (socket says 172.21.15.5:59640) 2024-04-25T10:08:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.372+0000 7f84ef400640 1 -- 172.21.15.5:0/888017679 learned_addr learned my addr 172.21.15.5:0/888017679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ef400640 1 -- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb78d370 msgr2=0x559efb7aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ef400640 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb78d370 0x559efb7aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ef400640 1 -- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7a1000 msgr2=0x559efb7ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ee000640 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7a1000 0x559efb7ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ef400640 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7a1000 0x559efb7ab530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ef400640 1 -- 172.21.15.5:0/888017679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efb7aa390 con 0x559efb7aac10 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ee000640 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7a1000 0x559efb7ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84ef400640 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb75c6e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f84e800b4a0 tx=0x7f84e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2d01e02afaeba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84daa00640 1 -- 172.21.15.5:0/888017679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8018040 con 0x559efb7aac10 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84daa00640 1 -- 172.21.15.5:0/888017679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8010a10 con 0x559efb7aac10 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.373+0000 7f84daa00640 1 -- 172.21.15.5:0/888017679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8010de0 con 0x559efb7aac10 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 -- 172.21.15.5:0/888017679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 msgr2=0x559efb75c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb75c6e0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f84e800b4a0 tx=0x7f84e8015000 comp rx=0 tx=0).stop 2024-04-25T10:08:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 -- 172.21.15.5:0/888017679 shutdown_connections 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb75c6e0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb7a1000 0x559efb7ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 --2- 172.21.15.5:0/888017679 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb78d370 0x559efb7aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 -- 172.21.15.5:0/888017679 >> 172.21.15.5:0/888017679 conn(0x559efb763050 msgr2=0x559efb789070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 -- 172.21.15.5:0/888017679 shutdown_connections 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.374+0000 7f84f8774740 1 -- 172.21.15.5:0/888017679 wait complete. 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.375+0000 7f84f8774740 1 -- start start 2024-04-25T10:08:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.375+0000 7f84f8774740 1 Processor -- start 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb7fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb7aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb78f490 con 0x559efb7aac10 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efb7b91d0 con 0x559efb78d370 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559efb78fa90 con 0x559efb7a1000 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84f8774740 10 client.0 _ll_register_callbacks cb 0x559efb75dd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb7aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb7aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59642/0 (socket says 172.21.15.5:59642) 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84ef400640 1 -- 172.21.15.5:0/1407101500 learned_addr learned my addr 172.21.15.5:0/1407101500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84ee000640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.376+0000 7f84eea00640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb7fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ef400640 1 -- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 msgr2=0x559efb7b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ef400640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ef400640 1 -- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 msgr2=0x559efb7fbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ef400640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb7fbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ef400640 1 -- 172.21.15.5:0/1407101500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efb78e290 con 0x559efb7aac10 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ee000640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84eea00640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb7fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84ef400640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb7aaff0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f84e8000c00 tx=0x7f84e00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d078dc0a1b8c1f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84cfe00640 1 -- 172.21.15.5:0/1407101500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e8018050 con 0x559efb7aac10 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84e8010a10 con 0x559efb7aac10 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.377+0000 7f84cfe00640 1 -- 172.21.15.5:0/1407101500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f84e8004230 con 0x559efb7aac10 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84cfe00640 1 -- 172.21.15.5:0/1407101500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e8038050 con 0x559efb7aac10 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 10 client.7167 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559efb78de10 con 0x559efb7aac10 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84cfe00640 1 -- 172.21.15.5:0/1407101500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e8010a10 con 0x559efb7aac10 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84cfe00640 1 -- 172.21.15.5:0/1407101500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f84e803b020 con 0x559efb7aac10 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 10 client.7167 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 10 client.7167 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 20 client.7167 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb75ce00 0x559efb812b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 4 client.7167 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.378+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x559efb6d3020 con 0x559efb75ce00 2024-04-25T10:08:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.379+0000 7f84eea00640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb75ce00 0x559efb812b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.380+0000 7f84eea00640 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb75ce00 0x559efb812b30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3e59b467f00885 server_cookie=1eff9908c132390d in_seq=0 out_seq=0 2024-04-25T10:08:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.380+0000 7f84cfe00640 10 client.7167 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.380+0000 7f84cfe00640 1 -- 172.21.15.5:0/1407101500 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x559efb6d3020 con 0x559efb75ce00 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.380+0000 7f84cfe00640 10 client.7167 handle_command_reply: tid=0 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.380+0000 7f84f8774740 1 client.7167 shutdown 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 msgr2=0x559efb7aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb7aaff0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f84e8000c00 tx=0x7f84e00014a0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 shutdown_connections 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x559efb75ce00 0x559efb812b30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559efb7aac10 0x559efb7aaff0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x559efb7a1000 0x559efb7b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84cfe00640 0 client.7167 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 --2- 172.21.15.5:0/1407101500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559efb78d370 0x559efb7fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 >> 172.21.15.5:0/1407101500 conn(0x559efb763050 msgr2=0x559efb789070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 shutdown_connections 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 1 -- 172.21.15.5:0/1407101500 wait complete. 2024-04-25T10:08:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.381+0000 7f84f8774740 20 client.7167 trim_cache size 0 max 16384 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "dentry", 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 2055017340, 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627776, 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "frag": "*", 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "dname": "file_to_be_damaged", 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_id": "head", 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T10:08:04.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:08:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84d006b4d0 msgr2=0x7f84d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84d006b4d0 0x7f84d006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f84e0007fd0 tx=0x7f84e0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 msgr2=0x7f84f01f0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 0x7f84f01f0a80 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f84dc034040 tx=0x7f84dc001a40 comp rx=0 tx=0).stop 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 shutdown_connections 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84f01f6cd0 0x7f84f01f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f00aa3d0 0x7f84f01f0a80 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84d006b4d0 0x7f84d006d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 --2- 172.21.15.5:0/3227191865 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84d4004800 0x7f84f00cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 >> 172.21.15.5:0/3227191865 conn(0x7f84f00726b0 msgr2=0x7f84f015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 shutdown_connections 2024-04-25T10:08:04.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.382+0000 7f84f5800640 1 -- 172.21.15.5:0/3227191865 wait complete. 2024-04-25T10:08:04.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:08:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.609+0000 7faef3200640 1 Processor -- start 2024-04-25T10:08:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.609+0000 7faef3200640 1 -- start start 2024-04-25T10:08:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.611+0000 7faef3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faef40d0b10 0x7faef40d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.611+0000 7faef3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faef40ce700 0x7faef40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.611+0000 7faef3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef406c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.611+0000 7faef3200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef40d2f10 con 0x7faef40d0b10 2024-04-25T10:08:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.611+0000 7faef3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef406cb40 con 0x7faef40ce700 2024-04-25T10:08:04.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.611+0000 7faef3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef40d1c30 con 0x7faef406bdc0 2024-04-25T10:08:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faef40d0b10 0x7faef40d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faef40d0b10 0x7faef40d29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59656/0 (socket says 172.21.15.5:59656) 2024-04-25T10:08:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 -- 172.21.15.5:0/835034921 learned_addr learned my addr 172.21.15.5:0/835034921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef2800640 1 --2- 172.21.15.5:0/835034921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef406c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1400640 1 --2- 172.21.15.5:0/835034921 >> v2:172.21.15.80:3300/0 conn(0x7faef40ce700 0x7faef40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 -- 172.21.15.5:0/835034921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 msgr2=0x7faef406c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 --2- 172.21.15.5:0/835034921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef406c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 -- 172.21.15.5:0/835034921 >> v2:172.21.15.80:3300/0 conn(0x7faef40ce700 msgr2=0x7faef40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 --2- 172.21.15.5:0/835034921 >> v2:172.21.15.80:3300/0 conn(0x7faef40ce700 0x7faef40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.612+0000 7faef1e00640 1 -- 172.21.15.5:0/835034921 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40d1f10 con 0x7faef40d0b10 2024-04-25T10:08:04.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.614+0000 7faef1e00640 1 --2- 172.21.15.5:0/835034921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40d0b10 0x7faef40d29d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7faef40b30f0 tx=0x7faee402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647a5abe27cf243b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.614+0000 7faef0a00640 1 -- 172.21.15.5:0/835034921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4037070 con 0x7faef40d0b10 2024-04-25T10:08:04.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.614+0000 7faef0a00640 1 -- 172.21.15.5:0/835034921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee4032440 con 0x7faef40d0b10 2024-04-25T10:08:04.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.614+0000 7faef0a00640 1 -- 172.21.15.5:0/835034921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4032810 con 0x7faef40d0b10 2024-04-25T10:08:04.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 -- 172.21.15.5:0/835034921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40d0b10 msgr2=0x7faef40d29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 --2- 172.21.15.5:0/835034921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40d0b10 0x7faef40d29d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7faef40b30f0 tx=0x7faee402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 -- 172.21.15.5:0/835034921 shutdown_connections 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 --2- 172.21.15.5:0/835034921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef406c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 --2- 172.21.15.5:0/835034921 >> v2:172.21.15.80:3300/0 conn(0x7faef40ce700 0x7faef40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 --2- 172.21.15.5:0/835034921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40d0b10 0x7faef40d29d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 -- 172.21.15.5:0/835034921 >> 172.21.15.5:0/835034921 conn(0x7faef40b53d0 msgr2=0x7faef40b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 -- 172.21.15.5:0/835034921 shutdown_connections 2024-04-25T10:08:04.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.615+0000 7faef3200640 1 -- 172.21.15.5:0/835034921 wait complete. 2024-04-25T10:08:04.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.617+0000 7faef3200640 1 Processor -- start 2024-04-25T10:08:04.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.617+0000 7faef3200640 1 -- start start 2024-04-25T10:08:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef41e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 0x7faef40af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef40d0b10 0x7faef40ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef406c730 con 0x7faef40ce700 2024-04-25T10:08:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef40d24d0 con 0x7faef40d0b10 2024-04-25T10:08:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef406c8b0 con 0x7faef406bdc0 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 0x7faef40af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 0x7faef40af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59658/0 (socket says 172.21.15.5:59658) 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef1400640 1 -- 172.21.15.5:0/3658109405 learned_addr learned my addr 172.21.15.5:0/3658109405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.618+0000 7faef1e00640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef41e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef2800640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef40d0b10 0x7faef40ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef1400640 1 -- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 msgr2=0x7faef41e6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef1400640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef41e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef1400640 1 -- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef40d0b10 msgr2=0x7faef40ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef1400640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef40d0b10 0x7faef40ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef1400640 1 -- 172.21.15.5:0/3658109405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40ab220 con 0x7faef40ce700 2024-04-25T10:08:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef1400640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 0x7faef40af0e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faee000d800 tx=0x7faee000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000b6c0 con 0x7faef40ce700 2024-04-25T10:08:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee0019070 con 0x7faef40ce700 2024-04-25T10:08:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000ba80 con 0x7faef40ce700 2024-04-25T10:08:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef40ab4e0 con 0x7faef40ce700 2024-04-25T10:08:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.619+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef40abaa0 con 0x7faef40ce700 2024-04-25T10:08:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.620+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef40677d0 con 0x7faef40ce700 2024-04-25T10:08:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.622+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faee000bc20 con 0x7faef40ce700 2024-04-25T10:08:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.622+0000 7faeea000640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faed406b4d0 0x7faed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.623+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faee008a8d0 con 0x7faef40ce700 2024-04-25T10:08:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.623+0000 7faef1e00640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faed406b4d0 0x7faed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.624+0000 7faef1e00640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faed406b4d0 0x7faed406d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faee4034000 tx=0x7faee402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.627+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faee0059560 con 0x7faef40ce700 2024-04-25T10:08:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.869+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faef4068f60 con 0x7faef40ce700 2024-04-25T10:08:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.870+0000 7faeea000640 1 -- 172.21.15.5:0/3658109405 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7faee0058e60 con 0x7faef40ce700 2024-04-25T10:08:04.876 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:04.876 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faed406b4d0 msgr2=0x7faed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faed406b4d0 0x7faed406d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7faee4034000 tx=0x7faee402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 msgr2=0x7faef40af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 0x7faef40af0e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7faee000d800 tx=0x7faee000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 shutdown_connections 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faed406b4d0 0x7faed406d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef40d0b10 0x7faef40ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faef40ce700 0x7faef40af0e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 --2- 172.21.15.5:0/3658109405 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faef406bdc0 0x7faef41e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 >> 172.21.15.5:0/3658109405 conn(0x7faef40b53d0 msgr2=0x7faef41e6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 shutdown_connections 2024-04-25T10:08:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:04.873+0000 7faef3200640 1 -- 172.21.15.5:0/3658109405 wait complete. 2024-04-25T10:08:04.899 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:08:04.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:04.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T10:08:05.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.100+0000 7f11bd600640 1 Processor -- start 2024-04-25T10:08:05.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.100+0000 7f11bd600640 1 -- start start 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11bd600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f11b815a580 0x7f11b815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11bd600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f11b806d020 0x7f11b806d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11bd600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b80b33f0 con 0x7f11b8071890 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11bd600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b80b2630 con 0x7f11b815a580 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11bd600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b80b2910 con 0x7f11b806d020 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11b7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f11b8071890 0x7f11b8071c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50236/0 (socket says 172.21.15.5:50236) 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11b7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f11b815a580 0x7f11b815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11b7e00640 1 -- 172.21.15.5:0/2915316175 learned_addr learned my addr 172.21.15.5:0/2915316175 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:05.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.102+0000 7f11bcc00640 1 --2- 172.21.15.5:0/2915316175 >> v2:172.21.15.80:3300/0 conn(0x7f11b806d020 0x7f11b806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.103+0000 7f11b7400640 1 -- 172.21.15.5:0/2915316175 >> v2:172.21.15.80:3300/0 conn(0x7f11b806d020 msgr2=0x7f11b806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.103+0000 7f11b7400640 1 --2- 172.21.15.5:0/2915316175 >> v2:172.21.15.80:3300/0 conn(0x7f11b806d020 0x7f11b806d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.103+0000 7f11b7400640 1 -- 172.21.15.5:0/2915316175 >> v1:172.21.15.5:6789/0 conn(0x7f11b8071890 legacy=0x7f11b8071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.103+0000 7f11b6a00640 1 -- 172.21.15.5:0/2915316175 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884381810 0 0) 0x7f11b80b33f0 con 0x7f11b8071890 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.103+0000 7f11b7400640 1 -- 172.21.15.5:0/2915316175 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80b2bf0 con 0x7f11b815a580 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.104+0000 7f11b7400640 1 --2- 172.21.15.5:0/2915316175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b815a580 0x7f11b815aa30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009d90 tx=0x7f11a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca97e6064ac4539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.104+0000 7f11b6a00640 1 -- 172.21.15.5:0/2915316175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0007740 con 0x7f11b815a580 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.104+0000 7f11b6a00640 1 -- 172.21.15.5:0/2915316175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a00078e0 con 0x7f11b815a580 2024-04-25T10:08:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.104+0000 7f11b6a00640 1 -- 172.21.15.5:0/2915316175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0007cb0 con 0x7f11b815a580 2024-04-25T10:08:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 -- 172.21.15.5:0/2915316175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b815a580 msgr2=0x7f11b815aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 --2- 172.21.15.5:0/2915316175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b815a580 0x7f11b815aa30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009d90 tx=0x7f11a002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 -- 172.21.15.5:0/2915316175 shutdown_connections 2024-04-25T10:08:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 --2- 172.21.15.5:0/2915316175 >> v2:172.21.15.80:3300/0 conn(0x7f11b806d020 0x7f11b806d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 --2- 172.21.15.5:0/2915316175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b815a580 0x7f11b815aa30 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 -- 172.21.15.5:0/2915316175 >> 172.21.15.5:0/2915316175 conn(0x7f11b8066f20 msgr2=0x7f11b806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 -- 172.21.15.5:0/2915316175 shutdown_connections 2024-04-25T10:08:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.105+0000 7f11bd600640 1 -- 172.21.15.5:0/2915316175 wait complete. 2024-04-25T10:08:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.107+0000 7f11bd600640 1 Processor -- start 2024-04-25T10:08:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.107+0000 7f11bd600640 1 -- start start 2024-04-25T10:08:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11bd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 0x7f11b814d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11bd600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11b8071890 0x7f11b8158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11bd600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b815a580 0x7f11b8139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11bd600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8154480 con 0x7f11b806d020 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11bd600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b815af70 con 0x7f11b815a580 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11bd600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8154600 con 0x7f11b8071890 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 0x7f11b814d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 0x7f11b814d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59676/0 (socket says 172.21.15.5:59676) 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.108+0000 7f11b7e00640 1 -- 172.21.15.5:0/3101934712 learned_addr learned my addr 172.21.15.5:0/3101934712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11b7e00640 1 -- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11b8071890 msgr2=0x7f11b8158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11b8071890 0x7f11b8158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11b7e00640 1 -- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b815a580 msgr2=0x7f11b8139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b815a580 0x7f11b8139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11b7e00640 1 -- 172.21.15.5:0/3101934712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b814d7b0 con 0x7f11b806d020 2024-04-25T10:08:05.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 0x7f11b814d0d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f11a800c4d0 tx=0x7f11a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.109+0000 7f11afe00640 1 -- 172.21.15.5:0/3101934712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8018070 con 0x7f11b806d020 2024-04-25T10:08:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.110+0000 7f11afe00640 1 -- 172.21.15.5:0/3101934712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8007500 con 0x7f11b806d020 2024-04-25T10:08:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.110+0000 7f11afe00640 1 -- 172.21.15.5:0/3101934712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8004a20 con 0x7f11b806d020 2024-04-25T10:08:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.110+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b814b160 con 0x7f11b806d020 2024-04-25T10:08:05.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.110+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b814b720 con 0x7f11b806d020 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.112+0000 7f11afe00640 1 -- 172.21.15.5:0/3101934712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11a801b070 con 0x7f11b806d020 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.113+0000 7f11afe00640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f119406b4d0 0x7f119406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.113+0000 7f11afe00640 1 -- 172.21.15.5:0/3101934712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f11a8089790 con 0x7f11b806d020 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.113+0000 7f11c19dc740 1 Processor -- start 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 -- start start 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11b7400640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f119406b4d0 0x7f119406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 0x55a423557260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423527000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423527540 0x55a42354f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42348fb50 con 0x55a423556e80 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a423438fb0 con 0x55a4235578a0 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.114+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42348bbf0 con 0x55a423527540 2024-04-25T10:08:05.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 0x55a423557260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 0x55a423557260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59682/0 (socket says 172.21.15.5:59682) 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 -- 172.21.15.5:0/1602523456 learned_addr learned my addr 172.21.15.5:0/1602523456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7400640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f119406b4d0 0x7f119406d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f11a0034000 tx=0x7f11a002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11bcc00640 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423527540 0x55a42354f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 -- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423527540 msgr2=0x55a42354f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7400640 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423527000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423527540 0x55a42354f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 -- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 msgr2=0x55a423527000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423527000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.115+0000 7f11b7e00640 1 -- 172.21.15.5:0/1602523456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42326ca50 con 0x55a423556e80 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.116+0000 7f11b7e00640 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 0x55a423557260 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f11a805fe70 tx=0x7f11a8057d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d585930112d96ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.116+0000 7f11a7e00640 1 -- 172.21.15.5:0/1602523456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8055d30 con 0x55a423556e80 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.116+0000 7f11a7e00640 1 -- 172.21.15.5:0/1602523456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8055f90 con 0x55a423556e80 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.116+0000 7f11a7e00640 1 -- 172.21.15.5:0/1602523456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8004bc0 con 0x55a423556e80 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.116+0000 7f11c19dc740 1 -- 172.21.15.5:0/1602523456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 msgr2=0x55a423557260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.116+0000 7f11c19dc740 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 0x55a423557260 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f11a805fe70 tx=0x7f11a8057d50 comp rx=0 tx=0).stop 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 -- 172.21.15.5:0/1602523456 shutdown_connections 2024-04-25T10:08:05.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423527540 0x55a42354f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423527000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 --2- 172.21.15.5:0/1602523456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423556e80 0x55a423557260 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 -- 172.21.15.5:0/1602523456 >> 172.21.15.5:0/1602523456 conn(0x55a423529270 msgr2=0x55a423571c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 -- 172.21.15.5:0/1602523456 shutdown_connections 2024-04-25T10:08:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 -- 172.21.15.5:0/1602523456 wait complete. 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.117+0000 7f11c19dc740 1 -- start start 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.118+0000 7f11c19dc740 1 Processor -- start 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423548500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423548a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a423554c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4235535d0 con 0x55a423546100 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42342e590 con 0x55a4235578a0 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a423553870 con 0x55a423556e80 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11c19dc740 10 client.0 _ll_register_callbacks cb 0x55a4231a3ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11bcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a423554c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11bcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a423554c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59694/0 (socket says 172.21.15.5:59694) 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11bcc00640 1 -- 172.21.15.5:0/3688310993 learned_addr learned my addr 172.21.15.5:0/3688310993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.119+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423548500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11b7400640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423548a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11bcc00640 1 -- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 msgr2=0x55a423548500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11bcc00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423548500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11bcc00640 1 -- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 msgr2=0x55a423548a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11bcc00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423548a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11bcc00640 1 -- 172.21.15.5:0/3688310993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4231643e0 con 0x55a423546100 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423548500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11bcc00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a423554c70 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009f80 tx=0x7f11b000e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc248f61184ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11a4c00640 1 -- 172.21.15.5:0/3688310993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0017020 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11a4c00640 1 -- 172.21.15.5:0/3688310993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f11b0013410 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.120+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11a8055f90 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11a4c00640 1 -- 172.21.15.5:0/3688310993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b0021050 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 10 client.7188 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55a4232cd1e0 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11a4c00640 1 -- 172.21.15.5:0/3688310993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8055f90 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11a4c00640 1 -- 172.21.15.5:0/3688310993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11b0024020 con 0x55a423546100 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 10 client.7188 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 10 client.7188 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 20 client.7188 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:05.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a423558460 0x55a42354a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 4 client.7188 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:05.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.121+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a42349d090 con 0x55a423558460 2024-04-25T10:08:05.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.122+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a423558460 0x55a42354a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.123+0000 7f11b7e00640 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a423558460 0x55a42354a830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a23ba2196cf2f9 server_cookie=8eb1c919394e4a23 in_seq=0 out_seq=0 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.123+0000 7f11a4c00640 10 client.7188 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.124+0000 7f11a4c00640 1 -- 172.21.15.5:0/3688310993 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a42349d090 con 0x55a423558460 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11a4c00640 10 client.7188 handle_command_reply: tid=0 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 client.7188 shutdown 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 msgr2=0x55a423554c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a423554c70 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009f80 tx=0x7f11b000e600 comp rx=0 tx=0).stop 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 shutdown_connections 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a423558460 0x55a42354a830 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a423554c70 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423548a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 --2- 172.21.15.5:0/3688310993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423548500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11a4c00640 0 client.7188 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 >> 172.21.15.5:0/3688310993 conn(0x55a423527540 msgr2=0x55a42354c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 shutdown_connections 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 1 -- 172.21.15.5:0/3688310993 wait complete. 2024-04-25T10:08:05.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.125+0000 7f11c19dc740 20 client.7188 trim_cache size 0 max 16384 2024-04-25T10:08:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.142+0000 7f11c19dc740 1 Processor -- start 2024-04-25T10:08:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.142+0000 7f11c19dc740 1 -- start start 2024-04-25T10:08:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a4235535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423549d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423554350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11c19dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a42352a180 con 0x55a423546100 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b82001c0 con 0x55a4235578a0 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81ff360 con 0x55a423556e80 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.143+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a4235535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a4235535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59708/0 (socket says 172.21.15.5:59708) 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 -- 172.21.15.5:0/4148961371 learned_addr learned my addr 172.21.15.5:0/4148961371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11bcc00640 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423554350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7400640 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423549d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 -- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 msgr2=0x55a423549d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423549d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 -- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 msgr2=0x55a423554350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423554350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 -- 172.21.15.5:0/4148961371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a42313d4a0 con 0x55a423546100 2024-04-25T10:08:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11b7e00640 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a4235535d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f11a805fe70 tx=0x7f11a8057b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb0fdc273063c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.144+0000 7f11a7400640 1 -- 172.21.15.5:0/4148961371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8054300 con 0x55a423546100 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11a7400640 1 -- 172.21.15.5:0/4148961371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8004bc0 con 0x55a423546100 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11a7400640 1 -- 172.21.15.5:0/4148961371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a80544a0 con 0x55a423546100 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 -- 172.21.15.5:0/4148961371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 msgr2=0x55a4235535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a4235535d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f11a805fe70 tx=0x7f11a8057b90 comp rx=0 tx=0).stop 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 -- 172.21.15.5:0/4148961371 shutdown_connections 2024-04-25T10:08:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a4235578a0 0x55a423554350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a423556e80 0x55a423549d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 --2- 172.21.15.5:0/4148961371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a4235535d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 -- 172.21.15.5:0/4148961371 >> 172.21.15.5:0/4148961371 conn(0x55a423527540 msgr2=0x55a4235221a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 -- 172.21.15.5:0/4148961371 shutdown_connections 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.145+0000 7f11c19dc740 1 -- 172.21.15.5:0/4148961371 wait complete. 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.146+0000 7f11c19dc740 1 -- start start 2024-04-25T10:08:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.146+0000 7f11c19dc740 1 Processor -- start 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a42354b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a423556e80 0x55a4235d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a4235578a0 0x55a423551900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81fedc0 con 0x55a423546100 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b8062290 con 0x55a423556e80 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a42352a180 con 0x55a4235578a0 2024-04-25T10:08:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11c19dc740 10 client.0 _ll_register_callbacks cb 0x7f11a8053f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a42354b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a42354b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59720/0 (socket says 172.21.15.5:59720) 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.147+0000 7f11b7e00640 1 -- 172.21.15.5:0/900905123 learned_addr learned my addr 172.21.15.5:0/900905123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7e00640 1 -- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a4235578a0 msgr2=0x55a423551900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11bcc00640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a4235578a0 0x55a423551900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7400640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a423556e80 0x55a4235d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7e00640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a4235578a0 0x55a423551900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7e00640 1 -- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a423556e80 msgr2=0x55a4235d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7e00640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a423556e80 0x55a4235d6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7e00640 1 -- 172.21.15.5:0/900905123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4235dca20 con 0x55a423546100 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11b7e00640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a42354b800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f11a800ce90 tx=0x7f11a8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39b113c84a85a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11a4c00640 1 -- 172.21.15.5:0/900905123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8055f90 con 0x55a423546100 2024-04-25T10:08:05.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11a4c00640 1 -- 172.21.15.5:0/900905123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f11a8004d70 con 0x55a423546100 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.148+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11a8004bc0 con 0x55a423546100 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11a4c00640 1 -- 172.21.15.5:0/900905123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11a8054300 con 0x55a423546100 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 10 client.7194 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55a42357a010 con 0x55a423546100 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11a4c00640 1 -- 172.21.15.5:0/900905123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a8004bc0 con 0x55a423546100 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11a4c00640 1 -- 172.21.15.5:0/900905123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f11a8076b60 con 0x55a423546100 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 10 client.7194 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 10 client.7194 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 20 client.7194 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a4235e09a0 0x55a423551e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 4 client.7194 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:08:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.149+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x55a4234973e0 con 0x55a4235e09a0 2024-04-25T10:08:05.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.150+0000 7f11b7400640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a4235e09a0 0x55a423551e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.151+0000 7f11b7400640 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a4235e09a0 0x55a423551e40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=205c49d394753621 server_cookie=470438dab0518e1a in_seq=0 out_seq=0 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.151+0000 7f11a4c00640 10 client.7194 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11a4c00640 1 -- 172.21.15.5:0/900905123 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a4234973e0 con 0x55a4235e09a0 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11a4c00640 10 client.7194 handle_command_reply: tid=0 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 client.7194 shutdown 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 msgr2=0x55a42354b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a42354b800 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f11a800ce90 tx=0x7f11a8000ed0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 shutdown_connections 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55a4235e09a0 0x55a423551e40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a4235578a0 0x55a423551900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a423556e80 0x55a4235d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 --2- 172.21.15.5:0/900905123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423546100 0x55a42354b800 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 >> 172.21.15.5:0/900905123 conn(0x55a423527540 msgr2=0x55a423547140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11a4c00640 0 client.7194 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 shutdown_connections 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 1 -- 172.21.15.5:0/900905123 wait complete. 2024-04-25T10:08:05.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.153+0000 7f11c19dc740 20 client.7194 trim_cache size 0 max 16384 2024-04-25T10:08:05.165 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:08:05.165 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:08:05.165 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8a01ef40-d393-41ca-87e5-262f941ca6a5", 2024-04-25T10:08:05.165 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:08:05.165 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f119406b4d0 msgr2=0x7f119406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f119406b4d0 0x7f119406d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f11a0034000 tx=0x7f11a002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 msgr2=0x7f11b814d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 0x7f11b814d0d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f11a800c4d0 tx=0x7f11a800c990 comp rx=0 tx=0).stop 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 shutdown_connections 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f119406b4d0 0x7f119406d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b815a580 0x7f11b8139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11b8071890 0x7f11b8158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 --2- 172.21.15.5:0/3101934712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11b806d020 0x7f11b814d0d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 >> 172.21.15.5:0/3101934712 conn(0x7f11b8066f20 msgr2=0x7f11b814dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 shutdown_connections 2024-04-25T10:08:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.154+0000 7f11bd600640 1 -- 172.21.15.5:0/3101934712 wait complete. 2024-04-25T10:08:05.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:05.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:08:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.386+0000 7f870fe00640 1 Processor -- start 2024-04-25T10:08:05.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.386+0000 7f870fe00640 1 -- start start 2024-04-25T10:08:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.387+0000 7f870fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87101616f0 0x7f8710161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.387+0000 7f870fe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8710162030 con 0x7f87100b45c0 2024-04-25T10:08:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.387+0000 7f870fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871015ffa0 con 0x7f871015ec80 2024-04-25T10:08:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.387+0000 7f870fe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87100b4a30 con 0x7f87101616f0 2024-04-25T10:08:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.388+0000 7f870e000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f87100b45c0 0x7f8710160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50246/0 (socket says 172.21.15.5:50246) 2024-04-25T10:08:05.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.388+0000 7f870e000640 1 -- 172.21.15.5:0/1040691594 learned_addr learned my addr 172.21.15.5:0/1040691594 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:05.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.388+0000 7f870f400640 1 --2- 172.21.15.5:0/1040691594 >> v2:172.21.15.80:3300/0 conn(0x7f87101616f0 0x7f8710161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.388+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933633117 0 0) 0x7f8710162030 con 0x7f87100b45c0 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.388+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f4003520 con 0x7f87100b45c0 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2360598749 0 0) 0x7f86f4003520 con 0x7f87100b45c0 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 >> v2:172.21.15.80:3300/0 conn(0x7f87101616f0 msgr2=0x7f8710161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 --2- 172.21.15.5:0/1040691594 >> v2:172.21.15.80:3300/0 conn(0x7f87101616f0 0x7f8710161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 >> v1:172.21.15.80:6789/0 conn(0x7f871015ec80 legacy=0x7f87100d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710160240 con 0x7f87100b45c0 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f86fc0017f0 con 0x7f87100b45c0 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 >> v1:172.21.15.5:6789/0 conn(0x7f87100b45c0 legacy=0x7f8710160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86f4003a00 0x7f86f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f4004800 0x7f86f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f40051c0 0x7f86f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b4a30 con 0x7f86f4004800 2024-04-25T10:08:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f400c0c0 con 0x7f86f40051c0 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f400c2c0 con 0x7f86f4003a00 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.389+0000 7f870e000640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f4004800 0x7f86f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870f400640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f40051c0 0x7f86f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870e000640 1 -- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86f4003a00 msgr2=0x7f86f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870ea00640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86f4003a00 0x7f86f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870e000640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f86f4003a00 0x7f86f4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870e000640 1 -- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f40051c0 msgr2=0x7f86f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870e000640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f40051c0 0x7f86f400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870e000640 1 -- 172.21.15.5:0/1040691594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4003520 con 0x7f86f4004800 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870f400640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f40051c0 0x7f86f400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.390+0000 7f870f400640 1 -- 172.21.15.5:0/1040691594 reap_dead start 2024-04-25T10:08:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.391+0000 7f870e000640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f4004800 0x7f86f4004c80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c160 tx=0x7f86f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced30beb481e5937 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.391+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc007c20 con 0x7f86f4004800 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.391+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc007290 con 0x7f86f4004800 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.391+0000 7f870d600640 1 -- 172.21.15.5:0/1040691594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0057a0 con 0x7f86f4004800 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 -- 172.21.15.5:0/1040691594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f4004800 msgr2=0x7f86f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f4004800 0x7f86f4004c80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c160 tx=0x7f86f4005830 comp rx=0 tx=0).stop 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 -- 172.21.15.5:0/1040691594 shutdown_connections 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 --2- 172.21.15.5:0/1040691594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f86f4004800 0x7f86f4004c80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 -- 172.21.15.5:0/1040691594 >> 172.21.15.5:0/1040691594 conn(0x7f871008af80 msgr2=0x7f87100d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 -- 172.21.15.5:0/1040691594 shutdown_connections 2024-04-25T10:08:05.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.392+0000 7f870fe00640 1 -- 172.21.15.5:0/1040691594 wait complete. 2024-04-25T10:08:05.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.394+0000 7f870fe00640 1 Processor -- start 2024-04-25T10:08:05.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.394+0000 7f870fe00640 1 -- start start 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f4004800 0x7f87101f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 0x7f87100ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8710159b00 0x7f87100cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871015f060 con 0x7f87101f4eb0 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710160800 con 0x7f86f4004800 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b4c50 con 0x7f8710159b00 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 0x7f87100ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 0x7f87100ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59736/0 (socket says 172.21.15.5:59736) 2024-04-25T10:08:05.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870e000640 1 -- 172.21.15.5:0/1361460593 learned_addr learned my addr 172.21.15.5:0/1361460593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870ea00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f4004800 0x7f87101f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.395+0000 7f870f400640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8710159b00 0x7f87100cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870e000640 1 -- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8710159b00 msgr2=0x7f87100cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870e000640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8710159b00 0x7f87100cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870e000640 1 -- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f4004800 msgr2=0x7f87101f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870e000640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f4004800 0x7f87101f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870e000640 1 -- 172.21.15.5:0/1361460593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100ca770 con 0x7f87101f4eb0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870f400640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8710159b00 0x7f87100cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870e000640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 0x7f87100ccc70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c250 tx=0x7f8710160410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f8706a00640 1 -- 172.21.15.5:0/1361460593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc043070 con 0x7f87101f4eb0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f8706a00640 1 -- 172.21.15.5:0/1361460593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc038040 con 0x7f87101f4eb0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100ca9e0 con 0x7f87101f4eb0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f8706a00640 1 -- 172.21.15.5:0/1361460593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0059e0 con 0x7f87101f4eb0 2024-04-25T10:08:05.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.396+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87100cafa0 con 0x7f87101f4eb0 2024-04-25T10:08:05.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.399+0000 7f8706a00640 1 -- 172.21.15.5:0/1361460593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86fc032430 con 0x7f87101f4eb0 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.400+0000 7f8706a00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.400+0000 7f8718791740 1 Processor -- start 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.400+0000 7f8706a00640 1 -- 172.21.15.5:0/1361460593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f86fc0a9f20 con 0x7f87101f4eb0 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.400+0000 7f8718791740 1 -- start start 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f870ea00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f8718791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760eda6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760eda5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760edcfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f8718791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55760ed08c70 con 0x55760eda4d70 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55760ecb1cf0 con 0x55760ede1440 2024-04-25T10:08:05.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.401+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55760ed04d10 con 0x55760edcf830 2024-04-25T10:08:05.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760eda6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760eda6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59744/0 (socket says 172.21.15.5:59744) 2024-04-25T10:08:05.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 -- 172.21.15.5:0/1558298257 learned_addr learned my addr 172.21.15.5:0/1558298257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870e000640 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760eda5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870f400640 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760edcfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 -- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 msgr2=0x55760edcfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760edcfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 -- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 msgr2=0x55760eda5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760eda5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.402+0000 7f870ea00640 1 -- 172.21.15.5:0/1558298257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55760eae77d0 con 0x55760eda4d70 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.403+0000 7f870ea00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86f006b4d0 0x7f86f006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87101f13a0 tx=0x7f8700001a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.403+0000 7f870ea00640 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760eda6ed0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8700005490 tx=0x7f870001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19450bec230fd370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.403+0000 7f86fb400640 1 -- 172.21.15.5:0/1558298257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000f780 con 0x55760eda4d70 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.403+0000 7f86fb400640 1 -- 172.21.15.5:0/1558298257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000f920 con 0x55760eda4d70 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.403+0000 7f86fb400640 1 -- 172.21.15.5:0/1558298257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000fd10 con 0x55760eda4d70 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 -- 172.21.15.5:0/1558298257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 msgr2=0x55760eda6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760eda6ed0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8700005490 tx=0x7f870001a000 comp rx=0 tx=0).stop 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 -- 172.21.15.5:0/1558298257 shutdown_connections 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760edcfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760eda5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 --2- 172.21.15.5:0/1558298257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760eda6ed0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 -- 172.21.15.5:0/1558298257 >> 172.21.15.5:0/1558298257 conn(0x55760ed6ca30 msgr2=0x55760edea830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 -- 172.21.15.5:0/1558298257 shutdown_connections 2024-04-25T10:08:05.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.404+0000 7f8718791740 1 -- 172.21.15.5:0/1558298257 wait complete. 2024-04-25T10:08:05.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.405+0000 7f8718791740 1 -- start start 2024-04-25T10:08:05.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.405+0000 7f8718791740 1 Processor -- start 2024-04-25T10:08:05.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760edeb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ede5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ede5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55760eca73a0 con 0x55760eda4d70 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55760edcd380 con 0x55760ede1440 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55760edd1160 con 0x55760edcf830 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f8718791740 10 client.0 _ll_register_callbacks cb 0x55760ecc4b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f870ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760edeb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f870ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760edeb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59758/0 (socket says 172.21.15.5:59758) 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.406+0000 7f870ea00640 1 -- 172.21.15.5:0/659564861 learned_addr learned my addr 172.21.15.5:0/659564861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870e000640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ede5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870f400640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ede5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870ea00640 1 -- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 msgr2=0x55760ede5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870ea00640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ede5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870ea00640 1 -- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 msgr2=0x55760ede5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870ea00640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ede5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870ea00640 1 -- 172.21.15.5:0/659564861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55760ea1d3f0 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f870ea00640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760edeb0b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f87000052f0 tx=0x7f87000011e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da15c5a2ae70fa4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f86efe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f86efe00640 1 -- 172.21.15.5:0/659564861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700030060 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f86efe00640 1 -- 172.21.15.5:0/659564861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8700007cb0 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.407+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f870000f920 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f86efe00640 1 -- 172.21.15.5:0/659564861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8700036050 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 10 client.7206 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55760e9dd0f0 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f86efe00640 1 -- 172.21.15.5:0/659564861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000f920 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f86efe00640 1 -- 172.21.15.5:0/659564861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8700020680 con 0x55760eda4d70 2024-04-25T10:08:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 10 client.7206 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 10 client.7206 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 20 client.7206 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760edcc760 0x55760edd0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 4 client.7206 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.408+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55760ed161b0 con 0x55760edcc760 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.409+0000 7f870e000640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760edcc760 0x55760edd0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.410+0000 7f870e000640 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760edcc760 0x55760edd0180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748d93c8a38e1141 server_cookie=d410eb76f5655f6a in_seq=0 out_seq=0 2024-04-25T10:08:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.410+0000 7f86efe00640 10 client.7206 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.411+0000 7f86efe00640 1 -- 172.21.15.5:0/659564861 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55760ed161b0 con 0x55760edcc760 2024-04-25T10:08:05.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.411+0000 7f86efe00640 10 client.7206 handle_command_reply: tid=0 2024-04-25T10:08:05.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.411+0000 7f8718791740 1 client.7206 shutdown 2024-04-25T10:08:05.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.411+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 msgr2=0x55760edeb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760edeb0b0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f87000052f0 tx=0x7f87000011e0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 shutdown_connections 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760edcc760 0x55760edd0180 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ede5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ede5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 --2- 172.21.15.5:0/659564861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760edeb0b0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f86efe00640 0 client.7206 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 >> 172.21.15.5:0/659564861 conn(0x55760ed6ca30 msgr2=0x55760eded970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 shutdown_connections 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 1 -- 172.21.15.5:0/659564861 wait complete. 2024-04-25T10:08:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.412+0000 7f8718791740 20 client.7206 trim_cache size 0 max 16384 2024-04-25T10:08:05.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.429+0000 7f8718791740 1 Processor -- start 2024-04-25T10:08:05.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.429+0000 7f8718791740 1 -- start start 2024-04-25T10:08:05.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f8718791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760ededa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ee4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ee4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f8718791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8710200a30 con 0x55760eda4d70 2024-04-25T10:08:05.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b3da0 con 0x55760ede1440 2024-04-25T10:08:05.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b31a0 con 0x55760edcf830 2024-04-25T10:08:05.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f870ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760ededa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f870ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760ededa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59774/0 (socket says 172.21.15.5:59774) 2024-04-25T10:08:05.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f870ea00640 1 -- 172.21.15.5:0/3802754120 learned_addr learned my addr 172.21.15.5:0/3802754120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f870f400640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ee4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f870ea00640 1 -- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 msgr2=0x55760ee4e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.430+0000 7f870e000640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ee4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870ea00640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ee4e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870ea00640 1 -- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 msgr2=0x55760ee4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870ea00640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ee4ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870ea00640 1 -- 172.21.15.5:0/3802754120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55760eda4980 con 0x55760eda4d70 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870e000640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ee4e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870f400640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ee4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f870ea00640 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760ededa90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8700001f00 tx=0x7f8700024050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e1e17087ab8d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f86faa00640 1 -- 172.21.15.5:0/3802754120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700020680 con 0x55760eda4d70 2024-04-25T10:08:05.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f86faa00640 1 -- 172.21.15.5:0/3802754120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000f780 con 0x55760eda4d70 2024-04-25T10:08:05.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.431+0000 7f86faa00640 1 -- 172.21.15.5:0/3802754120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000f930 con 0x55760eda4d70 2024-04-25T10:08:05.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 -- 172.21.15.5:0/3802754120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 msgr2=0x55760ededa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760ededa90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8700001f00 tx=0x7f8700024050 comp rx=0 tx=0).stop 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 -- 172.21.15.5:0/3802754120 shutdown_connections 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760ede1440 0x55760ee4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760edcf830 0x55760ee4e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 --2- 172.21.15.5:0/3802754120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760eda4d70 0x55760ededa90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 -- 172.21.15.5:0/3802754120 >> 172.21.15.5:0/3802754120 conn(0x55760ed6ca30 msgr2=0x55760edcee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 -- 172.21.15.5:0/3802754120 shutdown_connections 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.432+0000 7f8718791740 1 -- 172.21.15.5:0/3802754120 wait complete. 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.433+0000 7f8718791740 1 -- start start 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.433+0000 7f8718791740 1 Processor -- start 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760eda4d70 0x55760edcaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760edcf830 0x55760ededa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 0x55760ededfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100b37a0 con 0x55760ede1440 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55760edd3a10 con 0x55760edcf830 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55760ee55b80 con 0x55760eda4d70 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f8718791740 10 client.0 _ll_register_callbacks cb 0x7f8700003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f870f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 0x55760ededfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f870f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 0x55760ededfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59780/0 (socket says 172.21.15.5:59780) 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.434+0000 7f870f400640 1 -- 172.21.15.5:0/3319728067 learned_addr learned my addr 172.21.15.5:0/3319728067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870ea00640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760eda4d70 0x55760edcaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870e000640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760edcf830 0x55760ededa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870f400640 1 -- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760eda4d70 msgr2=0x55760edcaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870f400640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760eda4d70 0x55760edcaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870f400640 1 -- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760edcf830 msgr2=0x55760ededa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870f400640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760edcf830 0x55760ededa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870f400640 1 -- 172.21.15.5:0/3319728067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55760edcafe0 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f870f400640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 0x55760ededfd0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f871008cf80 tx=0x7f8708004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735383ac1286d242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.435+0000 7f86efe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f86efe00640 1 -- 172.21.15.5:0/3319728067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870801b070 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f86efe00640 1 -- 172.21.15.5:0/3319728067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8708007c10 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f870000f780 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f86efe00640 1 -- 172.21.15.5:0/3319728067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8708026050 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 10 client.7212 fetch_fsmap learned FSMap version 52 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55760ee55980 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f86efe00640 1 -- 172.21.15.5:0/3319728067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000f780 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f86efe00640 1 -- 172.21.15.5:0/3319728067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8708029020 con 0x55760ede1440 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 10 client.7212 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 10 client.7212 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 20 client.7212 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760ee548e0 0x55760ee54cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 4 client.7212 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.436+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55760ed10500 con 0x55760ee548e0 2024-04-25T10:08:05.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.437+0000 7f870ea00640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760ee548e0 0x55760ee54cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.438+0000 7f870ea00640 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760ee548e0 0x55760ee54cf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25bf11e7afa08a4 server_cookie=d66feb4241c0752c in_seq=0 out_seq=0 2024-04-25T10:08:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.438+0000 7f86efe00640 10 client.7212 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.438+0000 7f86efe00640 1 -- 172.21.15.5:0/3319728067 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55760ed10500 con 0x55760ee548e0 2024-04-25T10:08:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f86efe00640 10 client.7212 handle_command_reply: tid=0 2024-04-25T10:08:05.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 client.7212 shutdown 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 msgr2=0x55760ededfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 0x55760ededfd0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f871008cf80 tx=0x7f8708004700 comp rx=0 tx=0).stop 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 shutdown_connections 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55760ee548e0 0x55760ee54cf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55760ede1440 0x55760ededfd0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55760edcf830 0x55760ededa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 --2- 172.21.15.5:0/3319728067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55760eda4d70 0x55760edcaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 >> 172.21.15.5:0/3319728067 conn(0x55760ed6ca30 msgr2=0x55760edca100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f86efe00640 0 client.7212 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 shutdown_connections 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 1 -- 172.21.15.5:0/3319728067 wait complete. 2024-04-25T10:08:05.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.439+0000 7f8718791740 20 client.7212 trim_cache size 0 max 16384 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: "8a01ef40-d393-41ca-87e5-262f941ca6a5": { 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/subdir", 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8a01ef40-d393-41ca-87e5-262f941ca6a5", 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:05.449 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:08:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86f006b4d0 msgr2=0x7f86f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86f006b4d0 0x7f86f006d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87101f13a0 tx=0x7f8700001a00 comp rx=0 tx=0).stop 2024-04-25T10:08:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 msgr2=0x7f87100ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 0x7f87100ccc70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00c250 tx=0x7f8710160410 comp rx=0 tx=0).stop 2024-04-25T10:08:05.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 shutdown_connections 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8710159b00 0x7f87100cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101f4eb0 0x7f87100ccc70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f86f006b4d0 0x7f86f006d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 --2- 172.21.15.5:0/1361460593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86f4004800 0x7f87101f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 >> 172.21.15.5:0/1361460593 conn(0x7f871008af80 msgr2=0x7f871015e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 shutdown_connections 2024-04-25T10:08:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:05.440+0000 7f870fe00640 1 -- 172.21.15.5:0/1361460593 wait complete. 2024-04-25T10:08:05.463 INFO:tasks.cephfs.filesystem:scrub status for tag:8a01ef40-d393-41ca-87e5-262f941ca6a5 - {'path': '/subdir', 'tag': '8a01ef40-d393-41ca-87e5-262f941ca6a5', 'options': 'recursive,repair'} 2024-04-25T10:08:24.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:24.934 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:35.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:35.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:08:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.685+0000 7fca59800640 1 Processor -- start 2024-04-25T10:08:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.685+0000 7fca59800640 1 -- start start 2024-04-25T10:08:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.687+0000 7fca59800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca54161d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.687+0000 7fca59800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca540b33f0 con 0x7fca540b37d0 2024-04-25T10:08:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.687+0000 7fca59800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca540b27e0 con 0x7fca54071000 2024-04-25T10:08:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.687+0000 7fca59800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca540b2ac0 con 0x7fca541619b0 2024-04-25T10:08:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.688+0000 7fca53400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fca540b37d0 0x7fca54162ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52236/0 (socket says 172.21.15.5:52236) 2024-04-25T10:08:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.688+0000 7fca53400640 1 -- 172.21.15.5:0/994532878 learned_addr learned my addr 172.21.15.5:0/994532878 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.688+0000 7fca53e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca54161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.688+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998729903 0 0) 0x7fca540b33f0 con 0x7fca540b37d0 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca38003520 con 0x7fca540b37d0 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936163240 0 0) 0x7fca540b27e0 con 0x7fca54071000 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca540b33f0 con 0x7fca54071000 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555455029 0 0) 0x7fca38003520 con 0x7fca540b37d0 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 msgr2=0x7fca54161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca54161d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 >> v1:172.21.15.80:6789/0 conn(0x7fca54071000 legacy=0x7fca5406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540b2da0 con 0x7fca540b37d0 2024-04-25T10:08:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fca3c001670 con 0x7fca540b37d0 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 >> v1:172.21.15.5:6789/0 conn(0x7fca540b37d0 legacy=0x7fca54162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca38003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca38004800 0x7fca38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca540b2ac0 con 0x7fca38003b70 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3800ba70 con 0x7fca38004800 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.689+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3800bc40 con 0x7fca541619b0 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca38003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53400640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca38004800 0x7fca38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca58e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53e00640 1 -- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 msgr2=0x7fca380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53e00640 1 -- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca38004800 msgr2=0x7fca38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca38004800 0x7fca38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca53e00640 1 -- 172.21.15.5:0/994532878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38003520 con 0x7fca38003b70 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.690+0000 7fca58e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541619b0 0x7fca380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.691+0000 7fca53e00640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca38003f70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ac0 tx=0x7fca38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca5f9c1f6cc4d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.692+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44036070 con 0x7fca38003b70 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.692+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44031400 con 0x7fca38003b70 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.692+0000 7fca52a00640 1 -- 172.21.15.5:0/994532878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44009690 con 0x7fca38003b70 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.692+0000 7fca59800640 1 -- 172.21.15.5:0/994532878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 msgr2=0x7fca38003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.692+0000 7fca59800640 1 --2- 172.21.15.5:0/994532878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca38003f70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ac0 tx=0x7fca38005830 comp rx=0 tx=0).stop 2024-04-25T10:08:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.692+0000 7fca58e00640 1 -- 172.21.15.5:0/994532878 reap_dead start 2024-04-25T10:08:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.693+0000 7fca59800640 1 -- 172.21.15.5:0/994532878 shutdown_connections 2024-04-25T10:08:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.693+0000 7fca59800640 1 -- 172.21.15.5:0/994532878 >> 172.21.15.5:0/994532878 conn(0x7fca54066f20 msgr2=0x7fca5406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.693+0000 7fca59800640 1 -- 172.21.15.5:0/994532878 shutdown_connections 2024-04-25T10:08:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.693+0000 7fca59800640 1 -- 172.21.15.5:0/994532878 wait complete. 2024-04-25T10:08:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.694+0000 7fca59800640 1 Processor -- start 2024-04-25T10:08:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.695+0000 7fca59800640 1 -- start start 2024-04-25T10:08:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca59800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca541ed570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca59800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541ec040 0x7fca541564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca59800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca54158e70 0x7fca54164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca59800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca540b2420 con 0x7fca38003b70 2024-04-25T10:08:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca59800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5406e730 con 0x7fca54158e70 2024-04-25T10:08:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca59800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca540b25a0 con 0x7fca541ec040 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca541ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca541ed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49472/0 (socket says 172.21.15.5:49472) 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53e00640 1 -- 172.21.15.5:0/1931819879 learned_addr learned my addr 172.21.15.5:0/1931819879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53400640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541ec040 0x7fca541564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca58e00640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca54158e70 0x7fca54164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53e00640 1 -- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541ec040 msgr2=0x7fca541564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53e00640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541ec040 0x7fca541564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.696+0000 7fca53e00640 1 -- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca54158e70 msgr2=0x7fca54164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca53e00640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca54158e70 0x7fca54164500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca53e00640 1 -- 172.21.15.5:0/1931819879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca541ec4d0 con 0x7fca38003b70 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca53e00640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca541ed570 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ac0 tx=0x7fca44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca4be00640 1 -- 172.21.15.5:0/1931819879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44041020 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca4be00640 1 -- 172.21.15.5:0/1931819879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4403c470 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca4be00640 1 -- 172.21.15.5:0/1931819879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44036040 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca541ec7b0 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.697+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca541ecd70 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.700+0000 7fca4be00640 1 -- 172.21.15.5:0/1931819879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca44044020 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.701+0000 7fca4be00640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca3006b4d0 0x7fca3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.701+0000 7fca4be00640 1 -- 172.21.15.5:0/1931819879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fca440aaca0 con 0x7fca38003b70 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.701+0000 7fca5dbe4740 1 Processor -- start 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.702+0000 7fca53400640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca3006b4d0 0x7fca3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.702+0000 7fca5dbe4740 1 -- start start 2024-04-25T10:08:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f19d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08f100c70 con 0x55b08f1c7830 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08f0a9cf0 con 0x55b08f19cd70 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b08f0fcd10 con 0x55b08f1d9440 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca53400640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca3006b4d0 0x7fca3006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca54161db0 tx=0x7fca3c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca58e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca58e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49482/0 (socket says 172.21.15.5:49482) 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca58e00640 1 -- 172.21.15.5:0/99302002 learned_addr learned my addr 172.21.15.5:0/99302002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.703+0000 7fca53400640 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f19d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca53e00640 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca58e00640 1 -- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 msgr2=0x55b08f19d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca58e00640 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f19d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca58e00640 1 -- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 msgr2=0x55b08f19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca58e00640 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca58e00640 1 -- 172.21.15.5:0/99302002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08eedf7d0 con 0x55b08f1c7830 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca58e00640 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1c7cb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fca540cc960 tx=0x7fca4c002010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb11fb0246dec5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca48c00640 1 -- 172.21.15.5:0/99302002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c019070 con 0x55b08f1c7830 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca48c00640 1 -- 172.21.15.5:0/99302002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c003d80 con 0x55b08f1c7830 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.704+0000 7fca48c00640 1 -- 172.21.15.5:0/99302002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c006d90 con 0x55b08f1c7830 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 -- 172.21.15.5:0/99302002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 msgr2=0x55b08f1c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1c7cb0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fca540cc960 tx=0x7fca4c002010 comp rx=0 tx=0).stop 2024-04-25T10:08:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 -- 172.21.15.5:0/99302002 shutdown_connections 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1c7cb0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f19d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/99302002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 -- 172.21.15.5:0/99302002 >> 172.21.15.5:0/99302002 conn(0x55b08f164a30 msgr2=0x55b08f1e2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 -- 172.21.15.5:0/99302002 shutdown_connections 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.705+0000 7fca5dbe4740 1 -- 172.21.15.5:0/99302002 wait complete. 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.706+0000 7fca5dbe4740 1 -- start start 2024-04-25T10:08:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.706+0000 7fca5dbe4740 1 Processor -- start 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.707+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.707+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.707+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1b7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08f09f3a0 con 0x55b08f1c7830 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08f1c5380 con 0x55b08f19cd70 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b08f1ca7e0 con 0x55b08f1d9440 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca5dbe4740 10 client.0 _ll_register_callbacks cb 0x55b08f0bcb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1e3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49488/0 (socket says 172.21.15.5:49488) 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 -- 172.21.15.5:0/3612475310 learned_addr learned my addr 172.21.15.5:0/3612475310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53e00640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca58e00640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 -- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 msgr2=0x55b08f1b7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1b7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 -- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 msgr2=0x55b08f1e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.708+0000 7fca53400640 1 -- 172.21.15.5:0/3612475310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b08ee153f0 con 0x55b08f1c7830 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca53400640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1e3960 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fca54152c70 tx=0x7fca540b29b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506ae716f328eabc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca41600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca41600640 1 -- 172.21.15.5:0/3612475310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c028060 con 0x55b08f1c7830 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca41600640 1 -- 172.21.15.5:0/3612475310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fca3c023c90 con 0x55b08f1c7830 2024-04-25T10:08:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca4c003d80 con 0x55b08f1c7830 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca41600640 1 -- 172.21.15.5:0/3612475310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca3c004e30 con 0x55b08f1c7830 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca5dbe4740 10 client.7224 fetch_fsmap learned FSMap version 53 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.709+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55b08edd50f0 con 0x55b08f1c7830 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca41600640 1 -- 172.21.15.5:0/3612475310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c003d80 con 0x55b08f1c7830 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca41600640 1 -- 172.21.15.5:0/3612475310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca3c014600 con 0x55b08f1c7830 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca5dbe4740 10 client.7224 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca5dbe4740 10 client.7224 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca5dbe4740 20 client.7224 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f1db730 0x55b08f1c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca5dbe4740 4 client.7224 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b08f10e1b0 con 0x55b08f1db730 2024-04-25T10:08:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.710+0000 7fca53e00640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f1db730 0x55b08f1c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.711+0000 7fca41600640 10 client.7224 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.711+0000 7fca53e00640 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f1db730 0x55b08f1c7c30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5ddd1c3b47de7d server_cookie=9ef3d60eb907fbed in_seq=0 out_seq=0 2024-04-25T10:08:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.713+0000 7fca41600640 1 -- 172.21.15.5:0/3612475310 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b08f10e1b0 con 0x55b08f1db730 2024-04-25T10:08:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.713+0000 7fca41600640 10 client.7224 handle_command_reply: tid=0 2024-04-25T10:08:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.713+0000 7fca5dbe4740 1 client.7224 shutdown 2024-04-25T10:08:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 msgr2=0x55b08f1e3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1e3960 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fca54152c70 tx=0x7fca540b29b0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 shutdown_connections 2024-04-25T10:08:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f1db730 0x55b08f1c7c30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1e3960 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3612475310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca41600640 0 client.7224 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:35.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 >> 172.21.15.5:0/3612475310 conn(0x55b08f164a30 msgr2=0x55b08f1c28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 shutdown_connections 2024-04-25T10:08:35.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3612475310 wait complete. 2024-04-25T10:08:35.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.714+0000 7fca5dbe4740 20 client.7224 trim_cache size 0 max 16384 2024-04-25T10:08:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.731+0000 7fca5dbe4740 1 Processor -- start 2024-04-25T10:08:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.731+0000 7fca5dbe4740 1 -- start start 2024-04-25T10:08:35.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca542050b0 con 0x55b08f1c7830 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca541fbf90 con 0x55b08f19cd70 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca541f7e50 con 0x55b08f1d9440 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca53400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca53400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49502/0 (socket says 172.21.15.5:49502) 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.732+0000 7fca53400640 1 -- 172.21.15.5:0/525951570 learned_addr learned my addr 172.21.15.5:0/525951570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53e00640 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca58e00640 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53400640 1 -- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 msgr2=0x55b08f1e22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53400640 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1e22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53400640 1 -- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 msgr2=0x55b08f1cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53400640 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53400640 1 -- 172.21.15.5:0/525951570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08f1e5ec0 con 0x55b08f1c7830 2024-04-25T10:08:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca58e00640 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca53400640 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1cb310 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fca3c001be0 tx=0x7fca3c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2225e49ac2efca7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca43e00640 1 -- 172.21.15.5:0/525951570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c03a070 con 0x55b08f1c7830 2024-04-25T10:08:35.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca43e00640 1 -- 172.21.15.5:0/525951570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c005420 con 0x55b08f1c7830 2024-04-25T10:08:35.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.733+0000 7fca43e00640 1 -- 172.21.15.5:0/525951570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c014400 con 0x55b08f1c7830 2024-04-25T10:08:35.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 -- 172.21.15.5:0/525951570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 msgr2=0x55b08f1cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1cb310 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fca3c001be0 tx=0x7fca3c021000 comp rx=0 tx=0).stop 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 -- 172.21.15.5:0/525951570 shutdown_connections 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1d9440 0x55b08f1e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f1c7830 0x55b08f1cb310 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/525951570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f19cd70 0x55b08f1cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 -- 172.21.15.5:0/525951570 >> 172.21.15.5:0/525951570 conn(0x55b08f164a30 msgr2=0x55b08f1c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 -- 172.21.15.5:0/525951570 shutdown_connections 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.734+0000 7fca5dbe4740 1 -- 172.21.15.5:0/525951570 wait complete. 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.735+0000 7fca5dbe4740 1 -- start start 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.735+0000 7fca5dbe4740 1 Processor -- start 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 0x55b08f233140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1c7830 0x55b08f1cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f1d9440 0x55b08f1cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08f0fce90 con 0x55b08f19cd70 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca541fb990 con 0x55b08f1d9440 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b08f193c40 con 0x55b08f1c7830 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca5dbe4740 10 client.0 _ll_register_callbacks cb 0x7fca3c001d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 0x55b08f233140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 0x55b08f233140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49510/0 (socket says 172.21.15.5:49510) 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.736+0000 7fca53e00640 1 -- 172.21.15.5:0/3790071808 learned_addr learned my addr 172.21.15.5:0/3790071808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53400640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1c7830 0x55b08f1cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca58e00640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f1d9440 0x55b08f1cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53e00640 1 -- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1c7830 msgr2=0x55b08f1cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53e00640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1c7830 0x55b08f1cad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53e00640 1 -- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f1d9440 msgr2=0x55b08f1cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53e00640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f1d9440 0x55b08f1cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53e00640 1 -- 172.21.15.5:0/3790071808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b08f1da960 con 0x55b08f19cd70 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53400640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1c7830 0x55b08f1cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca53e00640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 0x55b08f233140 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fca44079250 tx=0x7fca4403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78837de51f7098b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca41600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca41600640 1 -- 172.21.15.5:0/3790071808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44087060 con 0x55b08f19cd70 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca41600640 1 -- 172.21.15.5:0/3790071808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fca4407d120 con 0x55b08f19cd70 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca3c005420 con 0x55b08f19cd70 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca41600640 1 -- 172.21.15.5:0/3790071808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca4407d410 con 0x55b08f19cd70 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.737+0000 7fca5dbe4740 10 client.7230 fetch_fsmap learned FSMap version 53 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55b08f1da440 con 0x55b08f19cd70 2024-04-25T10:08:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca41600640 1 -- 172.21.15.5:0/3790071808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c005420 con 0x55b08f19cd70 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca41600640 1 -- 172.21.15.5:0/3790071808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca4408f040 con 0x55b08f19cd70 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 10 client.7230 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 10 client.7230 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 20 client.7230 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f24dde0 0x55b08f1d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 4 client.7230 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b08f1e6430 con 0x55b08f24dde0 2024-04-25T10:08:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.738+0000 7fca53400640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f24dde0 0x55b08f1d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca53400640 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f24dde0 0x55b08f1d81c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cad2d8a285ed1ab server_cookie=7fd95b7aaea62df1 in_seq=0 out_seq=0 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca41600640 10 client.7230 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca41600640 1 -- 172.21.15.5:0/3790071808 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fca3c03b050 con 0x55b08f24dde0 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca41600640 10 client.7230 handle_command_reply: tid=0 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 client.7230 shutdown 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 msgr2=0x55b08f233140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 0x55b08f233140 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fca44079250 tx=0x7fca4403d000 comp rx=0 tx=0).stop 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 shutdown_connections 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x55b08f24dde0 0x55b08f1d81c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b08f1d9440 0x55b08f1cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b08f1c7830 0x55b08f1cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 --2- 172.21.15.5:0/3790071808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b08f19cd70 0x55b08f233140 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca41600640 0 client.7230 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 >> 172.21.15.5:0/3790071808 conn(0x55b08f164a30 msgr2=0x55b08f1c4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.740+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 shutdown_connections 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca5dbe4740 1 -- 172.21.15.5:0/3790071808 wait complete. 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca5dbe4740 20 client.7230 trim_cache size 0 max 16384 2024-04-25T10:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca3006b4d0 msgr2=0x7fca3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca3006b4d0 0x7fca3006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca54161db0 tx=0x7fca3c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 msgr2=0x7fca541ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca541ed570 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ac0 tx=0x7fca44004730 comp rx=0 tx=0).stop 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 shutdown_connections 2024-04-25T10:08:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca3006b4d0 0x7fca3006d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca54158e70 0x7fca54164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca541ec040 0x7fca541564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 --2- 172.21.15.5:0/1931819879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca38003b70 0x7fca541ed570 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.741+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 >> 172.21.15.5:0/1931819879 conn(0x7fca54066f20 msgr2=0x7fca5406d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.742+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 shutdown_connections 2024-04-25T10:08:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.742+0000 7fca59800640 1 -- 172.21.15.5:0/1931819879 wait complete. 2024-04-25T10:08:35.766 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:08:35.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:35.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.969+0000 7f1b1b200640 1 Processor -- start 2024-04-25T10:08:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.969+0000 7f1b1b200640 1 -- start start 2024-04-25T10:08:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.970+0000 7f1b1b200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1b1c154a10 0x7f1b1c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b1b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b1c0cdca0 0x7f1b1c0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b1b200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c0b4890 con 0x7f1b1c070f90 2024-04-25T10:08:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b1b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cd200 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b1b200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c06e6f0 con 0x7f1b1c154a10 2024-04-25T10:08:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b13e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b1c0cdca0 0x7f1b1c0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b13e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b1c0cdca0 0x7f1b1c0b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49524/0 (socket says 172.21.15.5:49524) 2024-04-25T10:08:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b13e00640 1 -- 172.21.15.5:0/2950479940 learned_addr learned my addr 172.21.15.5:0/2950479940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b19e00640 1 --2- 172.21.15.5:0/2950479940 >> v2:172.21.15.80:3300/0 conn(0x7f1b1c154a10 0x7f1b1c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b13e00640 1 -- 172.21.15.5:0/2950479940 >> v2:172.21.15.80:3300/0 conn(0x7f1b1c154a10 msgr2=0x7f1b1c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b13e00640 1 --2- 172.21.15.5:0/2950479940 >> v2:172.21.15.80:3300/0 conn(0x7f1b1c154a10 0x7f1b1c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.971+0000 7f1b13e00640 1 -- 172.21.15.5:0/2950479940 >> v1:172.21.15.80:6789/0 conn(0x7f1b1c070f90 legacy=0x7f1b1c0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.972+0000 7f1b13e00640 1 -- 172.21.15.5:0/2950479940 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c06e9d0 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.972+0000 7f1b19e00640 1 --2- 172.21.15.5:0/2950479940 >> v2:172.21.15.80:3300/0 conn(0x7f1b1c154a10 0x7f1b1c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.973+0000 7f1b13e00640 1 --2- 172.21.15.5:0/2950479940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c0b4350 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0ccb70 tx=0x7f1b0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb31e2d7e3ad16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.973+0000 7f1b19400640 1 -- 172.21.15.5:0/2950479940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04037070 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.973+0000 7f1b19400640 1 -- 172.21.15.5:0/2950479940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04032440 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.973+0000 7f1b19400640 1 -- 172.21.15.5:0/2950479940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04032810 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 -- 172.21.15.5:0/2950479940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 msgr2=0x7f1b1c0b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 --2- 172.21.15.5:0/2950479940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c0b4350 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0ccb70 tx=0x7f1b0402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 -- 172.21.15.5:0/2950479940 shutdown_connections 2024-04-25T10:08:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 --2- 172.21.15.5:0/2950479940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c0b4350 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 --2- 172.21.15.5:0/2950479940 >> v2:172.21.15.80:3300/0 conn(0x7f1b1c154a10 0x7f1b1c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 -- 172.21.15.5:0/2950479940 >> 172.21.15.5:0/2950479940 conn(0x7f1b1c066f20 msgr2=0x7f1b1c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 -- 172.21.15.5:0/2950479940 shutdown_connections 2024-04-25T10:08:35.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.974+0000 7f1b1b200640 1 -- 172.21.15.5:0/2950479940 wait complete. 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.976+0000 7f1b1b200640 1 Processor -- start 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.976+0000 7f1b1b200640 1 -- start start 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b1b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b1c070f90 0x7f1b1c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b1b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b1b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b1c154a10 0x7f1b1c15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b1b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cd780 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b1b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cd4a0 con 0x7f1b1c070f90 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b1b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c06cff0 con 0x7f1b1c154a10 2024-04-25T10:08:35.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49540/0 (socket says 172.21.15.5:49540) 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.977+0000 7f1b13e00640 1 -- 172.21.15.5:0/343705557 learned_addr learned my addr 172.21.15.5:0/343705557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b19e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b1c070f90 0x7f1b1c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b13e00640 1 -- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b1c154a10 msgr2=0x7f1b1c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b1a800640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b1c154a10 0x7f1b1c15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b13e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b1c154a10 0x7f1b1c15c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b13e00640 1 -- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b1c070f90 msgr2=0x7f1b1c141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b13e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b1c070f90 0x7f1b1c141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b13e00640 1 -- 172.21.15.5:0/343705557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c142800 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b19e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b1c070f90 0x7f1b1c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b1a800640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b1c154a10 0x7f1b1c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b13e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c15bf70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000c00 tx=0x7f1b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04041020 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04030970 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04037040 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.978+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c157c10 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.979+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c142d80 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.980+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c068f60 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.981+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b04044020 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.982+0000 7f1b12000640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1af806b520 0x7f1af806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.982+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1b040aada0 con 0x7f1b1c0cdca0 2024-04-25T10:08:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.982+0000 7f1b19e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1af806b520 0x7f1af806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.984+0000 7f1b19e00640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1af806b520 0x7f1af806d9e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c006fd0 tx=0x7f1b0c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:35.986+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b0404b030 con 0x7f1b1c0cdca0 2024-04-25T10:08:36.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.228+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b1c0cce40 con 0x7f1b1c0cdca0 2024-04-25T10:08:36.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.229+0000 7f1b12000640 1 -- 172.21.15.5:0/343705557 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f1b04079ab0 con 0x7f1b1c0cdca0 2024-04-25T10:08:36.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:36.235 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:08:09.396742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:36.236 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 53 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1af806b520 msgr2=0x7f1af806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1af806b520 0x7f1af806d9e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c006fd0 tx=0x7f1b0c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 msgr2=0x7f1b1c15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c15bf70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000c00 tx=0x7f1b040046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 shutdown_connections 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b1c154a10 0x7f1b1c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b1c0cdca0 0x7f1b1c15bf70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1af806b520 0x7f1af806d9e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 --2- 172.21.15.5:0/343705557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b1c070f90 0x7f1b1c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 >> 172.21.15.5:0/343705557 conn(0x7f1b1c066f20 msgr2=0x7f1b1c071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.232+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 shutdown_connections 2024-04-25T10:08:36.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.233+0000 7f1b1b200640 1 -- 172.21.15.5:0/343705557 wait complete. 2024-04-25T10:08:36.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-25T10:08:36.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.457+0000 7f75fee00640 1 Processor -- start 2024-04-25T10:08:36.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.457+0000 7f75fee00640 1 -- start start 2024-04-25T10:08:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.459+0000 7f75fee00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f75f80b48c0 0x7f75f815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.459+0000 7f75fee00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f81606e0 con 0x7f75f8161d90 2024-04-25T10:08:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.459+0000 7f75fee00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f81609c0 con 0x7f75f80d3f90 2024-04-25T10:08:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.459+0000 7f75fee00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f8160ca0 con 0x7f75f80b48c0 2024-04-25T10:08:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.460+0000 7f75fe400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75f8161d90 0x7f75f80b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52248/0 (socket says 172.21.15.5:52248) 2024-04-25T10:08:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.460+0000 7f75fe400640 1 -- 172.21.15.5:0/3817330534 learned_addr learned my addr 172.21.15.5:0/3817330534 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.460+0000 7f75fda00640 1 --2- 172.21.15.5:0/3817330534 >> v2:172.21.15.80:3300/0 conn(0x7f75f80b48c0 0x7f75f815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.460+0000 7f75f7e00640 1 -- 172.21.15.5:0/3817330534 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087027963 0 0) 0x7f75f81606e0 con 0x7f75f8161d90 2024-04-25T10:08:36.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.460+0000 7f75f7e00640 1 -- 172.21.15.5:0/3817330534 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e0003520 con 0x7f75f8161d90 2024-04-25T10:08:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.461+0000 7f75fda00640 1 -- 172.21.15.5:0/3817330534 >> v1:172.21.15.80:6789/0 conn(0x7f75f80d3f90 legacy=0x7f75f8161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.461+0000 7f75f7e00640 1 -- 172.21.15.5:0/3817330534 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3944732680 0 0) 0x7f75e0003520 con 0x7f75f8161d90 2024-04-25T10:08:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.461+0000 7f75fda00640 1 -- 172.21.15.5:0/3817330534 >> v1:172.21.15.5:6789/0 conn(0x7f75f8161d90 legacy=0x7f75f80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.461+0000 7f75fda00640 1 -- 172.21.15.5:0/3817330534 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8160f80 con 0x7f75f80b48c0 2024-04-25T10:08:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.462+0000 7f75fda00640 1 --2- 172.21.15.5:0/3817330534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80b48c0 0x7f75f815fe60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75e80097a0 tx=0x7f75e802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9105140e84bd97fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.463+0000 7f75f7e00640 1 -- 172.21.15.5:0/3817330534 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8037070 con 0x7f75f80b48c0 2024-04-25T10:08:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.463+0000 7f75f7e00640 1 -- 172.21.15.5:0/3817330534 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e802abc0 con 0x7f75f80b48c0 2024-04-25T10:08:36.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.463+0000 7f75f7e00640 1 -- 172.21.15.5:0/3817330534 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f00019f0 con 0x7f75f80b48c0 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 -- 172.21.15.5:0/3817330534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80b48c0 msgr2=0x7f75f815fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 --2- 172.21.15.5:0/3817330534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80b48c0 0x7f75f815fe60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f75e80097a0 tx=0x7f75e802a130 comp rx=0 tx=0).stop 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 -- 172.21.15.5:0/3817330534 shutdown_connections 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 --2- 172.21.15.5:0/3817330534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80b48c0 0x7f75f815fe60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 -- 172.21.15.5:0/3817330534 >> 172.21.15.5:0/3817330534 conn(0x7f75f808af80 msgr2=0x7f75f80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 -- 172.21.15.5:0/3817330534 shutdown_connections 2024-04-25T10:08:36.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.464+0000 7f75fee00640 1 -- 172.21.15.5:0/3817330534 wait complete. 2024-04-25T10:08:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.466+0000 7f75fee00640 1 Processor -- start 2024-04-25T10:08:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.466+0000 7f75fee00640 1 -- start start 2024-04-25T10:08:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fee00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f80b48c0 0x7f75f8153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80d3f90 0x7f75f80b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 0x7f75f80b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f81603a0 con 0x7f75f8161d90 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b4ca0 con 0x7f75f80d3f90 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fee00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b3280 con 0x7f75f80b48c0 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fe400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 0x7f75f80b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fe400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 0x7f75f80b7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49546/0 (socket says 172.21.15.5:49546) 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.467+0000 7f75fe400640 1 -- 172.21.15.5:0/3343197392 learned_addr learned my addr 172.21.15.5:0/3343197392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fda00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f80b48c0 0x7f75f8153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fd000640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80d3f90 0x7f75f80b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fe400640 1 -- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f80b48c0 msgr2=0x7f75f8153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fe400640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f80b48c0 0x7f75f8153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fe400640 1 -- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80d3f90 msgr2=0x7f75f80b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fe400640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80d3f90 0x7f75f80b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fe400640 1 -- 172.21.15.5:0/3343197392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80b9100 con 0x7f75f8161d90 2024-04-25T10:08:36.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75fe400640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 0x7f75f80b7de0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f75f000cbc0 tx=0x7f75f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75f5600640 1 -- 172.21.15.5:0/3343197392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0017070 con 0x7f75f8161d90 2024-04-25T10:08:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.468+0000 7f75f5600640 1 -- 172.21.15.5:0/3343197392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f000ad40 con 0x7f75f8161d90 2024-04-25T10:08:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.469+0000 7f75f5600640 1 -- 172.21.15.5:0/3343197392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0012620 con 0x7f75f8161d90 2024-04-25T10:08:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.469+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f8153eb0 con 0x7f75f8161d90 2024-04-25T10:08:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.469+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f80c7500 con 0x7f75f8161d90 2024-04-25T10:08:36.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.471+0000 7f75f5600640 1 -- 172.21.15.5:0/3343197392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75f001a070 con 0x7f75f8161d90 2024-04-25T10:08:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.472+0000 7f75f5600640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75dc06b500 0x7f75dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.473+0000 7f75f5600640 1 -- 172.21.15.5:0/3343197392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f75f008a260 con 0x7f75f8161d90 2024-04-25T10:08:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.473+0000 7f760326c740 1 Processor -- start 2024-04-25T10:08:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.473+0000 7f760326c740 1 -- start start 2024-04-25T10:08:36.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.473+0000 7f75fda00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75dc06b500 0x7f75dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49c5d60 0x563cb49c6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49e3600 0x563cb49e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f760326c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 0x563cb49e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f760326c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb48f5ae0 con 0x563cb499ba40 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb489e7c0 con 0x563cb49e3600 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb48f1b80 con 0x563cb49c5d60 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.474+0000 7f75fe400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 0x563cb49e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 0x563cb49e1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49560/0 (socket says 172.21.15.5:49560) 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 -- 172.21.15.5:0/1471718324 learned_addr learned my addr 172.21.15.5:0/1471718324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fd000640 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49e3600 0x563cb49e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fda00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75dc06b500 0x7f75dc06d9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f75e8034000 tx=0x7f75e802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 -- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49c5d60 msgr2=0x563cb49c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49c5d60 0x563cb49c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 -- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49e3600 msgr2=0x563cb49e3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49e3600 0x563cb49e3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 -- 172.21.15.5:0/1471718324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb46d67c0 con 0x563cb499ba40 2024-04-25T10:08:36.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.475+0000 7f75fe400640 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 0x563cb49e1cc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f75f000afd0 tx=0x7f75f00588b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3702618e205c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f75e6000640 1 -- 172.21.15.5:0/1471718324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0056af0 con 0x563cb499ba40 2024-04-25T10:08:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f75e6000640 1 -- 172.21.15.5:0/1471718324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0054e60 con 0x563cb499ba40 2024-04-25T10:08:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f75e6000640 1 -- 172.21.15.5:0/1471718324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0060a70 con 0x563cb499ba40 2024-04-25T10:08:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 -- 172.21.15.5:0/1471718324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 msgr2=0x563cb49e1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 0x563cb49e1cc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f75f000afd0 tx=0x7f75f00588b0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 -- 172.21.15.5:0/1471718324 shutdown_connections 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb499ba40 0x563cb49e1cc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49e3600 0x563cb49e3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 --2- 172.21.15.5:0/1471718324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49c5d60 0x563cb49c6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.476+0000 7f760326c740 1 -- 172.21.15.5:0/1471718324 >> 172.21.15.5:0/1471718324 conn(0x563cb4998370 msgr2=0x563cb49e0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.477+0000 7f760326c740 1 -- 172.21.15.5:0/1471718324 shutdown_connections 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.477+0000 7f760326c740 1 -- 172.21.15.5:0/1471718324 wait complete. 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.477+0000 7f760326c740 1 -- start start 2024-04-25T10:08:36.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.478+0000 7f760326c740 1 Processor -- start 2024-04-25T10:08:36.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.478+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.478+0000 7f760326c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 0x563cb49d7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.478+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49d9760 0x563cb49f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f760326c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb499c7e0 con 0x563cb49e3600 2024-04-25T10:08:36.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb4893d40 con 0x563cb49c5d60 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb49962a0 con 0x563cb49d9760 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f760326c740 10 client.0 _ll_register_callbacks cb 0x563cb4609ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 0x563cb49d7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 0x563cb49d7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49564/0 (socket says 172.21.15.5:49564) 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 -- 172.21.15.5:0/2307891577 learned_addr learned my addr 172.21.15.5:0/2307891577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 -- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49d9760 msgr2=0x563cb49f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49d9760 0x563cb49f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fda00640 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 -- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 msgr2=0x563cb49b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.479+0000 7f75fd000640 1 -- 172.21.15.5:0/2307891577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cb4736f50 con 0x563cb49e3600 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f75fd000640 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 0x563cb49d7c20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f75ec003b30 tx=0x7f75ec001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf4435e7d15863a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f75daa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:36.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f75daa00640 1 -- 172.21.15.5:0/2307891577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec01a060 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f75daa00640 1 -- 172.21.15.5:0/2307891577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f75ec001bb0 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75f0054e60 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f75daa00640 1 -- 172.21.15.5:0/2307891577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec020050 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f760326c740 10 client.7248 fetch_fsmap learned FSMap version 53 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x563cb45ca0b0 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.480+0000 7f75daa00640 1 -- 172.21.15.5:0/2307891577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0054e60 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f75daa00640 1 -- 172.21.15.5:0/2307891577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75ec023020 con 0x563cb49e3600 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f760326c740 10 client.7248 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f760326c740 10 client.7248 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f760326c740 20 client.7248 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f760326c740 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb49f0e00 0x563cb49c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f760326c740 4 client.7248 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cb4954b60 con 0x563cb49f0e00 2024-04-25T10:08:36.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.481+0000 7f75fda00640 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb49f0e00 0x563cb49c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.482+0000 7f75fda00640 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb49f0e00 0x563cb49c6c50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3c0360a9171076 server_cookie=29e9cd54c1a84637 in_seq=0 out_seq=0 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.482+0000 7f75daa00640 10 client.7248 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f75daa00640 1 -- 172.21.15.5:0/2307891577 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cb4954b60 con 0x563cb49f0e00 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f75daa00640 10 client.7248 handle_command_reply: tid=0 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f760326c740 1 client.7248 shutdown 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 msgr2=0x563cb49d7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f760326c740 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 0x563cb49d7c20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f75ec003b30 tx=0x7f75ec001170 comp rx=0 tx=0).stop 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 shutdown_connections 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f760326c740 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb49f0e00 0x563cb49c6c50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.484+0000 7f760326c740 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49d9760 0x563cb49f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f760326c740 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49e3600 0x563cb49d7c20 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f760326c740 1 --2- 172.21.15.5:0/2307891577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f75daa00640 0 client.7248 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 >> 172.21.15.5:0/2307891577 conn(0x563cb499ba40 msgr2=0x563cb49e4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 shutdown_connections 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f760326c740 1 -- 172.21.15.5:0/2307891577 wait complete. 2024-04-25T10:08:36.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.485+0000 7f760326c740 20 client.7248 trim_cache size 0 max 16384 2024-04-25T10:08:36.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.498+0000 7f760326c740 1 Processor -- start 2024-04-25T10:08:36.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.498+0000 7f760326c740 1 -- start start 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f760326c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb499be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb499c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f760326c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f806f690 con 0x563cb49d9760 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80c8bb0 con 0x563cb49c5d60 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f806f090 con 0x563cb49e3600 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f75fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb499be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f75fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb499be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49572/0 (socket says 172.21.15.5:49572) 2024-04-25T10:08:36.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.499+0000 7f75fd000640 1 -- 172.21.15.5:0/684252274 learned_addr learned my addr 172.21.15.5:0/684252274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fe400640 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb499c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fda00640 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fd000640 1 -- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 msgr2=0x563cb499c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fd000640 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb499c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fd000640 1 -- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 msgr2=0x563cb49b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fd000640 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fd000640 1 -- 172.21.15.5:0/684252274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb49c62b0 con 0x563cb49d9760 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fe400640 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb499c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75fd000640 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb499be80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f75ec007e70 tx=0x7f75ec00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3872d4ff1480cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75e5600640 1 -- 172.21.15.5:0/684252274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec003640 con 0x563cb49d9760 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75e5600640 1 -- 172.21.15.5:0/684252274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec028070 con 0x563cb49d9760 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.500+0000 7f75e5600640 1 -- 172.21.15.5:0/684252274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec001670 con 0x563cb49d9760 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 -- 172.21.15.5:0/684252274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 msgr2=0x563cb499be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb499be80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f75ec007e70 tx=0x7f75ec00e000 comp rx=0 tx=0).stop 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 -- 172.21.15.5:0/684252274 shutdown_connections 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb499c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb499be80 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 --2- 172.21.15.5:0/684252274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 -- 172.21.15.5:0/684252274 >> 172.21.15.5:0/684252274 conn(0x563cb499ba40 msgr2=0x563cb49e2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 -- 172.21.15.5:0/684252274 shutdown_connections 2024-04-25T10:08:36.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.501+0000 7f760326c740 1 -- 172.21.15.5:0/684252274 wait complete. 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.502+0000 7f760326c740 1 -- start start 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.502+0000 7f760326c740 1 Processor -- start 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb49dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb4a4b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb49c28c0 con 0x563cb49d9760 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb49d8160 con 0x563cb49c5d60 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563cb49d9c30 con 0x563cb49e3600 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f760326c740 10 client.0 _ll_register_callbacks cb 0x563cb4996780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:36.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.503+0000 7f75fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb49dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb49dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49582/0 (socket says 172.21.15.5:49582) 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 -- 172.21.15.5:0/912298792 learned_addr learned my addr 172.21.15.5:0/912298792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fda00640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fe400640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb4a4b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 -- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 msgr2=0x563cb4a4b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb4a4b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 -- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 msgr2=0x563cb49dd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49dd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 -- 172.21.15.5:0/912298792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cb49e4fd0 con 0x563cb49d9760 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75fd000640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb49dea40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f75ec015900 tx=0x7f75ec000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c01a1dcae4da1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.504+0000 7f75daa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f75daa00640 1 -- 172.21.15.5:0/912298792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec015460 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f75daa00640 1 -- 172.21.15.5:0/912298792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f75ec010040 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75ec028070 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f75daa00640 1 -- 172.21.15.5:0/912298792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec01a020 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 10 client.7254 fetch_fsmap learned FSMap version 53 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x563cb49e4210 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f75daa00640 1 -- 172.21.15.5:0/912298792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec028070 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f75daa00640 1 -- 172.21.15.5:0/912298792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75ec023040 con 0x563cb49d9760 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 10 client.7254 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 10 client.7254 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:08:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 20 client.7254 populate_metadata read hostname 'smithi005' 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb499c020 0x563cb499c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 4 client.7254 mds_command: new command op to 6025 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.505+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 --> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x563cb4903020 con 0x563cb499c020 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.506+0000 7f75fda00640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb499c020 0x563cb499c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.507+0000 7f75fda00640 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb499c020 0x563cb499c400 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3be288cc3f5e22 server_cookie=d4ee7e501796de07 in_seq=0 out_seq=0 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.507+0000 7f75daa00640 10 client.7254 ms_handle_connect on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f75daa00640 1 -- 172.21.15.5:0/912298792 <== mds.0 v2:172.21.15.80:6834/3658047546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cb4903020 con 0x563cb499c020 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f75daa00640 10 client.7254 handle_command_reply: tid=0 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 client.7254 shutdown 2024-04-25T10:08:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 msgr2=0x563cb49dea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb49dea40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f75ec015900 tx=0x7f75ec000f20 comp rx=0 tx=0).stop 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 shutdown_connections 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:6834/3658047546,v1:172.21.15.80:6835/3658047546] conn(0x563cb499c020 0x563cb499c400 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563cb49e3600 0x563cb4a4b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cb49d9760 0x563cb49dea40 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 --2- 172.21.15.5:0/912298792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563cb49c5d60 0x563cb49dd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f75daa00640 0 client.7254 ms_handle_reset on v2:172.21.15.80:6834/3658047546 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 >> 172.21.15.5:0/912298792 conn(0x563cb499ba40 msgr2=0x563cb49b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 shutdown_connections 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 1 -- 172.21.15.5:0/912298792 wait complete. 2024-04-25T10:08:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.508+0000 7f760326c740 20 client.7254 trim_cache size 0 max 16384 2024-04-25T10:08:36.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:36.518 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-25T10:08:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75dc06b500 msgr2=0x7f75dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75dc06b500 0x7f75dc06d9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f75e8034000 tx=0x7f75e802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 msgr2=0x7f75f80b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 0x7f75f80b7de0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f75f000cbc0 tx=0x7f75f000a430 comp rx=0 tx=0).stop 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 shutdown_connections 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f8161d90 0x7f75f80b7de0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75dc06b500 0x7f75dc06d9c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f80d3f90 0x7f75f80b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 --2- 172.21.15.5:0/3343197392 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f80b48c0 0x7f75f8153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 >> 172.21.15.5:0/3343197392 conn(0x7f75f808af80 msgr2=0x7f75f80ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 shutdown_connections 2024-04-25T10:08:36.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.509+0000 7f75fee00640 1 -- 172.21.15.5:0/3343197392 wait complete. 2024-04-25T10:08:36.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:08:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.718+0000 7fcded200640 1 Processor -- start 2024-04-25T10:08:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.719+0000 7fcded200640 1 -- start start 2024-04-25T10:08:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.720+0000 7fcded200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcde80cee10 0x7fcde80d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.720+0000 7fcded200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcde806bdc0 0x7fcde806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.720+0000 7fcded200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde806cb40 con 0x7fcde80d0b10 2024-04-25T10:08:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.720+0000 7fcded200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80d2f10 con 0x7fcde806bdc0 2024-04-25T10:08:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.720+0000 7fcded200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80d1c30 con 0x7fcde80cee10 2024-04-25T10:08:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde7400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcde80d0b10 0x7fcde80ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:52252/0 (socket says 172.21.15.5:52252) 2024-04-25T10:08:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde7400640 1 -- 172.21.15.5:0/2435277471 learned_addr learned my addr 172.21.15.5:0/2435277471 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde7e00640 1 --2- 172.21.15.5:0/2435277471 >> v2:172.21.15.5:3300/0 conn(0x7fcde806bdc0 0x7fcde806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde6a00640 1 --2- 172.21.15.5:0/2435277471 >> v2:172.21.15.80:3300/0 conn(0x7fcde80cee10 0x7fcde80d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde6000640 1 -- 172.21.15.5:0/2435277471 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130394570 0 0) 0x7fcde806cb40 con 0x7fcde80d0b10 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde6000640 1 -- 172.21.15.5:0/2435277471 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc8003520 con 0x7fcde80d0b10 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde7e00640 1 -- 172.21.15.5:0/2435277471 >> v2:172.21.15.80:3300/0 conn(0x7fcde80cee10 msgr2=0x7fcde80d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.721+0000 7fcde7e00640 1 --2- 172.21.15.5:0/2435277471 >> v2:172.21.15.80:3300/0 conn(0x7fcde80cee10 0x7fcde80d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.722+0000 7fcde7e00640 1 -- 172.21.15.5:0/2435277471 >> v1:172.21.15.5:6789/0 conn(0x7fcde80d0b10 legacy=0x7fcde80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.722+0000 7fcde7e00640 1 -- 172.21.15.5:0/2435277471 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80d1f10 con 0x7fcde806bdc0 2024-04-25T10:08:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.723+0000 7fcde7e00640 1 --2- 172.21.15.5:0/2435277471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde806bdc0 0x7fcde806c1d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcde80d1890 tx=0x7fcde002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3924be2a51ef75b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.723+0000 7fcde6000640 1 -- 172.21.15.5:0/2435277471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0037070 con 0x7fcde806bdc0 2024-04-25T10:08:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.723+0000 7fcde6000640 1 -- 172.21.15.5:0/2435277471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0032440 con 0x7fcde806bdc0 2024-04-25T10:08:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.723+0000 7fcde6000640 1 -- 172.21.15.5:0/2435277471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00327f0 con 0x7fcde806bdc0 2024-04-25T10:08:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.723+0000 7fcded200640 1 -- 172.21.15.5:0/2435277471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde806bdc0 msgr2=0x7fcde806c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.723+0000 7fcded200640 1 --2- 172.21.15.5:0/2435277471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde806bdc0 0x7fcde806c1d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcde80d1890 tx=0x7fcde002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.724+0000 7fcded200640 1 -- 172.21.15.5:0/2435277471 shutdown_connections 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.724+0000 7fcded200640 1 --2- 172.21.15.5:0/2435277471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde806bdc0 0x7fcde806c1d0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.724+0000 7fcded200640 1 --2- 172.21.15.5:0/2435277471 >> v2:172.21.15.80:3300/0 conn(0x7fcde80cee10 0x7fcde80d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.724+0000 7fcded200640 1 -- 172.21.15.5:0/2435277471 >> 172.21.15.5:0/2435277471 conn(0x7fcde80b53d0 msgr2=0x7fcde80b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.724+0000 7fcded200640 1 -- 172.21.15.5:0/2435277471 shutdown_connections 2024-04-25T10:08:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.724+0000 7fcded200640 1 -- 172.21.15.5:0/2435277471 wait complete. 2024-04-25T10:08:36.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.725+0000 7fcded200640 1 Processor -- start 2024-04-25T10:08:36.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.726+0000 7fcded200640 1 -- start start 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcded200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcde806bdc0 0x7fcde81f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcded200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 0x7fcde80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcded200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcde80d0b10 0x7fcde80ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcded200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde806c710 con 0x7fcde80cee10 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcded200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80d24d0 con 0x7fcde806bdc0 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcded200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde806c890 con 0x7fcde80d0b10 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 0x7fcde80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 0x7fcde80cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49606/0 (socket says 172.21.15.5:49606) 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 -- 172.21.15.5:0/3812171861 learned_addr learned my addr 172.21.15.5:0/3812171861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 -- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcde80d0b10 msgr2=0x7fcde80ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcde80d0b10 0x7fcde80ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 -- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcde806bdc0 msgr2=0x7fcde81f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcde806bdc0 0x7fcde81f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.727+0000 7fcde6a00640 1 -- 172.21.15.5:0/3812171861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde815a8f0 con 0x7fcde80cee10 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.728+0000 7fcde6a00640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 0x7fcde80cc980 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000efc0 tx=0x7fcdd000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.728+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0018020 con 0x7fcde80cee10 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.728+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd0014480 con 0x7fcde80cee10 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.728+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0014830 con 0x7fcde80cee10 2024-04-25T10:08:36.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.728+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde815abd0 con 0x7fcde80cee10 2024-04-25T10:08:36.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.728+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde815b190 con 0x7fcde80cee10 2024-04-25T10:08:36.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.729+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde80677d0 con 0x7fcde80cee10 2024-04-25T10:08:36.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.730+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdd0005d10 con 0x7fcde80cee10 2024-04-25T10:08:36.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.732+0000 7fcddf400640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdc406b4d0 0x7fcdc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.732+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcdd008b2f0 con 0x7fcde80cee10 2024-04-25T10:08:36.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.733+0000 7fcde7400640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdc406b4d0 0x7fcdc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.734+0000 7fcde7400640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdc406b4d0 0x7fcdc406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000f60 tx=0x7fcdd80078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.737+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdd0022030 con 0x7fcde80cee10 2024-04-25T10:08:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.974+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcde8068f60 con 0x7fcde80cee10 2024-04-25T10:08:36.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.975+0000 7fcddf400640 1 -- 172.21.15.5:0/3812171861 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcdd0010040 con 0x7fcde80cee10 2024-04-25T10:08:36.981 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:36.981 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:08:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.977+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdc406b4d0 msgr2=0x7fcdc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.977+0000 7fcded200640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdc406b4d0 0x7fcdc406d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000f60 tx=0x7fcdd80078f0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.977+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 msgr2=0x7fcde80cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.977+0000 7fcded200640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 0x7fcde80cc980 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000efc0 tx=0x7fcdd000c490 comp rx=0 tx=0).stop 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 shutdown_connections 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcde80d0b10 0x7fcde80ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcde80cee10 0x7fcde80cc980 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdc406b4d0 0x7fcdc406d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 --2- 172.21.15.5:0/3812171861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcde806bdc0 0x7fcde81f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 >> 172.21.15.5:0/3812171861 conn(0x7fcde80b53d0 msgr2=0x7fcde80b57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 shutdown_connections 2024-04-25T10:08:36.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:36.978+0000 7fcded200640 1 -- 172.21.15.5:0/3812171861 wait complete. 2024-04-25T10:08:37.002 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:08:37.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.202+0000 7f22dae00640 1 Processor -- start 2024-04-25T10:08:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.202+0000 7f22dae00640 1 -- start start 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22dae00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22d40cda10 0x7f22d415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22dae00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d4071d30 con 0x7f22d4141da0 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22dae00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d406d620 con 0x7f22d415ce00 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22dae00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d4142180 con 0x7f22d40cda10 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22d9a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f22d4141da0 0x7f22d406e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40552/0 (socket says 172.21.15.5:40552) 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22d9a00640 1 -- 172.21.15.5:0/1504757117 learned_addr learned my addr 172.21.15.5:0/1504757117 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:37.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.204+0000 7f22da400640 1 --2- 172.21.15.5:0/1504757117 >> v2:172.21.15.5:3300/0 conn(0x7f22d40cda10 0x7f22d415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.205+0000 7f22d3e00640 1 -- 172.21.15.5:0/1504757117 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176947191 0 0) 0x7f22d4071d30 con 0x7f22d4141da0 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.205+0000 7f22d3e00640 1 -- 172.21.15.5:0/1504757117 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22bc003520 con 0x7f22d4141da0 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.205+0000 7f22da400640 1 -- 172.21.15.5:0/1504757117 >> v1:172.21.15.80:6789/0 conn(0x7f22d415ce00 legacy=0x7f22d40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.205+0000 7f22da400640 1 -- 172.21.15.5:0/1504757117 >> v1:172.21.15.5:6789/0 conn(0x7f22d4141da0 legacy=0x7f22d406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.205+0000 7f22da400640 1 -- 172.21.15.5:0/1504757117 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d406d900 con 0x7f22d40cda10 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.206+0000 7f22da400640 1 --2- 172.21.15.5:0/1504757117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d40cda10 0x7f22d415d280 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0097a0 tx=0x7f22cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d38e8fe91b3990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.206+0000 7f22d3e00640 1 -- 172.21.15.5:0/1504757117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc037070 con 0x7f22d40cda10 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.206+0000 7f22d3e00640 1 -- 172.21.15.5:0/1504757117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc032440 con 0x7f22d40cda10 2024-04-25T10:08:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.206+0000 7f22d3e00640 1 -- 172.21.15.5:0/1504757117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0327f0 con 0x7f22d40cda10 2024-04-25T10:08:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 -- 172.21.15.5:0/1504757117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d40cda10 msgr2=0x7f22d415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 --2- 172.21.15.5:0/1504757117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d40cda10 0x7f22d415d280 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0097a0 tx=0x7f22cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 -- 172.21.15.5:0/1504757117 shutdown_connections 2024-04-25T10:08:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 --2- 172.21.15.5:0/1504757117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d40cda10 0x7f22d415d280 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 -- 172.21.15.5:0/1504757117 >> 172.21.15.5:0/1504757117 conn(0x7f22d4066f20 msgr2=0x7f22d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 -- 172.21.15.5:0/1504757117 shutdown_connections 2024-04-25T10:08:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.207+0000 7f22dae00640 1 -- 172.21.15.5:0/1504757117 wait complete. 2024-04-25T10:08:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.209+0000 7f22dae00640 1 Processor -- start 2024-04-25T10:08:37.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.209+0000 7f22dae00640 1 -- start start 2024-04-25T10:08:37.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22dae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22d40cda10 0x7f22d413f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22dae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22d4141da0 0x7f22d415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22dae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 0x7f22d415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22dae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4071000 con 0x7f22d415ce00 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22dae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406e810 con 0x7f22d4141da0 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22dae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4071180 con 0x7f22d40cda10 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22da400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 0x7f22d415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22da400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 0x7f22d415b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44378/0 (socket says 172.21.15.5:44378) 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.210+0000 7f22da400640 1 -- 172.21.15.5:0/3158990614 learned_addr learned my addr 172.21.15.5:0/3158990614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22d9000640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22d4141da0 0x7f22d415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22d9a00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22d40cda10 0x7f22d413f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22da400640 1 -- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22d40cda10 msgr2=0x7f22d413f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22da400640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22d40cda10 0x7f22d413f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22da400640 1 -- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22d4141da0 msgr2=0x7f22d415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22da400640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22d4141da0 0x7f22d415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22da400640 1 -- 172.21.15.5:0/3158990614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d40b3520 con 0x7f22d415ce00 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22d9000640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22d4141da0 0x7f22d415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22d9a00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22d40cda10 0x7f22d413f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22da400640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 0x7f22d415b170 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0097a0 tx=0x7f22cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc037040 con 0x7f22d415ce00 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc042070 con 0x7f22d415ce00 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.211+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d40b3800 con 0x7f22d415ce00 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.212+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc030840 con 0x7f22d415ce00 2024-04-25T10:08:37.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.212+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d40b3dc0 con 0x7f22d415ce00 2024-04-25T10:08:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.213+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4068f60 con 0x7f22d415ce00 2024-04-25T10:08:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.214+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22cc0309e0 con 0x7f22d415ce00 2024-04-25T10:08:37.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.215+0000 7f22d1600640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.215+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f22cc0aadc0 con 0x7f22d415ce00 2024-04-25T10:08:37.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.216+0000 7f22d9a00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.217+0000 7f22d9a00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f22b406b4d0 0x7f22b406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f22c8005c00 tx=0x7f22c8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.219+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22cc0799d0 con 0x7f22d415ce00 2024-04-25T10:08:37.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.452+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22d413eee0 con 0x7f22d415ce00 2024-04-25T10:08:37.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.452+0000 7f22d1600640 1 -- 172.21.15.5:0/3158990614 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22cc0792d0 con 0x7f22d415ce00 2024-04-25T10:08:37.459 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f22b406b4d0 msgr2=0x7f22b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f22b406b4d0 0x7f22b406d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f22c8005c00 tx=0x7f22c8009040 comp rx=0 tx=0).stop 2024-04-25T10:08:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 msgr2=0x7f22d415b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 0x7f22d415b170 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0097a0 tx=0x7f22cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 shutdown_connections 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22d415ce00 0x7f22d415b170 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f22b406b4d0 0x7f22b406d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22d4141da0 0x7f22d415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 --2- 172.21.15.5:0/3158990614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22d40cda10 0x7f22d413f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 >> 172.21.15.5:0/3158990614 conn(0x7f22d4066f20 msgr2=0x7f22d4140230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 shutdown_connections 2024-04-25T10:08:37.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:37.455+0000 7f22dae00640 1 -- 172.21.15.5:0/3158990614 wait complete. 2024-04-25T10:08:37.477 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:08:37.477 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:37.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:08:37.477 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:37.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:37.478 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:08:37.478 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:08:37.478 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:37.478 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:37.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:37.478 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:08:37.527 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:08:37.528 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42208sec preferred_lft 42208sec 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:08:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:37.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:37.534 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:37.616 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:37.622 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:08:37.677 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:08:37.677 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:37.677 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:08:37.677 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:08:37.677 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:37.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:37.678 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:08:37.678 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:08:37.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:08:37.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:08:37.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:08:37.679 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:08:37.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:37.830 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:08:37.883 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:37.883 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:37.884 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:37.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:37.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:08:37.960 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:37.960 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:38.010 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:38.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:38.011 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:38.058 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:08:38.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:38.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:38.189 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:38.189 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:38.190 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:08:38.212 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:38.212 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:38.212 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:08:38.212 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:08:38.213 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:38.243 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:38.243 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:38.244 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:08:38.244 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:08:38.244 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:38.244 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:08:38.244 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:08:38.244 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:08:38.246 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:08:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:08:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:08:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:08:38.246 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:08:38.251 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:08:38.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:38.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:38.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:38.332 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:08:38.392 INFO:teuthology.orchestra.run.smithi005.stdout:client6139 2024-04-25T10:08:38.392 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:08:38.392 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:38.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:38.443 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:08:38.547 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:08:38.547 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.658+0000 7f2135430f40 1 Processor -- start 2024-04-25T10:08:38.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.659+0000 7f2135430f40 1 -- start start 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2135430f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255857ce0 0x557255876460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2135430f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 0x5572558769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2135430f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557255879030 0x557255876ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2135430f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572557e4270 con 0x5572558580c0 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2135430f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557255740dd0 con 0x557255857ce0 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2135430f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572557e5590 con 0x557255879030 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2131c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 0x5572558769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2131c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 0x5572558769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44398/0 (socket says 172.21.15.5:44398) 2024-04-25T10:08:38.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.660+0000 7f2131c00640 1 -- 172.21.15.5:0/2352118091 learned_addr learned my addr 172.21.15.5:0/2352118091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:38.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2132600640 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255857ce0 0x557255876460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2131c00640 1 -- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557255879030 msgr2=0x557255876ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2131c00640 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557255879030 0x557255876ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2131c00640 1 -- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255857ce0 msgr2=0x557255876460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2131c00640 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255857ce0 0x557255876460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2131c00640 1 -- 172.21.15.5:0/2352118091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557255857030 con 0x5572558580c0 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f2131c00640 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 0x5572558769a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f212000b2c0 tx=0x7f212000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa45c180ea31ae23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f212be00640 1 -- 172.21.15.5:0/2352118091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120008070 con 0x5572558580c0 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f212be00640 1 -- 172.21.15.5:0/2352118091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000d500 con 0x5572558580c0 2024-04-25T10:08:38.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.661+0000 7f212be00640 1 -- 172.21.15.5:0/2352118091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000d930 con 0x5572558580c0 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 -- 172.21.15.5:0/2352118091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 msgr2=0x5572558769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 0x5572558769a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f212000b2c0 tx=0x7f212000b780 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 -- 172.21.15.5:0/2352118091 shutdown_connections 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557255879030 0x557255876ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572558580c0 0x5572558769a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 --2- 172.21.15.5:0/2352118091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255857ce0 0x557255876460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 -- 172.21.15.5:0/2352118091 >> 172.21.15.5:0/2352118091 conn(0x557255798c60 msgr2=0x5572557cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 -- 172.21.15.5:0/2352118091 shutdown_connections 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.662+0000 7f2135430f40 1 -- 172.21.15.5:0/2352118091 wait complete. 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.663+0000 7f2135430f40 1 Processor -- start 2024-04-25T10:08:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.663+0000 7f2135430f40 1 -- start start 2024-04-25T10:08:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2135430f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557255857ce0 0x5572557cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2135430f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5572558580c0 0x557255870e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2135430f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255879030 0x5572559057a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2135430f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572557c9be0 con 0x557255857ce0 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2135430f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572557ca230 con 0x557255879030 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2135430f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572557c9ec0 con 0x5572558580c0 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2132600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557255857ce0 0x5572557cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2132600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557255857ce0 0x5572557cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44402/0 (socket says 172.21.15.5:44402) 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.664+0000 7f2132600640 1 -- 172.21.15.5:0/1458491310 learned_addr learned my addr 172.21.15.5:0/1458491310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2131c00640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5572558580c0 0x557255870e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2132600640 1 -- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5572558580c0 msgr2=0x557255870e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2132600640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5572558580c0 0x557255870e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2132600640 1 -- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255879030 msgr2=0x5572559057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2133000640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255879030 0x5572559057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2132600640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557255879030 0x5572559057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2132600640 1 -- 172.21.15.5:0/1458491310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725586ba50 con 0x557255857ce0 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.665+0000 7f2132600640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557255857ce0 0x5572557cb7d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2124008480 tx=0x7f2124008940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.666+0000 7f212aa00640 1 -- 172.21.15.5:0/1458491310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212400e040 con 0x557255857ce0 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.666+0000 7f212aa00640 1 -- 172.21.15.5:0/1458491310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124017070 con 0x557255857ce0 2024-04-25T10:08:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.666+0000 7f212aa00640 1 -- 172.21.15.5:0/1458491310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124011850 con 0x557255857ce0 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.666+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557255904cc0 con 0x557255857ce0 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.666+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55725586bf00 con 0x557255857ce0 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.668+0000 7f212aa00640 1 -- 172.21.15.5:0/1458491310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21240119f0 con 0x557255857ce0 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.669+0000 7f212aa00640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2114069eb0 0x7f211406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.669+0000 7f212aa00640 1 -- 172.21.15.5:0/1458491310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2124088f50 con 0x557255857ce0 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.669+0000 7f2135430f40 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5572557cbd10 0x55725590eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.669+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78) v9 -- 0x55725586c450 con 0x5572557cbd10 2024-04-25T10:08:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.669+0000 7f2131c00640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2114069eb0 0x7f211406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.670+0000 7f2133000640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5572557cbd10 0x55725590eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.671+0000 7f2131c00640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2114069eb0 0x7f211406c370 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f212000b2c0 tx=0x7f2120009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.671+0000 7f2133000640 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5572557cbd10 0x55725590eed0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2133000640 1 -- 172.21.15.5:0/1458491310 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v78'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f212c05c040 con 0x5572557cbd10 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5572557cbd10 msgr2=0x55725590eed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5572557cbd10 0x55725590eed0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2114069eb0 msgr2=0x7f211406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2114069eb0 0x7f211406c370 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f212000b2c0 tx=0x7f2120009000 comp rx=0 tx=0).stop 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557255857ce0 msgr2=0x5572557cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 --2- 172.21.15.5:0/1458491310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557255857ce0 0x5572557cb7d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2124008480 tx=0x7f2124008940 comp rx=0 tx=0).stop 2024-04-25T10:08:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2133000640 1 -- 172.21.15.5:0/1458491310 reap_dead start 2024-04-25T10:08:38.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 shutdown_connections 2024-04-25T10:08:38.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 >> 172.21.15.5:0/1458491310 conn(0x557255798c60 msgr2=0x557255876cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 shutdown_connections 2024-04-25T10:08:38.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:38.674+0000 7f2135430f40 1 -- 172.21.15.5:0/1458491310 wait complete. 2024-04-25T10:08:38.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:38.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:38.785 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:38.894 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:38.894 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:08:38.894 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:38.957 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:08:38.957 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:39.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.223+0000 7f87da800640 1 Processor -- start 2024-04-25T10:08:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.223+0000 7f87da800640 1 -- start start 2024-04-25T10:08:39.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.225+0000 7f87da800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87d4070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.225+0000 7f87da800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d40cd2a0 con 0x7f87d40714d0 2024-04-25T10:08:39.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.225+0000 7f87da800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d40cd580 con 0x7f87d41652f0 2024-04-25T10:08:39.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.225+0000 7f87da800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40ce340 con 0x7f87d406e980 2024-04-25T10:08:39.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.225+0000 7f87d9e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f87d40714d0 0x7f87d406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40568/0 (socket says 172.21.15.5:40568) 2024-04-25T10:08:39.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.225+0000 7f87d9e00640 1 -- 172.21.15.5:0/3206169718 learned_addr learned my addr 172.21.15.5:0/3206169718 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:39.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87d4070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461165464 0 0) 0x7f87d40cd2a0 con 0x7f87d40714d0 2024-04-25T10:08:39.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87bc003520 con 0x7f87d40714d0 2024-04-25T10:08:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525707948 0 0) 0x7f87bc003520 con 0x7f87d40714d0 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 msgr2=0x7f87d4070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87d4070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 >> v1:172.21.15.80:6789/0 conn(0x7f87d41652f0 legacy=0x7f87d40cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40cd860 con 0x7f87d40714d0 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.226+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f87cc0019f0 con 0x7f87d40714d0 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 >> v1:172.21.15.5:6789/0 conn(0x7f87d40714d0 legacy=0x7f87d406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87bc0039e0 0x7f87bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40ce340 con 0x7f87bc004800 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc00ba70 con 0x7f87bc0039e0 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc00bc70 con 0x7f87d406e980 2024-04-25T10:08:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d9e00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d8a00640 1 -- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 msgr2=0x7f87bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d8a00640 1 -- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87bc0039e0 msgr2=0x7f87bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87bc0039e0 0x7f87bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d8a00640 1 -- 172.21.15.5:0/3206169718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc003520 con 0x7f87bc004800 2024-04-25T10:08:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.227+0000 7f87d9e00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d406e980 0x7f87bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87bc004c80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009a90 tx=0x7f87bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb52db896c750d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0036070 con 0x7f87bc004800 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0031400 con 0x7f87bc004800 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87d3e00640 1 -- 172.21.15.5:0/3206169718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0009690 con 0x7f87bc004800 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87da800640 1 -- 172.21.15.5:0/3206169718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 msgr2=0x7f87bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87da800640 1 --2- 172.21.15.5:0/3206169718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87bc004c80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009a90 tx=0x7f87bc005830 comp rx=0 tx=0).stop 2024-04-25T10:08:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.229+0000 7f87d9e00640 1 -- 172.21.15.5:0/3206169718 reap_dead start 2024-04-25T10:08:39.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.230+0000 7f87da800640 1 -- 172.21.15.5:0/3206169718 shutdown_connections 2024-04-25T10:08:39.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.230+0000 7f87da800640 1 -- 172.21.15.5:0/3206169718 >> 172.21.15.5:0/3206169718 conn(0x7f87d4066f20 msgr2=0x7f87d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.230+0000 7f87da800640 1 -- 172.21.15.5:0/3206169718 shutdown_connections 2024-04-25T10:08:39.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.230+0000 7f87da800640 1 -- 172.21.15.5:0/3206169718 wait complete. 2024-04-25T10:08:39.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.232+0000 7f87da800640 1 Processor -- start 2024-04-25T10:08:39.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.232+0000 7f87da800640 1 -- start start 2024-04-25T10:08:39.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87da800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87d415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87da800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d40adc50 0x7f87d40ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87da800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d40b00e0 0x7f87d40b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87da800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4070f90 con 0x7f87bc004800 2024-04-25T10:08:39.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87da800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d40cdeb0 con 0x7f87d40adc50 2024-04-25T10:08:39.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87da800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d4071110 con 0x7f87d40b00e0 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87d415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87d415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44422/0 (socket says 172.21.15.5:44422) 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 -- 172.21.15.5:0/3156135851 learned_addr learned my addr 172.21.15.5:0/3156135851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 -- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d40b00e0 msgr2=0x7f87d40b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d40b00e0 0x7f87d40b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 -- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d40adc50 msgr2=0x7f87d40ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d40adc50 0x7f87d40ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.233+0000 7f87d9400640 1 -- 172.21.15.5:0/3156135851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d41fb010 con 0x7f87bc004800 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.234+0000 7f87d9400640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87d415f4a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c460 tx=0x7f87c800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.234+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80052d0 con 0x7f87bc004800 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.234+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8005490 con 0x7f87bc004800 2024-04-25T10:08:39.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.234+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8012870 con 0x7f87bc004800 2024-04-25T10:08:39.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.234+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d41fb2f0 con 0x7f87bc004800 2024-04-25T10:08:39.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.234+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d41fb8b0 con 0x7f87bc004800 2024-04-25T10:08:39.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.235+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d4068f60 con 0x7f87bc004800 2024-04-25T10:08:39.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.236+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87c8012a10 con 0x7f87bc004800 2024-04-25T10:08:39.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.237+0000 7f87d1600640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f87b406b4d0 0x7f87b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.237+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f87c808a060 con 0x7f87bc004800 2024-04-25T10:08:39.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.238+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f87b406b4d0 0x7f87b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.239+0000 7f87d8a00640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f87b406b4d0 0x7f87b406d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f87c0033000 tx=0x7f87c002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.242+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87c8058c60 con 0x7f87bc004800 2024-04-25T10:08:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.446+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87d40cce40 con 0x7f87bc004800 2024-04-25T10:08:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.448+0000 7f87d1600640 1 -- 172.21.15.5:0/3156135851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16906 (secure 0 0 0) 0x7f87c8058560 con 0x7f87bc004800 2024-04-25T10:08:39.455 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:39.455 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:22.531478+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1098619647":"2024-04-26T10:07:22.531455+0000","172.21.15.80:6834/1098619647":"2024-04-26T10:07:22.531455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:39.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f87b406b4d0 msgr2=0x7f87b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f87b406b4d0 0x7f87b406d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f87c0033000 tx=0x7f87c002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:39.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 msgr2=0x7f87d415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87d415f4a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c460 tx=0x7f87c800c920 comp rx=0 tx=0).stop 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 shutdown_connections 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f87d40b00e0 0x7f87d40b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f87b406b4d0 0x7f87b406d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d40adc50 0x7f87d40ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 --2- 172.21.15.5:0/3156135851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87bc004800 0x7f87d415f4a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 >> 172.21.15.5:0/3156135851 conn(0x7f87d4066f20 msgr2=0x7f87d40af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 shutdown_connections 2024-04-25T10:08:39.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.451+0000 7f87da800640 1 -- 172.21.15.5:0/3156135851 wait complete. 2024-04-25T10:08:39.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:07.362188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:08.532434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:39.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.680+0000 7f7387e00640 1 Processor -- start 2024-04-25T10:08:39.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.680+0000 7f7387e00640 1 -- start start 2024-04-25T10:08:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7387e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f738815ec80 0x7f73880b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7387e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73880d4000 0x7f7388160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7387e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738815ffa0 con 0x7f7388161240 2024-04-25T10:08:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7387e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7388160240 con 0x7f738815ec80 2024-04-25T10:08:39.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7387e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7388160520 con 0x7f73880d4000 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7386a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f738815ec80 0x7f73880b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7386a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f738815ec80 0x7f73880b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44432/0 (socket says 172.21.15.5:44432) 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7386a00640 1 -- 172.21.15.5:0/3794022187 learned_addr learned my addr 172.21.15.5:0/3794022187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.682+0000 7f7386000640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73880d4000 0x7f7388160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.683+0000 7f7386a00640 1 -- 172.21.15.5:0/3794022187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73880d4000 msgr2=0x7f7388160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.683+0000 7f7386a00640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73880d4000 0x7f7388160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.683+0000 7f7386a00640 1 -- 172.21.15.5:0/3794022187 >> v1:172.21.15.80:6789/0 conn(0x7f7388161240 legacy=0x7f73881616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.683+0000 7f7386a00640 1 -- 172.21.15.5:0/3794022187 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7388160800 con 0x7f738815ec80 2024-04-25T10:08:39.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.683+0000 7f7386000640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73880d4000 0x7f7388160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.684+0000 7f7386a00640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b45c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7378009860 tx=0x7f737802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107d9a170a28a237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.684+0000 7f7385600640 1 -- 172.21.15.5:0/3794022187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378037070 con 0x7f738815ec80 2024-04-25T10:08:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.684+0000 7f7385600640 1 -- 172.21.15.5:0/3794022187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7378032440 con 0x7f738815ec80 2024-04-25T10:08:39.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.684+0000 7f7385600640 1 -- 172.21.15.5:0/3794022187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378032810 con 0x7f738815ec80 2024-04-25T10:08:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 -- 172.21.15.5:0/3794022187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 msgr2=0x7f73880b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b45c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7378009860 tx=0x7f737802a250 comp rx=0 tx=0).stop 2024-04-25T10:08:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 -- 172.21.15.5:0/3794022187 shutdown_connections 2024-04-25T10:08:39.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73880d4000 0x7f7388160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 --2- 172.21.15.5:0/3794022187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b45c0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 -- 172.21.15.5:0/3794022187 >> 172.21.15.5:0/3794022187 conn(0x7f738808af80 msgr2=0x7f73880d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 -- 172.21.15.5:0/3794022187 shutdown_connections 2024-04-25T10:08:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.685+0000 7f7387e00640 1 -- 172.21.15.5:0/3794022187 wait complete. 2024-04-25T10:08:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.688+0000 7f7387e00640 1 Processor -- start 2024-04-25T10:08:39.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.688+0000 7f7387e00640 1 -- start start 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7387e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73880d4000 0x7f7388143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7387e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7387e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7388161240 0x7f7388145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7387e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880d4490 con 0x7f738815ec80 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7387e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880b4c50 con 0x7f73880d4000 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7387e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7388161e00 con 0x7f7388161240 2024-04-25T10:08:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7386000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7386000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44442/0 (socket says 172.21.15.5:44442) 2024-04-25T10:08:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.689+0000 7f7386000640 1 -- 172.21.15.5:0/3627443002 learned_addr learned my addr 172.21.15.5:0/3627443002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386a00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73880d4000 0x7f7388143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7387400640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7388161240 0x7f7388145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386000640 1 -- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7388161240 msgr2=0x7f7388145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386000640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7388161240 0x7f7388145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386000640 1 -- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73880d4000 msgr2=0x7f7388143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386000640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73880d4000 0x7f7388143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386000640 1 -- 172.21.15.5:0/3627443002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7388143790 con 0x7f738815ec80 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386a00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73880d4000 0x7f7388143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.690+0000 7f7386000640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b66c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f737400e840 tx=0x7f737400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.691+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400f040 con 0x7f738815ec80 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.691+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7374004030 con 0x7f738815ec80 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.691+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374004420 con 0x7f738815ec80 2024-04-25T10:08:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.691+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73880b9390 con 0x7f738815ec80 2024-04-25T10:08:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.691+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73880b8af0 con 0x7f738815ec80 2024-04-25T10:08:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.692+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7388068f60 con 0x7f738815ec80 2024-04-25T10:08:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.693+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7374005990 con 0x7f738815ec80 2024-04-25T10:08:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.694+0000 7f737ea00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f736806b4d0 0x7f736806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.694+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f737408a550 con 0x7f738815ec80 2024-04-25T10:08:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.695+0000 7f7386a00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f736806b4d0 0x7f736806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.696+0000 7f7386a00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f736806b4d0 0x7f736806d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7378034000 tx=0x7f737802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.698+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7374020020 con 0x7f738815ec80 2024-04-25T10:08:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.897+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73880b9560 con 0x7f738815ec80 2024-04-25T10:08:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.899+0000 7f737ea00640 1 -- 172.21.15.5:0/3627443002 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7388143790 con 0x7f738815ec80 2024-04-25T10:08:39.905 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:08:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f736806b4d0 msgr2=0x7f736806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f736806b4d0 0x7f736806d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7378034000 tx=0x7f737802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 msgr2=0x7f73880b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b66c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f737400e840 tx=0x7f737400ed00 comp rx=0 tx=0).stop 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 shutdown_connections 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7388161240 0x7f7388145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f738815ec80 0x7f73880b66c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f736806b4d0 0x7f736806d990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 --2- 172.21.15.5:0/3627443002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73880d4000 0x7f7388143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.901+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 >> 172.21.15.5:0/3627443002 conn(0x7f738808af80 msgr2=0x7f7388073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.902+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 shutdown_connections 2024-04-25T10:08:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:39.902+0000 7f7387e00640 1 -- 172.21.15.5:0/3627443002 wait complete. 2024-04-25T10:08:39.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.135+0000 7f170a400640 1 Processor -- start 2024-04-25T10:08:40.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.135+0000 7f170a400640 1 -- start start 2024-04-25T10:08:40.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f170a400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1704161c30 0x7f17040b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f170a400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1704162010 0x7f1704162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f170a400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17040cd4c0 con 0x7f17040b2650 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f170a400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17040b4bd0 con 0x7f1704162010 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f170a400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17041632e0 con 0x7f1704161c30 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f1702a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1704162010 0x7f1704162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f1703e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f17040b2650 0x7f1704071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40572/0 (socket says 172.21.15.5:40572) 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f1703e00640 1 -- 172.21.15.5:0/3153743764 learned_addr learned my addr 172.21.15.5:0/3153743764 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.137+0000 7f1703400640 1 --2- 172.21.15.5:0/3153743764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1704161c30 0x7f17040b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.138+0000 7f1702a00640 1 -- 172.21.15.5:0/3153743764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1704161c30 msgr2=0x7f17040b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.138+0000 7f1702a00640 1 --2- 172.21.15.5:0/3153743764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1704161c30 0x7f17040b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.138+0000 7f1702a00640 1 -- 172.21.15.5:0/3153743764 >> v1:172.21.15.5:6789/0 conn(0x7f17040b2650 legacy=0x7f1704071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.138+0000 7f1702a00640 1 -- 172.21.15.5:0/3153743764 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17040cd760 con 0x7f1704162010 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.139+0000 7f1702a00640 1 --2- 172.21.15.5:0/3153743764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f1704162da0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009d90 tx=0x7f16f002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa642b5283227951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.139+0000 7f1702000640 1 -- 172.21.15.5:0/3153743764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0007740 con 0x7f1704162010 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.139+0000 7f1702000640 1 -- 172.21.15.5:0/3153743764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f00078e0 con 0x7f1704162010 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.139+0000 7f1702000640 1 -- 172.21.15.5:0/3153743764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0007cb0 con 0x7f1704162010 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 -- 172.21.15.5:0/3153743764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 msgr2=0x7f1704162da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 --2- 172.21.15.5:0/3153743764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f1704162da0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009d90 tx=0x7f16f002a660 comp rx=0 tx=0).stop 2024-04-25T10:08:40.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 -- 172.21.15.5:0/3153743764 shutdown_connections 2024-04-25T10:08:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 --2- 172.21.15.5:0/3153743764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f1704162da0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 --2- 172.21.15.5:0/3153743764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1704161c30 0x7f17040b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 -- 172.21.15.5:0/3153743764 >> 172.21.15.5:0/3153743764 conn(0x7f170406ab60 msgr2=0x7f170406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 -- 172.21.15.5:0/3153743764 shutdown_connections 2024-04-25T10:08:40.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.140+0000 7f170a400640 1 -- 172.21.15.5:0/3153743764 wait complete. 2024-04-25T10:08:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.142+0000 7f170a400640 1 Processor -- start 2024-04-25T10:08:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.143+0000 7f170a400640 1 -- start start 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f170a400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040b2650 0x7f170415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f170a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1704161c30 0x7f17041ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f170a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f17040af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f170a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1704071840 con 0x7f1704162010 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f170a400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17040cda00 con 0x7f1704161c30 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f170a400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17040719c0 con 0x7f17040b2650 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f1703e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f17040af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f1703e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f17040af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44454/0 (socket says 172.21.15.5:44454) 2024-04-25T10:08:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.144+0000 7f1703e00640 1 -- 172.21.15.5:0/115608642 learned_addr learned my addr 172.21.15.5:0/115608642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:40.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040b2650 0x7f170415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1702a00640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1704161c30 0x7f17041ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703e00640 1 -- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040b2650 msgr2=0x7f170415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703e00640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040b2650 0x7f170415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703e00640 1 -- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1704161c30 msgr2=0x7f17041ec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703e00640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1704161c30 0x7f17041ec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703e00640 1 -- 172.21.15.5:0/115608642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170415a8b0 con 0x7f1704162010 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f1703e00640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f17040af2c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c420 tx=0x7f16fc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc017070 con 0x7f1704162010 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.145+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc005ce0 con 0x7f1704162010 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.146+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0072d0 con 0x7f1704162010 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.146+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17041ea1f0 con 0x7f1704162010 2024-04-25T10:08:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.146+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17041ea7b0 con 0x7f1704162010 2024-04-25T10:08:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.147+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1704068f60 con 0x7f1704162010 2024-04-25T10:08:40.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.148+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16fc01a070 con 0x7f1704162010 2024-04-25T10:08:40.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.149+0000 7f16fb400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e406b4d0 0x7f16e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.149+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f16fc089630 con 0x7f1704162010 2024-04-25T10:08:40.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.150+0000 7f1703400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e406b4d0 0x7f16e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.151+0000 7f1703400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e406b4d0 0x7f16e406d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f16f4006fd0 tx=0x7f16f4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.156+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16fc057c70 con 0x7f1704162010 2024-04-25T10:08:40.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.351+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17040711d0 con 0x7f1704162010 2024-04-25T10:08:40.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.352+0000 7f16fb400640 1 -- 172.21.15.5:0/115608642 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16fc05bd10 con 0x7f1704162010 2024-04-25T10:08:40.358 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:08:40.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e406b4d0 msgr2=0x7f16e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e406b4d0 0x7f16e406d990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f16f4006fd0 tx=0x7f16f4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:40.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 msgr2=0x7f17040af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f17040af2c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c420 tx=0x7f16fc00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 shutdown_connections 2024-04-25T10:08:40.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1704162010 0x7f17040af2c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e406b4d0 0x7f16e406d990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1704161c30 0x7f17041ec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 --2- 172.21.15.5:0/115608642 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17040b2650 0x7f170415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 >> 172.21.15.5:0/115608642 conn(0x7f170406ab60 msgr2=0x7f17040b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 shutdown_connections 2024-04-25T10:08:40.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.355+0000 7f170a400640 1 -- 172.21.15.5:0/115608642 wait complete. 2024-04-25T10:08:40.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:40.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.591+0000 7f3caca00640 1 Processor -- start 2024-04-25T10:08:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.591+0000 7f3caca00640 1 -- start start 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3caca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ca80b48c0 0x7f3ca815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3caca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80d3f90 0x7f3ca8161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3caca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3ca8161bc0 0x7f3ca80d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3caca00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca81606e0 con 0x7f3ca80b48c0 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3caca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca81609c0 con 0x7f3ca8161bc0 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3caca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca8160ca0 con 0x7f3ca80d3f90 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ca80b48c0 0x7f3ca815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ca80b48c0 0x7f3ca815fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44460/0 (socket says 172.21.15.5:44460) 2024-04-25T10:08:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 -- 172.21.15.5:0/3017276594 learned_addr learned my addr 172.21.15.5:0/3017276594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 -- 172.21.15.5:0/3017276594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80d3f90 msgr2=0x7f3ca8161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 --2- 172.21.15.5:0/3017276594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80d3f90 0x7f3ca8161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 -- 172.21.15.5:0/3017276594 >> v2:172.21.15.80:3300/0 conn(0x7f3ca8161bc0 msgr2=0x7f3ca80d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 --2- 172.21.15.5:0/3017276594 >> v2:172.21.15.80:3300/0 conn(0x7f3ca8161bc0 0x7f3ca80d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.593+0000 7f3ca7400640 1 -- 172.21.15.5:0/3017276594 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8160f80 con 0x7f3ca80b48c0 2024-04-25T10:08:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.594+0000 7f3ca7400640 1 --2- 172.21.15.5:0/3017276594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca80b48c0 0x7f3ca815fe60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80d35b0 tx=0x7f3c9802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c646f93fa235855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.595+0000 7f3ca6000640 1 -- 172.21.15.5:0/3017276594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c98036070 con 0x7f3ca80b48c0 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.595+0000 7f3ca6000640 1 -- 172.21.15.5:0/3017276594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c98031600 con 0x7f3ca80b48c0 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.595+0000 7f3ca6000640 1 -- 172.21.15.5:0/3017276594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c980319d0 con 0x7f3ca80b48c0 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 -- 172.21.15.5:0/3017276594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca80b48c0 msgr2=0x7f3ca815fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 --2- 172.21.15.5:0/3017276594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca80b48c0 0x7f3ca815fe60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80d35b0 tx=0x7f3c9802a600 comp rx=0 tx=0).stop 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 -- 172.21.15.5:0/3017276594 shutdown_connections 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 --2- 172.21.15.5:0/3017276594 >> v2:172.21.15.80:3300/0 conn(0x7f3ca8161bc0 0x7f3ca80d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 --2- 172.21.15.5:0/3017276594 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80d3f90 0x7f3ca8161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 --2- 172.21.15.5:0/3017276594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca80b48c0 0x7f3ca815fe60 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 -- 172.21.15.5:0/3017276594 >> 172.21.15.5:0/3017276594 conn(0x7f3ca808af80 msgr2=0x7f3ca80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 -- 172.21.15.5:0/3017276594 shutdown_connections 2024-04-25T10:08:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.596+0000 7f3caca00640 1 -- 172.21.15.5:0/3017276594 wait complete. 2024-04-25T10:08:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.598+0000 7f3caca00640 1 Processor -- start 2024-04-25T10:08:40.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.598+0000 7f3caca00640 1 -- start start 2024-04-25T10:08:40.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3caca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80b48c0 0x7f3ca81f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3caca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ca80d3f90 0x7f3ca81f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3caca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 0x7f3ca81f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3caca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca81603a0 con 0x7f3ca8161bc0 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3caca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca80b4ca0 con 0x7f3ca80d3f90 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3caca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca80d1e00 con 0x7f3ca80b48c0 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 0x7f3ca81f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 0x7f3ca81f35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44464/0 (socket says 172.21.15.5:44464) 2024-04-25T10:08:40.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca7e00640 1 -- 172.21.15.5:0/1442390045 learned_addr learned my addr 172.21.15.5:0/1442390045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca7400640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80b48c0 0x7f3ca81f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca6a00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ca80d3f90 0x7f3ca81f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca7e00640 1 -- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80b48c0 msgr2=0x7f3ca81f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.599+0000 7f3ca7e00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80b48c0 0x7f3ca81f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3ca7e00640 1 -- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ca80d3f90 msgr2=0x7f3ca81f46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3ca7e00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ca80d3f90 0x7f3ca81f46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3ca7e00640 1 -- 172.21.15.5:0/1442390045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca81f4c20 con 0x7f3ca8161bc0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3ca6a00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ca80d3f90 0x7f3ca81f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3ca7e00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 0x7f3ca81f35b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b410 tx=0x7f3ca000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0015020 con 0x7f3ca8161bc0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca0011850 con 0x7f3ca8161bc0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011c40 con 0x7f3ca8161bc0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca81f3af0 con 0x7f3ca8161bc0 2024-04-25T10:08:40.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.600+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca81f5a00 con 0x7f3ca8161bc0 2024-04-25T10:08:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.602+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca8068f60 con 0x7f3ca8161bc0 2024-04-25T10:08:40.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.603+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ca0018020 con 0x7f3ca8161bc0 2024-04-25T10:08:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.603+0000 7f3c9f400640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3c8406b4d0 0x7f3c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.604+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3ca008a3c0 con 0x7f3ca8161bc0 2024-04-25T10:08:40.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.604+0000 7f3ca7400640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3c8406b4d0 0x7f3c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.605+0000 7f3ca7400640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3c8406b4d0 0x7f3c8406d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3c98033000 tx=0x7f3c98009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.608+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ca001f020 con 0x7f3ca8161bc0 2024-04-25T10:08:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.809+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ca80d0dd0 con 0x7f3ca8161bc0 2024-04-25T10:08:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.810+0000 7f3c9f400640 1 -- 172.21.15.5:0/1442390045 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ca81f3af0 con 0x7f3ca8161bc0 2024-04-25T10:08:40.815 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.812+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3c8406b4d0 msgr2=0x7f3c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.812+0000 7f3caca00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3c8406b4d0 0x7f3c8406d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3c98033000 tx=0x7f3c98009000 comp rx=0 tx=0).stop 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.812+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 msgr2=0x7f3ca81f35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.812+0000 7f3caca00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 0x7f3ca81f35b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000b410 tx=0x7f3ca000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 shutdown_connections 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ca8161bc0 0x7f3ca81f35b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3c8406b4d0 0x7f3c8406d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ca80d3f90 0x7f3ca81f46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 --2- 172.21.15.5:0/1442390045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ca80b48c0 0x7f3ca81f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 >> 172.21.15.5:0/1442390045 conn(0x7f3ca808af80 msgr2=0x7f3ca815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 shutdown_connections 2024-04-25T10:08:40.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:40.813+0000 7f3caca00640 1 -- 172.21.15.5:0/1442390045 wait complete. 2024-04-25T10:08:40.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:08:41.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.047+0000 7f78b1400640 1 Processor -- start 2024-04-25T10:08:41.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.047+0000 7f78b1400640 1 -- start start 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78b1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78ac141da0 0x7f78ac06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78b1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78b1400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78ac15d340 0x7f78ac0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78b1400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac142180 con 0x7f78ac141da0 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78b1400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac071d30 con 0x7f78ac15d340 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78b1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac06d620 con 0x7f78ac06e640 2024-04-25T10:08:41.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.048+0000 7f78abe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78ac141da0 0x7f78ac06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78ac141da0 0x7f78ac06e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44468/0 (socket says 172.21.15.5:44468) 2024-04-25T10:08:41.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 -- 172.21.15.5:0/1261160990 learned_addr learned my addr 172.21.15.5:0/1261160990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:41.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78a3e00640 1 --2- 172.21.15.5:0/1261160990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78b0a00640 1 --2- 172.21.15.5:0/1261160990 >> v2:172.21.15.80:3300/0 conn(0x7f78ac15d340 0x7f78ac0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 -- 172.21.15.5:0/1261160990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 msgr2=0x7f78ac15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 --2- 172.21.15.5:0/1261160990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac15ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 -- 172.21.15.5:0/1261160990 >> v2:172.21.15.80:3300/0 conn(0x7f78ac15d340 msgr2=0x7f78ac0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 --2- 172.21.15.5:0/1261160990 >> v2:172.21.15.80:3300/0 conn(0x7f78ac15d340 0x7f78ac0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.049+0000 7f78abe00640 1 -- 172.21.15.5:0/1261160990 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac06d900 con 0x7f78ac141da0 2024-04-25T10:08:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.050+0000 7f78abe00640 1 --2- 172.21.15.5:0/1261160990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac06e100 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0b2460 tx=0x7f789c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9d42da2aac8635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.050+0000 7f78ab400640 1 -- 172.21.15.5:0/1261160990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c037070 con 0x7f78ac141da0 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.050+0000 7f78ab400640 1 -- 172.21.15.5:0/1261160990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c032440 con 0x7f78ac141da0 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.050+0000 7f78ab400640 1 -- 172.21.15.5:0/1261160990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c032810 con 0x7f78ac141da0 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 -- 172.21.15.5:0/1261160990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 msgr2=0x7f78ac06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 --2- 172.21.15.5:0/1261160990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac06e100 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0b2460 tx=0x7f789c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 -- 172.21.15.5:0/1261160990 shutdown_connections 2024-04-25T10:08:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 --2- 172.21.15.5:0/1261160990 >> v2:172.21.15.80:3300/0 conn(0x7f78ac15d340 0x7f78ac0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 --2- 172.21.15.5:0/1261160990 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 --2- 172.21.15.5:0/1261160990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac06e100 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 -- 172.21.15.5:0/1261160990 >> 172.21.15.5:0/1261160990 conn(0x7f78ac066f20 msgr2=0x7f78ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 -- 172.21.15.5:0/1261160990 shutdown_connections 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.051+0000 7f78b1400640 1 -- 172.21.15.5:0/1261160990 wait complete. 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.053+0000 7f78b1400640 1 Processor -- start 2024-04-25T10:08:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.053+0000 7f78b1400640 1 -- start start 2024-04-25T10:08:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78b1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac12fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78b1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac15d340 0x7f78ac15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78b1400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0cd9c0 con 0x7f78ac141da0 2024-04-25T10:08:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78b1400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0cd840 con 0x7f78ac15d340 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78b1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0cdb40 con 0x7f78ac06e640 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac12fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac12fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44474/0 (socket says 172.21.15.5:44474) 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.054+0000 7f78a3e00640 1 -- 172.21.15.5:0/3458055490 learned_addr learned my addr 172.21.15.5:0/3458055490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a3e00640 1 -- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 msgr2=0x7f78ac0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a3e00640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a3e00640 1 -- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac15d340 msgr2=0x7f78ac15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a3e00640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac15d340 0x7f78ac15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a3e00640 1 -- 172.21.15.5:0/3458055490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac13bc80 con 0x7f78ac141da0 2024-04-25T10:08:41.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a3e00640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac12fbc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f789400c820 tx=0x7f789400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894004030 con 0x7f78ac141da0 2024-04-25T10:08:41.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78940041d0 con 0x7f78ac141da0 2024-04-25T10:08:41.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894007830 con 0x7f78ac141da0 2024-04-25T10:08:41.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.055+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac13c840 con 0x7f78ac141da0 2024-04-25T10:08:41.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.056+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac0b0b40 con 0x7f78ac141da0 2024-04-25T10:08:41.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.057+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac068f60 con 0x7f78ac141da0 2024-04-25T10:08:41.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.058+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78940079d0 con 0x7f78ac141da0 2024-04-25T10:08:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.059+0000 7f78a8c00640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f788806b4d0 0x7f788806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.059+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f789408a420 con 0x7f78ac141da0 2024-04-25T10:08:41.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.059+0000 7f78abe00640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f788806b4d0 0x7f788806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.060+0000 7f78abe00640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f788806b4d0 0x7f788806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f789c034000 tx=0x7f789c02c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.063+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7894015020 con 0x7f78ac141da0 2024-04-25T10:08:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.278+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78ac0ccae0 con 0x7f78ac141da0 2024-04-25T10:08:41.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.279+0000 7f78a8c00640 1 -- 172.21.15.5:0/3458055490 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) v1 ==== 80+0+122 (secure 0 0 0) 0x7f7894059020 con 0x7f78ac141da0 2024-04-25T10:08:41.285 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6835/1098619647 2024-04-26T10:07:22.531455+0000 2024-04-25T10:08:41.285 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6834/1098619647 2024-04-26T10:07:22.531455+0000 2024-04-25T10:08:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f788806b4d0 msgr2=0x7f788806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f788806b4d0 0x7f788806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f789c034000 tx=0x7f789c02c000 comp rx=0 tx=0).stop 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 msgr2=0x7f78ac12fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac12fbc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f789400c820 tx=0x7f789400cce0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 shutdown_connections 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f788806b4d0 0x7f788806d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac15d340 0x7f78ac15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac141da0 0x7f78ac12fbc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 --2- 172.21.15.5:0/3458055490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac06e640 0x7f78ac0b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 >> 172.21.15.5:0/3458055490 conn(0x7f78ac066f20 msgr2=0x7f78ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 shutdown_connections 2024-04-25T10:08:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.282+0000 7f78b1400640 1 -- 172.21.15.5:0/3458055490 wait complete. 2024-04-25T10:08:41.306 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:08:41.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:41.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:41.398 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:08:41.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:41.398 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:08:41.453 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:08:41.454 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:41.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:41.511 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:08:41.512 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:41.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:41.572 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:08:41.573 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:41.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:41.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.844+0000 7fd3b3000640 1 Processor -- start 2024-04-25T10:08:41.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.844+0000 7fd3b3000640 1 -- start start 2024-04-25T10:08:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b3000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3ac15ce00 0x7fd3ac15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b3000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd3ac0cd300 0x7fd3ac0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b3000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ac071d30 con 0x7fd3ac141da0 2024-04-25T10:08:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b3000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac0cdcd0 con 0x7fd3ac15ce00 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b3000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac142180 con 0x7fd3ac0cd300 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b1200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3ac15ce00 0x7fd3ac15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b1200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3ac15ce00 0x7fd3ac15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44482/0 (socket says 172.21.15.5:44482) 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.846+0000 7fd3b1200640 1 -- 172.21.15.5:0/3930847887 learned_addr learned my addr 172.21.15.5:0/3930847887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.847+0000 7fd3b2600640 1 --2- 172.21.15.5:0/3930847887 >> v2:172.21.15.80:3300/0 conn(0x7fd3ac0cd300 0x7fd3ac0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.847+0000 7fd3b1200640 1 -- 172.21.15.5:0/3930847887 >> v2:172.21.15.80:3300/0 conn(0x7fd3ac0cd300 msgr2=0x7fd3ac0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.847+0000 7fd3b1200640 1 --2- 172.21.15.5:0/3930847887 >> v2:172.21.15.80:3300/0 conn(0x7fd3ac0cd300 0x7fd3ac0cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.847+0000 7fd3b1200640 1 -- 172.21.15.5:0/3930847887 >> v1:172.21.15.80:6789/0 conn(0x7fd3ac141da0 legacy=0x7fd3ac06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.847+0000 7fd3b1200640 1 -- 172.21.15.5:0/3930847887 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac06d620 con 0x7fd3ac15ce00 2024-04-25T10:08:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.848+0000 7fd3b1200640 1 --2- 172.21.15.5:0/3930847887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac15d280 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac15f220 tx=0x7fd39802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d602846ca68138df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.848+0000 7fd3abe00640 1 -- 172.21.15.5:0/3930847887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398037070 con 0x7fd3ac15ce00 2024-04-25T10:08:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.848+0000 7fd3abe00640 1 -- 172.21.15.5:0/3930847887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd398032440 con 0x7fd3ac15ce00 2024-04-25T10:08:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.848+0000 7fd3abe00640 1 -- 172.21.15.5:0/3930847887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398032810 con 0x7fd3ac15ce00 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 -- 172.21.15.5:0/3930847887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 msgr2=0x7fd3ac15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3930847887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac15d280 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac15f220 tx=0x7fd39802a230 comp rx=0 tx=0).stop 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 -- 172.21.15.5:0/3930847887 shutdown_connections 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3930847887 >> v2:172.21.15.80:3300/0 conn(0x7fd3ac0cd300 0x7fd3ac0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3930847887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac15d280 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 -- 172.21.15.5:0/3930847887 >> 172.21.15.5:0/3930847887 conn(0x7fd3ac066f20 msgr2=0x7fd3ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 -- 172.21.15.5:0/3930847887 shutdown_connections 2024-04-25T10:08:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.849+0000 7fd3b3000640 1 -- 172.21.15.5:0/3930847887 wait complete. 2024-04-25T10:08:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.851+0000 7fd3b3000640 1 Processor -- start 2024-04-25T10:08:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.851+0000 7fd3b3000640 1 -- start start 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b3000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ac0cd300 0x7fd3ac13a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b3000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ac141da0 0x7fd3ac143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b3000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b3000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac06e810 con 0x7fd3ac15ce00 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b3000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac06dbe0 con 0x7fd3ac0cd300 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b3000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac071000 con 0x7fd3ac141da0 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac142300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44490/0 (socket says 172.21.15.5:44490) 2024-04-25T10:08:41.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.852+0000 7fd3b2600640 1 -- 172.21.15.5:0/3401642276 learned_addr learned my addr 172.21.15.5:0/3401642276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b1c00640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ac0cd300 0x7fd3ac13a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b1200640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ac141da0 0x7fd3ac143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b2600640 1 -- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ac141da0 msgr2=0x7fd3ac143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b2600640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ac141da0 0x7fd3ac143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b2600640 1 -- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ac0cd300 msgr2=0x7fd3ac13a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b2600640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ac0cd300 0x7fd3ac13a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b2600640 1 -- 172.21.15.5:0/3401642276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac142e70 con 0x7fd3ac15ce00 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b1c00640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ac0cd300 0x7fd3ac13a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.853+0000 7fd3b2600640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac142300 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4007e40 tx=0x7fd3a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.854+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4014620 con 0x7fd3ac15ce00 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.854+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a4019070 con 0x7fd3ac15ce00 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.854+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4014a20 con 0x7fd3ac15ce00 2024-04-25T10:08:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.854+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac143150 con 0x7fd3ac15ce00 2024-04-25T10:08:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.854+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac143710 con 0x7fd3ac15ce00 2024-04-25T10:08:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.855+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac068f60 con 0x7fd3ac15ce00 2024-04-25T10:08:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.856+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3a4014bc0 con 0x7fd3ac15ce00 2024-04-25T10:08:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.857+0000 7fd3a9600640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd38c06b4d0 0x7fd38c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.857+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd3a408adc0 con 0x7fd3ac15ce00 2024-04-25T10:08:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.858+0000 7fd3b1c00640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd38c06b4d0 0x7fd38c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.859+0000 7fd3b1c00640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd38c06b4d0 0x7fd38c06d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0b1aa0 tx=0x7fd3a0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:41.861+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3a40599c0 con 0x7fd3ac15ce00 2024-04-25T10:08:42.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.107+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ac06ac50 con 0x7fd3ac15ce00 2024-04-25T10:08:42.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.109+0000 7fd3a9600640 1 -- 172.21.15.5:0/3401642276 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4930 (secure 0 0 0) 0x7fd3a40592c0 con 0x7fd3ac15ce00 2024-04-25T10:08:42.115 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:42.115 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:08:09.396742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 53 2024-04-25T10:08:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd38c06b4d0 msgr2=0x7fd38c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd38c06b4d0 0x7fd38c06d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0b1aa0 tx=0x7fd3a0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 msgr2=0x7fd3ac142300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac142300 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4007e40 tx=0x7fd3a400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 shutdown_connections 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ac15ce00 0x7fd3ac142300 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ac141da0 0x7fd3ac143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd38c06b4d0 0x7fd38c06d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 --2- 172.21.15.5:0/3401642276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ac0cd300 0x7fd3ac13a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 >> 172.21.15.5:0/3401642276 conn(0x7fd3ac066f20 msgr2=0x7fd3ac073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 shutdown_connections 2024-04-25T10:08:42.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.112+0000 7fd3b3000640 1 -- 172.21.15.5:0/3401642276 wait complete. 2024-04-25T10:08:42.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.343+0000 7f184ae00640 1 Processor -- start 2024-04-25T10:08:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.343+0000 7f184ae00640 1 -- start start 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f184ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 0x7f1844070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f184ae00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18440714d0 0x7f184406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f184ae00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440cd2a0 con 0x7f18441652f0 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f184ae00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440ce340 con 0x7f18440714d0 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f184ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440cdeb0 con 0x7f184406e980 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f1849a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f18441652f0 0x7f18440cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40578/0 (socket says 172.21.15.5:40578) 2024-04-25T10:08:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.345+0000 7f1849a00640 1 -- 172.21.15.5:0/813710104 learned_addr learned my addr 172.21.15.5:0/813710104 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f184a400640 1 --2- 172.21.15.5:0/813710104 >> v2:172.21.15.80:3300/0 conn(0x7f18440714d0 0x7f184406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456841520 0 0) 0x7f18440cd2a0 con 0x7f18441652f0 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f182c003520 con 0x7f18441652f0 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1849000640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 0x7f1844070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2667493987 0 0) 0x7f182c003520 con 0x7f18441652f0 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 msgr2=0x7f1844070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 0x7f1844070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 >> v2:172.21.15.80:3300/0 conn(0x7f18440714d0 msgr2=0x7f184406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 --2- 172.21.15.5:0/813710104 >> v2:172.21.15.80:3300/0 conn(0x7f18440714d0 0x7f184406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440cd580 con 0x7f18441652f0 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1838001670 con 0x7f18441652f0 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 >> v1:172.21.15.5:6789/0 conn(0x7f18441652f0 legacy=0x7f18440cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0039b0 0x7f182c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.346+0000 7f1843e00640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 0x7f182c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f1843e00640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f182c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440ce340 con 0x7f182c004d40 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440cdeb0 con 0x7f182c0039b0 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c00b9c0 con 0x7f184406e980 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f184a400640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f182c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f1849a00640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0039b0 0x7f182c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f1849000640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 0x7f182c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f184a400640 1 -- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 msgr2=0x7f182c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f184a400640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f184406e980 0x7f182c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f184a400640 1 -- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0039b0 msgr2=0x7f182c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f184a400640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0039b0 0x7f182c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.347+0000 7f184a400640 1 -- 172.21.15.5:0/813710104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c003520 con 0x7f182c004d40 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.348+0000 7f184a400640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f182c0051c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e170 tx=0x7f182c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b85b25fdf22649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.349+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c034020 con 0x7f182c004d40 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.349+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c030430 con 0x7f182c004d40 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.349+0000 7f1843e00640 1 -- 172.21.15.5:0/813710104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c009c10 con 0x7f182c004d40 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.349+0000 7f184ae00640 1 -- 172.21.15.5:0/813710104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 msgr2=0x7f182c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.349+0000 7f184ae00640 1 --2- 172.21.15.5:0/813710104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f182c0051c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e170 tx=0x7f182c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.349+0000 7f184a400640 1 -- 172.21.15.5:0/813710104 reap_dead start 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.350+0000 7f184ae00640 1 -- 172.21.15.5:0/813710104 shutdown_connections 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.350+0000 7f184ae00640 1 -- 172.21.15.5:0/813710104 >> 172.21.15.5:0/813710104 conn(0x7f1844066f20 msgr2=0x7f184406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.350+0000 7f184ae00640 1 -- 172.21.15.5:0/813710104 shutdown_connections 2024-04-25T10:08:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.350+0000 7f184ae00640 1 -- 172.21.15.5:0/813710104 wait complete. 2024-04-25T10:08:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.351+0000 7f184ae00640 1 Processor -- start 2024-04-25T10:08:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.352+0000 7f184ae00640 1 -- start start 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.352+0000 7f184ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f18441634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.352+0000 7f184ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1844159740 0x7f1844164100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.352+0000 7f184ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184414f400 0x7f18441f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f184ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1844070f90 con 0x7f182c004d40 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f184ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440cdb40 con 0x7f184414f400 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f184ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1844071110 con 0x7f1844159740 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f18441634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f18441634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44504/0 (socket says 172.21.15.5:44504) 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 -- 172.21.15.5:0/208122169 learned_addr learned my addr 172.21.15.5:0/208122169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 -- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1844159740 msgr2=0x7f1844164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1844159740 0x7f1844164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 -- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184414f400 msgr2=0x7f18441f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184414f400 0x7f18441f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.353+0000 7f1849a00640 1 -- 172.21.15.5:0/208122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844164640 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.354+0000 7f1849a00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f18441634f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f183800e840 tx=0x7f183800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.354+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800c7d0 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.354+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1838018070 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.354+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800cbb0 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.354+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1844162730 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.354+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1844162cf0 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.355+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18440cce40 con 0x7f182c004d40 2024-04-25T10:08:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.356+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f183801d460 con 0x7f182c004d40 2024-04-25T10:08:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.357+0000 7f1841600640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182406b500 0x7f182406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.357+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f183808a280 con 0x7f182c004d40 2024-04-25T10:08:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.358+0000 7f1849000640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182406b500 0x7f182406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.359+0000 7f1849000640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182406b500 0x7f182406d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f184414fd80 tx=0x7f183000b2e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.361+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1838058f10 con 0x7f182c004d40 2024-04-25T10:08:42.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.565+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18440ae8f0 con 0x7f182c004d40 2024-04-25T10:08:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.567+0000 7f1841600640 1 -- 172.21.15.5:0/208122169 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16906 (secure 0 0 0) 0x7f1838058810 con 0x7f182c004d40 2024-04-25T10:08:42.573 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:42.573 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:22.531478+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1098619647":"2024-04-26T10:07:22.531455+0000","172.21.15.80:6834/1098619647":"2024-04-26T10:07:22.531455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.569+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182406b500 msgr2=0x7f182406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.569+0000 7f184ae00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182406b500 0x7f182406d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f184414fd80 tx=0x7f183000b2e0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.569+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 msgr2=0x7f18441634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.569+0000 7f184ae00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f18441634f0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f183800e840 tx=0x7f183800ed00 comp rx=0 tx=0).stop 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 shutdown_connections 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f182406b500 0x7f182406d9c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f184414f400 0x7f18441f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1844159740 0x7f1844164100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 --2- 172.21.15.5:0/208122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f182c004d40 0x7f18441634f0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 >> 172.21.15.5:0/208122169 conn(0x7f1844066f20 msgr2=0x7f18440b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 shutdown_connections 2024-04-25T10:08:42.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.570+0000 7f184ae00640 1 -- 172.21.15.5:0/208122169 wait complete. 2024-04-25T10:08:42.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:07.362188+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:08.532434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:42.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.795+0000 7f36b2800640 1 Processor -- start 2024-04-25T10:08:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.796+0000 7f36b2800640 1 -- start start 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.797+0000 7f36b2800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36ac143f30 0x7f36ac06da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.797+0000 7f36b2800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36ac06ec70 0x7f36ac071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.797+0000 7f36b2800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0ade10 con 0x7f36ac0aebd0 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.797+0000 7f36b2800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0ae0f0 con 0x7f36ac06ec70 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b2800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0ae3d0 con 0x7f36ac143f30 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b0a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36ac06ec70 0x7f36ac071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b1e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36ac0aebd0 0x7f36ac161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40582/0 (socket says 172.21.15.5:40582) 2024-04-25T10:08:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b1e00640 1 -- 172.21.15.5:0/2878251302 learned_addr learned my addr 172.21.15.5:0/2878251302 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b1400640 1 --2- 172.21.15.5:0/2878251302 >> v2:172.21.15.80:3300/0 conn(0x7f36ac143f30 0x7f36ac06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b0a00640 1 -- 172.21.15.5:0/2878251302 >> v2:172.21.15.80:3300/0 conn(0x7f36ac143f30 msgr2=0x7f36ac06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b0a00640 1 --2- 172.21.15.5:0/2878251302 >> v2:172.21.15.80:3300/0 conn(0x7f36ac143f30 0x7f36ac06da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b0a00640 1 -- 172.21.15.5:0/2878251302 >> v1:172.21.15.5:6789/0 conn(0x7f36ac0aebd0 legacy=0x7f36ac161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b0a00640 1 -- 172.21.15.5:0/2878251302 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0ae6b0 con 0x7f36ac06ec70 2024-04-25T10:08:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.798+0000 7f36b1400640 1 --2- 172.21.15.5:0/2878251302 >> v2:172.21.15.80:3300/0 conn(0x7f36ac143f30 0x7f36ac06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.799+0000 7f36b0a00640 1 --2- 172.21.15.5:0/2878251302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac06ec70 0x7f36ac071000 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f36ac072480 tx=0x7f369c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209085ddde47af3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36abe00640 1 -- 172.21.15.5:0/2878251302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c037070 con 0x7f36ac06ec70 2024-04-25T10:08:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36abe00640 1 -- 172.21.15.5:0/2878251302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c032440 con 0x7f36ac06ec70 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36abe00640 1 -- 172.21.15.5:0/2878251302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c02ae60 con 0x7f36ac06ec70 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36b2800640 1 -- 172.21.15.5:0/2878251302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac06ec70 msgr2=0x7f36ac071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36b2800640 1 --2- 172.21.15.5:0/2878251302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac06ec70 0x7f36ac071000 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f36ac072480 tx=0x7f369c02a110 comp rx=0 tx=0).stop 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36b2800640 1 -- 172.21.15.5:0/2878251302 shutdown_connections 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36b2800640 1 --2- 172.21.15.5:0/2878251302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac06ec70 0x7f36ac071000 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36b2800640 1 --2- 172.21.15.5:0/2878251302 >> v2:172.21.15.80:3300/0 conn(0x7f36ac143f30 0x7f36ac06da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.800+0000 7f36b2800640 1 -- 172.21.15.5:0/2878251302 >> 172.21.15.5:0/2878251302 conn(0x7f36ac066f20 msgr2=0x7f36ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.801+0000 7f36b2800640 1 -- 172.21.15.5:0/2878251302 shutdown_connections 2024-04-25T10:08:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.801+0000 7f36b2800640 1 -- 172.21.15.5:0/2878251302 wait complete. 2024-04-25T10:08:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.802+0000 7f36b2800640 1 Processor -- start 2024-04-25T10:08:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.802+0000 7f36b2800640 1 -- start start 2024-04-25T10:08:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.803+0000 7f36b2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36ac06ec70 0x7f36ac1f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.803+0000 7f36b2800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36ac0aebd0 0x7f36ac1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.803+0000 7f36b2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 0x7f36ac140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.803+0000 7f36b2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac06e0f0 con 0x7f36ac143f30 2024-04-25T10:08:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.803+0000 7f36b2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac06df70 con 0x7f36ac06ec70 2024-04-25T10:08:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.803+0000 7f36b2800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac071540 con 0x7f36ac0aebd0 2024-04-25T10:08:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 0x7f36ac140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 0x7f36ac140b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44520/0 (socket says 172.21.15.5:44520) 2024-04-25T10:08:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 -- 172.21.15.5:0/472969048 learned_addr learned my addr 172.21.15.5:0/472969048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1400640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36ac06ec70 0x7f36ac1f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b0a00640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36ac0aebd0 0x7f36ac1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 -- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36ac0aebd0 msgr2=0x7f36ac1f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36ac0aebd0 0x7f36ac1f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 -- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36ac06ec70 msgr2=0x7f36ac1f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36ac06ec70 0x7f36ac1f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b1e00640 1 -- 172.21.15.5:0/472969048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0c4430 con 0x7f36ac143f30 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.804+0000 7f36b0a00640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36ac0aebd0 0x7f36ac1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.805+0000 7f36b1e00640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 0x7f36ac140b30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f36a400cb20 tx=0x7f36a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.805+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4017070 con 0x7f36ac143f30 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.805+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a400ac40 con 0x7f36ac143f30 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.805+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a4012400 con 0x7f36ac143f30 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.805+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac0c46d0 con 0x7f36ac143f30 2024-04-25T10:08:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.805+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac142230 con 0x7f36ac143f30 2024-04-25T10:08:42.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.806+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac068f60 con 0x7f36ac143f30 2024-04-25T10:08:42.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.807+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36a401a070 con 0x7f36ac143f30 2024-04-25T10:08:42.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.809+0000 7f36a9600640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.810+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f36a408a060 con 0x7f36ac143f30 2024-04-25T10:08:42.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.810+0000 7f36b1400640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.811+0000 7f36b1400640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369006b4d0 0x7f369006d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f36a000a950 tx=0x7f36a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:42.814+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36a4058610 con 0x7f36ac143f30 2024-04-25T10:08:43.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.017+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36ac071320 con 0x7f36ac143f30 2024-04-25T10:08:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.017+0000 7f36a9600640 1 -- 172.21.15.5:0/472969048 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36a405c6b0 con 0x7f36ac143f30 2024-04-25T10:08:43.023 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:08:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369006b4d0 msgr2=0x7f369006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369006b4d0 0x7f369006d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f36a000a950 tx=0x7f36a0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 msgr2=0x7f36ac140b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 0x7f36ac140b30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f36a400cb20 tx=0x7f36a400a060 comp rx=0 tx=0).stop 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 shutdown_connections 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36ac143f30 0x7f36ac140b30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36ac0aebd0 0x7f36ac1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f369006b4d0 0x7f369006d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 --2- 172.21.15.5:0/472969048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36ac06ec70 0x7f36ac1f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.020+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 >> 172.21.15.5:0/472969048 conn(0x7f36ac066f20 msgr2=0x7f36ac0b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.021+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 shutdown_connections 2024-04-25T10:08:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.021+0000 7f36b2800640 1 -- 172.21.15.5:0/472969048 wait complete. 2024-04-25T10:08:43.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.249+0000 7fb32a800640 1 Processor -- start 2024-04-25T10:08:43.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.249+0000 7fb32a800640 1 -- start start 2024-04-25T10:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.251+0000 7fb32a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb3240b1ab0 0x7fb3240afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.251+0000 7fb32a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3240b0420 0x7fb32406af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.251+0000 7fb32a800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3240d1220 con 0x7fb32406ab60 2024-04-25T10:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.251+0000 7fb32a800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3240b5130 con 0x7fb3240b1ab0 2024-04-25T10:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.251+0000 7fb32a800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240afcc0 con 0x7fb3240b0420 2024-04-25T10:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb329400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb32406ab60 0x7fb3240d0b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40594/0 (socket says 172.21.15.5:40594) 2024-04-25T10:08:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb329400640 1 -- 172.21.15.5:0/447905103 learned_addr learned my addr 172.21.15.5:0/447905103 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb328a00640 1 --2- 172.21.15.5:0/447905103 >> v2:172.21.15.5:3300/0 conn(0x7fb3240b1ab0 0x7fb3240afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb329e00640 1 --2- 172.21.15.5:0/447905103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3240b0420 0x7fb32406af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb323e00640 1 -- 172.21.15.5:0/447905103 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209012617 0 0) 0x7fb3240d1220 con 0x7fb32406ab60 2024-04-25T10:08:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb323e00640 1 -- 172.21.15.5:0/447905103 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb30c003520 con 0x7fb32406ab60 2024-04-25T10:08:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb328a00640 1 -- 172.21.15.5:0/447905103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3240b0420 msgr2=0x7fb32406af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb328a00640 1 --2- 172.21.15.5:0/447905103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3240b0420 0x7fb32406af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb328a00640 1 -- 172.21.15.5:0/447905103 >> v1:172.21.15.5:6789/0 conn(0x7fb32406ab60 legacy=0x7fb3240d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.252+0000 7fb328a00640 1 -- 172.21.15.5:0/447905103 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240b5360 con 0x7fb3240b1ab0 2024-04-25T10:08:43.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.254+0000 7fb328a00640 1 --2- 172.21.15.5:0/447905103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb3240afee0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb3100097a0 tx=0x7fb31002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6c58fc7eddab84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.254+0000 7fb323e00640 1 -- 172.21.15.5:0/447905103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310037070 con 0x7fb3240b1ab0 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.254+0000 7fb323e00640 1 -- 172.21.15.5:0/447905103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb310032440 con 0x7fb3240b1ab0 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.254+0000 7fb323e00640 1 -- 172.21.15.5:0/447905103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310032810 con 0x7fb3240b1ab0 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 -- 172.21.15.5:0/447905103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 msgr2=0x7fb3240afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 --2- 172.21.15.5:0/447905103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb3240afee0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb3100097a0 tx=0x7fb31002a180 comp rx=0 tx=0).stop 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 -- 172.21.15.5:0/447905103 shutdown_connections 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 --2- 172.21.15.5:0/447905103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb3240b0420 0x7fb32406af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 --2- 172.21.15.5:0/447905103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb3240afee0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 -- 172.21.15.5:0/447905103 >> 172.21.15.5:0/447905103 conn(0x7fb3240b5ab0 msgr2=0x7fb3240999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 -- 172.21.15.5:0/447905103 shutdown_connections 2024-04-25T10:08:43.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.255+0000 7fb32a800640 1 -- 172.21.15.5:0/447905103 wait complete. 2024-04-25T10:08:43.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.257+0000 7fb32a800640 1 Processor -- start 2024-04-25T10:08:43.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.257+0000 7fb32a800640 1 -- start start 2024-04-25T10:08:43.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb32a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb32406ab60 0x7fb32415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb32a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3240b0420 0x7fb32413b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb32a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb32413be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb32a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240d0310 con 0x7fb3240b1ab0 2024-04-25T10:08:43.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb32a800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240d00c0 con 0x7fb3240b0420 2024-04-25T10:08:43.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb32a800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240d05f0 con 0x7fb32406ab60 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb329e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb32413be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb329e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb32413be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44542/0 (socket says 172.21.15.5:44542) 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.258+0000 7fb329e00640 1 -- 172.21.15.5:0/1953156185 learned_addr learned my addr 172.21.15.5:0/1953156185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb328a00640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3240b0420 0x7fb32413b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb329e00640 1 -- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb32406ab60 msgr2=0x7fb32415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb329e00640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb32406ab60 0x7fb32415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb329e00640 1 -- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3240b0420 msgr2=0x7fb32413b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb329e00640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3240b0420 0x7fb32413b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb329e00640 1 -- 172.21.15.5:0/1953156185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324155200 con 0x7fb3240b1ab0 2024-04-25T10:08:43.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb328a00640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3240b0420 0x7fb32413b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:43.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb329e00640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb32413be70 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009dd0 tx=0x7fb31c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c018020 con 0x7fb3240b1ab0 2024-04-25T10:08:43.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c014480 con 0x7fb3240b1ab0 2024-04-25T10:08:43.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32413c7f0 con 0x7fb3240b1ab0 2024-04-25T10:08:43.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.259+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c014870 con 0x7fb3240b1ab0 2024-04-25T10:08:43.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.260+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32413cdb0 con 0x7fb3240b1ab0 2024-04-25T10:08:43.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.261+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3240af4d0 con 0x7fb3240b1ab0 2024-04-25T10:08:43.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.262+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb31c005d10 con 0x7fb3240b1ab0 2024-04-25T10:08:43.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.263+0000 7fb321600640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb30406b4d0 0x7fb30406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.263+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb31c08af90 con 0x7fb3240b1ab0 2024-04-25T10:08:43.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.263+0000 7fb329400640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb30406b4d0 0x7fb30406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.264+0000 7fb329400640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb30406b4d0 0x7fb30406d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb318000f60 tx=0x7fb3180078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.266+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3240af4d0 con 0x7fb3240b1ab0 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.464+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb324068f60 con 0x7fb3240b1ab0 2024-04-25T10:08:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.465+0000 7fb321600640 1 -- 172.21.15.5:0/1953156185 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb31c022070 con 0x7fb3240b1ab0 2024-04-25T10:08:43.471 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:08:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb30406b4d0 msgr2=0x7fb30406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb30406b4d0 0x7fb30406d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb318000f60 tx=0x7fb3180078f0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 msgr2=0x7fb32413be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb32413be70 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009dd0 tx=0x7fb31c00c490 comp rx=0 tx=0).stop 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 shutdown_connections 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3240b1ab0 0x7fb32413be70 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb30406b4d0 0x7fb30406d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3240b0420 0x7fb32413b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 --2- 172.21.15.5:0/1953156185 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb32406ab60 0x7fb32415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.468+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 >> 172.21.15.5:0/1953156185 conn(0x7fb3240b5ab0 msgr2=0x7fb324073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.469+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 shutdown_connections 2024-04-25T10:08:43.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.469+0000 7fb32a800640 1 -- 172.21.15.5:0/1953156185 wait complete. 2024-04-25T10:08:43.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.695+0000 7f476ac00640 1 Processor -- start 2024-04-25T10:08:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.696+0000 7f476ac00640 1 -- start start 2024-04-25T10:08:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.697+0000 7f476ac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47640b48c0 0x7f476415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.697+0000 7f476ac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4764161d90 0x7f47640b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.697+0000 7f476ac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47641606e0 con 0x7f47640d3f90 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.697+0000 7f476ac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47641609c0 con 0x7f47640b48c0 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.697+0000 7f476ac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4764160ca0 con 0x7f4764161d90 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47640b48c0 0x7f476415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47640b48c0 0x7f476415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44546/0 (socket says 172.21.15.5:44546) 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 -- 172.21.15.5:0/4079559552 learned_addr learned my addr 172.21.15.5:0/4079559552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f476a200640 1 --2- 172.21.15.5:0/4079559552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4764161d90 0x7f47640b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 -- 172.21.15.5:0/4079559552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4764161d90 msgr2=0x7f47640b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 --2- 172.21.15.5:0/4079559552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4764161d90 0x7f47640b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 -- 172.21.15.5:0/4079559552 >> v1:172.21.15.80:6789/0 conn(0x7f47640d3f90 legacy=0x7f4764161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.698+0000 7f4769800640 1 -- 172.21.15.5:0/4079559552 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764160f80 con 0x7f47640b48c0 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.699+0000 7f4769800640 1 --2- 172.21.15.5:0/4079559552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47640b48c0 0x7f476415fe60 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4758009870 tx=0x7f475802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d6dd5b77658ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.700+0000 7f4763e00640 1 -- 172.21.15.5:0/4079559552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758037070 con 0x7f47640b48c0 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.700+0000 7f4763e00640 1 -- 172.21.15.5:0/4079559552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758032440 con 0x7f47640b48c0 2024-04-25T10:08:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.700+0000 7f4763e00640 1 -- 172.21.15.5:0/4079559552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758032810 con 0x7f47640b48c0 2024-04-25T10:08:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.700+0000 7f476ac00640 1 -- 172.21.15.5:0/4079559552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47640b48c0 msgr2=0x7f476415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.700+0000 7f476ac00640 1 --2- 172.21.15.5:0/4079559552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47640b48c0 0x7f476415fe60 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4758009870 tx=0x7f475802a180 comp rx=0 tx=0).stop 2024-04-25T10:08:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.701+0000 7f476ac00640 1 -- 172.21.15.5:0/4079559552 shutdown_connections 2024-04-25T10:08:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.701+0000 7f476ac00640 1 --2- 172.21.15.5:0/4079559552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4764161d90 0x7f47640b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.701+0000 7f476ac00640 1 --2- 172.21.15.5:0/4079559552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47640b48c0 0x7f476415fe60 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.701+0000 7f476ac00640 1 -- 172.21.15.5:0/4079559552 >> 172.21.15.5:0/4079559552 conn(0x7f476408af80 msgr2=0x7f47640d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.701+0000 7f476ac00640 1 -- 172.21.15.5:0/4079559552 shutdown_connections 2024-04-25T10:08:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.701+0000 7f476ac00640 1 -- 172.21.15.5:0/4079559552 wait complete. 2024-04-25T10:08:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.703+0000 7f476ac00640 1 Processor -- start 2024-04-25T10:08:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.703+0000 7f476ac00640 1 -- start start 2024-04-25T10:08:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476ac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47640b48c0 0x7f4764150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476ac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47640d3f90 0x7f47640ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476ac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 0x7f47640cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476ac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47641603a0 con 0x7f4764161d90 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476ac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640b4ca0 con 0x7f47640b48c0 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476ac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47640b30b0 con 0x7f47640d3f90 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 0x7f47640cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 0x7f47640cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44562/0 (socket says 172.21.15.5:44562) 2024-04-25T10:08:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f476a200640 1 -- 172.21.15.5:0/828838812 learned_addr learned my addr 172.21.15.5:0/828838812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f4769800640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47640b48c0 0x7f4764150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.705+0000 7f4768e00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47640d3f90 0x7f47640ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476a200640 1 -- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47640d3f90 msgr2=0x7f47640ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476a200640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47640d3f90 0x7f47640ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476a200640 1 -- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47640b48c0 msgr2=0x7f4764150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476a200640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47640b48c0 0x7f4764150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476a200640 1 -- 172.21.15.5:0/828838812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640cb040 con 0x7f4764161d90 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476a200640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 0x7f47640cf9d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f475c007d40 tx=0x7f475c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c014420 con 0x7f4764161d90 2024-04-25T10:08:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c019070 con 0x7f4764161d90 2024-04-25T10:08:43.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c014820 con 0x7f4764161d90 2024-04-25T10:08:43.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.706+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47640cb320 con 0x7f4764161d90 2024-04-25T10:08:43.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.707+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47640cb8e0 con 0x7f4764161d90 2024-04-25T10:08:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.708+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47640d2660 con 0x7f4764161d90 2024-04-25T10:08:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.709+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f475c0149c0 con 0x7f4764161d90 2024-04-25T10:08:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.710+0000 7f4761600640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f474406b500 0x7f474406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.710+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f475c08abb0 con 0x7f4764161d90 2024-04-25T10:08:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.710+0000 7f4769800640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f474406b500 0x7f474406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.712+0000 7f4769800640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f474406b500 0x7f474406d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4758034000 tx=0x7f475802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.715+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47640d2660 con 0x7f4764161d90 2024-04-25T10:08:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.917+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47640d0dd0 con 0x7f4764161d90 2024-04-25T10:08:43.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.922+0000 7f4761600640 1 -- 172.21.15.5:0/828838812 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f475c05f070 con 0x7f4764161d90 2024-04-25T10:08:43.928 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:08:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.925+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f474406b500 msgr2=0x7f474406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.925+0000 7f476ac00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f474406b500 0x7f474406d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4758034000 tx=0x7f475802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.925+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 msgr2=0x7f47640cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.925+0000 7f476ac00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 0x7f47640cf9d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f475c007d40 tx=0x7f475c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 shutdown_connections 2024-04-25T10:08:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4764161d90 0x7f47640cf9d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47640d3f90 0x7f47640ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f474406b500 0x7f474406d9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 --2- 172.21.15.5:0/828838812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47640b48c0 0x7f4764150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 >> 172.21.15.5:0/828838812 conn(0x7f476408af80 msgr2=0x7f476415def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 shutdown_connections 2024-04-25T10:08:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:43.926+0000 7f476ac00640 1 -- 172.21.15.5:0/828838812 wait complete. 2024-04-25T10:08:43.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.157+0000 7f51bc400640 1 Processor -- start 2024-04-25T10:08:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.158+0000 7f51bc400640 1 -- start start 2024-04-25T10:08:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51bc400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b41529e0 0x7f51b40c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51bc400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f51b40c6410 0x7f51b40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51bc400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b415bdc0 con 0x7f51b4071ed0 2024-04-25T10:08:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51bc400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b40b2340 con 0x7f51b40c6410 2024-04-25T10:08:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51bc400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415c060 con 0x7f51b41529e0 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51ba400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f51b4071ed0 0x7f51b40c4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40602/0 (socket says 172.21.15.5:40602) 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.159+0000 7f51ba400640 1 -- 172.21.15.5:0/3355741028 learned_addr learned my addr 172.21.15.5:0/3355741028 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b9a00640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b41529e0 0x7f51b40c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b9000640 1 --2- 172.21.15.5:0/3355741028 >> v2:172.21.15.80:3300/0 conn(0x7f51b40c6410 0x7f51b40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362881911 0 0) 0x7f51b415bdc0 con 0x7f51b4071ed0 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5198003520 con 0x7f51b4071ed0 2024-04-25T10:08:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528987435 0 0) 0x7f5198003520 con 0x7f51b4071ed0 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b41529e0 msgr2=0x7f51b40c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b41529e0 0x7f51b40c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 >> v2:172.21.15.80:3300/0 conn(0x7f51b40c6410 msgr2=0x7f51b40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 --2- 172.21.15.5:0/3355741028 >> v2:172.21.15.80:3300/0 conn(0x7f51b40c6410 0x7f51b40ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b415c340 con 0x7f51b4071ed0 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f51ac001670 con 0x7f51b4071ed0 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 >> v1:172.21.15.5:6789/0 conn(0x7f51b4071ed0 legacy=0x7f51b40c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.160+0000 7f51b3e00640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51980039b0 0x7f5198003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b3e00640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5198004800 0x7f5198004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b3e00640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5198005180 0x7f519800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40b2340 con 0x7f5198005180 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415c060 con 0x7f51980039b0 2024-04-25T10:08:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519800bf00 con 0x7f5198004800 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51ba400640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5198005180 0x7f519800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b9a00640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51980039b0 0x7f5198003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b9000640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5198004800 0x7f5198004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51ba400640 1 -- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5198004800 msgr2=0x7f5198004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51ba400640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5198004800 0x7f5198004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51ba400640 1 -- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51980039b0 msgr2=0x7f5198003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51ba400640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51980039b0 0x7f5198003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51ba400640 1 -- 172.21.15.5:0/3355741028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198003520 con 0x7f5198005180 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.161+0000 7f51b9000640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5198004800 0x7f5198004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.162+0000 7f51ba400640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5198005180 0x7f519800b9c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f51ac001050 tx=0x7f5198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a9abdecdffd3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007ca0 con 0x7f5198005180 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac005ad0 con 0x7f5198005180 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51b3e00640 1 -- 172.21.15.5:0/3355741028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007070 con 0x7f5198005180 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51bc400640 1 -- 172.21.15.5:0/3355741028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5198005180 msgr2=0x7f519800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51bc400640 1 --2- 172.21.15.5:0/3355741028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5198005180 0x7f519800b9c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f51ac001050 tx=0x7f5198005830 comp rx=0 tx=0).stop 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51ba400640 1 -- 172.21.15.5:0/3355741028 reap_dead start 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51bc400640 1 -- 172.21.15.5:0/3355741028 shutdown_connections 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51bc400640 1 -- 172.21.15.5:0/3355741028 >> 172.21.15.5:0/3355741028 conn(0x7f51b4071150 msgr2=0x7f51b4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.163+0000 7f51bc400640 1 -- 172.21.15.5:0/3355741028 shutdown_connections 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.164+0000 7f51bc400640 1 -- 172.21.15.5:0/3355741028 wait complete. 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.165+0000 7f51bc400640 1 Processor -- start 2024-04-25T10:08:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.165+0000 7f51bc400640 1 -- start start 2024-04-25T10:08:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.166+0000 7f51bc400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51b41f0630 0x7f51b40c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.166+0000 7f51bc400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b4155820 0x7f51b40ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.166+0000 7f51bc400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 0x7f51b41f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.166+0000 7f51bc400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b406e160 con 0x7f51b41f32f0 2024-04-25T10:08:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.166+0000 7f51bc400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415c900 con 0x7f51b41f0630 2024-04-25T10:08:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.166+0000 7f51bc400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b406e2e0 con 0x7f51b4155820 2024-04-25T10:08:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 0x7f51b41f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 0x7f51b41f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44578/0 (socket says 172.21.15.5:44578) 2024-04-25T10:08:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 -- 172.21.15.5:0/185880834 learned_addr learned my addr 172.21.15.5:0/185880834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51b9a00640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51b41f0630 0x7f51b40c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51b9000640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b4155820 0x7f51b40ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 -- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b4155820 msgr2=0x7f51b40ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b4155820 0x7f51b40ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 -- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51b41f0630 msgr2=0x7f51b40c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51b41f0630 0x7f51b40c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 -- 172.21.15.5:0/185880834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4154a60 con 0x7f51b41f32f0 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.167+0000 7f51ba400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 0x7f51b41f3780 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.168+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac001bf0 con 0x7f51b41f32f0 2024-04-25T10:08:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.168+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac042070 con 0x7f51b41f32f0 2024-04-25T10:08:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.168+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007270 con 0x7f51b41f32f0 2024-04-25T10:08:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.168+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b4154d40 con 0x7f51b41f32f0 2024-04-25T10:08:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.168+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b41f0a10 con 0x7f51b41f32f0 2024-04-25T10:08:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.169+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b4068fd0 con 0x7f51b41f32f0 2024-04-25T10:08:44.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.170+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51ac007410 con 0x7f51b41f32f0 2024-04-25T10:08:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.171+0000 7f51b1600640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f519406b500 0x7f519406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.172+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f51ac0aa740 con 0x7f51b41f32f0 2024-04-25T10:08:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.172+0000 7f51b9a00640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f519406b500 0x7f519406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.173+0000 7f51b9a00640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f519406b500 0x7f519406d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f51b4158dd0 tx=0x7f51a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.176+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51ac0792b0 con 0x7f51b41f32f0 2024-04-25T10:08:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.407+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51b4157650 con 0x7f51b41f32f0 2024-04-25T10:08:44.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.407+0000 7f51b1600640 1 -- 172.21.15.5:0/185880834 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f51ac078bb0 con 0x7f51b41f32f0 2024-04-25T10:08:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:44.414 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:08:09.396742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:44.414 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 53 2024-04-25T10:08:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f519406b500 msgr2=0x7f519406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f519406b500 0x7f519406d9c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f51b4158dd0 tx=0x7f51a4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 msgr2=0x7f51b41f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 0x7f51b41f3780 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac000fe0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 shutdown_connections 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51b41f32f0 0x7f51b41f3780 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51b4155820 0x7f51b40ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f519406b500 0x7f519406d9c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 --2- 172.21.15.5:0/185880834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51b41f0630 0x7f51b40c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 >> 172.21.15.5:0/185880834 conn(0x7f51b4071150 msgr2=0x7f51b415af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 shutdown_connections 2024-04-25T10:08:44.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.410+0000 7f51bc400640 1 -- 172.21.15.5:0/185880834 wait complete. 2024-04-25T10:08:44.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.630+0000 7f3901000640 1 Processor -- start 2024-04-25T10:08:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.630+0000 7f3901000640 1 -- start start 2024-04-25T10:08:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.631+0000 7f3901000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 0x7f38fc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.631+0000 7f3901000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38fc0d2380 0x7f38fc0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.631+0000 7f3901000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0cf410 con 0x7f38fc0d0600 2024-04-25T10:08:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.631+0000 7f3901000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0d27c0 con 0x7f38fc0d2380 2024-04-25T10:08:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.631+0000 7f3901000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b28b0 con 0x7f38fc161200 2024-04-25T10:08:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.632+0000 7f38fb400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 0x7f38fc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.632+0000 7f38faa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38fc0d2380 0x7f38fc0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.632+0000 7f38fb400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 0x7f38fc0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:60704/0 (socket says 172.21.15.5:60704) 2024-04-25T10:08:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.632+0000 7f38fb400640 1 -- 172.21.15.5:0/1367278801 learned_addr learned my addr 172.21.15.5:0/1367278801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.633+0000 7f38fb400640 1 -- 172.21.15.5:0/1367278801 >> v2:172.21.15.80:3300/0 conn(0x7f38fc0d2380 msgr2=0x7f38fc0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.633+0000 7f38fb400640 1 --2- 172.21.15.5:0/1367278801 >> v2:172.21.15.80:3300/0 conn(0x7f38fc0d2380 0x7f38fc0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.633+0000 7f38fb400640 1 -- 172.21.15.5:0/1367278801 >> v1:172.21.15.80:6789/0 conn(0x7f38fc0d0600 legacy=0x7f38fc0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.633+0000 7f38fb400640 1 -- 172.21.15.5:0/1367278801 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0cf6f0 con 0x7f38fc161200 2024-04-25T10:08:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.634+0000 7f38fb400640 1 --2- 172.21.15.5:0/1367278801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 0x7f38fc0b4bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009870 tx=0x7f38ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b885fc07e871e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.635+0000 7f38fa000640 1 -- 172.21.15.5:0/1367278801 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec037070 con 0x7f38fc161200 2024-04-25T10:08:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.635+0000 7f38fa000640 1 -- 172.21.15.5:0/1367278801 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec032440 con 0x7f38fc161200 2024-04-25T10:08:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.635+0000 7f38fa000640 1 -- 172.21.15.5:0/1367278801 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec032810 con 0x7f38fc161200 2024-04-25T10:08:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.635+0000 7f3901000640 1 -- 172.21.15.5:0/1367278801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 msgr2=0x7f38fc0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.635+0000 7f3901000640 1 --2- 172.21.15.5:0/1367278801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 0x7f38fc0b4bb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009870 tx=0x7f38ec02a270 comp rx=0 tx=0).stop 2024-04-25T10:08:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.636+0000 7f3901000640 1 -- 172.21.15.5:0/1367278801 shutdown_connections 2024-04-25T10:08:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.636+0000 7f3901000640 1 --2- 172.21.15.5:0/1367278801 >> v2:172.21.15.80:3300/0 conn(0x7f38fc0d2380 0x7f38fc0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.636+0000 7f3901000640 1 --2- 172.21.15.5:0/1367278801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc161200 0x7f38fc0b4bb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.636+0000 7f3901000640 1 -- 172.21.15.5:0/1367278801 >> 172.21.15.5:0/1367278801 conn(0x7f38fc08af80 msgr2=0x7f38fc0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.636+0000 7f3901000640 1 -- 172.21.15.5:0/1367278801 shutdown_connections 2024-04-25T10:08:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.636+0000 7f3901000640 1 -- 172.21.15.5:0/1367278801 wait complete. 2024-04-25T10:08:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.638+0000 7f3901000640 1 Processor -- start 2024-04-25T10:08:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.638+0000 7f3901000640 1 -- start start 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f3901000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc0d0600 0x7f38fc0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f3901000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38fc0d2380 0x7f38fc15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f3901000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 0x7f38fc0cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f3901000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b3f90 con 0x7f38fc161200 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f3901000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0d42b0 con 0x7f38fc0d2380 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f3901000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0b4110 con 0x7f38fc0d0600 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f38fbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 0x7f38fc0cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f38fbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 0x7f38fc0cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44584/0 (socket says 172.21.15.5:44584) 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f38fbe00640 1 -- 172.21.15.5:0/3588029562 learned_addr learned my addr 172.21.15.5:0/3588029562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f38fb400640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc0d0600 0x7f38fc0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.639+0000 7f38faa00640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38fc0d2380 0x7f38fc15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38fbe00640 1 -- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc0d0600 msgr2=0x7f38fc0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38fbe00640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc0d0600 0x7f38fc0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38fbe00640 1 -- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38fc0d2380 msgr2=0x7f38fc15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38fbe00640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38fc0d2380 0x7f38fc15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38fbe00640 1 -- 172.21.15.5:0/3588029562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0ccd20 con 0x7f38fc161200 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38fbe00640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 0x7f38fc0cc340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f38f400c6b0 tx=0x7f38f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4003b70 con 0x7f38fc161200 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f4003d30 con 0x7f38fc161200 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0ca920 con 0x7f38fc161200 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.640+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4005310 con 0x7f38fc161200 2024-04-25T10:08:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.641+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0caee0 con 0x7f38fc161200 2024-04-25T10:08:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.642+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc0cb0e0 con 0x7f38fc161200 2024-04-25T10:08:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.643+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38f40054b0 con 0x7f38fc161200 2024-04-25T10:08:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.644+0000 7f38f3400640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f38d806b550 0x7f38d806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.644+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f38f408a230 con 0x7f38fc161200 2024-04-25T10:08:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.644+0000 7f38fb400640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f38d806b550 0x7f38d806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.645+0000 7f38fb400640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f38d806b550 0x7f38d806da10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009870 tx=0x7f38ec02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.648+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38fc0cb0e0 con 0x7f38fc161200 2024-04-25T10:08:44.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.840+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38fc068f60 con 0x7f38fc161200 2024-04-25T10:08:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.842+0000 7f38f3400640 1 -- 172.21.15.5:0/3588029562 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16906 (secure 0 0 0) 0x7f38f4058eb0 con 0x7f38fc161200 2024-04-25T10:08:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:44.848 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:22.531478+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1098619647":"2024-04-26T10:07:22.531455+0000","172.21.15.80:6834/1098619647":"2024-04-26T10:07:22.531455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.844+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f38d806b550 msgr2=0x7f38d806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.844+0000 7f3901000640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f38d806b550 0x7f38d806da10 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009870 tx=0x7f38ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.844+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 msgr2=0x7f38fc0cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.844+0000 7f3901000640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 0x7f38fc0cc340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f38f400c6b0 tx=0x7f38f400cb70 comp rx=0 tx=0).stop 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 shutdown_connections 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc161200 0x7f38fc0cc340 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f38d806b550 0x7f38d806da10 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38fc0d2380 0x7f38fc15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 --2- 172.21.15.5:0/3588029562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f38fc0d0600 0x7f38fc0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 >> 172.21.15.5:0/3588029562 conn(0x7f38fc08af80 msgr2=0x7f38fc0b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 shutdown_connections 2024-04-25T10:08:44.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:44.845+0000 7f3901000640 1 -- 172.21.15.5:0/3588029562 wait complete. 2024-04-25T10:08:44.869 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:44.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.079+0000 7f2f9fe00640 1 Processor -- start 2024-04-25T10:08:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.080+0000 7f2f9fe00640 1 -- start start 2024-04-25T10:08:45.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2fa00d07d0 0x7f2fa00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9fe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa00cf410 con 0x7f2fa00d2380 2024-04-25T10:08:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa00cf6f0 con 0x7f2fa0161200 2024-04-25T10:08:45.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa00b28b0 con 0x7f2fa00d07d0 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2fa00d2380 0x7f2fa00b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40616/0 (socket says 172.21.15.5:40616) 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9f400640 1 -- 172.21.15.5:0/825288316 learned_addr learned my addr 172.21.15.5:0/825288316 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.081+0000 7f2f9e000640 1 --2- 172.21.15.5:0/825288316 >> v2:172.21.15.5:3300/0 conn(0x7f2fa00d07d0 0x7f2fa00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.082+0000 7f2f9d600640 1 -- 172.21.15.5:0/825288316 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034886213 0 0) 0x7f2fa00cf410 con 0x7f2fa00d2380 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.082+0000 7f2f9d600640 1 -- 172.21.15.5:0/825288316 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f80003520 con 0x7f2fa00d2380 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.082+0000 7f2f9e000640 1 -- 172.21.15.5:0/825288316 >> v1:172.21.15.80:6789/0 conn(0x7f2fa0161200 legacy=0x7f2fa00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.082+0000 7f2f9e000640 1 -- 172.21.15.5:0/825288316 >> v1:172.21.15.5:6789/0 conn(0x7f2fa00d2380 legacy=0x7f2fa00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.082+0000 7f2f9e000640 1 -- 172.21.15.5:0/825288316 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00d3fd0 con 0x7f2fa00d07d0 2024-04-25T10:08:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.083+0000 7f2f9e000640 1 --2- 172.21.15.5:0/825288316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa00d07d0 0x7f2fa00b4bb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009d90 tx=0x7f2f8802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96413250cbce72c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9d600640 1 -- 172.21.15.5:0/825288316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88007740 con 0x7f2fa00d07d0 2024-04-25T10:08:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9d600640 1 -- 172.21.15.5:0/825288316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f880078e0 con 0x7f2fa00d07d0 2024-04-25T10:08:45.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9d600640 1 -- 172.21.15.5:0/825288316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88007cb0 con 0x7f2fa00d07d0 2024-04-25T10:08:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 -- 172.21.15.5:0/825288316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa00d07d0 msgr2=0x7f2fa00b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/825288316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa00d07d0 0x7f2fa00b4bb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009d90 tx=0x7f2f8802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 -- 172.21.15.5:0/825288316 shutdown_connections 2024-04-25T10:08:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/825288316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa00d07d0 0x7f2fa00b4bb0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 -- 172.21.15.5:0/825288316 >> 172.21.15.5:0/825288316 conn(0x7f2fa008af80 msgr2=0x7f2fa00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 -- 172.21.15.5:0/825288316 shutdown_connections 2024-04-25T10:08:45.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.084+0000 7f2f9fe00640 1 -- 172.21.15.5:0/825288316 wait complete. 2024-04-25T10:08:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.086+0000 7f2f9fe00640 1 Processor -- start 2024-04-25T10:08:45.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.086+0000 7f2f9fe00640 1 -- start start 2024-04-25T10:08:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fa00d07d0 0x7f2fa0152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fa00d2380 0x7f2fa006bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 0x7f2fa006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00b3810 con 0x7f2fa0161200 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00b3690 con 0x7f2fa00d2380 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00b3990 con 0x7f2fa00d07d0 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 0x7f2fa006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.087+0000 7f2f9f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 0x7f2fa006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44592/0 (socket says 172.21.15.5:44592) 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 -- 172.21.15.5:0/294097429 learned_addr learned my addr 172.21.15.5:0/294097429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9e000640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fa00d2380 0x7f2fa006bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 -- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fa00d07d0 msgr2=0x7f2fa0152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9ea00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fa00d07d0 0x7f2fa0152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fa00d07d0 0x7f2fa0152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 -- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fa00d2380 msgr2=0x7f2fa006bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fa00d2380 0x7f2fa006bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 -- 172.21.15.5:0/294097429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0153080 con 0x7f2fa0161200 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9e000640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fa00d2380 0x7f2fa006bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:45.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9ea00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fa00d07d0 0x7f2fa0152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f9f400640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 0x7f2fa006c310 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c460 tx=0x7f2f9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.088+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98017070 con 0x7f2fa0161200 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.089+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f98005ce0 con 0x7f2fa0161200 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.089+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa00b6470 con 0x7f2fa0161200 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.089+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980072b0 con 0x7f2fa0161200 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.089+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa00b6a30 con 0x7f2fa0161200 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.090+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa0068f60 con 0x7f2fa0161200 2024-04-25T10:08:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.091+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f9801a070 con 0x7f2fa0161200 2024-04-25T10:08:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.092+0000 7f2f97400640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f7c06b4d0 0x7f2f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.092+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2f98089810 con 0x7f2fa0161200 2024-04-25T10:08:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.092+0000 7f2f9ea00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f7c06b4d0 0x7f2f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.093+0000 7f2f9ea00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f7c06b4d0 0x7f2f7c06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2f900015e0 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.096+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fa0153080 con 0x7f2fa0161200 2024-04-25T10:08:45.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.337+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fa00d0590 con 0x7f2fa0161200 2024-04-25T10:08:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.338+0000 7f2f97400640 1 -- 172.21.15.5:0/294097429 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f2f98058410 con 0x7f2fa0161200 2024-04-25T10:08:45.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:45.344 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:08:09.396742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 53 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f7c06b4d0 msgr2=0x7f2f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f7c06b4d0 0x7f2f7c06d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2f900015e0 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 msgr2=0x7f2fa006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 0x7f2fa006c310 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800c460 tx=0x7f2f9800c920 comp rx=0 tx=0).stop 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 shutdown_connections 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fa0161200 0x7f2fa006c310 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f7c06b4d0 0x7f2f7c06d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2fa00d2380 0x7f2fa006bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 --2- 172.21.15.5:0/294097429 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2fa00d07d0 0x7f2fa0152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 >> 172.21.15.5:0/294097429 conn(0x7f2fa008af80 msgr2=0x7f2fa00b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 shutdown_connections 2024-04-25T10:08:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.342+0000 7f2f9fe00640 1 -- 172.21.15.5:0/294097429 wait complete. 2024-04-25T10:08:45.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.571+0000 7f2e22200640 1 Processor -- start 2024-04-25T10:08:45.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.571+0000 7f2e22200640 1 -- start start 2024-04-25T10:08:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.573+0000 7f2e22200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 0x7f2e1c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.573+0000 7f2e22200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c071d30 con 0x7f2e1c0cd300 2024-04-25T10:08:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.573+0000 7f2e22200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c06d620 con 0x7f2e1c141da0 2024-04-25T10:08:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.573+0000 7f2e22200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c142180 con 0x7f2e1c15ce00 2024-04-25T10:08:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.573+0000 7f2e21800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2e1c0cd300 0x7f2e1c071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:40632/0 (socket says 172.21.15.5:40632) 2024-04-25T10:08:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.573+0000 7f2e21800640 1 -- 172.21.15.5:0/2903304837 learned_addr learned my addr 172.21.15.5:0/2903304837 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1be00640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 0x7f2e1c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888472641 0 0) 0x7f2e1c071d30 con 0x7f2e1c0cd300 2024-04-25T10:08:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e04003520 con 0x7f2e1c0cd300 2024-04-25T10:08:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328264996 0 0) 0x7f2e04003520 con 0x7f2e1c0cd300 2024-04-25T10:08:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 msgr2=0x7f2e1c15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 0x7f2e1c15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 >> v1:172.21.15.80:6789/0 conn(0x7f2e1c141da0 legacy=0x7f2e1c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c06d900 con 0x7f2e1c0cd300 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2e140019f0 con 0x7f2e1c0cd300 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 >> v1:172.21.15.5:6789/0 conn(0x7f2e1c0cd300 legacy=0x7f2e1c071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 0x7f2e040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04003f60 0x7f2e04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.574+0000 7f2e1b400640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e04004d40 0x7f2e040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c142180 con 0x7f2e04004d40 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0400baa0 con 0x7f2e04003f60 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0400bca0 con 0x7f2e1c15ce00 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e21800640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e04004d40 0x7f2e040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e20e00640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 0x7f2e040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e1be00640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04003f60 0x7f2e04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e21800640 1 -- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 msgr2=0x7f2e040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e21800640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c15ce00 0x7f2e040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e21800640 1 -- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04003f60 msgr2=0x7f2e04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e21800640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04003f60 0x7f2e04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e21800640 1 -- 172.21.15.5:0/2903304837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e04003520 con 0x7f2e04004d40 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.575+0000 7f2e1be00640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04003f60 0x7f2e04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.576+0000 7f2e21800640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e04004d40 0x7f2e040051c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400a860 tx=0x7f2e04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bb50f3609458af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400a480 con 0x7f2e04004d40 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1402f070 con 0x7f2e04004d40 2024-04-25T10:08:45.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e1b400640 1 -- 172.21.15.5:0/2903304837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1402bd20 con 0x7f2e04004d40 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e22200640 1 -- 172.21.15.5:0/2903304837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e04004d40 msgr2=0x7f2e040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e22200640 1 --2- 172.21.15.5:0/2903304837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e04004d40 0x7f2e040051c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400a860 tx=0x7f2e04005830 comp rx=0 tx=0).stop 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e21800640 1 -- 172.21.15.5:0/2903304837 reap_dead start 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e22200640 1 -- 172.21.15.5:0/2903304837 shutdown_connections 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e22200640 1 -- 172.21.15.5:0/2903304837 >> 172.21.15.5:0/2903304837 conn(0x7f2e1c066f20 msgr2=0x7f2e1c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e22200640 1 -- 172.21.15.5:0/2903304837 shutdown_connections 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.577+0000 7f2e22200640 1 -- 172.21.15.5:0/2903304837 wait complete. 2024-04-25T10:08:45.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.579+0000 7f2e22200640 1 Processor -- start 2024-04-25T10:08:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.579+0000 7f2e22200640 1 -- start start 2024-04-25T10:08:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e22200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04004d40 0x7f2e1c13ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e22200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 0x7f2e1c0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e22200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c0b3020 0x7f2e1c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e22200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c071710 con 0x7f2e1c0b1690 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e22200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c06e810 con 0x7f2e04004d40 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e22200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c071890 con 0x7f2e1c0b3020 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e1be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 0x7f2e1c0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e1be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 0x7f2e1c0b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44614/0 (socket says 172.21.15.5:44614) 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.580+0000 7f2e1be00640 1 -- 172.21.15.5:0/3813021605 learned_addr learned my addr 172.21.15.5:0/3813021605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e20e00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04004d40 0x7f2e1c13ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e1be00640 1 -- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c0b3020 msgr2=0x7f2e1c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e1be00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c0b3020 0x7f2e1c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e1be00640 1 -- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04004d40 msgr2=0x7f2e1c13ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e1be00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04004d40 0x7f2e1c13ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e1be00640 1 -- 172.21.15.5:0/3813021605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c1426a0 con 0x7f2e1c0b1690 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e20e00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04004d40 0x7f2e1c13ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e1be00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 0x7f2e1c0b1b10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2e080108e0 tx=0x7f2e08010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0800f040 con 0x7f2e1c0b1690 2024-04-25T10:08:45.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0800eba0 con 0x7f2e1c0b1690 2024-04-25T10:08:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0801c400 con 0x7f2e1c0b1690 2024-04-25T10:08:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.581+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c142980 con 0x7f2e1c0b1690 2024-04-25T10:08:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.582+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c142f40 con 0x7f2e1c0b1690 2024-04-25T10:08:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.583+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c068f60 con 0x7f2e1c0b1690 2024-04-25T10:08:45.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.584+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e0801c5a0 con 0x7f2e1c0b1690 2024-04-25T10:08:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.585+0000 7f2e18c00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dfc06b4d0 0x7f2dfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.585+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2e0808ab50 con 0x7f2e1c0b1690 2024-04-25T10:08:45.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.586+0000 7f2e20e00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dfc06b4d0 0x7f2dfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.587+0000 7f2e20e00640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dfc06b4d0 0x7f2dfc06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2e100016b0 tx=0x7f2e10009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:45.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.590+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e08018030 con 0x7f2e1c0b1690 2024-04-25T10:08:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.824+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e1c0ccae0 con 0x7f2e1c0b1690 2024-04-25T10:08:45.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.825+0000 7f2e18c00640 1 -- 172.21.15.5:0/3813021605 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f2e080597e0 con 0x7f2e1c0b1690 2024-04-25T10:08:45.831 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:45.831 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:09.493208+0000","modified":"2024-04-25T10:08:09.396742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6025":{"gid":6025,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.80:6835/3658047546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3658047546},{"type":"v1","addr":"172.21.15.80:6835","nonce":3658047546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:45.831 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 53 2024-04-25T10:08:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dfc06b4d0 msgr2=0x7f2dfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dfc06b4d0 0x7f2dfc06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2e100016b0 tx=0x7f2e10009040 comp rx=0 tx=0).stop 2024-04-25T10:08:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 msgr2=0x7f2e1c0b1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 0x7f2e1c0b1b10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2e080108e0 tx=0x7f2e08010da0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 shutdown_connections 2024-04-25T10:08:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e1c0b3020 0x7f2e1c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e1c0b1690 0x7f2e1c0b1b10 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dfc06b4d0 0x7f2dfc06d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 --2- 172.21.15.5:0/3813021605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e04004d40 0x7f2e1c13ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 >> 172.21.15.5:0/3813021605 conn(0x7f2e1c066f20 msgr2=0x7f2e1c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 shutdown_connections 2024-04-25T10:08:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:45.828+0000 7f2e22200640 1 -- 172.21.15.5:0/3813021605 wait complete. 2024-04-25T10:08:45.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.048+0000 7fa977a00640 1 Processor -- start 2024-04-25T10:08:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.048+0000 7fa977a00640 1 -- start start 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa977a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9701529e0 0x7fa9700c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa977a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9700c6410 0x7fa9700ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa977a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa970071ed0 0x7fa9700ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa977a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9700b2340 con 0x7fa970071ed0 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa977a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97015bdc0 con 0x7fa9700c6410 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa977a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97015c060 con 0x7fa9701529e0 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa975a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa970071ed0 0x7fa9700ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa975a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa970071ed0 0x7fa9700ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44628/0 (socket says 172.21.15.5:44628) 2024-04-25T10:08:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.050+0000 7fa975a00640 1 -- 172.21.15.5:0/4205902062 learned_addr learned my addr 172.21.15.5:0/4205902062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.051+0000 7fa975000640 1 --2- 172.21.15.5:0/4205902062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9701529e0 0x7fa9700c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.051+0000 7fa975a00640 1 -- 172.21.15.5:0/4205902062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9701529e0 msgr2=0x7fa9700c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.051+0000 7fa975a00640 1 --2- 172.21.15.5:0/4205902062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9701529e0 0x7fa9700c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.051+0000 7fa975a00640 1 -- 172.21.15.5:0/4205902062 >> v2:172.21.15.80:3300/0 conn(0x7fa9700c6410 msgr2=0x7fa9700ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.051+0000 7fa975a00640 1 --2- 172.21.15.5:0/4205902062 >> v2:172.21.15.80:3300/0 conn(0x7fa9700c6410 0x7fa9700ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.051+0000 7fa975a00640 1 -- 172.21.15.5:0/4205902062 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97015c340 con 0x7fa970071ed0 2024-04-25T10:08:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.052+0000 7fa975a00640 1 --2- 172.21.15.5:0/4205902062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa9700ce890 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa9700cf5a0 tx=0x7fa96802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa036d8fe6111735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.052+0000 7fa96fe00640 1 -- 172.21.15.5:0/4205902062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968036070 con 0x7fa970071ed0 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.052+0000 7fa96fe00640 1 -- 172.21.15.5:0/4205902062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968031600 con 0x7fa970071ed0 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.052+0000 7fa96fe00640 1 -- 172.21.15.5:0/4205902062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9680319d0 con 0x7fa970071ed0 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 -- 172.21.15.5:0/4205902062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 msgr2=0x7fa9700ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 --2- 172.21.15.5:0/4205902062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa9700ce890 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa9700cf5a0 tx=0x7fa96802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 -- 172.21.15.5:0/4205902062 shutdown_connections 2024-04-25T10:08:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 --2- 172.21.15.5:0/4205902062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa9700ce890 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 --2- 172.21.15.5:0/4205902062 >> v2:172.21.15.80:3300/0 conn(0x7fa9700c6410 0x7fa9700ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 --2- 172.21.15.5:0/4205902062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9701529e0 0x7fa9700c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 -- 172.21.15.5:0/4205902062 >> 172.21.15.5:0/4205902062 conn(0x7fa970071150 msgr2=0x7fa970071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 -- 172.21.15.5:0/4205902062 shutdown_connections 2024-04-25T10:08:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.053+0000 7fa977a00640 1 -- 172.21.15.5:0/4205902062 wait complete. 2024-04-25T10:08:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.055+0000 7fa977a00640 1 Processor -- start 2024-04-25T10:08:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.055+0000 7fa977a00640 1 -- start start 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.056+0000 7fa977a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa97014a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.056+0000 7fa977a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9700c6410 0x7fa9700b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.056+0000 7fa977a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa970156330 0x7fa9701468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.056+0000 7fa977a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700c4660 con 0x7fa970071ed0 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.056+0000 7fa977a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97015c900 con 0x7fa970156330 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.056+0000 7fa977a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700c47e0 con 0x7fa9700c6410 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa97014a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa97014a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44644/0 (socket says 172.21.15.5:44644) 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 -- 172.21.15.5:0/3507747120 learned_addr learned my addr 172.21.15.5:0/3507747120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 -- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9700c6410 msgr2=0x7fa9700b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9700c6410 0x7fa9700b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 -- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa970156330 msgr2=0x7fa9701468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa970156330 0x7fa9701468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.057+0000 7fa975000640 1 -- 172.21.15.5:0/3507747120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970158cd0 con 0x7fa970071ed0 2024-04-25T10:08:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.058+0000 7fa975000640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa97014a0d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa96000d5f0 tx=0x7fa96000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.058+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9600125f0 con 0x7fa970071ed0 2024-04-25T10:08:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.058+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa960019070 con 0x7fa970071ed0 2024-04-25T10:08:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.058+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9600129d0 con 0x7fa970071ed0 2024-04-25T10:08:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.058+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa970158fb0 con 0x7fa970071ed0 2024-04-25T10:08:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.058+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9700b4cd0 con 0x7fa970071ed0 2024-04-25T10:08:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.060+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9700b4820 con 0x7fa970071ed0 2024-04-25T10:08:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.061+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa960012b70 con 0x7fa970071ed0 2024-04-25T10:08:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.061+0000 7fa96d600640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa94c06b550 0x7fa94c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.062+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa96008a730 con 0x7fa970071ed0 2024-04-25T10:08:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.062+0000 7fa967e00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa94c06b550 0x7fa94c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.064+0000 7fa967e00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa94c06b550 0x7fa94c06da10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa970155c40 tx=0x7fa958006c90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.068+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa960059440 con 0x7fa970071ed0 2024-04-25T10:08:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.271+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa970068fd0 con 0x7fa970071ed0 2024-04-25T10:08:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.273+0000 7fa96d600640 1 -- 172.21.15.5:0/3507747120 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16906 (secure 0 0 0) 0x7fa960058d40 con 0x7fa970071ed0 2024-04-25T10:08:46.279 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:46.279 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:07:22.531478+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:07.362188+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:08.532434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1098619647":"2024-04-26T10:07:22.531455+0000","172.21.15.80:6834/1098619647":"2024-04-26T10:07:22.531455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.275+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa94c06b550 msgr2=0x7fa94c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.275+0000 7fa977a00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa94c06b550 0x7fa94c06da10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa970155c40 tx=0x7fa958006c90 comp rx=0 tx=0).stop 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.275+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 msgr2=0x7fa97014a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.275+0000 7fa977a00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa97014a0d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa96000d5f0 tx=0x7fa96000dab0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 shutdown_connections 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa94c06b550 0x7fa94c06da10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa970156330 0x7fa9701468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa9700c6410 0x7fa9700b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 --2- 172.21.15.5:0/3507747120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa970071ed0 0x7fa97014a0d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 >> 172.21.15.5:0/3507747120 conn(0x7fa970071150 msgr2=0x7fa97015a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 shutdown_connections 2024-04-25T10:08:46.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.276+0000 7fa977a00640 1 -- 172.21.15.5:0/3507747120 wait complete. 2024-04-25T10:08:46.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:08:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.505+0000 7faf67400640 1 Processor -- start 2024-04-25T10:08:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.505+0000 7faf67400640 1 -- start start 2024-04-25T10:08:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf67400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faf60144100 0x7faf600cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf67400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faf6006e970 0x7faf6015b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf67400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf600cdbf0 con 0x7faf600ced20 2024-04-25T10:08:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf67400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf601444e0 con 0x7faf60144100 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf67400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6015ac70 con 0x7faf6006e970 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf64a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faf60144100 0x7faf600cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf64a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faf60144100 0x7faf600cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44652/0 (socket says 172.21.15.5:44652) 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf64a00640 1 -- 172.21.15.5:0/1598468207 learned_addr learned my addr 172.21.15.5:0/1598468207 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.507+0000 7faf65400640 1 --2- 172.21.15.5:0/1598468207 >> v2:172.21.15.80:3300/0 conn(0x7faf6006e970 0x7faf6015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.508+0000 7faf64a00640 1 -- 172.21.15.5:0/1598468207 >> v2:172.21.15.80:3300/0 conn(0x7faf6006e970 msgr2=0x7faf6015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.508+0000 7faf64a00640 1 --2- 172.21.15.5:0/1598468207 >> v2:172.21.15.80:3300/0 conn(0x7faf6006e970 0x7faf6015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.508+0000 7faf64a00640 1 -- 172.21.15.5:0/1598468207 >> v1:172.21.15.80:6789/0 conn(0x7faf600ced20 legacy=0x7faf6006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.508+0000 7faf64a00640 1 -- 172.21.15.5:0/1598468207 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600cded0 con 0x7faf60144100 2024-04-25T10:08:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.508+0000 7faf65400640 1 --2- 172.21.15.5:0/1598468207 >> v2:172.21.15.80:3300/0 conn(0x7faf6006e970 0x7faf6015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.509+0000 7faf64a00640 1 --2- 172.21.15.5:0/1598468207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf60144100 0x7faf600cd610 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7faf600b2780 tx=0x7faf5002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9c2760834c809d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.509+0000 7faf5fe00640 1 -- 172.21.15.5:0/1598468207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500325a0 con 0x7faf60144100 2024-04-25T10:08:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.509+0000 7faf5fe00640 1 -- 172.21.15.5:0/1598468207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf50038070 con 0x7faf60144100 2024-04-25T10:08:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.509+0000 7faf5fe00640 1 -- 172.21.15.5:0/1598468207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50032960 con 0x7faf60144100 2024-04-25T10:08:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 -- 172.21.15.5:0/1598468207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf60144100 msgr2=0x7faf600cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 --2- 172.21.15.5:0/1598468207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf60144100 0x7faf600cd610 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7faf600b2780 tx=0x7faf5002c8e0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 -- 172.21.15.5:0/1598468207 shutdown_connections 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 --2- 172.21.15.5:0/1598468207 >> v2:172.21.15.80:3300/0 conn(0x7faf6006e970 0x7faf6015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 --2- 172.21.15.5:0/1598468207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf60144100 0x7faf600cd610 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 -- 172.21.15.5:0/1598468207 >> 172.21.15.5:0/1598468207 conn(0x7faf60071150 msgr2=0x7faf60071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 -- 172.21.15.5:0/1598468207 shutdown_connections 2024-04-25T10:08:46.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.510+0000 7faf67400640 1 -- 172.21.15.5:0/1598468207 wait complete. 2024-04-25T10:08:46.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.512+0000 7faf67400640 1 Processor -- start 2024-04-25T10:08:46.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.512+0000 7faf67400640 1 -- start start 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf67400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf6006e970 0x7faf60159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf67400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 0x7faf6013ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf67400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faf60144100 0x7faf60141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf67400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600cea00 con 0x7faf600ced20 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf67400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf600ce490 con 0x7faf6006e970 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf67400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6015b670 con 0x7faf60144100 2024-04-25T10:08:46.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 0x7faf6013ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 0x7faf6013ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44668/0 (socket says 172.21.15.5:44668) 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf57e00640 1 -- 172.21.15.5:0/2271052925 learned_addr learned my addr 172.21.15.5:0/2271052925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf64a00640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf6006e970 0x7faf60159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf65400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faf60144100 0x7faf60141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf57e00640 1 -- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faf60144100 msgr2=0x7faf60141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.514+0000 7faf57e00640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faf60144100 0x7faf60141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf57e00640 1 -- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf6006e970 msgr2=0x7faf60159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf57e00640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf6006e970 0x7faf60159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf57e00640 1 -- 172.21.15.5:0/2271052925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6015a2e0 con 0x7faf600ced20 2024-04-25T10:08:46.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf65400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faf60144100 0x7faf60141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf57e00640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 0x7faf6013ac70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c430 tx=0x7faf4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011790 con 0x7faf600ced20 2024-04-25T10:08:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf48017070 con 0x7faf600ced20 2024-04-25T10:08:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48011bb0 con 0x7faf600ced20 2024-04-25T10:08:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60138b90 con 0x7faf600ced20 2024-04-25T10:08:46.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.515+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf60139150 con 0x7faf600ced20 2024-04-25T10:08:46.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.516+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60068fd0 con 0x7faf600ced20 2024-04-25T10:08:46.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.517+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf4801c460 con 0x7faf600ced20 2024-04-25T10:08:46.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.518+0000 7faf5d600640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faf3c06b4d0 0x7faf3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.519+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faf4808a360 con 0x7faf600ced20 2024-04-25T10:08:46.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.519+0000 7faf64a00640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faf3c06b4d0 0x7faf3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.520+0000 7faf64a00640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faf3c06b4d0 0x7faf3c06d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf50037000 tx=0x7faf50032000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.523+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf48058f60 con 0x7faf600ced20 2024-04-25T10:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.759+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faf600ccd20 con 0x7faf600ced20 2024-04-25T10:08:46.765 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:08:46.760+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.764+0000 7faf5d600640 1 -- 172.21.15.5:0/2271052925 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf48058860 con 0x7faf600ced20 2024-04-25T10:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faf3c06b4d0 msgr2=0x7faf3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faf3c06b4d0 0x7faf3c06d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faf50037000 tx=0x7faf50032000 comp rx=0 tx=0).stop 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 msgr2=0x7faf6013ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 0x7faf6013ac70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c430 tx=0x7faf4800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 shutdown_connections 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faf60144100 0x7faf60141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf600ced20 0x7faf6013ac70 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faf3c06b4d0 0x7faf3c06d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 --2- 172.21.15.5:0/2271052925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faf6006e970 0x7faf60159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 >> 172.21.15.5:0/2271052925 conn(0x7faf60071150 msgr2=0x7faf600b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 shutdown_connections 2024-04-25T10:08:46.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.767+0000 7faf67400640 1 -- 172.21.15.5:0/2271052925 wait complete. 2024-04-25T10:08:46.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:08:46.805 INFO:tasks.ceph.mds.d.smithi080.stderr: -4217> 2024-04-25T10:07:28.391+0000 7f392ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:46.890 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:08:46.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.989+0000 7f75f8c00640 1 Processor -- start 2024-04-25T10:08:46.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.989+0000 7f75f8c00640 1 -- start start 2024-04-25T10:08:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f8c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f75f4143d10 0x7f75f415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f8c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f75f4071710 0x7f75f40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f8c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f415f5e0 con 0x7f75f40adac0 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f8c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f406d250 con 0x7f75f4143d10 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f8c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40ae680 con 0x7f75f4071710 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f2a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75f40adac0 0x7f75f4071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60430/0 (socket says 172.21.15.5:60430) 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f2a00640 1 -- 172.21.15.5:0/1570016713 learned_addr learned my addr 172.21.15.5:0/1570016713 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.991+0000 7f75f3400640 1 --2- 172.21.15.5:0/1570016713 >> v2:172.21.15.5:3300/0 conn(0x7f75f4143d10 0x7f75f415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f3e00640 1 --2- 172.21.15.5:0/1570016713 >> v2:172.21.15.80:3300/0 conn(0x7f75f4071710 0x7f75f40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f2000640 1 -- 172.21.15.5:0/1570016713 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937673695 0 0) 0x7f75f415f5e0 con 0x7f75f40adac0 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f2000640 1 -- 172.21.15.5:0/1570016713 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d8003520 con 0x7f75f40adac0 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f3400640 1 -- 172.21.15.5:0/1570016713 >> v2:172.21.15.80:3300/0 conn(0x7f75f4071710 msgr2=0x7f75f40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f3400640 1 --2- 172.21.15.5:0/1570016713 >> v2:172.21.15.80:3300/0 conn(0x7f75f4071710 0x7f75f40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f3400640 1 -- 172.21.15.5:0/1570016713 >> v1:172.21.15.5:6789/0 conn(0x7f75f40adac0 legacy=0x7f75f4071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.992+0000 7f75f3400640 1 -- 172.21.15.5:0/1570016713 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f40ae960 con 0x7f75f4143d10 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.993+0000 7f75f3400640 1 --2- 172.21.15.5:0/1570016713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4143d10 0x7f75f415f0a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f75f40cbbe0 tx=0x7f75e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfae0c7fb8b892ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.993+0000 7f75f2000640 1 -- 172.21.15.5:0/1570016713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4037070 con 0x7f75f4143d10 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.993+0000 7f75f2000640 1 -- 172.21.15.5:0/1570016713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e4032440 con 0x7f75f4143d10 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.993+0000 7f75f2000640 1 -- 172.21.15.5:0/1570016713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40327f0 con 0x7f75f4143d10 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 -- 172.21.15.5:0/1570016713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4143d10 msgr2=0x7f75f415f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1570016713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4143d10 0x7f75f415f0a0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f75f40cbbe0 tx=0x7f75e402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 -- 172.21.15.5:0/1570016713 shutdown_connections 2024-04-25T10:08:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1570016713 >> v2:172.21.15.80:3300/0 conn(0x7f75f4071710 0x7f75f40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1570016713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4143d10 0x7f75f415f0a0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 -- 172.21.15.5:0/1570016713 >> 172.21.15.5:0/1570016713 conn(0x7f75f4066f20 msgr2=0x7f75f406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 -- 172.21.15.5:0/1570016713 shutdown_connections 2024-04-25T10:08:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.994+0000 7f75f8c00640 1 -- 172.21.15.5:0/1570016713 wait complete. 2024-04-25T10:08:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.996+0000 7f75f8c00640 1 Processor -- start 2024-04-25T10:08:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.996+0000 7f75f8c00640 1 -- start start 2024-04-25T10:08:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 0x7f75f413fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f40adac0 0x7f75f4161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f4143d10 0x7f75f415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f8c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41440f0 con 0x7f75f4071710 2024-04-25T10:08:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f8c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f406ea40 con 0x7f75f40adac0 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f8c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f406d6e0 con 0x7f75f4143d10 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 0x7f75f413fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 0x7f75f413fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59660/0 (socket says 172.21.15.5:59660) 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f3400640 1 -- 172.21.15.5:0/1646564560 learned_addr learned my addr 172.21.15.5:0/1646564560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f2a00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f40adac0 0x7f75f4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.997+0000 7f75f3e00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f4143d10 0x7f75f415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f3400640 1 -- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f4143d10 msgr2=0x7f75f415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f3400640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f4143d10 0x7f75f415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f3400640 1 -- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f40adac0 msgr2=0x7f75f4161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f3400640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f40adac0 0x7f75f4161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f3400640 1 -- 172.21.15.5:0/1646564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f415ff30 con 0x7f75f4071710 2024-04-25T10:08:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f2a00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f40adac0 0x7f75f4161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f3400640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 0x7f75f413fe10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f75e402a1e0 tx=0x7f75e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4037040 con 0x7f75f4071710 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e40304c0 con 0x7f75f4071710 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4030890 con 0x7f75f4071710 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.998+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f4141b50 con 0x7f75f4071710 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:46.999+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f4142110 con 0x7f75f4071710 2024-04-25T10:08:47.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.000+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f4068f60 con 0x7f75f4071710 2024-04-25T10:08:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.001+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75e4044020 con 0x7f75f4071710 2024-04-25T10:08:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.001+0000 7f75eb400640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75d406b4d0 0x7f75d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.002+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f75e40aa7c0 con 0x7f75f4071710 2024-04-25T10:08:47.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.002+0000 7f75f2a00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75d406b4d0 0x7f75d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.003+0000 7f75f2a00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75d406b4d0 0x7f75d406d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f75e0007fd0 tx=0x7f75e0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.006+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75e40793f0 con 0x7f75f4071710 2024-04-25T10:08:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.246+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f75f40b2350 con 0x7f75f4071710 2024-04-25T10:08:47.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.774+0000 7f75eb400640 1 -- 172.21.15.5:0/1646564560 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75e407f0f0 con 0x7f75f4071710 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75d406b4d0 msgr2=0x7f75d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75d406b4d0 0x7f75d406d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f75e0007fd0 tx=0x7f75e0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 msgr2=0x7f75f413fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 0x7f75f413fe10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f75e402a1e0 tx=0x7f75e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 shutdown_connections 2024-04-25T10:08:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75f4143d10 0x7f75f415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f75d406b4d0 0x7f75d406d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75f40adac0 0x7f75f4161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 --2- 172.21.15.5:0/1646564560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75f4071710 0x7f75f413fe10 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 >> 172.21.15.5:0/1646564560 conn(0x7f75f4066f20 msgr2=0x7f75f4071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 shutdown_connections 2024-04-25T10:08:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:47.779+0000 7f75f8c00640 1 -- 172.21.15.5:0/1646564560 wait complete. 2024-04-25T10:08:47.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:08:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.017+0000 7fd09ec00640 1 Processor -- start 2024-04-25T10:08:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.018+0000 7fd09ec00640 1 -- start start 2024-04-25T10:08:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.019+0000 7fd09ec00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0980d0820 0x7fd0980b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.019+0000 7fd09ec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 0x7fd098162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.019+0000 7fd09ec00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09806c1f0 con 0x7fd09815db10 2024-04-25T10:08:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.019+0000 7fd09ec00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09806bfb0 con 0x7fd0980d0820 2024-04-25T10:08:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.019+0000 7fd09ec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09806c4d0 con 0x7fd09806bb00 2024-04-25T10:08:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.020+0000 7fd09d800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd09815db10 0x7fd0980d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60438/0 (socket says 172.21.15.5:60438) 2024-04-25T10:08:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.020+0000 7fd09d800640 1 -- 172.21.15.5:0/3229101007 learned_addr learned my addr 172.21.15.5:0/3229101007 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.020+0000 7fd09e200640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 0x7fd098162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.020+0000 7fd09ce00640 1 --2- 172.21.15.5:0/3229101007 >> v2:172.21.15.80:3300/0 conn(0x7fd0980d0820 0x7fd0980b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.020+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302361613 0 0) 0x7fd09806c1f0 con 0x7fd09815db10 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.020+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd07c003520 con 0x7fd09815db10 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 639315313 0 0) 0x7fd07c003520 con 0x7fd09815db10 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 msgr2=0x7fd098162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 0x7fd098162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 >> v2:172.21.15.80:3300/0 conn(0x7fd0980d0820 msgr2=0x7fd0980b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 --2- 172.21.15.5:0/3229101007 >> v2:172.21.15.80:3300/0 conn(0x7fd0980d0820 0x7fd0980b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09806c7b0 con 0x7fd09815db10 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd0880019f0 con 0x7fd09815db10 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 >> v1:172.21.15.5:6789/0 conn(0x7fd09815db10 legacy=0x7fd0980d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd07c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 0x7fd07c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09806bfb0 con 0x7fd07c0039b0 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09806c4d0 con 0x7fd07c004d40 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c00b9c0 con 0x7fd09806bb00 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.021+0000 7fd09d800640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd07c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09e200640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09ce00640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 0x7fd07c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09d800640 1 -- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 msgr2=0x7fd07c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09d800640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd09806bb00 0x7fd07c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09d800640 1 -- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd07c004d40 msgr2=0x7fd07c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09d800640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd07c004d40 0x7fd07c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.022+0000 7fd09d800640 1 -- 172.21.15.5:0/3229101007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c003520 con 0x7fd07c0039b0 2024-04-25T10:08:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.023+0000 7fd09d800640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd07c003dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd08800a890 tx=0x7fd07c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bf2b5c2a72ef6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.023+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08800a4e0 con 0x7fd07c0039b0 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.023+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08802f070 con 0x7fd07c0039b0 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.023+0000 7fd097e00640 1 -- 172.21.15.5:0/3229101007 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08802bcf0 con 0x7fd07c0039b0 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09ec00640 1 -- 172.21.15.5:0/3229101007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 msgr2=0x7fd07c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09ec00640 1 --2- 172.21.15.5:0/3229101007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd07c003dd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd08800a890 tx=0x7fd07c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09e200640 1 -- 172.21.15.5:0/3229101007 reap_dead start 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09ec00640 1 -- 172.21.15.5:0/3229101007 shutdown_connections 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09ec00640 1 -- 172.21.15.5:0/3229101007 >> 172.21.15.5:0/3229101007 conn(0x7fd0980b5210 msgr2=0x7fd0980b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09ec00640 1 -- 172.21.15.5:0/3229101007 shutdown_connections 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.024+0000 7fd09ec00640 1 -- 172.21.15.5:0/3229101007 wait complete. 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.026+0000 7fd09ec00640 1 Processor -- start 2024-04-25T10:08:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.026+0000 7fd09ec00640 1 -- start start 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd098152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09ec00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0980ca7d0 0x7fd0980c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09ec00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0980c8780 0x7fd09815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09815e070 con 0x7fd07c0039b0 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09ec00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09815def0 con 0x7fd0980c8780 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09ec00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd098162280 con 0x7fd0980ca7d0 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd098152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd098152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59684/0 (socket says 172.21.15.5:59684) 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.027+0000 7fd09d800640 1 -- 172.21.15.5:0/560298488 learned_addr learned my addr 172.21.15.5:0/560298488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09d800640 1 -- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0980ca7d0 msgr2=0x7fd0980c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09d800640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0980ca7d0 0x7fd0980c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09d800640 1 -- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0980c8780 msgr2=0x7fd09815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09e200640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0980c8780 0x7fd09815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09d800640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0980c8780 0x7fd09815fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09d800640 1 -- 172.21.15.5:0/560298488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980c8e00 con 0x7fd07c0039b0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd09d800640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd098152460 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd08800a890 tx=0x7fd088001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.028+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08802ba90 con 0x7fd07c0039b0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.029+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08802bc30 con 0x7fd07c0039b0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.029+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0980c90e0 con 0x7fd07c0039b0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.029+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088004e30 con 0x7fd07c0039b0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.029+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09815e3d0 con 0x7fd07c0039b0 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.030+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09815e1f0 con 0x7fd07c0039b0 2024-04-25T10:08:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.031+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd088034470 con 0x7fd07c0039b0 2024-04-25T10:08:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.032+0000 7fd095600640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd07806b5a0 0x7fd07806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.032+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd0880aa980 con 0x7fd07c0039b0 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.033+0000 7fd09ce00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd07806b5a0 0x7fd07806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.034+0000 7fd09ce00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd07806b5a0 0x7fd07806da60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd09815c740 tx=0x7fd084008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.036+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd09815e1f0 con 0x7fd07c0039b0 2024-04-25T10:08:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.236+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0980677d0 con 0x7fd07c0039b0 2024-04-25T10:08:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.948+0000 7fd095600640 1 -- 172.21.15.5:0/560298488 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd088079630 con 0x7fd07c0039b0 2024-04-25T10:08:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd07806b5a0 msgr2=0x7fd07806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd07806b5a0 0x7fd07806da60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd09815c740 tx=0x7fd084008040 comp rx=0 tx=0).stop 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 msgr2=0x7fd098152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd098152460 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd08800a890 tx=0x7fd088001dc0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 shutdown_connections 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd07806b5a0 0x7fd07806da60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0980c8780 0x7fd09815fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0980ca7d0 0x7fd0980c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 --2- 172.21.15.5:0/560298488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c0039b0 0x7fd098152460 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.952+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 >> 172.21.15.5:0/560298488 conn(0x7fd0980b5210 msgr2=0x7fd0980d36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.953+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 shutdown_connections 2024-04-25T10:08:48.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:48.953+0000 7fd09ec00640 1 -- 172.21.15.5:0/560298488 wait complete. 2024-04-25T10:08:48.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.165+0000 7f667f000640 1 Processor -- start 2024-04-25T10:08:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.165+0000 7f667f000640 1 -- start start 2024-04-25T10:08:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.166+0000 7f667f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 0x7f667806e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.166+0000 7f667f000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f667806ed60 0x7f66780cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.166+0000 7f667f000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667806d780 con 0x7f66780cce30 2024-04-25T10:08:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.166+0000 7f667f000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66781423f0 con 0x7f667806ed60 2024-04-25T10:08:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.166+0000 7f667f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66780cc210 con 0x7f6678142010 2024-04-25T10:08:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.167+0000 7f6677e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 0x7f667806e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.167+0000 7f6677e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 0x7f667806e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:46498/0 (socket says 172.21.15.5:46498) 2024-04-25T10:08:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.167+0000 7f6677e00640 1 -- 172.21.15.5:0/2131573908 learned_addr learned my addr 172.21.15.5:0/2131573908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.168+0000 7f6677400640 1 --2- 172.21.15.5:0/2131573908 >> v2:172.21.15.80:3300/0 conn(0x7f667806ed60 0x7f66780cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.168+0000 7f6677e00640 1 -- 172.21.15.5:0/2131573908 >> v2:172.21.15.80:3300/0 conn(0x7f667806ed60 msgr2=0x7f66780cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.168+0000 7f6677e00640 1 --2- 172.21.15.5:0/2131573908 >> v2:172.21.15.80:3300/0 conn(0x7f667806ed60 0x7f66780cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.168+0000 7f6677e00640 1 -- 172.21.15.5:0/2131573908 >> v1:172.21.15.80:6789/0 conn(0x7f66780cce30 legacy=0x7f66781426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.168+0000 7f6677e00640 1 -- 172.21.15.5:0/2131573908 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667806da60 con 0x7f6678142010 2024-04-25T10:08:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.169+0000 7f6677e00640 1 --2- 172.21.15.5:0/2131573908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 0x7f667806e820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6678142570 tx=0x7f666802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1447c16694c285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.170+0000 7f6676a00640 1 -- 172.21.15.5:0/2131573908 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668037070 con 0x7f6678142010 2024-04-25T10:08:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.170+0000 7f6676a00640 1 -- 172.21.15.5:0/2131573908 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6668032440 con 0x7f6678142010 2024-04-25T10:08:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.170+0000 7f6676a00640 1 -- 172.21.15.5:0/2131573908 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668032810 con 0x7f6678142010 2024-04-25T10:08:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 -- 172.21.15.5:0/2131573908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 msgr2=0x7f667806e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 --2- 172.21.15.5:0/2131573908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 0x7f667806e820 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6678142570 tx=0x7f666802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 -- 172.21.15.5:0/2131573908 shutdown_connections 2024-04-25T10:08:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 --2- 172.21.15.5:0/2131573908 >> v2:172.21.15.80:3300/0 conn(0x7f667806ed60 0x7f66780cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 --2- 172.21.15.5:0/2131573908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678142010 0x7f667806e820 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 -- 172.21.15.5:0/2131573908 >> 172.21.15.5:0/2131573908 conn(0x7f6678066f80 msgr2=0x7f667806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 -- 172.21.15.5:0/2131573908 shutdown_connections 2024-04-25T10:08:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.171+0000 7f667f000640 1 -- 172.21.15.5:0/2131573908 wait complete. 2024-04-25T10:08:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.173+0000 7f667f000640 1 Processor -- start 2024-04-25T10:08:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.173+0000 7f667f000640 1 -- start start 2024-04-25T10:08:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.174+0000 7f667f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f667806ed60 0x7f667813fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.174+0000 7f667f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 0x7f667813db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.174+0000 7f667f000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6678142010 0x7f667813e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.174+0000 7f667f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667806e300 con 0x7f66780cce30 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f667f000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667806e020 con 0x7f6678142010 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f667f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6678142db0 con 0x7f667806ed60 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f6677400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 0x7f667813db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f6677400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 0x7f667813db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59688/0 (socket says 172.21.15.5:59688) 2024-04-25T10:08:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f6677400640 1 -- 172.21.15.5:0/2202578363 learned_addr learned my addr 172.21.15.5:0/2202578363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f667d000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6678142010 0x7f667813e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.175+0000 7f6677e00640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f667806ed60 0x7f667813fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f6677400640 1 -- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f667806ed60 msgr2=0x7f667813fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f6677400640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f667806ed60 0x7f667813fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f6677400640 1 -- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6678142010 msgr2=0x7f667813e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f6677400640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6678142010 0x7f667813e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f6677400640 1 -- 172.21.15.5:0/2202578363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66781f45d0 con 0x7f66780cce30 2024-04-25T10:08:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f6677400640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 0x7f667813db60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f666000d800 tx=0x7f666000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666000b6c0 con 0x7f66780cce30 2024-04-25T10:08:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6660019070 con 0x7f66780cce30 2024-04-25T10:08:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666000ba80 con 0x7f66780cce30 2024-04-25T10:08:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.176+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66781f4bb0 con 0x7f66780cce30 2024-04-25T10:08:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.177+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66781f5170 con 0x7f66780cce30 2024-04-25T10:08:49.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.178+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678069040 con 0x7f66780cce30 2024-04-25T10:08:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.179+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f666000bc20 con 0x7f66780cce30 2024-04-25T10:08:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.180+0000 7f666fe00640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f665406b4d0 0x7f665406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.180+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f666008a6f0 con 0x7f66780cce30 2024-04-25T10:08:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.181+0000 7f6677e00640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f665406b4d0 0x7f665406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.182+0000 7f6677e00640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f665406b4d0 0x7f665406d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6668034000 tx=0x7f666802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.187+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f666001c360 con 0x7f66780cce30 2024-04-25T10:08:49.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.384+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6678067d70 con 0x7f66780cce30 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.961+0000 7f666fe00640 1 -- 172.21.15.5:0/2202578363 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f66600594e0 con 0x7f66780cce30 2024-04-25T10:08:49.968 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.967+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f665406b4d0 msgr2=0x7f665406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.967+0000 7f667f000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f665406b4d0 0x7f665406d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6668034000 tx=0x7f666802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.967+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 msgr2=0x7f667813db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.967+0000 7f667f000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 0x7f667813db60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f666000d800 tx=0x7f666000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.968+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 shutdown_connections 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.968+0000 7f667f000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f665406b4d0 0x7f665406d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.968+0000 7f667f000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6678142010 0x7f667813e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.968+0000 7f667f000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66780cce30 0x7f667813db60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.968+0000 7f667f000640 1 --2- 172.21.15.5:0/2202578363 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f667806ed60 0x7f667813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.968+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 >> 172.21.15.5:0/2202578363 conn(0x7f6678066f80 msgr2=0x7f667815b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.969+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 shutdown_connections 2024-04-25T10:08:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:49.969+0000 7f667f000640 1 -- 172.21.15.5:0/2202578363 wait complete. 2024-04-25T10:08:49.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.207+0000 7ff265600640 1 Processor -- start 2024-04-25T10:08:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.207+0000 7ff265600640 1 -- start start 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.209+0000 7ff265600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff26015ed20 0x7ff260144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.209+0000 7ff265600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff260071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.209+0000 7ff265600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26015f3e0 con 0x7ff260071540 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.209+0000 7ff265600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2600719b0 con 0x7ff26015ed20 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.209+0000 7ff265600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff260144780 con 0x7ff260144e90 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff26015ed20 0x7ff260144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff264c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff260071540 0x7ff26006d020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60446/0 (socket says 172.21.15.5:60446) 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff26015ed20 0x7ff260144950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59696/0 (socket says 172.21.15.5:59696) 2024-04-25T10:08:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff264c00640 1 -- 172.21.15.5:0/2875039726 learned_addr learned my addr 172.21.15.5:0/2875039726 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25fe00640 1 -- 172.21.15.5:0/2875039726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 msgr2=0x7ff260071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25f400640 1 --2- 172.21.15.5:0/2875039726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff260071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25fe00640 1 --2- 172.21.15.5:0/2875039726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff260071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25fe00640 1 -- 172.21.15.5:0/2875039726 >> v1:172.21.15.5:6789/0 conn(0x7ff260071540 legacy=0x7ff26006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25fe00640 1 -- 172.21.15.5:0/2875039726 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26015f680 con 0x7ff26015ed20 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.210+0000 7ff25ea00640 1 -- 172.21.15.5:0/2875039726 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395286222 0 0) 0x7ff26015f3e0 con 0x7ff260071540 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.211+0000 7ff25fe00640 1 --2- 172.21.15.5:0/2875039726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff26015ed20 0x7ff260144950 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff250009d90 tx=0x7ff25002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390dae4a7e278dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.212+0000 7ff25ea00640 1 -- 172.21.15.5:0/2875039726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250031600 con 0x7ff26015ed20 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.212+0000 7ff25ea00640 1 -- 172.21.15.5:0/2875039726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2500317a0 con 0x7ff26015ed20 2024-04-25T10:08:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.212+0000 7ff25ea00640 1 -- 172.21.15.5:0/2875039726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250031b70 con 0x7ff26015ed20 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.212+0000 7ff265600640 1 -- 172.21.15.5:0/2875039726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff26015ed20 msgr2=0x7ff260144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.212+0000 7ff265600640 1 --2- 172.21.15.5:0/2875039726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff26015ed20 0x7ff260144950 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff250009d90 tx=0x7ff25002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.213+0000 7ff265600640 1 -- 172.21.15.5:0/2875039726 shutdown_connections 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.213+0000 7ff265600640 1 --2- 172.21.15.5:0/2875039726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff260071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.213+0000 7ff265600640 1 --2- 172.21.15.5:0/2875039726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff26015ed20 0x7ff260144950 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.213+0000 7ff265600640 1 -- 172.21.15.5:0/2875039726 >> 172.21.15.5:0/2875039726 conn(0x7ff260066f20 msgr2=0x7ff26006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.213+0000 7ff265600640 1 -- 172.21.15.5:0/2875039726 shutdown_connections 2024-04-25T10:08:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.213+0000 7ff265600640 1 -- 172.21.15.5:0/2875039726 wait complete. 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.215+0000 7ff265600640 1 Processor -- start 2024-04-25T10:08:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.215+0000 7ff265600640 1 -- start start 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff265600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 0x7ff2601e5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff265600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff2601e5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff265600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26015ed20 0x7ff26012eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff265600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2600cdd50 con 0x7ff260071540 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff265600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26015fc40 con 0x7ff26015ed20 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff265600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26015cdd0 con 0x7ff260144e90 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.216+0000 7ff25fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 0x7ff2601e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 0x7ff2601e5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59702/0 (socket says 172.21.15.5:59702) 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 -- 172.21.15.5:0/3311182032 learned_addr learned my addr 172.21.15.5:0/3311182032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 -- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 msgr2=0x7ff2601e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff2601e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 -- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26015ed20 msgr2=0x7ff26012eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26015ed20 0x7ff26012eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 -- 172.21.15.5:0/3311182032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600cbe50 con 0x7ff260071540 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.217+0000 7ff25fe00640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 0x7ff2601e5010 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff250009d90 tx=0x7ff250004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.218+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25003f070 con 0x7ff260071540 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.218+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26012cbf0 con 0x7ff260071540 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.218+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff250030be0 con 0x7ff260071540 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.218+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250043410 con 0x7ff260071540 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.218+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26012d1b0 con 0x7ff260071540 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.219+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff260068f60 con 0x7ff260071540 2024-04-25T10:08:50.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.220+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2500435b0 con 0x7ff260071540 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.221+0000 7ff257e00640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff24006b4d0 0x7ff24006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.221+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff2500ab490 con 0x7ff260071540 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.221+0000 7ff25f400640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff24006b4d0 0x7ff24006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.223+0000 7ff25f400640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff24006b4d0 0x7ff24006d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006fd0 tx=0x7ff24c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.225+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff25007a470 con 0x7ff260071540 2024-04-25T10:08:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.426+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff26006ec00 con 0x7ff260071540 2024-04-25T10:08:50.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.433+0000 7ff257e00640 1 -- 172.21.15.5:0/3311182032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff250079d70 con 0x7ff260071540 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff24006b4d0 msgr2=0x7ff24006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff24006b4d0 0x7ff24006d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006fd0 tx=0x7ff24c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 msgr2=0x7ff2601e5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 0x7ff2601e5010 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff250009d90 tx=0x7ff250004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 shutdown_connections 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff24006b4d0 0x7ff24006d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26015ed20 0x7ff26012eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff260144e90 0x7ff2601e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 --2- 172.21.15.5:0/3311182032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260071540 0x7ff2601e5010 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 >> 172.21.15.5:0/3311182032 conn(0x7ff260066f20 msgr2=0x7ff260143ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.436+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 shutdown_connections 2024-04-25T10:08:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.437+0000 7ff265600640 1 -- 172.21.15.5:0/3311182032 wait complete. 2024-04-25T10:08:50.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:08:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.649+0000 7ff4ba400640 1 Processor -- start 2024-04-25T10:08:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.649+0000 7ff4ba400640 1 -- start start 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b406e970 0x7ff4b40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4ba400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff4b40ced20 0x7ff4b415b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4ba400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40cdbf0 con 0x7ff4b4144100 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4ba400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b41444e0 con 0x7ff4b40ced20 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4ba400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415ac70 con 0x7ff4b406e970 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4b3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff4b40ced20 0x7ff4b415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4b3400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff4b4144100 0x7ff4b406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60462/0 (socket says 172.21.15.5:60462) 2024-04-25T10:08:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.650+0000 7ff4b3400640 1 -- 172.21.15.5:0/2260031194 learned_addr learned my addr 172.21.15.5:0/2260031194 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.651+0000 7ff4b2a00640 1 --2- 172.21.15.5:0/2260031194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b406e970 0x7ff4b40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.651+0000 7ff4b3e00640 1 -- 172.21.15.5:0/2260031194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b406e970 msgr2=0x7ff4b40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.651+0000 7ff4b3e00640 1 --2- 172.21.15.5:0/2260031194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b406e970 0x7ff4b40cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.651+0000 7ff4b3e00640 1 -- 172.21.15.5:0/2260031194 >> v1:172.21.15.5:6789/0 conn(0x7ff4b4144100 legacy=0x7ff4b406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.651+0000 7ff4b3e00640 1 -- 172.21.15.5:0/2260031194 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40cded0 con 0x7ff4b40ced20 2024-04-25T10:08:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.651+0000 7ff4b2000640 1 -- 172.21.15.5:0/2260031194 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82305821 0 0) 0x7ff4b40cdbf0 con 0x7ff4b4144100 2024-04-25T10:08:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.652+0000 7ff4b3e00640 1 --2- 172.21.15.5:0/2260031194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ced20 0x7ff4b415b130 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0097a0 tx=0x7ff4ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2bad67125f8acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4b2000640 1 -- 172.21.15.5:0/2260031194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac037070 con 0x7ff4b40ced20 2024-04-25T10:08:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4b2000640 1 -- 172.21.15.5:0/2260031194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac032440 con 0x7ff4b40ced20 2024-04-25T10:08:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4b2000640 1 -- 172.21.15.5:0/2260031194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac032810 con 0x7ff4b40ced20 2024-04-25T10:08:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 -- 172.21.15.5:0/2260031194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ced20 msgr2=0x7ff4b415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2260031194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ced20 0x7ff4b415b130 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0097a0 tx=0x7ff4ac02a180 comp rx=0 tx=0).stop 2024-04-25T10:08:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 -- 172.21.15.5:0/2260031194 shutdown_connections 2024-04-25T10:08:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2260031194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ced20 0x7ff4b415b130 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2260031194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b406e970 0x7ff4b40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 -- 172.21.15.5:0/2260031194 >> 172.21.15.5:0/2260031194 conn(0x7ff4b4071150 msgr2=0x7ff4b4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 -- 172.21.15.5:0/2260031194 shutdown_connections 2024-04-25T10:08:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.653+0000 7ff4ba400640 1 -- 172.21.15.5:0/2260031194 wait complete. 2024-04-25T10:08:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.655+0000 7ff4ba400640 1 Processor -- start 2024-04-25T10:08:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.655+0000 7ff4ba400640 1 -- start start 2024-04-25T10:08:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 0x7ff4b413a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40ced20 0x7ff4b4157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4ba400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4144100 0x7ff4b413df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4ba400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40cea00 con 0x7ff4b406e970 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4ba400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40ce490 con 0x7ff4b40ced20 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4ba400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415b670 con 0x7ff4b4144100 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.656+0000 7ff4b3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 0x7ff4b413a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 0x7ff4b413a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59718/0 (socket says 172.21.15.5:59718) 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 -- 172.21.15.5:0/2055447712 learned_addr learned my addr 172.21.15.5:0/2055447712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3e00640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4144100 0x7ff4b413df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b2a00640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40ced20 0x7ff4b4157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 -- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4144100 msgr2=0x7ff4b413df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4144100 0x7ff4b413df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 -- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40ced20 msgr2=0x7ff4b4157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40ced20 0x7ff4b4157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 -- 172.21.15.5:0/2055447712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b413ca70 con 0x7ff4b406e970 2024-04-25T10:08:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.657+0000 7ff4b3400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 0x7ff4b413a190 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c7f0 tx=0x7ff4a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.658+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4018070 con 0x7ff4b406e970 2024-04-25T10:08:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.658+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400a640 con 0x7ff4b406e970 2024-04-25T10:08:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.658+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400aa30 con 0x7ff4b406e970 2024-04-25T10:08:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.658+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b413cd50 con 0x7ff4b406e970 2024-04-25T10:08:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.658+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b413d310 con 0x7ff4b406e970 2024-04-25T10:08:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.659+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b4068fd0 con 0x7ff4b406e970 2024-04-25T10:08:50.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.660+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4a401b070 con 0x7ff4b406e970 2024-04-25T10:08:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.661+0000 7ff4ab400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b4d0 0x7ff49406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.661+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff4a4089800 con 0x7ff4b406e970 2024-04-25T10:08:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.662+0000 7ff4b2a00640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b4d0 0x7ff49406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.663+0000 7ff4b2a00640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b4d0 0x7ff49406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4b41435a0 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.665+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4a4058700 con 0x7ff4b406e970 2024-04-25T10:08:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.941+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7ff4b413a6d0 con 0x7ff4b406e970 2024-04-25T10:08:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.964+0000 7ff4ab400640 1 -- 172.21.15.5:0/2055447712 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v352) v1 ==== 166+0+0 (secure 0 0 0) 0x7ff4a4058000 con 0x7ff4b406e970 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b4d0 msgr2=0x7ff49406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b4d0 0x7ff49406d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff4b41435a0 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 msgr2=0x7ff4b413a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 0x7ff4b413a190 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c7f0 tx=0x7ff4a400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 shutdown_connections 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4144100 0x7ff4b413df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b4d0 0x7ff49406d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40ced20 0x7ff4b4157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 --2- 172.21.15.5:0/2055447712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b406e970 0x7ff4b413a190 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 >> 172.21.15.5:0/2055447712 conn(0x7ff4b4071150 msgr2=0x7ff4b40ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 shutdown_connections 2024-04-25T10:08:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:50.968+0000 7ff4ba400640 1 -- 172.21.15.5:0/2055447712 wait complete. 2024-04-25T10:08:50.994 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:08:50.994 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:08:50.995 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:08:50.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:51.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.202+0000 7f716b200640 1 Processor -- start 2024-04-25T10:08:51.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.203+0000 7f716b200640 1 -- start start 2024-04-25T10:08:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.204+0000 7f716b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f716c15ba20 0x7f716c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.204+0000 7f716b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.204+0000 7f716b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c0cc060 con 0x7f716c09ab40 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.204+0000 7f716b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c15be00 con 0x7f716c15ba20 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.204+0000 7f716b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0d0320 con 0x7f716c0999f0 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f716c15ba20 0x7f716c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f716c15ba20 0x7f716c0cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59726/0 (socket says 172.21.15.5:59726) 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 -- 172.21.15.5:0/1265279774 learned_addr learned my addr 172.21.15.5:0/1265279774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 -- 172.21.15.5:0/1265279774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 msgr2=0x7f716c099e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 --2- 172.21.15.5:0/1265279774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 -- 172.21.15.5:0/1265279774 >> v1:172.21.15.5:6789/0 conn(0x7f716c09ab40 legacy=0x7f716c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.205+0000 7f7169e00640 1 -- 172.21.15.5:0/1265279774 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0cc300 con 0x7f716c15ba20 2024-04-25T10:08:51.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.206+0000 7f7169e00640 1 --2- 172.21.15.5:0/1265279774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0cd0c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f715c009870 tx=0x7f715c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589c1d9980477c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f7168a00640 1 -- 172.21.15.5:0/1265279774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c037070 con 0x7f716c15ba20 2024-04-25T10:08:51.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f7168a00640 1 -- 172.21.15.5:0/1265279774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c02ae50 con 0x7f716c15ba20 2024-04-25T10:08:51.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f7168a00640 1 -- 172.21.15.5:0/1265279774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c032860 con 0x7f716c15ba20 2024-04-25T10:08:51.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f716b200640 1 -- 172.21.15.5:0/1265279774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 msgr2=0x7f716c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f716b200640 1 --2- 172.21.15.5:0/1265279774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0cd0c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f715c009870 tx=0x7f715c02a300 comp rx=0 tx=0).stop 2024-04-25T10:08:51.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f716b200640 1 -- 172.21.15.5:0/1265279774 shutdown_connections 2024-04-25T10:08:51.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f716b200640 1 --2- 172.21.15.5:0/1265279774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f716b200640 1 --2- 172.21.15.5:0/1265279774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0cd0c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.207+0000 7f716b200640 1 -- 172.21.15.5:0/1265279774 >> 172.21.15.5:0/1265279774 conn(0x7f716c0b2f60 msgr2=0x7f716c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.208+0000 7f716b200640 1 -- 172.21.15.5:0/1265279774 shutdown_connections 2024-04-25T10:08:51.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.208+0000 7f716b200640 1 -- 172.21.15.5:0/1265279774 wait complete. 2024-04-25T10:08:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.209+0000 7f716b200640 1 Processor -- start 2024-04-25T10:08:51.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.210+0000 7f716b200640 1 -- start start 2024-04-25T10:08:51.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.210+0000 7f716b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c14c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f716c09ab40 0x7f716c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0ccba0 con 0x7f716c15ba20 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0cc8c0 con 0x7f716c09ab40 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0b0a50 con 0x7f716c0999f0 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59734/0 (socket says 172.21.15.5:59734) 2024-04-25T10:08:51.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 -- 172.21.15.5:0/3330468809 learned_addr learned my addr 172.21.15.5:0/3330468809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f7169400640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f716c09ab40 0x7f716c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f7169e00640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c14c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 -- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 msgr2=0x7f716c14c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c14c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 -- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f716c09ab40 msgr2=0x7f716c0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f716c09ab40 0x7f716c0cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f716a800640 1 -- 172.21.15.5:0/3330468809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c1f4670 con 0x7f716c15ba20 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f7169e00640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c14c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.211+0000 7f7169400640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f716c09ab40 0x7f716c0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.212+0000 7f716a800640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0b4f30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7164009fd0 tx=0x7f716400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.212+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011830 con 0x7f716c15ba20 2024-04-25T10:08:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.212+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164016070 con 0x7f716c15ba20 2024-04-25T10:08:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.212+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164011bf0 con 0x7f716c15ba20 2024-04-25T10:08:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.212+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c0ca530 con 0x7f716c15ba20 2024-04-25T10:08:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.212+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1f4950 con 0x7f716c15ba20 2024-04-25T10:08:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.213+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c068f60 con 0x7f716c15ba20 2024-04-25T10:08:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.214+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f716401b460 con 0x7f716c15ba20 2024-04-25T10:08:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.216+0000 7f7162000640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f714806b520 0x7f714806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.216+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7164089de0 con 0x7f716c15ba20 2024-04-25T10:08:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.216+0000 7f7169e00640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f714806b520 0x7f714806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.217+0000 7f7169e00640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f714806b520 0x7f714806d9e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f715c034000 tx=0x7f715c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.219+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7164058df0 con 0x7f716c15ba20 2024-04-25T10:08:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.426+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f716c0b3cc0 con 0x7f716c15ba20 2024-04-25T10:08:51.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.428+0000 7f7162000640 1 -- 172.21.15.5:0/3330468809 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+13316 (secure 0 0 0) 0x7f71640586f0 con 0x7f716c15ba20 2024-04-25T10:08:51.434 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:51.434 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:08:49.949282+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6834/3658047546":"2024-04-26T10:08:46.760895+0000","172.21.15.80:6835/1098619647":"2024-04-26T10:07:22.531455+0000","172.21.15.80:6835/3658047546":"2024-04-26T10:08:46.760895+0000","172.21.15.80:6834/1098619647":"2024-04-26T10:07:22.531455+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.430+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f714806b520 msgr2=0x7f714806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.430+0000 7f716b200640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f714806b520 0x7f714806d9e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f715c034000 tx=0x7f715c02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.430+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 msgr2=0x7f716c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.430+0000 7f716b200640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0b4f30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7164009fd0 tx=0x7f716400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 shutdown_connections 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716c15ba20 0x7f716c0b4f30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f714806b520 0x7f714806d9e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f716c09ab40 0x7f716c0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 --2- 172.21.15.5:0/3330468809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f716c0999f0 0x7f716c14c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 >> 172.21.15.5:0/3330468809 conn(0x7f716c0b2f60 msgr2=0x7f716c06c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 shutdown_connections 2024-04-25T10:08:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.431+0000 7f716b200640 1 -- 172.21.15.5:0/3330468809 wait complete. 2024-04-25T10:08:51.457 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:51.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:51.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.671+0000 7f45a6200640 1 Processor -- start 2024-04-25T10:08:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.671+0000 7f45a6200640 1 -- start start 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.672+0000 7f45a6200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45a00b48c0 0x7f45a015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.672+0000 7f45a6200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a00d3f90 0x7f45a0161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.672+0000 7f45a6200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a01606e0 con 0x7f45a0161bc0 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.672+0000 7f45a6200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a01609c0 con 0x7f45a00b48c0 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.672+0000 7f45a6200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0160ca0 con 0x7f45a00d3f90 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a4e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45a00b48c0 0x7f45a015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a5800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f45a0161bc0 0x7f45a00d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60474/0 (socket says 172.21.15.5:60474) 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a5800640 1 -- 172.21.15.5:0/727784073 learned_addr learned my addr 172.21.15.5:0/727784073 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f459fe00640 1 --2- 172.21.15.5:0/727784073 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a00d3f90 0x7f45a0161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a4e00640 1 -- 172.21.15.5:0/727784073 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a00d3f90 msgr2=0x7f45a0161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a4e00640 1 --2- 172.21.15.5:0/727784073 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a00d3f90 0x7f45a0161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a4e00640 1 -- 172.21.15.5:0/727784073 >> v1:172.21.15.5:6789/0 conn(0x7f45a0161bc0 legacy=0x7f45a00d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f459f400640 1 -- 172.21.15.5:0/727784073 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545677703 0 0) 0x7f45a01606e0 con 0x7f45a0161bc0 2024-04-25T10:08:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.673+0000 7f45a4e00640 1 -- 172.21.15.5:0/727784073 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a0160f80 con 0x7f45a00b48c0 2024-04-25T10:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.674+0000 7f45a4e00640 1 --2- 172.21.15.5:0/727784073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a015fe60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f45900097a0 tx=0x7f459002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9252987f8febb4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f459f400640 1 -- 172.21.15.5:0/727784073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590037070 con 0x7f45a00b48c0 2024-04-25T10:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f459f400640 1 -- 172.21.15.5:0/727784073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459002ae50 con 0x7f45a00b48c0 2024-04-25T10:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f459f400640 1 -- 172.21.15.5:0/727784073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590032660 con 0x7f45a00b48c0 2024-04-25T10:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f45a6200640 1 -- 172.21.15.5:0/727784073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 msgr2=0x7f45a015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f45a6200640 1 --2- 172.21.15.5:0/727784073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a015fe60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f45900097a0 tx=0x7f459002a100 comp rx=0 tx=0).stop 2024-04-25T10:08:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f45a6200640 1 -- 172.21.15.5:0/727784073 shutdown_connections 2024-04-25T10:08:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f45a6200640 1 --2- 172.21.15.5:0/727784073 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a00d3f90 0x7f45a0161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f45a6200640 1 --2- 172.21.15.5:0/727784073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a015fe60 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.675+0000 7f45a6200640 1 -- 172.21.15.5:0/727784073 >> 172.21.15.5:0/727784073 conn(0x7f45a008af80 msgr2=0x7f45a00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.676+0000 7f45a6200640 1 -- 172.21.15.5:0/727784073 shutdown_connections 2024-04-25T10:08:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.676+0000 7f45a6200640 1 -- 172.21.15.5:0/727784073 wait complete. 2024-04-25T10:08:51.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.677+0000 7f45a6200640 1 Processor -- start 2024-04-25T10:08:51.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.678+0000 7f45a6200640 1 -- start start 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.678+0000 7f45a6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a01e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a6200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a00d3f90 0x7f45a00ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a6200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a0161bc0 0x7f45a015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a01603a0 con 0x7f45a00b48c0 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a6200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b4ca0 con 0x7f45a00d3f90 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a6200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00b3b80 con 0x7f45a0161bc0 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a01e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a01e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59760/0 (socket says 172.21.15.5:59760) 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 -- 172.21.15.5:0/3604210562 learned_addr learned my addr 172.21.15.5:0/3604210562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f459fe00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a00d3f90 0x7f45a00ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a5800640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a0161bc0 0x7f45a015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 -- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a0161bc0 msgr2=0x7f45a015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a0161bc0 0x7f45a015f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 -- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a00d3f90 msgr2=0x7f45a00ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a00d3f90 0x7f45a00ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f45a4e00640 1 -- 172.21.15.5:0/3604210562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a00cd4c0 con 0x7f45a00b48c0 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.679+0000 7f459fe00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a00d3f90 0x7f45a00ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.680+0000 7f45a4e00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a01e9490 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4590035040 tx=0x7f45900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.680+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590041070 con 0x7f45a00b48c0 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.680+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459003c670 con 0x7f45a00b48c0 2024-04-25T10:08:51.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.680+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590037040 con 0x7f45a00b48c0 2024-04-25T10:08:51.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.680+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a00caab0 con 0x7f45a00b48c0 2024-04-25T10:08:51.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.680+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a00cca60 con 0x7f45a00b48c0 2024-04-25T10:08:51.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.682+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a0068f60 con 0x7f45a00b48c0 2024-04-25T10:08:51.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.683+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f459003a070 con 0x7f45a00b48c0 2024-04-25T10:08:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.683+0000 7f459cc00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f457c06b4d0 0x7f457c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.684+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f45900aa240 con 0x7f45a00b48c0 2024-04-25T10:08:51.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.684+0000 7f459fe00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f457c06b4d0 0x7f457c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.685+0000 7f459fe00640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f457c06b4d0 0x7f457c06d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f45a006dd20 tx=0x7f458800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.688+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45900791e0 con 0x7f45a00b48c0 2024-04-25T10:08:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.885+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45a00b5f40 con 0x7f45a00b48c0 2024-04-25T10:08:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.886+0000 7f459cc00640 1 -- 172.21.15.5:0/3604210562 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4590078ae0 con 0x7f45a00b48c0 2024-04-25T10:08:51.891 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f457c06b4d0 msgr2=0x7f457c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f457c06b4d0 0x7f457c06d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f45a006dd20 tx=0x7f458800b040 comp rx=0 tx=0).stop 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 msgr2=0x7f45a01e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a01e9490 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4590035040 tx=0x7f45900046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 shutdown_connections 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45a0161bc0 0x7f45a015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f457c06b4d0 0x7f457c06d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a00d3f90 0x7f45a00ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 --2- 172.21.15.5:0/3604210562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a00b48c0 0x7f45a01e9490 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 >> 172.21.15.5:0/3604210562 conn(0x7f45a008af80 msgr2=0x7f45a00d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 shutdown_connections 2024-04-25T10:08:51.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:51.888+0000 7f45a6200640 1 -- 172.21.15.5:0/3604210562 wait complete. 2024-04-25T10:08:51.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:08:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.117+0000 7f56efe00640 1 Processor -- start 2024-04-25T10:08:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.117+0000 7f56efe00640 1 -- start start 2024-04-25T10:08:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56efe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56f0143d30 0x7f56f015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56efe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56f00ceea0 0x7f56f00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56efe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f00b4880 con 0x7f56f00b3ae0 2024-04-25T10:08:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56efe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00b45a0 con 0x7f56f0143d30 2024-04-25T10:08:52.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56efe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f015e2c0 con 0x7f56f00ceea0 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56eea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56f0143d30 0x7f56f015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56eea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56f0143d30 0x7f56f015eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59764/0 (socket says 172.21.15.5:59764) 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56eea00640 1 -- 172.21.15.5:0/1639208474 learned_addr learned my addr 172.21.15.5:0/1639208474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.119+0000 7f56ef400640 1 --2- 172.21.15.5:0/1639208474 >> v2:172.21.15.80:3300/0 conn(0x7f56f00ceea0 0x7f56f00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.120+0000 7f56eea00640 1 -- 172.21.15.5:0/1639208474 >> v2:172.21.15.80:3300/0 conn(0x7f56f00ceea0 msgr2=0x7f56f00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.120+0000 7f56eea00640 1 --2- 172.21.15.5:0/1639208474 >> v2:172.21.15.80:3300/0 conn(0x7f56f00ceea0 0x7f56f00b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.120+0000 7f56eea00640 1 -- 172.21.15.5:0/1639208474 >> v1:172.21.15.80:6789/0 conn(0x7f56f00b3ae0 legacy=0x7f56f00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.120+0000 7f56eea00640 1 -- 172.21.15.5:0/1639208474 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f015e5a0 con 0x7f56f0143d30 2024-04-25T10:08:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.120+0000 7f56ef400640 1 --2- 172.21.15.5:0/1639208474 >> v2:172.21.15.80:3300/0 conn(0x7f56f00ceea0 0x7f56f00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.121+0000 7f56eea00640 1 --2- 172.21.15.5:0/1639208474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f015eda0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f56f006f4a0 tx=0x7f56e002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5349dc165efa7e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.121+0000 7f56ee000640 1 -- 172.21.15.5:0/1639208474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00325a0 con 0x7f56f0143d30 2024-04-25T10:08:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.121+0000 7f56ee000640 1 -- 172.21.15.5:0/1639208474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0038070 con 0x7f56f0143d30 2024-04-25T10:08:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.121+0000 7f56ee000640 1 -- 172.21.15.5:0/1639208474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0032960 con 0x7f56f0143d30 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 -- 172.21.15.5:0/1639208474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 msgr2=0x7f56f015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 --2- 172.21.15.5:0/1639208474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f015eda0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f56f006f4a0 tx=0x7f56e002c8e0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 -- 172.21.15.5:0/1639208474 shutdown_connections 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 --2- 172.21.15.5:0/1639208474 >> v2:172.21.15.80:3300/0 conn(0x7f56f00ceea0 0x7f56f00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 --2- 172.21.15.5:0/1639208474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f015eda0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 -- 172.21.15.5:0/1639208474 >> 172.21.15.5:0/1639208474 conn(0x7f56f0066f20 msgr2=0x7f56f006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 -- 172.21.15.5:0/1639208474 shutdown_connections 2024-04-25T10:08:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.122+0000 7f56efe00640 1 -- 172.21.15.5:0/1639208474 wait complete. 2024-04-25T10:08:52.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.124+0000 7f56efe00640 1 Processor -- start 2024-04-25T10:08:52.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.124+0000 7f56efe00640 1 -- start start 2024-04-25T10:08:52.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56efe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56f00b3ae0 0x7f56f00c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56efe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56f00ceea0 0x7f56f00caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f00cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56efe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0144290 con 0x7f56f0143d30 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56efe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0144110 con 0x7f56f00ceea0 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56efe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f006e430 con 0x7f56f00b3ae0 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f00cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f00cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59776/0 (socket says 172.21.15.5:59776) 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56ef400640 1 -- 172.21.15.5:0/633439089 learned_addr learned my addr 172.21.15.5:0/633439089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56e6000640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56f00ceea0 0x7f56f00caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.125+0000 7f56eea00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56f00b3ae0 0x7f56f00c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56ef400640 1 -- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56f00b3ae0 msgr2=0x7f56f00c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56ef400640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56f00b3ae0 0x7f56f00c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56ef400640 1 -- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56f00ceea0 msgr2=0x7f56f00caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56ef400640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56f00ceea0 0x7f56f00caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56ef400640 1 -- 172.21.15.5:0/633439089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00c4320 con 0x7f56f0143d30 2024-04-25T10:08:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56ef400640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f00cb530 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f56e800ced0 tx=0x7f56e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8003b90 con 0x7f56f0143d30 2024-04-25T10:08:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8003d50 con 0x7f56f0143d30 2024-04-25T10:08:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8004260 con 0x7f56f0143d30 2024-04-25T10:08:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.126+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f00c45c0 con 0x7f56f0143d30 2024-04-25T10:08:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.127+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f00c4b80 con 0x7f56f0143d30 2024-04-25T10:08:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.128+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f0068f60 con 0x7f56f0143d30 2024-04-25T10:08:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.129+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56e8004400 con 0x7f56f0143d30 2024-04-25T10:08:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.130+0000 7f56e7e00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.130+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f56e808ab70 con 0x7f56f0143d30 2024-04-25T10:08:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.130+0000 7f56eea00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.132+0000 7f56eea00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56e0037000 tx=0x7f56e003e040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.134+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56e8059b30 con 0x7f56f0143d30 2024-04-25T10:08:52.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.407+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f56f0158d20 con 0x7f56f0143d30 2024-04-25T10:08:52.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.977+0000 7f56e7e00640 1 -- 172.21.15.5:0/633439089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v354) v1 ==== 170+0+0 (secure 0 0 0) 0x7f56e8059430 con 0x7f56f0143d30 2024-04-25T10:08:52.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56cc06b4d0 msgr2=0x7f56cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56e0037000 tx=0x7f56e003e040 comp rx=0 tx=0).stop 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 msgr2=0x7f56f00cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f00cb530 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f56e800ced0 tx=0x7f56e800a710 comp rx=0 tx=0).stop 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 shutdown_connections 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56f0143d30 0x7f56f00cb530 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56f00ceea0 0x7f56f00caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 --2- 172.21.15.5:0/633439089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56f00b3ae0 0x7f56f00c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 >> 172.21.15.5:0/633439089 conn(0x7f56f0066f20 msgr2=0x7f56f0159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 shutdown_connections 2024-04-25T10:08:52.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:52.981+0000 7f56efe00640 1 -- 172.21.15.5:0/633439089 wait complete. 2024-04-25T10:08:53.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:08:53.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.225+0000 7fc2f2200640 1 Processor -- start 2024-04-25T10:08:53.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.225+0000 7fc2f2200640 1 -- start start 2024-04-25T10:08:53.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.226+0000 7fc2f2200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec0cc840 0x7fc2ec06e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.226+0000 7fc2f2200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2ec15b500 0x7fc2ec071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.226+0000 7fc2f2200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec0ce570 con 0x7fc2ec06e670 2024-04-25T10:08:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.226+0000 7fc2f2200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ec06ead0 con 0x7fc2ec15b500 2024-04-25T10:08:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.226+0000 7fc2f2200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0ce850 con 0x7fc2ec0cc840 2024-04-25T10:08:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2ec15b500 0x7fc2ec071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2ec15b500 0x7fc2ec071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59790/0 (socket says 172.21.15.5:59790) 2024-04-25T10:08:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 -- 172.21.15.5:0/2547228380 learned_addr learned my addr 172.21.15.5:0/2547228380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f0e00640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec0cc840 0x7fc2ec06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 -- 172.21.15.5:0/2547228380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec0cc840 msgr2=0x7fc2ec06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec0cc840 0x7fc2ec06e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 -- 172.21.15.5:0/2547228380 >> v1:172.21.15.80:6789/0 conn(0x7fc2ec06e670 legacy=0x7fc2ec15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f1800640 1 -- 172.21.15.5:0/2547228380 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0ceb30 con 0x7fc2ec15b500 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.227+0000 7fc2f0e00640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec0cc840 0x7fc2ec06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.229+0000 7fc2f1800640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec15b500 0x7fc2ec071000 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009e50 tx=0x7fc2e402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaf6e9d0e541860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.229+0000 7fc2eb400640 1 -- 172.21.15.5:0/2547228380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4036070 con 0x7fc2ec15b500 2024-04-25T10:08:53.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.229+0000 7fc2eb400640 1 -- 172.21.15.5:0/2547228380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e4031600 con 0x7fc2ec15b500 2024-04-25T10:08:53.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.229+0000 7fc2eb400640 1 -- 172.21.15.5:0/2547228380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40319d0 con 0x7fc2ec15b500 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.229+0000 7fc2f2200640 1 -- 172.21.15.5:0/2547228380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec15b500 msgr2=0x7fc2ec071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.229+0000 7fc2f2200640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec15b500 0x7fc2ec071000 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009e50 tx=0x7fc2e402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.230+0000 7fc2f2200640 1 -- 172.21.15.5:0/2547228380 shutdown_connections 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.230+0000 7fc2f2200640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec15b500 0x7fc2ec071000 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.230+0000 7fc2f2200640 1 --2- 172.21.15.5:0/2547228380 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec0cc840 0x7fc2ec06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.230+0000 7fc2f2200640 1 -- 172.21.15.5:0/2547228380 >> 172.21.15.5:0/2547228380 conn(0x7fc2ec066f20 msgr2=0x7fc2ec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.230+0000 7fc2f2200640 1 -- 172.21.15.5:0/2547228380 shutdown_connections 2024-04-25T10:08:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.230+0000 7fc2f2200640 1 -- 172.21.15.5:0/2547228380 wait complete. 2024-04-25T10:08:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.231+0000 7fc2f2200640 1 Processor -- start 2024-04-25T10:08:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.232+0000 7fc2f2200640 1 -- start start 2024-04-25T10:08:53.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.232+0000 7fc2f2200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2ec06e670 0x7fc2ec143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f2200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 0x7fc2ec1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f2200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec15b500 0x7fc2ec156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f2200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0ce1d0 con 0x7fc2ec0cc840 2024-04-25T10:08:53.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f2200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0b2d20 con 0x7fc2ec06e670 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f2200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0ce350 con 0x7fc2ec15b500 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 0x7fc2ec1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 0x7fc2ec1540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59800/0 (socket says 172.21.15.5:59800) 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 -- 172.21.15.5:0/1470054310 learned_addr learned my addr 172.21.15.5:0/1470054310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f1800640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec15b500 0x7fc2ec156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f0e00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2ec06e670 0x7fc2ec143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 -- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec15b500 msgr2=0x7fc2ec156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec15b500 0x7fc2ec156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 -- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2ec06e670 msgr2=0x7fc2ec143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2ec06e670 0x7fc2ec143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2ebe00640 1 -- 172.21.15.5:0/1470054310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec143b60 con 0x7fc2ec0cc840 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f1800640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec15b500 0x7fc2ec156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.233+0000 7fc2f0e00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2ec06e670 0x7fc2ec143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.234+0000 7fc2ebe00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 0x7fc2ec1540e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c430 tx=0x7fc2d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.234+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8011790 con 0x7fc2ec0cc840 2024-04-25T10:08:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.234+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d8017070 con 0x7fc2ec0cc840 2024-04-25T10:08:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.234+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8011bb0 con 0x7fc2ec0cc840 2024-04-25T10:08:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.234+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec0b1880 con 0x7fc2ec0cc840 2024-04-25T10:08:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.234+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec0b1e40 con 0x7fc2ec0cc840 2024-04-25T10:08:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.235+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec0cd7f0 con 0x7fc2ec0cc840 2024-04-25T10:08:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.236+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2d801c460 con 0x7fc2ec0cc840 2024-04-25T10:08:53.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.238+0000 7fc2e8c00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2cc06b4d0 0x7fc2cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.238+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc2d8089fe0 con 0x7fc2ec0cc840 2024-04-25T10:08:53.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.239+0000 7fc2f0e00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2cc06b4d0 0x7fc2cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.240+0000 7fc2f0e00640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2cc06b4d0 0x7fc2cc06d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0cb730 tx=0x7fc2dc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.242+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2ec0cd7f0 con 0x7fc2ec0cc840 2024-04-25T10:08:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.440+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc2ec068f60 con 0x7fc2ec0cc840 2024-04-25T10:08:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.446+0000 7fc2e8c00640 1 -- 172.21.15.5:0/1470054310 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2d8058f40 con 0x7fc2ec0cc840 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2cc06b4d0 msgr2=0x7fc2cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2cc06b4d0 0x7fc2cc06d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0cb730 tx=0x7fc2dc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 msgr2=0x7fc2ec1540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 0x7fc2ec1540e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800c430 tx=0x7fc2d800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 shutdown_connections 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2ec15b500 0x7fc2ec156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2ec0cc840 0x7fc2ec1540e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2cc06b4d0 0x7fc2cc06d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 --2- 172.21.15.5:0/1470054310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2ec06e670 0x7fc2ec143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 >> 172.21.15.5:0/1470054310 conn(0x7fc2ec066f20 msgr2=0x7fc2ec0ac8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 shutdown_connections 2024-04-25T10:08:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:53.449+0000 7fc2f2200640 1 -- 172.21.15.5:0/1470054310 wait complete. 2024-04-25T10:08:53.479 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:53.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:53.596 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:53.707 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:53.707 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:08:53.707 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:53.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:53.768 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:08:53.768 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:53.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:53.826 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:08:53.827 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:53.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:53.885 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:08:53.885 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:53.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:53.942 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:08:53.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:54.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.159+0000 7f271b400640 1 Processor -- start 2024-04-25T10:08:54.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.159+0000 7f271b400640 1 -- start start 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.160+0000 7f271b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f271415da60 0x7f27140d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.160+0000 7f271b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f271b400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271415e8a0 con 0x7f271415de40 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f271b400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271415eb80 con 0x7f271415da60 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f271b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271415ee60 con 0x7f271415e2d0 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2718a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f271415da60 0x7f27140d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2713e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f271415de40 0x7f2714161d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60486/0 (socket says 172.21.15.5:60486) 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2718a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f271415da60 0x7f27140d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59806/0 (socket says 172.21.15.5:59806) 2024-04-25T10:08:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2713e00640 1 -- 172.21.15.5:0/267035422 learned_addr learned my addr 172.21.15.5:0/267035422 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2719400640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2718a00640 1 -- 172.21.15.5:0/267035422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 msgr2=0x7f27140ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2718a00640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.162+0000 7f2718a00640 1 -- 172.21.15.5:0/267035422 >> v1:172.21.15.5:6789/0 conn(0x7f271415de40 legacy=0x7f2714161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.161+0000 7f2713400640 1 -- 172.21.15.5:0/267035422 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930070411 0 0) 0x7f271415e8a0 con 0x7f271415de40 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.162+0000 7f2718a00640 1 -- 172.21.15.5:0/267035422 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2714161850 con 0x7f271415da60 2024-04-25T10:08:54.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.162+0000 7f2719400640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:54.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.163+0000 7f2718a00640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415da60 0x7f27140d0aa0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f27040097a0 tx=0x7f270402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167abc68cde48a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.163+0000 7f2713400640 1 -- 172.21.15.5:0/267035422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704037070 con 0x7f271415da60 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.163+0000 7f2713400640 1 -- 172.21.15.5:0/267035422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270402ae50 con 0x7f271415da60 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.163+0000 7f2713400640 1 -- 172.21.15.5:0/267035422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704032660 con 0x7f271415da60 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 -- 172.21.15.5:0/267035422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415da60 msgr2=0x7f27140d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415da60 0x7f27140d0aa0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f27040097a0 tx=0x7f270402a100 comp rx=0 tx=0).stop 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 -- 172.21.15.5:0/267035422 shutdown_connections 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 --2- 172.21.15.5:0/267035422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415da60 0x7f27140d0aa0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 -- 172.21.15.5:0/267035422 >> 172.21.15.5:0/267035422 conn(0x7f271406bfe0 msgr2=0x7f271406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 -- 172.21.15.5:0/267035422 shutdown_connections 2024-04-25T10:08:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.164+0000 7f271b400640 1 -- 172.21.15.5:0/267035422 wait complete. 2024-04-25T10:08:54.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.166+0000 7f271b400640 1 Processor -- start 2024-04-25T10:08:54.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.166+0000 7f271b400640 1 -- start start 2024-04-25T10:08:54.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f271b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f271415da60 0x7f27140aa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f271b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 0x7f27140b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f271b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f271b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140cebe0 con 0x7f271415de40 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f271b400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140cea60 con 0x7f271415da60 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f271b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27140ced60 con 0x7f271415e2d0 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f2713e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 0x7f27140b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f2713e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 0x7f27140b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59812/0 (socket says 172.21.15.5:59812) 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f2713e00640 1 -- 172.21.15.5:0/2791234944 learned_addr learned my addr 172.21.15.5:0/2791234944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.167+0000 7f2718a00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f271415da60 0x7f27140aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2719400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2713e00640 1 -- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 msgr2=0x7f27140cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2713e00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2713e00640 1 -- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f271415da60 msgr2=0x7f27140aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2713e00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f271415da60 0x7f27140aa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2713e00640 1 -- 172.21.15.5:0/2791234944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271414cb60 con 0x7f271415de40 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2719400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2713e00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 0x7f27140b1b50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c430 tx=0x7f26fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0074e0 con 0x7f271415de40 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc0047e0 con 0x7f271415de40 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004bd0 con 0x7f271415de40 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271414ce40 con 0x7f271415de40 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.168+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271414d400 con 0x7f271415de40 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.170+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27140cf7f0 con 0x7f271415de40 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.171+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26fc011070 con 0x7f271415de40 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.171+0000 7f2710c00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f26f006b500 0x7f26f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.172+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f26fc089e00 con 0x7f271415de40 2024-04-25T10:08:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.172+0000 7f2718a00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f26f006b500 0x7f26f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.174+0000 7f2718a00640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f26f006b500 0x7f26f006d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2704034000 tx=0x7f270402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.177+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27140cf7f0 con 0x7f271415de40 2024-04-25T10:08:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.414+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2714068fd0 con 0x7f271415de40 2024-04-25T10:08:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.415+0000 7f2710c00640 1 -- 172.21.15.5:0/2791234944 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3687 (secure 0 0 0) 0x7f26fc00e280 con 0x7f271415de40 2024-04-25T10:08:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:54.422 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/489236722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":489236722},{"type":"v1","addr":"172.21.15.5:6834","nonce":489236722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/111384147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":111384147},{"type":"v1","addr":"172.21.15.80:6837","nonce":111384147}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5917,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2218256745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2218256745},{"type":"v1","addr":"172.21.15.5:6835","nonce":2218256745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3335610705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3335610705},{"type":"v1","addr":"172.21.15.80:6835","nonce":3335610705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2024-04-25T10:08:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 55 2024-04-25T10:08:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f26f006b500 msgr2=0x7f26f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f26f006b500 0x7f26f006d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2704034000 tx=0x7f270402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 msgr2=0x7f27140b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 0x7f27140b1b50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00c430 tx=0x7f26fc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 shutdown_connections 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f271415e2d0 0x7f27140cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f271415de40 0x7f27140b1b50 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f26f006b500 0x7f26f006d9c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 --2- 172.21.15.5:0/2791234944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f271415da60 0x7f27140aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.418+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 >> 172.21.15.5:0/2791234944 conn(0x7f271406bfe0 msgr2=0x7f2714160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.419+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 shutdown_connections 2024-04-25T10:08:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.419+0000 7f271b400640 1 -- 172.21.15.5:0/2791234944 wait complete. 2024-04-25T10:08:54.444 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:08:54.444 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:08:54.444 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:54.445 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:54.445 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:08:54.445 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:08:54.445 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:54.445 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:54.445 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:08:54.445 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:08:54.446 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:54.446 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:54.446 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:08:54.446 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:08:54.446 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:54.446 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:54.464 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:54.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:08:54.465 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:54.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:08:54.466 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:54.466 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:08:54.467 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:54.467 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:08:54.512 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:08:54.513 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:54.514 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:08:54.515 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:54.518 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:08:54.518 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:54.520 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:08:54.520 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:54.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:08:54.667 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:08:54.667 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:08:54.667 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:08:54.668 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.696+0000 7fb9c7e00640 1 Processor -- start 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.696+0000 7fb9c7e00640 1 -- start start 2024-04-25T10:08:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.697+0000 7fb9c7e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9c8161200 0x7fb9c80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9c7e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80cf410 con 0x7fb9c80d2380 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9c7e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c80cf6f0 con 0x7fb9c80d07d0 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9c7e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c80d27c0 con 0x7fb9c8161200 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9be000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb9c80d2380 0x7fb9c80d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60500/0 (socket says 172.21.15.5:60500) 2024-04-25T10:08:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9be000640 1 -- 172.21.15.5:0/736102262 learned_addr learned my addr 172.21.15.5:0/736102262 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9c6a00640 1 --2- 172.21.15.5:0/736102262 >> v2:172.21.15.80:3300/0 conn(0x7fb9c8161200 0x7fb9c80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9c6000640 1 -- 172.21.15.5:0/736102262 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775638530 0 0) 0x7fb9c80cf410 con 0x7fb9c80d2380 2024-04-25T10:08:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.698+0000 7fb9c6000640 1 -- 172.21.15.5:0/736102262 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ac003520 con 0x7fb9c80d2380 2024-04-25T10:08:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.699+0000 7fb9c6a00640 1 -- 172.21.15.5:0/736102262 >> v1:172.21.15.80:6789/0 conn(0x7fb9c80d07d0 legacy=0x7fb9c80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.699+0000 7fb9c6000640 1 -- 172.21.15.5:0/736102262 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584550596 0 0) 0x7fb9c80cf6f0 con 0x7fb9c80d07d0 2024-04-25T10:08:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.699+0000 7fb9c6a00640 1 -- 172.21.15.5:0/736102262 >> v1:172.21.15.5:6789/0 conn(0x7fb9c80d2380 legacy=0x7fb9c80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.699+0000 7fb9c6a00640 1 -- 172.21.15.5:0/736102262 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80d3fd0 con 0x7fb9c8161200 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.700+0000 7fb9c6a00640 1 --2- 172.21.15.5:0/736102262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c8161200 0x7fb9c80b4bb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00097a0 tx=0x7fb9b002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789ad66a187bfc98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.700+0000 7fb9c6000640 1 -- 172.21.15.5:0/736102262 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0037070 con 0x7fb9c8161200 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.700+0000 7fb9c6000640 1 -- 172.21.15.5:0/736102262 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0032440 con 0x7fb9c8161200 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.700+0000 7fb9c6000640 1 -- 172.21.15.5:0/736102262 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0032810 con 0x7fb9c8161200 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 -- 172.21.15.5:0/736102262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c8161200 msgr2=0x7fb9c80b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/736102262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c8161200 0x7fb9c80b4bb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00097a0 tx=0x7fb9b002a230 comp rx=0 tx=0).stop 2024-04-25T10:08:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 -- 172.21.15.5:0/736102262 shutdown_connections 2024-04-25T10:08:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/736102262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c8161200 0x7fb9c80b4bb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 -- 172.21.15.5:0/736102262 >> 172.21.15.5:0/736102262 conn(0x7fb9c808af80 msgr2=0x7fb9c80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 -- 172.21.15.5:0/736102262 shutdown_connections 2024-04-25T10:08:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.701+0000 7fb9c7e00640 1 -- 172.21.15.5:0/736102262 wait complete. 2024-04-25T10:08:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.703+0000 7fb9c7e00640 1 Processor -- start 2024-04-25T10:08:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.703+0000 7fb9c7e00640 1 -- start start 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c80d07d0 0x7fb9c80b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9c80d2380 0x7fb9c81f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 0x7fb9c814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80b3690 con 0x7fb9c8161200 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80b28b0 con 0x7fb9c80d07d0 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c80b3810 con 0x7fb9c80d2380 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 0x7fb9c814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 0x7fb9c814d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59848/0 (socket says 172.21.15.5:59848) 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c7400640 1 -- 172.21.15.5:0/2686322553 learned_addr learned my addr 172.21.15.5:0/2686322553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9be000640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9c80d2380 0x7fb9c81f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.704+0000 7fb9c6a00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c80d07d0 0x7fb9c80b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7400640 1 -- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9c80d2380 msgr2=0x7fb9c81f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7400640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9c80d2380 0x7fb9c81f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7400640 1 -- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c80d07d0 msgr2=0x7fb9c80b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7400640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c80d07d0 0x7fb9c80b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7400640 1 -- 172.21.15.5:0/2686322553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80b9280 con 0x7fb9c8161200 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c6a00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c80d07d0 0x7fb9c80b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7400640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 0x7fb9c814d200 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c760 tx=0x7fb9c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0003da0 con 0x7fb9c8161200 2024-04-25T10:08:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0005070 con 0x7fb9c8161200 2024-04-25T10:08:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0005460 con 0x7fb9c8161200 2024-04-25T10:08:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c806c8e0 con 0x7fb9c8161200 2024-04-25T10:08:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.705+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c815d490 con 0x7fb9c8161200 2024-04-25T10:08:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.706+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c80ccdc0 con 0x7fb9c8161200 2024-04-25T10:08:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.707+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9c0005600 con 0x7fb9c8161200 2024-04-25T10:08:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.708+0000 7fb9bf400640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a406b4d0 0x7fb9a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.709+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb9c0089ff0 con 0x7fb9c8161200 2024-04-25T10:08:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.709+0000 7fb9c6a00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a406b4d0 0x7fb9a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.710+0000 7fb9c6a00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a406b4d0 0x7fb9a406d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0034000 tx=0x7fb9b002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.713+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9c0058fa0 con 0x7fb9c8161200 2024-04-25T10:08:54.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:54.924+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb9c80d0590 con 0x7fb9c8161200 2024-04-25T10:08:55.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.003+0000 7fb9bf400640 1 -- 172.21.15.5:0/2686322553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb9c00588a0 con 0x7fb9c8161200 2024-04-25T10:08:55.010 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.006+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a406b4d0 msgr2=0x7fb9a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.006+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a406b4d0 0x7fb9a406d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0034000 tx=0x7fb9b002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.006+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 msgr2=0x7fb9c814d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.006+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 0x7fb9c814d200 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c760 tx=0x7fb9c000cc20 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 shutdown_connections 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c8161200 0x7fb9c814d200 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9c80d2380 0x7fb9c81f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a406b4d0 0x7fb9a406d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 --2- 172.21.15.5:0/2686322553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9c80d07d0 0x7fb9c80b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 >> 172.21.15.5:0/2686322553 conn(0x7fb9c808af80 msgr2=0x7fb9c806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 shutdown_connections 2024-04-25T10:08:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.007+0000 7fb9c7e00640 1 -- 172.21.15.5:0/2686322553 wait complete. 2024-04-25T10:08:55.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:08:55.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:55.086 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:55.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.225+0000 7f111fe00640 1 Processor -- start 2024-04-25T10:08:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.225+0000 7f111fe00640 1 -- start start 2024-04-25T10:08:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.226+0000 7f111fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d1590 0x7f11200d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.226+0000 7f111fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f11200d0570 0x7f11200d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.226+0000 7f111fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11200d23f0 con 0x7f112015e4c0 2024-04-25T10:08:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.227+0000 7f111fe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11200cea20 con 0x7f11200d0570 2024-04-25T10:08:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.227+0000 7f111fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11200ce6f0 con 0x7f11200d1590 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.227+0000 7f111f400640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f112015e4c0 0x7f11200b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:45442/0 (socket says 172.21.15.5:45442) 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.227+0000 7f111f400640 1 -- 172.21.15.5:0/3115693644 learned_addr learned my addr 172.21.15.5:0/3115693644 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.227+0000 7f111e000640 1 --2- 172.21.15.5:0/3115693644 >> v2:172.21.15.80:3300/0 conn(0x7f11200d0570 0x7f11200d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.227+0000 7f111ea00640 1 --2- 172.21.15.5:0/3115693644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d1590 0x7f11200d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.228+0000 7f111d600640 1 -- 172.21.15.5:0/3115693644 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370696816 0 0) 0x7f11200d23f0 con 0x7f112015e4c0 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.228+0000 7f111d600640 1 -- 172.21.15.5:0/3115693644 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1104003520 con 0x7f112015e4c0 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.228+0000 7f111ea00640 1 -- 172.21.15.5:0/3115693644 >> v2:172.21.15.80:3300/0 conn(0x7f11200d0570 msgr2=0x7f11200d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.228+0000 7f111ea00640 1 --2- 172.21.15.5:0/3115693644 >> v2:172.21.15.80:3300/0 conn(0x7f11200d0570 0x7f11200d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.228+0000 7f111ea00640 1 -- 172.21.15.5:0/3115693644 >> v1:172.21.15.80:6789/0 conn(0x7f112015e4c0 legacy=0x7f11200b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.228+0000 7f111ea00640 1 -- 172.21.15.5:0/3115693644 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112015ea10 con 0x7f11200d1590 2024-04-25T10:08:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.229+0000 7f111ea00640 1 --2- 172.21.15.5:0/3115693644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d1590 0x7f11200d1970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11100097a0 tx=0x7f111002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddae55caca4f218f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.230+0000 7f111d600640 1 -- 172.21.15.5:0/3115693644 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110037070 con 0x7f11200d1590 2024-04-25T10:08:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.230+0000 7f111d600640 1 -- 172.21.15.5:0/3115693644 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110032440 con 0x7f11200d1590 2024-04-25T10:08:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.230+0000 7f111d600640 1 -- 172.21.15.5:0/3115693644 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100327f0 con 0x7f11200d1590 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 -- 172.21.15.5:0/3115693644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d1590 msgr2=0x7f11200d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 --2- 172.21.15.5:0/3115693644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d1590 0x7f11200d1970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11100097a0 tx=0x7f111002a180 comp rx=0 tx=0).stop 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 -- 172.21.15.5:0/3115693644 shutdown_connections 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 --2- 172.21.15.5:0/3115693644 >> v2:172.21.15.80:3300/0 conn(0x7f11200d0570 0x7f11200d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 --2- 172.21.15.5:0/3115693644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d1590 0x7f11200d1970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 -- 172.21.15.5:0/3115693644 >> 172.21.15.5:0/3115693644 conn(0x7f11200726e0 msgr2=0x7f1120072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 -- 172.21.15.5:0/3115693644 shutdown_connections 2024-04-25T10:08:55.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.231+0000 7f111fe00640 1 -- 172.21.15.5:0/3115693644 wait complete. 2024-04-25T10:08:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.233+0000 7f111fe00640 1 Processor -- start 2024-04-25T10:08:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.233+0000 7f111fe00640 1 -- start start 2024-04-25T10:08:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d0570 0x7f11200c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11200d1590 0x7f112014e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 0x7f11200cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11200d2950 con 0x7f112015e4c0 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11200d27d0 con 0x7f11200d1590 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11200d3040 con 0x7f11200d0570 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 0x7f11200cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 0x7f11200cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59854/0 (socket says 172.21.15.5:59854) 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.234+0000 7f111f400640 1 -- 172.21.15.5:0/3106120134 learned_addr learned my addr 172.21.15.5:0/3106120134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111ea00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d0570 0x7f11200c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111e000640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11200d1590 0x7f112014e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111f400640 1 -- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d0570 msgr2=0x7f11200c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111f400640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d0570 0x7f11200c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111f400640 1 -- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11200d1590 msgr2=0x7f112014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111f400640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11200d1590 0x7f112014e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111f400640 1 -- 172.21.15.5:0/3106120134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11201f3680 con 0x7f112015e4c0 2024-04-25T10:08:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f111f400640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 0x7f11200cc900 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f111800caa0 tx=0x7f111800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.235+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118017070 con 0x7f112015e4c0 2024-04-25T10:08:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.236+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111800ac00 con 0x7f112015e4c0 2024-04-25T10:08:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.236+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11201f3960 con 0x7f112015e4c0 2024-04-25T10:08:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.236+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118012400 con 0x7f112015e4c0 2024-04-25T10:08:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.236+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11201f3f20 con 0x7f112015e4c0 2024-04-25T10:08:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.237+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1120068080 con 0x7f112015e4c0 2024-04-25T10:08:55.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.238+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f111801a070 con 0x7f112015e4c0 2024-04-25T10:08:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.239+0000 7f1117400640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110006b4d0 0x7f110006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.239+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1118089e30 con 0x7f112015e4c0 2024-04-25T10:08:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.239+0000 7f111ea00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110006b4d0 0x7f110006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.241+0000 7f111ea00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110006b4d0 0x7f110006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1110034000 tx=0x7f111002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.243+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1118058e60 con 0x7f112015e4c0 2024-04-25T10:08:55.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.512+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f11201f4260 con 0x7f112015e4c0 2024-04-25T10:08:55.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.513+0000 7f1117400640 1 -- 172.21.15.5:0/3106120134 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f11201f4260 con 0x7f112015e4c0 2024-04-25T10:08:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:55.518 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:08:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:08:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:08:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:08:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:08:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:08:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:08:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110006b4d0 msgr2=0x7f110006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110006b4d0 0x7f110006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1110034000 tx=0x7f111002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 msgr2=0x7f11200cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 0x7f11200cc900 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f111800caa0 tx=0x7f111800cf60 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 shutdown_connections 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f112015e4c0 0x7f11200cc900 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f110006b4d0 0x7f110006d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11200d1590 0x7f112014e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.515+0000 7f111fe00640 1 --2- 172.21.15.5:0/3106120134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11200d0570 0x7f11200c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.516+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 >> 172.21.15.5:0/3106120134 conn(0x7f11200726e0 msgr2=0x7f11200c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.516+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 shutdown_connections 2024-04-25T10:08:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.516+0000 7f111fe00640 1 -- 172.21.15.5:0/3106120134 wait complete. 2024-04-25T10:08:55.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.738+0000 7fef85c00640 1 Processor -- start 2024-04-25T10:08:55.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.738+0000 7fef85c00640 1 -- start start 2024-04-25T10:08:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.739+0000 7fef85c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef800d0a50 0x7fef800d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.739+0000 7fef85c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800b53b0 0x7fef800d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.739+0000 7fef85c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef800d1e80 con 0x7fef8015db00 2024-04-25T10:08:55.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.739+0000 7fef85c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800b5930 con 0x7fef800d0a50 2024-04-25T10:08:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.739+0000 7fef85c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800b5c10 con 0x7fef800b53b0 2024-04-25T10:08:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef800d0a50 0x7fef800d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef800d0a50 0x7fef800d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59868/0 (socket says 172.21.15.5:59868) 2024-04-25T10:08:55.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 -- 172.21.15.5:0/2881606764 learned_addr learned my addr 172.21.15.5:0/2881606764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:55.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 -- 172.21.15.5:0/2881606764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800b53b0 msgr2=0x7fef800d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef7f400640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800b53b0 0x7fef800d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800b53b0 0x7fef800d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 -- 172.21.15.5:0/2881606764 >> v1:172.21.15.80:6789/0 conn(0x7fef8015db00 legacy=0x7fef800d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:55.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef77e00640 1 -- 172.21.15.5:0/2881606764 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800b5ef0 con 0x7fef800d0a50 2024-04-25T10:08:55.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.740+0000 7fef7f400640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800b53b0 0x7fef800d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:55.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.742+0000 7fef77e00640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800d0a50 0x7fef800d2330 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fef68009870 tx=0x7fef6802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8307b2a750a76b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.742+0000 7fef7e000640 1 -- 172.21.15.5:0/2881606764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68037070 con 0x7fef800d0a50 2024-04-25T10:08:55.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.742+0000 7fef7e000640 1 -- 172.21.15.5:0/2881606764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef68032440 con 0x7fef800d0a50 2024-04-25T10:08:55.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.742+0000 7fef7e000640 1 -- 172.21.15.5:0/2881606764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68032810 con 0x7fef800d0a50 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 -- 172.21.15.5:0/2881606764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800d0a50 msgr2=0x7fef800d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800d0a50 0x7fef800d2330 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fef68009870 tx=0x7fef6802a210 comp rx=0 tx=0).stop 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 -- 172.21.15.5:0/2881606764 shutdown_connections 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800b53b0 0x7fef800d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 --2- 172.21.15.5:0/2881606764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800d0a50 0x7fef800d2330 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 -- 172.21.15.5:0/2881606764 >> 172.21.15.5:0/2881606764 conn(0x7fef80072290 msgr2=0x7fef800726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 -- 172.21.15.5:0/2881606764 shutdown_connections 2024-04-25T10:08:55.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.743+0000 7fef85c00640 1 -- 172.21.15.5:0/2881606764 wait complete. 2024-04-25T10:08:55.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.745+0000 7fef85c00640 1 Processor -- start 2024-04-25T10:08:55.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.745+0000 7fef85c00640 1 -- start start 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef85c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 0x7fef800ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef85c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800d0a50 0x7fef80153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef85c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef8015db00 0x7fef800c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef85c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800d3660 con 0x7fef800b53b0 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef85c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800b61d0 con 0x7fef8015db00 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef85c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800ce2b0 con 0x7fef800d0a50 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 0x7fef800ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 0x7fef800ccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59878/0 (socket says 172.21.15.5:59878) 2024-04-25T10:08:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 -- 172.21.15.5:0/2329571179 learned_addr learned my addr 172.21.15.5:0/2329571179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 -- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800d0a50 msgr2=0x7fef80153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800d0a50 0x7fef80153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 -- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef8015db00 msgr2=0x7fef800c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.746+0000 7fef7ea00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef8015db00 0x7fef800c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef7ea00640 1 -- 172.21.15.5:0/2329571179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800cd390 con 0x7fef800b53b0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef7ea00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 0x7fef800ccb40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fef7000b420 tx=0x7fef7000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70011830 con 0x7fef800b53b0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70016070 con 0x7fef800b53b0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70011c10 con 0x7fef800b53b0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef800cd670 con 0x7fef800b53b0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.747+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8015e290 con 0x7fef800b53b0 2024-04-25T10:08:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.749+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef80073ce0 con 0x7fef800b53b0 2024-04-25T10:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.749+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef7001b460 con 0x7fef800b53b0 2024-04-25T10:08:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.750+0000 7fef76a00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fef5c06b4d0 0x7fef5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.751+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fef70089b70 con 0x7fef800b53b0 2024-04-25T10:08:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.751+0000 7fef77e00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fef5c06b4d0 0x7fef5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.753+0000 7fef77e00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fef5c06b4d0 0x7fef5c06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fef68034000 tx=0x7fef6802c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.755+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef80073ce0 con 0x7fef800b53b0 2024-04-25T10:08:55.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.955+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef80068fd0 con 0x7fef800b53b0 2024-04-25T10:08:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.957+0000 7fef76a00640 1 -- 172.21.15.5:0/2329571179 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+13057 (secure 0 0 0) 0x7fef70020020 con 0x7fef800b53b0 2024-04-25T10:08:55.963 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:55.963 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:08:55.018709+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fef5c06b4d0 msgr2=0x7fef5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fef5c06b4d0 0x7fef5c06d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fef68034000 tx=0x7fef6802c000 comp rx=0 tx=0).stop 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 msgr2=0x7fef800ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 0x7fef800ccb40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fef7000b420 tx=0x7fef7000b8e0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 shutdown_connections 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fef5c06b4d0 0x7fef5c06d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef8015db00 0x7fef800c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef800d0a50 0x7fef80153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 --2- 172.21.15.5:0/2329571179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef800b53b0 0x7fef800ccb40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.959+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 >> 172.21.15.5:0/2329571179 conn(0x7fef80072290 msgr2=0x7fef800d37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.960+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 shutdown_connections 2024-04-25T10:08:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:55.960+0000 7fef85c00640 1 -- 172.21.15.5:0/2329571179 wait complete. 2024-04-25T10:08:55.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:55.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.185+0000 7efd59e00640 1 Processor -- start 2024-04-25T10:08:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.186+0000 7efd59e00640 1 -- start start 2024-04-25T10:08:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.187+0000 7efd59e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd54161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.187+0000 7efd59e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd541606e0 con 0x7efd54161bc0 2024-04-25T10:08:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.187+0000 7efd59e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd541609c0 con 0x7efd540b48c0 2024-04-25T10:08:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.187+0000 7efd59e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd54160ca0 con 0x7efd540d3f90 2024-04-25T10:08:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.188+0000 7efd59400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efd54161bc0 0x7efd540b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60508/0 (socket says 172.21.15.5:60508) 2024-04-25T10:08:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.188+0000 7efd59400640 1 -- 172.21.15.5:0/2320779695 learned_addr learned my addr 172.21.15.5:0/2320779695 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.188+0000 7efd4be00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd54161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.188+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289924177 0 0) 0x7efd541606e0 con 0x7efd54161bc0 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.188+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd38003520 con 0x7efd54161bc0 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880565154 0 0) 0x7efd38003520 con 0x7efd54161bc0 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 msgr2=0x7efd54161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd54161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 >> v1:172.21.15.80:6789/0 conn(0x7efd540b48c0 legacy=0x7efd5415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54160f80 con 0x7efd54161bc0 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7efd4c0019f0 con 0x7efd54161bc0 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 >> v1:172.21.15.5:6789/0 conn(0x7efd54161bc0 legacy=0x7efd540b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38003f60 0x7efd38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd38004d40 0x7efd380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd54160ca0 con 0x7efd38004d40 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3800bb50 con 0x7efd38003f60 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3800bd50 con 0x7efd540d3f90 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd59400640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd38004d40 0x7efd380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.189+0000 7efd4be00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38003f60 0x7efd38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd58a00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd59400640 1 -- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 msgr2=0x7efd380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd59400640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd59400640 1 -- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38003f60 msgr2=0x7efd38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd59400640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38003f60 0x7efd38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd59400640 1 -- 172.21.15.5:0/2320779695 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38003520 con 0x7efd38004d40 2024-04-25T10:08:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.190+0000 7efd58a00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540d3f90 0x7efd380039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.191+0000 7efd59400640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd38004d40 0x7efd380051c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00a860 tx=0x7efd38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86c39c030d8fd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.191+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00a480 con 0x7efd38004d40 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.191+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c02f070 con 0x7efd38004d40 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.191+0000 7efd53e00640 1 -- 172.21.15.5:0/2320779695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02bce0 con 0x7efd38004d40 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59e00640 1 -- 172.21.15.5:0/2320779695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd38004d40 msgr2=0x7efd380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59e00640 1 --2- 172.21.15.5:0/2320779695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd38004d40 0x7efd380051c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00a860 tx=0x7efd38005830 comp rx=0 tx=0).stop 2024-04-25T10:08:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59400640 1 -- 172.21.15.5:0/2320779695 reap_dead start 2024-04-25T10:08:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59e00640 1 -- 172.21.15.5:0/2320779695 shutdown_connections 2024-04-25T10:08:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59e00640 1 -- 172.21.15.5:0/2320779695 >> 172.21.15.5:0/2320779695 conn(0x7efd5408af80 msgr2=0x7efd540d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59e00640 1 -- 172.21.15.5:0/2320779695 shutdown_connections 2024-04-25T10:08:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.192+0000 7efd59e00640 1 -- 172.21.15.5:0/2320779695 wait complete. 2024-04-25T10:08:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.194+0000 7efd59e00640 1 Processor -- start 2024-04-25T10:08:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.194+0000 7efd59e00640 1 -- start start 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38004d40 0x7efd5415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540b8ba0 0x7efd540c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 0x7efd540cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd54161680 con 0x7efd540cf350 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd540b4ca0 con 0x7efd38004d40 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd54161800 con 0x7efd540b8ba0 2024-04-25T10:08:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 0x7efd540cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 0x7efd540cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59902/0 (socket says 172.21.15.5:59902) 2024-04-25T10:08:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd59400640 1 -- 172.21.15.5:0/1873695374 learned_addr learned my addr 172.21.15.5:0/1873695374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd58a00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38004d40 0x7efd5415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.195+0000 7efd4be00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540b8ba0 0x7efd540c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.196+0000 7efd59400640 1 -- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540b8ba0 msgr2=0x7efd540c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.196+0000 7efd59400640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540b8ba0 0x7efd540c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.196+0000 7efd59400640 1 -- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38004d40 msgr2=0x7efd5415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.196+0000 7efd59400640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38004d40 0x7efd5415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.196+0000 7efd59400640 1 -- 172.21.15.5:0/1873695374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd541f3910 con 0x7efd540cf350 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.196+0000 7efd59400640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 0x7efd540cf800 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00a980 tx=0x7efd541608b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.197+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00a480 con 0x7efd540cf350 2024-04-25T10:08:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.197+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c02bc90 con 0x7efd540cf350 2024-04-25T10:08:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.197+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c034670 con 0x7efd540cf350 2024-04-25T10:08:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.197+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd541f3bf0 con 0x7efd540cf350 2024-04-25T10:08:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.197+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd541f41b0 con 0x7efd540cf350 2024-04-25T10:08:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.198+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd541f43b0 con 0x7efd540cf350 2024-04-25T10:08:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.199+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd4c034810 con 0x7efd540cf350 2024-04-25T10:08:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.200+0000 7efd51600640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd3006b520 0x7efd3006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.200+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efd4c0aa2a0 con 0x7efd540cf350 2024-04-25T10:08:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.201+0000 7efd58a00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd3006b520 0x7efd3006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.202+0000 7efd58a00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd3006b520 0x7efd3006d9e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd54073b50 tx=0x7efd44001190 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.204+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd4c078d90 con 0x7efd540cf350 2024-04-25T10:08:56.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.408+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd54068f60 con 0x7efd540cf350 2024-04-25T10:08:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.409+0000 7efd51600640 1 -- 172.21.15.5:0/1873695374 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd4c07ce30 con 0x7efd540cf350 2024-04-25T10:08:56.415 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.412+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd3006b520 msgr2=0x7efd3006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.412+0000 7efd59e00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd3006b520 0x7efd3006d9e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd54073b50 tx=0x7efd44001190 comp rx=0 tx=0).stop 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.412+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 msgr2=0x7efd540cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.412+0000 7efd59e00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 0x7efd540cf800 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7efd4c00a980 tx=0x7efd541608b0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 shutdown_connections 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd540cf350 0x7efd540cf800 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efd540b8ba0 0x7efd540c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efd3006b520 0x7efd3006d9e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 --2- 172.21.15.5:0/1873695374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd38004d40 0x7efd5415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 >> 172.21.15.5:0/1873695374 conn(0x7efd5408af80 msgr2=0x7efd5415de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 shutdown_connections 2024-04-25T10:08:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.413+0000 7efd59e00640 1 -- 172.21.15.5:0/1873695374 wait complete. 2024-04-25T10:08:56.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:08:56.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.645+0000 7f9c8f000640 1 Processor -- start 2024-04-25T10:08:56.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.645+0000 7f9c8f000640 1 -- start start 2024-04-25T10:08:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.646+0000 7f9c8f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d0570 0x7f9c880d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.646+0000 7f9c8f000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c8815e4c0 0x7f9c881613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.646+0000 7f9c8f000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c880ce6f0 con 0x7f9c880d1590 2024-04-25T10:08:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.646+0000 7f9c8f000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880cea20 con 0x7f9c8815e4c0 2024-04-25T10:08:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.646+0000 7f9c8f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8815ea10 con 0x7f9c880d0570 2024-04-25T10:08:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8e600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c8815e4c0 0x7f9c881613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8dc00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9c880d1590 0x7f9c880d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60510/0 (socket says 172.21.15.5:60510) 2024-04-25T10:08:56.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8dc00640 1 -- 172.21.15.5:0/2382060393 learned_addr learned my addr 172.21.15.5:0/2382060393 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:56.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8d200640 1 --2- 172.21.15.5:0/2382060393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d0570 0x7f9c880d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8e600640 1 -- 172.21.15.5:0/2382060393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d0570 msgr2=0x7f9c880d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:56.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8e600640 1 --2- 172.21.15.5:0/2382060393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d0570 0x7f9c880d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8e600640 1 -- 172.21.15.5:0/2382060393 >> v1:172.21.15.5:6789/0 conn(0x7f9c880d1590 legacy=0x7f9c880d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.647+0000 7f9c8e600640 1 -- 172.21.15.5:0/2382060393 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8815ecf0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.649+0000 7f9c8e600640 1 --2- 172.21.15.5:0/2382060393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c881613c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009d90 tx=0x7f9c8002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd284e5eedd71f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.649+0000 7f9c87e00640 1 -- 172.21.15.5:0/2382060393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80007740 con 0x7f9c8815e4c0 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.649+0000 7f9c87e00640 1 -- 172.21.15.5:0/2382060393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c800078e0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.649+0000 7f9c87e00640 1 -- 172.21.15.5:0/2382060393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80007cb0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 -- 172.21.15.5:0/2382060393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 msgr2=0x7f9c881613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 --2- 172.21.15.5:0/2382060393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c881613c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009d90 tx=0x7f9c8002a660 comp rx=0 tx=0).stop 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 -- 172.21.15.5:0/2382060393 shutdown_connections 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 --2- 172.21.15.5:0/2382060393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c881613c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 --2- 172.21.15.5:0/2382060393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d0570 0x7f9c880d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 -- 172.21.15.5:0/2382060393 >> 172.21.15.5:0/2382060393 conn(0x7f9c880726e0 msgr2=0x7f9c88072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 -- 172.21.15.5:0/2382060393 shutdown_connections 2024-04-25T10:08:56.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.650+0000 7f9c8f000640 1 -- 172.21.15.5:0/2382060393 wait complete. 2024-04-25T10:08:56.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.652+0000 7f9c8f000640 1 Processor -- start 2024-04-25T10:08:56.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.652+0000 7f9c8f000640 1 -- start start 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8f000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c880d0570 0x7f9c880c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8f000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d1590 0x7f9c880c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c880cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880d31c0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8f000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880d3040 con 0x7f9c880d0570 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8f000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88161900 con 0x7f9c880d1590 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c880cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c880cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59930/0 (socket says 172.21.15.5:59930) 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.653+0000 7f9c8e600640 1 -- 172.21.15.5:0/3165340448 learned_addr learned my addr 172.21.15.5:0/3165340448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8dc00640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c880d0570 0x7f9c880c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8d200640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d1590 0x7f9c880c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8e600640 1 -- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d1590 msgr2=0x7f9c880c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8e600640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d1590 0x7f9c880c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8e600640 1 -- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c880d0570 msgr2=0x7f9c880c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8e600640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c880d0570 0x7f9c880c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8e600640 1 -- 172.21.15.5:0/3165340448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880c7a40 con 0x7f9c8815e4c0 2024-04-25T10:08:56.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8d200640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d1590 0x7f9c880c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:56.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.654+0000 7f9c8e600640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c880cc6d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009eb0 tx=0x7f9c80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.655+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8003f070 con 0x7f9c8815e4c0 2024-04-25T10:08:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.655+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c80031bc0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.655+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80044460 con 0x7f9c8815e4c0 2024-04-25T10:08:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.655+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880c7d20 con 0x7f9c8815e4c0 2024-04-25T10:08:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.655+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c880c82e0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.656+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88068080 con 0x7f9c8815e4c0 2024-04-25T10:08:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.657+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c80007790 con 0x7f9c8815e4c0 2024-04-25T10:08:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.658+0000 7f9c85600640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6806b4d0 0x7f9c6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:56.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.658+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9c800ab8c0 con 0x7f9c8815e4c0 2024-04-25T10:08:56.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.659+0000 7f9c8dc00640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6806b4d0 0x7f9c6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:56.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.660+0000 7f9c8dc00640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6806b4d0 0x7f9c6806d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000f60 tx=0x7f9c7c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:56.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.663+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c8007a900 con 0x7f9c8815e4c0 2024-04-25T10:08:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.868+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9c88068f60 con 0x7f9c8815e4c0 2024-04-25T10:08:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.868+0000 7f9c85600640 1 -- 172.21.15.5:0/3165340448 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9c8007a200 con 0x7f9c8815e4c0 2024-04-25T10:08:56.874 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:08:56.874 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:08:56.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:08:56.874 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:08:56.874 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:08:56.875 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:08:56.875 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6806b4d0 msgr2=0x7f9c6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6806b4d0 0x7f9c6806d990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000f60 tx=0x7f9c7c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 msgr2=0x7f9c880cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c880cc6d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009eb0 tx=0x7f9c80004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 shutdown_connections 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c8815e4c0 0x7f9c880cc6d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c880d1590 0x7f9c880c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c6806b4d0 0x7f9c6806d990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 --2- 172.21.15.5:0/3165340448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c880d0570 0x7f9c880c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.871+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 >> 172.21.15.5:0/3165340448 conn(0x7f9c880726e0 msgr2=0x7f9c880b80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.872+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 shutdown_connections 2024-04-25T10:08:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:56.872+0000 7f9c8f000640 1 -- 172.21.15.5:0/3165340448 wait complete. 2024-04-25T10:08:56.897 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:08:56.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:08:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.093+0000 7f8740a00640 1 Processor -- start 2024-04-25T10:08:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.093+0000 7f8740a00640 1 -- start start 2024-04-25T10:08:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.095+0000 7f8740a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f873c144e20 0x7f873c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.095+0000 7f8740a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.095+0000 7f8740a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c06e850 con 0x7f873c071000 2024-04-25T10:08:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.095+0000 7f8740a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c15fdf0 con 0x7f873c144e20 2024-04-25T10:08:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.095+0000 7f8740a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1600d0 con 0x7f873c15f270 2024-04-25T10:08:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.095+0000 7f873aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873b400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f873c144e20 0x7f873c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c15f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49670/0 (socket says 172.21.15.5:49670) 2024-04-25T10:08:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873aa00640 1 -- 172.21.15.5:0/2872704186 learned_addr learned my addr 172.21.15.5:0/2872704186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873aa00640 1 -- 172.21.15.5:0/2872704186 >> v2:172.21.15.80:3300/0 conn(0x7f873c144e20 msgr2=0x7f873c15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873aa00640 1 --2- 172.21.15.5:0/2872704186 >> v2:172.21.15.80:3300/0 conn(0x7f873c144e20 0x7f873c15c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873aa00640 1 -- 172.21.15.5:0/2872704186 >> v1:172.21.15.80:6789/0 conn(0x7f873c071000 legacy=0x7f873c0cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.096+0000 7f873aa00640 1 -- 172.21.15.5:0/2872704186 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c15bd90 con 0x7f873c15f270 2024-04-25T10:08:57.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.098+0000 7f873aa00640 1 --2- 172.21.15.5:0/2872704186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c15f720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8728009870 tx=0x7f872802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80059d8a613f76c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.098+0000 7f873a000640 1 -- 172.21.15.5:0/2872704186 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728037070 con 0x7f873c15f270 2024-04-25T10:08:57.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.098+0000 7f873a000640 1 -- 172.21.15.5:0/2872704186 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728032440 con 0x7f873c15f270 2024-04-25T10:08:57.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.098+0000 7f873a000640 1 -- 172.21.15.5:0/2872704186 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728032810 con 0x7f873c15f270 2024-04-25T10:08:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 -- 172.21.15.5:0/2872704186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 msgr2=0x7f873c15f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 --2- 172.21.15.5:0/2872704186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c15f720 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8728009870 tx=0x7f872802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 -- 172.21.15.5:0/2872704186 shutdown_connections 2024-04-25T10:08:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 --2- 172.21.15.5:0/2872704186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c15f720 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 --2- 172.21.15.5:0/2872704186 >> v2:172.21.15.80:3300/0 conn(0x7f873c144e20 0x7f873c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 -- 172.21.15.5:0/2872704186 >> 172.21.15.5:0/2872704186 conn(0x7f873c066f20 msgr2=0x7f873c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 -- 172.21.15.5:0/2872704186 shutdown_connections 2024-04-25T10:08:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.099+0000 7f8740a00640 1 -- 172.21.15.5:0/2872704186 wait complete. 2024-04-25T10:08:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.101+0000 7f8740a00640 1 Processor -- start 2024-04-25T10:08:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.101+0000 7f8740a00640 1 -- start start 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.102+0000 7f8740a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f873c071000 0x7f873c0ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.102+0000 7f8740a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 0x7f873c15b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.102+0000 7f8740a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c0addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.102+0000 7f8740a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c145380 con 0x7f873c144e20 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.102+0000 7f8740a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c145200 con 0x7f873c071000 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.102+0000 7f8740a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c071430 con 0x7f873c15f270 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 0x7f873c15b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 0x7f873c15b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35962/0 (socket says 172.21.15.5:35962) 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 -- 172.21.15.5:0/1550251516 learned_addr learned my addr 172.21.15.5:0/1550251516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873b400640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f873c071000 0x7f873c0ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873be00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c0addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 -- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 msgr2=0x7f873c0addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c0addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 -- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f873c071000 msgr2=0x7f873c0ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f873c071000 0x7f873c0ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 -- 172.21.15.5:0/1550251516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0acd10 con 0x7f873c144e20 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873be00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.103+0000 7f873aa00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 0x7f873c15b520 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f87280090b0 tx=0x7f87280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.104+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728041030 con 0x7f873c144e20 2024-04-25T10:08:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.104+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728030440 con 0x7f873c144e20 2024-04-25T10:08:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.104+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728037040 con 0x7f873c144e20 2024-04-25T10:08:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.104+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c0acff0 con 0x7f873c144e20 2024-04-25T10:08:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.104+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c0ad5b0 con 0x7f873c144e20 2024-04-25T10:08:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.105+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c06b7b0 con 0x7f873c144e20 2024-04-25T10:08:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.106+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8728044070 con 0x7f873c144e20 2024-04-25T10:08:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.108+0000 7f8733400640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f871c06b4d0 0x7f871c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.108+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f87280aa770 con 0x7f873c144e20 2024-04-25T10:08:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.108+0000 7f873b400640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f871c06b4d0 0x7f871c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.110+0000 7f873b400640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f871c06b4d0 0x7f871c06d990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f872c009ad0 tx=0x7f872c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.112+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f873c06b7b0 con 0x7f873c144e20 2024-04-25T10:08:57.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:57.311+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f873c068f60 con 0x7f873c144e20 2024-04-25T10:08:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.040+0000 7f8733400640 1 -- 172.21.15.5:0/1550251516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8728079830 con 0x7f873c144e20 2024-04-25T10:08:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:08:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f871c06b4d0 msgr2=0x7f871c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f871c06b4d0 0x7f871c06d990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f872c009ad0 tx=0x7f872c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 msgr2=0x7f873c15b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 0x7f873c15b520 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f87280090b0 tx=0x7f87280046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 shutdown_connections 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f873c15f270 0x7f873c0addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f873c144e20 0x7f873c15b520 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f871c06b4d0 0x7f871c06d990 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 --2- 172.21.15.5:0/1550251516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f873c071000 0x7f873c0ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.043+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 >> 172.21.15.5:0/1550251516 conn(0x7f873c066f20 msgr2=0x7f873c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.044+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 shutdown_connections 2024-04-25T10:08:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.044+0000 7f8740a00640 1 -- 172.21.15.5:0/1550251516 wait complete. 2024-04-25T10:08:58.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:08:58.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.271+0000 7fcdf8a00640 1 Processor -- start 2024-04-25T10:08:58.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.271+0000 7fcdf8a00640 1 -- start start 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf8a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcdf40cdaa0 0x7fcdf415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf8a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 0x7fcdf415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf8a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf4144cd0 con 0x7fcdf41448f0 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf8a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40cdf30 con 0x7fcdf40cdaa0 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf8a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415f460 con 0x7fcdf415fe30 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf2800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcdf41448f0 0x7fcdf415bc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33288/0 (socket says 172.21.15.5:33288) 2024-04-25T10:08:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.273+0000 7fcdf2800640 1 -- 172.21.15.5:0/1041108726 learned_addr learned my addr 172.21.15.5:0/1041108726 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/1041108726 >> v2:172.21.15.80:3300/0 conn(0x7fcdf40cdaa0 0x7fcdf415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf3200640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 0x7fcdf415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675789108 0 0) 0x7fcdf4144cd0 con 0x7fcdf41448f0 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd8003520 con 0x7fcdf41448f0 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2176083350 0 0) 0x7fcdd8003520 con 0x7fcdf41448f0 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 msgr2=0x7fcdf415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 0x7fcdf415ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 >> v2:172.21.15.80:3300/0 conn(0x7fcdf40cdaa0 msgr2=0x7fcdf415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 --2- 172.21.15.5:0/1041108726 >> v2:172.21.15.80:3300/0 conn(0x7fcdf40cdaa0 0x7fcdf415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf415c440 con 0x7fcdf41448f0 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fcde40019f0 con 0x7fcdf41448f0 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 >> v1:172.21.15.5:6789/0 conn(0x7fcdf41448f0 legacy=0x7fcdf415bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.274+0000 7fcdf1400640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd80039b0 0x7fcdd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1400640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdd8004800 0x7fcdd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1400640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 0x7fcdd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40cdf30 con 0x7fcdd8004800 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415f460 con 0x7fcdd80039b0 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd800b9c0 con 0x7fcdf415fe30 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdd8004800 0x7fcdd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf2800640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd80039b0 0x7fcdd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1e00640 1 -- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 msgr2=0x7fcdd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf415fe30 0x7fcdd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1e00640 1 -- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd80039b0 msgr2=0x7fcdd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd80039b0 0x7fcdd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.275+0000 7fcdf1e00640 1 -- 172.21.15.5:0/1041108726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8003520 con 0x7fcdd8004800 2024-04-25T10:08:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.276+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdd8004800 0x7fcdd8004c80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ac0 tx=0x7fcdd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4fda832a260ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.277+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0036070 con 0x7fcdd8004800 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.277+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0031400 con 0x7fcdd8004800 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.277+0000 7fcdf1400640 1 -- 172.21.15.5:0/1041108726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0009690 con 0x7fcdd8004800 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.277+0000 7fcdf8a00640 1 -- 172.21.15.5:0/1041108726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdd8004800 msgr2=0x7fcdd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.277+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/1041108726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdd8004800 0x7fcdd8004c80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ac0 tx=0x7fcdd8005830 comp rx=0 tx=0).stop 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.277+0000 7fcdf3200640 1 -- 172.21.15.5:0/1041108726 reap_dead start 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.278+0000 7fcdf8a00640 1 -- 172.21.15.5:0/1041108726 shutdown_connections 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.278+0000 7fcdf8a00640 1 -- 172.21.15.5:0/1041108726 >> 172.21.15.5:0/1041108726 conn(0x7fcdf4066f20 msgr2=0x7fcdf406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.278+0000 7fcdf8a00640 1 -- 172.21.15.5:0/1041108726 shutdown_connections 2024-04-25T10:08:58.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.278+0000 7fcdf8a00640 1 -- 172.21.15.5:0/1041108726 wait complete. 2024-04-25T10:08:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.280+0000 7fcdf8a00640 1 Processor -- start 2024-04-25T10:08:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.280+0000 7fcdf8a00640 1 -- start start 2024-04-25T10:08:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd8004800 0x7fcdf40b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 0x7fcdf413bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf8a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf40aeb40 0x7fcdf40b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf8a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415ce20 con 0x7fcdf413fe70 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf8a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415ca00 con 0x7fcdd8004800 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf8a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415cfa0 con 0x7fcdf40aeb40 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 0x7fcdf413bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 0x7fcdf413bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35982/0 (socket says 172.21.15.5:35982) 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf1e00640 1 -- 172.21.15.5:0/3850947875 learned_addr learned my addr 172.21.15.5:0/3850947875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.281+0000 7fcdf2800640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd8004800 0x7fcdf40b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf3200640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf40aeb40 0x7fcdf40b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf1e00640 1 -- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf40aeb40 msgr2=0x7fcdf40b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf40aeb40 0x7fcdf40b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf1e00640 1 -- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd8004800 msgr2=0x7fcdf40b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd8004800 0x7fcdf40b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf1e00640 1 -- 172.21.15.5:0/3850947875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40af4b0 con 0x7fcdf413fe70 2024-04-25T10:08:58.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdf1e00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 0x7fcdf413bd20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ac0 tx=0x7fcde0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.282+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0041020 con 0x7fcdf413fe70 2024-04-25T10:08:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.283+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde003c470 con 0x7fcdf413fe70 2024-04-25T10:08:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.283+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0036040 con 0x7fcdf413fe70 2024-04-25T10:08:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.283+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40af790 con 0x7fcdf413fe70 2024-04-25T10:08:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.283+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf40afd50 con 0x7fcdf413fe70 2024-04-25T10:08:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.284+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4068f60 con 0x7fcdf413fe70 2024-04-25T10:08:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.285+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcde0044020 con 0x7fcdf413fe70 2024-04-25T10:08:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.287+0000 7fcdeaa00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.287+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fcde00aaa90 con 0x7fcdf413fe70 2024-04-25T10:08:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.288+0000 7fcdf2800640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.289+0000 7fcdf2800640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdd406b4d0 0x7fcdd406d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcde4005db0 tx=0x7fcde4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.292+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcde004b030 con 0x7fcdf413fe70 2024-04-25T10:08:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:58.493+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcdf4072180 con 0x7fcdf413fe70 2024-04-25T10:08:59.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.056+0000 7fcdeaa00640 1 -- 172.21.15.5:0/3850947875 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcde00798c0 con 0x7fcdf413fe70 2024-04-25T10:08:59.062 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.059+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdd406b4d0 msgr2=0x7fcdd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.059+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdd406b4d0 0x7fcdd406d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcde4005db0 tx=0x7fcde4009040 comp rx=0 tx=0).stop 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.059+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 msgr2=0x7fcdf413bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.059+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 0x7fcdf413bd20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ac0 tx=0x7fcde0004730 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 shutdown_connections 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcdf40aeb40 0x7fcdf40b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf413fe70 0x7fcdf413bd20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcdd406b4d0 0x7fcdd406d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 --2- 172.21.15.5:0/3850947875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcdd8004800 0x7fcdf40b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 >> 172.21.15.5:0/3850947875 conn(0x7fcdf4066f20 msgr2=0x7fcdf40adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 shutdown_connections 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.060+0000 7fcdf8a00640 1 -- 172.21.15.5:0/3850947875 wait complete. 2024-04-25T10:08:59.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:08:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.278+0000 7f170b200640 1 Processor -- start 2024-04-25T10:08:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.279+0000 7f170b200640 1 -- start start 2024-04-25T10:08:59.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.280+0000 7f170b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f170c161d90 0x7f170c1689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.280+0000 7f170b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c1606e0 con 0x7f170c0b48c0 2024-04-25T10:08:59.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.280+0000 7f170b200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c1609c0 con 0x7f170c0d3f90 2024-04-25T10:08:59.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.280+0000 7f170b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c160ca0 con 0x7f170c161d90 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f170a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f170c161d90 0x7f170c1689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f1709e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f170c0b48c0 0x7f170c15fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33290/0 (socket says 172.21.15.5:33290) 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f1709e00640 1 -- 172.21.15.5:0/1209392876 learned_addr learned my addr 172.21.15.5:0/1209392876 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f170a800640 1 -- 172.21.15.5:0/1209392876 >> v1:172.21.15.80:6789/0 conn(0x7f170c0d3f90 legacy=0x7f170c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f170a800640 1 -- 172.21.15.5:0/1209392876 >> v1:172.21.15.5:6789/0 conn(0x7f170c0b48c0 legacy=0x7f170c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f170a800640 1 -- 172.21.15.5:0/1209392876 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c160f80 con 0x7f170c161d90 2024-04-25T10:08:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.281+0000 7f1708a00640 1 -- 172.21.15.5:0/1209392876 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695897319 0 0) 0x7f170c1606e0 con 0x7f170c0b48c0 2024-04-25T10:08:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.282+0000 7f170a800640 1 --2- 172.21.15.5:0/1209392876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c1689b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f17040097a0 tx=0x7f170402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89dc22d17faf929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f1708a00640 1 -- 172.21.15.5:0/1209392876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704037070 con 0x7f170c161d90 2024-04-25T10:08:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f1708a00640 1 -- 172.21.15.5:0/1209392876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1704032440 con 0x7f170c161d90 2024-04-25T10:08:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f1708a00640 1 -- 172.21.15.5:0/1209392876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704032810 con 0x7f170c161d90 2024-04-25T10:08:59.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f170b200640 1 -- 172.21.15.5:0/1209392876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 msgr2=0x7f170c1689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f170b200640 1 --2- 172.21.15.5:0/1209392876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c1689b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f17040097a0 tx=0x7f170402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f170b200640 1 -- 172.21.15.5:0/1209392876 shutdown_connections 2024-04-25T10:08:59.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f170b200640 1 --2- 172.21.15.5:0/1209392876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c1689b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.283+0000 7f170b200640 1 -- 172.21.15.5:0/1209392876 >> 172.21.15.5:0/1209392876 conn(0x7f170c08af80 msgr2=0x7f170c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.284+0000 7f170b200640 1 -- 172.21.15.5:0/1209392876 shutdown_connections 2024-04-25T10:08:59.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.284+0000 7f170b200640 1 -- 172.21.15.5:0/1209392876 wait complete. 2024-04-25T10:08:59.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.285+0000 7f170b200640 1 Processor -- start 2024-04-25T10:08:59.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.285+0000 7f170b200640 1 -- start start 2024-04-25T10:08:59.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.286+0000 7f170b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f170c0b48c0 0x7f170c15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.286+0000 7f170b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f170c0d3f90 0x7f170c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.286+0000 7f170b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.286+0000 7f170b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0b3b80 con 0x7f170c161d90 2024-04-25T10:08:59.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.286+0000 7f170b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0b4ca0 con 0x7f170c0b48c0 2024-04-25T10:08:59.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.286+0000 7f170b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0b3d00 con 0x7f170c0d3f90 2024-04-25T10:08:59.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35996/0 (socket says 172.21.15.5:35996) 2024-04-25T10:08:59.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 -- 172.21.15.5:0/374316381 learned_addr learned my addr 172.21.15.5:0/374316381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:08:59.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 -- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f170c0d3f90 msgr2=0x7f170c154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f1709400640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f170c0d3f90 0x7f170c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f1709e00640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f170c0b48c0 0x7f170c15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f170c0d3f90 0x7f170c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 -- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f170c0b48c0 msgr2=0x7f170c15c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f170c0b48c0 0x7f170c15c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f170a800640 1 -- 172.21.15.5:0/374316381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c155e70 con 0x7f170c161d90 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.287+0000 7f1709e00640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f170c0b48c0 0x7f170c15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.288+0000 7f170a800640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c0cb380 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f17040097a0 tx=0x7f17040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.288+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704037040 con 0x7f170c161d90 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.288+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1704042070 con 0x7f170c161d90 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.288+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704030b30 con 0x7f170c161d90 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.288+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c15d610 con 0x7f170c161d90 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.288+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c15d330 con 0x7f170c161d90 2024-04-25T10:08:59.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.289+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c068f60 con 0x7f170c161d90 2024-04-25T10:08:59.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.290+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1704030cd0 con 0x7f170c161d90 2024-04-25T10:08:59.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.292+0000 7f1702000640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e806b4d0 0x7f16e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.292+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f17040aafa0 con 0x7f170c161d90 2024-04-25T10:08:59.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.293+0000 7f1709e00640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e806b4d0 0x7f16e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.294+0000 7f1709e00640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e806b4d0 0x7f16e806d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000f60 tx=0x7f16fc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.297+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1704079600 con 0x7f170c161d90 2024-04-25T10:08:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:08:59.533+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f170c0d0dd0 con 0x7f170c161d90 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.060+0000 7f1702000640 1 -- 172.21.15.5:0/374316381 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f170407d6a0 con 0x7f170c161d90 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e806b4d0 msgr2=0x7f16e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e806b4d0 0x7f16e806d990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f16fc000f60 tx=0x7f16fc009040 comp rx=0 tx=0).stop 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 msgr2=0x7f170c0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c0cb380 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f17040097a0 tx=0x7f17040046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 shutdown_connections 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f170c161d90 0x7f170c0cb380 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f170c0d3f90 0x7f170c154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f16e806b4d0 0x7f16e806d990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 --2- 172.21.15.5:0/374316381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f170c0b48c0 0x7f170c15c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.068+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 >> 172.21.15.5:0/374316381 conn(0x7f170c08af80 msgr2=0x7f170c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.070+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 shutdown_connections 2024-04-25T10:09:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.070+0000 7f170b200640 1 -- 172.21.15.5:0/374316381 wait complete. 2024-04-25T10:09:00.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.288+0000 7f0f63a00640 1 Processor -- start 2024-04-25T10:09:00.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.288+0000 7f0f63a00640 1 -- start start 2024-04-25T10:09:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.290+0000 7f0f63a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 0x7f0f5c0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.290+0000 7f0f63a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.290+0000 7f0f63a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0d1e80 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.290+0000 7f0f63a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b5930 con 0x7f0f5c15db00 2024-04-25T10:09:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.290+0000 7f0f63a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b5c10 con 0x7f0f5c0ce2b0 2024-04-25T10:09:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.290+0000 7f0f61a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 0x7f0f5c0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:49718/0 (socket says 172.21.15.5:49718) 2024-04-25T10:09:00.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 0x7f0f5c0d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:54220/0 (socket says 172.21.15.5:54220) 2024-04-25T10:09:00.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61a00640 1 -- 172.21.15.5:0/3207875595 learned_addr learned my addr 172.21.15.5:0/3207875595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:00.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61a00640 1 -- 172.21.15.5:0/3207875595 >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 msgr2=0x7f0f5c0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:00.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61a00640 1 --2- 172.21.15.5:0/3207875595 >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 0x7f0f5c0d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61a00640 1 -- 172.21.15.5:0/3207875595 >> v1:172.21.15.80:6789/0 conn(0x7f0f5c0b53b0 legacy=0x7f0f5c161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61a00640 1 -- 172.21.15.5:0/3207875595 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0b5ef0 con 0x7f0f5c0ce2b0 2024-04-25T10:09:00.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.291+0000 7f0f61000640 1 --2- 172.21.15.5:0/3207875595 >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 0x7f0f5c0d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:00.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.293+0000 7f0f61a00640 1 --2- 172.21.15.5:0/3207875595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0d0880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009e50 tx=0x7f0f5402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8592a99ca969d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.293+0000 7f0f5be00640 1 -- 172.21.15.5:0/3207875595 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54036070 con 0x7f0f5c0ce2b0 2024-04-25T10:09:00.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.293+0000 7f0f5be00640 1 -- 172.21.15.5:0/3207875595 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54031600 con 0x7f0f5c0ce2b0 2024-04-25T10:09:00.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.293+0000 7f0f5be00640 1 -- 172.21.15.5:0/3207875595 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f540319d0 con 0x7f0f5c0ce2b0 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 -- 172.21.15.5:0/3207875595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 msgr2=0x7f0f5c0d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 --2- 172.21.15.5:0/3207875595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0d0880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009e50 tx=0x7f0f5402a710 comp rx=0 tx=0).stop 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 -- 172.21.15.5:0/3207875595 shutdown_connections 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 --2- 172.21.15.5:0/3207875595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0d0880 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 --2- 172.21.15.5:0/3207875595 >> v2:172.21.15.80:3300/0 conn(0x7f0f5c15db00 0x7f0f5c0d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 -- 172.21.15.5:0/3207875595 >> 172.21.15.5:0/3207875595 conn(0x7f0f5c072290 msgr2=0x7f0f5c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 -- 172.21.15.5:0/3207875595 shutdown_connections 2024-04-25T10:09:00.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.294+0000 7f0f63a00640 1 -- 172.21.15.5:0/3207875595 wait complete. 2024-04-25T10:09:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.296+0000 7f0f63a00640 1 Processor -- start 2024-04-25T10:09:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.296+0000 7f0f63a00640 1 -- start start 2024-04-25T10:09:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f63a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 0x7f0f5c1f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f63a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f63a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c15db00 0x7f0f5c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f63a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0d3660 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f63a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b61d0 con 0x7f0f5c0ce2b0 2024-04-25T10:09:00.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f63a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b6c10 con 0x7f0f5c15db00 2024-04-25T10:09:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f61000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 0x7f0f5c1f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f61000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 0x7f0f5c1f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36004/0 (socket says 172.21.15.5:36004) 2024-04-25T10:09:00.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.297+0000 7f0f61000640 1 -- 172.21.15.5:0/961923866 learned_addr learned my addr 172.21.15.5:0/961923866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f61000640 1 -- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c15db00 msgr2=0x7f0f5c0c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f61000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c15db00 0x7f0f5c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f61000640 1 -- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0ce2b0 msgr2=0x7f0f5c0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f61000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f61000640 1 -- 172.21.15.5:0/961923866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0c8b00 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f61000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 0x7f0f5c1f3620 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400e990 tx=0x7f0f4400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44017070 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400c960 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400cd10 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.298+0000 7f0f63a00640 1 -- 172.21.15.5:0/961923866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c0c8de0 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.299+0000 7f0f63a00640 1 -- 172.21.15.5:0/961923866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c0c93a0 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.300+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c005290 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.301+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f4401a070 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.302+0000 7f0f59600640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3806b500 0x7f0f3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.302+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f4408a230 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.303+0000 7f0f53e00640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3806b500 0x7f0f3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.304+0000 7f0f53e00640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3806b500 0x7f0f3806d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000f60 tx=0x7f0f4c0079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.307+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f44058f10 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.499+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f2c005580 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.502+0000 7f0f59600640 1 -- 172.21.15.5:0/961923866 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16838 (secure 0 0 0) 0x7f0f44058810 con 0x7f0f5c0b53b0 2024-04-25T10:09:00.508 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:00.509 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:08:59.534974+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3806b500 msgr2=0x7f0f3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3806b500 0x7f0f3806d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000f60 tx=0x7f0f4c0079d0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 msgr2=0x7f0f5c1f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 0x7f0f5c1f3620 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400e990 tx=0x7f0f4400ee50 comp rx=0 tx=0).stop 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 shutdown_connections 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f5c15db00 0x7f0f5c0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3806b500 0x7f0f3806d9c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0ce2b0 0x7f0f5c0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 --2- 172.21.15.5:0/961923866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5c0b53b0 0x7f0f5c1f3620 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 >> 172.21.15.5:0/961923866 conn(0x7f0f5c072290 msgr2=0x7f0f5c0b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.505+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 shutdown_connections 2024-04-25T10:09:00.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.506+0000 7f0f52000640 1 -- 172.21.15.5:0/961923866 wait complete. 2024-04-25T10:09:00.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.734+0000 7fd2ef200640 1 Processor -- start 2024-04-25T10:09:00.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.734+0000 7fd2ef200640 1 -- start start 2024-04-25T10:09:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.736+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 0x7fd2e80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.736+0000 7fd2ef200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80cf410 con 0x7fd2e80d07d0 2024-04-25T10:09:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.736+0000 7fd2ef200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e80cf6f0 con 0x7fd2e8161200 2024-04-25T10:09:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.736+0000 7fd2ef200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80b28b0 con 0x7fd2e80b4bb0 2024-04-25T10:09:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.736+0000 7fd2ed400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd2e80d07d0 0x7fd2e80b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33294/0 (socket says 172.21.15.5:33294) 2024-04-25T10:09:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.736+0000 7fd2ed400640 1 -- 172.21.15.5:0/3872007237 learned_addr learned my addr 172.21.15.5:0/3872007237 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2ee800640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 0x7fd2e80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410764405 0 0) 0x7fd2e80cf410 con 0x7fd2e80d07d0 2024-04-25T10:09:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d0003520 con 0x7fd2e80d07d0 2024-04-25T10:09:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 379601538 0 0) 0x7fd2d0003520 con 0x7fd2e80d07d0 2024-04-25T10:09:00.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 msgr2=0x7fd2e80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 0x7fd2e80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 >> v1:172.21.15.80:6789/0 conn(0x7fd2e8161200 legacy=0x7fd2e80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e80d3fd0 con 0x7fd2e80d07d0 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd2d8001670 con 0x7fd2e80d07d0 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 >> v1:172.21.15.5:6789/0 conn(0x7fd2e80d07d0 legacy=0x7fd2e80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2d00039e0 0x7fd2d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 0x7fd2d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0004d40 0x7fd2d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80b28b0 con 0x7fd2d0004d40 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d000ba70 con 0x7fd2d00039e0 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.737+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d000bc70 con 0x7fd2e80b4bb0 2024-04-25T10:09:00.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ee800640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0004d40 0x7fd2d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ede00640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2d00039e0 0x7fd2d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ee800640 1 -- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 msgr2=0x7fd2d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ee800640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e80b4bb0 0x7fd2d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ee800640 1 -- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2d00039e0 msgr2=0x7fd2d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ee800640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2d00039e0 0x7fd2d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.738+0000 7fd2ee800640 1 -- 172.21.15.5:0/3872007237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0003520 con 0x7fd2d0004d40 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.739+0000 7fd2ee800640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0004d40 0x7fd2d00051c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009f90 tx=0x7fd2d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a45235b9915c033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4009c10 con 0x7fd2d0004d40 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e4036070 con 0x7fd2d0004d40 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2eca00640 1 -- 172.21.15.5:0/3872007237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4031880 con 0x7fd2d0004d40 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ef200640 1 -- 172.21.15.5:0/3872007237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0004d40 msgr2=0x7fd2d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ef200640 1 --2- 172.21.15.5:0/3872007237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2d0004d40 0x7fd2d00051c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009f90 tx=0x7fd2d0005830 comp rx=0 tx=0).stop 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ee800640 1 -- 172.21.15.5:0/3872007237 reap_dead start 2024-04-25T10:09:00.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ef200640 1 -- 172.21.15.5:0/3872007237 shutdown_connections 2024-04-25T10:09:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ef200640 1 -- 172.21.15.5:0/3872007237 >> 172.21.15.5:0/3872007237 conn(0x7fd2e808af80 msgr2=0x7fd2e80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ef200640 1 -- 172.21.15.5:0/3872007237 shutdown_connections 2024-04-25T10:09:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.740+0000 7fd2ef200640 1 -- 172.21.15.5:0/3872007237 wait complete. 2024-04-25T10:09:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.742+0000 7fd2ef200640 1 Processor -- start 2024-04-25T10:09:00.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.742+0000 7fd2ef200640 1 -- start start 2024-04-25T10:09:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e8150ab0 0x7fd2e80b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 0x7fd2e809ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ef200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2e80b8560 0x7fd2e809f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ef200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80d2500 con 0x7fd2e80b80e0 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ef200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80d2380 con 0x7fd2e80b8560 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ef200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80d2680 con 0x7fd2e8150ab0 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.743+0000 7fd2ed400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 0x7fd2e809ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 0x7fd2e809ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36020/0 (socket says 172.21.15.5:36020) 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 -- 172.21.15.5:0/1787703040 learned_addr learned my addr 172.21.15.5:0/1787703040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ee800640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2e80b8560 0x7fd2e809f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ede00640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e8150ab0 0x7fd2e80b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 -- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e8150ab0 msgr2=0x7fd2e80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e8150ab0 0x7fd2e80b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 -- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2e80b8560 msgr2=0x7fd2e809f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2e80b8560 0x7fd2e809f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 -- 172.21.15.5:0/1787703040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e809f730 con 0x7fd2e80b80e0 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.744+0000 7fd2ed400640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 0x7fd2e809ecb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c7a0 tx=0x7fd2d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.745+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8018070 con 0x7fd2e80b80e0 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.745+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d80074e0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.745+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8004a20 con 0x7fd2e80b80e0 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.745+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e815d3e0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.745+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e80b8ec0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.746+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e8068f60 con 0x7fd2e80b80e0 2024-04-25T10:09:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.747+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2d8004bc0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.748+0000 7fd2e2000640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2cc06b4d0 0x7fd2cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:00.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.748+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2d8089af0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.749+0000 7fd2ede00640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2cc06b4d0 0x7fd2cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:00.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.750+0000 7fd2ede00640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2cc06b4d0 0x7fd2cc06d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007fd0 tx=0x7fd2dc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:00.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.752+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2d80586b0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.951+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2e80d0560 con 0x7fd2e80b80e0 2024-04-25T10:09:00.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.953+0000 7fd2e2000640 1 -- 172.21.15.5:0/1787703040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16838 (secure 0 0 0) 0x7fd2d8057fb0 con 0x7fd2e80b80e0 2024-04-25T10:09:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:00.960 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:08:59.534974+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2cc06b4d0 msgr2=0x7fd2cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2cc06b4d0 0x7fd2cc06d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc007fd0 tx=0x7fd2dc009040 comp rx=0 tx=0).stop 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 msgr2=0x7fd2e809ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 0x7fd2e809ecb0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c7a0 tx=0x7fd2d800cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 shutdown_connections 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2cc06b4d0 0x7fd2cc06d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2e80b8560 0x7fd2e809f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2e80b80e0 0x7fd2e809ecb0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 --2- 172.21.15.5:0/1787703040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2e8150ab0 0x7fd2e80b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.956+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 >> 172.21.15.5:0/1787703040 conn(0x7fd2e808af80 msgr2=0x7fd2e81604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.957+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 shutdown_connections 2024-04-25T10:09:00.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:00.957+0000 7fd2ef200640 1 -- 172.21.15.5:0/1787703040 wait complete. 2024-04-25T10:09:00.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:09:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.197+0000 7f8658a00640 1 Processor -- start 2024-04-25T10:09:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.198+0000 7f8658a00640 1 -- start start 2024-04-25T10:09:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.199+0000 7f8658a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654144fe0 0x7f865406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.199+0000 7f8658a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f865406d130 0x7f8654071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8658a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86541453c0 con 0x7f8654071670 2024-04-25T10:09:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8658a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86540a8560 con 0x7f865406d130 2024-04-25T10:09:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8658a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865406d6a0 con 0x7f8654144fe0 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f865406d130 0x7f8654071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f865406d130 0x7f8654071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36022/0 (socket says 172.21.15.5:36022) 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 -- 172.21.15.5:0/536293201 learned_addr learned my addr 172.21.15.5:0/536293201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8652800640 1 --1- 172.21.15.5:0/536293201 >> v1:172.21.15.5:6789/0 conn(0x7f8654071670 0x7f86540cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33308/0 (socket says 172.21.15.5:33308) 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 -- 172.21.15.5:0/536293201 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654144fe0 msgr2=0x7f865406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651e00640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654144fe0 0x7f865406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654144fe0 0x7f865406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 -- 172.21.15.5:0/536293201 >> v1:172.21.15.5:6789/0 conn(0x7f8654071670 legacy=0x7f86540cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8650a00640 1 -- 172.21.15.5:0/536293201 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091814370 0 0) 0x7f86541453c0 con 0x7f8654071670 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651400640 1 -- 172.21.15.5:0/536293201 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865406d980 con 0x7f865406d130 2024-04-25T10:09:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.200+0000 7f8651e00640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654144fe0 0x7f865406e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.201+0000 7f8651400640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f865406d130 0x7f8654071130 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f863c009870 tx=0x7f863c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45148bff7b9583d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8650a00640 1 -- 172.21.15.5:0/536293201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c037070 con 0x7f865406d130 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8650a00640 1 -- 172.21.15.5:0/536293201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c032440 con 0x7f865406d130 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8650a00640 1 -- 172.21.15.5:0/536293201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c032810 con 0x7f865406d130 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8658a00640 1 -- 172.21.15.5:0/536293201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f865406d130 msgr2=0x7f8654071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8658a00640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f865406d130 0x7f8654071130 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f863c009870 tx=0x7f863c02a250 comp rx=0 tx=0).stop 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8658a00640 1 -- 172.21.15.5:0/536293201 shutdown_connections 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8658a00640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f865406d130 0x7f8654071130 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8658a00640 1 --2- 172.21.15.5:0/536293201 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654144fe0 0x7f865406e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.202+0000 7f8658a00640 1 -- 172.21.15.5:0/536293201 >> 172.21.15.5:0/536293201 conn(0x7f865406ab60 msgr2=0x7f865406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.203+0000 7f8658a00640 1 -- 172.21.15.5:0/536293201 shutdown_connections 2024-04-25T10:09:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.203+0000 7f8658a00640 1 -- 172.21.15.5:0/536293201 wait complete. 2024-04-25T10:09:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.204+0000 7f8658a00640 1 Processor -- start 2024-04-25T10:09:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.205+0000 7f8658a00640 1 -- start start 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8658a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f865406d130 0x7f865415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8658a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654071670 0x7f86540b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8658a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 0x7f86540b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8658a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865415be50 con 0x7f8654144fe0 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8658a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865406df40 con 0x7f865406d130 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8658a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865406eaa0 con 0x7f8654071670 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 0x7f86540b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 0x7f86540b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36030/0 (socket says 172.21.15.5:36030) 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 -- 172.21.15.5:0/2593952505 learned_addr learned my addr 172.21.15.5:0/2593952505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8651400640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654071670 0x7f86540b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8651e00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f865406d130 0x7f865415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 -- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654071670 msgr2=0x7f86540b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654071670 0x7f86540b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 -- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f865406d130 msgr2=0x7f865415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f865406d130 0x7f865415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8652800640 1 -- 172.21.15.5:0/2593952505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540ca260 con 0x7f8654144fe0 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8651400640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654071670 0x7f86540b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.206+0000 7f8651e00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f865406d130 0x7f865415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.207+0000 7f8652800640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 0x7f86540b15a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c480 tx=0x7f864c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.207+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c018070 con 0x7f8654144fe0 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.207+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c007500 con 0x7f8654144fe0 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.207+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c004a20 con 0x7f8654144fe0 2024-04-25T10:09:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.207+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86540ca540 con 0x7f8654144fe0 2024-04-25T10:09:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.207+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86540cab00 con 0x7f8654144fe0 2024-04-25T10:09:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.208+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654068f60 con 0x7f8654144fe0 2024-04-25T10:09:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.209+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f864c01b070 con 0x7f8654144fe0 2024-04-25T10:09:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.211+0000 7f864a000640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.211+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f864c0896d0 con 0x7f8654144fe0 2024-04-25T10:09:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.211+0000 7f8651e00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:01.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.213+0000 7f8651e00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f863006b4d0 0x7f863006d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8644007f40 tx=0x7f8644007490 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:01.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.215+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f864c0583b0 con 0x7f8654144fe0 2024-04-25T10:09:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:01.454+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f865406e3a0 con 0x7f8654144fe0 2024-04-25T10:09:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.101+0000 7f864a000640 1 -- 172.21.15.5:0/2593952505 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7f864c057cb0 con 0x7f8654144fe0 2024-04-25T10:09:02.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f863006b4d0 msgr2=0x7f863006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f863006b4d0 0x7f863006d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8644007f40 tx=0x7f8644007490 comp rx=0 tx=0).stop 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 msgr2=0x7f86540b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 0x7f86540b15a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f864c00c480 tx=0x7f864c00c940 comp rx=0 tx=0).stop 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 shutdown_connections 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8654144fe0 0x7f86540b15a0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8654071670 0x7f86540b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f863006b4d0 0x7f863006d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 --2- 172.21.15.5:0/2593952505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f865406d130 0x7f865415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 >> 172.21.15.5:0/2593952505 conn(0x7f865406ab60 msgr2=0x7f86540afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 shutdown_connections 2024-04-25T10:09:02.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.105+0000 7f8658a00640 1 -- 172.21.15.5:0/2593952505 wait complete. 2024-04-25T10:09:02.132 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:09:02.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:09:02.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.340+0000 7f1ac6c00640 1 Processor -- start 2024-04-25T10:09:02.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.340+0000 7f1ac6c00640 1 -- start start 2024-04-25T10:09:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ac6c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ac0144fe0 0x7f1ac006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ac6c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ac006d130 0x7f1ac0071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ac6c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac01453c0 con 0x7f1ac0071670 2024-04-25T10:09:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ac6c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac00a8560 con 0x7f1ac006d130 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ac6c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac006d6a0 con 0x7f1ac0144fe0 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ab7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ac006d130 0x7f1ac0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ab7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1ac006d130 0x7f1ac0071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36046/0 (socket says 172.21.15.5:36046) 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1ab7400640 1 -- 172.21.15.5:0/2085296719 learned_addr learned my addr 172.21.15.5:0/2085296719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.342+0000 7f1abfe00640 1 --2- 172.21.15.5:0/2085296719 >> v2:172.21.15.80:3300/0 conn(0x7f1ac0144fe0 0x7f1ac006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.343+0000 7f1ab7400640 1 -- 172.21.15.5:0/2085296719 >> v2:172.21.15.80:3300/0 conn(0x7f1ac0144fe0 msgr2=0x7f1ac006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.343+0000 7f1ab7400640 1 --2- 172.21.15.5:0/2085296719 >> v2:172.21.15.80:3300/0 conn(0x7f1ac0144fe0 0x7f1ac006e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.343+0000 7f1ab7400640 1 -- 172.21.15.5:0/2085296719 >> v1:172.21.15.5:6789/0 conn(0x7f1ac0071670 legacy=0x7f1ac00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.343+0000 7f1ab7400640 1 -- 172.21.15.5:0/2085296719 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac006d980 con 0x7f1ac006d130 2024-04-25T10:09:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.344+0000 7f1ab7400640 1 --2- 172.21.15.5:0/2085296719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac006d130 0x7f1ac0071130 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00728b0 tx=0x7f1aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4b47bd717c1a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.344+0000 7f1abf400640 1 -- 172.21.15.5:0/2085296719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8037070 con 0x7f1ac006d130 2024-04-25T10:09:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.344+0000 7f1abf400640 1 -- 172.21.15.5:0/2085296719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa8032440 con 0x7f1ac006d130 2024-04-25T10:09:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.344+0000 7f1abf400640 1 -- 172.21.15.5:0/2085296719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8032810 con 0x7f1ac006d130 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 -- 172.21.15.5:0/2085296719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac006d130 msgr2=0x7f1ac0071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/2085296719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac006d130 0x7f1ac0071130 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00728b0 tx=0x7f1aa802a270 comp rx=0 tx=0).stop 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 -- 172.21.15.5:0/2085296719 shutdown_connections 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/2085296719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac006d130 0x7f1ac0071130 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/2085296719 >> v2:172.21.15.80:3300/0 conn(0x7f1ac0144fe0 0x7f1ac006e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 -- 172.21.15.5:0/2085296719 >> 172.21.15.5:0/2085296719 conn(0x7f1ac006ab60 msgr2=0x7f1ac006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 -- 172.21.15.5:0/2085296719 shutdown_connections 2024-04-25T10:09:02.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.345+0000 7f1ac6c00640 1 -- 172.21.15.5:0/2085296719 wait complete. 2024-04-25T10:09:02.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.348+0000 7f1ac6c00640 1 Processor -- start 2024-04-25T10:09:02.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.348+0000 7f1ac6c00640 1 -- start start 2024-04-25T10:09:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.349+0000 7f1ac6c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1ac006d130 0x7f1ac015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.349+0000 7f1ac6c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 0x7f1ac01424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.349+0000 7f1ac6c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac0144fe0 0x7f1ac0142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.349+0000 7f1ac6c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015be50 con 0x7f1ac0071670 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.349+0000 7f1ac6c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac006df40 con 0x7f1ac0144fe0 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.349+0000 7f1ac6c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac006eaa0 con 0x7f1ac006d130 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 0x7f1ac01424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 0x7f1ac01424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36062/0 (socket says 172.21.15.5:36062) 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 -- 172.21.15.5:0/1403952143 learned_addr learned my addr 172.21.15.5:0/1403952143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1abfe00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1ac006d130 0x7f1ac015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ac4c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac0144fe0 0x7f1ac0142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 -- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1ac006d130 msgr2=0x7f1ac015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1ac006d130 0x7f1ac015afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 -- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac0144fe0 msgr2=0x7f1ac0142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac0144fe0 0x7f1ac0142a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 -- 172.21.15.5:0/1403952143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00b1f60 con 0x7f1ac0071670 2024-04-25T10:09:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.350+0000 7f1ab7400640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 0x7f1ac01424d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8000c00 tx=0x7f1aa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.351+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8041020 con 0x7f1ac0071670 2024-04-25T10:09:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.351+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa80306c0 con 0x7f1ac0071670 2024-04-25T10:09:02.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.351+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac01e3e70 con 0x7f1ac0071670 2024-04-25T10:09:02.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.351+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8037040 con 0x7f1ac0071670 2024-04-25T10:09:02.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.351+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01e4430 con 0x7f1ac0071670 2024-04-25T10:09:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.353+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac0068f60 con 0x7f1ac0071670 2024-04-25T10:09:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.353+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1aa8044020 con 0x7f1ac0071670 2024-04-25T10:09:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.354+0000 7f1abcc00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a9c06b4d0 0x7f1a9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.354+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1aa80aac30 con 0x7f1ac0071670 2024-04-25T10:09:02.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.355+0000 7f1abfe00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a9c06b4d0 0x7f1a9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.356+0000 7f1abfe00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a9c06b4d0 0x7f1a9c06d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0006fd0 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.359+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1aa804b030 con 0x7f1ac0071670 2024-04-25T10:09:02.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:02.598+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1ac006e1f0 con 0x7f1ac0071670 2024-04-25T10:09:03.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.113+0000 7f1abcc00640 1 -- 172.21.15.5:0/1403952143 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1aa8079880 con 0x7f1ac0071670 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a9c06b4d0 msgr2=0x7f1a9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a9c06b4d0 0x7f1a9c06d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0006fd0 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 msgr2=0x7f1ac01424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 0x7f1ac01424d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8000c00 tx=0x7f1aa80046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 shutdown_connections 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1a9c06b4d0 0x7f1a9c06d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ac0144fe0 0x7f1ac0142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ac0071670 0x7f1ac01424d0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 --2- 172.21.15.5:0/1403952143 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1ac006d130 0x7f1ac015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 >> 172.21.15.5:0/1403952143 conn(0x7f1ac006ab60 msgr2=0x7f1ac00b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.117+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 shutdown_connections 2024-04-25T10:09:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.118+0000 7f1ac6c00640 1 -- 172.21.15.5:0/1403952143 wait complete. 2024-04-25T10:09:03.146 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:09:03.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.335+0000 7ff8e7e00640 1 Processor -- start 2024-04-25T10:09:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.335+0000 7ff8e7e00640 1 -- start start 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff8e815ce00 0x7ff8e815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e7e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff8e80cd300 0x7ff8e80cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e7e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e8071d30 con 0x7ff8e8141da0 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e7e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e80cdcd0 con 0x7ff8e815ce00 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e7e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e8142180 con 0x7ff8e80cd300 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e6a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff8e8141da0 0x7ff8e806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33334/0 (socket says 172.21.15.5:33334) 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e6a00640 1 -- 172.21.15.5:0/3454078389 learned_addr learned my addr 172.21.15.5:0/3454078389 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.337+0000 7ff8e6000640 1 --2- 172.21.15.5:0/3454078389 >> v2:172.21.15.5:3300/0 conn(0x7ff8e815ce00 0x7ff8e815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e5600640 1 -- 172.21.15.5:0/3454078389 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3620847205 0 0) 0x7ff8e8071d30 con 0x7ff8e8141da0 2024-04-25T10:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e7400640 1 --2- 172.21.15.5:0/3454078389 >> v2:172.21.15.80:3300/0 conn(0x7ff8e80cd300 0x7ff8e80cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e5600640 1 -- 172.21.15.5:0/3454078389 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8cc003520 con 0x7ff8e8141da0 2024-04-25T10:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e6000640 1 -- 172.21.15.5:0/3454078389 >> v2:172.21.15.80:3300/0 conn(0x7ff8e80cd300 msgr2=0x7ff8e80cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e6000640 1 --2- 172.21.15.5:0/3454078389 >> v2:172.21.15.80:3300/0 conn(0x7ff8e80cd300 0x7ff8e80cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e6000640 1 -- 172.21.15.5:0/3454078389 >> v1:172.21.15.5:6789/0 conn(0x7ff8e8141da0 legacy=0x7ff8e806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.338+0000 7ff8e6000640 1 -- 172.21.15.5:0/3454078389 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e806d620 con 0x7ff8e815ce00 2024-04-25T10:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.339+0000 7ff8e6000640 1 --2- 172.21.15.5:0/3454078389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e815ce00 0x7ff8e815d280 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff8e815f220 tx=0x7ff8d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6f7b61985d93ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.339+0000 7ff8e5600640 1 -- 172.21.15.5:0/3454078389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4037070 con 0x7ff8e815ce00 2024-04-25T10:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e5600640 1 -- 172.21.15.5:0/3454078389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d4032440 con 0x7ff8e815ce00 2024-04-25T10:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e5600640 1 -- 172.21.15.5:0/3454078389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40327f0 con 0x7ff8e815ce00 2024-04-25T10:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 -- 172.21.15.5:0/3454078389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e815ce00 msgr2=0x7ff8e815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/3454078389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e815ce00 0x7ff8e815d280 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff8e815f220 tx=0x7ff8d402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 -- 172.21.15.5:0/3454078389 shutdown_connections 2024-04-25T10:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/3454078389 >> v2:172.21.15.80:3300/0 conn(0x7ff8e80cd300 0x7ff8e80cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/3454078389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e815ce00 0x7ff8e815d280 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 -- 172.21.15.5:0/3454078389 >> 172.21.15.5:0/3454078389 conn(0x7ff8e8066f20 msgr2=0x7ff8e806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 -- 172.21.15.5:0/3454078389 shutdown_connections 2024-04-25T10:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.340+0000 7ff8e7e00640 1 -- 172.21.15.5:0/3454078389 wait complete. 2024-04-25T10:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.342+0000 7ff8e7e00640 1 Processor -- start 2024-04-25T10:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.342+0000 7ff8e7e00640 1 -- start start 2024-04-25T10:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 0x7ff8e81f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8e8141da0 0x7ff8e81f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e7e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8e815ce00 0x7ff8e81f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e806e810 con 0x7ff8e80cd300 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e7e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e806dbe0 con 0x7ff8e8141da0 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e7e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8071000 con 0x7ff8e815ce00 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.343+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 0x7ff8e81f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 0x7ff8e81f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36074/0 (socket says 172.21.15.5:36074) 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 -- 172.21.15.5:0/2343261761 learned_addr learned my addr 172.21.15.5:0/2343261761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6000640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8e8141da0 0x7ff8e81f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e7400640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8e815ce00 0x7ff8e81f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 -- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8e815ce00 msgr2=0x7ff8e81f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8e815ce00 0x7ff8e81f7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 -- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8e8141da0 msgr2=0x7ff8e81f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8e8141da0 0x7ff8e81f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 -- 172.21.15.5:0/2343261761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e80c9430 con 0x7ff8e80cd300 2024-04-25T10:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.344+0000 7ff8e6a00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 0x7ff8e81f6c90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c460 tx=0x7ff8d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.345+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8017070 con 0x7ff8e80cd300 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.345+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8005ce0 con 0x7ff8e80cd300 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.345+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80072f0 con 0x7ff8e80cd300 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.345+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e80cac80 con 0x7ff8e80cd300 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.345+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e80c9090 con 0x7ff8e80cd300 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.346+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8068f60 con 0x7ff8e80cd300 2024-04-25T10:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.347+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8d801a070 con 0x7ff8e80cd300 2024-04-25T10:09:03.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.348+0000 7ff8dea00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff8c806b4d0 0x7ff8c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.348+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8d8089880 con 0x7ff8e80cd300 2024-04-25T10:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.349+0000 7ff8e6000640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff8c806b4d0 0x7ff8c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.350+0000 7ff8e6000640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff8c806b4d0 0x7ff8c806d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4034000 tx=0x7ff8d402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.352+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8d80584d0 con 0x7ff8e80cd300 2024-04-25T10:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.592+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e806d2b0 con 0x7ff8e80cd300 2024-04-25T10:09:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.593+0000 7ff8dea00640 1 -- 172.21.15.5:0/2343261761 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4928 (secure 0 0 0) 0x7ff8d8057dd0 con 0x7ff8e80cd300 2024-04-25T10:09:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:03.599 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:02.600363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7488":{"gid":7488,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/3627376661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3627376661},{"type":"v1","addr":"172.21.15.5:6834","nonce":3627376661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 62 2024-04-25T10:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff8c806b4d0 msgr2=0x7ff8c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff8c806b4d0 0x7ff8c806d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4034000 tx=0x7ff8d402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 msgr2=0x7ff8e81f6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 0x7ff8e81f6c90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c460 tx=0x7ff8d800c920 comp rx=0 tx=0).stop 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 shutdown_connections 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff8e815ce00 0x7ff8e81f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff8c806b4d0 0x7ff8c806d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff8e8141da0 0x7ff8e81f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 --2- 172.21.15.5:0/2343261761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8e80cd300 0x7ff8e81f6c90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.596+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 >> 172.21.15.5:0/2343261761 conn(0x7ff8e8066f20 msgr2=0x7ff8e806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.597+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 shutdown_connections 2024-04-25T10:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.597+0000 7ff8e7e00640 1 -- 172.21.15.5:0/2343261761 wait complete. 2024-04-25T10:09:03.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.826+0000 7f0092600640 1 Processor -- start 2024-04-25T10:09:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.826+0000 7f0092600640 1 -- start start 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0092600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f008c15de70 0x7f008c15e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0092600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c06f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0092600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f008c1613b0 0x7f008c161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0092600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c09a7a0 con 0x7f008c1613b0 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0092600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c15ec40 con 0x7f008c15de70 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0092600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c15e790 con 0x7f008c06f260 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0091c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f008c1613b0 0x7f008c161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.828+0000 7f0091c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f008c1613b0 0x7f008c161860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36086/0 (socket says 172.21.15.5:36086) 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091c00640 1 -- 172.21.15.5:0/3655428921 learned_addr learned my addr 172.21.15.5:0/3655428921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:03.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091200640 1 --2- 172.21.15.5:0/3655428921 >> v2:172.21.15.80:3300/0 conn(0x7f008c15de70 0x7f008c15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f008be00640 1 --2- 172.21.15.5:0/3655428921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c06f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091c00640 1 -- 172.21.15.5:0/3655428921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 msgr2=0x7f008c06f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091c00640 1 --2- 172.21.15.5:0/3655428921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c06f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091c00640 1 -- 172.21.15.5:0/3655428921 >> v2:172.21.15.80:3300/0 conn(0x7f008c15de70 msgr2=0x7f008c15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091c00640 1 --2- 172.21.15.5:0/3655428921 >> v2:172.21.15.80:3300/0 conn(0x7f008c15de70 0x7f008c15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.829+0000 7f0091c00640 1 -- 172.21.15.5:0/3655428921 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c099b90 con 0x7f008c1613b0 2024-04-25T10:09:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.830+0000 7f0091c00640 1 --2- 172.21.15.5:0/3655428921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c161860 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f008c0b3ba0 tx=0x7f008402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65ca5e0bfad305a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.830+0000 7f008b400640 1 -- 172.21.15.5:0/3655428921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084036070 con 0x7f008c1613b0 2024-04-25T10:09:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.830+0000 7f008b400640 1 -- 172.21.15.5:0/3655428921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084031600 con 0x7f008c1613b0 2024-04-25T10:09:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.830+0000 7f008b400640 1 -- 172.21.15.5:0/3655428921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840319d0 con 0x7f008c1613b0 2024-04-25T10:09:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 -- 172.21.15.5:0/3655428921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 msgr2=0x7f008c161860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 --2- 172.21.15.5:0/3655428921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c161860 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f008c0b3ba0 tx=0x7f008402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 -- 172.21.15.5:0/3655428921 shutdown_connections 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 --2- 172.21.15.5:0/3655428921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c161860 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 --2- 172.21.15.5:0/3655428921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 --2- 172.21.15.5:0/3655428921 >> v2:172.21.15.80:3300/0 conn(0x7f008c15de70 0x7f008c15e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 -- 172.21.15.5:0/3655428921 >> 172.21.15.5:0/3655428921 conn(0x7f008c0b2fd0 msgr2=0x7f008c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 -- 172.21.15.5:0/3655428921 shutdown_connections 2024-04-25T10:09:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.831+0000 7f0092600640 1 -- 172.21.15.5:0/3655428921 wait complete. 2024-04-25T10:09:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.833+0000 7f0092600640 1 Processor -- start 2024-04-25T10:09:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.833+0000 7f0092600640 1 -- start start 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0092600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c0cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0092600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f008c15de70 0x7f008c14b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0092600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0092600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c09a430 con 0x7f008c1613b0 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0092600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c09a150 con 0x7f008c15de70 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0092600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c15dab0 con 0x7f008c06f260 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.834+0000 7f0091c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c15ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36088/0 (socket says 172.21.15.5:36088) 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 -- 172.21.15.5:0/2575723791 learned_addr learned my addr 172.21.15.5:0/2575723791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091200640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c0cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f008be00640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f008c15de70 0x7f008c14b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 -- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 msgr2=0x7f008c0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c0cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 -- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f008c15de70 msgr2=0x7f008c14b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f008c15de70 0x7f008c14b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 -- 172.21.15.5:0/2575723791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c0cd8a0 con 0x7f008c1613b0 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f008be00640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f008c15de70 0x7f008c14b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:03.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.835+0000 7f0091c00640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c15ab50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0084009e20 tx=0x7f0084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.836+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084041070 con 0x7f008c1613b0 2024-04-25T10:09:03.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.836+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008403c650 con 0x7f008c1613b0 2024-04-25T10:09:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.836+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c0cbf80 con 0x7f008c1613b0 2024-04-25T10:09:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.836+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084036040 con 0x7f008c1613b0 2024-04-25T10:09:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.836+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c0cc540 con 0x7f008c1613b0 2024-04-25T10:09:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.837+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c068f60 con 0x7f008c1613b0 2024-04-25T10:09:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.838+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0084044070 con 0x7f008c1613b0 2024-04-25T10:09:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.839+0000 7f0088c00640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f006c06b4d0 0x7f006c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.839+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00840aae80 con 0x7f008c1613b0 2024-04-25T10:09:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.840+0000 7f0091200640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f006c06b4d0 0x7f006c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.841+0000 7f0091200640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f006c06b4d0 0x7f006c06d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0080006fd0 tx=0x7f0080008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:03.843+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0084079ad0 con 0x7f008c1613b0 2024-04-25T10:09:04.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.044+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f008c0b3e30 con 0x7f008c1613b0 2024-04-25T10:09:04.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.046+0000 7f0088c00640 1 -- 172.21.15.5:0/2575723791 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16825 (secure 0 0 0) 0x7f00840793d0 con 0x7f008c1613b0 2024-04-25T10:09:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:04.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:01.074532+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f006c06b4d0 msgr2=0x7f006c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f006c06b4d0 0x7f006c06d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0080006fd0 tx=0x7f0080008040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 msgr2=0x7f008c15ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c15ab50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0084009e20 tx=0x7f0084004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 shutdown_connections 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f008c1613b0 0x7f008c15ab50 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f006c06b4d0 0x7f006c06d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f008c15de70 0x7f008c14b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 --2- 172.21.15.5:0/2575723791 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f008c06f260 0x7f008c0cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 >> 172.21.15.5:0/2575723791 conn(0x7f008c0b2fd0 msgr2=0x7f008c06c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 shutdown_connections 2024-04-25T10:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.049+0000 7f0092600640 1 -- 172.21.15.5:0/2575723791 wait complete. 2024-04-25T10:09:04.075 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:09:04.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:04.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.280+0000 7fe0c3200640 1 Processor -- start 2024-04-25T10:09:04.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.280+0000 7fe0c3200640 1 -- start start 2024-04-25T10:09:04.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.281+0000 7fe0c3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0c4161200 0x7fe0c40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.281+0000 7fe0c3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d2380 0x7fe0c40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.281+0000 7fe0c3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40cf410 con 0x7fe0c40d0600 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.281+0000 7fe0c3200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40d27c0 con 0x7fe0c4161200 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.281+0000 7fe0c3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b28b0 con 0x7fe0c40d2380 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.281+0000 7fe0c1e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0c4161200 0x7fe0c40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0c4161200 0x7fe0c40b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36100/0 (socket says 172.21.15.5:36100) 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1e00640 1 -- 172.21.15.5:0/3612646673 learned_addr learned my addr 172.21.15.5:0/3612646673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1400640 1 --2- 172.21.15.5:0/3612646673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d2380 0x7fe0c40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1e00640 1 -- 172.21.15.5:0/3612646673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d2380 msgr2=0x7fe0c40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1e00640 1 --2- 172.21.15.5:0/3612646673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d2380 0x7fe0c40d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1e00640 1 -- 172.21.15.5:0/3612646673 >> v1:172.21.15.80:6789/0 conn(0x7fe0c40d0600 legacy=0x7fe0c40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.282+0000 7fe0c1e00640 1 -- 172.21.15.5:0/3612646673 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40cf6f0 con 0x7fe0c4161200 2024-04-25T10:09:04.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.283+0000 7fe0c1e00640 1 --2- 172.21.15.5:0/3612646673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c4161200 0x7fe0c40b4bb0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e80 tx=0x7fe0b402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9035ec0ac7623ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c0a00640 1 -- 172.21.15.5:0/3612646673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4036070 con 0x7fe0c4161200 2024-04-25T10:09:04.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c0a00640 1 -- 172.21.15.5:0/3612646673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4031600 con 0x7fe0c4161200 2024-04-25T10:09:04.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c0a00640 1 -- 172.21.15.5:0/3612646673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40319d0 con 0x7fe0c4161200 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c3200640 1 -- 172.21.15.5:0/3612646673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c4161200 msgr2=0x7fe0c40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3612646673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c4161200 0x7fe0c40b4bb0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009e80 tx=0x7fe0b402a750 comp rx=0 tx=0).stop 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c3200640 1 -- 172.21.15.5:0/3612646673 shutdown_connections 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3612646673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d2380 0x7fe0c40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3612646673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c4161200 0x7fe0c40b4bb0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.284+0000 7fe0c3200640 1 -- 172.21.15.5:0/3612646673 >> 172.21.15.5:0/3612646673 conn(0x7fe0c408af80 msgr2=0x7fe0c40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.285+0000 7fe0c3200640 1 -- 172.21.15.5:0/3612646673 shutdown_connections 2024-04-25T10:09:04.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.285+0000 7fe0c3200640 1 -- 172.21.15.5:0/3612646673 wait complete. 2024-04-25T10:09:04.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.286+0000 7fe0c3200640 1 Processor -- start 2024-04-25T10:09:04.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.287+0000 7fe0c3200640 1 -- start start 2024-04-25T10:09:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d0600 0x7fe0c4150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c40d2380 0x7fe0c40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 0x7fe0c409ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b3f90 con 0x7fe0c40ccc00 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40d42b0 con 0x7fe0c40d2380 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40b4110 con 0x7fe0c40d0600 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 0x7fe0c409ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 0x7fe0c409ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36114/0 (socket says 172.21.15.5:36114) 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c2800640 1 -- 172.21.15.5:0/3759181595 learned_addr learned my addr 172.21.15.5:0/3759181595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c1400640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c40d2380 0x7fe0c40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.288+0000 7fe0c1e00640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d0600 0x7fe0c4150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c2800640 1 -- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d0600 msgr2=0x7fe0c4150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c2800640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d0600 0x7fe0c4150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c2800640 1 -- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c40d2380 msgr2=0x7fe0c40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c2800640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c40d2380 0x7fe0c40cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c2800640 1 -- 172.21.15.5:0/3759181595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40cada0 con 0x7fe0c40ccc00 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c1400640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c40d2380 0x7fe0c40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c2800640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 0x7fe0c409ee10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00c390 tx=0x7fe0bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc017070 con 0x7fe0c40ccc00 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc0052d0 con 0x7fe0c40ccc00 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc003810 con 0x7fe0c40ccc00 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c40cc140 con 0x7fe0c40ccc00 2024-04-25T10:09:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.289+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c40cb320 con 0x7fe0c40ccc00 2024-04-25T10:09:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.291+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c4068f60 con 0x7fe0c40ccc00 2024-04-25T10:09:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.292+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0bc01a070 con 0x7fe0c40ccc00 2024-04-25T10:09:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.293+0000 7fe0ba000640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0a406b4d0 0x7fe0a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.294+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0bc089650 con 0x7fe0c40ccc00 2024-04-25T10:09:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.294+0000 7fe0c1e00640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0a406b4d0 0x7fe0a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.295+0000 7fe0c1e00640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0a406b4d0 0x7fe0a406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4033000 tx=0x7fe0b4009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.298+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0bc0583d0 con 0x7fe0c40ccc00 2024-04-25T10:09:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.494+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0c40cb560 con 0x7fe0a406b4d0 2024-04-25T10:09:04.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.514+0000 7fe0ba000640 1 -- 172.21.15.5:0/3759181595 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251044 (secure 0 0 0) 0x7fe0c40cb560 con 0x7fe0a406b4d0 2024-04-25T10:09:04.524 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:09:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0a406b4d0 msgr2=0x7fe0a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0a406b4d0 0x7fe0a406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4033000 tx=0x7fe0b4009000 comp rx=0 tx=0).stop 2024-04-25T10:09:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 msgr2=0x7fe0c409ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 0x7fe0c409ee10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00c390 tx=0x7fe0bc00c850 comp rx=0 tx=0).stop 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 shutdown_connections 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c40ccc00 0x7fe0c409ee10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0a406b4d0 0x7fe0a406d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0c40d2380 0x7fe0c40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 --2- 172.21.15.5:0/3759181595 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0c40d0600 0x7fe0c4150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 >> 172.21.15.5:0/3759181595 conn(0x7fe0c408af80 msgr2=0x7fe0c406d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.520+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 shutdown_connections 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.521+0000 7fe0c3200640 1 -- 172.21.15.5:0/3759181595 wait complete. 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-25T10:09:02.770211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259084,"kb_used_data":9628,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715636,"statfs":{"total":773094113280,"available":772828811264,"internally_reserved":0,"allocated":9859072,"data_stored":5108797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.725145"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115301+0000","last_change":"2024-04-25T10:09:00.107093+0000","last_active":"2024-04-25T10:09:00.115301+0000","last_peered":"2024-04-25T10:09:00.115301+0000","last_clean":"2024-04-25T10:09:00.115301+0000","last_became_active":"2024-04-25T10:09:00.106825+0000","last_became_peered":"2024-04-25T10:09:00.106825+0000","last_unstale":"2024-04-25T10:09:00.115301+0000","last_undegraded":"2024-04-25T10:09:00.115301+0000","last_fullsized":"2024-04-25T10:09:00.115301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:19.534934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.101384+0000","last_change":"2024-04-25T10:09:00.098271+0000","last_active":"2024-04-25T10:09:00.101384+0000","last_peered":"2024-04-25T10:09:00.101384+0000","last_clean":"2024-04-25T10:09:00.101384+0000","last_became_active":"2024-04-25T10:09:00.097364+0000","last_became_peered":"2024-04-25T10:09:00.097364+0000","last_unstale":"2024-04-25T10:09:00.101384+0000","last_undegraded":"2024-04-25T10:09:00.101384+0000","last_fullsized":"2024-04-25T10:09:00.101384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:18.272172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114060+0000","last_change":"2024-04-25T10:09:00.110468+0000","last_active":"2024-04-25T10:09:00.114060+0000","last_peered":"2024-04-25T10:09:00.114060+0000","last_clean":"2024-04-25T10:09:00.114060+0000","last_became_active":"2024-04-25T10:09:00.105042+0000","last_became_peered":"2024-04-25T10:09:00.105042+0000","last_unstale":"2024-04-25T10:09:00.114060+0000","last_undegraded":"2024-04-25T10:09:00.114060+0000","last_fullsized":"2024-04-25T10:09:00.114060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:37.168689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114893+0000","last_change":"2024-04-25T10:09:00.101968+0000","last_active":"2024-04-25T10:09:00.114893+0000","last_peered":"2024-04-25T10:09:00.114893+0000","last_clean":"2024-04-25T10:09:00.114893+0000","last_became_active":"2024-04-25T10:09:00.096794+0000","last_became_peered":"2024-04-25T10:09:00.096794+0000","last_unstale":"2024-04-25T10:09:00.114893+0000","last_undegraded":"2024-04-25T10:09:00.114893+0000","last_fullsized":"2024-04-25T10:09:00.114893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:39.761726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092230+0000","last_change":"2024-04-25T10:09:00.089257+0000","last_active":"2024-04-25T10:09:00.092230+0000","last_peered":"2024-04-25T10:09:00.092230+0000","last_clean":"2024-04-25T10:09:00.092230+0000","last_became_active":"2024-04-25T10:09:00.088429+0000","last_became_peered":"2024-04-25T10:09:00.088429+0000","last_unstale":"2024-04-25T10:09:00.092230+0000","last_undegraded":"2024-04-25T10:09:00.092230+0000","last_fullsized":"2024-04-25T10:09:00.092230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:23.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.073426+0000","last_change":"2024-04-25T10:08:59.073426+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.073426+0000","last_undegraded":"2024-04-25T10:08:59.073426+0000","last_fullsized":"2024-04-25T10:08:59.073426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103772+0000","last_change":"2024-04-25T10:09:00.102056+0000","last_active":"2024-04-25T10:09:00.103772+0000","last_peered":"2024-04-25T10:09:00.103772+0000","last_clean":"2024-04-25T10:09:00.103772+0000","last_became_active":"2024-04-25T10:09:00.101790+0000","last_became_peered":"2024-04-25T10:09:00.101790+0000","last_unstale":"2024-04-25T10:09:00.103772+0000","last_undegraded":"2024-04-25T10:09:00.103772+0000","last_fullsized":"2024-04-25T10:09:00.103772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:05.614089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.076122+0000","last_change":"2024-04-25T10:08:59.076122+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.076122+0000","last_undegraded":"2024-04-25T10:08:59.076122+0000","last_fullsized":"2024-04-25T10:08:59.076122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.094324+0000","last_change":"2024-04-25T10:09:00.092182+0000","last_active":"2024-04-25T10:09:00.094324+0000","last_peered":"2024-04-25T10:09:00.094324+0000","last_clean":"2024-04-25T10:09:00.094324+0000","last_became_active":"2024-04-25T10:09:00.090969+0000","last_became_peered":"2024-04-25T10:09:00.090969+0000","last_unstale":"2024-04-25T10:09:00.094324+0000","last_undegraded":"2024-04-25T10:09:00.094324+0000","last_fullsized":"2024-04-25T10:09:00.094324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.039093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117947+0000","last_change":"2024-04-25T10:09:00.115320+0000","last_active":"2024-04-25T10:09:00.117947+0000","last_peered":"2024-04-25T10:09:00.117947+0000","last_clean":"2024-04-25T10:09:00.117947+0000","last_became_active":"2024-04-25T10:09:00.112216+0000","last_became_peered":"2024-04-25T10:09:00.112216+0000","last_unstale":"2024-04-25T10:09:00.117947+0000","last_undegraded":"2024-04-25T10:09:00.117947+0000","last_fullsized":"2024-04-25T10:09:00.117947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:01.634967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103055+0000","last_change":"2024-04-25T10:09:00.099685+0000","last_active":"2024-04-25T10:09:00.103055+0000","last_peered":"2024-04-25T10:09:00.103055+0000","last_clean":"2024-04-25T10:09:00.103055+0000","last_became_active":"2024-04-25T10:09:00.099391+0000","last_became_peered":"2024-04-25T10:09:00.099391+0000","last_unstale":"2024-04-25T10:09:00.103055+0000","last_undegraded":"2024-04-25T10:09:00.103055+0000","last_fullsized":"2024-04-25T10:09:00.103055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:14.198166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114919+0000","last_change":"2024-04-25T10:09:00.109297+0000","last_active":"2024-04-25T10:09:00.114919+0000","last_peered":"2024-04-25T10:09:00.114919+0000","last_clean":"2024-04-25T10:09:00.114919+0000","last_became_active":"2024-04-25T10:09:00.108965+0000","last_became_peered":"2024-04-25T10:09:00.108965+0000","last_unstale":"2024-04-25T10:09:00.114919+0000","last_undegraded":"2024-04-25T10:09:00.114919+0000","last_fullsized":"2024-04-25T10:09:00.114919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:26.119223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117003+0000","last_change":"2024-04-25T10:09:00.107706+0000","last_active":"2024-04-25T10:09:00.117003+0000","last_peered":"2024-04-25T10:09:00.117003+0000","last_clean":"2024-04-25T10:09:00.117003+0000","last_became_active":"2024-04-25T10:09:00.107433+0000","last_became_peered":"2024-04-25T10:09:00.107433+0000","last_unstale":"2024-04-25T10:09:00.117003+0000","last_undegraded":"2024-04-25T10:09:00.117003+0000","last_fullsized":"2024-04-25T10:09:00.117003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:02.507363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.101043+0000","last_change":"2024-04-25T10:09:00.098596+0000","last_active":"2024-04-25T10:09:00.101043+0000","last_peered":"2024-04-25T10:09:00.101043+0000","last_clean":"2024-04-25T10:09:00.101043+0000","last_became_active":"2024-04-25T10:09:00.098267+0000","last_became_peered":"2024-04-25T10:09:00.098267+0000","last_unstale":"2024-04-25T10:09:00.101043+0000","last_undegraded":"2024-04-25T10:09:00.101043+0000","last_fullsized":"2024-04-25T10:09:00.101043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:23.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.104235+0000","last_change":"2024-04-25T10:09:00.095973+0000","last_active":"2024-04-25T10:09:00.104235+0000","last_peered":"2024-04-25T10:09:00.104235+0000","last_clean":"2024-04-25T10:09:00.104235+0000","last_became_active":"2024-04-25T10:09:00.095690+0000","last_became_peered":"2024-04-25T10:09:00.095690+0000","last_unstale":"2024-04-25T10:09:00.104235+0000","last_undegraded":"2024-04-25T10:09:00.104235+0000","last_fullsized":"2024-04-25T10:09:00.104235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:12.225717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.073232+0000","last_change":"2024-04-25T10:08:59.073232+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.073232+0000","last_undegraded":"2024-04-25T10:08:59.073232+0000","last_fullsized":"2024-04-25T10:08:59.073232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.072995+0000","last_change":"2024-04-25T10:08:59.072995+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.072995+0000","last_undegraded":"2024-04-25T10:08:59.072995+0000","last_fullsized":"2024-04-25T10:08:59.072995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114007+0000","last_change":"2024-04-25T10:09:00.106048+0000","last_active":"2024-04-25T10:09:00.114007+0000","last_peered":"2024-04-25T10:09:00.114007+0000","last_clean":"2024-04-25T10:09:00.114007+0000","last_became_active":"2024-04-25T10:09:00.105793+0000","last_became_peered":"2024-04-25T10:09:00.105793+0000","last_unstale":"2024-04-25T10:09:00.114007+0000","last_undegraded":"2024-04-25T10:09:00.114007+0000","last_fullsized":"2024-04-25T10:09:00.114007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:11.049854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.102990+0000","last_change":"2024-04-25T10:09:00.099526+0000","last_active":"2024-04-25T10:09:00.102990+0000","last_peered":"2024-04-25T10:09:00.102990+0000","last_clean":"2024-04-25T10:09:00.102990+0000","last_became_active":"2024-04-25T10:09:00.099234+0000","last_became_peered":"2024-04-25T10:09:00.099234+0000","last_unstale":"2024-04-25T10:09:00.102990+0000","last_undegraded":"2024-04-25T10:09:00.102990+0000","last_fullsized":"2024-04-25T10:09:00.102990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:02.507363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.066271+0000","last_change":"2024-04-25T10:08:59.066271+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.066271+0000","last_undegraded":"2024-04-25T10:08:59.066271+0000","last_fullsized":"2024-04-25T10:08:59.066271+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092209+0000","last_change":"2024-04-25T10:09:00.088779+0000","last_active":"2024-04-25T10:09:00.092209+0000","last_peered":"2024-04-25T10:09:00.092209+0000","last_clean":"2024-04-25T10:09:00.092209+0000","last_became_active":"2024-04-25T10:09:00.087994+0000","last_became_peered":"2024-04-25T10:09:00.087994+0000","last_unstale":"2024-04-25T10:09:00.092209+0000","last_undegraded":"2024-04-25T10:09:00.092209+0000","last_fullsized":"2024-04-25T10:09:00.092209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:23.373405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.098850+0000","last_change":"2024-04-25T10:09:00.094739+0000","last_active":"2024-04-25T10:09:00.098850+0000","last_peered":"2024-04-25T10:09:00.098850+0000","last_clean":"2024-04-25T10:09:00.098850+0000","last_became_active":"2024-04-25T10:09:00.093298+0000","last_became_peered":"2024-04-25T10:09:00.093298+0000","last_unstale":"2024-04-25T10:09:00.098850+0000","last_undegraded":"2024-04-25T10:09:00.098850+0000","last_fullsized":"2024-04-25T10:09:00.098850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:54.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103551+0000","last_change":"2024-04-25T10:09:00.102614+0000","last_active":"2024-04-25T10:09:00.103551+0000","last_peered":"2024-04-25T10:09:00.103551+0000","last_clean":"2024-04-25T10:09:00.103551+0000","last_became_active":"2024-04-25T10:09:00.102006+0000","last_became_peered":"2024-04-25T10:09:00.102006+0000","last_unstale":"2024-04-25T10:09:00.103551+0000","last_undegraded":"2024-04-25T10:09:00.103551+0000","last_fullsized":"2024-04-25T10:09:00.103551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:25.068236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115372+0000","last_change":"2024-04-25T10:09:00.105944+0000","last_active":"2024-04-25T10:09:00.115372+0000","last_peered":"2024-04-25T10:09:00.115372+0000","last_clean":"2024-04-25T10:09:00.115372+0000","last_became_active":"2024-04-25T10:09:00.104706+0000","last_became_peered":"2024-04-25T10:09:00.104706+0000","last_unstale":"2024-04-25T10:09:00.115372+0000","last_undegraded":"2024-04-25T10:09:00.115372+0000","last_fullsized":"2024-04-25T10:09:00.115372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:59.579367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.100991+0000","last_change":"2024-04-25T10:09:00.093829+0000","last_active":"2024-04-25T10:09:00.100991+0000","last_peered":"2024-04-25T10:09:00.100991+0000","last_clean":"2024-04-25T10:09:00.100991+0000","last_became_active":"2024-04-25T10:09:00.089291+0000","last_became_peered":"2024-04-25T10:09:00.089291+0000","last_unstale":"2024-04-25T10:09:00.100991+0000","last_undegraded":"2024-04-25T10:09:00.100991+0000","last_fullsized":"2024-04-25T10:09:00.100991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:30.601882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103273+0000","last_change":"2024-04-25T10:09:00.101540+0000","last_active":"2024-04-25T10:09:00.103273+0000","last_peered":"2024-04-25T10:09:00.103273+0000","last_clean":"2024-04-25T10:09:00.103273+0000","last_became_active":"2024-04-25T10:09:00.101169+0000","last_became_peered":"2024-04-25T10:09:00.101169+0000","last_unstale":"2024-04-25T10:09:00.103273+0000","last_undegraded":"2024-04-25T10:09:00.103273+0000","last_fullsized":"2024-04-25T10:09:00.103273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:46.287236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103592+0000","last_change":"2024-04-25T10:09:00.098104+0000","last_active":"2024-04-25T10:09:00.103592+0000","last_peered":"2024-04-25T10:09:00.103592+0000","last_clean":"2024-04-25T10:09:00.103592+0000","last_became_active":"2024-04-25T10:09:00.097730+0000","last_became_peered":"2024-04-25T10:09:00.097730+0000","last_unstale":"2024-04-25T10:09:00.103592+0000","last_undegraded":"2024-04-25T10:09:00.103592+0000","last_fullsized":"2024-04-25T10:09:00.103592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.628186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117174+0000","last_change":"2024-04-25T10:09:00.107218+0000","last_active":"2024-04-25T10:09:00.117174+0000","last_peered":"2024-04-25T10:09:00.117174+0000","last_clean":"2024-04-25T10:09:00.117174+0000","last_became_active":"2024-04-25T10:09:00.106742+0000","last_became_peered":"2024-04-25T10:09:00.106742+0000","last_unstale":"2024-04-25T10:09:00.117174+0000","last_undegraded":"2024-04-25T10:09:00.117174+0000","last_fullsized":"2024-04-25T10:09:00.117174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:21.577126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116977+0000","last_change":"2024-04-25T10:09:00.116341+0000","last_active":"2024-04-25T10:09:00.116977+0000","last_peered":"2024-04-25T10:09:00.116977+0000","last_clean":"2024-04-25T10:09:00.116977+0000","last_became_active":"2024-04-25T10:09:00.115996+0000","last_became_peered":"2024-04-25T10:09:00.115996+0000","last_unstale":"2024-04-25T10:09:00.116977+0000","last_undegraded":"2024-04-25T10:09:00.116977+0000","last_fullsized":"2024-04-25T10:09:00.116977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:29.189977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092051+0000","last_change":"2024-04-25T10:09:00.088170+0000","last_active":"2024-04-25T10:09:00.092051+0000","last_peered":"2024-04-25T10:09:00.092051+0000","last_clean":"2024-04-25T10:09:00.092051+0000","last_became_active":"2024-04-25T10:09:00.087877+0000","last_became_peered":"2024-04-25T10:09:00.087877+0000","last_unstale":"2024-04-25T10:09:00.092051+0000","last_undegraded":"2024-04-25T10:09:00.092051+0000","last_fullsized":"2024-04-25T10:09:00.092051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:00.826258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.113806+0000","last_change":"2024-04-25T10:09:00.112242+0000","last_active":"2024-04-25T10:09:00.113806+0000","last_peered":"2024-04-25T10:09:00.113806+0000","last_clean":"2024-04-25T10:09:00.113806+0000","last_became_active":"2024-04-25T10:09:00.111951+0000","last_became_peered":"2024-04-25T10:09:00.111951+0000","last_unstale":"2024-04-25T10:09:00.113806+0000","last_undegraded":"2024-04-25T10:09:00.113806+0000","last_fullsized":"2024-04-25T10:09:00.113806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:20.776889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.070634+0000","last_change":"2024-04-25T10:08:59.070634+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.070634+0000","last_undegraded":"2024-04-25T10:08:59.070634+0000","last_fullsized":"2024-04-25T10:08:59.070634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115077+0000","last_change":"2024-04-25T10:09:00.104938+0000","last_active":"2024-04-25T10:09:00.115077+0000","last_peered":"2024-04-25T10:09:00.115077+0000","last_clean":"2024-04-25T10:09:00.115077+0000","last_became_active":"2024-04-25T10:09:00.104620+0000","last_became_peered":"2024-04-25T10:09:00.104620+0000","last_unstale":"2024-04-25T10:09:00.115077+0000","last_undegraded":"2024-04-25T10:09:00.115077+0000","last_fullsized":"2024-04-25T10:09:00.115077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:06.117065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.095377+0000","last_change":"2024-04-25T10:08:59.097251+0000","last_active":"2024-04-25T10:09:00.095377+0000","last_peered":"2024-04-25T10:09:00.095377+0000","last_clean":"2024-04-25T10:09:00.095377+0000","last_became_active":"2024-04-25T10:08:59.096348+0000","last_became_peered":"2024-04-25T10:08:59.096348+0000","last_unstale":"2024-04-25T10:09:00.095377+0000","last_undegraded":"2024-04-25T10:09:00.095377+0000","last_fullsized":"2024-04-25T10:09:00.095377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:05.109244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.072346+0000","last_change":"2024-04-25T10:08:59.072346+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.072346+0000","last_undegraded":"2024-04-25T10:08:59.072346+0000","last_fullsized":"2024-04-25T10:08:59.072346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.080936+0000","last_change":"2024-04-25T10:08:59.084943+0000","last_active":"2024-04-25T10:09:00.080936+0000","last_peered":"2024-04-25T10:09:00.080936+0000","last_clean":"2024-04-25T10:09:00.080936+0000","last_became_active":"2024-04-25T10:08:59.084653+0000","last_became_peered":"2024-04-25T10:08:59.084653+0000","last_unstale":"2024-04-25T10:09:00.080936+0000","last_undegraded":"2024-04-25T10:09:00.080936+0000","last_fullsized":"2024-04-25T10:09:00.080936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:59.810916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.111875+0000","last_change":"2024-04-25T10:09:00.109835+0000","last_active":"2024-04-25T10:09:00.111875+0000","last_peered":"2024-04-25T10:09:00.111875+0000","last_clean":"2024-04-25T10:09:00.111875+0000","last_became_active":"2024-04-25T10:09:00.109484+0000","last_became_peered":"2024-04-25T10:09:00.109484+0000","last_unstale":"2024-04-25T10:09:00.111875+0000","last_undegraded":"2024-04-25T10:09:00.111875+0000","last_fullsized":"2024-04-25T10:09:00.111875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:35.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.101961+0000","last_change":"2024-04-25T10:08:59.097447+0000","last_active":"2024-04-25T10:09:00.101961+0000","last_peered":"2024-04-25T10:09:00.101961+0000","last_clean":"2024-04-25T10:09:00.101961+0000","last_became_active":"2024-04-25T10:08:59.097138+0000","last_became_peered":"2024-04-25T10:08:59.097138+0000","last_unstale":"2024-04-25T10:09:00.101961+0000","last_undegraded":"2024-04-25T10:09:00.101961+0000","last_fullsized":"2024-04-25T10:09:00.101961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.807099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114090+0000","last_change":"2024-04-25T10:09:00.110066+0000","last_active":"2024-04-25T10:09:00.114090+0000","last_peered":"2024-04-25T10:09:00.114090+0000","last_clean":"2024-04-25T10:09:00.114090+0000","last_became_active":"2024-04-25T10:09:00.109800+0000","last_became_peered":"2024-04-25T10:09:00.109800+0000","last_unstale":"2024-04-25T10:09:00.114090+0000","last_undegraded":"2024-04-25T10:09:00.114090+0000","last_fullsized":"2024-04-25T10:09:00.114090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:53.932367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093494+0000","last_change":"2024-04-25T10:08:59.088737+0000","last_active":"2024-04-25T10:09:00.093494+0000","last_peered":"2024-04-25T10:09:00.093494+0000","last_clean":"2024-04-25T10:09:00.093494+0000","last_became_active":"2024-04-25T10:08:59.088445+0000","last_became_peered":"2024-04-25T10:08:59.088445+0000","last_unstale":"2024-04-25T10:09:00.093494+0000","last_undegraded":"2024-04-25T10:09:00.093494+0000","last_fullsized":"2024-04-25T10:09:00.093494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:05.109244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.119310+0000","last_change":"2024-04-25T10:09:00.117302+0000","last_active":"2024-04-25T10:09:00.119310+0000","last_peered":"2024-04-25T10:09:00.119310+0000","last_clean":"2024-04-25T10:09:00.119310+0000","last_became_active":"2024-04-25T10:09:00.117010+0000","last_became_peered":"2024-04-25T10:09:00.117010+0000","last_unstale":"2024-04-25T10:09:00.119310+0000","last_undegraded":"2024-04-25T10:09:00.119310+0000","last_fullsized":"2024-04-25T10:09:00.119310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:20.776889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.100339+0000","last_change":"2024-04-25T10:08:59.085838+0000","last_active":"2024-04-25T10:09:00.100339+0000","last_peered":"2024-04-25T10:09:00.100339+0000","last_clean":"2024-04-25T10:09:00.100339+0000","last_became_active":"2024-04-25T10:08:59.085523+0000","last_became_peered":"2024-04-25T10:08:59.085523+0000","last_unstale":"2024-04-25T10:09:00.100339+0000","last_undegraded":"2024-04-25T10:09:00.100339+0000","last_fullsized":"2024-04-25T10:09:00.100339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:25.880429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.100684+0000","last_change":"2024-04-25T10:09:00.097516+0000","last_active":"2024-04-25T10:09:00.100684+0000","last_peered":"2024-04-25T10:09:00.100684+0000","last_clean":"2024-04-25T10:09:00.100684+0000","last_became_active":"2024-04-25T10:09:00.097152+0000","last_became_peered":"2024-04-25T10:09:00.097152+0000","last_unstale":"2024-04-25T10:09:00.100684+0000","last_undegraded":"2024-04-25T10:09:00.100684+0000","last_fullsized":"2024-04-25T10:09:00.100684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:40.206207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116714+0000","last_change":"2024-04-25T10:08:59.096460+0000","last_active":"2024-04-25T10:09:00.116714+0000","last_peered":"2024-04-25T10:09:00.116714+0000","last_clean":"2024-04-25T10:09:00.116714+0000","last_became_active":"2024-04-25T10:08:59.096134+0000","last_became_peered":"2024-04-25T10:08:59.096134+0000","last_unstale":"2024-04-25T10:09:00.116714+0000","last_undegraded":"2024-04-25T10:09:00.116714+0000","last_fullsized":"2024-04-25T10:09:00.116714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:53.483811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.072313+0000","last_change":"2024-04-25T10:08:59.072313+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.072313+0000","last_undegraded":"2024-04-25T10:08:59.072313+0000","last_fullsized":"2024-04-25T10:08:59.072313+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093092+0000","last_change":"2024-04-25T10:08:59.090145+0000","last_active":"2024-04-25T10:09:00.093092+0000","last_peered":"2024-04-25T10:09:00.093092+0000","last_clean":"2024-04-25T10:09:00.093092+0000","last_became_active":"2024-04-25T10:08:59.089810+0000","last_became_peered":"2024-04-25T10:08:59.089810+0000","last_unstale":"2024-04-25T10:09:00.093092+0000","last_undegraded":"2024-04-25T10:09:00.093092+0000","last_fullsized":"2024-04-25T10:09:00.093092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:14.705434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115603+0000","last_change":"2024-04-25T10:09:00.110294+0000","last_active":"2024-04-25T10:09:00.115603+0000","last_peered":"2024-04-25T10:09:00.115603+0000","last_clean":"2024-04-25T10:09:00.115603+0000","last_became_active":"2024-04-25T10:09:00.107927+0000","last_became_peered":"2024-04-25T10:09:00.107927+0000","last_unstale":"2024-04-25T10:09:00.115603+0000","last_undegraded":"2024-04-25T10:09:00.115603+0000","last_fullsized":"2024-04-25T10:09:00.115603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:46.230653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.080507+0000","last_change":"2024-04-25T10:08:59.094440+0000","last_active":"2024-04-25T10:09:00.080507+0000","last_peered":"2024-04-25T10:09:00.080507+0000","last_clean":"2024-04-25T10:09:00.080507+0000","last_became_active":"2024-04-25T10:08:59.094115+0000","last_became_peered":"2024-04-25T10:08:59.094115+0000","last_unstale":"2024-04-25T10:09:00.080507+0000","last_undegraded":"2024-04-25T10:09:00.080507+0000","last_fullsized":"2024-04-25T10:09:00.080507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:59.041914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.112275+0000","last_change":"2024-04-25T10:09:00.110091+0000","last_active":"2024-04-25T10:09:00.112275+0000","last_peered":"2024-04-25T10:09:00.112275+0000","last_clean":"2024-04-25T10:09:00.112275+0000","last_became_active":"2024-04-25T10:09:00.109803+0000","last_became_peered":"2024-04-25T10:09:00.109803+0000","last_unstale":"2024-04-25T10:09:00.112275+0000","last_undegraded":"2024-04-25T10:09:00.112275+0000","last_fullsized":"2024-04-25T10:09:00.112275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:59.232732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.083824+0000","last_change":"2024-04-25T10:08:59.099038+0000","last_active":"2024-04-25T10:09:00.083824+0000","last_peered":"2024-04-25T10:09:00.083824+0000","last_clean":"2024-04-25T10:09:00.083824+0000","last_became_active":"2024-04-25T10:08:59.097931+0000","last_became_peered":"2024-04-25T10:08:59.097931+0000","last_unstale":"2024-04-25T10:09:00.083824+0000","last_undegraded":"2024-04-25T10:09:00.083824+0000","last_fullsized":"2024-04-25T10:09:00.083824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.981495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092721+0000","last_change":"2024-04-25T10:09:00.087851+0000","last_active":"2024-04-25T10:09:00.092721+0000","last_peered":"2024-04-25T10:09:00.092721+0000","last_clean":"2024-04-25T10:09:00.092721+0000","last_became_active":"2024-04-25T10:09:00.087550+0000","last_became_peered":"2024-04-25T10:09:00.087550+0000","last_unstale":"2024-04-25T10:09:00.092721+0000","last_undegraded":"2024-04-25T10:09:00.092721+0000","last_fullsized":"2024-04-25T10:09:00.092721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:18.740243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093832+0000","last_change":"2024-04-25T10:08:59.094892+0000","last_active":"2024-04-25T10:09:00.093832+0000","last_peered":"2024-04-25T10:09:00.093832+0000","last_clean":"2024-04-25T10:09:00.093832+0000","last_became_active":"2024-04-25T10:08:59.094612+0000","last_became_peered":"2024-04-25T10:08:59.094612+0000","last_unstale":"2024-04-25T10:09:00.093832+0000","last_undegraded":"2024-04-25T10:09:00.093832+0000","last_fullsized":"2024-04-25T10:09:00.093832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:38.150543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115194+0000","last_change":"2024-04-25T10:09:00.104125+0000","last_active":"2024-04-25T10:09:00.115194+0000","last_peered":"2024-04-25T10:09:00.115194+0000","last_clean":"2024-04-25T10:09:00.115194+0000","last_became_active":"2024-04-25T10:09:00.103800+0000","last_became_peered":"2024-04-25T10:09:00.103800+0000","last_unstale":"2024-04-25T10:09:00.115194+0000","last_undegraded":"2024-04-25T10:09:00.115194+0000","last_fullsized":"2024-04-25T10:09:00.115194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:18.670347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"88'3","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.104770+0000","last_change":"2024-04-25T10:08:59.092861+0000","last_active":"2024-04-25T10:09:00.104770+0000","last_peered":"2024-04-25T10:09:00.104770+0000","last_clean":"2024-04-25T10:09:00.104770+0000","last_became_active":"2024-04-25T10:08:59.092126+0000","last_became_peered":"2024-04-25T10:08:59.092126+0000","last_unstale":"2024-04-25T10:09:00.104770+0000","last_undegraded":"2024-04-25T10:09:00.104770+0000","last_fullsized":"2024-04-25T10:09:00.104770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:25.708966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.071777+0000","last_change":"2024-04-25T10:08:59.071777+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.071777+0000","last_undegraded":"2024-04-25T10:08:59.071777+0000","last_fullsized":"2024-04-25T10:08:59.071777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.120496+0000","last_change":"2024-04-25T10:08:59.093933+0000","last_active":"2024-04-25T10:09:00.120496+0000","last_peered":"2024-04-25T10:09:00.120496+0000","last_clean":"2024-04-25T10:09:00.120496+0000","last_became_active":"2024-04-25T10:08:59.092816+0000","last_became_peered":"2024-04-25T10:08:59.092816+0000","last_unstale":"2024-04-25T10:09:00.120496+0000","last_undegraded":"2024-04-25T10:09:00.120496+0000","last_fullsized":"2024-04-25T10:09:00.120496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:04.085717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115395+0000","last_change":"2024-04-25T10:09:00.106166+0000","last_active":"2024-04-25T10:09:00.115395+0000","last_peered":"2024-04-25T10:09:00.115395+0000","last_clean":"2024-04-25T10:09:00.115395+0000","last_became_active":"2024-04-25T10:09:00.105099+0000","last_became_peered":"2024-04-25T10:09:00.105099+0000","last_unstale":"2024-04-25T10:09:00.115395+0000","last_undegraded":"2024-04-25T10:09:00.115395+0000","last_fullsized":"2024-04-25T10:09:00.115395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:38.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115844+0000","last_change":"2024-04-25T10:08:59.095627+0000","last_active":"2024-04-25T10:09:00.115844+0000","last_peered":"2024-04-25T10:09:00.115844+0000","last_clean":"2024-04-25T10:09:00.115844+0000","last_became_active":"2024-04-25T10:08:59.095363+0000","last_became_peered":"2024-04-25T10:08:59.095363+0000","last_unstale":"2024-04-25T10:09:00.115844+0000","last_undegraded":"2024-04-25T10:09:00.115844+0000","last_fullsized":"2024-04-25T10:09:00.115844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:55.919386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093283+0000","last_change":"2024-04-25T10:09:00.088747+0000","last_active":"2024-04-25T10:09:00.093283+0000","last_peered":"2024-04-25T10:09:00.093283+0000","last_clean":"2024-04-25T10:09:00.093283+0000","last_became_active":"2024-04-25T10:09:00.088401+0000","last_became_peered":"2024-04-25T10:09:00.088401+0000","last_unstale":"2024-04-25T10:09:00.093283+0000","last_undegraded":"2024-04-25T10:09:00.093283+0000","last_fullsized":"2024-04-25T10:09:00.093283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:32.071482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.084912+0000","last_change":"2024-04-25T10:08:59.094246+0000","last_active":"2024-04-25T10:09:00.084912+0000","last_peered":"2024-04-25T10:09:00.084912+0000","last_clean":"2024-04-25T10:09:00.084912+0000","last_became_active":"2024-04-25T10:08:59.092895+0000","last_became_peered":"2024-04-25T10:08:59.092895+0000","last_unstale":"2024-04-25T10:09:00.084912+0000","last_undegraded":"2024-04-25T10:09:00.084912+0000","last_fullsized":"2024-04-25T10:09:00.084912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:01.492020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115371+0000","last_change":"2024-04-25T10:09:00.106894+0000","last_active":"2024-04-25T10:09:00.115371+0000","last_peered":"2024-04-25T10:09:00.115371+0000","last_clean":"2024-04-25T10:09:00.115371+0000","last_became_active":"2024-04-25T10:09:00.106578+0000","last_became_peered":"2024-04-25T10:09:00.106578+0000","last_unstale":"2024-04-25T10:09:00.115371+0000","last_undegraded":"2024-04-25T10:09:00.115371+0000","last_fullsized":"2024-04-25T10:09:00.115371+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:13.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092126+0000","last_change":"2024-04-25T10:08:59.094516+0000","last_active":"2024-04-25T10:09:00.092126+0000","last_peered":"2024-04-25T10:09:00.092126+0000","last_clean":"2024-04-25T10:09:00.092126+0000","last_became_active":"2024-04-25T10:08:59.094205+0000","last_became_peered":"2024-04-25T10:08:59.094205+0000","last_unstale":"2024-04-25T10:09:00.092126+0000","last_undegraded":"2024-04-25T10:09:00.092126+0000","last_fullsized":"2024-04-25T10:09:00.092126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:19.761546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116980+0000","last_change":"2024-04-25T10:09:00.103121+0000","last_active":"2024-04-25T10:09:00.116980+0000","last_peered":"2024-04-25T10:09:00.116980+0000","last_clean":"2024-04-25T10:09:00.116980+0000","last_became_active":"2024-04-25T10:09:00.102714+0000","last_became_peered":"2024-04-25T10:09:00.102714+0000","last_unstale":"2024-04-25T10:09:00.116980+0000","last_undegraded":"2024-04-25T10:09:00.116980+0000","last_fullsized":"2024-04-25T10:09:00.116980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:07.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.119410+0000","last_change":"2024-04-25T10:08:59.089451+0000","last_active":"2024-04-25T10:09:00.119410+0000","last_peered":"2024-04-25T10:09:00.119410+0000","last_clean":"2024-04-25T10:09:00.119410+0000","last_became_active":"2024-04-25T10:08:59.089022+0000","last_became_peered":"2024-04-25T10:08:59.089022+0000","last_unstale":"2024-04-25T10:09:00.119410+0000","last_undegraded":"2024-04-25T10:09:00.119410+0000","last_fullsized":"2024-04-25T10:09:00.119410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.547679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.112664+0000","last_change":"2024-04-25T10:09:00.101830+0000","last_active":"2024-04-25T10:09:00.112664+0000","last_peered":"2024-04-25T10:09:00.112664+0000","last_clean":"2024-04-25T10:09:00.112664+0000","last_became_active":"2024-04-25T10:09:00.101499+0000","last_became_peered":"2024-04-25T10:09:00.101499+0000","last_unstale":"2024-04-25T10:09:00.112664+0000","last_undegraded":"2024-04-25T10:09:00.112664+0000","last_fullsized":"2024-04-25T10:09:00.112664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:42.374443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.089711+0000","last_change":"2024-04-25T10:08:59.084911+0000","last_active":"2024-04-25T10:09:00.089711+0000","last_peered":"2024-04-25T10:09:00.089711+0000","last_clean":"2024-04-25T10:09:00.089711+0000","last_became_active":"2024-04-25T10:08:59.084610+0000","last_became_peered":"2024-04-25T10:08:59.084610+0000","last_unstale":"2024-04-25T10:09:00.089711+0000","last_undegraded":"2024-04-25T10:09:00.089711+0000","last_fullsized":"2024-04-25T10:09:00.089711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:41.359101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.084595+0000","last_change":"2024-04-25T10:09:00.083017+0000","last_active":"2024-04-25T10:09:00.084595+0000","last_peered":"2024-04-25T10:09:00.084595+0000","last_clean":"2024-04-25T10:09:00.084595+0000","last_became_active":"2024-04-25T10:09:00.082230+0000","last_became_peered":"2024-04-25T10:09:00.082230+0000","last_unstale":"2024-04-25T10:09:00.084595+0000","last_undegraded":"2024-04-25T10:09:00.084595+0000","last_fullsized":"2024-04-25T10:09:00.084595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:56.873015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.089084+0000","last_change":"2024-04-25T10:08:59.093883+0000","last_active":"2024-04-25T10:09:00.089084+0000","last_peered":"2024-04-25T10:09:00.089084+0000","last_clean":"2024-04-25T10:09:00.089084+0000","last_became_active":"2024-04-25T10:08:59.093505+0000","last_became_peered":"2024-04-25T10:08:59.093505+0000","last_unstale":"2024-04-25T10:09:00.089084+0000","last_undegraded":"2024-04-25T10:09:00.089084+0000","last_fullsized":"2024-04-25T10:09:00.089084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:31.056140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.112406+0000","last_change":"2024-04-25T10:09:00.110465+0000","last_active":"2024-04-25T10:09:00.112406+0000","last_peered":"2024-04-25T10:09:00.112406+0000","last_clean":"2024-04-25T10:09:00.112406+0000","last_became_active":"2024-04-25T10:09:00.109857+0000","last_became_peered":"2024-04-25T10:09:00.109857+0000","last_unstale":"2024-04-25T10:09:00.112406+0000","last_undegraded":"2024-04-25T10:09:00.112406+0000","last_fullsized":"2024-04-25T10:09:00.112406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:46.771136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.090158+0000","last_change":"2024-04-25T10:08:59.093106+0000","last_active":"2024-04-25T10:09:00.090158+0000","last_peered":"2024-04-25T10:09:00.090158+0000","last_clean":"2024-04-25T10:09:00.090158+0000","last_became_active":"2024-04-25T10:08:59.092805+0000","last_became_peered":"2024-04-25T10:08:59.092805+0000","last_unstale":"2024-04-25T10:09:00.090158+0000","last_undegraded":"2024-04-25T10:09:00.090158+0000","last_fullsized":"2024-04-25T10:09:00.090158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.807099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.067505+0000","last_change":"2024-04-25T10:08:59.067505+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.067505+0000","last_undegraded":"2024-04-25T10:08:59.067505+0000","last_fullsized":"2024-04-25T10:08:59.067505+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.104668+0000","last_change":"2024-04-25T10:08:59.093989+0000","last_active":"2024-04-25T10:09:00.104668+0000","last_peered":"2024-04-25T10:09:00.104668+0000","last_clean":"2024-04-25T10:09:00.104668+0000","last_became_active":"2024-04-25T10:08:59.093699+0000","last_became_peered":"2024-04-25T10:08:59.093699+0000","last_unstale":"2024-04-25T10:09:00.104668+0000","last_undegraded":"2024-04-25T10:09:00.104668+0000","last_fullsized":"2024-04-25T10:09:00.104668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.662452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.097103+0000","last_change":"2024-04-25T10:09:00.096022+0000","last_active":"2024-04-25T10:09:00.097103+0000","last_peered":"2024-04-25T10:09:00.097103+0000","last_clean":"2024-04-25T10:09:00.097103+0000","last_became_active":"2024-04-25T10:09:00.095715+0000","last_became_peered":"2024-04-25T10:09:00.095715+0000","last_unstale":"2024-04-25T10:09:00.097103+0000","last_undegraded":"2024-04-25T10:09:00.097103+0000","last_fullsized":"2024-04-25T10:09:00.097103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:23.373405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-25T10:08:59.099046+0000","last_change":"2024-04-25T10:08:59.094052+0000","last_active":"2024-04-25T10:08:59.099046+0000","last_peered":"2024-04-25T10:08:59.099046+0000","last_clean":"2024-04-25T10:08:59.099046+0000","last_became_active":"2024-04-25T10:08:59.093721+0000","last_became_peered":"2024-04-25T10:08:59.093721+0000","last_unstale":"2024-04-25T10:08:59.099046+0000","last_undegraded":"2024-04-25T10:08:59.099046+0000","last_fullsized":"2024-04-25T10:08:59.099046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:57.948367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.099081+0000","last_change":"2024-04-25T10:09:00.097745+0000","last_active":"2024-04-25T10:09:00.099081+0000","last_peered":"2024-04-25T10:09:00.099081+0000","last_clean":"2024-04-25T10:09:00.099081+0000","last_became_active":"2024-04-25T10:09:00.097414+0000","last_became_peered":"2024-04-25T10:09:00.097414+0000","last_unstale":"2024-04-25T10:09:00.099081+0000","last_undegraded":"2024-04-25T10:09:00.099081+0000","last_fullsized":"2024-04-25T10:09:00.099081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:21.577126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.071693+0000","last_change":"2024-04-25T10:08:59.095243+0000","last_active":"2024-04-25T10:09:00.071693+0000","last_peered":"2024-04-25T10:09:00.071693+0000","last_clean":"2024-04-25T10:09:00.071693+0000","last_became_active":"2024-04-25T10:08:59.094898+0000","last_became_peered":"2024-04-25T10:08:59.094898+0000","last_unstale":"2024-04-25T10:09:00.071693+0000","last_undegraded":"2024-04-25T10:09:00.071693+0000","last_fullsized":"2024-04-25T10:09:00.071693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.981495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115582+0000","last_change":"2024-04-25T10:09:00.108733+0000","last_active":"2024-04-25T10:09:00.115582+0000","last_peered":"2024-04-25T10:09:00.115582+0000","last_clean":"2024-04-25T10:09:00.115582+0000","last_became_active":"2024-04-25T10:09:00.107531+0000","last_became_peered":"2024-04-25T10:09:00.107531+0000","last_unstale":"2024-04-25T10:09:00.115582+0000","last_undegraded":"2024-04-25T10:09:00.115582+0000","last_fullsized":"2024-04-25T10:09:00.115582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:40.236525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.106445+0000","last_change":"2024-04-25T10:08:59.087309+0000","last_active":"2024-04-25T10:09:00.106445+0000","last_peered":"2024-04-25T10:09:00.106445+0000","last_clean":"2024-04-25T10:09:00.106445+0000","last_became_active":"2024-04-25T10:08:59.086733+0000","last_became_peered":"2024-04-25T10:08:59.086733+0000","last_unstale":"2024-04-25T10:09:00.106445+0000","last_undegraded":"2024-04-25T10:09:00.106445+0000","last_fullsized":"2024-04-25T10:09:00.106445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.098557+0000","last_change":"2024-04-25T10:09:00.094380+0000","last_active":"2024-04-25T10:09:00.098557+0000","last_peered":"2024-04-25T10:09:00.098557+0000","last_clean":"2024-04-25T10:09:00.098557+0000","last_became_active":"2024-04-25T10:09:00.094098+0000","last_became_peered":"2024-04-25T10:09:00.094098+0000","last_unstale":"2024-04-25T10:09:00.098557+0000","last_undegraded":"2024-04-25T10:09:00.098557+0000","last_fullsized":"2024-04-25T10:09:00.098557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:07.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.082207+0000","last_change":"2024-04-25T10:08:59.081444+0000","last_active":"2024-04-25T10:09:00.082207+0000","last_peered":"2024-04-25T10:09:00.082207+0000","last_clean":"2024-04-25T10:09:00.082207+0000","last_became_active":"2024-04-25T10:08:59.081110+0000","last_became_peered":"2024-04-25T10:08:59.081110+0000","last_unstale":"2024-04-25T10:09:00.082207+0000","last_undegraded":"2024-04-25T10:09:00.082207+0000","last_fullsized":"2024-04-25T10:09:00.082207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:28.169249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.099959+0000","last_change":"2024-04-25T10:09:00.093010+0000","last_active":"2024-04-25T10:09:00.099959+0000","last_peered":"2024-04-25T10:09:00.099959+0000","last_clean":"2024-04-25T10:09:00.099959+0000","last_became_active":"2024-04-25T10:09:00.089467+0000","last_became_peered":"2024-04-25T10:09:00.089467+0000","last_unstale":"2024-04-25T10:09:00.099959+0000","last_undegraded":"2024-04-25T10:09:00.099959+0000","last_fullsized":"2024-04-25T10:09:00.099959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:26.895772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.067553+0000","last_change":"2024-04-25T10:08:59.091283+0000","last_active":"2024-04-25T10:09:00.067553+0000","last_peered":"2024-04-25T10:09:00.067553+0000","last_clean":"2024-04-25T10:09:00.067553+0000","last_became_active":"2024-04-25T10:08:59.090450+0000","last_became_peered":"2024-04-25T10:08:59.090450+0000","last_unstale":"2024-04-25T10:09:00.067553+0000","last_undegraded":"2024-04-25T10:09:00.067553+0000","last_fullsized":"2024-04-25T10:09:00.067553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:00.853913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117042+0000","last_change":"2024-04-25T10:09:00.109416+0000","last_active":"2024-04-25T10:09:00.117042+0000","last_peered":"2024-04-25T10:09:00.117042+0000","last_clean":"2024-04-25T10:09:00.117042+0000","last_became_active":"2024-04-25T10:09:00.109099+0000","last_became_peered":"2024-04-25T10:09:00.109099+0000","last_unstale":"2024-04-25T10:09:00.117042+0000","last_undegraded":"2024-04-25T10:09:00.117042+0000","last_fullsized":"2024-04-25T10:09:00.117042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:32.510485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.079301+0000","last_change":"2024-04-25T10:08:59.086330+0000","last_active":"2024-04-25T10:09:00.079301+0000","last_peered":"2024-04-25T10:09:00.079301+0000","last_clean":"2024-04-25T10:09:00.079301+0000","last_became_active":"2024-04-25T10:08:59.085977+0000","last_became_peered":"2024-04-25T10:08:59.085977+0000","last_unstale":"2024-04-25T10:09:00.079301+0000","last_undegraded":"2024-04-25T10:09:00.079301+0000","last_fullsized":"2024-04-25T10:09:00.079301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:53.483811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.095286+0000","last_change":"2024-04-25T10:09:00.091254+0000","last_active":"2024-04-25T10:09:00.095286+0000","last_peered":"2024-04-25T10:09:00.095286+0000","last_clean":"2024-04-25T10:09:00.095286+0000","last_became_active":"2024-04-25T10:09:00.089920+0000","last_became_peered":"2024-04-25T10:09:00.089920+0000","last_unstale":"2024-04-25T10:09:00.095286+0000","last_undegraded":"2024-04-25T10:09:00.095286+0000","last_fullsized":"2024-04-25T10:09:00.095286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:27.242447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.065287+0000","last_change":"2024-04-25T10:08:59.087483+0000","last_active":"2024-04-25T10:09:00.065287+0000","last_peered":"2024-04-25T10:09:00.065287+0000","last_clean":"2024-04-25T10:09:00.065287+0000","last_became_active":"2024-04-25T10:08:59.086156+0000","last_became_peered":"2024-04-25T10:08:59.086156+0000","last_unstale":"2024-04-25T10:09:00.065287+0000","last_undegraded":"2024-04-25T10:09:00.065287+0000","last_fullsized":"2024-04-25T10:09:00.065287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.439344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.066100+0000","last_change":"2024-04-25T10:08:59.066100+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.066100+0000","last_undegraded":"2024-04-25T10:08:59.066100+0000","last_fullsized":"2024-04-25T10:08:59.066100+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.091763+0000","last_change":"2024-04-25T10:08:59.093509+0000","last_active":"2024-04-25T10:09:00.091763+0000","last_peered":"2024-04-25T10:09:00.091763+0000","last_clean":"2024-04-25T10:09:00.091763+0000","last_became_active":"2024-04-25T10:08:59.092849+0000","last_became_peered":"2024-04-25T10:08:59.092849+0000","last_unstale":"2024-04-25T10:09:00.091763+0000","last_undegraded":"2024-04-25T10:09:00.091763+0000","last_fullsized":"2024-04-25T10:09:00.091763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:31.495143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116671+0000","last_change":"2024-04-25T10:09:00.113592+0000","last_active":"2024-04-25T10:09:00.116671+0000","last_peered":"2024-04-25T10:09:00.116671+0000","last_clean":"2024-04-25T10:09:00.116671+0000","last_became_active":"2024-04-25T10:09:00.113121+0000","last_became_peered":"2024-04-25T10:09:00.113121+0000","last_unstale":"2024-04-25T10:09:00.116671+0000","last_undegraded":"2024-04-25T10:09:00.116671+0000","last_fullsized":"2024-04-25T10:09:00.116671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:58.240994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.072585+0000","last_change":"2024-04-25T10:08:59.096272+0000","last_active":"2024-04-25T10:09:00.072585+0000","last_peered":"2024-04-25T10:09:00.072585+0000","last_clean":"2024-04-25T10:09:00.072585+0000","last_became_active":"2024-04-25T10:08:59.095932+0000","last_became_peered":"2024-04-25T10:08:59.095932+0000","last_unstale":"2024-04-25T10:09:00.072585+0000","last_undegraded":"2024-04-25T10:09:00.072585+0000","last_fullsized":"2024-04-25T10:09:00.072585+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:00.619624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.107474+0000","last_change":"2024-04-25T10:09:00.106808+0000","last_active":"2024-04-25T10:09:00.107474+0000","last_peered":"2024-04-25T10:09:00.107474+0000","last_clean":"2024-04-25T10:09:00.107474+0000","last_became_active":"2024-04-25T10:09:00.106373+0000","last_became_peered":"2024-04-25T10:09:00.106373+0000","last_unstale":"2024-04-25T10:09:00.107474+0000","last_undegraded":"2024-04-25T10:09:00.107474+0000","last_fullsized":"2024-04-25T10:09:00.107474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:32.510485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.073065+0000","last_change":"2024-04-25T10:08:59.094065+0000","last_active":"2024-04-25T10:09:00.073065+0000","last_peered":"2024-04-25T10:09:00.073065+0000","last_clean":"2024-04-25T10:09:00.073065+0000","last_became_active":"2024-04-25T10:08:59.093019+0000","last_became_peered":"2024-04-25T10:08:59.093019+0000","last_unstale":"2024-04-25T10:09:00.073065+0000","last_undegraded":"2024-04-25T10:09:00.073065+0000","last_fullsized":"2024-04-25T10:09:00.073065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:06.887928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-04-25T10:08:59.067425+0000","last_change":"2024-04-25T10:08:59.067425+0000","last_active":"2024-04-25T10:08:59.041251+0000","last_peered":"2024-04-25T10:08:59.041251+0000","last_clean":"2024-04-25T10:08:59.041251+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:59.067425+0000","last_undegraded":"2024-04-25T10:08:59.067425+0000","last_fullsized":"2024-04-25T10:08:59.067425+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.080703+0000","last_change":"2024-04-25T10:08:59.082272+0000","last_active":"2024-04-25T10:09:00.080703+0000","last_peered":"2024-04-25T10:09:00.080703+0000","last_clean":"2024-04-25T10:09:00.080703+0000","last_became_active":"2024-04-25T10:08:59.082003+0000","last_became_peered":"2024-04-25T10:08:59.082003+0000","last_unstale":"2024-04-25T10:09:00.080703+0000","last_undegraded":"2024-04-25T10:09:00.080703+0000","last_fullsized":"2024-04-25T10:09:00.080703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:16.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.110594+0000","last_change":"2024-04-25T10:09:00.109500+0000","last_active":"2024-04-25T10:09:00.110594+0000","last_peered":"2024-04-25T10:09:00.110594+0000","last_clean":"2024-04-25T10:09:00.110594+0000","last_became_active":"2024-04-25T10:09:00.109224+0000","last_became_peered":"2024-04-25T10:09:00.109224+0000","last_unstale":"2024-04-25T10:09:00.110594+0000","last_undegraded":"2024-04-25T10:09:00.110594+0000","last_fullsized":"2024-04-25T10:09:00.110594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:01.668226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093932+0000","last_change":"2024-04-25T10:08:59.093748+0000","last_active":"2024-04-25T10:09:00.093932+0000","last_peered":"2024-04-25T10:09:00.093932+0000","last_clean":"2024-04-25T10:09:00.093932+0000","last_became_active":"2024-04-25T10:08:59.093009+0000","last_became_peered":"2024-04-25T10:08:59.093009+0000","last_unstale":"2024-04-25T10:09:00.093932+0000","last_undegraded":"2024-04-25T10:09:00.093932+0000","last_fullsized":"2024-04-25T10:09:00.093932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:40.357143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":295,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.083197+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:09:00.083197+0000","last_peered":"2024-04-25T10:09:00.083197+0000","last_clean":"2024-04-25T10:09:00.083197+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:09:00.083197+0000","last_undegraded":"2024-04-25T10:09:00.083197+0000","last_fullsized":"2024-04-25T10:09:00.083197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":491301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":493037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1081669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":493025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":491905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509529,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":1082355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1081344,"data_stored":491371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1064960,"data_stored":484134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:04.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:09:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.796+0000 7f974fe00640 1 Processor -- start 2024-04-25T10:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.797+0000 7f974fe00640 1 -- start start 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9750161200 0x7f97500b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97500d2380 0x7f97500d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f97500d0600 0x7f97500ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97500d27c0 con 0x7f9750161200 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974fe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97500b28b0 con 0x7f97500d0600 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97500cf410 con 0x7f97500d2380 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9750161200 0x7f97500b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974ea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9750161200 0x7f97500b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36116/0 (socket says 172.21.15.5:36116) 2024-04-25T10:09:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.798+0000 7f974ea00640 1 -- 172.21.15.5:0/1767936619 learned_addr learned my addr 172.21.15.5:0/1767936619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974e000640 1 --2- 172.21.15.5:0/1767936619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97500d2380 0x7f97500d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974f400640 1 --2- 172.21.15.5:0/1767936619 >> v2:172.21.15.80:3300/0 conn(0x7f97500d0600 0x7f97500ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974ea00640 1 -- 172.21.15.5:0/1767936619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97500d2380 msgr2=0x7f97500d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974ea00640 1 --2- 172.21.15.5:0/1767936619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97500d2380 0x7f97500d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974ea00640 1 -- 172.21.15.5:0/1767936619 >> v2:172.21.15.80:3300/0 conn(0x7f97500d0600 msgr2=0x7f97500ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974ea00640 1 --2- 172.21.15.5:0/1767936619 >> v2:172.21.15.80:3300/0 conn(0x7f97500d0600 0x7f97500ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974ea00640 1 -- 172.21.15.5:0/1767936619 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97500cf6f0 con 0x7f9750161200 2024-04-25T10:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.799+0000 7f974f400640 1 --2- 172.21.15.5:0/1767936619 >> v2:172.21.15.80:3300/0 conn(0x7f97500d0600 0x7f97500ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.800+0000 7f974ea00640 1 --2- 172.21.15.5:0/1767936619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9750161200 0x7f97500b4bb0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f97500b4a80 tx=0x7f974002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6af54dc071de42c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.800+0000 7f974d600640 1 -- 172.21.15.5:0/1767936619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740037070 con 0x7f9750161200 2024-04-25T10:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.800+0000 7f974d600640 1 -- 172.21.15.5:0/1767936619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740032440 con 0x7f9750161200 2024-04-25T10:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.800+0000 7f974d600640 1 -- 172.21.15.5:0/1767936619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740032810 con 0x7f9750161200 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 -- 172.21.15.5:0/1767936619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9750161200 msgr2=0x7f97500b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 --2- 172.21.15.5:0/1767936619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9750161200 0x7f97500b4bb0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f97500b4a80 tx=0x7f974002a210 comp rx=0 tx=0).stop 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 -- 172.21.15.5:0/1767936619 shutdown_connections 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 --2- 172.21.15.5:0/1767936619 >> v2:172.21.15.80:3300/0 conn(0x7f97500d0600 0x7f97500ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 --2- 172.21.15.5:0/1767936619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97500d2380 0x7f97500d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 --2- 172.21.15.5:0/1767936619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9750161200 0x7f97500b4bb0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 -- 172.21.15.5:0/1767936619 >> 172.21.15.5:0/1767936619 conn(0x7f975008af80 msgr2=0x7f97500d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 -- 172.21.15.5:0/1767936619 shutdown_connections 2024-04-25T10:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.801+0000 7f974fe00640 1 -- 172.21.15.5:0/1767936619 wait complete. 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.803+0000 7f974fe00640 1 Processor -- start 2024-04-25T10:09:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.803+0000 7f974fe00640 1 -- start start 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97500d0600 0x7f975009f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 0x7f975009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9750161200 0x7f97500cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500ce7c0 con 0x7f97500d2380 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97500d42b0 con 0x7f97500d0600 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97500ce940 con 0x7f9750161200 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.804+0000 7f974e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 0x7f975009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 0x7f975009d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36124/0 (socket says 172.21.15.5:36124) 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 -- 172.21.15.5:0/758226518 learned_addr learned my addr 172.21.15.5:0/758226518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974ea00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97500d0600 0x7f975009f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974f400640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9750161200 0x7f97500cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 -- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9750161200 msgr2=0x7f97500cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9750161200 0x7f97500cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 -- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97500d0600 msgr2=0x7f975009f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97500d0600 0x7f975009f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 -- 172.21.15.5:0/758226518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975009f940 con 0x7f97500d2380 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974f400640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9750161200 0x7f97500cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.805+0000 7f974e000640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 0x7f975009d770 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f973800d830 tx=0x7f973800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.806+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973800b6c0 con 0x7f97500d2380 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.806+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738019070 con 0x7f97500d2380 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.806+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973800ba80 con 0x7f97500d2380 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.806+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975009fc20 con 0x7f97500d2380 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.806+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97500a01e0 con 0x7f97500d2380 2024-04-25T10:09:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.807+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9750068f60 con 0x7f97500d2380 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.808+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f973801e460 con 0x7f97500d2380 2024-04-25T10:09:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.809+0000 7f9746a00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972c06b4d0 0x7f972c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.809+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f973808aae0 con 0x7f97500d2380 2024-04-25T10:09:04.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.810+0000 7f974ea00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972c06b4d0 0x7f972c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.811+0000 7f974ea00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972c06b4d0 0x7f972c06d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9740034000 tx=0x7f974002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:04.813+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9738059730 con 0x7f97500d2380 2024-04-25T10:09:05.005 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-04-25T10:09:04.998+0000 7f4942600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:05.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.009+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f975015d5f0 con 0x7f972c06b4d0 2024-04-25T10:09:05.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.028+0000 7f9746a00640 1 -- 172.21.15.5:0/758226518 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252018 (secure 0 0 0) 0x7f97400326f0 con 0x7f972c06b4d0 2024-04-25T10:09:05.038 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:05.039 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972c06b4d0 msgr2=0x7f972c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972c06b4d0 0x7f972c06d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9740034000 tx=0x7f974002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 msgr2=0x7f975009d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 0x7f975009d770 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f973800d830 tx=0x7f973800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 shutdown_connections 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9750161200 0x7f97500cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97500d2380 0x7f975009d770 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972c06b4d0 0x7f972c06d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 --2- 172.21.15.5:0/758226518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97500d0600 0x7f975009f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 >> 172.21.15.5:0/758226518 conn(0x7f975008af80 msgr2=0x7f97500d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 shutdown_connections 2024-04-25T10:09:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.034+0000 7f974fe00640 1 -- 172.21.15.5:0/758226518 wait complete. 2024-04-25T10:09:05.060 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-04-25T10:09:04.770832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259116,"kb_used_data":9652,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715604,"statfs":{"total":773094113280,"available":772828778496,"internally_reserved":0,"allocated":9883648,"data_stored":5122656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002137"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115301+0000","last_change":"2024-04-25T10:09:00.107093+0000","last_active":"2024-04-25T10:09:00.115301+0000","last_peered":"2024-04-25T10:09:00.115301+0000","last_clean":"2024-04-25T10:09:00.115301+0000","last_became_active":"2024-04-25T10:09:00.106825+0000","last_became_peered":"2024-04-25T10:09:00.106825+0000","last_unstale":"2024-04-25T10:09:00.115301+0000","last_undegraded":"2024-04-25T10:09:00.115301+0000","last_fullsized":"2024-04-25T10:09:00.115301+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:19.534934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.101384+0000","last_change":"2024-04-25T10:09:00.098271+0000","last_active":"2024-04-25T10:09:00.101384+0000","last_peered":"2024-04-25T10:09:00.101384+0000","last_clean":"2024-04-25T10:09:00.101384+0000","last_became_active":"2024-04-25T10:09:00.097364+0000","last_became_peered":"2024-04-25T10:09:00.097364+0000","last_unstale":"2024-04-25T10:09:00.101384+0000","last_undegraded":"2024-04-25T10:09:00.101384+0000","last_fullsized":"2024-04-25T10:09:00.101384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:18.272172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114060+0000","last_change":"2024-04-25T10:09:00.110468+0000","last_active":"2024-04-25T10:09:00.114060+0000","last_peered":"2024-04-25T10:09:00.114060+0000","last_clean":"2024-04-25T10:09:00.114060+0000","last_became_active":"2024-04-25T10:09:00.105042+0000","last_became_peered":"2024-04-25T10:09:00.105042+0000","last_unstale":"2024-04-25T10:09:00.114060+0000","last_undegraded":"2024-04-25T10:09:00.114060+0000","last_fullsized":"2024-04-25T10:09:00.114060+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:37.168689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114893+0000","last_change":"2024-04-25T10:09:00.101968+0000","last_active":"2024-04-25T10:09:00.114893+0000","last_peered":"2024-04-25T10:09:00.114893+0000","last_clean":"2024-04-25T10:09:00.114893+0000","last_became_active":"2024-04-25T10:09:00.096794+0000","last_became_peered":"2024-04-25T10:09:00.096794+0000","last_unstale":"2024-04-25T10:09:00.114893+0000","last_undegraded":"2024-04-25T10:09:00.114893+0000","last_fullsized":"2024-04-25T10:09:00.114893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:39.761726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092230+0000","last_change":"2024-04-25T10:09:00.089257+0000","last_active":"2024-04-25T10:09:00.092230+0000","last_peered":"2024-04-25T10:09:00.092230+0000","last_clean":"2024-04-25T10:09:00.092230+0000","last_became_active":"2024-04-25T10:09:00.088429+0000","last_became_peered":"2024-04-25T10:09:00.088429+0000","last_unstale":"2024-04-25T10:09:00.092230+0000","last_undegraded":"2024-04-25T10:09:00.092230+0000","last_fullsized":"2024-04-25T10:09:00.092230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:23.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.094988+0000","last_change":"2024-04-25T10:09:00.105255+0000","last_active":"2024-04-25T10:09:01.094988+0000","last_peered":"2024-04-25T10:09:01.094988+0000","last_clean":"2024-04-25T10:09:01.094988+0000","last_became_active":"2024-04-25T10:09:00.104173+0000","last_became_peered":"2024-04-25T10:09:00.104173+0000","last_unstale":"2024-04-25T10:09:01.094988+0000","last_undegraded":"2024-04-25T10:09:01.094988+0000","last_fullsized":"2024-04-25T10:09:01.094988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:25.581333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103772+0000","last_change":"2024-04-25T10:09:00.102056+0000","last_active":"2024-04-25T10:09:00.103772+0000","last_peered":"2024-04-25T10:09:00.103772+0000","last_clean":"2024-04-25T10:09:00.103772+0000","last_became_active":"2024-04-25T10:09:00.101790+0000","last_became_peered":"2024-04-25T10:09:00.101790+0000","last_unstale":"2024-04-25T10:09:00.103772+0000","last_undegraded":"2024-04-25T10:09:00.103772+0000","last_fullsized":"2024-04-25T10:09:00.103772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:05.614089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.093514+0000","last_change":"2024-04-25T10:09:00.088549+0000","last_active":"2024-04-25T10:09:01.093514+0000","last_peered":"2024-04-25T10:09:01.093514+0000","last_clean":"2024-04-25T10:09:01.093514+0000","last_became_active":"2024-04-25T10:09:00.086104+0000","last_became_peered":"2024-04-25T10:09:00.086104+0000","last_unstale":"2024-04-25T10:09:01.093514+0000","last_undegraded":"2024-04-25T10:09:01.093514+0000","last_fullsized":"2024-04-25T10:09:01.093514+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:27.731814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.094324+0000","last_change":"2024-04-25T10:09:00.092182+0000","last_active":"2024-04-25T10:09:00.094324+0000","last_peered":"2024-04-25T10:09:00.094324+0000","last_clean":"2024-04-25T10:09:00.094324+0000","last_became_active":"2024-04-25T10:09:00.090969+0000","last_became_peered":"2024-04-25T10:09:00.090969+0000","last_unstale":"2024-04-25T10:09:00.094324+0000","last_undegraded":"2024-04-25T10:09:00.094324+0000","last_fullsized":"2024-04-25T10:09:00.094324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:57.039093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117947+0000","last_change":"2024-04-25T10:09:00.115320+0000","last_active":"2024-04-25T10:09:00.117947+0000","last_peered":"2024-04-25T10:09:00.117947+0000","last_clean":"2024-04-25T10:09:00.117947+0000","last_became_active":"2024-04-25T10:09:00.112216+0000","last_became_peered":"2024-04-25T10:09:00.112216+0000","last_unstale":"2024-04-25T10:09:00.117947+0000","last_undegraded":"2024-04-25T10:09:00.117947+0000","last_fullsized":"2024-04-25T10:09:00.117947+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:01.634967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103055+0000","last_change":"2024-04-25T10:09:00.099685+0000","last_active":"2024-04-25T10:09:00.103055+0000","last_peered":"2024-04-25T10:09:00.103055+0000","last_clean":"2024-04-25T10:09:00.103055+0000","last_became_active":"2024-04-25T10:09:00.099391+0000","last_became_peered":"2024-04-25T10:09:00.099391+0000","last_unstale":"2024-04-25T10:09:00.103055+0000","last_undegraded":"2024-04-25T10:09:00.103055+0000","last_fullsized":"2024-04-25T10:09:00.103055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:14.198166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114919+0000","last_change":"2024-04-25T10:09:00.109297+0000","last_active":"2024-04-25T10:09:00.114919+0000","last_peered":"2024-04-25T10:09:00.114919+0000","last_clean":"2024-04-25T10:09:00.114919+0000","last_became_active":"2024-04-25T10:09:00.108965+0000","last_became_peered":"2024-04-25T10:09:00.108965+0000","last_unstale":"2024-04-25T10:09:00.114919+0000","last_undegraded":"2024-04-25T10:09:00.114919+0000","last_fullsized":"2024-04-25T10:09:00.114919+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:26.119223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117003+0000","last_change":"2024-04-25T10:09:00.107706+0000","last_active":"2024-04-25T10:09:00.117003+0000","last_peered":"2024-04-25T10:09:00.117003+0000","last_clean":"2024-04-25T10:09:00.117003+0000","last_became_active":"2024-04-25T10:09:00.107433+0000","last_became_peered":"2024-04-25T10:09:00.107433+0000","last_unstale":"2024-04-25T10:09:00.117003+0000","last_undegraded":"2024-04-25T10:09:00.117003+0000","last_fullsized":"2024-04-25T10:09:00.117003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:02.507363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.101043+0000","last_change":"2024-04-25T10:09:00.098596+0000","last_active":"2024-04-25T10:09:00.101043+0000","last_peered":"2024-04-25T10:09:00.101043+0000","last_clean":"2024-04-25T10:09:00.101043+0000","last_became_active":"2024-04-25T10:09:00.098267+0000","last_became_peered":"2024-04-25T10:09:00.098267+0000","last_unstale":"2024-04-25T10:09:00.101043+0000","last_undegraded":"2024-04-25T10:09:00.101043+0000","last_fullsized":"2024-04-25T10:09:00.101043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:23.671158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.104235+0000","last_change":"2024-04-25T10:09:00.095973+0000","last_active":"2024-04-25T10:09:00.104235+0000","last_peered":"2024-04-25T10:09:00.104235+0000","last_clean":"2024-04-25T10:09:00.104235+0000","last_became_active":"2024-04-25T10:09:00.095690+0000","last_became_peered":"2024-04-25T10:09:00.095690+0000","last_unstale":"2024-04-25T10:09:00.104235+0000","last_undegraded":"2024-04-25T10:09:00.104235+0000","last_fullsized":"2024-04-25T10:09:00.104235+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:12.225717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.093908+0000","last_change":"2024-04-25T10:09:00.103582+0000","last_active":"2024-04-25T10:09:01.093908+0000","last_peered":"2024-04-25T10:09:01.093908+0000","last_clean":"2024-04-25T10:09:01.093908+0000","last_became_active":"2024-04-25T10:09:00.102329+0000","last_became_peered":"2024-04-25T10:09:00.102329+0000","last_unstale":"2024-04-25T10:09:01.093908+0000","last_undegraded":"2024-04-25T10:09:01.093908+0000","last_fullsized":"2024-04-25T10:09:01.093908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:43.585207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.097202+0000","last_change":"2024-04-25T10:09:00.102729+0000","last_active":"2024-04-25T10:09:01.097202+0000","last_peered":"2024-04-25T10:09:01.097202+0000","last_clean":"2024-04-25T10:09:01.097202+0000","last_became_active":"2024-04-25T10:09:00.102009+0000","last_became_peered":"2024-04-25T10:09:00.102009+0000","last_unstale":"2024-04-25T10:09:01.097202+0000","last_undegraded":"2024-04-25T10:09:01.097202+0000","last_fullsized":"2024-04-25T10:09:01.097202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:10.003117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114007+0000","last_change":"2024-04-25T10:09:00.106048+0000","last_active":"2024-04-25T10:09:00.114007+0000","last_peered":"2024-04-25T10:09:00.114007+0000","last_clean":"2024-04-25T10:09:00.114007+0000","last_became_active":"2024-04-25T10:09:00.105793+0000","last_became_peered":"2024-04-25T10:09:00.105793+0000","last_unstale":"2024-04-25T10:09:00.114007+0000","last_undegraded":"2024-04-25T10:09:00.114007+0000","last_fullsized":"2024-04-25T10:09:00.114007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:11.049854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.102990+0000","last_change":"2024-04-25T10:09:00.099526+0000","last_active":"2024-04-25T10:09:00.102990+0000","last_peered":"2024-04-25T10:09:00.102990+0000","last_clean":"2024-04-25T10:09:00.102990+0000","last_became_active":"2024-04-25T10:09:00.099234+0000","last_became_peered":"2024-04-25T10:09:00.099234+0000","last_unstale":"2024-04-25T10:09:00.102990+0000","last_undegraded":"2024-04-25T10:09:00.102990+0000","last_fullsized":"2024-04-25T10:09:00.102990+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:02.507363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.097844+0000","last_change":"2024-04-25T10:09:00.108488+0000","last_active":"2024-04-25T10:09:01.097844+0000","last_peered":"2024-04-25T10:09:01.097844+0000","last_clean":"2024-04-25T10:09:01.097844+0000","last_became_active":"2024-04-25T10:09:00.108081+0000","last_became_peered":"2024-04-25T10:09:00.108081+0000","last_unstale":"2024-04-25T10:09:01.097844+0000","last_undegraded":"2024-04-25T10:09:01.097844+0000","last_fullsized":"2024-04-25T10:09:01.097844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:53.448169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092209+0000","last_change":"2024-04-25T10:09:00.088779+0000","last_active":"2024-04-25T10:09:00.092209+0000","last_peered":"2024-04-25T10:09:00.092209+0000","last_clean":"2024-04-25T10:09:00.092209+0000","last_became_active":"2024-04-25T10:09:00.087994+0000","last_became_peered":"2024-04-25T10:09:00.087994+0000","last_unstale":"2024-04-25T10:09:00.092209+0000","last_undegraded":"2024-04-25T10:09:00.092209+0000","last_fullsized":"2024-04-25T10:09:00.092209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:23.373405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.098850+0000","last_change":"2024-04-25T10:09:00.094739+0000","last_active":"2024-04-25T10:09:00.098850+0000","last_peered":"2024-04-25T10:09:00.098850+0000","last_clean":"2024-04-25T10:09:00.098850+0000","last_became_active":"2024-04-25T10:09:00.093298+0000","last_became_peered":"2024-04-25T10:09:00.093298+0000","last_unstale":"2024-04-25T10:09:00.098850+0000","last_undegraded":"2024-04-25T10:09:00.098850+0000","last_fullsized":"2024-04-25T10:09:00.098850+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:54.383047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103551+0000","last_change":"2024-04-25T10:09:00.102614+0000","last_active":"2024-04-25T10:09:00.103551+0000","last_peered":"2024-04-25T10:09:00.103551+0000","last_clean":"2024-04-25T10:09:00.103551+0000","last_became_active":"2024-04-25T10:09:00.102006+0000","last_became_peered":"2024-04-25T10:09:00.102006+0000","last_unstale":"2024-04-25T10:09:00.103551+0000","last_undegraded":"2024-04-25T10:09:00.103551+0000","last_fullsized":"2024-04-25T10:09:00.103551+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:25.068236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115372+0000","last_change":"2024-04-25T10:09:00.105944+0000","last_active":"2024-04-25T10:09:00.115372+0000","last_peered":"2024-04-25T10:09:00.115372+0000","last_clean":"2024-04-25T10:09:00.115372+0000","last_became_active":"2024-04-25T10:09:00.104706+0000","last_became_peered":"2024-04-25T10:09:00.104706+0000","last_unstale":"2024-04-25T10:09:00.115372+0000","last_undegraded":"2024-04-25T10:09:00.115372+0000","last_fullsized":"2024-04-25T10:09:00.115372+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:59.579367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.100991+0000","last_change":"2024-04-25T10:09:00.093829+0000","last_active":"2024-04-25T10:09:00.100991+0000","last_peered":"2024-04-25T10:09:00.100991+0000","last_clean":"2024-04-25T10:09:00.100991+0000","last_became_active":"2024-04-25T10:09:00.089291+0000","last_became_peered":"2024-04-25T10:09:00.089291+0000","last_unstale":"2024-04-25T10:09:00.100991+0000","last_undegraded":"2024-04-25T10:09:00.100991+0000","last_fullsized":"2024-04-25T10:09:00.100991+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:30.601882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103273+0000","last_change":"2024-04-25T10:09:00.101540+0000","last_active":"2024-04-25T10:09:00.103273+0000","last_peered":"2024-04-25T10:09:00.103273+0000","last_clean":"2024-04-25T10:09:00.103273+0000","last_became_active":"2024-04-25T10:09:00.101169+0000","last_became_peered":"2024-04-25T10:09:00.101169+0000","last_unstale":"2024-04-25T10:09:00.103273+0000","last_undegraded":"2024-04-25T10:09:00.103273+0000","last_fullsized":"2024-04-25T10:09:00.103273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:46.287236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.103592+0000","last_change":"2024-04-25T10:09:00.098104+0000","last_active":"2024-04-25T10:09:00.103592+0000","last_peered":"2024-04-25T10:09:00.103592+0000","last_clean":"2024-04-25T10:09:00.103592+0000","last_became_active":"2024-04-25T10:09:00.097730+0000","last_became_peered":"2024-04-25T10:09:00.097730+0000","last_unstale":"2024-04-25T10:09:00.103592+0000","last_undegraded":"2024-04-25T10:09:00.103592+0000","last_fullsized":"2024-04-25T10:09:00.103592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:37.628186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117174+0000","last_change":"2024-04-25T10:09:00.107218+0000","last_active":"2024-04-25T10:09:00.117174+0000","last_peered":"2024-04-25T10:09:00.117174+0000","last_clean":"2024-04-25T10:09:00.117174+0000","last_became_active":"2024-04-25T10:09:00.106742+0000","last_became_peered":"2024-04-25T10:09:00.106742+0000","last_unstale":"2024-04-25T10:09:00.117174+0000","last_undegraded":"2024-04-25T10:09:00.117174+0000","last_fullsized":"2024-04-25T10:09:00.117174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:21.577126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116977+0000","last_change":"2024-04-25T10:09:00.116341+0000","last_active":"2024-04-25T10:09:00.116977+0000","last_peered":"2024-04-25T10:09:00.116977+0000","last_clean":"2024-04-25T10:09:00.116977+0000","last_became_active":"2024-04-25T10:09:00.115996+0000","last_became_peered":"2024-04-25T10:09:00.115996+0000","last_unstale":"2024-04-25T10:09:00.116977+0000","last_undegraded":"2024-04-25T10:09:00.116977+0000","last_fullsized":"2024-04-25T10:09:00.116977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:29.189977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092051+0000","last_change":"2024-04-25T10:09:00.088170+0000","last_active":"2024-04-25T10:09:00.092051+0000","last_peered":"2024-04-25T10:09:00.092051+0000","last_clean":"2024-04-25T10:09:00.092051+0000","last_became_active":"2024-04-25T10:09:00.087877+0000","last_became_peered":"2024-04-25T10:09:00.087877+0000","last_unstale":"2024-04-25T10:09:00.092051+0000","last_undegraded":"2024-04-25T10:09:00.092051+0000","last_fullsized":"2024-04-25T10:09:00.092051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:00.826258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.113806+0000","last_change":"2024-04-25T10:09:00.112242+0000","last_active":"2024-04-25T10:09:00.113806+0000","last_peered":"2024-04-25T10:09:00.113806+0000","last_clean":"2024-04-25T10:09:00.113806+0000","last_became_active":"2024-04-25T10:09:00.111951+0000","last_became_peered":"2024-04-25T10:09:00.111951+0000","last_unstale":"2024-04-25T10:09:00.113806+0000","last_undegraded":"2024-04-25T10:09:00.113806+0000","last_fullsized":"2024-04-25T10:09:00.113806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:20.776889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.094286+0000","last_change":"2024-04-25T10:09:00.106485+0000","last_active":"2024-04-25T10:09:01.094286+0000","last_peered":"2024-04-25T10:09:01.094286+0000","last_clean":"2024-04-25T10:09:01.094286+0000","last_became_active":"2024-04-25T10:09:00.101773+0000","last_became_peered":"2024-04-25T10:09:00.101773+0000","last_unstale":"2024-04-25T10:09:01.094286+0000","last_undegraded":"2024-04-25T10:09:01.094286+0000","last_fullsized":"2024-04-25T10:09:01.094286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:22.226379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115077+0000","last_change":"2024-04-25T10:09:00.104938+0000","last_active":"2024-04-25T10:09:00.115077+0000","last_peered":"2024-04-25T10:09:00.115077+0000","last_clean":"2024-04-25T10:09:00.115077+0000","last_became_active":"2024-04-25T10:09:00.104620+0000","last_became_peered":"2024-04-25T10:09:00.104620+0000","last_unstale":"2024-04-25T10:09:00.115077+0000","last_undegraded":"2024-04-25T10:09:00.115077+0000","last_fullsized":"2024-04-25T10:09:00.115077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:06.117065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.095377+0000","last_change":"2024-04-25T10:08:59.097251+0000","last_active":"2024-04-25T10:09:00.095377+0000","last_peered":"2024-04-25T10:09:00.095377+0000","last_clean":"2024-04-25T10:09:00.095377+0000","last_became_active":"2024-04-25T10:08:59.096348+0000","last_became_peered":"2024-04-25T10:08:59.096348+0000","last_unstale":"2024-04-25T10:09:00.095377+0000","last_undegraded":"2024-04-25T10:09:00.095377+0000","last_fullsized":"2024-04-25T10:09:00.095377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:05.109244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.095922+0000","last_change":"2024-04-25T10:09:00.106049+0000","last_active":"2024-04-25T10:09:01.095922+0000","last_peered":"2024-04-25T10:09:01.095922+0000","last_clean":"2024-04-25T10:09:01.095922+0000","last_became_active":"2024-04-25T10:09:00.105621+0000","last_became_peered":"2024-04-25T10:09:00.105621+0000","last_unstale":"2024-04-25T10:09:01.095922+0000","last_undegraded":"2024-04-25T10:09:01.095922+0000","last_fullsized":"2024-04-25T10:09:01.095922+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:42.012970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.080936+0000","last_change":"2024-04-25T10:08:59.084943+0000","last_active":"2024-04-25T10:09:00.080936+0000","last_peered":"2024-04-25T10:09:00.080936+0000","last_clean":"2024-04-25T10:09:00.080936+0000","last_became_active":"2024-04-25T10:08:59.084653+0000","last_became_peered":"2024-04-25T10:08:59.084653+0000","last_unstale":"2024-04-25T10:09:00.080936+0000","last_undegraded":"2024-04-25T10:09:00.080936+0000","last_fullsized":"2024-04-25T10:09:00.080936+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:59.810916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.111875+0000","last_change":"2024-04-25T10:09:00.109835+0000","last_active":"2024-04-25T10:09:00.111875+0000","last_peered":"2024-04-25T10:09:00.111875+0000","last_clean":"2024-04-25T10:09:00.111875+0000","last_became_active":"2024-04-25T10:09:00.109484+0000","last_became_peered":"2024-04-25T10:09:00.109484+0000","last_unstale":"2024-04-25T10:09:00.111875+0000","last_undegraded":"2024-04-25T10:09:00.111875+0000","last_fullsized":"2024-04-25T10:09:00.111875+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:35.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.101961+0000","last_change":"2024-04-25T10:08:59.097447+0000","last_active":"2024-04-25T10:09:00.101961+0000","last_peered":"2024-04-25T10:09:00.101961+0000","last_clean":"2024-04-25T10:09:00.101961+0000","last_became_active":"2024-04-25T10:08:59.097138+0000","last_became_peered":"2024-04-25T10:08:59.097138+0000","last_unstale":"2024-04-25T10:09:00.101961+0000","last_undegraded":"2024-04-25T10:09:00.101961+0000","last_fullsized":"2024-04-25T10:09:00.101961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.807099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.114090+0000","last_change":"2024-04-25T10:09:00.110066+0000","last_active":"2024-04-25T10:09:00.114090+0000","last_peered":"2024-04-25T10:09:00.114090+0000","last_clean":"2024-04-25T10:09:00.114090+0000","last_became_active":"2024-04-25T10:09:00.109800+0000","last_became_peered":"2024-04-25T10:09:00.109800+0000","last_unstale":"2024-04-25T10:09:00.114090+0000","last_undegraded":"2024-04-25T10:09:00.114090+0000","last_fullsized":"2024-04-25T10:09:00.114090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:53.932367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093494+0000","last_change":"2024-04-25T10:08:59.088737+0000","last_active":"2024-04-25T10:09:00.093494+0000","last_peered":"2024-04-25T10:09:00.093494+0000","last_clean":"2024-04-25T10:09:00.093494+0000","last_became_active":"2024-04-25T10:08:59.088445+0000","last_became_peered":"2024-04-25T10:08:59.088445+0000","last_unstale":"2024-04-25T10:09:00.093494+0000","last_undegraded":"2024-04-25T10:09:00.093494+0000","last_fullsized":"2024-04-25T10:09:00.093494+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:05.109244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.119310+0000","last_change":"2024-04-25T10:09:00.117302+0000","last_active":"2024-04-25T10:09:00.119310+0000","last_peered":"2024-04-25T10:09:00.119310+0000","last_clean":"2024-04-25T10:09:00.119310+0000","last_became_active":"2024-04-25T10:09:00.117010+0000","last_became_peered":"2024-04-25T10:09:00.117010+0000","last_unstale":"2024-04-25T10:09:00.119310+0000","last_undegraded":"2024-04-25T10:09:00.119310+0000","last_fullsized":"2024-04-25T10:09:00.119310+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:20.776889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.100339+0000","last_change":"2024-04-25T10:08:59.085838+0000","last_active":"2024-04-25T10:09:00.100339+0000","last_peered":"2024-04-25T10:09:00.100339+0000","last_clean":"2024-04-25T10:09:00.100339+0000","last_became_active":"2024-04-25T10:08:59.085523+0000","last_became_peered":"2024-04-25T10:08:59.085523+0000","last_unstale":"2024-04-25T10:09:00.100339+0000","last_undegraded":"2024-04-25T10:09:00.100339+0000","last_fullsized":"2024-04-25T10:09:00.100339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:25.880429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.100684+0000","last_change":"2024-04-25T10:09:00.097516+0000","last_active":"2024-04-25T10:09:00.100684+0000","last_peered":"2024-04-25T10:09:00.100684+0000","last_clean":"2024-04-25T10:09:00.100684+0000","last_became_active":"2024-04-25T10:09:00.097152+0000","last_became_peered":"2024-04-25T10:09:00.097152+0000","last_unstale":"2024-04-25T10:09:00.100684+0000","last_undegraded":"2024-04-25T10:09:00.100684+0000","last_fullsized":"2024-04-25T10:09:00.100684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:40.206207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116714+0000","last_change":"2024-04-25T10:08:59.096460+0000","last_active":"2024-04-25T10:09:00.116714+0000","last_peered":"2024-04-25T10:09:00.116714+0000","last_clean":"2024-04-25T10:09:00.116714+0000","last_became_active":"2024-04-25T10:08:59.096134+0000","last_became_peered":"2024-04-25T10:08:59.096134+0000","last_unstale":"2024-04-25T10:09:00.116714+0000","last_undegraded":"2024-04-25T10:09:00.116714+0000","last_fullsized":"2024-04-25T10:09:00.116714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:53.483811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.095799+0000","last_change":"2024-04-25T10:09:00.104423+0000","last_active":"2024-04-25T10:09:01.095799+0000","last_peered":"2024-04-25T10:09:01.095799+0000","last_clean":"2024-04-25T10:09:01.095799+0000","last_became_active":"2024-04-25T10:09:00.100836+0000","last_became_peered":"2024-04-25T10:09:00.100836+0000","last_unstale":"2024-04-25T10:09:01.095799+0000","last_undegraded":"2024-04-25T10:09:01.095799+0000","last_fullsized":"2024-04-25T10:09:01.095799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:32.589797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093092+0000","last_change":"2024-04-25T10:08:59.090145+0000","last_active":"2024-04-25T10:09:00.093092+0000","last_peered":"2024-04-25T10:09:00.093092+0000","last_clean":"2024-04-25T10:09:00.093092+0000","last_became_active":"2024-04-25T10:08:59.089810+0000","last_became_peered":"2024-04-25T10:08:59.089810+0000","last_unstale":"2024-04-25T10:09:00.093092+0000","last_undegraded":"2024-04-25T10:09:00.093092+0000","last_fullsized":"2024-04-25T10:09:00.093092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:14.705434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115603+0000","last_change":"2024-04-25T10:09:00.110294+0000","last_active":"2024-04-25T10:09:00.115603+0000","last_peered":"2024-04-25T10:09:00.115603+0000","last_clean":"2024-04-25T10:09:00.115603+0000","last_became_active":"2024-04-25T10:09:00.107927+0000","last_became_peered":"2024-04-25T10:09:00.107927+0000","last_unstale":"2024-04-25T10:09:00.115603+0000","last_undegraded":"2024-04-25T10:09:00.115603+0000","last_fullsized":"2024-04-25T10:09:00.115603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:46.230653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.080507+0000","last_change":"2024-04-25T10:08:59.094440+0000","last_active":"2024-04-25T10:09:00.080507+0000","last_peered":"2024-04-25T10:09:00.080507+0000","last_clean":"2024-04-25T10:09:00.080507+0000","last_became_active":"2024-04-25T10:08:59.094115+0000","last_became_peered":"2024-04-25T10:08:59.094115+0000","last_unstale":"2024-04-25T10:09:00.080507+0000","last_undegraded":"2024-04-25T10:09:00.080507+0000","last_fullsized":"2024-04-25T10:09:00.080507+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:59.041914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.112275+0000","last_change":"2024-04-25T10:09:00.110091+0000","last_active":"2024-04-25T10:09:00.112275+0000","last_peered":"2024-04-25T10:09:00.112275+0000","last_clean":"2024-04-25T10:09:00.112275+0000","last_became_active":"2024-04-25T10:09:00.109803+0000","last_became_peered":"2024-04-25T10:09:00.109803+0000","last_unstale":"2024-04-25T10:09:00.112275+0000","last_undegraded":"2024-04-25T10:09:00.112275+0000","last_fullsized":"2024-04-25T10:09:00.112275+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:59.232732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.083824+0000","last_change":"2024-04-25T10:08:59.099038+0000","last_active":"2024-04-25T10:09:00.083824+0000","last_peered":"2024-04-25T10:09:00.083824+0000","last_clean":"2024-04-25T10:09:00.083824+0000","last_became_active":"2024-04-25T10:08:59.097931+0000","last_became_peered":"2024-04-25T10:08:59.097931+0000","last_unstale":"2024-04-25T10:09:00.083824+0000","last_undegraded":"2024-04-25T10:09:00.083824+0000","last_fullsized":"2024-04-25T10:09:00.083824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.981495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092721+0000","last_change":"2024-04-25T10:09:00.087851+0000","last_active":"2024-04-25T10:09:00.092721+0000","last_peered":"2024-04-25T10:09:00.092721+0000","last_clean":"2024-04-25T10:09:00.092721+0000","last_became_active":"2024-04-25T10:09:00.087550+0000","last_became_peered":"2024-04-25T10:09:00.087550+0000","last_unstale":"2024-04-25T10:09:00.092721+0000","last_undegraded":"2024-04-25T10:09:00.092721+0000","last_fullsized":"2024-04-25T10:09:00.092721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:18.740243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093832+0000","last_change":"2024-04-25T10:08:59.094892+0000","last_active":"2024-04-25T10:09:00.093832+0000","last_peered":"2024-04-25T10:09:00.093832+0000","last_clean":"2024-04-25T10:09:00.093832+0000","last_became_active":"2024-04-25T10:08:59.094612+0000","last_became_peered":"2024-04-25T10:08:59.094612+0000","last_unstale":"2024-04-25T10:09:00.093832+0000","last_undegraded":"2024-04-25T10:09:00.093832+0000","last_fullsized":"2024-04-25T10:09:00.093832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:38.150543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115194+0000","last_change":"2024-04-25T10:09:00.104125+0000","last_active":"2024-04-25T10:09:00.115194+0000","last_peered":"2024-04-25T10:09:00.115194+0000","last_clean":"2024-04-25T10:09:00.115194+0000","last_became_active":"2024-04-25T10:09:00.103800+0000","last_became_peered":"2024-04-25T10:09:00.103800+0000","last_unstale":"2024-04-25T10:09:00.115194+0000","last_undegraded":"2024-04-25T10:09:00.115194+0000","last_fullsized":"2024-04-25T10:09:00.115194+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:18.670347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"88'3","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.104770+0000","last_change":"2024-04-25T10:08:59.092861+0000","last_active":"2024-04-25T10:09:00.104770+0000","last_peered":"2024-04-25T10:09:00.104770+0000","last_clean":"2024-04-25T10:09:00.104770+0000","last_became_active":"2024-04-25T10:08:59.092126+0000","last_became_peered":"2024-04-25T10:08:59.092126+0000","last_unstale":"2024-04-25T10:09:00.104770+0000","last_undegraded":"2024-04-25T10:09:00.104770+0000","last_fullsized":"2024-04-25T10:09:00.104770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:25.708966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.097657+0000","last_change":"2024-04-25T10:09:00.086986+0000","last_active":"2024-04-25T10:09:01.097657+0000","last_peered":"2024-04-25T10:09:01.097657+0000","last_clean":"2024-04-25T10:09:01.097657+0000","last_became_active":"2024-04-25T10:09:00.086571+0000","last_became_peered":"2024-04-25T10:09:00.086571+0000","last_unstale":"2024-04-25T10:09:01.097657+0000","last_undegraded":"2024-04-25T10:09:01.097657+0000","last_fullsized":"2024-04-25T10:09:01.097657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:28.407001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.120496+0000","last_change":"2024-04-25T10:08:59.093933+0000","last_active":"2024-04-25T10:09:00.120496+0000","last_peered":"2024-04-25T10:09:00.120496+0000","last_clean":"2024-04-25T10:09:00.120496+0000","last_became_active":"2024-04-25T10:08:59.092816+0000","last_became_peered":"2024-04-25T10:08:59.092816+0000","last_unstale":"2024-04-25T10:09:00.120496+0000","last_undegraded":"2024-04-25T10:09:00.120496+0000","last_fullsized":"2024-04-25T10:09:00.120496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:04.085717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115395+0000","last_change":"2024-04-25T10:09:00.106166+0000","last_active":"2024-04-25T10:09:00.115395+0000","last_peered":"2024-04-25T10:09:00.115395+0000","last_clean":"2024-04-25T10:09:00.115395+0000","last_became_active":"2024-04-25T10:09:00.105099+0000","last_became_peered":"2024-04-25T10:09:00.105099+0000","last_unstale":"2024-04-25T10:09:00.115395+0000","last_undegraded":"2024-04-25T10:09:00.115395+0000","last_fullsized":"2024-04-25T10:09:00.115395+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:38.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115844+0000","last_change":"2024-04-25T10:08:59.095627+0000","last_active":"2024-04-25T10:09:00.115844+0000","last_peered":"2024-04-25T10:09:00.115844+0000","last_clean":"2024-04-25T10:09:00.115844+0000","last_became_active":"2024-04-25T10:08:59.095363+0000","last_became_peered":"2024-04-25T10:08:59.095363+0000","last_unstale":"2024-04-25T10:09:00.115844+0000","last_undegraded":"2024-04-25T10:09:00.115844+0000","last_fullsized":"2024-04-25T10:09:00.115844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:55.919386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093283+0000","last_change":"2024-04-25T10:09:00.088747+0000","last_active":"2024-04-25T10:09:00.093283+0000","last_peered":"2024-04-25T10:09:00.093283+0000","last_clean":"2024-04-25T10:09:00.093283+0000","last_became_active":"2024-04-25T10:09:00.088401+0000","last_became_peered":"2024-04-25T10:09:00.088401+0000","last_unstale":"2024-04-25T10:09:00.093283+0000","last_undegraded":"2024-04-25T10:09:00.093283+0000","last_fullsized":"2024-04-25T10:09:00.093283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:32.071482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.084912+0000","last_change":"2024-04-25T10:08:59.094246+0000","last_active":"2024-04-25T10:09:00.084912+0000","last_peered":"2024-04-25T10:09:00.084912+0000","last_clean":"2024-04-25T10:09:00.084912+0000","last_became_active":"2024-04-25T10:08:59.092895+0000","last_became_peered":"2024-04-25T10:08:59.092895+0000","last_unstale":"2024-04-25T10:09:00.084912+0000","last_undegraded":"2024-04-25T10:09:00.084912+0000","last_fullsized":"2024-04-25T10:09:00.084912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:01.492020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115371+0000","last_change":"2024-04-25T10:09:00.106894+0000","last_active":"2024-04-25T10:09:00.115371+0000","last_peered":"2024-04-25T10:09:00.115371+0000","last_clean":"2024-04-25T10:09:00.115371+0000","last_became_active":"2024-04-25T10:09:00.106578+0000","last_became_peered":"2024-04-25T10:09:00.106578+0000","last_unstale":"2024-04-25T10:09:00.115371+0000","last_undegraded":"2024-04-25T10:09:00.115371+0000","last_fullsized":"2024-04-25T10:09:00.115371+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:13.200401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.092126+0000","last_change":"2024-04-25T10:08:59.094516+0000","last_active":"2024-04-25T10:09:00.092126+0000","last_peered":"2024-04-25T10:09:00.092126+0000","last_clean":"2024-04-25T10:09:00.092126+0000","last_became_active":"2024-04-25T10:08:59.094205+0000","last_became_peered":"2024-04-25T10:08:59.094205+0000","last_unstale":"2024-04-25T10:09:00.092126+0000","last_undegraded":"2024-04-25T10:09:00.092126+0000","last_fullsized":"2024-04-25T10:09:00.092126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:19.761546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116980+0000","last_change":"2024-04-25T10:09:00.103121+0000","last_active":"2024-04-25T10:09:00.116980+0000","last_peered":"2024-04-25T10:09:00.116980+0000","last_clean":"2024-04-25T10:09:00.116980+0000","last_became_active":"2024-04-25T10:09:00.102714+0000","last_became_peered":"2024-04-25T10:09:00.102714+0000","last_unstale":"2024-04-25T10:09:00.116980+0000","last_undegraded":"2024-04-25T10:09:00.116980+0000","last_fullsized":"2024-04-25T10:09:00.116980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:07.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.119410+0000","last_change":"2024-04-25T10:08:59.089451+0000","last_active":"2024-04-25T10:09:00.119410+0000","last_peered":"2024-04-25T10:09:00.119410+0000","last_clean":"2024-04-25T10:09:00.119410+0000","last_became_active":"2024-04-25T10:08:59.089022+0000","last_became_peered":"2024-04-25T10:08:59.089022+0000","last_unstale":"2024-04-25T10:09:00.119410+0000","last_undegraded":"2024-04-25T10:09:00.119410+0000","last_fullsized":"2024-04-25T10:09:00.119410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.547679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.112664+0000","last_change":"2024-04-25T10:09:00.101830+0000","last_active":"2024-04-25T10:09:00.112664+0000","last_peered":"2024-04-25T10:09:00.112664+0000","last_clean":"2024-04-25T10:09:00.112664+0000","last_became_active":"2024-04-25T10:09:00.101499+0000","last_became_peered":"2024-04-25T10:09:00.101499+0000","last_unstale":"2024-04-25T10:09:00.112664+0000","last_undegraded":"2024-04-25T10:09:00.112664+0000","last_fullsized":"2024-04-25T10:09:00.112664+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:42.374443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.089711+0000","last_change":"2024-04-25T10:08:59.084911+0000","last_active":"2024-04-25T10:09:00.089711+0000","last_peered":"2024-04-25T10:09:00.089711+0000","last_clean":"2024-04-25T10:09:00.089711+0000","last_became_active":"2024-04-25T10:08:59.084610+0000","last_became_peered":"2024-04-25T10:08:59.084610+0000","last_unstale":"2024-04-25T10:09:00.089711+0000","last_undegraded":"2024-04-25T10:09:00.089711+0000","last_fullsized":"2024-04-25T10:09:00.089711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:41.359101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.084595+0000","last_change":"2024-04-25T10:09:00.083017+0000","last_active":"2024-04-25T10:09:00.084595+0000","last_peered":"2024-04-25T10:09:00.084595+0000","last_clean":"2024-04-25T10:09:00.084595+0000","last_became_active":"2024-04-25T10:09:00.082230+0000","last_became_peered":"2024-04-25T10:09:00.082230+0000","last_unstale":"2024-04-25T10:09:00.084595+0000","last_undegraded":"2024-04-25T10:09:00.084595+0000","last_fullsized":"2024-04-25T10:09:00.084595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:56.873015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.089084+0000","last_change":"2024-04-25T10:08:59.093883+0000","last_active":"2024-04-25T10:09:00.089084+0000","last_peered":"2024-04-25T10:09:00.089084+0000","last_clean":"2024-04-25T10:09:00.089084+0000","last_became_active":"2024-04-25T10:08:59.093505+0000","last_became_peered":"2024-04-25T10:08:59.093505+0000","last_unstale":"2024-04-25T10:09:00.089084+0000","last_undegraded":"2024-04-25T10:09:00.089084+0000","last_fullsized":"2024-04-25T10:09:00.089084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:31.056140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.112406+0000","last_change":"2024-04-25T10:09:00.110465+0000","last_active":"2024-04-25T10:09:00.112406+0000","last_peered":"2024-04-25T10:09:00.112406+0000","last_clean":"2024-04-25T10:09:00.112406+0000","last_became_active":"2024-04-25T10:09:00.109857+0000","last_became_peered":"2024-04-25T10:09:00.109857+0000","last_unstale":"2024-04-25T10:09:00.112406+0000","last_undegraded":"2024-04-25T10:09:00.112406+0000","last_fullsized":"2024-04-25T10:09:00.112406+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:46.771136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.090158+0000","last_change":"2024-04-25T10:08:59.093106+0000","last_active":"2024-04-25T10:09:00.090158+0000","last_peered":"2024-04-25T10:09:00.090158+0000","last_clean":"2024-04-25T10:09:00.090158+0000","last_became_active":"2024-04-25T10:08:59.092805+0000","last_became_peered":"2024-04-25T10:08:59.092805+0000","last_unstale":"2024-04-25T10:09:00.090158+0000","last_undegraded":"2024-04-25T10:09:00.090158+0000","last_fullsized":"2024-04-25T10:09:00.090158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:17.807099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.094659+0000","last_change":"2024-04-25T10:09:00.118480+0000","last_active":"2024-04-25T10:09:01.094659+0000","last_peered":"2024-04-25T10:09:01.094659+0000","last_clean":"2024-04-25T10:09:01.094659+0000","last_became_active":"2024-04-25T10:09:00.118085+0000","last_became_peered":"2024-04-25T10:09:00.118085+0000","last_unstale":"2024-04-25T10:09:01.094659+0000","last_undegraded":"2024-04-25T10:09:01.094659+0000","last_fullsized":"2024-04-25T10:09:01.094659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:51.973663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.104668+0000","last_change":"2024-04-25T10:08:59.093989+0000","last_active":"2024-04-25T10:09:00.104668+0000","last_peered":"2024-04-25T10:09:00.104668+0000","last_clean":"2024-04-25T10:09:00.104668+0000","last_became_active":"2024-04-25T10:08:59.093699+0000","last_became_peered":"2024-04-25T10:08:59.093699+0000","last_unstale":"2024-04-25T10:09:00.104668+0000","last_undegraded":"2024-04-25T10:09:00.104668+0000","last_fullsized":"2024-04-25T10:09:00.104668+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:00.662452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.097103+0000","last_change":"2024-04-25T10:09:00.096022+0000","last_active":"2024-04-25T10:09:00.097103+0000","last_peered":"2024-04-25T10:09:00.097103+0000","last_clean":"2024-04-25T10:09:00.097103+0000","last_became_active":"2024-04-25T10:09:00.095715+0000","last_became_peered":"2024-04-25T10:09:00.095715+0000","last_unstale":"2024-04-25T10:09:00.097103+0000","last_undegraded":"2024-04-25T10:09:00.097103+0000","last_fullsized":"2024-04-25T10:09:00.097103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:23.373405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.095738+0000","last_change":"2024-04-25T10:08:59.094052+0000","last_active":"2024-04-25T10:09:01.095738+0000","last_peered":"2024-04-25T10:09:01.095738+0000","last_clean":"2024-04-25T10:09:01.095738+0000","last_became_active":"2024-04-25T10:08:59.093721+0000","last_became_peered":"2024-04-25T10:08:59.093721+0000","last_unstale":"2024-04-25T10:09:01.095738+0000","last_undegraded":"2024-04-25T10:09:01.095738+0000","last_fullsized":"2024-04-25T10:09:01.095738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:16.149400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.099081+0000","last_change":"2024-04-25T10:09:00.097745+0000","last_active":"2024-04-25T10:09:00.099081+0000","last_peered":"2024-04-25T10:09:00.099081+0000","last_clean":"2024-04-25T10:09:00.099081+0000","last_became_active":"2024-04-25T10:09:00.097414+0000","last_became_peered":"2024-04-25T10:09:00.097414+0000","last_unstale":"2024-04-25T10:09:00.099081+0000","last_undegraded":"2024-04-25T10:09:00.099081+0000","last_fullsized":"2024-04-25T10:09:00.099081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:21.577126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.071693+0000","last_change":"2024-04-25T10:08:59.095243+0000","last_active":"2024-04-25T10:09:00.071693+0000","last_peered":"2024-04-25T10:09:00.071693+0000","last_clean":"2024-04-25T10:09:00.071693+0000","last_became_active":"2024-04-25T10:08:59.094898+0000","last_became_peered":"2024-04-25T10:08:59.094898+0000","last_unstale":"2024-04-25T10:09:00.071693+0000","last_undegraded":"2024-04-25T10:09:00.071693+0000","last_fullsized":"2024-04-25T10:09:00.071693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:17.981495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.115582+0000","last_change":"2024-04-25T10:09:00.108733+0000","last_active":"2024-04-25T10:09:00.115582+0000","last_peered":"2024-04-25T10:09:00.115582+0000","last_clean":"2024-04-25T10:09:00.115582+0000","last_became_active":"2024-04-25T10:09:00.107531+0000","last_became_peered":"2024-04-25T10:09:00.107531+0000","last_unstale":"2024-04-25T10:09:00.115582+0000","last_undegraded":"2024-04-25T10:09:00.115582+0000","last_fullsized":"2024-04-25T10:09:00.115582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:40.236525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.106445+0000","last_change":"2024-04-25T10:08:59.087309+0000","last_active":"2024-04-25T10:09:00.106445+0000","last_peered":"2024-04-25T10:09:00.106445+0000","last_clean":"2024-04-25T10:09:00.106445+0000","last_became_active":"2024-04-25T10:08:59.086733+0000","last_became_peered":"2024-04-25T10:08:59.086733+0000","last_unstale":"2024-04-25T10:09:00.106445+0000","last_undegraded":"2024-04-25T10:09:00.106445+0000","last_fullsized":"2024-04-25T10:09:00.106445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.367704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.098557+0000","last_change":"2024-04-25T10:09:00.094380+0000","last_active":"2024-04-25T10:09:00.098557+0000","last_peered":"2024-04-25T10:09:00.098557+0000","last_clean":"2024-04-25T10:09:00.098557+0000","last_became_active":"2024-04-25T10:09:00.094098+0000","last_became_peered":"2024-04-25T10:09:00.094098+0000","last_unstale":"2024-04-25T10:09:00.098557+0000","last_undegraded":"2024-04-25T10:09:00.098557+0000","last_fullsized":"2024-04-25T10:09:00.098557+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:07.616247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.082207+0000","last_change":"2024-04-25T10:08:59.081444+0000","last_active":"2024-04-25T10:09:00.082207+0000","last_peered":"2024-04-25T10:09:00.082207+0000","last_clean":"2024-04-25T10:09:00.082207+0000","last_became_active":"2024-04-25T10:08:59.081110+0000","last_became_peered":"2024-04-25T10:08:59.081110+0000","last_unstale":"2024-04-25T10:09:00.082207+0000","last_undegraded":"2024-04-25T10:09:00.082207+0000","last_fullsized":"2024-04-25T10:09:00.082207+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:28.169249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.099959+0000","last_change":"2024-04-25T10:09:00.093010+0000","last_active":"2024-04-25T10:09:00.099959+0000","last_peered":"2024-04-25T10:09:00.099959+0000","last_clean":"2024-04-25T10:09:00.099959+0000","last_became_active":"2024-04-25T10:09:00.089467+0000","last_became_peered":"2024-04-25T10:09:00.089467+0000","last_unstale":"2024-04-25T10:09:00.099959+0000","last_undegraded":"2024-04-25T10:09:00.099959+0000","last_fullsized":"2024-04-25T10:09:00.099959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:26.895772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.067553+0000","last_change":"2024-04-25T10:08:59.091283+0000","last_active":"2024-04-25T10:09:00.067553+0000","last_peered":"2024-04-25T10:09:00.067553+0000","last_clean":"2024-04-25T10:09:00.067553+0000","last_became_active":"2024-04-25T10:08:59.090450+0000","last_became_peered":"2024-04-25T10:08:59.090450+0000","last_unstale":"2024-04-25T10:09:00.067553+0000","last_undegraded":"2024-04-25T10:09:00.067553+0000","last_fullsized":"2024-04-25T10:09:00.067553+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:00.853913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.117042+0000","last_change":"2024-04-25T10:09:00.109416+0000","last_active":"2024-04-25T10:09:00.117042+0000","last_peered":"2024-04-25T10:09:00.117042+0000","last_clean":"2024-04-25T10:09:00.117042+0000","last_became_active":"2024-04-25T10:09:00.109099+0000","last_became_peered":"2024-04-25T10:09:00.109099+0000","last_unstale":"2024-04-25T10:09:00.117042+0000","last_undegraded":"2024-04-25T10:09:00.117042+0000","last_fullsized":"2024-04-25T10:09:00.117042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:32.510485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.079301+0000","last_change":"2024-04-25T10:08:59.086330+0000","last_active":"2024-04-25T10:09:00.079301+0000","last_peered":"2024-04-25T10:09:00.079301+0000","last_clean":"2024-04-25T10:09:00.079301+0000","last_became_active":"2024-04-25T10:08:59.085977+0000","last_became_peered":"2024-04-25T10:08:59.085977+0000","last_unstale":"2024-04-25T10:09:00.079301+0000","last_undegraded":"2024-04-25T10:09:00.079301+0000","last_fullsized":"2024-04-25T10:09:00.079301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:53.483811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.095286+0000","last_change":"2024-04-25T10:09:00.091254+0000","last_active":"2024-04-25T10:09:00.095286+0000","last_peered":"2024-04-25T10:09:00.095286+0000","last_clean":"2024-04-25T10:09:00.095286+0000","last_became_active":"2024-04-25T10:09:00.089920+0000","last_became_peered":"2024-04-25T10:09:00.089920+0000","last_unstale":"2024-04-25T10:09:00.095286+0000","last_undegraded":"2024-04-25T10:09:00.095286+0000","last_fullsized":"2024-04-25T10:09:00.095286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:27.242447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.065287+0000","last_change":"2024-04-25T10:08:59.087483+0000","last_active":"2024-04-25T10:09:00.065287+0000","last_peered":"2024-04-25T10:09:00.065287+0000","last_clean":"2024-04-25T10:09:00.065287+0000","last_became_active":"2024-04-25T10:08:59.086156+0000","last_became_peered":"2024-04-25T10:08:59.086156+0000","last_unstale":"2024-04-25T10:09:00.065287+0000","last_undegraded":"2024-04-25T10:09:00.065287+0000","last_fullsized":"2024-04-25T10:09:00.065287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:40.439344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.095017+0000","last_change":"2024-04-25T10:09:00.114927+0000","last_active":"2024-04-25T10:09:01.095017+0000","last_peered":"2024-04-25T10:09:01.095017+0000","last_clean":"2024-04-25T10:09:01.095017+0000","last_became_active":"2024-04-25T10:09:00.114101+0000","last_became_peered":"2024-04-25T10:09:00.114101+0000","last_unstale":"2024-04-25T10:09:01.095017+0000","last_undegraded":"2024-04-25T10:09:01.095017+0000","last_fullsized":"2024-04-25T10:09:01.095017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:03.846157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.091763+0000","last_change":"2024-04-25T10:08:59.093509+0000","last_active":"2024-04-25T10:09:00.091763+0000","last_peered":"2024-04-25T10:09:00.091763+0000","last_clean":"2024-04-25T10:09:00.091763+0000","last_became_active":"2024-04-25T10:08:59.092849+0000","last_became_peered":"2024-04-25T10:08:59.092849+0000","last_unstale":"2024-04-25T10:09:00.091763+0000","last_undegraded":"2024-04-25T10:09:00.091763+0000","last_fullsized":"2024-04-25T10:09:00.091763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:31.495143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.116671+0000","last_change":"2024-04-25T10:09:00.113592+0000","last_active":"2024-04-25T10:09:00.116671+0000","last_peered":"2024-04-25T10:09:00.116671+0000","last_clean":"2024-04-25T10:09:00.116671+0000","last_became_active":"2024-04-25T10:09:00.113121+0000","last_became_peered":"2024-04-25T10:09:00.113121+0000","last_unstale":"2024-04-25T10:09:00.116671+0000","last_undegraded":"2024-04-25T10:09:00.116671+0000","last_fullsized":"2024-04-25T10:09:00.116671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:58.240994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.072585+0000","last_change":"2024-04-25T10:08:59.096272+0000","last_active":"2024-04-25T10:09:00.072585+0000","last_peered":"2024-04-25T10:09:00.072585+0000","last_clean":"2024-04-25T10:09:00.072585+0000","last_became_active":"2024-04-25T10:08:59.095932+0000","last_became_peered":"2024-04-25T10:08:59.095932+0000","last_unstale":"2024-04-25T10:09:00.072585+0000","last_undegraded":"2024-04-25T10:09:00.072585+0000","last_fullsized":"2024-04-25T10:09:00.072585+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:00.619624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.107474+0000","last_change":"2024-04-25T10:09:00.106808+0000","last_active":"2024-04-25T10:09:00.107474+0000","last_peered":"2024-04-25T10:09:00.107474+0000","last_clean":"2024-04-25T10:09:00.107474+0000","last_became_active":"2024-04-25T10:09:00.106373+0000","last_became_peered":"2024-04-25T10:09:00.106373+0000","last_unstale":"2024-04-25T10:09:00.107474+0000","last_undegraded":"2024-04-25T10:09:00.107474+0000","last_fullsized":"2024-04-25T10:09:00.107474+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:32.510485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.073065+0000","last_change":"2024-04-25T10:08:59.094065+0000","last_active":"2024-04-25T10:09:00.073065+0000","last_peered":"2024-04-25T10:09:00.073065+0000","last_clean":"2024-04-25T10:09:00.073065+0000","last_became_active":"2024-04-25T10:08:59.093019+0000","last_became_peered":"2024-04-25T10:08:59.093019+0000","last_unstale":"2024-04-25T10:09:00.073065+0000","last_undegraded":"2024-04-25T10:09:00.073065+0000","last_fullsized":"2024-04-25T10:09:00.073065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:06.887928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:09:01.095211+0000","last_change":"2024-04-25T10:09:00.109462+0000","last_active":"2024-04-25T10:09:01.095211+0000","last_peered":"2024-04-25T10:09:01.095211+0000","last_clean":"2024-04-25T10:09:01.095211+0000","last_became_active":"2024-04-25T10:09:00.109103+0000","last_became_peered":"2024-04-25T10:09:00.109103+0000","last_unstale":"2024-04-25T10:09:01.095211+0000","last_undegraded":"2024-04-25T10:09:01.095211+0000","last_fullsized":"2024-04-25T10:09:01.095211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:21.683581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.080703+0000","last_change":"2024-04-25T10:08:59.082272+0000","last_active":"2024-04-25T10:09:00.080703+0000","last_peered":"2024-04-25T10:09:00.080703+0000","last_clean":"2024-04-25T10:09:00.080703+0000","last_became_active":"2024-04-25T10:08:59.082003+0000","last_became_peered":"2024-04-25T10:08:59.082003+0000","last_unstale":"2024-04-25T10:09:00.080703+0000","last_undegraded":"2024-04-25T10:09:00.080703+0000","last_fullsized":"2024-04-25T10:09:00.080703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:16.232770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.110594+0000","last_change":"2024-04-25T10:09:00.109500+0000","last_active":"2024-04-25T10:09:00.110594+0000","last_peered":"2024-04-25T10:09:00.110594+0000","last_clean":"2024-04-25T10:09:00.110594+0000","last_became_active":"2024-04-25T10:09:00.109224+0000","last_became_peered":"2024-04-25T10:09:00.109224+0000","last_unstale":"2024-04-25T10:09:00.110594+0000","last_undegraded":"2024-04-25T10:09:00.110594+0000","last_fullsized":"2024-04-25T10:09:00.110594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:59.041251+0000","last_clean_scrub_stamp":"2024-04-25T10:08:59.041251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:01.668226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.093932+0000","last_change":"2024-04-25T10:08:59.093748+0000","last_active":"2024-04-25T10:09:00.093932+0000","last_peered":"2024-04-25T10:09:00.093932+0000","last_clean":"2024-04-25T10:09:00.093932+0000","last_became_active":"2024-04-25T10:08:59.093009+0000","last_became_peered":"2024-04-25T10:08:59.093009+0000","last_unstale":"2024-04-25T10:09:00.093932+0000","last_undegraded":"2024-04-25T10:09:00.093932+0000","last_fullsized":"2024-04-25T10:09:00.093932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:58.025908+0000","last_clean_scrub_stamp":"2024-04-25T10:08:58.025908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:40.357143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":295,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T10:09:00.083197+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:09:00.083197+0000","last_peered":"2024-04-25T10:09:00.083197+0000","last_clean":"2024-04-25T10:09:00.083197+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:09:00.083197+0000","last_undegraded":"2024-04-25T10:09:00.083197+0000","last_fullsized":"2024-04-25T10:09:00.083197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509529,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1077248,"data_stored":491301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":493037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509530,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":1081669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509531,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1097728,"data_stored":493025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":491905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509529,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1683456,"data_stored":1082355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1081344,"data_stored":491371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":497993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:09:05.064 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:09:05.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:09:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.322+0000 7f1757e00640 1 Processor -- start 2024-04-25T10:09:05.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.322+0000 7f1757e00640 1 -- start start 2024-04-25T10:09:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1757e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f17580b48c0 0x7f175815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1757e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17581606e0 con 0x7f1758161d90 2024-04-25T10:09:05.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1757e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17581609c0 con 0x7f17580d3f90 2024-04-25T10:09:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1757e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1758160ca0 con 0x7f17580b48c0 2024-04-25T10:09:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1757400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1758161d90 0x7f17580b2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33344/0 (socket says 172.21.15.5:33344) 2024-04-25T10:09:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1757400640 1 -- 172.21.15.5:0/21404283 learned_addr learned my addr 172.21.15.5:0/21404283 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:05.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.324+0000 7f1756a00640 1 --2- 172.21.15.5:0/21404283 >> v2:172.21.15.80:3300/0 conn(0x7f17580b48c0 0x7f175815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111900634 0 0) 0x7f17581606e0 con 0x7f1758161d90 2024-04-25T10:09:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1738003520 con 0x7f1758161d90 2024-04-25T10:09:05.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2796907173 0 0) 0x7f1738003520 con 0x7f1758161d90 2024-04-25T10:09:05.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 >> v2:172.21.15.80:3300/0 conn(0x7f17580b48c0 msgr2=0x7f175815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 --2- 172.21.15.5:0/21404283 >> v2:172.21.15.80:3300/0 conn(0x7f17580b48c0 0x7f175815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 >> v1:172.21.15.80:6789/0 conn(0x7f17580d3f90 legacy=0x7f1758161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1758160f80 con 0x7f1758161d90 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1750001670 con 0x7f1758161d90 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 >> v1:172.21.15.5:6789/0 conn(0x7f1758161d90 legacy=0x7f17580b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1738003a00 0x7f1738003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1738004800 0x7f1738004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738005210 0x7f173800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1758160ca0 con 0x7f1738003a00 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173800c160 con 0x7f1738004800 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.325+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173800c360 con 0x7f1738005210 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756a00640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1738003a00 0x7f1738003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756000640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1738004800 0x7f1738004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756a00640 1 -- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738005210 msgr2=0x7f173800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1757400640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738005210 0x7f173800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756a00640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738005210 0x7f173800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756a00640 1 -- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1738004800 msgr2=0x7f1738004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756a00640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1738004800 0x7f1738004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1756a00640 1 -- 172.21.15.5:0/21404283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738003520 con 0x7f1738003a00 2024-04-25T10:09:05.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.326+0000 7f1757400640 1 -- 172.21.15.5:0/21404283 reap_dead start 2024-04-25T10:09:05.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.327+0000 7f1756a00640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1738003a00 0x7f1738003e20 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1748009a90 tx=0x7f1738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eca51264e03800c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.328+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748036070 con 0x7f1738003a00 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.328+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1748031400 con 0x7f1738003a00 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.328+0000 7f1755600640 1 -- 172.21.15.5:0/21404283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748009690 con 0x7f1738003a00 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.328+0000 7f1757e00640 1 -- 172.21.15.5:0/21404283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1738003a00 msgr2=0x7f1738003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.328+0000 7f1757e00640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1738003a00 0x7f1738003e20 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f1748009a90 tx=0x7f1738005830 comp rx=0 tx=0).stop 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.329+0000 7f1757e00640 1 -- 172.21.15.5:0/21404283 shutdown_connections 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.329+0000 7f1757e00640 1 --2- 172.21.15.5:0/21404283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1738003a00 0x7f1738003e20 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.329+0000 7f1757e00640 1 -- 172.21.15.5:0/21404283 >> 172.21.15.5:0/21404283 conn(0x7f175808af80 msgr2=0x7f17580d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.329+0000 7f1757e00640 1 -- 172.21.15.5:0/21404283 shutdown_connections 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.329+0000 7f1757e00640 1 -- 172.21.15.5:0/21404283 wait complete. 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.330+0000 7f1757e00640 1 Processor -- start 2024-04-25T10:09:05.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.331+0000 7f1757e00640 1 -- start start 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.331+0000 7f1757e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738003a00 0x7f17580c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1757e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 0x7f17581f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1757e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17580b6ff0 0x7f17581f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1757e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1758161680 con 0x7f17580b6b90 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1757e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580b4ca0 con 0x7f17580b6ff0 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1757e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1758161800 con 0x7f1738003a00 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 0x7f17581f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 0x7f17581f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36150/0 (socket says 172.21.15.5:36150) 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 -- 172.21.15.5:0/756173602 learned_addr learned my addr 172.21.15.5:0/756173602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756a00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738003a00 0x7f17580c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 -- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738003a00 msgr2=0x7f17580c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1757400640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17580b6ff0 0x7f17581f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738003a00 0x7f17580c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 -- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17580b6ff0 msgr2=0x7f17581f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17580b6ff0 0x7f17581f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756000640 1 -- 172.21.15.5:0/756173602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580b8790 con 0x7f17580b6b90 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.332+0000 7f1756a00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738003a00 0x7f17580c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.333+0000 7f1756000640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 0x7f17581f7000 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f174000e7f0 tx=0x7f174000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.333+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740003bd0 con 0x7f17580b6b90 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.333+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740018070 con 0x7f17580b6b90 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.333+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174000c6c0 con 0x7f17580b6b90 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.333+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17580cb320 con 0x7f17580b6b90 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.333+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580b8cf0 con 0x7f17580b6b90 2024-04-25T10:09:05.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.334+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1758068f60 con 0x7f17580b6b90 2024-04-25T10:09:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.335+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f174000c860 con 0x7f17580b6b90 2024-04-25T10:09:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.336+0000 7f174ea00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f173406b500 0x7f173406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.336+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f174008a3e0 con 0x7f17580b6b90 2024-04-25T10:09:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.337+0000 7f1756a00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f173406b500 0x7f173406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.338+0000 7f1756a00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f173406b500 0x7f173406d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1748033000 tx=0x7f1748008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.341+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17580b8790 con 0x7f17580b6b90 2024-04-25T10:09:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.613+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f175808d730 con 0x7f17580b6b90 2024-04-25T10:09:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.617+0000 7f174ea00640 1 -- 172.21.15.5:0/756173602 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1740059030 con 0x7f17580b6b90 2024-04-25T10:09:05.624 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:09:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:09:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:09:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:09:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:09:05.625 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:09:05.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f173406b500 msgr2=0x7f173406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f173406b500 0x7f173406d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1748033000 tx=0x7f1748008030 comp rx=0 tx=0).stop 2024-04-25T10:09:05.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 msgr2=0x7f17581f7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 0x7f17581f7000 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f174000e7f0 tx=0x7f174000ecb0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 shutdown_connections 2024-04-25T10:09:05.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f173406b500 0x7f173406d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17580b6ff0 0x7f17581f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17580b6b90 0x7f17581f7000 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 --2- 172.21.15.5:0/756173602 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1738003a00 0x7f17580c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 >> 172.21.15.5:0/756173602 conn(0x7f175808af80 msgr2=0x7f17580b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.620+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 shutdown_connections 2024-04-25T10:09:05.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.621+0000 7f1757e00640 1 -- 172.21.15.5:0/756173602 wait complete. 2024-04-25T10:09:05.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.847+0000 7f6cd5000640 1 Processor -- start 2024-04-25T10:09:05.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.847+0000 7f6cd5000640 1 -- start start 2024-04-25T10:09:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cd5000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd015db00 0x7f6cd00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cd5000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cd00b53b0 0x7f6cd00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cd5000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6cd00ce2b0 0x7f6cd0161670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cd5000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd00d1e80 con 0x7f6cd00b53b0 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cd5000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd00b5930 con 0x7f6cd00ce2b0 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cd5000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b5c10 con 0x7f6cd015db00 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cc7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cd00b53b0 0x7f6cd00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cc7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cd00b53b0 0x7f6cd00d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36156/0 (socket says 172.21.15.5:36156) 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.849+0000 7f6cc7e00640 1 -- 172.21.15.5:0/758621795 learned_addr learned my addr 172.21.15.5:0/758621795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6ccde00640 1 --2- 172.21.15.5:0/758621795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd015db00 0x7f6cd00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6cce800640 1 --2- 172.21.15.5:0/758621795 >> v2:172.21.15.80:3300/0 conn(0x7f6cd00ce2b0 0x7f6cd0161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6cc7e00640 1 -- 172.21.15.5:0/758621795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd015db00 msgr2=0x7f6cd00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6cc7e00640 1 --2- 172.21.15.5:0/758621795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd015db00 0x7f6cd00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6cc7e00640 1 -- 172.21.15.5:0/758621795 >> v2:172.21.15.80:3300/0 conn(0x7f6cd00ce2b0 msgr2=0x7f6cd0161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6cc7e00640 1 --2- 172.21.15.5:0/758621795 >> v2:172.21.15.80:3300/0 conn(0x7f6cd00ce2b0 0x7f6cd0161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.850+0000 7f6cc7e00640 1 -- 172.21.15.5:0/758621795 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00b5ef0 con 0x7f6cd00b53b0 2024-04-25T10:09:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.851+0000 7f6cc7e00640 1 --2- 172.21.15.5:0/758621795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd00b53b0 0x7f6cd00d0880 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00ce010 tx=0x7f6cb802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6d20e38d62ad65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.851+0000 7f6ccd400640 1 -- 172.21.15.5:0/758621795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8037070 con 0x7f6cd00b53b0 2024-04-25T10:09:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.851+0000 7f6ccd400640 1 -- 172.21.15.5:0/758621795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8032440 con 0x7f6cd00b53b0 2024-04-25T10:09:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.851+0000 7f6ccd400640 1 -- 172.21.15.5:0/758621795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8032810 con 0x7f6cd00b53b0 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 -- 172.21.15.5:0/758621795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd00b53b0 msgr2=0x7f6cd00d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 --2- 172.21.15.5:0/758621795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd00b53b0 0x7f6cd00d0880 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00ce010 tx=0x7f6cb802a1d0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 -- 172.21.15.5:0/758621795 shutdown_connections 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 --2- 172.21.15.5:0/758621795 >> v2:172.21.15.80:3300/0 conn(0x7f6cd00ce2b0 0x7f6cd0161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 --2- 172.21.15.5:0/758621795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd00b53b0 0x7f6cd00d0880 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 --2- 172.21.15.5:0/758621795 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd015db00 0x7f6cd00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 -- 172.21.15.5:0/758621795 >> 172.21.15.5:0/758621795 conn(0x7f6cd0072290 msgr2=0x7f6cd00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 -- 172.21.15.5:0/758621795 shutdown_connections 2024-04-25T10:09:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.852+0000 7f6cd5000640 1 -- 172.21.15.5:0/758621795 wait complete. 2024-04-25T10:09:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.854+0000 7f6cd5000640 1 Processor -- start 2024-04-25T10:09:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.854+0000 7f6cd5000640 1 -- start start 2024-04-25T10:09:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.855+0000 7f6cd5000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd00b53b0 0x7f6cd00cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.855+0000 7f6cd5000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd00ce2b0 0x7f6cd01f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.855+0000 7f6cd5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 0x7f6cd00b7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.855+0000 7f6cd5000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00d3660 con 0x7f6cd015db00 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.855+0000 7f6cd5000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b61d0 con 0x7f6cd00ce2b0 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.855+0000 7f6cd5000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b6c10 con 0x7f6cd00b53b0 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 0x7f6cd00b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 0x7f6cd00b7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36166/0 (socket says 172.21.15.5:36166) 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 -- 172.21.15.5:0/2318303365 learned_addr learned my addr 172.21.15.5:0/2318303365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cc7e00640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd00ce2b0 0x7f6cd01f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6ccde00640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd00b53b0 0x7f6cd00cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 -- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd00b53b0 msgr2=0x7f6cd00cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd00b53b0 0x7f6cd00cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 -- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd00ce2b0 msgr2=0x7f6cd01f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd00ce2b0 0x7f6cd01f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 -- 172.21.15.5:0/2318303365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd01f4580 con 0x7f6cd015db00 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.856+0000 7f6cce800640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 0x7f6cd00b7680 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8007e40 tx=0x7f6cc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.857+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8014620 con 0x7f6cd015db00 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.857+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8019070 con 0x7f6cd015db00 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.857+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8014a20 con 0x7f6cd015db00 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.857+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd0153cf0 con 0x7f6cd015db00 2024-04-25T10:09:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.857+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd00cad80 con 0x7f6cd015db00 2024-04-25T10:09:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.858+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd0068fd0 con 0x7f6cd015db00 2024-04-25T10:09:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.859+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cc8014bc0 con 0x7f6cd015db00 2024-04-25T10:09:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.860+0000 7f6cc6000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6cac06b4d0 0x7f6cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.860+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6cc808acb0 con 0x7f6cd015db00 2024-04-25T10:09:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.861+0000 7f6ccde00640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6cac06b4d0 0x7f6cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.862+0000 7f6ccde00640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6cac06b4d0 0x7f6cac06d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0006fd0 tx=0x7f6cc0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:05.865+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cc8059900 con 0x7f6cd015db00 2024-04-25T10:09:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.092+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cd0156be0 con 0x7f6cd015db00 2024-04-25T10:09:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.093+0000 7f6cc6000640 1 -- 172.21.15.5:0/2318303365 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f6cc8059200 con 0x7f6cd015db00 2024-04-25T10:09:06.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:06.101 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:02.600363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7488":{"gid":7488,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/3627376661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3627376661},{"type":"v1","addr":"172.21.15.5:6834","nonce":3627376661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 62 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6cac06b4d0 msgr2=0x7f6cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6cac06b4d0 0x7f6cac06d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0006fd0 tx=0x7f6cc0008040 comp rx=0 tx=0).stop 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 msgr2=0x7f6cd00b7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 0x7f6cd00b7680 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8007e40 tx=0x7f6cc800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 shutdown_connections 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd015db00 0x7f6cd00b7680 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6cac06b4d0 0x7f6cac06d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd00ce2b0 0x7f6cd01f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 --2- 172.21.15.5:0/2318303365 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd00b53b0 0x7f6cd00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 >> 172.21.15.5:0/2318303365 conn(0x7f6cd0072290 msgr2=0x7f6cd00c83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 shutdown_connections 2024-04-25T10:09:06.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.096+0000 7f6cd5000640 1 -- 172.21.15.5:0/2318303365 wait complete. 2024-04-25T10:09:06.524 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:09:02.600363+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7488': {'gid': 7488, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6834/3627376661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 3627376661}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 3627376661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:06.524 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:06.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:09:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.746+0000 7f5183e00640 1 Processor -- start 2024-04-25T10:09:06.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.746+0000 7f5183e00640 1 -- start start 2024-04-25T10:09:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5184141da0 0x7f518406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f518415d510 0x7f51840cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5184071d30 con 0x7f518406e640 2024-04-25T10:09:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5184142180 con 0x7f518415d510 2024-04-25T10:09:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518406d620 con 0x7f5184141da0 2024-04-25T10:09:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f518415d510 0x7f51840cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f518415d510 0x7f51840cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36168/0 (socket says 172.21.15.5:36168) 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5183400640 1 -- 172.21.15.5:0/163755180 learned_addr learned my addr 172.21.15.5:0/163755180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.748+0000 7f5182a00640 1 --2- 172.21.15.5:0/163755180 >> v2:172.21.15.80:3300/0 conn(0x7f5184141da0 0x7f518406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.749+0000 7f5183400640 1 -- 172.21.15.5:0/163755180 >> v2:172.21.15.80:3300/0 conn(0x7f5184141da0 msgr2=0x7f518406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.749+0000 7f5183400640 1 --2- 172.21.15.5:0/163755180 >> v2:172.21.15.80:3300/0 conn(0x7f5184141da0 0x7f518406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.749+0000 7f5183400640 1 -- 172.21.15.5:0/163755180 >> v1:172.21.15.80:6789/0 conn(0x7f518406e640 legacy=0x7f518415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.749+0000 7f5183400640 1 -- 172.21.15.5:0/163755180 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518406d900 con 0x7f518415d510 2024-04-25T10:09:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.750+0000 7f5183400640 1 --2- 172.21.15.5:0/163755180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f518415d510 0x7f51840cd300 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f518406bbb0 tx=0x7f517c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8770578da5abe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.750+0000 7f5181600640 1 -- 172.21.15.5:0/163755180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c037070 con 0x7f518415d510 2024-04-25T10:09:06.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.750+0000 7f5181600640 1 -- 172.21.15.5:0/163755180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c032440 con 0x7f518415d510 2024-04-25T10:09:06.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.750+0000 7f5181600640 1 -- 172.21.15.5:0/163755180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c032810 con 0x7f518415d510 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 -- 172.21.15.5:0/163755180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f518415d510 msgr2=0x7f51840cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 --2- 172.21.15.5:0/163755180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f518415d510 0x7f51840cd300 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f518406bbb0 tx=0x7f517c02a230 comp rx=0 tx=0).stop 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 -- 172.21.15.5:0/163755180 shutdown_connections 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 --2- 172.21.15.5:0/163755180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f518415d510 0x7f51840cd300 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 --2- 172.21.15.5:0/163755180 >> v2:172.21.15.80:3300/0 conn(0x7f5184141da0 0x7f518406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 -- 172.21.15.5:0/163755180 >> 172.21.15.5:0/163755180 conn(0x7f5184066f20 msgr2=0x7f518406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 -- 172.21.15.5:0/163755180 shutdown_connections 2024-04-25T10:09:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.751+0000 7f5183e00640 1 -- 172.21.15.5:0/163755180 wait complete. 2024-04-25T10:09:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.753+0000 7f5183e00640 1 Processor -- start 2024-04-25T10:09:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.753+0000 7f5183e00640 1 -- start start 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f518406e640 0x7f51841590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5184141da0 0x7f51841560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 0x7f518415a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840cd9c0 con 0x7f51840b1ce0 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840cd840 con 0x7f518406e640 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51840cdb40 con 0x7f5184141da0 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 0x7f518415a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 0x7f518415a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36174/0 (socket says 172.21.15.5:36174) 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.754+0000 7f5183400640 1 -- 172.21.15.5:0/2006948931 learned_addr learned my addr 172.21.15.5:0/2006948931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5182a00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f518406e640 0x7f51841590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5182000640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5184141da0 0x7f51841560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5183400640 1 -- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5184141da0 msgr2=0x7f51841560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5183400640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5184141da0 0x7f51841560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5183400640 1 -- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f518406e640 msgr2=0x7f51841590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5183400640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f518406e640 0x7f51841590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5183400640 1 -- 172.21.15.5:0/2006948931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184156900 con 0x7f51840b1ce0 2024-04-25T10:09:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5182000640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5184141da0 0x7f51841560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f5183400640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 0x7f518415a3b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f517c02a270 tx=0x7f517c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f517b400640 1 -- 172.21.15.5:0/2006948931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c041020 con 0x7f51840b1ce0 2024-04-25T10:09:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.755+0000 7f517b400640 1 -- 172.21.15.5:0/2006948931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c0306c0 con 0x7f51840b1ce0 2024-04-25T10:09:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.756+0000 7f517b400640 1 -- 172.21.15.5:0/2006948931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c037040 con 0x7f51840b1ce0 2024-04-25T10:09:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.756+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5184156ba0 con 0x7f51840b1ce0 2024-04-25T10:09:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.756+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5184157440 con 0x7f51840b1ce0 2024-04-25T10:09:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.758+0000 7f517b400640 1 -- 172.21.15.5:0/2006948931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f517c044020 con 0x7f51840b1ce0 2024-04-25T10:09:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.759+0000 7f517b400640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f516006b4d0 0x7f516006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.759+0000 7f517b400640 1 -- 172.21.15.5:0/2006948931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f517c0aab90 con 0x7f51840b1ce0 2024-04-25T10:09:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.760+0000 7f518cbe8740 1 Processor -- start 2024-04-25T10:09:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.760+0000 7f5182a00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f516006b4d0 0x7f516006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.760+0000 7f518cbe8740 1 -- start start 2024-04-25T10:09:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.760+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447fd6b30 0x557447ffe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 0x557447fd6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x557447fd52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f518cbe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557447f3a620 con 0x557447ffeeb0 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557447ee3ca0 con 0x55744801f7a0 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557447f366c0 con 0x557447fd6b30 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182a00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f516006b4d0 0x7f516006d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f516c006fd0 tx=0x7f516c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 0x557447fd6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 0x557447fd6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36186/0 (socket says 172.21.15.5:36186) 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 -- 172.21.15.5:0/1432210231 learned_addr learned my addr 172.21.15.5:0/1432210231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182a00640 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447fd6b30 0x557447ffe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5183400640 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x557447fd52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 -- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447fd6b30 msgr2=0x557447ffe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447fd6b30 0x557447ffe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 -- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 msgr2=0x557447fd52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x557447fd52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5182000640 1 -- 172.21.15.5:0/1432210231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557447eeaf40 con 0x557447ffeeb0 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.761+0000 7f5183400640 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x557447fd52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.762+0000 7f5182000640 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 0x557447fd6f10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5174007f00 tx=0x7f517400c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf801f250afc5507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.762+0000 7f5173400640 1 -- 172.21.15.5:0/1432210231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174014070 con 0x557447ffeeb0 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.762+0000 7f5173400640 1 -- 172.21.15.5:0/1432210231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400f850 con 0x557447ffeeb0 2024-04-25T10:09:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.762+0000 7f5173400640 1 -- 172.21.15.5:0/1432210231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517400fc20 con 0x557447ffeeb0 2024-04-25T10:09:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 -- 172.21.15.5:0/1432210231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 msgr2=0x557447fd6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 0x557447fd6f10 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5174007f00 tx=0x7f517400c690 comp rx=0 tx=0).stop 2024-04-25T10:09:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 -- 172.21.15.5:0/1432210231 shutdown_connections 2024-04-25T10:09:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x557447fd52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ffeeb0 0x557447fd6f10 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1432210231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447fd6b30 0x557447ffe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 -- 172.21.15.5:0/1432210231 >> 172.21.15.5:0/1432210231 conn(0x557447fd3d50 msgr2=0x55744801c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 -- 172.21.15.5:0/1432210231 shutdown_connections 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 -- 172.21.15.5:0/1432210231 wait complete. 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.763+0000 7f518cbe8740 1 -- start start 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.764+0000 7f518cbe8740 1 Processor -- start 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ffeeb0 0x557447ffb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ff5910 0x55744801efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557447ffe4e0 con 0x55744801f7a0 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557447ed91f0 con 0x557447ffeeb0 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557447ffded0 con 0x557447ff5910 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f518cbe8740 10 client.0 _ll_register_callbacks cb 0x557447ef6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36192/0 (socket says 172.21.15.5:36192) 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 -- 172.21.15.5:0/2516434578 learned_addr learned my addr 172.21.15.5:0/2516434578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5183400640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ff5910 0x55744801efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 -- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ff5910 msgr2=0x55744801efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182a00640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ffeeb0 0x557447ffb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ff5910 0x55744801efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 -- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ffeeb0 msgr2=0x557447ffb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ffeeb0 0x557447ffb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.765+0000 7f5182000640 1 -- 172.21.15.5:0/2516434578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557447ef8c50 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f5182000640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ea80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5174013000 tx=0x7f51740049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370d404bb1bc35bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f5167e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f5167e00640 1 -- 172.21.15.5:0/2516434578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174029060 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f517400f850 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f5167e00640 1 -- 172.21.15.5:0/2516434578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5174024470 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f5167e00640 1 -- 172.21.15.5:0/2516434578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517402f050 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f518cbe8740 10 client.7620 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.766+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x557447c4f2d0 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f5167e00640 1 -- 172.21.15.5:0/2516434578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517400f850 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f5167e00640 1 -- 172.21.15.5:0/2516434578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5174032020 con 0x55744801f7a0 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f518cbe8740 10 client.7620 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f518cbe8740 10 client.7620 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:06.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f518cbe8740 20 client.7620 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f518cbe8740 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x55744802d0a0 0x55744802d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f518cbe8740 4 client.7620 mds_command: new command op to 7488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 --> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557447f47b60 con 0x55744802d0a0 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.767+0000 7f5182a00640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x55744802d0a0 0x55744802d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.768+0000 7f5182a00640 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x55744802d0a0 0x55744802d480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09cea8999638eb8 server_cookie=5e94dcac83f576fa in_seq=0 out_seq=0 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.768+0000 7f5167e00640 10 client.7620 ms_handle_connect on v2:172.21.15.5:6832/3627376661 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f5167e00640 1 -- 172.21.15.5:0/2516434578 <== mds.0 v2:172.21.15.5:6832/3627376661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557447f47b60 con 0x55744802d0a0 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f5167e00640 10 client.7620 handle_command_reply: tid=0 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 client.7620 shutdown 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 msgr2=0x55744801ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ea80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5174013000 tx=0x7f51740049c0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 shutdown_connections 2024-04-25T10:09:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ff5910 0x55744801efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ea80 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x55744802d0a0 0x55744802d480 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 --2- 172.21.15.5:0/2516434578 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ffeeb0 0x557447ffb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 >> 172.21.15.5:0/2516434578 conn(0x557447fd6b30 msgr2=0x557447fd52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f5167e00640 0 client.7620 ms_handle_reset on v2:172.21.15.5:6832/3627376661 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 shutdown_connections 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.769+0000 7f518cbe8740 1 -- 172.21.15.5:0/2516434578 wait complete. 2024-04-25T10:09:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.770+0000 7f518cbe8740 20 client.7620 trim_cache size 0 max 16384 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.786+0000 7f518cbe8740 1 Processor -- start 2024-04-25T10:09:06.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.786+0000 7f518cbe8740 1 -- start start 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.787+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 0x557447ffdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.787+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744802d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.787+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x55744802dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.787+0000 7f518cbe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51841ff400 con 0x557447ff5910 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.787+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840b0cf0 con 0x55744801f7a0 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.787+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51841f4630 con 0x557447ffeeb0 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 0x557447ffdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 0x557447ffdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36198/0 (socket says 172.21.15.5:36198) 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 -- 172.21.15.5:0/3824139956 learned_addr learned my addr 172.21.15.5:0/3824139956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 -- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 msgr2=0x55744802d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5183400640 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x55744802dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744802d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 -- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 msgr2=0x55744802dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x55744802dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 -- 172.21.15.5:0/3824139956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557447d196b0 con 0x557447ff5910 2024-04-25T10:09:06.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.788+0000 7f5182a00640 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 0x557447ffdfd0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x55744802d020 tx=0x7f516c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d59178e92cf5c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f5172a00640 1 -- 172.21.15.5:0/3824139956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c02c070 con 0x557447ff5910 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f5172a00640 1 -- 172.21.15.5:0/3824139956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c014ce0 con 0x557447ff5910 2024-04-25T10:09:06.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f5172a00640 1 -- 172.21.15.5:0/3824139956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c026350 con 0x557447ff5910 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 -- 172.21.15.5:0/3824139956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 msgr2=0x557447ffdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 0x557447ffdfd0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x55744802d020 tx=0x7f516c015000 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 -- 172.21.15.5:0/3824139956 shutdown_connections 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55744801f7a0 0x55744802dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744802d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 --2- 172.21.15.5:0/3824139956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557447ff5910 0x557447ffdfd0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 -- 172.21.15.5:0/3824139956 >> 172.21.15.5:0/3824139956 conn(0x557447fd6b30 msgr2=0x557447fd1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 -- 172.21.15.5:0/3824139956 shutdown_connections 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.789+0000 7f518cbe8740 1 -- 172.21.15.5:0/3824139956 wait complete. 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.790+0000 7f518cbe8740 1 -- start start 2024-04-25T10:09:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.790+0000 7f518cbe8740 1 Processor -- start 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ff5910 0x557448086700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744801e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574480053a0 con 0x55744801f7a0 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557447ffc570 con 0x557447ff5910 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557447ff3ee0 con 0x557447ffeeb0 2024-04-25T10:09:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.791+0000 7f518cbe8740 10 client.0 _ll_register_callbacks cb 0x7f5174007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36200/0 (socket says 172.21.15.5:36200) 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 -- 172.21.15.5:0/1812703408 learned_addr learned my addr 172.21.15.5:0/1812703408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5182a00640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ff5910 0x557448086700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5182000640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744801e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 -- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 msgr2=0x55744801e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744801e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 -- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ff5910 msgr2=0x557448086700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ff5910 0x557448086700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 -- 172.21.15.5:0/1812703408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55744808a400 con 0x55744801f7a0 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5182000640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744801e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5183400640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ef00 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f517c075540 tx=0x7f517c078b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fa38ec2d75e693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.792+0000 7f5167e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f5167e00640 1 -- 172.21.15.5:0/1812703408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0a7060 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f5167e00640 1 -- 172.21.15.5:0/1812703408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f517c081560 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f516c014ce0 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f5167e00640 1 -- 172.21.15.5:0/1812703408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517c095050 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 10 client.7626 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55744802f8b0 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f5167e00640 1 -- 172.21.15.5:0/1812703408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c014ce0 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f5167e00640 1 -- 172.21.15.5:0/1812703408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f517c0aa040 con 0x55744801f7a0 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 10 client.7626 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 10 client.7626 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 20 client.7626 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x557447fd7860 0x557447fffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 4 client.7626 mds_command: new command op to 7488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.793+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 --> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557447f41eb0 con 0x557447fd7860 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.794+0000 7f5182a00640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x557447fd7860 0x557447fffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.794+0000 7f5182a00640 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x557447fd7860 0x557447fffc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426c6d67af54e126 server_cookie=e8a92bf2d2678c62 in_seq=0 out_seq=0 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.794+0000 7f5167e00640 10 client.7626 ms_handle_connect on v2:172.21.15.5:6832/3627376661 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.794+0000 7f5167e00640 1 -- 172.21.15.5:0/1812703408 <== mds.0 v2:172.21.15.5:6832/3627376661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x557447f41eb0 con 0x557447fd7860 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.794+0000 7f5167e00640 10 client.7626 handle_command_reply: tid=0 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 client.7626 shutdown 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 msgr2=0x55744801ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ef00 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f517c075540 tx=0x7f517c078b00 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 shutdown_connections 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55744801f7a0 0x55744801ef00 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557447ffeeb0 0x55744801e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.5:6832/3627376661,v1:172.21.15.5:6834/3627376661] conn(0x557447fd7860 0x557447fffc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 --2- 172.21.15.5:0/1812703408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557447ff5910 0x557448086700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 >> 172.21.15.5:0/1812703408 conn(0x557447fd6b30 msgr2=0x557447ffdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f5167e00640 0 client.7626 ms_handle_reset on v2:172.21.15.5:6832/3627376661 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 shutdown_connections 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 1 -- 172.21.15.5:0/1812703408 wait complete. 2024-04-25T10:09:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.795+0000 7f518cbe8740 20 client.7626 trim_cache size 0 max 16384 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7488, 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.7340244619999998, 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 62, 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T10:09:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 12.13055797 2024-04-25T10:09:06.808 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f516006b4d0 msgr2=0x7f516006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f516006b4d0 0x7f516006d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f516c006fd0 tx=0x7f516c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 msgr2=0x7f518415a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 0x7f518415a3b0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f517c02a270 tx=0x7f517c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 shutdown_connections 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51840b1ce0 0x7f518415a3b0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5184141da0 0x7f51841560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f516006b4d0 0x7f516006d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 --2- 172.21.15.5:0/2006948931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f518406e640 0x7f51841590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 >> 172.21.15.5:0/2006948931 conn(0x7f5184066f20 msgr2=0x7f51840b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 shutdown_connections 2024-04-25T10:09:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:06.796+0000 7f5183e00640 1 -- 172.21.15.5:0/2006948931 wait complete. 2024-04-25T10:09:06.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:07.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.015+0000 7f89ed400640 1 Processor -- start 2024-04-25T10:09:07.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.015+0000 7f89ed400640 1 -- start start 2024-04-25T10:09:07.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.017+0000 7f89ed400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e8141da0 0x7f89e806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.017+0000 7f89ed400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89e815d510 0x7f89e80cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.017+0000 7f89ed400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e8071d30 con 0x7f89e806e640 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.017+0000 7f89ed400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e8142180 con 0x7f89e815d510 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.017+0000 7f89ed400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e806d620 con 0x7f89e8141da0 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89e815d510 0x7f89e80cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f89e815d510 0x7f89e80cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42060/0 (socket says 172.21.15.5:42060) 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 -- 172.21.15.5:0/2186020031 learned_addr learned my addr 172.21.15.5:0/2186020031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89e7e00640 1 --2- 172.21.15.5:0/2186020031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e8141da0 0x7f89e806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 -- 172.21.15.5:0/2186020031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e8141da0 msgr2=0x7f89e806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 --2- 172.21.15.5:0/2186020031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e8141da0 0x7f89e806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 -- 172.21.15.5:0/2186020031 >> v1:172.21.15.80:6789/0 conn(0x7f89e806e640 legacy=0x7f89e815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.018+0000 7f89eca00640 1 -- 172.21.15.5:0/2186020031 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e806d900 con 0x7f89e815d510 2024-04-25T10:09:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.019+0000 7f89eca00640 1 --2- 172.21.15.5:0/2186020031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e815d510 0x7f89e80cd300 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e50 tx=0x7f89e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0f8dcf1c12fb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.020+0000 7f89e6a00640 1 -- 172.21.15.5:0/2186020031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0036070 con 0x7f89e815d510 2024-04-25T10:09:07.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.020+0000 7f89e6a00640 1 -- 172.21.15.5:0/2186020031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0031600 con 0x7f89e815d510 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.020+0000 7f89e6a00640 1 -- 172.21.15.5:0/2186020031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00319d0 con 0x7f89e815d510 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.020+0000 7f89ed400640 1 -- 172.21.15.5:0/2186020031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e815d510 msgr2=0x7f89e80cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.020+0000 7f89ed400640 1 --2- 172.21.15.5:0/2186020031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e815d510 0x7f89e80cd300 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e50 tx=0x7f89e002a710 comp rx=0 tx=0).stop 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.021+0000 7f89ed400640 1 -- 172.21.15.5:0/2186020031 shutdown_connections 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.021+0000 7f89ed400640 1 --2- 172.21.15.5:0/2186020031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e815d510 0x7f89e80cd300 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.021+0000 7f89ed400640 1 --2- 172.21.15.5:0/2186020031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e8141da0 0x7f89e806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.021+0000 7f89ed400640 1 -- 172.21.15.5:0/2186020031 >> 172.21.15.5:0/2186020031 conn(0x7f89e8066f20 msgr2=0x7f89e806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.021+0000 7f89ed400640 1 -- 172.21.15.5:0/2186020031 shutdown_connections 2024-04-25T10:09:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.021+0000 7f89ed400640 1 -- 172.21.15.5:0/2186020031 wait complete. 2024-04-25T10:09:07.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.022+0000 7f89ed400640 1 Processor -- start 2024-04-25T10:09:07.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.023+0000 7f89ed400640 1 -- start start 2024-04-25T10:09:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.023+0000 7f89ed400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e806e640 0x7f89e81f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.023+0000 7f89ed400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e8141da0 0x7f89e813f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.023+0000 7f89ed400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 0x7f89e80aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89ed400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80cd9c0 con 0x7f89e80a9d10 2024-04-25T10:09:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89ed400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80cd840 con 0x7f89e8141da0 2024-04-25T10:09:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89ed400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80cdb40 con 0x7f89e806e640 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 0x7f89e80aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 0x7f89e80aa1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42062/0 (socket says 172.21.15.5:42062) 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 -- 172.21.15.5:0/1104819548 learned_addr learned my addr 172.21.15.5:0/1104819548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 -- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e806e640 msgr2=0x7f89e81f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e806e640 0x7f89e81f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 -- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e8141da0 msgr2=0x7f89e813f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89e7400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e8141da0 0x7f89e813f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e8141da0 0x7f89e813f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89eca00640 1 -- 172.21.15.5:0/1104819548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e81f7980 con 0x7f89e80a9d10 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.024+0000 7f89e7400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e8141da0 0x7f89e813f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:07.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.025+0000 7f89eca00640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 0x7f89e80aa1d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e50 tx=0x7f89e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.025+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0041020 con 0x7f89e80a9d10 2024-04-25T10:09:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.025+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0007800 con 0x7f89e80a9d10 2024-04-25T10:09:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.025+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0036040 con 0x7f89e80a9d10 2024-04-25T10:09:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.025+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e80b26c0 con 0x7f89e80a9d10 2024-04-25T10:09:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.025+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e80b2c80 con 0x7f89e80a9d10 2024-04-25T10:09:07.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.026+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e8068f60 con 0x7f89e80a9d10 2024-04-25T10:09:07.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.027+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89e0044020 con 0x7f89e80a9d10 2024-04-25T10:09:07.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.028+0000 7f89dfe00640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c806b500 0x7f89c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:07.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.029+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89e00ab0b0 con 0x7f89e80a9d10 2024-04-25T10:09:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.029+0000 7f89e7e00640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c806b500 0x7f89c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.030+0000 7f89e7e00640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c806b500 0x7f89c806d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098f0 tx=0x7f89d8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:07.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.033+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89e004b030 con 0x7f89e80a9d10 2024-04-25T10:09:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.275+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f89e80b1d40 con 0x7f89e80a9d10 2024-04-25T10:09:07.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.276+0000 7f89dfe00640 1 -- 172.21.15.5:0/1104819548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89e0079d90 con 0x7f89e80a9d10 2024-04-25T10:09:07.283 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c806b500 msgr2=0x7f89c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c806b500 0x7f89c806d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f89d80098f0 tx=0x7f89d8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 msgr2=0x7f89e80aa1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 0x7f89e80aa1d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009e50 tx=0x7f89e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 shutdown_connections 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e80a9d10 0x7f89e80aa1d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89c806b500 0x7f89c806d9c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e8141da0 0x7f89e813f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 --2- 172.21.15.5:0/1104819548 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e806e640 0x7f89e81f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 >> 172.21.15.5:0/1104819548 conn(0x7f89e8066f20 msgr2=0x7f89e8070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 shutdown_connections 2024-04-25T10:09:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:07.279+0000 7f89ed400640 1 -- 172.21.15.5:0/1104819548 wait complete. 2024-04-25T10:09:07.304 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:07.305 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:07.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.305 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:09:07.354 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42178sec preferred_lft 42178sec 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:07.355 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:09:07.357 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:07.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.358 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:07.440 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:09:07.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.445 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:09:07.496 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:09:07.497 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:07.497 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:09:07.497 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:09:07.497 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:07.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.497 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:09:07.497 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:09:07.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:09:07.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:09:07.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:09:07.498 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:09:07.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.651 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:09:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:07.704 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:07.705 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:07.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:09:07.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:07.793 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:07.849 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:07.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:07.850 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:07.907 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:09:07.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:07.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:08.049 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:08.049 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:08.049 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:09:08.071 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:09:08.072 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:08.097 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:08.097 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:08.097 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:09:08.097 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:09:08.098 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:08.098 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:09:08.098 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:09:08.098 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:09:08.100 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:09:08.100 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:09:08.100 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:09:08.100 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:09:08.100 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:09:08.104 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:09:08.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:08.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:08.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:08.178 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:09:08.234 INFO:teuthology.orchestra.run.smithi005.stdout:client7635 2024-04-25T10:09:08.234 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:09:08.234 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:08.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:08.288 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T10:09:08.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:08.356 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T10:09:08.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:08.416 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T10:09:08.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:08.480 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T10:09:08.547 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:09:08.548 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:09:08.548 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:09:08.548 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:09:08.548 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T10:09:08.548 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:09:08.687 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:09:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:08.700 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:09:08.700 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:08.801 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:09:08.801 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:08.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.064+0000 7f65d1200640 1 Processor -- start 2024-04-25T10:09:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.064+0000 7f65d1200640 1 -- start start 2024-04-25T10:09:09.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.066+0000 7f65d1200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f65cc143d30 0x7f65cc15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.066+0000 7f65d1200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f65cc0b3ae0 0x7f65cc0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.066+0000 7f65d1200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc0b4880 con 0x7f65cc0ce790 2024-04-25T10:09:09.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.066+0000 7f65d1200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0b45a0 con 0x7f65cc0b3ae0 2024-04-25T10:09:09.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.066+0000 7f65d1200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc15e2c0 con 0x7f65cc143d30 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f65cc0b3ae0 0x7f65cc0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f65cc0b3ae0 0x7f65cc0b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42072/0 (socket says 172.21.15.5:42072) 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 -- 172.21.15.5:0/2152143529 learned_addr learned my addr 172.21.15.5:0/2152143529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65cb400640 1 --2- 172.21.15.5:0/2152143529 >> v2:172.21.15.80:3300/0 conn(0x7f65cc143d30 0x7f65cc15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 -- 172.21.15.5:0/2152143529 >> v2:172.21.15.80:3300/0 conn(0x7f65cc143d30 msgr2=0x7f65cc15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 --2- 172.21.15.5:0/2152143529 >> v2:172.21.15.80:3300/0 conn(0x7f65cc143d30 0x7f65cc15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 -- 172.21.15.5:0/2152143529 >> v1:172.21.15.80:6789/0 conn(0x7f65cc0ce790 legacy=0x7f65cc15abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.067+0000 7f65c3e00640 1 -- 172.21.15.5:0/2152143529 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc15e5a0 con 0x7f65cc0b3ae0 2024-04-25T10:09:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.068+0000 7f65c3e00640 1 --2- 172.21.15.5:0/2152143529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0b3ae0 0x7f65cc0b3f90 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f65cc072100 tx=0x7f65b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab1695e803741a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.069+0000 7f65caa00640 1 -- 172.21.15.5:0/2152143529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4037070 con 0x7f65cc0b3ae0 2024-04-25T10:09:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.069+0000 7f65caa00640 1 -- 172.21.15.5:0/2152143529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b4032440 con 0x7f65cc0b3ae0 2024-04-25T10:09:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.069+0000 7f65caa00640 1 -- 172.21.15.5:0/2152143529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4032810 con 0x7f65cc0b3ae0 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.069+0000 7f65d1200640 1 -- 172.21.15.5:0/2152143529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0b3ae0 msgr2=0x7f65cc0b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.069+0000 7f65d1200640 1 --2- 172.21.15.5:0/2152143529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0b3ae0 0x7f65cc0b3f90 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f65cc072100 tx=0x7f65b402a230 comp rx=0 tx=0).stop 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.070+0000 7f65d1200640 1 -- 172.21.15.5:0/2152143529 shutdown_connections 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.070+0000 7f65d1200640 1 --2- 172.21.15.5:0/2152143529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0b3ae0 0x7f65cc0b3f90 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.070+0000 7f65d1200640 1 --2- 172.21.15.5:0/2152143529 >> v2:172.21.15.80:3300/0 conn(0x7f65cc143d30 0x7f65cc15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.070+0000 7f65d1200640 1 -- 172.21.15.5:0/2152143529 >> 172.21.15.5:0/2152143529 conn(0x7f65cc066f20 msgr2=0x7f65cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.070+0000 7f65d1200640 1 -- 172.21.15.5:0/2152143529 shutdown_connections 2024-04-25T10:09:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.070+0000 7f65d1200640 1 -- 172.21.15.5:0/2152143529 wait complete. 2024-04-25T10:09:09.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.072+0000 7f65d1200640 1 Processor -- start 2024-04-25T10:09:09.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.072+0000 7f65d1200640 1 -- start start 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65d1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65cc0b3ae0 0x7f65cc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 0x7f65cc1e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65d1200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65cc143d30 0x7f65cc0ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65d1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc144290 con 0x7f65cc0ce790 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65d1200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc144110 con 0x7f65cc143d30 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65d1200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc06e430 con 0x7f65cc0b3ae0 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 0x7f65cc1e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.073+0000 7f65c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 0x7f65cc1e66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42078/0 (socket says 172.21.15.5:42078) 2024-04-25T10:09:09.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 -- 172.21.15.5:0/2327759778 learned_addr learned my addr 172.21.15.5:0/2327759778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65cb400640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65cc0b3ae0 0x7f65cc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 -- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65cc0b3ae0 msgr2=0x7f65cc157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65cc0b3ae0 0x7f65cc157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 -- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65cc143d30 msgr2=0x7f65cc0ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65cbe00640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65cc143d30 0x7f65cc0ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65cc143d30 0x7f65cc0ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 -- 172.21.15.5:0/2327759778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc1e7080 con 0x7f65cc0ce790 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.074+0000 7f65c3e00640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 0x7f65cc1e66a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.075+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4041020 con 0x7f65cc0ce790 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.075+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b40306c0 con 0x7f65cc0ce790 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.075+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b4037040 con 0x7f65cc0ce790 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.075+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc1e7360 con 0x7f65cc0ce790 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.075+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc0a8b70 con 0x7f65cc0ce790 2024-04-25T10:09:09.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.076+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc068f60 con 0x7f65cc0ce790 2024-04-25T10:09:09.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.077+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65b4044020 con 0x7f65cc0ce790 2024-04-25T10:09:09.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.078+0000 7f65c3400640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f65a806b4d0 0x7f65a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.078+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65b40aac60 con 0x7f65cc0ce790 2024-04-25T10:09:09.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.079+0000 7f65cb400640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f65a806b4d0 0x7f65a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.080+0000 7f65cb400640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f65a806b4d0 0x7f65a806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006fd0 tx=0x7f65bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.083+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65b404b030 con 0x7f65cc0ce790 2024-04-25T10:09:09.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.284+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65cc0cd4a0 con 0x7f65cc0ce790 2024-04-25T10:09:09.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.287+0000 7f65c3400640 1 -- 172.21.15.5:0/2327759778 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16825 (secure 0 0 0) 0x7f65b40798b0 con 0x7f65cc0ce790 2024-04-25T10:09:09.295 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:09.295 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:01.074532+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.290+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f65a806b4d0 msgr2=0x7f65a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.290+0000 7f65d1200640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f65a806b4d0 0x7f65a806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006fd0 tx=0x7f65bc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.290+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 msgr2=0x7f65cc1e66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.290+0000 7f65d1200640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 0x7f65cc1e66a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 shutdown_connections 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f65a806b4d0 0x7f65a806d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65cc143d30 0x7f65cc0ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65cc0ce790 0x7f65cc1e66a0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 --2- 172.21.15.5:0/2327759778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65cc0b3ae0 0x7f65cc157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 >> 172.21.15.5:0/2327759778 conn(0x7f65cc066f20 msgr2=0x7f65cc06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 shutdown_connections 2024-04-25T10:09:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.291+0000 7f65d1200640 1 -- 172.21.15.5:0/2327759778 wait complete. 2024-04-25T10:09:09.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:57.371047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:58.551736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:09.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.524+0000 7faef1800640 1 Processor -- start 2024-04-25T10:09:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.524+0000 7faef1800640 1 -- start start 2024-04-25T10:09:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.526+0000 7faef1800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 0x7faeec1651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.526+0000 7faef1800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec072010 con 0x7faeec06e840 2024-04-25T10:09:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.526+0000 7faef1800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec0ce570 con 0x7faeec0cc840 2024-04-25T10:09:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.526+0000 7faef1800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec15b910 con 0x7faeec15b500 2024-04-25T10:09:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.526+0000 7faee3400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faeec06e840 0x7faeec15adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50476/0 (socket says 172.21.15.5:50476) 2024-04-25T10:09:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faee3400640 1 -- 172.21.15.5:0/4230857474 learned_addr learned my addr 172.21.15.5:0/4230857474 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faef0e00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 0x7faeec1651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000293109 0 0) 0x7faeec072010 con 0x7faeec06e840 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed0003520 con 0x7faeec06e840 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480409840 0 0) 0x7faed0003520 con 0x7faeec06e840 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 msgr2=0x7faeec1651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 0x7faeec1651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 >> v1:172.21.15.80:6789/0 conn(0x7faeec0cc840 legacy=0x7faeec06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0ce850 con 0x7faeec06e840 2024-04-25T10:09:09.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.527+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7faed4001690 con 0x7faeec06e840 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 >> v1:172.21.15.5:6789/0 conn(0x7faeec06e840 legacy=0x7faeec15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faed0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 0x7faed0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faed0004d40 0x7faed00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec15b910 con 0x7faed00039e0 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed000dce0 con 0x7faed0004d40 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed000dee0 con 0x7faeec15b500 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeebe00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faed0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faee3400640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 0x7faed0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faef0e00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faed0004d40 0x7faed00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeebe00640 1 -- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 msgr2=0x7faed0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeebe00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec15b500 0x7faed0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeebe00640 1 -- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faed0004d40 msgr2=0x7faed00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeebe00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faed0004d40 0x7faed00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faeebe00640 1 -- 172.21.15.5:0/4230857474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0003520 con 0x7faed00039e0 2024-04-25T10:09:09.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.528+0000 7faef0e00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faed0004d40 0x7faed00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:09.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.529+0000 7faeebe00640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faed0003e00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7faedc001720 tx=0x7faed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5400131ed1cfc0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.530+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc001370 con 0x7faed00039e0 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.530+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc0052d0 con 0x7faed00039e0 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.530+0000 7faeeb400640 1 -- 172.21.15.5:0/4230857474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc02abc0 con 0x7faed00039e0 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.530+0000 7faef1800640 1 -- 172.21.15.5:0/4230857474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 msgr2=0x7faed0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.530+0000 7faef1800640 1 --2- 172.21.15.5:0/4230857474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faed0003e00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7faedc001720 tx=0x7faed0005830 comp rx=0 tx=0).stop 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.530+0000 7faef0e00640 1 -- 172.21.15.5:0/4230857474 reap_dead start 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.531+0000 7faef1800640 1 -- 172.21.15.5:0/4230857474 shutdown_connections 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.531+0000 7faef1800640 1 -- 172.21.15.5:0/4230857474 >> 172.21.15.5:0/4230857474 conn(0x7faeec066f20 msgr2=0x7faeec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.531+0000 7faef1800640 1 -- 172.21.15.5:0/4230857474 shutdown_connections 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.531+0000 7faef1800640 1 -- 172.21.15.5:0/4230857474 wait complete. 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.532+0000 7faef1800640 1 Processor -- start 2024-04-25T10:09:09.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 -- start start 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faeec14bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec0b0970 0x7faeec1521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec152700 0x7faeec14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0b2d20 con 0x7faed00039e0 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0cee10 con 0x7faeec152700 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.533+0000 7faef1800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0ce1d0 con 0x7faeec0b0970 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faeec14bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faeec14bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42096/0 (socket says 172.21.15.5:42096) 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 -- 172.21.15.5:0/3823837576 learned_addr learned my addr 172.21.15.5:0/3823837576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 -- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec0b0970 msgr2=0x7faeec1521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faee3400640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec0b0970 0x7faeec1521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faef0e00640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec152700 0x7faeec14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec0b0970 0x7faeec1521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 -- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec152700 msgr2=0x7faeec14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec152700 0x7faeec14d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.534+0000 7faeebe00640 1 -- 172.21.15.5:0/3823837576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec14c1b0 con 0x7faed00039e0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.535+0000 7faeebe00640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faeec14bc70 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7faedc008fd0 tx=0x7faedc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.535+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc042070 con 0x7faed00039e0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.535+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc005480 con 0x7faed00039e0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.535+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc037040 con 0x7faed00039e0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.535+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec14ece0 con 0x7faed00039e0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.535+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec14f2a0 con 0x7faed00039e0 2024-04-25T10:09:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.536+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec15de00 con 0x7faed00039e0 2024-04-25T10:09:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.537+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faedc045070 con 0x7faed00039e0 2024-04-25T10:09:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.539+0000 7faee8c00640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.539+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faedc0a9c20 con 0x7faed00039e0 2024-04-25T10:09:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.539+0000 7faee3400640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.541+0000 7faee3400640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faed4001220 tx=0x7faed4001130 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.543+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faeec14c1b0 con 0x7faed00039e0 2024-04-25T10:09:09.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.743+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faeec068f60 con 0x7faed00039e0 2024-04-25T10:09:09.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.744+0000 7faee8c00640 1 -- 172.21.15.5:0/3823837576 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7faedc0787e0 con 0x7faed00039e0 2024-04-25T10:09:09.751 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:09:09.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 msgr2=0x7faec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faed4001220 tx=0x7faed4001130 comp rx=0 tx=0).stop 2024-04-25T10:09:09.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 msgr2=0x7faeec14bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faeec14bc70 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7faedc008fd0 tx=0x7faedc001120 comp rx=0 tx=0).stop 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 shutdown_connections 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec152700 0x7faeec14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec0b0970 0x7faeec1521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 --2- 172.21.15.5:0/3823837576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed00039e0 0x7faeec14bc70 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 >> 172.21.15.5:0/3823837576 conn(0x7faeec066f20 msgr2=0x7faeec15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.746+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 shutdown_connections 2024-04-25T10:09:09.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.747+0000 7faef1800640 1 -- 172.21.15.5:0/3823837576 wait complete. 2024-04-25T10:09:09.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.979+0000 7fc498e00640 1 Processor -- start 2024-04-25T10:09:09.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.979+0000 7fc498e00640 1 -- start start 2024-04-25T10:09:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.981+0000 7fc498e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc49406d130 0x7fc494071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.981+0000 7fc498e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc494071670 0x7fc49415c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.981+0000 7fc498e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4941453c0 con 0x7fc494144fe0 2024-04-25T10:09:09.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.981+0000 7fc498e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940a8560 con 0x7fc49406d130 2024-04-25T10:09:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.981+0000 7fc498e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49406d6a0 con 0x7fc494071670 2024-04-25T10:09:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.981+0000 7fc491400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc49406d130 0x7fc494071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc491400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc49406d130 0x7fc494071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42100/0 (socket says 172.21.15.5:42100) 2024-04-25T10:09:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc491400640 1 -- 172.21.15.5:0/3186019423 learned_addr learned my addr 172.21.15.5:0/3186019423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc492800640 1 --2- 172.21.15.5:0/3186019423 >> v2:172.21.15.80:3300/0 conn(0x7fc494071670 0x7fc49415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc491400640 1 -- 172.21.15.5:0/3186019423 >> v2:172.21.15.80:3300/0 conn(0x7fc494071670 msgr2=0x7fc49415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc491400640 1 --2- 172.21.15.5:0/3186019423 >> v2:172.21.15.80:3300/0 conn(0x7fc494071670 0x7fc49415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc491400640 1 -- 172.21.15.5:0/3186019423 >> v1:172.21.15.80:6789/0 conn(0x7fc494144fe0 legacy=0x7fc49406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.982+0000 7fc491400640 1 -- 172.21.15.5:0/3186019423 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49406d980 con 0x7fc49406d130 2024-04-25T10:09:09.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.984+0000 7fc491400640 1 --2- 172.21.15.5:0/3186019423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc49406d130 0x7fc494071130 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc49415bd10 tx=0x7fc48002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215dea2033f4b2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc490a00640 1 -- 172.21.15.5:0/3186019423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480037070 con 0x7fc49406d130 2024-04-25T10:09:09.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc490a00640 1 -- 172.21.15.5:0/3186019423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc480032440 con 0x7fc49406d130 2024-04-25T10:09:09.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc490a00640 1 -- 172.21.15.5:0/3186019423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480032810 con 0x7fc49406d130 2024-04-25T10:09:09.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 -- 172.21.15.5:0/3186019423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc49406d130 msgr2=0x7fc494071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 --2- 172.21.15.5:0/3186019423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc49406d130 0x7fc494071130 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fc49415bd10 tx=0x7fc48002a270 comp rx=0 tx=0).stop 2024-04-25T10:09:09.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 -- 172.21.15.5:0/3186019423 shutdown_connections 2024-04-25T10:09:09.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 --2- 172.21.15.5:0/3186019423 >> v2:172.21.15.80:3300/0 conn(0x7fc494071670 0x7fc49415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 --2- 172.21.15.5:0/3186019423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc49406d130 0x7fc494071130 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 -- 172.21.15.5:0/3186019423 >> 172.21.15.5:0/3186019423 conn(0x7fc49406ab60 msgr2=0x7fc49406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:09.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.985+0000 7fc498e00640 1 -- 172.21.15.5:0/3186019423 shutdown_connections 2024-04-25T10:09:09.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.986+0000 7fc498e00640 1 -- 172.21.15.5:0/3186019423 wait complete. 2024-04-25T10:09:09.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.987+0000 7fc498e00640 1 Processor -- start 2024-04-25T10:09:09.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.987+0000 7fc498e00640 1 -- start start 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.988+0000 7fc498e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc49406d130 0x7fc49413cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.988+0000 7fc498e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 0x7fc49413eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.988+0000 7fc498e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc494144fe0 0x7fc4941417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.988+0000 7fc498e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49415be50 con 0x7fc494071670 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.988+0000 7fc498e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49406df40 con 0x7fc49406d130 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.988+0000 7fc498e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49415c8b0 con 0x7fc494144fe0 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 0x7fc49413eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 0x7fc49413eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42102/0 (socket says 172.21.15.5:42102) 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 -- 172.21.15.5:0/170715587 learned_addr learned my addr 172.21.15.5:0/170715587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 -- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc494144fe0 msgr2=0x7fc4941417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc494144fe0 0x7fc4941417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 -- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc49406d130 msgr2=0x7fc49413cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc49406d130 0x7fc49413cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc49406d130 0x7fc49413cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491400640 1 -- 172.21.15.5:0/170715587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494140320 con 0x7fc494071670 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.989+0000 7fc491e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc49406d130 0x7fc49413cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.990+0000 7fc491400640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 0x7fc49413eb80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc48002a2b0 tx=0x7fc4800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.990+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480041030 con 0x7fc494071670 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.990+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48003d420 con 0x7fc494071670 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.990+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480037040 con 0x7fc494071670 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.990+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49413f280 con 0x7fc494071670 2024-04-25T10:09:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.990+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49413f840 con 0x7fc494071670 2024-04-25T10:09:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.991+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc494068f60 con 0x7fc494071670 2024-04-25T10:09:09.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.992+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc480044070 con 0x7fc494071670 2024-04-25T10:09:10.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.994+0000 7fc48a000640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.994+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4800aa9f0 con 0x7fc494071670 2024-04-25T10:09:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.994+0000 7fc491e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.996+0000 7fc491e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc47406b4d0 0x7fc47406d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc4840015e0 tx=0x7fc484009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:09.998+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc480079760 con 0x7fc494071670 2024-04-25T10:09:10.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.208+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc49415b560 con 0x7fc494071670 2024-04-25T10:09:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.208+0000 7fc48a000640 1 -- 172.21.15.5:0/170715587 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc480079060 con 0x7fc494071670 2024-04-25T10:09:10.215 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc47406b4d0 msgr2=0x7fc47406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc47406b4d0 0x7fc47406d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc4840015e0 tx=0x7fc484009040 comp rx=0 tx=0).stop 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 msgr2=0x7fc49413eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 0x7fc49413eb80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fc48002a2b0 tx=0x7fc4800046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 shutdown_connections 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc494144fe0 0x7fc4941417f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc494071670 0x7fc49413eb80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc47406b4d0 0x7fc47406d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 --2- 172.21.15.5:0/170715587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc49406d130 0x7fc49413cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 >> 172.21.15.5:0/170715587 conn(0x7fc49406ab60 msgr2=0x7fc49413e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 shutdown_connections 2024-04-25T10:09:10.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.212+0000 7fc498e00640 1 -- 172.21.15.5:0/170715587 wait complete. 2024-04-25T10:09:10.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:10.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.439+0000 7f5d28400640 1 Processor -- start 2024-04-25T10:09:10.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.440+0000 7f5d28400640 1 -- start start 2024-04-25T10:09:10.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.441+0000 7f5d28400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d2006d130 0x7f5d20071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.441+0000 7f5d28400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d20071670 0x7f5d2015c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.441+0000 7f5d28400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d201453c0 con 0x7f5d20144fe0 2024-04-25T10:09:10.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.441+0000 7f5d28400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d200a8560 con 0x7f5d2006d130 2024-04-25T10:09:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.441+0000 7f5d28400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2006d6a0 con 0x7f5d20071670 2024-04-25T10:09:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d2006d130 0x7f5d20071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5d20144fe0 0x7f5d2006e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50490/0 (socket says 172.21.15.5:50490) 2024-04-25T10:09:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25a00640 1 -- 172.21.15.5:0/1859202925 learned_addr learned my addr 172.21.15.5:0/1859202925 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d26400640 1 --2- 172.21.15.5:0/1859202925 >> v2:172.21.15.80:3300/0 conn(0x7f5d20071670 0x7f5d2015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25000640 1 -- 172.21.15.5:0/1859202925 >> v2:172.21.15.80:3300/0 conn(0x7f5d20071670 msgr2=0x7f5d2015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25000640 1 --2- 172.21.15.5:0/1859202925 >> v2:172.21.15.80:3300/0 conn(0x7f5d20071670 0x7f5d2015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25000640 1 -- 172.21.15.5:0/1859202925 >> v1:172.21.15.5:6789/0 conn(0x7f5d20144fe0 legacy=0x7f5d2006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d25000640 1 -- 172.21.15.5:0/1859202925 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2006d980 con 0x7f5d2006d130 2024-04-25T10:09:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.442+0000 7f5d1fe00640 1 -- 172.21.15.5:0/1859202925 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232664232 0 0) 0x7f5d201453c0 con 0x7f5d20144fe0 2024-04-25T10:09:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.444+0000 7f5d25000640 1 --2- 172.21.15.5:0/1859202925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2006d130 0x7f5d20071130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2015bd10 tx=0x7f5d0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89acbd8eb0f10f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.444+0000 7f5d1fe00640 1 -- 172.21.15.5:0/1859202925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c037070 con 0x7f5d2006d130 2024-04-25T10:09:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.444+0000 7f5d1fe00640 1 -- 172.21.15.5:0/1859202925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c032440 con 0x7f5d2006d130 2024-04-25T10:09:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.444+0000 7f5d1fe00640 1 -- 172.21.15.5:0/1859202925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c032810 con 0x7f5d2006d130 2024-04-25T10:09:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.444+0000 7f5d28400640 1 -- 172.21.15.5:0/1859202925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2006d130 msgr2=0x7f5d20071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.444+0000 7f5d28400640 1 --2- 172.21.15.5:0/1859202925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2006d130 0x7f5d20071130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2015bd10 tx=0x7f5d0c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.445+0000 7f5d28400640 1 -- 172.21.15.5:0/1859202925 shutdown_connections 2024-04-25T10:09:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.445+0000 7f5d28400640 1 --2- 172.21.15.5:0/1859202925 >> v2:172.21.15.80:3300/0 conn(0x7f5d20071670 0x7f5d2015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.445+0000 7f5d28400640 1 --2- 172.21.15.5:0/1859202925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d2006d130 0x7f5d20071130 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.445+0000 7f5d28400640 1 -- 172.21.15.5:0/1859202925 >> 172.21.15.5:0/1859202925 conn(0x7f5d2006ab60 msgr2=0x7f5d2006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.445+0000 7f5d28400640 1 -- 172.21.15.5:0/1859202925 shutdown_connections 2024-04-25T10:09:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.445+0000 7f5d28400640 1 -- 172.21.15.5:0/1859202925 wait complete. 2024-04-25T10:09:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.447+0000 7f5d28400640 1 Processor -- start 2024-04-25T10:09:10.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.447+0000 7f5d28400640 1 -- start start 2024-04-25T10:09:10.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d28400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2006d130 0x7f5d2015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d28400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 0x7f5d201395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d28400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d20144fe0 0x7f5d2013afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d28400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2015be50 con 0x7f5d20071670 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d28400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2006df40 con 0x7f5d2006d130 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d28400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2015c8b0 con 0x7f5d20144fe0 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d25000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 0x7f5d201395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d25000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 0x7f5d201395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42114/0 (socket says 172.21.15.5:42114) 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d25000640 1 -- 172.21.15.5:0/4088287531 learned_addr learned my addr 172.21.15.5:0/4088287531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.448+0000 7f5d25a00640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2006d130 0x7f5d2015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25000640 1 -- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d20144fe0 msgr2=0x7f5d2013afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d26400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d20144fe0 0x7f5d2013afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25000640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d20144fe0 0x7f5d2013afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25000640 1 -- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2006d130 msgr2=0x7f5d2015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25000640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2006d130 0x7f5d2015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25000640 1 -- 172.21.15.5:0/4088287531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2015b450 con 0x7f5d20071670 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25a00640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2006d130 0x7f5d2015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d26400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d20144fe0 0x7f5d2013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d25000640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 0x7f5d201395d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c000c00 tx=0x7f5d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c037050 con 0x7f5d20071670 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c038040 con 0x7f5d20071670 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d20139b10 con 0x7f5d20071670 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.449+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c004470 con 0x7f5d20071670 2024-04-25T10:09:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.450+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2013c150 con 0x7f5d20071670 2024-04-25T10:09:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.451+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d20068f60 con 0x7f5d20071670 2024-04-25T10:09:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.452+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d0c030900 con 0x7f5d20071670 2024-04-25T10:09:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.453+0000 7f5d1d600640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0006b4d0 0x7f5d0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.453+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d0c0aaf50 con 0x7f5d20071670 2024-04-25T10:09:10.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.453+0000 7f5d25a00640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0006b4d0 0x7f5d0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.454+0000 7f5d25a00640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0006b4d0 0x7f5d0006d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d14007fd0 tx=0x7f5d14009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.457+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d0c079d50 con 0x7f5d20071670 2024-04-25T10:09:10.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.661+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d2013d1f0 con 0x7f5d20071670 2024-04-25T10:09:10.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.662+0000 7f5d1d600640 1 -- 172.21.15.5:0/4088287531 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d0c079650 con 0x7f5d20071670 2024-04-25T10:09:10.671 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0006b4d0 msgr2=0x7f5d0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0006b4d0 0x7f5d0006d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d14007fd0 tx=0x7f5d14009040 comp rx=0 tx=0).stop 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 msgr2=0x7f5d201395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 0x7f5d201395d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c000c00 tx=0x7f5d0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 shutdown_connections 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d20144fe0 0x7f5d2013afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d20071670 0x7f5d201395d0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d0006b4d0 0x7f5d0006d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 --2- 172.21.15.5:0/4088287531 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d2006d130 0x7f5d2015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 >> 172.21.15.5:0/4088287531 conn(0x7f5d2006ab60 msgr2=0x7f5d200afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 shutdown_connections 2024-04-25T10:09:10.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.665+0000 7f5d28400640 1 -- 172.21.15.5:0/4088287531 wait complete. 2024-04-25T10:09:10.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.897+0000 7f74c6400640 1 Processor -- start 2024-04-25T10:09:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.898+0000 7f74c6400640 1 -- start start 2024-04-25T10:09:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.899+0000 7f74c6400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74c00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.899+0000 7f74c6400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00c9f90 con 0x7f74c00cad50 2024-04-25T10:09:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.899+0000 7f74c6400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c00ca270 con 0x7f74c00ced80 2024-04-25T10:09:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.899+0000 7f74c6400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00ca550 con 0x7f74c01472c0 2024-04-25T10:09:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.900+0000 7f74bea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f74c00cad50 0x7f74c00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50502/0 (socket says 172.21.15.5:50502) 2024-04-25T10:09:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.900+0000 7f74bea00640 1 -- 172.21.15.5:0/4068614488 learned_addr learned my addr 172.21.15.5:0/4068614488 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.900+0000 7f74bf400640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74c00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.900+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795408959 0 0) 0x7f74c00c9f90 con 0x7f74c00cad50 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.900+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a4003520 con 0x7f74c00cad50 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4109455714 0 0) 0x7f74a4003520 con 0x7f74c00cad50 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 msgr2=0x7f74c00cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74c00cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 >> v1:172.21.15.80:6789/0 conn(0x7f74c00ced80 legacy=0x7f74c006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00ca830 con 0x7f74c00cad50 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f74ac001670 con 0x7f74c00cad50 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 >> v1:172.21.15.5:6789/0 conn(0x7f74c00cad50 legacy=0x7f74c00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74a40039e0 0x7f74a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74a4005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00ca550 con 0x7f74a40039e0 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400bb80 con 0x7f74a4004800 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400bd80 con 0x7f74c01472c0 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.901+0000 7f74bf400640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74a40039e0 0x7f74a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bea00640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bfe00640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74a4005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bf400640 1 -- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 msgr2=0x7f74a4005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bf400640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74a4005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bf400640 1 -- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74a4004800 msgr2=0x7f74a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bf400640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bf400640 1 -- 172.21.15.5:0/4068614488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4003520 con 0x7f74a40039e0 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bfe00640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74c01472c0 0x7f74a4005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.902+0000 7f74bea00640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74a4004800 0x7f74a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.903+0000 7f74bf400640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74a40039e0 0x7f74a4003e00 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009ac0 tx=0x7f74a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927b9243939333cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.903+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0036070 con 0x7f74a40039e0 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.903+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b0031400 con 0x7f74a40039e0 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.903+0000 7f74be000640 1 -- 172.21.15.5:0/4068614488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0009690 con 0x7f74a40039e0 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74c6400640 1 -- 172.21.15.5:0/4068614488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74a40039e0 msgr2=0x7f74a4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74c6400640 1 --2- 172.21.15.5:0/4068614488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74a40039e0 0x7f74a4003e00 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009ac0 tx=0x7f74a4005830 comp rx=0 tx=0).stop 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74bfe00640 1 -- 172.21.15.5:0/4068614488 reap_dead start 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74c6400640 1 -- 172.21.15.5:0/4068614488 shutdown_connections 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74c6400640 1 -- 172.21.15.5:0/4068614488 >> 172.21.15.5:0/4068614488 conn(0x7f74c0071150 msgr2=0x7f74c0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74c6400640 1 -- 172.21.15.5:0/4068614488 shutdown_connections 2024-04-25T10:09:10.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.904+0000 7f74c6400640 1 -- 172.21.15.5:0/4068614488 wait complete. 2024-04-25T10:09:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.906+0000 7f74c6400640 1 Processor -- start 2024-04-25T10:09:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.906+0000 7f74c6400640 1 -- start start 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74c6400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74a40039e0 0x7f74c013d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74c6400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74c013fe00 0x7f74c01e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74c6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 0x7f74c00b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74c6400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c006d9e0 con 0x7f74c013ce90 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74c6400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c006d860 con 0x7f74c013fe00 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74c6400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c006db60 con 0x7f74a40039e0 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 0x7f74c00b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 0x7f74c00b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42124/0 (socket says 172.21.15.5:42124) 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.907+0000 7f74bfe00640 1 -- 172.21.15.5:0/150750915 learned_addr learned my addr 172.21.15.5:0/150750915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bf400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74a40039e0 0x7f74c013d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bea00640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74c013fe00 0x7f74c01e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bfe00640 1 -- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74a40039e0 msgr2=0x7f74c013d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bfe00640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74a40039e0 0x7f74c013d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bfe00640 1 -- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74c013fe00 msgr2=0x7f74c01e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bfe00640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74c013fe00 0x7f74c01e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bfe00640 1 -- 172.21.15.5:0/150750915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c01e7770 con 0x7f74c013ce90 2024-04-25T10:09:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74bfe00640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 0x7f74c00b46b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f74c00b58d0 tx=0x7f74b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.908+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8006d50 con 0x7f74c013ce90 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.909+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8003970 con 0x7f74c013ce90 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.909+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8003da0 con 0x7f74c013ce90 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.909+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c01452c0 con 0x7f74c013ce90 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.909+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c0145880 con 0x7f74c013ce90 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.910+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c0145a80 con 0x7f74c013ce90 2024-04-25T10:09:10.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.911+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74b801b020 con 0x7f74c013ce90 2024-04-25T10:09:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.912+0000 7f74b7400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f74a006b500 0x7f74a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.912+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f74b8089ec0 con 0x7f74c013ce90 2024-04-25T10:09:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.913+0000 7f74bf400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f74a006b500 0x7f74a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:10.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.914+0000 7f74bf400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f74a006b500 0x7f74a006d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f74b0033000 tx=0x7f74b0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:10.917+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74c0145a80 con 0x7f74c013ce90 2024-04-25T10:09:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.127+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74c0072390 con 0x7f74c013ce90 2024-04-25T10:09:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.128+0000 7f74b7400640 1 -- 172.21.15.5:0/150750915 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7f74b8058ba0 con 0x7f74c013ce90 2024-04-25T10:09:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:09:11.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f74a006b500 msgr2=0x7f74a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f74a006b500 0x7f74a006d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f74b0033000 tx=0x7f74b0008030 comp rx=0 tx=0).stop 2024-04-25T10:09:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 msgr2=0x7f74c00b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 0x7f74c00b46b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f74c00b58d0 tx=0x7f74b800dc00 comp rx=0 tx=0).stop 2024-04-25T10:09:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 shutdown_connections 2024-04-25T10:09:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74c013ce90 0x7f74c00b46b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f74a006b500 0x7f74a006d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74c013fe00 0x7f74c01e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 --2- 172.21.15.5:0/150750915 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f74a40039e0 0x7f74c013d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 >> 172.21.15.5:0/150750915 conn(0x7f74c0071150 msgr2=0x7f74c00cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 shutdown_connections 2024-04-25T10:09:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.131+0000 7f74c6400640 1 -- 172.21.15.5:0/150750915 wait complete. 2024-04-25T10:09:11.160 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:09:11.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:11.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:11.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:09:11.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:11.283 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:11.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.509+0000 7f720a200640 1 Processor -- start 2024-04-25T10:09:11.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.509+0000 7f720a200640 1 -- start start 2024-04-25T10:09:11.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f720a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f720a200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72040cecd0 0x7f720415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f720a200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040b4880 con 0x7f7204143d30 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f720a200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040b45a0 con 0x7f72040cecd0 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f720a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720415e2c0 con 0x7f720415eda0 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f7209800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72040cecd0 0x7f720415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f7209800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f72040cecd0 0x7f720415abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42134/0 (socket says 172.21.15.5:42134) 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.511+0000 7f7209800640 1 -- 172.21.15.5:0/3271644197 learned_addr learned my addr 172.21.15.5:0/3271644197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.512+0000 7f7203e00640 1 --2- 172.21.15.5:0/3271644197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.512+0000 7f7209800640 1 -- 172.21.15.5:0/3271644197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 msgr2=0x7f72040ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.512+0000 7f7209800640 1 --2- 172.21.15.5:0/3271644197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.512+0000 7f7209800640 1 -- 172.21.15.5:0/3271644197 >> v1:172.21.15.80:6789/0 conn(0x7f7204143d30 legacy=0x7f72040b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.512+0000 7f7209800640 1 -- 172.21.15.5:0/3271644197 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720415e5a0 con 0x7f72040cecd0 2024-04-25T10:09:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.513+0000 7f7209800640 1 --2- 172.21.15.5:0/3271644197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040cecd0 0x7f720415abc0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00d150 tx=0x7f71fc02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a681766c62cfa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.513+0000 7f7203400640 1 -- 172.21.15.5:0/3271644197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc009600 con 0x7f72040cecd0 2024-04-25T10:09:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.513+0000 7f7203400640 1 -- 172.21.15.5:0/3271644197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc037070 con 0x7f72040cecd0 2024-04-25T10:09:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.513+0000 7f7203400640 1 -- 172.21.15.5:0/3271644197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0099c0 con 0x7f72040cecd0 2024-04-25T10:09:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 -- 172.21.15.5:0/3271644197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040cecd0 msgr2=0x7f720415abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 --2- 172.21.15.5:0/3271644197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040cecd0 0x7f720415abc0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00d150 tx=0x7f71fc02c930 comp rx=0 tx=0).stop 2024-04-25T10:09:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 -- 172.21.15.5:0/3271644197 shutdown_connections 2024-04-25T10:09:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 --2- 172.21.15.5:0/3271644197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72040cecd0 0x7f720415abc0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 --2- 172.21.15.5:0/3271644197 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 -- 172.21.15.5:0/3271644197 >> 172.21.15.5:0/3271644197 conn(0x7f7204066f20 msgr2=0x7f720406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 -- 172.21.15.5:0/3271644197 shutdown_connections 2024-04-25T10:09:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.514+0000 7f720a200640 1 -- 172.21.15.5:0/3271644197 wait complete. 2024-04-25T10:09:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.516+0000 7f720a200640 1 Processor -- start 2024-04-25T10:09:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.516+0000 7f720a200640 1 -- start start 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f720a200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72040cecd0 0x7f720415d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f720a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 0x7f72040b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f720a200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f720a200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b41f0 con 0x7f7204143d30 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f720a200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204144110 con 0x7f72040cecd0 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f720a200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72041317e0 con 0x7f720415eda0 2024-04-25T10:09:11.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f7203e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 0x7f72040b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f7203e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 0x7f72040b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42150/0 (socket says 172.21.15.5:42150) 2024-04-25T10:09:11.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.517+0000 7f7203e00640 1 -- 172.21.15.5:0/2940134539 learned_addr learned my addr 172.21.15.5:0/2940134539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7208e00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72040cecd0 0x7f720415d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7203e00640 1 -- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 msgr2=0x7f72040b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7203e00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7203e00640 1 -- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72040cecd0 msgr2=0x7f720415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7203e00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72040cecd0 0x7f720415d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7203e00640 1 -- 172.21.15.5:0/2940134539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7204130740 con 0x7f7204143d30 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7203e00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 0x7f72040b0c70 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f71f000d600 tx=0x7f71f000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.518+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00125f0 con 0x7f7204143d30 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.519+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0019070 con 0x7f7204143d30 2024-04-25T10:09:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.519+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00129f0 con 0x7f7204143d30 2024-04-25T10:09:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.519+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7204130a20 con 0x7f7204143d30 2024-04-25T10:09:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.519+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7204130fe0 con 0x7f7204143d30 2024-04-25T10:09:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.520+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72040b2e00 con 0x7f7204143d30 2024-04-25T10:09:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.521+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71f0012b90 con 0x7f7204143d30 2024-04-25T10:09:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.522+0000 7f7200c00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71e406b4d0 0x7f71e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.522+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71f008a8b0 con 0x7f7204143d30 2024-04-25T10:09:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.523+0000 7f7208e00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71e406b4d0 0x7f71e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.524+0000 7f7208e00640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71e406b4d0 0x7f71e406d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f72040b3030 tx=0x7f71f8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.527+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72040b2e00 con 0x7f7204143d30 2024-04-25T10:09:11.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.765+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7204068f60 con 0x7f7204143d30 2024-04-25T10:09:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.767+0000 7f7200c00640 1 -- 172.21.15.5:0/2940134539 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f71f0059500 con 0x7f7204143d30 2024-04-25T10:09:11.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:11.774 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:02.600363+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7488":{"gid":7488,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6834/3627376661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3627376661},{"type":"v1","addr":"172.21.15.5:6834","nonce":3627376661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 62 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.769+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71e406b4d0 msgr2=0x7f71e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.769+0000 7f720a200640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71e406b4d0 0x7f71e406d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f72040b3030 tx=0x7f71f8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.769+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 msgr2=0x7f72040b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.769+0000 7f720a200640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 0x7f72040b0c70 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f71f000d600 tx=0x7f71f000dac0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 shutdown_connections 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f720415eda0 0x7f72040b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7204143d30 0x7f72040b0c70 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f71e406b4d0 0x7f71e406d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 --2- 172.21.15.5:0/2940134539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72040cecd0 0x7f720415d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 >> 172.21.15.5:0/2940134539 conn(0x7f7204066f20 msgr2=0x7f720406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 shutdown_connections 2024-04-25T10:09:11.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:11.770+0000 7f720a200640 1 -- 172.21.15.5:0/2940134539 wait complete. 2024-04-25T10:09:11.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:09:12.048 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-04-25T10:09:12.048 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:09:12.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.253+0000 7f36eee00640 1 Processor -- start 2024-04-25T10:09:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.254+0000 7f36eee00640 1 -- start start 2024-04-25T10:09:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.255+0000 7f36eee00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36e81529e0 0x7f36e80c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.255+0000 7f36eee00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36e80c6410 0x7f36e80ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.255+0000 7f36eee00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e815bdc0 con 0x7f36e8071ed0 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.255+0000 7f36eee00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80b2340 con 0x7f36e81529e0 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.255+0000 7f36eee00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e815c060 con 0x7f36e80c6410 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36e81529e0 0x7f36e80c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36e81529e0 0x7f36e80c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42152/0 (socket says 172.21.15.5:42152) 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 -- 172.21.15.5:0/4190399277 learned_addr learned my addr 172.21.15.5:0/4190399277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 -- 172.21.15.5:0/4190399277 >> v2:172.21.15.80:3300/0 conn(0x7f36e80c6410 msgr2=0x7f36e80ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 --2- 172.21.15.5:0/4190399277 >> v2:172.21.15.80:3300/0 conn(0x7f36e80c6410 0x7f36e80ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 -- 172.21.15.5:0/4190399277 >> v1:172.21.15.80:6789/0 conn(0x7f36e8071ed0 legacy=0x7f36e80c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.256+0000 7f36e7e00640 1 -- 172.21.15.5:0/4190399277 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e815c340 con 0x7f36e81529e0 2024-04-25T10:09:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.257+0000 7f36e7e00640 1 --2- 172.21.15.5:0/4190399277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e81529e0 0x7f36e80c5ed0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f36e80b5810 tx=0x7f36d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f4ba77922ac0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36e6a00640 1 -- 172.21.15.5:0/4190399277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d8036070 con 0x7f36e81529e0 2024-04-25T10:09:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36e6a00640 1 -- 172.21.15.5:0/4190399277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d8031600 con 0x7f36e81529e0 2024-04-25T10:09:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36e6a00640 1 -- 172.21.15.5:0/4190399277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d80319d0 con 0x7f36e81529e0 2024-04-25T10:09:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36eee00640 1 -- 172.21.15.5:0/4190399277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e81529e0 msgr2=0x7f36e80c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36eee00640 1 --2- 172.21.15.5:0/4190399277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e81529e0 0x7f36e80c5ed0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f36e80b5810 tx=0x7f36d802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36eee00640 1 -- 172.21.15.5:0/4190399277 shutdown_connections 2024-04-25T10:09:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36eee00640 1 --2- 172.21.15.5:0/4190399277 >> v2:172.21.15.80:3300/0 conn(0x7f36e80c6410 0x7f36e80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36eee00640 1 --2- 172.21.15.5:0/4190399277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e81529e0 0x7f36e80c5ed0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.258+0000 7f36eee00640 1 -- 172.21.15.5:0/4190399277 >> 172.21.15.5:0/4190399277 conn(0x7f36e8071150 msgr2=0x7f36e8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.259+0000 7f36eee00640 1 -- 172.21.15.5:0/4190399277 shutdown_connections 2024-04-25T10:09:12.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.259+0000 7f36eee00640 1 -- 172.21.15.5:0/4190399277 wait complete. 2024-04-25T10:09:12.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.260+0000 7f36eee00640 1 Processor -- start 2024-04-25T10:09:12.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 -- start start 2024-04-25T10:09:12.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8071ed0 0x7f36e80ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 0x7f36e80cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36e80ce560 0x7f36e8158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e160 con 0x7f36e80c6410 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e815c900 con 0x7f36e8071ed0 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.261+0000 7f36eee00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e2e0 con 0x7f36e80ce560 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 0x7f36e80cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 0x7f36e80cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42156/0 (socket says 172.21.15.5:42156) 2024-04-25T10:09:12.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 -- 172.21.15.5:0/1671980769 learned_addr learned my addr 172.21.15.5:0/1671980769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7e00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8071ed0 0x7f36e80ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36ece00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36e80ce560 0x7f36e8158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 -- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36e80ce560 msgr2=0x7f36e8158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36e80ce560 0x7f36e8158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 -- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8071ed0 msgr2=0x7f36e80ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8071ed0 0x7f36e80ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7400640 1 -- 172.21.15.5:0/1671980769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80cd7a0 con 0x7f36e80c6410 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36e7e00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8071ed0 0x7f36e80ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.262+0000 7f36ece00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36e80ce560 0x7f36e8158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.263+0000 7f36e7400640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 0x7f36e80cb910 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f36d000b410 tx=0x7f36d000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.263+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0015020 con 0x7f36e80c6410 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.263+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0011850 con 0x7f36e80c6410 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.263+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0011c40 con 0x7f36e80c6410 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.263+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e80cda80 con 0x7f36e80c6410 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.263+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e80ce040 con 0x7f36e80c6410 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.264+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e80ce240 con 0x7f36e80c6410 2024-04-25T10:09:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.265+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36d0018020 con 0x7f36e80c6410 2024-04-25T10:09:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.266+0000 7f36dfe00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36c406b4d0 0x7f36c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.266+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36d008a3d0 con 0x7f36e80c6410 2024-04-25T10:09:12.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.267+0000 7f36e7e00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36c406b4d0 0x7f36c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.268+0000 7f36e7e00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36c406b4d0 0x7f36c406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f36d8033000 tx=0x7f36d8009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.271+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36e80ce240 con 0x7f36e80c6410 2024-04-25T10:09:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.510+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f36e8068fd0 con 0x7f36e80c6410 2024-04-25T10:09:12.518 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:09:12.511+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.514+0000 7f36dfe00640 1 -- 172.21.15.5:0/1671980769 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f36d001f020 con 0x7f36e80c6410 2024-04-25T10:09:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.518+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36c406b4d0 msgr2=0x7f36c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.518+0000 7f36eee00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36c406b4d0 0x7f36c406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f36d8033000 tx=0x7f36d8009000 comp rx=0 tx=0).stop 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.518+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 msgr2=0x7f36e80cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.518+0000 7f36eee00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 0x7f36e80cb910 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f36d000b410 tx=0x7f36d000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 shutdown_connections 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36e80ce560 0x7f36e8158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36e80c6410 0x7f36e80cb910 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36c406b4d0 0x7f36c406d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 --2- 172.21.15.5:0/1671980769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36e8071ed0 0x7f36e80ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 >> 172.21.15.5:0/1671980769 conn(0x7f36e8071150 msgr2=0x7f36e8158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 shutdown_connections 2024-04-25T10:09:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.519+0000 7f36eee00640 1 -- 172.21.15.5:0/1671980769 wait complete. 2024-04-25T10:09:12.550 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T10:09:12.557 INFO:tasks.ceph.mds.a.smithi005.stderr: -4811> 2024-04-25T10:09:04.998+0000 7f4942600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:12.624 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:09:12.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.657+0000 7ff7e1a0df40 1 Processor -- start 2024-04-25T10:09:12.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.657+0000 7ff7e1a0df40 1 -- start start 2024-04-25T10:09:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7e1a0df40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce6740e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7e1a0df40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce674620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7e1a0df40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7e1a0df40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ce5551c0 con 0x5588ce554db0 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7e1a0df40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ce557380 con 0x5588ce556bd0 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7e1a0df40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588ce557620 con 0x5588ce5c7b20 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7de000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce6740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7de000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce6740e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42198/0 (socket says 172.21.15.5:42198) 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.658+0000 7ff7de000640 1 -- 172.21.15.5:0/3158835354 learned_addr learned my addr 172.21.15.5:0/3158835354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7dea00640 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7d7e00640 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce674620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7de000640 1 -- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 msgr2=0x5588ce5c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7de000640 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7de000640 1 -- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 msgr2=0x5588ce674620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7de000640 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce674620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7de000640 1 -- 172.21.15.5:0/3158835354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ce4773b0 con 0x5588ce554db0 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7de000640 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce6740e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009750 tx=0x7ff7d0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1addedc654d3ac17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7dcc00640 1 -- 172.21.15.5:0/3158835354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0019070 con 0x5588ce554db0 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7dcc00640 1 -- 172.21.15.5:0/3158835354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0008890 con 0x5588ce554db0 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.659+0000 7ff7dcc00640 1 -- 172.21.15.5:0/3158835354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0008c60 con 0x5588ce554db0 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/3158835354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 msgr2=0x5588ce6740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce6740e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0009750 tx=0x7ff7d0005a00 comp rx=0 tx=0).stop 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/3158835354 shutdown_connections 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce674620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/3158835354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce6740e0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/3158835354 >> 172.21.15.5:0/3158835354 conn(0x5588ce509c60 msgr2=0x5588ce5e0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/3158835354 shutdown_connections 2024-04-25T10:09:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.660+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/3158835354 wait complete. 2024-04-25T10:09:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.661+0000 7ff7e1a0df40 1 Processor -- start 2024-04-25T10:09:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.661+0000 7ff7e1a0df40 1 -- start start 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7e1a0df40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce5c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7e1a0df40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce5c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7e1a0df40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7e1a0df40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ce53c360 con 0x5588ce554db0 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7e1a0df40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ce53d090 con 0x5588ce556bd0 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7e1a0df40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588ce53cc00 con 0x5588ce5c7b20 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7de000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce5c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7de000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce5c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42204/0 (socket says 172.21.15.5:42204) 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.662+0000 7ff7de000640 1 -- 172.21.15.5:0/2332493687 learned_addr learned my addr 172.21.15.5:0/2332493687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7d7e00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce5c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7de000640 1 -- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 msgr2=0x5588ce5c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7dea00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7de000640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5588ce5c7b20 0x5588ce5c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7de000640 1 -- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 msgr2=0x5588ce5c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7de000640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce5c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7de000640 1 -- 172.21.15.5:0/2332493687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ce5c8110 con 0x5588ce554db0 2024-04-25T10:09:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7d7e00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5588ce556bd0 0x5588ce5c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7de000640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce5c21d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0000c00 tx=0x7ff7d0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7d6a00640 1 -- 172.21.15.5:0/2332493687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0019040 con 0x5588ce554db0 2024-04-25T10:09:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7d6a00640 1 -- 172.21.15.5:0/2332493687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d0023070 con 0x5588ce554db0 2024-04-25T10:09:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7d6a00640 1 -- 172.21.15.5:0/2332493687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00146c0 con 0x5588ce554db0 2024-04-25T10:09:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.663+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5588ce5c2a20 con 0x5588ce554db0 2024-04-25T10:09:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.664+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588ce5dd130 con 0x5588ce554db0 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.665+0000 7ff7d6a00640 1 -- 172.21.15.5:0/2332493687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7d0014860 con 0x5588ce554db0 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.666+0000 7ff7d6a00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7bc069ee0 0x7ff7bc06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.666+0000 7ff7d6a00640 1 -- 172.21.15.5:0/2332493687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff7d008a790 con 0x5588ce554db0 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.667+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5588ce5c37c0 0x5588ce5c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.667+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e90) v9 -- 0x5588ce5c13d0 con 0x5588ce5c37c0 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.667+0000 7ff7d7e00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7bc069ee0 0x7ff7bc06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.667+0000 7ff7dea00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5588ce5c37c0 0x5588ce5c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.669+0000 7ff7dea00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5588ce5c37c0 0x5588ce5c3c20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.669+0000 7ff7d7e00640 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7bc069ee0 0x7ff7bc06c3a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8057000 tx=0x7ff7c8052000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.673+0000 7ff7dea00640 1 -- 172.21.15.5:0/2332493687 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v90'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff7d8011040 con 0x5588ce5c37c0 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5588ce5c37c0 msgr2=0x5588ce5c3c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5588ce5c37c0 0x5588ce5c3c20 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7bc069ee0 msgr2=0x7ff7bc06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff7bc069ee0 0x7ff7bc06c3a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8057000 tx=0x7ff7c8052000 comp rx=0 tx=0).stop 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 msgr2=0x5588ce5c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 --2- 172.21.15.5:0/2332493687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588ce554db0 0x5588ce5c21d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0000c00 tx=0x7ff7d0003020 comp rx=0 tx=0).stop 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7dea00640 1 -- 172.21.15.5:0/2332493687 reap_dead start 2024-04-25T10:09:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 shutdown_connections 2024-04-25T10:09:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 >> 172.21.15.5:0/2332493687 conn(0x5588ce509c60 msgr2=0x5588ce53aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 shutdown_connections 2024-04-25T10:09:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.674+0000 7ff7e1a0df40 1 -- 172.21.15.5:0/2332493687 wait complete. 2024-04-25T10:09:12.686 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T10:09:12.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.765+0000 7f0fd8c1a280 1 -- start start 2024-04-25T10:09:12.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.765+0000 7f0fd8c1a280 1 Processor -- start 2024-04-25T10:09:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.765+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc480ac00 0x555cc4876580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.765+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc480b800 0x555cc4876b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 0x555cc4877080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576180 con 0x555cc480bc00 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576300 con 0x555cc480b800 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576480 con 0x555cc480ac00 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 0x555cc4877080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 0x555cc4877080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42214/0 (socket says 172.21.15.5:42214) 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 -- 172.21.15.5:0/2750616027 learned_addr learned my addr 172.21.15.5:0/2750616027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd5600640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc480b800 0x555cc4876b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6000640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc480ac00 0x555cc4876580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 -- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc480ac00 msgr2=0x555cc4876580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc480ac00 0x555cc4876580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 -- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc480b800 msgr2=0x555cc4876b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc480b800 0x555cc4876b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6a00640 1 -- 172.21.15.5:0/2750616027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc483e780 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd6000640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc480ac00 0x555cc4876580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.766+0000 7f0fd5600640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc480b800 0x555cc4876b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 0x555cc4877080 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x555cc562f2f0 tx=0x555cc558cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c6d39707c228f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd2e00640 1 -- 172.21.15.5:0/2750616027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc48309c0 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd2e00640 1 -- 172.21.15.5:0/2750616027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555cc4836700 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd2e00640 1 -- 172.21.15.5:0/2750616027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc4830680 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cc483f4a0 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555cc483e000 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd8c1a280 4 waiting for MDS map... 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.767+0000 7f0fd2e00640 1 -- 172.21.15.5:0/2750616027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x555cc56d2200 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.768+0000 7f0fd2e00640 1 -- 172.21.15.5:0/2750616027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x555cc4865180 con 0x555cc480bc00 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.768+0000 7f0fd8c1a280 4 Got MDS map 63 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.768+0000 7f0fd8c1a280 10 main: JournalTool::main 2024-04-25T10:09:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.768+0000 7f0fd8c1a280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:09:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.768+0000 7f0fd8c1a280 1 Processor -- start 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 -- start start 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55edc00 0x555cc4877600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e000 0x555cc4877b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576600 con 0x555cc572e400 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576780 con 0x555cc55edc00 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.769+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576900 con 0x555cc572e000 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42230/0 (socket says 172.21.15.5:42230) 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 -- 172.21.15.5:0/1632310900 learned_addr learned my addr 172.21.15.5:0/1632310900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 -- 172.21.15.5:0/1632310900 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e000 msgr2=0x555cc4877b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e000 0x555cc4877b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 -- 172.21.15.5:0/1632310900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55edc00 msgr2=0x555cc4877600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55edc00 0x555cc4877600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 -- 172.21.15.5:0/1632310900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc483e3c0 con 0x555cc572e400 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd5600640 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 0x555cc4878100 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x555cc56de8a0 tx=0x555cc57380c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770bcd58737f6770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.770+0000 7f0fd0600640 1 -- 172.21.15.5:0/1632310900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc48309c0 con 0x555cc572e400 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd0600640 1 -- 172.21.15.5:0/1632310900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555cc4836fc0 con 0x555cc572e400 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd0600640 1 -- 172.21.15.5:0/1632310900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc4830680 con 0x555cc572e400 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/1632310900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 msgr2=0x555cc4878100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 0x555cc4878100 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x555cc56de8a0 tx=0x555cc57380c0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/1632310900 shutdown_connections 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e000 0x555cc4877b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/1632310900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55edc00 0x555cc4877600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/1632310900 >> 172.21.15.5:0/1632310900 conn(0x555cc55ed800 msgr2=0x555cc5587000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/1632310900 shutdown_connections 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.771+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/1632310900 wait complete. 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.772+0000 7f0fd8c1a280 1 Processor -- start 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.772+0000 7f0fd8c1a280 1 -- start start 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55ed800 0x555cc4879180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 0x555cc4878c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572ec00 0x555cc4878680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5577c80 con 0x555cc572f000 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5577b00 con 0x555cc55ed800 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc5577980 con 0x555cc572ec00 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 0x555cc4878c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 0x555cc4878c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42244/0 (socket says 172.21.15.5:42244) 2024-04-25T10:09:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd6000640 1 -- 172.21.15.5:0/4259174607 learned_addr learned my addr 172.21.15.5:0/4259174607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd6000640 1 -- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572ec00 msgr2=0x555cc4878680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.773+0000 7f0fd5600640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572ec00 0x555cc4878680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd6000640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572ec00 0x555cc4878680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd6000640 1 -- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55ed800 msgr2=0x555cc4879180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd6000640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55ed800 0x555cc4879180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd6000640 1 -- 172.21.15.5:0/4259174607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc483e960 con 0x555cc572f000 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd5600640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572ec00 0x555cc4878680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd6000640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 0x555cc4878c00 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x555cc56df560 tx=0x555cc57382a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fce800640 1 -- 172.21.15.5:0/4259174607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc48309c0 con 0x555cc572f000 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fce800640 1 -- 172.21.15.5:0/4259174607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555cc4836e00 con 0x555cc572f000 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fce800640 1 -- 172.21.15.5:0/4259174607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc4830680 con 0x555cc572f000 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555cc483f680 con 0x555cc572f000 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cc483f860 con 0x555cc572f000 2024-04-25T10:09:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.774+0000 7f0fd8c1a280 4 main: JournalTool: resolving pool 13 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.776+0000 7f0fce800640 1 -- 172.21.15.5:0/4259174607 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x555cc5740c00 con 0x555cc572f000 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.777+0000 7f0fce800640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x555cc574cc00 0x555cc4879700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.777+0000 7f0fce800640 1 -- 172.21.15.5:0/4259174607 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x555cc573ee00 con 0x555cc572f000 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.777+0000 7f0fd8c1a280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.777+0000 7f0fd8c1a280 4 main: Executing for rank 0 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.777+0000 7f0fd5600640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x555cc574cc00 0x555cc4879700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.778+0000 7f0fd8c1a280 1 -- start start 2024-04-25T10:09:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.778+0000 7f0fd8c1a280 1 Processor -- start 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.778+0000 7f0fd5600640 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x555cc574cc00 0x555cc4879700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x555cc56de8a0 tx=0x555cc5738000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc572e000 0x555cc4877b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd8c1a280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc574d000 0x555cc4877600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5577680 con 0x555cc574d000 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576600 con 0x555cc572e000 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd8c1a280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x555cc5576780 con 0x555cc572e400 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc574d000 0x555cc4877600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc574d000 0x555cc4877600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42254/0 (socket says 172.21.15.5:42254) 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.779+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 learned_addr learned my addr 172.21.15.5:0/396678323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd6000640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc572e000 0x555cc4877b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e400 msgr2=0x555cc4878100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd5600640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572e400 0x555cc4878100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc572e000 msgr2=0x555cc4877b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd5600640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc572e000 0x555cc4877b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cc483fa40 con 0x555cc574d000 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fd5600640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc574d000 0x555cc4877600 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x555cc4b25620 tx=0x555cc5738060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4ae2069f51674c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fcac00640 1 -- 172.21.15.5:0/396678323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc48304e0 con 0x555cc574d000 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fcac00640 1 -- 172.21.15.5:0/396678323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555cc4836c40 con 0x555cc574d000 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.780+0000 7f0fcac00640 1 -- 172.21.15.5:0/396678323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555cc577cb60 con 0x555cc574d000 2024-04-25T10:09:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cc483e3c0 con 0x555cc574d000 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555cc483fa40 con 0x555cc574d000 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd8c1a280 4 waiting for MDS map... 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fcac00640 1 -- 172.21.15.5:0/396678323 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x555cc4bd6000 con 0x555cc574d000 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fcac00640 1 -- 172.21.15.5:0/396678323 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x555cc4bde000 con 0x555cc574d000 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd8c1a280 4 Got MDS map 63 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x555cc574d400 0x555cc4bee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc574d800 con 0x555cc574d400 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.781+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x555cc574d400 0x555cc4bee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.782+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x555cc574d400 0x555cc4bee000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=4a09d1416cec53fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.785+0000 7f0fd6a00640 1 -- 172.21.15.5:0/396678323 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x555cc5588000 con 0x555cc574d400 2024-04-25T10:09:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.785+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x555cc574dc00 0x555cc4bee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.785+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bd5000 con 0x555cc574dc00 2024-04-25T10:09:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.786+0000 7f0fd6000640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x555cc574dc00 0x555cc4bee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.787+0000 7f0fd6000640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x555cc574dc00 0x555cc4bee580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=72856349dc5f4f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.788+0000 7f0fd6000640 1 -- 172.21.15.5:0/396678323 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x555cc5588500 con 0x555cc574dc00 2024-04-25T10:09:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.788+0000 7f0fc8e00640 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bfe000 con 0x555cc574dc00 2024-04-25T10:09:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.788+0000 7f0fc8e00640 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bfe400 con 0x555cc574d400 2024-04-25T10:09:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.788+0000 7f0fd6a00640 1 -- 172.21.15.5:0/396678323 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588000 con 0x555cc574d400 2024-04-25T10:09:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.789+0000 7f0fd6000640 1 -- 172.21.15.5:0/396678323 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x555cc5588500 con 0x555cc574dc00 2024-04-25T10:09:12.796 INFO:teuthology.orchestra.run.smithi005.stdout:old journal was 4212750~2030 2024-04-25T10:09:12.796 INFO:teuthology.orchestra.run.smithi005.stdout:new journal start will be 8388608 (4173828 bytes past old end) 2024-04-25T10:09:12.796 INFO:teuthology.orchestra.run.smithi005.stdout:writing journal head 2024-04-25T10:09:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.789+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bd5400 con 0x555cc574dc00 2024-04-25T10:09:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd6000640 1 -- 172.21.15.5:0/396678323 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v90'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588500 con 0x555cc574dc00 2024-04-25T10:09:12.800 INFO:teuthology.orchestra.run.smithi005.stdout:writing EResetJournal entry 2024-04-25T10:09:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bd5800 con 0x555cc574d400 2024-04-25T10:09:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x555cc4bd5c00 0x555cc4beeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bfec00 con 0x555cc4bd5c00 2024-04-25T10:09:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x555cc4bff000 0x555cc4bef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bff400 con 0x555cc4bff000 2024-04-25T10:09:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x555cc4bff800 0x555cc4bef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bffc00 con 0x555cc4bff800 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x555cc4c04000 0x555cc4befb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4c04400 con 0x555cc4c04000 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4c04800 con 0x555cc4bff000 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd6000640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x555cc4bff000 0x555cc4bef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.793+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x555cc4bd5c00 0x555cc4beeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.794+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x555cc4c04000 0x555cc4befb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.794+0000 7f0fd5600640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x555cc4bff800 0x555cc4bef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.794+0000 7f0fd6000640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x555cc4bff000 0x555cc4bef080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=b40071a6424bd375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.794+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x555cc4bd5c00 0x555cc4beeb00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=ba1dd72911bcf8f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.794+0000 7f0fd6a00640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x555cc4c04000 0x555cc4befb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=daa14847baa0b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.795+0000 7f0fd5600640 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x555cc4bff800 0x555cc4bef600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=a3098e2204813041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.796+0000 7f0fd6a00640 1 -- 172.21.15.5:0/396678323 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(6 200.00000002 [delete] v90'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588000 con 0x555cc574d400 2024-04-25T10:09:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.796+0000 7f0fd6000640 1 -- 172.21.15.5:0/396678323 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(8 200.00000004 [delete] v90'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588500 con 0x555cc4bff000 2024-04-25T10:09:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.797+0000 7f0fd6a00640 1 -- 172.21.15.5:0/396678323 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(10 200.00000006 [delete] v90'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588000 con 0x555cc4c04000 2024-04-25T10:09:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.797+0000 7f0fd6000640 1 -- 172.21.15.5:0/396678323 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(11 200.00000007 [delete] v90'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588500 con 0x555cc4bff000 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.798+0000 7f0fd6a00640 1 -- 172.21.15.5:0/396678323 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(7 200.00000003 [delete] v90'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588000 con 0x555cc4bd5c00 2024-04-25T10:09:12.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.798+0000 7f0fc8e00640 1 -- 172.21.15.5:0/396678323 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90) v9 -- 0x555cc4bfe800 con 0x555cc574d400 2024-04-25T10:09:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.802+0000 7f0fd6a00640 1 -- 172.21.15.5:0/396678323 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v90'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588000 con 0x555cc574d400 2024-04-25T10:09:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(9 200.00000005 [delete] v90'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555cc5588780 con 0x555cc4bff800 2024-04-25T10:09:12.812 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-04-25T10:09:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x555cc4c04000 msgr2=0x555cc4befb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x555cc4c04000 0x555cc4befb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x555cc4bff000 msgr2=0x555cc4bef080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x555cc4bff000 0x555cc4bef080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x555cc574d400 msgr2=0x555cc4bee000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x555cc574d400 0x555cc4bee000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x555cc4bd5c00 msgr2=0x555cc4beeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x555cc4bd5c00 0x555cc4beeb00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x555cc4bff800 msgr2=0x555cc4bef600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x555cc4bff800 0x555cc4bef600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x555cc574dc00 msgr2=0x555cc4bee580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x555cc574dc00 0x555cc4bee580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 reap_dead start 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 reap_dead start 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc574d000 msgr2=0x555cc4877600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/396678323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc574d000 0x555cc4877600 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x555cc4b25620 tx=0x555cc5738060 comp rx=0 tx=0).stop 2024-04-25T10:09:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd5600640 1 -- 172.21.15.5:0/396678323 reap_dead start 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 shutdown_connections 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.805+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 >> 172.21.15.5:0/396678323 conn(0x555cc572e800 msgr2=0x555cc5587800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 shutdown_connections 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/396678323 wait complete. 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x555cc574cc00 msgr2=0x555cc4879700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x555cc574cc00 0x555cc4879700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x555cc56de8a0 tx=0x555cc5738000 comp rx=0 tx=0).stop 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 msgr2=0x555cc4878c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 0x555cc4878c00 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x555cc56df560 tx=0x555cc57382a0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 shutdown_connections 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc572ec00 0x555cc4878680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc572f000 0x555cc4878c00 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x555cc574cc00 0x555cc4879700 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/4259174607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc55ed800 0x555cc4879180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.806+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 >> 172.21.15.5:0/4259174607 conn(0x555cc55edc00 msgr2=0x555cc5587000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 shutdown_connections 2024-04-25T10:09:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/4259174607 wait complete. 2024-04-25T10:09:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 msgr2=0x555cc4877080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 0x555cc4877080 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x555cc562f2f0 tx=0x555cc558cc00 comp rx=0 tx=0).stop 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 shutdown_connections 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cc480bc00 0x555cc4877080 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x555cc480b800 0x555cc4876b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 --2- 172.21.15.5:0/2750616027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x555cc480ac00 0x555cc4876580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 >> 172.21.15.5:0/2750616027 conn(0x555cc480b000 msgr2=0x555cc5586800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 shutdown_connections 2024-04-25T10:09:12.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:12.807+0000 7f0fd8c1a280 1 -- 172.21.15.5:0/2750616027 wait complete. 2024-04-25T10:09:12.819 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.133524, result: old journal was 4212750~2030 new journal start will be 8388608 (4173828 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T10:09:12.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:09:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.021+0000 7f8a96000640 1 Processor -- start 2024-04-25T10:09:13.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.021+0000 7f8a96000640 1 -- start start 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a96000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a900cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a96000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8a9015bc30 0x7f8a9015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a96000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a90144cd0 con 0x7f8a9015fe30 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a96000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9015f460 con 0x7f8a9015bc30 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a96000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015c440 con 0x7f8a901448f0 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a95600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8a9015fe30 0x7f8a9015ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50508/0 (socket says 172.21.15.5:50508) 2024-04-25T10:09:13.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.023+0000 7f8a95600640 1 -- 172.21.15.5:0/528836291 learned_addr learned my addr 172.21.15.5:0/528836291 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:13.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a94c00640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a900cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8fe00640 1 --2- 172.21.15.5:0/528836291 >> v2:172.21.15.80:3300/0 conn(0x7f8a9015bc30 0x7f8a9015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869374740 0 0) 0x7f8a90144cd0 con 0x7f8a9015fe30 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a78003520 con 0x7f8a9015fe30 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260764752 0 0) 0x7f8a78003520 con 0x7f8a9015fe30 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 msgr2=0x7f8a900cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a900cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 >> v2:172.21.15.80:3300/0 conn(0x7f8a9015bc30 msgr2=0x7f8a9015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 --2- 172.21.15.5:0/528836291 >> v2:172.21.15.80:3300/0 conn(0x7f8a9015bc30 0x7f8a9015f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9015c720 con 0x7f8a9015fe30 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8a880019f0 con 0x7f8a9015fe30 2024-04-25T10:09:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a94c00640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a900cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.024+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 >> v1:172.21.15.5:6789/0 conn(0x7f8a9015fe30 legacy=0x7f8a9015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8f400640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a780039b0 0x7f8a78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8f400640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8f400640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a78004d40 0x7f8a780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015f460 con 0x7f8a78004d40 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015c440 con 0x7f8a780039b0 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7800b9c0 con 0x7f8a901448f0 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a95600640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a78004d40 0x7f8a780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a8fe00640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a94c00640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a780039b0 0x7f8a78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a95600640 1 -- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 msgr2=0x7f8a78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a95600640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901448f0 0x7f8a78004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a95600640 1 -- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a780039b0 msgr2=0x7f8a78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a95600640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a780039b0 0x7f8a78003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.025+0000 7f8a95600640 1 -- 172.21.15.5:0/528836291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78003520 con 0x7f8a78004d40 2024-04-25T10:09:13.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.026+0000 7f8a95600640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a78004d40 0x7f8a780051c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800a7b0 tx=0x7f8a78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d9d40c927a0a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800a3d0 con 0x7f8a78004d40 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8802f070 con 0x7f8a78004d40 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a8f400640 1 -- 172.21.15.5:0/528836291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8802bc20 con 0x7f8a78004d40 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a96000640 1 -- 172.21.15.5:0/528836291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a78004d40 msgr2=0x7f8a780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a96000640 1 --2- 172.21.15.5:0/528836291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a78004d40 0x7f8a780051c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800a7b0 tx=0x7f8a78005830 comp rx=0 tx=0).stop 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a95600640 1 -- 172.21.15.5:0/528836291 reap_dead start 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a96000640 1 -- 172.21.15.5:0/528836291 shutdown_connections 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.027+0000 7f8a96000640 1 -- 172.21.15.5:0/528836291 >> 172.21.15.5:0/528836291 conn(0x7f8a90066f20 msgr2=0x7f8a9006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.028+0000 7f8a96000640 1 -- 172.21.15.5:0/528836291 shutdown_connections 2024-04-25T10:09:13.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.028+0000 7f8a96000640 1 -- 172.21.15.5:0/528836291 wait complete. 2024-04-25T10:09:13.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.029+0000 7f8a96000640 1 Processor -- start 2024-04-25T10:09:13.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.030+0000 7f8a96000640 1 -- start start 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a96000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a78004d40 0x7f8a9013ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a96000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901f4160 0x7f8a900ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a96000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 0x7f8a900b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a96000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015c090 con 0x7f8a90153880 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a96000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015d530 con 0x7f8a78004d40 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a96000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9013b2b0 con 0x7f8a901f4160 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 0x7f8a900b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 0x7f8a900b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42272/0 (socket says 172.21.15.5:42272) 2024-04-25T10:09:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 -- 172.21.15.5:0/1011479487 learned_addr learned my addr 172.21.15.5:0/1011479487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a94c00640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a78004d40 0x7f8a9013ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a8fe00640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901f4160 0x7f8a900ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 -- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901f4160 msgr2=0x7f8a900ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901f4160 0x7f8a900ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 -- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a78004d40 msgr2=0x7f8a9013ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.031+0000 7f8a95600640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a78004d40 0x7f8a9013ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a95600640 1 -- 172.21.15.5:0/1011479487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90154180 con 0x7f8a90153880 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a94c00640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a78004d40 0x7f8a9013ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a95600640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 0x7f8a900b2dd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800a8a0 tx=0x7f8a900cefa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800a3d0 con 0x7f8a90153880 2024-04-25T10:09:13.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8802cb30 con 0x7f8a90153880 2024-04-25T10:09:13.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a90154460 con 0x7f8a90153880 2024-04-25T10:09:13.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88001f20 con 0x7f8a90153880 2024-04-25T10:09:13.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.032+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a900ae840 con 0x7f8a90153880 2024-04-25T10:09:13.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.034+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90068f60 con 0x7f8a90153880 2024-04-25T10:09:13.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.035+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a88034470 con 0x7f8a90153880 2024-04-25T10:09:13.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.036+0000 7f8a8cc00640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7006b4d0 0x7f8a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.036+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8a880aa700 con 0x7f8a90153880 2024-04-25T10:09:13.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.036+0000 7f8a94c00640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7006b4d0 0x7f8a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.038+0000 7f8a94c00640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7006b4d0 0x7f8a7006d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8a90145320 tx=0x7f8a84008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.040+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a88079390 con 0x7f8a90153880 2024-04-25T10:09:13.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.280+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f8a90072180 con 0x7f8a90153880 2024-04-25T10:09:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.404+0000 7f8a8cc00640 1 -- 172.21.15.5:0/1011479487 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8a88078c90 con 0x7f8a90153880 2024-04-25T10:09:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.407+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7006b4d0 msgr2=0x7f8a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.407+0000 7f8a96000640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7006b4d0 0x7f8a7006d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8a90145320 tx=0x7f8a84008040 comp rx=0 tx=0).stop 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.407+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 msgr2=0x7f8a900b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.407+0000 7f8a96000640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 0x7f8a900b2dd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800a8a0 tx=0x7f8a900cefa0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 shutdown_connections 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a90153880 0x7f8a900b2dd0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a901f4160 0x7f8a900ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a7006b4d0 0x7f8a7006d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 --2- 172.21.15.5:0/1011479487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a78004d40 0x7f8a9013ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 >> 172.21.15.5:0/1011479487 conn(0x7f8a90066f20 msgr2=0x7f8a900b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 shutdown_connections 2024-04-25T10:09:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.408+0000 7f8a96000640 1 -- 172.21.15.5:0/1011479487 wait complete. 2024-04-25T10:09:13.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:13.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.616+0000 7f5063e00640 1 Processor -- start 2024-04-25T10:09:13.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.617+0000 7f5063e00640 1 -- start start 2024-04-25T10:09:13.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.618+0000 7f5063e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f50640cda10 0x7f506415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.618+0000 7f5063e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5064071d30 con 0x7f506415ce00 2024-04-25T10:09:13.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.618+0000 7f5063e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506406d620 con 0x7f5064141da0 2024-04-25T10:09:13.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.618+0000 7f5063e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5064142180 con 0x7f50640cda10 2024-04-25T10:09:13.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f5063400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f50640cda10 0x7f506415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f505a000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f506415ce00 0x7f50640cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50512/0 (socket says 172.21.15.5:50512) 2024-04-25T10:09:13.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f505a000640 1 -- 172.21.15.5:0/1822679959 learned_addr learned my addr 172.21.15.5:0/1822679959 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:13.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f5063400640 1 -- 172.21.15.5:0/1822679959 >> v1:172.21.15.80:6789/0 conn(0x7f5064141da0 legacy=0x7f506406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f5063400640 1 -- 172.21.15.5:0/1822679959 >> v1:172.21.15.5:6789/0 conn(0x7f506415ce00 legacy=0x7f50640cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f5063400640 1 -- 172.21.15.5:0/1822679959 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506406d900 con 0x7f50640cda10 2024-04-25T10:09:13.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.619+0000 7f5062000640 1 -- 172.21.15.5:0/1822679959 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323272387 0 0) 0x7f5064071d30 con 0x7f506415ce00 2024-04-25T10:09:13.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.620+0000 7f5063400640 1 --2- 172.21.15.5:0/1822679959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f506415d280 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f505c009d90 tx=0x7f505c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cadd07efc1c607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.621+0000 7f5062000640 1 -- 172.21.15.5:0/1822679959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007740 con 0x7f50640cda10 2024-04-25T10:09:13.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.621+0000 7f5062000640 1 -- 172.21.15.5:0/1822679959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c0078e0 con 0x7f50640cda10 2024-04-25T10:09:13.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.621+0000 7f5062000640 1 -- 172.21.15.5:0/1822679959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007cb0 con 0x7f50640cda10 2024-04-25T10:09:13.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.621+0000 7f5063e00640 1 -- 172.21.15.5:0/1822679959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 msgr2=0x7f506415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.621+0000 7f5063e00640 1 --2- 172.21.15.5:0/1822679959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f506415d280 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f505c009d90 tx=0x7f505c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.622+0000 7f5063e00640 1 -- 172.21.15.5:0/1822679959 shutdown_connections 2024-04-25T10:09:13.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.622+0000 7f5063e00640 1 --2- 172.21.15.5:0/1822679959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f506415d280 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.622+0000 7f5063e00640 1 -- 172.21.15.5:0/1822679959 >> 172.21.15.5:0/1822679959 conn(0x7f5064066f20 msgr2=0x7f506406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.622+0000 7f5063e00640 1 -- 172.21.15.5:0/1822679959 shutdown_connections 2024-04-25T10:09:13.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.622+0000 7f5063e00640 1 -- 172.21.15.5:0/1822679959 wait complete. 2024-04-25T10:09:13.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.623+0000 7f5063e00640 1 Processor -- start 2024-04-25T10:09:13.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.623+0000 7f5063e00640 1 -- start start 2024-04-25T10:09:13.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.624+0000 7f5063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f5064154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.624+0000 7f5063e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5064141da0 0x7f506415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.624+0000 7f5063e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f506415ce00 0x7f5064154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.624+0000 7f5063e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5064071000 con 0x7f50640cda10 2024-04-25T10:09:13.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.624+0000 7f5063e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506406e810 con 0x7f506415ce00 2024-04-25T10:09:13.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.624+0000 7f5063e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5064071180 con 0x7f5064141da0 2024-04-25T10:09:13.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f5064154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f5064154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42302/0 (socket says 172.21.15.5:42302) 2024-04-25T10:09:13.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 -- 172.21.15.5:0/739905019 learned_addr learned my addr 172.21.15.5:0/739905019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 -- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5064141da0 msgr2=0x7f506415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5064141da0 0x7f506415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 -- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f506415ce00 msgr2=0x7f5064154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f506415ce00 0x7f5064154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 -- 172.21.15.5:0/739905019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640cb500 con 0x7f50640cda10 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.625+0000 7f5062a00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f5064154310 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f505400c450 tx=0x7f505400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.626+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50540052d0 con 0x7f50640cda10 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.626+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5054005470 con 0x7f50640cda10 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.626+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054012870 con 0x7f50640cda10 2024-04-25T10:09:13.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.626+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50640cb840 con 0x7f50640cda10 2024-04-25T10:09:13.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.626+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50640cc080 con 0x7f50640cda10 2024-04-25T10:09:13.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.627+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50640ccae0 con 0x7f50640cda10 2024-04-25T10:09:13.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.628+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5054012a10 con 0x7f50640cda10 2024-04-25T10:09:13.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.629+0000 7f505be00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f504006b4d0 0x7f504006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.630+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5054089ff0 con 0x7f50640cda10 2024-04-25T10:09:13.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.630+0000 7f505a000640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f504006b4d0 0x7f504006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.631+0000 7f505a000640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f504006b4d0 0x7f504006d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c0079c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.634+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50640cb840 con 0x7f50640cda10 2024-04-25T10:09:13.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.876+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5064154850 con 0x7f50640cda10 2024-04-25T10:09:13.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.877+0000 7f505be00640 1 -- 172.21.15.5:0/739905019 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f5054058cc0 con 0x7f50640cda10 2024-04-25T10:09:13.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:13.884 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:13.402815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7683},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7683":{"gid":7683,"name":"a","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6834/7055779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":7055779},{"type":"v1","addr":"172.21.15.5:6834","nonce":7055779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:13.884 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 64 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.879+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f504006b4d0 msgr2=0x7f504006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.879+0000 7f5063e00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f504006b4d0 0x7f504006d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f504c000c00 tx=0x7f504c0079c0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.879+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 msgr2=0x7f5064154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.879+0000 7f5063e00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f5064154310 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f505400c450 tx=0x7f505400c910 comp rx=0 tx=0).stop 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 shutdown_connections 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f504006b4d0 0x7f504006d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f506415ce00 0x7f5064154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5064141da0 0x7f506415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 --2- 172.21.15.5:0/739905019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50640cda10 0x7f5064154310 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 >> 172.21.15.5:0/739905019 conn(0x7f5064066f20 msgr2=0x7f50640b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 shutdown_connections 2024-04-25T10:09:13.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:13.880+0000 7f5063e00640 1 -- 172.21.15.5:0/739905019 wait complete. 2024-04-25T10:09:14.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:09:13.402815+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7683}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7683': {'gid': 7683, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6834/7055779', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 7055779}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 7055779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:14.309 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7683:up:replay 2024-04-25T10:09:15.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.525+0000 7f9a95000640 1 Processor -- start 2024-04-25T10:09:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.525+0000 7f9a95000640 1 -- start start 2024-04-25T10:09:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.526+0000 7f9a95000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9a90141da0 0x7f9a9006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.526+0000 7f9a95000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9a9015d510 0x7f9a900cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.526+0000 7f9a95000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a90071d30 con 0x7f9a9006e640 2024-04-25T10:09:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.526+0000 7f9a95000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a90142180 con 0x7f9a90141da0 2024-04-25T10:09:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.526+0000 7f9a95000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9006d620 con 0x7f9a9015d510 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.526+0000 7f9a8f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9a90141da0 0x7f9a9006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9a90141da0 0x7f9a9006e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42308/0 (socket says 172.21.15.5:42308) 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8f400640 1 -- 172.21.15.5:0/2731403440 learned_addr learned my addr 172.21.15.5:0/2731403440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8fe00640 1 --2- 172.21.15.5:0/2731403440 >> v2:172.21.15.80:3300/0 conn(0x7f9a9015d510 0x7f9a900cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8f400640 1 -- 172.21.15.5:0/2731403440 >> v2:172.21.15.80:3300/0 conn(0x7f9a9015d510 msgr2=0x7f9a900cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8f400640 1 --2- 172.21.15.5:0/2731403440 >> v2:172.21.15.80:3300/0 conn(0x7f9a9015d510 0x7f9a900cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8f400640 1 -- 172.21.15.5:0/2731403440 >> v1:172.21.15.80:6789/0 conn(0x7f9a9006e640 legacy=0x7f9a9015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.527+0000 7f9a8f400640 1 -- 172.21.15.5:0/2731403440 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9006d900 con 0x7f9a90141da0 2024-04-25T10:09:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.528+0000 7f9a8f400640 1 --2- 172.21.15.5:0/2731403440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a90141da0 0x7f9a9006e100 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9a900ccab0 tx=0x7f9a8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd054506f5ffd2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a8e000640 1 -- 172.21.15.5:0/2731403440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80037070 con 0x7f9a90141da0 2024-04-25T10:09:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a8e000640 1 -- 172.21.15.5:0/2731403440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a80032440 con 0x7f9a90141da0 2024-04-25T10:09:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a8e000640 1 -- 172.21.15.5:0/2731403440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80032810 con 0x7f9a90141da0 2024-04-25T10:09:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 -- 172.21.15.5:0/2731403440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a90141da0 msgr2=0x7f9a9006e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 --2- 172.21.15.5:0/2731403440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a90141da0 0x7f9a9006e100 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9a900ccab0 tx=0x7f9a8002a230 comp rx=0 tx=0).stop 2024-04-25T10:09:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 -- 172.21.15.5:0/2731403440 shutdown_connections 2024-04-25T10:09:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 --2- 172.21.15.5:0/2731403440 >> v2:172.21.15.80:3300/0 conn(0x7f9a9015d510 0x7f9a900cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 --2- 172.21.15.5:0/2731403440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a90141da0 0x7f9a9006e100 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 -- 172.21.15.5:0/2731403440 >> 172.21.15.5:0/2731403440 conn(0x7f9a90066f20 msgr2=0x7f9a9006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 -- 172.21.15.5:0/2731403440 shutdown_connections 2024-04-25T10:09:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.529+0000 7f9a95000640 1 -- 172.21.15.5:0/2731403440 wait complete. 2024-04-25T10:09:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.531+0000 7f9a95000640 1 Processor -- start 2024-04-25T10:09:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.532+0000 7f9a95000640 1 -- start start 2024-04-25T10:09:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a95000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a9006e640 0x7f9a9015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a95000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a90141da0 0x7f9a900adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a95000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 0x7f9a900c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a95000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900cd9c0 con 0x7f9a900ae230 2024-04-25T10:09:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a95000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900cd840 con 0x7f9a90141da0 2024-04-25T10:09:15.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a95000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900cdb40 con 0x7f9a9006e640 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 0x7f9a900c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 0x7f9a900c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42320/0 (socket says 172.21.15.5:42320) 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a8fe00640 1 -- 172.21.15.5:0/1604916750 learned_addr learned my addr 172.21.15.5:0/1604916750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a8ea00640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a90141da0 0x7f9a900adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.533+0000 7f9a8f400640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a9006e640 0x7f9a9015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a8fe00640 1 -- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a9006e640 msgr2=0x7f9a9015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a8fe00640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a9006e640 0x7f9a9015a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a8fe00640 1 -- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a90141da0 msgr2=0x7f9a900adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a8fe00640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a90141da0 0x7f9a900adcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a8fe00640 1 -- 172.21.15.5:0/1604916750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900aca30 con 0x7f9a900ae230 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a8fe00640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 0x7f9a900c8730 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009dd0 tx=0x7f9a8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88018020 con 0x7f9a900ae230 2024-04-25T10:09:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a88014480 con 0x7f9a900ae230 2024-04-25T10:09:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.534+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88014850 con 0x7f9a900ae230 2024-04-25T10:09:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.535+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a900acd10 con 0x7f9a900ae230 2024-04-25T10:09:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.535+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a900ad2d0 con 0x7f9a900ae230 2024-04-25T10:09:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.536+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a900ccae0 con 0x7f9a900ae230 2024-04-25T10:09:15.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.537+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a88005d10 con 0x7f9a900ae230 2024-04-25T10:09:15.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.538+0000 7f9a87400640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.538+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9a8808b2e0 con 0x7f9a900ae230 2024-04-25T10:09:15.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.540+0000 7f9a8f400640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.540+0000 7f9a8f400640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a7006b4d0 0x7f9a7006d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9a80034000 tx=0x7f9a8002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.543+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a88022030 con 0x7f9a900ae230 2024-04-25T10:09:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.782+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a90140290 con 0x7f9a900ae230 2024-04-25T10:09:15.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.783+0000 7f9a87400640 1 -- 172.21.15.5:0/1604916750 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f9a88010040 con 0x7f9a900ae230 2024-04-25T10:09:15.790 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:15.790 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:14.413544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7683},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7683":{"gid":7683,"name":"a","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6834/7055779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":7055779},{"type":"v1","addr":"172.21.15.5:6834","nonce":7055779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:15.791 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 66 2024-04-25T10:09:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a7006b4d0 msgr2=0x7f9a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a7006b4d0 0x7f9a7006d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9a80034000 tx=0x7f9a8002e000 comp rx=0 tx=0).stop 2024-04-25T10:09:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 msgr2=0x7f9a900c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 0x7f9a900c8730 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009dd0 tx=0x7f9a8800c490 comp rx=0 tx=0).stop 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 shutdown_connections 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a900ae230 0x7f9a900c8730 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a90141da0 0x7f9a900adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 --2- 172.21.15.5:0/1604916750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a9006e640 0x7f9a9015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 >> 172.21.15.5:0/1604916750 conn(0x7f9a90066f20 msgr2=0x7f9a90140a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.786+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 shutdown_connections 2024-04-25T10:09:15.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:15.787+0000 7f9a95000640 1 -- 172.21.15.5:0/1604916750 wait complete. 2024-04-25T10:09:16.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:09:14.413544+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7683}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7683': {'gid': 7683, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6834/7055779', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 7055779}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 7055779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:16.216 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7683:up:rejoin 2024-04-25T10:09:17.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:17.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.435+0000 7f1f46400640 1 Processor -- start 2024-04-25T10:09:17.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.435+0000 7f1f46400640 1 -- start start 2024-04-25T10:09:17.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f46400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f400d07d0 0x7f1f400b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f46400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f400b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f46400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f400cf410 con 0x7f1f40161200 2024-04-25T10:09:17.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f46400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f400b28b0 con 0x7f1f400d07d0 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f46400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400cf6f0 con 0x7f1f400d2380 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f400d07d0 0x7f1f400b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f400d07d0 0x7f1f400b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45302/0 (socket says 172.21.15.5:45302) 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 -- 172.21.15.5:0/3550131809 learned_addr learned my addr 172.21.15.5:0/3550131809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 -- 172.21.15.5:0/3550131809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 msgr2=0x7f1f400b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 --2- 172.21.15.5:0/3550131809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f400b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 -- 172.21.15.5:0/3550131809 >> v1:172.21.15.80:6789/0 conn(0x7f1f40161200 legacy=0x7f1f400d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.437+0000 7f1f3fe00640 1 -- 172.21.15.5:0/3550131809 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400d3fd0 con 0x7f1f400d07d0 2024-04-25T10:09:17.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.439+0000 7f1f3fe00640 1 --2- 172.21.15.5:0/3550131809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400b4bb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009870 tx=0x7f1f2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d40350cafb610f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.439+0000 7f1f3f400640 1 -- 172.21.15.5:0/3550131809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28037070 con 0x7f1f400d07d0 2024-04-25T10:09:17.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.439+0000 7f1f3f400640 1 -- 172.21.15.5:0/3550131809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f28032440 con 0x7f1f400d07d0 2024-04-25T10:09:17.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.439+0000 7f1f3f400640 1 -- 172.21.15.5:0/3550131809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28032810 con 0x7f1f400d07d0 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 -- 172.21.15.5:0/3550131809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 msgr2=0x7f1f400b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 --2- 172.21.15.5:0/3550131809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400b4bb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009870 tx=0x7f1f2802a210 comp rx=0 tx=0).stop 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 -- 172.21.15.5:0/3550131809 shutdown_connections 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 --2- 172.21.15.5:0/3550131809 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f400b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 --2- 172.21.15.5:0/3550131809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400b4bb0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 -- 172.21.15.5:0/3550131809 >> 172.21.15.5:0/3550131809 conn(0x7f1f4008af80 msgr2=0x7f1f400d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 -- 172.21.15.5:0/3550131809 shutdown_connections 2024-04-25T10:09:17.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.440+0000 7f1f46400640 1 -- 172.21.15.5:0/3550131809 wait complete. 2024-04-25T10:09:17.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.442+0000 7f1f46400640 1 Processor -- start 2024-04-25T10:09:17.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.442+0000 7f1f46400640 1 -- start start 2024-04-25T10:09:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f46400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f46400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f4015d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f46400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f40161200 0x7f1f401f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f46400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400b3640 con 0x7f1f400d07d0 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f46400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400b34c0 con 0x7f1f40161200 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f46400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f400b37c0 con 0x7f1f400d2380 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f45000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f45000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45304/0 (socket says 172.21.15.5:45304) 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.443+0000 7f1f45000640 1 -- 172.21.15.5:0/1708981770 learned_addr learned my addr 172.21.15.5:0/1708981770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f3fe00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f4015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45a00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f40161200 0x7f1f401f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45000640 1 -- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 msgr2=0x7f1f4015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45000640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f4015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45000640 1 -- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f40161200 msgr2=0x7f1f401f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45000640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f40161200 0x7f1f401f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45000640 1 -- 172.21.15.5:0/1708981770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400cb7d0 con 0x7f1f400d07d0 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f3fe00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f4015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45a00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f40161200 0x7f1f401f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f45000640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400cc270 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000ba50 tx=0x7f1f3000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30019070 con 0x7f1f400d07d0 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3000cc20 con 0x7f1f400d07d0 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30014680 con 0x7f1f400d07d0 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.444+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f401f3fe0 con 0x7f1f400d07d0 2024-04-25T10:09:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.445+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f400cba90 con 0x7f1f400d07d0 2024-04-25T10:09:17.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.446+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f400cd430 con 0x7f1f400d07d0 2024-04-25T10:09:17.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.447+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f3001c070 con 0x7f1f400d07d0 2024-04-25T10:09:17.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.447+0000 7f1f3cc00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f1c06b4d0 0x7f1f1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.448+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1f3008a070 con 0x7f1f400d07d0 2024-04-25T10:09:17.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.448+0000 7f1f3fe00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f1c06b4d0 0x7f1f1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.449+0000 7f1f3fe00640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f1c06b4d0 0x7f1f1c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f28034000 tx=0x7f1f2802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.452+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f300586b0 con 0x7f1f400d07d0 2024-04-25T10:09:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.688+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f40068f60 con 0x7f1f400d07d0 2024-04-25T10:09:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.689+0000 7f1f3cc00640 1 -- 172.21.15.5:0/1708981770 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f1f3005c750 con 0x7f1f400d07d0 2024-04-25T10:09:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:17.695 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:15.422441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7683},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7683":{"gid":7683,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/7055779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":7055779},{"type":"v1","addr":"172.21.15.5:6834","nonce":7055779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 67 2024-04-25T10:09:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.691+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f1c06b4d0 msgr2=0x7f1f1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f1c06b4d0 0x7f1f1c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1f28034000 tx=0x7f1f2802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 msgr2=0x7f1f400cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400cc270 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000ba50 tx=0x7f1f3000bf10 comp rx=0 tx=0).stop 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 shutdown_connections 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f1c06b4d0 0x7f1f1c06d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f40161200 0x7f1f401f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f400d2380 0x7f1f4015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 --2- 172.21.15.5:0/1708981770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f400d07d0 0x7f1f400cc270 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 >> 172.21.15.5:0/1708981770 conn(0x7f1f4008af80 msgr2=0x7f1f400b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 shutdown_connections 2024-04-25T10:09:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:17.692+0000 7f1f46400640 1 -- 172.21.15.5:0/1708981770 wait complete. 2024-04-25T10:09:18.118 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:09:15.422441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7683}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7683': {'gid': 7683, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6834/7055779', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 7055779}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 7055779}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:18.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:18.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:09:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.397+0000 7f827f200640 1 Processor -- start 2024-04-25T10:09:18.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.397+0000 7f827f200640 1 -- start start 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.398+0000 7f827f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8280141da0 0x7f828006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f828006e640 0x7f828015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8280071d30 con 0x7f828015d340 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8280142180 con 0x7f8280141da0 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828006d620 con 0x7f828006e640 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8280141da0 0x7f828006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827e800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f828015d340 0x7f82800cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37044/0 (socket says 172.21.15.5:37044) 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827e800640 1 -- 172.21.15.5:0/77031154 learned_addr learned my addr 172.21.15.5:0/77031154 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f827de00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8280141da0 0x7f828006e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45308/0 (socket says 172.21.15.5:45308) 2024-04-25T10:09:18.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.399+0000 7f8277e00640 1 --2- 172.21.15.5:0/77031154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f828006e640 0x7f828015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.400+0000 7f827de00640 1 -- 172.21.15.5:0/77031154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f828006e640 msgr2=0x7f828015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.400+0000 7f827de00640 1 --2- 172.21.15.5:0/77031154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f828006e640 0x7f828015ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.400+0000 7f827de00640 1 -- 172.21.15.5:0/77031154 >> v1:172.21.15.5:6789/0 conn(0x7f828015d340 legacy=0x7f82800cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.400+0000 7f827de00640 1 -- 172.21.15.5:0/77031154 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828006d900 con 0x7f8280141da0 2024-04-25T10:09:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.400+0000 7f827d400640 1 -- 172.21.15.5:0/77031154 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426659131 0 0) 0x7f8280071d30 con 0x7f828015d340 2024-04-25T10:09:18.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.401+0000 7f827de00640 1 --2- 172.21.15.5:0/77031154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8280141da0 0x7f828006e100 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f82700097a0 tx=0x7f827002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274a65826d73cad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.401+0000 7f827d400640 1 -- 172.21.15.5:0/77031154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270037070 con 0x7f8280141da0 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.401+0000 7f827d400640 1 -- 172.21.15.5:0/77031154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8270032440 con 0x7f8280141da0 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.401+0000 7f827d400640 1 -- 172.21.15.5:0/77031154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270032810 con 0x7f8280141da0 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 -- 172.21.15.5:0/77031154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8280141da0 msgr2=0x7f828006e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 --2- 172.21.15.5:0/77031154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8280141da0 0x7f828006e100 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f82700097a0 tx=0x7f827002a180 comp rx=0 tx=0).stop 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 -- 172.21.15.5:0/77031154 shutdown_connections 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 --2- 172.21.15.5:0/77031154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f828006e640 0x7f828015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 --2- 172.21.15.5:0/77031154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8280141da0 0x7f828006e100 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 -- 172.21.15.5:0/77031154 >> 172.21.15.5:0/77031154 conn(0x7f8280066f20 msgr2=0x7f828006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 -- 172.21.15.5:0/77031154 shutdown_connections 2024-04-25T10:09:18.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.402+0000 7f827f200640 1 -- 172.21.15.5:0/77031154 wait complete. 2024-04-25T10:09:18.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.404+0000 7f827f200640 1 Processor -- start 2024-04-25T10:09:18.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.404+0000 7f827f200640 1 -- start start 2024-04-25T10:09:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 0x7f828013a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8280141da0 0x7f82800b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f828015d340 0x7f8280138d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82800cdb90 con 0x7f828006e640 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82800cda10 con 0x7f828015d340 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8280071000 con 0x7f8280141da0 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 0x7f828013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 0x7f828013a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45316/0 (socket says 172.21.15.5:45316) 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 -- 172.21.15.5:0/2441210721 learned_addr learned my addr 172.21.15.5:0/2441210721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 -- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8280141da0 msgr2=0x7f82800b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8280141da0 0x7f82800b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 -- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f828015d340 msgr2=0x7f8280138d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f828015d340 0x7f8280138d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.405+0000 7f827de00640 1 -- 172.21.15.5:0/2441210721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828013ac00 con 0x7f828006e640 2024-04-25T10:09:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.406+0000 7f827de00640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 0x7f828013a320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8270035040 tx=0x7f82700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.406+0000 7f8276000640 1 -- 172.21.15.5:0/2441210721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270037040 con 0x7f828006e640 2024-04-25T10:09:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.406+0000 7f8276000640 1 -- 172.21.15.5:0/2441210721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8270030770 con 0x7f828006e640 2024-04-25T10:09:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.406+0000 7f8276000640 1 -- 172.21.15.5:0/2441210721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270030b80 con 0x7f828006e640 2024-04-25T10:09:18.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.406+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828013e5a0 con 0x7f828006e640 2024-04-25T10:09:18.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.407+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828013eb60 con 0x7f828006e640 2024-04-25T10:09:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.409+0000 7f8276000640 1 -- 172.21.15.5:0/2441210721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8270044020 con 0x7f828006e640 2024-04-25T10:09:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.410+0000 7f8276000640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f825c06b500 0x7f825c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.411+0000 7f8276000640 1 -- 172.21.15.5:0/2441210721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f82700aa870 con 0x7f828006e640 2024-04-25T10:09:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.411+0000 7f828805f740 1 Processor -- start 2024-04-25T10:09:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.411+0000 7f8277e00640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f825c06b500 0x7f825c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.411+0000 7f828805f740 1 -- start start 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf4cb30 0x564f8cf74970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf4cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f828805f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f828805f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8ceb0620 con 0x564f8cf957a0 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8ce59ca0 con 0x564f8cf4cb30 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f8ceac6c0 con 0x564f8cf74eb0 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f8277e00640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f825c06b500 0x7f825c06d9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f828013e320 tx=0x7f8268008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.412+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf4b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45324/0 (socket says 172.21.15.5:45324) 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 -- 172.21.15.5:0/1559826845 learned_addr learned my addr 172.21.15.5:0/1559826845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827de00640 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf4cb30 0x564f8cf74970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f8277e00640 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf4cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 -- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 msgr2=0x564f8cf4cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf4cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 -- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf4cb30 msgr2=0x564f8cf74970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf4cb30 0x564f8cf74970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 -- 172.21.15.5:0/1559826845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f8ce60f40 con 0x564f8cf957a0 2024-04-25T10:09:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.413+0000 7f827e800640 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf4b2f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8278001cd0 tx=0x7f8278002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c08fc1cf39c7843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f826ea00640 1 -- 172.21.15.5:0/1559826845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800c840 con 0x564f8cf957a0 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f826ea00640 1 -- 172.21.15.5:0/1559826845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800ca00 con 0x564f8cf957a0 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f826ea00640 1 -- 172.21.15.5:0/1559826845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800ce30 con 0x564f8cf957a0 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f828805f740 1 -- 172.21.15.5:0/1559826845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 msgr2=0x564f8cf4b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f828805f740 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf4b2f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8278001cd0 tx=0x7f8278002190 comp rx=0 tx=0).stop 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f828805f740 1 -- 172.21.15.5:0/1559826845 shutdown_connections 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f828805f740 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf4b2f0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f828805f740 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf4cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.414+0000 7f828805f740 1 --2- 172.21.15.5:0/1559826845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf4cb30 0x564f8cf74970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.415+0000 7f828805f740 1 -- 172.21.15.5:0/1559826845 >> 172.21.15.5:0/1559826845 conn(0x564f8cf49d50 msgr2=0x564f8cf92730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.415+0000 7f828805f740 1 -- 172.21.15.5:0/1559826845 shutdown_connections 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.415+0000 7f828805f740 1 -- 172.21.15.5:0/1559826845 wait complete. 2024-04-25T10:09:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.415+0000 7f828805f740 1 -- start start 2024-04-25T10:09:18.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.415+0000 7f828805f740 1 Processor -- start 2024-04-25T10:09:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.416+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.416+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf957a0 0x564f8cf79a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.416+0000 7f828805f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 0x564f8cf73130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.416+0000 7f828805f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8cf744e0 con 0x564f8cf79fc0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8ce4f1f0 con 0x564f8cf957a0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f8cf73ed0 con 0x564f8cf74eb0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f828805f740 10 client.0 _ll_register_callbacks cb 0x564f8ce6cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 0x564f8cf73130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 0x564f8cf73130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45336/0 (socket says 172.21.15.5:45336) 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 -- 172.21.15.5:0/4212972014 learned_addr learned my addr 172.21.15.5:0/4212972014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827de00640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f8277e00640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf957a0 0x564f8cf79a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 -- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 msgr2=0x564f8cf916c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf916c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 -- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf957a0 msgr2=0x564f8cf79a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf957a0 0x564f8cf79a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f827e800640 1 -- 172.21.15.5:0/4212972014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f8ce6ec50 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.417+0000 7f8277e00640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf957a0 0x564f8cf79a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f827e800640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 0x564f8cf73130 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f8278007fd0 tx=0x7f8278001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255ea9ea3e508545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f8263400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f8263400640 1 -- 172.21.15.5:0/4212972014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278025060 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f8263400640 1 -- 172.21.15.5:0/4212972014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8278020cb0 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827800ca00 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f8263400640 1 -- 172.21.15.5:0/4212972014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82780072d0 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f828805f740 10 client.7749 fetch_fsmap learned FSMap version 67 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.418+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x564f8cbc52d0 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f8263400640 1 -- 172.21.15.5:0/4212972014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800ca00 con 0x564f8cf79fc0 2024-04-25T10:09:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f8263400640 1 -- 172.21.15.5:0/4212972014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8278007470 con 0x564f8cf79fc0 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f828805f740 10 client.7749 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f828805f740 10 client.7749 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f828805f740 20 client.7749 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f828805f740 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cf75770 0x564f8cf75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f828805f740 4 client.7749 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f8cebdb60 con 0x564f8cf75770 2024-04-25T10:09:18.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.419+0000 7f827de00640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cf75770 0x564f8cf75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.420+0000 7f827de00640 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cf75770 0x564f8cf75b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5840aa9319fbe89d server_cookie=620c76863842818d in_seq=0 out_seq=0 2024-04-25T10:09:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.420+0000 7f8263400640 10 client.7749 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f8263400640 1 -- 172.21.15.5:0/4212972014 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564f8cebdb60 con 0x564f8cf75770 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f8263400640 10 client.7749 handle_command_reply: tid=0 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 client.7749 shutdown 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 msgr2=0x564f8cf73130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 0x564f8cf73130 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f8278007fd0 tx=0x7f8278001400 comp rx=0 tx=0).stop 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 shutdown_connections 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cf75770 0x564f8cf75b70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf79fc0 0x564f8cf73130 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf957a0 0x564f8cf79a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 --2- 172.21.15.5:0/4212972014 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf916c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f8263400640 0 client.7749 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.421+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 >> 172.21.15.5:0/4212972014 conn(0x564f8cf4cb30 msgr2=0x564f8cf92530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.422+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 shutdown_connections 2024-04-25T10:09:18.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.422+0000 7f828805f740 1 -- 172.21.15.5:0/4212972014 wait complete. 2024-04-25T10:09:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.422+0000 7f828805f740 20 client.7749 trim_cache size 0 max 16384 2024-04-25T10:09:18.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.441+0000 7f828805f740 1 Processor -- start 2024-04-25T10:09:18.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.441+0000 7f828805f740 1 -- start start 2024-04-25T10:09:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.442+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf79590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.442+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cff38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.442+0000 7f828805f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cff3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.442+0000 7f828805f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82801ff2e0 con 0x564f8cf957a0 2024-04-25T10:09:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.442+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828013fdc0 con 0x564f8cf79fc0 2024-04-25T10:09:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.442+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82801fece0 con 0x564f8cf74eb0 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cff3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cff3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45338/0 (socket says 172.21.15.5:45338) 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 -- 172.21.15.5:0/4182101755 learned_addr learned my addr 172.21.15.5:0/4182101755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827de00640 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf79590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f8277e00640 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cff38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 -- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 msgr2=0x564f8cf79590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf79590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 -- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 msgr2=0x564f8cff38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cff38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.443+0000 7f827e800640 1 -- 172.21.15.5:0/4182101755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f8cc8f6b0 con 0x564f8cf957a0 2024-04-25T10:09:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.444+0000 7f827e800640 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cff3e20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8278001cd0 tx=0x7f827801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c657930ca593430d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.444+0000 7f826e000640 1 -- 172.21.15.5:0/4182101755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827803a070 con 0x564f8cf957a0 2024-04-25T10:09:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.444+0000 7f826e000640 1 -- 172.21.15.5:0/4182101755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800b030 con 0x564f8cf957a0 2024-04-25T10:09:18.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.444+0000 7f826e000640 1 -- 172.21.15.5:0/4182101755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827800c900 con 0x564f8cf957a0 2024-04-25T10:09:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 -- 172.21.15.5:0/4182101755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 msgr2=0x564f8cff3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cff3e20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8278001cd0 tx=0x7f827801e000 comp rx=0 tx=0).stop 2024-04-25T10:09:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 -- 172.21.15.5:0/4182101755 shutdown_connections 2024-04-25T10:09:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cff3e20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cff38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 --2- 172.21.15.5:0/4182101755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cf79590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 -- 172.21.15.5:0/4182101755 >> 172.21.15.5:0/4182101755 conn(0x564f8cf4cb30 msgr2=0x564f8cf95dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 -- 172.21.15.5:0/4182101755 shutdown_connections 2024-04-25T10:09:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.445+0000 7f828805f740 1 -- 172.21.15.5:0/4182101755 wait complete. 2024-04-25T10:09:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.446+0000 7f828805f740 1 -- start start 2024-04-25T10:09:18.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.446+0000 7f828805f740 1 Processor -- start 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cff2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cf8fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8cfa40f0 con 0x564f8cf957a0 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f8cf7aae0 con 0x564f8cf79fc0 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f8cf785a0 con 0x564f8cf74eb0 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f828805f740 10 client.0 _ll_register_callbacks cb 0x564f8cf74e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f827e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf90260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45344/0 (socket says 172.21.15.5:45344) 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.447+0000 7f827e800640 1 -- 172.21.15.5:0/1514210227 learned_addr learned my addr 172.21.15.5:0/1514210227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f8277e00640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cf8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827de00640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cff2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827e800640 1 -- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 msgr2=0x564f8cff2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827e800640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cff2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827e800640 1 -- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 msgr2=0x564f8cf8fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827e800640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cf8fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827e800640 1 -- 172.21.15.5:0/1514210227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f8cfa37a0 con 0x564f8cf957a0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827de00640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cff2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f827e800640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf90260 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8278001440 tx=0x7f8278001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e93f5c04db702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f8263400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f8263400640 1 -- 172.21.15.5:0/1514210227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827803a040 con 0x564f8cf957a0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f8263400640 1 -- 172.21.15.5:0/1514210227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f82780070d0 con 0x564f8cf957a0 2024-04-25T10:09:18.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.448+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827800b030 con 0x564f8cf957a0 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f8263400640 1 -- 172.21.15.5:0/1514210227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8278048050 con 0x564f8cf957a0 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 10 client.7755 fetch_fsmap learned FSMap version 67 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x564f8cf7aa50 con 0x564f8cf957a0 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f8263400640 1 -- 172.21.15.5:0/1514210227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800b030 con 0x564f8cf957a0 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f8263400640 1 -- 172.21.15.5:0/1514210227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8278007470 con 0x564f8cf957a0 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 10 client.7755 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 10 client.7755 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 20 client.7755 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cfa1270 0x564f8cfa1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 4 client.7755 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564f8cf89860 con 0x564f8cfa1270 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.449+0000 7f827de00640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cfa1270 0x564f8cfa1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.450+0000 7f8263400640 10 client.7755 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:18.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.450+0000 7f827de00640 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cfa1270 0x564f8cfa1650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49ed7d6be60d3bc server_cookie=31ddbad260b26ecc in_seq=0 out_seq=0 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.450+0000 7f8263400640 1 -- 172.21.15.5:0/1514210227 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564f8cf89860 con 0x564f8cfa1270 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.450+0000 7f8263400640 10 client.7755 handle_command_reply: tid=0 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.450+0000 7f828805f740 1 client.7755 shutdown 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 msgr2=0x564f8cf90260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf90260 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f8278001440 tx=0x7f8278001630 comp rx=0 tx=0).stop 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 shutdown_connections 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x564f8cfa1270 0x564f8cfa1650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f8cf957a0 0x564f8cf90260 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564f8cf79fc0 0x564f8cf8fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 --2- 172.21.15.5:0/1514210227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564f8cf74eb0 0x564f8cff2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 >> 172.21.15.5:0/1514210227 conn(0x564f8cf4cb30 msgr2=0x564f8cf95cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f8263400640 0 client.7755 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 shutdown_connections 2024-04-25T10:09:18.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 1 -- 172.21.15.5:0/1514210227 wait complete. 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.451+0000 7f828805f740 20 client.7755 trim_cache size 0 max 16384 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7683, 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.0426242669999999, 2024-04-25T10:09:18.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 67, 2024-04-25T10:09:18.462 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 90, 2024-04-25T10:09:18.462 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T10:09:18.462 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 5.8240080269999996 2024-04-25T10:09:18.462 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f825c06b500 msgr2=0x7f825c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f825c06b500 0x7f825c06d9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f828013e320 tx=0x7f8268008040 comp rx=0 tx=0).stop 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 msgr2=0x7f828013a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 0x7f828013a320 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8270035040 tx=0x7f82700046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 shutdown_connections 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f825c06b500 0x7f825c06d9c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f828015d340 0x7f8280138d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8280141da0 0x7f82800b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 --2- 172.21.15.5:0/2441210721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828006e640 0x7f828013a320 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 >> 172.21.15.5:0/2441210721 conn(0x7f8280066f20 msgr2=0x7f82800b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 shutdown_connections 2024-04-25T10:09:18.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.452+0000 7f827f200640 1 -- 172.21.15.5:0/2441210721 wait complete. 2024-04-25T10:09:18.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.671+0000 7f5681200640 1 Processor -- start 2024-04-25T10:09:18.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.671+0000 7f5681200640 1 -- start start 2024-04-25T10:09:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f5681200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f567c15d8a0 0x7f567c0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f5681200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c0b2f30 0x7f567c0d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f5681200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c15f410 con 0x7f567c1614e0 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f5681200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567c15ffe0 con 0x7f567c15d8a0 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f5681200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15f6b0 con 0x7f567c0b2f30 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f567b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f567c15d8a0 0x7f567c0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f567b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f567c15d8a0 0x7f567c0d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45352/0 (socket says 172.21.15.5:45352) 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.673+0000 7f567b400640 1 -- 172.21.15.5:0/1702490027 learned_addr learned my addr 172.21.15.5:0/1702490027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.674+0000 7f567aa00640 1 --2- 172.21.15.5:0/1702490027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c0b2f30 0x7f567c0d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.674+0000 7f567b400640 1 -- 172.21.15.5:0/1702490027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c0b2f30 msgr2=0x7f567c0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.674+0000 7f567b400640 1 --2- 172.21.15.5:0/1702490027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c0b2f30 0x7f567c0d3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.674+0000 7f567b400640 1 -- 172.21.15.5:0/1702490027 >> v1:172.21.15.80:6789/0 conn(0x7f567c1614e0 legacy=0x7f567c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:18.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.674+0000 7f567b400640 1 -- 172.21.15.5:0/1702490027 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c15f990 con 0x7f567c15d8a0 2024-04-25T10:09:18.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.675+0000 7f567b400640 1 --2- 172.21.15.5:0/1702490027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c15d8a0 0x7f567c0d2570 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f566c009860 tx=0x7f566c02a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f83378e215d1134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.675+0000 7f567a000640 1 -- 172.21.15.5:0/1702490027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c037070 con 0x7f567c15d8a0 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.675+0000 7f567a000640 1 -- 172.21.15.5:0/1702490027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c02ade0 con 0x7f567c15d8a0 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.675+0000 7f567a000640 1 -- 172.21.15.5:0/1702490027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c032660 con 0x7f567c15d8a0 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 -- 172.21.15.5:0/1702490027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c15d8a0 msgr2=0x7f567c0d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 --2- 172.21.15.5:0/1702490027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c15d8a0 0x7f567c0d2570 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f566c009860 tx=0x7f566c02a0d0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 -- 172.21.15.5:0/1702490027 shutdown_connections 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 --2- 172.21.15.5:0/1702490027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c0b2f30 0x7f567c0d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 --2- 172.21.15.5:0/1702490027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c15d8a0 0x7f567c0d2570 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 -- 172.21.15.5:0/1702490027 >> 172.21.15.5:0/1702490027 conn(0x7f567c08af80 msgr2=0x7f567c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 -- 172.21.15.5:0/1702490027 shutdown_connections 2024-04-25T10:09:18.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.676+0000 7f5681200640 1 -- 172.21.15.5:0/1702490027 wait complete. 2024-04-25T10:09:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.677+0000 7f5681200640 1 Processor -- start 2024-04-25T10:09:18.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.678+0000 7f5681200640 1 -- start start 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.678+0000 7f5681200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f567c0b2f30 0x7f567c0b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f5681200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c15d8a0 0x7f567c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f5681200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 0x7f567c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f5681200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15ec90 con 0x7f567c1614e0 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f5681200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c0d4460 con 0x7f567c0b2f30 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f5681200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15ee10 con 0x7f567c15d8a0 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 0x7f567c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 0x7f567c15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45364/0 (socket says 172.21.15.5:45364) 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 -- 172.21.15.5:0/1876804451 learned_addr learned my addr 172.21.15.5:0/1876804451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:18.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567b400640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f567c0b2f30 0x7f567c0b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567aa00640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c15d8a0 0x7f567c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 -- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c15d8a0 msgr2=0x7f567c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c15d8a0 0x7f567c0cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 -- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f567c0b2f30 msgr2=0x7f567c0b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f567c0b2f30 0x7f567c0b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567be00640 1 -- 172.21.15.5:0/1876804451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c0b8380 con 0x7f567c1614e0 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.679+0000 7f567aa00640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c15d8a0 0x7f567c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.680+0000 7f567be00640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 0x7f567c15c390 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f567400c390 tx=0x7f567400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.680+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674017070 con 0x7f567c1614e0 2024-04-25T10:09:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.680+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56740035f0 con 0x7f567c1614e0 2024-04-25T10:09:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.680+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740039e0 con 0x7f567c1614e0 2024-04-25T10:09:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.680+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c0c72b0 con 0x7f567c1614e0 2024-04-25T10:09:18.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.680+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c0cf600 con 0x7f567c1614e0 2024-04-25T10:09:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.681+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c068f60 con 0x7f567c1614e0 2024-04-25T10:09:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.682+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5674003b80 con 0x7f567c1614e0 2024-04-25T10:09:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.683+0000 7f5673400640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f565c06b4d0 0x7f565c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.683+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f56740896d0 con 0x7f567c1614e0 2024-04-25T10:09:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.684+0000 7f567b400640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f565c06b4d0 0x7f565c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:18.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.685+0000 7f567b400640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f565c06b4d0 0x7f565c06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:18.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.688+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56740582d0 con 0x7f567c1614e0 2024-04-25T10:09:18.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.928+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f567c0d0e50 con 0x7f567c1614e0 2024-04-25T10:09:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.929+0000 7f5673400640 1 -- 172.21.15.5:0/1876804451 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5674057bd0 con 0x7f567c1614e0 2024-04-25T10:09:18.935 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f565c06b4d0 msgr2=0x7f565c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f565c06b4d0 0x7f565c06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 msgr2=0x7f567c15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 0x7f567c15c390 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f567400c390 tx=0x7f567400c850 comp rx=0 tx=0).stop 2024-04-25T10:09:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 shutdown_connections 2024-04-25T10:09:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f567c1614e0 0x7f567c15c390 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f567c15d8a0 0x7f567c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f565c06b4d0 0x7f565c06d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 --2- 172.21.15.5:0/1876804451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f567c0b2f30 0x7f567c0b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 >> 172.21.15.5:0/1876804451 conn(0x7f567c08af80 msgr2=0x7f567c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 shutdown_connections 2024-04-25T10:09:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:18.932+0000 7f5681200640 1 -- 172.21.15.5:0/1876804451 wait complete. 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:18.958 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:18.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.959 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:09:19.016 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:19.016 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:19.016 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:19.016 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.016 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42167sec preferred_lft 42167sec 2024-04-25T10:09:19.017 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:09:19.018 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.019 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:19.019 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:19.019 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:09:19.019 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:19.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.020 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:19.107 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:09:19.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.110 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:09:19.165 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:09:19.165 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:19.165 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:09:19.165 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:09:19.165 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:19.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.166 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:09:19.166 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:09:19.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:09:19.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:09:19.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:09:19.167 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:09:19.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.333 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:09:19.390 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:19.391 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:19.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:19.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:09:19.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:19.483 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:19.538 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:19.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:19.539 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:19.590 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:09:19.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.591 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:19.729 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:19.729 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:19.729 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:09:19.752 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:19.753 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:19.753 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:09:19.753 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:09:19.753 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:09:19.779 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:09:19.780 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:09:19.781 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:09:19.781 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:09:19.781 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:09:19.781 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:09:19.781 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:09:19.785 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:09:19.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:19.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.858 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:09:19.913 INFO:teuthology.orchestra.run.smithi005.stdout:client7764 2024-04-25T10:09:19.913 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:09:19.913 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:19.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:19.963 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T10:09:20.068 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-04-25T10:09:20.062+0000 7f6b9dc00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:09:20.069 INFO:teuthology.orchestra.run.smithi005.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T10:09:20.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:20.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T10:09:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.287+0000 7fcbf5e00640 1 Processor -- start 2024-04-25T10:09:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.288+0000 7fcbf5e00640 1 -- start start 2024-04-25T10:09:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.289+0000 7fcbf5e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbf00d0820 0x7fcbf00b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.289+0000 7fcbf5e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf006c1f0 con 0x7fcbf006bb00 2024-04-25T10:09:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.289+0000 7fcbf5e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf006c4d0 con 0x7fcbf015db10 2024-04-25T10:09:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.289+0000 7fcbf5e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf006bfb0 con 0x7fcbf00d0820 2024-04-25T10:09:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.290+0000 7fcbf5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcbf006bb00 0x7fcbf00d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37050/0 (socket says 172.21.15.5:37050) 2024-04-25T10:09:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.290+0000 7fcbf5400640 1 -- 172.21.15.5:0/3795659396 learned_addr learned my addr 172.21.15.5:0/3795659396 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.290+0000 7fcbefe00640 1 --2- 172.21.15.5:0/3795659396 >> v2:172.21.15.80:3300/0 conn(0x7fcbf00d0820 0x7fcbf00b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.290+0000 7fcbef400640 1 -- 172.21.15.5:0/3795659396 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907041752 0 0) 0x7fcbf006c1f0 con 0x7fcbf006bb00 2024-04-25T10:09:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.290+0000 7fcbef400640 1 -- 172.21.15.5:0/3795659396 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd0003520 con 0x7fcbf006bb00 2024-04-25T10:09:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.291+0000 7fcbefe00640 1 -- 172.21.15.5:0/3795659396 >> v1:172.21.15.80:6789/0 conn(0x7fcbf015db10 legacy=0x7fcbf00d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.291+0000 7fcbefe00640 1 -- 172.21.15.5:0/3795659396 >> v1:172.21.15.5:6789/0 conn(0x7fcbf006bb00 legacy=0x7fcbf00d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.291+0000 7fcbef400640 1 -- 172.21.15.5:0/3795659396 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251108110 0 0) 0x7fcbd0003520 con 0x7fcbf006bb00 2024-04-25T10:09:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.291+0000 7fcbefe00640 1 -- 172.21.15.5:0/3795659396 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf006c7b0 con 0x7fcbf00d0820 2024-04-25T10:09:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.292+0000 7fcbefe00640 1 --2- 172.21.15.5:0/3795659396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf00d0820 0x7fcbf00b6ba0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009d90 tx=0x7fcbd802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31a6b886aad05ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.292+0000 7fcbef400640 1 -- 172.21.15.5:0/3795659396 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8001670 con 0x7fcbf00d0820 2024-04-25T10:09:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbef400640 1 -- 172.21.15.5:0/3795659396 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd8007540 con 0x7fcbf00d0820 2024-04-25T10:09:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbef400640 1 -- 172.21.15.5:0/3795659396 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd80078f0 con 0x7fcbf00d0820 2024-04-25T10:09:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbf5e00640 1 -- 172.21.15.5:0/3795659396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf00d0820 msgr2=0x7fcbf00b6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/3795659396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf00d0820 0x7fcbf00b6ba0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8009d90 tx=0x7fcbd802a680 comp rx=0 tx=0).stop 2024-04-25T10:09:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbf5e00640 1 -- 172.21.15.5:0/3795659396 shutdown_connections 2024-04-25T10:09:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/3795659396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf00d0820 0x7fcbf00b6ba0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbf5e00640 1 -- 172.21.15.5:0/3795659396 >> 172.21.15.5:0/3795659396 conn(0x7fcbf00b5210 msgr2=0x7fcbf00b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.293+0000 7fcbf5e00640 1 -- 172.21.15.5:0/3795659396 shutdown_connections 2024-04-25T10:09:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.294+0000 7fcbf5e00640 1 -- 172.21.15.5:0/3795659396 wait complete. 2024-04-25T10:09:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.295+0000 7fcbf5e00640 1 Processor -- start 2024-04-25T10:09:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.295+0000 7fcbf5e00640 1 -- start start 2024-04-25T10:09:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 0x7fcbf0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf5e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbf00d0820 0x7fcbf00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf015db10 0x7fcbf00b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf5e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015e070 con 0x7fcbf006bb00 2024-04-25T10:09:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf5e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015def0 con 0x7fcbf015db10 2024-04-25T10:09:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf5e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0162280 con 0x7fcbf00d0820 2024-04-25T10:09:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.296+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 0x7fcbf0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 0x7fcbf0155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45368/0 (socket says 172.21.15.5:45368) 2024-04-25T10:09:20.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1792641884 learned_addr learned my addr 172.21.15.5:0/1792641884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf015db10 0x7fcbf00b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbf00d0820 msgr2=0x7fcbf00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbf00d0820 0x7fcbf00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf015db10 msgr2=0x7fcbf00b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf015db10 0x7fcbf00b3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1792641884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00cb310 con 0x7fcbf006bb00 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.297+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 0x7fcbf0155cd0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c430 tx=0x7fcbe000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.298+0000 7fcbecc00640 1 -- 172.21.15.5:0/1792641884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0017070 con 0x7fcbf006bb00 2024-04-25T10:09:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.298+0000 7fcbecc00640 1 -- 172.21.15.5:0/1792641884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00052d0 con 0x7fcbf006bb00 2024-04-25T10:09:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.298+0000 7fcbecc00640 1 -- 172.21.15.5:0/1792641884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0003830 con 0x7fcbf006bb00 2024-04-25T10:09:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.298+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf00cb5f0 con 0x7fcbf006bb00 2024-04-25T10:09:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.298+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf00cbbb0 con 0x7fcbf006bb00 2024-04-25T10:09:20.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.300+0000 7fcbecc00640 1 -- 172.21.15.5:0/1792641884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbe001a070 con 0x7fcbf006bb00 2024-04-25T10:09:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.302+0000 7fcbecc00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcbcc06b4d0 0x7fcbcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.302+0000 7fcbecc00640 1 -- 172.21.15.5:0/1792641884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcbe0089740 con 0x7fcbf006bb00 2024-04-25T10:09:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.302+0000 7fcbefe00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcbcc06b4d0 0x7fcbcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.302+0000 7fcbfa1a6740 1 Processor -- start 2024-04-25T10:09:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.303+0000 7fcbfa1a6740 1 -- start start 2024-04-25T10:09:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.303+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d462c20 0x56276d4155b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.303+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d4450c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 0x56276d41a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276d37e920 con 0x56276d445600 2024-04-25T10:09:20.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276d327da0 con 0x56276d445d40 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbefe00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcbcc06b4d0 0x7fcbcc06d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8034000 tx=0x7fcbd802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56276d37a9c0 con 0x56276d462c20 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 0x56276d41a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 0x56276d41a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45384/0 (socket says 172.21.15.5:45384) 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 -- 172.21.15.5:0/4232992625 learned_addr learned my addr 172.21.15.5:0/4232992625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbefe00640 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d4450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d462c20 0x56276d4155b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 -- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d462c20 msgr2=0x56276d4155b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d462c20 0x56276d4155b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 -- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 msgr2=0x56276d4450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d4450c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.304+0000 7fcbf5400640 1 -- 172.21.15.5:0/4232992625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276d334290 con 0x56276d445600 2024-04-25T10:09:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.305+0000 7fcbf5400640 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 0x56276d41a410 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80079d0 tx=0x7fcbe8007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2db029fc5d3754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.305+0000 7fcbe5200640 1 -- 172.21.15.5:0/4232992625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8018070 con 0x56276d445600 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.305+0000 7fcbe5200640 1 -- 172.21.15.5:0/4232992625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8005ce0 con 0x56276d445600 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.305+0000 7fcbe5200640 1 -- 172.21.15.5:0/4232992625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe800da90 con 0x56276d445600 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/4232992625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 msgr2=0x56276d41a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 0x56276d41a410 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80079d0 tx=0x7fcbe8007e90 comp rx=0 tx=0).stop 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/4232992625 shutdown_connections 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d445600 0x56276d41a410 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d4450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/4232992625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d462c20 0x56276d4155b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/4232992625 >> 172.21.15.5:0/4232992625 conn(0x56276d417820 msgr2=0x56276d4600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/4232992625 shutdown_connections 2024-04-25T10:09:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/4232992625 wait complete. 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.306+0000 7fcbfa1a6740 1 -- start start 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.307+0000 7fcbfa1a6740 1 Processor -- start 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d440c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d470e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d4420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276d4199d0 con 0x56276d441c70 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276d31d260 con 0x56276d462c20 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56276d418dc0 con 0x56276d445d40 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbfa1a6740 10 client.0 _ll_register_callbacks cb 0x56276d33abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d4420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d4420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45390/0 (socket says 172.21.15.5:45390) 2024-04-25T10:09:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf5400640 1 -- 172.21.15.5:0/1993390176 learned_addr learned my addr 172.21.15.5:0/1993390176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d440c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf5400640 1 -- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 msgr2=0x56276d440c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbefe00640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d470e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d440c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.308+0000 7fcbf5400640 1 -- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 msgr2=0x56276d470e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d470e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbf5400640 1 -- 172.21.15.5:0/1993390176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56276d15f7b0 con 0x56276d441c70 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d4420f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8016040 tx=0x7fcbe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c8d48b50d18d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbdd600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbdd600640 1 -- 172.21.15.5:0/1993390176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8018030 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbdd600640 1 -- 172.21.15.5:0/1993390176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcbe8005ae0 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe8005ce0 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbdd600640 1 -- 172.21.15.5:0/1993390176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe802c050 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbfa1a6740 10 client.7776 fetch_fsmap learned FSMap version 67 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.309+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56276d1bff40 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbdd600640 1 -- 172.21.15.5:0/1993390176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8005ce0 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbdd600640 1 -- 172.21.15.5:0/1993390176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcbe802f020 con 0x56276d441c70 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbfa1a6740 10 client.7776 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbfa1a6740 10 client.7776 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbfa1a6740 20 client.7776 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d446700 0x56276d446ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbfa1a6740 4 client.7776 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:20.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56276d38be60 con 0x56276d446700 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.310+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d446700 0x56276d446ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.311+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d446700 0x56276d446ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4db224ac9e7c5eb server_cookie=7fa46f500a8029f2 in_seq=0 out_seq=0 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.311+0000 7fcbdd600640 10 client.7776 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbdd600640 1 -- 172.21.15.5:0/1993390176 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56276d38be60 con 0x56276d446700 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbdd600640 10 client.7776 handle_command_reply: tid=0 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 client.7776 shutdown 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 msgr2=0x56276d4420f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d4420f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8016040 tx=0x7fcbe80057f0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 shutdown_connections 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d446700 0x56276d446ae0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d4420f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d470e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1993390176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d440c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbdd600640 0 client.7776 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 >> 172.21.15.5:0/1993390176 conn(0x56276d445600 msgr2=0x56276d4163f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 shutdown_connections 2024-04-25T10:09:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1993390176 wait complete. 2024-04-25T10:09:20.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.312+0000 7fcbfa1a6740 20 client.7776 trim_cache size 0 max 16384 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.329+0000 7fcbfa1a6740 1 Processor -- start 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.329+0000 7fcbfa1a6740 1 -- start start 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.330+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d41b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.330+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d479980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.330+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d479ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.330+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0200110 con 0x56276d441c70 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.330+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01ff310 con 0x56276d462c20 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.330+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00b84a0 con 0x56276d445d40 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d41b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d41b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45404/0 (socket says 172.21.15.5:45404) 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1328045735 learned_addr learned my addr 172.21.15.5:0/1328045735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 msgr2=0x56276d479980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d479980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 msgr2=0x56276d479ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d479ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d479ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.331+0000 7fcbf4a00640 1 -- 172.21.15.5:0/1328045735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276d473060 con 0x56276d441c70 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.332+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d41b050 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00579d0 tx=0x7fcbe000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6671f6b5ee69466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.332+0000 7fcbdfe00640 1 -- 172.21.15.5:0/1328045735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0055d70 con 0x56276d441c70 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.332+0000 7fcbdfe00640 1 -- 172.21.15.5:0/1328045735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0087e50 con 0x56276d441c70 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.332+0000 7fcbdfe00640 1 -- 172.21.15.5:0/1328045735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0055fd0 con 0x56276d441c70 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.332+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1328045735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 msgr2=0x56276d41b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.332+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d41b050 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00579d0 tx=0x7fcbe000a000 comp rx=0 tx=0).stop 2024-04-25T10:09:20.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1328045735 shutdown_connections 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d462c20 0x56276d479ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d445d40 0x56276d479980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1328045735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d441c70 0x56276d41b050 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1328045735 >> 172.21.15.5:0/1328045735 conn(0x56276d445600 msgr2=0x56276d444eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1328045735 shutdown_connections 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1328045735 wait complete. 2024-04-25T10:09:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.333+0000 7fcbfa1a6740 1 -- start start 2024-04-25T10:09:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.334+0000 7fcbfa1a6740 1 Processor -- start 2024-04-25T10:09:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.334+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d441c70 0x56276d446ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d447020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbfa1a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 0x56276d4ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276d46dc00 con 0x56276d462c20 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01fedb0 con 0x56276d445d40 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbfa1a6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56276d459050 con 0x56276d441c70 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbfa1a6740 10 client.0 _ll_register_callbacks cb 0x7fcbe8016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 0x56276d4ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 0x56276d4ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45420/0 (socket says 172.21.15.5:45420) 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 -- 172.21.15.5:0/1945491778 learned_addr learned my addr 172.21.15.5:0/1945491778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbefe00640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d447020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d441c70 0x56276d446ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 -- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d441c70 msgr2=0x56276d446ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d441c70 0x56276d446ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 -- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 msgr2=0x56276d447020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d447020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbf5400640 1 -- 172.21.15.5:0/1945491778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56276d0530a0 con 0x56276d462c20 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.335+0000 7fcbefe00640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d447020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbf5400640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 0x56276d4ce890 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x56276d441340 tx=0x7fcbe8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c81c0fae342faeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbdd600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbdd600640 1 -- 172.21.15.5:0/1945491778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8018050 con 0x56276d462c20 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbdd600640 1 -- 172.21.15.5:0/1945491778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcbe8039c90 con 0x56276d462c20 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe0087e50 con 0x56276d462c20 2024-04-25T10:09:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbdd600640 1 -- 172.21.15.5:0/1945491778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe8013680 con 0x56276d462c20 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbfa1a6740 10 client.7782 fetch_fsmap learned FSMap version 67 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.336+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56276d41b6d0 con 0x56276d462c20 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbdd600640 1 -- 172.21.15.5:0/1945491778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0087e50 con 0x56276d462c20 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbdd600640 1 -- 172.21.15.5:0/1945491778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcbe802f040 con 0x56276d462c20 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbfa1a6740 10 client.7782 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbfa1a6740 10 client.7782 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbfa1a6740 20 client.7782 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d4b3be0 0x56276d4cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbfa1a6740 4 client.7782 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T10:09:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x56276d3861b0 con 0x56276d4b3be0 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.337+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d4b3be0 0x56276d4cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbf4a00640 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d4b3be0 0x56276d4cedd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0cb27acf044edf server_cookie=bbf8cc36266db942 in_seq=0 out_seq=0 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbdd600640 10 client.7782 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbdd600640 1 -- 172.21.15.5:0/1945491778 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56276d3861b0 con 0x56276d4b3be0 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbdd600640 10 client.7782 handle_command_reply: tid=0 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbfa1a6740 1 client.7782 shutdown 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 msgr2=0x56276d4ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.338+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 0x56276d4ce890 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x56276d441340 tx=0x7fcbe8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 shutdown_connections 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56276d4b3be0 0x56276d4cedd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56276d462c20 0x56276d4ce890 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56276d445d40 0x56276d447020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 --2- 172.21.15.5:0/1945491778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56276d441c70 0x56276d446ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbdd600640 0 client.7782 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 >> 172.21.15.5:0/1945491778 conn(0x56276d445600 msgr2=0x56276d45fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 shutdown_connections 2024-04-25T10:09:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 1 -- 172.21.15.5:0/1945491778 wait complete. 2024-04-25T10:09:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.339+0000 7fcbfa1a6740 20 client.7782 trim_cache size 0 max 16384 2024-04-25T10:09:20.349 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-04-25T10:09:20.339+0000 7f6b9dc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0) n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x5639aa759b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:09:20.349 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:20.350 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:09:20.350 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "cdce7018-10fb-4647-a0ab-defa01d5bdb8", 2024-04-25T10:09:20.350 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:09:20.350 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcbcc06b4d0 msgr2=0x7fcbcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcbcc06b4d0 0x7fcbcc06d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8034000 tx=0x7fcbd802e000 comp rx=0 tx=0).stop 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 msgr2=0x7fcbf0155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 0x7fcbf0155cd0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c430 tx=0x7fcbe000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 shutdown_connections 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcbcc06b4d0 0x7fcbcc06d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbf015db10 0x7fcbf00b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbf00d0820 0x7fcbf00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 --2- 172.21.15.5:0/1792641884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf006bb00 0x7fcbf0155cd0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 >> 172.21.15.5:0/1792641884 conn(0x7fcbf00b5210 msgr2=0x7fcbf00b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 shutdown_connections 2024-04-25T10:09:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.340+0000 7fcbf5e00640 1 -- 172.21.15.5:0/1792641884 wait complete. 2024-04-25T10:09:20.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:09:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.565+0000 7f8324e00640 1 Processor -- start 2024-04-25T10:09:20.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.565+0000 7f8324e00640 1 -- start start 2024-04-25T10:09:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.567+0000 7f8324e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8320143fb0 0x7f83200b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.567+0000 7f8324e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200b2ee0 con 0x7f83200ced20 2024-04-25T10:09:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.567+0000 7f8324e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200b3150 con 0x7f832006e240 2024-04-25T10:09:20.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.567+0000 7f8324e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83200b3430 con 0x7f8320143fb0 2024-04-25T10:09:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.568+0000 7f831e800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f83200ced20 0x7f832015ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37052/0 (socket says 172.21.15.5:37052) 2024-04-25T10:09:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.568+0000 7f831e800640 1 -- 172.21.15.5:0/2079180725 learned_addr learned my addr 172.21.15.5:0/2079180725 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.568+0000 7f831de00640 1 --2- 172.21.15.5:0/2079180725 >> v2:172.21.15.80:3300/0 conn(0x7f8320143fb0 0x7f83200b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.568+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246655518 0 0) 0x7f83200b2ee0 con 0x7f83200ced20 2024-04-25T10:09:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.568+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8304005730 con 0x7f83200ced20 2024-04-25T10:09:20.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.568+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985393910 0 0) 0x7f8304005730 con 0x7f83200ced20 2024-04-25T10:09:20.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 >> v2:172.21.15.80:3300/0 conn(0x7f8320143fb0 msgr2=0x7f83200b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 --2- 172.21.15.5:0/2079180725 >> v2:172.21.15.80:3300/0 conn(0x7f8320143fb0 0x7f83200b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 >> v1:172.21.15.80:6789/0 conn(0x7f832006e240 legacy=0x7f83200ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200b3710 con 0x7f83200ced20 2024-04-25T10:09:20.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f83180019f0 con 0x7f83200ced20 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 >> v1:172.21.15.5:6789/0 conn(0x7f83200ced20 legacy=0x7f832015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8304005bc0 0x7f8304005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8304006a10 0x7f8304006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83040073d0 0x7f830400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200b3430 con 0x7f83040073d0 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830400e270 con 0x7f8304005bc0 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830400e470 con 0x7f8304006a10 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831e800640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83040073d0 0x7f830400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831de00640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8304005bc0 0x7f8304005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831d400640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8304006a10 0x7f8304006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.569+0000 7f831e800640 1 -- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8304006a10 msgr2=0x7f8304006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.570+0000 7f831e800640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8304006a10 0x7f8304006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.570+0000 7f831e800640 1 -- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8304005bc0 msgr2=0x7f8304005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.570+0000 7f831e800640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8304005bc0 0x7f8304005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.570+0000 7f831e800640 1 -- 172.21.15.5:0/2079180725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304005730 con 0x7f83040073d0 2024-04-25T10:09:20.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.570+0000 7f831d400640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8304006a10 0x7f8304006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.570+0000 7f831e800640 1 -- 172.21.15.5:0/2079180725 reap_dead start 2024-04-25T10:09:20.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.571+0000 7f831e800640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83040073d0 0x7f830400dbd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f831800a860 tx=0x7f8304007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb82a2e74d5c2eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.571+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800a480 con 0x7f83040073d0 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.571+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831802f070 con 0x7f83040073d0 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.571+0000 7f831ca00640 1 -- 172.21.15.5:0/2079180725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831802bca0 con 0x7f83040073d0 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.571+0000 7f8324e00640 1 -- 172.21.15.5:0/2079180725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83040073d0 msgr2=0x7f830400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.571+0000 7f8324e00640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83040073d0 0x7f830400dbd0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f831800a860 tx=0x7f8304007a40 comp rx=0 tx=0).stop 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.572+0000 7f8324e00640 1 -- 172.21.15.5:0/2079180725 shutdown_connections 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.572+0000 7f8324e00640 1 --2- 172.21.15.5:0/2079180725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83040073d0 0x7f830400dbd0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.572+0000 7f8324e00640 1 -- 172.21.15.5:0/2079180725 >> 172.21.15.5:0/2079180725 conn(0x7f832006ab60 msgr2=0x7f832006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.572+0000 7f8324e00640 1 -- 172.21.15.5:0/2079180725 shutdown_connections 2024-04-25T10:09:20.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.572+0000 7f8324e00640 1 -- 172.21.15.5:0/2079180725 wait complete. 2024-04-25T10:09:20.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.573+0000 7f8324e00640 1 Processor -- start 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.574+0000 7f8324e00640 1 -- start start 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.574+0000 7f8324e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83040073d0 0x7f83200a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.574+0000 7f8324e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 0x7f832012efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f8324e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83200a8450 0x7f832012f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f8324e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200b3c30 con 0x7f83200a9fd0 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f8324e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83200cf2f0 con 0x7f83040073d0 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f8324e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83200b3db0 con 0x7f83200a8450 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 0x7f832012efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 0x7f832012efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45446/0 (socket says 172.21.15.5:45446) 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 -- 172.21.15.5:0/2847757723 learned_addr learned my addr 172.21.15.5:0/2847757723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 -- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83200a8450 msgr2=0x7f832012f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83200a8450 0x7f832012f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 -- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83040073d0 msgr2=0x7f83200a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83040073d0 0x7f83200a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.575+0000 7f831d400640 1 -- 172.21.15.5:0/2847757723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200a89f0 con 0x7f83200a9fd0 2024-04-25T10:09:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.576+0000 7f831d400640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 0x7f832012efe0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8308000f60 tx=0x7f830800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.576+0000 7f8316000640 1 -- 172.21.15.5:0/2847757723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308005480 con 0x7f83200a9fd0 2024-04-25T10:09:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.576+0000 7f8316000640 1 -- 172.21.15.5:0/2847757723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308017070 con 0x7f83200a9fd0 2024-04-25T10:09:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.576+0000 7f8316000640 1 -- 172.21.15.5:0/2847757723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308003810 con 0x7f83200a9fd0 2024-04-25T10:09:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.576+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83200a8cd0 con 0x7f83200a9fd0 2024-04-25T10:09:20.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.576+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83200a9290 con 0x7f83200a9fd0 2024-04-25T10:09:20.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.579+0000 7f8316000640 1 -- 172.21.15.5:0/2847757723 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83080039b0 con 0x7f83200a9fd0 2024-04-25T10:09:20.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.579+0000 7f8316000640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82fc06b500 0x7f82fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.580+0000 7f8316000640 1 -- 172.21.15.5:0/2847757723 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f830808a330 con 0x7f83200a9fd0 2024-04-25T10:09:20.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.580+0000 7f8327d89740 1 Processor -- start 2024-04-25T10:09:20.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.580+0000 7f831de00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82fc06b500 0x7f82fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.580+0000 7f8327d89740 1 -- start start 2024-04-25T10:09:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a2efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f8327d89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a2edf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a318af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f8327d89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a251c70 con 0x55b86a32a280 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a1facf0 con 0x55b86a2edbb0 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a24dd10 con 0x55b86a318670 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.581+0000 7f831de00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82fc06b500 0x7f82fc06d9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8310009890 tx=0x7f8310008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a2edf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a2edf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45462/0 (socket says 172.21.15.5:45462) 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 -- 172.21.15.5:0/4088306123 learned_addr learned my addr 172.21.15.5:0/4088306123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831de00640 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a2efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 -- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 msgr2=0x55b86a318af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a318af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 -- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 msgr2=0x55b86a2efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a2efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 -- 172.21.15.5:0/4088306123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b86a0307d0 con 0x55b86a32a280 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f831d400640 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a2edf90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8308088550 tx=0x7f8308059060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7b70a190555b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.582+0000 7f830ea00640 1 -- 172.21.15.5:0/4088306123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308070020 con 0x55b86a32a280 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f830ea00640 1 -- 172.21.15.5:0/4088306123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308060f40 con 0x55b86a32a280 2024-04-25T10:09:20.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f830ea00640 1 -- 172.21.15.5:0/4088306123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308056b30 con 0x55b86a32a280 2024-04-25T10:09:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 -- 172.21.15.5:0/4088306123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 msgr2=0x55b86a2edf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a2edf90 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8308088550 tx=0x7f8308059060 comp rx=0 tx=0).stop 2024-04-25T10:09:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 -- 172.21.15.5:0/4088306123 shutdown_connections 2024-04-25T10:09:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a318af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a2edf90 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 --2- 172.21.15.5:0/4088306123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a2efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 -- 172.21.15.5:0/4088306123 >> 172.21.15.5:0/4088306123 conn(0x55b86a2b5a30 msgr2=0x55b86a333670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 -- 172.21.15.5:0/4088306123 shutdown_connections 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.583+0000 7f8327d89740 1 -- 172.21.15.5:0/4088306123 wait complete. 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.584+0000 7f8327d89740 1 -- start start 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.584+0000 7f8327d89740 1 Processor -- start 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 0x55b86a31bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a318670 0x55b86a32fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a309a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a1f03a0 con 0x55b86a2edbb0 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a3161c0 con 0x55b86a318670 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a31a2d0 con 0x55b86a32a280 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.585+0000 7f8327d89740 10 client.0 _ll_register_callbacks cb 0x55b86a20db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 0x55b86a31bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 0x55b86a31bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45478/0 (socket says 172.21.15.5:45478) 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 -- 172.21.15.5:0/74144401 learned_addr learned my addr 172.21.15.5:0/74144401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 -- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 msgr2=0x55b86a309a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a309a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831d400640 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a318670 0x55b86a32fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 -- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a318670 msgr2=0x55b86a32fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a318670 0x55b86a32fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.586+0000 7f831de00640 1 -- 172.21.15.5:0/74144401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b869f663f0 con 0x55b86a2edbb0 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f831de00640 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 0x55b86a31bb50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8310011fd0 tx=0x7f8310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c72abc9473383e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8303400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8303400640 1 -- 172.21.15.5:0/74144401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310025070 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8303400640 1 -- 172.21.15.5:0/74144401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8310021620 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8308060f40 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8303400640 1 -- 172.21.15.5:0/74144401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8310030050 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8327d89740 10 client.7794 fetch_fsmap learned FSMap version 67 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.587+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55b869f260f0 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8303400640 1 -- 172.21.15.5:0/74144401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308060f40 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8303400640 1 -- 172.21.15.5:0/74144401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8310033020 con 0x55b86a2edbb0 2024-04-25T10:09:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8327d89740 10 client.7794 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:09:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8327d89740 10 client.7794 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8327d89740 20 client.7794 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8327d89740 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a3467c0 0x55b86a346be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8327d89740 4 client.7794 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b86a25f1b0 con 0x55b86a3467c0 2024-04-25T10:09:20.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.588+0000 7f831d400640 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a3467c0 0x55b86a346be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.589+0000 7f831d400640 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a3467c0 0x55b86a346be0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40aece882662a2a server_cookie=7af640490e15b679 in_seq=0 out_seq=0 2024-04-25T10:09:20.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.589+0000 7f8303400640 10 client.7794 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.589+0000 7f8303400640 1 -- 172.21.15.5:0/74144401 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b86a25f1b0 con 0x55b86a3467c0 2024-04-25T10:09:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.589+0000 7f8303400640 10 client.7794 handle_command_reply: tid=0 2024-04-25T10:09:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.589+0000 7f8327d89740 1 client.7794 shutdown 2024-04-25T10:09:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 msgr2=0x55b86a31bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 0x55b86a31bb50 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8310011fd0 tx=0x7f8310004730 comp rx=0 tx=0).stop 2024-04-25T10:09:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 shutdown_connections 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a3467c0 0x55b86a346be0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a309a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a318670 0x55b86a32fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 --2- 172.21.15.5:0/74144401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a2edbb0 0x55b86a31bb50 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8303400640 0 client.7794 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 >> 172.21.15.5:0/74144401 conn(0x55b86a2b5a30 msgr2=0x55b86a3159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 shutdown_connections 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 1 -- 172.21.15.5:0/74144401 wait complete. 2024-04-25T10:09:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.590+0000 7f8327d89740 20 client.7794 trim_cache size 0 max 16384 2024-04-25T10:09:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.607+0000 7f8327d89740 1 Processor -- start 2024-04-25T10:09:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.607+0000 7f8327d89740 1 -- start start 2024-04-25T10:09:20.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a34d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a317bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f8327d89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a397f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f8327d89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320200c10 con 0x55b86a32a280 2024-04-25T10:09:20.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201ffd90 con 0x55b86a2edbb0 2024-04-25T10:09:20.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83201ff190 con 0x55b86a318670 2024-04-25T10:09:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f831e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a397f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f831e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a397f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45486/0 (socket says 172.21.15.5:45486) 2024-04-25T10:09:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.608+0000 7f831e800640 1 -- 172.21.15.5:0/1063760543 learned_addr learned my addr 172.21.15.5:0/1063760543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831de00640 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a34d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831d400640 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a317bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831e800640 1 -- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 msgr2=0x55b86a317bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831e800640 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a317bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831e800640 1 -- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 msgr2=0x55b86a34d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831e800640 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a34d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831e800640 1 -- 172.21.15.5:0/1063760543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b86a31c8a0 con 0x55b86a32a280 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.609+0000 7f831e800640 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a397f20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f831802b530 tx=0x7f83200b38e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1644cd6da5bd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f830e000640 1 -- 172.21.15.5:0/1063760543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318001930 con 0x55b86a32a280 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f830e000640 1 -- 172.21.15.5:0/1063760543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831802f070 con 0x55b86a32a280 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f830e000640 1 -- 172.21.15.5:0/1063760543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318004d80 con 0x55b86a32a280 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 -- 172.21.15.5:0/1063760543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 msgr2=0x55b86a397f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a397f20 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f831802b530 tx=0x7f83200b38e0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 -- 172.21.15.5:0/1063760543 shutdown_connections 2024-04-25T10:09:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a32a280 0x55b86a397f20 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a318670 0x55b86a317bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 --2- 172.21.15.5:0/1063760543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a34d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 -- 172.21.15.5:0/1063760543 >> 172.21.15.5:0/1063760543 conn(0x55b86a2b5a30 msgr2=0x55b86a355e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 -- 172.21.15.5:0/1063760543 shutdown_connections 2024-04-25T10:09:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.610+0000 7f8327d89740 1 -- 172.21.15.5:0/1063760543 wait complete. 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.611+0000 7f8327d89740 1 -- start start 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.611+0000 7f8327d89740 1 Processor -- start 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a394980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 0x55b86a332bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a32ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201ff790 con 0x55b86a318670 2024-04-25T10:09:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a284ca0 con 0x55b86a2edbb0 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a335d90 con 0x55b86a32a280 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.612+0000 7f8327d89740 10 client.0 _ll_register_callbacks cb 0x7f83100095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 0x55b86a332bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 0x55b86a332bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45502/0 (socket says 172.21.15.5:45502) 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 -- 172.21.15.5:0/2188505331 learned_addr learned my addr 172.21.15.5:0/2188505331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831de00640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a394980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831e800640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a32ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 -- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 msgr2=0x55b86a32ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a32ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 -- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 msgr2=0x55b86a394980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a394980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 -- 172.21.15.5:0/2188505331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b86a32c770 con 0x55b86a318670 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831de00640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a394980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f831d400640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 0x55b86a332bc0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8308054c90 tx=0x7f8308001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c0aa79c0efc85c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.613+0000 7f8303400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8303400640 1 -- 172.21.15.5:0/2188505331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308075060 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8303400640 1 -- 172.21.15.5:0/2188505331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8308088240 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f831802f070 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8303400640 1 -- 172.21.15.5:0/2188505331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830807c050 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 10 client.7800 fetch_fsmap learned FSMap version 67 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x55b86a3951c0 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8303400640 1 -- 172.21.15.5:0/2188505331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831802f070 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8303400640 1 -- 172.21.15.5:0/2188505331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83080772b0 con 0x55b86a318670 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 10 client.7800 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 10 client.7800 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 20 client.7800 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a32f230 0x55b86a334d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 4 client.7800 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.614+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b86a259500 con 0x55b86a32f230 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.615+0000 7f831de00640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a32f230 0x55b86a334d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.615+0000 7f831de00640 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a32f230 0x55b86a334d80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143cd7ddfc0d5695 server_cookie=50d0dc513d1ebd8e in_seq=0 out_seq=0 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.615+0000 7f8303400640 10 client.7800 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8303400640 1 -- 172.21.15.5:0/2188505331 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55b86a259500 con 0x55b86a32f230 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8303400640 10 client.7800 handle_command_reply: tid=0 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 client.7800 shutdown 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 msgr2=0x55b86a332bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 0x55b86a332bc0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8308054c90 tx=0x7f8308001120 comp rx=0 tx=0).stop 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 shutdown_connections 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55b86a32f230 0x55b86a334d80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b86a32a280 0x55b86a32ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86a318670 0x55b86a332bc0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8303400640 0 client.7800 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 --2- 172.21.15.5:0/2188505331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b86a2edbb0 0x55b86a394980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 >> 172.21.15.5:0/2188505331 conn(0x55b86a2b5a30 msgr2=0x55b86a32db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 shutdown_connections 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 1 -- 172.21.15.5:0/2188505331 wait complete. 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.616+0000 7f8327d89740 20 client.7800 trim_cache size 0 max 16384 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "cdce7018-10fb-4647-a0ab-defa01d5bdb8": { 2024-04-25T10:09:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir", 2024-04-25T10:09:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cdce7018-10fb-4647-a0ab-defa01d5bdb8", 2024-04-25T10:09:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive" 2024-04-25T10:09:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:09:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:09:20.627 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82fc06b500 msgr2=0x7f82fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82fc06b500 0x7f82fc06d9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8310009890 tx=0x7f8310008040 comp rx=0 tx=0).stop 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 msgr2=0x7f832012efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 0x7f832012efe0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8308000f60 tx=0x7f830800d9f0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 shutdown_connections 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f83200a8450 0x7f832012f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83200a9fd0 0x7f832012efe0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82fc06b500 0x7f82fc06d9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 --2- 172.21.15.5:0/2847757723 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83040073d0 0x7f83200a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 >> 172.21.15.5:0/2847757723 conn(0x7f832006ab60 msgr2=0x7f832006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 shutdown_connections 2024-04-25T10:09:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:20.617+0000 7f8324e00640 1 -- 172.21.15.5:0/2847757723 wait complete. 2024-04-25T10:09:20.642 INFO:tasks.cephfs.filesystem:scrub status for tag:cdce7018-10fb-4647-a0ab-defa01d5bdb8 - {'path': '/dir', 'tag': 'cdce7018-10fb-4647-a0ab-defa01d5bdb8', 'options': 'recursive'} 2024-04-25T10:09:23.472 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:09:23.465+0000 7f92a8200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:09:25.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:25.217 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:50.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:09:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.872+0000 7f56d3e00640 1 Processor -- start 2024-04-25T10:09:50.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.872+0000 7f56d3e00640 1 -- start start 2024-04-25T10:09:50.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.874+0000 7f56d3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d4144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.874+0000 7f56d3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56d4144e90 0x7f56d4071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.874+0000 7f56d3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d415f3e0 con 0x7f56d4071540 2024-04-25T10:09:50.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.874+0000 7f56d3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d4144780 con 0x7f56d4144e90 2024-04-25T10:09:50.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.874+0000 7f56d3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4071980 con 0x7f56d415ed20 2024-04-25T10:09:50.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.875+0000 7f56d2000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56d4144e90 0x7f56d4071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.875+0000 7f56d2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d4144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.875+0000 7f56d2000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56d4144e90 0x7f56d4071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:56882/0 (socket says 172.21.15.5:56882) 2024-04-25T10:09:50.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.875+0000 7f56d2000640 1 -- 172.21.15.5:0/135352448 learned_addr learned my addr 172.21.15.5:0/135352448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:50.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.876+0000 7f56d2000640 1 -- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 msgr2=0x7f56d4144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.876+0000 7f56d2000640 1 --2- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d4144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.876+0000 7f56d2000640 1 -- 172.21.15.5:0/135352448 >> v1:172.21.15.80:6789/0 conn(0x7f56d4071540 legacy=0x7f56d406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.876+0000 7f56d2000640 1 -- 172.21.15.5:0/135352448 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d415f680 con 0x7f56d4144e90 2024-04-25T10:09:50.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.876+0000 7f56d2a00640 1 --2- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d4144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:50.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.877+0000 7f56d2000640 1 --2- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4144e90 0x7f56d4071000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009870 tx=0x7f56c002a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0188ec4782b9ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.878+0000 7f56d1600640 1 -- 172.21.15.5:0/135352448 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037070 con 0x7f56d4144e90 2024-04-25T10:09:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.878+0000 7f56d1600640 1 -- 172.21.15.5:0/135352448 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0032440 con 0x7f56d4144e90 2024-04-25T10:09:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.878+0000 7f56d1600640 1 -- 172.21.15.5:0/135352448 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0032810 con 0x7f56d4144e90 2024-04-25T10:09:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.878+0000 7f56d3e00640 1 -- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4144e90 msgr2=0x7f56d4071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.878+0000 7f56d3e00640 1 --2- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4144e90 0x7f56d4071000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009870 tx=0x7f56c002a1d0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.879+0000 7f56d3e00640 1 -- 172.21.15.5:0/135352448 shutdown_connections 2024-04-25T10:09:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.879+0000 7f56d3e00640 1 --2- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4144e90 0x7f56d4071000 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.879+0000 7f56d3e00640 1 --2- 172.21.15.5:0/135352448 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d4144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.879+0000 7f56d3e00640 1 -- 172.21.15.5:0/135352448 >> 172.21.15.5:0/135352448 conn(0x7f56d4066f20 msgr2=0x7f56d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.879+0000 7f56d3e00640 1 -- 172.21.15.5:0/135352448 shutdown_connections 2024-04-25T10:09:50.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.879+0000 7f56d3e00640 1 -- 172.21.15.5:0/135352448 wait complete. 2024-04-25T10:09:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.880+0000 7f56d3e00640 1 Processor -- start 2024-04-25T10:09:50.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.881+0000 7f56d3e00640 1 -- start start 2024-04-25T10:09:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.881+0000 7f56d3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4071540 0x7f56d41e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 0x7f56d41e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d40cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40cdd50 con 0x7f56d4144e90 2024-04-25T10:09:50.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d415fc40 con 0x7f56d4071540 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d415cdd0 con 0x7f56d415ed20 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 0x7f56d41e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 0x7f56d41e8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54804/0 (socket says 172.21.15.5:54804) 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 -- 172.21.15.5:0/3553275604 learned_addr learned my addr 172.21.15.5:0/3553275604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2a00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4071540 0x7f56d41e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 -- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 msgr2=0x7f56d40cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d40cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 -- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4071540 msgr2=0x7f56d41e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4071540 0x7f56d41e5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.882+0000 7f56d2000640 1 -- 172.21.15.5:0/3553275604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d412ec10 con 0x7f56d4144e90 2024-04-25T10:09:50.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.883+0000 7f56d2000640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 0x7f56d41e8b40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.883+0000 7f56caa00640 1 -- 172.21.15.5:0/3553275604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037040 con 0x7f56d4144e90 2024-04-25T10:09:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.883+0000 7f56caa00640 1 -- 172.21.15.5:0/3553275604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0042070 con 0x7f56d4144e90 2024-04-25T10:09:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.883+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d412eef0 con 0x7f56d4144e90 2024-04-25T10:09:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.883+0000 7f56caa00640 1 -- 172.21.15.5:0/3553275604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0030880 con 0x7f56d4144e90 2024-04-25T10:09:50.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.883+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d412f4b0 con 0x7f56d4144e90 2024-04-25T10:09:50.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.885+0000 7f56caa00640 1 -- 172.21.15.5:0/3553275604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56c0030a20 con 0x7f56d4144e90 2024-04-25T10:09:50.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.886+0000 7f56caa00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56b406b500 0x7f56b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.886+0000 7f56caa00640 1 -- 172.21.15.5:0/3553275604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f56c00aae20 con 0x7f56d4144e90 2024-04-25T10:09:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.887+0000 7f56d2a00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56b406b500 0x7f56b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.887+0000 7f56dc7c0740 1 Processor -- start 2024-04-25T10:09:50.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.887+0000 7f56dc7c0740 1 -- start start 2024-04-25T10:09:50.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e6f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56d2a00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56b406b500 0x7f56b406d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009950 tx=0x7f56c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e72c440 0x56011e6f0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e71a830 0x56011e71acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011e653c70 con 0x56011e6efd70 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011e5fccf0 con 0x56011e72c440 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.888+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011e64fd10 con 0x56011e71a830 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e6f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e6f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54820/0 (socket says 172.21.15.5:54820) 2024-04-25T10:09:50.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 -- 172.21.15.5:0/3277934707 learned_addr learned my addr 172.21.15.5:0/3277934707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2000640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e72c440 0x56011e6f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d3400640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e71a830 0x56011e71acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 -- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e71a830 msgr2=0x56011e71acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e71a830 0x56011e71acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 -- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e72c440 msgr2=0x56011e6f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e72c440 0x56011e6f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2a00640 1 -- 172.21.15.5:0/3277934707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56011e4327d0 con 0x56011e6efd70 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d2000640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e72c440 0x56011e6f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.889+0000 7f56d3400640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e71a830 0x56011e71acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.890+0000 7f56d2a00640 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e6f1ed0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f56c4010a00 tx=0x7f56c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b414870490c0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.890+0000 7f56bf400640 1 -- 172.21.15.5:0/3277934707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c401c050 con 0x56011e6efd70 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.890+0000 7f56bf400640 1 -- 172.21.15.5:0/3277934707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4026070 con 0x56011e6efd70 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.890+0000 7f56bf400640 1 -- 172.21.15.5:0/3277934707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c400f040 con 0x56011e6efd70 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 -- 172.21.15.5:0/3277934707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 msgr2=0x56011e6f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e6f1ed0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f56c4010a00 tx=0x7f56c4013000 comp rx=0 tx=0).stop 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 -- 172.21.15.5:0/3277934707 shutdown_connections 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e71a830 0x56011e71acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e72c440 0x56011e6f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/3277934707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e6f1ed0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 -- 172.21.15.5:0/3277934707 >> 172.21.15.5:0/3277934707 conn(0x56011e6b7a30 msgr2=0x56011e735830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 -- 172.21.15.5:0/3277934707 shutdown_connections 2024-04-25T10:09:50.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.891+0000 7f56dc7c0740 1 -- 172.21.15.5:0/3277934707 wait complete. 2024-04-25T10:09:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.892+0000 7f56dc7c0740 1 -- start start 2024-04-25T10:09:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.892+0000 7f56dc7c0740 1 Processor -- start 2024-04-25T10:09:50.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e71b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e71cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e72c440 0x56011e71d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011e5f23a0 con 0x56011e6efd70 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011e718380 con 0x56011e71a830 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011e7458a0 con 0x56011e72c440 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56dc7c0740 10 client.0 _ll_register_callbacks cb 0x56011e60fb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e71b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e71b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54824/0 (socket says 172.21.15.5:54824) 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.893+0000 7f56d2a00640 1 -- 172.21.15.5:0/2994871153 learned_addr learned my addr 172.21.15.5:0/2994871153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2000640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e71cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2a00640 1 -- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e72c440 msgr2=0x56011e71d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d3400640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e72c440 0x56011e71d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2a00640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e72c440 0x56011e71d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2a00640 1 -- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 msgr2=0x56011e71cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2a00640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e71cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2a00640 1 -- 172.21.15.5:0/2994871153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56011e3683f0 con 0x56011e6efd70 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2000640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e71cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d3400640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e72c440 0x56011e71d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56d2a00640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e71b6c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f56c4008010 tx=0x7f56c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4b9dad3f8e7d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.894+0000 7f56b3e00640 1 -- 172.21.15.5:0/2994871153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c402f070 con 0x56011e6efd70 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56b3e00640 1 -- 172.21.15.5:0/2994871153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f56c4003a20 con 0x56011e6efd70 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56c4026070 con 0x56011e6efd70 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56b3e00640 1 -- 172.21.15.5:0/2994871153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c401c020 con 0x56011e6efd70 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 10 client.7809 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x56011e3280f0 con 0x56011e6efd70 2024-04-25T10:09:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56b3e00640 1 -- 172.21.15.5:0/2994871153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4026070 con 0x56011e6efd70 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56b3e00640 1 -- 172.21.15.5:0/2994871153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56c4003c60 con 0x56011e6efd70 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 10 client.7809 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 10 client.7809 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 20 client.7809 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e70b390 0x56011e749110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 4 client.7809 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.895+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56011e6611b0 con 0x56011e70b390 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.896+0000 7f56d2000640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e70b390 0x56011e749110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.896+0000 7f56d2000640 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e70b390 0x56011e749110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12e50a06aaa401 server_cookie=61d3f9c040151e17 in_seq=0 out_seq=0 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.896+0000 7f56b3e00640 10 client.7809 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56b3e00640 1 -- 172.21.15.5:0/2994871153 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56011e6611b0 con 0x56011e70b390 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56b3e00640 10 client.7809 handle_command_reply: tid=0 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 client.7809 shutdown 2024-04-25T10:09:50.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 msgr2=0x56011e71b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e71b6c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f56c4008010 tx=0x7f56c40046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 shutdown_connections 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e70b390 0x56011e749110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e72c440 0x56011e71d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e71cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/2994871153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e6efd70 0x56011e71b6c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56b3e00640 0 client.7809 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 >> 172.21.15.5:0/2994871153 conn(0x56011e6b7a30 msgr2=0x56011e72e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 shutdown_connections 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.897+0000 7f56dc7c0740 1 -- 172.21.15.5:0/2994871153 wait complete. 2024-04-25T10:09:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.898+0000 7f56dc7c0740 20 client.7809 trim_cache size 0 max 16384 2024-04-25T10:09:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.914+0000 7f56dc7c0740 1 Processor -- start 2024-04-25T10:09:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.914+0000 7f56dc7c0740 1 -- start start 2024-04-25T10:09:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.915+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e74f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.915+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e74f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.915+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e7159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.915+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4200860 con 0x56011e72c440 2024-04-25T10:09:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.915+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d41ff9b0 con 0x56011e71a830 2024-04-25T10:09:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.915+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4131670 con 0x56011e6efd70 2024-04-25T10:09:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e7159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e7159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54834/0 (socket says 172.21.15.5:54834) 2024-04-25T10:09:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 -- 172.21.15.5:0/870784220 learned_addr learned my addr 172.21.15.5:0/870784220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d2a00640 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e74f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d2000640 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e74f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 -- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 msgr2=0x56011e74f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e74f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 -- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 msgr2=0x56011e74f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.916+0000 7f56d3400640 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e74f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.917+0000 7f56d3400640 1 -- 172.21.15.5:0/870784220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56011e7584b0 con 0x56011e72c440 2024-04-25T10:09:50.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.917+0000 7f56d3400640 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e7159e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f56d41f3a80 tx=0x7f56cc002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8871f419c0dbf35f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.917+0000 7f56bea00640 1 -- 172.21.15.5:0/870784220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00ec40 con 0x56011e72c440 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.917+0000 7f56bea00640 1 -- 172.21.15.5:0/870784220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc00c070 con 0x56011e72c440 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.917+0000 7f56bea00640 1 -- 172.21.15.5:0/870784220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc003970 con 0x56011e72c440 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 -- 172.21.15.5:0/870784220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 msgr2=0x56011e7159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e7159e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f56d41f3a80 tx=0x7f56cc002070 comp rx=0 tx=0).stop 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 -- 172.21.15.5:0/870784220 shutdown_connections 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e7159e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e74f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/870784220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e74f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 -- 172.21.15.5:0/870784220 >> 172.21.15.5:0/870784220 conn(0x56011e6b7a30 msgr2=0x56011e7385f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 -- 172.21.15.5:0/870784220 shutdown_connections 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.918+0000 7f56dc7c0740 1 -- 172.21.15.5:0/870784220 wait complete. 2024-04-25T10:09:50.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.919+0000 7f56dc7c0740 1 -- start start 2024-04-25T10:09:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.919+0000 7f56dc7c0740 1 Processor -- start 2024-04-25T10:09:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e7829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e7383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e72e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56011e6f1760 con 0x56011e72c440 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d41ff370 con 0x56011e71a830 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56011e70b1a0 con 0x56011e6efd70 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56dc7c0740 10 client.0 _ll_register_callbacks cb 0x7f56c4010900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e72e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e72e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54846/0 (socket says 172.21.15.5:54846) 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.920+0000 7f56d3400640 1 -- 172.21.15.5:0/676463361 learned_addr learned my addr 172.21.15.5:0/676463361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d2000640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e7383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d2a00640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e7829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d3400640 1 -- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 msgr2=0x56011e7829f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d3400640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e7829f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d3400640 1 -- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 msgr2=0x56011e7383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d3400640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e7383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d3400640 1 -- 172.21.15.5:0/676463361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56011e7a0270 con 0x56011e72c440 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d2a00640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e7829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56d3400640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e72e850 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x56011e748be0 tx=0x7f56cc001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b36f04cfada416b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.921+0000 7f56b3e00640 1 -- 172.21.15.5:0/676463361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc022070 con 0x56011e72c440 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56b3e00640 1 -- 172.21.15.5:0/676463361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f56cc003e00 con 0x56011e72c440 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56cc00c070 con 0x56011e72c440 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56b3e00640 1 -- 172.21.15.5:0/676463361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56cc0050d0 con 0x56011e72c440 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 10 client.7815 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:50.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x56011e785590 con 0x56011e72c440 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56b3e00640 1 -- 172.21.15.5:0/676463361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc00c070 con 0x56011e72c440 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56b3e00640 1 -- 172.21.15.5:0/676463361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56cc032020 con 0x56011e72c440 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 10 client.7815 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 10 client.7815 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 20 client.7815 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e72ed90 0x56011e7a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 4 client.7815 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.922+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56011e65b500 con 0x56011e72ed90 2024-04-25T10:09:50.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.923+0000 7f56d2a00640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e72ed90 0x56011e7a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:50.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.923+0000 7f56d2a00640 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e72ed90 0x56011e7a37f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea8222346125973 server_cookie=9c724055aff506e1 in_seq=0 out_seq=0 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.923+0000 7f56b3e00640 10 client.7815 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56b3e00640 1 -- 172.21.15.5:0/676463361 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56011e65b500 con 0x56011e72ed90 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56b3e00640 10 client.7815 handle_command_reply: tid=0 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 client.7815 shutdown 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 msgr2=0x56011e72e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e72e850 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x56011e748be0 tx=0x7f56cc001510 comp rx=0 tx=0).stop 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 shutdown_connections 2024-04-25T10:09:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x56011e72ed90 0x56011e7a37f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56011e72c440 0x56011e72e850 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56011e71a830 0x56011e7383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 --2- 172.21.15.5:0/676463361 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56011e6efd70 0x56011e7829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 >> 172.21.15.5:0/676463361 conn(0x56011e6b7a30 msgr2=0x56011e74f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56b3e00640 0 client.7815 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 shutdown_connections 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 1 -- 172.21.15.5:0/676463361 wait complete. 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.924+0000 7f56dc7c0740 20 client.7815 trim_cache size 0 max 16384 2024-04-25T10:09:50.934 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:09:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:09:50.935 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56b406b500 msgr2=0x7f56b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56b406b500 0x7f56b406d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009950 tx=0x7f56c4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 msgr2=0x7f56d41e8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 0x7f56d41e8b40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c00046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 shutdown_connections 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56d415ed20 0x7f56d40cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56d4144e90 0x7f56d41e8b40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56b406b500 0x7f56b406d9c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 --2- 172.21.15.5:0/3553275604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56d4071540 0x7f56d41e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 >> 172.21.15.5:0/3553275604 conn(0x7f56d4066f20 msgr2=0x7f56d4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 shutdown_connections 2024-04-25T10:09:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:50.925+0000 7f56d3e00640 1 -- 172.21.15.5:0/3553275604 wait complete. 2024-04-25T10:09:50.949 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:09:50.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.137+0000 7fc4dcc00640 1 Processor -- start 2024-04-25T10:09:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.138+0000 7fc4dcc00640 1 -- start start 2024-04-25T10:09:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4dcc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc4d8071ed0 0x7fc4d80ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4dcc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d815bdc0 con 0x7fc4d81529e0 2024-04-25T10:09:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4dcc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d815c060 con 0x7fc4d80ce890 2024-04-25T10:09:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4dcc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80b2340 con 0x7fc4d8071ed0 2024-04-25T10:09:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4d5e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc4d81529e0 0x7fc4d80c5ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58754/0 (socket says 172.21.15.5:58754) 2024-04-25T10:09:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4d5e00640 1 -- 172.21.15.5:0/2733835050 learned_addr learned my addr 172.21.15.5:0/2733835050 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.139+0000 7fc4cd400640 1 --2- 172.21.15.5:0/2733835050 >> v2:172.21.15.5:3300/0 conn(0x7fc4d8071ed0 0x7fc4d80ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.140+0000 7fc4d5400640 1 -- 172.21.15.5:0/2733835050 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245215722 0 0) 0x7fc4d815bdc0 con 0x7fc4d81529e0 2024-04-25T10:09:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.140+0000 7fc4d5400640 1 -- 172.21.15.5:0/2733835050 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4bc003520 con 0x7fc4d81529e0 2024-04-25T10:09:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.140+0000 7fc4cd400640 1 -- 172.21.15.5:0/2733835050 >> v1:172.21.15.80:6789/0 conn(0x7fc4d80ce890 legacy=0x7fc4d806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.140+0000 7fc4cd400640 1 -- 172.21.15.5:0/2733835050 >> v1:172.21.15.5:6789/0 conn(0x7fc4d81529e0 legacy=0x7fc4d80c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.140+0000 7fc4cd400640 1 -- 172.21.15.5:0/2733835050 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d815c340 con 0x7fc4d8071ed0 2024-04-25T10:09:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.141+0000 7fc4cd400640 1 --2- 172.21.15.5:0/2733835050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d8071ed0 0x7fc4d80ce350 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00097a0 tx=0x7fc4c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc656a82fad5351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.141+0000 7fc4d5400640 1 -- 172.21.15.5:0/2733835050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0037070 con 0x7fc4d8071ed0 2024-04-25T10:09:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.141+0000 7fc4d5400640 1 -- 172.21.15.5:0/2733835050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c0032440 con 0x7fc4d8071ed0 2024-04-25T10:09:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.141+0000 7fc4d5400640 1 -- 172.21.15.5:0/2733835050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0032810 con 0x7fc4d8071ed0 2024-04-25T10:09:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 -- 172.21.15.5:0/2733835050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d8071ed0 msgr2=0x7fc4d80ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/2733835050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d8071ed0 0x7fc4d80ce350 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00097a0 tx=0x7fc4c002a230 comp rx=0 tx=0).stop 2024-04-25T10:09:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 -- 172.21.15.5:0/2733835050 shutdown_connections 2024-04-25T10:09:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/2733835050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d8071ed0 0x7fc4d80ce350 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 -- 172.21.15.5:0/2733835050 >> 172.21.15.5:0/2733835050 conn(0x7fc4d8071150 msgr2=0x7fc4d8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 -- 172.21.15.5:0/2733835050 shutdown_connections 2024-04-25T10:09:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.142+0000 7fc4dcc00640 1 -- 172.21.15.5:0/2733835050 wait complete. 2024-04-25T10:09:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.144+0000 7fc4dcc00640 1 Processor -- start 2024-04-25T10:09:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.144+0000 7fc4dcc00640 1 -- start start 2024-04-25T10:09:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.145+0000 7fc4dcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4d8071ed0 0x7fc4d814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.145+0000 7fc4dcc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d80ce890 0x7fc4d81f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.145+0000 7fc4dcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 0x7fc4d814b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.145+0000 7fc4dcc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d815c900 con 0x7fc4d80b4420 2024-04-25T10:09:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.145+0000 7fc4dcc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80cecc0 con 0x7fc4d80ce890 2024-04-25T10:09:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.145+0000 7fc4dcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80c65e0 con 0x7fc4d8071ed0 2024-04-25T10:09:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 0x7fc4d814b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 0x7fc4d814b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54868/0 (socket says 172.21.15.5:54868) 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 -- 172.21.15.5:0/3208528525 learned_addr learned my addr 172.21.15.5:0/3208528525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4cd400640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d80ce890 0x7fc4d81f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d5e00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4d8071ed0 0x7fc4d814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 -- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4d8071ed0 msgr2=0x7fc4d814cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4d8071ed0 0x7fc4d814cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 -- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d80ce890 msgr2=0x7fc4d81f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d80ce890 0x7fc4d81f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.146+0000 7fc4d6800640 1 -- 172.21.15.5:0/3208528525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d814ba20 con 0x7fc4d80b4420 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.147+0000 7fc4d6800640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 0x7fc4d814b070 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c7a0 tx=0x7fc4d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.147+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0003b70 con 0x7fc4d80b4420 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.147+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d0003d30 con 0x7fc4d80b4420 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.147+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000aa90 con 0x7fc4d80b4420 2024-04-25T10:09:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.147+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d81f4e60 con 0x7fc4d80b4420 2024-04-25T10:09:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.147+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d81f5420 con 0x7fc4d80b4420 2024-04-25T10:09:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.148+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d8068fd0 con 0x7fc4d80b4420 2024-04-25T10:09:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.149+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4d000ac30 con 0x7fc4d80b4420 2024-04-25T10:09:51.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.150+0000 7fc4cea00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc4b406b500 0x7fc4b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.150+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7fc4d008a320 con 0x7fc4d80b4420 2024-04-25T10:09:51.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.151+0000 7fc4d5e00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc4b406b500 0x7fc4b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.152+0000 7fc4d5e00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc4b406b500 0x7fc4b406d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8005880 tx=0x7fc4c8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.155+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4d0058ed0 con 0x7fc4d80b4420 2024-04-25T10:09:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.394+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4d81f5660 con 0x7fc4d80b4420 2024-04-25T10:09:51.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.394+0000 7fc4cea00640 1 -- 172.21.15.5:0/3208528525 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fc4d00587d0 con 0x7fc4d80b4420 2024-04-25T10:09:51.400 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:51.401 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:23.419710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7683},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7683":{"gid":7683,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6834/7055779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":7055779},{"type":"v1","addr":"172.21.15.5:6834","nonce":7055779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:51.401 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 68 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc4b406b500 msgr2=0x7fc4b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc4b406b500 0x7fc4b406d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8005880 tx=0x7fc4c8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 msgr2=0x7fc4d814b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 0x7fc4d814b070 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c7a0 tx=0x7fc4d000cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 shutdown_connections 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4d80b4420 0x7fc4d814b070 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc4b406b500 0x7fc4b406d9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d80ce890 0x7fc4d81f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 --2- 172.21.15.5:0/3208528525 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc4d8071ed0 0x7fc4d814cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 >> 172.21.15.5:0/3208528525 conn(0x7fc4d8071150 msgr2=0x7fc4d8159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 shutdown_connections 2024-04-25T10:09:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.397+0000 7fc4dcc00640 1 -- 172.21.15.5:0/3208528525 wait complete. 2024-04-25T10:09:51.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:09:51.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.612+0000 7f2a0aa00640 1 Processor -- start 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.612+0000 7f2a0aa00640 1 -- start start 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.613+0000 7f2a0aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040b2c30 0x7f2a040ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.613+0000 7f2a0aa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a040d3760 0x7f2a040d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.613+0000 7f2a0aa00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a040d1a40 con 0x7f2a0415deb0 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.613+0000 7f2a0aa00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a040b3090 con 0x7f2a040d3760 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.613+0000 7f2a0aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040d1d20 con 0x7f2a040b2c30 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.614+0000 7f2a03e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a0415deb0 0x7f2a040b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58760/0 (socket says 172.21.15.5:58760) 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.614+0000 7f2a03e00640 1 -- 172.21.15.5:0/3722921111 learned_addr learned my addr 172.21.15.5:0/3722921111 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.614+0000 7f2a08a00640 1 --2- 172.21.15.5:0/3722921111 >> v2:172.21.15.5:3300/0 conn(0x7f2a040d3760 0x7f2a040d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.614+0000 7f2a03400640 1 --2- 172.21.15.5:0/3722921111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040b2c30 0x7f2a040ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.614+0000 7f2a02a00640 1 -- 172.21.15.5:0/3722921111 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929099650 0 0) 0x7f2a040d1a40 con 0x7f2a0415deb0 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.615+0000 7f2a02a00640 1 -- 172.21.15.5:0/3722921111 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29e8003520 con 0x7f2a0415deb0 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.615+0000 7f2a08a00640 1 -- 172.21.15.5:0/3722921111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040b2c30 msgr2=0x7f2a040ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.615+0000 7f2a08a00640 1 --2- 172.21.15.5:0/3722921111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040b2c30 0x7f2a040ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.615+0000 7f2a08a00640 1 -- 172.21.15.5:0/3722921111 >> v1:172.21.15.5:6789/0 conn(0x7f2a0415deb0 legacy=0x7f2a040b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.615+0000 7f2a08a00640 1 -- 172.21.15.5:0/3722921111 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a040d2000 con 0x7f2a040d3760 2024-04-25T10:09:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.616+0000 7f2a08a00640 1 --2- 172.21.15.5:0/3722921111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040d3760 0x7f2a040d3bf0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0097a0 tx=0x7f29fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54cf480e95fc983 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.616+0000 7f2a02a00640 1 -- 172.21.15.5:0/3722921111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc037070 con 0x7f2a040d3760 2024-04-25T10:09:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.616+0000 7f2a02a00640 1 -- 172.21.15.5:0/3722921111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc032440 con 0x7f2a040d3760 2024-04-25T10:09:51.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.616+0000 7f2a02a00640 1 -- 172.21.15.5:0/3722921111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc032810 con 0x7f2a040d3760 2024-04-25T10:09:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 -- 172.21.15.5:0/3722921111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040d3760 msgr2=0x7f2a040d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/3722921111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040d3760 0x7f2a040d3bf0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0097a0 tx=0x7f29fc02a180 comp rx=0 tx=0).stop 2024-04-25T10:09:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 -- 172.21.15.5:0/3722921111 shutdown_connections 2024-04-25T10:09:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/3722921111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040d3760 0x7f2a040d3bf0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/3722921111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040b2c30 0x7f2a040ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 -- 172.21.15.5:0/3722921111 >> 172.21.15.5:0/3722921111 conn(0x7f2a040726b0 msgr2=0x7f2a04072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 -- 172.21.15.5:0/3722921111 shutdown_connections 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.617+0000 7f2a0aa00640 1 -- 172.21.15.5:0/3722921111 wait complete. 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.619+0000 7f2a0aa00640 1 Processor -- start 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.619+0000 7f2a0aa00640 1 -- start start 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a0aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 0x7f2a041f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a0aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040d3760 0x7f2a040ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a0aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a0415deb0 0x7f2a040cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a0aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040b70f0 con 0x7f2a040b2c30 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a0aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040d25c0 con 0x7f2a0415deb0 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a0aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040d2ae0 con 0x7f2a040d3760 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 0x7f2a041f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 0x7f2a041f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54884/0 (socket says 172.21.15.5:54884) 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.620+0000 7f2a03e00640 1 -- 172.21.15.5:0/59684680 learned_addr learned my addr 172.21.15.5:0/59684680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03400640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040d3760 0x7f2a040ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a08a00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a0415deb0 0x7f2a040cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03e00640 1 -- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040d3760 msgr2=0x7f2a040ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03e00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040d3760 0x7f2a040ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03e00640 1 -- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a0415deb0 msgr2=0x7f2a040cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03e00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a0415deb0 0x7f2a040cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03e00640 1 -- 172.21.15.5:0/59684680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a040cc3f0 con 0x7f2a040b2c30 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f2a03e00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 0x7f2a041f4110 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f29f400cad0 tx=0x7f29f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.621+0000 7f29fbe00640 1 -- 172.21.15.5:0/59684680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4017070 con 0x7f2a040b2c30 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.622+0000 7f29fbe00640 1 -- 172.21.15.5:0/59684680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f400ac30 con 0x7f2a040b2c30 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.622+0000 7f29fbe00640 1 -- 172.21.15.5:0/59684680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4012400 con 0x7f2a040b2c30 2024-04-25T10:09:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.622+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a040cc6d0 con 0x7f2a040b2c30 2024-04-25T10:09:51.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.622+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a040ccc90 con 0x7f2a040b2c30 2024-04-25T10:09:51.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.624+0000 7f29fbe00640 1 -- 172.21.15.5:0/59684680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29f401a070 con 0x7f2a040b2c30 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.625+0000 7f29fbe00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29e006b4d0 0x7f29e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.625+0000 7f29fbe00640 1 -- 172.21.15.5:0/59684680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f29f408a290 con 0x7f2a040b2c30 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.626+0000 7f2a0d7e2740 1 Processor -- start 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.626+0000 7f2a03400640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29e006b4d0 0x7f29e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.626+0000 7f2a0d7e2740 1 -- start start 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d864ef370 0x558d864ef750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d8650d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 0x558d8650b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d86427ae0 con 0x558d864c5050 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d863d07c0 con 0x558d8650cc10 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d86423b80 con 0x558d864ef370 2024-04-25T10:09:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a08a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 0x558d8650b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a03400640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29e006b4d0 0x7f29e006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2a0415c4b0 tx=0x7f29ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a08a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 0x558d8650b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54886/0 (socket says 172.21.15.5:54886) 2024-04-25T10:09:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.627+0000 7f2a08a00640 1 -- 172.21.15.5:0/2732268250 learned_addr learned my addr 172.21.15.5:0/2732268250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a03e00640 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d864ef370 0x558d864ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a03400640 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d8650d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a08a00640 1 -- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d864ef370 msgr2=0x558d864ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a08a00640 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d864ef370 0x558d864ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a08a00640 1 -- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 msgr2=0x558d8650d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a08a00640 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d8650d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a08a00640 1 -- 172.21.15.5:0/2732268250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d862087c0 con 0x558d864c5050 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a03400640 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d8650d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f2a08a00640 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 0x558d8650b2d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f29fc035040 tx=0x7f29fc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acad1d9b00339e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f29f3e00640 1 -- 172.21.15.5:0/2732268250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc041070 con 0x558d864c5050 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f29f3e00640 1 -- 172.21.15.5:0/2732268250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc03c660 con 0x558d864c5050 2024-04-25T10:09:51.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.628+0000 7f29f3e00640 1 -- 172.21.15.5:0/2732268250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc037040 con 0x558d864c5050 2024-04-25T10:09:51.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2732268250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 msgr2=0x558d8650b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 0x558d8650b2d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f29fc035040 tx=0x7f29fc02c000 comp rx=0 tx=0).stop 2024-04-25T10:09:51.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2732268250 shutdown_connections 2024-04-25T10:09:51.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864c5050 0x558d8650b2d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d8650d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2732268250 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d864ef370 0x558d864ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2732268250 >> 172.21.15.5:0/2732268250 conn(0x558d864c1980 msgr2=0x558d8650a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2732268250 shutdown_connections 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.629+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2732268250 wait complete. 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.630+0000 7f2a0d7e2740 1 -- start start 2024-04-25T10:09:51.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.630+0000 7f2a0d7e2740 1 Processor -- start 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d864f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d8650b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d864c5df0 con 0x558d864ef370 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d863c5d40 con 0x558d8650cc10 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d864bf8b0 con 0x558d8650b100 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.631+0000 7f2a0d7e2740 10 client.0 _ll_register_callbacks cb 0x558d8613bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864e0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54896/0 (socket says 172.21.15.5:54896) 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 -- 172.21.15.5:0/3633781708 learned_addr learned my addr 172.21.15.5:0/3633781708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 -- 172.21.15.5:0/3633781708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 msgr2=0x558d8650b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d8650b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 -- 172.21.15.5:0/3633781708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 msgr2=0x558d864f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d864f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 -- 172.21.15.5:0/3633781708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d86268f50 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f2a03e00640 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864e0de0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f29f4088460 tx=0x7f29f4058800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d53794791467f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.632+0000 7f29f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f29f0c00640 1 -- 172.21.15.5:0/3633781708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4070020 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f29f0c00640 1 -- 172.21.15.5:0/3633781708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f29f40701c0 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29fc03c660 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f29f0c00640 1 -- 172.21.15.5:0/3633781708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29f40880d0 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 10 client.7839 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x558d860fc0b0 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f29f0c00640 1 -- 172.21.15.5:0/3633781708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc03c660 con 0x558d864ef370 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f29f0c00640 1 -- 172.21.15.5:0/3633781708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29f40609c0 con 0x558d864ef370 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 10 client.7839 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 10 client.7839 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 20 client.7839 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.633+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d8650d7c0 0x558d8650dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.634+0000 7f2a0d7e2740 4 client.7839 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.634+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d86486b60 con 0x558d8650d7c0 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.634+0000 7f2a03400640 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d8650d7c0 0x558d8650dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.634+0000 7f29f0c00640 10 client.7839 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.634+0000 7f2a03400640 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d8650d7c0 0x558d8650dba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a717c26375cdff server_cookie=a6455cb70e8129a4 in_seq=0 out_seq=0 2024-04-25T10:09:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f29f0c00640 1 -- 172.21.15.5:0/3633781708 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d86486b60 con 0x558d8650d7c0 2024-04-25T10:09:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f29f0c00640 10 client.7839 handle_command_reply: tid=0 2024-04-25T10:09:51.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 client.7839 shutdown 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 msgr2=0x558d864e0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864e0de0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f29f4088460 tx=0x7f29f4058800 comp rx=0 tx=0).stop 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 shutdown_connections 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d8650d7c0 0x558d8650dba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d8650b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d864f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/3633781708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864e0de0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f29f0c00640 0 client.7839 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.635+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 >> 172.21.15.5:0/3633781708 conn(0x558d864c5050 msgr2=0x558d864be6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.636+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 shutdown_connections 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.636+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/3633781708 wait complete. 2024-04-25T10:09:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.636+0000 7f2a0d7e2740 20 client.7839 trim_cache size 0 max 16384 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.649+0000 7f2a0d7e2740 1 Processor -- start 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.649+0000 7f2a0d7e2740 1 -- start start 2024-04-25T10:09:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d864c5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041ff6f0 con 0x558d864ef370 2024-04-25T10:09:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0415eed0 con 0x558d8650cc10 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a041ff0e0 con 0x558d8650b100 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864ee370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54902/0 (socket says 172.21.15.5:54902) 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 -- 172.21.15.5:0/2560861986 learned_addr learned my addr 172.21.15.5:0/2560861986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 -- 172.21.15.5:0/2560861986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 msgr2=0x558d864c5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 -- 172.21.15.5:0/2560861986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 msgr2=0x558d864c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d864c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.650+0000 7f2a03e00640 1 -- 172.21.15.5:0/2560861986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8650d520 con 0x558d864ef370 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a03e00640 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864ee370 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f29f4058440 tx=0x7f29f401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7201d6ede585a7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f29f3400640 1 -- 172.21.15.5:0/2560861986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4070040 con 0x558d864ef370 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f29f3400640 1 -- 172.21.15.5:0/2560861986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4081e50 con 0x558d864ef370 2024-04-25T10:09:51.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f29f3400640 1 -- 172.21.15.5:0/2560861986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4057b90 con 0x558d864ef370 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2560861986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 msgr2=0x558d864ee370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864ee370 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f29f4058440 tx=0x7f29f401e000 comp rx=0 tx=0).stop 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2560861986 shutdown_connections 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d8650cc10 0x558d864c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864c5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/2560861986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d864ef370 0x558d864ee370 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2560861986 >> 172.21.15.5:0/2560861986 conn(0x558d864c5050 msgr2=0x558d865027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.651+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2560861986 shutdown_connections 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.652+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/2560861986 wait complete. 2024-04-25T10:09:51.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.652+0000 7f2a0d7e2740 1 -- start start 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.652+0000 7f2a0d7e2740 1 Processor -- start 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d864ef370 0x558d86571b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864ebb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 0x558d86574450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d864df1a0 con 0x558d8650cc10 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d86501ad0 con 0x558d864ef370 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d864de7e0 con 0x558d8650b100 2024-04-25T10:09:51.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.653+0000 7f2a0d7e2740 10 client.0 _ll_register_callbacks cb 0x558d864bfd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:51.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 0x558d86574450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 0x558d86574450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54910/0 (socket says 172.21.15.5:54910) 2024-04-25T10:09:51.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 -- 172.21.15.5:0/966514021 learned_addr learned my addr 172.21.15.5:0/966514021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a03e00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d864ef370 0x558d86571b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a03400640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864ebb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 -- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 msgr2=0x558d864ebb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864ebb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 -- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d864ef370 msgr2=0x558d86571b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d864ef370 0x558d86571b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 -- 172.21.15.5:0/966514021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d865005c0 con 0x558d8650cc10 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a03e00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d864ef370 0x558d86571b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f2a08a00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 0x558d86574450 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0098c0 tx=0x7f29fc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484245d94ce78c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.654+0000 7f29f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f29f0c00640 1 -- 172.21.15.5:0/966514021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc04b060 con 0x558d8650cc10 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f29f0c00640 1 -- 172.21.15.5:0/966514021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f29fc003a20 con 0x558d8650cc10 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29f4081e50 con 0x558d8650cc10 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f29f0c00640 1 -- 172.21.15.5:0/966514021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29fc041030 con 0x558d8650cc10 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f2a0d7e2740 10 client.7845 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x558d864eb760 con 0x558d8650cc10 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.655+0000 7f29f0c00640 1 -- 172.21.15.5:0/966514021 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4081e50 con 0x558d8650cc10 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f29f0c00640 1 -- 172.21.15.5:0/966514021 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29fc003c20 con 0x558d8650cc10 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a0d7e2740 10 client.7845 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a0d7e2740 10 client.7845 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a0d7e2740 20 client.7845 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d864c5880 0x558d864c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a0d7e2740 4 client.7845 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x558d86435020 con 0x558d864c5880 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a03e00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d864c5880 0x558d864c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f2a03e00640 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d864c5880 0x558d864c2f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0941a6374539fb server_cookie=3268dbc03ac3be27 in_seq=0 out_seq=0 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.656+0000 7f29f0c00640 10 client.7845 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f29f0c00640 1 -- 172.21.15.5:0/966514021 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+418 (crc 0 0 0) 0x558d86435020 con 0x558d864c5880 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f29f0c00640 10 client.7845 handle_command_reply: tid=0 2024-04-25T10:09:51.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 client.7845 shutdown 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 msgr2=0x558d86574450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 0x558d86574450 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0098c0 tx=0x7f29fc0043b0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 shutdown_connections 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x558d864c5880 0x558d864c2f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d8650cc10 0x558d86574450 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558d8650b100 0x558d864ebb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 --2- 172.21.15.5:0/966514021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558d864ef370 0x558d86571b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f29f0c00640 0 client.7845 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 >> 172.21.15.5:0/966514021 conn(0x558d864c5050 msgr2=0x558d865027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 shutdown_connections 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 1 -- 172.21.15.5:0/966514021 wait complete. 2024-04-25T10:09:51.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.657+0000 7f2a0d7e2740 20 client.7845 trim_cache size 0 max 16384 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 414853104, 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/testdir/hardlink" 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-04-25T10:09:51.667 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 723922232, 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627776, 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir" 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "dir_frag", 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 3115835007, 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627776, 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "frag": "*", 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir" 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:09:51.668 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:09:51.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29e006b4d0 msgr2=0x7f29e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29e006b4d0 0x7f29e006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2a0415c4b0 tx=0x7f29ec008040 comp rx=0 tx=0).stop 2024-04-25T10:09:51.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 msgr2=0x7f2a041f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 0x7f2a041f4110 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f29f400cad0 tx=0x7f29f400cf90 comp rx=0 tx=0).stop 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 shutdown_connections 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f29e006b4d0 0x7f29e006d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a0415deb0 0x7f2a040cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a040d3760 0x7f2a040ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 --2- 172.21.15.5:0/59684680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a040b2c30 0x7f2a041f4110 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 >> 172.21.15.5:0/59684680 conn(0x7f2a040726b0 msgr2=0x7f2a0415d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 shutdown_connections 2024-04-25T10:09:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.658+0000 7f2a0aa00640 1 -- 172.21.15.5:0/59684680 wait complete. 2024-04-25T10:09:51.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:09:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.882+0000 7fbfe5400640 1 Processor -- start 2024-04-25T10:09:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.882+0000 7fbfe5400640 1 -- start start 2024-04-25T10:09:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfe5400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfe0071100 0x7fbfe00714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfe5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe00cd2c0 0x7fbfe00cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfe5400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe0071a20 con 0x7fbfe006d170 2024-04-25T10:09:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfe5400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe00cdd80 con 0x7fbfe0071100 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfe5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00ce060 con 0x7fbfe00cd2c0 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfde800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfe0071100 0x7fbfe00714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfde800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbfe0071100 0x7fbfe00714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54914/0 (socket says 172.21.15.5:54914) 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.884+0000 7fbfde800640 1 -- 172.21.15.5:0/3658938755 learned_addr learned my addr 172.21.15.5:0/3658938755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.885+0000 7fbfd5e00640 1 --2- 172.21.15.5:0/3658938755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe00cd2c0 0x7fbfe00cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.885+0000 7fbfde800640 1 -- 172.21.15.5:0/3658938755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe00cd2c0 msgr2=0x7fbfe00cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.885+0000 7fbfde800640 1 --2- 172.21.15.5:0/3658938755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe00cd2c0 0x7fbfe00cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.885+0000 7fbfde800640 1 -- 172.21.15.5:0/3658938755 >> v1:172.21.15.80:6789/0 conn(0x7fbfe006d170 legacy=0x7fbfe006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.885+0000 7fbfde800640 1 -- 172.21.15.5:0/3658938755 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe015c090 con 0x7fbfe0071100 2024-04-25T10:09:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.886+0000 7fbfde800640 1 --2- 172.21.15.5:0/3658938755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0071100 0x7fbfe00714e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009870 tx=0x7fbfd002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d848a76032aa31bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.886+0000 7fbfdde00640 1 -- 172.21.15.5:0/3658938755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0037070 con 0x7fbfe0071100 2024-04-25T10:09:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.886+0000 7fbfdde00640 1 -- 172.21.15.5:0/3658938755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd0032440 con 0x7fbfe0071100 2024-04-25T10:09:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.886+0000 7fbfdde00640 1 -- 172.21.15.5:0/3658938755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0032810 con 0x7fbfe0071100 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 -- 172.21.15.5:0/3658938755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0071100 msgr2=0x7fbfe00714e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3658938755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0071100 0x7fbfe00714e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009870 tx=0x7fbfd002a180 comp rx=0 tx=0).stop 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 -- 172.21.15.5:0/3658938755 shutdown_connections 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3658938755 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe00cd2c0 0x7fbfe00cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3658938755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe0071100 0x7fbfe00714e0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 -- 172.21.15.5:0/3658938755 >> 172.21.15.5:0/3658938755 conn(0x7fbfe006ab60 msgr2=0x7fbfe006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 -- 172.21.15.5:0/3658938755 shutdown_connections 2024-04-25T10:09:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.887+0000 7fbfe5400640 1 -- 172.21.15.5:0/3658938755 wait complete. 2024-04-25T10:09:51.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.889+0000 7fbfe5400640 1 Processor -- start 2024-04-25T10:09:51.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.889+0000 7fbfe5400640 1 -- start start 2024-04-25T10:09:51.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfe5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 0x7fbfe01f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfe5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe0071100 0x7fbfe01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfe5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfe00cd2c0 0x7fbfe01e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfe5400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe00cc260 con 0x7fbfe006d170 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfe5400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe015bd80 con 0x7fbfe00cd2c0 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfe5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe0145270 con 0x7fbfe0071100 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfde800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 0x7fbfe01f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfde800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 0x7fbfe01f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54922/0 (socket says 172.21.15.5:54922) 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.890+0000 7fbfde800640 1 -- 172.21.15.5:0/3381065886 learned_addr learned my addr 172.21.15.5:0/3381065886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfd5e00640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe0071100 0x7fbfe01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfdf200640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfe00cd2c0 0x7fbfe01e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfde800640 1 -- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe0071100 msgr2=0x7fbfe01f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfde800640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe0071100 0x7fbfe01f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfde800640 1 -- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfe00cd2c0 msgr2=0x7fbfe01e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfde800640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfe00cd2c0 0x7fbfe01e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfde800640 1 -- 172.21.15.5:0/3381065886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe01e7ec0 con 0x7fbfe006d170 2024-04-25T10:09:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfdf200640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfe00cd2c0 0x7fbfe01e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.891+0000 7fbfde800640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 0x7fbfe01f70b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0000c00 tx=0x7fbfd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.892+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0041020 con 0x7fbfe006d170 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.892+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd00306c0 con 0x7fbfe006d170 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.892+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe01e81a0 con 0x7fbfe006d170 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.892+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0037040 con 0x7fbfe006d170 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.892+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01e8760 con 0x7fbfe006d170 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.893+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe0074bb0 con 0x7fbfe006d170 2024-04-25T10:09:51.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.894+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfd0044020 con 0x7fbfe006d170 2024-04-25T10:09:51.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.895+0000 7fbfd7400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfbc06b500 0x7fbfbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.895+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7fbfd00aad10 con 0x7fbfe006d170 2024-04-25T10:09:51.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.896+0000 7fbfd5e00640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfbc06b500 0x7fbfbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.897+0000 7fbfd5e00640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfbc06b500 0x7fbfbc06d9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbfe006b600 tx=0x7fbfc8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:51.900+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfd00b0020 con 0x7fbfe006d170 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.139+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbfe015b8f0 con 0x7fbfe006d170 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.140+0000 7fbfd7400640 1 -- 172.21.15.5:0/3381065886 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fbfd004b030 con 0x7fbfe006d170 2024-04-25T10:09:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:52.146 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:09:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfbc06b500 msgr2=0x7fbfbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfbc06b500 0x7fbfbc06d9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbfe006b600 tx=0x7fbfc8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 msgr2=0x7fbfe01f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 0x7fbfe01f70b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0000c00 tx=0x7fbfd00046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 shutdown_connections 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbfbc06b500 0x7fbfbc06d9c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbfe00cd2c0 0x7fbfe01e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbfe0071100 0x7fbfe01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 --2- 172.21.15.5:0/3381065886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfe006d170 0x7fbfe01f70b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 >> 172.21.15.5:0/3381065886 conn(0x7fbfe006ab60 msgr2=0x7fbfe00b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 shutdown_connections 2024-04-25T10:09:52.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.142+0000 7fbfe5400640 1 -- 172.21.15.5:0/3381065886 wait complete. 2024-04-25T10:09:52.167 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:09:52.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:09:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.372+0000 7f4e2d400640 1 Processor -- start 2024-04-25T10:09:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.373+0000 7f4e2d400640 1 -- start start 2024-04-25T10:09:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.374+0000 7f4e2d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e2815db10 0x7f4e280b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.374+0000 7f4e2d400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e280d0820 0x7f4e28162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.374+0000 7f4e2d400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2806bf40 con 0x7f4e2806bb00 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e2d400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2806c1f0 con 0x7f4e2815db10 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e2d400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2806c4d0 con 0x7f4e280d0820 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e27e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e2815db10 0x7f4e280b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e27e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e2815db10 0x7f4e280b6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54932/0 (socket says 172.21.15.5:54932) 2024-04-25T10:09:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e27e00640 1 -- 172.21.15.5:0/808262885 learned_addr learned my addr 172.21.15.5:0/808262885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e280d0820 0x7f4e28162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e27e00640 1 -- 172.21.15.5:0/808262885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e280d0820 msgr2=0x7f4e28162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e27e00640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e280d0820 0x7f4e28162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.375+0000 7f4e27e00640 1 -- 172.21.15.5:0/808262885 >> v1:172.21.15.5:6789/0 conn(0x7f4e2806bb00 legacy=0x7f4e280d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.376+0000 7f4e27e00640 1 -- 172.21.15.5:0/808262885 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2806c7b0 con 0x7f4e2815db10 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.376+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e280d0820 0x7f4e28162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.377+0000 7f4e27e00640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2815db10 0x7f4e280b6ba0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009870 tx=0x7f4e1802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414caf699deefa93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.377+0000 7f4e27400640 1 -- 172.21.15.5:0/808262885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18037070 con 0x7f4e2815db10 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.377+0000 7f4e27400640 1 -- 172.21.15.5:0/808262885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e18032440 con 0x7f4e2815db10 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.377+0000 7f4e27400640 1 -- 172.21.15.5:0/808262885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18032810 con 0x7f4e2815db10 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 -- 172.21.15.5:0/808262885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2815db10 msgr2=0x7f4e280b6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2815db10 0x7f4e280b6ba0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009870 tx=0x7f4e1802a250 comp rx=0 tx=0).stop 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 -- 172.21.15.5:0/808262885 shutdown_connections 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e280d0820 0x7f4e28162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 --2- 172.21.15.5:0/808262885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2815db10 0x7f4e280b6ba0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 -- 172.21.15.5:0/808262885 >> 172.21.15.5:0/808262885 conn(0x7f4e280b5210 msgr2=0x7f4e280b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 -- 172.21.15.5:0/808262885 shutdown_connections 2024-04-25T10:09:52.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.378+0000 7f4e2d400640 1 -- 172.21.15.5:0/808262885 wait complete. 2024-04-25T10:09:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.380+0000 7f4e2d400640 1 Processor -- start 2024-04-25T10:09:52.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.381+0000 7f4e2d400640 1 -- start start 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e2d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 0x7f4e2814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e2d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e280d0820 0x7f4e281f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e2d400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e2815db10 0x7f4e280cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e2d400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2815e070 con 0x7f4e2806bb00 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e2d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2815def0 con 0x7f4e280d0820 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e2d400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280d20b0 con 0x7f4e2815db10 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 0x7f4e2814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 0x7f4e2814cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54938/0 (socket says 172.21.15.5:54938) 2024-04-25T10:09:52.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.382+0000 7f4e27e00640 1 -- 172.21.15.5:0/225657922 learned_addr learned my addr 172.21.15.5:0/225657922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e27e00640 1 -- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e2815db10 msgr2=0x7f4e280cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e280d0820 0x7f4e281f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e27e00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e2815db10 0x7f4e280cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e27e00640 1 -- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e280d0820 msgr2=0x7f4e281f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e27e00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e280d0820 0x7f4e281f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e27e00640 1 -- 172.21.15.5:0/225657922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e281f45a0 con 0x7f4e2806bb00 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e27e00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 0x7f4e2814cba0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000c00 tx=0x7f4e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e24c00640 1 -- 172.21.15.5:0/225657922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18037040 con 0x7f4e2806bb00 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e24c00640 1 -- 172.21.15.5:0/225657922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e18042070 con 0x7f4e2806bb00 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2815a210 con 0x7f4e2806bb00 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.383+0000 7f4e24c00640 1 -- 172.21.15.5:0/225657922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18030bd0 con 0x7f4e2806bb00 2024-04-25T10:09:52.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.384+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2815a7d0 con 0x7f4e2806bb00 2024-04-25T10:09:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.386+0000 7f4e24c00640 1 -- 172.21.15.5:0/225657922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e18047460 con 0x7f4e2806bb00 2024-04-25T10:09:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.387+0000 7f4e24c00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.387+0000 7f4e24c00640 1 -- 172.21.15.5:0/225657922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f4e180ab170 con 0x7f4e2806bb00 2024-04-25T10:09:52.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.388+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.388+0000 7f4e317ae740 1 Processor -- start 2024-04-25T10:09:52.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.388+0000 7f4e317ae740 1 -- start start 2024-04-25T10:09:52.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e0406b4d0 0x7f4e0406d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4e2815f440 tx=0x7f4e10009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c99c370 0x55ad1c99c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9b9c10 0x55ad1c9ba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 0x55ad1c9b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e317ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c8d4980 con 0x55ad1c972050 2024-04-25T10:09:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c87dde0 con 0x55ad1c9b9c10 2024-04-25T10:09:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.389+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c8d0a20 con 0x55ad1c99c370 2024-04-25T10:09:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 0x55ad1c9b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 0x55ad1c9b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54952/0 (socket says 172.21.15.5:54952) 2024-04-25T10:09:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 -- 172.21.15.5:0/1360591175 learned_addr learned my addr 172.21.15.5:0/1360591175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9b9c10 0x55ad1c9ba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e27e00640 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c99c370 0x55ad1c99c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 -- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c99c370 msgr2=0x55ad1c99c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c99c370 0x55ad1c99c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 -- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9b9c10 msgr2=0x55ad1c9ba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9b9c10 0x55ad1c9ba160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 -- 172.21.15.5:0/1360591175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad1c89be30 con 0x55ad1c972050 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.390+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 0x55ad1c9b82d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4e20008e20 tx=0x7f4e2000d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30c080b499baf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.391+0000 7f4e17e00640 1 -- 172.21.15.5:0/1360591175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000e850 con 0x55ad1c972050 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.391+0000 7f4e17e00640 1 -- 172.21.15.5:0/1360591175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2000e9f0 con 0x55ad1c972050 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.391+0000 7f4e17e00640 1 -- 172.21.15.5:0/1360591175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000ede0 con 0x55ad1c972050 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.391+0000 7f4e317ae740 1 -- 172.21.15.5:0/1360591175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 msgr2=0x55ad1c9b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.391+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 0x55ad1c9b82d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4e20008e20 tx=0x7f4e2000d7a0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 -- 172.21.15.5:0/1360591175 shutdown_connections 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c972050 0x55ad1c9b82d0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9b9c10 0x55ad1c9ba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1360591175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c99c370 0x55ad1c99c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 -- 172.21.15.5:0/1360591175 >> 172.21.15.5:0/1360591175 conn(0x55ad1c96e980 msgr2=0x55ad1c9b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 -- 172.21.15.5:0/1360591175 shutdown_connections 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.392+0000 7f4e317ae740 1 -- 172.21.15.5:0/1360591175 wait complete. 2024-04-25T10:09:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.393+0000 7f4e317ae740 1 -- start start 2024-04-25T10:09:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.393+0000 7f4e317ae740 1 Processor -- start 2024-04-25T10:09:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c9855f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c985db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9bab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c972df0 con 0x55ad1c99c370 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c8733c0 con 0x55ad1c9ba700 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c96c8b0 con 0x55ad1c9b9c10 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e317ae740 10 client.0 _ll_register_callbacks cb 0x55ad1c5e8c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c9855f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c9855f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54958/0 (socket says 172.21.15.5:54958) 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.394+0000 7f4e27e00640 1 -- 172.21.15.5:0/1300388580 learned_addr learned my addr 172.21.15.5:0/1300388580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c985db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e27e00640 1 -- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 msgr2=0x55ad1c985db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e27e00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c985db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e27e00640 1 -- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 msgr2=0x55ad1c9bab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e27e00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9bab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e27e00640 1 -- 172.21.15.5:0/1300388580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad1c6b17e0 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e27e00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c9855f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4e1802aaa0 tx=0x7f4e18045420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd77ddfbbb414072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e14c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e14c00640 1 -- 172.21.15.5:0/1300388580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18091170 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e14c00640 1 -- 172.21.15.5:0/1300388580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f4e18079880 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e2000e9f0 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.395+0000 7f4e14c00640 1 -- 172.21.15.5:0/1300388580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e1807d770 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 10 client.7866 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55ad1c711f70 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e14c00640 1 -- 172.21.15.5:0/1300388580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2000e9f0 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e14c00640 1 -- 172.21.15.5:0/1300388580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e1803a030 con 0x55ad1c99c370 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 10 client.7866 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 10 client.7866 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 20 client.7866 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:52.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c989770 0x55ad1c989bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 4 client.7866 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:52.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad1c8e1ec0 con 0x55ad1c989770 2024-04-25T10:09:52.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.396+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c989770 0x55ad1c989bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.397+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c989770 0x55ad1c989bc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9afbdf43b0423e6 server_cookie=3bf1c3d20b17d17d in_seq=0 out_seq=0 2024-04-25T10:09:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.397+0000 7f4e14c00640 10 client.7866 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e14c00640 1 -- 172.21.15.5:0/1300388580 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad1c8e1ec0 con 0x55ad1c989770 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e14c00640 10 client.7866 handle_command_reply: tid=0 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 client.7866 shutdown 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 msgr2=0x55ad1c9855f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c9855f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4e1802aaa0 tx=0x7f4e18045420 comp rx=0 tx=0).stop 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 shutdown_connections 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c989770 0x55ad1c989bc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9bab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c985db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 --2- 172.21.15.5:0/1300388580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c9855f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e14c00640 0 client.7866 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 >> 172.21.15.5:0/1300388580 conn(0x55ad1c972050 msgr2=0x55ad1c9b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 shutdown_connections 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 1 -- 172.21.15.5:0/1300388580 wait complete. 2024-04-25T10:09:52.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.398+0000 7f4e317ae740 20 client.7866 trim_cache size 0 max 16384 2024-04-25T10:09:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.415+0000 7f4e317ae740 1 Processor -- start 2024-04-25T10:09:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.415+0000 7f4e317ae740 1 -- start start 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.416+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c985400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.416+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c9c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.416+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.416+0000 7f4e317ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e282004a0 con 0x55ad1c99c370 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.416+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281ff5c0 con 0x55ad1c9ba700 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.416+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281605c0 con 0x55ad1c9b9c10 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c985400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c985400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54966/0 (socket says 172.21.15.5:54966) 2024-04-25T10:09:52.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 -- 172.21.15.5:0/888174591 learned_addr learned my addr 172.21.15.5:0/888174591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c9c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 -- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 msgr2=0x55ad1c9c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c9c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 -- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 msgr2=0x55ad1c9aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 -- 172.21.15.5:0/888174591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad1c5a9170 con 0x55ad1c99c370 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c9c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.417+0000 7f4e27e00640 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c985400 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4e1802aa70 tx=0x7f4e18008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b216c1db20e892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e17400640 1 -- 172.21.15.5:0/888174591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18009210 con 0x55ad1c99c370 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e17400640 1 -- 172.21.15.5:0/888174591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e18096070 con 0x55ad1c99c370 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e17400640 1 -- 172.21.15.5:0/888174591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18091170 con 0x55ad1c99c370 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 -- 172.21.15.5:0/888174591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 msgr2=0x55ad1c985400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c985400 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4e1802aa70 tx=0x7f4e18008000 comp rx=0 tx=0).stop 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 -- 172.21.15.5:0/888174591 shutdown_connections 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c9ba700 0x55ad1c9aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1c9c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 --2- 172.21.15.5:0/888174591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c99c370 0x55ad1c985400 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.418+0000 7f4e317ae740 1 -- 172.21.15.5:0/888174591 >> 172.21.15.5:0/888174591 conn(0x55ad1c972050 msgr2=0x55ad1c9b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.419+0000 7f4e317ae740 1 -- 172.21.15.5:0/888174591 shutdown_connections 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.419+0000 7f4e317ae740 1 -- 172.21.15.5:0/888174591 wait complete. 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.419+0000 7f4e317ae740 1 -- start start 2024-04-25T10:09:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.419+0000 7f4e317ae740 1 Processor -- start 2024-04-25T10:09:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c99c370 0x55ad1c9af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1ca23030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 0x55ad1c96ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c5a9170 con 0x55ad1c9ba700 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281fefc0 con 0x55ad1c99c370 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad1c96de40 con 0x55ad1c9b9c10 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.420+0000 7f4e317ae740 10 client.0 _ll_register_callbacks cb 0x55ad1c96cd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 0x55ad1c96ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 0x55ad1c96ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54978/0 (socket says 172.21.15.5:54978) 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 -- 172.21.15.5:0/424233172 learned_addr learned my addr 172.21.15.5:0/424233172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e27e00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c99c370 0x55ad1c9af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e1fe00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1ca23030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 -- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 msgr2=0x55ad1ca23030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1ca23030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 -- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c99c370 msgr2=0x55ad1c9af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c99c370 0x55ad1c9af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e2ca00640 1 -- 172.21.15.5:0/424233172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad1c9ad6d0 con 0x55ad1c9ba700 2024-04-25T10:09:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.421+0000 7f4e27e00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c99c370 0x55ad1c9af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e2ca00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 0x55ad1c96ba60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4e20018040 tx=0x7f4e20005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105e244ae7c9311f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e14c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e14c00640 1 -- 172.21.15.5:0/424233172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20025030 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e14c00640 1 -- 172.21.15.5:0/424233172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f4e2000e650 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e18096070 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e14c00640 1 -- 172.21.15.5:0/424233172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e2000e890 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e317ae740 10 client.7872 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.422+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55ad1c98a890 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e14c00640 1 -- 172.21.15.5:0/424233172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e18096070 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e14c00640 1 -- 172.21.15.5:0/424233172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4e2000ea30 con 0x55ad1c9ba700 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e317ae740 10 client.7872 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e317ae740 10 client.7872 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e317ae740 20 client.7872 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e317ae740 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c96c0b0 0x55ad1c9b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e317ae740 4 client.7872 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:09:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55ad1c8dc210 con 0x55ad1c96c0b0 2024-04-25T10:09:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.423+0000 7f4e27e00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c96c0b0 0x55ad1c9b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e27e00640 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c96c0b0 0x55ad1c9b81a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92d98325c12a13c server_cookie=88a1194d6e865098 in_seq=0 out_seq=0 2024-04-25T10:09:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e14c00640 10 client.7872 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e14c00640 1 -- 172.21.15.5:0/424233172 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ad1c8dc210 con 0x55ad1c96c0b0 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e14c00640 10 client.7872 handle_command_reply: tid=0 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 client.7872 shutdown 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 msgr2=0x55ad1c96ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 0x55ad1c96ba60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4e20018040 tx=0x7f4e20005db0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 shutdown_connections 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55ad1c96c0b0 0x55ad1c9b81a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad1c9ba700 0x55ad1c96ba60 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ad1c9b9c10 0x55ad1ca23030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 --2- 172.21.15.5:0/424233172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ad1c99c370 0x55ad1c9af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e14c00640 0 client.7872 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.424+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 >> 172.21.15.5:0/424233172 conn(0x55ad1c972050 msgr2=0x55ad1c9b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.425+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 shutdown_connections 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.425+0000 7f4e317ae740 1 -- 172.21.15.5:0/424233172 wait complete. 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.425+0000 7f4e317ae740 20 client.7872 trim_cache size 0 max 16384 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "05a9d737-e488-4769-81bd-82cc53c08fff", 2024-04-25T10:09:52.435 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:09:52.436 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:52.436 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-04-25T10:09:52.425+0000 7f6b9dc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0)->f(v1 m2024-04-25T10:09:08.406925+0000) n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1)->n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5639aa759b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e0406b4d0 msgr2=0x7f4e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e0406b4d0 0x7f4e0406d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4e2815f440 tx=0x7f4e10009040 comp rx=0 tx=0).stop 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 msgr2=0x7f4e2814cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 0x7f4e2814cba0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000c00 tx=0x7f4e180046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 shutdown_connections 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e2815db10 0x7f4e280cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e0406b4d0 0x7f4e0406d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e280d0820 0x7f4e281f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 --2- 172.21.15.5:0/225657922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2806bb00 0x7f4e2814cba0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 >> 172.21.15.5:0/225657922 conn(0x7f4e280b5210 msgr2=0x7f4e28152740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 shutdown_connections 2024-04-25T10:09:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.426+0000 7f4e2d400640 1 -- 172.21.15.5:0/225657922 wait complete. 2024-04-25T10:09:52.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:09:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.652+0000 7f2da3800640 1 Processor -- start 2024-04-25T10:09:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.653+0000 7f2da3800640 1 -- start start 2024-04-25T10:09:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.654+0000 7f2da3800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d9c06d130 0x7f2d9c071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.654+0000 7f2da3800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d9c071670 0x7f2d9c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.654+0000 7f2da3800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c1453c0 con 0x7f2d9c144fe0 2024-04-25T10:09:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.654+0000 7f2da3800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c0a8560 con 0x7f2d9c071670 2024-04-25T10:09:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.654+0000 7f2da3800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c06d6a0 con 0x7f2d9c06d130 2024-04-25T10:09:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d9c071670 0x7f2d9c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d9c071670 0x7f2d9c15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54980/0 (socket says 172.21.15.5:54980) 2024-04-25T10:09:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 -- 172.21.15.5:0/3154547999 learned_addr learned my addr 172.21.15.5:0/3154547999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3154547999 >> v2:172.21.15.80:3300/0 conn(0x7f2d9c06d130 0x7f2d9c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 -- 172.21.15.5:0/3154547999 >> v2:172.21.15.80:3300/0 conn(0x7f2d9c06d130 msgr2=0x7f2d9c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 --2- 172.21.15.5:0/3154547999 >> v2:172.21.15.80:3300/0 conn(0x7f2d9c06d130 0x7f2d9c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 -- 172.21.15.5:0/3154547999 >> v1:172.21.15.80:6789/0 conn(0x7f2d9c144fe0 legacy=0x7f2d9c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2da1800640 1 -- 172.21.15.5:0/3154547999 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c06d980 con 0x7f2d9c071670 2024-04-25T10:09:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.655+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3154547999 >> v2:172.21.15.80:3300/0 conn(0x7f2d9c06d130 0x7f2d9c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.656+0000 7f2da1800640 1 --2- 172.21.15.5:0/3154547999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c071670 0x7f2d9c15c370 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0cec80 tx=0x7f2d9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2e957311f46d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.657+0000 7f2d9b400640 1 -- 172.21.15.5:0/3154547999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94036070 con 0x7f2d9c071670 2024-04-25T10:09:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.657+0000 7f2d9b400640 1 -- 172.21.15.5:0/3154547999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94031600 con 0x7f2d9c071670 2024-04-25T10:09:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.657+0000 7f2d9b400640 1 -- 172.21.15.5:0/3154547999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940319d0 con 0x7f2d9c071670 2024-04-25T10:09:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.657+0000 7f2da3800640 1 -- 172.21.15.5:0/3154547999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c071670 msgr2=0x7f2d9c15c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.657+0000 7f2da3800640 1 --2- 172.21.15.5:0/3154547999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c071670 0x7f2d9c15c370 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0cec80 tx=0x7f2d9402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.658+0000 7f2da3800640 1 -- 172.21.15.5:0/3154547999 shutdown_connections 2024-04-25T10:09:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.658+0000 7f2da3800640 1 --2- 172.21.15.5:0/3154547999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c071670 0x7f2d9c15c370 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.658+0000 7f2da3800640 1 --2- 172.21.15.5:0/3154547999 >> v2:172.21.15.80:3300/0 conn(0x7f2d9c06d130 0x7f2d9c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.658+0000 7f2da3800640 1 -- 172.21.15.5:0/3154547999 >> 172.21.15.5:0/3154547999 conn(0x7f2d9c06ab60 msgr2=0x7f2d9c06af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.658+0000 7f2da3800640 1 -- 172.21.15.5:0/3154547999 shutdown_connections 2024-04-25T10:09:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.658+0000 7f2da3800640 1 -- 172.21.15.5:0/3154547999 wait complete. 2024-04-25T10:09:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.660+0000 7f2da3800640 1 Processor -- start 2024-04-25T10:09:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.660+0000 7f2da3800640 1 -- start start 2024-04-25T10:09:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 0x7f2d9c1f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da3800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d9c071670 0x7f2d9c15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da3800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d9c144fe0 0x7f2d9c1426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da3800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c15be50 con 0x7f2d9c06d130 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da3800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c06df40 con 0x7f2d9c071670 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da3800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c15c8b0 con 0x7f2d9c144fe0 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 0x7f2d9c1f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 0x7f2d9c1f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54984/0 (socket says 172.21.15.5:54984) 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.661+0000 7f2da0e00640 1 -- 172.21.15.5:0/3405088393 learned_addr learned my addr 172.21.15.5:0/3405088393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d9c071670 0x7f2d9c15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da0e00640 1 -- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d9c144fe0 msgr2=0x7f2d9c1426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d9c144fe0 0x7f2d9c1426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da0e00640 1 -- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d9c071670 msgr2=0x7f2d9c15b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d9c071670 0x7f2d9c15b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da0e00640 1 -- 172.21.15.5:0/3405088393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c1f7c70 con 0x7f2d9c06d130 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d9c071670 0x7f2d9c15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 0x7f2d9c1f7730 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000b410 tx=0x7f2d9000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2d98c00640 1 -- 172.21.15.5:0/3405088393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90015020 con 0x7f2d9c06d130 2024-04-25T10:09:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2d98c00640 1 -- 172.21.15.5:0/3405088393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90011830 con 0x7f2d9c06d130 2024-04-25T10:09:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c0b1600 con 0x7f2d9c06d130 2024-04-25T10:09:52.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2d98c00640 1 -- 172.21.15.5:0/3405088393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90011c20 con 0x7f2d9c06d130 2024-04-25T10:09:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.662+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c1e40b0 con 0x7f2d9c06d130 2024-04-25T10:09:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.664+0000 7f2d98c00640 1 -- 172.21.15.5:0/3405088393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d90018020 con 0x7f2d9c06d130 2024-04-25T10:09:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.665+0000 7f2d98c00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.666+0000 7f2d98c00640 1 -- 172.21.15.5:0/3405088393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f2d9008a460 con 0x7f2d9c06d130 2024-04-25T10:09:52.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.666+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.666+0000 7f2da6625740 1 Processor -- start 2024-04-25T10:09:52.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.667+0000 7f2da6625740 1 -- start start 2024-04-25T10:09:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.667+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006fd0 tx=0x7f2d88008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.667+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f29fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f29df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da6625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da6625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83f201c70 con 0x55f83f2c8670 2024-04-25T10:09:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83f1aacf0 con 0x55f83f2da280 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f83f1fdd10 con 0x55f83f29dbb0 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54998/0 (socket says 172.21.15.5:54998) 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 -- 172.21.15.5:0/3040141284 learned_addr learned my addr 172.21.15.5:0/3040141284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f29df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f29fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 -- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 msgr2=0x55f83f29fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f29fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 -- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 msgr2=0x55f83f29df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f29df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da1800640 1 -- 172.21.15.5:0/3040141284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f83efe07d0 con 0x55f83f2c8670 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.668+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f29fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.669+0000 7f2da1800640 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2c8af0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2d9402ace0 tx=0x7f2d9402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fa07e4c2533e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.669+0000 7f2d8d600640 1 -- 172.21.15.5:0/3040141284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94036050 con 0x55f83f2c8670 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.669+0000 7f2d8d600640 1 -- 172.21.15.5:0/3040141284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94042070 con 0x55f83f2c8670 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.669+0000 7f2d8d600640 1 -- 172.21.15.5:0/3040141284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94009510 con 0x55f83f2c8670 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 -- 172.21.15.5:0/3040141284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 msgr2=0x55f83f2c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2c8af0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2d9402ace0 tx=0x7f2d9402c000 comp rx=0 tx=0).stop 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 -- 172.21.15.5:0/3040141284 shutdown_connections 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2c8af0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f29df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 --2- 172.21.15.5:0/3040141284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f29fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 -- 172.21.15.5:0/3040141284 >> 172.21.15.5:0/3040141284 conn(0x55f83f265a30 msgr2=0x55f83f2e3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 -- 172.21.15.5:0/3040141284 shutdown_connections 2024-04-25T10:09:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.670+0000 7f2da6625740 1 -- 172.21.15.5:0/3040141284 wait complete. 2024-04-25T10:09:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.671+0000 7f2da6625740 1 -- start start 2024-04-25T10:09:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.671+0000 7f2da6625740 1 Processor -- start 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f29dbb0 0x55f83f2c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2da280 0x55f83f2de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83f1a03a0 con 0x55f83f2c8670 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83f2c61c0 con 0x55f83f29dbb0 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f83f2b5320 con 0x55f83f2da280 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2da6625740 10 client.0 _ll_register_callbacks cb 0x55f83f1bdb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2ddb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55006/0 (socket says 172.21.15.5:55006) 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.672+0000 7f2d9be00640 1 -- 172.21.15.5:0/1052254770 learned_addr learned my addr 172.21.15.5:0/1052254770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2da0e00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f29dbb0 0x55f83f2c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2da1800640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2da280 0x55f83f2de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d9be00640 1 -- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2da280 msgr2=0x55f83f2de0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d9be00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2da280 0x55f83f2de0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d9be00640 1 -- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f29dbb0 msgr2=0x55f83f2c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d9be00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f29dbb0 0x55f83f2c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d9be00640 1 -- 172.21.15.5:0/1052254770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f83ef163f0 con 0x55f83f2c8670 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2da0e00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f29dbb0 0x55f83f2c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d9be00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2ddb80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2d880126c0 tx=0x7f2d88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd2b88ee95d5536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d82000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d82000640 1 -- 172.21.15.5:0/1052254770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8801f610 con 0x55f83f2c8670 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2d82000640 1 -- 172.21.15.5:0/1052254770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f2d8801f7b0 con 0x55f83f2c8670 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.673+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d94042070 con 0x55f83f2c8670 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2d82000640 1 -- 172.21.15.5:0/1052254770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8802c050 con 0x55f83f2c8670 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 10 client.7884 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55f83eed60f0 con 0x55f83f2c8670 2024-04-25T10:09:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2d82000640 1 -- 172.21.15.5:0/1052254770 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94042070 con 0x55f83f2c8670 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2d82000640 1 -- 172.21.15.5:0/1052254770 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d8802f020 con 0x55f83f2c8670 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 10 client.7884 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 10 client.7884 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 20 client.7884 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f2b64b0 0x55f83f2b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 4 client.7884 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.674+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f83f20f1b0 con 0x55f83f2b64b0 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.675+0000 7f2da0e00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f2b64b0 0x55f83f2b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.675+0000 7f2da0e00640 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f2b64b0 0x55f83f2b68f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beda0ab0be827b1f server_cookie=edb4c371b6475945 in_seq=0 out_seq=0 2024-04-25T10:09:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.675+0000 7f2d82000640 10 client.7884 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2d82000640 1 -- 172.21.15.5:0/1052254770 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f83f20f1b0 con 0x55f83f2b64b0 2024-04-25T10:09:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2d82000640 10 client.7884 handle_command_reply: tid=0 2024-04-25T10:09:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 client.7884 shutdown 2024-04-25T10:09:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 msgr2=0x55f83f2ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2ddb80 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2d880126c0 tx=0x7f2d88004730 comp rx=0 tx=0).stop 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 shutdown_connections 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f2b64b0 0x55f83f2b68f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2da280 0x55f83f2de0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2c8670 0x55f83f2ddb80 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 --2- 172.21.15.5:0/1052254770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f29dbb0 0x55f83f2c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2d82000640 0 client.7884 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 >> 172.21.15.5:0/1052254770 conn(0x55f83f265a30 msgr2=0x55f83f2c5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 shutdown_connections 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.676+0000 7f2da6625740 1 -- 172.21.15.5:0/1052254770 wait complete. 2024-04-25T10:09:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.677+0000 7f2da6625740 20 client.7884 trim_cache size 0 max 16384 2024-04-25T10:09:52.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.693+0000 7f2da6625740 1 Processor -- start 2024-04-25T10:09:52.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.693+0000 7f2da6625740 1 -- start start 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da6625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 0x55f83f2cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2c8670 0x55f83f2c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f2c7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da6625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c2006d0 con 0x55f83f29dbb0 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1ff820 con 0x55f83f2da280 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c0adfb0 con 0x55f83f2c8670 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 0x55f83f2cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 0x55f83f2cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55014/0 (socket says 172.21.15.5:55014) 2024-04-25T10:09:52.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.694+0000 7f2da0e00640 1 -- 172.21.15.5:0/537503610 learned_addr learned my addr 172.21.15.5:0/537503610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da0e00640 1 -- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2c8670 msgr2=0x55f83f2c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2d9be00640 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2c8670 0x55f83f2c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da1800640 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f2c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da0e00640 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2c8670 0x55f83f2c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da0e00640 1 -- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 msgr2=0x55f83f2c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da0e00640 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f2c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da0e00640 1 -- 172.21.15.5:0/537503610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f83f2b5de0 con 0x55f83f29dbb0 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2d9be00640 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2c8670 0x55f83f2c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2da0e00640 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 0x55f83f2cba10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2d90058890 tx=0x7f2d90088580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7e6a2a76e038c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.695+0000 7f2d8cc00640 1 -- 172.21.15.5:0/537503610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90088070 con 0x55f83f29dbb0 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2d8cc00640 1 -- 172.21.15.5:0/537503610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90011dc0 con 0x55f83f29dbb0 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2d8cc00640 1 -- 172.21.15.5:0/537503610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d90054f70 con 0x55f83f29dbb0 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 -- 172.21.15.5:0/537503610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 msgr2=0x55f83f2cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 0x55f83f2cba10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2d90058890 tx=0x7f2d90088580 comp rx=0 tx=0).stop 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 -- 172.21.15.5:0/537503610 shutdown_connections 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2da280 0x55f83f2c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f2c8670 0x55f83f2c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 --2- 172.21.15.5:0/537503610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f29dbb0 0x55f83f2cba10 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 -- 172.21.15.5:0/537503610 >> 172.21.15.5:0/537503610 conn(0x55f83f265a30 msgr2=0x55f83f2e4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 -- 172.21.15.5:0/537503610 shutdown_connections 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.696+0000 7f2da6625740 1 -- 172.21.15.5:0/537503610 wait complete. 2024-04-25T10:09:52.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.697+0000 7f2da6625740 1 -- start start 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.697+0000 7f2da6625740 1 Processor -- start 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f34d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2c8670 0x55f83f29c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 0x55f83f2b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1ff1f0 con 0x55f83f2da280 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f83f297ce0 con 0x55f83f2c8670 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f83f29ce60 con 0x55f83f29dbb0 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.698+0000 7f2da6625740 10 client.0 _ll_register_callbacks cb 0x7f2d88008700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 0x55f83f2b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 0x55f83f2b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55030/0 (socket says 172.21.15.5:55030) 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 -- 172.21.15.5:0/3419168278 learned_addr learned my addr 172.21.15.5:0/3419168278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f34d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2d9be00640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2c8670 0x55f83f29c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 -- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 msgr2=0x55f83f34d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f34d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 -- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2c8670 msgr2=0x55f83f29c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2c8670 0x55f83f29c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.699+0000 7f2da1800640 1 -- 172.21.15.5:0/3419168278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f83f2c9500 con 0x55f83f2da280 2024-04-25T10:09:52.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2da1800640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 0x55f83f2b25d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2d9403f040 tx=0x7f2d94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e128fac90e311232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2d82000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:52.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2d82000640 1 -- 172.21.15.5:0/3419168278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94053060 con 0x55f83f2da280 2024-04-25T10:09:52.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2d82000640 1 -- 172.21.15.5:0/3419168278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f2d94004780 con 0x55f83f2da280 2024-04-25T10:09:52.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d90011dc0 con 0x55f83f2da280 2024-04-25T10:09:52.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2d82000640 1 -- 172.21.15.5:0/3419168278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d94059050 con 0x55f83f2da280 2024-04-25T10:09:52.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2da6625740 10 client.7890 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:52.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.700+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55f83f29c8b0 con 0x55f83f2da280 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2d82000640 1 -- 172.21.15.5:0/3419168278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d90011dc0 con 0x55f83f2da280 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2d82000640 1 -- 172.21.15.5:0/3419168278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d9405c020 con 0x55f83f2da280 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da6625740 10 client.7890 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da6625740 10 client.7890 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da6625740 20 client.7890 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da6625740 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f29f630 0x55f83f2c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da6625740 4 client.7890 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f83f209500 con 0x55f83f29f630 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.701+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f29f630 0x55f83f2c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da0e00640 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f29f630 0x55f83f2c58e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47c1ff2303e2084 server_cookie=ff3b4fdf21084d0c in_seq=0 out_seq=0 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2d82000640 10 client.7890 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2d82000640 1 -- 172.21.15.5:0/3419168278 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f83f209500 con 0x55f83f29f630 2024-04-25T10:09:52.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2d82000640 10 client.7890 handle_command_reply: tid=0 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 client.7890 shutdown 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 msgr2=0x55f83f2b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 0x55f83f2b25d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2d9403f040 tx=0x7f2d94004ab0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 shutdown_connections 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55f83f29f630 0x55f83f2c58e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f83f2da280 0x55f83f2b25d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f83f2c8670 0x55f83f29c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 --2- 172.21.15.5:0/3419168278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f83f29dbb0 0x55f83f34d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2d82000640 0 client.7890 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 >> 172.21.15.5:0/3419168278 conn(0x55f83f265a30 msgr2=0x55f83f2f55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 shutdown_connections 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 1 -- 172.21.15.5:0/3419168278 wait complete. 2024-04-25T10:09:52.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.702+0000 7f2da6625740 20 client.7890 trim_cache size 0 max 16384 2024-04-25T10:09:52.714 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:09:52.714 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:09:52.714 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:09:52.714 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d7c06b4d0 msgr2=0x7f2d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2d88006fd0 tx=0x7f2d88008040 comp rx=0 tx=0).stop 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 msgr2=0x7f2d9c1f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 0x7f2d9c1f7730 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2d9000b410 tx=0x7f2d9000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 shutdown_connections 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d9c144fe0 0x7f2d9c1426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d7c06b4d0 0x7f2d7c06d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d9c071670 0x7f2d9c15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 --2- 172.21.15.5:0/3405088393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9c06d130 0x7f2d9c1f7730 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 >> 172.21.15.5:0/3405088393 conn(0x7f2d9c06ab60 msgr2=0x7f2d9c15d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 shutdown_connections 2024-04-25T10:09:52.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.703+0000 7f2da3800640 1 -- 172.21.15.5:0/3405088393 wait complete. 2024-04-25T10:09:52.727 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:09:52.727 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.928+0000 7f7c95a00640 1 Processor -- start 2024-04-25T10:09:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.928+0000 7f7c95a00640 1 -- start start 2024-04-25T10:09:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.930+0000 7f7c95a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c9015eda0 0x7f7c900ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.930+0000 7f7c95a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900b4880 con 0x7f7c90143d30 2024-04-25T10:09:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.930+0000 7f7c95a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900b45a0 con 0x7f7c900cecd0 2024-04-25T10:09:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.930+0000 7f7c95a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9015e2c0 con 0x7f7c9015eda0 2024-04-25T10:09:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.930+0000 7f7c8fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c90143d30 0x7f7c900b3ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2024-04-25T10:09:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.930+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1446862804 learned_addr learned my addr 172.21.15.5:0/1446862804 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:52.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8f400640 1 --2- 172.21.15.5:0/1446862804 >> v2:172.21.15.80:3300/0 conn(0x7f7c9015eda0 0x7f7c900ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767235621 0 0) 0x7f7c900b4880 con 0x7f7c90143d30 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c74003520 con 0x7f7c90143d30 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1983261147 0 0) 0x7f7c74003520 con 0x7f7c90143d30 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 >> v2:172.21.15.80:3300/0 conn(0x7f7c9015eda0 msgr2=0x7f7c900ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 --2- 172.21.15.5:0/1446862804 >> v2:172.21.15.80:3300/0 conn(0x7f7c9015eda0 0x7f7c900ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 >> v1:172.21.15.80:6789/0 conn(0x7f7c900cecd0 legacy=0x7f7c9015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9015e5a0 con 0x7f7c90143d30 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f7c80001670 con 0x7f7c90143d30 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 >> v1:172.21.15.5:6789/0 conn(0x7f7c90143d30 legacy=0x7f7c900b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c74003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c74004800 0x7f7c74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c740052e0 0x7f7c7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9015e2c0 con 0x7f7c740039b0 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7400c190 con 0x7f7c740052e0 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.931+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7400c390 con 0x7f7c74004800 2024-04-25T10:09:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c74003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8f400640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c74004800 0x7f7c74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c74004800 msgr2=0x7f7c74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c74004800 0x7f7c74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c740052e0 msgr2=0x7f7c7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c740052e0 0x7f7c7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1446862804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74003520 con 0x7f7c740039b0 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c95000640 1 -- 172.21.15.5:0/1446862804 reap_dead start 2024-04-25T10:09:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.932+0000 7f7c8f400640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c74004800 0x7f7c74004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.933+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c74003db0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c210 tx=0x7f7c74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda3949d86585dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80007d10 con 0x7f7c740039b0 2024-04-25T10:09:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c800058d0 con 0x7f7c740039b0 2024-04-25T10:09:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c8ea00640 1 -- 172.21.15.5:0/1446862804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80005d20 con 0x7f7c740039b0 2024-04-25T10:09:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c95a00640 1 -- 172.21.15.5:0/1446862804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 msgr2=0x7f7c74003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c74003db0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c210 tx=0x7f7c74005830 comp rx=0 tx=0).stop 2024-04-25T10:09:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c95a00640 1 -- 172.21.15.5:0/1446862804 shutdown_connections 2024-04-25T10:09:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1446862804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c74003db0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c95a00640 1 -- 172.21.15.5:0/1446862804 >> 172.21.15.5:0/1446862804 conn(0x7f7c90066f20 msgr2=0x7f7c9006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.934+0000 7f7c95a00640 1 -- 172.21.15.5:0/1446862804 shutdown_connections 2024-04-25T10:09:52.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.935+0000 7f7c95a00640 1 -- 172.21.15.5:0/1446862804 wait complete. 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.936+0000 7f7c95a00640 1 Processor -- start 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.936+0000 7f7c95a00640 1 -- start start 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.937+0000 7f7c95a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c9015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.937+0000 7f7c95a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c900a8450 0x7f7c90140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.937+0000 7f7c95a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c900ad710 0x7f7c900adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.937+0000 7f7c95a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90144290 con 0x7f7c740039b0 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.937+0000 7f7c95a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c90144110 con 0x7f7c900ad710 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.937+0000 7f7c95a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9006e430 con 0x7f7c900a8450 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c9015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c9015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55038/0 (socket says 172.21.15.5:55038) 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1309867756 learned_addr learned my addr 172.21.15.5:0/1309867756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c900a8450 msgr2=0x7f7c90140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c900a8450 0x7f7c90140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c900ad710 msgr2=0x7f7c900adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c900ad710 0x7f7c900adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 -- 172.21.15.5:0/1309867756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c900abf30 con 0x7f7c740039b0 2024-04-25T10:09:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.938+0000 7f7c8fe00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c9015cc60 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c330 tx=0x7f7c9015e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.939+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80040020 con 0x7f7c740039b0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.939+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c800049f0 con 0x7f7c740039b0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.939+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c900a89c0 con 0x7f7c740039b0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.939+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80001c50 con 0x7f7c740039b0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.939+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c900a8f80 con 0x7f7c740039b0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.940+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90158ef0 con 0x7f7c740039b0 2024-04-25T10:09:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.941+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c80043020 con 0x7f7c740039b0 2024-04-25T10:09:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.942+0000 7f7c87e00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c7006b500 0x7f7c7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.942+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f7c800aa300 con 0x7f7c740039b0 2024-04-25T10:09:52.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.942+0000 7f7c8f400640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c7006b500 0x7f7c7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.944+0000 7f7c8f400640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c7006b500 0x7f7c7006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c006a90 tx=0x7f7c7c006a20 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:52.946+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c90158ef0 con 0x7f7c740039b0 2024-04-25T10:09:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.185+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c90068f60 con 0x7f7c740039b0 2024-04-25T10:09:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.186+0000 7f7c87e00640 1 -- 172.21.15.5:0/1309867756 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f7c80078eb0 con 0x7f7c740039b0 2024-04-25T10:09:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:53.193 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:23.419710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7683},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7683":{"gid":7683,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6834/7055779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":7055779},{"type":"v1","addr":"172.21.15.5:6834","nonce":7055779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 68 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c7006b500 msgr2=0x7f7c7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c7006b500 0x7f7c7006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c006a90 tx=0x7f7c7c006a20 comp rx=0 tx=0).stop 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 msgr2=0x7f7c9015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c9015cc60 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c330 tx=0x7f7c9015e770 comp rx=0 tx=0).stop 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 shutdown_connections 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7c7006b500 0x7f7c7006d9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c900ad710 0x7f7c900adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7c900a8450 0x7f7c90140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 --2- 172.21.15.5:0/1309867756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c740039b0 0x7f7c9015cc60 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 >> 172.21.15.5:0/1309867756 conn(0x7f7c90066f20 msgr2=0x7f7c901587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 shutdown_connections 2024-04-25T10:09:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.189+0000 7f7c95a00640 1 -- 172.21.15.5:0/1309867756 wait complete. 2024-04-25T10:09:53.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:09:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.426+0000 7f69f3e00640 1 Processor -- start 2024-04-25T10:09:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.426+0000 7f69f3e00640 1 -- start start 2024-04-25T10:09:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.427+0000 7f69f3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69f40b48c0 0x7f69f415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.427+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40d3f90 0x7f69f4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f41606e0 con 0x7f69f4161bc0 2024-04-25T10:09:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f41609c0 con 0x7f69f40b48c0 2024-04-25T10:09:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f4160ca0 con 0x7f69f40d3f90 2024-04-25T10:09:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f2a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69f40b48c0 0x7f69f415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f2a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69f40b48c0 0x7f69f415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:57156/0 (socket says 172.21.15.5:57156) 2024-04-25T10:09:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f2a00640 1 -- 172.21.15.5:0/2227715212 learned_addr learned my addr 172.21.15.5:0/2227715212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.428+0000 7f69f2000640 1 --2- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40d3f90 0x7f69f4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.429+0000 7f69f2a00640 1 -- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40d3f90 msgr2=0x7f69f4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.429+0000 7f69f2a00640 1 --2- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40d3f90 0x7f69f4161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.429+0000 7f69f2a00640 1 -- 172.21.15.5:0/2227715212 >> v1:172.21.15.80:6789/0 conn(0x7f69f4161bc0 legacy=0x7f69f40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.429+0000 7f69f2a00640 1 -- 172.21.15.5:0/2227715212 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4160f80 con 0x7f69f40b48c0 2024-04-25T10:09:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.430+0000 7f69f2a00640 1 --2- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40b48c0 0x7f69f415fe60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009870 tx=0x7f69e402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80bee20064d5a39a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.431+0000 7f69f1600640 1 -- 172.21.15.5:0/2227715212 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4037070 con 0x7f69f40b48c0 2024-04-25T10:09:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.431+0000 7f69f1600640 1 -- 172.21.15.5:0/2227715212 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4032440 con 0x7f69f40b48c0 2024-04-25T10:09:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.431+0000 7f69f1600640 1 -- 172.21.15.5:0/2227715212 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4032810 con 0x7f69f40b48c0 2024-04-25T10:09:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.431+0000 7f69f3e00640 1 -- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40b48c0 msgr2=0x7f69f415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.431+0000 7f69f3e00640 1 --2- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40b48c0 0x7f69f415fe60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009870 tx=0x7f69e402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.432+0000 7f69f3e00640 1 -- 172.21.15.5:0/2227715212 shutdown_connections 2024-04-25T10:09:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.432+0000 7f69f3e00640 1 --2- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40d3f90 0x7f69f4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.432+0000 7f69f3e00640 1 --2- 172.21.15.5:0/2227715212 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40b48c0 0x7f69f415fe60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.432+0000 7f69f3e00640 1 -- 172.21.15.5:0/2227715212 >> 172.21.15.5:0/2227715212 conn(0x7f69f408af80 msgr2=0x7f69f40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.432+0000 7f69f3e00640 1 -- 172.21.15.5:0/2227715212 shutdown_connections 2024-04-25T10:09:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.432+0000 7f69f3e00640 1 -- 172.21.15.5:0/2227715212 wait complete. 2024-04-25T10:09:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.433+0000 7f69f3e00640 1 Processor -- start 2024-04-25T10:09:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.434+0000 7f69f3e00640 1 -- start start 2024-04-25T10:09:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.434+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40b48c0 0x7f69f40cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.434+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40d3f90 0x7f69f40c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 0x7f69f4153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41603a0 con 0x7f69f4161bc0 2024-04-25T10:09:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40b4ca0 con 0x7f69f40d3f90 2024-04-25T10:09:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40b3b80 con 0x7f69f40b48c0 2024-04-25T10:09:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 0x7f69f4153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 0x7f69f4153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55042/0 (socket says 172.21.15.5:55042) 2024-04-25T10:09:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 -- 172.21.15.5:0/499933120 learned_addr learned my addr 172.21.15.5:0/499933120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f2a00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40b48c0 0x7f69f40cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f2000640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40d3f90 0x7f69f40c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 -- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40b48c0 msgr2=0x7f69f40cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40b48c0 0x7f69f40cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 -- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40d3f90 msgr2=0x7f69f40c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40d3f90 0x7f69f40c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.435+0000 7f69f3400640 1 -- 172.21.15.5:0/499933120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4153e90 con 0x7f69f4161bc0 2024-04-25T10:09:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.436+0000 7f69f3400640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 0x7f69f4153420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c6e0 tx=0x7f69ec00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.436+0000 7f69eaa00640 1 -- 172.21.15.5:0/499933120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec003b70 con 0x7f69f4161bc0 2024-04-25T10:09:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.436+0000 7f69eaa00640 1 -- 172.21.15.5:0/499933120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec003d30 con 0x7f69f4161bc0 2024-04-25T10:09:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.436+0000 7f69eaa00640 1 -- 172.21.15.5:0/499933120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec005310 con 0x7f69f4161bc0 2024-04-25T10:09:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.436+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f4153960 con 0x7f69f4161bc0 2024-04-25T10:09:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.436+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f40c8ed0 con 0x7f69f4161bc0 2024-04-25T10:09:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.439+0000 7f69eaa00640 1 -- 172.21.15.5:0/499933120 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69ec0054b0 con 0x7f69f4161bc0 2024-04-25T10:09:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.440+0000 7f69eaa00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d006b4d0 0x7f69d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.441+0000 7f69eaa00640 1 -- 172.21.15.5:0/499933120 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f69ec08a3a0 con 0x7f69f4161bc0 2024-04-25T10:09:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.441+0000 7f69fc5e9740 1 Processor -- start 2024-04-25T10:09:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.441+0000 7f69fc5e9740 1 -- start start 2024-04-25T10:09:53.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.441+0000 7f69f2a00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d006b4d0 0x7f69d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e347c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e349a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 0x55a6e34984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e33b4ae0 con 0x55a6e3452230 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e335d7c0 con 0x55a6e3499df0 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6e33b0b80 con 0x55a6e347c550 2024-04-25T10:09:53.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 0x55a6e34984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 0x55a6e34984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55048/0 (socket says 172.21.15.5:55048) 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.442+0000 7f69f3400640 1 -- 172.21.15.5:0/1314028426 learned_addr learned my addr 172.21.15.5:0/1314028426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f2a00640 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e347c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f2000640 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e349a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f3400640 1 -- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 msgr2=0x55a6e347c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f3400640 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e347c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f3400640 1 -- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 msgr2=0x55a6e349a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f3400640 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e349a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f2a00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d006b4d0 0x7f69d006d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f69e4034000 tx=0x7f69e402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f3400640 1 -- 172.21.15.5:0/1314028426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e31957c0 con 0x55a6e3452230 2024-04-25T10:09:53.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69f3400640 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 0x55a6e34984b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f69ec054c20 tx=0x7f69ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b480e876a43a9c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69e3400640 1 -- 172.21.15.5:0/1314028426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec060ed0 con 0x55a6e3452230 2024-04-25T10:09:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69e3400640 1 -- 172.21.15.5:0/1314028426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec056860 con 0x55a6e3452230 2024-04-25T10:09:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.443+0000 7f69e3400640 1 -- 172.21.15.5:0/1314028426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec08a070 con 0x55a6e3452230 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1314028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 msgr2=0x55a6e34984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 0x55a6e34984b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f69ec054c20 tx=0x7f69ec011000 comp rx=0 tx=0).stop 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1314028426 shutdown_connections 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3452230 0x55a6e34984b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e349a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1314028426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e347c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1314028426 >> 172.21.15.5:0/1314028426 conn(0x55a6e344eb60 msgr2=0x55a6e3497360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1314028426 shutdown_connections 2024-04-25T10:09:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.444+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1314028426 wait complete. 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.445+0000 7f69fc5e9740 1 -- start start 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.445+0000 7f69fc5e9740 1 Processor -- start 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e346de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 0x55a6e347fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e344b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e3452fd0 con 0x55a6e3499df0 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e3352d40 con 0x55a6e3490b10 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6e344ca90 con 0x55a6e347c550 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.446+0000 7f69fc5e9740 10 client.0 _ll_register_callbacks cb 0x55a6e30c8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 0x55a6e347fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 0x55a6e347fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55058/0 (socket says 172.21.15.5:55058) 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 -- 172.21.15.5:0/1862045589 learned_addr learned my addr 172.21.15.5:0/1862045589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2a00640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e346de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f3400640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e344b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 -- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 msgr2=0x55a6e346de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e346de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 -- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 msgr2=0x55a6e344b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e344b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 -- 172.21.15.5:0/1862045589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6e31f5f50 con 0x55a6e3499df0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69f2000640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 0x55a6e347fff0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009cd0 tx=0x7f69dc0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9398bd0852e460e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.447+0000 7f69d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69d7e00640 1 -- 172.21.15.5:0/1862045589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc004230 con 0x55a6e3499df0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69d7e00640 1 -- 172.21.15.5:0/1862045589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f69dc0043d0 con 0x55a6e3499df0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69ec056860 con 0x55a6e3499df0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69d7e00640 1 -- 172.21.15.5:0/1862045589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69dc024050 con 0x55a6e3499df0 2024-04-25T10:09:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 10 client.7905 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55a6e30890b0 con 0x55a6e3499df0 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69d7e00640 1 -- 172.21.15.5:0/1862045589 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec056860 con 0x55a6e3499df0 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69d7e00640 1 -- 172.21.15.5:0/1862045589 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f69dc027020 con 0x55a6e3499df0 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 10 client.7905 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 10 client.7905 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 20 client.7905 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e34a7350 0x55a6e34b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 4 client.7905 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.448+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6e3413b60 con 0x55a6e34a7350 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.449+0000 7f69f2a00640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e34a7350 0x55a6e34b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.449+0000 7f69f2a00640 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e34a7350 0x55a6e34b4a10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3705735bb1d5165 server_cookie=f0825077b4dc31e1 in_seq=0 out_seq=0 2024-04-25T10:09:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.449+0000 7f69d7e00640 10 client.7905 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69d7e00640 1 -- 172.21.15.5:0/1862045589 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a6e3413b60 con 0x55a6e34a7350 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69d7e00640 10 client.7905 handle_command_reply: tid=0 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69fc5e9740 1 client.7905 shutdown 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 msgr2=0x55a6e347fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 0x55a6e347fff0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009cd0 tx=0x7f69dc0136b0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 shutdown_connections 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.450+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e34a7350 0x55a6e34b4a10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e344b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e3499df0 0x55a6e347fff0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/1862045589 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e347c550 0x55a6e346de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69d7e00640 0 client.7905 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 >> 172.21.15.5:0/1862045589 conn(0x55a6e3452230 msgr2=0x55a6e347ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 shutdown_connections 2024-04-25T10:09:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 1 -- 172.21.15.5:0/1862045589 wait complete. 2024-04-25T10:09:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.451+0000 7f69fc5e9740 20 client.7905 trim_cache size 0 max 16384 2024-04-25T10:09:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.464+0000 7f69fc5e9740 1 Processor -- start 2024-04-25T10:09:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.464+0000 7f69fc5e9740 1 -- start start 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e3453600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e348ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3499df0 0x55a6e347b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f4200880 con 0x55a6e347c550 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41ff9a0 con 0x55a6e3490b10 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41ff340 con 0x55a6e3499df0 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e3453600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e3453600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55060/0 (socket says 172.21.15.5:55060) 2024-04-25T10:09:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.465+0000 7f69f2a00640 1 -- 172.21.15.5:0/3617551077 learned_addr learned my addr 172.21.15.5:0/3617551077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69f2a00640 1 -- 172.21.15.5:0/3617551077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3499df0 msgr2=0x55a6e347b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69f2a00640 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3499df0 0x55a6e347b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69f2a00640 1 -- 172.21.15.5:0/3617551077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 msgr2=0x55a6e348ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69f2a00640 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e348ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69f2a00640 1 -- 172.21.15.5:0/3617551077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6e344baf0 con 0x55a6e347c550 2024-04-25T10:09:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69f2a00640 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e3453600 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f69e40090b0 tx=0x7f69e402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3b252b83205b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69e2a00640 1 -- 172.21.15.5:0/3617551077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4051030 con 0x55a6e347c550 2024-04-25T10:09:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69e2a00640 1 -- 172.21.15.5:0/3617551077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4043040 con 0x55a6e347c550 2024-04-25T10:09:53.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.466+0000 7f69e2a00640 1 -- 172.21.15.5:0/3617551077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4037040 con 0x55a6e347c550 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3617551077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 msgr2=0x55a6e3453600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e3453600 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f69e40090b0 tx=0x7f69e402c000 comp rx=0 tx=0).stop 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3617551077 shutdown_connections 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3499df0 0x55a6e347b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3490b10 0x55a6e348ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3617551077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e3453600 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3617551077 >> 172.21.15.5:0/3617551077 conn(0x55a6e3452230 msgr2=0x55a6e347fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3617551077 shutdown_connections 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.467+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3617551077 wait complete. 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.468+0000 7f69fc5e9740 1 -- start start 2024-04-25T10:09:53.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.468+0000 7f69fc5e9740 1 Processor -- start 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e346cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3490b10 0x55a6e35023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e348ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e34a71d0 con 0x55a6e347c550 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6e347f040 con 0x55a6e3499df0 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6e34a7740 con 0x55a6e3490b10 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69fc5e9740 10 client.0 _ll_register_callbacks cb 0x55a6e344cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e346cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e346cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55072/0 (socket says 172.21.15.5:55072) 2024-04-25T10:09:53.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.469+0000 7f69f2a00640 1 -- 172.21.15.5:0/3778485935 learned_addr learned my addr 172.21.15.5:0/3778485935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f3400640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e348ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2000640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3490b10 0x55a6e35023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2a00640 1 -- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3490b10 msgr2=0x55a6e35023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2a00640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3490b10 0x55a6e35023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2a00640 1 -- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 msgr2=0x55a6e348ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2a00640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e348ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2a00640 1 -- 172.21.15.5:0/3778485935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6e34b1160 con 0x55a6e347c550 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69f2a00640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e346cde0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009990 tx=0x7f69e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857577963028a7cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69d7e00640 1 -- 172.21.15.5:0/3778485935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4037030 con 0x55a6e347c550 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69d7e00640 1 -- 172.21.15.5:0/3778485935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f69e404d470 con 0x55a6e347c550 2024-04-25T10:09:53.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.470+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69e4043040 con 0x55a6e347c550 2024-04-25T10:09:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69d7e00640 1 -- 172.21.15.5:0/3778485935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69e4060050 con 0x55a6e347c550 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 10 client.7911 fetch_fsmap learned FSMap version 68 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55a6e3501720 con 0x55a6e347c550 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69d7e00640 1 -- 172.21.15.5:0/3778485935 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e4043040 con 0x55a6e347c550 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69d7e00640 1 -- 172.21.15.5:0/3778485935 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f69e4063020 con 0x55a6e347c550 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 10 client.7911 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 10 client.7911 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 20 client.7911 populate_metadata read hostname 'smithi005' 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e348d230 0x55a6e347d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 4 client.7911 mds_command: new command op to 7683 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 --> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55a6e33c2020 con 0x55a6e348d230 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.471+0000 7f69f2000640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e348d230 0x55a6e347d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.472+0000 7f69f2000640 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e348d230 0x55a6e347d210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce96a89ccfd878c8 server_cookie=cba01163c52e529e in_seq=0 out_seq=0 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.472+0000 7f69d7e00640 10 client.7911 ms_handle_connect on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.472+0000 7f69d7e00640 1 -- 172.21.15.5:0/3778485935 <== mds.0 v2:172.21.15.5:6832/7055779 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+143 (crc 0 0 0) 0x55a6e33c2020 con 0x55a6e348d230 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.472+0000 7f69d7e00640 10 client.7911 handle_command_reply: tid=0 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.472+0000 7f69fc5e9740 1 client.7911 shutdown 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 msgr2=0x55a6e346cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e346cde0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009990 tx=0x7f69e4004730 comp rx=0 tx=0).stop 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 shutdown_connections 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:6832/7055779,v1:172.21.15.5:6834/7055779] conn(0x55a6e348d230 0x55a6e347d210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a6e3499df0 0x55a6e348ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a6e3490b10 0x55a6e35023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 --2- 172.21.15.5:0/3778485935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6e347c550 0x55a6e346cde0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69d7e00640 0 client.7911 ms_handle_reset on v2:172.21.15.5:6832/7055779 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 >> 172.21.15.5:0/3778485935 conn(0x55a6e3452230 msgr2=0x55a6e347b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 shutdown_connections 2024-04-25T10:09:53.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 1 -- 172.21.15.5:0/3778485935 wait complete. 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.473+0000 7f69fc5e9740 20 client.7911 trim_cache size 0 max 16384 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "damage_type": "backtrace", 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 414853104, 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 1099511627777, 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/testdir/hardlink" 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:09:53.483 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d006b4d0 msgr2=0x7f69d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d006b4d0 0x7f69d006d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f69e4034000 tx=0x7f69e402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 msgr2=0x7f69f4153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 0x7f69f4153420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c6e0 tx=0x7f69ec00cba0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 shutdown_connections 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f4161bc0 0x7f69f4153420 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d006b4d0 0x7f69d006d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f40d3f90 0x7f69f40c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 --2- 172.21.15.5:0/499933120 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f40b48c0 0x7f69f40cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 >> 172.21.15.5:0/499933120 conn(0x7f69f408af80 msgr2=0x7f69f415e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 shutdown_connections 2024-04-25T10:09:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.474+0000 7f69f3e00640 1 -- 172.21.15.5:0/499933120 wait complete. 2024-04-25T10:09:53.498 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:09:53.498 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.544 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:09:53.544 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:53.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.850+0000 7f97dac00640 1 Processor -- start 2024-04-25T10:09:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.850+0000 7f97dac00640 1 -- start start 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97dac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f97d406e970 0x7f97d40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97dac00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40cdbf0 con 0x7f97d40ced20 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97dac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40cded0 con 0x7f97d4144100 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97dac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d41444e0 con 0x7f97d406e970 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97d3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f97d406e970 0x7f97d40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97d8c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f97d40ced20 0x7f97d415b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58788/0 (socket says 172.21.15.5:58788) 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97d3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f97d406e970 0x7f97d40cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55082/0 (socket says 172.21.15.5:55082) 2024-04-25T10:09:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.852+0000 7f97d8c00640 1 -- 172.21.15.5:0/668057396 learned_addr learned my addr 172.21.15.5:0/668057396 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.853+0000 7f97d3400640 1 -- 172.21.15.5:0/668057396 >> v1:172.21.15.80:6789/0 conn(0x7f97d4144100 legacy=0x7f97d406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.853+0000 7f97d3400640 1 -- 172.21.15.5:0/668057396 >> v1:172.21.15.5:6789/0 conn(0x7f97d40ced20 legacy=0x7f97d415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.853+0000 7f97d3400640 1 -- 172.21.15.5:0/668057396 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40ce1b0 con 0x7f97d406e970 2024-04-25T10:09:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.853+0000 7f97d2a00640 1 -- 172.21.15.5:0/668057396 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7682205 0 0) 0x7f97d40cdbf0 con 0x7f97d40ced20 2024-04-25T10:09:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.854+0000 7f97d3400640 1 --2- 172.21.15.5:0/668057396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d406e970 0x7f97d40cd610 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0097a0 tx=0x7f97bc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a8bf7f86c8517f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.854+0000 7f97d2a00640 1 -- 172.21.15.5:0/668057396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc037070 con 0x7f97d406e970 2024-04-25T10:09:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.854+0000 7f97d2a00640 1 -- 172.21.15.5:0/668057396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc02ae50 con 0x7f97d406e970 2024-04-25T10:09:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.854+0000 7f97d2a00640 1 -- 172.21.15.5:0/668057396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc032660 con 0x7f97d406e970 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 -- 172.21.15.5:0/668057396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d406e970 msgr2=0x7f97d40cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 --2- 172.21.15.5:0/668057396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d406e970 0x7f97d40cd610 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0097a0 tx=0x7f97bc02a140 comp rx=0 tx=0).stop 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 -- 172.21.15.5:0/668057396 shutdown_connections 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 --2- 172.21.15.5:0/668057396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d406e970 0x7f97d40cd610 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 -- 172.21.15.5:0/668057396 >> 172.21.15.5:0/668057396 conn(0x7f97d4071150 msgr2=0x7f97d4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 -- 172.21.15.5:0/668057396 shutdown_connections 2024-04-25T10:09:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.855+0000 7f97dac00640 1 -- 172.21.15.5:0/668057396 wait complete. 2024-04-25T10:09:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.857+0000 7f97dac00640 1 Processor -- start 2024-04-25T10:09:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.857+0000 7f97dac00640 1 -- start start 2024-04-25T10:09:53.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97dac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97d406e970 0x7f97d41e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97dac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 0x7f97d41e1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97dac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97d4144100 0x7f97d41597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97dac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40cea00 con 0x7f97d40ced20 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97dac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d415ac70 con 0x7f97d4144100 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97dac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d415b840 con 0x7f97d406e970 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 0x7f97d41e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 0x7f97d41e1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55090/0 (socket says 172.21.15.5:55090) 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.858+0000 7f97d3400640 1 -- 172.21.15.5:0/2419109651 learned_addr learned my addr 172.21.15.5:0/2419109651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d8c00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97d4144100 0x7f97d41597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d3400640 1 -- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97d406e970 msgr2=0x7f97d41e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d3400640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97d406e970 0x7f97d41e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d3400640 1 -- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97d4144100 msgr2=0x7f97d41597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d3400640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97d4144100 0x7f97d41597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d3400640 1 -- 172.21.15.5:0/2419109651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d41e1c20 con 0x7f97d40ced20 2024-04-25T10:09:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97d3400640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 0x7f97d41e1280 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098c0 tx=0x7f97bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc042070 con 0x7f97d40ced20 2024-04-25T10:09:53.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc03e670 con 0x7f97d40ced20 2024-04-25T10:09:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d41e1f00 con 0x7f97d40ced20 2024-04-25T10:09:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.859+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc037040 con 0x7f97d40ced20 2024-04-25T10:09:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.860+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d41e24c0 con 0x7f97d40ced20 2024-04-25T10:09:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.861+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d4067530 con 0x7f97d40ced20 2024-04-25T10:09:53.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.862+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97bc045070 con 0x7f97d40ced20 2024-04-25T10:09:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.863+0000 7f97cbe00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f97b006b4d0 0x7f97b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:53.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.863+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f97bc0aae50 con 0x7f97d40ced20 2024-04-25T10:09:53.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.863+0000 7f97d3e00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f97b006b4d0 0x7f97b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:53.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.865+0000 7f97d3e00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f97b006b4d0 0x7f97b006d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006fd0 tx=0x7f97c4006bc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:53.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:53.867+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97d4067530 con 0x7f97d40ced20 2024-04-25T10:09:54.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.066+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97d4068fd0 con 0x7f97d40ced20 2024-04-25T10:09:54.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.070+0000 7f97cbe00640 1 -- 172.21.15.5:0/2419109651 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17325 (secure 0 0 0) 0x7f97d4068fd0 con 0x7f97d40ced20 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:42.586429+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.20","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.28","mappings":[{"from":1,"to":3}]},{"pgid":"14.30","mappings":[{"from":7,"to":0}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/3627376661":"2024-04-26T10:09:12.513203+0000","172.21.15.5:6832/3627376661":"2024-04-26T10:09:12.513203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f97b006b4d0 msgr2=0x7f97b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f97b006b4d0 0x7f97b006d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f97c4006fd0 tx=0x7f97c4006bc0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 msgr2=0x7f97d41e1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 0x7f97d41e1280 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098c0 tx=0x7f97bc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 shutdown_connections 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f97b006b4d0 0x7f97b006d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97d4144100 0x7f97d41597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97d40ced20 0x7f97d41e1280 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 --2- 172.21.15.5:0/2419109651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97d406e970 0x7f97d41e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.072+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 >> 172.21.15.5:0/2419109651 conn(0x7f97d4071150 msgr2=0x7f97d40b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.073+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 shutdown_connections 2024-04-25T10:09:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.073+0000 7f97dac00640 1 -- 172.21.15.5:0/2419109651 wait complete. 2024-04-25T10:09:54.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:57.371047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:58.551736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:54.099 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:54.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.311+0000 7f3fb6a00640 1 Processor -- start 2024-04-25T10:09:54.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.311+0000 7f3fb6a00640 1 -- start start 2024-04-25T10:09:54.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb6a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3fb015ec80 0x7f3fb00b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb01616f0 0x7f3fb0161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb6a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb0162030 con 0x7f3fb00d4000 2024-04-25T10:09:54.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb6a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb015ffa0 con 0x7f3fb015ec80 2024-04-25T10:09:54.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb6a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0160240 con 0x7f3fb01616f0 2024-04-25T10:09:54.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb5600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3fb015ec80 0x7f3fb00b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb5600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3fb015ec80 0x7f3fb00b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:57226/0 (socket says 172.21.15.5:57226) 2024-04-25T10:09:54.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb5600640 1 -- 172.21.15.5:0/1463827585 learned_addr learned my addr 172.21.15.5:0/1463827585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:54.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.313+0000 7f3fb6000640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb01616f0 0x7f3fb0161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.314+0000 7f3fb5600640 1 -- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb01616f0 msgr2=0x7f3fb0161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.314+0000 7f3fb5600640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb01616f0 0x7f3fb0161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.314+0000 7f3fb5600640 1 -- 172.21.15.5:0/1463827585 >> v1:172.21.15.80:6789/0 conn(0x7f3fb00d4000 legacy=0x7f3fb0160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.314+0000 7f3fb5600640 1 -- 172.21.15.5:0/1463827585 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0160520 con 0x7f3fb015ec80 2024-04-25T10:09:54.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.314+0000 7f3fb6000640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb01616f0 0x7f3fb0161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:54.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.316+0000 7f3fb5600640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00b45c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009850 tx=0x7f3fa402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dacf22083e4da0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.316+0000 7f3fafe00640 1 -- 172.21.15.5:0/1463827585 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004030 con 0x7f3fb015ec80 2024-04-25T10:09:54.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.317+0000 7f3fafe00640 1 -- 172.21.15.5:0/1463827585 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa40041d0 con 0x7f3fb015ec80 2024-04-25T10:09:54.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.317+0000 7f3fafe00640 1 -- 172.21.15.5:0/1463827585 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4003610 con 0x7f3fb015ec80 2024-04-25T10:09:54.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.317+0000 7f3fb6a00640 1 -- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 msgr2=0x7f3fb00b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.317+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00b45c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009850 tx=0x7f3fa402a250 comp rx=0 tx=0).stop 2024-04-25T10:09:54.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.318+0000 7f3fb6a00640 1 -- 172.21.15.5:0/1463827585 shutdown_connections 2024-04-25T10:09:54.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.318+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb01616f0 0x7f3fb0161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.318+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/1463827585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00b45c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.318+0000 7f3fb6a00640 1 -- 172.21.15.5:0/1463827585 >> 172.21.15.5:0/1463827585 conn(0x7f3fb008af80 msgr2=0x7f3fb00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.318+0000 7f3fb6a00640 1 -- 172.21.15.5:0/1463827585 shutdown_connections 2024-04-25T10:09:54.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.318+0000 7f3fb6a00640 1 -- 172.21.15.5:0/1463827585 wait complete. 2024-04-25T10:09:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.320+0000 7f3fb6a00640 1 Processor -- start 2024-04-25T10:09:54.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.320+0000 7f3fb6a00640 1 -- start start 2024-04-25T10:09:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.321+0000 7f3fb6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb00d4000 0x7f3fb00cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.321+0000 7f3fb6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.321+0000 7f3fb6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 0x7f3fb01f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.321+0000 7f3fb6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb015f060 con 0x7f3fb01616f0 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb6a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb0161410 con 0x7f3fb015ec80 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb6a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00b4c50 con 0x7f3fb00d4000 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 0x7f3fb01f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 0x7f3fb01f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55098/0 (socket says 172.21.15.5:55098) 2024-04-25T10:09:54.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb6000640 1 -- 172.21.15.5:0/2676402572 learned_addr learned my addr 172.21.15.5:0/2676402572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb4c00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.322+0000 7f3fb5600640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb00d4000 0x7f3fb00cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6000640 1 -- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb00d4000 msgr2=0x7f3fb00cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6000640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb00d4000 0x7f3fb00cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6000640 1 -- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 msgr2=0x7f3fb00cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6000640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6000640 1 -- 172.21.15.5:0/2676402572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00c8170 con 0x7f3fb01616f0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6000640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 0x7f3fb01f71b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009e60 tx=0x7f3fa800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8018030 con 0x7f3fb01616f0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa800cdf0 con 0x7f3fb01616f0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb00cb930 con 0x7f3fb01616f0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8003bb0 con 0x7f3fb01616f0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.323+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb00c86d0 con 0x7f3fb01616f0 2024-04-25T10:09:54.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.325+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb0068f60 con 0x7f3fb01616f0 2024-04-25T10:09:54.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.326+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fa801b070 con 0x7f3fb01616f0 2024-04-25T10:09:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.327+0000 7f3fad600640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f9006b4d0 0x7f3f9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.327+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f3fa808ae90 con 0x7f3fb01616f0 2024-04-25T10:09:54.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.328+0000 7f3fb5600640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f9006b4d0 0x7f3f9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.329+0000 7f3fb5600640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f9006b4d0 0x7f3f9006d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4033000 tx=0x7f3fa402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.331+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fb00c8170 con 0x7f3fb01616f0 2024-04-25T10:09:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.533+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fb00d0e50 con 0x7f3fb01616f0 2024-04-25T10:09:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.534+0000 7f3fad600640 1 -- 172.21.15.5:0/2676402572 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fa8010040 con 0x7f3fb01616f0 2024-04-25T10:09:54.540 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:09:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f9006b4d0 msgr2=0x7f3f9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f9006b4d0 0x7f3f9006d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4033000 tx=0x7f3fa402e000 comp rx=0 tx=0).stop 2024-04-25T10:09:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 msgr2=0x7f3fb01f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 0x7f3fb01f71b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009e60 tx=0x7f3fa800c490 comp rx=0 tx=0).stop 2024-04-25T10:09:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 shutdown_connections 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb01616f0 0x7f3fb01f71b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3f9006b4d0 0x7f3f9006d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fb015ec80 0x7f3fb00cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 --2- 172.21.15.5:0/2676402572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fb00d4000 0x7f3fb00cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.536+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 >> 172.21.15.5:0/2676402572 conn(0x7f3fb008af80 msgr2=0x7f3fb00cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.537+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 shutdown_connections 2024-04-25T10:09:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.537+0000 7f3fb6a00640 1 -- 172.21.15.5:0/2676402572 wait complete. 2024-04-25T10:09:54.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.778+0000 7f2f53e00640 1 Processor -- start 2024-04-25T10:09:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.778+0000 7f2f53e00640 1 -- start start 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f53e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f540b48c0 0x7f2f5415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f54161d90 0x7f2f540b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f53e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f541606e0 con 0x7f2f540d3f90 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f53e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f541609c0 con 0x7f2f540b48c0 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f53e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f54160ca0 con 0x7f2f54161d90 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f52a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f540b48c0 0x7f2f5415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f52a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f540b48c0 0x7f2f5415fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55102/0 (socket says 172.21.15.5:55102) 2024-04-25T10:09:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.780+0000 7f2f52a00640 1 -- 172.21.15.5:0/1405451497 learned_addr learned my addr 172.21.15.5:0/1405451497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.781+0000 7f2f53400640 1 --2- 172.21.15.5:0/1405451497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f54161d90 0x7f2f540b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.781+0000 7f2f52a00640 1 -- 172.21.15.5:0/1405451497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f54161d90 msgr2=0x7f2f540b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.781+0000 7f2f52a00640 1 --2- 172.21.15.5:0/1405451497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f54161d90 0x7f2f540b2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.781+0000 7f2f52a00640 1 -- 172.21.15.5:0/1405451497 >> v1:172.21.15.80:6789/0 conn(0x7f2f540d3f90 legacy=0x7f2f54161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.781+0000 7f2f52a00640 1 -- 172.21.15.5:0/1405451497 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54160f80 con 0x7f2f540b48c0 2024-04-25T10:09:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.782+0000 7f2f52a00640 1 --2- 172.21.15.5:0/1405451497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f5415fe60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214aa06c4faa48aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.782+0000 7f2f51600640 1 -- 172.21.15.5:0/1405451497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f540b48c0 2024-04-25T10:09:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.782+0000 7f2f51600640 1 -- 172.21.15.5:0/1405451497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f540b48c0 2024-04-25T10:09:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.782+0000 7f2f51600640 1 -- 172.21.15.5:0/1405451497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c032810 con 0x7f2f540b48c0 2024-04-25T10:09:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.783+0000 7f2f53e00640 1 -- 172.21.15.5:0/1405451497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 msgr2=0x7f2f5415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.783+0000 7f2f53e00640 1 --2- 172.21.15.5:0/1405451497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f5415fe60 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c02a180 comp rx=0 tx=0).stop 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.783+0000 7f2f53e00640 1 -- 172.21.15.5:0/1405451497 shutdown_connections 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.783+0000 7f2f53e00640 1 --2- 172.21.15.5:0/1405451497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f54161d90 0x7f2f540b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.783+0000 7f2f53e00640 1 --2- 172.21.15.5:0/1405451497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f5415fe60 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.783+0000 7f2f53e00640 1 -- 172.21.15.5:0/1405451497 >> 172.21.15.5:0/1405451497 conn(0x7f2f5408af80 msgr2=0x7f2f540d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.784+0000 7f2f53e00640 1 -- 172.21.15.5:0/1405451497 shutdown_connections 2024-04-25T10:09:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.784+0000 7f2f53e00640 1 -- 172.21.15.5:0/1405451497 wait complete. 2024-04-25T10:09:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.785+0000 7f2f53e00640 1 Processor -- start 2024-04-25T10:09:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.785+0000 7f2f53e00640 1 -- start start 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.786+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f54150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.786+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f540d3f90 0x7f2f540ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.786+0000 7f2f53e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f54161d90 0x7f2f540cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.786+0000 7f2f53e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f541603a0 con 0x7f2f540b48c0 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.786+0000 7f2f53e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b4ca0 con 0x7f2f54161d90 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.786+0000 7f2f53e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f540b30b0 con 0x7f2f540d3f90 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f54150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f54150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55114/0 (socket says 172.21.15.5:55114) 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 -- 172.21.15.5:0/4098511706 learned_addr learned my addr 172.21.15.5:0/4098511706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 -- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f540d3f90 msgr2=0x7f2f540ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f540d3f90 0x7f2f540ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 -- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f54161d90 msgr2=0x7f2f540cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f54161d90 0x7f2f540cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.787+0000 7f2f52a00640 1 -- 172.21.15.5:0/4098511706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f540cb040 con 0x7f2f540b48c0 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.788+0000 7f2f52a00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f54150be0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.788+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c041020 con 0x7f2f540b48c0 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.788+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c030970 con 0x7f2f540b48c0 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.788+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037040 con 0x7f2f540b48c0 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.788+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f540cb320 con 0x7f2f540b48c0 2024-04-25T10:09:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.788+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540cb8e0 con 0x7f2f540b48c0 2024-04-25T10:09:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.789+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f540d2660 con 0x7f2f540b48c0 2024-04-25T10:09:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.790+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f3c044020 con 0x7f2f540b48c0 2024-04-25T10:09:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.791+0000 7f2f4aa00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f3006b550 0x7f2f3006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.791+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f2f3c0aae30 con 0x7f2f540b48c0 2024-04-25T10:09:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.792+0000 7f2f52000640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f3006b550 0x7f2f3006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.793+0000 7f2f52000640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f3006b550 0x7f2f3006da10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f540c9e30 tx=0x7f2f44007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:54.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.796+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f540d2660 con 0x7f2f540b48c0 2024-04-25T10:09:55.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.997+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f540d0dd0 con 0x7f2f540b48c0 2024-04-25T10:09:55.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:54.998+0000 7f2f4aa00640 1 -- 172.21.15.5:0/4098511706 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f3c04b030 con 0x7f2f540b48c0 2024-04-25T10:09:55.005 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f3006b550 msgr2=0x7f2f3006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f3006b550 0x7f2f3006da10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f540c9e30 tx=0x7f2f44007a00 comp rx=0 tx=0).stop 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 msgr2=0x7f2f54150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f54150be0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009870 tx=0x7f2f3c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 shutdown_connections 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2f3006b550 0x7f2f3006da10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f54161d90 0x7f2f540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2f540d3f90 0x7f2f540ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 --2- 172.21.15.5:0/4098511706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f540b48c0 0x7f2f54150be0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 >> 172.21.15.5:0/4098511706 conn(0x7f2f5408af80 msgr2=0x7f2f5415def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 shutdown_connections 2024-04-25T10:09:55.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.001+0000 7f2f53e00640 1 -- 172.21.15.5:0/4098511706 wait complete. 2024-04-25T10:09:55.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.241+0000 7fd43e200640 1 Processor -- start 2024-04-25T10:09:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.241+0000 7fd43e200640 1 -- start start 2024-04-25T10:09:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.243+0000 7fd43e200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4381465d0 0x7fd43815e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.243+0000 7fd43e200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd438071840 0x7fd43815f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.243+0000 7fd43e200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43815eed0 con 0x7fd4380ae860 2024-04-25T10:09:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.243+0000 7fd43e200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43806f080 con 0x7fd438071840 2024-04-25T10:09:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.243+0000 7fd43e200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43806f360 con 0x7fd4381465d0 2024-04-25T10:09:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd438071840 0x7fd43815f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd438071840 0x7fd43815f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55120/0 (socket says 172.21.15.5:55120) 2024-04-25T10:09:55.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 -- 172.21.15.5:0/2129182796 learned_addr learned my addr 172.21.15.5:0/2129182796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437400640 1 --2- 172.21.15.5:0/2129182796 >> v2:172.21.15.80:3300/0 conn(0x7fd4381465d0 0x7fd43815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 -- 172.21.15.5:0/2129182796 >> v2:172.21.15.80:3300/0 conn(0x7fd4381465d0 msgr2=0x7fd43815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 --2- 172.21.15.5:0/2129182796 >> v2:172.21.15.80:3300/0 conn(0x7fd4381465d0 0x7fd43815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 -- 172.21.15.5:0/2129182796 >> v1:172.21.15.80:6789/0 conn(0x7fd4380ae860 legacy=0x7fd438071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437e00640 1 -- 172.21.15.5:0/2129182796 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380ae060 con 0x7fd438071840 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.244+0000 7fd437400640 1 --2- 172.21.15.5:0/2129182796 >> v2:172.21.15.80:3300/0 conn(0x7fd4381465d0 0x7fd43815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:55.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.245+0000 7fd437e00640 1 --2- 172.21.15.5:0/2129182796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd43815f0d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd4380af190 tx=0x7fd43002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e42c9ea4ca2b429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.246+0000 7fd436a00640 1 -- 172.21.15.5:0/2129182796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430036070 con 0x7fd438071840 2024-04-25T10:09:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.246+0000 7fd436a00640 1 -- 172.21.15.5:0/2129182796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430031600 con 0x7fd438071840 2024-04-25T10:09:55.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.246+0000 7fd436a00640 1 -- 172.21.15.5:0/2129182796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300319d0 con 0x7fd438071840 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 -- 172.21.15.5:0/2129182796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 msgr2=0x7fd43815f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 --2- 172.21.15.5:0/2129182796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd43815f0d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd4380af190 tx=0x7fd43002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 -- 172.21.15.5:0/2129182796 shutdown_connections 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 --2- 172.21.15.5:0/2129182796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd43815f0d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 --2- 172.21.15.5:0/2129182796 >> v2:172.21.15.80:3300/0 conn(0x7fd4381465d0 0x7fd43815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 -- 172.21.15.5:0/2129182796 >> 172.21.15.5:0/2129182796 conn(0x7fd43806ab60 msgr2=0x7fd43806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 -- 172.21.15.5:0/2129182796 shutdown_connections 2024-04-25T10:09:55.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.247+0000 7fd43e200640 1 -- 172.21.15.5:0/2129182796 wait complete. 2024-04-25T10:09:55.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.249+0000 7fd43e200640 1 Processor -- start 2024-04-25T10:09:55.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.249+0000 7fd43e200640 1 -- start start 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd43e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd438140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd43e200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4380ae860 0x7fd4381e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd43e200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4381465d0 0x7fd4381e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd43e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806e7d0 con 0x7fd438071840 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd43e200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4381469b0 con 0x7fd4380ae860 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd43e200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806ece0 con 0x7fd4381465d0 2024-04-25T10:09:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.250+0000 7fd437400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd438140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd438140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55130/0 (socket says 172.21.15.5:55130) 2024-04-25T10:09:55.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 -- 172.21.15.5:0/947185480 learned_addr learned my addr 172.21.15.5:0/947185480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 -- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4381465d0 msgr2=0x7fd4381e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4381465d0 0x7fd4381e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 -- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4380ae860 msgr2=0x7fd4381e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4380ae860 0x7fd4381e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 -- 172.21.15.5:0/947185480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4381e8a30 con 0x7fd438071840 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.251+0000 7fd437400640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd438140930 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd42800c7f0 tx=0x7fd42800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.252+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428015070 con 0x7fd438071840 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.252+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd428007a30 con 0x7fd438071840 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.252+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428007e00 con 0x7fd438071840 2024-04-25T10:09:55.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.252+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4380c5f40 con 0x7fd438071840 2024-04-25T10:09:55.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.252+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4380c5c40 con 0x7fd438071840 2024-04-25T10:09:55.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.253+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd438068f60 con 0x7fd438071840 2024-04-25T10:09:55.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.254+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd428018070 con 0x7fd438071840 2024-04-25T10:09:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.255+0000 7fd42fe00640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd41406b4d0 0x7fd41406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.255+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7fd42808a480 con 0x7fd438071840 2024-04-25T10:09:55.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.256+0000 7fd42ea00640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd41406b4d0 0x7fd41406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.257+0000 7fd42ea00640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd41406b4d0 0x7fd41406d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd438071710 tx=0x7fd4200079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.259+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd428059030 con 0x7fd438071840 2024-04-25T10:09:55.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:55.384 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.460+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd43806e140 con 0x7fd438071840 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.461+0000 7fd42fe00640 1 -- 172.21.15.5:0/947185480 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd428058930 con 0x7fd438071840 2024-04-25T10:09:55.467 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd41406b4d0 msgr2=0x7fd41406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd41406b4d0 0x7fd41406d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd438071710 tx=0x7fd4200079b0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 msgr2=0x7fd438140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd438140930 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd42800c7f0 tx=0x7fd42800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 shutdown_connections 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4381465d0 0x7fd4381e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd41406b4d0 0x7fd41406d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4380ae860 0x7fd4381e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 --2- 172.21.15.5:0/947185480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd438071840 0x7fd438140930 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 >> 172.21.15.5:0/947185480 conn(0x7fd43806ab60 msgr2=0x7fd438145460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 shutdown_connections 2024-04-25T10:09:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.463+0000 7fd43e200640 1 -- 172.21.15.5:0/947185480 wait complete. 2024-04-25T10:09:55.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.692+0000 7f4922a00640 1 Processor -- start 2024-04-25T10:09:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.693+0000 7f4922a00640 1 -- start start 2024-04-25T10:09:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4922a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f491c0b4bb0 0x7f491c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4922a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c0cf410 con 0x7f491c161200 2024-04-25T10:09:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4922a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491c0cf6f0 con 0x7f491c0d07d0 2024-04-25T10:09:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4922a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f491c0b28b0 con 0x7f491c0b4bb0 2024-04-25T10:09:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4921600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f491c161200 0x7f491c0d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58804/0 (socket says 172.21.15.5:58804) 2024-04-25T10:09:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4921600640 1 -- 172.21.15.5:0/592900940 learned_addr learned my addr 172.21.15.5:0/592900940 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f4922000640 1 --2- 172.21.15.5:0/592900940 >> v2:172.21.15.80:3300/0 conn(0x7f491c0b4bb0 0x7f491c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.695+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535138137 0 0) 0x7f491c0cf410 con 0x7f491c161200 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4904003520 con 0x7f491c161200 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4039579636 0 0) 0x7f4904003520 con 0x7f491c161200 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 >> v2:172.21.15.80:3300/0 conn(0x7f491c0b4bb0 msgr2=0x7f491c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 --2- 172.21.15.5:0/592900940 >> v2:172.21.15.80:3300/0 conn(0x7f491c0b4bb0 0x7f491c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 >> v1:172.21.15.80:6789/0 conn(0x7f491c0d07d0 legacy=0x7f491c0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0d3fd0 con 0x7f491c161200 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4910001670 con 0x7f491c161200 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 >> v1:172.21.15.5:6789/0 conn(0x7f491c161200 legacy=0x7f491c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4904003a00 0x7f4904003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904004800 0x7f4904004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49040051c0 0x7f490400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0b28b0 con 0x7f4904003a00 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490400c090 con 0x7f49040051c0 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490400c290 con 0x7f4904004800 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.696+0000 7f4921600640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4904003a00 0x7f4904003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4921600640 1 -- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904004800 msgr2=0x7f4904004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4920c00640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904004800 0x7f4904004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4921600640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904004800 0x7f4904004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4921600640 1 -- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49040051c0 msgr2=0x7f490400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4922000640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49040051c0 0x7f490400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4921600640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49040051c0 0x7f490400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4921600640 1 -- 172.21.15.5:0/592900940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4904003520 con 0x7f4904003a00 2024-04-25T10:09:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.697+0000 7f4922000640 1 -- 172.21.15.5:0/592900940 reap_dead start 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.698+0000 7f4921600640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4904003a00 0x7f4904003e00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f491000c160 tx=0x7f4904005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b895ab89f2bb59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.698+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910007c80 con 0x7f4904003a00 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.698+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910005af0 con 0x7f4904003a00 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.698+0000 7f491be00640 1 -- 172.21.15.5:0/592900940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910007070 con 0x7f4904003a00 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 -- 172.21.15.5:0/592900940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4904003a00 msgr2=0x7f4904003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4904003a00 0x7f4904003e00 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f491000c160 tx=0x7f4904005830 comp rx=0 tx=0).stop 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 -- 172.21.15.5:0/592900940 shutdown_connections 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 --2- 172.21.15.5:0/592900940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4904003a00 0x7f4904003e00 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 -- 172.21.15.5:0/592900940 >> 172.21.15.5:0/592900940 conn(0x7f491c08af80 msgr2=0x7f491c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 -- 172.21.15.5:0/592900940 shutdown_connections 2024-04-25T10:09:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.699+0000 7f4922a00640 1 -- 172.21.15.5:0/592900940 wait complete. 2024-04-25T10:09:55.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.701+0000 7f4922a00640 1 Processor -- start 2024-04-25T10:09:55.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.701+0000 7f4922a00640 1 -- start start 2024-04-25T10:09:55.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4922a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904003a00 0x7f491c0a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4922a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 0x7f491c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4922a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f491c1f46f0 0x7f491c0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4922a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0d2500 con 0x7f491c1f4270 2024-04-25T10:09:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4922a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0d2380 con 0x7f491c1f46f0 2024-04-25T10:09:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4922a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c0d2680 con 0x7f4904003a00 2024-04-25T10:09:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4920c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 0x7f491c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4920c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 0x7f491c0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55150/0 (socket says 172.21.15.5:55150) 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.702+0000 7f4920c00640 1 -- 172.21.15.5:0/4290651216 learned_addr learned my addr 172.21.15.5:0/4290651216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4922000640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f491c1f46f0 0x7f491c0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4921600640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904003a00 0x7f491c0a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4920c00640 1 -- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904003a00 msgr2=0x7f491c0a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4920c00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904003a00 0x7f491c0a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4920c00640 1 -- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f491c1f46f0 msgr2=0x7f491c0cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4920c00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f491c1f46f0 0x7f491c0cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4920c00640 1 -- 172.21.15.5:0/4290651216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c0b6560 con 0x7f491c1f4270 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4920c00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 0x7f491c0cb950 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f490800d510 tx=0x7f490800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908016070 con 0x7f491c1f4270 2024-04-25T10:09:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908012800 con 0x7f491c1f4270 2024-04-25T10:09:55.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908012bf0 con 0x7f491c1f4270 2024-04-25T10:09:55.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.703+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c0ca8b0 con 0x7f491c1f4270 2024-04-25T10:09:55.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.704+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c0cae70 con 0x7f491c1f4270 2024-04-25T10:09:55.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.705+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c068f60 con 0x7f491c1f4270 2024-04-25T10:09:55.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.706+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4908019070 con 0x7f491c1f4270 2024-04-25T10:09:55.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.707+0000 7f4919600640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:55.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.707+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f490808a2f0 con 0x7f491c1f4270 2024-04-25T10:09:55.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.708+0000 7f4921600640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:55.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.709+0000 7f4921600640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4910032000 tx=0x7f491002d000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:55.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.712+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4908058ea0 con 0x7f491c1f4270 2024-04-25T10:09:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.925+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f491c0c9bf0 con 0x7f491c1f4270 2024-04-25T10:09:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.926+0000 7f4919600640 1 -- 172.21.15.5:0/4290651216 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v92) v1 ==== 80+0+120 (secure 0 0 0) 0x7f49080587a0 con 0x7f491c1f4270 2024-04-25T10:09:55.933 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6834/3627376661 2024-04-26T10:09:12.513203+0000 2024-04-25T10:09:55.933 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6832/3627376661 2024-04-26T10:09:12.513203+0000 2024-04-25T10:09:55.934 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 msgr2=0x7f48fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4910032000 tx=0x7f491002d000 comp rx=0 tx=0).stop 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 msgr2=0x7f491c0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 0x7f491c0cb950 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f490800d510 tx=0x7f490800d9d0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 shutdown_connections 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f491c1f46f0 0x7f491c0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c1f4270 0x7f491c0cb950 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 --2- 172.21.15.5:0/4290651216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4904003a00 0x7f491c0a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 >> 172.21.15.5:0/4290651216 conn(0x7f491c08af80 msgr2=0x7f491c0b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 shutdown_connections 2024-04-25T10:09:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:55.929+0000 7f4922a00640 1 -- 172.21.15.5:0/4290651216 wait complete. 2024-04-25T10:09:55.955 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:09:55.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:55.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:56.055 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:09:56.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:56.055 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:56.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.308+0000 7f0833000640 1 Processor -- start 2024-04-25T10:09:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.309+0000 7f0833000640 1 -- start start 2024-04-25T10:09:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.310+0000 7f0833000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.310+0000 7f0833000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f082c0d07d0 0x7f082c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.310+0000 7f0833000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c0cf410 con 0x7f082c0d2380 2024-04-25T10:09:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.310+0000 7f0833000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c0d27c0 con 0x7f082c0d07d0 2024-04-25T10:09:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.310+0000 7f0833000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0b28b0 con 0x7f082c161200 2024-04-25T10:09:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f082c0d07d0 0x7f082c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f082c0d07d0 0x7f082c0b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55162/0 (socket says 172.21.15.5:55162) 2024-04-25T10:09:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 -- 172.21.15.5:0/2487988032 learned_addr learned my addr 172.21.15.5:0/2487988032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0831c00640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 -- 172.21.15.5:0/2487988032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 msgr2=0x7f082c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 -- 172.21.15.5:0/2487988032 >> v1:172.21.15.80:6789/0 conn(0x7f082c0d2380 legacy=0x7f082c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0832600640 1 -- 172.21.15.5:0/2487988032 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c0cf6f0 con 0x7f082c0d07d0 2024-04-25T10:09:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.311+0000 7f0831c00640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.312+0000 7f0832600640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d07d0 0x7f082c0b2f80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f08240098a0 tx=0x7f082402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6bbfbff8477f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.313+0000 7f082be00640 1 -- 172.21.15.5:0/2487988032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824037070 con 0x7f082c0d07d0 2024-04-25T10:09:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.313+0000 7f082be00640 1 -- 172.21.15.5:0/2487988032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0824032440 con 0x7f082c0d07d0 2024-04-25T10:09:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.313+0000 7f082be00640 1 -- 172.21.15.5:0/2487988032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0824032810 con 0x7f082c0d07d0 2024-04-25T10:09:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.313+0000 7f0833000640 1 -- 172.21.15.5:0/2487988032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d07d0 msgr2=0x7f082c0b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.313+0000 7f0833000640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d07d0 0x7f082c0b2f80 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f08240098a0 tx=0x7f082402a250 comp rx=0 tx=0).stop 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.314+0000 7f0833000640 1 -- 172.21.15.5:0/2487988032 shutdown_connections 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.314+0000 7f0833000640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d07d0 0x7f082c0b2f80 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.314+0000 7f0833000640 1 --2- 172.21.15.5:0/2487988032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.314+0000 7f0833000640 1 -- 172.21.15.5:0/2487988032 >> 172.21.15.5:0/2487988032 conn(0x7f082c08af80 msgr2=0x7f082c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.314+0000 7f0833000640 1 -- 172.21.15.5:0/2487988032 shutdown_connections 2024-04-25T10:09:56.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.314+0000 7f0833000640 1 -- 172.21.15.5:0/2487988032 wait complete. 2024-04-25T10:09:56.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.315+0000 7f0833000640 1 Processor -- start 2024-04-25T10:09:56.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.316+0000 7f0833000640 1 -- start start 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.316+0000 7f0833000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f082c0d07d0 0x7f082c14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.316+0000 7f0833000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 0x7f082c0c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.316+0000 7f0833000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0833000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0b34c0 con 0x7f082c0d2380 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0833000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0d42b0 con 0x7f082c0d07d0 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0833000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0b3640 con 0x7f082c161200 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 0x7f082c0c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 0x7f082c0c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55164/0 (socket says 172.21.15.5:55164) 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 -- 172.21.15.5:0/6688790 learned_addr learned my addr 172.21.15.5:0/6688790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 -- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 msgr2=0x7f082c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 -- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f082c0d07d0 msgr2=0x7f082c14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831c00640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f082c0d07d0 0x7f082c14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f082c0d07d0 0x7f082c14d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.317+0000 7f0831200640 1 -- 172.21.15.5:0/6688790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c1500a0 con 0x7f082c0d2380 2024-04-25T10:09:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.318+0000 7f0831200640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 0x7f082c0c87b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f082000c7f0 tx=0x7f082000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.318+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820016020 con 0x7f082c0d2380 2024-04-25T10:09:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.318+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082000aa50 con 0x7f082c0d2380 2024-04-25T10:09:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.318+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082000ae40 con 0x7f082c0d2380 2024-04-25T10:09:56.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.318+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c150380 con 0x7f082c0d2380 2024-04-25T10:09:56.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.318+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c0ca7f0 con 0x7f082c0d2380 2024-04-25T10:09:56.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.319+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c068f60 con 0x7f082c0d2380 2024-04-25T10:09:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.321+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0820019020 con 0x7f082c0d2380 2024-04-25T10:09:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.322+0000 7f0829600640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f081006b4d0 0x7f081006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.322+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7f082008a500 con 0x7f082c0d2380 2024-04-25T10:09:56.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.322+0000 7f0831c00640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f081006b4d0 0x7f081006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.324+0000 7f0831c00640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f081006b4d0 0x7f081006d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f081c00b840 tx=0x7f081c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.326+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0820020030 con 0x7f082c0d2380 2024-04-25T10:09:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.566+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f082c0b2da0 con 0x7f082c0d2380 2024-04-25T10:09:56.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.568+0000 7f0829600640 1 -- 172.21.15.5:0/6688790 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f08200590b0 con 0x7f082c0d2380 2024-04-25T10:09:56.575 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:56.575 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:23.419710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7683},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7683":{"gid":7683,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.5:6834/7055779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":7055779},{"type":"v1","addr":"172.21.15.5:6834","nonce":7055779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:56.575 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 68 2024-04-25T10:09:56.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f081006b4d0 msgr2=0x7f081006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f081006b4d0 0x7f081006d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f081c00b840 tx=0x7f081c008040 comp rx=0 tx=0).stop 2024-04-25T10:09:56.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 msgr2=0x7f082c0c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 0x7f082c0c87b0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f082000c7f0 tx=0x7f082000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 shutdown_connections 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f082c161200 0x7f082c0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c0d2380 0x7f082c0c87b0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f081006b4d0 0x7f081006d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.570+0000 7f0833000640 1 --2- 172.21.15.5:0/6688790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f082c0d07d0 0x7f082c14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.571+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 >> 172.21.15.5:0/6688790 conn(0x7f082c08af80 msgr2=0x7f082c0d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.571+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 shutdown_connections 2024-04-25T10:09:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:56.571+0000 7f0833000640 1 -- 172.21.15.5:0/6688790 wait complete. 2024-04-25T10:09:56.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:09:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-04-25T10:09:56.858 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:09:56.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.059+0000 7fad61400640 1 Processor -- start 2024-04-25T10:09:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.060+0000 7fad61400640 1 -- start start 2024-04-25T10:09:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.061+0000 7fad61400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad5c15d920 0x7fad5c161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.061+0000 7fad61400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fad5c0ced30 0x7fad5c0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.061+0000 7fad61400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c161fa0 con 0x7fad5c0b9250 2024-04-25T10:09:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.061+0000 7fad61400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c0d1de0 con 0x7fad5c15d920 2024-04-25T10:09:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.061+0000 7fad61400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c162560 con 0x7fad5c0ced30 2024-04-25T10:09:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad5c15d920 0x7fad5c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad5c15d920 0x7fad5c161630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45020/0 (socket says 172.21.15.5:45020) 2024-04-25T10:09:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5a800640 1 -- 172.21.15.5:0/3027332140 learned_addr learned my addr 172.21.15.5:0/3027332140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5b200640 1 --2- 172.21.15.5:0/3027332140 >> v2:172.21.15.80:3300/0 conn(0x7fad5c0ced30 0x7fad5c0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5a800640 1 -- 172.21.15.5:0/3027332140 >> v2:172.21.15.80:3300/0 conn(0x7fad5c0ced30 msgr2=0x7fad5c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5a800640 1 --2- 172.21.15.5:0/3027332140 >> v2:172.21.15.80:3300/0 conn(0x7fad5c0ced30 0x7fad5c0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.062+0000 7fad5a800640 1 -- 172.21.15.5:0/3027332140 >> v1:172.21.15.80:6789/0 conn(0x7fad5c0b9250 legacy=0x7fad5c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.063+0000 7fad5a800640 1 -- 172.21.15.5:0/3027332140 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c162280 con 0x7fad5c15d920 2024-04-25T10:09:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.063+0000 7fad5b200640 1 --2- 172.21.15.5:0/3027332140 >> v2:172.21.15.80:3300/0 conn(0x7fad5c0ced30 0x7fad5c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.064+0000 7fad5a800640 1 --2- 172.21.15.5:0/3027332140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c15d920 0x7fad5c161630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0cf190 tx=0x7fad4c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9311004736110d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.065+0000 7fad59e00640 1 -- 172.21.15.5:0/3027332140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c037070 con 0x7fad5c15d920 2024-04-25T10:09:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.065+0000 7fad59e00640 1 -- 172.21.15.5:0/3027332140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c032440 con 0x7fad5c15d920 2024-04-25T10:09:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.065+0000 7fad59e00640 1 -- 172.21.15.5:0/3027332140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c032810 con 0x7fad5c15d920 2024-04-25T10:09:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.065+0000 7fad61400640 1 -- 172.21.15.5:0/3027332140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c15d920 msgr2=0x7fad5c161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.065+0000 7fad61400640 1 --2- 172.21.15.5:0/3027332140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c15d920 0x7fad5c161630 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0cf190 tx=0x7fad4c02a160 comp rx=0 tx=0).stop 2024-04-25T10:09:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.066+0000 7fad61400640 1 -- 172.21.15.5:0/3027332140 shutdown_connections 2024-04-25T10:09:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.066+0000 7fad61400640 1 --2- 172.21.15.5:0/3027332140 >> v2:172.21.15.80:3300/0 conn(0x7fad5c0ced30 0x7fad5c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.066+0000 7fad61400640 1 --2- 172.21.15.5:0/3027332140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c15d920 0x7fad5c161630 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.066+0000 7fad61400640 1 -- 172.21.15.5:0/3027332140 >> 172.21.15.5:0/3027332140 conn(0x7fad5c06bfe0 msgr2=0x7fad5c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.066+0000 7fad61400640 1 -- 172.21.15.5:0/3027332140 shutdown_connections 2024-04-25T10:09:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.066+0000 7fad61400640 1 -- 172.21.15.5:0/3027332140 wait complete. 2024-04-25T10:09:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.068+0000 7fad61400640 1 Processor -- start 2024-04-25T10:09:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.068+0000 7fad61400640 1 -- start start 2024-04-25T10:09:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad61400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad5c0b9250 0x7fad5c1e8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 0x7fad5c0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad61400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad5c15d920 0x7fad5c14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad61400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c161cf0 con 0x7fad5c0ced30 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad61400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c161b70 con 0x7fad5c15d920 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad61400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c15dd00 con 0x7fad5c0b9250 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 0x7fad5c0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 0x7fad5c0cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45036/0 (socket says 172.21.15.5:45036) 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad53e00640 1 -- 172.21.15.5:0/1651980725 learned_addr learned my addr 172.21.15.5:0/1651980725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.069+0000 7fad5a800640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad5c0b9250 0x7fad5c1e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad5b200640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad5c15d920 0x7fad5c14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad53e00640 1 -- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad5c0b9250 msgr2=0x7fad5c1e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad53e00640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad5c0b9250 0x7fad5c1e8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad53e00640 1 -- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad5c15d920 msgr2=0x7fad5c14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad53e00640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad5c15d920 0x7fad5c14fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad53e00640 1 -- 172.21.15.5:0/1651980725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0ca9c0 con 0x7fad5c0ced30 2024-04-25T10:09:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad53e00640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 0x7fad5c0cba20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c3a0 tx=0x7fad4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440052d0 con 0x7fad5c0ced30 2024-04-25T10:09:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44005490 con 0x7fad5c0ced30 2024-04-25T10:09:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44007260 con 0x7fad5c0ced30 2024-04-25T10:09:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.070+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c1e8980 con 0x7fad5c0ced30 2024-04-25T10:09:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.071+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c0caf40 con 0x7fad5c0ced30 2024-04-25T10:09:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.072+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c068fd0 con 0x7fad5c0ced30 2024-04-25T10:09:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.073+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad44007400 con 0x7fad5c0ced30 2024-04-25T10:09:57.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.074+0000 7fad52a00640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad3806b4d0 0x7fad3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.074+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6448+0+0 (secure 0 0 0) 0x7fad440899a0 con 0x7fad5c0ced30 2024-04-25T10:09:57.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.075+0000 7fad5a800640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad3806b4d0 0x7fad3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.076+0000 7fad5a800640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad3806b4d0 0x7fad3806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fad4c034000 tx=0x7fad4c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.079+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad44058550 con 0x7fad5c0ced30 2024-04-25T10:09:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.319+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad5c0d14f0 con 0x7fad5c0ced30 2024-04-25T10:09:57.327 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:09:57.320+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.323+0000 7fad52a00640 1 -- 172.21.15.5:0/1651980725 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad44057e50 con 0x7fad5c0ced30 2024-04-25T10:09:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.327+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad3806b4d0 msgr2=0x7fad3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad3806b4d0 0x7fad3806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fad4c034000 tx=0x7fad4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 msgr2=0x7fad5c0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 0x7fad5c0cba20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c3a0 tx=0x7fad4400c860 comp rx=0 tx=0).stop 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 shutdown_connections 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fad3806b4d0 0x7fad3806d990 secure :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fad4c034000 tx=0x7fad4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad5c15d920 0x7fad5c14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c0ced30 0x7fad5c0cba20 secure :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c3a0 tx=0x7fad4400c860 comp rx=0 tx=0).stop 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 --2- 172.21.15.5:0/1651980725 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad5c0b9250 0x7fad5c1e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.328+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 >> 172.21.15.5:0/1651980725 conn(0x7fad5c06bfe0 msgr2=0x7fad5c0c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.329+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 shutdown_connections 2024-04-25T10:09:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.329+0000 7fad61400640 1 -- 172.21.15.5:0/1651980725 wait complete. 2024-04-25T10:09:57.345 INFO:tasks.ceph.mds.a.smithi005.stderr: -2874> 2024-04-25T10:09:20.062+0000 7f6b9dc00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:09:57.345 INFO:tasks.ceph.mds.a.smithi005.stderr: -2708> 2024-04-25T10:09:20.339+0000 7f6b9dc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0) n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x5639aa759b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:09:57.349 INFO:tasks.ceph.mds.a.smithi005.stderr: -935> 2024-04-25T10:09:52.425+0000 7f6b9dc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0)->f(v1 m2024-04-25T10:09:08.406925+0000) n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1)->n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5639aa759b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:08.406925+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:08.536922+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:09:57.357 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:09:57.414 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:09:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.443+0000 7f07fae33740 1 -- start start 2024-04-25T10:09:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.443+0000 7f07fae33740 1 Processor -- start 2024-04-25T10:09:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07fae33740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379a58c00 0x557379af8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07fae33740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379a59800 0x557379af8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07fae33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 0x557379af9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07fae33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737a822600 con 0x557379a59c00 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07fae33740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737a822480 con 0x557379a59800 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07fae33740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55737a822300 con 0x557379a58c00 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07f8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 0x557379af9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07f8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 0x557379af9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45074/0 (socket says 172.21.15.5:45074) 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.444+0000 7f07f8a00640 1 -- 172.21.15.5:0/490397435 learned_addr learned my addr 172.21.15.5:0/490397435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f7600640 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379a59800 0x557379af8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8000640 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379a58c00 0x557379af8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8a00640 1 -- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379a58c00 msgr2=0x557379af8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8a00640 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379a58c00 0x557379af8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8a00640 1 -- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379a59800 msgr2=0x557379af8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8a00640 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379a59800 0x557379af8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8a00640 1 -- 172.21.15.5:0/490397435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557379a85680 con 0x557379a59c00 2024-04-25T10:09:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8000640 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379a58c00 0x557379af8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f8a00640 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 0x557379af9080 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x55737a85bcb0 tx=0x55737a7ba240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1a417b1b1a93a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f4e00640 1 -- 172.21.15.5:0/490397435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557379a7e680 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f4e00640 1 -- 172.21.15.5:0/490397435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557379aa2c40 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.445+0000 7f07f4e00640 1 -- 172.21.15.5:0/490397435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557379a7eb60 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557379a85860 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557379a852c0 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07fae33740 4 waiting for MDS map... 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07f4e00640 1 -- 172.21.15.5:0/490397435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x557379d58600 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07f4e00640 1 -- 172.21.15.5:0/490397435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 69) v1 ==== 2723+0+0 (secure 0 0 0) 0x557379ab1180 con 0x557379a59c00 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07fae33740 4 Got MDS map 69 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07fae33740 4 datascan.main: Using metadata pool output 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.446+0000 7f07fae33740 4 datascan.main: connecting to RADOS... 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.447+0000 7f07fae33740 1 Processor -- start 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.447+0000 7f07fae33740 1 -- start start 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07fae33740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49000 0x557379af9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07fae33740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379af9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07fae33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 0x557379afa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07fae33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737a822180 con 0x557379d49800 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07fae33740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557379ab8480 con 0x557379d49400 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07fae33740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557379ab8900 con 0x557379d49000 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07f7600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 0x557379afa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07f7600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 0x557379afa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45090/0 (socket says 172.21.15.5:45090) 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07f7600640 1 -- 172.21.15.5:0/736932497 learned_addr learned my addr 172.21.15.5:0/736932497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.448+0000 7f07f8000640 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379af9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49000 0x557379af9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 -- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49000 msgr2=0x557379af9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49000 0x557379af9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 -- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 msgr2=0x557379af9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379af9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 -- 172.21.15.5:0/736932497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557379a854a0 con 0x557379d49800 2024-04-25T10:09:57.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f7600640 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 0x557379afa100 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x557379d531a0 tx=0x557379d74c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ada3760ad90522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f2600640 1 -- 172.21.15.5:0/736932497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557379a7e680 con 0x557379d49800 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f2600640 1 -- 172.21.15.5:0/736932497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557379aa2540 con 0x557379d49800 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.449+0000 7f07f2600640 1 -- 172.21.15.5:0/736932497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557379a7eb60 con 0x557379d49800 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 -- 172.21.15.5:0/736932497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 msgr2=0x557379afa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 0x557379afa100 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x557379d531a0 tx=0x557379d74c60 comp rx=0 tx=0).stop 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 -- 172.21.15.5:0/736932497 shutdown_connections 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49800 0x557379afa100 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379af9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 --2- 172.21.15.5:0/736932497 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49000 0x557379af9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 -- 172.21.15.5:0/736932497 >> 172.21.15.5:0/736932497 conn(0x557379d48c00 msgr2=0x557379b5c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 -- 172.21.15.5:0/736932497 shutdown_connections 2024-04-25T10:09:57.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 -- 172.21.15.5:0/736932497 wait complete. 2024-04-25T10:09:57.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.450+0000 7f07fae33740 1 Processor -- start 2024-04-25T10:09:57.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 -- start start 2024-04-25T10:09:57.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49000 0x557379afb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379afac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49800 0x557379afa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737a823380 con 0x557379d49000 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737a822180 con 0x557379d49400 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.451+0000 7f07fae33740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55737a823500 con 0x557379d49800 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49000 0x557379afb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49000 0x557379afb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45098/0 (socket says 172.21.15.5:45098) 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 learned_addr learned my addr 172.21.15.5:0/2214544111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379afac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49800 msgr2=0x557379afa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379d49800 0x557379afa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 msgr2=0x557379afac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379d49400 0x557379afac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557379a85680 con 0x557379d49000 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.452+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49000 0x557379afb180 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x557379d53e60 tx=0x55737a7ba060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.453+0000 7f07f0800640 1 -- 172.21.15.5:0/2214544111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557379a7e4e0 con 0x557379d49000 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.453+0000 7f07f0800640 1 -- 172.21.15.5:0/2214544111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557379aa2a80 con 0x557379d49000 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.453+0000 7f07f0800640 1 -- 172.21.15.5:0/2214544111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557379a7e680 con 0x557379d49000 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.453+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557379a85860 con 0x557379d49000 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.453+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557379a850e0 con 0x557379d49000 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.453+0000 7f07fae33740 4 datascan.init: resolving metadata pool 13 2024-04-25T10:09:57.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07f0800640 1 -- 172.21.15.5:0/2214544111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x557379dc8c00 con 0x557379d49000 2024-04-25T10:09:57.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07f0800640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x557379dd6800 0x557379afb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07f0800640 1 -- 172.21.15.5:0/2214544111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x557379dc6a00 con 0x557379d49000 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07fae33740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07fae33740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07fae33740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x557379dd6c00 0x557379afa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.455+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379dd7000 con 0x557379dd6c00 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.456+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x557379dd6800 0x557379afb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.456+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x557379dd6c00 0x557379afa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.457+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x557379dd6800 0x557379afb700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55737a7ea1e0 tx=0x55737a7e8900 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.457+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x557379dd6c00 0x557379afa100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.464+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.464+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8000 con 0x557379dd6c00 2024-04-25T10:09:57.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.466+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== osd_op_reply(2 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.466+0000 7f07f2600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x557379ed8400 0x557379ede000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.466+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8800 con 0x557379ed8400 2024-04-25T10:09:57.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.466+0000 7f07f8a00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x557379ed8400 0x557379ede000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.467+0000 7f07f8a00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x557379ed8400 0x557379ede000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.474+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.474+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9000 con 0x557379dd6c00 2024-04-25T10:09:57.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.475+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 3 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.475+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8c00 con 0x557379dd6c00 2024-04-25T10:09:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.476+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 4 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.476+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9400 con 0x557379ed8400 2024-04-25T10:09:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.477+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(6 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.477+0000 7f07f2600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x557379ed8000 0x557379ede580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.477+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9000 con 0x557379ed8000 2024-04-25T10:09:57.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.478+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x557379ed8000 0x557379ede580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.479+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x557379ed8000 0x557379ede580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.484+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.485+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8c00 con 0x557379dd6c00 2024-04-25T10:09:57.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.486+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 5 ==== osd_op_reply(8 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.486+0000 7f07f2600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x557379dd7000 0x557379edeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.486+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea000 con 0x557379dd7000 2024-04-25T10:09:57.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.486+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x557379dd7000 0x557379edeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.487+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x557379dd7000 0x557379edeb00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.490+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd7000 2024-04-25T10:09:57.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.490+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9800 con 0x557379ed8000 2024-04-25T10:09:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.491+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.491+0000 7f07f2600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557379eea400 0x557379edf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.491+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea800 con 0x557379eea400 2024-04-25T10:09:57.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.492+0000 7f07f8a00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557379eea400 0x557379edf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.492+0000 7f07f8a00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557379eea400 0x557379edf080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.493+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.493+0000 7f07f1c00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x557379ed9c00 0x557379ee0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.493+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8800 con 0x557379ed9c00 2024-04-25T10:09:57.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.494+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x557379ed9c00 0x557379ee0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.495+0000 7f07f8000640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x557379ed9c00 0x557379ee0100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.496+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.496+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeac00 con 0x557379ed8000 2024-04-25T10:09:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.497+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(13 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.497+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9400 con 0x557379dd6c00 2024-04-25T10:09:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.498+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 6 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.498+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9000 con 0x557379ed8400 2024-04-25T10:09:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.499+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(15 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.499+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379d48800 con 0x557379ed9c00 2024-04-25T10:09:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.500+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.500+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9800 con 0x557379dd7000 2024-04-25T10:09:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.501+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd7000 2024-04-25T10:09:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.501+0000 7f07f1c00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x557379d48400 0x557379ee0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.501+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea800 con 0x557379d48400 2024-04-25T10:09:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.501+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x557379d48400 0x557379ee0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.502+0000 7f07f7600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x557379d48400 0x557379ee0680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.506+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379d48400 2024-04-25T10:09:57.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.506+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8800 con 0x557379ed8400 2024-04-25T10:09:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.507+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 4 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.507+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb000 con 0x557379d48400 2024-04-25T10:09:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.507+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379d48400 2024-04-25T10:09:57.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.508+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55737a825400 con 0x557379dd7000 2024-04-25T10:09:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.508+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd7000 2024-04-25T10:09:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.508+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb400 con 0x557379dd7000 2024-04-25T10:09:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.509+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(22 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd7000 2024-04-25T10:09:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.509+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55737a825000 con 0x557379dd6c00 2024-04-25T10:09:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.510+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 7 ==== osd_op_reply(23 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.510+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb800 con 0x557379dd6c00 2024-04-25T10:09:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.511+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 8 ==== osd_op_reply(24 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.511+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea000 con 0x557379ed8000 2024-04-25T10:09:57.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.512+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 4 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.512+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eebc00 con 0x557379eea400 2024-04-25T10:09:57.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.513+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(26 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.513+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea800 con 0x557379ed8400 2024-04-25T10:09:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.514+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 5 ==== osd_op_reply(27 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.514+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8000 con 0x557379ed9c00 2024-04-25T10:09:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.516+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 3 ==== osd_op_reply(28 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.516+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8c00 con 0x557379eea400 2024-04-25T10:09:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.517+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 3 ==== osd_op_reply(29 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.517+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8400 con 0x557379ed9c00 2024-04-25T10:09:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.518+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 4 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.518+0000 7f07f2600640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x557379ed9400 0x557379edf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.518+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9800 con 0x557379ed9400 2024-04-25T10:09:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.518+0000 7f07f8a00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x557379ed9400 0x557379edf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.519+0000 7f07f8a00640 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x557379ed9400 0x557379edf600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.520+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(31 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.520+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8800 con 0x557379eea400 2024-04-25T10:09:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.521+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 4 ==== osd_op_reply(32 [pgnls start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.521+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef9800 con 0x557379ed9c00 2024-04-25T10:09:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.522+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 5 ==== osd_op_reply(33 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.522+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8c00 con 0x557379dd7000 2024-04-25T10:09:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.523+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.2 v2:172.21.15.5:6822/3928063993 5 ==== osd_op_reply(34 [pgnls start_epoch 93 out=49b] v89'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd7000 2024-04-25T10:09:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.523+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9000 con 0x557379d48400 2024-04-25T10:09:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.524+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.0 v2:172.21.15.5:6809/1760674718 3 ==== osd_op_reply(35 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379d48400 2024-04-25T10:09:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.524+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef9000 con 0x557379ed9c00 2024-04-25T10:09:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.525+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 6 ==== osd_op_reply(36 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.525+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8800 con 0x557379d48400 2024-04-25T10:09:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.526+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.0 v2:172.21.15.5:6809/1760674718 4 ==== osd_op_reply(37 [pgnls start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379d48400 2024-04-25T10:09:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.526+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef9400 con 0x557379eea400 2024-04-25T10:09:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.527+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 5 ==== osd_op_reply(38 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.527+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8c00 con 0x557379ed9400 2024-04-25T10:09:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.528+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(39 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.528+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb000 con 0x557379ed9400 2024-04-25T10:09:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.529+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(40 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.529+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed9800 con 0x557379ed8000 2024-04-25T10:09:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.530+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 5 ==== osd_op_reply(41 [pgnls start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.530+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb400 con 0x557379ed9c00 2024-04-25T10:09:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.532+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 7 ==== osd_op_reply(42 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.532+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eebc00 con 0x557379ed9400 2024-04-25T10:09:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.533+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 4 ==== osd_op_reply(43 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.533+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb800 con 0x557379ed9400 2024-04-25T10:09:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.533+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 5 ==== osd_op_reply(44 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.534+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea800 con 0x557379ed9400 2024-04-25T10:09:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.534+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 6 ==== osd_op_reply(45 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.535+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeac00 con 0x557379dd7000 2024-04-25T10:09:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.536+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.2 v2:172.21.15.5:6822/3928063993 6 ==== osd_op_reply(46 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd7000 2024-04-25T10:09:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.536+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8800 con 0x557379dd6c00 2024-04-25T10:09:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.537+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 9 ==== osd_op_reply(47 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.537+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379d48800 con 0x557379ed9c00 2024-04-25T10:09:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.538+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 8 ==== osd_op_reply(48 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.538+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef9400 con 0x557379dd6c00 2024-04-25T10:09:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.540+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 10 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.540+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55737a825400 con 0x557379eea400 2024-04-25T10:09:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.541+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 6 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.541+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb000 con 0x557379ed8000 2024-04-25T10:09:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.542+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 6 ==== osd_op_reply(51 [pgnls start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.542+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55737a825000 con 0x557379ed8000 2024-04-25T10:09:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.543+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.5 v2:172.21.15.80:6800/161144277 7 ==== osd_op_reply(52 [pgnls start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed8000 2024-04-25T10:09:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.543+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eebc00 con 0x557379ed8400 2024-04-25T10:09:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.544+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 6 ==== osd_op_reply(53 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.544+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8c00 con 0x557379eea400 2024-04-25T10:09:57.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.545+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 7 ==== osd_op_reply(54 [pgnls start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.545+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeb800 con 0x557379ed8400 2024-04-25T10:09:57.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.546+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 7 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.546+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8000 con 0x557379eea400 2024-04-25T10:09:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.547+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 8 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.547+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea800 con 0x557379eea400 2024-04-25T10:09:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.548+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 9 ==== osd_op_reply(57 [pgnls start_epoch 93 out=81b] v89'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.549+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379dd7400 con 0x557379eea400 2024-04-25T10:09:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.550+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.550+0000 7f07fae33740 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:09:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.550+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379dd7800 con 0x557379eea400 2024-04-25T10:09:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.553+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 11 ==== osd_op_reply(59 10000000001.00000000 [call] v93'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.554+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379dd7c00 con 0x557379eea400 2024-04-25T10:09:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.554+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 12 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.555+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eea000 con 0x557379dd6c00 2024-04-25T10:09:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.556+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 11 ==== osd_op_reply(61 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.556+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef8400 con 0x557379ed9400 2024-04-25T10:09:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.557+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.3 v2:172.21.15.5:6808/3976284996 7 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed9400 2024-04-25T10:09:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.557+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ed8c00 con 0x557379eea400 2024-04-25T10:09:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.558+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.1 v2:172.21.15.5:6800/692601986 13 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379eea400 2024-04-25T10:09:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.558+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef9800 con 0x557379dd6c00 2024-04-25T10:09:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.559+0000 7f07f7600640 1 -- 172.21.15.5:0/2214544111 <== osd.4 v2:172.21.15.80:6808/2045277337 12 ==== osd_op_reply(64 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6000 con 0x557379dd6c00 2024-04-25T10:09:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.559+0000 7f07f2600640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379eeac00 con 0x557379ed9c00 2024-04-25T10:09:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.560+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 <== osd.7 v2:172.21.15.80:6816/4166609972 9 ==== osd_op_reply(65 [pgnls start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6780 con 0x557379ed9c00 2024-04-25T10:09:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.561+0000 7f07f1c00640 1 -- 172.21.15.5:0/2214544111 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x557379ef9c00 con 0x557379ed8400 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07f8a00640 1 -- 172.21.15.5:0/2214544111 <== osd.6 v2:172.21.15.80:6824/3989257232 8 ==== osd_op_reply(66 [pgnls start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55737a7d6500 con 0x557379ed8400 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x557379d48400 msgr2=0x557379ee0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x557379d48400 0x557379ee0680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557379eea400 msgr2=0x557379edf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x557379eea400 0x557379edf080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x557379dd7000 msgr2=0x557379edeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x557379dd7000 0x557379edeb00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x557379ed9400 msgr2=0x557379edf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x557379ed9400 0x557379edf600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 reap_dead start 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x557379dd6c00 msgr2=0x557379afa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x557379dd6c00 0x557379afa100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x557379ed8000 msgr2=0x557379ede580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x557379ed8000 0x557379ede580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x557379ed8400 msgr2=0x557379ede000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x557379ed8400 0x557379ede000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x557379ed9c00 msgr2=0x557379ee0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x557379ed9c00 0x557379ee0100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x557379dd6800 msgr2=0x557379afb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x557379dd6800 0x557379afb700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55737a7ea1e0 tx=0x55737a7e8900 comp rx=0 tx=0).stop 2024-04-25T10:09:57.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49000 msgr2=0x557379afb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.562+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 reap_dead start 2024-04-25T10:09:57.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07fae33740 1 --2- 172.21.15.5:0/2214544111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379d49000 0x557379afb180 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x557379d53e60 tx=0x55737a7ba060 comp rx=0 tx=0).stop 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 reap_dead start 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 reap_dead start 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07f8000640 1 -- 172.21.15.5:0/2214544111 reap_dead start 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 shutdown_connections 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.563+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 >> 172.21.15.5:0/2214544111 conn(0x557379d48c00 msgr2=0x557379b5c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.564+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 shutdown_connections 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.564+0000 7f07fae33740 1 -- 172.21.15.5:0/2214544111 wait complete. 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 msgr2=0x557379af9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 0x557379af9080 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x55737a85bcb0 tx=0x55737a7ba240 comp rx=0 tx=0).stop 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 shutdown_connections 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557379a59c00 0x557379af9080 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x557379a59800 0x557379af8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 --2- 172.21.15.5:0/490397435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557379a58c00 0x557379af8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 >> 172.21.15.5:0/490397435 conn(0x557379a59000 msgr2=0x557379b5d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 shutdown_connections 2024-04-25T10:09:57.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.565+0000 7f07fae33740 1 -- 172.21.15.5:0/490397435 wait complete. 2024-04-25T10:09:57.581 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.223822, result: 2024-04-25T10:09:57.581 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T10:09:57.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.675+0000 7f5855a0d740 1 -- start start 2024-04-25T10:09:57.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.675+0000 7f5855a0d740 1 Processor -- start 2024-04-25T10:09:57.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 0x55ba61ee8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba61e49800 0x55ba61ee8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba61e49c00 0x55ba61ee9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5855a0d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d06180 con 0x55ba61e48c00 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5855a0d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d06300 con 0x55ba61e49c00 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5855a0d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d06480 con 0x55ba61e49800 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5853000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 0x55ba61ee8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5853000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 0x55ba61ee8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45110/0 (socket says 172.21.15.5:45110) 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5853000640 1 -- 172.21.15.5:0/1181469324 learned_addr learned my addr 172.21.15.5:0/1181469324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.676+0000 7f5853a00640 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba61e49c00 0x55ba61ee9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5852600640 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba61e49800 0x55ba61ee8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5853000640 1 -- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba61e49800 msgr2=0x55ba61ee8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5853000640 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba61e49800 0x55ba61ee8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5853000640 1 -- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba61e49c00 msgr2=0x55ba61ee9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5853000640 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba61e49c00 0x55ba61ee9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5853000640 1 -- 172.21.15.5:0/1181469324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba61e75680 con 0x55ba61e48c00 2024-04-25T10:09:57.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5852600640 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba61e49800 0x55ba61ee8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:57.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5853000640 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 0x55ba61ee8580 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x55ba62c5d890 tx=0x55ba62baa600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb7bcab72ac1106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f584fe00640 1 -- 172.21.15.5:0/1181469324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ba61e6e680 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f584fe00640 1 -- 172.21.15.5:0/1181469324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba61e92540 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f584fe00640 1 -- 172.21.15.5:0/1181469324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ba61e6eb60 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba61e754a0 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ba61e75a40 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.677+0000 7f5855a0d740 4 waiting for MDS map... 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.678+0000 7f584fe00640 1 -- 172.21.15.5:0/1181469324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x55ba62d16600 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.678+0000 7f584fe00640 1 -- 172.21.15.5:0/1181469324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 69) v1 ==== 2723+0+0 (secure 0 0 0) 0x55ba61ea1180 con 0x55ba61e48c00 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.678+0000 7f5855a0d740 4 Got MDS map 69 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.678+0000 7f5855a0d740 4 datascan.main: Using metadata pool output 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.678+0000 7f5855a0d740 4 datascan.main: connecting to RADOS... 2024-04-25T10:09:57.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.679+0000 7f5855a0d740 1 Processor -- start 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.679+0000 7f5855a0d740 1 -- start start 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 0x55ba61ee9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61ee9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13800 0x55ba61eea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5855a0d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d06600 con 0x55ba62d13000 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5855a0d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d06780 con 0x55ba62d13400 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5855a0d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d06900 con 0x55ba62d13800 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5853a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 0x55ba61ee9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5853a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 0x55ba61ee9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45126/0 (socket says 172.21.15.5:45126) 2024-04-25T10:09:57.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.680+0000 7f5853a00640 1 -- 172.21.15.5:0/3010125504 learned_addr learned my addr 172.21.15.5:0/3010125504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5852600640 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13800 0x55ba61eea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853000640 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61ee9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853a00640 1 -- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13800 msgr2=0x55ba61eea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853a00640 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13800 0x55ba61eea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853a00640 1 -- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 msgr2=0x55ba61ee9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853a00640 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61ee9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853a00640 1 -- 172.21.15.5:0/3010125504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba61e75860 con 0x55ba62d13000 2024-04-25T10:09:57.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f5853a00640 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 0x55ba61ee9600 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x55ba62c26b70 tx=0x55ba62d32c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbfd2a68e41791d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f584d600640 1 -- 172.21.15.5:0/3010125504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ba61e6e680 con 0x55ba62d13000 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f584d600640 1 -- 172.21.15.5:0/3010125504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba61e92e00 con 0x55ba62d13000 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.681+0000 7f584d600640 1 -- 172.21.15.5:0/3010125504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ba61e6eb60 con 0x55ba62d13000 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 -- 172.21.15.5:0/3010125504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 msgr2=0x55ba61ee9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 0x55ba61ee9600 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x55ba62c26b70 tx=0x55ba62d32c00 comp rx=0 tx=0).stop 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 -- 172.21.15.5:0/3010125504 shutdown_connections 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13800 0x55ba61eea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61ee9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 --2- 172.21.15.5:0/3010125504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13000 0x55ba61ee9600 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 -- 172.21.15.5:0/3010125504 >> 172.21.15.5:0/3010125504 conn(0x55ba62d12c00 msgr2=0x55ba61f37800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 -- 172.21.15.5:0/3010125504 shutdown_connections 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.682+0000 7f5855a0d740 1 -- 172.21.15.5:0/3010125504 wait complete. 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.683+0000 7f5855a0d740 1 Processor -- start 2024-04-25T10:09:57.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.683+0000 7f5855a0d740 1 -- start start 2024-04-25T10:09:57.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13000 0x55ba61eeb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61eeac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5855a0d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 0x55ba61eea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5855a0d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d07c80 con 0x55ba62d13800 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5855a0d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d07b00 con 0x55ba62d13400 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5855a0d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba62d07980 con 0x55ba62d13000 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5852600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 0x55ba61eea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5852600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 0x55ba61eea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45134/0 (socket says 172.21.15.5:45134) 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.684+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 learned_addr learned my addr 172.21.15.5:0/17543921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61eeac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13000 0x55ba61eeb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13000 msgr2=0x55ba61eeb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba62d13000 0x55ba61eeb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 msgr2=0x55ba61eeac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba62d13400 0x55ba61eeac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba61e75680 con 0x55ba62d13800 2024-04-25T10:09:57.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 0x55ba61eea680 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x55ba62c5d230 tx=0x55ba62baa3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f584b800640 1 -- 172.21.15.5:0/17543921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ba61e6e9c0 con 0x55ba62d13800 2024-04-25T10:09:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f584b800640 1 -- 172.21.15.5:0/17543921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba61e92c40 con 0x55ba62d13800 2024-04-25T10:09:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.685+0000 7f584b800640 1 -- 172.21.15.5:0/17543921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ba61e6e680 con 0x55ba62d13800 2024-04-25T10:09:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.686+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba61e754a0 con 0x55ba62d13800 2024-04-25T10:09:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.686+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba61e75a40 con 0x55ba62d13800 2024-04-25T10:09:57.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.686+0000 7f5855a0d740 4 datascan.init: resolving metadata pool 13 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.687+0000 7f584b800640 1 -- 172.21.15.5:0/17543921 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55ba62d90c00 con 0x55ba62d13800 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f584b800640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55ba62d92800 0x55ba61eeb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f584b800640 1 -- 172.21.15.5:0/17543921 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x55ba62d8ea00 con 0x55ba62d13800 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f5855a0d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f5855a0d740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f5855a0d740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:09:57.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55ba62d92c00 0x55ba61eea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.688+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93000 con 0x55ba62d92c00 2024-04-25T10:09:57.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.689+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55ba62d92800 0x55ba61eeb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.689+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55ba62d92c00 0x55ba61eea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.690+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55ba62d92800 0x55ba61eeb700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55ba62c26b70 tx=0x55ba62baa120 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.690+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55ba62d92c00 0x55ba61eea100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.691+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.691+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55ba62d93400 0x55ba61ee9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.691+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93800 con 0x55ba62d93400 2024-04-25T10:09:57.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.692+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55ba62d93400 0x55ba61ee9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.692+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55ba62d93400 0x55ba61ee9b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.693+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.693+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55ba62d93c00 0x55ba61ee9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.693+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6000 con 0x55ba62d93c00 2024-04-25T10:09:57.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.693+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55ba62d93c00 0x55ba61ee9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.694+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55ba62d93c00 0x55ba61ee9600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.696+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.696+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7800 con 0x55ba62d93c00 2024-04-25T10:09:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.697+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.697+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12800 con 0x55ba62d92c00 2024-04-25T10:09:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.698+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.698+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93000 con 0x55ba62d93c00 2024-04-25T10:09:57.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.699+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.699+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6000 con 0x55ba62d93c00 2024-04-25T10:09:57.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.700+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.700+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12400 con 0x55ba62d92c00 2024-04-25T10:09:57.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.701+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(8 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.701+0000 7f584d600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55ba621d7800 0x55ba621e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.701+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7c00 con 0x55ba621d7800 2024-04-25T10:09:57.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.701+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55ba621d7800 0x55ba621e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.703+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55ba621d7800 0x55ba621e4000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.703+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.703+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c000 con 0x55ba62d93c00 2024-04-25T10:09:57.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.704+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 5 ==== osd_op_reply(10 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.705+0000 7f584d600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55ba62c3cc00 0x55ba621e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.705+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62c3cc00 2024-04-25T10:09:57.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.705+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55ba62c3cc00 0x55ba621e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.706+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55ba62c3cc00 0x55ba621e4580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.706+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(11 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.706+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba621d7800 2024-04-25T10:09:57.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.707+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.707+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d400 con 0x55ba62d93400 2024-04-25T10:09:57.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.708+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.708+0000 7f584cc00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55ba62c3c800 0x55ba621e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.708+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62c3c800 2024-04-25T10:09:57.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.709+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55ba62c3c800 0x55ba621e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.710+0000 7f5853000640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55ba62c3c800 0x55ba621e5080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.711+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.711+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d800 con 0x55ba621d7800 2024-04-25T10:09:57.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.711+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.712+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d400 con 0x55ba62d93c00 2024-04-25T10:09:57.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.713+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 6 ==== osd_op_reply(16 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.713+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c000 con 0x55ba62d92c00 2024-04-25T10:09:57.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.714+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.714+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d92400 con 0x55ba62c3c800 2024-04-25T10:09:57.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.715+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.715+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12800 con 0x55ba62c3cc00 2024-04-25T10:09:57.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.715+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.715+0000 7f584cc00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55ba62d93800 0x55ba621e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.715+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62d93800 2024-04-25T10:09:57.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.716+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55ba62d93800 0x55ba621e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.716+0000 7f5852600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55ba62d93800 0x55ba621e5600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.717+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.717+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12400 con 0x55ba62d92c00 2024-04-25T10:09:57.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.718+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.718+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3dc00 con 0x55ba62d93800 2024-04-25T10:09:57.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.719+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.719+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6000 con 0x55ba62c3cc00 2024-04-25T10:09:57.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.719+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.719+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba62c3cc00 2024-04-25T10:09:57.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.720+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.720+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93000 con 0x55ba62d93c00 2024-04-25T10:09:57.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.720+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.721+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d800 con 0x55ba62d93c00 2024-04-25T10:09:57.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.721+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.721+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4000 con 0x55ba621d7800 2024-04-25T10:09:57.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.722+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 4 ==== osd_op_reply(27 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.722+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62d93400 2024-04-25T10:09:57.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.723+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.723+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4400 con 0x55ba62d92c00 2024-04-25T10:09:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.724+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.724+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7c00 con 0x55ba62c3c800 2024-04-25T10:09:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.725+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 3 ==== osd_op_reply(30 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.725+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4800 con 0x55ba62d93400 2024-04-25T10:09:57.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.725+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.725+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c2b400 con 0x55ba62c3c800 2024-04-25T10:09:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.726+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.726+0000 7f584d600640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55ba621f4c00 0x55ba621e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.726+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5000 con 0x55ba621f4c00 2024-04-25T10:09:57.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.727+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55ba621f4c00 0x55ba621e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.727+0000 7f5853a00640 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55ba621f4c00 0x55ba621e4b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.728+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.728+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba62d93400 2024-04-25T10:09:57.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.728+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.728+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5400 con 0x55ba62c3c800 2024-04-25T10:09:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.729+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.729+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62c3cc00 2024-04-25T10:09:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.730+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 93 out=49b] v89'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.730+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4800 con 0x55ba62d93800 2024-04-25T10:09:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.730+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.730+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d92400 con 0x55ba62c3c800 2024-04-25T10:09:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.731+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.731+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4400 con 0x55ba62d93800 2024-04-25T10:09:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.731+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.732+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93000 con 0x55ba62d93400 2024-04-25T10:09:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.732+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 6 ==== osd_op_reply(40 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.732+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5800 con 0x55ba621f4c00 2024-04-25T10:09:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.733+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(41 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.733+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12800 con 0x55ba621f4c00 2024-04-25T10:09:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.733+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.733+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5c00 con 0x55ba621d7800 2024-04-25T10:09:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.734+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.734+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12400 con 0x55ba62c3c800 2024-04-25T10:09:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.735+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 7 ==== osd_op_reply(44 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.735+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d400 con 0x55ba621f4c00 2024-04-25T10:09:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.736+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.736+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6000 con 0x55ba621f4c00 2024-04-25T10:09:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.737+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.737+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c000 con 0x55ba621f4c00 2024-04-25T10:09:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.737+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.737+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12800 con 0x55ba62c3cc00 2024-04-25T10:09:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.738+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 6 ==== osd_op_reply(48 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.738+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d800 con 0x55ba62d93c00 2024-04-25T10:09:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.739+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 9 ==== osd_op_reply(49 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.739+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5000 con 0x55ba62c3c800 2024-04-25T10:09:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.740+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 8 ==== osd_op_reply(50 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.740+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200000 con 0x55ba62d93c00 2024-04-25T10:09:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.741+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 10 ==== osd_op_reply(51 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.741+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5800 con 0x55ba62d93400 2024-04-25T10:09:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.742+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 7 ==== osd_op_reply(52 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.742+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200400 con 0x55ba621d7800 2024-04-25T10:09:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.743+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.743+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93000 con 0x55ba621d7800 2024-04-25T10:09:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.744+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.744+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200800 con 0x55ba62d92c00 2024-04-25T10:09:57.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.745+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 7 ==== osd_op_reply(55 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.745+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba62d93400 2024-04-25T10:09:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.746+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 8 ==== osd_op_reply(56 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.746+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200c00 con 0x55ba62d92c00 2024-04-25T10:09:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.747+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.747+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62d93400 2024-04-25T10:09:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.747+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.747+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 93 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201000 con 0x55ba62d93400 2024-04-25T10:09:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.748+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 10 ==== osd_op_reply(59 [pgnls-filter start_epoch 93 out=81b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.748+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6400 con 0x55ba62d93c00 2024-04-25T10:09:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.750+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 11 ==== osd_op_reply(60 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.750+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201400 con 0x55ba62d93c00 2024-04-25T10:09:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.751+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 12 ==== osd_op_reply(61 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.751+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d400 con 0x55ba62d92c00 2024-04-25T10:09:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.752+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.752+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201800 con 0x55ba62d93c00 2024-04-25T10:09:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.753+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 13 ==== osd_op_reply(63 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.753+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c000 con 0x55ba62d93c00 2024-04-25T10:09:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.754+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.754+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201c00 con 0x55ba62d92c00 2024-04-25T10:09:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.756+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 10 ==== osd_op_reply(65 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.756+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba621d7800 2024-04-25T10:09:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.757+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.757+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5800 con 0x55ba62d93c00 2024-04-25T10:09:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.759+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 15 ==== osd_op_reply(67 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.759+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12800 con 0x55ba62c3cc00 2024-04-25T10:09:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.759+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.759+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4000 con 0x55ba621d7800 2024-04-25T10:09:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.760+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 9 ==== osd_op_reply(69 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.761+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12400 con 0x55ba62d93400 2024-04-25T10:09:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.761+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 11 ==== osd_op_reply(70 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.762+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4800 con 0x55ba62c3c800 2024-04-25T10:09:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.763+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 9 ==== osd_op_reply(71 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.763+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d92400 con 0x55ba621d7800 2024-04-25T10:09:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.764+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.764+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4400 con 0x55ba62d93c00 2024-04-25T10:09:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.765+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 16 ==== osd_op_reply(73 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.765+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c2b400 con 0x55ba62d92c00 2024-04-25T10:09:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.766+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 11 ==== osd_op_reply(74 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.766+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5c00 con 0x55ba62c3c800 2024-04-25T10:09:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.767+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.768+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6000 con 0x55ba62c3cc00 2024-04-25T10:09:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.768+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.769+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5400 con 0x55ba62d93800 2024-04-25T10:09:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.769+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 5 ==== osd_op_reply(77 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.769+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7c00 con 0x55ba62d92c00 2024-04-25T10:09:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.771+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.771+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62d93800 2024-04-25T10:09:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.772+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 6 ==== osd_op_reply(79 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.772+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200800 con 0x55ba62c3cc00 2024-04-25T10:09:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.772+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 9 ==== osd_op_reply(80 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.773+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3dc00 con 0x55ba62c3cc00 2024-04-25T10:09:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.774+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.774+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200c00 con 0x55ba62d93c00 2024-04-25T10:09:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.775+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 17 ==== osd_op_reply(82 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.775+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d800 con 0x55ba62d93c00 2024-04-25T10:09:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.776+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 18 ==== osd_op_reply(83 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.776+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201400 con 0x55ba621d7800 2024-04-25T10:09:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.777+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 11 ==== osd_op_reply(84 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.777+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201400 con 0x55ba62d93400 2024-04-25T10:09:57.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.778+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.779+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201800 con 0x55ba62d92c00 2024-04-25T10:09:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.780+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 13 ==== osd_op_reply(86 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.780+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200400 con 0x55ba62c3c800 2024-04-25T10:09:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.781+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.781+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201c00 con 0x55ba62d93400 2024-04-25T10:09:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.782+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.782+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200000 con 0x55ba62c3c800 2024-04-25T10:09:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.783+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.783+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5800 con 0x55ba621f4c00 2024-04-25T10:09:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.784+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 7 ==== osd_op_reply(90 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.784+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba62d93400 2024-04-25T10:09:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.785+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 14 ==== osd_op_reply(91 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.785+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5000 con 0x55ba62c3c800 2024-04-25T10:09:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.787+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 13 ==== osd_op_reply(92 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.787+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62c3cc00 2024-04-25T10:09:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.788+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 11 ==== osd_op_reply(93 [pgnls-filter start_epoch 93 out=49b] v89'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.788+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d400 con 0x55ba62d93800 2024-04-25T10:09:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.789+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 7 ==== osd_op_reply(94 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.789+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4000 con 0x55ba62c3c800 2024-04-25T10:09:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.791+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 14 ==== osd_op_reply(95 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.791+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c000 con 0x55ba62d93800 2024-04-25T10:09:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.791+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.0 v2:172.21.15.5:6809/1760674718 8 ==== osd_op_reply(96 [pgnls-filter start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba62d93800 2024-04-25T10:09:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.791+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5400 con 0x55ba62d93400 2024-04-25T10:09:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.792+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 15 ==== osd_op_reply(97 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.792+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6400 con 0x55ba621f4c00 2024-04-25T10:09:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.793+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 8 ==== osd_op_reply(98 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.793+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d93000 con 0x55ba621f4c00 2024-04-25T10:09:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.794+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 9 ==== osd_op_reply(99 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.794+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d800 con 0x55ba621d7800 2024-04-25T10:09:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.795+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 12 ==== osd_op_reply(100 [pgnls-filter start_epoch 93 out=49b] v89'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.795+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d92400 con 0x55ba62c3c800 2024-04-25T10:09:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.796+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 15 ==== osd_op_reply(101 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.796+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3dc00 con 0x55ba621f4c00 2024-04-25T10:09:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.797+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 10 ==== osd_op_reply(102 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.797+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200000 con 0x55ba621f4c00 2024-04-25T10:09:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.798+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 11 ==== osd_op_reply(103 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.798+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7c00 con 0x55ba621f4c00 2024-04-25T10:09:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.799+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 12 ==== osd_op_reply(104 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.799+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200400 con 0x55ba62c3cc00 2024-04-25T10:09:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.800+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.2 v2:172.21.15.5:6822/3928063993 12 ==== osd_op_reply(105 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62c3cc00 2024-04-25T10:09:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.800+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6000 con 0x55ba62d93c00 2024-04-25T10:09:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.802+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 19 ==== osd_op_reply(106 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.802+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200800 con 0x55ba62c3c800 2024-04-25T10:09:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.803+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 16 ==== osd_op_reply(107 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.803+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12800 con 0x55ba62d93c00 2024-04-25T10:09:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.804+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 20 ==== osd_op_reply(108 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.804+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200c00 con 0x55ba62d93400 2024-04-25T10:09:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.805+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 93 out=49b] v89'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.805+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62d12400 con 0x55ba621d7800 2024-04-25T10:09:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.806+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 93 out=49b] v89'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.806+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201000 con 0x55ba621d7800 2024-04-25T10:09:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.807+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 <== osd.5 v2:172.21.15.80:6800/161144277 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 93 out=49b] v89'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2780 con 0x55ba621d7800 2024-04-25T10:09:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.808+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4800 con 0x55ba62d92c00 2024-04-25T10:09:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.809+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 14 ==== osd_op_reply(112 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.809+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201400 con 0x55ba62d93400 2024-04-25T10:09:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.810+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 17 ==== osd_op_reply(113 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.810+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f4400 con 0x55ba62d92c00 2024-04-25T10:09:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.811+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 15 ==== osd_op_reply(114 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.811+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201800 con 0x55ba62d93400 2024-04-25T10:09:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.812+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 18 ==== osd_op_reply(115 [pgnls-filter start_epoch 93 out=49b] v89'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.812+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621f5c00 con 0x55ba62d93400 2024-04-25T10:09:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.813+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 19 ==== osd_op_reply(116 [pgnls-filter start_epoch 93 out=81b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.813+0000 7f5855a0d740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:09:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.813+0000 7f5855a0d740 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:09:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.813+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6c00 con 0x55ba62d93400 2024-04-25T10:09:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.814+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 20 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.814+0000 7f5855a0d740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:09:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.814+0000 7f5855a0d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T10:09:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.814+0000 7f5855a0d740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T10:09:57.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.814+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7000 con 0x55ba62c3c800 2024-04-25T10:09:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.816+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.816+0000 7f5855a0d740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T10:09:57.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.816+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d7400 con 0x55ba62d93400 2024-04-25T10:09:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.816+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 21 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+581 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.817+0000 7f5855a0d740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T10:09:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.817+0000 7f5855a0d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:09:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.817+0000 7f5855a0d740 20 datascan.get_frag_of: frag is * 2024-04-25T10:09:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.817+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200000 con 0x55ba62c3c800 2024-04-25T10:09:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.818+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.818+0000 7f5855a0d740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T10:09:57.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.818+0000 7f5855a0d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T10:09:57.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.818+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3c400 con 0x55ba62c3c800 2024-04-25T10:09:57.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.819+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.819+0000 7f5855a0d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:09:57.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.819+0000 7f5855a0d740 20 datascan.read_dentry: file_head not found in result 2024-04-25T10:09:57.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.819+0000 7f5855a0d740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T10:09:57.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.819+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3d000 con 0x55ba62c3c800 2024-04-25T10:09:57.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.823+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v93'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.823+0000 7f5855a0d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:09:57.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.823+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c3dc00 con 0x55ba62d93400 2024-04-25T10:09:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.824+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 22 ==== osd_op_reply(123 [pgnls-filter start_epoch 93 out=49b] v89'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.824+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62c2b400 con 0x55ba62d93c00 2024-04-25T10:09:57.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.825+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.826+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62201c00 con 0x55ba621f4c00 2024-04-25T10:09:57.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.827+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.3 v2:172.21.15.5:6808/3976284996 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 93 out=49b] v89'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba621f4c00 2024-04-25T10:09:57.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.827+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200400 con 0x55ba62d93400 2024-04-25T10:09:57.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.828+0000 7f5853a00640 1 -- 172.21.15.5:0/17543921 <== osd.1 v2:172.21.15.5:6800/692601986 23 ==== osd_op_reply(126 [pgnls-filter start_epoch 93 out=49b] v89'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2500 con 0x55ba62d93400 2024-04-25T10:09:57.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.828+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6c00 con 0x55ba62d93c00 2024-04-25T10:09:57.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.829+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.4 v2:172.21.15.80:6808/2045277337 22 ==== osd_op_reply(127 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d93c00 2024-04-25T10:09:57.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.829+0000 7f584cc00640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba62200c00 con 0x55ba62c3c800 2024-04-25T10:09:57.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.830+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.7 v2:172.21.15.80:6816/4166609972 21 ==== osd_op_reply(128 [pgnls-filter start_epoch 93 out=49b] v89'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62c3c800 2024-04-25T10:09:57.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.830+0000 7f584d600640 1 -- 172.21.15.5:0/17543921 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 93 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55ba621d6800 con 0x55ba62d92c00 2024-04-25T10:09:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.831+0000 7f5853000640 1 -- 172.21.15.5:0/17543921 <== osd.6 v2:172.21.15.80:6824/3989257232 16 ==== osd_op_reply(129 [pgnls-filter start_epoch 93 out=49b] v89'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba62bc2000 con 0x55ba62d92c00 2024-04-25T10:09:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.831+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55ba62d93800 msgr2=0x55ba621e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.831+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55ba62d93800 0x55ba621e5600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.831+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55ba62d93400 msgr2=0x55ba61ee9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.831+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55ba62d93400 0x55ba61ee9b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.831+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55ba62c3cc00 msgr2=0x55ba621e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55ba62c3cc00 0x55ba621e4580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55ba621f4c00 msgr2=0x55ba621e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55ba621f4c00 0x55ba621e4b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55ba62d93c00 msgr2=0x55ba61ee9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55ba62d93c00 0x55ba61ee9600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55ba621d7800 msgr2=0x55ba621e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55ba621d7800 0x55ba621e4000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55ba62d92c00 msgr2=0x55ba61eea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55ba62d92c00 0x55ba61eea100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55ba62c3c800 msgr2=0x55ba621e5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55ba62c3c800 0x55ba621e5080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 reap_dead start 2024-04-25T10:09:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 reap_dead start 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55ba62d92800 msgr2=0x55ba61eeb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5852600640 1 -- 172.21.15.5:0/17543921 reap_dead start 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55ba62d92800 0x55ba61eeb700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55ba62c26b70 tx=0x55ba62baa120 comp rx=0 tx=0).stop 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 msgr2=0x55ba61eea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.832+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 0x55ba61eea680 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x55ba62c5d230 tx=0x55ba62baa3c0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.833+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 shutdown_connections 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.833+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba62d13800 0x55ba61eea680 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.833+0000 7f5855a0d740 1 --2- 172.21.15.5:0/17543921 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55ba62d92800 0x55ba61eeb700 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.833+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 >> 172.21.15.5:0/17543921 conn(0x55ba62d12c00 msgr2=0x55ba61f37800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.833+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 shutdown_connections 2024-04-25T10:09:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.833+0000 7f5855a0d740 1 -- 172.21.15.5:0/17543921 wait complete. 2024-04-25T10:09:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.834+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 msgr2=0x55ba61ee8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.834+0000 7f5855a0d740 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 0x55ba61ee8580 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x55ba62c5d890 tx=0x55ba62baa600 comp rx=0 tx=0).stop 2024-04-25T10:09:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 shutdown_connections 2024-04-25T10:09:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55ba61e49c00 0x55ba61ee9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55ba61e49800 0x55ba61ee8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 --2- 172.21.15.5:0/1181469324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba61e48c00 0x55ba61ee8580 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 >> 172.21.15.5:0/1181469324 conn(0x55ba61e49000 msgr2=0x55ba61f37000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 shutdown_connections 2024-04-25T10:09:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.835+0000 7f5855a0d740 1 -- 172.21.15.5:0/1181469324 wait complete. 2024-04-25T10:09:57.855 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.274447, result: 2024-04-25T10:09:57.856 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:09:57.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.962+0000 7f3ddf622740 1 -- start start 2024-04-25T10:09:57.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.962+0000 7f3ddf622740 1 Processor -- start 2024-04-25T10:09:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 0x55f752702b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f752663c00 0x55f752703080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f753443400 0x55f752703600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddf622740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f753524000 con 0x55f752662c00 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddf622740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f753524180 con 0x55f753443400 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddf622740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f753524300 con 0x55f752663c00 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.963+0000 7f3ddcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 0x55f752702b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 0x55f752702b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45162/0 (socket says 172.21.15.5:45162) 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 -- 172.21.15.5:0/1482907395 learned_addr learned my addr 172.21.15.5:0/1482907395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddc200640 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f752663c00 0x55f752703080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddd600640 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f753443400 0x55f752703600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 -- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f752663c00 msgr2=0x55f752703080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f752663c00 0x55f752703080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 -- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f753443400 msgr2=0x55f752703600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f753443400 0x55f752703600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 -- 172.21.15.5:0/1482907395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f75268f680 con 0x55f752662c00 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.964+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 0x55f752702b00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x55f753477a40 tx=0x55f7533c4600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c6569e51a935f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3dd9a00640 1 -- 172.21.15.5:0/1482907395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f752688680 con 0x55f752662c00 2024-04-25T10:09:57.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3dd9a00640 1 -- 172.21.15.5:0/1482907395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f7526ac540 con 0x55f752662c00 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3dd9a00640 1 -- 172.21.15.5:0/1482907395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f752688b60 con 0x55f752662c00 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f75268f860 con 0x55f752662c00 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f75268f2c0 con 0x55f752662c00 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3ddf622740 4 waiting for MDS map... 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3dd9a00640 1 -- 172.21.15.5:0/1482907395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x55f753532600 con 0x55f752662c00 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3dd9a00640 1 -- 172.21.15.5:0/1482907395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 69) v1 ==== 2723+0+0 (secure 0 0 0) 0x55f7526bb180 con 0x55f752662c00 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3ddf622740 4 Got MDS map 69 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3ddf622740 4 datascan.main: Using metadata pool output 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.965+0000 7f3ddf622740 4 datascan.main: connecting to RADOS... 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.966+0000 7f3ddf622740 1 Processor -- start 2024-04-25T10:09:57.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.966+0000 7f3ddf622740 1 -- start start 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f75358c000 0x55f752704680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddf622740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f753524480 con 0x55f753442800 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddf622740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f753524600 con 0x55f752663800 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddf622740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f753524780 con 0x55f75358c000 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45174/0 (socket says 172.21.15.5:45174) 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.967+0000 7f3ddd600640 1 -- 172.21.15.5:0/394974881 learned_addr learned my addr 172.21.15.5:0/394974881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddc200640 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f75358c000 0x55f752704680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddd600640 1 -- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f75358c000 msgr2=0x55f752704680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddd600640 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f75358c000 0x55f752704680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddd600640 1 -- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f752663800 msgr2=0x55f752704100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddd600640 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddd600640 1 -- 172.21.15.5:0/394974881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f75268f4a0 con 0x55f753442800 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3ddd600640 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 0x55f752703b80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x55f7534327e0 tx=0x55f753544c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94a57257a1922c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.968+0000 7f3dd7200640 1 -- 172.21.15.5:0/394974881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f752688680 con 0x55f753442800 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3dd7200640 1 -- 172.21.15.5:0/394974881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f7526ace00 con 0x55f753442800 2024-04-25T10:09:57.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3dd7200640 1 -- 172.21.15.5:0/394974881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f752688b60 con 0x55f753442800 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 -- 172.21.15.5:0/394974881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 msgr2=0x55f752703b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 0x55f752703b80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x55f7534327e0 tx=0x55f753544c00 comp rx=0 tx=0).stop 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 -- 172.21.15.5:0/394974881 shutdown_connections 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f75358c000 0x55f752704680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 --2- 172.21.15.5:0/394974881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 -- 172.21.15.5:0/394974881 >> 172.21.15.5:0/394974881 conn(0x55f753443000 msgr2=0x55f752751800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 -- 172.21.15.5:0/394974881 shutdown_connections 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.969+0000 7f3ddf622740 1 -- 172.21.15.5:0/394974881 wait complete. 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.970+0000 7f3ddf622740 1 Processor -- start 2024-04-25T10:09:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.970+0000 7f3ddf622740 1 -- start start 2024-04-25T10:09:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddf622740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f75358d000 0x55f752704680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddf622740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f753525e00 con 0x55f752663800 2024-04-25T10:09:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddf622740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f753525c80 con 0x55f75358d000 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddf622740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f753525b00 con 0x55f753442800 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45186/0 (socket says 172.21.15.5:45186) 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.971+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 learned_addr learned my addr 172.21.15.5:0/2217024971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f75358d000 0x55f752704680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f753442800 msgr2=0x55f752703b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f75358d000 msgr2=0x55f752704680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f75358d000 0x55f752704680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f75268f680 con 0x55f752663800 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f75358d000 0x55f752704680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f753442800 0x55f752703b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 0x55f752704100 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x55f753433aa0 tx=0x55f7533c4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3dd5400640 1 -- 172.21.15.5:0/2217024971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f7526884e0 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3dd5400640 1 -- 172.21.15.5:0/2217024971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f7526acc40 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.972+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f75268f860 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.973+0000 7f3dd5400640 1 -- 172.21.15.5:0/2217024971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f752688680 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.973+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f75268f0e0 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.973+0000 7f3ddf622740 4 datascan.init: resolving metadata pool 13 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.974+0000 7f3dd5400640 1 -- 172.21.15.5:0/2217024971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55f7535a4c00 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.975+0000 7f3dd5400640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55f7529ce000 0x55f752705700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.975+0000 7f3dd5400640 1 -- 172.21.15.5:0/2217024971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x55f7535a2a00 con 0x55f752663800 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.975+0000 7f3ddf622740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.975+0000 7f3ddf622740 4 datascan.main: resolving metadata pool 13 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.975+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55f75358cc00 0x55f752704c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.975+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c800 con 0x55f75358cc00 2024-04-25T10:09:57.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.976+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55f7529ce000 0x55f752705700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.976+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55f75358cc00 0x55f752704c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.976+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55f75358cc00 0x55f752704c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.977+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55f7529ce000 0x55f752705700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55f753477d10 tx=0x55f7533c4060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.977+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(1 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.977+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55f75358c800 0x55f752702580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.977+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cf000 con 0x55f75358c800 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.978+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55f75358c800 0x55f752702580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.979+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55f75358c800 0x55f752702580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.980+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(2 [pgnls start_epoch 93 out=177b] v93'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:57.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.980+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f7529cf000 0x55f7529e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.980+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8000 con 0x55f7529cf000 2024-04-25T10:09:57.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.981+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f7529cf000 0x55f7529e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.982+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f7529cf000 0x55f7529e4000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.982+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(3 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf000 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.982+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8000 con 0x55f75358cc00 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.983+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(4 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.983+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f7529e8000 0x55f7529e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.983+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cf400 con 0x55f7529e8000 2024-04-25T10:09:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.983+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f7529e8000 0x55f7529e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.984+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f7529e8000 0x55f7529e4580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.984+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(5 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.984+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55f7529cf400 0x55f7529e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.984+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cf800 con 0x55f7529cf400 2024-04-25T10:09:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.985+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55f7529cf400 0x55f7529e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.986+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55f7529cf400 0x55f7529e4b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.987+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(6 [pgnls start_epoch 93 out=149b] v92'26 uv26 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:57.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.987+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cf800 con 0x55f7529cf400 2024-04-25T10:09:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.988+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(7 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.988+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55f7529cf800 0x55f7529e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.988+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8400 con 0x55f7529cf800 2024-04-25T10:09:57.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.988+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55f7529cf800 0x55f7529e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.989+0000 7f3ddcc00640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55f7529cf800 0x55f7529e5600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.989+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(8 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.989+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8400 con 0x55f75358cc00 2024-04-25T10:09:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.990+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 3 ==== osd_op_reply(9 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:57.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.990+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8400 con 0x55f7529cf400 2024-04-25T10:09:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.991+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(10 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.991+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55f7529e8400 0x55f7529e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.991+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529e8400 2024-04-25T10:09:57.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.991+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55f7529e8400 0x55f7529e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:57.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.993+0000 7f3ddc200640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55f7529e8400 0x55f7529e5b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.993+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(11 [pgnls start_epoch 93 out=81b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8400 2024-04-25T10:09:58.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.994+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529cf000 2024-04-25T10:09:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.995+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(12 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf000 2024-04-25T10:09:58.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.995+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529e8000 2024-04-25T10:09:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.995+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(13 [pgnls start_epoch 93 out=77b] v92'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.995+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f75358c800 2024-04-25T10:09:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.996+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== osd_op_reply(14 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.996+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529cf800 2024-04-25T10:09:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.997+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== osd_op_reply(15 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.997+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529cf800 2024-04-25T10:09:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.997+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 3 ==== osd_op_reply(16 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.997+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f75358cc00 2024-04-25T10:09:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.998+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 4 ==== osd_op_reply(17 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.998+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529cf400 2024-04-25T10:09:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.999+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 4 ==== osd_op_reply(18 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:57.999+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f7529cf800 2024-04-25T10:09:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.000+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 4 ==== osd_op_reply(19 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.000+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8800 con 0x55f75358c800 2024-04-25T10:09:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.001+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 3 ==== osd_op_reply(20 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.001+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55f7529e8800 0x55f7529e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.001+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8800 2024-04-25T10:09:58.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.001+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55f7529e8800 0x55f7529e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.002+0000 7f3ddd600640 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55f7529e8800 0x55f7529e6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.003+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(21 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529e8800 2024-04-25T10:09:58.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.003+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8000 2024-04-25T10:09:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.003+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(22 [pgnls start_epoch 93 out=97b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.003+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529cf800 2024-04-25T10:09:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.004+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 5 ==== osd_op_reply(23 [pgnls start_epoch 93 out=74b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.004+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f75358cc00 2024-04-25T10:09:58.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.005+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 5 ==== osd_op_reply(24 [pgnls start_epoch 93 out=73b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.005+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529cf400 2024-04-25T10:09:58.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.006+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 5 ==== osd_op_reply(25 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.006+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f75358cc00 2024-04-25T10:09:58.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.007+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 6 ==== osd_op_reply(26 [pgnls start_epoch 93 out=98b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.007+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8000 2024-04-25T10:09:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.007+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 4 ==== osd_op_reply(27 [pgnls start_epoch 93 out=76b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.007+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8800 2024-04-25T10:09:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.008+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(28 [pgnls start_epoch 93 out=73b] v92'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529e8800 2024-04-25T10:09:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.008+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8000 2024-04-25T10:09:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.009+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 5 ==== osd_op_reply(29 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.009+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f75358cc00 2024-04-25T10:09:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.009+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 7 ==== osd_op_reply(30 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.009+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8800 2024-04-25T10:09:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.010+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(31 [pgnls start_epoch 93 out=73b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529e8800 2024-04-25T10:09:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.010+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f75358cc00 2024-04-25T10:09:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.010+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 8 ==== osd_op_reply(32 [pgnls start_epoch 93 out=101b] v92'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.011+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:09:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.011+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c400 con 0x55f75358c800 2024-04-25T10:09:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.012+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 4 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.012+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:09:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.012+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c000 con 0x55f75358c800 2024-04-25T10:09:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.013+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 5 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.013+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:09:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.013+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9c00 con 0x55f75358c800 2024-04-25T10:09:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.015+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 6 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.015+0000 7f3ddf622740 20 datascan.scan_links: injected ino detected 2024-04-25T10:09:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.015+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:09:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.015+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9800 con 0x55f75358c800 2024-04-25T10:09:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.016+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 7 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.016+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:09:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.016+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9400 con 0x55f75358c800 2024-04-25T10:09:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 8 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:09:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:09:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.018+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9000 con 0x55f7529cf400 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.019+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 6 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv24 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.019+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:09:58.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.019+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9c00 con 0x55f7529cf800 2024-04-25T10:09:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.020+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.020+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T10:09:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.020+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c400 con 0x55f7529e8400 2024-04-25T10:09:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.021+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8400 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.021+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.021+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.021+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.021+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c000 con 0x55f7529cf800 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:09:58.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06000 con 0x55f7529e8000 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 6 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.022+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:09:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06400 con 0x55f75358cc00 2024-04-25T10:09:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 9 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:09:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:09:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:09:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.023+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06800 con 0x55f75358cc00 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 10 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:09:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.024+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06c00 con 0x55f7529e8000 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddf622740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:09:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.025+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07000 con 0x55f75358cc00 2024-04-25T10:09:58.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.026+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 11 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.026+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.027+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 12 ==== osd_op_reply(47 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.027+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358c800 2024-04-25T10:09:58.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.029+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 9 ==== osd_op_reply(48 [pgnls start_epoch 93 out=177b] v93'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.029+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf000 2024-04-25T10:09:58.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.030+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(49 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf000 2024-04-25T10:09:58.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.030+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.030+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 13 ==== osd_op_reply(50 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.030+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8000 2024-04-25T10:09:58.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.031+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 8 ==== osd_op_reply(51 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.031+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf400 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.032+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 7 ==== osd_op_reply(52 [pgnls start_epoch 93 out=149b] v92'26 uv26 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.032+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf400 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.034+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 8 ==== osd_op_reply(53 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.034+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf800 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.035+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 8 ==== osd_op_reply(54 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.035+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.035+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 14 ==== osd_op_reply(55 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.035+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf400 2024-04-25T10:09:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.036+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 9 ==== osd_op_reply(56 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.037+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8400 2024-04-25T10:09:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.038+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.4 v2:172.21.15.80:6808/2045277337 3 ==== osd_op_reply(57 [pgnls start_epoch 93 out=81b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8400 2024-04-25T10:09:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.038+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf000 2024-04-25T10:09:58.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.040+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.5 v2:172.21.15.80:6800/161144277 4 ==== osd_op_reply(58 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf000 2024-04-25T10:09:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.040+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8000 2024-04-25T10:09:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.040+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 9 ==== osd_op_reply(59 [pgnls start_epoch 93 out=77b] v92'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.040+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358c800 2024-04-25T10:09:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.041+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 10 ==== osd_op_reply(60 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.041+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf800 2024-04-25T10:09:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.042+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 9 ==== osd_op_reply(61 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.042+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf800 2024-04-25T10:09:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.043+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 10 ==== osd_op_reply(62 [pgnls start_epoch 93 out=49b] v90'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.043+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.044+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 15 ==== osd_op_reply(63 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.044+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf400 2024-04-25T10:09:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.045+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 10 ==== osd_op_reply(64 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.045+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf800 2024-04-25T10:09:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.045+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 11 ==== osd_op_reply(65 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.046+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358c800 2024-04-25T10:09:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.046+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 11 ==== osd_op_reply(66 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.047+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8800 2024-04-25T10:09:58.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.047+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(67 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529e8800 2024-04-25T10:09:58.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.048+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8000 2024-04-25T10:09:58.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.049+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 10 ==== osd_op_reply(68 [pgnls start_epoch 93 out=97b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.049+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf800 2024-04-25T10:09:58.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.049+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 12 ==== osd_op_reply(69 [pgnls start_epoch 93 out=74b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.049+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.050+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 16 ==== osd_op_reply(70 [pgnls start_epoch 93 out=73b] v92'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.050+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529cf400 2024-04-25T10:09:58.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.051+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 11 ==== osd_op_reply(71 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.051+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.052+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 17 ==== osd_op_reply(72 [pgnls start_epoch 93 out=98b] v92'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.052+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8000 2024-04-25T10:09:58.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.053+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 11 ==== osd_op_reply(73 [pgnls start_epoch 93 out=76b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.053+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8800 2024-04-25T10:09:58.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.054+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.2 v2:172.21.15.5:6822/3928063993 5 ==== osd_op_reply(74 [pgnls start_epoch 93 out=73b] v92'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529e8800 2024-04-25T10:09:58.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.054+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8000 2024-04-25T10:09:58.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.055+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 12 ==== osd_op_reply(75 [pgnls start_epoch 93 out=73b] v92'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.055+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.056+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 18 ==== osd_op_reply(76 [pgnls start_epoch 93 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.056+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f7529e8800 2024-04-25T10:09:58.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.057+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.2 v2:172.21.15.5:6822/3928063993 6 ==== osd_op_reply(77 [pgnls start_epoch 93 out=73b] v88'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529e8800 2024-04-25T10:09:58.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.057+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 93 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07400 con 0x55f75358cc00 2024-04-25T10:09:58.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.058+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 19 ==== osd_op_reply(78 [pgnls start_epoch 93 out=101b] v92'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.058+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:09:58.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.059+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07800 con 0x55f75358c800 2024-04-25T10:09:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.060+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 12 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.060+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:09:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.060+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07c00 con 0x55f75358c800 2024-04-25T10:09:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.061+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 13 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.061+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:09:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.061+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06000 con 0x55f75358c800 2024-04-25T10:09:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.062+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 14 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.062+0000 7f3ddf622740 20 datascan.scan_links: injected ino detected 2024-04-25T10:09:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.062+0000 7f3ddf622740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T10:09:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.062+0000 7f3ddf622740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:09:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.062+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:09:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.062+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06400 con 0x55f75358c800 2024-04-25T10:09:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.063+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 15 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.063+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:09:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.063+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06800 con 0x55f75358c800 2024-04-25T10:09:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 16 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:09:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.065+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06c00 con 0x55f7529cf400 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.066+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.6 v2:172.21.15.80:6824/3989257232 12 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv24 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc500 con 0x55f7529cf400 2024-04-25T10:09:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.066+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:09:58.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.066+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c400 con 0x55f7529cf800 2024-04-25T10:09:58.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.067+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.067+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T10:09:58.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.067+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e8c00 con 0x55f7529e8400 2024-04-25T10:09:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.068+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.4 v2:172.21.15.80:6808/2045277337 4 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8400 2024-04-25T10:09:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.068+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:09:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.068+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:09:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.068+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:09:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.068+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cfc00 con 0x55f7529cf800 2024-04-25T10:09:58.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.069+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 14 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.069+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:09:58.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.069+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07800 con 0x55f7529e8000 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 13 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:09:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.070+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a07c00 con 0x55f75358cc00 2024-04-25T10:09:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.071+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 20 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.071+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:09:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.071+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:09:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.071+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:09:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.071+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06000 con 0x55f75358cc00 2024-04-25T10:09:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 21 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:09:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:09:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T10:09:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:09:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:09:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.073+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f752a06400 con 0x55f7529e8000 2024-04-25T10:09:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.3 v2:172.21.15.5:6808/3976284996 14 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f7533dc000 con 0x55f7529e8000 2024-04-25T10:09:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:09:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:09:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:09:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddf622740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:09:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddf622740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:09:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.074+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9c00 con 0x55f75358cc00 2024-04-25T10:09:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.075+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 22 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1163 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.075+0000 7f3ddf622740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T10:09:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.075+0000 7f3ddf622740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:09:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.075+0000 7f3ddf622740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T10:09:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.075+0000 7f3ddf622740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T10:09:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.075+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9800 con 0x55f75358c800 2024-04-25T10:09:58.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.076+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 17 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.076+0000 7f3ddf622740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:09:58.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.076+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9400 con 0x55f75358c800 2024-04-25T10:09:58.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.080+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 18 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v93'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.080+0000 7f3ddf622740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:09:58.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.080+0000 7f3ddf622740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:09:58.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.080+0000 7f3ddf622740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:09:58.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.080+0000 7f3ddf622740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T10:09:58.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.080+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529e9000 con 0x55f75358c800 2024-04-25T10:09:58.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.081+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 19 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.081+0000 7f3ddf622740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:09:58.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.081+0000 7f3ddf622740 20 datascan.scan_links: first is now 2 2024-04-25T10:09:58.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.081+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cec00 con 0x55f75358c800 2024-04-25T10:09:58.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.085+0000 7f3ddd600640 1 -- 172.21.15.5:0/2217024971 <== osd.7 v2:172.21.15.80:6816/4166609972 20 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v93'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f7533dc500 con 0x55f75358c800 2024-04-25T10:09:58.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.086+0000 7f3ddf622740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:09:58.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.086+0000 7f3ddf622740 10 datascan.scan_links: updating inotable 2024-04-25T10:09:58.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.086+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f7529cfc00 con 0x55f75358cc00 2024-04-25T10:09:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.087+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 23 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.087+0000 7f3ddf622740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T10:09:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.087+0000 7f3ddf622740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T10:09:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.087+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c000 con 0x55f75358cc00 2024-04-25T10:09:58.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.091+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 <== osd.1 v2:172.21.15.5:6800/692601986 24 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v93'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f7533dc000 con 0x55f75358cc00 2024-04-25T10:09:58.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.091+0000 7f3ddf622740 10 datascan.scan_links: updating snaptable 2024-04-25T10:09:58.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.091+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v9 -- 0x55f75358c400 con 0x55f7529cf800 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.092+0000 7f3ddcc00640 1 -- 172.21.15.5:0/2217024971 <== osd.0 v2:172.21.15.5:6809/1760674718 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55f7533dc780 con 0x55f7529cf800 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55f7529cf800 msgr2=0x55f7529e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55f7529cf800 0x55f7529e5600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55f75358cc00 msgr2=0x55f752704c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55f75358cc00 0x55f752704c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55f7529e8800 msgr2=0x55f7529e6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55f7529e8800 0x55f7529e6100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f7529e8000 msgr2=0x55f7529e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55f7529e8000 0x55f7529e4580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55f7529e8400 msgr2=0x55f7529e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55f7529e8400 0x55f7529e5b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f7529cf000 msgr2=0x55f7529e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f7529cf000 0x55f7529e4000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55f7529cf400 msgr2=0x55f7529e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55f7529cf400 0x55f7529e4b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55f75358c800 msgr2=0x55f752702580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55f75358c800 0x55f752702580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 reap_dead start 2024-04-25T10:09:58.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 reap_dead start 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 reap_dead start 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55f7529ce000 msgr2=0x55f752705700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddc200640 1 -- 172.21.15.5:0/2217024971 reap_dead start 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55f7529ce000 0x55f752705700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55f753477d10 tx=0x55f7533c4060 comp rx=0 tx=0).stop 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.093+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 msgr2=0x55f752704100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 0x55f752704100 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x55f753433aa0 tx=0x55f7533c4120 comp rx=0 tx=0).stop 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 shutdown_connections 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55f7529ce000 0x55f752705700 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 --2- 172.21.15.5:0/2217024971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752663800 0x55f752704100 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 >> 172.21.15.5:0/2217024971 conn(0x55f753443000 msgr2=0x55f752751800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 shutdown_connections 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.094+0000 7f3ddf622740 1 -- 172.21.15.5:0/2217024971 wait complete. 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.095+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 msgr2=0x55f752702b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.095+0000 7f3ddf622740 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 0x55f752702b00 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x55f753477a40 tx=0x55f7533c4600 comp rx=0 tx=0).stop 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 shutdown_connections 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f753443400 0x55f752703600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f752663c00 0x55f752703080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 --2- 172.21.15.5:0/1482907395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f752662c00 0x55f752702b00 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 >> 172.21.15.5:0/1482907395 conn(0x55f752663000 msgr2=0x55f752751000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 shutdown_connections 2024-04-25T10:09:58.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.096+0000 7f3ddf622740 1 -- 172.21.15.5:0/1482907395 wait complete. 2024-04-25T10:09:58.116 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.259978, result: 2024-04-25T10:09:58.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:09:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.326+0000 7f76c5200640 1 Processor -- start 2024-04-25T10:09:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.326+0000 7f76c5200640 1 -- start start 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76c5200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76c0144fe0 0x7f76c006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76c5200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c0071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76c5200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c01453c0 con 0x7f76c0071670 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76c5200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c00a8560 con 0x7f76c0144fe0 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76c5200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c006d6a0 con 0x7f76c006d130 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76bde00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76c0144fe0 0x7f76c006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76be800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f76c0071670 0x7f76c00cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44788/0 (socket says 172.21.15.5:44788) 2024-04-25T10:09:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.328+0000 7f76be800640 1 -- 172.21.15.5:0/2987467908 learned_addr learned my addr 172.21.15.5:0/2987467908 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.329+0000 7f76bd400640 1 --2- 172.21.15.5:0/2987467908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.329+0000 7f76bde00640 1 -- 172.21.15.5:0/2987467908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 msgr2=0x7f76c0071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.329+0000 7f76bde00640 1 --2- 172.21.15.5:0/2987467908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c0071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.329+0000 7f76bde00640 1 -- 172.21.15.5:0/2987467908 >> v1:172.21.15.5:6789/0 conn(0x7f76c0071670 legacy=0x7f76c00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.329+0000 7f76bde00640 1 -- 172.21.15.5:0/2987467908 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c006d980 con 0x7f76c0144fe0 2024-04-25T10:09:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.330+0000 7f76bde00640 1 --2- 172.21.15.5:0/2987467908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0144fe0 0x7f76c006e560 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009d90 tx=0x7f76b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68396f7784a61232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.330+0000 7f76bca00640 1 -- 172.21.15.5:0/2987467908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0007740 con 0x7f76c0144fe0 2024-04-25T10:09:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.330+0000 7f76bca00640 1 -- 172.21.15.5:0/2987467908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00078e0 con 0x7f76c0144fe0 2024-04-25T10:09:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.330+0000 7f76bca00640 1 -- 172.21.15.5:0/2987467908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0007cb0 con 0x7f76c0144fe0 2024-04-25T10:09:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 -- 172.21.15.5:0/2987467908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0144fe0 msgr2=0x7f76c006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 --2- 172.21.15.5:0/2987467908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0144fe0 0x7f76c006e560 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009d90 tx=0x7f76b002a660 comp rx=0 tx=0).stop 2024-04-25T10:09:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 -- 172.21.15.5:0/2987467908 shutdown_connections 2024-04-25T10:09:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 --2- 172.21.15.5:0/2987467908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c0071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 --2- 172.21.15.5:0/2987467908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0144fe0 0x7f76c006e560 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 -- 172.21.15.5:0/2987467908 >> 172.21.15.5:0/2987467908 conn(0x7f76c006ab60 msgr2=0x7f76c006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.331+0000 7f76c5200640 1 -- 172.21.15.5:0/2987467908 shutdown_connections 2024-04-25T10:09:58.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.332+0000 7f76c5200640 1 -- 172.21.15.5:0/2987467908 wait complete. 2024-04-25T10:09:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.333+0000 7f76c5200640 1 Processor -- start 2024-04-25T10:09:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.334+0000 7f76c5200640 1 -- start start 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76c5200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c01f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76c5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 0x7f76c01f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76c5200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76c0144fe0 0x7f76c00c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76c5200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c015be50 con 0x7f76c0071670 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76c5200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c006df40 con 0x7f76c0144fe0 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76c5200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c006eaa0 con 0x7f76c006d130 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 0x7f76c01f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 0x7f76c01f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45204/0 (socket says 172.21.15.5:45204) 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 -- 172.21.15.5:0/3421343836 learned_addr learned my addr 172.21.15.5:0/3421343836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bde00640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c01f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76be800640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76c0144fe0 0x7f76c00c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 -- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 msgr2=0x7f76c01f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c01f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 -- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76c0144fe0 msgr2=0x7f76c00c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76c0144fe0 0x7f76c00c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.335+0000 7f76bd400640 1 -- 172.21.15.5:0/3421343836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00abee0 con 0x7f76c0071670 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.336+0000 7f76bd400640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 0x7f76c01f4140 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f76a800d800 tx=0x7f76a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.336+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800b6c0 con 0x7f76c0071670 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.336+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a8019070 con 0x7f76c0071670 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.336+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c00ac1c0 con 0x7f76c0071670 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.336+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800ba80 con 0x7f76c0071670 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.336+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c00ac780 con 0x7f76c0071670 2024-04-25T10:09:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.338+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c0068f60 con 0x7f76c0071670 2024-04-25T10:09:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.338+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76a800bc20 con 0x7f76c0071670 2024-04-25T10:09:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.339+0000 7f76b6000640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f769c06b4d0 0x7f769c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.339+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f76a808aa00 con 0x7f76c0071670 2024-04-25T10:09:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.340+0000 7f76bde00640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f769c06b4d0 0x7f769c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.341+0000 7f76bde00640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f769c06b4d0 0x7f769c06d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76b0034000 tx=0x7f76b0033040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.343+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76a8059530 con 0x7f76c0071670 2024-04-25T10:09:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:58.582+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f76c00ae660 con 0x7f76c0071670 2024-04-25T10:09:59.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.339+0000 7f76b6000640 1 -- 172.21.15.5:0/3421343836 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) v1 ==== 162+0+0 (secure 0 0 0) 0x7f76a8058e30 con 0x7f76c0071670 2024-04-25T10:09:59.346 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:09:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f769c06b4d0 msgr2=0x7f769c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f769c06b4d0 0x7f769c06d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76b0034000 tx=0x7f76b0033040 comp rx=0 tx=0).stop 2024-04-25T10:09:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 msgr2=0x7f76c01f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 0x7f76c01f4140 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f76a800d800 tx=0x7f76a800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 shutdown_connections 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f769c06b4d0 0x7f769c06d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76c0144fe0 0x7f76c00c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76c0071670 0x7f76c01f4140 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 --2- 172.21.15.5:0/3421343836 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76c006d130 0x7f76c01f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.342+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 >> 172.21.15.5:0/3421343836 conn(0x7f76c006ab60 msgr2=0x7f76c00b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.343+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 shutdown_connections 2024-04-25T10:09:59.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.343+0000 7f76c5200640 1 -- 172.21.15.5:0/3421343836 wait complete. 2024-04-25T10:09:59.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.564+0000 7fbe36200640 1 Processor -- start 2024-04-25T10:09:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.564+0000 7fbe36200640 1 -- start start 2024-04-25T10:09:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.566+0000 7fbe36200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 0x7fbe300ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.566+0000 7fbe36200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300c9f90 con 0x7fbe300cbb50 2024-04-25T10:09:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.566+0000 7fbe36200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300ca270 con 0x7fbe301472c0 2024-04-25T10:09:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.566+0000 7fbe36200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300ca550 con 0x7fbe300cad50 2024-04-25T10:09:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.566+0000 7fbe2ea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbe300cbb50 0x7fbe300ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44792/0 (socket says 172.21.15.5:44792) 2024-04-25T10:09:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.566+0000 7fbe2ea00640 1 -- 172.21.15.5:0/3490896963 learned_addr learned my addr 172.21.15.5:0/3490896963 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:09:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 0x7fbe300ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838499193 0 0) 0x7fbe300c9f90 con 0x7fbe300cbb50 2024-04-25T10:09:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe14003520 con 0x7fbe300cbb50 2024-04-25T10:09:59.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235452841 0 0) 0x7fbe14003520 con 0x7fbe300cbb50 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 msgr2=0x7fbe300ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 0x7fbe300ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 >> v1:172.21.15.80:6789/0 conn(0x7fbe301472c0 legacy=0x7fbe300cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300ca830 con 0x7fbe300cbb50 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fbe1c001690 con 0x7fbe300cbb50 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 >> v1:172.21.15.5:6789/0 conn(0x7fbe300cbb50 legacy=0x7fbe300ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 0x7fbe140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14003fb0 0x7fbe14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.567+0000 7fbe2e000640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe14004d40 0x7fbe140051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300ca550 con 0x7fbe14004d40 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1400bc50 con 0x7fbe14003fb0 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1400be50 con 0x7fbe300cad50 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe14004d40 0x7fbe140051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2ea00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14003fb0 0x7fbe14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2f400640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 0x7fbe140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2fe00640 1 -- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 msgr2=0x7fbe140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe300cad50 0x7fbe140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2fe00640 1 -- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14003fb0 msgr2=0x7fbe14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14003fb0 0x7fbe14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2fe00640 1 -- 172.21.15.5:0/3490896963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14003520 con 0x7fbe14004d40 2024-04-25T10:09:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.568+0000 7fbe2ea00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14003fb0 0x7fbe14004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.570+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe14004d40 0x7fbe140051a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fc0 tx=0x7fbe14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbde998cf6eefbc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.570+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28009c10 con 0x7fbe14004d40 2024-04-25T10:09:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.570+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28036070 con 0x7fbe14004d40 2024-04-25T10:09:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.570+0000 7fbe2e000640 1 -- 172.21.15.5:0/3490896963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28031880 con 0x7fbe14004d40 2024-04-25T10:09:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.570+0000 7fbe36200640 1 -- 172.21.15.5:0/3490896963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe14004d40 msgr2=0x7fbe140051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.570+0000 7fbe36200640 1 --2- 172.21.15.5:0/3490896963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe14004d40 0x7fbe140051a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fc0 tx=0x7fbe14005830 comp rx=0 tx=0).stop 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.571+0000 7fbe2fe00640 1 -- 172.21.15.5:0/3490896963 reap_dead start 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.571+0000 7fbe36200640 1 -- 172.21.15.5:0/3490896963 shutdown_connections 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.571+0000 7fbe36200640 1 -- 172.21.15.5:0/3490896963 >> 172.21.15.5:0/3490896963 conn(0x7fbe30071150 msgr2=0x7fbe30071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.571+0000 7fbe36200640 1 -- 172.21.15.5:0/3490896963 shutdown_connections 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.571+0000 7fbe36200640 1 -- 172.21.15.5:0/3490896963 wait complete. 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.573+0000 7fbe36200640 1 Processor -- start 2024-04-25T10:09:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.573+0000 7fbe36200640 1 -- start start 2024-04-25T10:09:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe36200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14004d40 0x7fbe301f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe36200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe301370d0 0x7fbe301f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe36200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 0x7fbe30145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe36200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300cef00 con 0x7fbe30145700 2024-04-25T10:09:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe36200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300ced80 con 0x7fbe14004d40 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe36200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe300cf080 con 0x7fbe301370d0 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 0x7fbe30145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 0x7fbe30145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45220/0 (socket says 172.21.15.5:45220) 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 -- 172.21.15.5:0/832349276 learned_addr learned my addr 172.21.15.5:0/832349276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2f400640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14004d40 0x7fbe301f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2ea00640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe301370d0 0x7fbe301f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 -- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe301370d0 msgr2=0x7fbe301f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe301370d0 0x7fbe301f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 -- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14004d40 msgr2=0x7fbe301f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14004d40 0x7fbe301f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2fe00640 1 -- 172.21.15.5:0/832349276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe301f4fa0 con 0x7fbe30145700 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.574+0000 7fbe2ea00640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe301370d0 0x7fbe301f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.575+0000 7fbe2fe00640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 0x7fbe30145b80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e230 tx=0x7fbe28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.575+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28009c10 con 0x7fbe30145700 2024-04-25T10:09:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.575+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28031dd0 con 0x7fbe30145700 2024-04-25T10:09:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.575+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe30134a10 con 0x7fbe30145700 2024-04-25T10:09:59.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.575+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28003bd0 con 0x7fbe30145700 2024-04-25T10:09:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.575+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30134fd0 con 0x7fbe30145700 2024-04-25T10:09:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.576+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30068fd0 con 0x7fbe30145700 2024-04-25T10:09:59.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.578+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe28005d10 con 0x7fbe30145700 2024-04-25T10:09:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.578+0000 7fbe27400640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe1006b4d0 0x7fbe1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.579+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fbe280abcd0 con 0x7fbe30145700 2024-04-25T10:09:59.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.579+0000 7fbe2f400640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe1006b4d0 0x7fbe1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:59.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.580+0000 7fbe2f400640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe1006b4d0 0x7fbe1006d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000c00 tx=0x7fbe20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.583+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe2804a060 con 0x7fbe30145700 2024-04-25T10:09:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.829+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe30072390 con 0x7fbe30145700 2024-04-25T10:09:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.830+0000 7fbe27400640 1 -- 172.21.15.5:0/832349276 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fbe301f4fa0 con 0x7fbe30145700 2024-04-25T10:09:59.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:09:59.836 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:09:59.344692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 72 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe1006b4d0 msgr2=0x7fbe1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe1006b4d0 0x7fbe1006d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe20000c00 tx=0x7fbe20009040 comp rx=0 tx=0).stop 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 msgr2=0x7fbe30145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 0x7fbe30145b80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e230 tx=0x7fbe28009750 comp rx=0 tx=0).stop 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 shutdown_connections 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe30145700 0x7fbe30145b80 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbe301370d0 0x7fbe301f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbe1006b4d0 0x7fbe1006d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 --2- 172.21.15.5:0/832349276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe14004d40 0x7fbe301f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 >> 172.21.15.5:0/832349276 conn(0x7fbe30071150 msgr2=0x7fbe300b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 shutdown_connections 2024-04-25T10:09:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:09:59.832+0000 7fbe36200640 1 -- 172.21.15.5:0/832349276 wait complete. 2024-04-25T10:10:00.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:09:59.344692+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6834/841689019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 841689019}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 841689019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:00.261 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7959:up:replay 2024-04-25T10:10:01.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.482+0000 7f4a96000640 1 Processor -- start 2024-04-25T10:10:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.482+0000 7f4a96000640 1 -- start start 2024-04-25T10:10:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.483+0000 7f4a96000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4a900d04a0 0x7f4a900d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.483+0000 7f4a96000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a900b53c0 con 0x7f4a900ce530 2024-04-25T10:10:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.483+0000 7f4a96000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a900b56a0 con 0x7f4a9015db80 2024-04-25T10:10:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.483+0000 7f4a96000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a900b5170 con 0x7f4a900d04a0 2024-04-25T10:10:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.484+0000 7f4a95600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4a900ce530 0x7f4a90161740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44802/0 (socket says 172.21.15.5:44802) 2024-04-25T10:10:01.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.484+0000 7f4a95600640 1 -- 172.21.15.5:0/2916459433 learned_addr learned my addr 172.21.15.5:0/2916459433 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.484+0000 7f4a8fe00640 1 --2- 172.21.15.5:0/2916459433 >> v2:172.21.15.80:3300/0 conn(0x7f4a900d04a0 0x7f4a900d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.484+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104468831 0 0) 0x7f4a900b53c0 con 0x7f4a900ce530 2024-04-25T10:10:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.484+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a74003520 con 0x7f4a900ce530 2024-04-25T10:10:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898670790 0 0) 0x7f4a74003520 con 0x7f4a900ce530 2024-04-25T10:10:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 >> v2:172.21.15.80:3300/0 conn(0x7f4a900d04a0 msgr2=0x7f4a900d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 --2- 172.21.15.5:0/2916459433 >> v2:172.21.15.80:3300/0 conn(0x7f4a900d04a0 0x7f4a900d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 >> v1:172.21.15.80:6789/0 conn(0x7f4a9015db80 legacy=0x7f4a900b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a900b5980 con 0x7f4a900ce530 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4a88001670 con 0x7f4a900ce530 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 >> v1:172.21.15.5:6789/0 conn(0x7f4a900ce530 legacy=0x7f4a90161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a74004800 0x7f4a74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900b5170 con 0x7f4a740051c0 2024-04-25T10:10:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7400c060 con 0x7f4a740039b0 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7400c260 con 0x7f4a74004800 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a95600640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a94c00640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a8fe00640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a74004800 0x7f4a74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a95600640 1 -- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a74004800 msgr2=0x7f4a74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.485+0000 7f4a95600640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a74004800 0x7f4a74004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.486+0000 7f4a95600640 1 -- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a740039b0 msgr2=0x7f4a74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.486+0000 7f4a95600640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a740039b0 0x7f4a74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.486+0000 7f4a95600640 1 -- 172.21.15.5:0/2916459433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74003520 con 0x7f4a740051c0 2024-04-25T10:10:01.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.486+0000 7f4a95600640 1 -- 172.21.15.5:0/2916459433 reap_dead start 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.487+0000 7f4a95600640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c210 tx=0x7f4a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba0304b15060d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.487+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88007d30 con 0x7f4a740051c0 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.487+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a88005ad0 con 0x7f4a740051c0 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.487+0000 7f4a8f400640 1 -- 172.21.15.5:0/2916459433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a88007070 con 0x7f4a740051c0 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 -- 172.21.15.5:0/2916459433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 msgr2=0x7f4a7400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c210 tx=0x7f4a74005830 comp rx=0 tx=0).stop 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 -- 172.21.15.5:0/2916459433 shutdown_connections 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 --2- 172.21.15.5:0/2916459433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a7400b9c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 -- 172.21.15.5:0/2916459433 >> 172.21.15.5:0/2916459433 conn(0x7f4a9006c1c0 msgr2=0x7f4a9006c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 -- 172.21.15.5:0/2916459433 shutdown_connections 2024-04-25T10:10:01.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.488+0000 7f4a96000640 1 -- 172.21.15.5:0/2916459433 wait complete. 2024-04-25T10:10:01.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.490+0000 7f4a96000640 1 Processor -- start 2024-04-25T10:10:01.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.490+0000 7f4a96000640 1 -- start start 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a96000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a900c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a96000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a9014f070 0x7f4a9015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a96000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a90160320 0x7f4a901607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a96000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900d0920 con 0x7f4a740051c0 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a96000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900b5f40 con 0x7f4a9014f070 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a96000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a900d0aa0 con 0x7f4a90160320 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a900c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a900c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45236/0 (socket says 172.21.15.5:45236) 2024-04-25T10:10:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 -- 172.21.15.5:0/4262015662 learned_addr learned my addr 172.21.15.5:0/4262015662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 -- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a90160320 msgr2=0x7f4a901607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a90160320 0x7f4a901607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 -- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a9014f070 msgr2=0x7f4a9015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a9014f070 0x7f4a9015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.491+0000 7f4a94c00640 1 -- 172.21.15.5:0/4262015662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9015ee90 con 0x7f4a740051c0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.492+0000 7f4a94c00640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a900c7220 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000e850 tx=0x7f4a8000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.492+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80003b80 con 0x7f4a740051c0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.492+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a80018070 con 0x7f4a740051c0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.492+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800050d0 con 0x7f4a740051c0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.492+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a900c78e0 con 0x7f4a740051c0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.492+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a900c7ea0 con 0x7f4a740051c0 2024-04-25T10:10:01.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.493+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a90068f60 con 0x7f4a740051c0 2024-04-25T10:10:01.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.494+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a80005270 con 0x7f4a740051c0 2024-04-25T10:10:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.495+0000 7f4a8cc00640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a6c06b500 0x7f4a6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.495+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f4a8008a480 con 0x7f4a740051c0 2024-04-25T10:10:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.496+0000 7f4a8fe00640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a6c06b500 0x7f4a6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:01.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.497+0000 7f4a8fe00640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a6c06b500 0x7f4a6c06d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009950 tx=0x7f4a78008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:01.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.500+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a80058f30 con 0x7f4a740051c0 2024-04-25T10:10:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.739+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a900d1400 con 0x7f4a740051c0 2024-04-25T10:10:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.740+0000 7f4a8cc00640 1 -- 172.21.15.5:0/4262015662 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f4a8001be40 con 0x7f4a740051c0 2024-04-25T10:10:01.747 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:01.747 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:00.357513+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 74 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a6c06b500 msgr2=0x7f4a6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a6c06b500 0x7f4a6c06d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009950 tx=0x7f4a78008040 comp rx=0 tx=0).stop 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 msgr2=0x7f4a900c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a900c7220 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000e850 tx=0x7f4a8000ed10 comp rx=0 tx=0).stop 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 shutdown_connections 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4a90160320 0x7f4a901607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4a6c06b500 0x7f4a6c06d9c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a9014f070 0x7f4a9015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 --2- 172.21.15.5:0/4262015662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a740051c0 0x7f4a900c7220 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 >> 172.21.15.5:0/4262015662 conn(0x7f4a9006c1c0 msgr2=0x7f4a900d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 shutdown_connections 2024-04-25T10:10:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:01.743+0000 7f4a96000640 1 -- 172.21.15.5:0/4262015662 wait complete. 2024-04-25T10:10:02.171 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:10:00.357513+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6834/841689019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 841689019}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 841689019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:02.171 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7959:up:rejoin 2024-04-25T10:10:03.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:03.341 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-04-25T10:10:03.334+0000 7f6eac400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.392+0000 7f8a02a00640 1 Processor -- start 2024-04-25T10:10:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.392+0000 7f8a02a00640 1 -- start start 2024-04-25T10:10:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.393+0000 7f8a02a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89fc154a10 0x7f89fc15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.393+0000 7f8a02a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0cd200 con 0x7f89fc0cdca0 2024-04-25T10:10:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.393+0000 7f8a02a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc06e6f0 con 0x7f89fc070f90 2024-04-25T10:10:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.393+0000 7f8a02a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc06e9d0 con 0x7f89fc154a10 2024-04-25T10:10:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.393+0000 7f8a00c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f89fc0cdca0 0x7f89fc0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44812/0 (socket says 172.21.15.5:44812) 2024-04-25T10:10:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.393+0000 7f8a00c00640 1 -- 172.21.15.5:0/3024080999 learned_addr learned my addr 172.21.15.5:0/3024080999 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f8a01600640 1 --2- 172.21.15.5:0/3024080999 >> v2:172.21.15.80:3300/0 conn(0x7f89fc154a10 0x7f89fc15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106984806 0 0) 0x7f89fc0cd200 con 0x7f89fc0cdca0 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e4003520 con 0x7f89fc0cdca0 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 57127616 0 0) 0x7f89e4003520 con 0x7f89fc0cdca0 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 >> v2:172.21.15.80:3300/0 conn(0x7f89fc154a10 msgr2=0x7f89fc15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 --2- 172.21.15.5:0/3024080999 >> v2:172.21.15.80:3300/0 conn(0x7f89fc154a10 0x7f89fc15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 >> v1:172.21.15.80:6789/0 conn(0x7f89fc070f90 legacy=0x7f89fc06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc06ecb0 con 0x7f89fc0cdca0 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f89e8001690 con 0x7f89fc0cdca0 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 >> v1:172.21.15.5:6789/0 conn(0x7f89fc0cdca0 legacy=0x7f89fc0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.394+0000 7f89fbe00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f89fbe00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f89fbe00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4005210 0x7f89e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc06e9d0 con 0x7f89e4003a00 2024-04-25T10:10:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c160 con 0x7f89e4005210 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c360 con 0x7f89e4004800 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a01600640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a00c00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a02000640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4005210 0x7f89e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a01600640 1 -- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4004800 msgr2=0x7f89e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a01600640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a01600640 1 -- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4005210 msgr2=0x7f89e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a01600640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4005210 0x7f89e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a01600640 1 -- 172.21.15.5:0/3024080999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4003520 con 0x7f89e4003a00 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a02000640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89e4005210 0x7f89e400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a02000640 1 -- 172.21.15.5:0/3024080999 reap_dead start 2024-04-25T10:10:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.395+0000 7f8a00c00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4004800 0x7f89e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.396+0000 7f8a01600640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009a90 tx=0x7f89e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d53e428e8014e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0036070 con 0x7f89e4003a00 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f0031400 con 0x7f89e4003a00 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f89fbe00640 1 -- 172.21.15.5:0/3024080999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0009690 con 0x7f89e4003a00 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f8a02a00640 1 -- 172.21.15.5:0/3024080999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e4003a00 msgr2=0x7f89e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f8a02a00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009a90 tx=0x7f89e4005830 comp rx=0 tx=0).stop 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f8a02a00640 1 -- 172.21.15.5:0/3024080999 shutdown_connections 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f8a02a00640 1 --2- 172.21.15.5:0/3024080999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.397+0000 7f8a02a00640 1 -- 172.21.15.5:0/3024080999 >> 172.21.15.5:0/3024080999 conn(0x7f89fc066f20 msgr2=0x7f89fc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.398+0000 7f8a02a00640 1 -- 172.21.15.5:0/3024080999 shutdown_connections 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.398+0000 7f8a02a00640 1 -- 172.21.15.5:0/3024080999 wait complete. 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 Processor -- start 2024-04-25T10:10:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 -- start start 2024-04-25T10:10:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4003a00 0x7f89fc15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89fc15b9d0 0x7f89fc142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 0x7f89fc158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc15e8c0 con 0x7f89fc15b0c0 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0cd780 con 0x7f89fc15b9d0 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.400+0000 7f8a02a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc15ea40 con 0x7f89e4003a00 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 0x7f89fc158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 0x7f89fc158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45252/0 (socket says 172.21.15.5:45252) 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 -- 172.21.15.5:0/604762350 learned_addr learned my addr 172.21.15.5:0/604762350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a01600640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4003a00 0x7f89fc15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a00c00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89fc15b9d0 0x7f89fc142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 -- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4003a00 msgr2=0x7f89fc15cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4003a00 0x7f89fc15cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 -- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89fc15b9d0 msgr2=0x7f89fc142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89fc15b9d0 0x7f89fc142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a02000640 1 -- 172.21.15.5:0/604762350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc1431d0 con 0x7f89fc15b0c0 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.401+0000 7f8a00c00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89fc15b9d0 0x7f89fc142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.402+0000 7f8a02000640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 0x7f89fc158470 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06f250 tx=0x7f89f400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.402+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4018070 con 0x7f89fc15b0c0 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.402+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4003da0 con 0x7f89fc15b0c0 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.402+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f4006d90 con 0x7f89fc15b0c0 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.402+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc15bed0 con 0x7f89fc15b0c0 2024-04-25T10:10:03.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.402+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc15c490 con 0x7f89fc15b0c0 2024-04-25T10:10:03.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.403+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc06b930 con 0x7f89fc15b0c0 2024-04-25T10:10:03.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.404+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89f4005070 con 0x7f89fc15b0c0 2024-04-25T10:10:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.405+0000 7f89f9600640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89dc06b4d0 0x7f89dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.405+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f89f40895f0 con 0x7f89fc15b0c0 2024-04-25T10:10:03.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.406+0000 7f8a01600640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89dc06b4d0 0x7f89dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:03.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.408+0000 7f8a01600640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89dc06b4d0 0x7f89dc06d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89f0033000 tx=0x7f89f0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:03.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.410+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89f4058120 con 0x7f89fc15b0c0 2024-04-25T10:10:03.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.649+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89fc0b1cd0 con 0x7f89fc15b0c0 2024-04-25T10:10:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.651+0000 7f89f9600640 1 -- 172.21.15.5:0/604762350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f89f4057a20 con 0x7f89fc15b0c0 2024-04-25T10:10:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:03.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:01.369044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:03.658 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-04-25T10:10:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.652+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89dc06b4d0 msgr2=0x7f89dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.652+0000 7f8a02a00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89dc06b4d0 0x7f89dc06d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89f0033000 tx=0x7f89f0008030 comp rx=0 tx=0).stop 2024-04-25T10:10:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.652+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 msgr2=0x7f89fc158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.652+0000 7f8a02a00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 0x7f89fc158470 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06f250 tx=0x7f89f400d9d0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 shutdown_connections 2024-04-25T10:10:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89fc15b0c0 0x7f89fc158470 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f89dc06b4d0 0x7f89dc06d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89fc15b9d0 0x7f89fc142c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 --2- 172.21.15.5:0/604762350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89e4003a00 0x7f89fc15cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 >> 172.21.15.5:0/604762350 conn(0x7f89fc066f20 msgr2=0x7f89fc0af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 shutdown_connections 2024-04-25T10:10:03.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:03.653+0000 7f8a02a00640 1 -- 172.21.15.5:0/604762350 wait complete. 2024-04-25T10:10:04.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:59.535833+0000', 'modified': '2024-04-25T10:10:01.369044+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7959}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7959': {'gid': 7959, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6834/841689019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6832', 'nonce': 841689019}, {'type': 'v1', 'addr': '172.21.15.5:6834', 'nonce': 841689019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:04.082 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:10:04.083 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:10:04.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.289+0000 7fb9cb200640 1 Processor -- start 2024-04-25T10:10:04.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.289+0000 7fb9cb200640 1 -- start start 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9cb200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9cc15eda0 0x7fb9cc0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9cb200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9cc0cecd0 0x7fb9cc15abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9cb200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0b4880 con 0x7fb9cc143d30 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9cb200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc0b45a0 con 0x7fb9cc0cecd0 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9cb200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc15e2c0 con 0x7fb9cc15eda0 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9ca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9cc0cecd0 0x7fb9cc15abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9ca800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9cc0cecd0 0x7fb9cc15abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45262/0 (socket says 172.21.15.5:45262) 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9ca800640 1 -- 172.21.15.5:0/1380389162 learned_addr learned my addr 172.21.15.5:0/1380389162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1380389162 >> v2:172.21.15.80:3300/0 conn(0x7fb9cc15eda0 0x7fb9cc0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9ca800640 1 -- 172.21.15.5:0/1380389162 >> v2:172.21.15.80:3300/0 conn(0x7fb9cc15eda0 msgr2=0x7fb9cc0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1380389162 >> v2:172.21.15.80:3300/0 conn(0x7fb9cc15eda0 0x7fb9cc0ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.291+0000 7fb9ca800640 1 -- 172.21.15.5:0/1380389162 >> v1:172.21.15.80:6789/0 conn(0x7fb9cc143d30 legacy=0x7fb9cc0b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.292+0000 7fb9ca800640 1 -- 172.21.15.5:0/1380389162 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc15e5a0 con 0x7fb9cc0cecd0 2024-04-25T10:10:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.293+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1380389162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc15abc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0cd0d0 tx=0x7fb9c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c05bc85a964f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.293+0000 7fb9c8a00640 1 -- 172.21.15.5:0/1380389162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4036070 con 0x7fb9cc0cecd0 2024-04-25T10:10:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.293+0000 7fb9c8a00640 1 -- 172.21.15.5:0/1380389162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4031600 con 0x7fb9cc0cecd0 2024-04-25T10:10:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.293+0000 7fb9c8a00640 1 -- 172.21.15.5:0/1380389162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c40319d0 con 0x7fb9cc0cecd0 2024-04-25T10:10:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 -- 172.21.15.5:0/1380389162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc0cecd0 msgr2=0x7fb9cc15abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 --2- 172.21.15.5:0/1380389162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc15abc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0cd0d0 tx=0x7fb9c402a660 comp rx=0 tx=0).stop 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 -- 172.21.15.5:0/1380389162 shutdown_connections 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 --2- 172.21.15.5:0/1380389162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc15abc0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 --2- 172.21.15.5:0/1380389162 >> v2:172.21.15.80:3300/0 conn(0x7fb9cc15eda0 0x7fb9cc0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 -- 172.21.15.5:0/1380389162 >> 172.21.15.5:0/1380389162 conn(0x7fb9cc066f20 msgr2=0x7fb9cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 -- 172.21.15.5:0/1380389162 shutdown_connections 2024-04-25T10:10:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.294+0000 7fb9cb200640 1 -- 172.21.15.5:0/1380389162 wait complete. 2024-04-25T10:10:04.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.296+0000 7fb9cb200640 1 Processor -- start 2024-04-25T10:10:04.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.296+0000 7fb9cb200640 1 -- start start 2024-04-25T10:10:04.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc0a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 0x7fb9cc141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9cb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9cc15eda0 0x7fb9cc1385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9cb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0b41f0 con 0x7fb9cc143d30 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9cb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc144110 con 0x7fb9cc0cecd0 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9cb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15d1a0 con 0x7fb9cc15eda0 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 0x7fb9cc141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 0x7fb9cc141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45266/0 (socket says 172.21.15.5:45266) 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9400640 1 -- 172.21.15.5:0/860992255 learned_addr learned my addr 172.21.15.5:0/860992255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc0a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9400640 1 -- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9cc15eda0 msgr2=0x7fb9cc1385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9400640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9cc15eda0 0x7fb9cc1385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.297+0000 7fb9c9400640 1 -- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9cc0cecd0 msgr2=0x7fb9cc0a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c9400640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc0a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c9400640 1 -- 172.21.15.5:0/860992255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc15d480 con 0x7fb9cc143d30 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc0a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:04.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c9400640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 0x7fb9cc141dc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400d5f0 tx=0x7fb9b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c2000640 1 -- 172.21.15.5:0/860992255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40125f0 con 0x7fb9cc143d30 2024-04-25T10:10:04.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c2000640 1 -- 172.21.15.5:0/860992255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4019070 con 0x7fb9cc143d30 2024-04-25T10:10:04.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9c2000640 1 -- 172.21.15.5:0/860992255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40129d0 con 0x7fb9cc143d30 2024-04-25T10:10:04.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc15d760 con 0x7fb9cc143d30 2024-04-25T10:10:04.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.298+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc0a9e90 con 0x7fb9cc143d30 2024-04-25T10:10:04.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.300+0000 7fb9c2000640 1 -- 172.21.15.5:0/860992255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9b4012b70 con 0x7fb9cc143d30 2024-04-25T10:10:04.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.301+0000 7fb9c2000640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a806b4d0 0x7fb9a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.302+0000 7fb9c2000640 1 -- 172.21.15.5:0/860992255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fb9b408a9a0 con 0x7fb9cc143d30 2024-04-25T10:10:04.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.302+0000 7fb9d3c07740 1 Processor -- start 2024-04-25T10:10:04.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.302+0000 7fb9d3c07740 1 -- start start 2024-04-25T10:10:04.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.302+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a806b4d0 0x7fb9a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb1ab30 0x55dfdeb42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdeb1af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb192f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdea7e620 con 0x55dfdeb637a0 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdea27ca0 con 0x55dfdeb42eb0 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfdea7a6c0 con 0x55dfdeb1ab30 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb192f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb192f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45274/0 (socket says 172.21.15.5:45274) 2024-04-25T10:10:04.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9ca800640 1 -- 172.21.15.5:0/673725439 learned_addr learned my addr 172.21.15.5:0/673725439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a806b4d0 0x7fb9a806d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc007fd0 tx=0x7fb9bc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9ca800640 1 -- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb1ab30 msgr2=0x55dfdeb42970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9ca800640 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb1ab30 0x55dfdeb42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9ca800640 1 -- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 msgr2=0x55dfdeb1af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.303+0000 7fb9c9400640 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdeb1af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.304+0000 7fb9ca800640 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdeb1af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.304+0000 7fb9ca800640 1 -- 172.21.15.5:0/673725439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfdea2ef40 con 0x55dfdeb637a0 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.304+0000 7fb9ca800640 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb192f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009f40 tx=0x7fb9c402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51752b8801268777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.304+0000 7fb9baa00640 1 -- 172.21.15.5:0/673725439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4036050 con 0x55dfdeb637a0 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.304+0000 7fb9baa00640 1 -- 172.21.15.5:0/673725439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4042070 con 0x55dfdeb637a0 2024-04-25T10:10:04.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.304+0000 7fb9baa00640 1 -- 172.21.15.5:0/673725439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4009510 con 0x55dfdeb637a0 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 -- 172.21.15.5:0/673725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 msgr2=0x55dfdeb192f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb192f0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009f40 tx=0x7fb9c402c000 comp rx=0 tx=0).stop 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 -- 172.21.15.5:0/673725439 shutdown_connections 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb192f0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdeb1af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/673725439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb1ab30 0x55dfdeb42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 -- 172.21.15.5:0/673725439 >> 172.21.15.5:0/673725439 conn(0x55dfdeb17d50 msgr2=0x55dfdeb60730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 -- 172.21.15.5:0/673725439 shutdown_connections 2024-04-25T10:10:04.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.305+0000 7fb9d3c07740 1 -- 172.21.15.5:0/673725439 wait complete. 2024-04-25T10:10:04.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.306+0000 7fb9d3c07740 1 -- start start 2024-04-25T10:10:04.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.306+0000 7fb9d3c07740 1 Processor -- start 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb42eb0 0x55dfdeb395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb637a0 0x55dfdeb28e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 0x55dfdeb63de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdeb424e0 con 0x55dfdeb293b0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdea1d1f0 con 0x55dfdeb637a0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfdeb41ed0 con 0x55dfdeb42eb0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9d3c07740 10 client.0 _ll_register_callbacks cb 0x55dfdea3aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 0x55dfdeb63de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 0x55dfdeb63de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45276/0 (socket says 172.21.15.5:45276) 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.307+0000 7fb9ca800640 1 -- 172.21.15.5:0/1778354183 learned_addr learned my addr 172.21.15.5:0/1778354183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb637a0 0x55dfdeb28e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb42eb0 0x55dfdeb395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9ca800640 1 -- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb42eb0 msgr2=0x55dfdeb395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb42eb0 0x55dfdeb395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9ca800640 1 -- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb637a0 msgr2=0x55dfdeb28e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb637a0 0x55dfdeb28e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9ca800640 1 -- 172.21.15.5:0/1778354183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfdea3cc50 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb637a0 0x55dfdeb28e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 0x55dfdeb63de0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40093e0 tx=0x7fb9c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19af860e3e5687b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9af400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9af400640 1 -- 172.21.15.5:0/1778354183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4004800 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9af400640 1 -- 172.21.15.5:0/1778354183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fb9c403d450 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.308+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9c4042070 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9af400640 1 -- 172.21.15.5:0/1778354183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c4036020 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 10 client.8034 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55dfde7932d0 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9af400640 1 -- 172.21.15.5:0/1778354183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4042070 con 0x55dfdeb293b0 2024-04-25T10:10:04.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9af400640 1 -- 172.21.15.5:0/1778354183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9c4008040 con 0x55dfdeb293b0 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 10 client.8034 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 10 client.8034 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 20 client.8034 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb379d0 0x55dfdeb3ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 4 client.8034 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfdea8bb60 con 0x55dfdeb379d0 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.309+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb379d0 0x55dfdeb3ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.310+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb379d0 0x55dfdeb3ec10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329905eb6e43efc6 server_cookie=d6d4119e1e545e85 in_seq=0 out_seq=0 2024-04-25T10:10:04.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.310+0000 7fb9af400640 10 client.8034 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9af400640 1 -- 172.21.15.5:0/1778354183 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfdea8bb60 con 0x55dfdeb379d0 2024-04-25T10:10:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9af400640 10 client.8034 handle_command_reply: tid=0 2024-04-25T10:10:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 client.8034 shutdown 2024-04-25T10:10:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 msgr2=0x55dfdeb63de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 0x55dfdeb63de0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40093e0 tx=0x7fb9c4004ab0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 shutdown_connections 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb293b0 0x55dfdeb63de0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb379d0 0x55dfdeb3ec10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb637a0 0x55dfdeb28e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1778354183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb42eb0 0x55dfdeb395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9af400640 0 client.8034 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 >> 172.21.15.5:0/1778354183 conn(0x55dfdeb1ab30 msgr2=0x55dfdeb60530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 shutdown_connections 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1778354183 wait complete. 2024-04-25T10:10:04.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.311+0000 7fb9d3c07740 20 client.8034 trim_cache size 0 max 16384 2024-04-25T10:10:04.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.328+0000 7fb9d3c07740 1 Processor -- start 2024-04-25T10:10:04.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.328+0000 7fb9d3c07740 1 -- start start 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.329+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb293b0 0x55dfdebc1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.329+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 0x55dfdebc1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.329+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb637a0 0x55dfdeb72e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.329+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1ff810 con 0x55dfdeb42eb0 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.329+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1fec70 con 0x55dfdeb293b0 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.329+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1ff1f0 con 0x55dfdeb637a0 2024-04-25T10:10:04.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 0x55dfdebc1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 0x55dfdebc1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45284/0 (socket says 172.21.15.5:45284) 2024-04-25T10:10:04.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 -- 172.21.15.5:0/1860895220 learned_addr learned my addr 172.21.15.5:0/1860895220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb293b0 0x55dfdebc1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 -- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb637a0 msgr2=0x55dfdeb72e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb637a0 0x55dfdeb72e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 -- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb293b0 msgr2=0x55dfdebc1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb293b0 0x55dfdebc1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 -- 172.21.15.5:0/1860895220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfde85d6b0 con 0x55dfdeb42eb0 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 0x55dfdebc1e80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4088a90 tx=0x7fb9b40586a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390425be1a83e0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.330+0000 7fb9ba000640 1 -- 172.21.15.5:0/1860895220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b405ce90 con 0x55dfdeb42eb0 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9ba000640 1 -- 172.21.15.5:0/1860895220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4061450 con 0x55dfdeb42eb0 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9ba000640 1 -- 172.21.15.5:0/1860895220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4057040 con 0x55dfdeb42eb0 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1860895220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 msgr2=0x55dfdebc1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 0x55dfdebc1e80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4088a90 tx=0x7fb9b40586a0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1860895220 shutdown_connections 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb637a0 0x55dfdeb72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb42eb0 0x55dfdebc1e80 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1860895220 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb293b0 0x55dfdebc1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1860895220 >> 172.21.15.5:0/1860895220 conn(0x55dfdeb1ab30 msgr2=0x55dfdeb83680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1860895220 shutdown_connections 2024-04-25T10:10:04.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.331+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1860895220 wait complete. 2024-04-25T10:10:04.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.332+0000 7fb9d3c07740 1 -- start start 2024-04-25T10:10:04.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.332+0000 7fb9d3c07740 1 Processor -- start 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb293b0 0x55dfdeb3de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdebba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb40d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdeb37850 con 0x55dfdeb637a0 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfdeb3a7f0 con 0x55dfdeb42eb0 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfdeb37e50 con 0x55dfdeb293b0 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9d3c07740 10 client.0 _ll_register_callbacks cb 0x7fb9c4009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:04.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb40d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb40d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45296/0 (socket says 172.21.15.5:45296) 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.333+0000 7fb9ca800640 1 -- 172.21.15.5:0/1553154708 learned_addr learned my addr 172.21.15.5:0/1553154708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb293b0 0x55dfdeb3de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9c9400640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdebba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9ca800640 1 -- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb293b0 msgr2=0x55dfdeb3de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb293b0 0x55dfdeb3de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9ca800640 1 -- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 msgr2=0x55dfdebba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdebba7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9ca800640 1 -- 172.21.15.5:0/1553154708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfdeb40270 con 0x55dfdeb637a0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9ca800640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb40d60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40093b0 tx=0x7fb9c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb86df9e1976b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9af400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9af400640 1 -- 172.21.15.5:0/1553154708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c403d660 con 0x55dfdeb637a0 2024-04-25T10:10:04.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9af400640 1 -- 172.21.15.5:0/1553154708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fb9c403d800 con 0x55dfdeb637a0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9b4061450 con 0x55dfdeb637a0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9af400640 1 -- 172.21.15.5:0/1553154708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c4066050 con 0x55dfdeb637a0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9d3c07740 10 client.8040 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.334+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55dfdeb3d3e0 con 0x55dfdeb637a0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9af400640 1 -- 172.21.15.5:0/1553154708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4061450 con 0x55dfdeb637a0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9af400640 1 -- 172.21.15.5:0/1553154708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9c4069020 con 0x55dfdeb637a0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9d3c07740 10 client.8040 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9d3c07740 10 client.8040 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9d3c07740 20 client.8040 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb198f0 0x55dfdeb2b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9d3c07740 4 client.8040 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dfdea85eb0 con 0x55dfdeb198f0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.335+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb198f0 0x55dfdeb2b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9c9e00640 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb198f0 0x55dfdeb2b810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fbcb0193abc151 server_cookie=7374c9ada2f7d353 in_seq=0 out_seq=0 2024-04-25T10:10:04.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9af400640 10 client.8040 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9af400640 1 -- 172.21.15.5:0/1553154708 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55dfdea85eb0 con 0x55dfdeb198f0 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9af400640 10 client.8040 handle_command_reply: tid=0 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 client.8040 shutdown 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 msgr2=0x55dfdeb40d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb40d60 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40093b0 tx=0x7fb9c4004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 shutdown_connections 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfdeb637a0 0x55dfdeb40d60 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55dfdeb198f0 0x55dfdeb2b810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55dfdeb42eb0 0x55dfdebba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 --2- 172.21.15.5:0/1553154708 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55dfdeb293b0 0x55dfdeb3de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 >> 172.21.15.5:0/1553154708 conn(0x55dfdeb1ab30 msgr2=0x55dfdebc1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.336+0000 7fb9af400640 0 client.8040 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 shutdown_connections 2024-04-25T10:10:04.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9d3c07740 1 -- 172.21.15.5:0/1553154708 wait complete. 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9d3c07740 20 client.8040 trim_cache size 0 max 16384 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7959, 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:10:04.346 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:10:04.347 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.9886369720000001, 2024-04-25T10:10:04.347 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 75, 2024-04-25T10:10:04.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 93, 2024-04-25T10:10:04.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T10:10:04.347 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 6.9221449279999998 2024-04-25T10:10:04.347 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a806b4d0 msgr2=0x7fb9a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a806b4d0 0x7fb9a806d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc007fd0 tx=0x7fb9bc009040 comp rx=0 tx=0).stop 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 msgr2=0x7fb9cc141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 0x7fb9cc141dc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400d5f0 tx=0x7fb9b400dab0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 shutdown_connections 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9cc15eda0 0x7fb9cc1385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9cc143d30 0x7fb9cc141dc0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb9a806b4d0 0x7fb9a806d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 --2- 172.21.15.5:0/860992255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9cc0cecd0 0x7fb9cc0a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 >> 172.21.15.5:0/860992255 conn(0x7fb9cc066f20 msgr2=0x7fb9cc06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.337+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 shutdown_connections 2024-04-25T10:10:04.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.338+0000 7fb9cb200640 1 -- 172.21.15.5:0/860992255 wait complete. 2024-04-25T10:10:04.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:10:04.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.537+0000 7fc987e00640 1 Processor -- start 2024-04-25T10:10:04.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.537+0000 7fc987e00640 1 -- start start 2024-04-25T10:10:04.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.538+0000 7fc987e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc988161200 0x7fc9880b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.538+0000 7fc987e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d2380 0x7fc9880d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.538+0000 7fc987e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9880cf410 con 0x7fc9880d0600 2024-04-25T10:10:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.538+0000 7fc987e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880d27c0 con 0x7fc988161200 2024-04-25T10:10:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.538+0000 7fc987e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880b28b0 con 0x7fc9880d2380 2024-04-25T10:10:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc988161200 0x7fc9880b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d2380 0x7fc9880d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc988161200 0x7fc9880b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:39756/0 (socket says 172.21.15.5:39756) 2024-04-25T10:10:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986a00640 1 -- 172.21.15.5:0/95817280 learned_addr learned my addr 172.21.15.5:0/95817280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986a00640 1 -- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d2380 msgr2=0x7fc9880d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986a00640 1 --2- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d2380 0x7fc9880d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.539+0000 7fc986a00640 1 -- 172.21.15.5:0/95817280 >> v1:172.21.15.80:6789/0 conn(0x7fc9880d0600 legacy=0x7fc9880ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.540+0000 7fc986a00640 1 -- 172.21.15.5:0/95817280 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880cf6f0 con 0x7fc988161200 2024-04-25T10:10:04.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.541+0000 7fc986a00640 1 --2- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc988161200 0x7fc9880b4bb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098a0 tx=0x7fc97802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f91c7ccd961d0d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.541+0000 7fc985600640 1 -- 172.21.15.5:0/95817280 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978037070 con 0x7fc988161200 2024-04-25T10:10:04.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.541+0000 7fc985600640 1 -- 172.21.15.5:0/95817280 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978032440 con 0x7fc988161200 2024-04-25T10:10:04.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc985600640 1 -- 172.21.15.5:0/95817280 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978032810 con 0x7fc988161200 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 -- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc988161200 msgr2=0x7fc9880b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 --2- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc988161200 0x7fc9880b4bb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc9780098a0 tx=0x7fc97802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 -- 172.21.15.5:0/95817280 shutdown_connections 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 --2- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d2380 0x7fc9880d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 --2- 172.21.15.5:0/95817280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc988161200 0x7fc9880b4bb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 -- 172.21.15.5:0/95817280 >> 172.21.15.5:0/95817280 conn(0x7fc98808af80 msgr2=0x7fc9880d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 -- 172.21.15.5:0/95817280 shutdown_connections 2024-04-25T10:10:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.542+0000 7fc987e00640 1 -- 172.21.15.5:0/95817280 wait complete. 2024-04-25T10:10:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.544+0000 7fc987e00640 1 Processor -- start 2024-04-25T10:10:04.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.544+0000 7fc987e00640 1 -- start start 2024-04-25T10:10:04.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d0600 0x7fc9880b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9880d2380 0x7fc98814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 0x7fc9880b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880b3f90 con 0x7fc988161200 2024-04-25T10:10:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880d42b0 con 0x7fc9880d2380 2024-04-25T10:10:04.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880b4110 con 0x7fc9880d0600 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.545+0000 7fc987400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 0x7fc9880b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 0x7fc9880b86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45306/0 (socket says 172.21.15.5:45306) 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 -- 172.21.15.5:0/365498490 learned_addr learned my addr 172.21.15.5:0/365498490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc986a00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d0600 0x7fc9880b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc986000640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9880d2380 0x7fc98814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 -- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d0600 msgr2=0x7fc9880b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d0600 0x7fc9880b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 -- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9880d2380 msgr2=0x7fc98814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9880d2380 0x7fc98814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.546+0000 7fc987400640 1 -- 172.21.15.5:0/365498490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880b6810 con 0x7fc988161200 2024-04-25T10:10:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.547+0000 7fc987400640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 0x7fc9880b86a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c6b0 tx=0x7fc98000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.547+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980003b70 con 0x7fc988161200 2024-04-25T10:10:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.547+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980003d30 con 0x7fc988161200 2024-04-25T10:10:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.547+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980005310 con 0x7fc988161200 2024-04-25T10:10:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.547+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9880b6ab0 con 0x7fc988161200 2024-04-25T10:10:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.547+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9880b7070 con 0x7fc988161200 2024-04-25T10:10:04.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.548+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc988068f60 con 0x7fc988161200 2024-04-25T10:10:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.549+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9800054b0 con 0x7fc988161200 2024-04-25T10:10:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.550+0000 7fc97f400640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc96406b4d0 0x7fc96406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.551+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc98008a260 con 0x7fc988161200 2024-04-25T10:10:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.551+0000 7fc986a00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc96406b4d0 0x7fc96406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:04.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.552+0000 7fc986a00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc96406b4d0 0x7fc96406d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc978034000 tx=0x7fc97802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:04.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.555+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc980058750 con 0x7fc988161200 2024-04-25T10:10:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.799+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9880b9530 con 0x7fc988161200 2024-04-25T10:10:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.800+0000 7fc97f400640 1 -- 172.21.15.5:0/365498490 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc98005c7f0 con 0x7fc988161200 2024-04-25T10:10:04.805 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:10:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc96406b4d0 msgr2=0x7fc96406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc96406b4d0 0x7fc96406d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc978034000 tx=0x7fc97802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 msgr2=0x7fc9880b86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 0x7fc9880b86a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c6b0 tx=0x7fc98000cb70 comp rx=0 tx=0).stop 2024-04-25T10:10:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 shutdown_connections 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc988161200 0x7fc9880b86a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc96406b4d0 0x7fc96406d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9880d2380 0x7fc98814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 --2- 172.21.15.5:0/365498490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9880d0600 0x7fc9880b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.802+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 >> 172.21.15.5:0/365498490 conn(0x7fc98808af80 msgr2=0x7fc988160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.803+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 shutdown_connections 2024-04-25T10:10:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:04.803+0000 7fc987e00640 1 -- 172.21.15.5:0/365498490 wait complete. 2024-04-25T10:10:04.827 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:10:04.827 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:10:04.827 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:10:04.828 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:10:04.828 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:04.828 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:10:04.828 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:10:04.828 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:10:04.828 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:04.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:04.828 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:10:04.881 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:10:04.881 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:10:04.881 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:10:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42121sec preferred_lft 42121sec 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:04.883 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:04.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:04.884 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:10:04.956 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:10:04.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:04.960 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:10:05.009 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:10:05.009 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:10:05.010 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:10:05.010 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:10:05.010 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:10:05.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.010 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:10:05.010 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:10:05.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:10:05.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:10:05.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:10:05.010 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:10:05.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.161 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:10:05.212 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:10:05.213 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:10:05.214 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:05.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:10:05.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:05.304 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:05.362 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:05.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:05.362 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:05.414 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:10:05.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:05.535 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:05.535 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:05.535 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:10:05.557 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:05.557 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:05.557 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:10:05.557 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:10:05.557 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:10:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:10:05.587 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:10:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:10:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:10:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:10:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:10:05.591 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:10:05.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:05.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.670 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:10:05.728 INFO:teuthology.orchestra.run.smithi005.stdout:client5804 2024-04-25T10:10:05.728 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:10:05.729 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:05.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:05.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:10:06.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.001+0000 7fb822800640 1 Processor -- start 2024-04-25T10:10:06.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.002+0000 7fb822800640 1 -- start start 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.003+0000 7fb822800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 0x7fb81c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.003+0000 7fb822800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb81c071460 0x7fb81c140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.003+0000 7fb822800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c15b050 con 0x7fb81c071000 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.003+0000 7fb822800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c13fcf0 con 0x7fb81c071460 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.003+0000 7fb822800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c15b330 con 0x7fb81c15dff0 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.004+0000 7fb820a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb81c071000 0x7fb81c0cb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44814/0 (socket says 172.21.15.5:44814) 2024-04-25T10:10:06.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.004+0000 7fb820a00640 1 -- 172.21.15.5:0/2111161280 learned_addr learned my addr 172.21.15.5:0/2111161280 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.004+0000 7fb821400640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 0x7fb81c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.004+0000 7fb821e00640 1 --2- 172.21.15.5:0/2111161280 >> v2:172.21.15.80:3300/0 conn(0x7fb81c071460 0x7fb81c140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.004+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959900445 0 0) 0x7fb81c15b050 con 0x7fb81c071000 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.004+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb804003520 con 0x7fb81c071000 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401664093 0 0) 0x7fb804003520 con 0x7fb81c071000 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 msgr2=0x7fb81c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 0x7fb81c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 >> v2:172.21.15.80:3300/0 conn(0x7fb81c071460 msgr2=0x7fb81c140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 --2- 172.21.15.5:0/2111161280 >> v2:172.21.15.80:3300/0 conn(0x7fb81c071460 0x7fb81c140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c15b610 con 0x7fb81c071000 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb808001670 con 0x7fb81c071000 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 >> v1:172.21.15.5:6789/0 conn(0x7fb81c071000 legacy=0x7fb81c0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8040039b0 0x7fb804003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804004800 0x7fb804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 0x7fb8040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c13fcf0 con 0x7fb804004800 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c15b330 con 0x7fb8040039b0 2024-04-25T10:10:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80400b9c0 con 0x7fb81c15dff0 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb820a00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804004800 0x7fb804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.005+0000 7fb821400640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8040039b0 0x7fb804003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.006+0000 7fb821e00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 0x7fb8040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.006+0000 7fb820a00640 1 -- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 msgr2=0x7fb8040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.006+0000 7fb820a00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c15dff0 0x7fb8040051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.006+0000 7fb820a00640 1 -- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8040039b0 msgr2=0x7fb804003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.006+0000 7fb820a00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8040039b0 0x7fb804003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.006+0000 7fb820a00640 1 -- 172.21.15.5:0/2111161280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804003520 con 0x7fb804004800 2024-04-25T10:10:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.007+0000 7fb820a00640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804004800 0x7fb804004c80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb808001050 tx=0x7fb804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f658441e60a6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.007+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80802e070 con 0x7fb804004800 2024-04-25T10:10:06.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.007+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808005af0 con 0x7fb804004800 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.007+0000 7fb81be00640 1 -- 172.21.15.5:0/2111161280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808007d70 con 0x7fb804004800 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb822800640 1 -- 172.21.15.5:0/2111161280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804004800 msgr2=0x7fb804004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb822800640 1 --2- 172.21.15.5:0/2111161280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb804004800 0x7fb804004c80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb808001050 tx=0x7fb804005830 comp rx=0 tx=0).stop 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb821e00640 1 -- 172.21.15.5:0/2111161280 reap_dead start 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb822800640 1 -- 172.21.15.5:0/2111161280 shutdown_connections 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb822800640 1 -- 172.21.15.5:0/2111161280 >> 172.21.15.5:0/2111161280 conn(0x7fb81c066f20 msgr2=0x7fb81c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb822800640 1 -- 172.21.15.5:0/2111161280 shutdown_connections 2024-04-25T10:10:06.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.008+0000 7fb822800640 1 -- 172.21.15.5:0/2111161280 wait complete. 2024-04-25T10:10:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.010+0000 7fb822800640 1 Processor -- start 2024-04-25T10:10:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.010+0000 7fb822800640 1 -- start start 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb822800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb804004800 0x7fb81c13cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb822800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 0x7fb81c157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb822800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c14f660 0x7fb81c0cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb822800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0cbc90 con 0x7fb81c14f1e0 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb822800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0cbb10 con 0x7fb804004800 2024-04-25T10:10:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb822800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1406c0 con 0x7fb81c14f660 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb820a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 0x7fb81c157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb820a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 0x7fb81c157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45318/0 (socket says 172.21.15.5:45318) 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.011+0000 7fb820a00640 1 -- 172.21.15.5:0/3535717639 learned_addr learned my addr 172.21.15.5:0/3535717639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb821400640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb804004800 0x7fb81c13cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb821e00640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c14f660 0x7fb81c0cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb820a00640 1 -- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c14f660 msgr2=0x7fb81c0cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb820a00640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c14f660 0x7fb81c0cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb820a00640 1 -- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb804004800 msgr2=0x7fb81c13cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb820a00640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb804004800 0x7fb81c13cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb820a00640 1 -- 172.21.15.5:0/3535717639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0ad7e0 con 0x7fb81c14f1e0 2024-04-25T10:10:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.012+0000 7fb820a00640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 0x7fb81c157790 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb808034040 tx=0x7fb808000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.013+0000 7fb819600640 1 -- 172.21.15.5:0/3535717639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808042070 con 0x7fb81c14f1e0 2024-04-25T10:10:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.013+0000 7fb819600640 1 -- 172.21.15.5:0/3535717639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb808007d50 con 0x7fb81c14f1e0 2024-04-25T10:10:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.013+0000 7fb819600640 1 -- 172.21.15.5:0/3535717639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808001cc0 con 0x7fb81c14f1e0 2024-04-25T10:10:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.013+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c0adac0 con 0x7fb81c14f1e0 2024-04-25T10:10:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.013+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0ae080 con 0x7fb81c14f1e0 2024-04-25T10:10:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.015+0000 7fb819600640 1 -- 172.21.15.5:0/3535717639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb808045070 con 0x7fb81c14f1e0 2024-04-25T10:10:06.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.016+0000 7fb819600640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.016+0000 7fb819600640 1 -- 172.21.15.5:0/3535717639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fb8080aae60 con 0x7fb81c14f1e0 2024-04-25T10:10:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.017+0000 7fb826c41740 1 Processor -- start 2024-04-25T10:10:06.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.017+0000 7fb826c41740 1 -- start start 2024-04-25T10:10:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.017+0000 7fb821400640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb826c41740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e7e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e36050 0x55cd77e7c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb826c41740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd77d98980 con 0x55cd77e60370 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd77d41de0 con 0x55cd77e36050 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd77d94a20 con 0x55cd77e7dc10 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e60750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45322/0 (socket says 172.21.15.5:45322) 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb821400640 1 -- 172.21.15.5:0/2339255654 learned_addr learned my addr 172.21.15.5:0/2339255654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb821400640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb8100098b0 tx=0x7fb810008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.018+0000 7fb820a00640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e7e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821e00640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e36050 0x55cd77e7c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821400640 1 -- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 msgr2=0x55cd77e7e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821400640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e7e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821400640 1 -- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e36050 msgr2=0x55cd77e7c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821400640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e36050 0x55cd77e7c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821400640 1 -- 172.21.15.5:0/2339255654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd77d5fe30 con 0x55cd77e60370 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb820a00640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e7e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821e00640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e36050 0x55cd77e7c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb821400640 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e60750 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb810013c00 tx=0x7fb810016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3f94e27d935dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb80dc00640 1 -- 172.21.15.5:0/2339255654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810016b60 con 0x55cd77e60370 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb80dc00640 1 -- 172.21.15.5:0/2339255654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81001c030 con 0x55cd77e60370 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.019+0000 7fb80dc00640 1 -- 172.21.15.5:0/2339255654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810004020 con 0x55cd77e60370 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 -- 172.21.15.5:0/2339255654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 msgr2=0x55cd77e60750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e60750 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fb810013c00 tx=0x7fb810016000 comp rx=0 tx=0).stop 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 -- 172.21.15.5:0/2339255654 shutdown_connections 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e36050 0x55cd77e7c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e7e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 --2- 172.21.15.5:0/2339255654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e60750 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 -- 172.21.15.5:0/2339255654 >> 172.21.15.5:0/2339255654 conn(0x55cd77e32980 msgr2=0x55cd77e7b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 -- 172.21.15.5:0/2339255654 shutdown_connections 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.020+0000 7fb826c41740 1 -- 172.21.15.5:0/2339255654 wait complete. 2024-04-25T10:10:06.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.021+0000 7fb826c41740 1 -- start start 2024-04-25T10:10:06.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.021+0000 7fb826c41740 1 Processor -- start 2024-04-25T10:10:06.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e5ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e5e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e50f70 0x55cd77e5ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd77e36df0 con 0x55cd77e60370 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd77d373c0 con 0x55cd77e50f70 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd77e308b0 con 0x55cd77e7dc10 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.022+0000 7fb826c41740 10 client.0 _ll_register_callbacks cb 0x55cd77aacc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e5ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e5ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45330/0 (socket says 172.21.15.5:45330) 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 -- 172.21.15.5:0/1248052961 learned_addr learned my addr 172.21.15.5:0/1248052961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821e00640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e50f70 0x55cd77e5ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb820a00640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e5e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 -- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 msgr2=0x55cd77e5e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e5e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 -- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e50f70 msgr2=0x55cd77e5ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e50f70 0x55cd77e5ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821400640 1 -- 172.21.15.5:0/1248052961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd77b757e0 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.023+0000 7fb821e00640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e50f70 0x55cd77e5ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb821400640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e5ce50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb8100168f0 tx=0x7fb810004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfea452afbaced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb802000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb802000640 1 -- 172.21.15.5:0/1248052961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810023850 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb802000640 1 -- 172.21.15.5:0/1248052961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fb8100239f0 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81001c030 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb802000640 1 -- 172.21.15.5:0/1248052961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb81003d050 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb826c41740 10 client.8055 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55cd77bd5f70 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.024+0000 7fb802000640 1 -- 172.21.15.5:0/1248052961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81001c030 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb802000640 1 -- 172.21.15.5:0/1248052961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb810040020 con 0x55cd77e60370 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb826c41740 10 client.8055 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb826c41740 10 client.8055 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb826c41740 20 client.8055 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb826c41740 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e72900 0x55cd77e72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb826c41740 4 client.8055 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:06.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd77da5ec0 con 0x55cd77e72900 2024-04-25T10:10:06.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.025+0000 7fb820a00640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e72900 0x55cd77e72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.026+0000 7fb820a00640 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e72900 0x55cd77e72d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd14d2de501a724 server_cookie=d87d4563aebba976 in_seq=0 out_seq=0 2024-04-25T10:10:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.026+0000 7fb802000640 10 client.8055 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.026+0000 7fb802000640 1 -- 172.21.15.5:0/1248052961 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd77da5ec0 con 0x55cd77e72900 2024-04-25T10:10:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.026+0000 7fb802000640 10 client.8055 handle_command_reply: tid=0 2024-04-25T10:10:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.026+0000 7fb826c41740 1 client.8055 shutdown 2024-04-25T10:10:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 msgr2=0x55cd77e5ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e5ce50 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb8100168f0 tx=0x7fb810004730 comp rx=0 tx=0).stop 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 shutdown_connections 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e72900 0x55cd77e72d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e50f70 0x55cd77e5ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e7dc10 0x55cd77e5e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 --2- 172.21.15.5:0/1248052961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e60370 0x55cd77e5ce50 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 >> 172.21.15.5:0/1248052961 conn(0x55cd77e36050 msgr2=0x55cd77e7ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb802000640 0 client.8055 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 shutdown_connections 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 1 -- 172.21.15.5:0/1248052961 wait complete. 2024-04-25T10:10:06.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.027+0000 7fb826c41740 20 client.8055 trim_cache size 0 max 16384 2024-04-25T10:10:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.044+0000 7fb826c41740 1 Processor -- start 2024-04-25T10:10:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.044+0000 7fb826c41740 1 -- start start 2024-04-25T10:10:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb826c41740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 0x55cd77e5c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e60370 0x55cd77e35890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e7dc10 0x55cd77e72af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb826c41740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c200e50 con 0x55cd77e50f70 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1fffa0 con 0x55cd77e7dc10 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1ff380 con 0x55cd77e60370 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 0x55cd77e5c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 0x55cd77e5c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45342/0 (socket says 172.21.15.5:45342) 2024-04-25T10:10:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 -- 172.21.15.5:0/909744905 learned_addr learned my addr 172.21.15.5:0/909744905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 -- 172.21.15.5:0/909744905 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e60370 msgr2=0x55cd77e35890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e60370 0x55cd77e35890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 -- 172.21.15.5:0/909744905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e7dc10 msgr2=0x55cd77e72af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e7dc10 0x55cd77e72af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.045+0000 7fb821400640 1 -- 172.21.15.5:0/909744905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd77a6d170 con 0x55cd77e50f70 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.046+0000 7fb821400640 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 0x55cd77e5c640 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb81000b010 tx=0x7fb81002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f9ed7614c3b5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.046+0000 7fb80d200640 1 -- 172.21.15.5:0/909744905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810023650 con 0x55cd77e50f70 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.046+0000 7fb80d200640 1 -- 172.21.15.5:0/909744905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb810046070 con 0x55cd77e50f70 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.046+0000 7fb80d200640 1 -- 172.21.15.5:0/909744905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810023a10 con 0x55cd77e50f70 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 -- 172.21.15.5:0/909744905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 msgr2=0x55cd77e5c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 0x55cd77e5c640 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb81000b010 tx=0x7fb81002d000 comp rx=0 tx=0).stop 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 -- 172.21.15.5:0/909744905 shutdown_connections 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e7dc10 0x55cd77e72af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e60370 0x55cd77e35890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 --2- 172.21.15.5:0/909744905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e50f70 0x55cd77e5c640 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 -- 172.21.15.5:0/909744905 >> 172.21.15.5:0/909744905 conn(0x55cd77e36050 msgr2=0x55cd77e5e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 -- 172.21.15.5:0/909744905 shutdown_connections 2024-04-25T10:10:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.047+0000 7fb826c41740 1 -- 172.21.15.5:0/909744905 wait complete. 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.048+0000 7fb826c41740 1 -- start start 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.048+0000 7fb826c41740 1 Processor -- start 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e50f70 0x55cd77e91b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e60370 0x55cd77e5c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 0x55cd77e5c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd77e5d090 con 0x55cd77e7dc10 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1ff980 con 0x55cd77e60370 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd77e5cb80 con 0x55cd77e50f70 2024-04-25T10:10:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb826c41740 10 client.0 _ll_register_callbacks cb 0x55cd77e30d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.049+0000 7fb821e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 0x55cd77e5c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 0x55cd77e5c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45358/0 (socket says 172.21.15.5:45358) 2024-04-25T10:10:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 -- 172.21.15.5:0/545201216 learned_addr learned my addr 172.21.15.5:0/545201216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb820a00640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e60370 0x55cd77e5c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821400640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e50f70 0x55cd77e91b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 -- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e50f70 msgr2=0x55cd77e91b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e50f70 0x55cd77e91b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 -- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e60370 msgr2=0x55cd77e5c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e60370 0x55cd77e5c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 -- 172.21.15.5:0/545201216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd77e311b0 con 0x55cd77e7dc10 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb821e00640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 0x55cd77e5c580 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb81c06ba60 tx=0x7fb814009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cef6ff46382e6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.050+0000 7fb802000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb802000640 1 -- 172.21.15.5:0/545201216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81401d060 con 0x55cd77e7dc10 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb810046070 con 0x55cd77e7dc10 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb802000640 1 -- 172.21.15.5:0/545201216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fb814008b40 con 0x55cd77e7dc10 2024-04-25T10:10:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb802000640 1 -- 172.21.15.5:0/545201216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814023050 con 0x55cd77e7dc10 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 10 client.8061 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55cd77e36ae0 con 0x55cd77e7dc10 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb802000640 1 -- 172.21.15.5:0/545201216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb810046070 con 0x55cd77e7dc10 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb802000640 1 -- 172.21.15.5:0/545201216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb814026020 con 0x55cd77e7dc10 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 10 client.8061 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 10 client.8061 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 20 client.8061 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e9d510 0x55cd77ee57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 4 client.8061 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.051+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55cd77da0210 con 0x55cd77e9d510 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.052+0000 7fb821400640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e9d510 0x55cd77ee57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.052+0000 7fb821400640 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e9d510 0x55cd77ee57e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d718ebcfedf916 server_cookie=8e75fd7d0ed49ace in_seq=0 out_seq=0 2024-04-25T10:10:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.052+0000 7fb802000640 10 client.8061 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb802000640 1 -- 172.21.15.5:0/545201216 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cd77da0210 con 0x55cd77e9d510 2024-04-25T10:10:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb802000640 10 client.8061 handle_command_reply: tid=0 2024-04-25T10:10:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 client.8061 shutdown 2024-04-25T10:10:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 msgr2=0x55cd77e5c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 0x55cd77e5c580 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb81c06ba60 tx=0x7fb814009c70 comp rx=0 tx=0).stop 2024-04-25T10:10:06.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 shutdown_connections 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd77e7dc10 0x55cd77e5c580 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x55cd77e9d510 0x55cd77ee57e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cd77e60370 0x55cd77e5c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 --2- 172.21.15.5:0/545201216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cd77e50f70 0x55cd77e91b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb802000640 0 client.8061 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 >> 172.21.15.5:0/545201216 conn(0x55cd77e36050 msgr2=0x55cd77e2fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 shutdown_connections 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.053+0000 7fb826c41740 1 -- 172.21.15.5:0/545201216 wait complete. 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb826c41740 20 client.8061 trim_cache size 0 max 16384 2024-04-25T10:10:06.063 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:10:06.064 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:10:06.064 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "074bb7c7-555b-44b0-bd2e-ec23cdbad987", 2024-04-25T10:10:06.064 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:10:06.064 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb7fc06b4d0 msgr2=0x7fb7fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb8100098b0 tx=0x7fb810008040 comp rx=0 tx=0).stop 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 msgr2=0x7fb81c157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 0x7fb81c157790 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb808034040 tx=0x7fb808000fe0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 shutdown_connections 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb81c14f660 0x7fb81c0cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb81c14f1e0 0x7fb81c157790 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb7fc06b4d0 0x7fb7fc06d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 --2- 172.21.15.5:0/3535717639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb804004800 0x7fb81c13cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.054+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 >> 172.21.15.5:0/3535717639 conn(0x7fb81c066f20 msgr2=0x7fb81c13dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.055+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 shutdown_connections 2024-04-25T10:10:06.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.055+0000 7fb822800640 1 -- 172.21.15.5:0/3535717639 wait complete. 2024-04-25T10:10:06.066 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-04-25T10:10:06.057+0000 7f6ea7400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e255958100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:10:06.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:06.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:10:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.275+0000 7f93bd600640 1 Processor -- start 2024-04-25T10:10:06.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.276+0000 7f93bd600640 1 -- start start 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93bd600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f93b8144fe0 0x7f93b806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93bd600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f93b806d130 0x7f93b8071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93bd600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b81453c0 con 0x7f93b8071670 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93bd600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b80a8560 con 0x7f93b8144fe0 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93bd600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b806d6a0 con 0x7f93b806d130 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93b6a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f93b8144fe0 0x7f93b806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93b6a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f93b8144fe0 0x7f93b806e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45366/0 (socket says 172.21.15.5:45366) 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93b6a00640 1 -- 172.21.15.5:0/2252896437 learned_addr learned my addr 172.21.15.5:0/2252896437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.278+0000 7f93afe00640 1 --2- 172.21.15.5:0/2252896437 >> v2:172.21.15.80:3300/0 conn(0x7f93b806d130 0x7f93b8071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.279+0000 7f93b6a00640 1 -- 172.21.15.5:0/2252896437 >> v2:172.21.15.80:3300/0 conn(0x7f93b806d130 msgr2=0x7f93b8071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.279+0000 7f93b6a00640 1 --2- 172.21.15.5:0/2252896437 >> v2:172.21.15.80:3300/0 conn(0x7f93b806d130 0x7f93b8071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.279+0000 7f93b6a00640 1 -- 172.21.15.5:0/2252896437 >> v1:172.21.15.80:6789/0 conn(0x7f93b8071670 legacy=0x7f93b80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.279+0000 7f93b6a00640 1 -- 172.21.15.5:0/2252896437 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b806d980 con 0x7f93b8144fe0 2024-04-25T10:10:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.280+0000 7f93b6a00640 1 --2- 172.21.15.5:0/2252896437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8144fe0 0x7f93b806e560 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f93b80cec80 tx=0x7f93a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170965dc77d7a085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93b6000640 1 -- 172.21.15.5:0/2252896437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8037070 con 0x7f93b8144fe0 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93b6000640 1 -- 172.21.15.5:0/2252896437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a8032440 con 0x7f93b8144fe0 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93b6000640 1 -- 172.21.15.5:0/2252896437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8032810 con 0x7f93b8144fe0 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93bd600640 1 -- 172.21.15.5:0/2252896437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8144fe0 msgr2=0x7f93b806e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93bd600640 1 --2- 172.21.15.5:0/2252896437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8144fe0 0x7f93b806e560 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f93b80cec80 tx=0x7f93a802a230 comp rx=0 tx=0).stop 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93bd600640 1 -- 172.21.15.5:0/2252896437 shutdown_connections 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93bd600640 1 --2- 172.21.15.5:0/2252896437 >> v2:172.21.15.80:3300/0 conn(0x7f93b806d130 0x7f93b8071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93bd600640 1 --2- 172.21.15.5:0/2252896437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8144fe0 0x7f93b806e560 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.281+0000 7f93bd600640 1 -- 172.21.15.5:0/2252896437 >> 172.21.15.5:0/2252896437 conn(0x7f93b806ab60 msgr2=0x7f93b806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.282+0000 7f93bd600640 1 -- 172.21.15.5:0/2252896437 shutdown_connections 2024-04-25T10:10:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.282+0000 7f93bd600640 1 -- 172.21.15.5:0/2252896437 wait complete. 2024-04-25T10:10:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.283+0000 7f93bd600640 1 Processor -- start 2024-04-25T10:10:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 -- start start 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f93b806d130 0x7f93b815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 0x7f93b80cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93b8144fe0 0x7f93b80b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b815be50 con 0x7f93b8071670 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b806df40 con 0x7f93b8144fe0 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.284+0000 7f93bd600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b806eaa0 con 0x7f93b806d130 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 0x7f93b80cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 0x7f93b80cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45378/0 (socket says 172.21.15.5:45378) 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 -- 172.21.15.5:0/1230406920 learned_addr learned my addr 172.21.15.5:0/1230406920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93b6a00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f93b806d130 0x7f93b815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93b7400640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93b8144fe0 0x7f93b80b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 -- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f93b806d130 msgr2=0x7f93b815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f93b806d130 0x7f93b815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 -- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93b8144fe0 msgr2=0x7f93b80b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93b8144fe0 0x7f93b80b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.285+0000 7f93afe00640 1 -- 172.21.15.5:0/1230406920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80c9250 con 0x7f93b8071670 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.286+0000 7f93afe00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 0x7f93b80cadf0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f93a000d800 tx=0x7f93a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.286+0000 7f93aea00640 1 -- 172.21.15.5:0/1230406920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000b6c0 con 0x7f93b8071670 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.286+0000 7f93aea00640 1 -- 172.21.15.5:0/1230406920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a0019070 con 0x7f93b8071670 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.286+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b80c9530 con 0x7f93b8071670 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.286+0000 7f93aea00640 1 -- 172.21.15.5:0/1230406920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000ba80 con 0x7f93b8071670 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.286+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b80c9af0 con 0x7f93b8071670 2024-04-25T10:10:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.288+0000 7f93aea00640 1 -- 172.21.15.5:0/1230406920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93a000bc20 con 0x7f93b8071670 2024-04-25T10:10:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.289+0000 7f93aea00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f939406b4d0 0x7f939406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.289+0000 7f93aea00640 1 -- 172.21.15.5:0/1230406920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f93a008aa00 con 0x7f93b8071670 2024-04-25T10:10:06.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.290+0000 7f93b6a00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f939406b4d0 0x7f939406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.290+0000 7f93c0419740 1 Processor -- start 2024-04-25T10:10:06.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.290+0000 7f93c0419740 1 -- start start 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f26d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93c0419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93b6a00640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f939406b4d0 0x7f939406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f93a8034000 tx=0x7f93a802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93c0419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561993e88c70 con 0x561993f4f670 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561993e31cf0 con 0x561993f61280 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.291+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x561993e84d10 con 0x561993f24bb0 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f4faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45382/0 (socket says 172.21.15.5:45382) 2024-04-25T10:10:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 -- 172.21.15.5:0/576587932 learned_addr learned my addr 172.21.15.5:0/576587932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b6a00640 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f26d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93afe00640 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 -- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 msgr2=0x561993f26d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f26d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 -- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 msgr2=0x561993f24f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f24f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 -- 172.21.15.5:0/576587932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561993c677d0 con 0x561993f4f670 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93afe00640 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f24f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.292+0000 7f93b7400640 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f4faf0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f93b00043f0 tx=0x7f93b00048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3784f55f163bcbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.293+0000 7f93a7000640 1 -- 172.21.15.5:0/576587932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0006d60 con 0x561993f4f670 2024-04-25T10:10:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.293+0000 7f93a7000640 1 -- 172.21.15.5:0/576587932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0018070 con 0x561993f4f670 2024-04-25T10:10:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.293+0000 7f93a7000640 1 -- 172.21.15.5:0/576587932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000ea80 con 0x561993f4f670 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.293+0000 7f93c0419740 1 -- 172.21.15.5:0/576587932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 msgr2=0x561993f4faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.293+0000 7f93c0419740 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f4faf0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f93b00043f0 tx=0x7f93b00048b0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 -- 172.21.15.5:0/576587932 shutdown_connections 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f4faf0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f24f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 --2- 172.21.15.5:0/576587932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f26d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 -- 172.21.15.5:0/576587932 >> 172.21.15.5:0/576587932 conn(0x561993eeca30 msgr2=0x561993f6a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 -- 172.21.15.5:0/576587932 shutdown_connections 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 -- 172.21.15.5:0/576587932 wait complete. 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.294+0000 7f93c0419740 1 -- start start 2024-04-25T10:10:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.295+0000 7f93c0419740 1 Processor -- start 2024-04-25T10:10:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.295+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f64d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f4f670 0x561993f4c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93c0419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 0x561993f635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93c0419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561993e273a0 con 0x561993f61280 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x561993f4d1c0 con 0x561993f4f670 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x561993f52dc0 con 0x561993f24bb0 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93c0419740 10 client.0 _ll_register_callbacks cb 0x561993e44b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 0x561993f635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 0x561993f635b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45388/0 (socket says 172.21.15.5:45388) 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 -- 172.21.15.5:0/4279933828 learned_addr learned my addr 172.21.15.5:0/4279933828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b6a00640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f64d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93afe00640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f4f670 0x561993f4c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 -- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 msgr2=0x561993f64d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f64d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 -- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f4f670 msgr2=0x561993f4c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f4f670 0x561993f4c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.296+0000 7f93b7400640 1 -- 172.21.15.5:0/4279933828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561993b9d3f0 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f93b7400640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 0x561993f635b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f93b000e5f0 tx=0x7f93b0003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd06459bd971c22d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f939b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f939b400640 1 -- 172.21.15.5:0/4279933828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0026430 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f939b400640 1 -- 172.21.15.5:0/4279933828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f93b0013420 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93b0018070 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f939b400640 1 -- 172.21.15.5:0/4279933828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93b0013620 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f93c0419740 10 client.8073 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.297+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x561993b5d0f0 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f939b400640 1 -- 172.21.15.5:0/4279933828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0018070 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f939b400640 1 -- 172.21.15.5:0/4279933828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93b002e050 con 0x561993f61280 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93c0419740 10 client.8073 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93c0419740 10 client.8073 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93c0419740 20 client.8073 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93c0419740 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f63cb0 0x561993f7df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93c0419740 4 client.8073 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561993e961b0 con 0x561993f63cb0 2024-04-25T10:10:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.298+0000 7f93b6a00640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f63cb0 0x561993f7df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.299+0000 7f93b6a00640 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f63cb0 0x561993f7df30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19caef454d1a16b3 server_cookie=9ad6a31917839c3e in_seq=0 out_seq=0 2024-04-25T10:10:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.299+0000 7f939b400640 10 client.8073 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.299+0000 7f939b400640 1 -- 172.21.15.5:0/4279933828 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561993e961b0 con 0x561993f63cb0 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.299+0000 7f939b400640 10 client.8073 handle_command_reply: tid=0 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 client.8073 shutdown 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 msgr2=0x561993f635b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 0x561993f635b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f93b000e5f0 tx=0x7f93b0003a70 comp rx=0 tx=0).stop 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 shutdown_connections 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f61280 0x561993f635b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f63cb0 0x561993f7df30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f4f670 0x561993f4c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 --2- 172.21.15.5:0/4279933828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993f64d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f939b400640 0 client.8073 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 >> 172.21.15.5:0/4279933828 conn(0x561993eeca30 msgr2=0x561993f4ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 shutdown_connections 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 1 -- 172.21.15.5:0/4279933828 wait complete. 2024-04-25T10:10:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.300+0000 7f93c0419740 20 client.8073 trim_cache size 0 max 16384 2024-04-25T10:10:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.317+0000 7f93c0419740 1 Processor -- start 2024-04-25T10:10:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.317+0000 7f93c0419740 1 -- start start 2024-04-25T10:10:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.318+0000 7f93c0419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 0x561993f6dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.318+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f4f670 0x561993f79d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.318+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f7a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.318+0000 7f93c0419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8200880 con 0x561993f24bb0 2024-04-25T10:10:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.318+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81ff9a0 con 0x561993f61280 2024-04-25T10:10:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.318+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80c7430 con 0x561993f4f670 2024-04-25T10:10:06.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 0x561993f6dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 0x561993f6dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45392/0 (socket says 172.21.15.5:45392) 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 -- 172.21.15.5:0/3892412815 learned_addr learned my addr 172.21.15.5:0/3892412815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93afe00640 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f4f670 0x561993f79d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 -- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f4f670 msgr2=0x561993f79d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f4f670 0x561993f79d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 -- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 msgr2=0x561993f7a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b7400640 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f7a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f7a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.319+0000 7f93b6a00640 1 -- 172.21.15.5:0/3892412815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561993f63950 con 0x561993f24bb0 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.320+0000 7f93b6a00640 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 0x561993f6dce0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009630 tx=0x7f93a802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf96509accb56f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.320+0000 7f93a6600640 1 -- 172.21.15.5:0/3892412815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8037040 con 0x561993f24bb0 2024-04-25T10:10:06.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.320+0000 7f93a6600640 1 -- 172.21.15.5:0/3892412815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a804c280 con 0x561993f24bb0 2024-04-25T10:10:06.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.320+0000 7f93a6600640 1 -- 172.21.15.5:0/3892412815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a804c630 con 0x561993f24bb0 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.320+0000 7f93c0419740 1 -- 172.21.15.5:0/3892412815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 msgr2=0x561993f6dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.320+0000 7f93c0419740 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 0x561993f6dce0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009630 tx=0x7f93a802c000 comp rx=0 tx=0).stop 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 -- 172.21.15.5:0/3892412815 shutdown_connections 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993f7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f4f670 0x561993f79d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 --2- 172.21.15.5:0/3892412815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f24bb0 0x561993f6dce0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 -- 172.21.15.5:0/3892412815 >> 172.21.15.5:0/3892412815 conn(0x561993eeca30 msgr2=0x561993f4b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 -- 172.21.15.5:0/3892412815 shutdown_connections 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 -- 172.21.15.5:0/3892412815 wait complete. 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.321+0000 7f93c0419740 1 -- start start 2024-04-25T10:10:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.322+0000 7f93c0419740 1 Processor -- start 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993fcf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993fd5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561993f647b0 con 0x561993f4f670 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81ff360 con 0x561993f61280 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x561993f7a7a0 con 0x561993f24bb0 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93c0419740 10 client.0 _ll_register_callbacks cb 0x7f93b0004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f23790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45396/0 (socket says 172.21.15.5:45396) 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93afe00640 1 -- 172.21.15.5:0/3994472986 learned_addr learned my addr 172.21.15.5:0/3994472986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93b7400640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993fd5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93afe00640 1 -- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 msgr2=0x561993fcf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93b6a00640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993fcf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.323+0000 7f93afe00640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993fcf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93afe00640 1 -- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 msgr2=0x561993fd5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93afe00640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993fd5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93afe00640 1 -- 172.21.15.5:0/3994472986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561993fd55c0 con 0x561993f4f670 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93b7400640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993fd5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93afe00640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f23790 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f93a0058b30 tx=0x7f93a0059530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2a4174edb92dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f939b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f939b400640 1 -- 172.21.15.5:0/3994472986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0087060 con 0x561993f4f670 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f939b400640 1 -- 172.21.15.5:0/3994472986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f93a00554a0 con 0x561993f4f670 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a804c280 con 0x561993f4f670 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f939b400640 1 -- 172.21.15.5:0/3994472986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a0070070 con 0x561993f4f670 2024-04-25T10:10:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93c0419740 10 client.8079 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.324+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x561993f23d30 con 0x561993f4f670 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f939b400640 1 -- 172.21.15.5:0/3994472986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a804c280 con 0x561993f4f670 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f939b400640 1 -- 172.21.15.5:0/3994472986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f93a0013070 con 0x561993f4f670 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93c0419740 10 client.8079 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93c0419740 10 client.8079 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93c0419740 20 client.8079 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93c0419740 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f52f50 0x561993f7cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93c0419740 4 client.8079 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561993e90500 con 0x561993f52f50 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.325+0000 7f93b6a00640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f52f50 0x561993f7cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93b6a00640 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f52f50 0x561993f7cd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7701d2d9d56e4355 server_cookie=a441e9dab5bb1e56 in_seq=0 out_seq=0 2024-04-25T10:10:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f939b400640 10 client.8079 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f939b400640 1 -- 172.21.15.5:0/3994472986 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x561993e90500 con 0x561993f52f50 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f939b400640 10 client.8079 handle_command_reply: tid=0 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 client.8079 shutdown 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 msgr2=0x561993f23790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f23790 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f93a0058b30 tx=0x7f93a0059530 comp rx=0 tx=0).stop 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 shutdown_connections 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x561993f52f50 0x561993f7cd50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x561993f61280 0x561993fd5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561993f4f670 0x561993f23790 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 --2- 172.21.15.5:0/3994472986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x561993f24bb0 0x561993fcf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f939b400640 0 client.8079 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 >> 172.21.15.5:0/3994472986 conn(0x561993eeca30 msgr2=0x561993f6dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 shutdown_connections 2024-04-25T10:10:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 1 -- 172.21.15.5:0/3994472986 wait complete. 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.326+0000 7f93c0419740 20 client.8079 trim_cache size 0 max 16384 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "074bb7c7-555b-44b0-bd2e-ec23cdbad987": { 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/dir", 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "074bb7c7-555b-44b0-bd2e-ec23cdbad987", 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-04-25T10:10:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:06.337 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:10:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f939406b4d0 msgr2=0x7f939406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f939406b4d0 0x7f939406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f93a8034000 tx=0x7f93a802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 msgr2=0x7f93b80cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 0x7f93b80cadf0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f93a000d800 tx=0x7f93a000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 shutdown_connections 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f939406b4d0 0x7f939406d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f93b8144fe0 0x7f93b80b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b8071670 0x7f93b80cadf0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 --2- 172.21.15.5:0/1230406920 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f93b806d130 0x7f93b815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 >> 172.21.15.5:0/1230406920 conn(0x7f93b806ab60 msgr2=0x7f93b8143ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 shutdown_connections 2024-04-25T10:10:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:06.327+0000 7f93bd600640 1 -- 172.21.15.5:0/1230406920 wait complete. 2024-04-25T10:10:06.349 INFO:tasks.cephfs.filesystem:scrub status for tag:074bb7c7-555b-44b0-bd2e-ec23cdbad987 - {'path': '/dir', 'tag': '074bb7c7-555b-44b0-bd2e-ec23cdbad987', 'options': 'recursive,repair'} 2024-04-25T10:10:25.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:25.550 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:36.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:36.351 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:10:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.571+0000 7fc75e400640 1 Processor -- start 2024-04-25T10:10:36.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.572+0000 7fc75e400640 1 -- start start 2024-04-25T10:10:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.574+0000 7fc75e400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc7580b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.574+0000 7fc75e400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc7580d0f40 0x7fc7580b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.574+0000 7fc75e400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7580b5780 con 0x7fc75806ab60 2024-04-25T10:10:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.574+0000 7fc75e400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75815dd70 con 0x7fc7580d0f40 2024-04-25T10:10:36.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.574+0000 7fc75e400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75806b9b0 con 0x7fc75815d990 2024-04-25T10:10:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc757e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc7580d0f40 0x7fc7580b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc75d000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc7580b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc757e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc7580d0f40 0x7fc7580b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:59608/0 (socket says 172.21.15.5:59608) 2024-04-25T10:10:36.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc757e00640 1 -- 172.21.15.5:0/2454961659 learned_addr learned my addr 172.21.15.5:0/2454961659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc757e00640 1 -- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 msgr2=0x7fc7580b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc757e00640 1 --2- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc7580b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.575+0000 7fc757e00640 1 -- 172.21.15.5:0/2454961659 >> v1:172.21.15.80:6789/0 conn(0x7fc75806ab60 legacy=0x7fc758161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.576+0000 7fc757e00640 1 -- 172.21.15.5:0/2454961659 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7580d0180 con 0x7fc7580d0f40 2024-04-25T10:10:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.577+0000 7fc757e00640 1 --2- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc7580b50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc7440098a0 tx=0x7fc74402a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc364a82afebb09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.577+0000 7fc757400640 1 -- 172.21.15.5:0/2454961659 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744037070 con 0x7fc7580d0f40 2024-04-25T10:10:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.577+0000 7fc757400640 1 -- 172.21.15.5:0/2454961659 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744032440 con 0x7fc7580d0f40 2024-04-25T10:10:36.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.577+0000 7fc757400640 1 -- 172.21.15.5:0/2454961659 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744032810 con 0x7fc7580d0f40 2024-04-25T10:10:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 -- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 msgr2=0x7fc7580b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 --2- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc7580b50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc7440098a0 tx=0x7fc74402a200 comp rx=0 tx=0).stop 2024-04-25T10:10:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 -- 172.21.15.5:0/2454961659 shutdown_connections 2024-04-25T10:10:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 --2- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc7580b50c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 --2- 172.21.15.5:0/2454961659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc7580b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 -- 172.21.15.5:0/2454961659 >> 172.21.15.5:0/2454961659 conn(0x7fc7580b5ab0 msgr2=0x7fc7580999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 -- 172.21.15.5:0/2454961659 shutdown_connections 2024-04-25T10:10:36.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.578+0000 7fc75e400640 1 -- 172.21.15.5:0/2454961659 wait complete. 2024-04-25T10:10:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.580+0000 7fc75e400640 1 Processor -- start 2024-04-25T10:10:36.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.580+0000 7fc75e400640 1 -- start start 2024-04-25T10:10:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75e400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 0x7fc758159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75e400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc758156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75e400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc75809da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75e400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580d0a20 con 0x7fc75806ab60 2024-04-25T10:10:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75e400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7580d0740 con 0x7fc7580d0f40 2024-04-25T10:10:36.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75e400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75806aff0 con 0x7fc75815d990 2024-04-25T10:10:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.581+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 0x7fc758159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 0x7fc758159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47608/0 (socket says 172.21.15.5:47608) 2024-04-25T10:10:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 -- 172.21.15.5:0/2093461403 learned_addr learned my addr 172.21.15.5:0/2093461403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc757e00640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc758156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 -- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 msgr2=0x7fc75809da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc75809da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 -- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 msgr2=0x7fc758156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc758156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 -- 172.21.15.5:0/2093461403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc758157520 con 0x7fc75806ab60 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc757e00640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc758156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.582+0000 7fc75d000640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 0x7fc758159f30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fc74800e870 tx=0x7fc74800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.583+0000 7fc754c00640 1 -- 172.21.15.5:0/2093461403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748017070 con 0x7fc75806ab60 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.583+0000 7fc754c00640 1 -- 172.21.15.5:0/2093461403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74800c940 con 0x7fc75806ab60 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.583+0000 7fc754c00640 1 -- 172.21.15.5:0/2093461403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74800cd30 con 0x7fc75806ab60 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.583+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7581577c0 con 0x7fc75806ab60 2024-04-25T10:10:36.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.583+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7581f4840 con 0x7fc75806ab60 2024-04-25T10:10:36.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.585+0000 7fc754c00640 1 -- 172.21.15.5:0/2093461403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc74801a070 con 0x7fc75806ab60 2024-04-25T10:10:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.586+0000 7fc754c00640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73806b500 0x7fc73806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.586+0000 7fc754c00640 1 -- 172.21.15.5:0/2093461403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc74808a770 con 0x7fc75806ab60 2024-04-25T10:10:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.587+0000 7fc757e00640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73806b500 0x7fc73806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.587+0000 7fc76282d740 1 Processor -- start 2024-04-25T10:10:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.587+0000 7fc76282d740 1 -- start start 2024-04-25T10:10:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc76282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042852ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc757e00640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73806b500 0x7fc73806d9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc744034000 tx=0x7fc74402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x557042851150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x55704287bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc76282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570427b4c70 con 0x557042850d70 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55704275dcf0 con 0x55704287b830 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570427b0d10 con 0x55704288d440 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042852ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042852ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47616/0 (socket says 172.21.15.5:47616) 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.588+0000 7fc75d000640 1 -- 172.21.15.5:0/1863893929 learned_addr learned my addr 172.21.15.5:0/1863893929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc757e00640 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x557042851150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75da00640 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x55704287bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75d000640 1 -- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 msgr2=0x557042851150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75d000640 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x557042851150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75d000640 1 -- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 msgr2=0x55704287bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75d000640 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x55704287bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75d000640 1 -- 172.21.15.5:0/1863893929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570425937d0 con 0x557042850d70 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc757e00640 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x557042851150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc75d000640 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042852ed0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc748060d80 tx=0x7fc7480585c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2794bf448ee14b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc74d600640 1 -- 172.21.15.5:0/1863893929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748021050 con 0x557042850d70 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc74d600640 1 -- 172.21.15.5:0/1863893929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74805ccf0 con 0x557042850d70 2024-04-25T10:10:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.589+0000 7fc74d600640 1 -- 172.21.15.5:0/1863893929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480612b0 con 0x557042850d70 2024-04-25T10:10:36.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 -- 172.21.15.5:0/1863893929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 msgr2=0x557042852ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042852ed0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc748060d80 tx=0x7fc7480585c0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 -- 172.21.15.5:0/1863893929 shutdown_connections 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x55704287bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x557042851150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 --2- 172.21.15.5:0/1863893929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042852ed0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 -- 172.21.15.5:0/1863893929 >> 172.21.15.5:0/1863893929 conn(0x557042818a30 msgr2=0x557042896830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 -- 172.21.15.5:0/1863893929 shutdown_connections 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.590+0000 7fc76282d740 1 -- 172.21.15.5:0/1863893929 wait complete. 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.591+0000 7fc76282d740 1 -- start start 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.591+0000 7fc76282d740 1 Processor -- start 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x55704287e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x557042877f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x5570428979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570427533a0 con 0x557042850d70 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x557042879380 con 0x55704287b830 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557042869500 con 0x55704288d440 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc76282d740 10 client.0 _ll_register_callbacks cb 0x557042770b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.592+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x55704287e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x55704287e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47622/0 (socket says 172.21.15.5:47622) 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 -- 172.21.15.5:0/118050425 learned_addr learned my addr 172.21.15.5:0/118050425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 -- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 msgr2=0x5570428979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75da00640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x5570428979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc757e00640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x557042877f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x5570428979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 -- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 msgr2=0x557042877f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x557042877f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 -- 172.21.15.5:0/118050425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570424c93f0 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc75d000640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x55704287e6c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc748054fb0 tx=0x7fc748000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e69615017436827 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.593+0000 7fc742000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc742000640 1 -- 172.21.15.5:0/118050425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748057ff0 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc742000640 1 -- 172.21.15.5:0/118050425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc748058190 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc74805ccf0 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc742000640 1 -- 172.21.15.5:0/118050425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7480548f0 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 10 client.8088 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5570424890f0 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc742000640 1 -- 172.21.15.5:0/118050425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74805ccf0 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc742000640 1 -- 172.21.15.5:0/118050425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc748054a90 con 0x557042850d70 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 10 client.8088 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 10 client.8088 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 20 client.8088 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042897f10 0x557042867b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 4 client.8088 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.594+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570427c21b0 con 0x557042897f10 2024-04-25T10:10:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.595+0000 7fc757e00640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042897f10 0x557042867b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.595+0000 7fc757e00640 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042897f10 0x557042867b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228058b0a67c9b35 server_cookie=5f2b5f0ab42d8266 in_seq=0 out_seq=0 2024-04-25T10:10:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.595+0000 7fc742000640 10 client.8088 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc742000640 1 -- 172.21.15.5:0/118050425 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5570427c21b0 con 0x557042897f10 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc742000640 10 client.8088 handle_command_reply: tid=0 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc76282d740 1 client.8088 shutdown 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 msgr2=0x55704287e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc76282d740 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x55704287e6c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc748054fb0 tx=0x7fc748000bc0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 shutdown_connections 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc76282d740 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704288d440 0x5570428979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.596+0000 7fc76282d740 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042897f10 0x557042867b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc76282d740 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704287b830 0x557042877f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc76282d740 1 --2- 172.21.15.5:0/118050425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x55704287e6c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 >> 172.21.15.5:0/118050425 conn(0x557042818a30 msgr2=0x557042879560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc742000640 0 client.8088 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 shutdown_connections 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc76282d740 1 -- 172.21.15.5:0/118050425 wait complete. 2024-04-25T10:10:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.597+0000 7fc76282d740 20 client.8088 trim_cache size 0 max 16384 2024-04-25T10:10:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.613+0000 7fc76282d740 1 Processor -- start 2024-04-25T10:10:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 -- start start 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557042850d70 0x55704287f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 0x55704284b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7582008a0 con 0x55704287b830 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581ff9c0 con 0x55704288d440 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.614+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581fede0 con 0x557042850d70 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47638/0 (socket says 172.21.15.5:47638) 2024-04-25T10:10:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 -- 172.21.15.5:0/3424197945 learned_addr learned my addr 172.21.15.5:0/3424197945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 -- 172.21.15.5:0/3424197945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557042850d70 msgr2=0x55704287f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557042850d70 0x55704287f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 -- 172.21.15.5:0/3424197945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 msgr2=0x55704284b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 0x55704284b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 -- 172.21.15.5:0/3424197945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570428a6660 con 0x55704287b830 2024-04-25T10:10:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.615+0000 7fc757e00640 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 0x55704284abd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc74402a240 tx=0x7fc74402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1672fb7253e89f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc74cc00640 1 -- 172.21.15.5:0/3424197945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744037040 con 0x55704287b830 2024-04-25T10:10:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc74cc00640 1 -- 172.21.15.5:0/3424197945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74404f030 con 0x55704287b830 2024-04-25T10:10:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc74cc00640 1 -- 172.21.15.5:0/3424197945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744003610 con 0x55704287b830 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 -- 172.21.15.5:0/3424197945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 msgr2=0x55704284abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 0x55704284abd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc74402a240 tx=0x7fc74402c000 comp rx=0 tx=0).stop 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 -- 172.21.15.5:0/3424197945 shutdown_connections 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 0x55704284b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 --2- 172.21.15.5:0/3424197945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x557042850d70 0x55704287f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 -- 172.21.15.5:0/3424197945 >> 172.21.15.5:0/3424197945 conn(0x557042818a30 msgr2=0x557042879470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.616+0000 7fc76282d740 1 -- 172.21.15.5:0/3424197945 shutdown_connections 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.617+0000 7fc76282d740 1 -- 172.21.15.5:0/3424197945 wait complete. 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.617+0000 7fc76282d740 1 -- start start 2024-04-25T10:10:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.617+0000 7fc76282d740 1 Processor -- start 2024-04-25T10:10:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042900800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 0x55704284b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570427c21b0 con 0x557042850d70 2024-04-25T10:10:36.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7581ff380 con 0x55704288d440 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x557042869330 con 0x55704287b830 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.618+0000 7fc76282d740 10 client.0 _ll_register_callbacks cb 0x7fc74806f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042900800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042900800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47644/0 (socket says 172.21.15.5:47644) 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 -- 172.21.15.5:0/337618436 learned_addr learned my addr 172.21.15.5:0/337618436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 -- 172.21.15.5:0/337618436 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704287b830 msgr2=0x55704284abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 -- 172.21.15.5:0/337618436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 msgr2=0x55704284b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 0x55704284b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 -- 172.21.15.5:0/337618436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557042897d40 con 0x557042850d70 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc75d000640 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042900800 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc748060d80 tx=0x7fc74805f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08f8d5fbeda642c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.619+0000 7fc742000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:36.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc742000640 1 -- 172.21.15.5:0/337618436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748021070 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc742000640 1 -- 172.21.15.5:0/337618436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc748057b00 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc74404f030 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc742000640 1 -- 172.21.15.5:0/337618436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc74805ccf0 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 10 client.8094 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x557042901180 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc742000640 1 -- 172.21.15.5:0/337618436 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74404f030 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc742000640 1 -- 172.21.15.5:0/337618436 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc748077a70 con 0x557042850d70 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 10 client.8094 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 10 client.8094 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 20 client.8094 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042900d40 0x557042867b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 4 client.8094 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.620+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5570427bc500 con 0x557042900d40 2024-04-25T10:10:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.621+0000 7fc757e00640 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042900d40 0x557042867b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.621+0000 7fc757e00640 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042900d40 0x557042867b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7893dbc204339001 server_cookie=90c93528d061848 in_seq=0 out_seq=0 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.621+0000 7fc742000640 10 client.8094 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.621+0000 7fc742000640 1 -- 172.21.15.5:0/337618436 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5570427bc500 con 0x557042900d40 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.621+0000 7fc742000640 10 client.8094 handle_command_reply: tid=0 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.621+0000 7fc76282d740 1 client.8094 shutdown 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 msgr2=0x557042900800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042900800 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc748060d80 tx=0x7fc74805f050 comp rx=0 tx=0).stop 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 shutdown_connections 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x557042900d40 0x557042867b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55704288d440 0x55704284b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55704287b830 0x55704284abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc742000640 0 client.8094 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 --2- 172.21.15.5:0/337618436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557042850d70 0x557042900800 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 >> 172.21.15.5:0/337618436 conn(0x557042818a30 msgr2=0x5570428b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 shutdown_connections 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 1 -- 172.21.15.5:0/337618436 wait complete. 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.622+0000 7fc76282d740 20 client.8094 trim_cache size 0 max 16384 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:10:36.631 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:10:36.632 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73806b500 msgr2=0x7fc73806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73806b500 0x7fc73806d9c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc744034000 tx=0x7fc74402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 msgr2=0x7fc758159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 0x7fc758159f30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fc74800e870 tx=0x7fc74800ed30 comp rx=0 tx=0).stop 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 shutdown_connections 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc75815d990 0x7fc75809da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc73806b500 0x7fc73806d9c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7580d0f40 0x7fc758156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 --2- 172.21.15.5:0/2093461403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc75806ab60 0x7fc758159f30 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 >> 172.21.15.5:0/2093461403 conn(0x7fc7580b5ab0 msgr2=0x7fc7580af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 shutdown_connections 2024-04-25T10:10:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.623+0000 7fc75e400640 1 -- 172.21.15.5:0/2093461403 wait complete. 2024-04-25T10:10:36.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:36.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.840+0000 7f11aae00640 1 Processor -- start 2024-04-25T10:10:36.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.840+0000 7f11aae00640 1 -- start start 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.841+0000 7f11aae00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f11a415db00 0x7f11a40d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.841+0000 7f11aae00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f11a40ce2b0 0x7f11a40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.841+0000 7f11aae00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a40d1e80 con 0x7f11a40b53b0 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11aae00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a40b5930 con 0x7f11a415db00 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11aae00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a40b5c10 con 0x7f11a40ce2b0 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f11a415db00 0x7f11a40d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f11a415db00 0x7f11a40d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57458/0 (socket says 172.21.15.5:57458) 2024-04-25T10:10:36.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 -- 172.21.15.5:0/1558460329 learned_addr learned my addr 172.21.15.5:0/1558460329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a8e00640 1 --2- 172.21.15.5:0/1558460329 >> v2:172.21.15.80:3300/0 conn(0x7f11a40ce2b0 0x7f11a40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 -- 172.21.15.5:0/1558460329 >> v2:172.21.15.80:3300/0 conn(0x7f11a40ce2b0 msgr2=0x7f11a40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 --2- 172.21.15.5:0/1558460329 >> v2:172.21.15.80:3300/0 conn(0x7f11a40ce2b0 0x7f11a40d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 -- 172.21.15.5:0/1558460329 >> v1:172.21.15.80:6789/0 conn(0x7f11a40b53b0 legacy=0x7f11a4161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.842+0000 7f11a3e00640 1 -- 172.21.15.5:0/1558460329 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40b5ef0 con 0x7f11a415db00 2024-04-25T10:10:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.843+0000 7f11a3e00640 1 --2- 172.21.15.5:0/1558460329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a40d0340 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f11a40cdf10 tx=0x7f119402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb866657db65aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.844+0000 7f11a2a00640 1 -- 172.21.15.5:0/1558460329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194037070 con 0x7f11a415db00 2024-04-25T10:10:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.844+0000 7f11a2a00640 1 -- 172.21.15.5:0/1558460329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194032440 con 0x7f11a415db00 2024-04-25T10:10:36.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.844+0000 7f11a2a00640 1 -- 172.21.15.5:0/1558460329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194032810 con 0x7f11a415db00 2024-04-25T10:10:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.844+0000 7f11aae00640 1 -- 172.21.15.5:0/1558460329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 msgr2=0x7f11a40d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.844+0000 7f11aae00640 1 --2- 172.21.15.5:0/1558460329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a40d0340 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f11a40cdf10 tx=0x7f119402a230 comp rx=0 tx=0).stop 2024-04-25T10:10:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.845+0000 7f11aae00640 1 -- 172.21.15.5:0/1558460329 shutdown_connections 2024-04-25T10:10:36.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.845+0000 7f11aae00640 1 --2- 172.21.15.5:0/1558460329 >> v2:172.21.15.80:3300/0 conn(0x7f11a40ce2b0 0x7f11a40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.845+0000 7f11aae00640 1 --2- 172.21.15.5:0/1558460329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a40d0340 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.845+0000 7f11aae00640 1 -- 172.21.15.5:0/1558460329 >> 172.21.15.5:0/1558460329 conn(0x7f11a4072290 msgr2=0x7f11a40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.845+0000 7f11aae00640 1 -- 172.21.15.5:0/1558460329 shutdown_connections 2024-04-25T10:10:36.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.845+0000 7f11aae00640 1 -- 172.21.15.5:0/1558460329 wait complete. 2024-04-25T10:10:36.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.847+0000 7f11aae00640 1 Processor -- start 2024-04-25T10:10:36.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.847+0000 7f11aae00640 1 -- start start 2024-04-25T10:10:36.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.848+0000 7f11aae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11a40b53b0 0x7f11a40cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11aae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11a40ce2b0 0x7f11a415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11aae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a41e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11aae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d3660 con 0x7f11a415db00 2024-04-25T10:10:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11aae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40b61d0 con 0x7f11a40ce2b0 2024-04-25T10:10:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11aae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40b6c10 con 0x7f11a40b53b0 2024-04-25T10:10:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a41e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a41e62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57466/0 (socket says 172.21.15.5:57466) 2024-04-25T10:10:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 -- 172.21.15.5:0/650441061 learned_addr learned my addr 172.21.15.5:0/650441061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:36.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a3400640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11a40ce2b0 0x7f11a415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a3e00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11a40b53b0 0x7f11a40cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 -- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11a40b53b0 msgr2=0x7f11a40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11a40b53b0 0x7f11a40cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 -- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11a40ce2b0 msgr2=0x7f11a415cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11a40ce2b0 0x7f11a415cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.849+0000 7f11a8e00640 1 -- 172.21.15.5:0/650441061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4160110 con 0x7f11a415db00 2024-04-25T10:10:36.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.850+0000 7f11a8e00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a41e62e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f119c007e70 tx=0x7f119c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.850+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c014620 con 0x7f11a415db00 2024-04-25T10:10:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.850+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c019070 con 0x7f11a415db00 2024-04-25T10:10:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.850+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c014a20 con 0x7f11a415db00 2024-04-25T10:10:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.850+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a41603f0 con 0x7f11a415db00 2024-04-25T10:10:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.850+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a41609b0 con 0x7f11a415db00 2024-04-25T10:10:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.852+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a4068fd0 con 0x7f11a415db00 2024-04-25T10:10:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.853+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f119c014bc0 con 0x7f11a415db00 2024-04-25T10:10:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.854+0000 7f119be00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f118406b4d0 0x7f118406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.854+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f119c08ad60 con 0x7f11a415db00 2024-04-25T10:10:36.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.855+0000 7f11a3e00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f118406b4d0 0x7f118406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.856+0000 7f11a3e00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f118406b4d0 0x7f118406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1194034000 tx=0x7f119402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:36.859+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f119c059930 con 0x7f11a415db00 2024-04-25T10:10:37.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.097+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11a40cb8f0 con 0x7f11a415db00 2024-04-25T10:10:37.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.098+0000 7f119be00640 1 -- 172.21.15.5:0/650441061 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f119c059230 con 0x7f11a415db00 2024-04-25T10:10:37.104 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:37.104 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:01.369044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:37.104 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f118406b4d0 msgr2=0x7f118406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f118406b4d0 0x7f118406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1194034000 tx=0x7f119402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 msgr2=0x7f11a41e62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a41e62e0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f119c007e70 tx=0x7f119c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 shutdown_connections 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a415db00 0x7f11a41e62e0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f118406b4d0 0x7f118406d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11a40ce2b0 0x7f11a415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 --2- 172.21.15.5:0/650441061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f11a40b53b0 0x7f11a40cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.101+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 >> 172.21.15.5:0/650441061 conn(0x7f11a4072290 msgr2=0x7f11a40b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.102+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 shutdown_connections 2024-04-25T10:10:37.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.102+0000 7f11aae00640 1 -- 172.21.15.5:0/650441061 wait complete. 2024-04-25T10:10:37.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:10:37.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.318+0000 7fc098a00640 1 Processor -- start 2024-04-25T10:10:37.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.318+0000 7fc098a00640 1 -- start start 2024-04-25T10:10:37.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.319+0000 7fc098a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 0x7fc0940d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.319+0000 7fc098a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940cbcb0 con 0x7fc09415d990 2024-04-25T10:10:37.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.319+0000 7fc098a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940b1df0 con 0x7fc0940d1ac0 2024-04-25T10:10:37.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.319+0000 7fc098a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940b1b90 con 0x7fc0940b29c0 2024-04-25T10:10:37.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.319+0000 7fc091e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc09415d990 0x7fc0940b1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45996/0 (socket says 172.21.15.5:45996) 2024-04-25T10:10:37.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.319+0000 7fc091e00640 1 -- 172.21.15.5:0/906770422 learned_addr learned my addr 172.21.15.5:0/906770422 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc091400640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 0x7fc0940d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126492259 0 0) 0x7fc0940cbcb0 con 0x7fc09415d990 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003520 con 0x7fc09415d990 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 365843974 0 0) 0x7fc078003520 con 0x7fc09415d990 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 msgr2=0x7fc0940d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 0x7fc0940d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 >> v1:172.21.15.80:6789/0 conn(0x7fc0940d1ac0 legacy=0x7fc09406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940b20d0 con 0x7fc09415d990 2024-04-25T10:10:37.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fc084001670 con 0x7fc09415d990 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 >> v1:172.21.15.5:6789/0 conn(0x7fc09415d990 legacy=0x7fc0940b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc078004800 0x7fc078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 0x7fc07800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940b1b90 con 0x7fc078004800 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078005520 con 0x7fc0780039e0 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.320+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800bf00 con 0x7fc0940b29c0 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091400640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc078004800 0x7fc078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091e00640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc092800640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 0x7fc07800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091400640 1 -- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 msgr2=0x7fc07800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091400640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940b29c0 0x7fc07800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091400640 1 -- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0780039e0 msgr2=0x7fc078003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091400640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0780039e0 0x7fc078003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.321+0000 7fc091400640 1 -- 172.21.15.5:0/906770422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078003520 con 0x7fc078004800 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.322+0000 7fc091400640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc078004800 0x7fc078004c80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc08000ba00 tx=0x7fc078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ceadaaf09cb22e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.323+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080030070 con 0x7fc078004800 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.323+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08002adb0 con 0x7fc078004800 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.323+0000 7fc090a00640 1 -- 172.21.15.5:0/906770422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000b600 con 0x7fc078004800 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.323+0000 7fc098a00640 1 -- 172.21.15.5:0/906770422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc078004800 msgr2=0x7fc078004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.323+0000 7fc098a00640 1 --2- 172.21.15.5:0/906770422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc078004800 0x7fc078004c80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc08000ba00 tx=0x7fc078005830 comp rx=0 tx=0).stop 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.323+0000 7fc092800640 1 -- 172.21.15.5:0/906770422 reap_dead start 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.324+0000 7fc098a00640 1 -- 172.21.15.5:0/906770422 shutdown_connections 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.324+0000 7fc098a00640 1 -- 172.21.15.5:0/906770422 >> 172.21.15.5:0/906770422 conn(0x7fc09408af80 msgr2=0x7fc09408b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.324+0000 7fc098a00640 1 -- 172.21.15.5:0/906770422 shutdown_connections 2024-04-25T10:10:37.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.324+0000 7fc098a00640 1 -- 172.21.15.5:0/906770422 wait complete. 2024-04-25T10:10:37.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.325+0000 7fc098a00640 1 Processor -- start 2024-04-25T10:10:37.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.326+0000 7fc098a00640 1 -- start start 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.326+0000 7fc098a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc078004800 0x7fc0940b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc098a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 0x7fc0941571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc098a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc094157700 0x7fc0940aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc098a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cf9d0 con 0x7fc0940c9c00 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc098a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cfd20 con 0x7fc078004800 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc098a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940d1250 con 0x7fc094157700 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc091400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 0x7fc0941571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc091400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 0x7fc0941571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57480/0 (socket says 172.21.15.5:57480) 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc091400640 1 -- 172.21.15.5:0/3219944391 learned_addr learned my addr 172.21.15.5:0/3219944391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:37.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc091e00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc078004800 0x7fc0940b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.327+0000 7fc092800640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc094157700 0x7fc0940aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091400640 1 -- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc094157700 msgr2=0x7fc0940aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091400640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc094157700 0x7fc0940aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091400640 1 -- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc078004800 msgr2=0x7fc0940b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091400640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc078004800 0x7fc0940b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091400640 1 -- 172.21.15.5:0/3219944391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094157b90 con 0x7fc0940c9c00 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091e00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc078004800 0x7fc0940b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc091400640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 0x7fc0941571c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc08000bb20 tx=0x7fc080004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc08a000640 1 -- 172.21.15.5:0/3219944391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080043020 con 0x7fc0940c9c00 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc08a000640 1 -- 172.21.15.5:0/3219944391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08003f680 con 0x7fc0940c9c00 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc08a000640 1 -- 172.21.15.5:0/3219944391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000b600 con 0x7fc0940c9c00 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.328+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0940af9b0 con 0x7fc0940c9c00 2024-04-25T10:10:37.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.329+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0940b4700 con 0x7fc0940c9c00 2024-04-25T10:10:37.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.331+0000 7fc08a000640 1 -- 172.21.15.5:0/3219944391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc080046020 con 0x7fc0940c9c00 2024-04-25T10:10:37.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.332+0000 7fc08a000640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b500 0x7fc07406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.332+0000 7fc08a000640 1 -- 172.21.15.5:0/3219944391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc0800ab160 con 0x7fc0940c9c00 2024-04-25T10:10:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.332+0000 7fc09b84f740 1 Processor -- start 2024-04-25T10:10:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 -- start start 2024-04-25T10:10:37.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc091e00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b500 0x7fc07406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a612750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a630160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 0x56073a62e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073a54aae0 con 0x56073a5e8050 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073a4f37c0 con 0x56073a612370 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.333+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56073a546b80 con 0x56073a62fc10 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 0x56073a62e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 0x56073a62e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57492/0 (socket says 172.21.15.5:57492) 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 -- 172.21.15.5:0/1561995379 learned_addr learned my addr 172.21.15.5:0/1561995379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc091e00640 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a612750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 -- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 msgr2=0x56073a630160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a630160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 -- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 msgr2=0x56073a612750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a612750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 -- 172.21.15.5:0/1561995379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073a32b7c0 con 0x56073a5e8050 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc091e00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b500 0x7fc07406d9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc084001050 tx=0x7fc084009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.334+0000 7fc092800640 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 0x56073a62e2d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000f80 tx=0x7fc08c0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c729358828516e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc07e600640 1 -- 172.21.15.5:0/1561995379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c003b80 con 0x56073a5e8050 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc07e600640 1 -- 172.21.15.5:0/1561995379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c018070 con 0x56073a5e8050 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc07e600640 1 -- 172.21.15.5:0/1561995379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00f8a0 con 0x56073a5e8050 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 -- 172.21.15.5:0/1561995379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 msgr2=0x56073a62e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 0x56073a62e2d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000f80 tx=0x7fc08c0020e0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 -- 172.21.15.5:0/1561995379 shutdown_connections 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a5e8050 0x56073a62e2d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a630160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1561995379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a612750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.335+0000 7fc09b84f740 1 -- 172.21.15.5:0/1561995379 >> 172.21.15.5:0/1561995379 conn(0x56073a5e4980 msgr2=0x56073a62d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.336+0000 7fc09b84f740 1 -- 172.21.15.5:0/1561995379 shutdown_connections 2024-04-25T10:10:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.336+0000 7fc09b84f740 1 -- 172.21.15.5:0/1561995379 wait complete. 2024-04-25T10:10:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.336+0000 7fc09b84f740 1 -- start start 2024-04-25T10:10:37.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.336+0000 7fc09b84f740 1 Processor -- start 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.337+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a626270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.337+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a624c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.337+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a63cff0 0x56073a6039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.337+0000 7fc09b84f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073a5e8df0 con 0x56073a612370 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.337+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073a4e8d40 con 0x56073a63cff0 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.337+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56073a5e28b0 con 0x56073a62fc10 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc09b84f740 10 client.0 _ll_register_callbacks cb 0x56073a25eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:37.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a626270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a626270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57506/0 (socket says 172.21.15.5:57506) 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 -- 172.21.15.5:0/1800496318 learned_addr learned my addr 172.21.15.5:0/1800496318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091400640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a624c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 -- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 msgr2=0x56073a624c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc092800640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a63cff0 0x56073a6039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a624c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 -- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a63cff0 msgr2=0x56073a6039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a63cff0 0x56073a6039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 -- 172.21.15.5:0/1800496318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56073a38bf50 con 0x56073a612370 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc091e00640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a626270 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc084004a40 tx=0x7fc084001c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704bcb8cbf13b931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.338+0000 7fc072a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc072a00640 1 -- 172.21.15.5:0/1800496318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08401c070 con 0x56073a612370 2024-04-25T10:10:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc072a00640 1 -- 172.21.15.5:0/1800496318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc084021d90 con 0x56073a612370 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc08c018070 con 0x56073a612370 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc072a00640 1 -- 172.21.15.5:0/1800496318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc084016670 con 0x56073a612370 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc09b84f740 10 client.8112 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x56073a21f0b0 con 0x56073a612370 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc072a00640 1 -- 172.21.15.5:0/1800496318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08401a020 con 0x56073a612370 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc072a00640 1 -- 172.21.15.5:0/1800496318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc084016810 con 0x56073a612370 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc09b84f740 10 client.8112 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.339+0000 7fc09b84f740 10 client.8112 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc09b84f740 20 client.8112 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a613ed0 0x56073a64aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc09b84f740 4 client.8112 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:37.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56073a5a9b60 con 0x56073a613ed0 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc091400640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a613ed0 0x56073a64aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc091400640 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a613ed0 0x56073a64aaa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a542ef93060a3e0 server_cookie=59685730bcc8a979 in_seq=0 out_seq=0 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.340+0000 7fc072a00640 10 client.8112 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc072a00640 1 -- 172.21.15.5:0/1800496318 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56073a5a9b60 con 0x56073a613ed0 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc072a00640 10 client.8112 handle_command_reply: tid=0 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 client.8112 shutdown 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 msgr2=0x56073a626270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a626270 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fc084004a40 tx=0x7fc084001c50 comp rx=0 tx=0).stop 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 shutdown_connections 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a613ed0 0x56073a64aaa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a63cff0 0x56073a6039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a62fc10 0x56073a624c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 --2- 172.21.15.5:0/1800496318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a626270 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 >> 172.21.15.5:0/1800496318 conn(0x56073a5e8050 msgr2=0x56073a5e16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.341+0000 7fc072a00640 0 client.8112 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.342+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 shutdown_connections 2024-04-25T10:10:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.342+0000 7fc09b84f740 1 -- 172.21.15.5:0/1800496318 wait complete. 2024-04-25T10:10:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.342+0000 7fc09b84f740 20 client.8112 trim_cache size 0 max 16384 2024-04-25T10:10:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.355+0000 7fc09b84f740 1 Processor -- start 2024-04-25T10:10:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.355+0000 7fc09b84f740 1 -- start start 2024-04-25T10:10:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a5e8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a62fc10 0x56073a6923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a6928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc09b84f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094200120 con 0x56073a612370 2024-04-25T10:10:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940b5ea0 con 0x56073a62fc10 2024-04-25T10:10:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0941ffb00 con 0x56073a63cff0 2024-04-25T10:10:37.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc091e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a5e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc091e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a5e8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57516/0 (socket says 172.21.15.5:57516) 2024-04-25T10:10:37.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc091e00640 1 -- 172.21.15.5:0/2834966117 learned_addr learned my addr 172.21.15.5:0/2834966117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:37.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc091400640 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a62fc10 0x56073a6923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.356+0000 7fc092800640 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a6928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc091e00640 1 -- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 msgr2=0x56073a6928f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc091e00640 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a6928f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc091e00640 1 -- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a62fc10 msgr2=0x56073a6923b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc091e00640 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a62fc10 0x56073a6923b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc091e00640 1 -- 172.21.15.5:0/2834966117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56073a627510 con 0x56073a612370 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc091e00640 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a5e8640 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fc084001cc0 tx=0x7fc08401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1682d320f3db4348 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc07dc00640 1 -- 172.21.15.5:0/2834966117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084030070 con 0x56073a612370 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc07dc00640 1 -- 172.21.15.5:0/2834966117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084031040 con 0x56073a612370 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.357+0000 7fc07dc00640 1 -- 172.21.15.5:0/2834966117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084016810 con 0x56073a612370 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 -- 172.21.15.5:0/2834966117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 msgr2=0x56073a5e8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a5e8640 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fc084001cc0 tx=0x7fc08401c000 comp rx=0 tx=0).stop 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 -- 172.21.15.5:0/2834966117 shutdown_connections 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a6928f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a62fc10 0x56073a6923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2834966117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a612370 0x56073a5e8640 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 -- 172.21.15.5:0/2834966117 >> 172.21.15.5:0/2834966117 conn(0x56073a5e8050 msgr2=0x56073a64eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 -- 172.21.15.5:0/2834966117 shutdown_connections 2024-04-25T10:10:37.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.358+0000 7fc09b84f740 1 -- 172.21.15.5:0/2834966117 wait complete. 2024-04-25T10:10:37.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.359+0000 7fc09b84f740 1 -- start start 2024-04-25T10:10:37.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.359+0000 7fc09b84f740 1 Processor -- start 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a6036c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 0x56073a63c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a68ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073a613d50 con 0x56073a62fc10 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073a615dc0 con 0x56073a612370 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56073a6142c0 con 0x56073a63cff0 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc09b84f740 10 client.0 _ll_register_callbacks cb 0x56073a5e2d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc091400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 0x56073a63c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc091400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 0x56073a63c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57530/0 (socket says 172.21.15.5:57530) 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.360+0000 7fc091400640 1 -- 172.21.15.5:0/2058470476 learned_addr learned my addr 172.21.15.5:0/2058470476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091e00640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a6036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc092800640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a68ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091400640 1 -- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 msgr2=0x56073a68ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091400640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a68ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091400640 1 -- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 msgr2=0x56073a6036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091400640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a6036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091400640 1 -- 172.21.15.5:0/2058470476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56073a6240e0 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091e00640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a6036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc091400640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 0x56073a63c620 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc080079290 tx=0x7fc08000b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31732018b4b4869a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc072a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc072a00640 1 -- 172.21.15.5:0/2058470476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080091070 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc072a00640 1 -- 172.21.15.5:0/2058470476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc080077890 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.361+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc084031040 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc072a00640 1 -- 172.21.15.5:0/2058470476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc080075c00 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 10 client.8118 fetch_fsmap learned FSMap version 75 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x56073a630ce0 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc072a00640 1 -- 172.21.15.5:0/2058470476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084031040 con 0x56073a62fc10 2024-04-25T10:10:37.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc072a00640 1 -- 172.21.15.5:0/2058470476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc080081810 con 0x56073a62fc10 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 10 client.8118 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 10 client.8118 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 20 client.8118 populate_metadata read hostname 'smithi005' 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a625ea0 0x56073a645430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 4 client.8118 mds_command: new command op to 7959 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 --> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x56073a558020 con 0x56073a625ea0 2024-04-25T10:10:37.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.362+0000 7fc091e00640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a625ea0 0x56073a645430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc091e00640 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a625ea0 0x56073a645430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b4652853ec13ec server_cookie=dd0bbb83665bf1bd in_seq=0 out_seq=0 2024-04-25T10:10:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc072a00640 10 client.8118 ms_handle_connect on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc072a00640 1 -- 172.21.15.5:0/2058470476 <== mds.0 v2:172.21.15.5:6832/841689019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56073a558020 con 0x56073a625ea0 2024-04-25T10:10:37.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc072a00640 10 client.8118 handle_command_reply: tid=0 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 client.8118 shutdown 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 msgr2=0x56073a63c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 0x56073a63c620 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc080079290 tx=0x7fc08000b7a0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 shutdown_connections 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56073a63cff0 0x56073a68ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56073a62fc10 0x56073a63c620 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.5:6832/841689019,v1:172.21.15.5:6834/841689019] conn(0x56073a625ea0 0x56073a645430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 --2- 172.21.15.5:0/2058470476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56073a612370 0x56073a6036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 >> 172.21.15.5:0/2058470476 conn(0x56073a5e8050 msgr2=0x56073a60eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.363+0000 7fc072a00640 0 client.8118 ms_handle_reset on v2:172.21.15.5:6832/841689019 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 shutdown_connections 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc09b84f740 1 -- 172.21.15.5:0/2058470476 wait complete. 2024-04-25T10:10:37.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc09b84f740 20 client.8118 trim_cache size 0 max 16384 2024-04-25T10:10:37.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:37.373 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b500 msgr2=0x7fc07406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc098a00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b500 0x7fc07406d9c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc084001050 tx=0x7fc084009040 comp rx=0 tx=0).stop 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 msgr2=0x7fc0941571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.364+0000 7fc098a00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 0x7fc0941571c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc08000bb20 tx=0x7fc080004730 comp rx=0 tx=0).stop 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 shutdown_connections 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc094157700 0x7fc0940aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940c9c00 0x7fc0941571c0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07406b500 0x7fc07406d9c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 --2- 172.21.15.5:0/3219944391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc078004800 0x7fc0940b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 >> 172.21.15.5:0/3219944391 conn(0x7fc09408af80 msgr2=0x7fc094159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 shutdown_connections 2024-04-25T10:10:37.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.365+0000 7fc098a00640 1 -- 172.21.15.5:0/3219944391 wait complete. 2024-04-25T10:10:37.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:10:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.586+0000 7f573d400640 1 Processor -- start 2024-04-25T10:10:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.587+0000 7f573d400640 1 -- start start 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.588+0000 7f573d400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5738071100 0x7f57380714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.588+0000 7f573d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f573815d8a0 0x7f57380cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.588+0000 7f573d400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5738071a20 con 0x7f57380cd2c0 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f573d400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57380cdd80 con 0x7f573815d8a0 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f573d400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57380ce060 con 0x7f5738071100 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f572de00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f57380cd2c0 0x7f573806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46008/0 (socket says 172.21.15.5:46008) 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f572de00640 1 -- 172.21.15.5:0/2040897574 learned_addr learned my addr 172.21.15.5:0/2040897574 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5737200640 1 --2- 172.21.15.5:0/2040897574 >> v2:172.21.15.5:3300/0 conn(0x7f573815d8a0 0x7f57380cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5736800640 1 --2- 172.21.15.5:0/2040897574 >> v2:172.21.15.80:3300/0 conn(0x7f5738071100 0x7f57380714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5735e00640 1 -- 172.21.15.5:0/2040897574 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521142908 0 0) 0x7f5738071a20 con 0x7f57380cd2c0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5735e00640 1 -- 172.21.15.5:0/2040897574 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c003520 con 0x7f57380cd2c0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5737200640 1 -- 172.21.15.5:0/2040897574 >> v2:172.21.15.80:3300/0 conn(0x7f5738071100 msgr2=0x7f57380714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5737200640 1 --2- 172.21.15.5:0/2040897574 >> v2:172.21.15.80:3300/0 conn(0x7f5738071100 0x7f57380714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5737200640 1 -- 172.21.15.5:0/2040897574 >> v1:172.21.15.5:6789/0 conn(0x7f57380cd2c0 legacy=0x7f573806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.589+0000 7f5737200640 1 -- 172.21.15.5:0/2040897574 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573815c090 con 0x7f573815d8a0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.590+0000 7f5737200640 1 --2- 172.21.15.5:0/2040897574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f573815d8a0 0x7f57380cd740 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5738072310 tx=0x7f573002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6761438627acc905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.591+0000 7f5735e00640 1 -- 172.21.15.5:0/2040897574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730007740 con 0x7f573815d8a0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.591+0000 7f5735e00640 1 -- 172.21.15.5:0/2040897574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57300078e0 con 0x7f573815d8a0 2024-04-25T10:10:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.591+0000 7f5735e00640 1 -- 172.21.15.5:0/2040897574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730007cb0 con 0x7f573815d8a0 2024-04-25T10:10:37.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.591+0000 7f573d400640 1 -- 172.21.15.5:0/2040897574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f573815d8a0 msgr2=0x7f57380cd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.591+0000 7f573d400640 1 --2- 172.21.15.5:0/2040897574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f573815d8a0 0x7f57380cd740 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5738072310 tx=0x7f573002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.592+0000 7f573d400640 1 -- 172.21.15.5:0/2040897574 shutdown_connections 2024-04-25T10:10:37.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.592+0000 7f573d400640 1 --2- 172.21.15.5:0/2040897574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f573815d8a0 0x7f57380cd740 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.592+0000 7f573d400640 1 --2- 172.21.15.5:0/2040897574 >> v2:172.21.15.80:3300/0 conn(0x7f5738071100 0x7f57380714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.592+0000 7f573d400640 1 -- 172.21.15.5:0/2040897574 >> 172.21.15.5:0/2040897574 conn(0x7f573806ab60 msgr2=0x7f573806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.592+0000 7f573d400640 1 -- 172.21.15.5:0/2040897574 shutdown_connections 2024-04-25T10:10:37.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.592+0000 7f573d400640 1 -- 172.21.15.5:0/2040897574 wait complete. 2024-04-25T10:10:37.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.594+0000 7f573d400640 1 Processor -- start 2024-04-25T10:10:37.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.594+0000 7f573d400640 1 -- start start 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.595+0000 7f573d400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5738071100 0x7f573813f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.595+0000 7f573d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 0x7f57381e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.595+0000 7f573d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f573815d8a0 0x7f57381e5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.595+0000 7f573d400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380cc260 con 0x7f57380cd2c0 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.595+0000 7f573d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573815bd80 con 0x7f573815d8a0 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.595+0000 7f573d400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5738145270 con 0x7f5738071100 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 0x7f57381e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 0x7f57381e4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57548/0 (socket says 172.21.15.5:57548) 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 -- 172.21.15.5:0/3511676577 learned_addr learned my addr 172.21.15.5:0/3511676577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f5736800640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5738071100 0x7f573813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 -- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5738071100 msgr2=0x7f573813f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5738071100 0x7f573813f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 -- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f573815d8a0 msgr2=0x7f57381e5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f573815d8a0 0x7f57381e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 -- 172.21.15.5:0/3511676577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57381f7790 con 0x7f57380cd2c0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.596+0000 7f572de00640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 0x7f57381e4c30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f572000c390 tx=0x7f572000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.597+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720017070 con 0x7f57380cd2c0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.597+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5720005ce0 con 0x7f57380cd2c0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.597+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57200072f0 con 0x7f57380cd2c0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.597+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57381f33a0 con 0x7f57380cd2c0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.597+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57381f3960 con 0x7f57380cd2c0 2024-04-25T10:10:37.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.598+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5738074bb0 con 0x7f57380cd2c0 2024-04-25T10:10:37.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.599+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f572001a070 con 0x7f57380cd2c0 2024-04-25T10:10:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.600+0000 7f572f400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f571406b4d0 0x7f571406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.600+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f5720089a00 con 0x7f57380cd2c0 2024-04-25T10:10:37.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.600+0000 7f5736800640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f571406b4d0 0x7f571406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.602+0000 7f5736800640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f571406b4d0 0x7f571406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5728006fd0 tx=0x7f5728008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.604+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57200584a0 con 0x7f57380cd2c0 2024-04-25T10:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.832+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f57381f3d90 con 0x7f57380cd2c0 2024-04-25T10:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.833+0000 7f572f400640 1 -- 172.21.15.5:0/3511676577 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5720057da0 con 0x7f57380cd2c0 2024-04-25T10:10:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:37.839 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f571406b4d0 msgr2=0x7f571406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f571406b4d0 0x7f571406d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5728006fd0 tx=0x7f5728008040 comp rx=0 tx=0).stop 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 msgr2=0x7f57381e4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 0x7f57381e4c30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f572000c390 tx=0x7f572000c850 comp rx=0 tx=0).stop 2024-04-25T10:10:37.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 shutdown_connections 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f571406b4d0 0x7f571406d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f573815d8a0 0x7f57381e5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57380cd2c0 0x7f57381e4c30 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 --2- 172.21.15.5:0/3511676577 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5738071100 0x7f573813f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 >> 172.21.15.5:0/3511676577 conn(0x7f573806ab60 msgr2=0x7f57380714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 shutdown_connections 2024-04-25T10:10:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:37.836+0000 7f573d400640 1 -- 172.21.15.5:0/3511676577 wait complete. 2024-04-25T10:10:37.859 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:10:37.859 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:37.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:37.955 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:38.073 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:38.073 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:10:38.073 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:38.130 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:10:38.130 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:38.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.388+0000 7f5e89400640 1 Processor -- start 2024-04-25T10:10:38.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.389+0000 7f5e89400640 1 -- start start 2024-04-25T10:10:38.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.390+0000 7f5e89400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5e841652f0 0x7f5e840cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.390+0000 7f5e89400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 0x7f5e8406cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.390+0000 7f5e89400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e840cd2a0 con 0x7f5e8406e980 2024-04-25T10:10:38.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.390+0000 7f5e89400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e840ce340 con 0x7f5e841652f0 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.390+0000 7f5e89400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cdeb0 con 0x7f5e840716a0 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.391+0000 7f5e83400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5e8406e980 0x7f5e84070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46020/0 (socket says 172.21.15.5:46020) 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.391+0000 7f5e83400640 1 -- 172.21.15.5:0/2017016940 learned_addr learned my addr 172.21.15.5:0/2017016940 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.391+0000 7f5e88a00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 0x7f5e8406cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.391+0000 7f5e83e00640 1 --2- 172.21.15.5:0/2017016940 >> v2:172.21.15.80:3300/0 conn(0x7f5e841652f0 0x7f5e840cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.391+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579182901 0 0) 0x7f5e840cd2a0 con 0x7f5e8406e980 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.391+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e68003520 con 0x7f5e8406e980 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1849774712 0 0) 0x7f5e68003520 con 0x7f5e8406e980 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 msgr2=0x7f5e8406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 0x7f5e8406cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 >> v2:172.21.15.80:3300/0 conn(0x7f5e841652f0 msgr2=0x7f5e840cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 --2- 172.21.15.5:0/2017016940 >> v2:172.21.15.80:3300/0 conn(0x7f5e841652f0 0x7f5e840cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840cd580 con 0x7f5e8406e980 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f5e70001670 con 0x7f5e8406e980 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 >> v1:172.21.15.5:6789/0 conn(0x7f5e8406e980 legacy=0x7f5e84070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68004800 0x7f5e68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 0x7f5e680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840ce340 con 0x7f5e68004800 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cdeb0 con 0x7f5e680039b0 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6800b9c0 con 0x7f5e840716a0 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e83400640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68004800 0x7f5e68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.392+0000 7f5e88a00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 0x7f5e680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83e00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83400640 1 -- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 msgr2=0x7f5e680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83400640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e840716a0 0x7f5e680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83400640 1 -- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e680039b0 msgr2=0x7f5e68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83400640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83400640 1 -- 172.21.15.5:0/2017016940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68003520 con 0x7f5e68004800 2024-04-25T10:10:38.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.393+0000 7f5e83e00640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e83400640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68004800 0x7f5e68004c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c160 tx=0x7f5e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511d5b6e7847533e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70007c80 con 0x7f5e68004800 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70005ad0 con 0x7f5e68004800 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e82a00640 1 -- 172.21.15.5:0/2017016940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70007070 con 0x7f5e68004800 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e89400640 1 -- 172.21.15.5:0/2017016940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68004800 msgr2=0x7f5e68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e89400640 1 --2- 172.21.15.5:0/2017016940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e68004800 0x7f5e68004c80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000c160 tx=0x7f5e68005830 comp rx=0 tx=0).stop 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.394+0000 7f5e88a00640 1 -- 172.21.15.5:0/2017016940 reap_dead start 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.395+0000 7f5e89400640 1 -- 172.21.15.5:0/2017016940 shutdown_connections 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.395+0000 7f5e89400640 1 -- 172.21.15.5:0/2017016940 >> 172.21.15.5:0/2017016940 conn(0x7f5e84066f20 msgr2=0x7f5e8406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.395+0000 7f5e89400640 1 -- 172.21.15.5:0/2017016940 shutdown_connections 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.395+0000 7f5e89400640 1 -- 172.21.15.5:0/2017016940 wait complete. 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.396+0000 7f5e89400640 1 Processor -- start 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.397+0000 7f5e89400640 1 -- start start 2024-04-25T10:10:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e89400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e68004800 0x7f5e8414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e89400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e840c4320 0x7f5e840c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e89400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 0x7f5e8415efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e89400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cc8d0 con 0x7f5e8415eb60 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e89400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e840cdb40 con 0x7f5e840c4320 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e89400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8406cff0 con 0x7f5e68004800 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 0x7f5e8415efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 0x7f5e8415efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57558/0 (socket says 172.21.15.5:57558) 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 -- 172.21.15.5:0/3194296403 learned_addr learned my addr 172.21.15.5:0/3194296403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e83e00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e68004800 0x7f5e8414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e83400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e840c4320 0x7f5e840c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 -- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e68004800 msgr2=0x7f5e8414f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e68004800 0x7f5e8414f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 -- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e840c4320 msgr2=0x7f5e840c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.398+0000 7f5e88a00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e840c4320 0x7f5e840c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e88a00640 1 -- 172.21.15.5:0/3194296403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840c4de0 con 0x7f5e8415eb60 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e88a00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 0x7f5e8415efe0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5e840722d0 tx=0x7f5e7c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c011e10 con 0x7f5e8415eb60 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c01b070 con 0x7f5e8415eb60 2024-04-25T10:10:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c003990 con 0x7f5e8415eb60 2024-04-25T10:10:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e840c50c0 con 0x7f5e8415eb60 2024-04-25T10:10:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.399+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e840c5680 con 0x7f5e8415eb60 2024-04-25T10:10:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.401+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84068f60 con 0x7f5e8415eb60 2024-04-25T10:10:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.401+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e7c005d10 con 0x7f5e8415eb60 2024-04-25T10:10:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.402+0000 7f5e7be00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e6406b500 0x7f5e6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.402+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f5e7c08aa30 con 0x7f5e8415eb60 2024-04-25T10:10:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.403+0000 7f5e83e00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e6406b500 0x7f5e6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.404+0000 7f5e83e00640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e6406b500 0x7f5e6406d9c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009b20 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.407+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e7c025020 con 0x7f5e8415eb60 2024-04-25T10:10:38.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.612+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e840cce40 con 0x7f5e8415eb60 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.616+0000 7f5e7be00640 1 -- 172.21.15.5:0/3194296403 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17447 (secure 0 0 0) 0x7f5e7c014350 con 0x7f5e8415eb60 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":93,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:57.322201+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.20","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.28","mappings":[{"from":1,"to":3}]},{"pgid":"14.30","mappings":[{"from":7,"to":0}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6832/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6834/3627376661":"2024-04-26T10:09:12.513203+0000","172.21.15.5:6832/3627376661":"2024-04-26T10:09:12.513203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.618+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e6406b500 msgr2=0x7f5e6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.618+0000 7f5e89400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e6406b500 0x7f5e6406d9c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009b20 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 msgr2=0x7f5e8415efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 0x7f5e8415efe0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f5e840722d0 tx=0x7f5e7c0115a0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 shutdown_connections 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8415eb60 0x7f5e8415efe0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e6406b500 0x7f5e6406d9c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e840c4320 0x7f5e840c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 --2- 172.21.15.5:0/3194296403 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e68004800 0x7f5e8414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 >> 172.21.15.5:0/3194296403 conn(0x7f5e84066f20 msgr2=0x7f5e8406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 shutdown_connections 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.619+0000 7f5e89400640 1 -- 172.21.15.5:0/3194296403 wait complete. 2024-04-25T10:10:38.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:57.371047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:58.551736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:38.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:38.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.854+0000 7f0f60800640 1 Processor -- start 2024-04-25T10:10:38.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.855+0000 7f0f60800640 1 -- start start 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.856+0000 7f0f60800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58144fe0 0x7f0f5806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.856+0000 7f0f60800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f5806d130 0x7f0f58071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.856+0000 7f0f60800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f581453c0 con 0x7f0f58071670 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.856+0000 7f0f60800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580a8560 con 0x7f0f5806d130 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.856+0000 7f0f60800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5806d6a0 con 0x7f0f58144fe0 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f5806d130 0x7f0f58071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f5806d130 0x7f0f58071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57570/0 (socket says 172.21.15.5:57570) 2024-04-25T10:10:38.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 -- 172.21.15.5:0/3418942993 learned_addr learned my addr 172.21.15.5:0/3418942993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3418942993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58144fe0 0x7f0f5806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 -- 172.21.15.5:0/3418942993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58144fe0 msgr2=0x7f0f5806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3418942993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58144fe0 0x7f0f5806e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 -- 172.21.15.5:0/3418942993 >> v1:172.21.15.80:6789/0 conn(0x7f0f58071670 legacy=0x7f0f580cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.857+0000 7f0f5d400640 1 -- 172.21.15.5:0/3418942993 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5806d980 con 0x7f0f5806d130 2024-04-25T10:10:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.858+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3418942993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58071130 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009870 tx=0x7f0f4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14cd0280ebb669e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.859+0000 7f0f5ca00640 1 -- 172.21.15.5:0/3418942993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48037070 con 0x7f0f5806d130 2024-04-25T10:10:38.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.859+0000 7f0f5ca00640 1 -- 172.21.15.5:0/3418942993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48032440 con 0x7f0f5806d130 2024-04-25T10:10:38.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.859+0000 7f0f5ca00640 1 -- 172.21.15.5:0/3418942993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48032810 con 0x7f0f5806d130 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 -- 172.21.15.5:0/3418942993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 msgr2=0x7f0f58071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 --2- 172.21.15.5:0/3418942993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58071130 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009870 tx=0x7f0f4802a210 comp rx=0 tx=0).stop 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 -- 172.21.15.5:0/3418942993 shutdown_connections 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 --2- 172.21.15.5:0/3418942993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58071130 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 --2- 172.21.15.5:0/3418942993 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58144fe0 0x7f0f5806e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 -- 172.21.15.5:0/3418942993 >> 172.21.15.5:0/3418942993 conn(0x7f0f5806ab60 msgr2=0x7f0f5806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 -- 172.21.15.5:0/3418942993 shutdown_connections 2024-04-25T10:10:38.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.860+0000 7f0f60800640 1 -- 172.21.15.5:0/3418942993 wait complete. 2024-04-25T10:10:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.862+0000 7f0f60800640 1 Processor -- start 2024-04-25T10:10:38.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.863+0000 7f0f60800640 1 -- start start 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f60800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f60800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58071670 0x7f0f581e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f60800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f58144fe0 0x7f0f58143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f60800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815be50 con 0x7f0f5806d130 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f60800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5806df40 con 0x7f0f58144fe0 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f60800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5806eaa0 con 0x7f0f58071670 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57578/0 (socket says 172.21.15.5:57578) 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f5de00640 1 -- 172.21.15.5:0/1653122057 learned_addr learned my addr 172.21.15.5:0/1653122057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f5d400640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58071670 0x7f0f581e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.864+0000 7f0f5e800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f58144fe0 0x7f0f58143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5de00640 1 -- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58071670 msgr2=0x7f0f581e4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5de00640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58071670 0x7f0f581e4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5de00640 1 -- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f58144fe0 msgr2=0x7f0f58143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5de00640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f58144fe0 0x7f0f58143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5de00640 1 -- 172.21.15.5:0/1653122057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f581e57a0 con 0x7f0f5806d130 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5e800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f58144fe0 0x7f0f58143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5d400640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58071670 0x7f0f581e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f5de00640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58134160 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d5f0 tx=0x7f0f4c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0125f0 con 0x7f0f5806d130 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c019070 con 0x7f0f5806d130 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581e5a80 con 0x7f0f5806d130 2024-04-25T10:10:38.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0129d0 con 0x7f0f5806d130 2024-04-25T10:10:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.865+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581e6040 con 0x7f0f5806d130 2024-04-25T10:10:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.867+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f58068f60 con 0x7f0f5806d130 2024-04-25T10:10:38.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.868+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f4c012b70 con 0x7f0f5806d130 2024-04-25T10:10:38.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.869+0000 7f0f52000640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3c06b4d0 0x7f0f3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.869+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f0f4c08a9f0 con 0x7f0f5806d130 2024-04-25T10:10:38.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.869+0000 7f0f5d400640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3c06b4d0 0x7f0f3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.870+0000 7f0f5d400640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3c06b4d0 0x7f0f3c06d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0f48034000 tx=0x7f0f4802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:38.873+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f4c059490 con 0x7f0f5806d130 2024-04-25T10:10:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.073+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f580adbb0 con 0x7f0f5806d130 2024-04-25T10:10:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.074+0000 7f0f52000640 1 -- 172.21.15.5:0/1653122057 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f4c058d90 con 0x7f0f5806d130 2024-04-25T10:10:39.080 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3c06b4d0 msgr2=0x7f0f3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3c06b4d0 0x7f0f3c06d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0f48034000 tx=0x7f0f4802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 msgr2=0x7f0f58134160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58134160 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d5f0 tx=0x7f0f4c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 shutdown_connections 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0f3c06b4d0 0x7f0f3c06d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f58144fe0 0x7f0f58143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0f58071670 0x7f0f581e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 --2- 172.21.15.5:0/1653122057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f5806d130 0x7f0f58134160 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 >> 172.21.15.5:0/1653122057 conn(0x7f0f5806ab60 msgr2=0x7f0f580ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.077+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 shutdown_connections 2024-04-25T10:10:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.078+0000 7f0f60800640 1 -- 172.21.15.5:0/1653122057 wait complete. 2024-04-25T10:10:39.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.315+0000 7f0378800640 1 Processor -- start 2024-04-25T10:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.315+0000 7f0378800640 1 -- start start 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0378800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0370161c30 0x7f03700b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0378800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0370162010 0x7f0370162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0378800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03700b2650 0x7f0370071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0378800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03700b4bd0 con 0x7f03700b2650 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0378800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03701632e0 con 0x7f0370162010 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0378800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03700cd4c0 con 0x7f0370161c30 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.317+0000 7f0376800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03700b2650 0x7f0370071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03700b2650 0x7f0370071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57588/0 (socket says 172.21.15.5:57588) 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 -- 172.21.15.5:0/1370939835 learned_addr learned my addr 172.21.15.5:0/1370939835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0375e00640 1 --2- 172.21.15.5:0/1370939835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0370161c30 0x7f03700b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0375400640 1 --2- 172.21.15.5:0/1370939835 >> v2:172.21.15.80:3300/0 conn(0x7f0370162010 0x7f0370162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 -- 172.21.15.5:0/1370939835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0370161c30 msgr2=0x7f03700b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 --2- 172.21.15.5:0/1370939835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0370161c30 0x7f03700b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 -- 172.21.15.5:0/1370939835 >> v2:172.21.15.80:3300/0 conn(0x7f0370162010 msgr2=0x7f0370162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 --2- 172.21.15.5:0/1370939835 >> v2:172.21.15.80:3300/0 conn(0x7f0370162010 0x7f0370162da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.318+0000 7f0376800640 1 -- 172.21.15.5:0/1370939835 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03700cd760 con 0x7f03700b2650 2024-04-25T10:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.320+0000 7f0376800640 1 --2- 172.21.15.5:0/1370939835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03700b2650 0x7f0370071130 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f037006b910 tx=0x7f036c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf03fe6de25d3aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.320+0000 7f0374a00640 1 -- 172.21.15.5:0/1370939835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c036070 con 0x7f03700b2650 2024-04-25T10:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.320+0000 7f0374a00640 1 -- 172.21.15.5:0/1370939835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c031600 con 0x7f03700b2650 2024-04-25T10:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.320+0000 7f0374a00640 1 -- 172.21.15.5:0/1370939835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c0319d0 con 0x7f03700b2650 2024-04-25T10:10:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 -- 172.21.15.5:0/1370939835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03700b2650 msgr2=0x7f0370071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 --2- 172.21.15.5:0/1370939835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03700b2650 0x7f0370071130 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f037006b910 tx=0x7f036c02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 -- 172.21.15.5:0/1370939835 shutdown_connections 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 --2- 172.21.15.5:0/1370939835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03700b2650 0x7f0370071130 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 --2- 172.21.15.5:0/1370939835 >> v2:172.21.15.80:3300/0 conn(0x7f0370162010 0x7f0370162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 --2- 172.21.15.5:0/1370939835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0370161c30 0x7f03700b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 -- 172.21.15.5:0/1370939835 >> 172.21.15.5:0/1370939835 conn(0x7f037006ab60 msgr2=0x7f037006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 -- 172.21.15.5:0/1370939835 shutdown_connections 2024-04-25T10:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.321+0000 7f0378800640 1 -- 172.21.15.5:0/1370939835 wait complete. 2024-04-25T10:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.323+0000 7f0378800640 1 Processor -- start 2024-04-25T10:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.323+0000 7f0378800640 1 -- start start 2024-04-25T10:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0378800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03700b2650 0x7f03701486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0378800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 0x7f03701e8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0378800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0370162010 0x7f037015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0378800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03700cda00 con 0x7f0370161c30 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0378800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03700b46a0 con 0x7f0370162010 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0378800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03700b2ae0 con 0x7f03700b2650 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0375400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 0x7f03701e8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0375400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 0x7f03701e8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57592/0 (socket says 172.21.15.5:57592) 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0375400640 1 -- 172.21.15.5:0/2532156959 learned_addr learned my addr 172.21.15.5:0/2532156959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.324+0000 7f0375e00640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03700b2650 0x7f03701486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0376800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0370162010 0x7f037015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0375400640 1 -- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03700b2650 msgr2=0x7f03701486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0375400640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03700b2650 0x7f03701486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0375400640 1 -- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0370162010 msgr2=0x7f037015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0375400640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0370162010 0x7f037015f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0375400640 1 -- 172.21.15.5:0/2532156959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037015cee0 con 0x7f0370161c30 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0376800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0370162010 0x7f037015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f0375400640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 0x7f03701e8bc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f036000d5f0 tx=0x7f036000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.325+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600125f0 con 0x7f0370161c30 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.326+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360019070 con 0x7f0370161c30 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.326+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600129d0 con 0x7f0370161c30 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.326+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037015d1c0 con 0x7f0370161c30 2024-04-25T10:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.326+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037015da60 con 0x7f0370161c30 2024-04-25T10:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.327+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037015d5b0 con 0x7f0370161c30 2024-04-25T10:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.328+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0360012b70 con 0x7f0370161c30 2024-04-25T10:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.329+0000 7f036a000640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f035406b500 0x7f035406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.329+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f036008a980 con 0x7f0370161c30 2024-04-25T10:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.330+0000 7f0375e00640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f035406b500 0x7f035406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.331+0000 7f0375e00640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f035406b500 0x7f035406d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0364006fd0 tx=0x7f0364008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.334+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0360059420 con 0x7f0370161c30 2024-04-25T10:10:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.530+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0370068f60 con 0x7f0370161c30 2024-04-25T10:10:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.534+0000 7f036a000640 1 -- 172.21.15.5:0/2532156959 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0360058d20 con 0x7f0370161c30 2024-04-25T10:10:39.540 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:10:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.536+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f035406b500 msgr2=0x7f035406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f035406b500 0x7f035406d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f0364006fd0 tx=0x7f0364008040 comp rx=0 tx=0).stop 2024-04-25T10:10:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 msgr2=0x7f03701e8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 0x7f03701e8bc0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f036000d5f0 tx=0x7f036000dab0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 shutdown_connections 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f035406b500 0x7f035406d9c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0370162010 0x7f037015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0370161c30 0x7f03701e8bc0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 --2- 172.21.15.5:0/2532156959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f03700b2650 0x7f03701486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 >> 172.21.15.5:0/2532156959 conn(0x7f037006ab60 msgr2=0x7f037006af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 shutdown_connections 2024-04-25T10:10:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.537+0000 7f0378800640 1 -- 172.21.15.5:0/2532156959 wait complete. 2024-04-25T10:10:39.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.767+0000 7f2a6b200640 1 Processor -- start 2024-04-25T10:10:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.768+0000 7f2a6b200640 1 -- start start 2024-04-25T10:10:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.769+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a6c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.769+0000 7f2a6b200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2a6c161d90 0x7f2a6c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.769+0000 7f2a6b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c1606e0 con 0x7f2a6c0d3f90 2024-04-25T10:10:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.769+0000 7f2a6b200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6c1609c0 con 0x7f2a6c161d90 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.769+0000 7f2a6b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c160ca0 con 0x7f2a6c0b48c0 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.770+0000 7f2a69400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a6c0d3f90 0x7f2a6c161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46030/0 (socket says 172.21.15.5:46030) 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.770+0000 7f2a69400640 1 -- 172.21.15.5:0/1988813646 learned_addr learned my addr 172.21.15.5:0/1988813646 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.770+0000 7f2a6a800640 1 --2- 172.21.15.5:0/1988813646 >> v2:172.21.15.80:3300/0 conn(0x7f2a6c161d90 0x7f2a6c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.770+0000 7f2a69e00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a6c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.770+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803102270 0 0) 0x7f2a6c1606e0 con 0x7f2a6c0d3f90 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.770+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c003520 con 0x7f2a6c0d3f90 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2885136250 0 0) 0x7f2a4c003520 con 0x7f2a6c0d3f90 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 msgr2=0x7f2a6c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a6c15fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 >> v2:172.21.15.80:3300/0 conn(0x7f2a6c161d90 msgr2=0x7f2a6c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 --2- 172.21.15.5:0/1988813646 >> v2:172.21.15.80:3300/0 conn(0x7f2a6c161d90 0x7f2a6c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c160f80 con 0x7f2a6c0d3f90 2024-04-25T10:10:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2a54001b70 con 0x7f2a6c0d3f90 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 >> v1:172.21.15.5:6789/0 conn(0x7f2a6c0d3f90 legacy=0x7f2a6c161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a4c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003f30 0x7f2a4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c004d40 0x7f2a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c1609c0 con 0x7f2a4c003f30 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c160ca0 con 0x7f2a4c004d40 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c00ba10 con 0x7f2a6c0b48c0 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a69400640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003f30 0x7f2a4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.771+0000 7f2a6a800640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c004d40 0x7f2a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69e00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a4c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69400640 1 -- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 msgr2=0x7f2a4c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69400640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a4c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69400640 1 -- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c004d40 msgr2=0x7f2a4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69400640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c004d40 0x7f2a4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69400640 1 -- 172.21.15.5:0/1988813646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c003520 con 0x7f2a4c003f30 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a6a800640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c004d40 0x7f2a4c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.772+0000 7f2a69e00640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0b48c0 0x7f2a4c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.773+0000 7f2a69400640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003f30 0x7f2a4c004800 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c590 tx=0x7f2a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b60e2d3be26c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.773+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a54030070 con 0x7f2a4c003f30 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.773+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a54005e00 con 0x7f2a4c003f30 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.773+0000 7f2a68a00640 1 -- 172.21.15.5:0/1988813646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400c180 con 0x7f2a4c003f30 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6b200640 1 -- 172.21.15.5:0/1988813646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003f30 msgr2=0x7f2a4c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1988813646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003f30 0x7f2a4c004800 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400c590 tx=0x7f2a4c005830 comp rx=0 tx=0).stop 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6a800640 1 -- 172.21.15.5:0/1988813646 reap_dead start 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6b200640 1 -- 172.21.15.5:0/1988813646 shutdown_connections 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6b200640 1 -- 172.21.15.5:0/1988813646 >> 172.21.15.5:0/1988813646 conn(0x7f2a6c08af80 msgr2=0x7f2a6c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6b200640 1 -- 172.21.15.5:0/1988813646 shutdown_connections 2024-04-25T10:10:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.774+0000 7f2a6b200640 1 -- 172.21.15.5:0/1988813646 wait complete. 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.775+0000 7f2a6b200640 1 Processor -- start 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.776+0000 7f2a6b200640 1 -- start start 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.776+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c003f30 0x7f2a6c0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.776+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0c6ff0 0x7f2a6c0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 0x7f2a6c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c15fe60 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c0b4ca0 con 0x7f2a4c003f30 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c15ffe0 con 0x7f2a6c0c6ff0 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 0x7f2a6c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 0x7f2a6c0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57610/0 (socket says 172.21.15.5:57610) 2024-04-25T10:10:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 -- 172.21.15.5:0/1036099663 learned_addr learned my addr 172.21.15.5:0/1036099663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a69400640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0c6ff0 0x7f2a6c0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 -- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0c6ff0 msgr2=0x7f2a6c0c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a69e00640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c003f30 0x7f2a6c0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0c6ff0 0x7f2a6c0c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 -- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c003f30 msgr2=0x7f2a6c0c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c003f30 0x7f2a6c0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.777+0000 7f2a6a800640 1 -- 172.21.15.5:0/1036099663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c147750 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.778+0000 7f2a6a800640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 0x7f2a6c0cce60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2a64010790 tx=0x7f2a64010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.778+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400f040 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.778+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6400e870 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.778+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400ec40 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.778+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c147a30 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.778+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c147ff0 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.779+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c068f60 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.780+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a6401c400 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.781+0000 7f2a62000640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a4806b4d0 0x7f2a4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.781+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f2a6408a880 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.782+0000 7f2a69e00640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a4806b4d0 0x7f2a4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.783+0000 7f2a69e00640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a4806b4d0 0x7f2a4806d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098f0 tx=0x7f2a5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.786+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a64018030 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.983+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a6c0d0dd0 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.984+0000 7f2a62000640 1 -- 172.21.15.5:0/1036099663 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a64058d60 con 0x7f2a6c0cc9e0 2024-04-25T10:10:39.990 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a4806b4d0 msgr2=0x7f2a4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a4806b4d0 0x7f2a4806d990 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0098f0 tx=0x7f2a5c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 msgr2=0x7f2a6c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 0x7f2a6c0cce60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2a64010790 tx=0x7f2a64010c50 comp rx=0 tx=0).stop 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 shutdown_connections 2024-04-25T10:10:39.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a6c0cc9e0 0x7f2a6c0cce60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a6c0c6ff0 0x7f2a6c0c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a4806b4d0 0x7f2a4806d990 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 --2- 172.21.15.5:0/1036099663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a4c003f30 0x7f2a6c0c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 >> 172.21.15.5:0/1036099663 conn(0x7f2a6c08af80 msgr2=0x7f2a6c154bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 shutdown_connections 2024-04-25T10:10:39.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:39.988+0000 7f2a6b200640 1 -- 172.21.15.5:0/1036099663 wait complete. 2024-04-25T10:10:40.014 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.211+0000 7f378e200640 1 Processor -- start 2024-04-25T10:10:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.211+0000 7f378e200640 1 -- start start 2024-04-25T10:10:40.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.212+0000 7f378e200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3788161200 0x7f37880b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.212+0000 7f378e200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f37880d2380 0x7f37880d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.212+0000 7f378e200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37880cf410 con 0x7f37880d0600 2024-04-25T10:10:40.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.212+0000 7f378e200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880d27c0 con 0x7f37880d2380 2024-04-25T10:10:40.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.212+0000 7f378e200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880d0a70 con 0x7f3788161200 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f378d800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f37880d0600 0x7f37880ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46042/0 (socket says 172.21.15.5:46042) 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f378d800640 1 -- 172.21.15.5:0/57221911 learned_addr learned my addr 172.21.15.5:0/57221911 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787e00640 1 --2- 172.21.15.5:0/57221911 >> v2:172.21.15.5:3300/0 conn(0x7f37880d2380 0x7f37880d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f378ce00640 1 --2- 172.21.15.5:0/57221911 >> v2:172.21.15.80:3300/0 conn(0x7f3788161200 0x7f37880b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787400640 1 -- 172.21.15.5:0/57221911 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433317198 0 0) 0x7f37880cf410 con 0x7f37880d0600 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787400640 1 -- 172.21.15.5:0/57221911 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f376c003520 con 0x7f37880d0600 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787e00640 1 -- 172.21.15.5:0/57221911 >> v2:172.21.15.80:3300/0 conn(0x7f3788161200 msgr2=0x7f37880b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787e00640 1 --2- 172.21.15.5:0/57221911 >> v2:172.21.15.80:3300/0 conn(0x7f3788161200 0x7f37880b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787e00640 1 -- 172.21.15.5:0/57221911 >> v1:172.21.15.5:6789/0 conn(0x7f37880d0600 legacy=0x7f37880ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.213+0000 7f3787e00640 1 -- 172.21.15.5:0/57221911 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880cf6f0 con 0x7f37880d2380 2024-04-25T10:10:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.215+0000 7f3787e00640 1 --2- 172.21.15.5:0/57221911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37880d2380 0x7f37880d00c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f378808bed0 tx=0x7f377002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be97e50bd2202a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.215+0000 7f3787400640 1 -- 172.21.15.5:0/57221911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3770037070 con 0x7f37880d2380 2024-04-25T10:10:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.215+0000 7f3787400640 1 -- 172.21.15.5:0/57221911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3770032440 con 0x7f37880d2380 2024-04-25T10:10:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.215+0000 7f3787400640 1 -- 172.21.15.5:0/57221911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37700327f0 con 0x7f37880d2380 2024-04-25T10:10:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 -- 172.21.15.5:0/57221911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37880d2380 msgr2=0x7f37880d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 --2- 172.21.15.5:0/57221911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37880d2380 0x7f37880d00c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f378808bed0 tx=0x7f377002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 -- 172.21.15.5:0/57221911 shutdown_connections 2024-04-25T10:10:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 --2- 172.21.15.5:0/57221911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37880d2380 0x7f37880d00c0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 --2- 172.21.15.5:0/57221911 >> v2:172.21.15.80:3300/0 conn(0x7f3788161200 0x7f37880b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 -- 172.21.15.5:0/57221911 >> 172.21.15.5:0/57221911 conn(0x7f378808af80 msgr2=0x7f37880d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 -- 172.21.15.5:0/57221911 shutdown_connections 2024-04-25T10:10:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.216+0000 7f378e200640 1 -- 172.21.15.5:0/57221911 wait complete. 2024-04-25T10:10:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.218+0000 7f378e200640 1 Processor -- start 2024-04-25T10:10:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.218+0000 7f378e200640 1 -- start start 2024-04-25T10:10:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.219+0000 7f378e200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37880d0600 0x7f37880c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.219+0000 7f378e200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f37880d2380 0x7f37880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.219+0000 7f378e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 0x7f37881575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.219+0000 7f378e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37880d42b0 con 0x7f3788161200 2024-04-25T10:10:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.219+0000 7f378e200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37880b28b0 con 0x7f37880d0600 2024-04-25T10:10:40.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.219+0000 7f378e200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880b3f90 con 0x7f37880d2380 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 0x7f37881575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 0x7f37881575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57622/0 (socket says 172.21.15.5:57622) 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 -- 172.21.15.5:0/1545638280 learned_addr learned my addr 172.21.15.5:0/1545638280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378ce00640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37880d0600 0x7f37880c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f3787e00640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f37880d2380 0x7f37880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 -- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f37880d2380 msgr2=0x7f37880b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f37880d2380 0x7f37880b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 -- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37880d0600 msgr2=0x7f37880c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37880d0600 0x7f37880c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.220+0000 7f378d800640 1 -- 172.21.15.5:0/1545638280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880b90c0 con 0x7f3788161200 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.221+0000 7f378d800640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 0x7f37881575b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f378000cb40 tx=0x7f378000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.221+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780017070 con 0x7f3788161200 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.221+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378000aca0 con 0x7f3788161200 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.221+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37880c7850 con 0x7f3788161200 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.221+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780012620 con 0x7f3788161200 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.221+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37880c7e10 con 0x7f3788161200 2024-04-25T10:10:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.222+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3788068f60 con 0x7f3788161200 2024-04-25T10:10:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.223+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f378001a070 con 0x7f3788161200 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.224+0000 7f3784c00640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.224+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f378008a3b0 con 0x7f3788161200 2024-04-25T10:10:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.225+0000 7f378ce00640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.226+0000 7f378ce00640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f376406b4d0 0x7f376406d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f37881f70e0 tx=0x7f3778008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:40.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.229+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3780058ee0 con 0x7f3788161200 2024-04-25T10:10:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.441+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37880b9530 con 0x7f3788161200 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.442+0000 7f3784c00640 1 -- 172.21.15.5:0/1545638280 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v93) v1 ==== 80+0+234 (secure 0 0 0) 0x7f37800587e0 con 0x7f3788161200 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6834/7055779 2024-04-26T10:09:57.322177+0000 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6832/7055779 2024-04-26T10:09:57.322177+0000 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6834/3627376661 2024-04-26T10:09:12.513203+0000 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6832/3627376661 2024-04-26T10:09:12.513203+0000 2024-04-25T10:10:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:listed 4 entries 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.444+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f376406b4d0 msgr2=0x7f376406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.444+0000 7f378e200640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f376406b4d0 0x7f376406d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f37881f70e0 tx=0x7f3778008040 comp rx=0 tx=0).stop 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.444+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 msgr2=0x7f37881575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.444+0000 7f378e200640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 0x7f37881575b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f378000cb40 tx=0x7f378000a060 comp rx=0 tx=0).stop 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 shutdown_connections 2024-04-25T10:10:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3788161200 0x7f37881575b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f37880d2380 0x7f37880b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f376406b4d0 0x7f376406d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 --2- 172.21.15.5:0/1545638280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37880d0600 0x7f37880c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 >> 172.21.15.5:0/1545638280 conn(0x7f378808af80 msgr2=0x7f37880d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 shutdown_connections 2024-04-25T10:10:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:40.445+0000 7f378e200640 1 -- 172.21.15.5:0/1545638280 wait complete. 2024-04-25T10:10:40.471 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:10:40.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:40.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:40.570 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:10:40.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:40.570 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:40.623 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:10:40.623 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:40.682 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:40.682 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:10:40.682 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:40.744 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:40.744 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:10:40.744 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:40.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:40.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:41.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.013+0000 7f5c90a00640 1 Processor -- start 2024-04-25T10:10:41.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.013+0000 7f5c90a00640 1 -- start start 2024-04-25T10:10:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c90a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5c8c154a10 0x7f5c8c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c90a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c90a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c8c070f90 0x7f5c8c071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c90a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c0b4890 con 0x7f5c8c070f90 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c90a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c0cd200 con 0x7f5c8c154a10 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c90a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c06e6f0 con 0x7f5c8c0cdca0 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c8b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c8c070f90 0x7f5c8c071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c8b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c8c070f90 0x7f5c8c071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57626/0 (socket says 172.21.15.5:57626) 2024-04-25T10:10:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.015+0000 7f5c8b200640 1 -- 172.21.15.5:0/2522392646 learned_addr learned my addr 172.21.15.5:0/2522392646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c89e00640 1 --2- 172.21.15.5:0/2522392646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8a800640 1 --2- 172.21.15.5:0/2522392646 >> v2:172.21.15.80:3300/0 conn(0x7f5c8c154a10 0x7f5c8c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8b200640 1 -- 172.21.15.5:0/2522392646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 msgr2=0x7f5c8c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8b200640 1 --2- 172.21.15.5:0/2522392646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c0b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8b200640 1 -- 172.21.15.5:0/2522392646 >> v2:172.21.15.80:3300/0 conn(0x7f5c8c154a10 msgr2=0x7f5c8c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8b200640 1 --2- 172.21.15.5:0/2522392646 >> v2:172.21.15.80:3300/0 conn(0x7f5c8c154a10 0x7f5c8c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8b200640 1 -- 172.21.15.5:0/2522392646 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c06e9d0 con 0x7f5c8c070f90 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.016+0000 7f5c8a800640 1 --2- 172.21.15.5:0/2522392646 >> v2:172.21.15.80:3300/0 conn(0x7f5c8c154a10 0x7f5c8c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.017+0000 7f5c8b200640 1 --2- 172.21.15.5:0/2522392646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c071420 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ccb70 tx=0x7f5c8402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6ebec5e1a431c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.017+0000 7f5c89400640 1 -- 172.21.15.5:0/2522392646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84036070 con 0x7f5c8c070f90 2024-04-25T10:10:41.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.017+0000 7f5c89400640 1 -- 172.21.15.5:0/2522392646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c84031600 con 0x7f5c8c070f90 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.017+0000 7f5c89400640 1 -- 172.21.15.5:0/2522392646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c840319d0 con 0x7f5c8c070f90 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 -- 172.21.15.5:0/2522392646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 msgr2=0x7f5c8c071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 --2- 172.21.15.5:0/2522392646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c071420 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ccb70 tx=0x7f5c8402a610 comp rx=0 tx=0).stop 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 -- 172.21.15.5:0/2522392646 shutdown_connections 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 --2- 172.21.15.5:0/2522392646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c071420 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 --2- 172.21.15.5:0/2522392646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c0b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 --2- 172.21.15.5:0/2522392646 >> v2:172.21.15.80:3300/0 conn(0x7f5c8c154a10 0x7f5c8c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 -- 172.21.15.5:0/2522392646 >> 172.21.15.5:0/2522392646 conn(0x7f5c8c066f20 msgr2=0x7f5c8c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 -- 172.21.15.5:0/2522392646 shutdown_connections 2024-04-25T10:10:41.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.018+0000 7f5c90a00640 1 -- 172.21.15.5:0/2522392646 wait complete. 2024-04-25T10:10:41.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.020+0000 7f5c90a00640 1 Processor -- start 2024-04-25T10:10:41.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.020+0000 7f5c90a00640 1 -- start start 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c90a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c90a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c1f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c90a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c8c154a10 0x7f5c8c0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c90a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0cd780 con 0x7f5c8c070f90 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c90a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0cd4a0 con 0x7f5c8c154a10 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c90a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c06cff0 con 0x7f5c8c0cdca0 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.021+0000 7f5c8a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c15d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57628/0 (socket says 172.21.15.5:57628) 2024-04-25T10:10:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 -- 172.21.15.5:0/171562475 learned_addr learned my addr 172.21.15.5:0/171562475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c89e00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c1f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 -- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 msgr2=0x7f5c8c1f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8b200640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c8c154a10 0x7f5c8c0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c1f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 -- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c8c154a10 msgr2=0x7f5c8c0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c8c154a10 0x7f5c8c0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 -- 172.21.15.5:0/171562475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c0c82f0 con 0x7f5c8c070f90 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.022+0000 7f5c8a800640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c15d7f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c7f0 tx=0x7f5c7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.023+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c015070 con 0x7f5c8c070f90 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.023+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c007820 con 0x7f5c8c070f90 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.023+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c007bd0 con 0x7f5c8c070f90 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.023+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c0c85d0 con 0x7f5c8c070f90 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.023+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c0c8b90 con 0x7f5c8c070f90 2024-04-25T10:10:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.024+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c0c7fa0 con 0x7f5c8c070f90 2024-04-25T10:10:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.025+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c7c018070 con 0x7f5c8c070f90 2024-04-25T10:10:41.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.026+0000 7f5c82a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c6c06b4d0 0x7f5c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.026+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f5c7c08a6c0 con 0x7f5c8c070f90 2024-04-25T10:10:41.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.027+0000 7f5c89e00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c6c06b4d0 0x7f5c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.028+0000 7f5c89e00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c6c06b4d0 0x7f5c6c06d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ca150 tx=0x7f5c78008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.030+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c7c0591f0 con 0x7f5c8c070f90 2024-04-25T10:10:41.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.270+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c8c0b2d70 con 0x7f5c8c070f90 2024-04-25T10:10:41.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.272+0000 7f5c82a00640 1 -- 172.21.15.5:0/171562475 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f5c7c059e20 con 0x7f5c8c070f90 2024-04-25T10:10:41.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:41.278 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:01.369044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-04-25T10:10:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.274+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c6c06b4d0 msgr2=0x7f5c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.274+0000 7f5c90a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c6c06b4d0 0x7f5c6c06d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0ca150 tx=0x7f5c78008040 comp rx=0 tx=0).stop 2024-04-25T10:10:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.274+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 msgr2=0x7f5c8c15d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.274+0000 7f5c90a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c15d7f0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c7f0 tx=0x7f5c7c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 shutdown_connections 2024-04-25T10:10:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5c6c06b4d0 0x7f5c6c06d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c8c154a10 0x7f5c8c0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c8c0cdca0 0x7f5c8c1f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 --2- 172.21.15.5:0/171562475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c8c070f90 0x7f5c8c15d7f0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 >> 172.21.15.5:0/171562475 conn(0x7f5c8c066f20 msgr2=0x7f5c8c0af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 shutdown_connections 2024-04-25T10:10:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.275+0000 7f5c90a00640 1 -- 172.21.15.5:0/171562475 wait complete. 2024-04-25T10:10:41.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:41.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.499+0000 7fe8af200640 1 Processor -- start 2024-04-25T10:10:41.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.499+0000 7fe8af200640 1 -- start start 2024-04-25T10:10:41.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.500+0000 7fe8af200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b0161200 0x7fe8b00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.500+0000 7fe8af200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00cf410 con 0x7fe8b00d07d0 2024-04-25T10:10:41.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.500+0000 7fe8af200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00cf6f0 con 0x7fe8b00d2380 2024-04-25T10:10:41.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.500+0000 7fe8af200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d27c0 con 0x7fe8b0161200 2024-04-25T10:10:41.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.501+0000 7fe8ae800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe8b00d07d0 0x7fe8b00b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46050/0 (socket says 172.21.15.5:46050) 2024-04-25T10:10:41.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.501+0000 7fe8ae800640 1 -- 172.21.15.5:0/897176515 learned_addr learned my addr 172.21.15.5:0/897176515 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:41.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.501+0000 7fe8ade00640 1 --2- 172.21.15.5:0/897176515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b0161200 0x7fe8b00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.501+0000 7fe8aca00640 1 -- 172.21.15.5:0/897176515 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971481517 0 0) 0x7fe8b00cf410 con 0x7fe8b00d07d0 2024-04-25T10:10:41.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.502+0000 7fe8aca00640 1 -- 172.21.15.5:0/897176515 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe894003520 con 0x7fe8b00d07d0 2024-04-25T10:10:41.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.502+0000 7fe8ade00640 1 -- 172.21.15.5:0/897176515 >> v1:172.21.15.80:6789/0 conn(0x7fe8b00d2380 legacy=0x7fe8b00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.502+0000 7fe8aca00640 1 -- 172.21.15.5:0/897176515 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525967431 0 0) 0x7fe8b00cf6f0 con 0x7fe8b00d2380 2024-04-25T10:10:41.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.502+0000 7fe8ade00640 1 -- 172.21.15.5:0/897176515 >> v1:172.21.15.5:6789/0 conn(0x7fe8b00d07d0 legacy=0x7fe8b00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.502+0000 7fe8ade00640 1 -- 172.21.15.5:0/897176515 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00d3fd0 con 0x7fe8b0161200 2024-04-25T10:10:41.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.503+0000 7fe8ade00640 1 --2- 172.21.15.5:0/897176515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b0161200 0x7fe8b00b4bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00097d0 tx=0x7fe8a002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23806bdcfa52f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8aca00640 1 -- 172.21.15.5:0/897176515 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0037070 con 0x7fe8b0161200 2024-04-25T10:10:41.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8aca00640 1 -- 172.21.15.5:0/897176515 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0032440 con 0x7fe8b0161200 2024-04-25T10:10:41.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8aca00640 1 -- 172.21.15.5:0/897176515 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0032810 con 0x7fe8b0161200 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8af200640 1 -- 172.21.15.5:0/897176515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b0161200 msgr2=0x7fe8b00b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8af200640 1 --2- 172.21.15.5:0/897176515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b0161200 0x7fe8b00b4bb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00097d0 tx=0x7fe8a002a210 comp rx=0 tx=0).stop 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8af200640 1 -- 172.21.15.5:0/897176515 shutdown_connections 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8af200640 1 --2- 172.21.15.5:0/897176515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b0161200 0x7fe8b00b4bb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8af200640 1 -- 172.21.15.5:0/897176515 >> 172.21.15.5:0/897176515 conn(0x7fe8b008af80 msgr2=0x7fe8b00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.504+0000 7fe8af200640 1 -- 172.21.15.5:0/897176515 shutdown_connections 2024-04-25T10:10:41.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.505+0000 7fe8af200640 1 -- 172.21.15.5:0/897176515 wait complete. 2024-04-25T10:10:41.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.506+0000 7fe8af200640 1 Processor -- start 2024-04-25T10:10:41.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.506+0000 7fe8af200640 1 -- start start 2024-04-25T10:10:41.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.507+0000 7fe8af200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d07d0 0x7fe8b009ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.507+0000 7fe8af200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 0x7fe8b00b8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.507+0000 7fe8af200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b0161200 0x7fe8b009d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.507+0000 7fe8af200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b3690 con 0x7fe8b00d2380 2024-04-25T10:10:41.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.507+0000 7fe8af200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b28b0 con 0x7fe8b0161200 2024-04-25T10:10:41.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.507+0000 7fe8af200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b3810 con 0x7fe8b00d07d0 2024-04-25T10:10:41.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 0x7fe8b00b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 0x7fe8b00b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57634/0 (socket says 172.21.15.5:57634) 2024-04-25T10:10:41.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 -- 172.21.15.5:0/3039926529 learned_addr learned my addr 172.21.15.5:0/3039926529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ade00640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d07d0 0x7fe8b009ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ae800640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b0161200 0x7fe8b009d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 -- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d07d0 msgr2=0x7fe8b009ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d07d0 0x7fe8b009ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 -- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b0161200 msgr2=0x7fe8b009d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b0161200 0x7fe8b009d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ad400640 1 -- 172.21.15.5:0/3039926529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b009de70 con 0x7fe8b00d2380 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.508+0000 7fe8ae800640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b0161200 0x7fe8b009d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.509+0000 7fe8ad400640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 0x7fe8b00b8520 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c460 tx=0x7fe89c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.509+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c005480 con 0x7fe8b00d2380 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.509+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c0035f0 con 0x7fe8b00d2380 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.509+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c003a00 con 0x7fe8b00d2380 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.509+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b009e150 con 0x7fe8b00d2380 2024-04-25T10:10:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.509+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b009e710 con 0x7fe8b00d2380 2024-04-25T10:10:41.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.511+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0068f60 con 0x7fe8b00d2380 2024-04-25T10:10:41.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.511+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe89c003ba0 con 0x7fe8b00d2380 2024-04-25T10:10:41.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.512+0000 7fe8a6000640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe89006b670 0x7fe89006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.512+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fe89c08a2b0 con 0x7fe8b00d2380 2024-04-25T10:10:41.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.513+0000 7fe8ade00640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe89006b670 0x7fe89006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.514+0000 7fe8ade00640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe89006b670 0x7fe89006db30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0034000 tx=0x7fe8a002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.516+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe89c058d50 con 0x7fe8b00d2380 2024-04-25T10:10:41.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.715+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8b008b360 con 0x7fe8b00d2380 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.716+0000 7fe8a6000640 1 -- 172.21.15.5:0/3039926529 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17447 (secure 0 0 0) 0x7fe89c058650 con 0x7fe8b00d2380 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:41.723 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":93,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:57.322201+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.20","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.28","mappings":[{"from":1,"to":3}]},{"pgid":"14.30","mappings":[{"from":7,"to":0}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6832/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6834/3627376661":"2024-04-26T10:09:12.513203+0000","172.21.15.5:6832/3627376661":"2024-04-26T10:09:12.513203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe89006b670 msgr2=0x7fe89006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe89006b670 0x7fe89006db30 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0034000 tx=0x7fe8a002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 msgr2=0x7fe8b00b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 0x7fe8b00b8520 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fe89c00c460 tx=0x7fe89c00c920 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 shutdown_connections 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe89006b670 0x7fe89006db30 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b0161200 0x7fe8b009d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00d2380 0x7fe8b00b8520 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 --2- 172.21.15.5:0/3039926529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d07d0 0x7fe8b009ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.719+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 >> 172.21.15.5:0/3039926529 conn(0x7fe8b008af80 msgr2=0x7fe8b00b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.720+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 shutdown_connections 2024-04-25T10:10:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.720+0000 7fe8af200640 1 -- 172.21.15.5:0/3039926529 wait complete. 2024-04-25T10:10:41.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:57.371047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:58.551736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:41.747 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.960+0000 7f36ef200640 1 Processor -- start 2024-04-25T10:10:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.960+0000 7f36ef200640 1 -- start start 2024-04-25T10:10:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36ef200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36f015ce20 0x7f36f015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36ef200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f0144cd0 con 0x7f36f00cdaa0 2024-04-25T10:10:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36ef200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36f015c440 con 0x7f36f01448f0 2024-04-25T10:10:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36ef200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36f00cdf30 con 0x7f36f015ce20 2024-04-25T10:10:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36e7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36f00cdaa0 0x7f36f015f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46064/0 (socket says 172.21.15.5:46064) 2024-04-25T10:10:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36e7e00640 1 -- 172.21.15.5:0/1692014453 learned_addr learned my addr 172.21.15.5:0/1692014453 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.962+0000 7f36ee800640 1 --2- 172.21.15.5:0/1692014453 >> v2:172.21.15.80:3300/0 conn(0x7f36f015ce20 0x7f36f015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908959491 0 0) 0x7f36f0144cd0 con 0x7f36f00cdaa0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d4003520 con 0x7f36f00cdaa0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187820584 0 0) 0x7f36d4003520 con 0x7f36f00cdaa0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 >> v2:172.21.15.80:3300/0 conn(0x7f36f015ce20 msgr2=0x7f36f015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 --2- 172.21.15.5:0/1692014453 >> v2:172.21.15.80:3300/0 conn(0x7f36f015ce20 0x7f36f015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 >> v1:172.21.15.80:6789/0 conn(0x7f36f01448f0 legacy=0x7f36f015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f015c720 con 0x7f36f00cdaa0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f36d8001670 con 0x7f36f00cdaa0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 >> v1:172.21.15.5:6789/0 conn(0x7f36f00cdaa0 legacy=0x7f36f015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d40039b0 0x7f36d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d4004800 0x7f36d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d40051c0 0x7f36d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f00cdf30 con 0x7f36d40039b0 2024-04-25T10:10:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d400c060 con 0x7f36d40051c0 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.963+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d400c260 con 0x7f36d4004800 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ede00640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d40039b0 0x7f36d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ede00640 1 -- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d4004800 msgr2=0x7f36d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ede00640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d4004800 0x7f36d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ede00640 1 -- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d40051c0 msgr2=0x7f36d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ee800640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d40051c0 0x7f36d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ede00640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d40051c0 0x7f36d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ede00640 1 -- 172.21.15.5:0/1692014453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4003520 con 0x7f36d40039b0 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ee800640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36d40051c0 0x7f36d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.964+0000 7f36ee800640 1 -- 172.21.15.5:0/1692014453 reap_dead start 2024-04-25T10:10:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.965+0000 7f36ede00640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d40039b0 0x7f36d4003dd0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f36e0001750 tx=0x7f36d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697a7986a998a8d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e0001370 con 0x7f36d40039b0 2024-04-25T10:10:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e00052d0 con 0x7f36d40039b0 2024-04-25T10:10:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ed400640 1 -- 172.21.15.5:0/1692014453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e002abc0 con 0x7f36d40039b0 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ef200640 1 -- 172.21.15.5:0/1692014453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d40039b0 msgr2=0x7f36d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ef200640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d40039b0 0x7f36d4003dd0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f36e0001750 tx=0x7f36d4005830 comp rx=0 tx=0).stop 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ef200640 1 -- 172.21.15.5:0/1692014453 shutdown_connections 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ef200640 1 --2- 172.21.15.5:0/1692014453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d40039b0 0x7f36d4003dd0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.966+0000 7f36ef200640 1 -- 172.21.15.5:0/1692014453 >> 172.21.15.5:0/1692014453 conn(0x7f36f0066f20 msgr2=0x7f36f006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.967+0000 7f36ef200640 1 -- 172.21.15.5:0/1692014453 shutdown_connections 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.967+0000 7f36ef200640 1 -- 172.21.15.5:0/1692014453 wait complete. 2024-04-25T10:10:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.968+0000 7f36ef200640 1 Processor -- start 2024-04-25T10:10:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.968+0000 7f36ef200640 1 -- start start 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.969+0000 7f36ef200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d40039b0 0x7f36f00ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.969+0000 7f36ef200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f0132ce0 0x7f36f0131f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.969+0000 7f36ef200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 0x7f36f01ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.969+0000 7f36ef200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f015d2b0 con 0x7f36f01324d0 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.969+0000 7f36ef200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f015f460 con 0x7f36f0132ce0 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.969+0000 7f36ef200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36f015d430 con 0x7f36d40039b0 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 0x7f36f01ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 0x7f36f01ee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57640/0 (socket says 172.21.15.5:57640) 2024-04-25T10:10:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 -- 172.21.15.5:0/1266305006 learned_addr learned my addr 172.21.15.5:0/1266305006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:41.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36e7e00640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f0132ce0 0x7f36f0131f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 -- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d40039b0 msgr2=0x7f36f00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d40039b0 0x7f36f00ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 -- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f0132ce0 msgr2=0x7f36f0131f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f0132ce0 0x7f36f0131f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 -- 172.21.15.5:0/1266305006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f00ac0e0 con 0x7f36f01324d0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36e7e00640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f0132ce0 0x7f36f0131f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.970+0000 7f36ee800640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 0x7f36f01ee610 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f36f006ec20 tx=0x7f36e800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.971+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8016420 con 0x7f36f01324d0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.971+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e801b070 con 0x7f36f01324d0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.971+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80167c0 con 0x7f36f01324d0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.971+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f012ff30 con 0x7f36f01324d0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.971+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f01304f0 con 0x7f36f01324d0 2024-04-25T10:10:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.972+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f0068f60 con 0x7f36f01324d0 2024-04-25T10:10:41.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.973+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f36e8005d10 con 0x7f36f01324d0 2024-04-25T10:10:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.974+0000 7f36e6000640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36cc06b470 0x7f36cc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.974+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f36e808b160 con 0x7f36f01324d0 2024-04-25T10:10:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.975+0000 7f36ede00640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36cc06b470 0x7f36cc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.976+0000 7f36ede00640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36cc06b470 0x7f36cc06d930 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f36e0032000 tx=0x7f36e002c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:41.979+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36e8025020 con 0x7f36f01324d0 2024-04-25T10:10:42.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.176+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36f0144cd0 con 0x7f36f01324d0 2024-04-25T10:10:42.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.177+0000 7f36e6000640 1 -- 172.21.15.5:0/1266305006 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36e8059b70 con 0x7f36f01324d0 2024-04-25T10:10:42.183 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:10:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.179+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36cc06b470 msgr2=0x7f36cc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36cc06b470 0x7f36cc06d930 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f36e0032000 tx=0x7f36e002c000 comp rx=0 tx=0).stop 2024-04-25T10:10:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 msgr2=0x7f36f01ee610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 0x7f36f01ee610 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f36f006ec20 tx=0x7f36e800f520 comp rx=0 tx=0).stop 2024-04-25T10:10:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 shutdown_connections 2024-04-25T10:10:42.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36f01324d0 0x7f36f01ee610 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f36cc06b470 0x7f36cc06d930 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f0132ce0 0x7f36f0131f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 --2- 172.21.15.5:0/1266305006 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f36d40039b0 0x7f36f00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 >> 172.21.15.5:0/1266305006 conn(0x7f36f0066f20 msgr2=0x7f36f0071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 shutdown_connections 2024-04-25T10:10:42.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.180+0000 7f36ef200640 1 -- 172.21.15.5:0/1266305006 wait complete. 2024-04-25T10:10:42.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.415+0000 7f65a6600640 1 Processor -- start 2024-04-25T10:10:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.415+0000 7f65a6600640 1 -- start start 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f65a6600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 0x7f65a0071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f65a6600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f65a00aece0 0x7f65a0071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f65a6600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a006f080 con 0x7f65a01465d0 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f65a6600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a006f360 con 0x7f65a00aece0 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f65a6600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00ae060 con 0x7f65a00ae860 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f659f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65a01465d0 0x7f65a015e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46068/0 (socket says 172.21.15.5:46068) 2024-04-25T10:10:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.417+0000 7f659f400640 1 -- 172.21.15.5:0/1928887575 learned_addr learned my addr 172.21.15.5:0/1928887575 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659fe00640 1 --2- 172.21.15.5:0/1928887575 >> v2:172.21.15.80:3300/0 conn(0x7f65a00aece0 0x7f65a0071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f6596a00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 0x7f65a0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762172561 0 0) 0x7f65a006f080 con 0x7f65a01465d0 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6584003520 con 0x7f65a01465d0 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817766037 0 0) 0x7f6584003520 con 0x7f65a01465d0 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 msgr2=0x7f65a0071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 0x7f65a0071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 >> v2:172.21.15.80:3300/0 conn(0x7f65a00aece0 msgr2=0x7f65a0071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 --2- 172.21.15.5:0/1928887575 >> v2:172.21.15.80:3300/0 conn(0x7f65a00aece0 0x7f65a0071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00ae340 con 0x7f65a01465d0 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f6590001670 con 0x7f65a01465d0 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 >> v1:172.21.15.5:6789/0 conn(0x7f65a01465d0 legacy=0x7f65a015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65840039b0 0x7f6584003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584004800 0x7f6584004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 0x7f65840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006f360 con 0x7f6584004800 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00ae060 con 0x7f65840039b0 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.418+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658400b9c0 con 0x7f65a00ae860 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f6596a00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584004800 0x7f6584004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f659f400640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65840039b0 0x7f6584003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f659fe00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 0x7f65840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f6596a00640 1 -- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 msgr2=0x7f65840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f6596a00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00ae860 0x7f65840051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f6596a00640 1 -- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65840039b0 msgr2=0x7f6584003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f6596a00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65840039b0 0x7f6584003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f6596a00640 1 -- 172.21.15.5:0/1928887575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584003520 con 0x7f6584004800 2024-04-25T10:10:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.419+0000 7f659f400640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65840039b0 0x7f6584003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:42.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.420+0000 7f6596a00640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584004800 0x7f6584004c80 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6588009ac0 tx=0x7f6584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae375cd5b084d61b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.420+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588036070 con 0x7f6584004800 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.420+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588031400 con 0x7f6584004800 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.420+0000 7f659ea00640 1 -- 172.21.15.5:0/1928887575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588009690 con 0x7f6584004800 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f65a6600640 1 -- 172.21.15.5:0/1928887575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584004800 msgr2=0x7f6584004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f65a6600640 1 --2- 172.21.15.5:0/1928887575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584004800 0x7f6584004c80 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f6588009ac0 tx=0x7f6584005830 comp rx=0 tx=0).stop 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f659fe00640 1 -- 172.21.15.5:0/1928887575 reap_dead start 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f65a6600640 1 -- 172.21.15.5:0/1928887575 shutdown_connections 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f65a6600640 1 -- 172.21.15.5:0/1928887575 >> 172.21.15.5:0/1928887575 conn(0x7f65a006ab60 msgr2=0x7f65a006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f65a6600640 1 -- 172.21.15.5:0/1928887575 shutdown_connections 2024-04-25T10:10:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.421+0000 7f65a6600640 1 -- 172.21.15.5:0/1928887575 wait complete. 2024-04-25T10:10:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.423+0000 7f65a6600640 1 Processor -- start 2024-04-25T10:10:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.423+0000 7f65a6600640 1 -- start start 2024-04-25T10:10:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f65a6600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6584004800 0x7f65a013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f65a6600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 0x7f65a00b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f65a6600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00b1690 0x7f65a00b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f65a6600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006ece0 con 0x7f65a015ca10 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f65a6600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006e7d0 con 0x7f6584004800 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f65a6600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006ee60 con 0x7f65a00b1690 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f6596a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 0x7f65a00b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f6596a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 0x7f65a00b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57664/0 (socket says 172.21.15.5:57664) 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.424+0000 7f6596a00640 1 -- 172.21.15.5:0/685166162 learned_addr learned my addr 172.21.15.5:0/685166162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f659f400640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6584004800 0x7f65a013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f659fe00640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00b1690 0x7f65a00b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f6596a00640 1 -- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00b1690 msgr2=0x7f65a00b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f6596a00640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00b1690 0x7f65a00b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f6596a00640 1 -- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6584004800 msgr2=0x7f65a013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f6596a00640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6584004800 0x7f65a013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f6596a00640 1 -- 172.21.15.5:0/685166162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a013e120 con 0x7f65a015ca10 2024-04-25T10:10:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.425+0000 7f6596a00640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 0x7f65a00b1150 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6588009ac0 tx=0x7f6588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.426+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588040020 con 0x7f65a015ca10 2024-04-25T10:10:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.426+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6588003bc0 con 0x7f65a015ca10 2024-04-25T10:10:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.426+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a013f980 con 0x7f65a015ca10 2024-04-25T10:10:42.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.426+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588036040 con 0x7f65a015ca10 2024-04-25T10:10:42.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.426+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a013e680 con 0x7f65a015ca10 2024-04-25T10:10:42.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.427+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a00b1aa0 con 0x7f65a015ca10 2024-04-25T10:10:42.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.428+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6588043020 con 0x7f65a015ca10 2024-04-25T10:10:42.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.429+0000 7f6597e00640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f657c06b550 0x7f657c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.429+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f65880aa5d0 con 0x7f65a015ca10 2024-04-25T10:10:42.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.430+0000 7f659f400640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f657c06b550 0x7f657c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.431+0000 7f659f400640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f657c06b550 0x7f657c06da10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6590000ff0 tx=0x7f6590009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.433+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65a00b1aa0 con 0x7f65a015ca10 2024-04-25T10:10:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.630+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65a0144de0 con 0x7f65a015ca10 2024-04-25T10:10:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.630+0000 7f6597e00640 1 -- 172.21.15.5:0/685166162 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f658804a070 con 0x7f65a015ca10 2024-04-25T10:10:42.637 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:10:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.632+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f657c06b550 msgr2=0x7f657c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f657c06b550 0x7f657c06da10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6590000ff0 tx=0x7f6590009040 comp rx=0 tx=0).stop 2024-04-25T10:10:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 msgr2=0x7f65a00b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 0x7f65a00b1150 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f6588009ac0 tx=0x7f6588004730 comp rx=0 tx=0).stop 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 shutdown_connections 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65a00b1690 0x7f65a00b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65a015ca10 0x7f65a00b1150 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f657c06b550 0x7f657c06da10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 --2- 172.21.15.5:0/685166162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6584004800 0x7f65a013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 >> 172.21.15.5:0/685166162 conn(0x7f65a006ab60 msgr2=0x7f65a015e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 shutdown_connections 2024-04-25T10:10:42.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.633+0000 7f65a6600640 1 -- 172.21.15.5:0/685166162 wait complete. 2024-04-25T10:10:42.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.861+0000 7ff2f8a00640 1 Processor -- start 2024-04-25T10:10:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.861+0000 7ff2f8a00640 1 -- start start 2024-04-25T10:10:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.862+0000 7ff2f8a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff2f40b48c0 0x7ff2f415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.862+0000 7ff2f8a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2f40d3f90 0x7ff2f4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.862+0000 7ff2f8a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f41606e0 con 0x7ff2f4161bc0 2024-04-25T10:10:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.862+0000 7ff2f8a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f4161ff0 con 0x7ff2f40d3f90 2024-04-25T10:10:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.862+0000 7ff2f8a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f41609c0 con 0x7ff2f40b48c0 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2f40d3f90 0x7ff2f4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff2f40d3f90 0x7ff2f4161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57676/0 (socket says 172.21.15.5:57676) 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 -- 172.21.15.5:0/444061178 learned_addr learned my addr 172.21.15.5:0/444061178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f3400640 1 --2- 172.21.15.5:0/444061178 >> v2:172.21.15.80:3300/0 conn(0x7ff2f40b48c0 0x7ff2f415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 -- 172.21.15.5:0/444061178 >> v2:172.21.15.80:3300/0 conn(0x7ff2f40b48c0 msgr2=0x7ff2f415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 --2- 172.21.15.5:0/444061178 >> v2:172.21.15.80:3300/0 conn(0x7ff2f40b48c0 0x7ff2f415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 -- 172.21.15.5:0/444061178 >> v1:172.21.15.80:6789/0 conn(0x7ff2f4161bc0 legacy=0x7ff2f40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.863+0000 7ff2f2a00640 1 -- 172.21.15.5:0/444061178 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4160ca0 con 0x7ff2f40d3f90 2024-04-25T10:10:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.864+0000 7ff2f2a00640 1 --2- 172.21.15.5:0/444061178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f40d3f90 0x7ff2f4161680 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff2f408ba40 tx=0x7ff2dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152a5b974b1d57a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f2000640 1 -- 172.21.15.5:0/444061178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc036070 con 0x7ff2f40d3f90 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f2000640 1 -- 172.21.15.5:0/444061178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2dc031600 con 0x7ff2f40d3f90 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f2000640 1 -- 172.21.15.5:0/444061178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc0319d0 con 0x7ff2f40d3f90 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f8a00640 1 -- 172.21.15.5:0/444061178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f40d3f90 msgr2=0x7ff2f4161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/444061178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f40d3f90 0x7ff2f4161680 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff2f408ba40 tx=0x7ff2dc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f8a00640 1 -- 172.21.15.5:0/444061178 shutdown_connections 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/444061178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f40d3f90 0x7ff2f4161680 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/444061178 >> v2:172.21.15.80:3300/0 conn(0x7ff2f40b48c0 0x7ff2f415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.865+0000 7ff2f8a00640 1 -- 172.21.15.5:0/444061178 >> 172.21.15.5:0/444061178 conn(0x7ff2f408af80 msgr2=0x7ff2f40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.866+0000 7ff2f8a00640 1 -- 172.21.15.5:0/444061178 shutdown_connections 2024-04-25T10:10:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.866+0000 7ff2f8a00640 1 -- 172.21.15.5:0/444061178 wait complete. 2024-04-25T10:10:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.867+0000 7ff2f8a00640 1 Processor -- start 2024-04-25T10:10:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.867+0000 7ff2f8a00640 1 -- start start 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f8a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff2f40b48c0 0x7ff2f415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2f40d3f90 0x7ff2f41f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 0x7ff2f40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f8a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f40b4ca0 con 0x7ff2f4161bc0 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f8a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f4161260 con 0x7ff2f40d3f90 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f8a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f41603a0 con 0x7ff2f40b48c0 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.868+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 0x7ff2f40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 0x7ff2f40cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57680/0 (socket says 172.21.15.5:57680) 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 -- 172.21.15.5:0/1501025337 learned_addr learned my addr 172.21.15.5:0/1501025337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f2a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2f40d3f90 0x7ff2f41f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3400640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff2f40b48c0 0x7ff2f415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 -- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff2f40b48c0 msgr2=0x7ff2f415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff2f40b48c0 0x7ff2f415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 -- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2f40d3f90 msgr2=0x7ff2f41f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2f40d3f90 0x7ff2f41f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 -- 172.21.15.5:0/1501025337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f415a670 con 0x7ff2f4161bc0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f2a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2f40d3f90 0x7ff2f41f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2f3e00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 0x7ff2f40cc7e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c440 tx=0x7ff2ec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.869+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec011790 con 0x7ff2f4161bc0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.870+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec017070 con 0x7ff2f4161bc0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.870+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f415a950 con 0x7ff2f4161bc0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.870+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec011b70 con 0x7ff2f4161bc0 2024-04-25T10:10:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.870+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f415af10 con 0x7ff2f4161bc0 2024-04-25T10:10:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.871+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f4068f60 con 0x7ff2f4161bc0 2024-04-25T10:10:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.872+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2ec011d10 con 0x7ff2f4161bc0 2024-04-25T10:10:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.873+0000 7ff2eb400640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff2d006b4d0 0x7ff2d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.873+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7ff2ec08a3f0 con 0x7ff2f4161bc0 2024-04-25T10:10:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.873+0000 7ff2f3400640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff2d006b4d0 0x7ff2d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:42.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.874+0000 7ff2f3400640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff2d006b4d0 0x7ff2d006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4006fd0 tx=0x7ff2e4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:42.877+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2ec058e90 con 0x7ff2f4161bc0 2024-04-25T10:10:43.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.081+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2f40d0260 con 0x7ff2f4161bc0 2024-04-25T10:10:43.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.082+0000 7ff2eb400640 1 -- 172.21.15.5:0/1501025337 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2ec058790 con 0x7ff2f4161bc0 2024-04-25T10:10:43.089 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:10:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff2d006b4d0 msgr2=0x7ff2d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff2d006b4d0 0x7ff2d006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4006fd0 tx=0x7ff2e4008040 comp rx=0 tx=0).stop 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 msgr2=0x7ff2f40cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 0x7ff2f40cc7e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00c440 tx=0x7ff2ec00c900 comp rx=0 tx=0).stop 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 shutdown_connections 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f4161bc0 0x7ff2f40cc7e0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff2d006b4d0 0x7ff2d006d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff2f40d3f90 0x7ff2f41f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 --2- 172.21.15.5:0/1501025337 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff2f40b48c0 0x7ff2f415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 >> 172.21.15.5:0/1501025337 conn(0x7ff2f408af80 msgr2=0x7ff2f40d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 shutdown_connections 2024-04-25T10:10:43.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.086+0000 7ff2f8a00640 1 -- 172.21.15.5:0/1501025337 wait complete. 2024-04-25T10:10:43.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.309+0000 7fe826e00640 1 Processor -- start 2024-04-25T10:10:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.309+0000 7fe826e00640 1 -- start start 2024-04-25T10:10:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe826e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe8200cea60 0x7fe820166420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe826e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82015bdc0 con 0x7fe820071ed0 2024-04-25T10:10:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe826e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82015c060 con 0x7fe8201529e0 2024-04-25T10:10:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe826e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8200b2340 con 0x7fe8200cea60 2024-04-25T10:10:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe817e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe820071ed0 0x7fe8200ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46082/0 (socket says 172.21.15.5:46082) 2024-04-25T10:10:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe817e00640 1 -- 172.21.15.5:0/3992354653 learned_addr learned my addr 172.21.15.5:0/3992354653 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.311+0000 7fe824e00640 1 --2- 172.21.15.5:0/3992354653 >> v2:172.21.15.80:3300/0 conn(0x7fe8200cea60 0x7fe820166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.312+0000 7fe81f400640 1 -- 172.21.15.5:0/3992354653 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386036863 0 0) 0x7fe82015bdc0 con 0x7fe820071ed0 2024-04-25T10:10:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.312+0000 7fe81f400640 1 -- 172.21.15.5:0/3992354653 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe804003520 con 0x7fe820071ed0 2024-04-25T10:10:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.312+0000 7fe824e00640 1 -- 172.21.15.5:0/3992354653 >> v1:172.21.15.80:6789/0 conn(0x7fe8201529e0 legacy=0x7fe8200c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.312+0000 7fe824e00640 1 -- 172.21.15.5:0/3992354653 >> v1:172.21.15.5:6789/0 conn(0x7fe820071ed0 legacy=0x7fe8200ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.312+0000 7fe81f400640 1 -- 172.21.15.5:0/3992354653 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1563509445 0 0) 0x7fe804003520 con 0x7fe820071ed0 2024-04-25T10:10:43.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.312+0000 7fe824e00640 1 -- 172.21.15.5:0/3992354653 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82015c340 con 0x7fe8200cea60 2024-04-25T10:10:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.313+0000 7fe824e00640 1 --2- 172.21.15.5:0/3992354653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8200cea60 0x7fe820166420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe818009d90 tx=0x7fe81802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d04bfe2f1c9fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.314+0000 7fe81f400640 1 -- 172.21.15.5:0/3992354653 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808001670 con 0x7fe8200cea60 2024-04-25T10:10:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.314+0000 7fe81f400640 1 -- 172.21.15.5:0/3992354653 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818007540 con 0x7fe8200cea60 2024-04-25T10:10:43.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.314+0000 7fe81f400640 1 -- 172.21.15.5:0/3992354653 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818007910 con 0x7fe8200cea60 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 -- 172.21.15.5:0/3992354653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8200cea60 msgr2=0x7fe820166420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 --2- 172.21.15.5:0/3992354653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8200cea60 0x7fe820166420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe818009d90 tx=0x7fe81802a680 comp rx=0 tx=0).stop 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 -- 172.21.15.5:0/3992354653 shutdown_connections 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 --2- 172.21.15.5:0/3992354653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8200cea60 0x7fe820166420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 -- 172.21.15.5:0/3992354653 >> 172.21.15.5:0/3992354653 conn(0x7fe820071150 msgr2=0x7fe820071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 -- 172.21.15.5:0/3992354653 shutdown_connections 2024-04-25T10:10:43.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.315+0000 7fe826e00640 1 -- 172.21.15.5:0/3992354653 wait complete. 2024-04-25T10:10:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.317+0000 7fe826e00640 1 Processor -- start 2024-04-25T10:10:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.317+0000 7fe826e00640 1 -- start start 2024-04-25T10:10:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe826e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe820071ed0 0x7fe8201438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe826e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8200cea60 0x7fe8200b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe826e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 0x7fe820149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe826e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200c65e0 con 0x7fe8200ccb50 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe826e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82015c900 con 0x7fe820071ed0 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe826e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82006e160 con 0x7fe8200cea60 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe824e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 0x7fe820149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe824e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 0x7fe820149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57682/0 (socket says 172.21.15.5:57682) 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.318+0000 7fe824e00640 1 -- 172.21.15.5:0/1396138732 learned_addr learned my addr 172.21.15.5:0/1396138732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe81fe00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe820071ed0 0x7fe8201438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe824e00640 1 -- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8200cea60 msgr2=0x7fe8200b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe824e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8200cea60 0x7fe8200b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe824e00640 1 -- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe820071ed0 msgr2=0x7fe8201438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe824e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe820071ed0 0x7fe8201438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe824e00640 1 -- 172.21.15.5:0/1396138732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8201e5f40 con 0x7fe8200ccb50 2024-04-25T10:10:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe824e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 0x7fe820149860 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe818009d90 tx=0x7fe818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818040070 con 0x7fe8200ccb50 2024-04-25T10:10:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818031bc0 con 0x7fe8200ccb50 2024-04-25T10:10:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818009450 con 0x7fe8200ccb50 2024-04-25T10:10:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.319+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8200b54c0 con 0x7fe8200ccb50 2024-04-25T10:10:43.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.320+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8201e64c0 con 0x7fe8200ccb50 2024-04-25T10:10:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.321+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe820068fd0 con 0x7fe8200ccb50 2024-04-25T10:10:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.322+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe818039040 con 0x7fe8200ccb50 2024-04-25T10:10:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.323+0000 7fe81cc00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.323+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fe8180aba00 con 0x7fe8200ccb50 2024-04-25T10:10:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.323+0000 7fe81fe00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.324+0000 7fe81fe00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe810000f60 tx=0x7fe810009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.327+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8180303a0 con 0x7fe8200ccb50 2024-04-25T10:10:43.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.567+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe82006c1a0 con 0x7fe8200ccb50 2024-04-25T10:10:43.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.568+0000 7fe81cc00640 1 -- 172.21.15.5:0/1396138732 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fe81807a530 con 0x7fe8200ccb50 2024-04-25T10:10:43.574 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:43.574 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:01.369044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:43.575 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.570+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe7fc06b500 msgr2=0x7fe7fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.570+0000 7fe826e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe810000f60 tx=0x7fe810009040 comp rx=0 tx=0).stop 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.570+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 msgr2=0x7fe820149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.570+0000 7fe826e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 0x7fe820149860 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe818009d90 tx=0x7fe818004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 shutdown_connections 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8200ccb50 0x7fe820149860 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8200cea60 0x7fe8200b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 --2- 172.21.15.5:0/1396138732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe820071ed0 0x7fe8201438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 >> 172.21.15.5:0/1396138732 conn(0x7fe820071150 msgr2=0x7fe82015a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 shutdown_connections 2024-04-25T10:10:43.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.571+0000 7fe826e00640 1 -- 172.21.15.5:0/1396138732 wait complete. 2024-04-25T10:10:43.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:43.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.807+0000 7fdd3ca00640 1 Processor -- start 2024-04-25T10:10:43.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.807+0000 7fdd3ca00640 1 -- start start 2024-04-25T10:10:43.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd3ca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd38161c50 0x7fdd380b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd3ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd3806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd3ca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd380b5520 0x7fdd3806b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd3ca00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd380b0500 con 0x7fdd38161c50 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd3ca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd380af8f0 con 0x7fdd380b5520 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd3ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd380afbd0 con 0x7fdd380b50c0 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd37400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd38161c50 0x7fdd380b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd37400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd38161c50 0x7fdd380b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57698/0 (socket says 172.21.15.5:57698) 2024-04-25T10:10:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.809+0000 7fdd37400640 1 -- 172.21.15.5:0/3160505750 learned_addr learned my addr 172.21.15.5:0/3160505750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd37e00640 1 --2- 172.21.15.5:0/3160505750 >> v2:172.21.15.80:3300/0 conn(0x7fdd380b5520 0x7fdd3806b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd36a00640 1 --2- 172.21.15.5:0/3160505750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd3806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd37400640 1 -- 172.21.15.5:0/3160505750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 msgr2=0x7fdd3806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd37400640 1 --2- 172.21.15.5:0/3160505750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd3806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd37400640 1 -- 172.21.15.5:0/3160505750 >> v2:172.21.15.80:3300/0 conn(0x7fdd380b5520 msgr2=0x7fdd3806b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd37400640 1 --2- 172.21.15.5:0/3160505750 >> v2:172.21.15.80:3300/0 conn(0x7fdd380b5520 0x7fdd3806b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.810+0000 7fdd37400640 1 -- 172.21.15.5:0/3160505750 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd380afeb0 con 0x7fdd38161c50 2024-04-25T10:10:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.811+0000 7fdd37400640 1 --2- 172.21.15.5:0/3160505750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd38161c50 0x7fdd380b0990 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdd380b0470 tx=0x7fdd2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce71121c88f735fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.811+0000 7fdd36000640 1 -- 172.21.15.5:0/3160505750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28037070 con 0x7fdd38161c50 2024-04-25T10:10:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.811+0000 7fdd36000640 1 -- 172.21.15.5:0/3160505750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd28032440 con 0x7fdd38161c50 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.811+0000 7fdd36000640 1 -- 172.21.15.5:0/3160505750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd28032810 con 0x7fdd38161c50 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 -- 172.21.15.5:0/3160505750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd38161c50 msgr2=0x7fdd380b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/3160505750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd38161c50 0x7fdd380b0990 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fdd380b0470 tx=0x7fdd2802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 -- 172.21.15.5:0/3160505750 shutdown_connections 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/3160505750 >> v2:172.21.15.80:3300/0 conn(0x7fdd380b5520 0x7fdd3806b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/3160505750 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd3806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/3160505750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd38161c50 0x7fdd380b0990 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 -- 172.21.15.5:0/3160505750 >> 172.21.15.5:0/3160505750 conn(0x7fdd380b5ab0 msgr2=0x7fdd380999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 -- 172.21.15.5:0/3160505750 shutdown_connections 2024-04-25T10:10:43.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.812+0000 7fdd3ca00640 1 -- 172.21.15.5:0/3160505750 wait complete. 2024-04-25T10:10:43.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.814+0000 7fdd3ca00640 1 Processor -- start 2024-04-25T10:10:43.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.814+0000 7fdd3ca00640 1 -- start start 2024-04-25T10:10:43.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd3ca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd38149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd3ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 0x7fdd3806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd3ca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd38161c50 0x7fdd3806f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd3ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd380af4a0 con 0x7fdd380b5520 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd3ca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38162030 con 0x7fdd38161c50 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd3ca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd380af620 con 0x7fdd380b50c0 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd36a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 0x7fdd3806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd36a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 0x7fdd3806ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57706/0 (socket says 172.21.15.5:57706) 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd36a00640 1 -- 172.21.15.5:0/2563935078 learned_addr learned my addr 172.21.15.5:0/2563935078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd36a00640 1 -- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 msgr2=0x7fdd38149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd37400640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd38149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.815+0000 7fdd37e00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd38161c50 0x7fdd3806f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd36a00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd38149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd36a00640 1 -- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd38161c50 msgr2=0x7fdd3806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd36a00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd38161c50 0x7fdd3806f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd36a00640 1 -- 172.21.15.5:0/2563935078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3809bcb0 con 0x7fdd380b5520 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd37e00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd38161c50 0x7fdd3806f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd36a00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 0x7fdd3806ed50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000d830 tx=0x7fdd2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20018020 con 0x7fdd380b5520 2024-04-25T10:10:43.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd200096a0 con 0x7fdd380b5520 2024-04-25T10:10:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd20009a70 con 0x7fdd380b5520 2024-04-25T10:10:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3809bf90 con 0x7fdd380b5520 2024-04-25T10:10:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.816+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3809c550 con 0x7fdd380b5520 2024-04-25T10:10:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.817+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd38068f60 con 0x7fdd380b5520 2024-04-25T10:10:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.819+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd2001b020 con 0x7fdd380b5520 2024-04-25T10:10:43.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.819+0000 7fdd2f400640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd1406b4d0 0x7fdd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:43.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.820+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fdd2008aa20 con 0x7fdd380b5520 2024-04-25T10:10:43.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.820+0000 7fdd37400640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd1406b4d0 0x7fdd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.821+0000 7fdd37400640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd1406b4d0 0x7fdd1406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdd28034000 tx=0x7fdd2802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:43.824+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd20022030 con 0x7fdd380b5520 2024-04-25T10:10:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.022+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd380ae620 con 0x7fdd380b5520 2024-04-25T10:10:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.026+0000 7fdd2f400640 1 -- 172.21.15.5:0/2563935078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17447 (secure 0 0 0) 0x7fdd200102e0 con 0x7fdd380b5520 2024-04-25T10:10:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:44.033 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":93,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:57.322201+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.20","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.28","mappings":[{"from":1,"to":3}]},{"pgid":"14.30","mappings":[{"from":7,"to":0}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6832/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6834/3627376661":"2024-04-26T10:09:12.513203+0000","172.21.15.5:6832/3627376661":"2024-04-26T10:09:12.513203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.028+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd1406b4d0 msgr2=0x7fdd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.028+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd1406b4d0 0x7fdd1406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdd28034000 tx=0x7fdd2802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.028+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 msgr2=0x7fdd3806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.028+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 0x7fdd3806ed50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000d830 tx=0x7fdd2000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 shutdown_connections 2024-04-25T10:10:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdd1406b4d0 0x7fdd1406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd38161c50 0x7fdd3806f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd380b5520 0x7fdd3806ed50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 --2- 172.21.15.5:0/2563935078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd380b50c0 0x7fdd38149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 >> 172.21.15.5:0/2563935078 conn(0x7fdd380b5ab0 msgr2=0x7fdd3806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 shutdown_connections 2024-04-25T10:10:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.029+0000 7fdd3ca00640 1 -- 172.21.15.5:0/2563935078 wait complete. 2024-04-25T10:10:44.055 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:10:44.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.265+0000 7f4c42a00640 1 Processor -- start 2024-04-25T10:10:44.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.265+0000 7f4c42a00640 1 -- start start 2024-04-25T10:10:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.267+0000 7f4c42a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 0x7f4c3c0b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.267+0000 7f4c42a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c15bb60 con 0x7f4c3c0b29d0 2024-04-25T10:10:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.267+0000 7f4c42a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3c0cc230 con 0x7f4c3c0cffd0 2024-04-25T10:10:44.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.267+0000 7f4c42a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0cc510 con 0x7f4c3c15b780 2024-04-25T10:10:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.268+0000 7f4c40c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4c3c0b29d0 0x7f4c3c0cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46096/0 (socket says 172.21.15.5:46096) 2024-04-25T10:10:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.268+0000 7f4c40c00640 1 -- 172.21.15.5:0/3607863282 learned_addr learned my addr 172.21.15.5:0/3607863282 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.268+0000 7f4c41600640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 0x7f4c3c0b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.268+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476029219 0 0) 0x7f4c3c15bb60 con 0x7f4c3c0b29d0 2024-04-25T10:10:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.268+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c24003520 con 0x7f4c3c0b29d0 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221345472 0 0) 0x7f4c24003520 con 0x7f4c3c0b29d0 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 msgr2=0x7f4c3c0b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 0x7f4c3c0b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 >> v1:172.21.15.80:6789/0 conn(0x7f4c3c0cffd0 legacy=0x7f4c3c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0cc7f0 con 0x7f4c3c0b29d0 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4c28001670 con 0x7f4c3c0b29d0 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 >> v1:172.21.15.5:6789/0 conn(0x7f4c3c0b29d0 legacy=0x7f4c3c0cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c24004800 0x7f4c24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 0x7f4c240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0cc510 con 0x7f4c240039e0 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2400bb00 con 0x7f4c24004800 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2400bd00 con 0x7f4c3c15b780 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.269+0000 7f4c41600640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.270+0000 7f4c41600640 1 -- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 msgr2=0x7f4c240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.270+0000 7f4c41600640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c15b780 0x7f4c240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.270+0000 7f4c41600640 1 -- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c24004800 msgr2=0x7f4c24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.270+0000 7f4c41600640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c24004800 0x7f4c24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.270+0000 7f4c41600640 1 -- 172.21.15.5:0/3607863282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c24003520 con 0x7f4c240039e0 2024-04-25T10:10:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.271+0000 7f4c41600640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009a90 tx=0x7f4c24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04016f1be46f849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.271+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30036070 con 0x7f4c240039e0 2024-04-25T10:10:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.271+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c30031400 con 0x7f4c240039e0 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.271+0000 7f4c3be00640 1 -- 172.21.15.5:0/3607863282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30009690 con 0x7f4c240039e0 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.271+0000 7f4c42a00640 1 -- 172.21.15.5:0/3607863282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 msgr2=0x7f4c24003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.271+0000 7f4c42a00640 1 --2- 172.21.15.5:0/3607863282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009a90 tx=0x7f4c24005830 comp rx=0 tx=0).stop 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.272+0000 7f4c42000640 1 -- 172.21.15.5:0/3607863282 reap_dead start 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.272+0000 7f4c42a00640 1 -- 172.21.15.5:0/3607863282 shutdown_connections 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.272+0000 7f4c42a00640 1 -- 172.21.15.5:0/3607863282 >> 172.21.15.5:0/3607863282 conn(0x7f4c3c0d1470 msgr2=0x7f4c3c0d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.272+0000 7f4c42a00640 1 -- 172.21.15.5:0/3607863282 shutdown_connections 2024-04-25T10:10:44.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.272+0000 7f4c42a00640 1 -- 172.21.15.5:0/3607863282 wait complete. 2024-04-25T10:10:44.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.274+0000 7f4c42a00640 1 Processor -- start 2024-04-25T10:10:44.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.274+0000 7f4c42a00640 1 -- start start 2024-04-25T10:10:44.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c42a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c3c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c42a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c1509d0 0x7f4c3c14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c42a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c3c14f860 0x7f4c3c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c42a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0cbe50 con 0x7f4c240039e0 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c42a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0ccdb0 con 0x7f4c3c14f860 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c42a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3c0cbfd0 con 0x7f4c3c1509d0 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c3c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c3c15cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57726/0 (socket says 172.21.15.5:57726) 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 -- 172.21.15.5:0/2785630306 learned_addr learned my addr 172.21.15.5:0/2785630306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 -- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c1509d0 msgr2=0x7f4c3c14f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c1509d0 0x7f4c3c14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 -- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c3c14f860 msgr2=0x7f4c3c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c3c14f860 0x7f4c3c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.275+0000 7f4c41600640 1 -- 172.21.15.5:0/2785630306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c15eb70 con 0x7f4c240039e0 2024-04-25T10:10:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.276+0000 7f4c41600640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c3c15cf90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4c30034040 tx=0x7f4c30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.276+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30040020 con 0x7f4c240039e0 2024-04-25T10:10:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.276+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c30003940 con 0x7f4c240039e0 2024-04-25T10:10:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.276+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c30036040 con 0x7f4c240039e0 2024-04-25T10:10:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.276+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3c15e890 con 0x7f4c240039e0 2024-04-25T10:10:44.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.276+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3c0c5bd0 con 0x7f4c240039e0 2024-04-25T10:10:44.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.277+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3c068f60 con 0x7f4c240039e0 2024-04-25T10:10:44.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.278+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c30043020 con 0x7f4c240039e0 2024-04-25T10:10:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.279+0000 7f4c39600640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c1c06b500 0x7f4c1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.279+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f4c300aa690 con 0x7f4c240039e0 2024-04-25T10:10:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.280+0000 7f4c40c00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c1c06b500 0x7f4c1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.281+0000 7f4c40c00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c1c06b500 0x7f4c1c06d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0cf980 tx=0x7f4c28001b30 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.284+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c3004a070 con 0x7f4c240039e0 2024-04-25T10:10:44.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.532+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c3c0cdef0 con 0x7f4c240039e0 2024-04-25T10:10:44.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.533+0000 7f4c39600640 1 -- 172.21.15.5:0/2785630306 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f4c300791c0 con 0x7f4c240039e0 2024-04-25T10:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:01.369044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:44.540 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.536+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c1c06b500 msgr2=0x7f4c1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.536+0000 7f4c42a00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c1c06b500 0x7f4c1c06d9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0cf980 tx=0x7f4c28001b30 comp rx=0 tx=0).stop 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.536+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 msgr2=0x7f4c3c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c3c15cf90 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f4c30034040 tx=0x7f4c30004730 comp rx=0 tx=0).stop 2024-04-25T10:10:44.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 shutdown_connections 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c1c06b500 0x7f4c1c06d9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c3c14f860 0x7f4c3c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c3c1509d0 0x7f4c3c14f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 --2- 172.21.15.5:0/2785630306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240039e0 0x7f4c3c15cf90 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 >> 172.21.15.5:0/2785630306 conn(0x7f4c3c0d1470 msgr2=0x7f4c3c06ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 shutdown_connections 2024-04-25T10:10:44.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.537+0000 7f4c42a00640 1 -- 172.21.15.5:0/2785630306 wait complete. 2024-04-25T10:10:44.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:44.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.769+0000 7fc19f200640 1 Processor -- start 2024-04-25T10:10:44.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.770+0000 7fc19f200640 1 -- start start 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.771+0000 7fc19f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc1a00d07d0 0x7fc1a00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.771+0000 7fc19f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a00b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.771+0000 7fc19f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a00cf410 con 0x7fc1a0161200 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.771+0000 7fc19f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00b28b0 con 0x7fc1a00d07d0 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.771+0000 7fc19f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00cf6f0 con 0x7fc1a00d2380 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19de00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc1a0161200 0x7fc1a00d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46098/0 (socket says 172.21.15.5:46098) 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19de00640 1 -- 172.21.15.5:0/1192735640 learned_addr learned my addr 172.21.15.5:0/1192735640 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc1a00d07d0 0x7fc1a00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19ca00640 1 -- 172.21.15.5:0/1192735640 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088878198 0 0) 0x7fc1a00cf410 con 0x7fc1a0161200 2024-04-25T10:10:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19e800640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a00b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19ca00640 1 -- 172.21.15.5:0/1192735640 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003520 con 0x7fc1a0161200 2024-04-25T10:10:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19d400640 1 -- 172.21.15.5:0/1192735640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 msgr2=0x7fc1a00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19d400640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a00b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19d400640 1 -- 172.21.15.5:0/1192735640 >> v1:172.21.15.5:6789/0 conn(0x7fc1a0161200 legacy=0x7fc1a00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19d400640 1 -- 172.21.15.5:0/1192735640 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00d3fd0 con 0x7fc1a00d07d0 2024-04-25T10:10:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.772+0000 7fc19e800640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a00b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.774+0000 7fc19d400640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b4bb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d90 tx=0x7fc18802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed76d9f275e5dd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.774+0000 7fc19ca00640 1 -- 172.21.15.5:0/1192735640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188007740 con 0x7fc1a00d07d0 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.774+0000 7fc19ca00640 1 -- 172.21.15.5:0/1192735640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1880078e0 con 0x7fc1a00d07d0 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.774+0000 7fc19ca00640 1 -- 172.21.15.5:0/1192735640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188007cb0 con 0x7fc1a00d07d0 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.774+0000 7fc19f200640 1 -- 172.21.15.5:0/1192735640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00d07d0 msgr2=0x7fc1a00b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.774+0000 7fc19f200640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b4bb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc188009d90 tx=0x7fc18802a660 comp rx=0 tx=0).stop 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.775+0000 7fc19f200640 1 -- 172.21.15.5:0/1192735640 shutdown_connections 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.775+0000 7fc19f200640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a00b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.775+0000 7fc19f200640 1 --2- 172.21.15.5:0/1192735640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b4bb0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.775+0000 7fc19f200640 1 -- 172.21.15.5:0/1192735640 >> 172.21.15.5:0/1192735640 conn(0x7fc1a008af80 msgr2=0x7fc1a00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:44.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.775+0000 7fc19f200640 1 -- 172.21.15.5:0/1192735640 shutdown_connections 2024-04-25T10:10:44.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.775+0000 7fc19f200640 1 -- 172.21.15.5:0/1192735640 wait complete. 2024-04-25T10:10:44.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.777+0000 7fc19f200640 1 Processor -- start 2024-04-25T10:10:44.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.777+0000 7fc19f200640 1 -- start start 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.778+0000 7fc19f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.778+0000 7fc19f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a0152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 0x7fc1a00cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00b3640 con 0x7fc1a0161200 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00b34c0 con 0x7fc1a00d07d0 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a00b37c0 con 0x7fc1a00d2380 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 0x7fc1a00cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 0x7fc1a00cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57746/0 (socket says 172.21.15.5:57746) 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 -- 172.21.15.5:0/475923897 learned_addr learned my addr 172.21.15.5:0/475923897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19d400640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a0152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19de00640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 -- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 msgr2=0x7fc1a0152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a0152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 -- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1a00d07d0 msgr2=0x7fc1a00b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19e800640 1 -- 172.21.15.5:0/475923897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00ccc30 con 0x7fc1a0161200 2024-04-25T10:10:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.779+0000 7fc19d400640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a0152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.780+0000 7fc19e800640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 0x7fc1a00cc6f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc19800c820 tx=0x7fc19800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.780+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198015070 con 0x7fc1a0161200 2024-04-25T10:10:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.780+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc198007a30 con 0x7fc1a0161200 2024-04-25T10:10:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.780+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198007e20 con 0x7fc1a0161200 2024-04-25T10:10:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.780+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a00b7230 con 0x7fc1a0161200 2024-04-25T10:10:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.780+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a00b77f0 con 0x7fc1a0161200 2024-04-25T10:10:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.782+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a0068f60 con 0x7fc1a0161200 2024-04-25T10:10:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.783+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc198018070 con 0x7fc1a0161200 2024-04-25T10:10:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.784+0000 7fc196000640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc17c06b4d0 0x7fc17c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.784+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fc19808a4f0 con 0x7fc1a0161200 2024-04-25T10:10:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.785+0000 7fc19de00640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc17c06b4d0 0x7fc17c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.786+0000 7fc19de00640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc17c06b4d0 0x7fc17c06d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc190000f60 tx=0x7fc190009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:44.788+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc198058f90 con 0x7fc1a0161200 2024-04-25T10:10:45.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.028+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1a00b2f30 con 0x7fc1a0161200 2024-04-25T10:10:45.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.029+0000 7fc196000640 1 -- 172.21.15.5:0/475923897 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fc198058890 con 0x7fc1a0161200 2024-04-25T10:10:45.036 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:45.036 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:59.535833+0000","modified":"2024-04-25T10:10:01.369044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7959},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7959":{"gid":7959,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6834/841689019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":841689019},{"type":"v1","addr":"172.21.15.5:6834","nonce":841689019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:45.036 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 75 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc17c06b4d0 msgr2=0x7fc17c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc17c06b4d0 0x7fc17c06d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc190000f60 tx=0x7fc190009040 comp rx=0 tx=0).stop 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 msgr2=0x7fc1a00cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 0x7fc1a00cc6f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc19800c820 tx=0x7fc19800cce0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 shutdown_connections 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a0161200 0x7fc1a00cc6f0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1a00d2380 0x7fc1a0152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc17c06b4d0 0x7fc17c06d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 --2- 172.21.15.5:0/475923897 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1a00d07d0 0x7fc1a00b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 >> 172.21.15.5:0/475923897 conn(0x7fc1a008af80 msgr2=0x7fc1a00d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.032+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 shutdown_connections 2024-04-25T10:10:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.033+0000 7fc19f200640 1 -- 172.21.15.5:0/475923897 wait complete. 2024-04-25T10:10:45.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:45.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.259+0000 7f1c45e00640 1 Processor -- start 2024-04-25T10:10:45.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.260+0000 7f1c45e00640 1 -- start start 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.261+0000 7f1c45e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c400d07d0 0x7f1c400b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.261+0000 7f1c45e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c400d2380 0x7f1c400b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.261+0000 7f1c45e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c400cf410 con 0x7f1c40161200 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.261+0000 7f1c45e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c400b28b0 con 0x7f1c400d07d0 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.261+0000 7f1c45e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c400cf6f0 con 0x7f1c400d2380 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c400d07d0 0x7f1c400b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c400d07d0 0x7f1c400b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57748/0 (socket says 172.21.15.5:57748) 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 -- 172.21.15.5:0/1195953607 learned_addr learned my addr 172.21.15.5:0/1195953607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c44a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1c40161200 0x7f1c400d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46102/0 (socket says 172.21.15.5:46102) 2024-04-25T10:10:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c45400640 1 --2- 172.21.15.5:0/1195953607 >> v2:172.21.15.80:3300/0 conn(0x7f1c400d2380 0x7f1c400b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 -- 172.21.15.5:0/1195953607 >> v2:172.21.15.80:3300/0 conn(0x7f1c400d2380 msgr2=0x7f1c400b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 --2- 172.21.15.5:0/1195953607 >> v2:172.21.15.80:3300/0 conn(0x7f1c400d2380 0x7f1c400b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 -- 172.21.15.5:0/1195953607 >> v1:172.21.15.5:6789/0 conn(0x7f1c40161200 legacy=0x7f1c400d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c3fe00640 1 -- 172.21.15.5:0/1195953607 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400d3fd0 con 0x7f1c400d07d0 2024-04-25T10:10:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.262+0000 7f1c45400640 1 --2- 172.21.15.5:0/1195953607 >> v2:172.21.15.80:3300/0 conn(0x7f1c400d2380 0x7f1c400b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:45.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.263+0000 7f1c3fe00640 1 --2- 172.21.15.5:0/1195953607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d07d0 0x7f1c400b4bb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1c400cfc80 tx=0x7f1c2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe843f5f3ebdfac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.264+0000 7f1c3f400640 1 -- 172.21.15.5:0/1195953607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28037070 con 0x7f1c400d07d0 2024-04-25T10:10:45.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.264+0000 7f1c3f400640 1 -- 172.21.15.5:0/1195953607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28032440 con 0x7f1c400d07d0 2024-04-25T10:10:45.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.264+0000 7f1c3f400640 1 -- 172.21.15.5:0/1195953607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28032810 con 0x7f1c400d07d0 2024-04-25T10:10:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.264+0000 7f1c45e00640 1 -- 172.21.15.5:0/1195953607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d07d0 msgr2=0x7f1c400b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.264+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1195953607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d07d0 0x7f1c400b4bb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1c400cfc80 tx=0x7f1c2802a270 comp rx=0 tx=0).stop 2024-04-25T10:10:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.265+0000 7f1c45e00640 1 -- 172.21.15.5:0/1195953607 shutdown_connections 2024-04-25T10:10:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.265+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1195953607 >> v2:172.21.15.80:3300/0 conn(0x7f1c400d2380 0x7f1c400b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.265+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1195953607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c400d07d0 0x7f1c400b4bb0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.265+0000 7f1c45e00640 1 -- 172.21.15.5:0/1195953607 >> 172.21.15.5:0/1195953607 conn(0x7f1c4008af80 msgr2=0x7f1c400d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.265+0000 7f1c45e00640 1 -- 172.21.15.5:0/1195953607 shutdown_connections 2024-04-25T10:10:45.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.265+0000 7f1c45e00640 1 -- 172.21.15.5:0/1195953607 wait complete. 2024-04-25T10:10:45.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.266+0000 7f1c45e00640 1 Processor -- start 2024-04-25T10:10:45.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.267+0000 7f1c45e00640 1 -- start start 2024-04-25T10:10:45.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c400d07d0 0x7f1c40150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c400d2380 0x7f1c4009d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 0x7f1c4009da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b3640 con 0x7f1c40161200 2024-04-25T10:10:45.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b34c0 con 0x7f1c400d07d0 2024-04-25T10:10:45.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c400b37c0 con 0x7f1c400d2380 2024-04-25T10:10:45.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 0x7f1c4009da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 0x7f1c4009da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57752/0 (socket says 172.21.15.5:57752) 2024-04-25T10:10:45.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 -- 172.21.15.5:0/1592878588 learned_addr learned my addr 172.21.15.5:0/1592878588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:45.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c44a00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c400d07d0 0x7f1c40150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c3fe00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c400d2380 0x7f1c4009d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 -- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c400d2380 msgr2=0x7f1c4009d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c400d2380 0x7f1c4009d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 -- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c400d07d0 msgr2=0x7f1c40150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c400d07d0 0x7f1c40150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c45400640 1 -- 172.21.15.5:0/1592878588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c400a0360 con 0x7f1c40161200 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.268+0000 7f1c3fe00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c400d2380 0x7f1c4009d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:45.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.269+0000 7f1c45400640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 0x7f1c4009da10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800cf20 tx=0x7f1c3800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.269+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3801a070 con 0x7f1c40161200 2024-04-25T10:10:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.269+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800d840 con 0x7f1c40161200 2024-04-25T10:10:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.269+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400a0640 con 0x7f1c40161200 2024-04-25T10:10:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.269+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800dc30 con 0x7f1c40161200 2024-04-25T10:10:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.269+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c400a0c00 con 0x7f1c40161200 2024-04-25T10:10:45.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.270+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c40068f60 con 0x7f1c40161200 2024-04-25T10:10:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.271+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c3801d070 con 0x7f1c40161200 2024-04-25T10:10:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.273+0000 7f1c3cc00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c1c06b4d0 0x7f1c1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.273+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f1c3808ae10 con 0x7f1c40161200 2024-04-25T10:10:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.273+0000 7f1c44a00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c1c06b4d0 0x7f1c1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.275+0000 7f1c44a00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c1c06b4d0 0x7f1c1c06d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1c30000f60 tx=0x7f1c30009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.277+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c3808e030 con 0x7f1c40161200 2024-04-25T10:10:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.476+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c400ca340 con 0x7f1c40161200 2024-04-25T10:10:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.479+0000 7f1c3cc00640 1 -- 172.21.15.5:0/1592878588 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17447 (secure 0 0 0) 0x7f1c38059940 con 0x7f1c40161200 2024-04-25T10:10:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:45.487 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":93,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:09:57.322201+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:57.371047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:58.551736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.a","mappings":[{"from":6,"to":5}]},{"pgid":"14.11","mappings":[{"from":1,"to":0}]},{"pgid":"14.12","mappings":[{"from":7,"to":3}]},{"pgid":"14.20","mappings":[{"from":7,"to":3}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.28","mappings":[{"from":1,"to":3}]},{"pgid":"14.30","mappings":[{"from":7,"to":0}]},{"pgid":"14.35","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6832/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6834/3627376661":"2024-04-26T10:09:12.513203+0000","172.21.15.5:6832/3627376661":"2024-04-26T10:09:12.513203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.482+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c1c06b4d0 msgr2=0x7f1c1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c1c06b4d0 0x7f1c1c06d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1c30000f60 tx=0x7f1c30009040 comp rx=0 tx=0).stop 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 msgr2=0x7f1c4009da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 0x7f1c4009da10 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1c3800cf20 tx=0x7f1c3800a7d0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 shutdown_connections 2024-04-25T10:10:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c40161200 0x7f1c4009da10 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c400d2380 0x7f1c4009d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c1c06b4d0 0x7f1c1c06d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 --2- 172.21.15.5:0/1592878588 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c400d07d0 0x7f1c40150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 >> 172.21.15.5:0/1592878588 conn(0x7f1c4008af80 msgr2=0x7f1c401604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 shutdown_connections 2024-04-25T10:10:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.483+0000 7f1c45e00640 1 -- 172.21.15.5:0/1592878588 wait complete. 2024-04-25T10:10:45.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:10:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.724+0000 7f791ce00640 1 Processor -- start 2024-04-25T10:10:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.725+0000 7f791ce00640 1 -- start start 2024-04-25T10:10:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f791ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79180d07d0 0x7f79180b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f791ce00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79180d2380 0x7f79180b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f791ce00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79180cf410 con 0x7f7918161200 2024-04-25T10:10:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f791ce00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180b28b0 con 0x7f79180d07d0 2024-04-25T10:10:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f791ce00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79180cf6f0 con 0x7f79180d2380 2024-04-25T10:10:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f7916a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79180d07d0 0x7f79180b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f7916a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79180d07d0 0x7f79180b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57758/0 (socket says 172.21.15.5:57758) 2024-04-25T10:10:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.726+0000 7f7916a00640 1 -- 172.21.15.5:0/2288270560 learned_addr learned my addr 172.21.15.5:0/2288270560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.727+0000 7f7917e00640 1 --2- 172.21.15.5:0/2288270560 >> v2:172.21.15.80:3300/0 conn(0x7f79180d2380 0x7f79180b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.727+0000 7f7916a00640 1 -- 172.21.15.5:0/2288270560 >> v2:172.21.15.80:3300/0 conn(0x7f79180d2380 msgr2=0x7f79180b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.727+0000 7f7916a00640 1 --2- 172.21.15.5:0/2288270560 >> v2:172.21.15.80:3300/0 conn(0x7f79180d2380 0x7f79180b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.727+0000 7f7916a00640 1 -- 172.21.15.5:0/2288270560 >> v1:172.21.15.80:6789/0 conn(0x7f7918161200 legacy=0x7f79180d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.727+0000 7f7916a00640 1 -- 172.21.15.5:0/2288270560 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180d3fd0 con 0x7f79180d07d0 2024-04-25T10:10:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.728+0000 7f7916a00640 1 --2- 172.21.15.5:0/2288270560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d07d0 0x7f79180b4bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f79180cfc80 tx=0x7f790402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efb53b0707be0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.728+0000 7f7916000640 1 -- 172.21.15.5:0/2288270560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904037070 con 0x7f79180d07d0 2024-04-25T10:10:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.728+0000 7f7916000640 1 -- 172.21.15.5:0/2288270560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7904032440 con 0x7f79180d07d0 2024-04-25T10:10:45.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.728+0000 7f7916000640 1 -- 172.21.15.5:0/2288270560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904032810 con 0x7f79180d07d0 2024-04-25T10:10:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 -- 172.21.15.5:0/2288270560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d07d0 msgr2=0x7f79180b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 --2- 172.21.15.5:0/2288270560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d07d0 0x7f79180b4bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f79180cfc80 tx=0x7f790402a230 comp rx=0 tx=0).stop 2024-04-25T10:10:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 -- 172.21.15.5:0/2288270560 shutdown_connections 2024-04-25T10:10:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 --2- 172.21.15.5:0/2288270560 >> v2:172.21.15.80:3300/0 conn(0x7f79180d2380 0x7f79180b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 --2- 172.21.15.5:0/2288270560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d07d0 0x7f79180b4bb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 -- 172.21.15.5:0/2288270560 >> 172.21.15.5:0/2288270560 conn(0x7f791808af80 msgr2=0x7f79180d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 -- 172.21.15.5:0/2288270560 shutdown_connections 2024-04-25T10:10:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.729+0000 7f791ce00640 1 -- 172.21.15.5:0/2288270560 wait complete. 2024-04-25T10:10:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.731+0000 7f791ce00640 1 Processor -- start 2024-04-25T10:10:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.731+0000 7f791ce00640 1 -- start start 2024-04-25T10:10:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f791ce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79180d07d0 0x7f79180ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f791ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 0x7f791814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f791ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7918161200 0x7f79180c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f791ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180b3640 con 0x7f79180d2380 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f791ce00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79180b34c0 con 0x7f7918161200 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f791ce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79180b37c0 con 0x7f79180d07d0 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 0x7f791814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 0x7f791814d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57764/0 (socket says 172.21.15.5:57764) 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 -- 172.21.15.5:0/482275826 learned_addr learned my addr 172.21.15.5:0/482275826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 -- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79180d07d0 msgr2=0x7f79180ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79180d07d0 0x7f79180ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 -- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7918161200 msgr2=0x7f79180c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7918161200 0x7f79180c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.732+0000 7f7916a00640 1 -- 172.21.15.5:0/482275826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79180cbc60 con 0x7f79180d2380 2024-04-25T10:10:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.733+0000 7f7916a00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 0x7f791814d730 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7904000c00 tx=0x7f79040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.733+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904041020 con 0x7f79180d2380 2024-04-25T10:10:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.733+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7904030970 con 0x7f79180d2380 2024-04-25T10:10:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.733+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904037040 con 0x7f79180d2380 2024-04-25T10:10:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.733+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79180cbf40 con 0x7f79180d2380 2024-04-25T10:10:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.733+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79180cc500 con 0x7f79180d2380 2024-04-25T10:10:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.735+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7918068f60 con 0x7f79180d2380 2024-04-25T10:10:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.735+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7904044020 con 0x7f79180d2380 2024-04-25T10:10:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.736+0000 7f790f400640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f806b4d0 0x7f78f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.737+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f79040aafc0 con 0x7f79180d2380 2024-04-25T10:10:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.737+0000 7f7917400640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f806b4d0 0x7f78f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:45.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.739+0000 7f7917400640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f806b4d0 0x7f78f806d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7908009650 tx=0x7f7908006ba0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:45.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.741+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f790404b030 con 0x7f79180d2380 2024-04-25T10:10:45.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.982+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f791806acf0 con 0x7f79180d2380 2024-04-25T10:10:45.989 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:10:45.983+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:10:45.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.987+0000 7f790f400640 1 -- 172.21.15.5:0/482275826 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7904079a60 con 0x7f79180d2380 2024-04-25T10:10:45.993 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:10:45.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f806b4d0 msgr2=0x7f78f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f806b4d0 0x7f78f806d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7908009650 tx=0x7f7908006ba0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 msgr2=0x7f791814d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:45.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 0x7f791814d730 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f7904000c00 tx=0x7f79040046c0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 shutdown_connections 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78f806b4d0 0x7f78f806d990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7918161200 0x7f79180c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79180d2380 0x7f791814d730 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 --2- 172.21.15.5:0/482275826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79180d07d0 0x7f79180ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.990+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 >> 172.21.15.5:0/482275826 conn(0x7f791808af80 msgr2=0x7f791815ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.991+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 shutdown_connections 2024-04-25T10:10:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:45.991+0000 7f791ce00640 1 -- 172.21.15.5:0/482275826 wait complete. 2024-04-25T10:10:46.003 INFO:tasks.ceph.mds.a.smithi005.stderr: -2511> 2024-04-25T10:10:03.334+0000 7f6eac400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:46.004 INFO:tasks.ceph.mds.a.smithi005.stderr: -2023> 2024-04-25T10:10:06.057+0000 7f6ea7400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e255958100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:10:46.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:10:46.069 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:10:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.199+0000 7f4b79400640 1 Processor -- start 2024-04-25T10:10:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.200+0000 7f4b79400640 1 -- start start 2024-04-25T10:10:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.201+0000 7f4b79400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b7415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.201+0000 7f4b79400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7406f080 con 0x7f4b741465d0 2024-04-25T10:10:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.201+0000 7f4b79400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7406f360 con 0x7f4b740ae860 2024-04-25T10:10:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.201+0000 7f4b79400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740ae060 con 0x7f4b74071840 2024-04-25T10:10:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.202+0000 7f4b72800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4b741465d0 0x7f4b7415e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46112/0 (socket says 172.21.15.5:46112) 2024-04-25T10:10:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.202+0000 7f4b72800640 1 -- 172.21.15.5:0/2770306413 learned_addr learned my addr 172.21.15.5:0/2770306413 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.202+0000 7f4b73200640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b7415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.202+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521768511 0 0) 0x7f4b7406f080 con 0x7f4b741465d0 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.202+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b58003520 con 0x7f4b741465d0 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204153011 0 0) 0x7f4b7406f360 con 0x7f4b740ae860 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b7406f080 con 0x7f4b740ae860 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2970787128 0 0) 0x7f4b58003520 con 0x7f4b741465d0 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 msgr2=0x7f4b7415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b7415f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 >> v1:172.21.15.80:6789/0 conn(0x7f4b740ae860 legacy=0x7f4b74071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740ae340 con 0x7f4b741465d0 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4b64001670 con 0x7f4b741465d0 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 >> v1:172.21.15.5:6789/0 conn(0x7f4b741465d0 legacy=0x7f4b7415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b58003b70 0x7f4b58003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740ae060 con 0x7f4b58003b70 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5800ba70 con 0x7f4b58004800 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5800bc70 con 0x7f4b74071840 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.203+0000 7f4b72800640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b58003b70 0x7f4b58003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b73200640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b72800640 1 -- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 msgr2=0x7f4b580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b72800640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b72800640 1 -- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58004800 msgr2=0x7f4b58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b6be00640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b72800640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b72800640 1 -- 172.21.15.5:0/2770306413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b58003520 con 0x7f4b58003b70 2024-04-25T10:10:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b73200640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b74071840 0x7f4b580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.204+0000 7f4b6be00640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58004800 0x7f4b58004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.205+0000 7f4b72800640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b58003b70 0x7f4b58003f70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4b64001eb0 tx=0x7f4b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df3f501bee59b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.205+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64007ce0 con 0x7f4b58003b70 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.205+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64005ad0 con 0x7f4b58003b70 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.205+0000 7f4b71e00640 1 -- 172.21.15.5:0/2770306413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64007070 con 0x7f4b58003b70 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b79400640 1 -- 172.21.15.5:0/2770306413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b58003b70 msgr2=0x7f4b58003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b79400640 1 --2- 172.21.15.5:0/2770306413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b58003b70 0x7f4b58003f70 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4b64001eb0 tx=0x7f4b58005830 comp rx=0 tx=0).stop 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b73200640 1 -- 172.21.15.5:0/2770306413 reap_dead start 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b79400640 1 -- 172.21.15.5:0/2770306413 shutdown_connections 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b79400640 1 -- 172.21.15.5:0/2770306413 >> 172.21.15.5:0/2770306413 conn(0x7f4b7406ab60 msgr2=0x7f4b7406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b79400640 1 -- 172.21.15.5:0/2770306413 shutdown_connections 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.206+0000 7f4b79400640 1 -- 172.21.15.5:0/2770306413 wait complete. 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.208+0000 7f4b79400640 1 Processor -- start 2024-04-25T10:10:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.208+0000 7f4b79400640 1 -- start start 2024-04-25T10:10:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.209+0000 7f4b79400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58003b70 0x7f4b740b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.209+0000 7f4b79400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b741e61a0 0x7f4b740b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.209+0000 7f4b79400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 0x7f4b7415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.209+0000 7f4b79400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7406ece0 con 0x7f4b7415cdf0 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.209+0000 7f4b79400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7406e7d0 con 0x7f4b58003b70 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.209+0000 7f4b79400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7406ee60 con 0x7f4b741e61a0 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 0x7f4b7415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 0x7f4b7415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57796/0 (socket says 172.21.15.5:57796) 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 -- 172.21.15.5:0/1937084943 learned_addr learned my addr 172.21.15.5:0/1937084943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b72800640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58003b70 0x7f4b740b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b6be00640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b741e61a0 0x7f4b740b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 -- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b741e61a0 msgr2=0x7f4b740b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b741e61a0 0x7f4b740b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 -- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58003b70 msgr2=0x7f4b740b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58003b70 0x7f4b740b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 -- 172.21.15.5:0/1937084943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b741f6f30 con 0x7f4b7415cdf0 2024-04-25T10:10:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.210+0000 7f4b73200640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 0x7f4b7415d270 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4b740cf570 tx=0x7f4b6c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.211+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c01c070 con 0x7f4b7415cdf0 2024-04-25T10:10:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.211+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c017400 con 0x7f4b7415cdf0 2024-04-25T10:10:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.211+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0177d0 con 0x7f4b7415cdf0 2024-04-25T10:10:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.211+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b741f7210 con 0x7f4b7415cdf0 2024-04-25T10:10:46.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.211+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b741f77d0 con 0x7f4b7415cdf0 2024-04-25T10:10:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.212+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b74068f60 con 0x7f4b7415cdf0 2024-04-25T10:10:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.213+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b6c01f070 con 0x7f4b7415cdf0 2024-04-25T10:10:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.214+0000 7f4b6aa00640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:46.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.214+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6620+0+0 (secure 0 0 0) 0x7f4b6c08a780 con 0x7f4b7415cdf0 2024-04-25T10:10:46.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.215+0000 7f4b72800640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.216+0000 7f4b72800640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4b5006b4d0 0x7f4b5006d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4b64001eb0 tx=0x7f4b64009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.218+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b6c012070 con 0x7f4b7415cdf0 2024-04-25T10:10:46.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.457+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4b741e80d0 con 0x7f4b7415cdf0 2024-04-25T10:10:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.990+0000 7f4b6aa00640 1 -- 172.21.15.5:0/1937084943 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b6c059140 con 0x7f4b7415cdf0 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.994+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4b5006b4d0 msgr2=0x7f4b5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.994+0000 7f4b79400640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4b5006b4d0 0x7f4b5006d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4b64001eb0 tx=0x7f4b64009030 comp rx=0 tx=0).stop 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.994+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 msgr2=0x7f4b7415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.994+0000 7f4b79400640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 0x7f4b7415d270 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4b740cf570 tx=0x7f4b6c00f4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.994+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 shutdown_connections 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.994+0000 7f4b79400640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b7415cdf0 0x7f4b7415d270 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.995+0000 7f4b79400640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4b741e61a0 0x7f4b740b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.995+0000 7f4b79400640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.995+0000 7f4b79400640 1 --2- 172.21.15.5:0/1937084943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b58003b70 0x7f4b740b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.995+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 >> 172.21.15.5:0/1937084943 conn(0x7f4b7406ab60 msgr2=0x7f4b74145730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.995+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 shutdown_connections 2024-04-25T10:10:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:46.995+0000 7f4b79400640 1 -- 172.21.15.5:0/1937084943 wait complete. 2024-04-25T10:10:47.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:10:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.232+0000 7fb10b800640 1 Processor -- start 2024-04-25T10:10:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.233+0000 7fb10b800640 1 -- start start 2024-04-25T10:10:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb10b800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb10415e5f0 0x7fb10415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb10b800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb104162450 con 0x7fb104161520 2024-04-25T10:10:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb10b800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb104161bb0 con 0x7fb10415d980 2024-04-25T10:10:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb10b800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb104161e90 con 0x7fb10415e5f0 2024-04-25T10:10:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb103e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb104161520 0x7fb1040d1a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59986/0 (socket says 172.21.15.5:59986) 2024-04-25T10:10:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb103e00640 1 -- 172.21.15.5:0/3672783245 learned_addr learned my addr 172.21.15.5:0/3672783245 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb109800640 1 --2- 172.21.15.5:0/3672783245 >> v2:172.21.15.80:3300/0 conn(0x7fb10415e5f0 0x7fb10415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746882413 0 0) 0x7fb104162450 con 0x7fb104161520 2024-04-25T10:10:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.235+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e8003520 con 0x7fb104161520 2024-04-25T10:10:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 956530812 0 0) 0x7fb0e8003520 con 0x7fb104161520 2024-04-25T10:10:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 >> v2:172.21.15.80:3300/0 conn(0x7fb10415e5f0 msgr2=0x7fb10415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 --2- 172.21.15.5:0/3672783245 >> v2:172.21.15.80:3300/0 conn(0x7fb10415e5f0 0x7fb10415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 >> v1:172.21.15.80:6789/0 conn(0x7fb10415d980 legacy=0x7fb1040b9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104162170 con 0x7fb104161520 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb0ec001670 con 0x7fb104161520 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 >> v1:172.21.15.5:6789/0 conn(0x7fb104161520 legacy=0x7fb1040d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e80039b0 0x7fb0e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e8004800 0x7fb0e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80051c0 0x7fb0e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104161e90 con 0x7fb0e80039b0 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e800c060 con 0x7fb0e80051c0 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e800c260 con 0x7fb0e8004800 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.236+0000 7fb108e00640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e80039b0 0x7fb0e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb103e00640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e8004800 0x7fb0e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb109800640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80051c0 0x7fb0e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb108e00640 1 -- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e8004800 msgr2=0x7fb0e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb108e00640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb0e8004800 0x7fb0e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb108e00640 1 -- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80051c0 msgr2=0x7fb0e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb108e00640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80051c0 0x7fb0e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb108e00640 1 -- 172.21.15.5:0/3672783245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8003520 con 0x7fb0e80039b0 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb109800640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80051c0 0x7fb0e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.237+0000 7fb109800640 1 -- 172.21.15.5:0/3672783245 reap_dead start 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.238+0000 7fb108e00640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e80039b0 0x7fb0e8003dd0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4001e50 tx=0x7fb0e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28527be55d3eabda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.238+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4001a70 con 0x7fb0e80039b0 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.238+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f402ae40 con 0x7fb0e80039b0 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.238+0000 7fb103400640 1 -- 172.21.15.5:0/3672783245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4003bb0 con 0x7fb0e80039b0 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 -- 172.21.15.5:0/3672783245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e80039b0 msgr2=0x7fb0e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e80039b0 0x7fb0e8003dd0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4001e50 tx=0x7fb0e8005830 comp rx=0 tx=0).stop 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 -- 172.21.15.5:0/3672783245 shutdown_connections 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 --2- 172.21.15.5:0/3672783245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0e80039b0 0x7fb0e8003dd0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 -- 172.21.15.5:0/3672783245 >> 172.21.15.5:0/3672783245 conn(0x7fb10406c3d0 msgr2=0x7fb10406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 -- 172.21.15.5:0/3672783245 shutdown_connections 2024-04-25T10:10:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.239+0000 7fb10b800640 1 -- 172.21.15.5:0/3672783245 wait complete. 2024-04-25T10:10:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.241+0000 7fb10b800640 1 Processor -- start 2024-04-25T10:10:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.241+0000 7fb10b800640 1 -- start start 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb10b800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80039b0 0x7fb1040c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb10b800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 0x7fb1040d2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb10b800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1040d2850 0x7fb1041474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb10b800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10415ea00 con 0x7fb1040c9c10 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb10b800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10415dd60 con 0x7fb0e80039b0 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb10b800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10415eb80 con 0x7fb1040d2850 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb103e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 0x7fb1040d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb103e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 0x7fb1040d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53356/0 (socket says 172.21.15.5:53356) 2024-04-25T10:10:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.242+0000 7fb103e00640 1 -- 172.21.15.5:0/2364507289 learned_addr learned my addr 172.21.15.5:0/2364507289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb108e00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80039b0 0x7fb1040c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb109800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1040d2850 0x7fb1041474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb103e00640 1 -- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1040d2850 msgr2=0x7fb1041474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb103e00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1040d2850 0x7fb1041474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb103e00640 1 -- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80039b0 msgr2=0x7fb1040c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb103e00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80039b0 0x7fb1040c5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb103e00640 1 -- 172.21.15.5:0/2364507289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040c6380 con 0x7fb1040c9c10 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb103e00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 0x7fb1040d2310 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e700 tx=0x7fb0ec00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00ee10 con 0x7fb1040c9c10 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.243+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec018070 con 0x7fb1040c9c10 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.244+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec00cab0 con 0x7fb1040c9c10 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.244+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1040c8e50 con 0x7fb1040c9c10 2024-04-25T10:10:47.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.244+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1040c9410 con 0x7fb1040c9c10 2024-04-25T10:10:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.245+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb104069040 con 0x7fb1040c9c10 2024-04-25T10:10:47.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.246+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0ec00cc50 con 0x7fb1040c9c10 2024-04-25T10:10:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.247+0000 7fb100c00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0e006b4d0 0x7fb0e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.247+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6620+0+0 (secure 0 0 0) 0x7fb0ec08a390 con 0x7fb1040c9c10 2024-04-25T10:10:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.247+0000 7fb108e00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0e006b4d0 0x7fb0e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:47.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.249+0000 7fb108e00640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0e006b4d0 0x7fb0e006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4032000 tx=0x7fb0f402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:47.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.251+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1040c6380 con 0x7fb1040c9c10 2024-04-25T10:10:47.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.453+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1040d2c30 con 0x7fb1040c9c10 2024-04-25T10:10:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.994+0000 7fb100c00640 1 -- 172.21.15.5:0/2364507289 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0ec058de0 con 0x7fb1040c9c10 2024-04-25T10:10:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:10:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0e006b4d0 msgr2=0x7fb0e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0e006b4d0 0x7fb0e006d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4032000 tx=0x7fb0f402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 msgr2=0x7fb1040d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 0x7fb1040d2310 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00e700 tx=0x7fb0ec00ebc0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 shutdown_connections 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb1040d2850 0x7fb1041474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1040c9c10 0x7fb1040d2310 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb0e006b4d0 0x7fb0e006d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 --2- 172.21.15.5:0/2364507289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0e80039b0 0x7fb1040c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.998+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 >> 172.21.15.5:0/2364507289 conn(0x7fb10406c3d0 msgr2=0x7fb10415ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.999+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 shutdown_connections 2024-04-25T10:10:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:47.999+0000 7fb10b800640 1 -- 172.21.15.5:0/2364507289 wait complete. 2024-04-25T10:10:48.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:10:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.246+0000 7f4c68a00640 1 Processor -- start 2024-04-25T10:10:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.246+0000 7f4c68a00640 1 -- start start 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c68a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4c640cc8f0 0x7f4c640ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c68a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6406d780 con 0x7f4c641426a0 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c68a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6406da60 con 0x7f4c64142010 2024-04-25T10:10:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c68a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c641423f0 con 0x7f4c640cc8f0 2024-04-25T10:10:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c62800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4c641426a0 0x7f4c64142af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60002/0 (socket says 172.21.15.5:60002) 2024-04-25T10:10:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c62800640 1 -- 172.21.15.5:0/3992165965 learned_addr learned my addr 172.21.15.5:0/3992165965 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.248+0000 7f4c5be00640 1 --2- 172.21.15.5:0/3992165965 >> v2:172.21.15.80:3300/0 conn(0x7f4c640cc8f0 0x7f4c640ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136212049 0 0) 0x7f4c6406d780 con 0x7f4c641426a0 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c48003520 con 0x7f4c641426a0 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2859604434 0 0) 0x7f4c48003520 con 0x7f4c641426a0 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 >> v2:172.21.15.80:3300/0 conn(0x7f4c640cc8f0 msgr2=0x7f4c640ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 --2- 172.21.15.5:0/3992165965 >> v2:172.21.15.80:3300/0 conn(0x7f4c640cc8f0 0x7f4c640ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 >> v1:172.21.15.80:6789/0 conn(0x7f4c64142010 legacy=0x7f4c6406e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6406dd40 con 0x7f4c641426a0 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4c5c0019f0 con 0x7f4c641426a0 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 >> v1:172.21.15.5:6789/0 conn(0x7f4c641426a0 legacy=0x7f4c64142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c480039b0 0x7f4c48003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c48004800 0x7f4c48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c480051c0 0x7f4c4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c641423f0 con 0x7f4c48004800 2024-04-25T10:10:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4800c090 con 0x7f4c480051c0 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.249+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4800c290 con 0x7f4c480039b0 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c5be00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c48004800 0x7f4c48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c62800640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c480051c0 0x7f4c4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c5be00640 1 -- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c480039b0 msgr2=0x7f4c48003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c61e00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c480039b0 0x7f4c48003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c5be00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c480039b0 0x7f4c48003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c5be00640 1 -- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c480051c0 msgr2=0x7f4c4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c5be00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c480051c0 0x7f4c4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c5be00640 1 -- 172.21.15.5:0/3992165965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48003520 con 0x7f4c48004800 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c62800640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c480051c0 0x7f4c4800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.250+0000 7f4c62800640 1 -- 172.21.15.5:0/3992165965 reap_dead start 2024-04-25T10:10:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.251+0000 7f4c5be00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c48004800 0x7f4c48004c80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ac0 tx=0x7f4c48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c21036ce663270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.251+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c036070 con 0x7f4c48004800 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.251+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c031400 con 0x7f4c48004800 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.251+0000 7f4c61400640 1 -- 172.21.15.5:0/3992165965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c009690 con 0x7f4c48004800 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 -- 172.21.15.5:0/3992165965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c48004800 msgr2=0x7f4c48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c48004800 0x7f4c48004c80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009ac0 tx=0x7f4c48005830 comp rx=0 tx=0).stop 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 -- 172.21.15.5:0/3992165965 shutdown_connections 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 --2- 172.21.15.5:0/3992165965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c48004800 0x7f4c48004c80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 -- 172.21.15.5:0/3992165965 >> 172.21.15.5:0/3992165965 conn(0x7f4c64066f80 msgr2=0x7f4c6406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 -- 172.21.15.5:0/3992165965 shutdown_connections 2024-04-25T10:10:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.252+0000 7f4c68a00640 1 -- 172.21.15.5:0/3992165965 wait complete. 2024-04-25T10:10:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.254+0000 7f4c68a00640 1 Processor -- start 2024-04-25T10:10:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.254+0000 7f4c68a00640 1 -- start start 2024-04-25T10:10:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.255+0000 7f4c68a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c48004800 0x7f4c6413d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.255+0000 7f4c68a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c64140320 0x7f4c640b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.255+0000 7f4c68a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 0x7f4c6413e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.255+0000 7f4c68a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6406e300 con 0x7f4c6413e540 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.255+0000 7f4c68a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640cc210 con 0x7f4c48004800 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.255+0000 7f4c68a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640ccd80 con 0x7f4c64140320 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 0x7f4c6413e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 0x7f4c6413e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53380/0 (socket says 172.21.15.5:53380) 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 -- 172.21.15.5:0/1945171178 learned_addr learned my addr 172.21.15.5:0/1945171178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c61e00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c48004800 0x7f4c6413d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c5be00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c64140320 0x7f4c640b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 -- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c64140320 msgr2=0x7f4c640b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c64140320 0x7f4c640b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 -- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c48004800 msgr2=0x7f4c6413d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c48004800 0x7f4c6413d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 -- 172.21.15.5:0/1945171178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c641f6a90 con 0x7f4c6413e540 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.256+0000 7f4c62800640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 0x7f4c6413e9c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4c640ce3e0 tx=0x7f4c5c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.257+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c019070 con 0x7f4c6413e540 2024-04-25T10:10:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.257+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c00bb20 con 0x7f4c6413e540 2024-04-25T10:10:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.257+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c007030 con 0x7f4c6413e540 2024-04-25T10:10:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.257+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c641f6d70 con 0x7f4c6413e540 2024-04-25T10:10:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.257+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c641f7850 con 0x7f4c6413e540 2024-04-25T10:10:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.258+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c64069040 con 0x7f4c6413e540 2024-04-25T10:10:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.259+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c5c0071d0 con 0x7f4c6413e540 2024-04-25T10:10:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.260+0000 7f4c5a000640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c4006b4d0 0x7f4c4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.260+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6153+0+0 (secure 0 0 0) 0x7f4c5c08a230 con 0x7f4c6413e540 2024-04-25T10:10:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.261+0000 7f4c61e00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c4006b4d0 0x7f4c4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.262+0000 7f4c61e00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c4006b4d0 0x7f4c4006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4c64050820 tx=0x7f4c54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.264+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c5c058e50 con 0x7f4c6413e540 2024-04-25T10:10:48.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.465+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c6415b9c0 con 0x7f4c6413e540 2024-04-25T10:10:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:48.998+0000 7f4c5a000640 1 -- 172.21.15.5:0/1945171178 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4c5c058750 con 0x7f4c6413e540 2024-04-25T10:10:49.004 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.001+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c4006b4d0 msgr2=0x7f4c4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.001+0000 7f4c68a00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c4006b4d0 0x7f4c4006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4c64050820 tx=0x7f4c54009040 comp rx=0 tx=0).stop 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.001+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 msgr2=0x7f4c6413e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.001+0000 7f4c68a00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 0x7f4c6413e9c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4c640ce3e0 tx=0x7f4c5c00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 shutdown_connections 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c6413e540 0x7f4c6413e9c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c64140320 0x7f4c640b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c4006b4d0 0x7f4c4006d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 --2- 172.21.15.5:0/1945171178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c48004800 0x7f4c6413d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 >> 172.21.15.5:0/1945171178 conn(0x7f4c64066f80 msgr2=0x7f4c640ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.002+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 shutdown_connections 2024-04-25T10:10:49.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.003+0000 7f4c68a00640 1 -- 172.21.15.5:0/1945171178 wait complete. 2024-04-25T10:10:49.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:10:49.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.219+0000 7f901ce00640 1 Processor -- start 2024-04-25T10:10:49.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.219+0000 7f901ce00640 1 -- start start 2024-04-25T10:10:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.220+0000 7f901ce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f9018071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.220+0000 7f901ce00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901815f3e0 con 0x7f901815bdc0 2024-04-25T10:10:49.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.220+0000 7f901ce00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901815f680 con 0x7f901815ed20 2024-04-25T10:10:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.220+0000 7f901ce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9018144780 con 0x7f9018071000 2024-04-25T10:10:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.220+0000 7f9017e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f901815bdc0 0x7f901815c230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60010/0 (socket says 172.21.15.5:60010) 2024-04-25T10:10:49.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.220+0000 7f9017e00640 1 -- 172.21.15.5:0/2336214190 learned_addr learned my addr 172.21.15.5:0/2336214190 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:49.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938840275 0 0) 0x7f901815f3e0 con 0x7f901815bdc0 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ffc003520 con 0x7f901815bdc0 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016a00640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f9018071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858490091 0 0) 0x7f8ffc003520 con 0x7f901815bdc0 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 msgr2=0x7f9018071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f9018071490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 >> v1:172.21.15.80:6789/0 conn(0x7f901815ed20 legacy=0x7f9018144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901815f960 con 0x7f901815bdc0 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f9010001a10 con 0x7f901815bdc0 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.221+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 >> v1:172.21.15.5:6789/0 conn(0x7f901815bdc0 legacy=0x7f901815c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016000640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f8ffc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016000640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f8ffc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016000640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ffc004d40 0x7f8ffc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018144780 con 0x7f8ffc004010 2024-04-25T10:10:49.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc00bc00 con 0x7f8ffc004d40 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc00be00 con 0x7f9018071000 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016a00640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f8ffc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9017400640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f8ffc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016a00640 1 -- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 msgr2=0x7f8ffc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016a00640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f8ffc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016a00640 1 -- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ffc004d40 msgr2=0x7f8ffc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016a00640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ffc004d40 0x7f8ffc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9016a00640 1 -- 172.21.15.5:0/2336214190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc003520 con 0x7f8ffc004010 2024-04-25T10:10:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.222+0000 7f9017400640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9018071000 0x7f8ffc0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.223+0000 7f9016a00640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f8ffc004800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f900400ba30 tx=0x7f8ffc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa116914b1d1f1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004030070 con 0x7f8ffc004010 2024-04-25T10:10:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900402ade0 con 0x7f8ffc004010 2024-04-25T10:10:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f9016000640 1 -- 172.21.15.5:0/2336214190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400b600 con 0x7f8ffc004010 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f901ce00640 1 -- 172.21.15.5:0/2336214190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 msgr2=0x7f8ffc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f901ce00640 1 --2- 172.21.15.5:0/2336214190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f8ffc004800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f900400ba30 tx=0x7f8ffc005830 comp rx=0 tx=0).stop 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f9017e00640 1 -- 172.21.15.5:0/2336214190 reap_dead start 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f901ce00640 1 -- 172.21.15.5:0/2336214190 shutdown_connections 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f901ce00640 1 -- 172.21.15.5:0/2336214190 >> 172.21.15.5:0/2336214190 conn(0x7f9018066f20 msgr2=0x7f901806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.224+0000 7f901ce00640 1 -- 172.21.15.5:0/2336214190 shutdown_connections 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.225+0000 7f901ce00640 1 -- 172.21.15.5:0/2336214190 wait complete. 2024-04-25T10:10:49.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.226+0000 7f901ce00640 1 Processor -- start 2024-04-25T10:10:49.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.226+0000 7f901ce00640 1 -- start start 2024-04-25T10:10:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.227+0000 7f901ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f901815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.227+0000 7f901ce00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9018135600 0x7f90180c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.227+0000 7f901ce00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f90180b21b0 0x7f90180b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.227+0000 7f901ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018145060 con 0x7f8ffc004010 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.227+0000 7f901ce00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180cdd50 con 0x7f9018135600 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.227+0000 7f901ce00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90181451e0 con 0x7f90180b21b0 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f901815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f901815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53410/0 (socket says 172.21.15.5:53410) 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 -- 172.21.15.5:0/2467392155 learned_addr learned my addr 172.21.15.5:0/2467392155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9016a00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9018135600 0x7f90180c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017e00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f90180b21b0 0x7f90180b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 -- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f90180b21b0 msgr2=0x7f90180b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f90180b21b0 0x7f90180b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 -- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9018135600 msgr2=0x7f90180c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9018135600 0x7f90180c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9017400640 1 -- 172.21.15.5:0/2467392155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9018135b70 con 0x7f8ffc004010 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.228+0000 7f9016a00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9018135600 0x7f90180c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.229+0000 7f9017400640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f901815af60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f900800d7d0 tx=0x7f900800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.229+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008005070 con 0x7f8ffc004010 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.229+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008005230 con 0x7f8ffc004010 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.229+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008007030 con 0x7f8ffc004010 2024-04-25T10:10:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.229+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9018135e50 con 0x7f8ffc004010 2024-04-25T10:10:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.229+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9018136410 con 0x7f8ffc004010 2024-04-25T10:10:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.230+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90180ceea0 con 0x7f8ffc004010 2024-04-25T10:10:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.231+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f900801a020 con 0x7f8ffc004010 2024-04-25T10:10:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.232+0000 7f900f400640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ff806b4d0 0x7f8ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.232+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f9008089a30 con 0x7f8ffc004010 2024-04-25T10:10:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.233+0000 7f9016a00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ff806b4d0 0x7f8ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.234+0000 7f9016a00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ff806b4d0 0x7f8ff806d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f900400ba00 tx=0x7f9004030000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.237+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90180ceea0 con 0x7f8ffc004010 2024-04-25T10:10:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.446+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9018068f60 con 0x7f8ffc004010 2024-04-25T10:10:49.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.453+0000 7f900f400640 1 -- 172.21.15.5:0/2467392155 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90080588d0 con 0x7f8ffc004010 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.456+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ff806b4d0 msgr2=0x7f8ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.456+0000 7f901ce00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ff806b4d0 0x7f8ff806d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f900400ba00 tx=0x7f9004030000 comp rx=0 tx=0).stop 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.456+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 msgr2=0x7f901815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.456+0000 7f901ce00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f901815af60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f900800d7d0 tx=0x7f900800dc90 comp rx=0 tx=0).stop 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 shutdown_connections 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f90180b21b0 0x7f90180b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ff806b4d0 0x7f8ff806d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9018135600 0x7f90180c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 --2- 172.21.15.5:0/2467392155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ffc004010 0x7f901815af60 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 >> 172.21.15.5:0/2467392155 conn(0x7f9018066f20 msgr2=0x7f9018071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 shutdown_connections 2024-04-25T10:10:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.457+0000 7f901ce00640 1 -- 172.21.15.5:0/2467392155 wait complete. 2024-04-25T10:10:49.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:10:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.687+0000 7fc999800640 1 Processor -- start 2024-04-25T10:10:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.687+0000 7fc999800640 1 -- start start 2024-04-25T10:10:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc999800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9940b0d70 0x7fc9940b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc999800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9940b6010 0x7fc99406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc999800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc99406b0a0 0x7fc994161350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc999800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940b4f70 con 0x7fc99406b0a0 2024-04-25T10:10:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc999800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940b17d0 con 0x7fc9940b6010 2024-04-25T10:10:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc999800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940b1ab0 con 0x7fc9940b0d70 2024-04-25T10:10:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc998e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc99406b0a0 0x7fc994161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc998e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc99406b0a0 0x7fc994161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53416/0 (socket says 172.21.15.5:53416) 2024-04-25T10:10:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.689+0000 7fc998e00640 1 -- 172.21.15.5:0/1680835236 learned_addr learned my addr 172.21.15.5:0/1680835236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc98be00640 1 --2- 172.21.15.5:0/1680835236 >> v2:172.21.15.80:3300/0 conn(0x7fc9940b6010 0x7fc99406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc993e00640 1 --2- 172.21.15.5:0/1680835236 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9940b0d70 0x7fc9940b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc998e00640 1 -- 172.21.15.5:0/1680835236 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9940b0d70 msgr2=0x7fc9940b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc998e00640 1 --2- 172.21.15.5:0/1680835236 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9940b0d70 0x7fc9940b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc998e00640 1 -- 172.21.15.5:0/1680835236 >> v2:172.21.15.80:3300/0 conn(0x7fc9940b6010 msgr2=0x7fc99406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc998e00640 1 --2- 172.21.15.5:0/1680835236 >> v2:172.21.15.80:3300/0 conn(0x7fc9940b6010 0x7fc99406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc998e00640 1 -- 172.21.15.5:0/1680835236 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940b51e0 con 0x7fc99406b0a0 2024-04-25T10:10:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.690+0000 7fc98be00640 1 --2- 172.21.15.5:0/1680835236 >> v2:172.21.15.80:3300/0 conn(0x7fc9940b6010 0x7fc99406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.691+0000 7fc998e00640 1 --2- 172.21.15.5:0/1680835236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99406b0a0 0x7fc994161350 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc994072550 tx=0x7fc98c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba900a01a2dc4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.691+0000 7fc993400640 1 -- 172.21.15.5:0/1680835236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c036070 con 0x7fc99406b0a0 2024-04-25T10:10:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.691+0000 7fc993400640 1 -- 172.21.15.5:0/1680835236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c031600 con 0x7fc99406b0a0 2024-04-25T10:10:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.691+0000 7fc993400640 1 -- 172.21.15.5:0/1680835236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0319d0 con 0x7fc99406b0a0 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 -- 172.21.15.5:0/1680835236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99406b0a0 msgr2=0x7fc994161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 --2- 172.21.15.5:0/1680835236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99406b0a0 0x7fc994161350 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc994072550 tx=0x7fc98c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 -- 172.21.15.5:0/1680835236 shutdown_connections 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 --2- 172.21.15.5:0/1680835236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99406b0a0 0x7fc994161350 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 --2- 172.21.15.5:0/1680835236 >> v2:172.21.15.80:3300/0 conn(0x7fc9940b6010 0x7fc99406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 --2- 172.21.15.5:0/1680835236 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc9940b0d70 0x7fc9940b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 -- 172.21.15.5:0/1680835236 >> 172.21.15.5:0/1680835236 conn(0x7fc9940999f0 msgr2=0x7fc9940b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 -- 172.21.15.5:0/1680835236 shutdown_connections 2024-04-25T10:10:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.692+0000 7fc999800640 1 -- 172.21.15.5:0/1680835236 wait complete. 2024-04-25T10:10:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.694+0000 7fc999800640 1 Processor -- start 2024-04-25T10:10:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.694+0000 7fc999800640 1 -- start start 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc999800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99406b0a0 0x7fc99415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc999800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 0x7fc994160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc999800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9940b6010 0x7fc994160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc999800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994161890 con 0x7fc9940b0d70 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc999800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940b1d90 con 0x7fc9940b6010 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc999800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc994161a10 con 0x7fc99406b0a0 2024-04-25T10:10:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc98be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 0x7fc994160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc98be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 0x7fc994160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53424/0 (socket says 172.21.15.5:53424) 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.695+0000 7fc98be00640 1 -- 172.21.15.5:0/3314284680 learned_addr learned my addr 172.21.15.5:0/3314284680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc993e00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99406b0a0 0x7fc99415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc998e00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9940b6010 0x7fc994160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc98be00640 1 -- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99406b0a0 msgr2=0x7fc99415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc98be00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99406b0a0 0x7fc99415d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc98be00640 1 -- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9940b6010 msgr2=0x7fc994160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc98be00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9940b6010 0x7fc994160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc98be00640 1 -- 172.21.15.5:0/3314284680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9941388e0 con 0x7fc9940b0d70 2024-04-25T10:10:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc98be00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 0x7fc994160a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00c7f0 tx=0x7fc97c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c015070 con 0x7fc9940b0d70 2024-04-25T10:10:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97c007820 con 0x7fc9940b0d70 2024-04-25T10:10:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.696+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994137d60 con 0x7fc9940b0d70 2024-04-25T10:10:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.697+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97c007bd0 con 0x7fc9940b0d70 2024-04-25T10:10:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.697+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994138320 con 0x7fc9940b0d70 2024-04-25T10:10:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.698+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994068f60 con 0x7fc9940b0d70 2024-04-25T10:10:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.699+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc97c018070 con 0x7fc9940b0d70 2024-04-25T10:10:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.700+0000 7fc990c00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.700+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc97c08a270 con 0x7fc9940b0d70 2024-04-25T10:10:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.700+0000 7fc993e00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.701+0000 7fc993e00640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc98400a990 tx=0x7fc984008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.704+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc97c059110 con 0x7fc9940b0d70 2024-04-25T10:10:49.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:49.978+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fc9940b0740 con 0x7fc9940b0d70 2024-04-25T10:10:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.001+0000 7fc990c00640 1 -- 172.21.15.5:0/3314284680 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v442) v1 ==== 167+0+0 (secure 0 0 0) 0x7fc97c058a10 con 0x7fc9940b0d70 2024-04-25T10:10:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 msgr2=0x7fc97006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc98400a990 tx=0x7fc984008040 comp rx=0 tx=0).stop 2024-04-25T10:10:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 msgr2=0x7fc994160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 0x7fc994160a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc97c00c7f0 tx=0x7fc97c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 shutdown_connections 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc97006b4d0 0x7fc97006d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9940b6010 0x7fc994160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9940b0d70 0x7fc994160a80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 --2- 172.21.15.5:0/3314284680 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc99406b0a0 0x7fc99415d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 >> 172.21.15.5:0/3314284680 conn(0x7fc9940999f0 msgr2=0x7fc994145b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 shutdown_connections 2024-04-25T10:10:50.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.004+0000 7fc999800640 1 -- 172.21.15.5:0/3314284680 wait complete. 2024-04-25T10:10:50.030 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:10:50.031 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:10:50.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.242+0000 7f2df9800640 1 Processor -- start 2024-04-25T10:10:50.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.242+0000 7f2df9800640 1 -- start start 2024-04-25T10:10:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df415ce00 0x7f2df415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df9800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2df40cd300 0x7f2df40cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df9800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df4071d30 con 0x7f2df4141da0 2024-04-25T10:10:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df9800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df40cdcd0 con 0x7f2df40cd300 2024-04-25T10:10:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df9800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4142180 con 0x7f2df415ce00 2024-04-25T10:10:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2df40cd300 0x7f2df40cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2df40cd300 0x7f2df40cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53436/0 (socket says 172.21.15.5:53436) 2024-04-25T10:10:50.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 -- 172.21.15.5:0/4087882356 learned_addr learned my addr 172.21.15.5:0/4087882356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df3400640 1 --2- 172.21.15.5:0/4087882356 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df415ce00 0x7f2df415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 -- 172.21.15.5:0/4087882356 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df415ce00 msgr2=0x7f2df415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 --2- 172.21.15.5:0/4087882356 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df415ce00 0x7f2df415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 -- 172.21.15.5:0/4087882356 >> v1:172.21.15.80:6789/0 conn(0x7f2df4141da0 legacy=0x7f2df406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.244+0000 7f2df8e00640 1 -- 172.21.15.5:0/4087882356 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df406d620 con 0x7f2df40cd300 2024-04-25T10:10:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.245+0000 7f2df8e00640 1 --2- 172.21.15.5:0/4087882356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df40cd300 0x7f2df40cd790 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009e50 tx=0x7f2dec02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112b7a15b60a9dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df2a00640 1 -- 172.21.15.5:0/4087882356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec036070 con 0x7f2df40cd300 2024-04-25T10:10:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df2a00640 1 -- 172.21.15.5:0/4087882356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec031600 con 0x7f2df40cd300 2024-04-25T10:10:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df2a00640 1 -- 172.21.15.5:0/4087882356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0319d0 con 0x7f2df40cd300 2024-04-25T10:10:50.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 -- 172.21.15.5:0/4087882356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df40cd300 msgr2=0x7f2df40cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 --2- 172.21.15.5:0/4087882356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df40cd300 0x7f2df40cd790 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009e50 tx=0x7f2dec02a710 comp rx=0 tx=0).stop 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 -- 172.21.15.5:0/4087882356 shutdown_connections 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 --2- 172.21.15.5:0/4087882356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df40cd300 0x7f2df40cd790 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 --2- 172.21.15.5:0/4087882356 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df415ce00 0x7f2df415d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 -- 172.21.15.5:0/4087882356 >> 172.21.15.5:0/4087882356 conn(0x7f2df4066f20 msgr2=0x7f2df406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.246+0000 7f2df9800640 1 -- 172.21.15.5:0/4087882356 shutdown_connections 2024-04-25T10:10:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.247+0000 7f2df9800640 1 -- 172.21.15.5:0/4087882356 wait complete. 2024-04-25T10:10:50.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.248+0000 7f2df9800640 1 Processor -- start 2024-04-25T10:10:50.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.248+0000 7f2df9800640 1 -- start start 2024-04-25T10:10:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.249+0000 7f2df9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df40cd300 0x7f2df4139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.249+0000 7f2df9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 0x7f2df415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.249+0000 7f2df9800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2df415ce00 0x7f2df4137260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.249+0000 7f2df9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df406e810 con 0x7f2df4141da0 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.249+0000 7f2df9800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df406dbe0 con 0x7f2df415ce00 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.249+0000 7f2df9800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4071000 con 0x7f2df40cd300 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 0x7f2df415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 0x7f2df415abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53450/0 (socket says 172.21.15.5:53450) 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 -- 172.21.15.5:0/1456746040 learned_addr learned my addr 172.21.15.5:0/1456746040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3e00640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df40cd300 0x7f2df4139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df8e00640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2df415ce00 0x7f2df4137260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 -- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df40cd300 msgr2=0x7f2df4139560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df40cd300 0x7f2df4139560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 -- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2df415ce00 msgr2=0x7f2df4137260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2df415ce00 0x7f2df4137260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df3400640 1 -- 172.21.15.5:0/1456746040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df4142550 con 0x7f2df4141da0 2024-04-25T10:10:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.250+0000 7f2df8e00640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2df415ce00 0x7f2df4137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.251+0000 7f2df3400640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 0x7f2df415abc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2de000d5f0 tx=0x7f2de000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.251+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00125f0 con 0x7f2df4141da0 2024-04-25T10:10:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.251+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de0019070 con 0x7f2df4141da0 2024-04-25T10:10:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.251+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00129d0 con 0x7f2df4141da0 2024-04-25T10:10:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.251+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df4142830 con 0x7f2df4141da0 2024-04-25T10:10:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.251+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df415b4c0 con 0x7f2df4141da0 2024-04-25T10:10:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.252+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4068f60 con 0x7f2df4141da0 2024-04-25T10:10:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.253+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de0012b70 con 0x7f2df4141da0 2024-04-25T10:10:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.254+0000 7f2debe00640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dd406b4d0 0x7f2dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.254+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2de008a540 con 0x7f2df4141da0 2024-04-25T10:10:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.255+0000 7f2df3e00640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dd406b4d0 0x7f2dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.256+0000 7f2df3e00640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dd406b4d0 0x7f2dd406d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2df415b250 tx=0x7f2de4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.260+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2de0059470 con 0x7f2df4141da0 2024-04-25T10:10:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.459+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2df40ccae0 con 0x7f2df4141da0 2024-04-25T10:10:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.461+0000 7f2debe00640 1 -- 172.21.15.5:0/1456746040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+13432 (secure 0 0 0) 0x7f2de0058d70 con 0x7f2df4141da0 2024-04-25T10:10:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:50.468 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:10:48.995897+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/841689019":"2024-04-26T10:10:45.985048+0000","172.21.15.5:6832/841689019":"2024-04-26T10:10:45.985048+0000","172.21.15.5:6834/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6832/7055779":"2024-04-26T10:09:57.322177+0000","172.21.15.5:6834/3627376661":"2024-04-26T10:09:12.513203+0000","172.21.15.5:6832/3627376661":"2024-04-26T10:09:12.513203+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.464+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dd406b4d0 msgr2=0x7f2dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.464+0000 7f2df9800640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dd406b4d0 0x7f2dd406d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2df415b250 tx=0x7f2de4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:50.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.464+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 msgr2=0x7f2df415abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.464+0000 7f2df9800640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 0x7f2df415abc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2de000d5f0 tx=0x7f2de000dab0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.464+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 shutdown_connections 2024-04-25T10:10:50.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2dd406b4d0 0x7f2dd406d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2df415ce00 0x7f2df4137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df4141da0 0x7f2df415abc0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 --2- 172.21.15.5:0/1456746040 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2df40cd300 0x7f2df4139560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 >> 172.21.15.5:0/1456746040 conn(0x7f2df4066f20 msgr2=0x7f2df41404f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 shutdown_connections 2024-04-25T10:10:50.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.465+0000 7f2df9800640 1 -- 172.21.15.5:0/1456746040 wait complete. 2024-04-25T10:10:50.490 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:50.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:50.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.707+0000 7f0bb1c00640 1 Processor -- start 2024-04-25T10:10:50.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.707+0000 7f0bb1c00640 1 -- start start 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.708+0000 7f0bb1c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0bac06bc70 0x7f0bac0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.708+0000 7f0bb1c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 0x7f0bac0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.708+0000 7f0bb1c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac0cfd50 con 0x7f0bac15da60 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.708+0000 7f0bb1c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac0d0030 con 0x7f0bac06bc70 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.708+0000 7f0bb1c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0d0310 con 0x7f0bac0ce390 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.709+0000 7f0babe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0bac15da60 0x7f0bac0cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60018/0 (socket says 172.21.15.5:60018) 2024-04-25T10:10:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.709+0000 7f0babe00640 1 -- 172.21.15.5:0/917934234 learned_addr learned my addr 172.21.15.5:0/917934234 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.709+0000 7f0bab400640 1 --2- 172.21.15.5:0/917934234 >> v2:172.21.15.80:3300/0 conn(0x7f0bac06bc70 0x7f0bac0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.709+0000 7f0bb1200640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 0x7f0bac0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.709+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386618266 0 0) 0x7f0bac0cfd50 con 0x7f0bac15da60 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.709+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8c003520 con 0x7f0bac15da60 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 503074598 0 0) 0x7f0b8c003520 con 0x7f0bac15da60 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 msgr2=0x7f0bac0b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 0x7f0bac0b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 >> v2:172.21.15.80:3300/0 conn(0x7f0bac06bc70 msgr2=0x7f0bac0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 --2- 172.21.15.5:0/917934234 >> v2:172.21.15.80:3300/0 conn(0x7f0bac06bc70 0x7f0bac0cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac0d05f0 con 0x7f0bac15da60 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0b9c001670 con 0x7f0bac15da60 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 >> v1:172.21.15.5:6789/0 conn(0x7f0bac15da60 legacy=0x7f0bac0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 0x7f0b8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c004d40 0x7f0b8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0d0030 con 0x7f0b8c0039b0 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0d0310 con 0x7f0b8c004d40 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c00b9f0 con 0x7f0bac0ce390 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.710+0000 7f0babe00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0bab400640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 0x7f0b8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0bb1200640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c004d40 0x7f0b8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0babe00640 1 -- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 msgr2=0x7f0b8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0babe00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bac0ce390 0x7f0b8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0babe00640 1 -- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c004d40 msgr2=0x7f0b8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0babe00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b8c004d40 0x7f0b8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.711+0000 7f0babe00640 1 -- 172.21.15.5:0/917934234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c003520 con 0x7f0b8c0039b0 2024-04-25T10:10:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.712+0000 7f0babe00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00c290 tx=0x7f0b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd553971d758f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.712+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c007d50 con 0x7f0b8c0039b0 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.712+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c005ad0 con 0x7f0b8c0039b0 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.712+0000 7f0baaa00640 1 -- 172.21.15.5:0/917934234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c007070 con 0x7f0b8c0039b0 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1c00640 1 -- 172.21.15.5:0/917934234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0039b0 msgr2=0x7f0b8c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/917934234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b8c0039b0 0x7f0b8c003dd0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00c290 tx=0x7f0b8c005830 comp rx=0 tx=0).stop 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1200640 1 -- 172.21.15.5:0/917934234 reap_dead start 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1c00640 1 -- 172.21.15.5:0/917934234 shutdown_connections 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1c00640 1 -- 172.21.15.5:0/917934234 >> 172.21.15.5:0/917934234 conn(0x7f0bac0d1470 msgr2=0x7f0bac0d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1c00640 1 -- 172.21.15.5:0/917934234 shutdown_connections 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.713+0000 7f0bb1c00640 1 -- 172.21.15.5:0/917934234 wait complete. 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.715+0000 7f0bb1c00640 1 Processor -- start 2024-04-25T10:10:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.715+0000 7f0bb1c00640 1 -- start start 2024-04-25T10:10:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.716+0000 7f0bb1c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b8c0039b0 0x7f0bac0b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.716+0000 7f0bb1c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bac15c820 0x7f0bac0b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.716+0000 7f0bb1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 0x7f0bac0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.716+0000 7f0bb1c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0b1800 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.716+0000 7f0bb1c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0b1680 con 0x7f0bac15c820 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.716+0000 7f0bb1c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0b1980 con 0x7f0b8c0039b0 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 0x7f0bac0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 0x7f0bac0c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53466/0 (socket says 172.21.15.5:53466) 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 -- 172.21.15.5:0/2269775367 learned_addr learned my addr 172.21.15.5:0/2269775367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bab400640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bac15c820 0x7f0bac0b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 -- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b8c0039b0 msgr2=0x7f0bac0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b8c0039b0 0x7f0bac0b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 -- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bac15c820 msgr2=0x7f0bac0b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bac15c820 0x7f0bac0b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 -- 172.21.15.5:0/2269775367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac0c5040 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bab400640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bac15c820 0x7f0bac0b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.717+0000 7f0bb1200640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 0x7f0bac0c7420 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0bac1613b0 tx=0x7f0ba4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.718+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4019020 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.718+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4003610 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.718+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4003a00 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.718+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac0c7960 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.718+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac0b5060 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.719+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac068f60 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.720+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ba401c020 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.722+0000 7f0ba3e00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b8806b4d0 0x7f0b8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.722+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0ba408a2b0 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.722+0000 7f0babe00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b8806b4d0 0x7f0b8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.724+0000 7f0babe00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b8806b4d0 0x7f0b8806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c033000 tx=0x7f0b9c007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.726+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ba4023030 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.937+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bac06c9c0 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.938+0000 7f0ba3e00640 1 -- 172.21.15.5:0/2269775367 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bac0c7960 con 0x7f0bac0c6fa0 2024-04-25T10:10:50.943 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b8806b4d0 msgr2=0x7f0b8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b8806b4d0 0x7f0b8806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c033000 tx=0x7f0b9c007000 comp rx=0 tx=0).stop 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 msgr2=0x7f0bac0c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 0x7f0bac0c7420 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f0bac1613b0 tx=0x7f0ba4011920 comp rx=0 tx=0).stop 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 shutdown_connections 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bac0c6fa0 0x7f0bac0c7420 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0b8806b4d0 0x7f0b8806d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bac15c820 0x7f0bac0b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.941+0000 7f0bb1c00640 1 --2- 172.21.15.5:0/2269775367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0b8c0039b0 0x7f0bac0b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.942+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 >> 172.21.15.5:0/2269775367 conn(0x7f0bac0d1470 msgr2=0x7f0bac0b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.942+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 shutdown_connections 2024-04-25T10:10:50.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:50.942+0000 7f0bb1c00640 1 -- 172.21.15.5:0/2269775367 wait complete. 2024-04-25T10:10:50.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:10:51.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.174+0000 7fcba3200640 1 Processor -- start 2024-04-25T10:10:51.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.174+0000 7fcba3200640 1 -- start start 2024-04-25T10:10:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.175+0000 7fcba3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcba415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.175+0000 7fcba3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba4071d30 con 0x7fcba415ce00 2024-04-25T10:10:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.175+0000 7fcba3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba406d620 con 0x7fcba4141da0 2024-04-25T10:10:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.176+0000 7fcba3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba4142180 con 0x7fcba40cda10 2024-04-25T10:10:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.176+0000 7fcb99400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcba415ce00 0x7fcba40cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60032/0 (socket says 172.21.15.5:60032) 2024-04-25T10:10:51.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.176+0000 7fcb99400640 1 -- 172.21.15.5:0/1277351933 learned_addr learned my addr 172.21.15.5:0/1277351933 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.176+0000 7fcba2800640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcba415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.176+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590774065 0 0) 0x7fcba4071d30 con 0x7fcba415ce00 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.176+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb88003520 con 0x7fcba415ce00 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 529677358 0 0) 0x7fcb88003520 con 0x7fcba415ce00 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 msgr2=0x7fcba415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcba415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 >> v1:172.21.15.80:6789/0 conn(0x7fcba4141da0 legacy=0x7fcba406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba406d900 con 0x7fcba415ce00 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fcb8c001670 con 0x7fcba415ce00 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 >> v1:172.21.15.5:6789/0 conn(0x7fcba415ce00 legacy=0x7fcba40cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb880039e0 0x7fcb88003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb88004800 0x7fcb88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcb880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba4142180 con 0x7fcb88004800 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8800baa0 con 0x7fcb880039e0 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8800bca0 con 0x7fcba40cda10 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.177+0000 7fcb99400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb88004800 0x7fcb88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcba1e00640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb880039e0 0x7fcb88003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcba2800640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcb880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcb99400640 1 -- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 msgr2=0x7fcb880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcb99400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcb880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcb99400640 1 -- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb880039e0 msgr2=0x7fcb88003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcb99400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb880039e0 0x7fcb88003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcb99400640 1 -- 172.21.15.5:0/1277351933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb88003520 con 0x7fcb88004800 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.178+0000 7fcba2800640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40cda10 0x7fcb880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:51.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.179+0000 7fcb99400640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb88004800 0x7fcb88004c80 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c0e0 tx=0x7fcb88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc359ffdf8629b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.179+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c007c40 con 0x7fcb88004800 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.179+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c005af0 con 0x7fcb88004800 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.179+0000 7fcba1400640 1 -- 172.21.15.5:0/1277351933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c007070 con 0x7fcb88004800 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba3200640 1 -- 172.21.15.5:0/1277351933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb88004800 msgr2=0x7fcb88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba3200640 1 --2- 172.21.15.5:0/1277351933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb88004800 0x7fcb88004c80 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c0e0 tx=0x7fcb88005830 comp rx=0 tx=0).stop 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba2800640 1 -- 172.21.15.5:0/1277351933 reap_dead start 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba3200640 1 -- 172.21.15.5:0/1277351933 shutdown_connections 2024-04-25T10:10:51.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba3200640 1 -- 172.21.15.5:0/1277351933 >> 172.21.15.5:0/1277351933 conn(0x7fcba4066f20 msgr2=0x7fcba406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:51.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba3200640 1 -- 172.21.15.5:0/1277351933 shutdown_connections 2024-04-25T10:10:51.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.180+0000 7fcba3200640 1 -- 172.21.15.5:0/1277351933 wait complete. 2024-04-25T10:10:51.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.182+0000 7fcba3200640 1 Processor -- start 2024-04-25T10:10:51.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.182+0000 7fcba3200640 1 -- start start 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcba3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb88004800 0x7fcba4139b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcba3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 0x7fcba40b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcba3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40b3a40 0x7fcba40cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcba3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415d280 con 0x7fcba41f4810 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcba3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba406e810 con 0x7fcb88004800 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcba3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba415d400 con 0x7fcba40b3a40 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcb99400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 0x7fcba40b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcb99400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 0x7fcba40b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53494/0 (socket says 172.21.15.5:53494) 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.183+0000 7fcb99400640 1 -- 172.21.15.5:0/1616194340 learned_addr learned my addr 172.21.15.5:0/1616194340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcba1e00640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb88004800 0x7fcba4139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcba2800640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40b3a40 0x7fcba40cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb99400640 1 -- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40b3a40 msgr2=0x7fcba40cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb99400640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40b3a40 0x7fcba40cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb99400640 1 -- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb88004800 msgr2=0x7fcba4139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb99400640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb88004800 0x7fcba4139b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb99400640 1 -- 172.21.15.5:0/1616194340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba413be50 con 0x7fcba41f4810 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb99400640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 0x7fcba40b3500 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c0e0 tx=0x7fcba4071f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c043070 con 0x7fcba41f4810 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c038040 con 0x7fcba41f4810 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.184+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c007070 con 0x7fcba41f4810 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.185+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba40cc4f0 con 0x7fcba41f4810 2024-04-25T10:10:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.185+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba413c3a0 con 0x7fcba41f4810 2024-04-25T10:10:51.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.186+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba4068f60 con 0x7fcba41f4810 2024-04-25T10:10:51.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.187+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb8c007210 con 0x7fcba41f4810 2024-04-25T10:10:51.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.188+0000 7fcb9aa00640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb8006b500 0x7fcb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:51.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.188+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fcb8c0a9d80 con 0x7fcba41f4810 2024-04-25T10:10:51.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.189+0000 7fcba1e00640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb8006b500 0x7fcb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:51.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.190+0000 7fcba1e00640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb8006b500 0x7fcb8006d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcb94007fd0 tx=0x7fcb94009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:51.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.192+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb8c078cb0 con 0x7fcba41f4810 2024-04-25T10:10:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:51.473+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fcba406e480 con 0x7fcba41f4810 2024-04-25T10:10:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.014+0000 7fcb9aa00640 1 -- 172.21.15.5:0/1616194340 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v444) v1 ==== 149+0+0 (secure 0 0 0) 0x7fcb8c0785b0 con 0x7fcba41f4810 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb8006b500 msgr2=0x7fcb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb8006b500 0x7fcb8006d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcb94007fd0 tx=0x7fcb94009040 comp rx=0 tx=0).stop 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 msgr2=0x7fcba40b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 0x7fcba40b3500 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c00c0e0 tx=0x7fcba4071f00 comp rx=0 tx=0).stop 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 shutdown_connections 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcba40b3a40 0x7fcba40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba41f4810 0x7fcba40b3500 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcb8006b500 0x7fcb8006d9c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 --2- 172.21.15.5:0/1616194340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb88004800 0x7fcba4139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 >> 172.21.15.5:0/1616194340 conn(0x7fcba4066f20 msgr2=0x7fcba4143450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.017+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 shutdown_connections 2024-04-25T10:10:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.018+0000 7fcba3200640 1 -- 172.21.15.5:0/1616194340 wait complete. 2024-04-25T10:10:52.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:10:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.256+0000 7fa564a00640 1 Processor -- start 2024-04-25T10:10:52.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.257+0000 7fa564a00640 1 -- start start 2024-04-25T10:10:52.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.258+0000 7fa564a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa56015ec80 0x7fa5600b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.258+0000 7fa564a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5601616f0 0x7fa560161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.258+0000 7fa564a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa560162030 con 0x7fa5600d4000 2024-04-25T10:10:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.258+0000 7fa564a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56015ffa0 con 0x7fa56015ec80 2024-04-25T10:10:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.258+0000 7fa564a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa560160240 con 0x7fa5601616f0 2024-04-25T10:10:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.259+0000 7fa55e800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa56015ec80 0x7fa5600b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.259+0000 7fa55e800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa56015ec80 0x7fa5600b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:34872/0 (socket says 172.21.15.5:34872) 2024-04-25T10:10:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.259+0000 7fa55e800640 1 -- 172.21.15.5:0/880282123 learned_addr learned my addr 172.21.15.5:0/880282123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.259+0000 7fa55f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5601616f0 0x7fa560161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.260+0000 7fa55e800640 1 -- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5601616f0 msgr2=0x7fa560161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.260+0000 7fa55e800640 1 --2- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5601616f0 0x7fa560161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.260+0000 7fa55e800640 1 -- 172.21.15.5:0/880282123 >> v1:172.21.15.80:6789/0 conn(0x7fa5600d4000 legacy=0x7fa560160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.260+0000 7fa55e800640 1 -- 172.21.15.5:0/880282123 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560160520 con 0x7fa56015ec80 2024-04-25T10:10:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.262+0000 7fa55e800640 1 --2- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600b45c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa550009850 tx=0x7fa55002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2c06385fec7c46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.262+0000 7fa55d400640 1 -- 172.21.15.5:0/880282123 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550004030 con 0x7fa56015ec80 2024-04-25T10:10:52.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.262+0000 7fa55d400640 1 -- 172.21.15.5:0/880282123 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5500041d0 con 0x7fa56015ec80 2024-04-25T10:10:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.262+0000 7fa55d400640 1 -- 172.21.15.5:0/880282123 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550003610 con 0x7fa56015ec80 2024-04-25T10:10:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.263+0000 7fa564a00640 1 -- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 msgr2=0x7fa5600b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.263+0000 7fa564a00640 1 --2- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600b45c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa550009850 tx=0x7fa55002a230 comp rx=0 tx=0).stop 2024-04-25T10:10:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.264+0000 7fa564a00640 1 -- 172.21.15.5:0/880282123 shutdown_connections 2024-04-25T10:10:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.264+0000 7fa564a00640 1 --2- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5601616f0 0x7fa560161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.264+0000 7fa564a00640 1 --2- 172.21.15.5:0/880282123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600b45c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.264+0000 7fa564a00640 1 -- 172.21.15.5:0/880282123 >> 172.21.15.5:0/880282123 conn(0x7fa56008af80 msgr2=0x7fa5600d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.264+0000 7fa564a00640 1 -- 172.21.15.5:0/880282123 shutdown_connections 2024-04-25T10:10:52.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.264+0000 7fa564a00640 1 -- 172.21.15.5:0/880282123 wait complete. 2024-04-25T10:10:52.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.266+0000 7fa564a00640 1 Processor -- start 2024-04-25T10:10:52.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.266+0000 7fa564a00640 1 -- start start 2024-04-25T10:10:52.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.267+0000 7fa564a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5600d4000 0x7fa5600c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa564a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa564a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 0x7fa560158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa564a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56015f060 con 0x7fa5601616f0 2024-04-25T10:10:52.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa564a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa560161410 con 0x7fa56015ec80 2024-04-25T10:10:52.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa564a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600b4c50 con 0x7fa5600d4000 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa55f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 0x7fa560158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa55f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 0x7fa560158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53500/0 (socket says 172.21.15.5:53500) 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa55f200640 1 -- 172.21.15.5:0/1607380513 learned_addr learned my addr 172.21.15.5:0/1607380513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa55e800640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5600d4000 0x7fa5600c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.268+0000 7fa55de00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa55f200640 1 -- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5600d4000 msgr2=0x7fa5600c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa55f200640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5600d4000 0x7fa5600c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa55f200640 1 -- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 msgr2=0x7fa5600cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa55f200640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa55f200640 1 -- 172.21.15.5:0/1607380513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560157c70 con 0x7fa5601616f0 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa55f200640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 0x7fa560158d10 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fa55800efc0 tx=0x7fa55800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558018020 con 0x7fa5601616f0 2024-04-25T10:10:52.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa558014480 con 0x7fa5601616f0 2024-04-25T10:10:52.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa560157f50 con 0x7fa5601616f0 2024-04-25T10:10:52.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558014870 con 0x7fa5601616f0 2024-04-25T10:10:52.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.269+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa560158510 con 0x7fa5601616f0 2024-04-25T10:10:52.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.271+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa560068f60 con 0x7fa5601616f0 2024-04-25T10:10:52.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.272+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa558005d10 con 0x7fa5601616f0 2024-04-25T10:10:52.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.273+0000 7fa556a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa54006b4d0 0x7fa54006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:52.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.273+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa55808ad40 con 0x7fa5601616f0 2024-04-25T10:10:52.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.274+0000 7fa55e800640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa54006b4d0 0x7fa54006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:52.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.275+0000 7fa55e800640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa54006b4d0 0x7fa54006d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa550033000 tx=0x7fa55002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:52.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.277+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa558022070 con 0x7fa5601616f0 2024-04-25T10:10:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.478+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa5600d0e50 con 0x7fa5601616f0 2024-04-25T10:10:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.483+0000 7fa556a00640 1 -- 172.21.15.5:0/1607380513 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa558010040 con 0x7fa5601616f0 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.486+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa54006b4d0 msgr2=0x7fa54006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa54006b4d0 0x7fa54006d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa550033000 tx=0x7fa55002e000 comp rx=0 tx=0).stop 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 msgr2=0x7fa560158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 0x7fa560158d10 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fa55800efc0 tx=0x7fa55800c490 comp rx=0 tx=0).stop 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 shutdown_connections 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5601616f0 0x7fa560158d10 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa54006b4d0 0x7fa54006d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa56015ec80 0x7fa5600cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 --2- 172.21.15.5:0/1607380513 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5600d4000 0x7fa5600c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 >> 172.21.15.5:0/1607380513 conn(0x7fa56008af80 msgr2=0x7fa5600ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 shutdown_connections 2024-04-25T10:10:52.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:52.487+0000 7fa564a00640 1 -- 172.21.15.5:0/1607380513 wait complete. 2024-04-25T10:10:52.511 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:10:52.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:52.609 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:10:52.717 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:10:52.718 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:10:52.718 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:52.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:52.780 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:10:52.780 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:52.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:52.839 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:10:52.840 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:52.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:10:52.903 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:10:52.903 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:52.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:10:52.962 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:10:52.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:53.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.177+0000 7fa410c00640 1 Processor -- start 2024-04-25T10:10:53.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.178+0000 7fa410c00640 1 -- start start 2024-04-25T10:10:53.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.179+0000 7fa410c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa40c0cea60 0x7fa40c166420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.179+0000 7fa410c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c15bdc0 con 0x7fa40c071ed0 2024-04-25T10:10:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.179+0000 7fa410c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c15c060 con 0x7fa40c1529e0 2024-04-25T10:10:53.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.179+0000 7fa410c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0b2340 con 0x7fa40c0cea60 2024-04-25T10:10:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.180+0000 7fa409400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa40c071ed0 0x7fa40c0ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60036/0 (socket says 172.21.15.5:60036) 2024-04-25T10:10:53.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.180+0000 7fa409400640 1 -- 172.21.15.5:0/1492530325 learned_addr learned my addr 172.21.15.5:0/1492530325 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.180+0000 7fa40a800640 1 --2- 172.21.15.5:0/1492530325 >> v2:172.21.15.80:3300/0 conn(0x7fa40c0cea60 0x7fa40c166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.180+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274070390 0 0) 0x7fa40c15bdc0 con 0x7fa40c071ed0 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f0003520 con 0x7fa40c071ed0 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3120390668 0 0) 0x7fa3f0003520 con 0x7fa40c071ed0 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 >> v2:172.21.15.80:3300/0 conn(0x7fa40c0cea60 msgr2=0x7fa40c166420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 --2- 172.21.15.5:0/1492530325 >> v2:172.21.15.80:3300/0 conn(0x7fa40c0cea60 0x7fa40c166420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 >> v1:172.21.15.80:6789/0 conn(0x7fa40c1529e0 legacy=0x7fa40c0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c15c340 con 0x7fa40c071ed0 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fa3f8001670 con 0x7fa40c071ed0 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 >> v1:172.21.15.5:6789/0 conn(0x7fa40c071ed0 legacy=0x7fa40c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3f0003a00 0x7fa3f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3f0004800 0x7fa3f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa3f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0b2340 con 0x7fa3f00051c0 2024-04-25T10:10:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f000c0c0 con 0x7fa3f0003a00 2024-04-25T10:10:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.181+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f000c2c0 con 0x7fa3f0004800 2024-04-25T10:10:53.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa40a800640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa3f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa409e00640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3f0003a00 0x7fa3f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa40a800640 1 -- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3f0004800 msgr2=0x7fa3f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa40a800640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa3f0004800 0x7fa3f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa40a800640 1 -- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3f0003a00 msgr2=0x7fa3f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa40a800640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa3f0003a00 0x7fa3f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.182+0000 7fa40a800640 1 -- 172.21.15.5:0/1492530325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0003520 con 0x7fa3f00051c0 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.183+0000 7fa40a800640 1 -- 172.21.15.5:0/1492530325 reap_dead start 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.183+0000 7fa40a800640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa3f000b9c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa404009fc0 tx=0x7fa3f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0b88d06c5afc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.183+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404035020 con 0x7fa3f00051c0 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404003bd0 con 0x7fa3f00051c0 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa408a00640 1 -- 172.21.15.5:0/1492530325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404009c10 con 0x7fa3f00051c0 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 -- 172.21.15.5:0/1492530325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 msgr2=0x7fa3f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa3f000b9c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa404009fc0 tx=0x7fa3f0005830 comp rx=0 tx=0).stop 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 -- 172.21.15.5:0/1492530325 shutdown_connections 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 --2- 172.21.15.5:0/1492530325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa3f000b9c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 -- 172.21.15.5:0/1492530325 >> 172.21.15.5:0/1492530325 conn(0x7fa40c071150 msgr2=0x7fa40c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 -- 172.21.15.5:0/1492530325 shutdown_connections 2024-04-25T10:10:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.184+0000 7fa410c00640 1 -- 172.21.15.5:0/1492530325 wait complete. 2024-04-25T10:10:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.186+0000 7fa410c00640 1 Processor -- start 2024-04-25T10:10:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.186+0000 7fa410c00640 1 -- start start 2024-04-25T10:10:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa410c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa40c144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa410c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa40c141a50 0x7fa40c152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa410c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa40c153310 0x7fa40c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa410c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0ce350 con 0x7fa3f00051c0 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa410c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c15c900 con 0x7fa40c153310 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa410c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0ce4d0 con 0x7fa40c141a50 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.187+0000 7fa409e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa40c144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa40c144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53510/0 (socket says 172.21.15.5:53510) 2024-04-25T10:10:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 -- 172.21.15.5:0/2015633771 learned_addr learned my addr 172.21.15.5:0/2015633771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 -- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa40c141a50 msgr2=0x7fa40c152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa40a800640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa40c153310 0x7fa40c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa40c141a50 0x7fa40c152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 -- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa40c153310 msgr2=0x7fa40c0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa40c153310 0x7fa40c0c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 -- 172.21.15.5:0/2015633771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c153850 con 0x7fa3f00051c0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.188+0000 7fa409e00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa40c144e30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e4b0 tx=0x7fa3fc00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.189+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0052e0 con 0x7fa3f00051c0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.189+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc018070 con 0x7fa3f00051c0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.189+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c144350 con 0x7fa3f00051c0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.189+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0037f0 con 0x7fa3f00051c0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.189+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c144910 con 0x7fa3f00051c0 2024-04-25T10:10:53.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.190+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c068fd0 con 0x7fa3f00051c0 2024-04-25T10:10:53.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.191+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3fc003990 con 0x7fa3f00051c0 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.192+0000 7fa402000640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3ec06b4d0 0x7fa3ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.192+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa3fc089db0 con 0x7fa3f00051c0 2024-04-25T10:10:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.193+0000 7fa409400640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3ec06b4d0 0x7fa3ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.194+0000 7fa409400640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3ec06b4d0 0x7fa3ec06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa40c151af0 tx=0x7fa3f8001500 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.196+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3fc08e070 con 0x7fa3f00051c0 2024-04-25T10:10:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.434+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa40c073910 con 0x7fa3f00051c0 2024-04-25T10:10:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.435+0000 7fa402000640 1 -- 172.21.15.5:0/2015633771 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3687 (secure 0 0 0) 0x7fa3fc058c50 con 0x7fa3f00051c0 2024-04-25T10:10:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:53.441 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5615,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/2639922320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6835","nonce":2639922320},{"type":"v1","addr":"172.21.15.80:6837","nonce":2639922320}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6836/259356775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":259356775},{"type":"v1","addr":"172.21.15.80:6836","nonce":259356775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/884141512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":884141512},{"type":"v1","addr":"172.21.15.5:6835","nonce":884141512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/4126295168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4126295168},{"type":"v1","addr":"172.21.15.5:6834","nonce":4126295168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[]} 2024-04-25T10:10:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 77 2024-04-25T10:10:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3ec06b4d0 msgr2=0x7fa3ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3ec06b4d0 0x7fa3ec06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa40c151af0 tx=0x7fa3f8001500 comp rx=0 tx=0).stop 2024-04-25T10:10:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 msgr2=0x7fa40c144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa40c144e30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e4b0 tx=0x7fa3fc00e970 comp rx=0 tx=0).stop 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 shutdown_connections 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa3ec06b4d0 0x7fa3ec06d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa40c153310 0x7fa40c0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa40c141a50 0x7fa40c152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 --2- 172.21.15.5:0/2015633771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3f00051c0 0x7fa40c144e30 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 >> 172.21.15.5:0/2015633771 conn(0x7fa40c071150 msgr2=0x7fa40c15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 shutdown_connections 2024-04-25T10:10:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.438+0000 7fa410c00640 1 -- 172.21.15.5:0/2015633771 wait complete. 2024-04-25T10:10:53.464 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:10:53.464 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:10:53.465 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:10:53.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:53.465 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:10:53.465 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:10:53.465 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:10:53.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:53.465 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:10:53.466 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:10:53.466 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:10:53.466 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:53.466 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:10:53.466 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:10:53.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:10:53.466 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:53.493 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:10:53.493 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:10:53.495 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:10:53.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:10:53.496 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:10:53.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:10:53.498 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:10:53.498 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:10:53.544 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:10:53.545 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:53.545 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:10:53.545 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:53.548 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:10:53.548 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:53.550 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:10:53.551 DEBUG:teuthology.parallel:result is None 2024-04-25T10:10:53.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:10:53.694 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:10:53.697 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:10:53.697 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:10:53.697 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:10:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.736+0000 7ffa50000640 1 Processor -- start 2024-04-25T10:10:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.737+0000 7ffa50000640 1 -- start start 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa50000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa481472c0 0x7ffa480cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa50000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa480cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa50000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa480c9f90 con 0x7ffa480cad50 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa50000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480ca270 con 0x7ffa481472c0 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa50000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ca550 con 0x7ffa480ced80 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa4cc00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffa480cad50 0x7ffa480ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60046/0 (socket says 172.21.15.5:60046) 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa4cc00640 1 -- 172.21.15.5:0/1130246100 learned_addr learned my addr 172.21.15.5:0/1130246100 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa4d600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa481472c0 0x7ffa480cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.738+0000 7ffa4e000640 1 --2- 172.21.15.5:0/1130246100 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa480cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.739+0000 7ffa4d600640 1 -- 172.21.15.5:0/1130246100 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 msgr2=0x7ffa480cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.739+0000 7ffa4d600640 1 --2- 172.21.15.5:0/1130246100 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa480cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.739+0000 7ffa4d600640 1 -- 172.21.15.5:0/1130246100 >> v1:172.21.15.5:6789/0 conn(0x7ffa480cad50 legacy=0x7ffa480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.739+0000 7ffa4d600640 1 -- 172.21.15.5:0/1130246100 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480ca830 con 0x7ffa481472c0 2024-04-25T10:10:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.740+0000 7ffa4d600640 1 --2- 172.21.15.5:0/1130246100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa480cb440 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009d90 tx=0x7ffa3c02a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8146e364a4dc78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.740+0000 7ffa47e00640 1 -- 172.21.15.5:0/1130246100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c007740 con 0x7ffa481472c0 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.740+0000 7ffa47e00640 1 -- 172.21.15.5:0/1130246100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0078e0 con 0x7ffa481472c0 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.740+0000 7ffa47e00640 1 -- 172.21.15.5:0/1130246100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c007cb0 con 0x7ffa481472c0 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 -- 172.21.15.5:0/1130246100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 msgr2=0x7ffa480cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 --2- 172.21.15.5:0/1130246100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa480cb440 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009d90 tx=0x7ffa3c02a540 comp rx=0 tx=0).stop 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 -- 172.21.15.5:0/1130246100 shutdown_connections 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 --2- 172.21.15.5:0/1130246100 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa480cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 --2- 172.21.15.5:0/1130246100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa480cb440 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 -- 172.21.15.5:0/1130246100 >> 172.21.15.5:0/1130246100 conn(0x7ffa48071150 msgr2=0x7ffa48071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 -- 172.21.15.5:0/1130246100 shutdown_connections 2024-04-25T10:10:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.741+0000 7ffa50000640 1 -- 172.21.15.5:0/1130246100 wait complete. 2024-04-25T10:10:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.743+0000 7ffa50000640 1 Processor -- start 2024-04-25T10:10:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.743+0000 7ffa50000640 1 -- start start 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa50000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa480cad50 0x7ffa480b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa50000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa48142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa50000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa4813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa50000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4806d2d0 con 0x7ffa481472c0 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa50000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4806d150 con 0x7ffa480cad50 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa50000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4806d450 con 0x7ffa480ced80 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa4e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa4813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa4e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa4813ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53562/0 (socket says 172.21.15.5:53562) 2024-04-25T10:10:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa4e000640 1 -- 172.21.15.5:0/3805669878 learned_addr learned my addr 172.21.15.5:0/3805669878 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa4cc00640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa48142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.744+0000 7ffa4d600640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa480cad50 0x7ffa480b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4e000640 1 -- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 msgr2=0x7ffa48142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4e000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa48142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4e000640 1 -- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa480cad50 msgr2=0x7ffa480b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4e000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa480cad50 0x7ffa480b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4e000640 1 -- 172.21.15.5:0/3805669878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480b4070 con 0x7ffa481472c0 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4d600640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa480cad50 0x7ffa480b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa4e000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa4813ce20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e960 tx=0x7ffa4000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40017070 con 0x7ffa481472c0 2024-04-25T10:10:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4000c980 con 0x7ffa481472c0 2024-04-25T10:10:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000cd70 con 0x7ffa481472c0 2024-04-25T10:10:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.745+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa480b4350 con 0x7ffa481472c0 2024-04-25T10:10:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.746+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa480b4bf0 con 0x7ffa481472c0 2024-04-25T10:10:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.747+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa480b4df0 con 0x7ffa481472c0 2024-04-25T10:10:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.748+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa4001a070 con 0x7ffa481472c0 2024-04-25T10:10:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.749+0000 7ffa45600640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa2806b500 0x7ffa2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.749+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ffa4008a530 con 0x7ffa481472c0 2024-04-25T10:10:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.750+0000 7ffa4d600640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa2806b500 0x7ffa2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.751+0000 7ffa4d600640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa2806b500 0x7ffa2806d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c034000 tx=0x7ffa3c033040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.754+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa480b4df0 con 0x7ffa481472c0 2024-04-25T10:10:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:53.973+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffa48068fd0 con 0x7ffa481472c0 2024-04-25T10:10:55.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.056+0000 7ffa45600640 1 -- 172.21.15.5:0/3805669878 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffa40058cf0 con 0x7ffa481472c0 2024-04-25T10:10:55.061 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:10:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa2806b500 msgr2=0x7ffa2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa2806b500 0x7ffa2806d9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c034000 tx=0x7ffa3c033040 comp rx=0 tx=0).stop 2024-04-25T10:10:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 msgr2=0x7ffa4813ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa4813ce20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000e960 tx=0x7ffa4000ee20 comp rx=0 tx=0).stop 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 shutdown_connections 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa481472c0 0x7ffa4813ce20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa480ced80 0x7ffa48142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa2806b500 0x7ffa2806d9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 --2- 172.21.15.5:0/3805669878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa480cad50 0x7ffa480b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 >> 172.21.15.5:0/3805669878 conn(0x7ffa48071150 msgr2=0x7ffa480cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.060+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 shutdown_connections 2024-04-25T10:10:55.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.062+0000 7ffa50000640 1 -- 172.21.15.5:0/3805669878 wait complete. 2024-04-25T10:10:55.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:10:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.297+0000 7fc656000640 1 Processor -- start 2024-04-25T10:10:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.298+0000 7fc656000640 1 -- start start 2024-04-25T10:10:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.299+0000 7fc656000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc650141da0 0x7fc65015c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.299+0000 7fc656000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc65006d930 0x7fc6500cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.299+0000 7fc656000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6500cc290 0x7fc650070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.299+0000 7fc656000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500cc6f0 con 0x7fc6500cc290 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.299+0000 7fc656000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc650142180 con 0x7fc650141da0 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.299+0000 7fc656000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65006d2b0 con 0x7fc65006d930 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6500cc290 0x7fc650070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc6500cc290 0x7fc650070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53578/0 (socket says 172.21.15.5:53578) 2024-04-25T10:10:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 -- 172.21.15.5:0/2700759554 learned_addr learned my addr 172.21.15.5:0/2700759554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc654c00640 1 --2- 172.21.15.5:0/2700759554 >> v2:172.21.15.80:3300/0 conn(0x7fc650141da0 0x7fc65015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 -- 172.21.15.5:0/2700759554 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc65006d930 msgr2=0x7fc6500cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 --2- 172.21.15.5:0/2700759554 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc65006d930 0x7fc6500cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 -- 172.21.15.5:0/2700759554 >> v2:172.21.15.80:3300/0 conn(0x7fc650141da0 msgr2=0x7fc65015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 --2- 172.21.15.5:0/2700759554 >> v2:172.21.15.80:3300/0 conn(0x7fc650141da0 0x7fc65015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.300+0000 7fc655600640 1 -- 172.21.15.5:0/2700759554 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65015f310 con 0x7fc6500cc290 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.301+0000 7fc655600640 1 --2- 172.21.15.5:0/2700759554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc650070f90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc65006f4a0 tx=0x7fc64802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c409af079faf83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.302+0000 7fc64f400640 1 -- 172.21.15.5:0/2700759554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648036070 con 0x7fc6500cc290 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.302+0000 7fc64f400640 1 -- 172.21.15.5:0/2700759554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc648031600 con 0x7fc6500cc290 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.302+0000 7fc64f400640 1 -- 172.21.15.5:0/2700759554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6480319d0 con 0x7fc6500cc290 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.302+0000 7fc656000640 1 -- 172.21.15.5:0/2700759554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 msgr2=0x7fc650070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.302+0000 7fc656000640 1 --2- 172.21.15.5:0/2700759554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc650070f90 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fc65006f4a0 tx=0x7fc64802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 -- 172.21.15.5:0/2700759554 shutdown_connections 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 --2- 172.21.15.5:0/2700759554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc650070f90 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 --2- 172.21.15.5:0/2700759554 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc65006d930 0x7fc6500cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 --2- 172.21.15.5:0/2700759554 >> v2:172.21.15.80:3300/0 conn(0x7fc650141da0 0x7fc65015c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 -- 172.21.15.5:0/2700759554 >> 172.21.15.5:0/2700759554 conn(0x7fc6500b4f00 msgr2=0x7fc65006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 -- 172.21.15.5:0/2700759554 shutdown_connections 2024-04-25T10:10:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.303+0000 7fc656000640 1 -- 172.21.15.5:0/2700759554 wait complete. 2024-04-25T10:10:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.304+0000 7fc656000640 1 Processor -- start 2024-04-25T10:10:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.305+0000 7fc656000640 1 -- start start 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.305+0000 7fc656000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65006d930 0x7fc650159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.305+0000 7fc656000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc6500ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc656000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc650141da0 0x7fc650158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc656000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015fbb0 con 0x7fc6500cc290 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc656000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015f8d0 con 0x7fc65006d930 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc656000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015fe90 con 0x7fc650141da0 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc6500ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc6500ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53584/0 (socket says 172.21.15.5:53584) 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 -- 172.21.15.5:0/4185580830 learned_addr learned my addr 172.21.15.5:0/4185580830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc654c00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65006d930 0x7fc650159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 -- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc650141da0 msgr2=0x7fc650158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc655600640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc650141da0 0x7fc650158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc650141da0 0x7fc650158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 -- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65006d930 msgr2=0x7fc650159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65006d930 0x7fc650159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc64fe00640 1 -- 172.21.15.5:0/4185580830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500adac0 con 0x7fc6500cc290 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.306+0000 7fc654c00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65006d930 0x7fc650159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.307+0000 7fc64fe00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc6500ae860 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00b410 tx=0x7fc63c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.307+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c015020 con 0x7fc6500cc290 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.307+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c011850 con 0x7fc6500cc290 2024-04-25T10:10:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.307+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c011c40 con 0x7fc6500cc290 2024-04-25T10:10:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.307+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6500adda0 con 0x7fc6500cc290 2024-04-25T10:10:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.307+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6500aeda0 con 0x7fc6500cc290 2024-04-25T10:10:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.308+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc650067c40 con 0x7fc6500cc290 2024-04-25T10:10:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.309+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc63c018020 con 0x7fc6500cc290 2024-04-25T10:10:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.310+0000 7fc64cc00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc63006b4d0 0x7fc63006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.310+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc63c08a080 con 0x7fc6500cc290 2024-04-25T10:10:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.311+0000 7fc654c00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc63006b4d0 0x7fc63006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.312+0000 7fc654c00640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc63006b4d0 0x7fc63006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc644008890 tx=0x7fc644008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.314+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc650067c40 con 0x7fc6500cc290 2024-04-25T10:10:55.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.599+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc650068f60 con 0x7fc6500cc290 2024-04-25T10:10:55.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.600+0000 7fc64cc00640 1 -- 172.21.15.5:0/4185580830 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc63c01f020 con 0x7fc6500cc290 2024-04-25T10:10:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:10:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:10:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:10:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:10:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:10:55.620 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc63006b4d0 msgr2=0x7fc63006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc63006b4d0 0x7fc63006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc644008890 tx=0x7fc644008040 comp rx=0 tx=0).stop 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 msgr2=0x7fc6500ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc6500ae860 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00b410 tx=0x7fc63c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 shutdown_connections 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc650141da0 0x7fc650158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6500cc290 0x7fc6500ae860 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc63006b4d0 0x7fc63006d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 --2- 172.21.15.5:0/4185580830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc65006d930 0x7fc650159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 >> 172.21.15.5:0/4185580830 conn(0x7fc6500b4f00 msgr2=0x7fc65015c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 shutdown_connections 2024-04-25T10:10:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.603+0000 7fc656000640 1 -- 172.21.15.5:0/4185580830 wait complete. 2024-04-25T10:10:55.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:55.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:55.684 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.828+0000 7f2afcc00640 1 Processor -- start 2024-04-25T10:10:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.828+0000 7f2afcc00640 1 -- start start 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2afcc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2af815ce00 0x7f2af815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2afcc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2af80cd300 0x7f2af80cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2afcc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af8071d30 con 0x7f2af8141da0 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2afcc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80cdcd0 con 0x7f2af80cd300 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2afcc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af8142180 con 0x7f2af815ce00 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2af7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2af80cd300 0x7f2af80cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2af7400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2af8141da0 0x7f2af806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60058/0 (socket says 172.21.15.5:60058) 2024-04-25T10:10:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.830+0000 7f2af7400640 1 -- 172.21.15.5:0/406088745 learned_addr learned my addr 172.21.15.5:0/406088745 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:55.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.831+0000 7f2af6a00640 1 --2- 172.21.15.5:0/406088745 >> v2:172.21.15.80:3300/0 conn(0x7f2af815ce00 0x7f2af815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.831+0000 7f2af7e00640 1 -- 172.21.15.5:0/406088745 >> v2:172.21.15.80:3300/0 conn(0x7f2af815ce00 msgr2=0x7f2af815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:55.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.831+0000 7f2af7e00640 1 --2- 172.21.15.5:0/406088745 >> v2:172.21.15.80:3300/0 conn(0x7f2af815ce00 0x7f2af815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.831+0000 7f2af7e00640 1 -- 172.21.15.5:0/406088745 >> v1:172.21.15.5:6789/0 conn(0x7f2af8141da0 legacy=0x7f2af806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.831+0000 7f2af6000640 1 -- 172.21.15.5:0/406088745 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102445266 0 0) 0x7f2af8071d30 con 0x7f2af8141da0 2024-04-25T10:10:55.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.831+0000 7f2af7e00640 1 -- 172.21.15.5:0/406088745 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af806d620 con 0x7f2af80cd300 2024-04-25T10:10:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.832+0000 7f2af7e00640 1 --2- 172.21.15.5:0/406088745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af80cd300 0x7f2af80cd790 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2af806c800 tx=0x7f2af002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8341473b58541a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.832+0000 7f2af6000640 1 -- 172.21.15.5:0/406088745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0007740 con 0x7f2af80cd300 2024-04-25T10:10:55.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.832+0000 7f2af6000640 1 -- 172.21.15.5:0/406088745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af00078e0 con 0x7f2af80cd300 2024-04-25T10:10:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.832+0000 7f2af6000640 1 -- 172.21.15.5:0/406088745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0007cb0 con 0x7f2af80cd300 2024-04-25T10:10:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 -- 172.21.15.5:0/406088745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af80cd300 msgr2=0x7f2af80cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 --2- 172.21.15.5:0/406088745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af80cd300 0x7f2af80cd790 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2af806c800 tx=0x7f2af002a680 comp rx=0 tx=0).stop 2024-04-25T10:10:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 -- 172.21.15.5:0/406088745 shutdown_connections 2024-04-25T10:10:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 --2- 172.21.15.5:0/406088745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af80cd300 0x7f2af80cd790 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 --2- 172.21.15.5:0/406088745 >> v2:172.21.15.80:3300/0 conn(0x7f2af815ce00 0x7f2af815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 -- 172.21.15.5:0/406088745 >> 172.21.15.5:0/406088745 conn(0x7f2af8066f20 msgr2=0x7f2af806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 -- 172.21.15.5:0/406088745 shutdown_connections 2024-04-25T10:10:55.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.833+0000 7f2afcc00640 1 -- 172.21.15.5:0/406088745 wait complete. 2024-04-25T10:10:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.835+0000 7f2afcc00640 1 Processor -- start 2024-04-25T10:10:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.835+0000 7f2afcc00640 1 -- start start 2024-04-25T10:10:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2afcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2af80cd300 0x7f2af80cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2afcc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2af8141da0 0x7f2af8155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2afcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 0x7f2af8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2afcc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af806e810 con 0x7f2af815ce00 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2afcc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af806dbe0 con 0x7f2af8141da0 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2afcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8071000 con 0x7f2af80cd300 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2af7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 0x7f2af8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2af7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 0x7f2af8155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53602/0 (socket says 172.21.15.5:53602) 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.836+0000 7f2af7e00640 1 -- 172.21.15.5:0/3064517536 learned_addr learned my addr 172.21.15.5:0/3064517536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7400640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2af80cd300 0x7f2af80cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af6a00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2af8141da0 0x7f2af8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7e00640 1 -- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2af80cd300 msgr2=0x7f2af80cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7e00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2af80cd300 0x7f2af80cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7e00640 1 -- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2af8141da0 msgr2=0x7f2af8155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7e00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2af8141da0 0x7f2af8155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7e00640 1 -- 172.21.15.5:0/3064517536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af8158db0 con 0x7f2af815ce00 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af6a00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2af8141da0 0x7f2af8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2af7e00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 0x7f2af8155e60 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009e80 tx=0x7f2af0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.837+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0040070 con 0x7f2af815ce00 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.838+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af0031c40 con 0x7f2af815ce00 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.838+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0009450 con 0x7f2af815ce00 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.838+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af80c88d0 con 0x7f2af815ce00 2024-04-25T10:10:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.838+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80c8e90 con 0x7f2af815ce00 2024-04-25T10:10:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.839+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af8068f60 con 0x7f2af815ce00 2024-04-25T10:10:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.840+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2af0039040 con 0x7f2af815ce00 2024-04-25T10:10:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.841+0000 7f2aef400640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ad806b4d0 0x7f2ad806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.842+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2af00ab290 con 0x7f2af815ce00 2024-04-25T10:10:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.842+0000 7f2af7400640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ad806b4d0 0x7f2ad806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.844+0000 7f2af7400640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ad806b4d0 0x7f2ad806d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000f60 tx=0x7f2ae8007b70 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:55.846+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2af007a2c0 con 0x7f2af815ce00 2024-04-25T10:10:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.047+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2af80b4480 con 0x7f2af815ce00 2024-04-25T10:10:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.048+0000 7f2aef400640 1 -- 172.21.15.5:0/3064517536 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+13058 (secure 0 0 0) 0x7f2af0079bc0 con 0x7f2af815ce00 2024-04-25T10:10:56.053 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:56.054 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":101,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:10:55.052937+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:56.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ad806b4d0 msgr2=0x7f2ad806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ad806b4d0 0x7f2ad806d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000f60 tx=0x7f2ae8007b70 comp rx=0 tx=0).stop 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 msgr2=0x7f2af8155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 0x7f2af8155e60 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009e80 tx=0x7f2af0004a40 comp rx=0 tx=0).stop 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 shutdown_connections 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2af815ce00 0x7f2af8155e60 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ad806b4d0 0x7f2ad806d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2af8141da0 0x7f2af8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 --2- 172.21.15.5:0/3064517536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2af80cd300 0x7f2af80cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.051+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 >> 172.21.15.5:0/3064517536 conn(0x7f2af8066f20 msgr2=0x7f2af80b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.052+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 shutdown_connections 2024-04-25T10:10:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.052+0000 7f2afcc00640 1 -- 172.21.15.5:0/3064517536 wait complete. 2024-04-25T10:10:56.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:56.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.289+0000 7f3a53200640 1 Processor -- start 2024-04-25T10:10:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.290+0000 7f3a53200640 1 -- start start 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.291+0000 7f3a53200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a5415ec80 0x7f3a540b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.291+0000 7f3a53200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a541616f0 0x7f3a54161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.291+0000 7f3a53200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a54162030 con 0x7f3a540d4000 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.291+0000 7f3a53200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5415ffa0 con 0x7f3a5415ec80 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.291+0000 7f3a53200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54160240 con 0x7f3a541616f0 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a5415ec80 0x7f3a540b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a5415ec80 0x7f3a540b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53614/0 (socket says 172.21.15.5:53614) 2024-04-25T10:10:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 -- 172.21.15.5:0/2136864975 learned_addr learned my addr 172.21.15.5:0/2136864975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a52800640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a541616f0 0x7f3a54161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 -- 172.21.15.5:0/2136864975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a541616f0 msgr2=0x7f3a54161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a541616f0 0x7f3a54161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 -- 172.21.15.5:0/2136864975 >> v1:172.21.15.80:6789/0 conn(0x7f3a540d4000 legacy=0x7f3a54160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a51e00640 1 -- 172.21.15.5:0/2136864975 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54160520 con 0x7f3a5415ec80 2024-04-25T10:10:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.292+0000 7f3a52800640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a541616f0 0x7f3a54161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.293+0000 7f3a51e00640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5415ec80 0x7f3a540b45c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009850 tx=0x7f3a4402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a2a15791932001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a50a00640 1 -- 172.21.15.5:0/2136864975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44004030 con 0x7f3a5415ec80 2024-04-25T10:10:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a50a00640 1 -- 172.21.15.5:0/2136864975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a440041d0 con 0x7f3a5415ec80 2024-04-25T10:10:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a50a00640 1 -- 172.21.15.5:0/2136864975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44003610 con 0x7f3a5415ec80 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 -- 172.21.15.5:0/2136864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5415ec80 msgr2=0x7f3a540b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5415ec80 0x7f3a540b45c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009850 tx=0x7f3a4402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 -- 172.21.15.5:0/2136864975 shutdown_connections 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a541616f0 0x7f3a54161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 --2- 172.21.15.5:0/2136864975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a5415ec80 0x7f3a540b45c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 -- 172.21.15.5:0/2136864975 >> 172.21.15.5:0/2136864975 conn(0x7f3a5408af80 msgr2=0x7f3a540d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.294+0000 7f3a53200640 1 -- 172.21.15.5:0/2136864975 shutdown_connections 2024-04-25T10:10:56.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.295+0000 7f3a53200640 1 -- 172.21.15.5:0/2136864975 wait complete. 2024-04-25T10:10:56.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.296+0000 7f3a53200640 1 Processor -- start 2024-04-25T10:10:56.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.296+0000 7f3a53200640 1 -- start start 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.297+0000 7f3a53200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a540d4000 0x7f3a540b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.297+0000 7f3a53200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a5415ec80 0x7f3a5414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.297+0000 7f3a53200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 0x7f3a5414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.297+0000 7f3a53200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5415f060 con 0x7f3a541616f0 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.297+0000 7f3a53200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54161410 con 0x7f3a540d4000 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.297+0000 7f3a53200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a540b4c50 con 0x7f3a5415ec80 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 0x7f3a5414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 0x7f3a5414c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53628/0 (socket says 172.21.15.5:53628) 2024-04-25T10:10:56.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 -- 172.21.15.5:0/1407333347 learned_addr learned my addr 172.21.15.5:0/1407333347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:56.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 -- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a5415ec80 msgr2=0x7f3a5414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a51400640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a5415ec80 0x7f3a5414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a51e00640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a540d4000 0x7f3a540b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a5415ec80 0x7f3a5414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 -- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a540d4000 msgr2=0x7f3a540b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a540d4000 0x7f3a540b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 -- 172.21.15.5:0/1407333347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a540b7740 con 0x7f3a541616f0 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a51400640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a5415ec80 0x7f3a5414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.298+0000 7f3a52800640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 0x7f3a5414c1f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00c7c0 tx=0x7f3a4c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.299+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c015020 con 0x7f3a541616f0 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.299+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c004230 con 0x7f3a541616f0 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.299+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c007620 con 0x7f3a541616f0 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.299+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a540b7a20 con 0x7f3a541616f0 2024-04-25T10:10:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.299+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a540c7240 con 0x7f3a541616f0 2024-04-25T10:10:56.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.300+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a540c7440 con 0x7f3a541616f0 2024-04-25T10:10:56.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.301+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a4c018020 con 0x7f3a541616f0 2024-04-25T10:10:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.302+0000 7f3a4a000640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a3006b4d0 0x7f3a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.302+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3a4c08a040 con 0x7f3a541616f0 2024-04-25T10:10:56.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.303+0000 7f3a51e00640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a3006b4d0 0x7f3a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.304+0000 7f3a51e00640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a3006b4d0 0x7f3a3006d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3a44033000 tx=0x7f3a4402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.307+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a540c7440 con 0x7f3a541616f0 2024-04-25T10:10:56.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.505+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a54073840 con 0x7f3a541616f0 2024-04-25T10:10:56.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.506+0000 7f3a4a000640 1 -- 172.21.15.5:0/1407333347 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a4c01f020 con 0x7f3a541616f0 2024-04-25T10:10:56.511 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:10:56.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.508+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a3006b4d0 msgr2=0x7f3a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.508+0000 7f3a53200640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a3006b4d0 0x7f3a3006d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3a44033000 tx=0x7f3a4402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.508+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 msgr2=0x7f3a5414c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.508+0000 7f3a53200640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 0x7f3a5414c1f0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00c7c0 tx=0x7f3a4c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 shutdown_connections 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a541616f0 0x7f3a5414c1f0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a5415ec80 0x7f3a5414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a3006b4d0 0x7f3a3006d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 --2- 172.21.15.5:0/1407333347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a540d4000 0x7f3a540b8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 >> 172.21.15.5:0/1407333347 conn(0x7f3a5408af80 msgr2=0x7f3a540d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 shutdown_connections 2024-04-25T10:10:56.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.509+0000 7f3a53200640 1 -- 172.21.15.5:0/1407333347 wait complete. 2024-04-25T10:10:56.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:10:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.749+0000 7f2d73e00640 1 Processor -- start 2024-04-25T10:10:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.750+0000 7f2d73e00640 1 -- start start 2024-04-25T10:10:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 0x7f2d740d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d73e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d7415e4c0 0x7f2d741613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d73e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740ce6f0 con 0x7f2d740d1590 2024-04-25T10:10:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d73e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740cea20 con 0x7f2d7415e4c0 2024-04-25T10:10:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d73e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7415ea10 con 0x7f2d740d0570 2024-04-25T10:10:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d72a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2d740d1590 0x7f2d740d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60070/0 (socket says 172.21.15.5:60070) 2024-04-25T10:10:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.751+0000 7f2d72a00640 1 -- 172.21.15.5:0/3504855651 learned_addr learned my addr 172.21.15.5:0/3504855651 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d6be00640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 0x7f2d740d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738100613 0 0) 0x7f2d740ce6f0 con 0x7f2d740d1590 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003520 con 0x7f2d740d1590 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d73400640 1 --2- 172.21.15.5:0/3504855651 >> v2:172.21.15.80:3300/0 conn(0x7f2d7415e4c0 0x7f2d741613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4241445238 0 0) 0x7f2d58003520 con 0x7f2d740d1590 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 msgr2=0x7f2d740d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 0x7f2d740d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 >> v2:172.21.15.80:3300/0 conn(0x7f2d7415e4c0 msgr2=0x7f2d741613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 --2- 172.21.15.5:0/3504855651 >> v2:172.21.15.80:3300/0 conn(0x7f2d7415e4c0 0x7f2d741613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7415ecf0 con 0x7f2d740d1590 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2d64001670 con 0x7f2d740d1590 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 >> v1:172.21.15.5:6789/0 conn(0x7f2d740d1590 legacy=0x7f2d740d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580039b0 0x7f2d58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 0x7f2d58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d58004d40 0x7f2d580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740cea20 con 0x7f2d58004d40 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7415ea10 con 0x7f2d580039b0 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800ba70 con 0x7f2d740d0570 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.752+0000 7f2d73400640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d58004d40 0x7f2d580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d72a00640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580039b0 0x7f2d58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d6be00640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 0x7f2d58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d73400640 1 -- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 msgr2=0x7f2d58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d73400640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d740d0570 0x7f2d58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d73400640 1 -- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580039b0 msgr2=0x7f2d58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d73400640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d580039b0 0x7f2d58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.753+0000 7f2d73400640 1 -- 172.21.15.5:0/3504855651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58003520 con 0x7f2d58004d40 2024-04-25T10:10:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.754+0000 7f2d73400640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d58004d40 0x7f2d580051c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c007390 tx=0x7f2d58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54588261ac5140b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.754+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c009be0 con 0x7f2d58004d40 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.754+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c02e640 con 0x7f2d58004d40 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.754+0000 7f2d72000640 1 -- 172.21.15.5:0/3504855651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c02ea10 con 0x7f2d58004d40 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73e00640 1 -- 172.21.15.5:0/3504855651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d58004d40 msgr2=0x7f2d580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73e00640 1 --2- 172.21.15.5:0/3504855651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d58004d40 0x7f2d580051c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c007390 tx=0x7f2d58005830 comp rx=0 tx=0).stop 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73400640 1 -- 172.21.15.5:0/3504855651 reap_dead start 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73e00640 1 -- 172.21.15.5:0/3504855651 shutdown_connections 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73e00640 1 -- 172.21.15.5:0/3504855651 >> 172.21.15.5:0/3504855651 conn(0x7f2d740726e0 msgr2=0x7f2d74072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73e00640 1 -- 172.21.15.5:0/3504855651 shutdown_connections 2024-04-25T10:10:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.755+0000 7f2d73e00640 1 -- 172.21.15.5:0/3504855651 wait complete. 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.757+0000 7f2d73e00640 1 Processor -- start 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.757+0000 7f2d73e00640 1 -- start start 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.758+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58004d40 0x7f2d740b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.758+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d741e7230 0x7f2d741e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.758+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 0x7f2d74149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.758+0000 7f2d73e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740d31c0 con 0x7f2d74149a80 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.758+0000 7f2d73e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740d3040 con 0x7f2d741e7230 2024-04-25T10:10:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.758+0000 7f2d73e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740d2080 con 0x7f2d58004d40 2024-04-25T10:10:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 0x7f2d74149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 0x7f2d74149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53646/0 (socket says 172.21.15.5:53646) 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 -- 172.21.15.5:0/330974410 learned_addr learned my addr 172.21.15.5:0/330974410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d72a00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58004d40 0x7f2d740b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d6be00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d741e7230 0x7f2d741e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 -- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58004d40 msgr2=0x7f2d740b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58004d40 0x7f2d740b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 -- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d741e7230 msgr2=0x7f2d741e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d741e7230 0x7f2d741e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.759+0000 7f2d73400640 1 -- 172.21.15.5:0/330974410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d741e7bf0 con 0x7f2d74149a80 2024-04-25T10:10:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.760+0000 7f2d73400640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 0x7f2d74149ee0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0074b0 tx=0x7f2d6c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.760+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c009be0 con 0x7f2d74149a80 2024-04-25T10:10:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.760+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c02e440 con 0x7f2d74149a80 2024-04-25T10:10:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.760+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c038280 con 0x7f2d74149a80 2024-04-25T10:10:56.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.760+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7414ca60 con 0x7f2d74149a80 2024-04-25T10:10:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.760+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7414d020 con 0x7f2d74149a80 2024-04-25T10:10:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.761+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74068080 con 0x7f2d74149a80 2024-04-25T10:10:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.762+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d6c038870 con 0x7f2d74149a80 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.763+0000 7f2d6aa00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.764+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2d6c0aa2c0 con 0x7f2d74149a80 2024-04-25T10:10:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.764+0000 7f2d72a00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.765+0000 7f2d72a00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005a90 tx=0x7f2d64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.767+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d6c049070 con 0x7f2d74149a80 2024-04-25T10:10:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.973+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2d74068f60 con 0x7f2d74149a80 2024-04-25T10:10:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.973+0000 7f2d6aa00640 1 -- 172.21.15.5:0/330974410 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2d6c032030 con 0x7f2d74149a80 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:10:56.979 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 msgr2=0x7f2d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005a90 tx=0x7f2d64009040 comp rx=0 tx=0).stop 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 msgr2=0x7f2d74149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 0x7f2d74149ee0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0074b0 tx=0x7f2d6c004ab0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 shutdown_connections 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d74149a80 0x7f2d74149ee0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d5006b4d0 0x7f2d5006d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d741e7230 0x7f2d741e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 --2- 172.21.15.5:0/330974410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d58004d40 0x7f2d740b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.976+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 >> 172.21.15.5:0/330974410 conn(0x7f2d740726e0 msgr2=0x7f2d740b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.977+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 shutdown_connections 2024-04-25T10:10:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:56.977+0000 7f2d73e00640 1 -- 172.21.15.5:0/330974410 wait complete. 2024-04-25T10:10:57.001 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:10:57.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:10:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.216+0000 7f368d400640 1 Processor -- start 2024-04-25T10:10:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.216+0000 7f368d400640 1 -- start start 2024-04-25T10:10:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.218+0000 7f368d400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36880cc8f0 0x7f36880ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.218+0000 7f368d400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368806d780 con 0x7f36881426a0 2024-04-25T10:10:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.218+0000 7f368d400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368806da60 con 0x7f3688142010 2024-04-25T10:10:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.218+0000 7f368d400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36881423f0 con 0x7f36880cc8f0 2024-04-25T10:10:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.218+0000 7f3687200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f36881426a0 0x7f3688142af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44640/0 (socket says 172.21.15.5:44640) 2024-04-25T10:10:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.218+0000 7f3687200640 1 -- 172.21.15.5:0/1194552434 learned_addr learned my addr 172.21.15.5:0/1194552434 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685e00640 1 --2- 172.21.15.5:0/1194552434 >> v2:172.21.15.80:3300/0 conn(0x7f36880cc8f0 0x7f36880ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623260638 0 0) 0x7f368806d780 con 0x7f36881426a0 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c003520 con 0x7f36881426a0 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 724926590 0 0) 0x7f366c003520 con 0x7f36881426a0 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 >> v2:172.21.15.80:3300/0 conn(0x7f36880cc8f0 msgr2=0x7f36880ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 --2- 172.21.15.5:0/1194552434 >> v2:172.21.15.80:3300/0 conn(0x7f36880cc8f0 0x7f36880ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 >> v1:172.21.15.80:6789/0 conn(0x7f3688142010 legacy=0x7f368806e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368806dd40 con 0x7f36881426a0 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f3680001670 con 0x7f36881426a0 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.219+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 >> v1:172.21.15.5:6789/0 conn(0x7f36881426a0 legacy=0x7f3688142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685400640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f366c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685400640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685400640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36881423f0 con 0x7f366c003a00 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00c0f0 con 0x7f366c004800 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c00c2f0 con 0x7f366c0051c0 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3686800640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f366c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685e00640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3687200640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3686800640 1 -- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f366c0051c0 msgr2=0x7f366c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3686800640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3686800640 1 -- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f366c004800 msgr2=0x7f366c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3686800640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3686800640 1 -- 172.21.15.5:0/1194552434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c003520 con 0x7f366c003a00 2024-04-25T10:10:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3687200640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.220+0000 7f3685e00640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.221+0000 7f3687200640 1 -- 172.21.15.5:0/1194552434 reap_dead start 2024-04-25T10:10:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.222+0000 7f3686800640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f366c003e40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3678001e50 tx=0x7f366c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a69675ff289a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.222+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678001a70 con 0x7f366c003a00 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.222+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367802ae40 con 0x7f366c003a00 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.222+0000 7f3685400640 1 -- 172.21.15.5:0/1194552434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678003bb0 con 0x7f366c003a00 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.222+0000 7f368d400640 1 -- 172.21.15.5:0/1194552434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 msgr2=0x7f366c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.222+0000 7f368d400640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f366c003e40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3678001e50 tx=0x7f366c005830 comp rx=0 tx=0).stop 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.223+0000 7f368d400640 1 -- 172.21.15.5:0/1194552434 shutdown_connections 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.223+0000 7f368d400640 1 --2- 172.21.15.5:0/1194552434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f366c003e40 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.223+0000 7f368d400640 1 -- 172.21.15.5:0/1194552434 >> 172.21.15.5:0/1194552434 conn(0x7f3688066f80 msgr2=0x7f368806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.223+0000 7f368d400640 1 -- 172.21.15.5:0/1194552434 shutdown_connections 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.223+0000 7f368d400640 1 -- 172.21.15.5:0/1194552434 wait complete. 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.224+0000 7f368d400640 1 Processor -- start 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.225+0000 7f368d400640 1 -- start start 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.225+0000 7f368d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f36881e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.225+0000 7f368d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368815db10 0x7f368815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.225+0000 7f368d400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368815df90 0x7f368815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f368d400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368806e300 con 0x7f366c003a00 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f368d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36880cc210 con 0x7f368815db10 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f368d400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36880ccd80 con 0x7f368815df90 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3686800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f36881e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3686800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f36881e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41112/0 (socket says 172.21.15.5:41112) 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3686800640 1 -- 172.21.15.5:0/2085170693 learned_addr learned my addr 172.21.15.5:0/2085170693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3687200640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368815df90 0x7f368815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3685e00640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368815db10 0x7f368815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3686800640 1 -- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368815df90 msgr2=0x7f368815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.226+0000 7f3686800640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368815df90 0x7f368815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f3686800640 1 -- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368815db10 msgr2=0x7f368815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f3686800640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368815db10 0x7f368815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f3686800640 1 -- 172.21.15.5:0/2085170693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368815e540 con 0x7f366c003a00 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f3685e00640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368815db10 0x7f368815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f3686800640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f36881e71e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3678001f70 tx=0x7f36780014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367803f020 con 0x7f366c003a00 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367802ac40 con 0x7f366c003a00 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678007db0 con 0x7f366c003a00 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.227+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368815e820 con 0x7f366c003a00 2024-04-25T10:10:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.228+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36881e4b20 con 0x7f366c003a00 2024-04-25T10:10:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.229+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3688069040 con 0x7f366c003a00 2024-04-25T10:10:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.230+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3678042020 con 0x7f366c003a00 2024-04-25T10:10:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.230+0000 7f367ea00640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.231+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f36780a9f80 con 0x7f366c003a00 2024-04-25T10:10:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.231+0000 7f3685e00640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.232+0000 7f3685e00640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f36881e4630 tx=0x7f3670008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.235+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3678049030 con 0x7f366c003a00 2024-04-25T10:10:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:57.435+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f368815b7e0 con 0x7f366c003a00 2024-04-25T10:10:58.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.087+0000 7f367ea00640 1 -- 172.21.15.5:0/2085170693 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3678078f20 con 0x7f366c003a00 2024-04-25T10:10:58.093 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:10:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366406b4d0 msgr2=0x7f366406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f36881e4630 tx=0x7f3670008040 comp rx=0 tx=0).stop 2024-04-25T10:10:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 msgr2=0x7f36881e71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f36881e71e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f3678001f70 tx=0x7f36780014a0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 shutdown_connections 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368815df90 0x7f368815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368815db10 0x7f368815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 --2- 172.21.15.5:0/2085170693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366c003a00 0x7f36881e71e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.091+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 >> 172.21.15.5:0/2085170693 conn(0x7f3688066f80 msgr2=0x7f36880b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.092+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 shutdown_connections 2024-04-25T10:10:58.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.092+0000 7f368d400640 1 -- 172.21.15.5:0/2085170693 wait complete. 2024-04-25T10:10:58.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:10:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.326+0000 7f0db3e00640 1 Processor -- start 2024-04-25T10:10:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.326+0000 7f0db3e00640 1 -- start start 2024-04-25T10:10:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.328+0000 7f0db3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0db415b500 0x7f0db41651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.328+0000 7f0db3e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db4072010 con 0x7f0db40cc840 2024-04-25T10:10:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.328+0000 7f0db3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db40ce570 con 0x7f0db406e840 2024-04-25T10:10:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.328+0000 7f0db3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db415b910 con 0x7f0db415b500 2024-04-25T10:10:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.328+0000 7f0db2a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0db40cc840 0x7f0db406e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44654/0 (socket says 172.21.15.5:44654) 2024-04-25T10:10:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.328+0000 7f0db2a00640 1 -- 172.21.15.5:0/3968284211 learned_addr learned my addr 172.21.15.5:0/3968284211 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db3400640 1 --2- 172.21.15.5:0/3968284211 >> v2:172.21.15.80:3300/0 conn(0x7f0db415b500 0x7f0db41651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239540765 0 0) 0x7f0db4072010 con 0x7f0db40cc840 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d98003520 con 0x7f0db40cc840 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 446246717 0 0) 0x7f0d98003520 con 0x7f0db40cc840 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 >> v2:172.21.15.80:3300/0 conn(0x7f0db415b500 msgr2=0x7f0db41651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 --2- 172.21.15.5:0/3968284211 >> v2:172.21.15.80:3300/0 conn(0x7f0db415b500 0x7f0db41651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 >> v1:172.21.15.80:6789/0 conn(0x7f0db406e840 legacy=0x7f0db415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db40ce850 con 0x7f0db40cc840 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0da4001670 con 0x7f0db40cc840 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 >> v1:172.21.15.5:6789/0 conn(0x7f0db40cc840 legacy=0x7f0db406e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d98003a00 0x7f0d98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.329+0000 7f0db2000640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d98004800 0x7f0d98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db2000640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d980051c0 0x7f0d9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db415b910 con 0x7f0d980051c0 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9800c0c0 con 0x7f0d98003a00 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9800c2c0 con 0x7f0d98004800 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db3400640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d980051c0 0x7f0d9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0daa000640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d98004800 0x7f0d98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db2a00640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d98003a00 0x7f0d98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db3400640 1 -- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d98004800 msgr2=0x7f0d98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db3400640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d98004800 0x7f0d98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db3400640 1 -- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d98003a00 msgr2=0x7f0d98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db3400640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d98003a00 0x7f0d98003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.330+0000 7f0db3400640 1 -- 172.21.15.5:0/3968284211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98003520 con 0x7f0d980051c0 2024-04-25T10:10:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.331+0000 7f0db2a00640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d98003a00 0x7f0d98003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.331+0000 7f0db3400640 1 -- 172.21.15.5:0/3968284211 reap_dead start 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.332+0000 7f0db3400640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d980051c0 0x7f0d9800b9c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ac0 tx=0x7f0d98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be321f4ca088e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.332+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac009690 con 0x7f0d980051c0 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.332+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac004020 con 0x7f0d980051c0 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.332+0000 7f0db2000640 1 -- 172.21.15.5:0/3968284211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0043d0 con 0x7f0d980051c0 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.332+0000 7f0db3e00640 1 -- 172.21.15.5:0/3968284211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d980051c0 msgr2=0x7f0d9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.332+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d980051c0 0x7f0d9800b9c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0dac009ac0 tx=0x7f0d98005830 comp rx=0 tx=0).stop 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.333+0000 7f0db3e00640 1 -- 172.21.15.5:0/3968284211 shutdown_connections 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.333+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3968284211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d980051c0 0x7f0d9800b9c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.333+0000 7f0db3e00640 1 -- 172.21.15.5:0/3968284211 >> 172.21.15.5:0/3968284211 conn(0x7f0db4066f20 msgr2=0x7f0db406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.333+0000 7f0db3e00640 1 -- 172.21.15.5:0/3968284211 shutdown_connections 2024-04-25T10:10:58.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.333+0000 7f0db3e00640 1 -- 172.21.15.5:0/3968284211 wait complete. 2024-04-25T10:10:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.335+0000 7f0db3e00640 1 Processor -- start 2024-04-25T10:10:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.335+0000 7f0db3e00640 1 -- start start 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d980051c0 0x7f0db41f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 0x7f0db414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0db40aeee0 0x7f0db41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40b2d20 con 0x7f0db41545e0 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40cee10 con 0x7f0db40aeee0 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40ce1d0 con 0x7f0d980051c0 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 0x7f0db414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 0x7f0db414db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41122/0 (socket says 172.21.15.5:41122) 2024-04-25T10:10:58.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 -- 172.21.15.5:0/3799490520 learned_addr learned my addr 172.21.15.5:0/3799490520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db2a00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d980051c0 0x7f0db41f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0db3400640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0db40aeee0 0x7f0db41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 -- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d980051c0 msgr2=0x7f0db41f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d980051c0 0x7f0db41f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 -- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0db40aeee0 msgr2=0x7f0db41588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0db40aeee0 0x7f0db41588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.336+0000 7f0daa000640 1 -- 172.21.15.5:0/3799490520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4154d50 con 0x7f0db41545e0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.337+0000 7f0daa000640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 0x7f0db414db10 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e8f0 tx=0x7f0d9c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.337+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c003b80 con 0x7f0db41545e0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.337+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9c018070 con 0x7f0db41545e0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.337+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c0050d0 con 0x7f0db41545e0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.337+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db4154a70 con 0x7f0db41545e0 2024-04-25T10:10:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.337+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db41f3d40 con 0x7f0db41545e0 2024-04-25T10:10:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.338+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db4068f60 con 0x7f0db41545e0 2024-04-25T10:10:58.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.340+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d9c005270 con 0x7f0db41545e0 2024-04-25T10:10:58.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.341+0000 7f0dabe00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d9006b4d0 0x7f0d9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.341+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f0d9c08a0f0 con 0x7f0db41545e0 2024-04-25T10:10:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.341+0000 7f0db2a00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d9006b4d0 0x7f0d9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.343+0000 7f0db2a00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d9006b4d0 0x7f0d9006d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0da40017e0 tx=0x7f0da4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.346+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d9c059050 con 0x7f0db41545e0 2024-04-25T10:10:58.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:58.546+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0db4150570 con 0x7f0db41545e0 2024-04-25T10:10:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.102+0000 7f0dabe00640 1 -- 172.21.15.5:0/3799490520 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0d9c01be40 con 0x7f0db41545e0 2024-04-25T10:10:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d9006b4d0 msgr2=0x7f0d9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d9006b4d0 0x7f0d9006d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0da40017e0 tx=0x7f0da4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 msgr2=0x7f0db414db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 0x7f0db414db10 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00e8f0 tx=0x7f0d9c00edb0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 shutdown_connections 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0d9006b4d0 0x7f0d9006d990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0db40aeee0 0x7f0db41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0db41545e0 0x7f0db414db10 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 --2- 172.21.15.5:0/3799490520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0d980051c0 0x7f0db41f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.108+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 >> 172.21.15.5:0/3799490520 conn(0x7f0db4066f20 msgr2=0x7f0db4071890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.109+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 shutdown_connections 2024-04-25T10:10:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.109+0000 7f0db3e00640 1 -- 172.21.15.5:0/3799490520 wait complete. 2024-04-25T10:10:59.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:10:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.322+0000 7f89b8c00640 1 Processor -- start 2024-04-25T10:10:59.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.322+0000 7f89b8c00640 1 -- start start 2024-04-25T10:10:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.324+0000 7f89b8c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89b40ae860 0x7f89b4071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.324+0000 7f89b8c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b406f080 con 0x7f89b40aece0 2024-04-25T10:10:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.324+0000 7f89b8c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b406f360 con 0x7f89b41465d0 2024-04-25T10:10:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.324+0000 7f89b8c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40ae060 con 0x7f89b40ae860 2024-04-25T10:10:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.324+0000 7f89b2800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f89b40aece0 0x7f89b415f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44668/0 (socket says 172.21.15.5:44668) 2024-04-25T10:10:59.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.324+0000 7f89b2800640 1 -- 172.21.15.5:0/332009691 learned_addr learned my addr 172.21.15.5:0/332009691 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:10:59.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b1400640 1 --2- 172.21.15.5:0/332009691 >> v2:172.21.15.80:3300/0 conn(0x7f89b40ae860 0x7f89b4071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766171494 0 0) 0x7f89b406f080 con 0x7f89b40aece0 2024-04-25T10:10:59.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8998003520 con 0x7f89b40aece0 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709562583 0 0) 0x7f8998003520 con 0x7f89b40aece0 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 >> v2:172.21.15.80:3300/0 conn(0x7f89b40ae860 msgr2=0x7f89b4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 --2- 172.21.15.5:0/332009691 >> v2:172.21.15.80:3300/0 conn(0x7f89b40ae860 0x7f89b4071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 >> v1:172.21.15.80:6789/0 conn(0x7f89b41465d0 legacy=0x7f89b415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40ae340 con 0x7f89b40aece0 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f89ac0019f0 con 0x7f89b40aece0 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 >> v1:172.21.15.5:6789/0 conn(0x7f89b40aece0 legacy=0x7f89b415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.325+0000 7f89b0a00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89980039b0 0x7f8998003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b0a00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89980043b0 0x7f8998004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b0a00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8998005080 0x7f899800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40ae060 con 0x7f89980043b0 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899800bfe0 con 0x7f89980039b0 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899800c1e0 con 0x7f8998005080 2024-04-25T10:10:59.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1400640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89980043b0 0x7f8998004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1400640 1 -- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8998005080 msgr2=0x7f899800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1400640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8998005080 0x7f899800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1e00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89980039b0 0x7f8998003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1400640 1 -- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89980039b0 msgr2=0x7f8998003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1400640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89980039b0 0x7f8998003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1400640 1 -- 172.21.15.5:0/332009691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998003520 con 0x7f89980043b0 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b2800640 1 -- 172.21.15.5:0/332009691 reap_dead start 2024-04-25T10:10:59.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.326+0000 7f89b1e00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89980039b0 0x7f8998003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:59.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.327+0000 7f89b1400640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89980043b0 0x7f8998004880 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009ac0 tx=0x7f8998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c615b5f25fc837 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0036070 con 0x7f89980043b0 2024-04-25T10:10:59.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a0031400 con 0x7f89980043b0 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b0a00640 1 -- 172.21.15.5:0/332009691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0009690 con 0x7f89980043b0 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 -- 172.21.15.5:0/332009691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89980043b0 msgr2=0x7f8998004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89980043b0 0x7f8998004880 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009ac0 tx=0x7f8998005830 comp rx=0 tx=0).stop 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 -- 172.21.15.5:0/332009691 shutdown_connections 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 --2- 172.21.15.5:0/332009691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89980043b0 0x7f8998004880 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 -- 172.21.15.5:0/332009691 >> 172.21.15.5:0/332009691 conn(0x7f89b406ab60 msgr2=0x7f89b406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 -- 172.21.15.5:0/332009691 shutdown_connections 2024-04-25T10:10:59.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.328+0000 7f89b8c00640 1 -- 172.21.15.5:0/332009691 wait complete. 2024-04-25T10:10:59.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.330+0000 7f89b8c00640 1 Processor -- start 2024-04-25T10:10:59.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.330+0000 7f89b8c00640 1 -- start start 2024-04-25T10:10:59.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.331+0000 7f89b8c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89980043b0 0x7f89b40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.331+0000 7f89b8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 0x7f89b415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.331+0000 7f89b8c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b40b0f60 0x7f89b40b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.331+0000 7f89b8c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b406ece0 con 0x7f89b40afcb0 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.331+0000 7f89b8c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b406e7d0 con 0x7f89b40b0f60 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.331+0000 7f89b8c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b406ee60 con 0x7f89980043b0 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 0x7f89b415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 0x7f89b415d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41146/0 (socket says 172.21.15.5:41146) 2024-04-25T10:10:59.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 -- 172.21.15.5:0/3783655332 learned_addr learned my addr 172.21.15.5:0/3783655332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1e00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89980043b0 0x7f89b40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 -- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89980043b0 msgr2=0x7f89b40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b2800640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b40b0f60 0x7f89b40b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89980043b0 0x7f89b40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 -- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b40b0f60 msgr2=0x7f89b40b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b40b0f60 0x7f89b40b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1400640 1 -- 172.21.15.5:0/3783655332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40b0470 con 0x7f89b40afcb0 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b1e00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89980043b0 0x7f89b40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.332+0000 7f89b2800640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b40b0f60 0x7f89b40b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.333+0000 7f89b1400640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 0x7f89b415d190 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f89a0034040 tx=0x7f89a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.333+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a003f020 con 0x7f89b40afcb0 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.333+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a0037040 con 0x7f89b40afcb0 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.333+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a0036070 con 0x7f89b40afcb0 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.333+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b40b0750 con 0x7f89b40afcb0 2024-04-25T10:10:59.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.333+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b40cbe00 con 0x7f89b40afcb0 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.334+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b4068f60 con 0x7f89b40afcb0 2024-04-25T10:10:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.335+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89a0042020 con 0x7f89b40afcb0 2024-04-25T10:10:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.336+0000 7f89aa000640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f899406b4d0 0x7f899406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.336+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f89a00aa4b0 con 0x7f89b40afcb0 2024-04-25T10:10:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.337+0000 7f89b1e00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f899406b4d0 0x7f899406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.338+0000 7f89b1e00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f899406b4d0 0x7f899406d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f89b4050700 tx=0x7f89a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:59.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.341+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89a0049030 con 0x7f89b40afcb0 2024-04-25T10:10:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:10:59.577+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f89b40cf160 con 0x7f89b40afcb0 2024-04-25T10:11:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.130+0000 7f89aa000640 1 -- 172.21.15.5:0/3783655332 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v79) v1 ==== 174+0+0 (secure 0 0 0) 0x7f89a0079220 con 0x7f89b40afcb0 2024-04-25T10:11:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T10:11:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.143+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f899406b4d0 msgr2=0x7f899406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.143+0000 7f89b8c00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f899406b4d0 0x7f899406d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f89b4050700 tx=0x7f89a4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.143+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 msgr2=0x7f89b415d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.143+0000 7f89b8c00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 0x7f89b415d190 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f89a0034040 tx=0x7f89a0004730 comp rx=0 tx=0).stop 2024-04-25T10:11:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.144+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 shutdown_connections 2024-04-25T10:11:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.144+0000 7f89b8c00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f899406b4d0 0x7f899406d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.144+0000 7f89b8c00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b40b0f60 0x7f89b40b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.144+0000 7f89b8c00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89b40afcb0 0x7f89b415d190 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.144+0000 7f89b8c00640 1 --2- 172.21.15.5:0/3783655332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f89980043b0 0x7f89b40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.144+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 >> 172.21.15.5:0/3783655332 conn(0x7f89b406ab60 msgr2=0x7f89b4142480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.146+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 shutdown_connections 2024-04-25T10:11:00.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.146+0000 7f89b8c00640 1 -- 172.21.15.5:0/3783655332 wait complete. 2024-04-25T10:11:00.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.349+0000 7fa020a00640 1 Processor -- start 2024-04-25T10:11:00.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.350+0000 7fa020a00640 1 -- start start 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa020a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa01c161200 0x7fa01c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa020a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d07d0 0x7fa01c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa020a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c0cf410 con 0x7fa01c0d2380 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa020a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01c0d27c0 con 0x7fa01c161200 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa020a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0b28b0 con 0x7fa01c0d07d0 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa01a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa01c161200 0x7fa01c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa01a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa01c161200 0x7fa01c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41156/0 (socket says 172.21.15.5:41156) 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.351+0000 7fa01a800640 1 -- 172.21.15.5:0/2422019336 learned_addr learned my addr 172.21.15.5:0/2422019336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.352+0000 7fa01b200640 1 --2- 172.21.15.5:0/2422019336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d07d0 0x7fa01c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.352+0000 7fa01a800640 1 -- 172.21.15.5:0/2422019336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d07d0 msgr2=0x7fa01c0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.352+0000 7fa01a800640 1 --2- 172.21.15.5:0/2422019336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d07d0 0x7fa01c0b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.352+0000 7fa01a800640 1 -- 172.21.15.5:0/2422019336 >> v1:172.21.15.80:6789/0 conn(0x7fa01c0d2380 legacy=0x7fa01c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.352+0000 7fa01a800640 1 -- 172.21.15.5:0/2422019336 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0cf6f0 con 0x7fa01c161200 2024-04-25T10:11:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.353+0000 7fa01a800640 1 --2- 172.21.15.5:0/2422019336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c161200 0x7fa01c0b4bb0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0098a0 tx=0x7fa00c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39587538d7c31bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.353+0000 7fa019400640 1 -- 172.21.15.5:0/2422019336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c037070 con 0x7fa01c161200 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.353+0000 7fa019400640 1 -- 172.21.15.5:0/2422019336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c032440 con 0x7fa01c161200 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.353+0000 7fa019400640 1 -- 172.21.15.5:0/2422019336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c032810 con 0x7fa01c161200 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 -- 172.21.15.5:0/2422019336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c161200 msgr2=0x7fa01c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 --2- 172.21.15.5:0/2422019336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c161200 0x7fa01c0b4bb0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0098a0 tx=0x7fa00c02a290 comp rx=0 tx=0).stop 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 -- 172.21.15.5:0/2422019336 shutdown_connections 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 --2- 172.21.15.5:0/2422019336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d07d0 0x7fa01c0b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 --2- 172.21.15.5:0/2422019336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c161200 0x7fa01c0b4bb0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 -- 172.21.15.5:0/2422019336 >> 172.21.15.5:0/2422019336 conn(0x7fa01c08af80 msgr2=0x7fa01c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 -- 172.21.15.5:0/2422019336 shutdown_connections 2024-04-25T10:11:00.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.354+0000 7fa020a00640 1 -- 172.21.15.5:0/2422019336 wait complete. 2024-04-25T10:11:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.356+0000 7fa020a00640 1 Processor -- start 2024-04-25T10:11:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.356+0000 7fa020a00640 1 -- start start 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa020a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 0x7fa01c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa020a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d2380 0x7fa01c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa020a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01c161200 0x7fa01c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa020a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0b34c0 con 0x7fa01c0d07d0 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa020a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0d42b0 con 0x7fa01c161200 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa020a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0b3640 con 0x7fa01c0d2380 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa01a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 0x7fa01c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa01a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 0x7fa01c0c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41166/0 (socket says 172.21.15.5:41166) 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.357+0000 7fa01a800640 1 -- 172.21.15.5:0/348764078 learned_addr learned my addr 172.21.15.5:0/348764078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01b200640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01c161200 0x7fa01c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa019e00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d2380 0x7fa01c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01a800640 1 -- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d2380 msgr2=0x7fa01c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01a800640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d2380 0x7fa01c0ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01a800640 1 -- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01c161200 msgr2=0x7fa01c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01a800640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01c161200 0x7fa01c0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01a800640 1 -- 172.21.15.5:0/348764078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0c9870 con 0x7fa01c0d07d0 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa019e00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d2380 0x7fa01c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa01a800640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 0x7fa01c0c9330 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0098a0 tx=0x7fa00c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c041020 con 0x7fa01c0d07d0 2024-04-25T10:11:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c0306c0 con 0x7fa01c0d07d0 2024-04-25T10:11:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c037040 con 0x7fa01c0d07d0 2024-04-25T10:11:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c0b7150 con 0x7fa01c0d07d0 2024-04-25T10:11:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.358+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c0b7710 con 0x7fa01c0d07d0 2024-04-25T10:11:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.360+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c068f60 con 0x7fa01c0d07d0 2024-04-25T10:11:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.361+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa00c044020 con 0x7fa01c0d07d0 2024-04-25T10:11:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.362+0000 7fa012a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9ff806b500 0x7f9ff806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.362+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa00c0aaac0 con 0x7fa01c0d07d0 2024-04-25T10:11:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.363+0000 7fa019e00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9ff806b500 0x7f9ff806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.364+0000 7fa019e00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9ff806b500 0x7f9ff806d9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa01c159520 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.367+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa00c04b030 con 0x7fa01c0d07d0 2024-04-25T10:11:00.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.557+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa01c0ca760 con 0x7fa01c0d07d0 2024-04-25T10:11:00.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.560+0000 7fa012a00640 1 -- 172.21.15.5:0/348764078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16805 (secure 0 0 0) 0x7fa01c0c9870 con 0x7fa01c0d07d0 2024-04-25T10:11:00.585 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:00.585 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:10:59.578968+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.563+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9ff806b500 msgr2=0x7f9ff806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.563+0000 7fa020a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9ff806b500 0x7f9ff806d9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa01c159520 tx=0x7fa004009040 comp rx=0 tx=0).stop 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.563+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 msgr2=0x7fa01c0c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.563+0000 7fa020a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 0x7fa01c0c9330 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0098a0 tx=0x7fa00c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 shutdown_connections 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9ff806b500 0x7f9ff806d9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa01c161200 0x7fa01c0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa01c0d2380 0x7fa01c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 --2- 172.21.15.5:0/348764078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa01c0d07d0 0x7fa01c0c9330 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 >> 172.21.15.5:0/348764078 conn(0x7fa01c08af80 msgr2=0x7fa01c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 shutdown_connections 2024-04-25T10:11:00.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.564+0000 7fa020a00640 1 -- 172.21.15.5:0/348764078 wait complete. 2024-04-25T10:11:00.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.788+0000 7f9d81400640 1 Processor -- start 2024-04-25T10:11:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.788+0000 7f9d81400640 1 -- start start 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.789+0000 7f9d81400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9d7c0c44d0 0x7f9d7c0c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.789+0000 7f9d81400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9d7c15f190 0x7f9d7c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.789+0000 7f9d81400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c162750 con 0x7f9d7c0b1940 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.789+0000 7f9d81400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c09ace0 con 0x7f9d7c0c44d0 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.789+0000 7f9d81400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c162a30 con 0x7f9d7c15f190 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d7b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9d7c0c44d0 0x7f9d7c0c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d7be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9d7c0b1940 0x7f9d7c06bc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44670/0 (socket says 172.21.15.5:44670) 2024-04-25T10:11:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d7be00640 1 -- 172.21.15.5:0/1555974501 learned_addr learned my addr 172.21.15.5:0/1555974501 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d80a00640 1 --2- 172.21.15.5:0/1555974501 >> v2:172.21.15.80:3300/0 conn(0x7f9d7c15f190 0x7f9d7c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d7b400640 1 -- 172.21.15.5:0/1555974501 >> v2:172.21.15.80:3300/0 conn(0x7f9d7c15f190 msgr2=0x7f9d7c15f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d7b400640 1 --2- 172.21.15.5:0/1555974501 >> v2:172.21.15.80:3300/0 conn(0x7f9d7c15f190 0x7f9d7c15f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.790+0000 7f9d7b400640 1 -- 172.21.15.5:0/1555974501 >> v1:172.21.15.5:6789/0 conn(0x7f9d7c0b1940 legacy=0x7f9d7c06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.791+0000 7f9d7b400640 1 -- 172.21.15.5:0/1555974501 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c162d10 con 0x7f9d7c0c44d0 2024-04-25T10:11:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.791+0000 7f9d7aa00640 1 -- 172.21.15.5:0/1555974501 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253992597 0 0) 0x7f9d7c162750 con 0x7f9d7c0b1940 2024-04-25T10:11:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.792+0000 7f9d7b400640 1 --2- 172.21.15.5:0/1555974501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0c4980 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0cb950 tx=0x7f9d6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a68a6652f6873c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.792+0000 7f9d7aa00640 1 -- 172.21.15.5:0/1555974501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68037070 con 0x7f9d7c0c44d0 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.792+0000 7f9d7aa00640 1 -- 172.21.15.5:0/1555974501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68032440 con 0x7f9d7c0c44d0 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.792+0000 7f9d7aa00640 1 -- 172.21.15.5:0/1555974501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68032810 con 0x7f9d7c0c44d0 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.792+0000 7f9d81400640 1 -- 172.21.15.5:0/1555974501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c0c44d0 msgr2=0x7f9d7c0c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.792+0000 7f9d81400640 1 --2- 172.21.15.5:0/1555974501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0c4980 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0cb950 tx=0x7f9d6802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.793+0000 7f9d81400640 1 -- 172.21.15.5:0/1555974501 shutdown_connections 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.793+0000 7f9d81400640 1 --2- 172.21.15.5:0/1555974501 >> v2:172.21.15.80:3300/0 conn(0x7f9d7c15f190 0x7f9d7c15f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.793+0000 7f9d81400640 1 --2- 172.21.15.5:0/1555974501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0c4980 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.793+0000 7f9d81400640 1 -- 172.21.15.5:0/1555974501 >> 172.21.15.5:0/1555974501 conn(0x7f9d7c0b2ff0 msgr2=0x7f9d7c0b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.793+0000 7f9d81400640 1 -- 172.21.15.5:0/1555974501 shutdown_connections 2024-04-25T10:11:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.793+0000 7f9d81400640 1 -- 172.21.15.5:0/1555974501 wait complete. 2024-04-25T10:11:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.794+0000 7f9d81400640 1 Processor -- start 2024-04-25T10:11:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.794+0000 7f9d81400640 1 -- start start 2024-04-25T10:11:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.795+0000 7f9d81400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d7c0b1940 0x7f9d7c0b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d81400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d81400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 0x7f9d7c0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d81400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0b1d20 con 0x7f9d7c15f190 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d81400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c1632d0 con 0x7f9d7c0c44d0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d81400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0cbb50 con 0x7f9d7c0b1940 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 0x7f9d7c0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 0x7f9d7c0ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41186/0 (socket says 172.21.15.5:41186) 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 -- 172.21.15.5:0/762170194 learned_addr learned my addr 172.21.15.5:0/762170194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d7b400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 -- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d7c0b1940 msgr2=0x7f9d7c0b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d7be00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d7c0b1940 0x7f9d7c0b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d7c0b1940 0x7f9d7c0b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 -- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d7c0c44d0 msgr2=0x7f9d7c0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d80a00640 1 -- 172.21.15.5:0/762170194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c0c9b90 con 0x7f9d7c15f190 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.796+0000 7f9d7b400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.797+0000 7f9d80a00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 0x7f9d7c0ae1e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f9d74007e60 tx=0x7f9d7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.797+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74014420 con 0x7f9d7c15f190 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.797+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74019070 con 0x7f9d7c15f190 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.797+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c0c9e70 con 0x7f9d7c15f190 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.797+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74014820 con 0x7f9d7c15f190 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.797+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c0ca430 con 0x7f9d7c15f190 2024-04-25T10:11:00.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.798+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c067560 con 0x7f9d7c15f190 2024-04-25T10:11:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.799+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d740149c0 con 0x7f9d7c15f190 2024-04-25T10:11:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.800+0000 7f9d73e00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.801+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d7408ac40 con 0x7f9d7c15f190 2024-04-25T10:11:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.801+0000 7f9d7be00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.802+0000 7f9d7be00640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007fd0 tx=0x7f9d6c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:00.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:00.804+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d74059890 con 0x7f9d7c15f190 2024-04-25T10:11:01.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.010+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d7c073ce0 con 0x7f9d7c15f190 2024-04-25T10:11:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.012+0000 7f9d73e00640 1 -- 172.21.15.5:0/762170194 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16805 (secure 0 0 0) 0x7f9d7c073ce0 con 0x7f9d7c15f190 2024-04-25T10:11:01.018 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:01.018 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:10:59.578968+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d5c06b4d0 msgr2=0x7f9d5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c007fd0 tx=0x7f9d6c009040 comp rx=0 tx=0).stop 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 msgr2=0x7f9d7c0ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 0x7f9d7c0ae1e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f9d74007e60 tx=0x7f9d7400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 shutdown_connections 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d7c15f190 0x7f9d7c0ae1e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d7c0c44d0 0x7f9d7c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 --2- 172.21.15.5:0/762170194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9d7c0b1940 0x7f9d7c0b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.015+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 >> 172.21.15.5:0/762170194 conn(0x7f9d7c0b2ff0 msgr2=0x7f9d7c070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.016+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 shutdown_connections 2024-04-25T10:11:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.016+0000 7f9d81400640 1 -- 172.21.15.5:0/762170194 wait complete. 2024-04-25T10:11:01.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.247+0000 7feddca00640 1 Processor -- start 2024-04-25T10:11:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.248+0000 7feddca00640 1 -- start start 2024-04-25T10:11:01.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7feddca00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fedd80b48c0 0x7fedd815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7feddca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 0x7fedd80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7feddca00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd81606e0 con 0x7fedd80d3f90 2024-04-25T10:11:01.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7feddca00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd81609c0 con 0x7fedd80b48c0 2024-04-25T10:11:01.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7feddca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd8160ca0 con 0x7fedd8161d90 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7fedd5e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fedd80d3f90 0x7fedd8161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44686/0 (socket says 172.21.15.5:44686) 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7fedd5e00640 1 -- 172.21.15.5:0/3545619023 learned_addr learned my addr 172.21.15.5:0/3545619023 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.249+0000 7fedd7200640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 0x7fedd80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd6800640 1 --2- 172.21.15.5:0/3545619023 >> v2:172.21.15.80:3300/0 conn(0x7fedd80b48c0 0x7fedd815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203754097 0 0) 0x7fedd81606e0 con 0x7fedd80d3f90 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedbc003520 con 0x7fedd80d3f90 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738647681 0 0) 0x7fedbc003520 con 0x7fedd80d3f90 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 msgr2=0x7fedd80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 0x7fedd80b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 >> v2:172.21.15.80:3300/0 conn(0x7fedd80b48c0 msgr2=0x7fedd815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 --2- 172.21.15.5:0/3545619023 >> v2:172.21.15.80:3300/0 conn(0x7fedd80b48c0 0x7fedd815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd8160f80 con 0x7fedd80d3f90 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fedc00017f0 con 0x7fedd80d3f90 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd7200640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 0x7fedd80b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 >> v1:172.21.15.5:6789/0 conn(0x7fedd80d3f90 legacy=0x7fedd8161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 0x7fedbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedbc004d40 0x7fedbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd81609c0 con 0x7fedbc0039b0 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd8160ca0 con 0x7fedbc004d40 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc00baa0 con 0x7fedd8161d90 2024-04-25T10:11:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.250+0000 7fedd6800640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.251+0000 7fedd7200640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedbc004d40 0x7fedbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.251+0000 7fedd6800640 1 -- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 msgr2=0x7fedbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.251+0000 7fedd6800640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd8161d90 0x7fedbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.251+0000 7fedd6800640 1 -- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedbc004d40 msgr2=0x7fedbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.251+0000 7fedd6800640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedbc004d40 0x7fedbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.251+0000 7fedd6800640 1 -- 172.21.15.5:0/3545619023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc003520 con 0x7fedbc0039b0 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.252+0000 7fedd6800640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedbc003dd0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009a90 tx=0x7fedbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274f13ca53fcc202 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.252+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8036070 con 0x7fedbc0039b0 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.252+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8031400 con 0x7fedbc0039b0 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.252+0000 7fedd5400640 1 -- 172.21.15.5:0/3545619023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8009690 con 0x7fedbc0039b0 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7feddca00640 1 -- 172.21.15.5:0/3545619023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 msgr2=0x7fedbc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7feddca00640 1 --2- 172.21.15.5:0/3545619023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedbc003dd0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009a90 tx=0x7fedbc005830 comp rx=0 tx=0).stop 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7fedd7200640 1 -- 172.21.15.5:0/3545619023 reap_dead start 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7feddca00640 1 -- 172.21.15.5:0/3545619023 shutdown_connections 2024-04-25T10:11:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7feddca00640 1 -- 172.21.15.5:0/3545619023 >> 172.21.15.5:0/3545619023 conn(0x7fedd808af80 msgr2=0x7fedd80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7feddca00640 1 -- 172.21.15.5:0/3545619023 shutdown_connections 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.253+0000 7feddca00640 1 -- 172.21.15.5:0/3545619023 wait complete. 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.254+0000 7feddca00640 1 Processor -- start 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.255+0000 7feddca00640 1 -- start start 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.255+0000 7feddca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedd80ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7feddca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd80b89f0 0x7fedd8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7feddca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd80d1930 0x7fedd80d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7feddca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd81603a0 con 0x7fedbc0039b0 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7feddca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80b4ca0 con 0x7fedd80b89f0 2024-04-25T10:11:01.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7feddca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd80b2b70 con 0x7fedd80d1930 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedd80ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedd80ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41206/0 (socket says 172.21.15.5:41206) 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 -- 172.21.15.5:0/2598264842 learned_addr learned my addr 172.21.15.5:0/2598264842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 -- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd80d1930 msgr2=0x7fedd80d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd80d1930 0x7fedd80d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd5e00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd80b89f0 0x7fedd8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 -- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd80b89f0 msgr2=0x7fedd8154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd80b89f0 0x7fedd8154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd6800640 1 -- 172.21.15.5:0/2598264842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd80cacb0 con 0x7fedbc0039b0 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.256+0000 7fedd5e00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd80b89f0 0x7fedd8154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.257+0000 7fedd6800640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedd80ca6d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fedc8034040 tx=0x7fedc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.257+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8040020 con 0x7fedbc0039b0 2024-04-25T10:11:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.257+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc8003b40 con 0x7fedbc0039b0 2024-04-25T10:11:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.257+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8036040 con 0x7fedbc0039b0 2024-04-25T10:11:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.257+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd80caf90 con 0x7fedbc0039b0 2024-04-25T10:11:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.257+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd80cb550 con 0x7fedbc0039b0 2024-04-25T10:11:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.258+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd80d22f0 con 0x7fedbc0039b0 2024-04-25T10:11:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.259+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedc8043020 con 0x7fedbc0039b0 2024-04-25T10:11:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.260+0000 7fedcea00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedb406b500 0x7fedb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.261+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fedc80aa5c0 con 0x7fedbc0039b0 2024-04-25T10:11:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.261+0000 7fedd5e00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedb406b500 0x7fedb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:01.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.262+0000 7fedd5e00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedb406b500 0x7fedb406d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fedd80cd450 tx=0x7fedc0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:01.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.265+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedc804a070 con 0x7fedbc0039b0 2024-04-25T10:11:01.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:01.503+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fedd8068f60 con 0x7fedbc0039b0 2024-04-25T10:11:02.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.159+0000 7fedcea00640 1 -- 172.21.15.5:0/2598264842 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) v1 ==== 118+0+0 (secure 0 0 0) 0x7fedc80044e0 con 0x7fedbc0039b0 2024-04-25T10:11:02.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.163+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedb406b500 msgr2=0x7fedb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.163+0000 7feddca00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedb406b500 0x7fedb406d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fedd80cd450 tx=0x7fedc0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.163+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 msgr2=0x7fedd80ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.163+0000 7feddca00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedd80ca6d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fedc8034040 tx=0x7fedc8004730 comp rx=0 tx=0).stop 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 shutdown_connections 2024-04-25T10:11:02.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd80d1930 0x7fedd80d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fedb406b500 0x7fedb406d9c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd80b89f0 0x7fedd8154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 --2- 172.21.15.5:0/2598264842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedbc0039b0 0x7fedd80ca6d0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 >> 172.21.15.5:0/2598264842 conn(0x7fedd808af80 msgr2=0x7fedd806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 shutdown_connections 2024-04-25T10:11:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.164+0000 7feddca00640 1 -- 172.21.15.5:0/2598264842 wait complete. 2024-04-25T10:11:02.194 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:02.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.407+0000 7fd2f5600640 1 Processor -- start 2024-04-25T10:11:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.408+0000 7fd2f5600640 1 -- start start 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.409+0000 7fd2f5600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f006e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.409+0000 7fd2f5600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2f006e670 0x7fd2f015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2f5600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f0072010 con 0x7fd2f015b330 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2f5600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f006ead0 con 0x7fd2f006e670 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2f5600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00ce570 con 0x7fd2f00cc840 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2ef400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2f006e670 0x7fd2f015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2f4c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd2f015b330 0x7fd2f015e060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44692/0 (socket says 172.21.15.5:44692) 2024-04-25T10:11:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2f4c00640 1 -- 172.21.15.5:0/3818381069 learned_addr learned my addr 172.21.15.5:0/3818381069 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.410+0000 7fd2efe00640 1 --2- 172.21.15.5:0/3818381069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.411+0000 7fd2ef400640 1 -- 172.21.15.5:0/3818381069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 msgr2=0x7fd2f006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.411+0000 7fd2ef400640 1 --2- 172.21.15.5:0/3818381069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f006e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.411+0000 7fd2ef400640 1 -- 172.21.15.5:0/3818381069 >> v1:172.21.15.5:6789/0 conn(0x7fd2f015b330 legacy=0x7fd2f015e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.411+0000 7fd2ef400640 1 -- 172.21.15.5:0/3818381069 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00ce850 con 0x7fd2f006e670 2024-04-25T10:11:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.411+0000 7fd2eea00640 1 -- 172.21.15.5:0/3818381069 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725089604 0 0) 0x7fd2f0072010 con 0x7fd2f015b330 2024-04-25T10:11:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.412+0000 7fd2ef400640 1 --2- 172.21.15.5:0/3818381069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f015adf0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009d90 tx=0x7fd2dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acbebc53899abbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.412+0000 7fd2eea00640 1 -- 172.21.15.5:0/3818381069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc007740 con 0x7fd2f006e670 2024-04-25T10:11:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.412+0000 7fd2eea00640 1 -- 172.21.15.5:0/3818381069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc0078e0 con 0x7fd2f006e670 2024-04-25T10:11:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.412+0000 7fd2eea00640 1 -- 172.21.15.5:0/3818381069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc007cb0 con 0x7fd2f006e670 2024-04-25T10:11:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 -- 172.21.15.5:0/3818381069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 msgr2=0x7fd2f015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 --2- 172.21.15.5:0/3818381069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f015adf0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009d90 tx=0x7fd2dc02a660 comp rx=0 tx=0).stop 2024-04-25T10:11:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 -- 172.21.15.5:0/3818381069 shutdown_connections 2024-04-25T10:11:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 --2- 172.21.15.5:0/3818381069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f015adf0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 --2- 172.21.15.5:0/3818381069 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f006e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 -- 172.21.15.5:0/3818381069 >> 172.21.15.5:0/3818381069 conn(0x7fd2f0066f20 msgr2=0x7fd2f006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 -- 172.21.15.5:0/3818381069 shutdown_connections 2024-04-25T10:11:02.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.413+0000 7fd2f5600640 1 -- 172.21.15.5:0/3818381069 wait complete. 2024-04-25T10:11:02.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.415+0000 7fd2f5600640 1 Processor -- start 2024-04-25T10:11:02.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.415+0000 7fd2f5600640 1 -- start start 2024-04-25T10:11:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.416+0000 7fd2f5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f0153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.416+0000 7fd2f5600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.416+0000 7fd2f5600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2f015b330 0x7fd2f0159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2f5600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00b2d20 con 0x7fd2f006e670 2024-04-25T10:11:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2f5600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00cee10 con 0x7fd2f015b330 2024-04-25T10:11:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2f5600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00ce1d0 con 0x7fd2f00cc840 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f0153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f0153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41222/0 (socket says 172.21.15.5:41222) 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 -- 172.21.15.5:0/2837796742 learned_addr learned my addr 172.21.15.5:0/2837796742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 -- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 msgr2=0x7fd2f00ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 -- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2f015b330 msgr2=0x7fd2f0159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2f015b330 0x7fd2f0159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.417+0000 7fd2efe00640 1 -- 172.21.15.5:0/2837796742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00ad320 con 0x7fd2f006e670 2024-04-25T10:11:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.418+0000 7fd2efe00640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f0153f90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000d830 tx=0x7fd2e000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.418+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000b6c0 con 0x7fd2f006e670 2024-04-25T10:11:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.418+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e0019070 con 0x7fd2f006e670 2024-04-25T10:11:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.418+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e000ba80 con 0x7fd2f006e670 2024-04-25T10:11:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.418+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f00ad600 con 0x7fd2f006e670 2024-04-25T10:11:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.418+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f01544d0 con 0x7fd2f006e670 2024-04-25T10:11:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.419+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f015bba0 con 0x7fd2f006e670 2024-04-25T10:11:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.420+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2e001e460 con 0x7fd2f006e670 2024-04-25T10:11:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.421+0000 7fd2e7e00640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2d006b4d0 0x7fd2d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.421+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2e008ab40 con 0x7fd2f006e670 2024-04-25T10:11:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.422+0000 7fd2ef400640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2d006b4d0 0x7fd2d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.423+0000 7fd2ef400640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2d006b4d0 0x7fd2d006d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc034000 tx=0x7fd2dc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.426+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2e0059790 con 0x7fd2f006e670 2024-04-25T10:11:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:02.667+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd2f0068f60 con 0x7fd2f006e670 2024-04-25T10:11:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.170+0000 7fd2e7e00640 1 -- 172.21.15.5:0/2837796742 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd2e0059090 con 0x7fd2f006e670 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2d006b4d0 msgr2=0x7fd2d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2d006b4d0 0x7fd2d006d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc034000 tx=0x7fd2dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 msgr2=0x7fd2f0153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f0153f90 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd2e000d830 tx=0x7fd2e000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 shutdown_connections 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd2d006b4d0 0x7fd2d006d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2f015b330 0x7fd2f0159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd2f00cc840 0x7fd2f00ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 --2- 172.21.15.5:0/2837796742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2f006e670 0x7fd2f0153f90 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 >> 172.21.15.5:0/2837796742 conn(0x7fd2f0066f20 msgr2=0x7fd2f006df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 shutdown_connections 2024-04-25T10:11:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.174+0000 7fd2f5600640 1 -- 172.21.15.5:0/2837796742 wait complete. 2024-04-25T10:11:03.201 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:03.201 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.412+0000 7f1b35e00640 1 Processor -- start 2024-04-25T10:11:03.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.412+0000 7f1b35e00640 1 -- start start 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.413+0000 7f1b35e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b30143d30 0x7f1b3015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.414+0000 7f1b35e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b300b4880 con 0x7f1b300ceea0 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.414+0000 7f1b35e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b300b45a0 con 0x7f1b300b3ae0 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.414+0000 7f1b35e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3015e2c0 con 0x7f1b30143d30 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.414+0000 7f1b34a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1b30143d30 0x7f1b3015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.414+0000 7f1b35400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1b300ceea0 0x7f1b3015abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44704/0 (socket says 172.21.15.5:44704) 2024-04-25T10:11:03.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.414+0000 7f1b35400640 1 -- 172.21.15.5:0/3758655197 learned_addr learned my addr 172.21.15.5:0/3758655197 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.415+0000 7f1b34a00640 1 -- 172.21.15.5:0/3758655197 >> v1:172.21.15.80:6789/0 conn(0x7f1b300b3ae0 legacy=0x7f1b300ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.415+0000 7f1b34a00640 1 -- 172.21.15.5:0/3758655197 >> v1:172.21.15.5:6789/0 conn(0x7f1b300ceea0 legacy=0x7f1b3015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.415+0000 7f1b34a00640 1 -- 172.21.15.5:0/3758655197 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3015e5a0 con 0x7f1b30143d30 2024-04-25T10:11:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.416+0000 7f1b34a00640 1 --2- 172.21.15.5:0/3758655197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b3015eda0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1b200097a0 tx=0x7f1b2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed66df8c443ad44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.416+0000 7f1b2f400640 1 -- 172.21.15.5:0/3758655197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20037070 con 0x7f1b30143d30 2024-04-25T10:11:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.416+0000 7f1b2f400640 1 -- 172.21.15.5:0/3758655197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b20032440 con 0x7f1b30143d30 2024-04-25T10:11:03.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.416+0000 7f1b2f400640 1 -- 172.21.15.5:0/3758655197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20032810 con 0x7f1b30143d30 2024-04-25T10:11:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 -- 172.21.15.5:0/3758655197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 msgr2=0x7f1b3015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 --2- 172.21.15.5:0/3758655197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b3015eda0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f1b200097a0 tx=0x7f1b2002a230 comp rx=0 tx=0).stop 2024-04-25T10:11:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 -- 172.21.15.5:0/3758655197 shutdown_connections 2024-04-25T10:11:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 --2- 172.21.15.5:0/3758655197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b3015eda0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 -- 172.21.15.5:0/3758655197 >> 172.21.15.5:0/3758655197 conn(0x7f1b30066f20 msgr2=0x7f1b3006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 -- 172.21.15.5:0/3758655197 shutdown_connections 2024-04-25T10:11:03.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.417+0000 7f1b35e00640 1 -- 172.21.15.5:0/3758655197 wait complete. 2024-04-25T10:11:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.419+0000 7f1b35e00640 1 Processor -- start 2024-04-25T10:11:03.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.419+0000 7f1b35e00640 1 -- start start 2024-04-25T10:11:03.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b300b3ae0 0x7f1b301ed670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b300ceea0 0x7f1b300ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b300ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30144290 con 0x7f1b30143d30 2024-04-25T10:11:03.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30144110 con 0x7f1b300b3ae0 2024-04-25T10:11:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3006e430 con 0x7f1b300ceea0 2024-04-25T10:11:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b300ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b300ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41248/0 (socket says 172.21.15.5:41248) 2024-04-25T10:11:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.420+0000 7f1b35400640 1 -- 172.21.15.5:0/1798651870 learned_addr learned my addr 172.21.15.5:0/1798651870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:03.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b34a00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b300b3ae0 0x7f1b301ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b2fe00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b300ceea0 0x7f1b300ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b35400640 1 -- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b300ceea0 msgr2=0x7f1b300ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b35400640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b300ceea0 0x7f1b300ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b35400640 1 -- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b300b3ae0 msgr2=0x7f1b301ed670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b35400640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b300b3ae0 0x7f1b301ed670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b35400640 1 -- 172.21.15.5:0/1798651870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3015b9d0 con 0x7f1b30143d30 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.421+0000 7f1b35400640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b300ad9c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800cbc0 tx=0x7f1b2800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.422+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28017070 con 0x7f1b30143d30 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.422+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800ad40 con 0x7f1b30143d30 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.422+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3015bcb0 con 0x7f1b30143d30 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.422+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28012600 con 0x7f1b30143d30 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.422+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3015c270 con 0x7f1b30143d30 2024-04-25T10:11:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.423+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b300b36c0 con 0x7f1b30143d30 2024-04-25T10:11:03.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.424+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b2801a070 con 0x7f1b30143d30 2024-04-25T10:11:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.425+0000 7f1b2cc00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1b1006b4d0 0x7f1b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.425+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b2808a080 con 0x7f1b30143d30 2024-04-25T10:11:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.426+0000 7f1b34a00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1b1006b4d0 0x7f1b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.427+0000 7f1b34a00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1b1006b4d0 0x7f1b1006d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1b20034000 tx=0x7f1b2002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.429+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b300b36c0 con 0x7f1b30143d30 2024-04-25T10:11:03.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.667+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b30068f60 con 0x7f1b30143d30 2024-04-25T10:11:03.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.668+0000 7f1b2cc00640 1 -- 172.21.15.5:0/1798651870 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f1b28058690 con 0x7f1b30143d30 2024-04-25T10:11:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:03.673 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:02.669462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6715":{"gid":6715,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2001088034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2001088034},{"type":"v1","addr":"172.21.15.5:6835","nonce":2001088034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:03.673 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1b1006b4d0 msgr2=0x7f1b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1b1006b4d0 0x7f1b1006d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1b20034000 tx=0x7f1b2002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 msgr2=0x7f1b300ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b300ad9c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800cbc0 tx=0x7f1b2800a430 comp rx=0 tx=0).stop 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 shutdown_connections 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b30143d30 0x7f1b300ad9c0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1b300ceea0 0x7f1b300ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1b1006b4d0 0x7f1b1006d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 --2- 172.21.15.5:0/1798651870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b300b3ae0 0x7f1b301ed670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.670+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 >> 172.21.15.5:0/1798651870 conn(0x7f1b30066f20 msgr2=0x7f1b3006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.671+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 shutdown_connections 2024-04-25T10:11:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.671+0000 7f1b35e00640 1 -- 172.21.15.5:0/1798651870 wait complete. 2024-04-25T10:11:03.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.882+0000 7fccf9e00640 1 Processor -- start 2024-04-25T10:11:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.882+0000 7fccf9e00640 1 -- start start 2024-04-25T10:11:03.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.884+0000 7fccf9e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf4154a10 0x7fccf415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.884+0000 7fccf9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fccf40cdca0 0x7fccf40b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.884+0000 7fccf9e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf40b4890 con 0x7fccf4070f90 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.884+0000 7fccf9e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf40cd200 con 0x7fccf40cdca0 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.884+0000 7fccf9e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf406e6f0 con 0x7fccf4154a10 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf9400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fccf4070f90 0x7fccf406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44716/0 (socket says 172.21.15.5:44716) 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf9400640 1 -- 172.21.15.5:0/2568468790 learned_addr learned my addr 172.21.15.5:0/2568468790 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf3e00640 1 --2- 172.21.15.5:0/2568468790 >> v2:172.21.15.5:3300/0 conn(0x7fccf40cdca0 0x7fccf40b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf8a00640 1 --2- 172.21.15.5:0/2568468790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf4154a10 0x7fccf415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf3400640 1 -- 172.21.15.5:0/2568468790 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941330810 0 0) 0x7fccf40b4890 con 0x7fccf4070f90 2024-04-25T10:11:03.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf3e00640 1 -- 172.21.15.5:0/2568468790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf4154a10 msgr2=0x7fccf415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf3e00640 1 --2- 172.21.15.5:0/2568468790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf4154a10 0x7fccf415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf3e00640 1 -- 172.21.15.5:0/2568468790 >> v1:172.21.15.5:6789/0 conn(0x7fccf4070f90 legacy=0x7fccf406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.885+0000 7fccf3e00640 1 -- 172.21.15.5:0/2568468790 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf406e9d0 con 0x7fccf40cdca0 2024-04-25T10:11:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.886+0000 7fccf3e00640 1 --2- 172.21.15.5:0/2568468790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf40cdca0 0x7fccf40b4350 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fcce00097a0 tx=0x7fcce002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07b578b81776939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.886+0000 7fccf3400640 1 -- 172.21.15.5:0/2568468790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0037070 con 0x7fccf40cdca0 2024-04-25T10:11:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.886+0000 7fccf3400640 1 -- 172.21.15.5:0/2568468790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce002ae50 con 0x7fccf40cdca0 2024-04-25T10:11:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.886+0000 7fccf3400640 1 -- 172.21.15.5:0/2568468790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0032660 con 0x7fccf40cdca0 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 -- 172.21.15.5:0/2568468790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf40cdca0 msgr2=0x7fccf40b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 --2- 172.21.15.5:0/2568468790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf40cdca0 0x7fccf40b4350 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fcce00097a0 tx=0x7fcce002a100 comp rx=0 tx=0).stop 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 -- 172.21.15.5:0/2568468790 shutdown_connections 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 --2- 172.21.15.5:0/2568468790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf40cdca0 0x7fccf40b4350 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 --2- 172.21.15.5:0/2568468790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf4154a10 0x7fccf415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 -- 172.21.15.5:0/2568468790 >> 172.21.15.5:0/2568468790 conn(0x7fccf4066f20 msgr2=0x7fccf406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 -- 172.21.15.5:0/2568468790 shutdown_connections 2024-04-25T10:11:03.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.887+0000 7fccf9e00640 1 -- 172.21.15.5:0/2568468790 wait complete. 2024-04-25T10:11:03.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.889+0000 7fccf9e00640 1 Processor -- start 2024-04-25T10:11:03.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.889+0000 7fccf9e00640 1 -- start start 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccf4070f90 0x7fccf4147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf40cdca0 0x7fccf40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 0x7fccf40addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40cd780 con 0x7fccf4154a10 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40cd4a0 con 0x7fccf4070f90 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf406d700 con 0x7fccf40cdca0 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 0x7fccf40addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 0x7fccf40addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41266/0 (socket says 172.21.15.5:41266) 2024-04-25T10:11:03.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.890+0000 7fccf9400640 1 -- 172.21.15.5:0/749517265 learned_addr learned my addr 172.21.15.5:0/749517265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf8a00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccf4070f90 0x7fccf4147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf3e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf40cdca0 0x7fccf40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9400640 1 -- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf40cdca0 msgr2=0x7fccf40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9400640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf40cdca0 0x7fccf40ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9400640 1 -- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccf4070f90 msgr2=0x7fccf4147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9400640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccf4070f90 0x7fccf4147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9400640 1 -- 172.21.15.5:0/749517265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf41f4360 con 0x7fccf4154a10 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9400640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 0x7fccf40addf0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c7a0 tx=0x7fccec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec018070 con 0x7fccf4154a10 2024-04-25T10:11:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec00a640 con 0x7fccf4154a10 2024-04-25T10:11:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec00aab0 con 0x7fccf4154a10 2024-04-25T10:11:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf4156780 con 0x7fccf4154a10 2024-04-25T10:11:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.891+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf4156d40 con 0x7fccf4154a10 2024-04-25T10:11:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.893+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf4068f60 con 0x7fccf4154a10 2024-04-25T10:11:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.894+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccec00ac50 con 0x7fccf4154a10 2024-04-25T10:11:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.894+0000 7fccf0c00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fccd406b4d0 0x7fccd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.895+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccec089990 con 0x7fccf4154a10 2024-04-25T10:11:03.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.895+0000 7fccf8a00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fccd406b4d0 0x7fccd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.896+0000 7fccf8a00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fccd406b4d0 0x7fccd406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcce8006fd0 tx=0x7fcce8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:03.899+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccec058710 con 0x7fccf4154a10 2024-04-25T10:11:04.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.089+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccf40af380 con 0x7fccf4154a10 2024-04-25T10:11:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.091+0000 7fccf0c00640 1 -- 172.21.15.5:0/749517265 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16792 (secure 0 0 0) 0x7fccec058010 con 0x7fccf4154a10 2024-04-25T10:11:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:04.097 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:11:01.130972+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fccd406b4d0 msgr2=0x7fccd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fccd406b4d0 0x7fccd406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcce8006fd0 tx=0x7fcce8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 msgr2=0x7fccf40addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 0x7fccf40addf0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c7a0 tx=0x7fccec00cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 shutdown_connections 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccf4154a10 0x7fccf40addf0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fccf40cdca0 0x7fccf40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fccd406b4d0 0x7fccd406d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 --2- 172.21.15.5:0/749517265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fccf4070f90 0x7fccf4147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 >> 172.21.15.5:0/749517265 conn(0x7fccf4066f20 msgr2=0x7fccf40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.094+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 shutdown_connections 2024-04-25T10:11:04.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.095+0000 7fccf9e00640 1 -- 172.21.15.5:0/749517265 wait complete. 2024-04-25T10:11:04.118 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:04.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.311+0000 7fbab3e00640 1 Processor -- start 2024-04-25T10:11:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.311+0000 7fbab3e00640 1 -- start start 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 0x7fbab415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbab40cdca0 0x7fbab40b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab40b4890 con 0x7fbab4070f90 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbab40cd200 con 0x7fbab40cdca0 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab406e6f0 con 0x7fbab4154a10 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbab4070f90 0x7fbab406cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44718/0 (socket says 172.21.15.5:44718) 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.313+0000 7fbab3400640 1 -- 172.21.15.5:0/493708931 learned_addr learned my addr 172.21.15.5:0/493708931 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab2000640 1 --2- 172.21.15.5:0/493708931 >> v2:172.21.15.80:3300/0 conn(0x7fbab40cdca0 0x7fbab40b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab2a00640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 0x7fbab415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575548105 0 0) 0x7fbab40b4890 con 0x7fbab4070f90 2024-04-25T10:11:04.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba98003520 con 0x7fbab4070f90 2024-04-25T10:11:04.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361012323 0 0) 0x7fba98003520 con 0x7fbab4070f90 2024-04-25T10:11:04.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 msgr2=0x7fbab415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 0x7fbab415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 >> v2:172.21.15.80:3300/0 conn(0x7fbab40cdca0 msgr2=0x7fbab40b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 --2- 172.21.15.5:0/493708931 >> v2:172.21.15.80:3300/0 conn(0x7fbab40cdca0 0x7fbab40b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab406e9d0 con 0x7fbab4070f90 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fbaac001670 con 0x7fbab4070f90 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 >> v1:172.21.15.5:6789/0 conn(0x7fbab4070f90 legacy=0x7fbab406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba980039b0 0x7fba98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fba98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 0x7fba980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40cd200 con 0x7fba98004800 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab406e6f0 con 0x7fba980039b0 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.314+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9800ba10 con 0x7fbab4154a10 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2000640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fba98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2a00640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba980039b0 0x7fba98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2000640 1 -- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 msgr2=0x7fba980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2000640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab4154a10 0x7fba980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2000640 1 -- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba980039b0 msgr2=0x7fba98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2000640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba980039b0 0x7fba98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2000640 1 -- 172.21.15.5:0/493708931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba98003520 con 0x7fba98004800 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.315+0000 7fbab2a00640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba980039b0 0x7fba98003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.316+0000 7fbab2000640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fba98004c80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009ac0 tx=0x7fba98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db06fb0fe288e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.316+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x7fba98004800 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.316+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c031400 con 0x7fba98004800 2024-04-25T10:11:04.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.316+0000 7fbab1600640 1 -- 172.21.15.5:0/493708931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c009690 con 0x7fba98004800 2024-04-25T10:11:04.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3e00640 1 -- 172.21.15.5:0/493708931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 msgr2=0x7fba98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3e00640 1 --2- 172.21.15.5:0/493708931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fba98004c80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009ac0 tx=0x7fba98005830 comp rx=0 tx=0).stop 2024-04-25T10:11:04.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3400640 1 -- 172.21.15.5:0/493708931 reap_dead start 2024-04-25T10:11:04.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3e00640 1 -- 172.21.15.5:0/493708931 shutdown_connections 2024-04-25T10:11:04.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3e00640 1 -- 172.21.15.5:0/493708931 >> 172.21.15.5:0/493708931 conn(0x7fbab4066f20 msgr2=0x7fbab406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3e00640 1 -- 172.21.15.5:0/493708931 shutdown_connections 2024-04-25T10:11:04.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.317+0000 7fbab3e00640 1 -- 172.21.15.5:0/493708931 wait complete. 2024-04-25T10:11:04.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.319+0000 7fbab3e00640 1 Processor -- start 2024-04-25T10:11:04.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.319+0000 7fbab3e00640 1 -- start start 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fbab413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab40b1be0 0x7fbab40afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbab413d610 0x7fbab413da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40cd780 con 0x7fba98004800 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40cd4a0 con 0x7fbab413d610 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab406d700 con 0x7fbab40b1be0 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fbab413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fbab413ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41282/0 (socket says 172.21.15.5:41282) 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab2a00640 1 -- 172.21.15.5:0/4177577543 learned_addr learned my addr 172.21.15.5:0/4177577543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:04.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab3400640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbab413d610 0x7fbab413da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab2a00640 1 -- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab40b1be0 msgr2=0x7fbab40afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.320+0000 7fbab2000640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab40b1be0 0x7fbab40afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab2a00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab40b1be0 0x7fbab40afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab2a00640 1 -- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbab413d610 msgr2=0x7fbab413da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab2a00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbab413d610 0x7fbab413da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab2a00640 1 -- 172.21.15.5:0/4177577543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40b0470 con 0x7fba98004800 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab3400640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbab413d610 0x7fbab413da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab2a00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fbab413ccc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400f380 tx=0x7fbaa400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4003b90 con 0x7fba98004800 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa4003d30 con 0x7fba98004800 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4004220 con 0x7fba98004800 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab40b0750 con 0x7fba98004800 2024-04-25T10:11:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.321+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab40b0d10 con 0x7fba98004800 2024-04-25T10:11:04.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.322+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab4068f60 con 0x7fba98004800 2024-04-25T10:11:04.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.323+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaa4005d10 con 0x7fba98004800 2024-04-25T10:11:04.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.325+0000 7fbaab400640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fba9006b4d0 0x7fba9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.325+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbaa408a9e0 con 0x7fba98004800 2024-04-25T10:11:04.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.325+0000 7fbab2000640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fba9006b4d0 0x7fba9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.327+0000 7fbab2000640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fba9006b4d0 0x7fba9006d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fba9c033000 tx=0x7fba9c008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.329+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaa4012070 con 0x7fba98004800 2024-04-25T10:11:04.507 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:11:04.502+0000 7f1a28000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.522+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbab406adb0 con 0x7fba9006b4d0 2024-04-25T10:11:04.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.543+0000 7fbaab400640 1 -- 172.21.15.5:0/4177577543 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251270 (secure 0 0 0) 0x7fbab406adb0 con 0x7fba9006b4d0 2024-04-25T10:11:04.552 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:04.553 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:11:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fba9006b4d0 msgr2=0x7fba9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fba9006b4d0 0x7fba9006d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fba9c033000 tx=0x7fba9c008030 comp rx=0 tx=0).stop 2024-04-25T10:11:04.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 msgr2=0x7fbab413ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fbab413ccc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400f380 tx=0x7fbaa400f840 comp rx=0 tx=0).stop 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 shutdown_connections 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fba9006b4d0 0x7fba9006d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbab413d610 0x7fbab413da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbab40b1be0 0x7fbab40afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 --2- 172.21.15.5:0/4177577543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba98004800 0x7fbab413ccc0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 >> 172.21.15.5:0/4177577543 conn(0x7fbab4066f20 msgr2=0x7fbab415b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 shutdown_connections 2024-04-25T10:11:04.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.548+0000 7fbab3e00640 1 -- 172.21.15.5:0/4177577543 wait complete. 2024-04-25T10:11:04.572 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-04-25T10:11:02.816681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":197,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260660,"kb_used_data":11204,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714060,"statfs":{"total":773094113280,"available":772827197440,"internally_reserved":0,"allocated":11472896,"data_stored":5923851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.726218"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.126124+0000","last_change":"2024-04-25T10:10:59.126124+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.126124+0000","last_undegraded":"2024-04-25T10:10:59.126124+0000","last_fullsized":"2024-04-25T10:10:59.126124+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.152216+0000","last_change":"2024-04-25T10:11:00.150426+0000","last_active":"2024-04-25T10:11:00.152216+0000","last_peered":"2024-04-25T10:11:00.152216+0000","last_clean":"2024-04-25T10:11:00.152216+0000","last_became_active":"2024-04-25T10:11:00.150107+0000","last_became_peered":"2024-04-25T10:11:00.150107+0000","last_unstale":"2024-04-25T10:11:00.152216+0000","last_undegraded":"2024-04-25T10:11:00.152216+0000","last_fullsized":"2024-04-25T10:11:00.152216+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:22.263161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.158486+0000","last_change":"2024-04-25T10:11:00.154826+0000","last_active":"2024-04-25T10:11:00.158486+0000","last_peered":"2024-04-25T10:11:00.158486+0000","last_clean":"2024-04-25T10:11:00.158486+0000","last_became_active":"2024-04-25T10:11:00.154532+0000","last_became_peered":"2024-04-25T10:11:00.154532+0000","last_unstale":"2024-04-25T10:11:00.158486+0000","last_undegraded":"2024-04-25T10:11:00.158486+0000","last_fullsized":"2024-04-25T10:11:00.158486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.443869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.136192+0000","last_change":"2024-04-25T10:10:59.136192+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.136192+0000","last_undegraded":"2024-04-25T10:10:59.136192+0000","last_fullsized":"2024-04-25T10:10:59.136192+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.129662+0000","last_change":"2024-04-25T10:10:59.129662+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.129662+0000","last_undegraded":"2024-04-25T10:10:59.129662+0000","last_fullsized":"2024-04-25T10:10:59.129662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157728+0000","last_change":"2024-04-25T10:11:00.140959+0000","last_active":"2024-04-25T10:11:00.157728+0000","last_peered":"2024-04-25T10:11:00.157728+0000","last_clean":"2024-04-25T10:11:00.157728+0000","last_became_active":"2024-04-25T10:11:00.140581+0000","last_became_peered":"2024-04-25T10:11:00.140581+0000","last_unstale":"2024-04-25T10:11:00.157728+0000","last_undegraded":"2024-04-25T10:11:00.157728+0000","last_fullsized":"2024-04-25T10:11:00.157728+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:19.322670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157417+0000","last_change":"2024-04-25T10:11:00.145532+0000","last_active":"2024-04-25T10:11:00.157417+0000","last_peered":"2024-04-25T10:11:00.157417+0000","last_clean":"2024-04-25T10:11:00.157417+0000","last_became_active":"2024-04-25T10:11:00.144637+0000","last_became_peered":"2024-04-25T10:11:00.144637+0000","last_unstale":"2024-04-25T10:11:00.157417+0000","last_undegraded":"2024-04-25T10:11:00.157417+0000","last_fullsized":"2024-04-25T10:11:00.157417+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:52.819431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.163794+0000","last_change":"2024-04-25T10:11:00.157377+0000","last_active":"2024-04-25T10:11:00.163794+0000","last_peered":"2024-04-25T10:11:00.163794+0000","last_clean":"2024-04-25T10:11:00.163794+0000","last_became_active":"2024-04-25T10:11:00.156516+0000","last_became_peered":"2024-04-25T10:11:00.156516+0000","last_unstale":"2024-04-25T10:11:00.163794+0000","last_undegraded":"2024-04-25T10:11:00.163794+0000","last_fullsized":"2024-04-25T10:11:00.163794+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:59.451682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.143677+0000","last_change":"2024-04-25T10:11:00.138094+0000","last_active":"2024-04-25T10:11:00.143677+0000","last_peered":"2024-04-25T10:11:00.143677+0000","last_clean":"2024-04-25T10:11:00.143677+0000","last_became_active":"2024-04-25T10:11:00.137842+0000","last_became_peered":"2024-04-25T10:11:00.137842+0000","last_unstale":"2024-04-25T10:11:00.143677+0000","last_undegraded":"2024-04-25T10:11:00.143677+0000","last_fullsized":"2024-04-25T10:11:00.143677+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.163778+0000","last_change":"2024-04-25T10:11:00.157715+0000","last_active":"2024-04-25T10:11:00.163778+0000","last_peered":"2024-04-25T10:11:00.163778+0000","last_clean":"2024-04-25T10:11:00.163778+0000","last_became_active":"2024-04-25T10:11:00.157424+0000","last_became_peered":"2024-04-25T10:11:00.157424+0000","last_unstale":"2024-04-25T10:11:00.163778+0000","last_undegraded":"2024-04-25T10:11:00.163778+0000","last_fullsized":"2024-04-25T10:11:00.163778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:39.120461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.138090+0000","last_change":"2024-04-25T10:11:00.136356+0000","last_active":"2024-04-25T10:11:00.138090+0000","last_peered":"2024-04-25T10:11:00.138090+0000","last_clean":"2024-04-25T10:11:00.138090+0000","last_became_active":"2024-04-25T10:11:00.136031+0000","last_became_peered":"2024-04-25T10:11:00.136031+0000","last_unstale":"2024-04-25T10:11:00.138090+0000","last_undegraded":"2024-04-25T10:11:00.138090+0000","last_fullsized":"2024-04-25T10:11:00.138090+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:55.089613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157914+0000","last_change":"2024-04-25T10:11:00.150439+0000","last_active":"2024-04-25T10:11:00.157914+0000","last_peered":"2024-04-25T10:11:00.157914+0000","last_clean":"2024-04-25T10:11:00.157914+0000","last_became_active":"2024-04-25T10:11:00.150112+0000","last_became_peered":"2024-04-25T10:11:00.150112+0000","last_unstale":"2024-04-25T10:11:00.157914+0000","last_undegraded":"2024-04-25T10:11:00.157914+0000","last_fullsized":"2024-04-25T10:11:00.157914+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:40.806266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.130356+0000","last_change":"2024-04-25T10:10:59.130356+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.130356+0000","last_undegraded":"2024-04-25T10:10:59.130356+0000","last_fullsized":"2024-04-25T10:10:59.130356+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157765+0000","last_change":"2024-04-25T10:11:00.153014+0000","last_active":"2024-04-25T10:11:00.157765+0000","last_peered":"2024-04-25T10:11:00.157765+0000","last_clean":"2024-04-25T10:11:00.157765+0000","last_became_active":"2024-04-25T10:11:00.152679+0000","last_became_peered":"2024-04-25T10:11:00.152679+0000","last_unstale":"2024-04-25T10:11:00.157765+0000","last_undegraded":"2024-04-25T10:11:00.157765+0000","last_fullsized":"2024-04-25T10:11:00.157765+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.795378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.156741+0000","last_change":"2024-04-25T10:11:00.155617+0000","last_active":"2024-04-25T10:11:00.156741+0000","last_peered":"2024-04-25T10:11:00.156741+0000","last_clean":"2024-04-25T10:11:00.156741+0000","last_became_active":"2024-04-25T10:11:00.155328+0000","last_became_peered":"2024-04-25T10:11:00.155328+0000","last_unstale":"2024-04-25T10:11:00.156741+0000","last_undegraded":"2024-04-25T10:11:00.156741+0000","last_fullsized":"2024-04-25T10:11:00.156741+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:51.933926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157859+0000","last_change":"2024-04-25T10:11:00.151161+0000","last_active":"2024-04-25T10:11:00.157859+0000","last_peered":"2024-04-25T10:11:00.157859+0000","last_clean":"2024-04-25T10:11:00.157859+0000","last_became_active":"2024-04-25T10:11:00.150875+0000","last_became_peered":"2024-04-25T10:11:00.150875+0000","last_unstale":"2024-04-25T10:11:00.157859+0000","last_undegraded":"2024-04-25T10:11:00.157859+0000","last_fullsized":"2024-04-25T10:11:00.157859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:35.147741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.164646+0000","last_change":"2024-04-25T10:11:00.160614+0000","last_active":"2024-04-25T10:11:00.164646+0000","last_peered":"2024-04-25T10:11:00.164646+0000","last_clean":"2024-04-25T10:11:00.164646+0000","last_became_active":"2024-04-25T10:11:00.160239+0000","last_became_peered":"2024-04-25T10:11:00.160239+0000","last_unstale":"2024-04-25T10:11:00.164646+0000","last_undegraded":"2024-04-25T10:11:00.164646+0000","last_fullsized":"2024-04-25T10:11:00.164646+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:15.501374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.137842+0000","last_change":"2024-04-25T10:11:00.133296+0000","last_active":"2024-04-25T10:11:00.137842+0000","last_peered":"2024-04-25T10:11:00.137842+0000","last_clean":"2024-04-25T10:11:00.137842+0000","last_became_active":"2024-04-25T10:11:00.132946+0000","last_became_peered":"2024-04-25T10:11:00.132946+0000","last_unstale":"2024-04-25T10:11:00.137842+0000","last_undegraded":"2024-04-25T10:11:00.137842+0000","last_fullsized":"2024-04-25T10:11:00.137842+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:09.127558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.129192+0000","last_change":"2024-04-25T10:10:59.129192+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.129192+0000","last_undegraded":"2024-04-25T10:10:59.129192+0000","last_fullsized":"2024-04-25T10:10:59.129192+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161578+0000","last_change":"2024-04-25T10:11:00.156924+0000","last_active":"2024-04-25T10:11:00.161578+0000","last_peered":"2024-04-25T10:11:00.161578+0000","last_clean":"2024-04-25T10:11:00.161578+0000","last_became_active":"2024-04-25T10:11:00.156508+0000","last_became_peered":"2024-04-25T10:11:00.156508+0000","last_unstale":"2024-04-25T10:11:00.161578+0000","last_undegraded":"2024-04-25T10:11:00.161578+0000","last_fullsized":"2024-04-25T10:11:00.161578+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:16.312678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.147390+0000","last_change":"2024-04-25T10:11:00.142496+0000","last_active":"2024-04-25T10:11:00.147390+0000","last_peered":"2024-04-25T10:11:00.147390+0000","last_clean":"2024-04-25T10:11:00.147390+0000","last_became_active":"2024-04-25T10:11:00.142186+0000","last_became_peered":"2024-04-25T10:11:00.142186+0000","last_unstale":"2024-04-25T10:11:00.147390+0000","last_undegraded":"2024-04-25T10:11:00.147390+0000","last_fullsized":"2024-04-25T10:11:00.147390+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.795378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.167269+0000","last_change":"2024-04-25T10:11:00.157922+0000","last_active":"2024-04-25T10:11:00.167269+0000","last_peered":"2024-04-25T10:11:00.167269+0000","last_clean":"2024-04-25T10:11:00.167269+0000","last_became_active":"2024-04-25T10:11:00.157628+0000","last_became_peered":"2024-04-25T10:11:00.157628+0000","last_unstale":"2024-04-25T10:11:00.167269+0000","last_undegraded":"2024-04-25T10:11:00.167269+0000","last_fullsized":"2024-04-25T10:11:00.167269+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:18.146132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155761+0000","last_change":"2024-04-25T10:11:00.152808+0000","last_active":"2024-04-25T10:11:00.155761+0000","last_peered":"2024-04-25T10:11:00.155761+0000","last_clean":"2024-04-25T10:11:00.155761+0000","last_became_active":"2024-04-25T10:11:00.152486+0000","last_became_peered":"2024-04-25T10:11:00.152486+0000","last_unstale":"2024-04-25T10:11:00.155761+0000","last_undegraded":"2024-04-25T10:11:00.155761+0000","last_fullsized":"2024-04-25T10:11:00.155761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:01.286881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.149004+0000","last_change":"2024-04-25T10:11:00.145699+0000","last_active":"2024-04-25T10:11:00.149004+0000","last_peered":"2024-04-25T10:11:00.149004+0000","last_clean":"2024-04-25T10:11:00.149004+0000","last_became_active":"2024-04-25T10:11:00.145381+0000","last_became_peered":"2024-04-25T10:11:00.145381+0000","last_unstale":"2024-04-25T10:11:00.149004+0000","last_undegraded":"2024-04-25T10:11:00.149004+0000","last_fullsized":"2024-04-25T10:11:00.149004+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:13.969166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161706+0000","last_change":"2024-04-25T10:11:00.160506+0000","last_active":"2024-04-25T10:11:00.161706+0000","last_peered":"2024-04-25T10:11:00.161706+0000","last_clean":"2024-04-25T10:11:00.161706+0000","last_became_active":"2024-04-25T10:11:00.159884+0000","last_became_peered":"2024-04-25T10:11:00.159884+0000","last_unstale":"2024-04-25T10:11:00.161706+0000","last_undegraded":"2024-04-25T10:11:00.161706+0000","last_fullsized":"2024-04-25T10:11:00.161706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:52.819431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161656+0000","last_change":"2024-04-25T10:11:00.158332+0000","last_active":"2024-04-25T10:11:00.161656+0000","last_peered":"2024-04-25T10:11:00.161656+0000","last_clean":"2024-04-25T10:11:00.161656+0000","last_became_active":"2024-04-25T10:11:00.157995+0000","last_became_peered":"2024-04-25T10:11:00.157995+0000","last_unstale":"2024-04-25T10:11:00.161656+0000","last_undegraded":"2024-04-25T10:11:00.161656+0000","last_fullsized":"2024-04-25T10:11:00.161656+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:19.322670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161321+0000","last_change":"2024-04-25T10:11:00.155073+0000","last_active":"2024-04-25T10:11:00.161321+0000","last_peered":"2024-04-25T10:11:00.161321+0000","last_clean":"2024-04-25T10:11:00.161321+0000","last_became_active":"2024-04-25T10:11:00.154740+0000","last_became_peered":"2024-04-25T10:11:00.154740+0000","last_unstale":"2024-04-25T10:11:00.161321+0000","last_undegraded":"2024-04-25T10:11:00.161321+0000","last_fullsized":"2024-04-25T10:11:00.161321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:24.194477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157938+0000","last_change":"2024-04-25T10:11:00.154427+0000","last_active":"2024-04-25T10:11:00.157938+0000","last_peered":"2024-04-25T10:11:00.157938+0000","last_clean":"2024-04-25T10:11:00.157938+0000","last_became_active":"2024-04-25T10:11:00.154085+0000","last_became_peered":"2024-04-25T10:11:00.154085+0000","last_unstale":"2024-04-25T10:11:00.157938+0000","last_undegraded":"2024-04-25T10:11:00.157938+0000","last_fullsized":"2024-04-25T10:11:00.157938+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:56.904985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.166857+0000","last_change":"2024-04-25T10:11:00.162836+0000","last_active":"2024-04-25T10:11:00.166857+0000","last_peered":"2024-04-25T10:11:00.166857+0000","last_clean":"2024-04-25T10:11:00.166857+0000","last_became_active":"2024-04-25T10:11:00.162437+0000","last_became_peered":"2024-04-25T10:11:00.162437+0000","last_unstale":"2024-04-25T10:11:00.166857+0000","last_undegraded":"2024-04-25T10:11:00.166857+0000","last_fullsized":"2024-04-25T10:11:00.166857+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:01.849997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.153706+0000","last_change":"2024-04-25T10:11:00.147790+0000","last_active":"2024-04-25T10:11:00.153706+0000","last_peered":"2024-04-25T10:11:00.153706+0000","last_clean":"2024-04-25T10:11:00.153706+0000","last_became_active":"2024-04-25T10:11:00.141161+0000","last_became_peered":"2024-04-25T10:11:00.141161+0000","last_unstale":"2024-04-25T10:11:00.153706+0000","last_undegraded":"2024-04-25T10:11:00.153706+0000","last_fullsized":"2024-04-25T10:11:00.153706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:14.906111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157390+0000","last_change":"2024-04-25T10:11:00.145300+0000","last_active":"2024-04-25T10:11:00.157390+0000","last_peered":"2024-04-25T10:11:00.157390+0000","last_clean":"2024-04-25T10:11:00.157390+0000","last_became_active":"2024-04-25T10:11:00.144145+0000","last_became_peered":"2024-04-25T10:11:00.144145+0000","last_unstale":"2024-04-25T10:11:00.157390+0000","last_undegraded":"2024-04-25T10:11:00.157390+0000","last_fullsized":"2024-04-25T10:11:00.157390+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:30.767315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.144375+0000","last_change":"2024-04-25T10:11:00.141264+0000","last_active":"2024-04-25T10:11:00.144375+0000","last_peered":"2024-04-25T10:11:00.144375+0000","last_clean":"2024-04-25T10:11:00.144375+0000","last_became_active":"2024-04-25T10:11:00.140864+0000","last_became_peered":"2024-04-25T10:11:00.140864+0000","last_unstale":"2024-04-25T10:11:00.144375+0000","last_undegraded":"2024-04-25T10:11:00.144375+0000","last_fullsized":"2024-04-25T10:11:00.144375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:35.595360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.164616+0000","last_change":"2024-04-25T10:11:00.160083+0000","last_active":"2024-04-25T10:11:00.164616+0000","last_peered":"2024-04-25T10:11:00.164616+0000","last_clean":"2024-04-25T10:11:00.164616+0000","last_became_active":"2024-04-25T10:11:00.159686+0000","last_became_peered":"2024-04-25T10:11:00.159686+0000","last_unstale":"2024-04-25T10:11:00.164616+0000","last_undegraded":"2024-04-25T10:11:00.164616+0000","last_fullsized":"2024-04-25T10:11:00.164616+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:55.102580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.173180+0000","last_change":"2024-04-25T10:10:59.143990+0000","last_active":"2024-04-25T10:11:00.173180+0000","last_peered":"2024-04-25T10:11:00.173180+0000","last_clean":"2024-04-25T10:11:00.173180+0000","last_became_active":"2024-04-25T10:10:59.143699+0000","last_became_peered":"2024-04-25T10:10:59.143699+0000","last_unstale":"2024-04-25T10:11:00.173180+0000","last_undegraded":"2024-04-25T10:11:00.173180+0000","last_fullsized":"2024-04-25T10:11:00.173180+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:04.645902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155787+0000","last_change":"2024-04-25T10:11:00.152888+0000","last_active":"2024-04-25T10:11:00.155787+0000","last_peered":"2024-04-25T10:11:00.155787+0000","last_clean":"2024-04-25T10:11:00.155787+0000","last_became_active":"2024-04-25T10:11:00.152463+0000","last_became_peered":"2024-04-25T10:11:00.152463+0000","last_unstale":"2024-04-25T10:11:00.155787+0000","last_undegraded":"2024-04-25T10:11:00.155787+0000","last_fullsized":"2024-04-25T10:11:00.155787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.632212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.173682+0000","last_change":"2024-04-25T10:10:59.131435+0000","last_active":"2024-04-25T10:11:00.173682+0000","last_peered":"2024-04-25T10:11:00.173682+0000","last_clean":"2024-04-25T10:11:00.173682+0000","last_became_active":"2024-04-25T10:10:59.131097+0000","last_became_peered":"2024-04-25T10:10:59.131097+0000","last_unstale":"2024-04-25T10:11:00.173682+0000","last_undegraded":"2024-04-25T10:11:00.173682+0000","last_fullsized":"2024-04-25T10:11:00.173682+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.751772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.144006+0000","last_change":"2024-04-25T10:11:00.139385+0000","last_active":"2024-04-25T10:11:00.144006+0000","last_peered":"2024-04-25T10:11:00.144006+0000","last_clean":"2024-04-25T10:11:00.144006+0000","last_became_active":"2024-04-25T10:11:00.139075+0000","last_became_peered":"2024-04-25T10:11:00.139075+0000","last_unstale":"2024-04-25T10:11:00.144006+0000","last_undegraded":"2024-04-25T10:11:00.144006+0000","last_fullsized":"2024-04-25T10:11:00.144006+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:51.395829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176816+0000","last_change":"2024-04-25T10:10:59.128480+0000","last_active":"2024-04-25T10:11:00.176816+0000","last_peered":"2024-04-25T10:11:00.176816+0000","last_clean":"2024-04-25T10:11:00.176816+0000","last_became_active":"2024-04-25T10:10:59.128031+0000","last_became_peered":"2024-04-25T10:10:59.128031+0000","last_unstale":"2024-04-25T10:11:00.176816+0000","last_undegraded":"2024-04-25T10:11:00.176816+0000","last_fullsized":"2024-04-25T10:11:00.176816+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:35.765834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.163438+0000","last_change":"2024-04-25T10:11:00.159359+0000","last_active":"2024-04-25T10:11:00.163438+0000","last_peered":"2024-04-25T10:11:00.163438+0000","last_clean":"2024-04-25T10:11:00.163438+0000","last_became_active":"2024-04-25T10:11:00.157853+0000","last_became_peered":"2024-04-25T10:11:00.157853+0000","last_unstale":"2024-04-25T10:11:00.163438+0000","last_undegraded":"2024-04-25T10:11:00.163438+0000","last_fullsized":"2024-04-25T10:11:00.163438+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:16.577842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.173631+0000","last_change":"2024-04-25T10:10:59.138124+0000","last_active":"2024-04-25T10:11:00.173631+0000","last_peered":"2024-04-25T10:11:00.173631+0000","last_clean":"2024-04-25T10:11:00.173631+0000","last_became_active":"2024-04-25T10:10:59.137742+0000","last_became_peered":"2024-04-25T10:10:59.137742+0000","last_unstale":"2024-04-25T10:11:00.173631+0000","last_undegraded":"2024-04-25T10:11:00.173631+0000","last_fullsized":"2024-04-25T10:11:00.173631+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:14.485830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.128930+0000","last_change":"2024-04-25T10:10:59.128930+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.128930+0000","last_undegraded":"2024-04-25T10:10:59.128930+0000","last_fullsized":"2024-04-25T10:10:59.128930+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176038+0000","last_change":"2024-04-25T10:10:59.134522+0000","last_active":"2024-04-25T10:11:00.176038+0000","last_peered":"2024-04-25T10:11:00.176038+0000","last_clean":"2024-04-25T10:11:00.176038+0000","last_became_active":"2024-04-25T10:10:59.134236+0000","last_became_peered":"2024-04-25T10:10:59.134236+0000","last_unstale":"2024-04-25T10:11:00.176038+0000","last_undegraded":"2024-04-25T10:11:00.176038+0000","last_fullsized":"2024-04-25T10:11:00.176038+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:02.935693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.126771+0000","last_change":"2024-04-25T10:10:59.126771+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.126771+0000","last_undegraded":"2024-04-25T10:10:59.126771+0000","last_fullsized":"2024-04-25T10:10:59.126771+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176414+0000","last_change":"2024-04-25T10:10:59.130166+0000","last_active":"2024-04-25T10:11:00.176414+0000","last_peered":"2024-04-25T10:11:00.176414+0000","last_clean":"2024-04-25T10:11:00.176414+0000","last_became_active":"2024-04-25T10:10:59.129889+0000","last_became_peered":"2024-04-25T10:10:59.129889+0000","last_unstale":"2024-04-25T10:11:00.176414+0000","last_undegraded":"2024-04-25T10:11:00.176414+0000","last_fullsized":"2024-04-25T10:11:00.176414+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:43.451259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.150890+0000","last_change":"2024-04-25T10:11:00.144604+0000","last_active":"2024-04-25T10:11:00.150890+0000","last_peered":"2024-04-25T10:11:00.150890+0000","last_clean":"2024-04-25T10:11:00.150890+0000","last_became_active":"2024-04-25T10:11:00.144322+0000","last_became_peered":"2024-04-25T10:11:00.144322+0000","last_unstale":"2024-04-25T10:11:00.150890+0000","last_undegraded":"2024-04-25T10:11:00.150890+0000","last_fullsized":"2024-04-25T10:11:00.150890+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:34.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.146134+0000","last_change":"2024-04-25T10:10:59.134683+0000","last_active":"2024-04-25T10:11:00.146134+0000","last_peered":"2024-04-25T10:11:00.146134+0000","last_clean":"2024-04-25T10:11:00.146134+0000","last_became_active":"2024-04-25T10:10:59.134304+0000","last_became_peered":"2024-04-25T10:10:59.134304+0000","last_unstale":"2024-04-25T10:11:00.146134+0000","last_undegraded":"2024-04-25T10:11:00.146134+0000","last_fullsized":"2024-04-25T10:11:00.146134+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:55.037750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.124802+0000","last_change":"2024-04-25T10:10:59.124802+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.124802+0000","last_undegraded":"2024-04-25T10:10:59.124802+0000","last_fullsized":"2024-04-25T10:10:59.124802+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.125656+0000","last_change":"2024-04-25T10:10:59.140846+0000","last_active":"2024-04-25T10:11:00.125656+0000","last_peered":"2024-04-25T10:11:00.125656+0000","last_clean":"2024-04-25T10:11:00.125656+0000","last_became_active":"2024-04-25T10:10:59.140452+0000","last_became_peered":"2024-04-25T10:10:59.140452+0000","last_unstale":"2024-04-25T10:11:00.125656+0000","last_undegraded":"2024-04-25T10:11:00.125656+0000","last_fullsized":"2024-04-25T10:11:00.125656+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:13.229979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.129630+0000","last_change":"2024-04-25T10:10:59.129630+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.129630+0000","last_undegraded":"2024-04-25T10:10:59.129630+0000","last_fullsized":"2024-04-25T10:10:59.129630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.125985+0000","last_change":"2024-04-25T10:10:59.138157+0000","last_active":"2024-04-25T10:11:00.125985+0000","last_peered":"2024-04-25T10:11:00.125985+0000","last_clean":"2024-04-25T10:11:00.125985+0000","last_became_active":"2024-04-25T10:10:59.136049+0000","last_became_peered":"2024-04-25T10:10:59.136049+0000","last_unstale":"2024-04-25T10:11:00.125985+0000","last_undegraded":"2024-04-25T10:11:00.125985+0000","last_fullsized":"2024-04-25T10:11:00.125985+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.178933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.127794+0000","last_change":"2024-04-25T10:10:59.127794+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.127794+0000","last_undegraded":"2024-04-25T10:10:59.127794+0000","last_fullsized":"2024-04-25T10:10:59.127794+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.136009+0000","last_change":"2024-04-25T10:10:59.145311+0000","last_active":"2024-04-25T10:11:00.136009+0000","last_peered":"2024-04-25T10:11:00.136009+0000","last_clean":"2024-04-25T10:11:00.136009+0000","last_became_active":"2024-04-25T10:10:59.144909+0000","last_became_peered":"2024-04-25T10:10:59.144909+0000","last_unstale":"2024-04-25T10:11:00.136009+0000","last_undegraded":"2024-04-25T10:11:00.136009+0000","last_fullsized":"2024-04-25T10:11:00.136009+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:54.087037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.148966+0000","last_change":"2024-04-25T10:11:00.145435+0000","last_active":"2024-04-25T10:11:00.148966+0000","last_peered":"2024-04-25T10:11:00.148966+0000","last_clean":"2024-04-25T10:11:00.148966+0000","last_became_active":"2024-04-25T10:11:00.145119+0000","last_became_peered":"2024-04-25T10:11:00.145119+0000","last_unstale":"2024-04-25T10:11:00.148966+0000","last_undegraded":"2024-04-25T10:11:00.148966+0000","last_fullsized":"2024-04-25T10:11:00.148966+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:47.947899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176824+0000","last_change":"2024-04-25T10:10:59.136845+0000","last_active":"2024-04-25T10:11:00.176824+0000","last_peered":"2024-04-25T10:11:00.176824+0000","last_clean":"2024-04-25T10:11:00.176824+0000","last_became_active":"2024-04-25T10:10:59.136562+0000","last_became_peered":"2024-04-25T10:10:59.136562+0000","last_unstale":"2024-04-25T10:11:00.176824+0000","last_undegraded":"2024-04-25T10:11:00.176824+0000","last_fullsized":"2024-04-25T10:11:00.176824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:34.132198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155825+0000","last_change":"2024-04-25T10:11:00.150168+0000","last_active":"2024-04-25T10:11:00.155825+0000","last_peered":"2024-04-25T10:11:00.155825+0000","last_clean":"2024-04-25T10:11:00.155825+0000","last_became_active":"2024-04-25T10:11:00.149765+0000","last_became_peered":"2024-04-25T10:11:00.149765+0000","last_unstale":"2024-04-25T10:11:00.155825+0000","last_undegraded":"2024-04-25T10:11:00.155825+0000","last_fullsized":"2024-04-25T10:11:00.155825+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:58.517016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.180164+0000","last_change":"2024-04-25T10:10:59.135798+0000","last_active":"2024-04-25T10:11:00.180164+0000","last_peered":"2024-04-25T10:11:00.180164+0000","last_clean":"2024-04-25T10:11:00.180164+0000","last_became_active":"2024-04-25T10:10:59.135518+0000","last_became_peered":"2024-04-25T10:10:59.135518+0000","last_unstale":"2024-04-25T10:11:00.180164+0000","last_undegraded":"2024-04-25T10:11:00.180164+0000","last_fullsized":"2024-04-25T10:11:00.180164+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:20.159764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.137928+0000","last_change":"2024-04-25T10:11:00.133469+0000","last_active":"2024-04-25T10:11:00.137928+0000","last_peered":"2024-04-25T10:11:00.137928+0000","last_clean":"2024-04-25T10:11:00.137928+0000","last_became_active":"2024-04-25T10:11:00.133134+0000","last_became_peered":"2024-04-25T10:11:00.133134+0000","last_unstale":"2024-04-25T10:11:00.137928+0000","last_undegraded":"2024-04-25T10:11:00.137928+0000","last_fullsized":"2024-04-25T10:11:00.137928+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:09.064575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.144996+0000","last_change":"2024-04-25T10:10:59.138623+0000","last_active":"2024-04-25T10:10:59.144996+0000","last_peered":"2024-04-25T10:10:59.144996+0000","last_clean":"2024-04-25T10:10:59.144996+0000","last_became_active":"2024-04-25T10:10:59.138336+0000","last_became_peered":"2024-04-25T10:10:59.138336+0000","last_unstale":"2024-04-25T10:10:59.144996+0000","last_undegraded":"2024-04-25T10:10:59.144996+0000","last_fullsized":"2024-04-25T10:10:59.144996+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:49.053697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.149065+0000","last_change":"2024-04-25T10:11:00.145458+0000","last_active":"2024-04-25T10:11:00.149065+0000","last_peered":"2024-04-25T10:11:00.149065+0000","last_clean":"2024-04-25T10:11:00.149065+0000","last_became_active":"2024-04-25T10:11:00.145153+0000","last_became_peered":"2024-04-25T10:11:00.145153+0000","last_unstale":"2024-04-25T10:11:00.149065+0000","last_undegraded":"2024-04-25T10:11:00.149065+0000","last_fullsized":"2024-04-25T10:11:00.149065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:18.146132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.134320+0000","last_change":"2024-04-25T10:10:59.144843+0000","last_active":"2024-04-25T10:11:00.134320+0000","last_peered":"2024-04-25T10:11:00.134320+0000","last_clean":"2024-04-25T10:11:00.134320+0000","last_became_active":"2024-04-25T10:10:59.144558+0000","last_became_peered":"2024-04-25T10:10:59.144558+0000","last_unstale":"2024-04-25T10:11:00.134320+0000","last_undegraded":"2024-04-25T10:11:00.134320+0000","last_fullsized":"2024-04-25T10:11:00.134320+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:55.889441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.167198+0000","last_change":"2024-04-25T10:11:00.157051+0000","last_active":"2024-04-25T10:11:00.167198+0000","last_peered":"2024-04-25T10:11:00.167198+0000","last_clean":"2024-04-25T10:11:00.167198+0000","last_became_active":"2024-04-25T10:11:00.156775+0000","last_became_peered":"2024-04-25T10:11:00.156775+0000","last_unstale":"2024-04-25T10:11:00.167198+0000","last_undegraded":"2024-04-25T10:11:00.167198+0000","last_fullsized":"2024-04-25T10:11:00.167198+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:51.395829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.148348+0000","last_change":"2024-04-25T10:10:59.142122+0000","last_active":"2024-04-25T10:10:59.148348+0000","last_peered":"2024-04-25T10:10:59.148348+0000","last_clean":"2024-04-25T10:10:59.148348+0000","last_became_active":"2024-04-25T10:10:59.141675+0000","last_became_peered":"2024-04-25T10:10:59.141675+0000","last_unstale":"2024-04-25T10:10:59.148348+0000","last_undegraded":"2024-04-25T10:10:59.148348+0000","last_fullsized":"2024-04-25T10:10:59.148348+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:08.620823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.164455+0000","last_change":"2024-04-25T10:11:00.160146+0000","last_active":"2024-04-25T10:11:00.164455+0000","last_peered":"2024-04-25T10:11:00.164455+0000","last_clean":"2024-04-25T10:11:00.164455+0000","last_became_active":"2024-04-25T10:11:00.159785+0000","last_became_peered":"2024-04-25T10:11:00.159785+0000","last_unstale":"2024-04-25T10:11:00.164455+0000","last_undegraded":"2024-04-25T10:11:00.164455+0000","last_fullsized":"2024-04-25T10:11:00.164455+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:56.904985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.177179+0000","last_change":"2024-04-25T10:10:59.129505+0000","last_active":"2024-04-25T10:11:00.177179+0000","last_peered":"2024-04-25T10:11:00.177179+0000","last_clean":"2024-04-25T10:11:00.177179+0000","last_became_active":"2024-04-25T10:10:59.129216+0000","last_became_peered":"2024-04-25T10:10:59.129216+0000","last_unstale":"2024-04-25T10:11:00.177179+0000","last_undegraded":"2024-04-25T10:11:00.177179+0000","last_fullsized":"2024-04-25T10:11:00.177179+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:37.060083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155814+0000","last_change":"2024-04-25T10:11:00.152775+0000","last_active":"2024-04-25T10:11:00.155814+0000","last_peered":"2024-04-25T10:11:00.155814+0000","last_clean":"2024-04-25T10:11:00.155814+0000","last_became_active":"2024-04-25T10:11:00.152437+0000","last_became_peered":"2024-04-25T10:11:00.152437+0000","last_unstale":"2024-04-25T10:11:00.155814+0000","last_undegraded":"2024-04-25T10:11:00.155814+0000","last_fullsized":"2024-04-25T10:11:00.155814+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:57.413264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.172691+0000","last_change":"2024-04-25T10:10:59.140294+0000","last_active":"2024-04-25T10:11:00.172691+0000","last_peered":"2024-04-25T10:11:00.172691+0000","last_clean":"2024-04-25T10:11:00.172691+0000","last_became_active":"2024-04-25T10:10:59.138679+0000","last_became_peered":"2024-04-25T10:10:59.138679+0000","last_unstale":"2024-04-25T10:11:00.172691+0000","last_undegraded":"2024-04-25T10:11:00.172691+0000","last_fullsized":"2024-04-25T10:11:00.172691+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:15.297135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161782+0000","last_change":"2024-04-25T10:11:00.154733+0000","last_active":"2024-04-25T10:11:00.161782+0000","last_peered":"2024-04-25T10:11:00.161782+0000","last_clean":"2024-04-25T10:11:00.161782+0000","last_became_active":"2024-04-25T10:11:00.154310+0000","last_became_peered":"2024-04-25T10:11:00.154310+0000","last_unstale":"2024-04-25T10:11:00.161782+0000","last_undegraded":"2024-04-25T10:11:00.161782+0000","last_fullsized":"2024-04-25T10:11:00.161782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:04.165775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.175794+0000","last_change":"2024-04-25T10:10:59.135411+0000","last_active":"2024-04-25T10:11:00.175794+0000","last_peered":"2024-04-25T10:11:00.175794+0000","last_clean":"2024-04-25T10:11:00.175794+0000","last_became_active":"2024-04-25T10:10:59.135122+0000","last_became_peered":"2024-04-25T10:10:59.135122+0000","last_unstale":"2024-04-25T10:11:00.175794+0000","last_undegraded":"2024-04-25T10:11:00.175794+0000","last_fullsized":"2024-04-25T10:11:00.175794+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:25.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161944+0000","last_change":"2024-04-25T10:11:00.154205+0000","last_active":"2024-04-25T10:11:00.161944+0000","last_peered":"2024-04-25T10:11:00.161944+0000","last_clean":"2024-04-25T10:11:00.161944+0000","last_became_active":"2024-04-25T10:11:00.153884+0000","last_became_peered":"2024-04-25T10:11:00.153884+0000","last_unstale":"2024-04-25T10:11:00.161944+0000","last_undegraded":"2024-04-25T10:11:00.161944+0000","last_fullsized":"2024-04-25T10:11:00.161944+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.125479+0000","last_change":"2024-04-25T10:10:59.134710+0000","last_active":"2024-04-25T10:11:00.125479+0000","last_peered":"2024-04-25T10:11:00.125479+0000","last_clean":"2024-04-25T10:11:00.125479+0000","last_became_active":"2024-04-25T10:10:59.134413+0000","last_became_peered":"2024-04-25T10:10:59.134413+0000","last_unstale":"2024-04-25T10:11:00.125479+0000","last_undegraded":"2024-04-25T10:11:00.125479+0000","last_fullsized":"2024-04-25T10:11:00.125479+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:03.150231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.133507+0000","last_change":"2024-04-25T10:10:59.133507+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.133507+0000","last_undegraded":"2024-04-25T10:10:59.133507+0000","last_fullsized":"2024-04-25T10:10:59.133507+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.175967+0000","last_change":"2024-04-25T10:10:59.141774+0000","last_active":"2024-04-25T10:11:00.175967+0000","last_peered":"2024-04-25T10:11:00.175967+0000","last_clean":"2024-04-25T10:11:00.175967+0000","last_became_active":"2024-04-25T10:10:59.141413+0000","last_became_peered":"2024-04-25T10:10:59.141413+0000","last_unstale":"2024-04-25T10:11:00.175967+0000","last_undegraded":"2024-04-25T10:11:00.175967+0000","last_fullsized":"2024-04-25T10:11:00.175967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:15.297135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.149430+0000","last_change":"2024-04-25T10:11:00.144994+0000","last_active":"2024-04-25T10:11:00.149430+0000","last_peered":"2024-04-25T10:11:00.149430+0000","last_clean":"2024-04-25T10:11:00.149430+0000","last_became_active":"2024-04-25T10:11:00.144706+0000","last_became_peered":"2024-04-25T10:11:00.144706+0000","last_unstale":"2024-04-25T10:11:00.149430+0000","last_undegraded":"2024-04-25T10:11:00.149430+0000","last_fullsized":"2024-04-25T10:11:00.149430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:09.127558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.135534+0000","last_change":"2024-04-25T10:10:59.131823+0000","last_active":"2024-04-25T10:11:00.135534+0000","last_peered":"2024-04-25T10:11:00.135534+0000","last_clean":"2024-04-25T10:11:00.135534+0000","last_became_active":"2024-04-25T10:10:59.131437+0000","last_became_peered":"2024-04-25T10:10:59.131437+0000","last_unstale":"2024-04-25T10:11:00.135534+0000","last_undegraded":"2024-04-25T10:11:00.135534+0000","last_fullsized":"2024-04-25T10:11:00.135534+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:18.307126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155821+0000","last_change":"2024-04-25T10:11:00.149678+0000","last_active":"2024-04-25T10:11:00.155821+0000","last_peered":"2024-04-25T10:11:00.155821+0000","last_clean":"2024-04-25T10:11:00.155821+0000","last_became_active":"2024-04-25T10:11:00.149313+0000","last_became_peered":"2024-04-25T10:11:00.149313+0000","last_unstale":"2024-04-25T10:11:00.155821+0000","last_undegraded":"2024-04-25T10:11:00.155821+0000","last_fullsized":"2024-04-25T10:11:00.155821+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:09.064575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.124809+0000","last_change":"2024-04-25T10:10:59.139124+0000","last_active":"2024-04-25T10:11:00.124809+0000","last_peered":"2024-04-25T10:11:00.124809+0000","last_clean":"2024-04-25T10:11:00.124809+0000","last_became_active":"2024-04-25T10:10:59.138797+0000","last_became_peered":"2024-04-25T10:10:59.138797+0000","last_unstale":"2024-04-25T10:11:00.124809+0000","last_undegraded":"2024-04-25T10:11:00.124809+0000","last_fullsized":"2024-04-25T10:11:00.124809+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:25.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.125598+0000","last_change":"2024-04-25T10:10:59.125598+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.125598+0000","last_undegraded":"2024-04-25T10:10:59.125598+0000","last_fullsized":"2024-04-25T10:10:59.125598+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.137457+0000","last_change":"2024-04-25T10:10:59.138623+0000","last_active":"2024-04-25T10:11:00.137457+0000","last_peered":"2024-04-25T10:11:00.137457+0000","last_clean":"2024-04-25T10:11:00.137457+0000","last_became_active":"2024-04-25T10:10:59.138336+0000","last_became_peered":"2024-04-25T10:10:59.138336+0000","last_unstale":"2024-04-25T10:11:00.137457+0000","last_undegraded":"2024-04-25T10:11:00.137457+0000","last_fullsized":"2024-04-25T10:11:00.137457+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:19.064015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.144396+0000","last_change":"2024-04-25T10:11:00.141868+0000","last_active":"2024-04-25T10:11:00.144396+0000","last_peered":"2024-04-25T10:11:00.144396+0000","last_clean":"2024-04-25T10:11:00.144396+0000","last_became_active":"2024-04-25T10:11:00.141601+0000","last_became_peered":"2024-04-25T10:11:00.141601+0000","last_unstale":"2024-04-25T10:11:00.144396+0000","last_undegraded":"2024-04-25T10:11:00.144396+0000","last_fullsized":"2024-04-25T10:11:00.144396+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:36.781377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176143+0000","last_change":"2024-04-25T10:10:59.146023+0000","last_active":"2024-04-25T10:11:00.176143+0000","last_peered":"2024-04-25T10:11:00.176143+0000","last_clean":"2024-04-25T10:11:00.176143+0000","last_became_active":"2024-04-25T10:10:59.145057+0000","last_became_peered":"2024-04-25T10:10:59.145057+0000","last_unstale":"2024-04-25T10:11:00.176143+0000","last_undegraded":"2024-04-25T10:11:00.176143+0000","last_fullsized":"2024-04-25T10:11:00.176143+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:39.790722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157577+0000","last_change":"2024-04-25T10:11:00.148986+0000","last_active":"2024-04-25T10:11:00.157577+0000","last_peered":"2024-04-25T10:11:00.157577+0000","last_clean":"2024-04-25T10:11:00.157577+0000","last_became_active":"2024-04-25T10:11:00.142173+0000","last_became_peered":"2024-04-25T10:11:00.142173+0000","last_unstale":"2024-04-25T10:11:00.157577+0000","last_undegraded":"2024-04-25T10:11:00.157577+0000","last_fullsized":"2024-04-25T10:11:00.157577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:53.016466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.136704+0000","last_change":"2024-04-25T10:10:59.139074+0000","last_active":"2024-04-25T10:11:00.136704+0000","last_peered":"2024-04-25T10:11:00.136704+0000","last_clean":"2024-04-25T10:11:00.136704+0000","last_became_active":"2024-04-25T10:10:59.138792+0000","last_became_peered":"2024-04-25T10:10:59.138792+0000","last_unstale":"2024-04-25T10:11:00.136704+0000","last_undegraded":"2024-04-25T10:11:00.136704+0000","last_fullsized":"2024-04-25T10:11:00.136704+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:51.803887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.158468+0000","last_change":"2024-04-25T10:11:00.156802+0000","last_active":"2024-04-25T10:11:00.158468+0000","last_peered":"2024-04-25T10:11:00.158468+0000","last_clean":"2024-04-25T10:11:00.158468+0000","last_became_active":"2024-04-25T10:11:00.156444+0000","last_became_peered":"2024-04-25T10:11:00.156444+0000","last_unstale":"2024-04-25T10:11:00.158468+0000","last_undegraded":"2024-04-25T10:11:00.158468+0000","last_fullsized":"2024-04-25T10:11:00.158468+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:21.936351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.132505+0000","last_change":"2024-04-25T10:10:59.136023+0000","last_active":"2024-04-25T10:11:00.132505+0000","last_peered":"2024-04-25T10:11:00.132505+0000","last_clean":"2024-04-25T10:11:00.132505+0000","last_became_active":"2024-04-25T10:10:59.135691+0000","last_became_peered":"2024-04-25T10:10:59.135691+0000","last_unstale":"2024-04-25T10:11:00.132505+0000","last_undegraded":"2024-04-25T10:11:00.132505+0000","last_fullsized":"2024-04-25T10:11:00.132505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:02.165953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.162387+0000","last_change":"2024-04-25T10:11:00.159090+0000","last_active":"2024-04-25T10:11:00.162387+0000","last_peered":"2024-04-25T10:11:00.162387+0000","last_clean":"2024-04-25T10:11:00.162387+0000","last_became_active":"2024-04-25T10:11:00.157455+0000","last_became_peered":"2024-04-25T10:11:00.157455+0000","last_unstale":"2024-04-25T10:11:00.162387+0000","last_undegraded":"2024-04-25T10:11:00.162387+0000","last_fullsized":"2024-04-25T10:11:00.162387+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:30.767315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.148190+0000","last_change":"2024-04-25T10:10:59.140664+0000","last_active":"2024-04-25T10:10:59.148190+0000","last_peered":"2024-04-25T10:10:59.148190+0000","last_clean":"2024-04-25T10:10:59.148190+0000","last_became_active":"2024-04-25T10:10:59.140358+0000","last_became_peered":"2024-04-25T10:10:59.140358+0000","last_unstale":"2024-04-25T10:10:59.148190+0000","last_undegraded":"2024-04-25T10:10:59.148190+0000","last_fullsized":"2024-04-25T10:10:59.148190+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:35.399217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.133941+0000","last_change":"2024-04-25T10:10:59.133941+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.133941+0000","last_undegraded":"2024-04-25T10:10:59.133941+0000","last_fullsized":"2024-04-25T10:10:59.133941+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.128833+0000","last_change":"2024-04-25T10:10:59.139722+0000","last_active":"2024-04-25T10:11:00.128833+0000","last_peered":"2024-04-25T10:11:00.128833+0000","last_clean":"2024-04-25T10:11:00.128833+0000","last_became_active":"2024-04-25T10:10:59.139406+0000","last_became_peered":"2024-04-25T10:10:59.139406+0000","last_unstale":"2024-04-25T10:11:00.128833+0000","last_undegraded":"2024-04-25T10:11:00.128833+0000","last_fullsized":"2024-04-25T10:11:00.128833+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:33.839986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-04-25T10:10:59.127624+0000","last_change":"2024-04-25T10:10:59.127624+0000","last_active":"2024-04-25T10:10:59.088607+0000","last_peered":"2024-04-25T10:10:59.088607+0000","last_clean":"2024-04-25T10:10:59.088607+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:10:59.127624+0000","last_undegraded":"2024-04-25T10:10:59.127624+0000","last_fullsized":"2024-04-25T10:10:59.127624+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.136919+0000","last_change":"2024-04-25T10:10:59.138750+0000","last_active":"2024-04-25T10:11:00.136919+0000","last_peered":"2024-04-25T10:11:00.136919+0000","last_clean":"2024-04-25T10:11:00.136919+0000","last_became_active":"2024-04-25T10:10:59.137574+0000","last_became_peered":"2024-04-25T10:10:59.137574+0000","last_unstale":"2024-04-25T10:11:00.136919+0000","last_undegraded":"2024-04-25T10:11:00.136919+0000","last_fullsized":"2024-04-25T10:11:00.136919+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:55.999917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.158677+0000","last_change":"2024-04-25T10:11:00.153550+0000","last_active":"2024-04-25T10:11:00.158677+0000","last_peered":"2024-04-25T10:11:00.158677+0000","last_clean":"2024-04-25T10:11:00.158677+0000","last_became_active":"2024-04-25T10:11:00.150240+0000","last_became_peered":"2024-04-25T10:11:00.150240+0000","last_unstale":"2024-04-25T10:11:00.158677+0000","last_undegraded":"2024-04-25T10:11:00.158677+0000","last_fullsized":"2024-04-25T10:11:00.158677+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.443869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.146291+0000","last_change":"2024-04-25T10:10:59.143574+0000","last_active":"2024-04-25T10:11:00.146291+0000","last_peered":"2024-04-25T10:11:00.146291+0000","last_clean":"2024-04-25T10:11:00.146291+0000","last_became_active":"2024-04-25T10:10:59.143241+0000","last_became_peered":"2024-04-25T10:10:59.143241+0000","last_unstale":"2024-04-25T10:11:00.146291+0000","last_undegraded":"2024-04-25T10:11:00.146291+0000","last_fullsized":"2024-04-25T10:11:00.146291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:36.300434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.143430+0000","last_change":"2024-04-25T10:11:00.137562+0000","last_active":"2024-04-25T10:11:00.143430+0000","last_peered":"2024-04-25T10:11:00.143430+0000","last_clean":"2024-04-25T10:11:00.143430+0000","last_became_active":"2024-04-25T10:11:00.137219+0000","last_became_peered":"2024-04-25T10:11:00.137219+0000","last_unstale":"2024-04-25T10:11:00.143430+0000","last_undegraded":"2024-04-25T10:11:00.143430+0000","last_fullsized":"2024-04-25T10:11:00.143430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:58.517016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.133619+0000","last_change":"2024-04-25T10:10:59.124080+0000","last_active":"2024-04-25T10:11:00.133619+0000","last_peered":"2024-04-25T10:11:00.133619+0000","last_clean":"2024-04-25T10:11:00.133619+0000","last_became_active":"2024-04-25T10:10:59.123508+0000","last_became_peered":"2024-04-25T10:10:59.123508+0000","last_unstale":"2024-04-25T10:11:00.133619+0000","last_undegraded":"2024-04-25T10:11:00.133619+0000","last_fullsized":"2024-04-25T10:11:00.133619+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.178933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157873+0000","last_change":"2024-04-25T10:11:00.145753+0000","last_active":"2024-04-25T10:11:00.157873+0000","last_peered":"2024-04-25T10:11:00.157873+0000","last_clean":"2024-04-25T10:11:00.157873+0000","last_became_active":"2024-04-25T10:11:00.145032+0000","last_became_peered":"2024-04-25T10:11:00.145032+0000","last_unstale":"2024-04-25T10:11:00.157873+0000","last_undegraded":"2024-04-25T10:11:00.157873+0000","last_fullsized":"2024-04-25T10:11:00.157873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:57.015460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T10:10:59.151040+0000","last_change":"2024-04-25T10:10:59.146698+0000","last_active":"2024-04-25T10:10:59.151040+0000","last_peered":"2024-04-25T10:10:59.151040+0000","last_clean":"2024-04-25T10:10:59.151040+0000","last_became_active":"2024-04-25T10:10:59.146402+0000","last_became_peered":"2024-04-25T10:10:59.146402+0000","last_unstale":"2024-04-25T10:10:59.151040+0000","last_undegraded":"2024-04-25T10:10:59.151040+0000","last_fullsized":"2024-04-25T10:10:59.151040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:48.718920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":325,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.146481+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:11:00.146481+0000","last_peered":"2024-04-25T10:11:00.146481+0000","last_clean":"2024-04-25T10:11:00.146481+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:11:00.146481+0000","last_undegraded":"2024-04-25T10:11:00.146481+0000","last_fullsized":"2024-04-25T10:11:00.146481+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":594052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":593400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1884160,"data_stored":1183892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509555,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1294336,"data_stored":595012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":593512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509553,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1875968,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":593982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1269760,"data_stored":586233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:04.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.816+0000 7f0c27000640 1 Processor -- start 2024-04-25T10:11:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.816+0000 7f0c27000640 1 -- start start 2024-04-25T10:11:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.818+0000 7f0c27000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 0x7f0c2006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.818+0000 7f0c27000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c20071d30 con 0x7f0c2015d510 2024-04-25T10:11:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.818+0000 7f0c27000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2006d620 con 0x7f0c2006e640 2024-04-25T10:11:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.818+0000 7f0c27000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20142180 con 0x7f0c20141da0 2024-04-25T10:11:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.819+0000 7f0c26600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0c2015d510 0x7f0c200cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44732/0 (socket says 172.21.15.5:44732) 2024-04-25T10:11:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.819+0000 7f0c26600640 1 -- 172.21.15.5:0/962164241 learned_addr learned my addr 172.21.15.5:0/962164241 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.819+0000 7f0c25c00640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 0x7f0c2006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.819+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995212764 0 0) 0x7f0c20071d30 con 0x7f0c2015d510 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.819+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003520 con 0x7f0c2015d510 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1769319005 0 0) 0x7f0c08003520 con 0x7f0c2015d510 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 msgr2=0x7f0c2006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 0x7f0c2006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 >> v1:172.21.15.80:6789/0 conn(0x7f0c2006e640 legacy=0x7f0c2015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2006d900 con 0x7f0c2015d510 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0c180019f0 con 0x7f0c2015d510 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 >> v1:172.21.15.5:6789/0 conn(0x7f0c2015d510 legacy=0x7f0c200cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c080039e0 0x7f0c08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 0x7f0c080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20142180 con 0x7f0c08004800 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800baa0 con 0x7f0c080039e0 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800bca0 con 0x7f0c20141da0 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c25200640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c26600640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 0x7f0c080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.820+0000 7f0c25c00640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c080039e0 0x7f0c08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.821+0000 7f0c25200640 1 -- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 msgr2=0x7f0c080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.821+0000 7f0c25200640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c20141da0 0x7f0c080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.821+0000 7f0c25200640 1 -- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c080039e0 msgr2=0x7f0c08003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.821+0000 7f0c25200640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c080039e0 0x7f0c08003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.821+0000 7f0c25200640 1 -- 172.21.15.5:0/962164241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08003520 con 0x7f0c08004800 2024-04-25T10:11:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.822+0000 7f0c25200640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0c2006eac0 tx=0x7f0c08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a56bf570122e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.822+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c001370 con 0x7f0c08004800 2024-04-25T10:11:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.822+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0052d0 con 0x7f0c08004800 2024-04-25T10:11:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.822+0000 7f0c1fe00640 1 -- 172.21.15.5:0/962164241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c02ab70 con 0x7f0c08004800 2024-04-25T10:11:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c27000640 1 -- 172.21.15.5:0/962164241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08004800 msgr2=0x7f0c08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c27000640 1 --2- 172.21.15.5:0/962164241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08004800 0x7f0c08004c80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0c2006eac0 tx=0x7f0c08005830 comp rx=0 tx=0).stop 2024-04-25T10:11:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c26600640 1 -- 172.21.15.5:0/962164241 reap_dead start 2024-04-25T10:11:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c27000640 1 -- 172.21.15.5:0/962164241 shutdown_connections 2024-04-25T10:11:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c27000640 1 -- 172.21.15.5:0/962164241 >> 172.21.15.5:0/962164241 conn(0x7f0c20066f20 msgr2=0x7f0c2006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c27000640 1 -- 172.21.15.5:0/962164241 shutdown_connections 2024-04-25T10:11:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.823+0000 7f0c27000640 1 -- 172.21.15.5:0/962164241 wait complete. 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.825+0000 7f0c27000640 1 Processor -- start 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.825+0000 7f0c27000640 1 -- start start 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c27000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c08004800 0x7f0c200c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c27000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c200716c0 0x7f0c201f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c27000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 0x7f0c201f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c27000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200cdb90 con 0x7f0c2015ac60 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c27000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200cda10 con 0x7f0c200716c0 2024-04-25T10:11:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c27000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2006e100 con 0x7f0c08004800 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 0x7f0c201f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 0x7f0c201f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41300/0 (socket says 172.21.15.5:41300) 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 -- 172.21.15.5:0/29390581 learned_addr learned my addr 172.21.15.5:0/29390581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c25c00640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c08004800 0x7f0c200c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c25200640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c200716c0 0x7f0c201f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 -- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c08004800 msgr2=0x7f0c200c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c08004800 0x7f0c200c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 -- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c200716c0 msgr2=0x7f0c201f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.826+0000 7f0c26600640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c200716c0 0x7f0c201f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c26600640 1 -- 172.21.15.5:0/29390581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200cbf60 con 0x7f0c2015ac60 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c25200640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c200716c0 0x7f0c201f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c26600640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 0x7f0c201f1a00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0c2006f020 tx=0x7f0c18007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1800ebf0 con 0x7f0c2015ac60 2024-04-25T10:11:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1800ed90 con 0x7f0c2015ac60 2024-04-25T10:11:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c200c7fd0 con 0x7f0c2015ac60 2024-04-25T10:11:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18014670 con 0x7f0c2015ac60 2024-04-25T10:11:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.827+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c200c8590 con 0x7f0c2015ac60 2024-04-25T10:11:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.828+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c20068f60 con 0x7f0c2015ac60 2024-04-25T10:11:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.829+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c18014810 con 0x7f0c2015ac60 2024-04-25T10:11:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.830+0000 7f0c1d600640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c0006b500 0x7f0c0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.830+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c1808a220 con 0x7f0c2015ac60 2024-04-25T10:11:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.831+0000 7f0c25c00640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c0006b500 0x7f0c0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.832+0000 7f0c25c00640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c0006b500 0x7f0c0006d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009a10 tx=0x7f0c14008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:04.834+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c200cbf60 con 0x7f0c2015ac60 2024-04-25T10:11:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.035+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c200c73a0 con 0x7f0c0006b500 2024-04-25T10:11:05.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.055+0000 7f0c1d600640 1 -- 172.21.15.5:0/29390581 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252435 (secure 0 0 0) 0x7f0c200c73a0 con 0x7f0c0006b500 2024-04-25T10:11:05.064 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:05.065 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c0006b500 msgr2=0x7f0c0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c0006b500 0x7f0c0006d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009a10 tx=0x7f0c14008040 comp rx=0 tx=0).stop 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 msgr2=0x7f0c201f1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 0x7f0c201f1a00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0c2006f020 tx=0x7f0c18007bc0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 shutdown_connections 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c2015ac60 0x7f0c201f1a00 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f0c0006b500 0x7f0c0006d9c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c200716c0 0x7f0c201f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 --2- 172.21.15.5:0/29390581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c08004800 0x7f0c200c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.060+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 >> 172.21.15.5:0/29390581 conn(0x7f0c20066f20 msgr2=0x7f0c200cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.061+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 shutdown_connections 2024-04-25T10:11:05.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.061+0000 7f0c27000640 1 -- 172.21.15.5:0/29390581 wait complete. 2024-04-25T10:11:05.084 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-04-25T10:11:04.817299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260684,"kb_used_data":11232,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714036,"statfs":{"total":773094113280,"available":772827172864,"internally_reserved":0,"allocated":11501568,"data_stored":5937744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002089"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.658359+0000","last_change":"2024-04-25T10:11:00.143560+0000","last_active":"2024-04-25T10:11:02.658359+0000","last_peered":"2024-04-25T10:11:02.658359+0000","last_clean":"2024-04-25T10:11:02.658359+0000","last_became_active":"2024-04-25T10:11:00.143198+0000","last_became_peered":"2024-04-25T10:11:00.143198+0000","last_unstale":"2024-04-25T10:11:02.658359+0000","last_undegraded":"2024-04-25T10:11:02.658359+0000","last_fullsized":"2024-04-25T10:11:02.658359+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:51.140597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.152216+0000","last_change":"2024-04-25T10:11:00.150426+0000","last_active":"2024-04-25T10:11:00.152216+0000","last_peered":"2024-04-25T10:11:00.152216+0000","last_clean":"2024-04-25T10:11:00.152216+0000","last_became_active":"2024-04-25T10:11:00.150107+0000","last_became_peered":"2024-04-25T10:11:00.150107+0000","last_unstale":"2024-04-25T10:11:00.152216+0000","last_undegraded":"2024-04-25T10:11:00.152216+0000","last_fullsized":"2024-04-25T10:11:00.152216+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:22.263161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.158486+0000","last_change":"2024-04-25T10:11:00.154826+0000","last_active":"2024-04-25T10:11:00.158486+0000","last_peered":"2024-04-25T10:11:00.158486+0000","last_clean":"2024-04-25T10:11:00.158486+0000","last_became_active":"2024-04-25T10:11:00.154532+0000","last_became_peered":"2024-04-25T10:11:00.154532+0000","last_unstale":"2024-04-25T10:11:00.158486+0000","last_undegraded":"2024-04-25T10:11:00.158486+0000","last_fullsized":"2024-04-25T10:11:00.158486+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.443869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.656983+0000","last_change":"2024-04-25T10:11:00.140825+0000","last_active":"2024-04-25T10:11:02.656983+0000","last_peered":"2024-04-25T10:11:02.656983+0000","last_clean":"2024-04-25T10:11:02.656983+0000","last_became_active":"2024-04-25T10:11:00.140546+0000","last_became_peered":"2024-04-25T10:11:00.140546+0000","last_unstale":"2024-04-25T10:11:02.656983+0000","last_undegraded":"2024-04-25T10:11:02.656983+0000","last_fullsized":"2024-04-25T10:11:02.656983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:05.744373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.656792+0000","last_change":"2024-04-25T10:11:00.131609+0000","last_active":"2024-04-25T10:11:02.656792+0000","last_peered":"2024-04-25T10:11:02.656792+0000","last_clean":"2024-04-25T10:11:02.656792+0000","last_became_active":"2024-04-25T10:11:00.131091+0000","last_became_peered":"2024-04-25T10:11:00.131091+0000","last_unstale":"2024-04-25T10:11:02.656792+0000","last_undegraded":"2024-04-25T10:11:02.656792+0000","last_fullsized":"2024-04-25T10:11:02.656792+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:40.313850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157728+0000","last_change":"2024-04-25T10:11:00.140959+0000","last_active":"2024-04-25T10:11:00.157728+0000","last_peered":"2024-04-25T10:11:00.157728+0000","last_clean":"2024-04-25T10:11:00.157728+0000","last_became_active":"2024-04-25T10:11:00.140581+0000","last_became_peered":"2024-04-25T10:11:00.140581+0000","last_unstale":"2024-04-25T10:11:00.157728+0000","last_undegraded":"2024-04-25T10:11:00.157728+0000","last_fullsized":"2024-04-25T10:11:00.157728+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:19.322670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157417+0000","last_change":"2024-04-25T10:11:00.145532+0000","last_active":"2024-04-25T10:11:00.157417+0000","last_peered":"2024-04-25T10:11:00.157417+0000","last_clean":"2024-04-25T10:11:00.157417+0000","last_became_active":"2024-04-25T10:11:00.144637+0000","last_became_peered":"2024-04-25T10:11:00.144637+0000","last_unstale":"2024-04-25T10:11:00.157417+0000","last_undegraded":"2024-04-25T10:11:00.157417+0000","last_fullsized":"2024-04-25T10:11:00.157417+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:52.819431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.163794+0000","last_change":"2024-04-25T10:11:00.157377+0000","last_active":"2024-04-25T10:11:00.163794+0000","last_peered":"2024-04-25T10:11:00.163794+0000","last_clean":"2024-04-25T10:11:00.163794+0000","last_became_active":"2024-04-25T10:11:00.156516+0000","last_became_peered":"2024-04-25T10:11:00.156516+0000","last_unstale":"2024-04-25T10:11:00.163794+0000","last_undegraded":"2024-04-25T10:11:00.163794+0000","last_fullsized":"2024-04-25T10:11:00.163794+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:59.451682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.143677+0000","last_change":"2024-04-25T10:11:00.138094+0000","last_active":"2024-04-25T10:11:00.143677+0000","last_peered":"2024-04-25T10:11:00.143677+0000","last_clean":"2024-04-25T10:11:00.143677+0000","last_became_active":"2024-04-25T10:11:00.137842+0000","last_became_peered":"2024-04-25T10:11:00.137842+0000","last_unstale":"2024-04-25T10:11:00.143677+0000","last_undegraded":"2024-04-25T10:11:00.143677+0000","last_fullsized":"2024-04-25T10:11:00.143677+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.163778+0000","last_change":"2024-04-25T10:11:00.157715+0000","last_active":"2024-04-25T10:11:00.163778+0000","last_peered":"2024-04-25T10:11:00.163778+0000","last_clean":"2024-04-25T10:11:00.163778+0000","last_became_active":"2024-04-25T10:11:00.157424+0000","last_became_peered":"2024-04-25T10:11:00.157424+0000","last_unstale":"2024-04-25T10:11:00.163778+0000","last_undegraded":"2024-04-25T10:11:00.163778+0000","last_fullsized":"2024-04-25T10:11:00.163778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:39.120461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.138090+0000","last_change":"2024-04-25T10:11:00.136356+0000","last_active":"2024-04-25T10:11:00.138090+0000","last_peered":"2024-04-25T10:11:00.138090+0000","last_clean":"2024-04-25T10:11:00.138090+0000","last_became_active":"2024-04-25T10:11:00.136031+0000","last_became_peered":"2024-04-25T10:11:00.136031+0000","last_unstale":"2024-04-25T10:11:00.138090+0000","last_undegraded":"2024-04-25T10:11:00.138090+0000","last_fullsized":"2024-04-25T10:11:00.138090+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:55.089613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157914+0000","last_change":"2024-04-25T10:11:00.150439+0000","last_active":"2024-04-25T10:11:00.157914+0000","last_peered":"2024-04-25T10:11:00.157914+0000","last_clean":"2024-04-25T10:11:00.157914+0000","last_became_active":"2024-04-25T10:11:00.150112+0000","last_became_peered":"2024-04-25T10:11:00.150112+0000","last_unstale":"2024-04-25T10:11:00.157914+0000","last_undegraded":"2024-04-25T10:11:00.157914+0000","last_fullsized":"2024-04-25T10:11:00.157914+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:40.806266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.656934+0000","last_change":"2024-04-25T10:11:00.161062+0000","last_active":"2024-04-25T10:11:02.656934+0000","last_peered":"2024-04-25T10:11:02.656934+0000","last_clean":"2024-04-25T10:11:02.656934+0000","last_became_active":"2024-04-25T10:11:00.160648+0000","last_became_peered":"2024-04-25T10:11:00.160648+0000","last_unstale":"2024-04-25T10:11:02.656934+0000","last_undegraded":"2024-04-25T10:11:02.656934+0000","last_fullsized":"2024-04-25T10:11:02.656934+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:55.238359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157765+0000","last_change":"2024-04-25T10:11:00.153014+0000","last_active":"2024-04-25T10:11:00.157765+0000","last_peered":"2024-04-25T10:11:00.157765+0000","last_clean":"2024-04-25T10:11:00.157765+0000","last_became_active":"2024-04-25T10:11:00.152679+0000","last_became_peered":"2024-04-25T10:11:00.152679+0000","last_unstale":"2024-04-25T10:11:00.157765+0000","last_undegraded":"2024-04-25T10:11:00.157765+0000","last_fullsized":"2024-04-25T10:11:00.157765+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.795378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.156741+0000","last_change":"2024-04-25T10:11:00.155617+0000","last_active":"2024-04-25T10:11:00.156741+0000","last_peered":"2024-04-25T10:11:00.156741+0000","last_clean":"2024-04-25T10:11:00.156741+0000","last_became_active":"2024-04-25T10:11:00.155328+0000","last_became_peered":"2024-04-25T10:11:00.155328+0000","last_unstale":"2024-04-25T10:11:00.156741+0000","last_undegraded":"2024-04-25T10:11:00.156741+0000","last_fullsized":"2024-04-25T10:11:00.156741+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:51.933926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157859+0000","last_change":"2024-04-25T10:11:00.151161+0000","last_active":"2024-04-25T10:11:00.157859+0000","last_peered":"2024-04-25T10:11:00.157859+0000","last_clean":"2024-04-25T10:11:00.157859+0000","last_became_active":"2024-04-25T10:11:00.150875+0000","last_became_peered":"2024-04-25T10:11:00.150875+0000","last_unstale":"2024-04-25T10:11:00.157859+0000","last_undegraded":"2024-04-25T10:11:00.157859+0000","last_fullsized":"2024-04-25T10:11:00.157859+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:35.147741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.164646+0000","last_change":"2024-04-25T10:11:00.160614+0000","last_active":"2024-04-25T10:11:00.164646+0000","last_peered":"2024-04-25T10:11:00.164646+0000","last_clean":"2024-04-25T10:11:00.164646+0000","last_became_active":"2024-04-25T10:11:00.160239+0000","last_became_peered":"2024-04-25T10:11:00.160239+0000","last_unstale":"2024-04-25T10:11:00.164646+0000","last_undegraded":"2024-04-25T10:11:00.164646+0000","last_fullsized":"2024-04-25T10:11:00.164646+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:15.501374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.137842+0000","last_change":"2024-04-25T10:11:00.133296+0000","last_active":"2024-04-25T10:11:00.137842+0000","last_peered":"2024-04-25T10:11:00.137842+0000","last_clean":"2024-04-25T10:11:00.137842+0000","last_became_active":"2024-04-25T10:11:00.132946+0000","last_became_peered":"2024-04-25T10:11:00.132946+0000","last_unstale":"2024-04-25T10:11:00.137842+0000","last_undegraded":"2024-04-25T10:11:00.137842+0000","last_fullsized":"2024-04-25T10:11:00.137842+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:09.127558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.657924+0000","last_change":"2024-04-25T10:11:00.133659+0000","last_active":"2024-04-25T10:11:02.657924+0000","last_peered":"2024-04-25T10:11:02.657924+0000","last_clean":"2024-04-25T10:11:02.657924+0000","last_became_active":"2024-04-25T10:11:00.133359+0000","last_became_peered":"2024-04-25T10:11:00.133359+0000","last_unstale":"2024-04-25T10:11:02.657924+0000","last_undegraded":"2024-04-25T10:11:02.657924+0000","last_fullsized":"2024-04-25T10:11:02.657924+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:44.613617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161578+0000","last_change":"2024-04-25T10:11:00.156924+0000","last_active":"2024-04-25T10:11:00.161578+0000","last_peered":"2024-04-25T10:11:00.161578+0000","last_clean":"2024-04-25T10:11:00.161578+0000","last_became_active":"2024-04-25T10:11:00.156508+0000","last_became_peered":"2024-04-25T10:11:00.156508+0000","last_unstale":"2024-04-25T10:11:00.161578+0000","last_undegraded":"2024-04-25T10:11:00.161578+0000","last_fullsized":"2024-04-25T10:11:00.161578+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:16.312678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.147390+0000","last_change":"2024-04-25T10:11:00.142496+0000","last_active":"2024-04-25T10:11:00.147390+0000","last_peered":"2024-04-25T10:11:00.147390+0000","last_clean":"2024-04-25T10:11:00.147390+0000","last_became_active":"2024-04-25T10:11:00.142186+0000","last_became_peered":"2024-04-25T10:11:00.142186+0000","last_unstale":"2024-04-25T10:11:00.147390+0000","last_undegraded":"2024-04-25T10:11:00.147390+0000","last_fullsized":"2024-04-25T10:11:00.147390+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:02.795378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.167269+0000","last_change":"2024-04-25T10:11:00.157922+0000","last_active":"2024-04-25T10:11:00.167269+0000","last_peered":"2024-04-25T10:11:00.167269+0000","last_clean":"2024-04-25T10:11:00.167269+0000","last_became_active":"2024-04-25T10:11:00.157628+0000","last_became_peered":"2024-04-25T10:11:00.157628+0000","last_unstale":"2024-04-25T10:11:00.167269+0000","last_undegraded":"2024-04-25T10:11:00.167269+0000","last_fullsized":"2024-04-25T10:11:00.167269+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:18.146132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155761+0000","last_change":"2024-04-25T10:11:00.152808+0000","last_active":"2024-04-25T10:11:00.155761+0000","last_peered":"2024-04-25T10:11:00.155761+0000","last_clean":"2024-04-25T10:11:00.155761+0000","last_became_active":"2024-04-25T10:11:00.152486+0000","last_became_peered":"2024-04-25T10:11:00.152486+0000","last_unstale":"2024-04-25T10:11:00.155761+0000","last_undegraded":"2024-04-25T10:11:00.155761+0000","last_fullsized":"2024-04-25T10:11:00.155761+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:01.286881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.149004+0000","last_change":"2024-04-25T10:11:00.145699+0000","last_active":"2024-04-25T10:11:00.149004+0000","last_peered":"2024-04-25T10:11:00.149004+0000","last_clean":"2024-04-25T10:11:00.149004+0000","last_became_active":"2024-04-25T10:11:00.145381+0000","last_became_peered":"2024-04-25T10:11:00.145381+0000","last_unstale":"2024-04-25T10:11:00.149004+0000","last_undegraded":"2024-04-25T10:11:00.149004+0000","last_fullsized":"2024-04-25T10:11:00.149004+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:13.969166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161706+0000","last_change":"2024-04-25T10:11:00.160506+0000","last_active":"2024-04-25T10:11:00.161706+0000","last_peered":"2024-04-25T10:11:00.161706+0000","last_clean":"2024-04-25T10:11:00.161706+0000","last_became_active":"2024-04-25T10:11:00.159884+0000","last_became_peered":"2024-04-25T10:11:00.159884+0000","last_unstale":"2024-04-25T10:11:00.161706+0000","last_undegraded":"2024-04-25T10:11:00.161706+0000","last_fullsized":"2024-04-25T10:11:00.161706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:52.819431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161656+0000","last_change":"2024-04-25T10:11:00.158332+0000","last_active":"2024-04-25T10:11:00.161656+0000","last_peered":"2024-04-25T10:11:00.161656+0000","last_clean":"2024-04-25T10:11:00.161656+0000","last_became_active":"2024-04-25T10:11:00.157995+0000","last_became_peered":"2024-04-25T10:11:00.157995+0000","last_unstale":"2024-04-25T10:11:00.161656+0000","last_undegraded":"2024-04-25T10:11:00.161656+0000","last_fullsized":"2024-04-25T10:11:00.161656+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:19.322670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161321+0000","last_change":"2024-04-25T10:11:00.155073+0000","last_active":"2024-04-25T10:11:00.161321+0000","last_peered":"2024-04-25T10:11:00.161321+0000","last_clean":"2024-04-25T10:11:00.161321+0000","last_became_active":"2024-04-25T10:11:00.154740+0000","last_became_peered":"2024-04-25T10:11:00.154740+0000","last_unstale":"2024-04-25T10:11:00.161321+0000","last_undegraded":"2024-04-25T10:11:00.161321+0000","last_fullsized":"2024-04-25T10:11:00.161321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:24.194477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157938+0000","last_change":"2024-04-25T10:11:00.154427+0000","last_active":"2024-04-25T10:11:00.157938+0000","last_peered":"2024-04-25T10:11:00.157938+0000","last_clean":"2024-04-25T10:11:00.157938+0000","last_became_active":"2024-04-25T10:11:00.154085+0000","last_became_peered":"2024-04-25T10:11:00.154085+0000","last_unstale":"2024-04-25T10:11:00.157938+0000","last_undegraded":"2024-04-25T10:11:00.157938+0000","last_fullsized":"2024-04-25T10:11:00.157938+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:56.904985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.166857+0000","last_change":"2024-04-25T10:11:00.162836+0000","last_active":"2024-04-25T10:11:00.166857+0000","last_peered":"2024-04-25T10:11:00.166857+0000","last_clean":"2024-04-25T10:11:00.166857+0000","last_became_active":"2024-04-25T10:11:00.162437+0000","last_became_peered":"2024-04-25T10:11:00.162437+0000","last_unstale":"2024-04-25T10:11:00.166857+0000","last_undegraded":"2024-04-25T10:11:00.166857+0000","last_fullsized":"2024-04-25T10:11:00.166857+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:01.849997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.153706+0000","last_change":"2024-04-25T10:11:00.147790+0000","last_active":"2024-04-25T10:11:00.153706+0000","last_peered":"2024-04-25T10:11:00.153706+0000","last_clean":"2024-04-25T10:11:00.153706+0000","last_became_active":"2024-04-25T10:11:00.141161+0000","last_became_peered":"2024-04-25T10:11:00.141161+0000","last_unstale":"2024-04-25T10:11:00.153706+0000","last_undegraded":"2024-04-25T10:11:00.153706+0000","last_fullsized":"2024-04-25T10:11:00.153706+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:14.906111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157390+0000","last_change":"2024-04-25T10:11:00.145300+0000","last_active":"2024-04-25T10:11:00.157390+0000","last_peered":"2024-04-25T10:11:00.157390+0000","last_clean":"2024-04-25T10:11:00.157390+0000","last_became_active":"2024-04-25T10:11:00.144145+0000","last_became_peered":"2024-04-25T10:11:00.144145+0000","last_unstale":"2024-04-25T10:11:00.157390+0000","last_undegraded":"2024-04-25T10:11:00.157390+0000","last_fullsized":"2024-04-25T10:11:00.157390+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:30.767315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.144375+0000","last_change":"2024-04-25T10:11:00.141264+0000","last_active":"2024-04-25T10:11:00.144375+0000","last_peered":"2024-04-25T10:11:00.144375+0000","last_clean":"2024-04-25T10:11:00.144375+0000","last_became_active":"2024-04-25T10:11:00.140864+0000","last_became_peered":"2024-04-25T10:11:00.140864+0000","last_unstale":"2024-04-25T10:11:00.144375+0000","last_undegraded":"2024-04-25T10:11:00.144375+0000","last_fullsized":"2024-04-25T10:11:00.144375+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:35.595360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.164616+0000","last_change":"2024-04-25T10:11:00.160083+0000","last_active":"2024-04-25T10:11:00.164616+0000","last_peered":"2024-04-25T10:11:00.164616+0000","last_clean":"2024-04-25T10:11:00.164616+0000","last_became_active":"2024-04-25T10:11:00.159686+0000","last_became_peered":"2024-04-25T10:11:00.159686+0000","last_unstale":"2024-04-25T10:11:00.164616+0000","last_undegraded":"2024-04-25T10:11:00.164616+0000","last_fullsized":"2024-04-25T10:11:00.164616+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:55.102580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"104'4","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.173180+0000","last_change":"2024-04-25T10:10:59.143990+0000","last_active":"2024-04-25T10:11:00.173180+0000","last_peered":"2024-04-25T10:11:00.173180+0000","last_clean":"2024-04-25T10:11:00.173180+0000","last_became_active":"2024-04-25T10:10:59.143699+0000","last_became_peered":"2024-04-25T10:10:59.143699+0000","last_unstale":"2024-04-25T10:11:00.173180+0000","last_undegraded":"2024-04-25T10:11:00.173180+0000","last_fullsized":"2024-04-25T10:11:00.173180+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:04.645902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155787+0000","last_change":"2024-04-25T10:11:00.152888+0000","last_active":"2024-04-25T10:11:00.155787+0000","last_peered":"2024-04-25T10:11:00.155787+0000","last_clean":"2024-04-25T10:11:00.155787+0000","last_became_active":"2024-04-25T10:11:00.152463+0000","last_became_peered":"2024-04-25T10:11:00.152463+0000","last_unstale":"2024-04-25T10:11:00.155787+0000","last_undegraded":"2024-04-25T10:11:00.155787+0000","last_fullsized":"2024-04-25T10:11:00.155787+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:53.632212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.173682+0000","last_change":"2024-04-25T10:10:59.131435+0000","last_active":"2024-04-25T10:11:00.173682+0000","last_peered":"2024-04-25T10:11:00.173682+0000","last_clean":"2024-04-25T10:11:00.173682+0000","last_became_active":"2024-04-25T10:10:59.131097+0000","last_became_peered":"2024-04-25T10:10:59.131097+0000","last_unstale":"2024-04-25T10:11:00.173682+0000","last_undegraded":"2024-04-25T10:11:00.173682+0000","last_fullsized":"2024-04-25T10:11:00.173682+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.751772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.144006+0000","last_change":"2024-04-25T10:11:00.139385+0000","last_active":"2024-04-25T10:11:00.144006+0000","last_peered":"2024-04-25T10:11:00.144006+0000","last_clean":"2024-04-25T10:11:00.144006+0000","last_became_active":"2024-04-25T10:11:00.139075+0000","last_became_peered":"2024-04-25T10:11:00.139075+0000","last_unstale":"2024-04-25T10:11:00.144006+0000","last_undegraded":"2024-04-25T10:11:00.144006+0000","last_fullsized":"2024-04-25T10:11:00.144006+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:51.395829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176816+0000","last_change":"2024-04-25T10:10:59.128480+0000","last_active":"2024-04-25T10:11:00.176816+0000","last_peered":"2024-04-25T10:11:00.176816+0000","last_clean":"2024-04-25T10:11:00.176816+0000","last_became_active":"2024-04-25T10:10:59.128031+0000","last_became_peered":"2024-04-25T10:10:59.128031+0000","last_unstale":"2024-04-25T10:11:00.176816+0000","last_undegraded":"2024-04-25T10:11:00.176816+0000","last_fullsized":"2024-04-25T10:11:00.176816+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:35.765834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.163438+0000","last_change":"2024-04-25T10:11:00.159359+0000","last_active":"2024-04-25T10:11:00.163438+0000","last_peered":"2024-04-25T10:11:00.163438+0000","last_clean":"2024-04-25T10:11:00.163438+0000","last_became_active":"2024-04-25T10:11:00.157853+0000","last_became_peered":"2024-04-25T10:11:00.157853+0000","last_unstale":"2024-04-25T10:11:00.163438+0000","last_undegraded":"2024-04-25T10:11:00.163438+0000","last_fullsized":"2024-04-25T10:11:00.163438+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:16.577842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.173631+0000","last_change":"2024-04-25T10:10:59.138124+0000","last_active":"2024-04-25T10:11:00.173631+0000","last_peered":"2024-04-25T10:11:00.173631+0000","last_clean":"2024-04-25T10:11:00.173631+0000","last_became_active":"2024-04-25T10:10:59.137742+0000","last_became_peered":"2024-04-25T10:10:59.137742+0000","last_unstale":"2024-04-25T10:11:00.173631+0000","last_undegraded":"2024-04-25T10:11:00.173631+0000","last_fullsized":"2024-04-25T10:11:00.173631+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:14.485830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.657737+0000","last_change":"2024-04-25T10:11:00.143212+0000","last_active":"2024-04-25T10:11:02.657737+0000","last_peered":"2024-04-25T10:11:02.657737+0000","last_clean":"2024-04-25T10:11:02.657737+0000","last_became_active":"2024-04-25T10:11:00.142724+0000","last_became_peered":"2024-04-25T10:11:00.142724+0000","last_unstale":"2024-04-25T10:11:02.657737+0000","last_undegraded":"2024-04-25T10:11:02.657737+0000","last_fullsized":"2024-04-25T10:11:02.657737+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:13.814444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176038+0000","last_change":"2024-04-25T10:10:59.134522+0000","last_active":"2024-04-25T10:11:00.176038+0000","last_peered":"2024-04-25T10:11:00.176038+0000","last_clean":"2024-04-25T10:11:00.176038+0000","last_became_active":"2024-04-25T10:10:59.134236+0000","last_became_peered":"2024-04-25T10:10:59.134236+0000","last_unstale":"2024-04-25T10:11:00.176038+0000","last_undegraded":"2024-04-25T10:11:00.176038+0000","last_fullsized":"2024-04-25T10:11:00.176038+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:02.935693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.658548+0000","last_change":"2024-04-25T10:11:00.161274+0000","last_active":"2024-04-25T10:11:02.658548+0000","last_peered":"2024-04-25T10:11:02.658548+0000","last_clean":"2024-04-25T10:11:02.658548+0000","last_became_active":"2024-04-25T10:11:00.160960+0000","last_became_peered":"2024-04-25T10:11:00.160960+0000","last_unstale":"2024-04-25T10:11:02.658548+0000","last_undegraded":"2024-04-25T10:11:02.658548+0000","last_fullsized":"2024-04-25T10:11:02.658548+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:41.240009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176414+0000","last_change":"2024-04-25T10:10:59.130166+0000","last_active":"2024-04-25T10:11:00.176414+0000","last_peered":"2024-04-25T10:11:00.176414+0000","last_clean":"2024-04-25T10:11:00.176414+0000","last_became_active":"2024-04-25T10:10:59.129889+0000","last_became_peered":"2024-04-25T10:10:59.129889+0000","last_unstale":"2024-04-25T10:11:00.176414+0000","last_undegraded":"2024-04-25T10:11:00.176414+0000","last_fullsized":"2024-04-25T10:11:00.176414+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:43.451259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.150890+0000","last_change":"2024-04-25T10:11:00.144604+0000","last_active":"2024-04-25T10:11:00.150890+0000","last_peered":"2024-04-25T10:11:00.150890+0000","last_clean":"2024-04-25T10:11:00.150890+0000","last_became_active":"2024-04-25T10:11:00.144322+0000","last_became_peered":"2024-04-25T10:11:00.144322+0000","last_unstale":"2024-04-25T10:11:00.150890+0000","last_undegraded":"2024-04-25T10:11:00.150890+0000","last_fullsized":"2024-04-25T10:11:00.150890+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:34.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.146134+0000","last_change":"2024-04-25T10:10:59.134683+0000","last_active":"2024-04-25T10:11:00.146134+0000","last_peered":"2024-04-25T10:11:00.146134+0000","last_clean":"2024-04-25T10:11:00.146134+0000","last_became_active":"2024-04-25T10:10:59.134304+0000","last_became_peered":"2024-04-25T10:10:59.134304+0000","last_unstale":"2024-04-25T10:11:00.146134+0000","last_undegraded":"2024-04-25T10:11:00.146134+0000","last_fullsized":"2024-04-25T10:11:00.146134+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:55.037750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.658662+0000","last_change":"2024-04-25T10:11:00.143289+0000","last_active":"2024-04-25T10:11:02.658662+0000","last_peered":"2024-04-25T10:11:02.658662+0000","last_clean":"2024-04-25T10:11:02.658662+0000","last_became_active":"2024-04-25T10:11:00.142461+0000","last_became_peered":"2024-04-25T10:11:00.142461+0000","last_unstale":"2024-04-25T10:11:02.658662+0000","last_undegraded":"2024-04-25T10:11:02.658662+0000","last_fullsized":"2024-04-25T10:11:02.658662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:23.025742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.125656+0000","last_change":"2024-04-25T10:10:59.140846+0000","last_active":"2024-04-25T10:11:00.125656+0000","last_peered":"2024-04-25T10:11:00.125656+0000","last_clean":"2024-04-25T10:11:00.125656+0000","last_became_active":"2024-04-25T10:10:59.140452+0000","last_became_peered":"2024-04-25T10:10:59.140452+0000","last_unstale":"2024-04-25T10:11:00.125656+0000","last_undegraded":"2024-04-25T10:11:00.125656+0000","last_fullsized":"2024-04-25T10:11:00.125656+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:13.229979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.655552+0000","last_change":"2024-04-25T10:11:00.159095+0000","last_active":"2024-04-25T10:11:02.655552+0000","last_peered":"2024-04-25T10:11:02.655552+0000","last_clean":"2024-04-25T10:11:02.655552+0000","last_became_active":"2024-04-25T10:11:00.158804+0000","last_became_peered":"2024-04-25T10:11:00.158804+0000","last_unstale":"2024-04-25T10:11:02.655552+0000","last_undegraded":"2024-04-25T10:11:02.655552+0000","last_fullsized":"2024-04-25T10:11:02.655552+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:37.963682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.125985+0000","last_change":"2024-04-25T10:10:59.138157+0000","last_active":"2024-04-25T10:11:00.125985+0000","last_peered":"2024-04-25T10:11:00.125985+0000","last_clean":"2024-04-25T10:11:00.125985+0000","last_became_active":"2024-04-25T10:10:59.136049+0000","last_became_peered":"2024-04-25T10:10:59.136049+0000","last_unstale":"2024-04-25T10:11:00.125985+0000","last_undegraded":"2024-04-25T10:11:00.125985+0000","last_fullsized":"2024-04-25T10:11:00.125985+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.178933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.656512+0000","last_change":"2024-04-25T10:11:00.140248+0000","last_active":"2024-04-25T10:11:02.656512+0000","last_peered":"2024-04-25T10:11:02.656512+0000","last_clean":"2024-04-25T10:11:02.656512+0000","last_became_active":"2024-04-25T10:11:00.139930+0000","last_became_peered":"2024-04-25T10:11:00.139930+0000","last_unstale":"2024-04-25T10:11:02.656512+0000","last_undegraded":"2024-04-25T10:11:02.656512+0000","last_fullsized":"2024-04-25T10:11:02.656512+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:48.638183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.136009+0000","last_change":"2024-04-25T10:10:59.145311+0000","last_active":"2024-04-25T10:11:00.136009+0000","last_peered":"2024-04-25T10:11:00.136009+0000","last_clean":"2024-04-25T10:11:00.136009+0000","last_became_active":"2024-04-25T10:10:59.144909+0000","last_became_peered":"2024-04-25T10:10:59.144909+0000","last_unstale":"2024-04-25T10:11:00.136009+0000","last_undegraded":"2024-04-25T10:11:00.136009+0000","last_fullsized":"2024-04-25T10:11:00.136009+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:54.087037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.148966+0000","last_change":"2024-04-25T10:11:00.145435+0000","last_active":"2024-04-25T10:11:00.148966+0000","last_peered":"2024-04-25T10:11:00.148966+0000","last_clean":"2024-04-25T10:11:00.148966+0000","last_became_active":"2024-04-25T10:11:00.145119+0000","last_became_peered":"2024-04-25T10:11:00.145119+0000","last_unstale":"2024-04-25T10:11:00.148966+0000","last_undegraded":"2024-04-25T10:11:00.148966+0000","last_fullsized":"2024-04-25T10:11:00.148966+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:47.947899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"104'3","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176824+0000","last_change":"2024-04-25T10:10:59.136845+0000","last_active":"2024-04-25T10:11:00.176824+0000","last_peered":"2024-04-25T10:11:00.176824+0000","last_clean":"2024-04-25T10:11:00.176824+0000","last_became_active":"2024-04-25T10:10:59.136562+0000","last_became_peered":"2024-04-25T10:10:59.136562+0000","last_unstale":"2024-04-25T10:11:00.176824+0000","last_undegraded":"2024-04-25T10:11:00.176824+0000","last_fullsized":"2024-04-25T10:11:00.176824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:34.132198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155825+0000","last_change":"2024-04-25T10:11:00.150168+0000","last_active":"2024-04-25T10:11:00.155825+0000","last_peered":"2024-04-25T10:11:00.155825+0000","last_clean":"2024-04-25T10:11:00.155825+0000","last_became_active":"2024-04-25T10:11:00.149765+0000","last_became_peered":"2024-04-25T10:11:00.149765+0000","last_unstale":"2024-04-25T10:11:00.155825+0000","last_undegraded":"2024-04-25T10:11:00.155825+0000","last_fullsized":"2024-04-25T10:11:00.155825+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:58.517016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"104'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.180164+0000","last_change":"2024-04-25T10:10:59.135798+0000","last_active":"2024-04-25T10:11:00.180164+0000","last_peered":"2024-04-25T10:11:00.180164+0000","last_clean":"2024-04-25T10:11:00.180164+0000","last_became_active":"2024-04-25T10:10:59.135518+0000","last_became_peered":"2024-04-25T10:10:59.135518+0000","last_unstale":"2024-04-25T10:11:00.180164+0000","last_undegraded":"2024-04-25T10:11:00.180164+0000","last_fullsized":"2024-04-25T10:11:00.180164+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:20.159764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.137928+0000","last_change":"2024-04-25T10:11:00.133469+0000","last_active":"2024-04-25T10:11:00.137928+0000","last_peered":"2024-04-25T10:11:00.137928+0000","last_clean":"2024-04-25T10:11:00.137928+0000","last_became_active":"2024-04-25T10:11:00.133134+0000","last_became_peered":"2024-04-25T10:11:00.133134+0000","last_unstale":"2024-04-25T10:11:00.137928+0000","last_undegraded":"2024-04-25T10:11:00.137928+0000","last_fullsized":"2024-04-25T10:11:00.137928+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:09.064575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.657266+0000","last_change":"2024-04-25T10:10:59.138623+0000","last_active":"2024-04-25T10:11:02.657266+0000","last_peered":"2024-04-25T10:11:02.657266+0000","last_clean":"2024-04-25T10:11:02.657266+0000","last_became_active":"2024-04-25T10:10:59.138336+0000","last_became_peered":"2024-04-25T10:10:59.138336+0000","last_unstale":"2024-04-25T10:11:02.657266+0000","last_undegraded":"2024-04-25T10:11:02.657266+0000","last_fullsized":"2024-04-25T10:11:02.657266+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:31.226148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.149065+0000","last_change":"2024-04-25T10:11:00.145458+0000","last_active":"2024-04-25T10:11:00.149065+0000","last_peered":"2024-04-25T10:11:00.149065+0000","last_clean":"2024-04-25T10:11:00.149065+0000","last_became_active":"2024-04-25T10:11:00.145153+0000","last_became_peered":"2024-04-25T10:11:00.145153+0000","last_unstale":"2024-04-25T10:11:00.149065+0000","last_undegraded":"2024-04-25T10:11:00.149065+0000","last_fullsized":"2024-04-25T10:11:00.149065+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:18.146132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.134320+0000","last_change":"2024-04-25T10:10:59.144843+0000","last_active":"2024-04-25T10:11:00.134320+0000","last_peered":"2024-04-25T10:11:00.134320+0000","last_clean":"2024-04-25T10:11:00.134320+0000","last_became_active":"2024-04-25T10:10:59.144558+0000","last_became_peered":"2024-04-25T10:10:59.144558+0000","last_unstale":"2024-04-25T10:11:00.134320+0000","last_undegraded":"2024-04-25T10:11:00.134320+0000","last_fullsized":"2024-04-25T10:11:00.134320+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:55.889441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.167198+0000","last_change":"2024-04-25T10:11:00.157051+0000","last_active":"2024-04-25T10:11:00.167198+0000","last_peered":"2024-04-25T10:11:00.167198+0000","last_clean":"2024-04-25T10:11:00.167198+0000","last_became_active":"2024-04-25T10:11:00.156775+0000","last_became_peered":"2024-04-25T10:11:00.156775+0000","last_unstale":"2024-04-25T10:11:00.167198+0000","last_undegraded":"2024-04-25T10:11:00.167198+0000","last_fullsized":"2024-04-25T10:11:00.167198+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:51.395829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.655967+0000","last_change":"2024-04-25T10:10:59.142122+0000","last_active":"2024-04-25T10:11:02.655967+0000","last_peered":"2024-04-25T10:11:02.655967+0000","last_clean":"2024-04-25T10:11:02.655967+0000","last_became_active":"2024-04-25T10:10:59.141675+0000","last_became_peered":"2024-04-25T10:10:59.141675+0000","last_unstale":"2024-04-25T10:11:02.655967+0000","last_undegraded":"2024-04-25T10:11:02.655967+0000","last_fullsized":"2024-04-25T10:11:02.655967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:29.958209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.164455+0000","last_change":"2024-04-25T10:11:00.160146+0000","last_active":"2024-04-25T10:11:00.164455+0000","last_peered":"2024-04-25T10:11:00.164455+0000","last_clean":"2024-04-25T10:11:00.164455+0000","last_became_active":"2024-04-25T10:11:00.159785+0000","last_became_peered":"2024-04-25T10:11:00.159785+0000","last_unstale":"2024-04-25T10:11:00.164455+0000","last_undegraded":"2024-04-25T10:11:00.164455+0000","last_fullsized":"2024-04-25T10:11:00.164455+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:56.904985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"104'8","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.177179+0000","last_change":"2024-04-25T10:10:59.129505+0000","last_active":"2024-04-25T10:11:00.177179+0000","last_peered":"2024-04-25T10:11:00.177179+0000","last_clean":"2024-04-25T10:11:00.177179+0000","last_became_active":"2024-04-25T10:10:59.129216+0000","last_became_peered":"2024-04-25T10:10:59.129216+0000","last_unstale":"2024-04-25T10:11:00.177179+0000","last_undegraded":"2024-04-25T10:11:00.177179+0000","last_fullsized":"2024-04-25T10:11:00.177179+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:37.060083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155814+0000","last_change":"2024-04-25T10:11:00.152775+0000","last_active":"2024-04-25T10:11:00.155814+0000","last_peered":"2024-04-25T10:11:00.155814+0000","last_clean":"2024-04-25T10:11:00.155814+0000","last_became_active":"2024-04-25T10:11:00.152437+0000","last_became_peered":"2024-04-25T10:11:00.152437+0000","last_unstale":"2024-04-25T10:11:00.155814+0000","last_undegraded":"2024-04-25T10:11:00.155814+0000","last_fullsized":"2024-04-25T10:11:00.155814+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:57.413264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.172691+0000","last_change":"2024-04-25T10:10:59.140294+0000","last_active":"2024-04-25T10:11:00.172691+0000","last_peered":"2024-04-25T10:11:00.172691+0000","last_clean":"2024-04-25T10:11:00.172691+0000","last_became_active":"2024-04-25T10:10:59.138679+0000","last_became_peered":"2024-04-25T10:10:59.138679+0000","last_unstale":"2024-04-25T10:11:00.172691+0000","last_undegraded":"2024-04-25T10:11:00.172691+0000","last_fullsized":"2024-04-25T10:11:00.172691+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:15.297135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161782+0000","last_change":"2024-04-25T10:11:00.154733+0000","last_active":"2024-04-25T10:11:00.161782+0000","last_peered":"2024-04-25T10:11:00.161782+0000","last_clean":"2024-04-25T10:11:00.161782+0000","last_became_active":"2024-04-25T10:11:00.154310+0000","last_became_peered":"2024-04-25T10:11:00.154310+0000","last_unstale":"2024-04-25T10:11:00.161782+0000","last_undegraded":"2024-04-25T10:11:00.161782+0000","last_fullsized":"2024-04-25T10:11:00.161782+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:04.165775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.175794+0000","last_change":"2024-04-25T10:10:59.135411+0000","last_active":"2024-04-25T10:11:00.175794+0000","last_peered":"2024-04-25T10:11:00.175794+0000","last_clean":"2024-04-25T10:11:00.175794+0000","last_became_active":"2024-04-25T10:10:59.135122+0000","last_became_peered":"2024-04-25T10:10:59.135122+0000","last_unstale":"2024-04-25T10:11:00.175794+0000","last_undegraded":"2024-04-25T10:11:00.175794+0000","last_fullsized":"2024-04-25T10:11:00.175794+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:25.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.161944+0000","last_change":"2024-04-25T10:11:00.154205+0000","last_active":"2024-04-25T10:11:00.161944+0000","last_peered":"2024-04-25T10:11:00.161944+0000","last_clean":"2024-04-25T10:11:00.161944+0000","last_became_active":"2024-04-25T10:11:00.153884+0000","last_became_peered":"2024-04-25T10:11:00.153884+0000","last_unstale":"2024-04-25T10:11:00.161944+0000","last_undegraded":"2024-04-25T10:11:00.161944+0000","last_fullsized":"2024-04-25T10:11:00.161944+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:15.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.125479+0000","last_change":"2024-04-25T10:10:59.134710+0000","last_active":"2024-04-25T10:11:00.125479+0000","last_peered":"2024-04-25T10:11:00.125479+0000","last_clean":"2024-04-25T10:11:00.125479+0000","last_became_active":"2024-04-25T10:10:59.134413+0000","last_became_peered":"2024-04-25T10:10:59.134413+0000","last_unstale":"2024-04-25T10:11:00.125479+0000","last_undegraded":"2024-04-25T10:11:00.125479+0000","last_fullsized":"2024-04-25T10:11:00.125479+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:03.150231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.657627+0000","last_change":"2024-04-25T10:11:00.142978+0000","last_active":"2024-04-25T10:11:02.657627+0000","last_peered":"2024-04-25T10:11:02.657627+0000","last_clean":"2024-04-25T10:11:02.657627+0000","last_became_active":"2024-04-25T10:11:00.142360+0000","last_became_peered":"2024-04-25T10:11:00.142360+0000","last_unstale":"2024-04-25T10:11:02.657627+0000","last_undegraded":"2024-04-25T10:11:02.657627+0000","last_fullsized":"2024-04-25T10:11:02.657627+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:59.613539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.175967+0000","last_change":"2024-04-25T10:10:59.141774+0000","last_active":"2024-04-25T10:11:00.175967+0000","last_peered":"2024-04-25T10:11:00.175967+0000","last_clean":"2024-04-25T10:11:00.175967+0000","last_became_active":"2024-04-25T10:10:59.141413+0000","last_became_peered":"2024-04-25T10:10:59.141413+0000","last_unstale":"2024-04-25T10:11:00.175967+0000","last_undegraded":"2024-04-25T10:11:00.175967+0000","last_fullsized":"2024-04-25T10:11:00.175967+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:15.297135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.149430+0000","last_change":"2024-04-25T10:11:00.144994+0000","last_active":"2024-04-25T10:11:00.149430+0000","last_peered":"2024-04-25T10:11:00.149430+0000","last_clean":"2024-04-25T10:11:00.149430+0000","last_became_active":"2024-04-25T10:11:00.144706+0000","last_became_peered":"2024-04-25T10:11:00.144706+0000","last_unstale":"2024-04-25T10:11:00.149430+0000","last_undegraded":"2024-04-25T10:11:00.149430+0000","last_fullsized":"2024-04-25T10:11:00.149430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:09.127558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.135534+0000","last_change":"2024-04-25T10:10:59.131823+0000","last_active":"2024-04-25T10:11:00.135534+0000","last_peered":"2024-04-25T10:11:00.135534+0000","last_clean":"2024-04-25T10:11:00.135534+0000","last_became_active":"2024-04-25T10:10:59.131437+0000","last_became_peered":"2024-04-25T10:10:59.131437+0000","last_unstale":"2024-04-25T10:11:00.135534+0000","last_undegraded":"2024-04-25T10:11:00.135534+0000","last_fullsized":"2024-04-25T10:11:00.135534+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:18.307126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.155821+0000","last_change":"2024-04-25T10:11:00.149678+0000","last_active":"2024-04-25T10:11:00.155821+0000","last_peered":"2024-04-25T10:11:00.155821+0000","last_clean":"2024-04-25T10:11:00.155821+0000","last_became_active":"2024-04-25T10:11:00.149313+0000","last_became_peered":"2024-04-25T10:11:00.149313+0000","last_unstale":"2024-04-25T10:11:00.155821+0000","last_undegraded":"2024-04-25T10:11:00.155821+0000","last_fullsized":"2024-04-25T10:11:00.155821+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:09.064575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.124809+0000","last_change":"2024-04-25T10:10:59.139124+0000","last_active":"2024-04-25T10:11:00.124809+0000","last_peered":"2024-04-25T10:11:00.124809+0000","last_clean":"2024-04-25T10:11:00.124809+0000","last_became_active":"2024-04-25T10:10:59.138797+0000","last_became_peered":"2024-04-25T10:10:59.138797+0000","last_unstale":"2024-04-25T10:11:00.124809+0000","last_undegraded":"2024-04-25T10:11:00.124809+0000","last_fullsized":"2024-04-25T10:11:00.124809+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:25.181071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.658363+0000","last_change":"2024-04-25T10:11:00.143579+0000","last_active":"2024-04-25T10:11:02.658363+0000","last_peered":"2024-04-25T10:11:02.658363+0000","last_clean":"2024-04-25T10:11:02.658363+0000","last_became_active":"2024-04-25T10:11:00.142814+0000","last_became_peered":"2024-04-25T10:11:00.142814+0000","last_unstale":"2024-04-25T10:11:02.658363+0000","last_undegraded":"2024-04-25T10:11:02.658363+0000","last_fullsized":"2024-04-25T10:11:02.658363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:55.161356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.137457+0000","last_change":"2024-04-25T10:10:59.138623+0000","last_active":"2024-04-25T10:11:00.137457+0000","last_peered":"2024-04-25T10:11:00.137457+0000","last_clean":"2024-04-25T10:11:00.137457+0000","last_became_active":"2024-04-25T10:10:59.138336+0000","last_became_peered":"2024-04-25T10:10:59.138336+0000","last_unstale":"2024-04-25T10:11:00.137457+0000","last_undegraded":"2024-04-25T10:11:00.137457+0000","last_fullsized":"2024-04-25T10:11:00.137457+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:19.064015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.144396+0000","last_change":"2024-04-25T10:11:00.141868+0000","last_active":"2024-04-25T10:11:00.144396+0000","last_peered":"2024-04-25T10:11:00.144396+0000","last_clean":"2024-04-25T10:11:00.144396+0000","last_became_active":"2024-04-25T10:11:00.141601+0000","last_became_peered":"2024-04-25T10:11:00.141601+0000","last_unstale":"2024-04-25T10:11:00.144396+0000","last_undegraded":"2024-04-25T10:11:00.144396+0000","last_fullsized":"2024-04-25T10:11:00.144396+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:36.781377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.176143+0000","last_change":"2024-04-25T10:10:59.146023+0000","last_active":"2024-04-25T10:11:00.176143+0000","last_peered":"2024-04-25T10:11:00.176143+0000","last_clean":"2024-04-25T10:11:00.176143+0000","last_became_active":"2024-04-25T10:10:59.145057+0000","last_became_peered":"2024-04-25T10:10:59.145057+0000","last_unstale":"2024-04-25T10:11:00.176143+0000","last_undegraded":"2024-04-25T10:11:00.176143+0000","last_fullsized":"2024-04-25T10:11:00.176143+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:39.790722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157577+0000","last_change":"2024-04-25T10:11:00.148986+0000","last_active":"2024-04-25T10:11:00.157577+0000","last_peered":"2024-04-25T10:11:00.157577+0000","last_clean":"2024-04-25T10:11:00.157577+0000","last_became_active":"2024-04-25T10:11:00.142173+0000","last_became_peered":"2024-04-25T10:11:00.142173+0000","last_unstale":"2024-04-25T10:11:00.157577+0000","last_undegraded":"2024-04-25T10:11:00.157577+0000","last_fullsized":"2024-04-25T10:11:00.157577+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:53.016466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.136704+0000","last_change":"2024-04-25T10:10:59.139074+0000","last_active":"2024-04-25T10:11:00.136704+0000","last_peered":"2024-04-25T10:11:00.136704+0000","last_clean":"2024-04-25T10:11:00.136704+0000","last_became_active":"2024-04-25T10:10:59.138792+0000","last_became_peered":"2024-04-25T10:10:59.138792+0000","last_unstale":"2024-04-25T10:11:00.136704+0000","last_undegraded":"2024-04-25T10:11:00.136704+0000","last_fullsized":"2024-04-25T10:11:00.136704+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:51.803887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.158468+0000","last_change":"2024-04-25T10:11:00.156802+0000","last_active":"2024-04-25T10:11:00.158468+0000","last_peered":"2024-04-25T10:11:00.158468+0000","last_clean":"2024-04-25T10:11:00.158468+0000","last_became_active":"2024-04-25T10:11:00.156444+0000","last_became_peered":"2024-04-25T10:11:00.156444+0000","last_unstale":"2024-04-25T10:11:00.158468+0000","last_undegraded":"2024-04-25T10:11:00.158468+0000","last_fullsized":"2024-04-25T10:11:00.158468+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:21.936351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.132505+0000","last_change":"2024-04-25T10:10:59.136023+0000","last_active":"2024-04-25T10:11:00.132505+0000","last_peered":"2024-04-25T10:11:00.132505+0000","last_clean":"2024-04-25T10:11:00.132505+0000","last_became_active":"2024-04-25T10:10:59.135691+0000","last_became_peered":"2024-04-25T10:10:59.135691+0000","last_unstale":"2024-04-25T10:11:00.132505+0000","last_undegraded":"2024-04-25T10:11:00.132505+0000","last_fullsized":"2024-04-25T10:11:00.132505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:02.165953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.162387+0000","last_change":"2024-04-25T10:11:00.159090+0000","last_active":"2024-04-25T10:11:00.162387+0000","last_peered":"2024-04-25T10:11:00.162387+0000","last_clean":"2024-04-25T10:11:00.162387+0000","last_became_active":"2024-04-25T10:11:00.157455+0000","last_became_peered":"2024-04-25T10:11:00.157455+0000","last_unstale":"2024-04-25T10:11:00.162387+0000","last_undegraded":"2024-04-25T10:11:00.162387+0000","last_fullsized":"2024-04-25T10:11:00.162387+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:30.767315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.657869+0000","last_change":"2024-04-25T10:10:59.140664+0000","last_active":"2024-04-25T10:11:02.657869+0000","last_peered":"2024-04-25T10:11:02.657869+0000","last_clean":"2024-04-25T10:11:02.657869+0000","last_became_active":"2024-04-25T10:10:59.140358+0000","last_became_peered":"2024-04-25T10:10:59.140358+0000","last_unstale":"2024-04-25T10:11:02.657869+0000","last_undegraded":"2024-04-25T10:11:02.657869+0000","last_fullsized":"2024-04-25T10:11:02.657869+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:18.959470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.657380+0000","last_change":"2024-04-25T10:11:00.140767+0000","last_active":"2024-04-25T10:11:02.657380+0000","last_peered":"2024-04-25T10:11:02.657380+0000","last_clean":"2024-04-25T10:11:02.657380+0000","last_became_active":"2024-04-25T10:11:00.140443+0000","last_became_peered":"2024-04-25T10:11:00.140443+0000","last_unstale":"2024-04-25T10:11:02.657380+0000","last_undegraded":"2024-04-25T10:11:02.657380+0000","last_fullsized":"2024-04-25T10:11:02.657380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.129348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.128833+0000","last_change":"2024-04-25T10:10:59.139722+0000","last_active":"2024-04-25T10:11:00.128833+0000","last_peered":"2024-04-25T10:11:00.128833+0000","last_clean":"2024-04-25T10:11:00.128833+0000","last_became_active":"2024-04-25T10:10:59.139406+0000","last_became_peered":"2024-04-25T10:10:59.139406+0000","last_unstale":"2024-04-25T10:11:00.128833+0000","last_undegraded":"2024-04-25T10:11:00.128833+0000","last_fullsized":"2024-04-25T10:11:00.128833+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:33.839986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.658318+0000","last_change":"2024-04-25T10:11:00.141243+0000","last_active":"2024-04-25T10:11:02.658318+0000","last_peered":"2024-04-25T10:11:02.658318+0000","last_clean":"2024-04-25T10:11:02.658318+0000","last_became_active":"2024-04-25T10:11:00.138874+0000","last_became_peered":"2024-04-25T10:11:00.138874+0000","last_unstale":"2024-04-25T10:11:02.658318+0000","last_undegraded":"2024-04-25T10:11:02.658318+0000","last_fullsized":"2024-04-25T10:11:02.658318+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:50.594153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.136919+0000","last_change":"2024-04-25T10:10:59.138750+0000","last_active":"2024-04-25T10:11:00.136919+0000","last_peered":"2024-04-25T10:11:00.136919+0000","last_clean":"2024-04-25T10:11:00.136919+0000","last_became_active":"2024-04-25T10:10:59.137574+0000","last_became_peered":"2024-04-25T10:10:59.137574+0000","last_unstale":"2024-04-25T10:11:00.136919+0000","last_undegraded":"2024-04-25T10:11:00.136919+0000","last_fullsized":"2024-04-25T10:11:00.136919+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:55.999917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.158677+0000","last_change":"2024-04-25T10:11:00.153550+0000","last_active":"2024-04-25T10:11:00.158677+0000","last_peered":"2024-04-25T10:11:00.158677+0000","last_clean":"2024-04-25T10:11:00.158677+0000","last_became_active":"2024-04-25T10:11:00.150240+0000","last_became_peered":"2024-04-25T10:11:00.150240+0000","last_unstale":"2024-04-25T10:11:00.158677+0000","last_undegraded":"2024-04-25T10:11:00.158677+0000","last_fullsized":"2024-04-25T10:11:00.158677+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.443869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.146291+0000","last_change":"2024-04-25T10:10:59.143574+0000","last_active":"2024-04-25T10:11:00.146291+0000","last_peered":"2024-04-25T10:11:00.146291+0000","last_clean":"2024-04-25T10:11:00.146291+0000","last_became_active":"2024-04-25T10:10:59.143241+0000","last_became_peered":"2024-04-25T10:10:59.143241+0000","last_unstale":"2024-04-25T10:11:00.146291+0000","last_undegraded":"2024-04-25T10:11:00.146291+0000","last_fullsized":"2024-04-25T10:11:00.146291+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:36.300434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.143430+0000","last_change":"2024-04-25T10:11:00.137562+0000","last_active":"2024-04-25T10:11:00.143430+0000","last_peered":"2024-04-25T10:11:00.143430+0000","last_clean":"2024-04-25T10:11:00.143430+0000","last_became_active":"2024-04-25T10:11:00.137219+0000","last_became_peered":"2024-04-25T10:11:00.137219+0000","last_unstale":"2024-04-25T10:11:00.143430+0000","last_undegraded":"2024-04-25T10:11:00.143430+0000","last_fullsized":"2024-04-25T10:11:00.143430+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:58.517016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.133619+0000","last_change":"2024-04-25T10:10:59.124080+0000","last_active":"2024-04-25T10:11:00.133619+0000","last_peered":"2024-04-25T10:11:00.133619+0000","last_clean":"2024-04-25T10:11:00.133619+0000","last_became_active":"2024-04-25T10:10:59.123508+0000","last_became_peered":"2024-04-25T10:10:59.123508+0000","last_unstale":"2024-04-25T10:11:00.133619+0000","last_undegraded":"2024-04-25T10:11:00.133619+0000","last_fullsized":"2024-04-25T10:11:00.133619+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:23.178933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.157873+0000","last_change":"2024-04-25T10:11:00.145753+0000","last_active":"2024-04-25T10:11:00.157873+0000","last_peered":"2024-04-25T10:11:00.157873+0000","last_clean":"2024-04-25T10:11:00.157873+0000","last_became_active":"2024-04-25T10:11:00.145032+0000","last_became_peered":"2024-04-25T10:11:00.145032+0000","last_unstale":"2024-04-25T10:11:00.157873+0000","last_undegraded":"2024-04-25T10:11:00.157873+0000","last_fullsized":"2024-04-25T10:11:00.157873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:59.088607+0000","last_clean_scrub_stamp":"2024-04-25T10:10:59.088607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:57.015460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T10:11:02.656757+0000","last_change":"2024-04-25T10:10:59.146698+0000","last_active":"2024-04-25T10:11:02.656757+0000","last_peered":"2024-04-25T10:11:02.656757+0000","last_clean":"2024-04-25T10:11:02.656757+0000","last_became_active":"2024-04-25T10:10:59.146402+0000","last_became_peered":"2024-04-25T10:10:59.146402+0000","last_unstale":"2024-04-25T10:11:02.656757+0000","last_undegraded":"2024-04-25T10:11:02.656757+0000","last_fullsized":"2024-04-25T10:11:02.656757+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:10:58.073064+0000","last_clean_scrub_stamp":"2024-04-25T10:10:58.073064+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:26.176362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":325,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-04-25T10:11:00.146481+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:11:00.146481+0000","last_peered":"2024-04-25T10:11:00.146481+0000","last_clean":"2024-04-25T10:11:00.146481+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:11:00.146481+0000","last_undegraded":"2024-04-25T10:11:00.146481+0000","last_fullsized":"2024-04-25T10:11:00.146481+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":594052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1282048,"data_stored":593400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1884160,"data_stored":1183892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509555,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1294336,"data_stored":595012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":593512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509553,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1875968,"data_stored":1183768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":593982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542261,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1298432,"data_stored":600126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:05.088 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:05.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.348+0000 7f78bbe00640 1 Processor -- start 2024-04-25T10:11:05.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.348+0000 7f78bbe00640 1 -- start start 2024-04-25T10:11:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78bbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78bc06ec70 0x7f78bc071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78bbe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78bc0ade10 con 0x7f78bc143f30 2024-04-25T10:11:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78bbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78bc0ae0f0 con 0x7f78bc0aebd0 2024-04-25T10:11:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78bbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78bc0ae3d0 con 0x7f78bc06ec70 2024-04-25T10:11:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78baa00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f78bc143f30 0x7f78bc06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44746/0 (socket says 172.21.15.5:44746) 2024-04-25T10:11:05.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78baa00640 1 -- 172.21.15.5:0/197928325 learned_addr learned my addr 172.21.15.5:0/197928325 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813311015 0 0) 0x7f78bc0ade10 con 0x7f78bc143f30 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78b3e00640 1 --2- 172.21.15.5:0/197928325 >> v2:172.21.15.80:3300/0 conn(0x7f78bc06ec70 0x7f78bc071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.350+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a0003520 con 0x7f78bc143f30 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3957025966 0 0) 0x7f78a0003520 con 0x7f78bc143f30 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 >> v2:172.21.15.80:3300/0 conn(0x7f78bc06ec70 msgr2=0x7f78bc071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 --2- 172.21.15.5:0/197928325 >> v2:172.21.15.80:3300/0 conn(0x7f78bc06ec70 0x7f78bc071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 >> v1:172.21.15.80:6789/0 conn(0x7f78bc0aebd0 legacy=0x7f78bc0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0ae6b0 con 0x7f78bc143f30 2024-04-25T10:11:05.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f78ac001670 con 0x7f78bc143f30 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 >> v1:172.21.15.5:6789/0 conn(0x7f78bc143f30 legacy=0x7f78bc06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78a0003a00 0x7f78a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0004800 0x7f78a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0005210 0x7f78a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc0ae3d0 con 0x7f78a0004800 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a000c160 con 0x7f78a0005210 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a000c360 con 0x7f78a0003a00 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.351+0000 7f78b3e00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0004800 0x7f78a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78baa00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78a0003a00 0x7f78a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78bb400640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0005210 0x7f78a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78b3e00640 1 -- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78a0003a00 msgr2=0x7f78a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78b3e00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78a0003a00 0x7f78a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78b3e00640 1 -- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0005210 msgr2=0x7f78a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:05.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78b3e00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0005210 0x7f78a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78b3e00640 1 -- 172.21.15.5:0/197928325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0003520 con 0x7f78a0004800 2024-04-25T10:11:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78baa00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78a0003a00 0x7f78a0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78bb400640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0005210 0x7f78a000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.352+0000 7f78bb400640 1 -- 172.21.15.5:0/197928325 reap_dead start 2024-04-25T10:11:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.353+0000 7f78b3e00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0004800 0x7f78a0004c80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f78bc06f090 tx=0x7f78a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dbcd2af987b961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.353+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4036070 con 0x7f78a0004800 2024-04-25T10:11:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.353+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a4031400 con 0x7f78a0004800 2024-04-25T10:11:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.353+0000 7f78ba000640 1 -- 172.21.15.5:0/197928325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4009690 con 0x7f78a0004800 2024-04-25T10:11:05.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 -- 172.21.15.5:0/197928325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0004800 msgr2=0x7f78a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0004800 0x7f78a0004c80 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f78bc06f090 tx=0x7f78a0005830 comp rx=0 tx=0).stop 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 -- 172.21.15.5:0/197928325 shutdown_connections 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 --2- 172.21.15.5:0/197928325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78a0004800 0x7f78a0004c80 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 -- 172.21.15.5:0/197928325 >> 172.21.15.5:0/197928325 conn(0x7f78bc066f20 msgr2=0x7f78bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 -- 172.21.15.5:0/197928325 shutdown_connections 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.354+0000 7f78bbe00640 1 -- 172.21.15.5:0/197928325 wait complete. 2024-04-25T10:11:05.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.356+0000 7f78bbe00640 1 Processor -- start 2024-04-25T10:11:05.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.356+0000 7f78bbe00640 1 -- start start 2024-04-25T10:11:05.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78bbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0004800 0x7f78bc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 0x7f78bc142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78bc142a70 0x7f78bc13beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78bbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc06dbb0 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78bbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc06da30 con 0x7f78a0004800 2024-04-25T10:11:05.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc06dd30 con 0x7f78bc142a70 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.357+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 0x7f78bc142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 0x7f78bc142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41324/0 (socket says 172.21.15.5:41324) 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 -- 172.21.15.5:0/2587017165 learned_addr learned my addr 172.21.15.5:0/2587017165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78bb400640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78bc142a70 0x7f78bc13beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78baa00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0004800 0x7f78bc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 -- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78bc142a70 msgr2=0x7f78bc13beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78bc142a70 0x7f78bc13beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 -- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0004800 msgr2=0x7f78bc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0004800 0x7f78bc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 -- 172.21.15.5:0/2587017165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0cd730 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.358+0000 7f78b3e00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 0x7f78bc142530 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f78a4000c00 tx=0x7f78a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.359+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a403f030 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.359+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a4037040 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.359+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc0cda10 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.359+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4036070 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.359+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc0cdfd0 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.360+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc068f60 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.361+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78a4042070 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.362+0000 7f78b2a00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f789806b500 0x7f789806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.362+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78a40a9990 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.363+0000 7f78baa00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f789806b500 0x7f789806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.364+0000 7f78baa00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f789806b500 0x7f789806d9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.366+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78a4077f90 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.633+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f78bc0712c0 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.638+0000 7f78b2a00640 1 -- 172.21.15.5:0/2587017165 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f78a407c030 con 0x7f78bc0ce7d0 2024-04-25T10:11:05.699 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:11:05.699 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:11:05.699 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:11:05.699 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:11:05.699 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:11:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f789806b500 msgr2=0x7f789806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f789806b500 0x7f789806d9c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac009040 comp rx=0 tx=0).stop 2024-04-25T10:11:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 msgr2=0x7f78bc142530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 0x7f78bc142530 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f78a4000c00 tx=0x7f78a4004730 comp rx=0 tx=0).stop 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 shutdown_connections 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78bc142a70 0x7f78bc13beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78bc0ce7d0 0x7f78bc142530 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f789806b500 0x7f789806d9c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 --2- 172.21.15.5:0/2587017165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78a0004800 0x7f78bc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 >> 172.21.15.5:0/2587017165 conn(0x7f78bc066f20 msgr2=0x7f78bc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 shutdown_connections 2024-04-25T10:11:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.641+0000 7f78bbe00640 1 -- 172.21.15.5:0/2587017165 wait complete. 2024-04-25T10:11:05.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:05.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.908+0000 7fd8cb000640 1 Processor -- start 2024-04-25T10:11:05.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.908+0000 7fd8cb000640 1 -- start start 2024-04-25T10:11:05.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.909+0000 7fd8cb000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.909+0000 7fd8cb000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8c40d07d0 0x7fd8c40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.909+0000 7fd8cb000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40cf410 con 0x7fd8c40d2380 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.909+0000 7fd8cb000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c40d27c0 con 0x7fd8c40d07d0 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.909+0000 7fd8cb000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b28b0 con 0x7fd8c4161200 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8ca600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd8c40d07d0 0x7fd8c40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8c9200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd8c40d2380 0x7fd8c40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44760/0 (socket says 172.21.15.5:44760) 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8c9200640 1 -- 172.21.15.5:0/3787624674 learned_addr learned my addr 172.21.15.5:0/3787624674 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8c9c00640 1 --2- 172.21.15.5:0/3787624674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8ca600640 1 -- 172.21.15.5:0/3787624674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 msgr2=0x7fd8c40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8ca600640 1 --2- 172.21.15.5:0/3787624674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8ca600640 1 -- 172.21.15.5:0/3787624674 >> v1:172.21.15.5:6789/0 conn(0x7fd8c40d2380 legacy=0x7fd8c40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8c3e00640 1 -- 172.21.15.5:0/3787624674 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986449900 0 0) 0x7fd8c40cf410 con 0x7fd8c40d2380 2024-04-25T10:11:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.910+0000 7fd8ca600640 1 -- 172.21.15.5:0/3787624674 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40cf6f0 con 0x7fd8c40d07d0 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.911+0000 7fd8ca600640 1 --2- 172.21.15.5:0/3787624674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d07d0 0x7fd8c40b2f80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009dc0 tx=0x7fd8bc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4832852eb8f3968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8c3e00640 1 -- 172.21.15.5:0/3787624674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc02ae60 con 0x7fd8c40d07d0 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8c3e00640 1 -- 172.21.15.5:0/3787624674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc007740 con 0x7fd8c40d07d0 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8c3e00640 1 -- 172.21.15.5:0/3787624674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc007b10 con 0x7fd8c40d07d0 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8cb000640 1 -- 172.21.15.5:0/3787624674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d07d0 msgr2=0x7fd8c40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8cb000640 1 --2- 172.21.15.5:0/3787624674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d07d0 0x7fd8c40b2f80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009dc0 tx=0x7fd8bc02a680 comp rx=0 tx=0).stop 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8cb000640 1 -- 172.21.15.5:0/3787624674 shutdown_connections 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8cb000640 1 --2- 172.21.15.5:0/3787624674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d07d0 0x7fd8c40b2f80 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8cb000640 1 --2- 172.21.15.5:0/3787624674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.912+0000 7fd8cb000640 1 -- 172.21.15.5:0/3787624674 >> 172.21.15.5:0/3787624674 conn(0x7fd8c408af80 msgr2=0x7fd8c40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.913+0000 7fd8cb000640 1 -- 172.21.15.5:0/3787624674 shutdown_connections 2024-04-25T10:11:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.913+0000 7fd8cb000640 1 -- 172.21.15.5:0/3787624674 wait complete. 2024-04-25T10:11:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.914+0000 7fd8cb000640 1 Processor -- start 2024-04-25T10:11:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.914+0000 7fd8cb000640 1 -- start start 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.915+0000 7fd8cb000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8c40d07d0 0x7fd8c4152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.915+0000 7fd8cb000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 0x7fd8c406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.915+0000 7fd8cb000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c406c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.915+0000 7fd8cb000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b34c0 con 0x7fd8c40d2380 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.915+0000 7fd8cb000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40d42b0 con 0x7fd8c40d07d0 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.915+0000 7fd8cb000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40b3640 con 0x7fd8c4161200 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 0x7fd8c406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 0x7fd8c406bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41342/0 (socket says 172.21.15.5:41342) 2024-04-25T10:11:05.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 -- 172.21.15.5:0/2376802940 learned_addr learned my addr 172.21.15.5:0/2376802940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9c00640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8c40d07d0 0x7fd8c4152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 -- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 msgr2=0x7fd8c406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8ca600640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 -- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8c40d07d0 msgr2=0x7fd8c4152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8c40d07d0 0x7fd8c4152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9200640 1 -- 172.21.15.5:0/2376802940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40ca820 con 0x7fd8c40d2380 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.916+0000 7fd8c9c00640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8c40d07d0 0x7fd8c4152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.917+0000 7fd8c9200640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 0x7fd8c406bdd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400c4d0 tx=0x7fd8b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.917+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4018070 con 0x7fd8c40d2380 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.917+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4007500 con 0x7fd8c40d2380 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.917+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c40caac0 con 0x7fd8c40d2380 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.917+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4004a20 con 0x7fd8c40d2380 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.917+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c40cb080 con 0x7fd8c40d2380 2024-04-25T10:11:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.918+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c40cdbe0 con 0x7fd8c40d2380 2024-04-25T10:11:05.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.919+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8b401b070 con 0x7fd8c40d2380 2024-04-25T10:11:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.921+0000 7fd8c1600640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd8a806b4d0 0x7fd8a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.921+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8b40896a0 con 0x7fd8c40d2380 2024-04-25T10:11:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.922+0000 7fd8c9c00640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd8a806b4d0 0x7fd8a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.923+0000 7fd8c9c00640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd8a806b4d0 0x7fd8a806d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8006fd0 tx=0x7fd8b8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:05.925+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8c40cdbe0 con 0x7fd8c40d2380 2024-04-25T10:11:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.160+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c4160570 con 0x7fd8c40d2380 2024-04-25T10:11:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.161+0000 7fd8c1600640 1 -- 172.21.15.5:0/2376802940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fd8b4058380 con 0x7fd8c40d2380 2024-04-25T10:11:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:06.167 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:02.669462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6715":{"gid":6715,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2001088034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2001088034},{"type":"v1","addr":"172.21.15.5:6835","nonce":2001088034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2024-04-25T10:11:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd8a806b4d0 msgr2=0x7fd8a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd8a806b4d0 0x7fd8a806d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8006fd0 tx=0x7fd8b8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 msgr2=0x7fd8c406bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 0x7fd8c406bdd0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400c4d0 tx=0x7fd8b400c990 comp rx=0 tx=0).stop 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 shutdown_connections 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8c4161200 0x7fd8c406c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c40d2380 0x7fd8c406bdd0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd8a806b4d0 0x7fd8a806d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 --2- 172.21.15.5:0/2376802940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8c40d07d0 0x7fd8c4152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 >> 172.21.15.5:0/2376802940 conn(0x7fd8c408af80 msgr2=0x7fd8c406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 shutdown_connections 2024-04-25T10:11:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.164+0000 7fd8cb000640 1 -- 172.21.15.5:0/2376802940 wait complete. 2024-04-25T10:11:06.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:10:59.580232+0000', 'modified': '2024-04-25T10:11:02.669462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6715}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6715': {'gid': 6715, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/2001088034', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 2001088034}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 2001088034}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:06.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:06.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:11:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.812+0000 7fdd18e00640 1 Processor -- start 2024-04-25T10:11:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.812+0000 7fdd18e00640 1 -- start start 2024-04-25T10:11:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.813+0000 7fdd18e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 0x7fdd140ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd18e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140c9f90 con 0x7fdd141472c0 2024-04-25T10:11:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd18e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140ca270 con 0x7fdd140cbb50 2024-04-25T10:11:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd18e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140ca550 con 0x7fdd140cad50 2024-04-25T10:11:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd11e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdd141472c0 0x7fdd140cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44768/0 (socket says 172.21.15.5:44768) 2024-04-25T10:11:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd11e00640 1 -- 172.21.15.5:0/2185786639 learned_addr learned my addr 172.21.15.5:0/2185786639 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186044325 0 0) 0x7fdd140c9f90 con 0x7fdd141472c0 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd12800640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 0x7fdd140ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.814+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf8003520 con 0x7fdd141472c0 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3834942057 0 0) 0x7fdcf8003520 con 0x7fdd141472c0 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 msgr2=0x7fdd140ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 0x7fdd140ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 >> v1:172.21.15.80:6789/0 conn(0x7fdd140cbb50 legacy=0x7fdd140ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140ca830 con 0x7fdd141472c0 2024-04-25T10:11:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fdd04001690 con 0x7fdd141472c0 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 >> v1:172.21.15.5:6789/0 conn(0x7fdd141472c0 legacy=0x7fdd140cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdcf8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 0x7fdcf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf8004d40 0x7fdcf80051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140ca550 con 0x7fdcf80039e0 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf800bc00 con 0x7fdcf8004d40 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf800be00 con 0x7fdd140cad50 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.815+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdcf8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 0x7fdcf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd12800640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf8004d40 0x7fdcf80051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd11e00640 1 -- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 msgr2=0x7fdcf8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd140cad50 0x7fdcf8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd11e00640 1 -- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf8004d40 msgr2=0x7fdcf80051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf8004d40 0x7fdcf80051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd11e00640 1 -- 172.21.15.5:0/2185786639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8003520 con 0x7fdcf80039e0 2024-04-25T10:11:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.816+0000 7fdd12800640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcf8004d40 0x7fdcf80051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.817+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdcf8003e00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fdd14071e70 tx=0x7fdcf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6922de1fbbc007b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.817+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04007c80 con 0x7fdcf80039e0 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.817+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04005ad0 con 0x7fdcf80039e0 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.817+0000 7fdd11400640 1 -- 172.21.15.5:0/2185786639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04007070 con 0x7fdcf80039e0 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd18e00640 1 -- 172.21.15.5:0/2185786639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 msgr2=0x7fdcf8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2185786639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdcf8003e00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fdd14071e70 tx=0x7fdcf8005830 comp rx=0 tx=0).stop 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd12800640 1 -- 172.21.15.5:0/2185786639 reap_dead start 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd18e00640 1 -- 172.21.15.5:0/2185786639 shutdown_connections 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd18e00640 1 -- 172.21.15.5:0/2185786639 >> 172.21.15.5:0/2185786639 conn(0x7fdd14071150 msgr2=0x7fdd14071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd18e00640 1 -- 172.21.15.5:0/2185786639 shutdown_connections 2024-04-25T10:11:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.818+0000 7fdd18e00640 1 -- 172.21.15.5:0/2185786639 wait complete. 2024-04-25T10:11:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.820+0000 7fdd18e00640 1 Processor -- start 2024-04-25T10:11:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.820+0000 7fdd18e00640 1 -- start start 2024-04-25T10:11:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd18e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdd140b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd18e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd1413e8c0 0x7fdd14140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd18e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd1413cf70 0x7fdd141e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd18e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cef00 con 0x7fdcf80039e0 2024-04-25T10:11:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd18e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140ced80 con 0x7fdd1413e8c0 2024-04-25T10:11:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd18e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd140cf080 con 0x7fdd1413cf70 2024-04-25T10:11:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdd140b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdd140b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41364/0 (socket says 172.21.15.5:41364) 2024-04-25T10:11:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.821+0000 7fdd11e00640 1 -- 172.21.15.5:0/2362195807 learned_addr learned my addr 172.21.15.5:0/2362195807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd11e00640 1 -- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd1413cf70 msgr2=0x7fdd141e88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd1413cf70 0x7fdd141e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd11e00640 1 -- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd1413e8c0 msgr2=0x7fdd14140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd1413e8c0 0x7fdd14140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd1413e8c0 0x7fdd14140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd11e00640 1 -- 172.21.15.5:0/2362195807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14140a90 con 0x7fdcf80039e0 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdd140b40f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c220 tx=0x7fdd04001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd0a000640 1 -- 172.21.15.5:0/2362195807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040015d0 con 0x7fdcf80039e0 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.822+0000 7fdd0a000640 1 -- 172.21.15.5:0/2362195807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd04042070 con 0x7fdcf80039e0 2024-04-25T10:11:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.823+0000 7fdd0a000640 1 -- 172.21.15.5:0/2362195807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040072b0 con 0x7fdcf80039e0 2024-04-25T10:11:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.823+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd140b3330 con 0x7fdcf80039e0 2024-04-25T10:11:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.823+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd140b3bd0 con 0x7fdcf80039e0 2024-04-25T10:11:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.825+0000 7fdd0a000640 1 -- 172.21.15.5:0/2362195807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd04007450 con 0x7fdcf80039e0 2024-04-25T10:11:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.826+0000 7fdd0a000640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdcf006b500 0x7fdcf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.826+0000 7fdd0a000640 1 -- 172.21.15.5:0/2362195807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd040aa400 con 0x7fdcf80039e0 2024-04-25T10:11:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.827+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdcf006b500 0x7fdcf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.827+0000 7fdd1bbdb740 1 Processor -- start 2024-04-25T10:11:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.827+0000 7fdd1bbdb740 1 -- start start 2024-04-25T10:11:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55973321d950 0x559733245790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdcf006b500 0x7fdcf006d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc007fd0 tx=0x7fdcfc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973321dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332665c0 0x55973321c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559733181620 con 0x559733245cd0 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973312aca0 con 0x5597332665c0 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55973317d6c0 con 0x55973321d950 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973321dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973321dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41370/0 (socket says 172.21.15.5:41370) 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.828+0000 7fdd0be00640 1 -- 172.21.15.5:0/2489660503 learned_addr learned my addr 172.21.15.5:0/2489660503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd11e00640 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55973321d950 0x559733245790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd12800640 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332665c0 0x55973321c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd0be00640 1 -- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55973321d950 msgr2=0x559733245790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55973321d950 0x559733245790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd0be00640 1 -- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332665c0 msgr2=0x55973321c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332665c0 0x55973321c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd0be00640 1 -- 172.21.15.5:0/2489660503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559733131f40 con 0x559733245cd0 2024-04-25T10:11:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd12800640 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332665c0 0x55973321c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:06.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd0be00640 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973321dd30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00f7f0 tx=0x7fdcfc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a140bfa2beebd71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd02a00640 1 -- 172.21.15.5:0/2489660503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc026070 con 0x559733245cd0 2024-04-25T10:11:06.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd02a00640 1 -- 172.21.15.5:0/2489660503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc01b050 con 0x559733245cd0 2024-04-25T10:11:06.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.829+0000 7fdd02a00640 1 -- 172.21.15.5:0/2489660503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc014b50 con 0x559733245cd0 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/2489660503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 msgr2=0x55973321dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973321dd30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00f7f0 tx=0x7fdcfc012000 comp rx=0 tx=0).stop 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/2489660503 shutdown_connections 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332665c0 0x55973321c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973321dd30 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/2489660503 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55973321d950 0x559733245790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/2489660503 >> 172.21.15.5:0/2489660503 conn(0x55973321ab70 msgr2=0x559733263550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/2489660503 shutdown_connections 2024-04-25T10:11:06.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.830+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/2489660503 wait complete. 2024-04-25T10:11:06.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.831+0000 7fdd1bbdb740 1 -- start start 2024-04-25T10:11:06.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.831+0000 7fdd1bbdb740 1 Processor -- start 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.832+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x559733262a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.832+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 0x55973323d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.832+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332294b0 0x55973323eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.832+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559733245300 con 0x5597332665c0 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.832+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597331201f0 con 0x559733245cd0 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.832+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559733244cf0 con 0x5597332294b0 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd1bbdb740 10 client.0 _ll_register_callbacks cb 0x55973313dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 0x55973323d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 0x55973323d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46296/0 (socket says 172.21.15.5:46296) 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 -- 172.21.15.5:0/4277640456 learned_addr learned my addr 172.21.15.5:0/4277640456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd12800640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332294b0 0x55973323eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd11e00640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x559733262a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 -- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332294b0 msgr2=0x55973323eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332294b0 0x55973323eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 -- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 msgr2=0x559733262a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x559733262a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd0be00640 1 -- 172.21.15.5:0/4277640456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55973313fc50 con 0x5597332665c0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.833+0000 7fdd12800640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332294b0 0x55973323eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdd0be00640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 0x55973323d490 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc024040 tx=0x7fdcfc001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cfd9b5a07e8f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdcf7400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdcf7400640 1 -- 172.21.15.5:0/4277640456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc026040 con 0x5597332665c0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdcf7400640 1 -- 172.21.15.5:0/4277640456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdcfc007230 con 0x5597332665c0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcfc01b050 con 0x5597332665c0 2024-04-25T10:11:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdcf7400640 1 -- 172.21.15.5:0/4277640456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcfc038050 con 0x5597332665c0 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdd1bbdb740 10 client.8463 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.834+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x559732e962d0 con 0x5597332665c0 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdcf7400640 1 -- 172.21.15.5:0/4277640456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc01b050 con 0x5597332665c0 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdcf7400640 1 -- 172.21.15.5:0/4277640456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdcfc03b020 con 0x5597332665c0 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd1bbdb740 10 client.8463 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd1bbdb740 10 client.8463 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd1bbdb740 20 client.8463 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55973323dc60 0x55973323e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd1bbdb740 4 client.8463 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55973318eb60 con 0x55973323dc60 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.835+0000 7fdd11e00640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55973323dc60 0x55973323e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.836+0000 7fdd11e00640 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55973323dc60 0x55973323e060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b7bf2401ef65a2 server_cookie=58d9e00f2ca414b8 in_seq=0 out_seq=0 2024-04-25T10:11:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.836+0000 7fdcf7400640 10 client.8463 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdcf7400640 1 -- 172.21.15.5:0/4277640456 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55973318eb60 con 0x55973323dc60 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdcf7400640 10 client.8463 handle_command_reply: tid=0 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 client.8463 shutdown 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 msgr2=0x55973323d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 0x55973323d490 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc024040 tx=0x7fdcfc001190 comp rx=0 tx=0).stop 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 shutdown_connections 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55973323dc60 0x55973323e060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332294b0 0x55973323eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332665c0 0x55973323d490 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/4277640456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x559733262a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdcf7400640 0 client.8463 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 >> 172.21.15.5:0/4277640456 conn(0x55973321d950 msgr2=0x559733266a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 shutdown_connections 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/4277640456 wait complete. 2024-04-25T10:11:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.837+0000 7fdd1bbdb740 20 client.8463 trim_cache size 0 max 16384 2024-04-25T10:11:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.855+0000 7fdd1bbdb740 1 Processor -- start 2024-04-25T10:11:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.855+0000 7fdd1bbdb740 1 -- start start 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332294b0 0x55973323b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973323d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x559733272f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd141ffc10 con 0x559733245cd0 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd141ff060 con 0x5597332294b0 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd141ff610 con 0x5597332665c0 2024-04-25T10:11:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973323d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973323d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46304/0 (socket says 172.21.15.5:46304) 2024-04-25T10:11:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd0be00640 1 -- 172.21.15.5:0/1409412899 learned_addr learned my addr 172.21.15.5:0/1409412899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd11e00640 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332294b0 0x55973323b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.856+0000 7fdd12800640 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x559733272f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd0be00640 1 -- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 msgr2=0x559733272f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd0be00640 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x559733272f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd0be00640 1 -- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332294b0 msgr2=0x55973323b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd0be00640 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332294b0 0x55973323b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd0be00640 1 -- 172.21.15.5:0/1409412899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559732f606b0 con 0x559733245cd0 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd0be00640 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973323d250 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc007810 tx=0x7fdcfc02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ac80c089c324cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd02000640 1 -- 172.21.15.5:0/1409412899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc021920 con 0x559733245cd0 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd02000640 1 -- 172.21.15.5:0/1409412899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc041070 con 0x559733245cd0 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.857+0000 7fdd02000640 1 -- 172.21.15.5:0/1409412899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc021cc0 con 0x559733245cd0 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/1409412899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 msgr2=0x55973323d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973323d250 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc007810 tx=0x7fdcfc02a000 comp rx=0 tx=0).stop 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/1409412899 shutdown_connections 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x559733272f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559733245cd0 0x55973323d250 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/1409412899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5597332294b0 0x55973323b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/1409412899 >> 172.21.15.5:0/1409412899 conn(0x55973321d950 msgr2=0x55973325d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/1409412899 shutdown_connections 2024-04-25T10:11:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.858+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/1409412899 wait complete. 2024-04-25T10:11:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.859+0000 7fdd1bbdb740 1 -- start start 2024-04-25T10:11:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.859+0000 7fdd1bbdb740 1 Processor -- start 2024-04-25T10:11:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 0x5597332d9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x55973323cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x55973323d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559733229e00 con 0x5597332294b0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x559733245300 con 0x559733245cd0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x559733229bc0 con 0x5597332665c0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd1bbdb740 10 client.0 _ll_register_callbacks cb 0x7fdcfc024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 0x5597332d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 0x5597332d9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46308/0 (socket says 172.21.15.5:46308) 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.860+0000 7fdd11e00640 1 -- 172.21.15.5:0/784891467 learned_addr learned my addr 172.21.15.5:0/784891467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd12800640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x55973323d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd0be00640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x55973323cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd11e00640 1 -- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 msgr2=0x55973323d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd11e00640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x55973323d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd11e00640 1 -- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 msgr2=0x55973323cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd11e00640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x55973323cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd11e00640 1 -- 172.21.15.5:0/784891467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597332d97a0 con 0x5597332294b0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd12800640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x55973323d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd11e00640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 0x5597332d9260 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdd04078970 tx=0x7fdd0400c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffe945545806cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdcf7400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdcf7400640 1 -- 172.21.15.5:0/784891467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040810f0 con 0x5597332294b0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdcf7400640 1 -- 172.21.15.5:0/784891467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdd04076ce0 con 0x5597332294b0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.861+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcfc041070 con 0x5597332294b0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdcf7400640 1 -- 172.21.15.5:0/784891467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd04097050 con 0x5597332294b0 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 10 client.8469 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55973323ae00 con 0x5597332294b0 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdcf7400640 1 -- 172.21.15.5:0/784891467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcfc041070 con 0x5597332294b0 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdcf7400640 1 -- 172.21.15.5:0/784891467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd04075050 con 0x5597332294b0 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 10 client.8469 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 10 client.8469 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 20 client.8469 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x559733260190 0x55973323e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 4 client.8469 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5597332409e0 con 0x559733260190 2024-04-25T10:11:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.862+0000 7fdd0be00640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x559733260190 0x55973323e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd0be00640 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x559733260190 0x55973323e040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446057269073315d server_cookie=56800d3387c68ab6 in_seq=0 out_seq=0 2024-04-25T10:11:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdcf7400640 10 client.8469 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:06.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdcf7400640 1 -- 172.21.15.5:0/784891467 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5597332409e0 con 0x559733260190 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdcf7400640 10 client.8469 handle_command_reply: tid=0 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 client.8469 shutdown 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 msgr2=0x5597332d9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 0x5597332d9260 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdd04078970 tx=0x7fdd0400c0e0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 shutdown_connections 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x559733260190 0x55973323e040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5597332665c0 0x55973323d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x559733245cd0 0x55973323cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 --2- 172.21.15.5:0/784891467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597332294b0 0x5597332d9260 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 >> 172.21.15.5:0/784891467 conn(0x55973321d950 msgr2=0x559733262840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.863+0000 7fdcf7400640 0 client.8469 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 shutdown_connections 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd1bbdb740 1 -- 172.21.15.5:0/784891467 wait complete. 2024-04-25T10:11:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd1bbdb740 20 client.8469 trim_cache size 0 max 16384 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 6715, 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.7270646359999997, 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 83, 2024-04-25T10:11:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 105, 2024-04-25T10:11:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 105, 2024-04-25T10:11:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 13.163868526 2024-04-25T10:11:06.874 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdcf006b500 msgr2=0x7fdcf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdcf006b500 0x7fdcf006d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc007fd0 tx=0x7fdcfc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 msgr2=0x7fdd140b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.864+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdd140b40f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400c220 tx=0x7fdd04001990 comp rx=0 tx=0).stop 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 shutdown_connections 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd1413cf70 0x7fdd141e88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdcf006b500 0x7fdcf006d9c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd1413e8c0 0x7fdd14140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 --2- 172.21.15.5:0/2362195807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf80039e0 0x7fdd140b40f0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 >> 172.21.15.5:0/2362195807 conn(0x7fdd14071150 msgr2=0x7fdd140cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 shutdown_connections 2024-04-25T10:11:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:06.865+0000 7fdd18e00640 1 -- 172.21.15.5:0/2362195807 wait complete. 2024-04-25T10:11:06.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.089+0000 7fe09da00640 1 Processor -- start 2024-04-25T10:11:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.089+0000 7fe09da00640 1 -- start start 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe09da00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe09815ce00 0x7fe09815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe09da00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0980cd300 0x7fe0980cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe09da00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe098071d30 con 0x7fe098141da0 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe09da00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0980cdcd0 con 0x7fe09815ce00 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe09da00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe098142180 con 0x7fe0980cd300 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe097400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe09815ce00 0x7fe09815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe097400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe09815ce00 0x7fe09815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46310/0 (socket says 172.21.15.5:46310) 2024-04-25T10:11:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.091+0000 7fe097400640 1 -- 172.21.15.5:0/2752738865 learned_addr learned my addr 172.21.15.5:0/2752738865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.092+0000 7fe09d000640 1 --2- 172.21.15.5:0/2752738865 >> v2:172.21.15.80:3300/0 conn(0x7fe0980cd300 0x7fe0980cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.092+0000 7fe097400640 1 -- 172.21.15.5:0/2752738865 >> v2:172.21.15.80:3300/0 conn(0x7fe0980cd300 msgr2=0x7fe0980cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.092+0000 7fe097400640 1 --2- 172.21.15.5:0/2752738865 >> v2:172.21.15.80:3300/0 conn(0x7fe0980cd300 0x7fe0980cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.092+0000 7fe097400640 1 -- 172.21.15.5:0/2752738865 >> v1:172.21.15.80:6789/0 conn(0x7fe098141da0 legacy=0x7fe09806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.092+0000 7fe097400640 1 -- 172.21.15.5:0/2752738865 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09806d620 con 0x7fe09815ce00 2024-04-25T10:11:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.092+0000 7fe09d000640 1 --2- 172.21.15.5:0/2752738865 >> v2:172.21.15.80:3300/0 conn(0x7fe0980cd300 0x7fe0980cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.093+0000 7fe097400640 1 --2- 172.21.15.5:0/2752738865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09815ce00 0x7fe09815d280 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fe09815f220 tx=0x7fe08402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c789f97548b03ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.093+0000 7fe096a00640 1 -- 172.21.15.5:0/2752738865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084037070 con 0x7fe09815ce00 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.093+0000 7fe096a00640 1 -- 172.21.15.5:0/2752738865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084032440 con 0x7fe09815ce00 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.093+0000 7fe096a00640 1 -- 172.21.15.5:0/2752738865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084032810 con 0x7fe09815ce00 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 -- 172.21.15.5:0/2752738865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09815ce00 msgr2=0x7fe09815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 --2- 172.21.15.5:0/2752738865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09815ce00 0x7fe09815d280 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fe09815f220 tx=0x7fe08402a230 comp rx=0 tx=0).stop 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 -- 172.21.15.5:0/2752738865 shutdown_connections 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 --2- 172.21.15.5:0/2752738865 >> v2:172.21.15.80:3300/0 conn(0x7fe0980cd300 0x7fe0980cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 --2- 172.21.15.5:0/2752738865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe09815ce00 0x7fe09815d280 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 -- 172.21.15.5:0/2752738865 >> 172.21.15.5:0/2752738865 conn(0x7fe098066f20 msgr2=0x7fe09806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 -- 172.21.15.5:0/2752738865 shutdown_connections 2024-04-25T10:11:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.094+0000 7fe09da00640 1 -- 172.21.15.5:0/2752738865 wait complete. 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.096+0000 7fe09da00640 1 Processor -- start 2024-04-25T10:11:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.096+0000 7fe09da00640 1 -- start start 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe09da00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0980cd300 0x7fe09812f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe09da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 0x7fe0980c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe09da00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09815ce00 0x7fe09815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe09da00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09806e810 con 0x7fe098141da0 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe09da00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09806dbe0 con 0x7fe0980cd300 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe09da00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe098071000 con 0x7fe09815ce00 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe097400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 0x7fe0980c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe097400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 0x7fe0980c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46312/0 (socket says 172.21.15.5:46312) 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.097+0000 7fe097400640 1 -- 172.21.15.5:0/2066464454 learned_addr learned my addr 172.21.15.5:0/2066464454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe097400640 1 -- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09815ce00 msgr2=0x7fe09815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe097400640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09815ce00 0x7fe09815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe097400640 1 -- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0980cd300 msgr2=0x7fe09812f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe097400640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0980cd300 0x7fe09812f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe097400640 1 -- 172.21.15.5:0/2066464454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980b2980 con 0x7fe098141da0 2024-04-25T10:11:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe097400640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 0x7fe0980c9420 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fe08402a270 tx=0x7fe0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084037040 con 0x7fe098141da0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084042070 con 0x7fe098141da0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084030bb0 con 0x7fe098141da0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.098+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0980b2c60 con 0x7fe098141da0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.099+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0980b3220 con 0x7fe098141da0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.100+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe098068f60 con 0x7fe098141da0 2024-04-25T10:11:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.101+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe084047460 con 0x7fe098141da0 2024-04-25T10:11:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.102+0000 7fe08fe00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe07806b4d0 0x7fe07806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.102+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0840ab170 con 0x7fe098141da0 2024-04-25T10:11:07.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.102+0000 7fe097e00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe07806b4d0 0x7fe07806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.104+0000 7fe097e00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe07806b4d0 0x7fe07806d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe088003cc0 tx=0x7fe0880079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.106+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe084045360 con 0x7fe098141da0 2024-04-25T10:11:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.351+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe098141130 con 0x7fe098141da0 2024-04-25T10:11:07.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.352+0000 7fe08fe00640 1 -- 172.21.15.5:0/2066464454 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe084079d30 con 0x7fe098141da0 2024-04-25T10:11:07.357 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.354+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe07806b4d0 msgr2=0x7fe07806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.354+0000 7fe09da00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe07806b4d0 0x7fe07806d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe088003cc0 tx=0x7fe0880079b0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.354+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 msgr2=0x7fe0980c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.354+0000 7fe09da00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 0x7fe0980c9420 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fe08402a270 tx=0x7fe0840046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 shutdown_connections 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe09815ce00 0x7fe09815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe098141da0 0x7fe0980c9420 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe07806b4d0 0x7fe07806d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 --2- 172.21.15.5:0/2066464454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0980cd300 0x7fe09812f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 >> 172.21.15.5:0/2066464454 conn(0x7fe098066f20 msgr2=0x7fe098073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 shutdown_connections 2024-04-25T10:11:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:07.355+0000 7fe09da00640 1 -- 172.21.15.5:0/2066464454 wait complete. 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:07.379 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:07.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:07.380 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:11:07.435 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42058sec preferred_lft 42058sec 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:11:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:07.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:07.439 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:07.521 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:11:07.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:07.526 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:11:07.580 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:11:07.580 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:07.580 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:11:07.580 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:11:07.580 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:07.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:07.581 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:11:07.581 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:11:07.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:11:07.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:11:07.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:11:07.581 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:11:07.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:07.741 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:11:07.796 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:07.796 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:07.797 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:07.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:07.797 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:11:07.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:07.891 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:07.949 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:07.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:07.950 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:08.004 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:08.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:08.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:08.136 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:08.136 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:08.137 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:11:08.164 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:08.164 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:08.164 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:11:08.165 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:11:08.165 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:08.191 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:08.191 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:08.192 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:11:08.192 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:11:08.192 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:08.192 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:11:08.192 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:11:08.192 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:08.194 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:08.195 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:08.200 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:08.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:08.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:08.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:08.277 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:08.355 INFO:teuthology.orchestra.run.smithi005.stdout:client5993 2024-04-25T10:11:08.356 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:08.356 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:08.414 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:08.415 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:08.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:08.468 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:11:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records in 2024-04-25T10:11:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records out 2024-04-25T10:11:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.15748 s, 40.0 MB/s 2024-04-25T10:11:08.692 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:08.692 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:08.748 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:08.748 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:08.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:08.999+0000 7faeeb200640 1 Processor -- start 2024-04-25T10:11:09.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:08.999+0000 7faeeb200640 1 -- start start 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faeeb200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faeec0cdca0 0x7faeec15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faeeb200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec0cd200 con 0x7faeec154a10 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faeeb200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeec06e6f0 con 0x7faeec070f90 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faeeb200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeec06e9d0 con 0x7faeec0cdca0 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faee9e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faeec154a10 0x7faeec0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34242/0 (socket says 172.21.15.5:34242) 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faee9e00640 1 -- 172.21.15.5:0/2827994643 learned_addr learned my addr 172.21.15.5:0/2827994643 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.001+0000 7faee1400640 1 --2- 172.21.15.5:0/2827994643 >> v2:172.21.15.5:3300/0 conn(0x7faeec0cdca0 0x7faeec15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.002+0000 7faee9400640 1 -- 172.21.15.5:0/2827994643 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911191350 0 0) 0x7faeec0cd200 con 0x7faeec154a10 2024-04-25T10:11:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.002+0000 7faee9400640 1 -- 172.21.15.5:0/2827994643 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed0003520 con 0x7faeec154a10 2024-04-25T10:11:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.002+0000 7faee1400640 1 -- 172.21.15.5:0/2827994643 >> v1:172.21.15.80:6789/0 conn(0x7faeec070f90 legacy=0x7faeec06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.002+0000 7faee1400640 1 -- 172.21.15.5:0/2827994643 >> v1:172.21.15.5:6789/0 conn(0x7faeec154a10 legacy=0x7faeec0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.002+0000 7faee1400640 1 -- 172.21.15.5:0/2827994643 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec06ecb0 con 0x7faeec0cdca0 2024-04-25T10:11:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.003+0000 7faee1400640 1 --2- 172.21.15.5:0/2827994643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec0cdca0 0x7faeec15e8c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1a7de40c17c6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.003+0000 7faee9400640 1 -- 172.21.15.5:0/2827994643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4037070 con 0x7faeec0cdca0 2024-04-25T10:11:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.003+0000 7faee9400640 1 -- 172.21.15.5:0/2827994643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed402ae50 con 0x7faeec0cdca0 2024-04-25T10:11:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.003+0000 7faee9400640 1 -- 172.21.15.5:0/2827994643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed4032640 con 0x7faeec0cdca0 2024-04-25T10:11:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 -- 172.21.15.5:0/2827994643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec0cdca0 msgr2=0x7faeec15e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 --2- 172.21.15.5:0/2827994643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec0cdca0 0x7faeec15e8c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7faed40097a0 tx=0x7faed402a140 comp rx=0 tx=0).stop 2024-04-25T10:11:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 -- 172.21.15.5:0/2827994643 shutdown_connections 2024-04-25T10:11:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 --2- 172.21.15.5:0/2827994643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec0cdca0 0x7faeec15e8c0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 -- 172.21.15.5:0/2827994643 >> 172.21.15.5:0/2827994643 conn(0x7faeec066f20 msgr2=0x7faeec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 -- 172.21.15.5:0/2827994643 shutdown_connections 2024-04-25T10:11:09.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.004+0000 7faeeb200640 1 -- 172.21.15.5:0/2827994643 wait complete. 2024-04-25T10:11:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.006+0000 7faeeb200640 1 Processor -- start 2024-04-25T10:11:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.006+0000 7faeeb200640 1 -- start start 2024-04-25T10:11:09.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeeb200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec070f90 0x7faeec15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeeb200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec0cdca0 0x7faeec159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeeb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 0x7faeec0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeeb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec06d700 con 0x7faeec154a10 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeeb200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec0cd780 con 0x7faeec0cdca0 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeeb200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec06d880 con 0x7faeec070f90 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeea800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 0x7faeec0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeea800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 0x7faeec0adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46334/0 (socket says 172.21.15.5:46334) 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faeea800640 1 -- 172.21.15.5:0/1838699200 learned_addr learned my addr 172.21.15.5:0/1838699200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.008+0000 7faee1400640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec0cdca0 0x7faeec159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faee9e00640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec070f90 0x7faeec15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faeea800640 1 -- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec070f90 msgr2=0x7faeec15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faeea800640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec070f90 0x7faeec15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faeea800640 1 -- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec0cdca0 msgr2=0x7faeec159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faeea800640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec0cdca0 0x7faeec159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faeea800640 1 -- 172.21.15.5:0/1838699200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec158a10 con 0x7faeec154a10 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faee1400640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec0cdca0 0x7faeec159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.009+0000 7faeea800640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 0x7faeec0adb20 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7faee400c410 tx=0x7faee400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.010+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4011790 con 0x7faeec154a10 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.010+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee4017070 con 0x7faeec154a10 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.010+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee4011bb0 con 0x7faeec154a10 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.010+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec158cf0 con 0x7faeec154a10 2024-04-25T10:11:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.010+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec1592b0 con 0x7faeec154a10 2024-04-25T10:11:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.011+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec06b8d0 con 0x7faeec154a10 2024-04-25T10:11:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.012+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faee401c460 con 0x7faeec154a10 2024-04-25T10:11:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.013+0000 7faee2a00640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.013+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faee408a050 con 0x7faeec154a10 2024-04-25T10:11:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.014+0000 7faee9e00640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.015+0000 7faee9e00640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7faedc001580 tx=0x7faedc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.018+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faee4021020 con 0x7faeec154a10 2024-04-25T10:11:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.222+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faeec06af70 con 0x7faeec154a10 2024-04-25T10:11:09.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.223+0000 7faee2a00640 1 -- 172.21.15.5:0/1838699200 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16792 (secure 0 0 0) 0x7faee4058ca0 con 0x7faeec154a10 2024-04-25T10:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:11:01.130972+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.226+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 msgr2=0x7faec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.226+0000 7faeeb200640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7faedc001580 tx=0x7faedc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.226+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 msgr2=0x7faeec0adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.226+0000 7faeeb200640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 0x7faeec0adb20 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7faee400c410 tx=0x7faee400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 shutdown_connections 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faeec154a10 0x7faeec0adb20 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faec806b4d0 0x7faec806d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeec0cdca0 0x7faeec159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 --2- 172.21.15.5:0/1838699200 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faeec070f90 0x7faeec15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 >> 172.21.15.5:0/1838699200 conn(0x7faeec066f20 msgr2=0x7faeec0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 shutdown_connections 2024-04-25T10:11:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.227+0000 7faeeb200640 1 -- 172.21.15.5:0/1838699200 wait complete. 2024-04-25T10:11:09.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:57.493997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.605360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:09.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.459+0000 7ffbb5c00640 1 Processor -- start 2024-04-25T10:11:09.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.459+0000 7ffbb5c00640 1 -- start start 2024-04-25T10:11:09.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.461+0000 7ffbb5c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbb006d130 0x7ffbb0071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.461+0000 7ffbb5c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb0071670 0x7ffbb015c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.461+0000 7ffbb5c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb01453c0 con 0x7ffbb0144fe0 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.461+0000 7ffbb5c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb00a8560 con 0x7ffbb006d130 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.461+0000 7ffbb5c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb006d6a0 con 0x7ffbb0071670 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.461+0000 7ffba6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbb006d130 0x7ffbb0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.462+0000 7ffba6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffbb006d130 0x7ffbb0071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46346/0 (socket says 172.21.15.5:46346) 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.462+0000 7ffba6000640 1 -- 172.21.15.5:0/3212343554 learned_addr learned my addr 172.21.15.5:0/3212343554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:09.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.462+0000 7ffba6000640 1 -- 172.21.15.5:0/3212343554 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb0071670 msgr2=0x7ffbb015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.462+0000 7ffba6000640 1 --2- 172.21.15.5:0/3212343554 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb0071670 0x7ffbb015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.462+0000 7ffba6000640 1 -- 172.21.15.5:0/3212343554 >> v1:172.21.15.80:6789/0 conn(0x7ffbb0144fe0 legacy=0x7ffbb006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.462+0000 7ffba6000640 1 -- 172.21.15.5:0/3212343554 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb006d980 con 0x7ffbb006d130 2024-04-25T10:11:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.463+0000 7ffba6000640 1 --2- 172.21.15.5:0/3212343554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb006d130 0x7ffbb0071130 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009870 tx=0x7ffb9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d274476e584134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.463+0000 7ffbae000640 1 -- 172.21.15.5:0/3212343554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98037070 con 0x7ffbb006d130 2024-04-25T10:11:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbae000640 1 -- 172.21.15.5:0/3212343554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98032440 con 0x7ffbb006d130 2024-04-25T10:11:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbae000640 1 -- 172.21.15.5:0/3212343554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98032810 con 0x7ffbb006d130 2024-04-25T10:11:09.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbb5c00640 1 -- 172.21.15.5:0/3212343554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb006d130 msgr2=0x7ffbb0071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/3212343554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb006d130 0x7ffbb0071130 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009870 tx=0x7ffb9802a210 comp rx=0 tx=0).stop 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbb5c00640 1 -- 172.21.15.5:0/3212343554 shutdown_connections 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/3212343554 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb0071670 0x7ffbb015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/3212343554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb006d130 0x7ffbb0071130 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.464+0000 7ffbb5c00640 1 -- 172.21.15.5:0/3212343554 >> 172.21.15.5:0/3212343554 conn(0x7ffbb006ab60 msgr2=0x7ffbb006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.465+0000 7ffbb5c00640 1 -- 172.21.15.5:0/3212343554 shutdown_connections 2024-04-25T10:11:09.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.465+0000 7ffbb5c00640 1 -- 172.21.15.5:0/3212343554 wait complete. 2024-04-25T10:11:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.467+0000 7ffbb5c00640 1 Processor -- start 2024-04-25T10:11:09.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.467+0000 7ffbb5c00640 1 -- start start 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbb5c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb006d130 0x7ffbb013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbb5c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbb0071670 0x7ffbb00b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbb5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 0x7ffbb00b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbb5c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb015be50 con 0x7ffbb0144fe0 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbb5c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb006df40 con 0x7ffbb0071670 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbb5c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb015c8b0 con 0x7ffbb006d130 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbaf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 0x7ffbb00b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbaf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 0x7ffbb00b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46352/0 (socket says 172.21.15.5:46352) 2024-04-25T10:11:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbaf400640 1 -- 172.21.15.5:0/116059156 learned_addr learned my addr 172.21.15.5:0/116059156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffba6000640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbb0071670 0x7ffbb00b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.469+0000 7ffbaea00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb006d130 0x7ffbb013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffbaf400640 1 -- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb006d130 msgr2=0x7ffbb013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffbaf400640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb006d130 0x7ffbb013fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffbaf400640 1 -- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbb0071670 msgr2=0x7ffbb00b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffbaf400640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbb0071670 0x7ffbb00b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffbaf400640 1 -- 172.21.15.5:0/116059156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb01e53a0 con 0x7ffbb0144fe0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffbaf400640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 0x7ffbb00b05c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ffba800b970 tx=0x7ffba800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.470+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8019070 con 0x7ffbb0144fe0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.471+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba800cb60 con 0x7ffbb0144fe0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.471+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb00afe00 con 0x7ffbb0144fe0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.471+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8014440 con 0x7ffbb0144fe0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.471+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb00b0b00 con 0x7ffbb0144fe0 2024-04-25T10:11:09.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.472+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb0068f60 con 0x7ffbb0144fe0 2024-04-25T10:11:09.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.473+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffba801c070 con 0x7ffbb0144fe0 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.474+0000 7ffba7400640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.475+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffba808a010 con 0x7ffbb0144fe0 2024-04-25T10:11:09.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.475+0000 7ffbaea00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.476+0000 7ffbaea00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00aeb60 tx=0x7ffba0009060 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.478+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffba80586a0 con 0x7ffbb0144fe0 2024-04-25T10:11:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.675+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbb00ae650 con 0x7ffbb0144fe0 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.676+0000 7ffba7400640 1 -- 172.21.15.5:0/116059156 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffba805c740 con 0x7ffbb0144fe0 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:11:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.678+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb8c06b4d0 msgr2=0x7ffb8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.678+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00aeb60 tx=0x7ffba0009060 comp rx=0 tx=0).stop 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.678+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 msgr2=0x7ffbb00b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.678+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 0x7ffbb00b05c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ffba800b970 tx=0x7ffba800be30 comp rx=0 tx=0).stop 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 shutdown_connections 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0144fe0 0x7ffbb00b05c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffb8c06b4d0 0x7ffb8c06d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffbb0071670 0x7ffbb00b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 --2- 172.21.15.5:0/116059156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffbb006d130 0x7ffbb013fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 >> 172.21.15.5:0/116059156 conn(0x7ffbb006ab60 msgr2=0x7ffbb015e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 shutdown_connections 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.679+0000 7ffbb5c00640 1 -- 172.21.15.5:0/116059156 wait complete. 2024-04-25T10:11:09.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:09.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.912+0000 7fce0ae00640 1 Processor -- start 2024-04-25T10:11:09.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.912+0000 7fce0ae00640 1 -- start start 2024-04-25T10:11:09.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.914+0000 7fce0ae00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fce041448f0 0x7fce040cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.914+0000 7fce0ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce0415bc30 0x7fce0415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.914+0000 7fce0ae00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce04144cd0 con 0x7fce0415fe30 2024-04-25T10:11:09.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.914+0000 7fce0ae00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0415f460 con 0x7fce041448f0 2024-04-25T10:11:09.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.914+0000 7fce0ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0415c440 con 0x7fce0415bc30 2024-04-25T10:11:09.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fce041448f0 0x7fce040cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fce041448f0 0x7fce040cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46358/0 (socket says 172.21.15.5:46358) 2024-04-25T10:11:09.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 -- 172.21.15.5:0/2733930379 learned_addr learned my addr 172.21.15.5:0/2733930379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:09.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09000640 1 --2- 172.21.15.5:0/2733930379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce0415bc30 0x7fce0415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 -- 172.21.15.5:0/2733930379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce0415bc30 msgr2=0x7fce0415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 --2- 172.21.15.5:0/2733930379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce0415bc30 0x7fce0415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 -- 172.21.15.5:0/2733930379 >> v1:172.21.15.80:6789/0 conn(0x7fce0415fe30 legacy=0x7fce0415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.915+0000 7fce09a00640 1 -- 172.21.15.5:0/2733930379 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0415c720 con 0x7fce041448f0 2024-04-25T10:11:09.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.916+0000 7fce09a00640 1 --2- 172.21.15.5:0/2733930379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce041448f0 0x7fce040cdaa0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009870 tx=0x7fcdf802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6d3b84510ac1e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce03e00640 1 -- 172.21.15.5:0/2733930379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8037070 con 0x7fce041448f0 2024-04-25T10:11:09.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce03e00640 1 -- 172.21.15.5:0/2733930379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf8032440 con 0x7fce041448f0 2024-04-25T10:11:09.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce03e00640 1 -- 172.21.15.5:0/2733930379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8032810 con 0x7fce041448f0 2024-04-25T10:11:09.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce0ae00640 1 -- 172.21.15.5:0/2733930379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce041448f0 msgr2=0x7fce040cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce0ae00640 1 --2- 172.21.15.5:0/2733930379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce041448f0 0x7fce040cdaa0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009870 tx=0x7fcdf802a210 comp rx=0 tx=0).stop 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce0ae00640 1 -- 172.21.15.5:0/2733930379 shutdown_connections 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce0ae00640 1 --2- 172.21.15.5:0/2733930379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce0415bc30 0x7fce0415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce0ae00640 1 --2- 172.21.15.5:0/2733930379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce041448f0 0x7fce040cdaa0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.917+0000 7fce0ae00640 1 -- 172.21.15.5:0/2733930379 >> 172.21.15.5:0/2733930379 conn(0x7fce04066f20 msgr2=0x7fce0406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.918+0000 7fce0ae00640 1 -- 172.21.15.5:0/2733930379 shutdown_connections 2024-04-25T10:11:09.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.918+0000 7fce0ae00640 1 -- 172.21.15.5:0/2733930379 wait complete. 2024-04-25T10:11:09.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.919+0000 7fce0ae00640 1 Processor -- start 2024-04-25T10:11:09.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.919+0000 7fce0ae00640 1 -- start start 2024-04-25T10:11:09.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.920+0000 7fce0ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 0x7fce041f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.920+0000 7fce0ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce0415fe30 0x7fce041f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.920+0000 7fce0ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce040b0fe0 0x7fce041f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.920+0000 7fce0ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0415c090 con 0x7fce0415bc30 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.920+0000 7fce0ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0415d530 con 0x7fce0415fe30 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.920+0000 7fce0ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0413e2a0 con 0x7fce040b0fe0 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 0x7fce041f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 0x7fce041f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46366/0 (socket says 172.21.15.5:46366) 2024-04-25T10:11:09.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 -- 172.21.15.5:0/1227127475 learned_addr learned my addr 172.21.15.5:0/1227127475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:09.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 -- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce040b0fe0 msgr2=0x7fce041f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce040b0fe0 0x7fce041f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 -- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce0415fe30 msgr2=0x7fce041f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce0415fe30 0x7fce041f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.921+0000 7fce09a00640 1 -- 172.21.15.5:0/1227127475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0413bcf0 con 0x7fce0415bc30 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.922+0000 7fce09a00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 0x7fce041f4b90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8000c00 tx=0x7fcdf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.922+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8041020 con 0x7fce0415bc30 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.922+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf8030740 con 0x7fce0415bc30 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.922+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8037040 con 0x7fce0415bc30 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.922+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0413bfd0 con 0x7fce0415bc30 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.922+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0413db80 con 0x7fce0415bc30 2024-04-25T10:11:09.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.923+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce04068f60 con 0x7fce0415bc30 2024-04-25T10:11:09.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.924+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdf8044020 con 0x7fce0415bc30 2024-04-25T10:11:09.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.925+0000 7fce01600640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcde406b4d0 0x7fcde406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.925+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdf80aa820 con 0x7fce0415bc30 2024-04-25T10:11:09.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.926+0000 7fce09000640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcde406b4d0 0x7fcde406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.927+0000 7fce09000640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcde406b4d0 0x7fcde406d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fce0413c530 tx=0x7fcdf0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:09.930+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdf804b070 con 0x7fce0415bc30 2024-04-25T10:11:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.126+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce04072180 con 0x7fce0415bc30 2024-04-25T10:11:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.127+0000 7fce01600640 1 -- 172.21.15.5:0/1227127475 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcdf8079470 con 0x7fce0415bc30 2024-04-25T10:11:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcde406b4d0 msgr2=0x7fcde406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcde406b4d0 0x7fcde406d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fce0413c530 tx=0x7fcdf0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 msgr2=0x7fce041f4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 0x7fce041f4b90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8000c00 tx=0x7fcdf80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 shutdown_connections 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce040b0fe0 0x7fce041f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcde406b4d0 0x7fcde406d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce0415fe30 0x7fce041f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 --2- 172.21.15.5:0/1227127475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0415bc30 0x7fce041f4b90 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 >> 172.21.15.5:0/1227127475 conn(0x7fce04066f20 msgr2=0x7fce041405f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.129+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 shutdown_connections 2024-04-25T10:11:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.130+0000 7fce0ae00640 1 -- 172.21.15.5:0/1227127475 wait complete. 2024-04-25T10:11:10.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:10.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.360+0000 7f2c02000640 1 Processor -- start 2024-04-25T10:11:10.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.361+0000 7f2c02000640 1 -- start start 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.362+0000 7f2c02000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2bfc0d3f90 0x7f2bfc161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.362+0000 7f2c02000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2bfc161bc0 0x7f2bfc0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.362+0000 7f2c02000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc1606e0 con 0x7f2bfc0b48c0 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.362+0000 7f2c02000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc161ff0 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.362+0000 7f2c02000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc1609c0 con 0x7f2bfc161bc0 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.362+0000 7f2bfbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2bfc0d3f90 0x7f2bfc161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2c00c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2bfc0b48c0 0x7f2bfc15fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34248/0 (socket says 172.21.15.5:34248) 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2c00c00640 1 -- 172.21.15.5:0/2358815044 learned_addr learned my addr 172.21.15.5:0/2358815044 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2c01600640 1 --2- 172.21.15.5:0/2358815044 >> v2:172.21.15.80:3300/0 conn(0x7f2bfc161bc0 0x7f2bfc0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2358815044 >> v2:172.21.15.80:3300/0 conn(0x7f2bfc161bc0 msgr2=0x7f2bfc0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2bfbe00640 1 --2- 172.21.15.5:0/2358815044 >> v2:172.21.15.80:3300/0 conn(0x7f2bfc161bc0 0x7f2bfc0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2358815044 >> v1:172.21.15.5:6789/0 conn(0x7f2bfc0b48c0 legacy=0x7f2bfc15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2358815044 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc160ca0 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.363+0000 7f2c01600640 1 --2- 172.21.15.5:0/2358815044 >> v2:172.21.15.80:3300/0 conn(0x7f2bfc161bc0 0x7f2bfc0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.364+0000 7f2bfbe00640 1 --2- 172.21.15.5:0/2358815044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc161680 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc08d2e0 tx=0x7f2be402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352e3aee0786100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.364+0000 7f2bfb400640 1 -- 172.21.15.5:0/2358815044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4037070 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.364+0000 7f2bfb400640 1 -- 172.21.15.5:0/2358815044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be4032440 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2bfb400640 1 -- 172.21.15.5:0/2358815044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be402ae60 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 -- 172.21.15.5:0/2358815044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 msgr2=0x7f2bfc161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 --2- 172.21.15.5:0/2358815044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc161680 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc08d2e0 tx=0x7f2be402a110 comp rx=0 tx=0).stop 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 -- 172.21.15.5:0/2358815044 shutdown_connections 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 --2- 172.21.15.5:0/2358815044 >> v2:172.21.15.80:3300/0 conn(0x7f2bfc161bc0 0x7f2bfc0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 --2- 172.21.15.5:0/2358815044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc161680 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 -- 172.21.15.5:0/2358815044 >> 172.21.15.5:0/2358815044 conn(0x7f2bfc08af80 msgr2=0x7f2bfc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 -- 172.21.15.5:0/2358815044 shutdown_connections 2024-04-25T10:11:10.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.365+0000 7f2c02000640 1 -- 172.21.15.5:0/2358815044 wait complete. 2024-04-25T10:11:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.367+0000 7f2c02000640 1 Processor -- start 2024-04-25T10:11:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.367+0000 7f2c02000640 1 -- start start 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2c02000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bfc0b48c0 0x7f2bfc153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2c02000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc0ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2c02000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bfc161bc0 0x7f2bfc0cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2c02000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b4ca0 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2c02000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc161260 con 0x7f2bfc0b48c0 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2c02000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b30b0 con 0x7f2bfc161bc0 2024-04-25T10:11:10.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.368+0000 7f2bfbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc0ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc0ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46382/0 (socket says 172.21.15.5:46382) 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2923886961 learned_addr learned my addr 172.21.15.5:0/2923886961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2c00c00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bfc0b48c0 0x7f2bfc153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bfc161bc0 msgr2=0x7f2bfc0cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bfc161bc0 0x7f2bfc0cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bfc0b48c0 msgr2=0x7f2bfc153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bfc0b48c0 0x7f2bfc153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 -- 172.21.15.5:0/2923886961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc0b0210 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2c00c00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bfc0b48c0 0x7f2bfc153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.369+0000 7f2bfbe00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc0ca6d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2be4000c00 tx=0x7f2be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.370+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4037040 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.370+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be4042070 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.370+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4030a50 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.370+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc0cca10 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.370+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc0cba10 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.371+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc068f60 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.372+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2be4030bf0 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.373+0000 7f2bf8c00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2bd806b4d0 0x7f2bd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.373+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2be40aae30 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.373+0000 7f2c00c00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2bd806b4d0 0x7f2bd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.375+0000 7f2c00c00640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2bd806b4d0 0x7f2bd806d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0012e0 tx=0x7f2bec009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.377+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2be4079c30 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.575+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bfc0d0dd0 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.576+0000 7f2bf8c00640 1 -- 172.21.15.5:0/2923886961 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2be4079530 con 0x7f2bfc0d3f90 2024-04-25T10:11:10.581 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:11:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2bd806b4d0 msgr2=0x7f2bd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2bd806b4d0 0x7f2bd806d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0012e0 tx=0x7f2bec009040 comp rx=0 tx=0).stop 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 msgr2=0x7f2bfc0ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc0ca6d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f2be4000c00 tx=0x7f2be40046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 shutdown_connections 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2bfc161bc0 0x7f2bfc0cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bfc0d3f90 0x7f2bfc0ca6d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2bd806b4d0 0x7f2bd806d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 --2- 172.21.15.5:0/2923886961 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2bfc0b48c0 0x7f2bfc153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 >> 172.21.15.5:0/2923886961 conn(0x7f2bfc08af80 msgr2=0x7f2bfc15eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 shutdown_connections 2024-04-25T10:11:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.579+0000 7f2c02000640 1 -- 172.21.15.5:0/2923886961 wait complete. 2024-04-25T10:11:10.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.808+0000 7f3ffe000640 1 Processor -- start 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.809+0000 7f3ffe000640 1 -- start start 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.810+0000 7f3ffe000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3ff8141da0 0x7f3ff806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.810+0000 7f3ffe000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ff806e640 0x7f3ff815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ffe000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff8071d30 con 0x7f3ff815d340 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ffe000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff8142180 con 0x7f3ff806e640 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ffe000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff806d620 con 0x7f3ff8141da0 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ff7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ff806e640 0x7f3ff815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ff7e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ff806e640 0x7f3ff815ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46384/0 (socket says 172.21.15.5:46384) 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ff7e00640 1 -- 172.21.15.5:0/1762643096 learned_addr learned my addr 172.21.15.5:0/1762643096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ff7e00640 1 -- 172.21.15.5:0/1762643096 >> v2:172.21.15.80:3300/0 conn(0x7f3ff8141da0 msgr2=0x7f3ff806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.811+0000 7f3ffcc00640 1 --2- 172.21.15.5:0/1762643096 >> v2:172.21.15.80:3300/0 conn(0x7f3ff8141da0 0x7f3ff806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.812+0000 7f3ff7e00640 1 --2- 172.21.15.5:0/1762643096 >> v2:172.21.15.80:3300/0 conn(0x7f3ff8141da0 0x7f3ff806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.812+0000 7f3ff7e00640 1 -- 172.21.15.5:0/1762643096 >> v1:172.21.15.80:6789/0 conn(0x7f3ff815d340 legacy=0x7f3ff80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.812+0000 7f3ff7e00640 1 -- 172.21.15.5:0/1762643096 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff806d900 con 0x7f3ff806e640 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.812+0000 7f3ffcc00640 1 --2- 172.21.15.5:0/1762643096 >> v2:172.21.15.80:3300/0 conn(0x7f3ff8141da0 0x7f3ff806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.813+0000 7f3ff7e00640 1 --2- 172.21.15.5:0/1762643096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff806e640 0x7f3ff815ce00 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3ff806bbb0 tx=0x7f3fe402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7fd2f799e9ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.813+0000 7f3ff7400640 1 -- 172.21.15.5:0/1762643096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4037070 con 0x7f3ff806e640 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.813+0000 7f3ff7400640 1 -- 172.21.15.5:0/1762643096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe4032440 con 0x7f3ff806e640 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.813+0000 7f3ff7400640 1 -- 172.21.15.5:0/1762643096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4032810 con 0x7f3ff806e640 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 -- 172.21.15.5:0/1762643096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff806e640 msgr2=0x7f3ff815ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 --2- 172.21.15.5:0/1762643096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff806e640 0x7f3ff815ce00 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f3ff806bbb0 tx=0x7f3fe402a230 comp rx=0 tx=0).stop 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 -- 172.21.15.5:0/1762643096 shutdown_connections 2024-04-25T10:11:10.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 --2- 172.21.15.5:0/1762643096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff806e640 0x7f3ff815ce00 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 --2- 172.21.15.5:0/1762643096 >> v2:172.21.15.80:3300/0 conn(0x7f3ff8141da0 0x7f3ff806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 -- 172.21.15.5:0/1762643096 >> 172.21.15.5:0/1762643096 conn(0x7f3ff8066f20 msgr2=0x7f3ff806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 -- 172.21.15.5:0/1762643096 shutdown_connections 2024-04-25T10:11:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.814+0000 7f3ffe000640 1 -- 172.21.15.5:0/1762643096 wait complete. 2024-04-25T10:11:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.816+0000 7f3ffe000640 1 Processor -- start 2024-04-25T10:11:10.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.817+0000 7f3ffe000640 1 -- start start 2024-04-25T10:11:10.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffe000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ff806e640 0x7f3ff80b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffe000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ff8141da0 0x7f3ff80c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffe000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 0x7f3ff8144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffe000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80cdb90 con 0x7f3ff815d340 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffe000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80cda10 con 0x7f3ff806e640 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffe000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8071000 con 0x7f3ff8141da0 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 0x7f3ff8144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 0x7f3ff8144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46388/0 (socket says 172.21.15.5:46388) 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ffd600640 1 -- 172.21.15.5:0/572918032 learned_addr learned my addr 172.21.15.5:0/572918032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.818+0000 7f3ff7e00640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ff8141da0 0x7f3ff80c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffcc00640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ff806e640 0x7f3ff80b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffd600640 1 -- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ff8141da0 msgr2=0x7f3ff80c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffd600640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ff8141da0 0x7f3ff80c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffd600640 1 -- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ff806e640 msgr2=0x7f3ff80b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffd600640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ff806e640 0x7f3ff80b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffd600640 1 -- 172.21.15.5:0/572918032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff80c9ff0 con 0x7f3ff815d340 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffd600640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 0x7f3ff8144290 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b9a0 tx=0x7f3ff000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0019070 con 0x7f3ff815d340 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000cba0 con 0x7f3ff815d340 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0014440 con 0x7f3ff815d340 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.819+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff80c8730 con 0x7f3ff815d340 2024-04-25T10:11:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.820+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff80c8cf0 con 0x7f3ff815d340 2024-04-25T10:11:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.821+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff8068f60 con 0x7f3ff815d340 2024-04-25T10:11:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.822+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ff001c070 con 0x7f3ff815d340 2024-04-25T10:11:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.823+0000 7f3ff4c00640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fd806b4d0 0x7f3fd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.823+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ff0089f10 con 0x7f3ff815d340 2024-04-25T10:11:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.823+0000 7f3ffcc00640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fd806b4d0 0x7f3fd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.825+0000 7f3ffcc00640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fd806b4d0 0x7f3fd806d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00a990 tx=0x7f3fec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:10.827+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ff0058c90 con 0x7f3ff815d340 2024-04-25T10:11:11.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.040+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3ff806ace0 con 0x7f3ff815d340 2024-04-25T10:11:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.041+0000 7f3ff4c00640 1 -- 172.21.15.5:0/572918032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3ff0058590 con 0x7f3ff815d340 2024-04-25T10:11:11.046 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fd806b4d0 msgr2=0x7f3fd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fd806b4d0 0x7f3fd806d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3fec00a990 tx=0x7f3fec008040 comp rx=0 tx=0).stop 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 msgr2=0x7f3ff8144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 0x7f3ff8144290 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b9a0 tx=0x7f3ff000be60 comp rx=0 tx=0).stop 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 shutdown_connections 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ff815d340 0x7f3ff8144290 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ff8141da0 0x7f3ff80c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3fd806b4d0 0x7f3fd806d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 --2- 172.21.15.5:0/572918032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ff806e640 0x7f3ff80b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 >> 172.21.15.5:0/572918032 conn(0x7f3ff8066f20 msgr2=0x7f3ff813fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 shutdown_connections 2024-04-25T10:11:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.044+0000 7f3ffe000640 1 -- 172.21.15.5:0/572918032 wait complete. 2024-04-25T10:11:11.068 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:11.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:11.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:11.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:11:11.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:11.173 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:11.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.425+0000 7f62bfe00640 1 Processor -- start 2024-04-25T10:11:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.426+0000 7f62bfe00640 1 -- start start 2024-04-25T10:11:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bfe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62c0143d30 0x7f62c015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bfe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f62c00b3ae0 0x7f62c00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bfe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c00b4880 con 0x7f62c00ce790 2024-04-25T10:11:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bfe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c00b45a0 con 0x7f62c0143d30 2024-04-25T10:11:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bfe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c015e2c0 con 0x7f62c00b3ae0 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62c0143d30 0x7f62c015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bea00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f62c0143d30 0x7f62c015eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46396/0 (socket says 172.21.15.5:46396) 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.427+0000 7f62bea00640 1 -- 172.21.15.5:0/2270172760 learned_addr learned my addr 172.21.15.5:0/2270172760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.428+0000 7f62b6000640 1 --2- 172.21.15.5:0/2270172760 >> v2:172.21.15.80:3300/0 conn(0x7f62c00b3ae0 0x7f62c00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.428+0000 7f62bea00640 1 -- 172.21.15.5:0/2270172760 >> v2:172.21.15.80:3300/0 conn(0x7f62c00b3ae0 msgr2=0x7f62c00b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.428+0000 7f62bea00640 1 --2- 172.21.15.5:0/2270172760 >> v2:172.21.15.80:3300/0 conn(0x7f62c00b3ae0 0x7f62c00b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.428+0000 7f62bea00640 1 -- 172.21.15.5:0/2270172760 >> v1:172.21.15.80:6789/0 conn(0x7f62c00ce790 legacy=0x7f62c015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.428+0000 7f62bea00640 1 -- 172.21.15.5:0/2270172760 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c015e5a0 con 0x7f62c0143d30 2024-04-25T10:11:11.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.429+0000 7f62bea00640 1 --2- 172.21.15.5:0/2270172760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c0143d30 0x7f62c015eda0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f62c00cd0d0 tx=0x7f62b002c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0144c44a8fe1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.429+0000 7f62be000640 1 -- 172.21.15.5:0/2270172760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0038070 con 0x7f62c0143d30 2024-04-25T10:11:11.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.429+0000 7f62be000640 1 -- 172.21.15.5:0/2270172760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b0031710 con 0x7f62c0143d30 2024-04-25T10:11:11.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.429+0000 7f62be000640 1 -- 172.21.15.5:0/2270172760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0031ae0 con 0x7f62c0143d30 2024-04-25T10:11:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 -- 172.21.15.5:0/2270172760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c0143d30 msgr2=0x7f62c015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 --2- 172.21.15.5:0/2270172760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c0143d30 0x7f62c015eda0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f62c00cd0d0 tx=0x7f62b002c440 comp rx=0 tx=0).stop 2024-04-25T10:11:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 -- 172.21.15.5:0/2270172760 shutdown_connections 2024-04-25T10:11:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 --2- 172.21.15.5:0/2270172760 >> v2:172.21.15.80:3300/0 conn(0x7f62c00b3ae0 0x7f62c00b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 --2- 172.21.15.5:0/2270172760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c0143d30 0x7f62c015eda0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 -- 172.21.15.5:0/2270172760 >> 172.21.15.5:0/2270172760 conn(0x7f62c0066f20 msgr2=0x7f62c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 -- 172.21.15.5:0/2270172760 shutdown_connections 2024-04-25T10:11:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.430+0000 7f62bfe00640 1 -- 172.21.15.5:0/2270172760 wait complete. 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.432+0000 7f62bfe00640 1 Processor -- start 2024-04-25T10:11:11.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.432+0000 7f62bfe00640 1 -- start start 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.433+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 0x7f62c0137d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.433+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62c00ce790 0x7f62c013bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.433+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62c0143d30 0x7f62c013c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.433+0000 7f62bfe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c0144290 con 0x7f62c00b3ae0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.433+0000 7f62bfe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c0144110 con 0x7f62c0143d30 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.433+0000 7f62bfe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c006e430 con 0x7f62c00ce790 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 0x7f62c0137d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 0x7f62c0137d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46410/0 (socket says 172.21.15.5:46410) 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 -- 172.21.15.5:0/482230152 learned_addr learned my addr 172.21.15.5:0/482230152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62b6000640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62c00ce790 0x7f62c013bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 -- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62c00ce790 msgr2=0x7f62c013bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62c00ce790 0x7f62c013bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 -- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62c0143d30 msgr2=0x7f62c013c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62c0143d30 0x7f62c013c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 -- 172.21.15.5:0/482230152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c015d410 con 0x7f62c00b3ae0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.434+0000 7f62bea00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 0x7f62c0137d00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f62b0000c00 tx=0x7f62b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.435+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0038040 con 0x7f62c00b3ae0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.435+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b0043070 con 0x7f62c00b3ae0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.435+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0033a50 con 0x7f62c00b3ae0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.435+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c00c91a0 con 0x7f62c00b3ae0 2024-04-25T10:11:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.435+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c00c9760 con 0x7f62c00b3ae0 2024-04-25T10:11:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.436+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c0067420 con 0x7f62c00b3ae0 2024-04-25T10:11:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.437+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62b0033bf0 con 0x7f62c00b3ae0 2024-04-25T10:11:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.438+0000 7f62b7400640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f629c06b4d0 0x7f629c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.438+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62b00ab220 con 0x7f62c00b3ae0 2024-04-25T10:11:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.438+0000 7f62b6000640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f629c06b4d0 0x7f629c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.440+0000 7f62b6000640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f629c06b4d0 0x7f629c06d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f62c01f67c0 tx=0x7f62a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.442+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62c0067420 con 0x7f62c00b3ae0 2024-04-25T10:11:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.669+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62c0068f60 con 0x7f62c00b3ae0 2024-04-25T10:11:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.670+0000 7f62b7400640 1 -- 172.21.15.5:0/482230152 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f62b0046360 con 0x7f62c00b3ae0 2024-04-25T10:11:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:11.676 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:02.669462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6715":{"gid":6715,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2001088034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2001088034},{"type":"v1","addr":"172.21.15.5:6835","nonce":2001088034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:11.676 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f629c06b4d0 msgr2=0x7f629c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f629c06b4d0 0x7f629c06d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f62c01f67c0 tx=0x7f62a8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 msgr2=0x7f62c0137d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 0x7f62c0137d00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f62b0000c00 tx=0x7f62b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 shutdown_connections 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f629c06b4d0 0x7f629c06d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62c0143d30 0x7f62c013c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f62c00ce790 0x7f62c013bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 --2- 172.21.15.5:0/482230152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c00b3ae0 0x7f62c0137d00 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 >> 172.21.15.5:0/482230152 conn(0x7f62c0066f20 msgr2=0x7f62c00cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 shutdown_connections 2024-04-25T10:11:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:11.673+0000 7f62bfe00640 1 -- 172.21.15.5:0/482230152 wait complete. 2024-04-25T10:11:11.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:11:11.932 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-04-25T10:11:11.932 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:11.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.126+0000 7f56a1400640 1 Processor -- start 2024-04-25T10:11:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.127+0000 7f56a1400640 1 -- start start 2024-04-25T10:11:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.128+0000 7f56a1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c072570 0x7f569c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.128+0000 7f56a1400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f569c0afad0 0x7f569c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.128+0000 7f56a1400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c0aed50 con 0x7f569c06ab60 2024-04-25T10:11:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.128+0000 7f56a1400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c0aeff0 con 0x7f569c0afad0 2024-04-25T10:11:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.128+0000 7f56a1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0af2d0 con 0x7f569c072570 2024-04-25T10:11:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.129+0000 7f569be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f569c06ab60 0x7f569c06af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34252/0 (socket says 172.21.15.5:34252) 2024-04-25T10:11:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.129+0000 7f569be00640 1 -- 172.21.15.5:0/2569520573 learned_addr learned my addr 172.21.15.5:0/2569520573 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.129+0000 7f569b400640 1 --2- 172.21.15.5:0/2569520573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c072570 0x7f569c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.129+0000 7f56a0a00640 1 --2- 172.21.15.5:0/2569520573 >> v2:172.21.15.80:3300/0 conn(0x7f569c0afad0 0x7f569c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.129+0000 7f569aa00640 1 -- 172.21.15.5:0/2569520573 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597040259 0 0) 0x7f569c0aed50 con 0x7f569c06ab60 2024-04-25T10:11:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.130+0000 7f569aa00640 1 -- 172.21.15.5:0/2569520573 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5680003520 con 0x7f569c06ab60 2024-04-25T10:11:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.130+0000 7f569aa00640 1 -- 172.21.15.5:0/2569520573 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 94753441 0 0) 0x7f5680003520 con 0x7f569c06ab60 2024-04-25T10:11:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.130+0000 7f569b400640 1 -- 172.21.15.5:0/2569520573 >> v2:172.21.15.80:3300/0 conn(0x7f569c0afad0 msgr2=0x7f569c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.130+0000 7f569b400640 1 --2- 172.21.15.5:0/2569520573 >> v2:172.21.15.80:3300/0 conn(0x7f569c0afad0 0x7f569c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.130+0000 7f569b400640 1 -- 172.21.15.5:0/2569520573 >> v1:172.21.15.5:6789/0 conn(0x7f569c06ab60 legacy=0x7f569c06af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.130+0000 7f569b400640 1 -- 172.21.15.5:0/2569520573 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0af5b0 con 0x7f569c072570 2024-04-25T10:11:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.131+0000 7f569b400640 1 --2- 172.21.15.5:0/2569520573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c072570 0x7f569c0b16f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56840097a0 tx=0x7f568402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d37bd1a77e198c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.132+0000 7f569aa00640 1 -- 172.21.15.5:0/2569520573 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684037070 con 0x7f569c072570 2024-04-25T10:11:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.132+0000 7f569aa00640 1 -- 172.21.15.5:0/2569520573 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568402aba0 con 0x7f569c072570 2024-04-25T10:11:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.132+0000 7f569aa00640 1 -- 172.21.15.5:0/2569520573 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c001670 con 0x7f569c072570 2024-04-25T10:11:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 -- 172.21.15.5:0/2569520573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c072570 msgr2=0x7f569c0b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 --2- 172.21.15.5:0/2569520573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c072570 0x7f569c0b16f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56840097a0 tx=0x7f568402a110 comp rx=0 tx=0).stop 2024-04-25T10:11:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 -- 172.21.15.5:0/2569520573 shutdown_connections 2024-04-25T10:11:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 --2- 172.21.15.5:0/2569520573 >> v2:172.21.15.80:3300/0 conn(0x7f569c0afad0 0x7f569c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 --2- 172.21.15.5:0/2569520573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c072570 0x7f569c0b16f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 -- 172.21.15.5:0/2569520573 >> 172.21.15.5:0/2569520573 conn(0x7f569c0b5ab0 msgr2=0x7f569c0999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 -- 172.21.15.5:0/2569520573 shutdown_connections 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.133+0000 7f56a1400640 1 -- 172.21.15.5:0/2569520573 wait complete. 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.135+0000 7f56a1400640 1 Processor -- start 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.135+0000 7f56a1400640 1 -- start start 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f569c06ab60 0x7f569c06f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 0x7f569c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c0afad0 0x7f569c09db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a1400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c1448d0 con 0x7f569c072570 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a1400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c144750 con 0x7f569c06ab60 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0b04a0 con 0x7f569c0afad0 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f569b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 0x7f569c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f569b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 0x7f569c0b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46426/0 (socket says 172.21.15.5:46426) 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f569b400640 1 -- 172.21.15.5:0/818551038 learned_addr learned my addr 172.21.15.5:0/818551038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f569be00640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f569c06ab60 0x7f569c06f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.136+0000 7f56a0a00640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c0afad0 0x7f569c09db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f569b400640 1 -- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c0afad0 msgr2=0x7f569c09db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f569b400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c0afad0 0x7f569c09db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f569b400640 1 -- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f569c06ab60 msgr2=0x7f569c06f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f569b400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f569c06ab60 0x7f569c06f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f569b400640 1 -- 172.21.15.5:0/818551038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c1f3790 con 0x7f569c072570 2024-04-25T10:11:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f569b400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 0x7f569c0b0aa0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5684035040 tx=0x7f56840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.137+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684041030 con 0x7f569c072570 2024-04-25T10:11:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.138+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568403d420 con 0x7f569c072570 2024-04-25T10:11:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.138+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c1f3a70 con 0x7f569c072570 2024-04-25T10:11:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.138+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684037040 con 0x7f569c072570 2024-04-25T10:11:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.138+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c1f4030 con 0x7f569c072570 2024-04-25T10:11:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.139+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c068f60 con 0x7f569c072570 2024-04-25T10:11:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.140+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5684044070 con 0x7f569c072570 2024-04-25T10:11:12.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.141+0000 7f5693e00640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f567806b4d0 0x7f567806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.141+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56840aa9d0 con 0x7f569c072570 2024-04-25T10:11:12.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.142+0000 7f569be00640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f567806b4d0 0x7f567806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.143+0000 7f569be00640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f567806b4d0 0x7f567806d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f568c005880 tx=0x7f568c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.146+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5684044e40 con 0x7f569c072570 2024-04-25T10:11:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.390+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f569c06d010 con 0x7f569c072570 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.392+0000 7f5693e00640 1 -- 172.21.15.5:0/818551038 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f5684079740 con 0x7f569c072570 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:02.669462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6715":{"gid":6715,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2001088034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2001088034},{"type":"v1","addr":"172.21.15.5:6835","nonce":2001088034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2024-04-25T10:11:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.394+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f567806b4d0 msgr2=0x7f567806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.394+0000 7f56a1400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f567806b4d0 0x7f567806d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f568c005880 tx=0x7f568c009040 comp rx=0 tx=0).stop 2024-04-25T10:11:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.394+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 msgr2=0x7f569c0b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.394+0000 7f56a1400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 0x7f569c0b0aa0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5684035040 tx=0x7f56840046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 shutdown_connections 2024-04-25T10:11:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f569c0afad0 0x7f569c09db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f569c072570 0x7f569c0b0aa0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f567806b4d0 0x7f567806d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 --2- 172.21.15.5:0/818551038 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f569c06ab60 0x7f569c06f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 >> 172.21.15.5:0/818551038 conn(0x7f569c0b5ab0 msgr2=0x7f569c06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 shutdown_connections 2024-04-25T10:11:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.395+0000 7f56a1400640 1 -- 172.21.15.5:0/818551038 wait complete. 2024-04-25T10:11:12.419 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T10:11:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.529+0000 7f31927b7f40 1 Processor -- start 2024-04-25T10:11:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.529+0000 7f31927b7f40 1 -- start start 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.530+0000 7f31927b7f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb945df80 0x562bb94ed920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.530+0000 7f31927b7f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 0x562bb94e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.530+0000 7f31927b7f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb94d0b20 0x562bb94eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.530+0000 7f31927b7f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb93badd0 con 0x562bb945fbd0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.530+0000 7f31927b7f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb9460380 con 0x562bb94d0b20 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.530+0000 7f31927b7f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bb94ed490 con 0x562bb945df80 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 0x562bb94e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 0x562bb94e8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46444/0 (socket says 172.21.15.5:46444) 2024-04-25T10:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 -- 172.21.15.5:0/2480752281 learned_addr learned my addr 172.21.15.5:0/2480752281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318fa00640 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb945df80 0x562bb94ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 -- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb945df80 msgr2=0x562bb94ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb945df80 0x562bb94ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 -- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb94d0b20 msgr2=0x562bb94eb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb94d0b20 0x562bb94eb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 -- 172.21.15.5:0/2480752281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bb94cc7f0 con 0x562bb945fbd0 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.531+0000 7f318f000640 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 0x562bb94e8b90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f317c00a7a0 tx=0x7f317c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b27068631a0cf03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f318dc00640 1 -- 172.21.15.5:0/2480752281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c019070 con 0x562bb945fbd0 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f318dc00640 1 -- 172.21.15.5:0/2480752281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c007600 con 0x562bb945fbd0 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f318dc00640 1 -- 172.21.15.5:0/2480752281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0079d0 con 0x562bb945fbd0 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 -- 172.21.15.5:0/2480752281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 msgr2=0x562bb94e8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 0x562bb94e8b90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f317c00a7a0 tx=0x7f317c00a890 comp rx=0 tx=0).stop 2024-04-25T10:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 -- 172.21.15.5:0/2480752281 shutdown_connections 2024-04-25T10:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb94d0b20 0x562bb94eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945fbd0 0x562bb94e8b90 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2480752281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb945df80 0x562bb94ed920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.532+0000 7f31927b7f40 1 -- 172.21.15.5:0/2480752281 >> 172.21.15.5:0/2480752281 conn(0x562bb9412c60 msgr2=0x562bb94ead90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.533+0000 7f31927b7f40 1 -- 172.21.15.5:0/2480752281 shutdown_connections 2024-04-25T10:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.533+0000 7f31927b7f40 1 -- 172.21.15.5:0/2480752281 wait complete. 2024-04-25T10:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.533+0000 7f31927b7f40 1 Processor -- start 2024-04-25T10:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 -- start start 2024-04-25T10:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945df80 0x562bb94cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb945fbd0 0x562bb94cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb94d0b20 0x562bb94eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb945d3b0 con 0x562bb945df80 2024-04-25T10:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb94ede60 con 0x562bb945fbd0 2024-04-25T10:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.534+0000 7f31927b7f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562bb945d530 con 0x562bb94d0b20 2024-04-25T10:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945df80 0x562bb94cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945df80 0x562bb94cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46448/0 (socket says 172.21.15.5:46448) 2024-04-25T10:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 -- 172.21.15.5:0/2752711869 learned_addr learned my addr 172.21.15.5:0/2752711869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f3190400640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb94d0b20 0x562bb94eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 -- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb94d0b20 msgr2=0x562bb94eb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562bb94d0b20 0x562bb94eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 -- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb945fbd0 msgr2=0x562bb94cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562bb945fbd0 0x562bb94cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.535+0000 7f318fa00640 1 -- 172.21.15.5:0/2752711869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bb9580dd0 con 0x562bb945df80 2024-04-25T10:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.536+0000 7f318fa00640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945df80 0x562bb94cb380 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f318000cc50 tx=0x7f318000a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.536+0000 7f3187e00640 1 -- 172.21.15.5:0/2752711869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000e640 con 0x562bb945df80 2024-04-25T10:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.536+0000 7f3187e00640 1 -- 172.21.15.5:0/2752711869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000e7e0 con 0x562bb945df80 2024-04-25T10:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.536+0000 7f3187e00640 1 -- 172.21.15.5:0/2752711869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318000ebb0 con 0x562bb945df80 2024-04-25T10:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.536+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562bb94ccaf0 con 0x562bb945df80 2024-04-25T10:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.536+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bb94cd0b0 con 0x562bb945df80 2024-04-25T10:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.538+0000 7f3187e00640 1 -- 172.21.15.5:0/2752711869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3180009030 con 0x562bb945df80 2024-04-25T10:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.539+0000 7f3187e00640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3170069eb0 0x7f317006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.539+0000 7f3187e00640 1 -- 172.21.15.5:0/2752711869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31800892c0 con 0x562bb945df80 2024-04-25T10:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.539+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x562bb94d30d0 0x562bb94d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.539+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) v9 -- 0x562bb94d39f0 con 0x562bb94d30d0 2024-04-25T10:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.540+0000 7f318f000640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3170069eb0 0x7f317006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.540+0000 7f3190400640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x562bb94d30d0 0x562bb94d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.541+0000 7f318f000640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3170069eb0 0x7f317006c370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x562bb9413a10 tx=0x7f317c00f000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.541+0000 7f3190400640 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x562bb94d30d0 0x562bb94d34b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f3190400640 1 -- 172.21.15.5:0/2752711869 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f3188052600 con 0x562bb94d30d0 2024-04-25T10:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x562bb94d30d0 msgr2=0x562bb94d34b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x562bb94d30d0 0x562bb94d34b0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3170069eb0 msgr2=0x7f317006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3170069eb0 0x7f317006c370 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x562bb9413a10 tx=0x7f317c00f000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945df80 msgr2=0x562bb94cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 --2- 172.21.15.5:0/2752711869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb945df80 0x562bb94cb380 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f318000cc50 tx=0x7f318000a6b0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f3190400640 1 -- 172.21.15.5:0/2752711869 reap_dead start 2024-04-25T10:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 shutdown_connections 2024-04-25T10:11:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.542+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 >> 172.21.15.5:0/2752711869 conn(0x562bb9412c60 msgr2=0x562bb94ead90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.543+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 shutdown_connections 2024-04-25T10:11:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.543+0000 7f31927b7f40 1 -- 172.21.15.5:0/2752711869 wait complete. 2024-04-25T10:11:12.561 INFO:teuthology.orchestra.run.smithi005.stdout:����� 2024-04-25T10:11:12.562 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:12.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.768+0000 7f5b4b200640 1 Processor -- start 2024-04-25T10:11:12.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.769+0000 7f5b4b200640 1 -- start start 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b4b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5b4c15d8a0 0x7f5b4c0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c1614e0 0x7f5b4c161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b4b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c15f410 con 0x7f5b4c0b2f30 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b4b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c15ffe0 con 0x7f5b4c15d8a0 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b4b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c15f6b0 con 0x7f5b4c1614e0 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b49e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5b4c15d8a0 0x7f5b4c0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b49400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5b4c0b2f30 0x7f5b4c0d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34282/0 (socket says 172.21.15.5:34282) 2024-04-25T10:11:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.770+0000 7f5b49400640 1 -- 172.21.15.5:0/1987196240 learned_addr learned my addr 172.21.15.5:0/1987196240 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b4a800640 1 --2- 172.21.15.5:0/1987196240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c1614e0 0x7f5b4c161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b48a00640 1 -- 172.21.15.5:0/1987196240 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407844665 0 0) 0x7f5b4c15f410 con 0x7f5b4c0b2f30 2024-04-25T10:11:12.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b48a00640 1 -- 172.21.15.5:0/1987196240 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b2c003520 con 0x7f5b4c0b2f30 2024-04-25T10:11:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b49e00640 1 -- 172.21.15.5:0/1987196240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c1614e0 msgr2=0x7f5b4c161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b49e00640 1 --2- 172.21.15.5:0/1987196240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c1614e0 0x7f5b4c161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b49e00640 1 -- 172.21.15.5:0/1987196240 >> v1:172.21.15.5:6789/0 conn(0x7f5b4c0b2f30 legacy=0x7f5b4c0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.771+0000 7f5b49e00640 1 -- 172.21.15.5:0/1987196240 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c15f990 con 0x7f5b4c15d8a0 2024-04-25T10:11:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.772+0000 7f5b49e00640 1 --2- 172.21.15.5:0/1987196240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c0d2570 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009790 tx=0x7f5b3c02a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2e42d4af95c5fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.772+0000 7f5b48a00640 1 -- 172.21.15.5:0/1987196240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c037070 con 0x7f5b4c15d8a0 2024-04-25T10:11:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b48a00640 1 -- 172.21.15.5:0/1987196240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c02ada0 con 0x7f5b4c15d8a0 2024-04-25T10:11:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b48a00640 1 -- 172.21.15.5:0/1987196240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c032640 con 0x7f5b4c15d8a0 2024-04-25T10:11:12.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 -- 172.21.15.5:0/1987196240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c15d8a0 msgr2=0x7f5b4c0d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 --2- 172.21.15.5:0/1987196240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c0d2570 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009790 tx=0x7f5b3c02a050 comp rx=0 tx=0).stop 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 -- 172.21.15.5:0/1987196240 shutdown_connections 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 --2- 172.21.15.5:0/1987196240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c1614e0 0x7f5b4c161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 --2- 172.21.15.5:0/1987196240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c0d2570 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 -- 172.21.15.5:0/1987196240 >> 172.21.15.5:0/1987196240 conn(0x7f5b4c08af80 msgr2=0x7f5b4c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.773+0000 7f5b4b200640 1 -- 172.21.15.5:0/1987196240 shutdown_connections 2024-04-25T10:11:12.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.774+0000 7f5b4b200640 1 -- 172.21.15.5:0/1987196240 wait complete. 2024-04-25T10:11:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.775+0000 7f5b4b200640 1 Processor -- start 2024-04-25T10:11:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.775+0000 7f5b4b200640 1 -- start start 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.776+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c0b2f30 0x7f5b4c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.776+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.776+0000 7f5b4b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 0x7f5b4c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.776+0000 7f5b4b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c15ee10 con 0x7f5b4c1614e0 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.776+0000 7f5b4b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c15ec90 con 0x7f5b4c15d8a0 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.776+0000 7f5b4b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0b2150 con 0x7f5b4c0b2f30 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 0x7f5b4c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 0x7f5b4c15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46456/0 (socket says 172.21.15.5:46456) 2024-04-25T10:11:12.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 -- 172.21.15.5:0/3605083745 learned_addr learned my addr 172.21.15.5:0/3605083745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b49e00640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c0b2f30 0x7f5b4c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 -- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c0b2f30 msgr2=0x7f5b4c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c0b2f30 0x7f5b4c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 -- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b4c15d8a0 msgr2=0x7f5b4c14d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b49400640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c14d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b4a800640 1 -- 172.21.15.5:0/3605083745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0ce450 con 0x7f5b4c1614e0 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.777+0000 7f5b49e00640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c0b2f30 0x7f5b4c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.778+0000 7f5b4a800640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 0x7f5b4c15ad80 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f5b44007e60 tx=0x7f5b4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.778+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44014420 con 0x7f5b4c1614e0 2024-04-25T10:11:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.778+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b44019070 con 0x7f5b4c1614e0 2024-04-25T10:11:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.778+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b44014820 con 0x7f5b4c1614e0 2024-04-25T10:11:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.778+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c15b2c0 con 0x7f5b4c1614e0 2024-04-25T10:11:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.778+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c0ce6c0 con 0x7f5b4c1614e0 2024-04-25T10:11:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.779+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c068f60 con 0x7f5b4c1614e0 2024-04-25T10:11:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.780+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b440149c0 con 0x7f5b4c1614e0 2024-04-25T10:11:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.781+0000 7f5b42000640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b2806b4d0 0x7f5b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.781+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b4408ac90 con 0x7f5b4c1614e0 2024-04-25T10:11:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.782+0000 7f5b49e00640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b2806b4d0 0x7f5b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.783+0000 7f5b49e00640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b2806b4d0 0x7f5b2806d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c034000 tx=0x7f5b3c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:12.785+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b440598e0 con 0x7f5b4c1614e0 2024-04-25T10:11:13.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.028+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b4c0d0e50 con 0x7f5b4c1614e0 2024-04-25T10:11:13.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.029+0000 7f5b42000640 1 -- 172.21.15.5:0/3605083745 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5b440591e0 con 0x7f5b4c1614e0 2024-04-25T10:11:13.035 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:13.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b2806b4d0 msgr2=0x7f5b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b2806b4d0 0x7f5b2806d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c034000 tx=0x7f5b3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 msgr2=0x7f5b4c15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 0x7f5b4c15ad80 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f5b44007e60 tx=0x7f5b4400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 shutdown_connections 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b4c1614e0 0x7f5b4c15ad80 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b2806b4d0 0x7f5b2806d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b4c15d8a0 0x7f5b4c14d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 --2- 172.21.15.5:0/3605083745 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b4c0b2f30 0x7f5b4c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.032+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 >> 172.21.15.5:0/3605083745 conn(0x7f5b4c08af80 msgr2=0x7f5b4c0b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.033+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 shutdown_connections 2024-04-25T10:11:13.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:13.033+0000 7f5b4b200640 1 -- 172.21.15.5:0/3605083745 wait complete. 2024-04-25T10:11:13.058 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:13.059 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:13.059 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:11:13.059 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:13.059 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:13.059 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:13.059 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:13.060 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:13.060 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:13.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.060 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:11:13.116 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:13.116 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:13.116 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:13.116 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.116 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:13.116 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.117 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:13.117 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:13.117 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:11:13.117 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:13.117 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:13.117 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:11:13.118 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:13.118 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:13.118 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:11:13.118 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:11:13.118 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42053sec preferred_lft 42053sec 2024-04-25T10:11:13.118 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:11:13.119 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.119 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:13.119 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:13.119 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:11:13.119 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:13.119 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:13.120 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:11:13.121 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:13.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.124 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:11:13.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.219 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:11:13.272 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:11:13.273 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:13.273 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:11:13.273 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:11:13.273 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:13.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.274 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:11:13.274 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:11:13.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:11:13.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:11:13.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:11:13.274 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:11:13.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.437 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:11:13.489 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:13.490 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:13.491 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:13.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:11:13.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:13.576 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:13.631 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:13.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:13.631 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:13.685 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:13.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:11:13.834 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:13.835 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:13.835 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:11:13.835 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:11:13.835 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:13.862 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:11:13.863 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:11:13.866 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:13.866 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:13.870 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:13.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:13.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:13.949 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:14.007 INFO:teuthology.orchestra.run.smithi005.stdout:client6808 2024-04-25T10:11:14.007 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:14.007 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:14.062 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:14.062 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:14.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.114 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:11:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records in 2024-04-25T10:11:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records out 2024-04-25T10:11:14.357 INFO:teuthology.orchestra.run.smithi005.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.181681 s, 34.6 MB/s 2024-04-25T10:11:14.358 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:14.359 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:14.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.418 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:11:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records in 2024-04-25T10:11:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:6+0 records out 2024-04-25T10:11:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.137428 s, 45.8 MB/s 2024-04-25T10:11:14.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.616 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:11:14.676 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-04-25T10:11:14.676 INFO:teuthology.orchestra.run.smithi005.stdout:./file2_sixmegs 2024-04-25T10:11:14.677 INFO:teuthology.orchestra.run.smithi005.stdout:./file3_sixmegs 2024-04-25T10:11:14.677 INFO:teuthology.orchestra.run.smithi005.stdout:./file1_sixmegs 2024-04-25T10:11:14.678 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:11:14.678 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:11:14.678 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:11:14.678 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:11:14.678 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:11:14.678 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:11:14.819 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:11:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:14.834 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:11:14.834 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:11:14.834 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:11:14.834 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:11:14.834 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T10:11:14.834 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:11:15.026 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:11:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:15.040 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:11:15.041 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:11:15.041 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:11:15.041 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:11:15.041 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T10:11:15.041 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:11:15.222 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-04-25T10:11:15.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:15.236 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:11:15.236 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:11:15.236 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:11:15.236 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:11:15.236 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T10:11:15.236 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:11:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:15.442 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:15.442 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:15.543 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:15.543 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:15.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.811+0000 7f4be3200640 1 Processor -- start 2024-04-25T10:11:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.812+0000 7f4be3200640 1 -- start start 2024-04-25T10:11:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.813+0000 7f4be3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be40d07d0 0x7f4be40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.813+0000 7f4be3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4be40d2380 0x7f4be40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.813+0000 7f4be3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be40cf410 con 0x7f4be4161200 2024-04-25T10:11:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.813+0000 7f4be3200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be40b28b0 con 0x7f4be40d2380 2024-04-25T10:11:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.813+0000 7f4be3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40cf6f0 con 0x7f4be40d07d0 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be1e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4be4161200 0x7f4be40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34296/0 (socket says 172.21.15.5:34296) 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be1e00640 1 -- 172.21.15.5:0/2370090056 learned_addr learned my addr 172.21.15.5:0/2370090056 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be2800640 1 --2- 172.21.15.5:0/2370090056 >> v2:172.21.15.5:3300/0 conn(0x7f4be40d2380 0x7f4be40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be1400640 1 --2- 172.21.15.5:0/2370090056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be40d07d0 0x7f4be40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be0a00640 1 -- 172.21.15.5:0/2370090056 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660553080 0 0) 0x7f4be40cf410 con 0x7f4be4161200 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be0a00640 1 -- 172.21.15.5:0/2370090056 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8003520 con 0x7f4be4161200 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be2800640 1 -- 172.21.15.5:0/2370090056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be40d07d0 msgr2=0x7f4be40b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be2800640 1 --2- 172.21.15.5:0/2370090056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be40d07d0 0x7f4be40b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be2800640 1 -- 172.21.15.5:0/2370090056 >> v1:172.21.15.5:6789/0 conn(0x7f4be4161200 legacy=0x7f4be40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.814+0000 7f4be2800640 1 -- 172.21.15.5:0/2370090056 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be40d3fd0 con 0x7f4be40d2380 2024-04-25T10:11:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.816+0000 7f4be2800640 1 --2- 172.21.15.5:0/2370090056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d2380 0x7f4be40b2f80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009d90 tx=0x7f4bdc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cfe4c11c2a8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.816+0000 7f4be0a00640 1 -- 172.21.15.5:0/2370090056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc007740 con 0x7f4be40d2380 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.816+0000 7f4be0a00640 1 -- 172.21.15.5:0/2370090056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc0078e0 con 0x7f4be40d2380 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.816+0000 7f4be0a00640 1 -- 172.21.15.5:0/2370090056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc007c90 con 0x7f4be40d2380 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.816+0000 7f4be3200640 1 -- 172.21.15.5:0/2370090056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d2380 msgr2=0x7f4be40b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.816+0000 7f4be3200640 1 --2- 172.21.15.5:0/2370090056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d2380 0x7f4be40b2f80 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009d90 tx=0x7f4bdc02a660 comp rx=0 tx=0).stop 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.817+0000 7f4be3200640 1 -- 172.21.15.5:0/2370090056 shutdown_connections 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.817+0000 7f4be3200640 1 --2- 172.21.15.5:0/2370090056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d2380 0x7f4be40b2f80 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.817+0000 7f4be3200640 1 --2- 172.21.15.5:0/2370090056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be40d07d0 0x7f4be40b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.817+0000 7f4be3200640 1 -- 172.21.15.5:0/2370090056 >> 172.21.15.5:0/2370090056 conn(0x7f4be408af80 msgr2=0x7f4be40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.817+0000 7f4be3200640 1 -- 172.21.15.5:0/2370090056 shutdown_connections 2024-04-25T10:11:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.817+0000 7f4be3200640 1 -- 172.21.15.5:0/2370090056 wait complete. 2024-04-25T10:11:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.818+0000 7f4be3200640 1 Processor -- start 2024-04-25T10:11:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.819+0000 7f4be3200640 1 -- start start 2024-04-25T10:11:15.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 0x7f4be415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be40d2380 0x7f4be40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be4161200 0x7f4be40b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40b3640 con 0x7f4be40d07d0 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40b34c0 con 0x7f4be40d2380 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be40b37c0 con 0x7f4be4161200 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 0x7f4be415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 0x7f4be415bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46480/0 (socket says 172.21.15.5:46480) 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 -- 172.21.15.5:0/275540145 learned_addr learned my addr 172.21.15.5:0/275540145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be2800640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be4161200 0x7f4be40b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 -- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be4161200 msgr2=0x7f4be40b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be4161200 0x7f4be40b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 -- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be40d2380 msgr2=0x7f4be40b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1400640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be40d2380 0x7f4be40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.820+0000 7f4be1e00640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be40d2380 0x7f4be40b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4be1e00640 1 -- 172.21.15.5:0/275540145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be415c2a0 con 0x7f4be40d07d0 2024-04-25T10:11:15.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4be1e00640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 0x7f4be415bd60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c390 tx=0x7f4bd400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4017070 con 0x7f4be40d07d0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd4005ce0 con 0x7f4be40d07d0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40072f0 con 0x7f4be40d07d0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be40cc900 con 0x7f4be40d07d0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.821+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be40ccec0 con 0x7f4be40d07d0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.823+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4068f60 con 0x7f4be40d07d0 2024-04-25T10:11:15.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.824+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bd401a070 con 0x7f4be40d07d0 2024-04-25T10:11:15.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.825+0000 7f4bda000640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4bc006b4d0 0x7f4bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.825+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4bd40897a0 con 0x7f4be40d07d0 2024-04-25T10:11:15.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.826+0000 7f4be1400640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4bc006b4d0 0x7f4bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.827+0000 7f4be1400640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4bc006b4d0 0x7f4bc006d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b5c60 tx=0x7f4bcc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:15.830+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bd4058480 con 0x7f4be40d07d0 2024-04-25T10:11:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.029+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4be40b5f70 con 0x7f4be40d07d0 2024-04-25T10:11:16.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.032+0000 7f4bda000640 1 -- 172.21.15.5:0/275540145 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16792 (secure 0 0 0) 0x7f4bd4057d80 con 0x7f4be40d07d0 2024-04-25T10:11:16.038 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:16.038 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:11:01.130972+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4bc006b4d0 msgr2=0x7f4bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4bc006b4d0 0x7f4bc006d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4be40b5c60 tx=0x7f4bcc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 msgr2=0x7f4be415bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 0x7f4be415bd60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c390 tx=0x7f4bd400c850 comp rx=0 tx=0).stop 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 shutdown_connections 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4be4161200 0x7f4be40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4bc006b4d0 0x7f4bc006d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be40d2380 0x7f4be40b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 --2- 172.21.15.5:0/275540145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4be40d07d0 0x7f4be415bd60 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 >> 172.21.15.5:0/275540145 conn(0x7f4be408af80 msgr2=0x7f4be41602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 shutdown_connections 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.035+0000 7f4be3200640 1 -- 172.21.15.5:0/275540145 wait complete. 2024-04-25T10:11:16.063 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:57.493997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.605360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:16.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.272+0000 7fecf7000640 1 Processor -- start 2024-04-25T10:11:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.273+0000 7fecf7000640 1 -- start start 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.274+0000 7fecf7000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf00cd5e0 0x7fecf009a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.274+0000 7fecf7000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecf006bf80 0x7fecf00999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.274+0000 7fecf7000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf015e180 con 0x7fecf0099f30 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.274+0000 7fecf7000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf00cbb50 con 0x7fecf006bf80 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.274+0000 7fecf7000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf009a380 con 0x7fecf00cd5e0 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf5200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecf006bf80 0x7fecf00999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf6600640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fecf0099f30 0x7fecf015d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34308/0 (socket says 172.21.15.5:34308) 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf6600640 1 -- 172.21.15.5:0/800398508 learned_addr learned my addr 172.21.15.5:0/800398508 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf5c00640 1 --2- 172.21.15.5:0/800398508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf00cd5e0 0x7fecf009a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf5200640 1 -- 172.21.15.5:0/800398508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf00cd5e0 msgr2=0x7fecf009a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf5200640 1 --2- 172.21.15.5:0/800398508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf00cd5e0 0x7fecf009a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf5200640 1 -- 172.21.15.5:0/800398508 >> v1:172.21.15.5:6789/0 conn(0x7fecf0099f30 legacy=0x7fecf015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecf5200640 1 -- 172.21.15.5:0/800398508 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf015e460 con 0x7fecf006bf80 2024-04-25T10:11:16.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.275+0000 7fecefe00640 1 -- 172.21.15.5:0/800398508 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112958438 0 0) 0x7fecf015e180 con 0x7fecf0099f30 2024-04-25T10:11:16.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.277+0000 7fecf5200640 1 --2- 172.21.15.5:0/800398508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf00999f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0097a0 tx=0x7fecdc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b2ea632babd134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.277+0000 7fecefe00640 1 -- 172.21.15.5:0/800398508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc037070 con 0x7fecf006bf80 2024-04-25T10:11:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.277+0000 7fecefe00640 1 -- 172.21.15.5:0/800398508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc032440 con 0x7fecf006bf80 2024-04-25T10:11:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.277+0000 7fecefe00640 1 -- 172.21.15.5:0/800398508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc032810 con 0x7fecf006bf80 2024-04-25T10:11:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 -- 172.21.15.5:0/800398508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 msgr2=0x7fecf00999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 --2- 172.21.15.5:0/800398508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf00999f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0097a0 tx=0x7fecdc02a180 comp rx=0 tx=0).stop 2024-04-25T10:11:16.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 -- 172.21.15.5:0/800398508 shutdown_connections 2024-04-25T10:11:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 --2- 172.21.15.5:0/800398508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf00999f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 --2- 172.21.15.5:0/800398508 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf00cd5e0 0x7fecf009a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 -- 172.21.15.5:0/800398508 >> 172.21.15.5:0/800398508 conn(0x7fecf00b2ff0 msgr2=0x7fecf00b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 -- 172.21.15.5:0/800398508 shutdown_connections 2024-04-25T10:11:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.278+0000 7fecf7000640 1 -- 172.21.15.5:0/800398508 wait complete. 2024-04-25T10:11:16.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.280+0000 7fecf7000640 1 Processor -- start 2024-04-25T10:11:16.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.280+0000 7fecf7000640 1 -- start start 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf7000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf01e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf7000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf0099f30 0x7fecf00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf7000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf00cd5e0 0x7fecf00c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf7000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf015ed90 con 0x7fecf006bf80 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf7000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf00cd9c0 con 0x7fecf00cd5e0 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf7000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf015ea20 con 0x7fecf0099f30 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf01e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf01e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46502/0 (socket says 172.21.15.5:46502) 2024-04-25T10:11:16.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.281+0000 7fecf5c00640 1 -- 172.21.15.5:0/2170146555 learned_addr learned my addr 172.21.15.5:0/2170146555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5200640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf0099f30 0x7fecf00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf6600640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf00cd5e0 0x7fecf00c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5c00640 1 -- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf0099f30 msgr2=0x7fecf00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5c00640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf0099f30 0x7fecf00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5c00640 1 -- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf00cd5e0 msgr2=0x7fecf00c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5c00640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf00cd5e0 0x7fecf00c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5c00640 1 -- 172.21.15.5:0/2170146555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf01e5330 con 0x7fecf006bf80 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf6600640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf00cd5e0 0x7fecf00c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:16.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7fecf5c00640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf01e63d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fece400d5f0 tx=0x7fece400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece40125f0 con 0x7fecf006bf80 2024-04-25T10:11:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.282+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece4019070 con 0x7fecf006bf80 2024-04-25T10:11:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.283+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf01e5610 con 0x7fecf006bf80 2024-04-25T10:11:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.283+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece40129d0 con 0x7fecf006bf80 2024-04-25T10:11:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.283+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf01e5bd0 con 0x7fecf006bf80 2024-04-25T10:11:16.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.284+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf0068f60 con 0x7fecf006bf80 2024-04-25T10:11:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.285+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fece4012b70 con 0x7fecf006bf80 2024-04-25T10:11:16.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.286+0000 7feced600640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd006b4d0 0x7fecd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.286+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fece408a8c0 con 0x7fecf006bf80 2024-04-25T10:11:16.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.286+0000 7fecf5200640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd006b4d0 0x7fecd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.288+0000 7fecf5200640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd006b4d0 0x7fecd006d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fecdc034000 tx=0x7fecdc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.290+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fece4059480 con 0x7fecf006bf80 2024-04-25T10:11:16.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.492+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecf00b3d50 con 0x7fecf006bf80 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.493+0000 7feced600640 1 -- 172.21.15.5:0/2170146555 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fece4058d80 con 0x7fecf006bf80 2024-04-25T10:11:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:11:16.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd006b4d0 msgr2=0x7fecd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd006b4d0 0x7fecd006d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fecdc034000 tx=0x7fecdc02e000 comp rx=0 tx=0).stop 2024-04-25T10:11:16.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 msgr2=0x7fecf01e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf01e63d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fece400d5f0 tx=0x7fece400dab0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 shutdown_connections 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd006b4d0 0x7fecd006d990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf00cd5e0 0x7fecf00c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf0099f30 0x7fecf00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 --2- 172.21.15.5:0/2170146555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf006bf80 0x7fecf01e63d0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 >> 172.21.15.5:0/2170146555 conn(0x7fecf00b2ff0 msgr2=0x7fecf00cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 shutdown_connections 2024-04-25T10:11:16.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.496+0000 7fecf7000640 1 -- 172.21.15.5:0/2170146555 wait complete. 2024-04-25T10:11:16.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.732+0000 7f842de00640 1 Processor -- start 2024-04-25T10:11:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.733+0000 7f842de00640 1 -- start start 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.734+0000 7f842de00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071460 0x7f84280b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.734+0000 7f842de00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842806d020 con 0x7f8428071000 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.734+0000 7f842de00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842806d2c0 con 0x7f842815f800 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.734+0000 7f842de00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842806ded0 con 0x7f8428071460 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.735+0000 7f8427e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8428071000 0x7f84280b3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:34314/0 (socket says 172.21.15.5:34314) 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.735+0000 7f8427e00640 1 -- 172.21.15.5:0/4180391604 learned_addr learned my addr 172.21.15.5:0/4180391604 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.735+0000 7f842d400640 1 --2- 172.21.15.5:0/4180391604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071460 0x7f84280b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.736+0000 7f8427400640 1 -- 172.21.15.5:0/4180391604 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837062488 0 0) 0x7f842806d020 con 0x7f8428071000 2024-04-25T10:11:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.736+0000 7f8427400640 1 -- 172.21.15.5:0/4180391604 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f840c003520 con 0x7f8428071000 2024-04-25T10:11:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.736+0000 7f842d400640 1 -- 172.21.15.5:0/4180391604 >> v1:172.21.15.80:6789/0 conn(0x7f842815f800 legacy=0x7f8428162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.736+0000 7f8427400640 1 -- 172.21.15.5:0/4180391604 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403891769 0 0) 0x7f842806d2c0 con 0x7f842815f800 2024-04-25T10:11:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.736+0000 7f842d400640 1 -- 172.21.15.5:0/4180391604 >> v1:172.21.15.5:6789/0 conn(0x7f8428071000 legacy=0x7f84280b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.736+0000 7f842d400640 1 -- 172.21.15.5:0/4180391604 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842806d5a0 con 0x7f8428071460 2024-04-25T10:11:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.737+0000 7f842d400640 1 --2- 172.21.15.5:0/4180391604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071460 0x7f84280b3f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84200097d0 tx=0x7f842002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce93bb31cf8216d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f8427400640 1 -- 172.21.15.5:0/4180391604 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420037070 con 0x7f8428071460 2024-04-25T10:11:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f8427400640 1 -- 172.21.15.5:0/4180391604 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420032440 con 0x7f8428071460 2024-04-25T10:11:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f8427400640 1 -- 172.21.15.5:0/4180391604 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84200327f0 con 0x7f8428071460 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f842de00640 1 -- 172.21.15.5:0/4180391604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071460 msgr2=0x7f84280b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f842de00640 1 --2- 172.21.15.5:0/4180391604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071460 0x7f84280b3f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84200097d0 tx=0x7f842002a210 comp rx=0 tx=0).stop 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f842de00640 1 -- 172.21.15.5:0/4180391604 shutdown_connections 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f842de00640 1 --2- 172.21.15.5:0/4180391604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071460 0x7f84280b3f50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.738+0000 7f842de00640 1 -- 172.21.15.5:0/4180391604 >> 172.21.15.5:0/4180391604 conn(0x7f8428066f20 msgr2=0x7f842806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.739+0000 7f842de00640 1 -- 172.21.15.5:0/4180391604 shutdown_connections 2024-04-25T10:11:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.739+0000 7f842de00640 1 -- 172.21.15.5:0/4180391604 wait complete. 2024-04-25T10:11:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.740+0000 7f842de00640 1 Processor -- start 2024-04-25T10:11:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.740+0000 7f842de00640 1 -- start start 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.741+0000 7f842de00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071000 0x7f8428157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.741+0000 7f842de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8428071460 0x7f84281f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.741+0000 7f842de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 0x7f842815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8428071890 con 0x7f842815f800 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842de00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842806db60 con 0x7f8428071460 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842de00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842806e960 con 0x7f8428071000 2024-04-25T10:11:16.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 0x7f842815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 0x7f842815b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46506/0 (socket says 172.21.15.5:46506) 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 -- 172.21.15.5:0/3907154227 learned_addr learned my addr 172.21.15.5:0/3907154227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842ca00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071000 0x7f8428157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 -- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071000 msgr2=0x7f8428157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f8427e00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8428071460 0x7f84281f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071000 0x7f8428157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 -- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8428071460 msgr2=0x7f84281f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8428071460 0x7f84281f7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842d400640 1 -- 172.21.15.5:0/3907154227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84281598e0 con 0x7f842815f800 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.742+0000 7f842ca00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071000 0x7f8428157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.743+0000 7f842d400640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 0x7f842815b150 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f842002a250 tx=0x7f84200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.743+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420037040 con 0x7f842815f800 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.743+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420042070 con 0x7f842815f800 2024-04-25T10:11:16.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.743+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420030940 con 0x7f842815f800 2024-04-25T10:11:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.743+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84281f82c0 con 0x7f842815f800 2024-04-25T10:11:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.743+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8428159ba0 con 0x7f842815f800 2024-04-25T10:11:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.744+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8428073ce0 con 0x7f842815f800 2024-04-25T10:11:16.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.745+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8420030ae0 con 0x7f842815f800 2024-04-25T10:11:16.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.746+0000 7f8424c00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f840806b500 0x7f840806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.746+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f84200aa720 con 0x7f842815f800 2024-04-25T10:11:16.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.747+0000 7f842ca00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f840806b500 0x7f840806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.748+0000 7f842ca00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f840806b500 0x7f840806d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8428163e30 tx=0x7f8418009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.750+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8428073ce0 con 0x7f842815f800 2024-04-25T10:11:16.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.949+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f842806ab60 con 0x7f842815f800 2024-04-25T10:11:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.950+0000 7f8424c00640 1 -- 172.21.15.5:0/3907154227 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) v1 ==== 104+0+11 (secure 0 0 0) 0x7f842806ab60 con 0x7f842815f800 2024-04-25T10:11:16.955 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:11:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f840806b500 msgr2=0x7f840806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f840806b500 0x7f840806d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8428163e30 tx=0x7f8418009040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 msgr2=0x7f842815b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 0x7f842815b150 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f842002a250 tx=0x7f84200046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 shutdown_connections 2024-04-25T10:11:16.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842815f800 0x7f842815b150 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f840806b500 0x7f840806d9c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8428071460 0x7f84281f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 --2- 172.21.15.5:0/3907154227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8428071000 0x7f8428157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 >> 172.21.15.5:0/3907154227 conn(0x7f8428066f20 msgr2=0x7f84281610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 shutdown_connections 2024-04-25T10:11:16.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:16.953+0000 7f842de00640 1 -- 172.21.15.5:0/3907154227 wait complete. 2024-04-25T10:11:16.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.192+0000 7fd96ae00640 1 Processor -- start 2024-04-25T10:11:17.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.192+0000 7fd96ae00640 1 -- start start 2024-04-25T10:11:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.193+0000 7fd96ae00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9640d0780 0x7fd9640d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.193+0000 7fd96ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640b9130 0x7fd9640d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.193+0000 7fd96ae00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd9640b9590 0x7fd9640d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.193+0000 7fd96ae00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9640d10a0 con 0x7fd9640d0780 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.193+0000 7fd96ae00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9640b7ef0 con 0x7fd9640b9590 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd96ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b81d0 con 0x7fd9640b9130 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9640d0780 0x7fd9640d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd9640d0780 0x7fd9640d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39806/0 (socket says 172.21.15.5:39806) 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 -- 172.21.15.5:0/3600304925 learned_addr learned my addr 172.21.15.5:0/3600304925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd968e00640 1 --2- 172.21.15.5:0/3600304925 >> v2:172.21.15.80:3300/0 conn(0x7fd9640b9590 0x7fd9640d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963400640 1 --2- 172.21.15.5:0/3600304925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640b9130 0x7fd9640d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 -- 172.21.15.5:0/3600304925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640b9130 msgr2=0x7fd9640d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 --2- 172.21.15.5:0/3600304925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640b9130 0x7fd9640d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 -- 172.21.15.5:0/3600304925 >> v2:172.21.15.80:3300/0 conn(0x7fd9640b9590 msgr2=0x7fd9640d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 --2- 172.21.15.5:0/3600304925 >> v2:172.21.15.80:3300/0 conn(0x7fd9640b9590 0x7fd9640d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd963e00640 1 -- 172.21.15.5:0/3600304925 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640b84b0 con 0x7fd9640d0780 2024-04-25T10:11:17.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.194+0000 7fd968e00640 1 --2- 172.21.15.5:0/3600304925 >> v2:172.21.15.80:3300/0 conn(0x7fd9640b9590 0x7fd9640d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.195+0000 7fd963e00640 1 --2- 172.21.15.5:0/3600304925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640d0780 0x7fd9640d0b60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd9640c72e0 tx=0x7fd95402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9605897ab2f613fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.196+0000 7fd962a00640 1 -- 172.21.15.5:0/3600304925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954037070 con 0x7fd9640d0780 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.196+0000 7fd962a00640 1 -- 172.21.15.5:0/3600304925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954032440 con 0x7fd9640d0780 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.196+0000 7fd962a00640 1 -- 172.21.15.5:0/3600304925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954032810 con 0x7fd9640d0780 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.196+0000 7fd96ae00640 1 -- 172.21.15.5:0/3600304925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640d0780 msgr2=0x7fd9640d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 --2- 172.21.15.5:0/3600304925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640d0780 0x7fd9640d0b60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd9640c72e0 tx=0x7fd95402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 -- 172.21.15.5:0/3600304925 shutdown_connections 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 --2- 172.21.15.5:0/3600304925 >> v2:172.21.15.80:3300/0 conn(0x7fd9640b9590 0x7fd9640d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 --2- 172.21.15.5:0/3600304925 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640b9130 0x7fd9640d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 --2- 172.21.15.5:0/3600304925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640d0780 0x7fd9640d0b60 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 -- 172.21.15.5:0/3600304925 >> 172.21.15.5:0/3600304925 conn(0x7fd9640726b0 msgr2=0x7fd964072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 -- 172.21.15.5:0/3600304925 shutdown_connections 2024-04-25T10:11:17.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.197+0000 7fd96ae00640 1 -- 172.21.15.5:0/3600304925 wait complete. 2024-04-25T10:11:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.199+0000 7fd96ae00640 1 Processor -- start 2024-04-25T10:11:17.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.199+0000 7fd96ae00640 1 -- start start 2024-04-25T10:11:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.200+0000 7fd96ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 0x7fd96415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd96ae00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9640b9590 0x7fd9640cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd96ae00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640d0780 0x7fd9640c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd96ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641611b0 con 0x7fd9640b9130 2024-04-25T10:11:17.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd96ae00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640b8a70 con 0x7fd9640b9590 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd96ae00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd964161330 con 0x7fd9640d0780 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 0x7fd96415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 0x7fd96415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39818/0 (socket says 172.21.15.5:39818) 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 -- 172.21.15.5:0/1643479517 learned_addr learned my addr 172.21.15.5:0/1643479517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd968e00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640d0780 0x7fd9640c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963400640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9640b9590 0x7fd9640cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 -- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640d0780 msgr2=0x7fd9640c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640d0780 0x7fd9640c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 -- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9640b9590 msgr2=0x7fd9640cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9640b9590 0x7fd9640cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.201+0000 7fd963e00640 1 -- 172.21.15.5:0/1643479517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9641f4360 con 0x7fd9640b9130 2024-04-25T10:11:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.202+0000 7fd963e00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 0x7fd96415aca0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd95402a220 tx=0x7fd9540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.202+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954037040 con 0x7fd9640b9130 2024-04-25T10:11:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.202+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954042070 con 0x7fd9640b9130 2024-04-25T10:11:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.202+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9640cab40 con 0x7fd9640b9130 2024-04-25T10:11:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.202+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954030bb0 con 0x7fd9640b9130 2024-04-25T10:11:17.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.202+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96415b1e0 con 0x7fd9640b9130 2024-04-25T10:11:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.204+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964068f60 con 0x7fd9640b9130 2024-04-25T10:11:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.205+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd954047460 con 0x7fd9640b9130 2024-04-25T10:11:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.206+0000 7fd95be00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd94006b4d0 0x7fd94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.206+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9540aafd0 con 0x7fd9640b9130 2024-04-25T10:11:17.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.206+0000 7fd963400640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd94006b4d0 0x7fd94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.207+0000 7fd963400640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd94006b4d0 0x7fd94006d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9640b1e90 tx=0x7fd94c0083c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.210+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd954045360 con 0x7fd9640b9130 2024-04-25T10:11:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.410+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd96415e0d0 con 0x7fd9640b9130 2024-04-25T10:11:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.411+0000 7fd95be00640 1 -- 172.21.15.5:0/1643479517 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd954079cb0 con 0x7fd9640b9130 2024-04-25T10:11:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:11:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd94006b4d0 msgr2=0x7fd94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd94006b4d0 0x7fd94006d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9640b1e90 tx=0x7fd94c0083c0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 msgr2=0x7fd96415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 0x7fd96415aca0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fd95402a220 tx=0x7fd9540046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 shutdown_connections 2024-04-25T10:11:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd9640d0780 0x7fd9640c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd94006b4d0 0x7fd94006d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9640b9590 0x7fd9640cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 --2- 172.21.15.5:0/1643479517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9640b9130 0x7fd96415aca0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 >> 172.21.15.5:0/1643479517 conn(0x7fd9640726b0 msgr2=0x7fd9640d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 shutdown_connections 2024-04-25T10:11:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.414+0000 7fd96ae00640 1 -- 172.21.15.5:0/1643479517 wait complete. 2024-04-25T10:11:17.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:17.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.651+0000 7f69ffc00640 1 Processor -- start 2024-04-25T10:11:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.652+0000 7f69ffc00640 1 -- start start 2024-04-25T10:11:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.653+0000 7f69ffc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f80d0aa0 0x7f69f8161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.653+0000 7f69ffc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f815e8a0 con 0x7f69f815da60 2024-04-25T10:11:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.653+0000 7f69ffc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f815eb80 con 0x7f69f81622b0 2024-04-25T10:11:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.653+0000 7f69ffc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815ee60 con 0x7f69f80d0aa0 2024-04-25T10:11:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.654+0000 7f69fd200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f69f815da60 0x7f69f815de40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53636/0 (socket says 172.21.15.5:53636) 2024-04-25T10:11:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.654+0000 7f69fd200640 1 -- 172.21.15.5:0/3976250958 learned_addr learned my addr 172.21.15.5:0/3976250958 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.654+0000 7f69f7e00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f80d0aa0 0x7f69f8161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.654+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375590207 0 0) 0x7f69f815e8a0 con 0x7f69f815da60 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.654+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69dc003520 con 0x7f69f815da60 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.654+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175693574 0 0) 0x7f69dc003520 con 0x7f69f815da60 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f80d0aa0 msgr2=0x7f69f8161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f80d0aa0 0x7f69f8161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 >> v1:172.21.15.80:6789/0 conn(0x7f69f81622b0 legacy=0x7f69f80ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f8161850 con 0x7f69f815da60 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f69e80019f0 con 0x7f69f815da60 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 >> v1:172.21.15.5:6789/0 conn(0x7f69f815da60 legacy=0x7f69f815de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69dc0039e0 0x7f69dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc004800 0x7f69dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69dc005210 0x7f69dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f815ee60 con 0x7f69dc005210 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc00c110 con 0x7f69dc0039e0 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc00c310 con 0x7f69dc004800 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69f7e00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69f80d0aa0 0x7f69f8161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.655+0000 7f69fdc00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69dc005210 0x7f69dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69fd200640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69dc0039e0 0x7f69dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69f7e00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc004800 0x7f69dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69fdc00640 1 -- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc004800 msgr2=0x7f69dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69fdc00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc004800 0x7f69dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69fdc00640 1 -- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69dc0039e0 msgr2=0x7f69dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69fdc00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69dc0039e0 0x7f69dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69fdc00640 1 -- 172.21.15.5:0/3976250958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc003520 con 0x7f69dc005210 2024-04-25T10:11:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.656+0000 7f69f7e00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc004800 0x7f69dc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.657+0000 7f69fdc00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69dc005210 0x7f69dc00b9c0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f69f0001750 tx=0x7f69dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ec9a7a0972c60f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.657+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0001370 con 0x7f69dc005210 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.657+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f00052d0 con 0x7f69dc005210 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.657+0000 7f69f7400640 1 -- 172.21.15.5:0/3976250958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f002abe0 con 0x7f69dc005210 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69ffc00640 1 -- 172.21.15.5:0/3976250958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69dc005210 msgr2=0x7f69dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69ffc00640 1 --2- 172.21.15.5:0/3976250958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69dc005210 0x7f69dc00b9c0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f69f0001750 tx=0x7f69dc005830 comp rx=0 tx=0).stop 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69fdc00640 1 -- 172.21.15.5:0/3976250958 reap_dead start 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69ffc00640 1 -- 172.21.15.5:0/3976250958 shutdown_connections 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69ffc00640 1 -- 172.21.15.5:0/3976250958 >> 172.21.15.5:0/3976250958 conn(0x7f69f806bfe0 msgr2=0x7f69f806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69ffc00640 1 -- 172.21.15.5:0/3976250958 shutdown_connections 2024-04-25T10:11:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.658+0000 7f69ffc00640 1 -- 172.21.15.5:0/3976250958 wait complete. 2024-04-25T10:11:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.660+0000 7f69ffc00640 1 Processor -- start 2024-04-25T10:11:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.660+0000 7f69ffc00640 1 -- start start 2024-04-25T10:11:17.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69ffc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc005210 0x7f69f80cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69ffc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f80cc860 0x7f69f80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69ffc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 0x7f69f8156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69ffc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80cedb0 con 0x7f69f8156910 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69ffc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80cec30 con 0x7f69f80cc860 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69ffc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8161d70 con 0x7f69dc005210 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.661+0000 7f69fdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 0x7f69f8156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 0x7f69f8156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39824/0 (socket says 172.21.15.5:39824) 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 -- 172.21.15.5:0/2010562218 learned_addr learned my addr 172.21.15.5:0/2010562218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69f7e00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f80cc860 0x7f69f80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fd200640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc005210 0x7f69f80cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 -- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc005210 msgr2=0x7f69f80cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc005210 0x7f69f80cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 -- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f80cc860 msgr2=0x7f69f80ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f80cc860 0x7f69f80ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 -- 172.21.15.5:0/2010562218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80d2f40 con 0x7f69f8156910 2024-04-25T10:11:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.662+0000 7f69fdc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 0x7f69f8156d90 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f69f0001870 tx=0x7f69f0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.663+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0037040 con 0x7f69f8156910 2024-04-25T10:11:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.663+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0040070 con 0x7f69f8156910 2024-04-25T10:11:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.663+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0001370 con 0x7f69f8156910 2024-04-25T10:11:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.663+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f80cd220 con 0x7f69f8156910 2024-04-25T10:11:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.663+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f80cd7b0 con 0x7f69f8156910 2024-04-25T10:11:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.664+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8068fd0 con 0x7f69f8156910 2024-04-25T10:11:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.665+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69f00052d0 con 0x7f69f8156910 2024-04-25T10:11:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.666+0000 7f69f4c00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d806b4d0 0x7f69d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.666+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69f00a9a60 con 0x7f69f8156910 2024-04-25T10:11:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.667+0000 7f69fd200640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d806b4d0 0x7f69d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.668+0000 7f69fd200640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d806b4d0 0x7f69d806d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f69e8002040 tx=0x7f69e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.670+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69f00787e0 con 0x7f69f8156910 2024-04-25T10:11:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.887+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f69f80d1560 con 0x7f69f8156910 2024-04-25T10:11:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.888+0000 7f69f4c00640 1 -- 172.21.15.5:0/2010562218 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) v1 ==== 80+0+0 (secure 0 0 0) 0x7f69f00780e0 con 0x7f69f8156910 2024-04-25T10:11:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:11:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d806b4d0 msgr2=0x7f69d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d806b4d0 0x7f69d806d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f69e8002040 tx=0x7f69e8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 msgr2=0x7f69f8156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 0x7f69f8156d90 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f69f0001870 tx=0x7f69f0001120 comp rx=0 tx=0).stop 2024-04-25T10:11:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 shutdown_connections 2024-04-25T10:11:17.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f8156910 0x7f69f8156d90 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f69d806b4d0 0x7f69d806d990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69f80cc860 0x7f69f80ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 --2- 172.21.15.5:0/2010562218 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69dc005210 0x7f69f80cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 >> 172.21.15.5:0/2010562218 conn(0x7f69f806bfe0 msgr2=0x7f69f80d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 shutdown_connections 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:17.891+0000 7f69ffc00640 1 -- 172.21.15.5:0/2010562218 wait complete. 2024-04-25T10:11:17.916 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:17.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:17.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:18.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:11:18.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:18.042 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:18.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.274+0000 7ff660e00640 1 Processor -- start 2024-04-25T10:11:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.274+0000 7ff660e00640 1 -- start start 2024-04-25T10:11:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff660e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c1529e0 0x7ff65c0c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff660e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff65c0c6410 0x7ff65c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff660e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c15bdc0 con 0x7ff65c071ed0 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff660e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c0b2340 con 0x7ff65c0c6410 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff660e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c15c060 con 0x7ff65c1529e0 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff653e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff65c0c6410 0x7ff65c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff65a800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff65c071ed0 0x7ff65c0c4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53638/0 (socket says 172.21.15.5:53638) 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff65a800640 1 -- 172.21.15.5:0/1190385207 learned_addr learned my addr 172.21.15.5:0/1190385207 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.276+0000 7ff659e00640 1 --2- 172.21.15.5:0/1190385207 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c1529e0 0x7ff65c0c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.277+0000 7ff653e00640 1 -- 172.21.15.5:0/1190385207 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c1529e0 msgr2=0x7ff65c0c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.277+0000 7ff653e00640 1 --2- 172.21.15.5:0/1190385207 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c1529e0 0x7ff65c0c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.277+0000 7ff653e00640 1 -- 172.21.15.5:0/1190385207 >> v1:172.21.15.5:6789/0 conn(0x7ff65c071ed0 legacy=0x7ff65c0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.277+0000 7ff653e00640 1 -- 172.21.15.5:0/1190385207 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c15c340 con 0x7ff65c0c6410 2024-04-25T10:11:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.277+0000 7ff659400640 1 -- 172.21.15.5:0/1190385207 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939420577 0 0) 0x7ff65c15bdc0 con 0x7ff65c071ed0 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.278+0000 7ff653e00640 1 --2- 172.21.15.5:0/1190385207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c0ce350 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff6440097a0 tx=0x7ff64402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47828f447b0ec740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.278+0000 7ff659400640 1 -- 172.21.15.5:0/1190385207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644037070 con 0x7ff65c0c6410 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.278+0000 7ff659400640 1 -- 172.21.15.5:0/1190385207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644032440 con 0x7ff65c0c6410 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.278+0000 7ff659400640 1 -- 172.21.15.5:0/1190385207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644032810 con 0x7ff65c0c6410 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 -- 172.21.15.5:0/1190385207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 msgr2=0x7ff65c0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 --2- 172.21.15.5:0/1190385207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c0ce350 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7ff6440097a0 tx=0x7ff64402a180 comp rx=0 tx=0).stop 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 -- 172.21.15.5:0/1190385207 shutdown_connections 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 --2- 172.21.15.5:0/1190385207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c0ce350 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 --2- 172.21.15.5:0/1190385207 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c1529e0 0x7ff65c0c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 -- 172.21.15.5:0/1190385207 >> 172.21.15.5:0/1190385207 conn(0x7ff65c071150 msgr2=0x7ff65c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 -- 172.21.15.5:0/1190385207 shutdown_connections 2024-04-25T10:11:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.279+0000 7ff660e00640 1 -- 172.21.15.5:0/1190385207 wait complete. 2024-04-25T10:11:18.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.281+0000 7ff660e00640 1 Processor -- start 2024-04-25T10:11:18.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.281+0000 7ff660e00640 1 -- start start 2024-04-25T10:11:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff660e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff65c071ed0 0x7ff65c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff660e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff660e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c157e60 0x7ff65c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff660e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c06e160 con 0x7ff65c0c6410 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff660e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c15c900 con 0x7ff65c071ed0 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff660e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c06e2e0 con 0x7ff65c157e60 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff653e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff653e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c14d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39844/0 (socket says 172.21.15.5:39844) 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.282+0000 7ff653e00640 1 -- 172.21.15.5:0/2348079490 learned_addr learned my addr 172.21.15.5:0/2348079490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff653e00640 1 -- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c157e60 msgr2=0x7ff65c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff653e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c157e60 0x7ff65c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff653e00640 1 -- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff65c071ed0 msgr2=0x7ff65c1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff659e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff65c071ed0 0x7ff65c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff653e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff65c071ed0 0x7ff65c1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff653e00640 1 -- 172.21.15.5:0/2348079490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c14e350 con 0x7ff65c0c6410 2024-04-25T10:11:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff653e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c14d990 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff644000c00 tx=0x7ff6440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644037040 con 0x7ff65c0c6410 2024-04-25T10:11:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644042070 con 0x7ff65c0c6410 2024-04-25T10:11:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644030bb0 con 0x7ff65c0c6410 2024-04-25T10:11:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.283+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c158390 con 0x7ff65c0c6410 2024-04-25T10:11:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.284+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c157040 con 0x7ff65c0c6410 2024-04-25T10:11:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.285+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c154780 con 0x7ff65c0c6410 2024-04-25T10:11:18.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.286+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff644047460 con 0x7ff65c0c6410 2024-04-25T10:11:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.287+0000 7ff652000640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff63806b4d0 0x7ff63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.287+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6440aaf50 con 0x7ff65c0c6410 2024-04-25T10:11:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.288+0000 7ff659e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff63806b4d0 0x7ff63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.289+0000 7ff659e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff63806b4d0 0x7ff63806d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.291+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff65c154780 con 0x7ff65c0c6410 2024-04-25T10:11:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.523+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff65c068fd0 con 0x7ff65c0c6410 2024-04-25T10:11:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.524+0000 7ff652000640 1 -- 172.21.15.5:0/2348079490 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7ff644032e20 con 0x7ff65c0c6410 2024-04-25T10:11:18.530 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:02.669462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6715},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6715":{"gid":6715,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2001088034","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2001088034},{"type":"v1","addr":"172.21.15.5:6835","nonce":2001088034}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 83 2024-04-25T10:11:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff63806b4d0 msgr2=0x7ff63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff63806b4d0 0x7ff63806d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff64c006fd0 tx=0x7ff64c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 msgr2=0x7ff65c14d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c14d990 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ff644000c00 tx=0x7ff6440046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 shutdown_connections 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff65c157e60 0x7ff65c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff65c0c6410 0x7ff65c14d990 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff63806b4d0 0x7ff63806d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 --2- 172.21.15.5:0/2348079490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff65c071ed0 0x7ff65c1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.526+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 >> 172.21.15.5:0/2348079490 conn(0x7ff65c071150 msgr2=0x7ff65c15b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.527+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 shutdown_connections 2024-04-25T10:11:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:18.527+0000 7ff660e00640 1 -- 172.21.15.5:0/2348079490 wait complete. 2024-04-25T10:11:18.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:11:18.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"message":"","return_code":0} 2024-04-25T10:11:18.789 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:18.790 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:11:18.790 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:18.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:18.845 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:11:18.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:18.845 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:18.896 INFO:teuthology.orchestra.run.smithi005.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:11:18.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:18.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:11:18.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:18.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:11:19.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.088+0000 7f47fba00640 1 Processor -- start 2024-04-25T10:11:19.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.089+0000 7f47fba00640 1 -- start start 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.090+0000 7f47fba00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47f40d3330 0x7f47f40d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.090+0000 7f47fba00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47f40b9290 0x7f47f40d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.090+0000 7f47fba00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40d2290 con 0x7f47f415de70 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.090+0000 7f47fba00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40d2030 con 0x7f47f40d3330 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.090+0000 7f47fba00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f40d2570 con 0x7f47f40b9290 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.090+0000 7f47f3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47f40d3330 0x7f47f40d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f47f40d3330 0x7f47f40d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39852/0 (socket says 172.21.15.5:39852) 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f3e00640 1 -- 172.21.15.5:0/4190477529 learned_addr learned my addr 172.21.15.5:0/4190477529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f9a00640 1 --2- 172.21.15.5:0/4190477529 >> v2:172.21.15.80:3300/0 conn(0x7f47f40b9290 0x7f47f40d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f3e00640 1 -- 172.21.15.5:0/4190477529 >> v2:172.21.15.80:3300/0 conn(0x7f47f40b9290 msgr2=0x7f47f40d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f3e00640 1 --2- 172.21.15.5:0/4190477529 >> v2:172.21.15.80:3300/0 conn(0x7f47f40b9290 0x7f47f40d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f3e00640 1 -- 172.21.15.5:0/4190477529 >> v1:172.21.15.80:6789/0 conn(0x7f47f415de70 legacy=0x7f47f40d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.091+0000 7f47f3e00640 1 -- 172.21.15.5:0/4190477529 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40d2850 con 0x7f47f40d3330 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.092+0000 7f47f3e00640 1 --2- 172.21.15.5:0/4190477529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f40d3800 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f47f40ca8d0 tx=0x7f47e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d2c3c1751bf3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.092+0000 7f47f3400640 1 -- 172.21.15.5:0/4190477529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0037070 con 0x7f47f40d3330 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.093+0000 7f47f3400640 1 -- 172.21.15.5:0/4190477529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e0032440 con 0x7f47f40d3330 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.093+0000 7f47f3400640 1 -- 172.21.15.5:0/4190477529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0032810 con 0x7f47f40d3330 2024-04-25T10:11:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.093+0000 7f47fba00640 1 -- 172.21.15.5:0/4190477529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 msgr2=0x7f47f40d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.093+0000 7f47fba00640 1 --2- 172.21.15.5:0/4190477529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f40d3800 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f47f40ca8d0 tx=0x7f47e002a230 comp rx=0 tx=0).stop 2024-04-25T10:11:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.094+0000 7f47fba00640 1 -- 172.21.15.5:0/4190477529 shutdown_connections 2024-04-25T10:11:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.094+0000 7f47fba00640 1 --2- 172.21.15.5:0/4190477529 >> v2:172.21.15.80:3300/0 conn(0x7f47f40b9290 0x7f47f40d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.094+0000 7f47fba00640 1 --2- 172.21.15.5:0/4190477529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f40d3800 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.094+0000 7f47fba00640 1 -- 172.21.15.5:0/4190477529 >> 172.21.15.5:0/4190477529 conn(0x7f47f40726b0 msgr2=0x7f47f4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.094+0000 7f47fba00640 1 -- 172.21.15.5:0/4190477529 shutdown_connections 2024-04-25T10:11:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.094+0000 7f47fba00640 1 -- 172.21.15.5:0/4190477529 wait complete. 2024-04-25T10:11:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.095+0000 7f47fba00640 1 Processor -- start 2024-04-25T10:11:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.096+0000 7f47fba00640 1 -- start start 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.096+0000 7f47fba00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f40b9290 0x7f47f41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.096+0000 7f47fba00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f41f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47fba00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47f415de70 0x7f47f415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47fba00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b6a10 con 0x7f47f40d3330 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47fba00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40d2e10 con 0x7f47f40b9290 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47fba00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b6b90 con 0x7f47f415de70 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f41f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f41f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39862/0 (socket says 172.21.15.5:39862) 2024-04-25T10:11:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 -- 172.21.15.5:0/824263994 learned_addr learned my addr 172.21.15.5:0/824263994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f9a00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47f415de70 0x7f47f415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f9000640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f40b9290 0x7f47f41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 -- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47f415de70 msgr2=0x7f47f415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47f415de70 0x7f47f415b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 -- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f40b9290 msgr2=0x7f47f41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f40b9290 0x7f47f41523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.097+0000 7f47f3e00640 1 -- 172.21.15.5:0/824263994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f415a0e0 con 0x7f47f40d3330 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.098+0000 7f47f3e00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f41f30d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f47e002a270 tx=0x7f47e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.098+0000 7f47f0c00640 1 -- 172.21.15.5:0/824263994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0037040 con 0x7f47f40d3330 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.098+0000 7f47f0c00640 1 -- 172.21.15.5:0/824263994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e0042070 con 0x7f47f40d3330 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.098+0000 7f47f0c00640 1 -- 172.21.15.5:0/824263994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0030bb0 con 0x7f47f40d3330 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.098+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f415a3c0 con 0x7f47f40d3330 2024-04-25T10:11:19.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.098+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f415a980 con 0x7f47f40d3330 2024-04-25T10:11:19.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.102+0000 7f47f0c00640 1 -- 172.21.15.5:0/824263994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47e0047460 con 0x7f47f40d3330 2024-04-25T10:11:19.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.103+0000 7f47f0c00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47d406b4d0 0x7f47d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.103+0000 7f47f0c00640 1 -- 172.21.15.5:0/824263994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47e00aaff0 con 0x7f47f40d3330 2024-04-25T10:11:19.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.103+0000 7f47fe858740 1 Processor -- start 2024-04-25T10:11:19.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.103+0000 7f47fe858740 1 -- start start 2024-04-25T10:11:19.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.103+0000 7f47f9000640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47d406b4d0 0x7f47d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c56740d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c56742af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47fe858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 0x55c567429100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47fe858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c567345980 con 0x55c5673e2e80 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5672eede0 con 0x55c56740d1a0 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c567341a20 con 0x55c56742aa40 2024-04-25T10:11:19.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.104+0000 7f47f9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 0x55c567429100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 0x55c567429100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39868/0 (socket says 172.21.15.5:39868) 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 -- 172.21.15.5:0/2406037558 learned_addr learned my addr 172.21.15.5:0/2406037558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9000640 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c56740d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f3e00640 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c56742af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 -- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 msgr2=0x55c56742af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c56742af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9000640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47d406b4d0 0x7f47d406d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f47e800a7a0 tx=0x7f47e8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 -- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 msgr2=0x55c56740d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c56740d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 -- 172.21.15.5:0/2406037558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c56730ce30 con 0x55c5673e2e80 2024-04-25T10:11:19.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.105+0000 7f47f9a00640 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 0x55c567429100 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f47ec008f40 tx=0x7f47ec00d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37dfab8f9ba7d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.106+0000 7f47e5600640 1 -- 172.21.15.5:0/2406037558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec003da0 con 0x55c5673e2e80 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.106+0000 7f47e5600640 1 -- 172.21.15.5:0/2406037558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec004030 con 0x55c5673e2e80 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.106+0000 7f47e5600640 1 -- 172.21.15.5:0/2406037558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec004420 con 0x55c5673e2e80 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 -- 172.21.15.5:0/2406037558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 msgr2=0x55c567429100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 0x55c567429100 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f47ec008f40 tx=0x7f47ec00d810 comp rx=0 tx=0).stop 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 -- 172.21.15.5:0/2406037558 shutdown_connections 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5673e2e80 0x55c567429100 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c56742af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 --2- 172.21.15.5:0/2406037558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c56740d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 -- 172.21.15.5:0/2406037558 >> 172.21.15.5:0/2406037558 conn(0x55c5673df7b0 msgr2=0x55c567427fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 -- 172.21.15.5:0/2406037558 shutdown_connections 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.107+0000 7f47fe858740 1 -- 172.21.15.5:0/2406037558 wait complete. 2024-04-25T10:11:19.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.108+0000 7f47fe858740 1 -- start start 2024-04-25T10:11:19.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.108+0000 7f47fe858740 1 Processor -- start 2024-04-25T10:11:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c567438ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c567421890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 0x55c5673fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5673e3c20 con 0x55c56740fc30 2024-04-25T10:11:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5672e43c0 con 0x55c56740d1a0 2024-04-25T10:11:19.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5673dd6e0 con 0x55c56742aa40 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47fe858740 10 client.0 _ll_register_callbacks cb 0x55c567059c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47f9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 0x55c5673fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47f9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 0x55c5673fe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39874/0 (socket says 172.21.15.5:39874) 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47f9a00640 1 -- 172.21.15.5:0/1438572946 learned_addr learned my addr 172.21.15.5:0/1438572946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.109+0000 7f47f9000640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c567438ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f3e00640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c567421890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9a00640 1 -- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 msgr2=0x55c567421890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9a00640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c567421890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9a00640 1 -- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 msgr2=0x55c567438ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9a00640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c567438ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9a00640 1 -- 172.21.15.5:0/1438572946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5671227e0 con 0x55c56740fc30 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9000640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c567438ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47f9a00640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 0x55c5673fe6b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00dd10 tx=0x7f47ec010070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d533b74f17c3d504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.110+0000 7f47da000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47da000640 1 -- 172.21.15.5:0/1438572946 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec028060 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47ec004030 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47da000640 1 -- 172.21.15.5:0/1438572946 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47ec0045f0 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47da000640 1 -- 172.21.15.5:0/1438572946 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47ec02e050 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 10 client.8601 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55c567182f70 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47da000640 1 -- 172.21.15.5:0/1438572946 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec004030 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47da000640 1 -- 172.21.15.5:0/1438572946 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47ec031020 con 0x55c56740fc30 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 10 client.8601 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 10 client.8601 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 20 client.8601 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c56741f060 0x55c56741f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 4 client.8601 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.111+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c567352ec0 con 0x55c56741f060 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.112+0000 7f47f9000640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c56741f060 0x55c56741f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.112+0000 7f47f9000640 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c56741f060 0x55c56741f490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2902bb0fc083ae58 server_cookie=9e05ce1ea629df99 in_seq=0 out_seq=0 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.112+0000 7f47da000640 10 client.8601 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.113+0000 7f47da000640 1 -- 172.21.15.5:0/1438572946 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c567352ec0 con 0x55c56741f060 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.113+0000 7f47da000640 10 client.8601 handle_command_reply: tid=0 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.113+0000 7f47fe858740 1 client.8601 shutdown 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.113+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 msgr2=0x55c5673fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.113+0000 7f47fe858740 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 0x55c5673fe6b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00dd10 tx=0x7f47ec010070 comp rx=0 tx=0).stop 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 shutdown_connections 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c56741f060 0x55c56741f490 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740fc30 0x55c5673fe6b0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c567421890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 --2- 172.21.15.5:0/1438572946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56740d1a0 0x55c567438ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47da000640 0 client.8601 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 >> 172.21.15.5:0/1438572946 conn(0x55c5673e2e80 msgr2=0x55c56742bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 shutdown_connections 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 1 -- 172.21.15.5:0/1438572946 wait complete. 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7f47fe858740 20 client.8601 trim_cache size 0 max 16384 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7fce6fc00640 1 Processor -- start 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.114+0000 7fce6fc00640 1 -- start start 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.115+0000 7fce6fc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fce6815da40 0x7fce680d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.115+0000 7fce6fc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce680d0780 0x7fce681616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.115+0000 7fce6fc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce68162080 con 0x7fce68161c10 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.115+0000 7fce6fc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce680d0c00 con 0x7fce6815da40 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.115+0000 7fce6fc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce681625e0 con 0x7fce680d0780 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6dc00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fce68161c10 0x7fce680b5410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:53642/0 (socket says 172.21.15.5:53642) 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6dc00640 1 -- 172.21.15.5:0/3373423371 learned_addr learned my addr 172.21.15.5:0/3373423371 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6d200640 1 --2- 172.21.15.5:0/3373423371 >> v2:172.21.15.5:3300/0 conn(0x7fce6815da40 0x7fce680d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce67e00640 1 --2- 172.21.15.5:0/3373423371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce680d0780 0x7fce681616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6d200640 1 -- 172.21.15.5:0/3373423371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce680d0780 msgr2=0x7fce681616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6d200640 1 --2- 172.21.15.5:0/3373423371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce680d0780 0x7fce681616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6d200640 1 -- 172.21.15.5:0/3373423371 >> v1:172.21.15.5:6789/0 conn(0x7fce68161c10 legacy=0x7fce680b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce6d200640 1 -- 172.21.15.5:0/3373423371 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68162300 con 0x7fce6815da40 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.116+0000 7fce67400640 1 -- 172.21.15.5:0/3373423371 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113766242 0 0) 0x7fce60001490 con 0x7fce68161c10 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.118+0000 7fce6d200640 1 --2- 172.21.15.5:0/3373423371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce6815da40 0x7fce680d0240 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0097a0 tx=0x7fce5c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2992078dbe4c6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.118+0000 7fce67400640 1 -- 172.21.15.5:0/3373423371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c037070 con 0x7fce6815da40 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.118+0000 7fce67400640 1 -- 172.21.15.5:0/3373423371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c032440 con 0x7fce6815da40 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.118+0000 7fce67400640 1 -- 172.21.15.5:0/3373423371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c032810 con 0x7fce6815da40 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.118+0000 7fce6fc00640 1 -- 172.21.15.5:0/3373423371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce6815da40 msgr2=0x7fce680d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.118+0000 7fce6fc00640 1 --2- 172.21.15.5:0/3373423371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce6815da40 0x7fce680d0240 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0097a0 tx=0x7fce5c02a180 comp rx=0 tx=0).stop 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.119+0000 7fce6fc00640 1 -- 172.21.15.5:0/3373423371 shutdown_connections 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.119+0000 7fce6fc00640 1 --2- 172.21.15.5:0/3373423371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce680d0780 0x7fce681616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.119+0000 7fce6fc00640 1 --2- 172.21.15.5:0/3373423371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce6815da40 0x7fce680d0240 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.119+0000 7fce6fc00640 1 -- 172.21.15.5:0/3373423371 >> 172.21.15.5:0/3373423371 conn(0x7fce68072290 msgr2=0x7fce680726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.119+0000 7fce6fc00640 1 -- 172.21.15.5:0/3373423371 shutdown_connections 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.119+0000 7fce6fc00640 1 -- 172.21.15.5:0/3373423371 wait complete. 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.120+0000 7fce6fc00640 1 Processor -- start 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.121+0000 7fce6fc00640 1 -- start start 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.121+0000 7fce6fc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce680d0780 0x7fce68151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.121+0000 7fce6fc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce6815da40 0x7fce681546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.121+0000 7fce6fc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce68161c10 0x7fce68154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.121+0000 7fce6fc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce680d1ec0 con 0x7fce680d0780 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.121+0000 7fce6fc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce680d1be0 con 0x7fce6815da40 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6fc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce680d21a0 con 0x7fce68161c10 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce680d0780 0x7fce68151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce680d0780 0x7fce68151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39892/0 (socket says 172.21.15.5:39892) 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 -- 172.21.15.5:0/2265648841 learned_addr learned my addr 172.21.15.5:0/2265648841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 -- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce68161c10 msgr2=0x7fce68154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 --2- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fce68161c10 0x7fce68154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 -- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce6815da40 msgr2=0x7fce681546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 --2- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce6815da40 0x7fce681546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.122+0000 7fce6d200640 1 -- 172.21.15.5:0/2265648841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce680cb950 con 0x7fce680d0780 2024-04-25T10:11:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.123+0000 7fce6d200640 1 --2- 172.21.15.5:0/2265648841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce680d0780 0x7fce68151fa0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0098c0 tx=0x7fce5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.123+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c037040 con 0x7fce680d0780 2024-04-25T10:11:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.123+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c030770 con 0x7fce680d0780 2024-04-25T10:11:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.123+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c030b40 con 0x7fce680d0780 2024-04-25T10:11:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.123+0000 7fce6fc00640 1 -- 172.21.15.5:0/2265648841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce680cd620 con 0x7fce680d0780 2024-04-25T10:11:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.123+0000 7fce6fc00640 1 -- 172.21.15.5:0/2265648841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce680cbeb0 con 0x7fce680d0780 2024-04-25T10:11:19.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.124+0000 7fce6fc00640 1 -- 172.21.15.5:0/2265648841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fce68068f60 con 0x7fce680d0780 2024-04-25T10:11:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.126+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce5c044020 con 0x7fce680d0780 2024-04-25T10:11:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.127+0000 7fce64c00640 1 --2- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce4806b4d0 0x7fce4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.127+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce5c0ab280 con 0x7fce680d0780 2024-04-25T10:11:19.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.128+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7fce680cb950 con 0x7fce680d0780 2024-04-25T10:11:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.128+0000 7fce6fc00640 1 -- 172.21.15.5:0/2265648841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fce680d1660 con 0x7fce680d0780 2024-04-25T10:11:19.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.128+0000 7fce67e00640 1 --2- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce4806b4d0 0x7fce4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.129+0000 7fce67e00640 1 --2- 172.21.15.5:0/2265648841 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fce4806b4d0 0x7fce4806d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fce6815c440 tx=0x7fce54008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.134+0000 7f47fe858740 1 Processor -- start 2024-04-25T10:11:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.134+0000 7f47fe858740 1 -- start start 2024-04-25T10:11:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.134+0000 7f47fe858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c5673d87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.134+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c5674416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c5674110d0 0x55c56744add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47fe858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f42002d0 con 0x55c56740d1a0 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41ff4c0 con 0x55c5674110d0 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41600f0 con 0x55c56742aa40 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c5673d87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c5673d87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39898/0 (socket says 172.21.15.5:39898) 2024-04-25T10:11:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 -- 172.21.15.5:0/1685589182 learned_addr learned my addr 172.21.15.5:0/1685589182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 -- 172.21.15.5:0/1685589182 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 msgr2=0x55c5674416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c5674416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 -- 172.21.15.5:0/1685589182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c5674110d0 msgr2=0x55c56744add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c5674110d0 0x55c56744add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 -- 172.21.15.5:0/1685589182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c56701a170 con 0x55c56740d1a0 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.135+0000 7f47f9000640 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c5673d87d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f47e800c520 tx=0x7f47e8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cbfd067972d702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.136+0000 7f47e4c00640 1 -- 172.21.15.5:0/1685589182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e80044c0 con 0x55c56740d1a0 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.136+0000 7f47e4c00640 1 -- 172.21.15.5:0/1685589182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800ce10 con 0x55c56740d1a0 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.136+0000 7f47e4c00640 1 -- 172.21.15.5:0/1685589182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8003810 con 0x55c56740d1a0 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.136+0000 7f47fe858740 1 -- 172.21.15.5:0/1685589182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 msgr2=0x55c5673d87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.136+0000 7f47fe858740 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c5673d87d0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f47e800c520 tx=0x7f47e8015000 comp rx=0 tx=0).stop 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 -- 172.21.15.5:0/1685589182 shutdown_connections 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c5674110d0 0x55c56744add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c56742aa40 0x55c5674416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 --2- 172.21.15.5:0/1685589182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c5673d87d0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 -- 172.21.15.5:0/1685589182 >> 172.21.15.5:0/1685589182 conn(0x55c5673e2e80 msgr2=0x55c5673fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 -- 172.21.15.5:0/1685589182 shutdown_connections 2024-04-25T10:11:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 -- 172.21.15.5:0/1685589182 wait complete. 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.137+0000 7f47fe858740 1 -- start start 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.138+0000 7f47fe858740 1 Processor -- start 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.138+0000 7f47fe858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c567492360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.138+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c5674110d0 0x55c5674299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.138+0000 7f47fe858740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56742aa40 0x55c567429f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47fe858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41606f0 con 0x55c56740d1a0 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c567449370 con 0x55c56742aa40 2024-04-25T10:11:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47fe858740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5673dd160 con 0x55c5674110d0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47fe858740 10 client.0 _ll_register_callbacks cb 0x55c5673ddbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c567492360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c567492360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39904/0 (socket says 172.21.15.5:39904) 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 -- 172.21.15.5:0/1426649089 learned_addr learned my addr 172.21.15.5:0/1426649089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 -- 172.21.15.5:0/1426649089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c5674110d0 msgr2=0x55c5674299d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c5674110d0 0x55c5674299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 -- 172.21.15.5:0/1426649089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56742aa40 msgr2=0x55c567429f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56742aa40 0x55c567429f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.139+0000 7f47f9000640 1 -- 172.21.15.5:0/1426649089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5673dc550 con 0x55c56740d1a0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47f9000640 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c567492360 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f47e8004410 tx=0x7f47e8004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340b30e74543521a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47da000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47da000640 1 -- 172.21.15.5:0/1426649089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8003670 con 0x55c56740d1a0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47e800ce10 con 0x55c56740d1a0 2024-04-25T10:11:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47da000640 1 -- 172.21.15.5:0/1426649089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f47e8003810 con 0x55c56740d1a0 2024-04-25T10:11:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47da000640 1 -- 172.21.15.5:0/1426649089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e803e050 con 0x55c56740d1a0 2024-04-25T10:11:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47fe858740 10 client.8616 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.140+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55c5673e4780 con 0x55c56740d1a0 2024-04-25T10:11:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47da000640 1 -- 172.21.15.5:0/1426649089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e800ce10 con 0x55c56740d1a0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47da000640 1 -- 172.21.15.5:0/1426649089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47e8041020 con 0x55c56740d1a0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47fe858740 10 client.8616 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47fe858740 10 client.8616 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47fe858740 20 client.8616 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47fe858740 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c5674374b0 0x55c5674378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47fe858740 4 client.8616 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55c56734d210 con 0x55c5674374b0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.141+0000 7f47f3e00640 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c5674374b0 0x55c5674378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47f3e00640 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c5674374b0 0x55c5674378e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8880bb54d514bb7c server_cookie=9b7b8584a70a127a in_seq=0 out_seq=0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47da000640 10 client.8616 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47da000640 1 -- 172.21.15.5:0/1426649089 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c56734d210 con 0x55c5674374b0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47da000640 10 client.8616 handle_command_reply: tid=0 2024-04-25T10:11:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47fe858740 1 client.8616 shutdown 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 msgr2=0x55c567492360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.142+0000 7f47fe858740 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c567492360 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f47e8004410 tx=0x7f47e8004680 comp rx=0 tx=0).stop 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 shutdown_connections 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55c5674374b0 0x55c5674378e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55c56742aa40 0x55c567429f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55c5674110d0 0x55c5674299d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 --2- 172.21.15.5:0/1426649089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c56740d1a0 0x55c567492360 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 >> 172.21.15.5:0/1426649089 conn(0x55c5673e2e80 msgr2=0x55c567441f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47da000640 0 client.8616 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 shutdown_connections 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 1 -- 172.21.15.5:0/1426649089 wait complete. 2024-04-25T10:11:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.143+0000 7f47fe858740 20 client.8616 trim_cache size 0 max 16384 2024-04-25T10:11:19.151 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:19.151 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:11:19.151 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "56ce0772-fef6-4e56-b48f-fcb3d91ff8c9", 2024-04-25T10:11:19.151 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:11:19.151 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47d406b4d0 msgr2=0x7f47d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47d406b4d0 0x7f47d406d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f47e800a7a0 tx=0x7f47e8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 msgr2=0x7f47f41f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f41f30d0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f47e002a270 tx=0x7f47e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 shutdown_connections 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f47f415de70 0x7f47f415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47f40d3330 0x7f47f41f30d0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f47d406b4d0 0x7f47d406d990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 --2- 172.21.15.5:0/824263994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f40b9290 0x7f47f41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 >> 172.21.15.5:0/824263994 conn(0x7f47f40726b0 msgr2=0x7f47f4154710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 shutdown_connections 2024-04-25T10:11:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.144+0000 7f47fba00640 1 -- 172.21.15.5:0/824263994 wait complete. 2024-04-25T10:11:19.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:19.166 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:11:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.339+0000 7f6c23e00640 1 Processor -- start 2024-04-25T10:11:19.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.339+0000 7f6c23e00640 1 -- start start 2024-04-25T10:11:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.341+0000 7f6c23e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c2415ed20 0x7f6c24144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.341+0000 7f6c23e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2415f3e0 con 0x7f6c24144e90 2024-04-25T10:11:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.341+0000 7f6c23e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2415f680 con 0x7f6c24071710 2024-04-25T10:11:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.341+0000 7f6c23e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c24144780 con 0x7f6c2415ed20 2024-04-25T10:11:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c2415ed20 0x7f6c24144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c2415ed20 0x7f6c24144950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39916/0 (socket says 172.21.15.5:39916) 2024-04-25T10:11:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22a00640 1 -- 172.21.15.5:0/1785306890 learned_addr learned my addr 172.21.15.5:0/1785306890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22a00640 1 -- 172.21.15.5:0/1785306890 >> v1:172.21.15.80:6789/0 conn(0x7f6c24071710 legacy=0x7f6c2415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22a00640 1 -- 172.21.15.5:0/1785306890 >> v1:172.21.15.5:6789/0 conn(0x7f6c24144e90 legacy=0x7f6c24071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22000640 1 -- 172.21.15.5:0/1785306890 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134901587 0 0) 0x7f6c2415f3e0 con 0x7f6c24144e90 2024-04-25T10:11:19.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.342+0000 7f6c22a00640 1 -- 172.21.15.5:0/1785306890 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2415f960 con 0x7f6c2415ed20 2024-04-25T10:11:19.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.344+0000 7f6c22a00640 1 --2- 172.21.15.5:0/1785306890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2415ed20 0x7f6c24144950 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009870 tx=0x7f6c1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21321c9a3d4942f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.344+0000 7f6c22000640 1 -- 172.21.15.5:0/1785306890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14037070 con 0x7f6c2415ed20 2024-04-25T10:11:19.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.344+0000 7f6c22000640 1 -- 172.21.15.5:0/1785306890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c14032440 con 0x7f6c2415ed20 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.344+0000 7f6c22000640 1 -- 172.21.15.5:0/1785306890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14032810 con 0x7f6c2415ed20 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.344+0000 7f6c23e00640 1 -- 172.21.15.5:0/1785306890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2415ed20 msgr2=0x7f6c24144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.344+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1785306890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2415ed20 0x7f6c24144950 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009870 tx=0x7f6c1402a180 comp rx=0 tx=0).stop 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.345+0000 7f6c23e00640 1 -- 172.21.15.5:0/1785306890 shutdown_connections 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.345+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1785306890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c2415ed20 0x7f6c24144950 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.345+0000 7f6c23e00640 1 -- 172.21.15.5:0/1785306890 >> 172.21.15.5:0/1785306890 conn(0x7f6c24066f20 msgr2=0x7f6c2406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.345+0000 7f6c23e00640 1 -- 172.21.15.5:0/1785306890 shutdown_connections 2024-04-25T10:11:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.345+0000 7f6c23e00640 1 -- 172.21.15.5:0/1785306890 wait complete. 2024-04-25T10:11:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.346+0000 7f6c23e00640 1 Processor -- start 2024-04-25T10:11:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.347+0000 7f6c23e00640 1 -- start start 2024-04-25T10:11:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.347+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c24071710 0x7f6c240b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 0x7f6c24152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c23e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c2415ed20 0x7f6c240ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c23e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2415c4d0 con 0x7f6c24144e90 2024-04-25T10:11:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c23e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cdd50 con 0x7f6c2415ed20 2024-04-25T10:11:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c23e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2415c650 con 0x7f6c24071710 2024-04-25T10:11:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c1be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 0x7f6c24152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c1be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 0x7f6c24152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39924/0 (socket says 172.21.15.5:39924) 2024-04-25T10:11:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c1be00640 1 -- 172.21.15.5:0/1560582639 learned_addr learned my addr 172.21.15.5:0/1560582639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c22a00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c24071710 0x7f6c240b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.348+0000 7f6c23400640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c2415ed20 0x7f6c240ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1be00640 1 -- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c24071710 msgr2=0x7f6c240b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1be00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c24071710 0x7f6c240b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1be00640 1 -- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c2415ed20 msgr2=0x7f6c240ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1be00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c2415ed20 0x7f6c240ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1be00640 1 -- 172.21.15.5:0/1560582639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2415b450 con 0x7f6c24144e90 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1be00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 0x7f6c24152dc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c4b0 tx=0x7f6c0c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1b400640 1 -- 172.21.15.5:0/1560582639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c017070 con 0x7f6c24144e90 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1b400640 1 -- 172.21.15.5:0/1560582639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c005480 con 0x7f6c24144e90 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c240b2bb0 con 0x7f6c24144e90 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c1b400640 1 -- 172.21.15.5:0/1560582639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c003830 con 0x7f6c24144e90 2024-04-25T10:11:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.349+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c240b3170 con 0x7f6c24144e90 2024-04-25T10:11:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.352+0000 7f6c1b400640 1 -- 172.21.15.5:0/1560582639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c0c01a070 con 0x7f6c24144e90 2024-04-25T10:11:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.352+0000 7f6c1b400640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c0006b4d0 0x7f6c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.353+0000 7f6c1b400640 1 -- 172.21.15.5:0/1560582639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c0c0897a0 con 0x7f6c24144e90 2024-04-25T10:11:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.353+0000 7f6c2cbf5740 1 Processor -- start 2024-04-25T10:11:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.353+0000 7f6c22a00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c0006b4d0 0x7f6c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.353+0000 7f6c2cbf5740 1 -- start start 2024-04-25T10:11:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abab7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abaf2440 0x5652abab6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 0x5652abae0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aba19c70 con 0x5652abae0830 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ab9c2cf0 con 0x5652abab5d70 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652aba15d10 con 0x5652abaf2440 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c22a00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c0006b4d0 0x7f6c0006d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c14034000 tx=0x7f6c1402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.354+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 0x5652abae0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 0x5652abae0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39938/0 (socket says 172.21.15.5:39938) 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 -- 172.21.15.5:0/3013973558 learned_addr learned my addr 172.21.15.5:0/3013973558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c1be00640 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abaf2440 0x5652abab6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 -- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abaf2440 msgr2=0x5652abab6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abaf2440 0x5652abab6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c22a00640 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abab7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 -- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 msgr2=0x5652abab7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abab7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 -- 172.21.15.5:0/3013973558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652ab7f87d0 con 0x5652abae0830 2024-04-25T10:11:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c1be00640 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abaf2440 0x5652abab6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c23400640 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 0x5652abae0cb0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c005430 tx=0x7f6c1c0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39daee4541372f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c13400640 1 -- 172.21.15.5:0/3013973558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c006d60 con 0x5652abae0830 2024-04-25T10:11:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c13400640 1 -- 172.21.15.5:0/3013973558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c018070 con 0x5652abae0830 2024-04-25T10:11:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.355+0000 7f6c13400640 1 -- 172.21.15.5:0/3013973558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00eaa0 con 0x5652abae0830 2024-04-25T10:11:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/3013973558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 msgr2=0x5652abae0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 0x5652abae0cb0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c005430 tx=0x7f6c1c0058f0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/3013973558 shutdown_connections 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abae0830 0x5652abae0cb0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abaf2440 0x5652abab6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/3013973558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abab7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/3013973558 >> 172.21.15.5:0/3013973558 conn(0x5652aba7da30 msgr2=0x5652abafb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/3013973558 shutdown_connections 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.356+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/3013973558 wait complete. 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.357+0000 7f6c2cbf5740 1 -- start start 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.357+0000 7f6c2cbf5740 1 Processor -- start 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abacfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abae0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaf4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652ab9b83a0 con 0x5652abaf2440 2024-04-25T10:11:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652abade380 con 0x5652abab5d70 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652abb0b440 con 0x5652abae0830 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.358+0000 7f6c2cbf5740 10 client.0 _ll_register_callbacks cb 0x5652ab9d5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaf4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaf4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39944/0 (socket says 172.21.15.5:39944) 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 -- 172.21.15.5:0/1861542948 learned_addr learned my addr 172.21.15.5:0/1861542948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c1be00640 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abae0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 -- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 msgr2=0x5652abae0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abae0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 -- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 msgr2=0x5652abacfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abacfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 -- 172.21.15.5:0/1861542948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652ab72e3f0 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c23400640 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaf4dd0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c005930 tx=0x7f6c1c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a442d563aea242a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.359+0000 7f6c07e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c07e00640 1 -- 172.21.15.5:0/1861542948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00ed90 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c07e00640 1 -- 172.21.15.5:0/1861542948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c1c027c70 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c1c018070 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c07e00640 1 -- 172.21.15.5:0/1861542948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c1c006e30 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 10 client.8631 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5652ab6ee0f0 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c07e00640 1 -- 172.21.15.5:0/1861542948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c018070 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c07e00640 1 -- 172.21.15.5:0/1861542948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c1c026430 con 0x5652abaf2440 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 10 client.8631 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 10 client.8631 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 20 client.8631 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.360+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb15ba0 0x5652abb17f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.361+0000 7f6c2cbf5740 4 client.8631 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.361+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652aba271b0 con 0x5652abb15ba0 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.361+0000 7f6c22a00640 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb15ba0 0x5652abb17f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.361+0000 7f6c22a00640 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb15ba0 0x5652abb17f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714b24c033889bf4 server_cookie=637b276746c4c70f in_seq=0 out_seq=0 2024-04-25T10:11:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.361+0000 7f6c07e00640 10 client.8631 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c07e00640 1 -- 172.21.15.5:0/1861542948 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652aba271b0 con 0x5652abb15ba0 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c07e00640 10 client.8631 handle_command_reply: tid=0 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 client.8631 shutdown 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 msgr2=0x5652abaf4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaf4dd0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c005930 tx=0x7f6c1c003a70 comp rx=0 tx=0).stop 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 shutdown_connections 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb15ba0 0x5652abb17f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaf4dd0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abae0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1861542948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abacfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c07e00640 0 client.8631 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.362+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 >> 172.21.15.5:0/1861542948 conn(0x5652aba7da30 msgr2=0x5652abafedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.363+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 shutdown_connections 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.363+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1861542948 wait complete. 2024-04-25T10:11:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.363+0000 7f6c2cbf5740 20 client.8631 trim_cache size 0 max 16384 2024-04-25T10:11:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.379+0000 7f6c2cbf5740 1 Processor -- start 2024-04-25T10:11:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.379+0000 7f6c2cbf5740 1 -- start start 2024-04-25T10:11:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abadad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abafecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241fffa0 con 0x5652abaf2440 2024-04-25T10:11:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241ff160 con 0x5652abab5d70 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241f3ed0 con 0x5652abae0830 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39952/0 (socket says 172.21.15.5:39952) 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.380+0000 7f6c23400640 1 -- 172.21.15.5:0/1437422811 learned_addr learned my addr 172.21.15.5:0/1437422811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c1be00640 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abafecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c23400640 1 -- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 msgr2=0x5652abafecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c23400640 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abafecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c23400640 1 -- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 msgr2=0x5652abadad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c23400640 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abadad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c23400640 1 -- 172.21.15.5:0/1437422811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652abb0b210 con 0x5652abaf2440 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c23400640 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaff1f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c007f50 tx=0x7f6c1c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babcad9b87638e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c12a00640 1 -- 172.21.15.5:0/1437422811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00ea70 con 0x5652abaf2440 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c12a00640 1 -- 172.21.15.5:0/1437422811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c018040 con 0x5652abaf2440 2024-04-25T10:11:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.381+0000 7f6c12a00640 1 -- 172.21.15.5:0/1437422811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c013420 con 0x5652abaf2440 2024-04-25T10:11:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1437422811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 msgr2=0x5652abaff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaff1f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c007f50 tx=0x7f6c1c01f070 comp rx=0 tx=0).stop 2024-04-25T10:11:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1437422811 shutdown_connections 2024-04-25T10:11:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abaff1f0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abafecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/1437422811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abadad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1437422811 >> 172.21.15.5:0/1437422811 conn(0x5652aba7da30 msgr2=0x5652abafbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1437422811 shutdown_connections 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.382+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/1437422811 wait complete. 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.383+0000 7f6c2cbf5740 1 -- start start 2024-04-25T10:11:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.383+0000 7f6c2cbf5740 1 Processor -- start 2024-04-25T10:11:19.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abafd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abb0a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abb68990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241f44d0 con 0x5652abaf2440 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652abab2a40 con 0x5652abab5d70 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652abb65810 con 0x5652abae0830 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c2cbf5740 10 client.0 _ll_register_callbacks cb 0x7f6c1c005280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abb68990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abb68990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39954/0 (socket says 172.21.15.5:39954) 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c23400640 1 -- 172.21.15.5:0/4132803964 learned_addr learned my addr 172.21.15.5:0/4132803964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.384+0000 7f6c1be00640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abb0a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c22a00640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abafd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c23400640 1 -- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 msgr2=0x5652abb0a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c23400640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abb0a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c23400640 1 -- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 msgr2=0x5652abafd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c23400640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abafd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c23400640 1 -- 172.21.15.5:0/4132803964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652abb1ca60 con 0x5652abaf2440 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c22a00640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abafd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c23400640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abb68990 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c006e00 tx=0x7f6c1c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929790446ea52336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c07e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c07e00640 1 -- 172.21.15.5:0/4132803964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c03a470 con 0x5652abaf2440 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c07e00640 1 -- 172.21.15.5:0/4132803964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c1c03a610 con 0x5652abaf2440 2024-04-25T10:11:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.385+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c1c018040 con 0x5652abaf2440 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c07e00640 1 -- 172.21.15.5:0/4132803964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c1c03a810 con 0x5652abaf2440 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 10 client.8637 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5652abb65550 con 0x5652abaf2440 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c07e00640 1 -- 172.21.15.5:0/4132803964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c018040 con 0x5652abaf2440 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c07e00640 1 -- 172.21.15.5:0/4132803964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c1c03a9b0 con 0x5652abaf2440 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 10 client.8637 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 10 client.8637 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 20 client.8637 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb0abb0 0x5652abb68ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 4 client.8637 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652aba21500 con 0x5652abb0abb0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.386+0000 7f6c22a00640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb0abb0 0x5652abb68ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c22a00640 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb0abb0 0x5652abb68ed0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920944d7f4eacfc7 server_cookie=9974e860aeb737fe in_seq=0 out_seq=0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c07e00640 10 client.8637 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c07e00640 1 -- 172.21.15.5:0/4132803964 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5652aba21500 con 0x5652abb0abb0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c07e00640 10 client.8637 handle_command_reply: tid=0 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c2cbf5740 1 client.8637 shutdown 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 msgr2=0x5652abb68990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.387+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abb68990 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c006e00 tx=0x7f6c1c009040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 shutdown_connections 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x5652abb0abb0 0x5652abb68ed0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652abaf2440 0x5652abb68990 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5652abae0830 0x5652abb0a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 --2- 172.21.15.5:0/4132803964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5652abab5d70 0x5652abafd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 >> 172.21.15.5:0/4132803964 conn(0x5652aba7da30 msgr2=0x5652abafecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c07e00640 0 client.8637 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 shutdown_connections 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 1 -- 172.21.15.5:0/4132803964 wait complete. 2024-04-25T10:11:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c2cbf5740 20 client.8637 trim_cache size 0 max 16384 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: "56ce0772-fef6-4e56-b48f-fcb3d91ff8c9": { 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "56ce0772-fef6-4e56-b48f-fcb3d91ff8c9", 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-04-25T10:11:19.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:11:19.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:11:19.404 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c0006b4d0 msgr2=0x7f6c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c0006b4d0 0x7f6c0006d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c14034000 tx=0x7f6c1402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 msgr2=0x7f6c24152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.388+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 0x7f6c24152dc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c4b0 tx=0x7f6c0c00c970 comp rx=0 tx=0).stop 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 shutdown_connections 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c0006b4d0 0x7f6c0006d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c2415ed20 0x7f6c240ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c24144e90 0x7f6c24152dc0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 --2- 172.21.15.5:0/1560582639 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c24071710 0x7f6c240b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 >> 172.21.15.5:0/1560582639 conn(0x7f6c24066f20 msgr2=0x7f6c2415c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 shutdown_connections 2024-04-25T10:11:19.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:19.389+0000 7f6c23e00640 1 -- 172.21.15.5:0/1560582639 wait complete. 2024-04-25T10:11:19.410 INFO:tasks.cephfs.filesystem:scrub status for tag:56ce0772-fef6-4e56-b48f-fcb3d91ff8c9 - {'path': '/', 'tag': '56ce0772-fef6-4e56-b48f-fcb3d91ff8c9', 'options': 'recursive,repair'} 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:20.255+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-25T10:11:19.143071+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7fce5c077a20 con 0x7fce680d0780 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: cluster: 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: id: 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: health: HEALTH_OK 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: services: 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: mgr: x(active, since 9m), standbys: y 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: data: 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: volumes: 1/1 healthy 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: usage: 285 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: pgs: 97 active+clean 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: io: 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: client: 85 B/s rd, 1.5 MiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:19.143071+0000 mds.c [INF] scrub queued for path: / 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:19.143075+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:19.143080+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:19.145042+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:11:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:24.269+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-25T10:11:23.704224+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7fce5c045e00 con 0x7fce680d0780 2024-04-25T10:11:24.274 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:23.704224+0000 mds.c [INF] scrub summary: idle 2024-04-25T10:11:25.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:25.815 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:42.340+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 9 ==== log(8 entries from seq 165 at 2024-04-25T10:11:41.642310+0000) v1 ==== 2208+0+0 (secure 0 0 0) 0x7fce5c077c80 con 0x7fce680d0780 2024-04-25T10:11:43.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:43.342+0000 7fce64c00640 1 -- 172.21.15.5:0/2265648841 <== mon.0 v2:172.21.15.5:3300/0 10 ==== log(4 entries from seq 1064 at 2024-04-25T10:11:42.323909+0000) v1 ==== 1081+0+0 (secure 0 0 0) 0x7fce5c0a9690 con 0x7fce680d0780 2024-04-25T10:11:49.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:11:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.640+0000 7fde6a000640 1 Processor -- start 2024-04-25T10:11:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.640+0000 7fde6a000640 1 -- start start 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde6a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640d3f90 0x7fde64161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde6a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde64161bc0 0x7fde640b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde6a000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde641606e0 con 0x7fde640b48c0 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde6a000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde641609c0 con 0x7fde64161bc0 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde6a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde64160ca0 con 0x7fde640d3f90 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde68c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fde640b48c0 0x7fde6415fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55520/0 (socket says 172.21.15.5:55520) 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde68c00640 1 -- 172.21.15.5:0/3245254354 learned_addr learned my addr 172.21.15.5:0/3245254354 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.642+0000 7fde69600640 1 --2- 172.21.15.5:0/3245254354 >> v2:172.21.15.5:3300/0 conn(0x7fde64161bc0 0x7fde640b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde63e00640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640d3f90 0x7fde64161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde63400640 1 -- 172.21.15.5:0/3245254354 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795281750 0 0) 0x7fde641606e0 con 0x7fde640b48c0 2024-04-25T10:11:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde63400640 1 -- 172.21.15.5:0/3245254354 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde4c003520 con 0x7fde640b48c0 2024-04-25T10:11:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde69600640 1 -- 172.21.15.5:0/3245254354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640d3f90 msgr2=0x7fde64161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde69600640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640d3f90 0x7fde64161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde69600640 1 -- 172.21.15.5:0/3245254354 >> v1:172.21.15.5:6789/0 conn(0x7fde640b48c0 legacy=0x7fde6415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde69600640 1 -- 172.21.15.5:0/3245254354 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde64160f80 con 0x7fde64161bc0 2024-04-25T10:11:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.643+0000 7fde63e00640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640d3f90 0x7fde64161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.644+0000 7fde69600640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640b2b70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0097a0 tx=0x7fde5c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b12b5ede681be38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.645+0000 7fde63400640 1 -- 172.21.15.5:0/3245254354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c037070 con 0x7fde64161bc0 2024-04-25T10:11:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.645+0000 7fde63400640 1 -- 172.21.15.5:0/3245254354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c02ae50 con 0x7fde64161bc0 2024-04-25T10:11:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.645+0000 7fde63400640 1 -- 172.21.15.5:0/3245254354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c032640 con 0x7fde64161bc0 2024-04-25T10:11:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.645+0000 7fde6a000640 1 -- 172.21.15.5:0/3245254354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 msgr2=0x7fde640b2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.645+0000 7fde6a000640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640b2b70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0097a0 tx=0x7fde5c02a100 comp rx=0 tx=0).stop 2024-04-25T10:11:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.646+0000 7fde6a000640 1 -- 172.21.15.5:0/3245254354 shutdown_connections 2024-04-25T10:11:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.646+0000 7fde6a000640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640b2b70 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.646+0000 7fde6a000640 1 --2- 172.21.15.5:0/3245254354 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640d3f90 0x7fde64161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.646+0000 7fde6a000640 1 -- 172.21.15.5:0/3245254354 >> 172.21.15.5:0/3245254354 conn(0x7fde6408af80 msgr2=0x7fde640d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.646+0000 7fde6a000640 1 -- 172.21.15.5:0/3245254354 shutdown_connections 2024-04-25T10:11:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.646+0000 7fde6a000640 1 -- 172.21.15.5:0/3245254354 wait complete. 2024-04-25T10:11:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.648+0000 7fde6a000640 1 Processor -- start 2024-04-25T10:11:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.648+0000 7fde6a000640 1 -- start start 2024-04-25T10:11:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde6a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640b48c0 0x7fde64155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde6a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde640d3f90 0x7fde640cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde6a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde6a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b30b0 con 0x7fde64161bc0 2024-04-25T10:11:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde6a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b4ca0 con 0x7fde640d3f90 2024-04-25T10:11:49.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde6a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b3230 con 0x7fde640b48c0 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40324/0 (socket says 172.21.15.5:40324) 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 -- 172.21.15.5:0/3156300948 learned_addr learned my addr 172.21.15.5:0/3156300948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 -- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640b48c0 msgr2=0x7fde64155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde68c00640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640b48c0 0x7fde64155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde63e00640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde640d3f90 0x7fde640cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640b48c0 0x7fde64155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 -- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde640d3f90 msgr2=0x7fde640cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde640d3f90 0x7fde640cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.649+0000 7fde69600640 1 -- 172.21.15.5:0/3156300948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640c70f0 con 0x7fde64161bc0 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde63e00640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde640d3f90 0x7fde640cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:49.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde69600640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640cc790 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fde5c035040 tx=0x7fde5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde60c00640 1 -- 172.21.15.5:0/3156300948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c041070 con 0x7fde64161bc0 2024-04-25T10:11:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde60c00640 1 -- 172.21.15.5:0/3156300948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c03c670 con 0x7fde64161bc0 2024-04-25T10:11:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde60c00640 1 -- 172.21.15.5:0/3156300948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c037040 con 0x7fde64161bc0 2024-04-25T10:11:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde640c73d0 con 0x7fde64161bc0 2024-04-25T10:11:49.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.650+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde640c7990 con 0x7fde64161bc0 2024-04-25T10:11:49.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.653+0000 7fde60c00640 1 -- 172.21.15.5:0/3156300948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde5c03a070 con 0x7fde64161bc0 2024-04-25T10:11:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.654+0000 7fde60c00640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde4406b4d0 0x7fde4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.654+0000 7fde60c00640 1 -- 172.21.15.5:0/3156300948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fde5c0aa6f0 con 0x7fde64161bc0 2024-04-25T10:11:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.654+0000 7fde6e3b6740 1 Processor -- start 2024-04-25T10:11:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.654+0000 7fde68c00640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde4406b4d0 0x7fde4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 -- start start 2024-04-25T10:11:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 0x55b3ae792ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae791150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7bb830 0x55b3ae7bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae6f4c70 con 0x55b3ae790d70 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae69dcf0 con 0x55b3ae7bb830 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.655+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae6f0d10 con 0x55b3ae7cd440 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 0x55b3ae792ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 0x55b3ae792ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40326/0 (socket says 172.21.15.5:40326) 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 -- 172.21.15.5:0/2258760955 learned_addr learned my addr 172.21.15.5:0/2258760955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde4406b4d0 0x7fde4406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fde580012e0 tx=0x7fde58009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 -- 172.21.15.5:0/2258760955 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 msgr2=0x55b3ae791150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae791150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 -- 172.21.15.5:0/2258760955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7bb830 msgr2=0x55b3ae7bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7bb830 0x55b3ae7bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 -- 172.21.15.5:0/2258760955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ae4d37d0 con 0x55b3ae790d70 2024-04-25T10:11:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.656+0000 7fde68c00640 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 0x55b3ae792ed0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fde580134d0 tx=0x7fde58010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc89327cf48201d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.657+0000 7fde55200640 1 -- 172.21.15.5:0/2258760955 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58024070 con 0x55b3ae790d70 2024-04-25T10:11:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.657+0000 7fde55200640 1 -- 172.21.15.5:0/2258760955 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58017030 con 0x55b3ae790d70 2024-04-25T10:11:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.657+0000 7fde55200640 1 -- 172.21.15.5:0/2258760955 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58019050 con 0x55b3ae790d70 2024-04-25T10:11:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.657+0000 7fde6e3b6740 1 -- 172.21.15.5:0/2258760955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 msgr2=0x55b3ae792ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.657+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 0x55b3ae792ed0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fde580134d0 tx=0x7fde58010000 comp rx=0 tx=0).stop 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 -- 172.21.15.5:0/2258760955 shutdown_connections 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7bb830 0x55b3ae7bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae791150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/2258760955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae790d70 0x55b3ae792ed0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 -- 172.21.15.5:0/2258760955 >> 172.21.15.5:0/2258760955 conn(0x55b3ae758a30 msgr2=0x55b3ae7d6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 -- 172.21.15.5:0/2258760955 shutdown_connections 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.658+0000 7fde6e3b6740 1 -- 172.21.15.5:0/2258760955 wait complete. 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.659+0000 7fde6e3b6740 1 -- start start 2024-04-25T10:11:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.659+0000 7fde6e3b6740 1 Processor -- start 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae7da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 0x55b3ae7d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7ce980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae6933a0 con 0x55b3ae7bb830 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae7b9380 con 0x55b3ae790d70 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae7d9c30 con 0x55b3ae7cd440 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde6e3b6740 10 client.0 _ll_register_callbacks cb 0x55b3ae6b0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde63e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 0x55b3ae7d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde63e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 0x55b3ae7d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40336/0 (socket says 172.21.15.5:40336) 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde63e00640 1 -- 172.21.15.5:0/3453760861 learned_addr learned my addr 172.21.15.5:0/3453760861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.660+0000 7fde68c00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae7da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde69600640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde63e00640 1 -- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 msgr2=0x55b3ae7ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde63e00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde63e00640 1 -- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 msgr2=0x55b3ae7da680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde63e00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae7da680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde63e00640 1 -- 172.21.15.5:0/3453760861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3ae4093f0 con 0x55b3ae7bb830 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde68c00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae7da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde63e00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 0x55b3ae7d0310 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fde50006a30 tx=0x7fde5000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa7097c4e25ee14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde49600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.661+0000 7fde49600640 1 -- 172.21.15.5:0/3453760861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde50018020 con 0x55b3ae7bb830 2024-04-25T10:11:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde49600640 1 -- 172.21.15.5:0/3453760861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fde50014680 con 0x55b3ae7bb830 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde58017030 con 0x55b3ae7bb830 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde49600640 1 -- 172.21.15.5:0/3453760861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde50014880 con 0x55b3ae7bb830 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 10 client.8652 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55b3ae3c90f0 con 0x55b3ae7bb830 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde49600640 1 -- 172.21.15.5:0/3453760861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58017030 con 0x55b3ae7bb830 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde49600640 1 -- 172.21.15.5:0/3453760861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde50014a20 con 0x55b3ae7bb830 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 10 client.8652 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 10 client.8652 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 20 client.8652 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7b3800 0x55b3ae7b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 4 client.8652 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.662+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3ae7021b0 con 0x55b3ae7b3800 2024-04-25T10:11:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.663+0000 7fde68c00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7b3800 0x55b3ae7b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.663+0000 7fde68c00640 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7b3800 0x55b3ae7b3be0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcc840258aee646 server_cookie=6407e5d6f4e72400 in_seq=0 out_seq=0 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.663+0000 7fde49600640 10 client.8652 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.664+0000 7fde49600640 1 -- 172.21.15.5:0/3453760861 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3ae7021b0 con 0x55b3ae7b3800 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.664+0000 7fde49600640 10 client.8652 handle_command_reply: tid=0 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.664+0000 7fde6e3b6740 1 client.8652 shutdown 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.664+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 msgr2=0x55b3ae7d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.664+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 0x55b3ae7d0310 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fde50006a30 tx=0x7fde5000f5b0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 shutdown_connections 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7b3800 0x55b3ae7b3be0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7bb830 0x55b3ae7d0310 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/3453760861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae7da680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde49600640 0 client.8652 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 >> 172.21.15.5:0/3453760861 conn(0x55b3ae758a30 msgr2=0x55b3ae7b9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 shutdown_connections 2024-04-25T10:11:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 1 -- 172.21.15.5:0/3453760861 wait complete. 2024-04-25T10:11:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.665+0000 7fde6e3b6740 20 client.8652 trim_cache size 0 max 16384 2024-04-25T10:11:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.682+0000 7fde6e3b6740 1 Processor -- start 2024-04-25T10:11:49.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.682+0000 7fde6e3b6740 1 -- start start 2024-04-25T10:11:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae790d70 0x55b3ae7be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7cd440 0x55b3ae7be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde64200110 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641ff2b0 con 0x55b3ae7cd440 2024-04-25T10:11:49.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b7110 con 0x55b3ae790d70 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde69600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde69600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40338/0 (socket says 172.21.15.5:40338) 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde69600640 1 -- 172.21.15.5:0/1346189719 learned_addr learned my addr 172.21.15.5:0/1346189719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde63e00640 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7cd440 0x55b3ae7be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.683+0000 7fde68c00640 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae790d70 0x55b3ae7be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde69600640 1 -- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae790d70 msgr2=0x55b3ae7be120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde69600640 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae790d70 0x55b3ae7be120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde69600640 1 -- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7cd440 msgr2=0x55b3ae7be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde69600640 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7cd440 0x55b3ae7be660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde69600640 1 -- 172.21.15.5:0/1346189719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3ae7da9f0 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde69600640 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf040 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02a6f0 tx=0x7fde5c078ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28292b88ad62ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde4be00640 1 -- 172.21.15.5:0/1346189719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c04a070 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde4be00640 1 -- 172.21.15.5:0/1346189719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c02ae50 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.684+0000 7fde4be00640 1 -- 172.21.15.5:0/1346189719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c078bd0 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1346189719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 msgr2=0x55b3ae7bf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf040 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02a6f0 tx=0x7fde5c078ad0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1346189719 shutdown_connections 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf040 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae7cd440 0x55b3ae7be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1346189719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae790d70 0x55b3ae7be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1346189719 >> 172.21.15.5:0/1346189719 conn(0x55b3ae758a30 msgr2=0x55b3ae7d8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1346189719 shutdown_connections 2024-04-25T10:11:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.685+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1346189719 wait complete. 2024-04-25T10:11:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.686+0000 7fde6e3b6740 1 -- start start 2024-04-25T10:11:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.686+0000 7fde6e3b6740 1 Processor -- start 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae82a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde641fedb0 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae7d95e0 con 0x55b3ae790d70 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3ae7d5520 con 0x55b3ae7cd440 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde6e3b6740 10 client.0 _ll_register_callbacks cb 0x7fde50009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.687+0000 7fde69600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40354/0 (socket says 172.21.15.5:40354) 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 -- 172.21.15.5:0/1251144668 learned_addr learned my addr 172.21.15.5:0/1251144668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde63e00640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde68c00640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae82a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 -- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 msgr2=0x55b3ae7bf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7bf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 -- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 msgr2=0x55b3ae82a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae82a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 -- 172.21.15.5:0/1251144668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3ae7cc260 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde69600640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf580 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02a6f0 tx=0x7fde5c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfffdb356d60503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde49600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde49600640 1 -- 172.21.15.5:0/1251144668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c081050 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde49600640 1 -- 172.21.15.5:0/1251144668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fde5c08cb90 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.688+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde5c02ae50 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde49600640 1 -- 172.21.15.5:0/1251144668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde5c0812e0 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 10 client.8658 fetch_fsmap learned FSMap version 83 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55b3ae7d5260 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde49600640 1 -- 172.21.15.5:0/1251144668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c02ae50 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde49600640 1 -- 172.21.15.5:0/1251144668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde5c0a5070 con 0x55b3ae7fa5c0 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 10 client.8658 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 10 client.8658 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 20 client.8658 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7ce8f0 0x55b3ae78f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 4 client.8658 mds_command: new command op to 6715 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 --> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b3ae6fc500 con 0x55b3ae7ce8f0 2024-04-25T10:11:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.689+0000 7fde68c00640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7ce8f0 0x55b3ae78f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde68c00640 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7ce8f0 0x55b3ae78f130 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e77b976b289556 server_cookie=a31ccb35e251d277 in_seq=0 out_seq=0 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde49600640 10 client.8658 ms_handle_connect on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde49600640 1 -- 172.21.15.5:0/1251144668 <== mds.0 v2:172.21.15.5:6834/2001088034 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b3ae6fc500 con 0x55b3ae7ce8f0 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde49600640 10 client.8658 handle_command_reply: tid=0 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde6e3b6740 1 client.8658 shutdown 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 msgr2=0x55b3ae7bf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.690+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf580 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02a6f0 tx=0x7fde5c009190 comp rx=0 tx=0).stop 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 shutdown_connections 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:6834/2001088034,v1:172.21.15.5:6835/2001088034] conn(0x55b3ae7ce8f0 0x55b3ae78f130 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3ae7fa5c0 0x55b3ae7bf580 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b3ae7cd440 0x55b3ae7bf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 --2- 172.21.15.5:0/1251144668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b3ae790d70 0x55b3ae82a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde49600640 0 client.8658 ms_handle_reset on v2:172.21.15.5:6834/2001088034 2024-04-25T10:11:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 >> 172.21.15.5:0/1251144668 conn(0x55b3ae758a30 msgr2=0x55b3ae7b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 shutdown_connections 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 1 -- 172.21.15.5:0/1251144668 wait complete. 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6e3b6740 20 client.8658 trim_cache size 0 max 16384 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:11:49.699 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde4406b4d0 msgr2=0x7fde4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6a000640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde4406b4d0 0x7fde4406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fde580012e0 tx=0x7fde58009040 comp rx=0 tx=0).stop 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 msgr2=0x7fde640cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.691+0000 7fde6a000640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640cc790 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fde5c035040 tx=0x7fde5c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 shutdown_connections 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde64161bc0 0x7fde640cc790 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fde4406b4d0 0x7fde4406d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde640d3f90 0x7fde640cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 --2- 172.21.15.5:0/3156300948 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde640b48c0 0x7fde64155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 >> 172.21.15.5:0/3156300948 conn(0x7fde6408af80 msgr2=0x7fde640d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 shutdown_connections 2024-04-25T10:11:49.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:49.692+0000 7fde6a000640 1 -- 172.21.15.5:0/3156300948 wait complete. 2024-04-25T10:11:49.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:49.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.019+0000 7ff4bac00640 1 Processor -- start 2024-04-25T10:11:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.019+0000 7ff4bac00640 1 -- start start 2024-04-25T10:11:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.021+0000 7ff4bac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.021+0000 7ff4bac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff4b40b3ae0 0x7ff4b40b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.021+0000 7ff4bac00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40b4880 con 0x7ff4b40ce790 2024-04-25T10:11:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.021+0000 7ff4bac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40b45a0 con 0x7ff4b40b3ae0 2024-04-25T10:11:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.021+0000 7ff4bac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b415e2c0 con 0x7ff4b4143d30 2024-04-25T10:11:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4ba200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff4b40ce790 0x7ff4b415abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55536/0 (socket says 172.21.15.5:55536) 2024-04-25T10:11:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4ba200640 1 -- 172.21.15.5:0/973563243 learned_addr learned my addr 172.21.15.5:0/973563243 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b8e00640 1 --2- 172.21.15.5:0/973563243 >> v2:172.21.15.5:3300/0 conn(0x7ff4b40b3ae0 0x7ff4b40b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b9800640 1 --2- 172.21.15.5:0/973563243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b3e00640 1 -- 172.21.15.5:0/973563243 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287066667 0 0) 0x7ff4b40b4880 con 0x7ff4b40ce790 2024-04-25T10:11:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b3e00640 1 -- 172.21.15.5:0/973563243 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff49c005730 con 0x7ff4b40ce790 2024-04-25T10:11:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b8e00640 1 -- 172.21.15.5:0/973563243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 msgr2=0x7ff4b415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b8e00640 1 --2- 172.21.15.5:0/973563243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.022+0000 7ff4b8e00640 1 -- 172.21.15.5:0/973563243 >> v1:172.21.15.5:6789/0 conn(0x7ff4b40ce790 legacy=0x7ff4b415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.023+0000 7ff4b8e00640 1 -- 172.21.15.5:0/973563243 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b415e5a0 con 0x7ff4b40b3ae0 2024-04-25T10:11:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.024+0000 7ff4b8e00640 1 --2- 172.21.15.5:0/973563243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b40b3f60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009d90 tx=0x7ff4a802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385ab36f340aa9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.024+0000 7ff4b3e00640 1 -- 172.21.15.5:0/973563243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8007740 con 0x7ff4b40b3ae0 2024-04-25T10:11:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.024+0000 7ff4b3e00640 1 -- 172.21.15.5:0/973563243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80078e0 con 0x7ff4b40b3ae0 2024-04-25T10:11:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.024+0000 7ff4b3e00640 1 -- 172.21.15.5:0/973563243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8007cb0 con 0x7ff4b40b3ae0 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.024+0000 7ff4bac00640 1 -- 172.21.15.5:0/973563243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40b3ae0 msgr2=0x7ff4b40b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.024+0000 7ff4bac00640 1 --2- 172.21.15.5:0/973563243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b40b3f60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009d90 tx=0x7ff4a802a660 comp rx=0 tx=0).stop 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.025+0000 7ff4bac00640 1 -- 172.21.15.5:0/973563243 shutdown_connections 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.025+0000 7ff4bac00640 1 --2- 172.21.15.5:0/973563243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b40b3f60 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.025+0000 7ff4bac00640 1 --2- 172.21.15.5:0/973563243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.025+0000 7ff4bac00640 1 -- 172.21.15.5:0/973563243 >> 172.21.15.5:0/973563243 conn(0x7ff4b4066f20 msgr2=0x7ff4b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.025+0000 7ff4bac00640 1 -- 172.21.15.5:0/973563243 shutdown_connections 2024-04-25T10:11:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.025+0000 7ff4bac00640 1 -- 172.21.15.5:0/973563243 wait complete. 2024-04-25T10:11:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.027+0000 7ff4bac00640 1 Processor -- start 2024-04-25T10:11:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.027+0000 7ff4bac00640 1 -- start start 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.028+0000 7ff4bac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b41e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.028+0000 7ff4bac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 0x7ff4b4130970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.028+0000 7ff4bac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b41e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.028+0000 7ff4bac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4144290 con 0x7ff4b40ce790 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.028+0000 7ff4bac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4144110 con 0x7ff4b40b3ae0 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.028+0000 7ff4bac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b406e430 con 0x7ff4b4143d30 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 0x7ff4b4130970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 0x7ff4b4130970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40360/0 (socket says 172.21.15.5:40360) 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 -- 172.21.15.5:0/1320743619 learned_addr learned my addr 172.21.15.5:0/1320743619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b9800640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b41e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4ba200640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b41e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 -- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 msgr2=0x7ff4b41e5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b41e5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 -- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40b3ae0 msgr2=0x7ff4b41e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b41e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4b8e00640 1 -- 172.21.15.5:0/1320743619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b41e3f20 con 0x7ff4b40ce790 2024-04-25T10:11:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.029+0000 7ff4ba200640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b41e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.030+0000 7ff4b8e00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 0x7ff4b4130970 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009d90 tx=0x7ff4a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.030+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8040070 con 0x7ff4b40ce790 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.030+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a8031c40 con 0x7ff4b40ce790 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.030+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8009450 con 0x7ff4b40ce790 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.030+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b41e4200 con 0x7ff4b40ce790 2024-04-25T10:11:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.030+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b41e47c0 con 0x7ff4b40ce790 2024-04-25T10:11:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.031+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b406aee0 con 0x7ff4b40ce790 2024-04-25T10:11:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.032+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4a8039040 con 0x7ff4b40ce790 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.033+0000 7ff4b1600640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b550 0x7ff49406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.033+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff4a80abbf0 con 0x7ff4b40ce790 2024-04-25T10:11:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.034+0000 7ff4b9800640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b550 0x7ff49406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.035+0000 7ff4b9800640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b550 0x7ff49406da10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009a30 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.038+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4a807a8e0 con 0x7ff4b40ce790 2024-04-25T10:11:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.283+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff4b406bf70 con 0x7ff4b40ce790 2024-04-25T10:11:50.288 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:11:50.284+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.288+0000 7ff4b1600640 1 -- 172.21.15.5:0/1320743619 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff4a807a1e0 con 0x7ff4b40ce790 2024-04-25T10:11:50.292 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b550 msgr2=0x7ff49406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b550 0x7ff49406da10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009a30 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 msgr2=0x7ff4b4130970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 0x7ff4b4130970 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009d90 tx=0x7ff4a8004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 shutdown_connections 2024-04-25T10:11:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff4b4143d30 0x7ff4b41e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4b40ce790 0x7ff4b4130970 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff49406b550 0x7ff49406da10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 --2- 172.21.15.5:0/1320743619 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4b40b3ae0 0x7ff4b41e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.291+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 >> 172.21.15.5:0/1320743619 conn(0x7ff4b4066f20 msgr2=0x7ff4b40cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.292+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 shutdown_connections 2024-04-25T10:11:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.292+0000 7ff4bac00640 1 -- 172.21.15.5:0/1320743619 wait complete. 2024-04-25T10:11:50.317 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T10:11:50.318 INFO:tasks.ceph.mds.c.smithi005.stderr: -7086> 2024-04-25T10:11:04.502+0000 7f1a28000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:50.377 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:11:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.403+0000 7f1ac3838280 1 -- start start 2024-04-25T10:11:50.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.403+0000 7f1ac3838280 1 Processor -- start 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779434800 0x563778739b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 0x563778739080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779435400 0x563778738b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac3838280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563779436480 con 0x563779435c00 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac3838280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563779436600 con 0x563779435400 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac3838280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563779436780 con 0x563779434800 2024-04-25T10:11:50.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 0x563778739080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 0x563778739080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40394/0 (socket says 172.21.15.5:40394) 2024-04-25T10:11:50.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.404+0000 7f1ac0e00640 1 -- 172.21.15.5:0/4150207219 learned_addr learned my addr 172.21.15.5:0/4150207219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac1800640 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779434800 0x563778739b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac2200640 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779435400 0x563778738b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac0e00640 1 -- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779434800 msgr2=0x563778739b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac0e00640 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779434800 0x563778739b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac0e00640 1 -- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779435400 msgr2=0x563778738b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac0e00640 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779435400 0x563778738b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac0e00640 1 -- 172.21.15.5:0/4150207219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563778700780 con 0x563779435c00 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1ac0e00640 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 0x563778739080 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x5637794e72c0 tx=0x563779494480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28faf1524242603 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1abe600640 1 -- 172.21.15.5:0/4150207219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5637786f2d00 con 0x563779435c00 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1abe600640 1 -- 172.21.15.5:0/4150207219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5637786f8700 con 0x563779435c00 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.405+0000 7f1abe600640 1 -- 172.21.15.5:0/4150207219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5637786f2680 con 0x563779435c00 2024-04-25T10:11:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563778700000 con 0x563779435c00 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563778701a40 con 0x563779435c00 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1ac3838280 4 waiting for MDS map... 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1abe600640 1 -- 172.21.15.5:0/4150207219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x5637794ca400 con 0x563779435c00 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1abe600640 1 -- 172.21.15.5:0/4150207219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 84) v1 ==== 2723+0+0 (secure 0 0 0) 0x563778727180 con 0x563779435c00 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1ac3838280 4 Got MDS map 84 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1ac3838280 10 main: JournalTool::main 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.406+0000 7f1ac3838280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.407+0000 7f1ac3838280 1 Processor -- start 2024-04-25T10:11:50.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.407+0000 7f1ac3838280 1 -- start start 2024-04-25T10:11:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac3838280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56377872e180 con 0x563779593000 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac3838280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563779436f00 con 0x563779592c00 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac3838280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563779437080 con 0x563779593400 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40406/0 (socket says 172.21.15.5:40406) 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac1800640 1 -- 172.21.15.5:0/3118334908 learned_addr learned my addr 172.21.15.5:0/3118334908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac1800640 1 -- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779593400 msgr2=0x56377873ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.408+0000 7f1ac1800640 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac1800640 1 -- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779592c00 msgr2=0x56377873a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac2200640 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac1800640 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac1800640 1 -- 172.21.15.5:0/3118334908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563778701860 con 0x563779593000 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac1800640 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 0x56377873a680 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x5637795ac600 tx=0x5637795b0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8064964d95ff6a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1abbe00640 1 -- 172.21.15.5:0/3118334908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5637786f2d00 con 0x563779593000 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1abbe00640 1 -- 172.21.15.5:0/3118334908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5637786f8fc0 con 0x563779593000 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1abbe00640 1 -- 172.21.15.5:0/3118334908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5637786f2680 con 0x563779593000 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac3838280 1 -- 172.21.15.5:0/3118334908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 msgr2=0x56377873a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.409+0000 7f1ac3838280 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 0x56377873a680 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x5637795ac600 tx=0x5637795b0c00 comp rx=0 tx=0).stop 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 -- 172.21.15.5:0/3118334908 shutdown_connections 2024-04-25T10:11:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 --2- 172.21.15.5:0/3118334908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 -- 172.21.15.5:0/3118334908 >> 172.21.15.5:0/3118334908 conn(0x563779435800 msgr2=0x563779449800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 -- 172.21.15.5:0/3118334908 shutdown_connections 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 -- 172.21.15.5:0/3118334908 wait complete. 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 Processor -- start 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.410+0000 7f1ac3838280 1 -- start start 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac3838280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac3838280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563779437380 con 0x563779593400 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac3838280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563779437200 con 0x563779593000 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac3838280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563779437500 con 0x563779592c00 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.411+0000 7f1ac0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40412/0 (socket says 172.21.15.5:40412) 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 -- 172.21.15.5:0/1726781801 learned_addr learned my addr 172.21.15.5:0/1726781801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac2200640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac1800640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 -- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779592c00 msgr2=0x56377873a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779592c00 0x56377873a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 -- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779593000 msgr2=0x56377873a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779593000 0x56377873a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563778700780 con 0x563779593400 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac0e00640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 0x56377873ac00 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x5637795ad920 tx=0x563779494180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1aba000640 1 -- 172.21.15.5:0/1726781801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5637786f2820 con 0x563779593400 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1aba000640 1 -- 172.21.15.5:0/1726781801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5637786f8c40 con 0x563779593400 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563778700000 con 0x563779593400 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.412+0000 7f1aba000640 1 -- 172.21.15.5:0/1726781801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5637786f2d00 con 0x563779593400 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.413+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563778701680 con 0x563779593400 2024-04-25T10:11:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.413+0000 7f1ac3838280 4 main: JournalTool: resolving pool 15 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.414+0000 7f1aba000640 1 -- 172.21.15.5:0/1726781801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x563779584c00 con 0x563779593400 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1aba000640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x563779601c00 0x56377873b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1aba000640 1 -- 172.21.15.5:0/1726781801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x563779455e00 con 0x563779593400 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac3838280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac3838280 4 main: Executing for rank 0 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac3838280 4 Filtering by inode '1099511627777' 2024-04-25T10:11:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x563779593800 0x56377873b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x563779593c00 con 0x563779593800 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac1800640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x563779593800 0x56377873b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.415+0000 7f1ac2200640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x563779601c00 0x56377873b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.416+0000 7f1ac1800640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x563779593800 0x56377873b180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.417+0000 7f1ac2200640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x563779601c00 0x56377873b700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5637787157a0 tx=0x5637794944e0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.418+0000 7f1ac1800640 1 -- 172.21.15.5:0/1726781801 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x56377944a000 con 0x563779593800 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.419+0000 7f1ac3838280 4 Pointer 400.00000000 is readable 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.419+0000 7f1ac3838280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.419+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x563779593c00 0x563778a48000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.419+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x563779600800 con 0x563779593c00 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.419+0000 7f1ac2200640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x563779593c00 0x563778a48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.419+0000 7f1ac2200640 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x563779593c00 0x563778a48000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac2200640 1 -- 172.21.15.5:0/1726781801 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x56377944a500 con 0x563779593c00 2024-04-25T10:11:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 10 Header 0x400000 0x407d9d 0x408119 2024-04-25T10:11:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 10 Starting journal scan from offset 0x407d9d 2024-04-25T10:11:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32157~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x563779600000 con 0x563779593c00 2024-04-25T10:11:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac2200640 1 -- 172.21.15.5:0/1726781801 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(3 200.00000001 [read 32157~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x56377944a500 con 0x563779593c00 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 10 Parsing data, 0x37c bytes available 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 10 Attempting decode at 0x407d9d 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 10 Consumed 0x37c bytes 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 10 Valid entry at 0x407d9d 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.422+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x563779600400 con 0x563779593c00 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac2200640 1 -- 172.21.15.5:0/1726781801 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56377944a500 con 0x563779593c00 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stdout:Events by type: 2024-04-25T10:11:50.430 INFO:teuthology.orchestra.run.smithi005.stdout:Errors: 0 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 4 Reached end of journal objects 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 4 Events scanned, 0 gaps 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 4 Found 1 valid events 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 4 Selected 0 events events for processing 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 4 0 events 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x563779593800 msgr2=0x56377873b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x563779593800 0x56377873b180 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x563779593c00 msgr2=0x563778a48000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x563779593c00 0x563778a48000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x563779601c00 msgr2=0x56377873b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x563779601c00 0x56377873b700 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5637787157a0 tx=0x5637794944e0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 msgr2=0x56377873ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 0x56377873ac00 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x5637795ad920 tx=0x563779494180 comp rx=0 tx=0).stop 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac1800640 1 -- 172.21.15.5:0/1726781801 reap_dead start 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 shutdown_connections 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 --2- 172.21.15.5:0/1726781801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779593400 0x56377873ac00 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 >> 172.21.15.5:0/1726781801 conn(0x563779435800 msgr2=0x563779449800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 shutdown_connections 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.423+0000 7f1ac3838280 1 -- 172.21.15.5:0/1726781801 wait complete. 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 msgr2=0x563778739080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 0x563778739080 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x5637794e72c0 tx=0x563779494480 comp rx=0 tx=0).stop 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 shutdown_connections 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563779435400 0x563778738b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563779435c00 0x563778739080 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 --2- 172.21.15.5:0/4150207219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563779434800 0x563778739b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 >> 172.21.15.5:0/4150207219 conn(0x5637786cd000 msgr2=0x563779449000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 shutdown_connections 2024-04-25T10:11:50.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.424+0000 7f1ac3838280 1 -- 172.21.15.5:0/4150207219 wait complete. 2024-04-25T10:11:50.435 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.117479, result: Events by type: Errors: 0 2024-04-25T10:11:50.435 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T10:11:50.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.516+0000 7f11a3fd5280 1 -- start start 2024-04-25T10:11:50.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.516+0000 7f11a3fd5280 1 Processor -- start 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4323fa800 0x55b431695b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b431629c00 0x55b431694b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 0x55b431695600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b432432000 con 0x55b4323fbc00 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b432432180 con 0x55b4323fa800 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b432432300 con 0x55b431629c00 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 0x55b431695600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.517+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 0x55b431695600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40430/0 (socket says 172.21.15.5:40430) 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 -- 172.21.15.5:0/568001235 learned_addr learned my addr 172.21.15.5:0/568001235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a1600640 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b431629c00 0x55b431694b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2000640 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4323fa800 0x55b431695b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 -- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b431629c00 msgr2=0x55b431694b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b431629c00 0x55b431694b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 -- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4323fa800 msgr2=0x55b431695b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4323fa800 0x55b431695b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 -- 172.21.15.5:0/568001235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43165c780 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f11a2a00640 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 0x55b431695600 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x55b432453380 tx=0x55b4323f8360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096889153684e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.518+0000 7f119ee00640 1 -- 172.21.15.5:0/568001235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b43164e680 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f119ee00640 1 -- 172.21.15.5:0/568001235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b431654700 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f119ee00640 1 -- 172.21.15.5:0/568001235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b43164e9c0 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43165c000 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b43165da40 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f11a3fd5280 4 waiting for MDS map... 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f119ee00640 1 -- 172.21.15.5:0/568001235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x55b432386a00 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f119ee00640 1 -- 172.21.15.5:0/568001235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 84) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b431683180 con 0x55b4323fbc00 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f11a3fd5280 4 Got MDS map 84 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f11a3fd5280 10 main: JournalTool::main 2024-04-25T10:11:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.519+0000 7f11a3fd5280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:11:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.520+0000 7f11a3fd5280 1 Processor -- start 2024-04-25T10:11:50.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.520+0000 7f11a3fd5280 1 -- start start 2024-04-25T10:11:50.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431695080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 0x55b431696100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432556000 0x55b431696680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b432432480 con 0x55b4323fa000 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b432432600 con 0x55b431629800 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b432432780 con 0x55b432556000 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431695080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431695080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40440/0 (socket says 172.21.15.5:40440) 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 -- 172.21.15.5:0/3550419399 learned_addr learned my addr 172.21.15.5:0/3550419399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 -- 172.21.15.5:0/3550419399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432556000 msgr2=0x55b431696680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432556000 0x55b431696680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 -- 172.21.15.5:0/3550419399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 msgr2=0x55b431696100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 0x55b431696100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.521+0000 7f11a1600640 1 -- 172.21.15.5:0/3550419399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43165d860 con 0x55b4323fa000 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a1600640 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431695080 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x55b4324f08a0 tx=0x55b43250cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d62756720a5128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f119c600640 1 -- 172.21.15.5:0/3550419399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b43164e680 con 0x55b4323fa000 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f119c600640 1 -- 172.21.15.5:0/3550419399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b431654c40 con 0x55b4323fa000 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f119c600640 1 -- 172.21.15.5:0/3550419399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b43164e9c0 con 0x55b4323fa000 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3550419399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 msgr2=0x55b431695080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431695080 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x55b4324f08a0 tx=0x55b43250cc00 comp rx=0 tx=0).stop 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3550419399 shutdown_connections 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432556000 0x55b431696680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 0x55b431696100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3550419399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431695080 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3550419399 >> 172.21.15.5:0/3550419399 conn(0x55b4323fa400 msgr2=0x55b4323a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.522+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3550419399 shutdown_connections 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.523+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3550419399 wait complete. 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.523+0000 7f11a3fd5280 1 Processor -- start 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.523+0000 7f11a3fd5280 1 -- start start 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 0x55b431697700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431697180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a3fd5280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432557400 0x55b431696c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b432433e00 con 0x55b4323fa000 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b432433c80 con 0x55b431629800 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a3fd5280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b432433b00 con 0x55b432557400 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431697180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431697180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40452/0 (socket says 172.21.15.5:40452) 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.524+0000 7f11a2000640 1 -- 172.21.15.5:0/3478180439 learned_addr learned my addr 172.21.15.5:0/3478180439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 0x55b431697700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432557400 0x55b431696c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a2000640 1 -- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432557400 msgr2=0x55b431696c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a2000640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432557400 0x55b431696c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a2000640 1 -- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 msgr2=0x55b431697700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a2000640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b431629800 0x55b431697700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a2000640 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43165d680 con 0x55b4323fa000 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b432557400 0x55b431696c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a2000640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431697180 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x55b4324f14a0 tx=0x55b43250cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f119a800640 1 -- 172.21.15.5:0/3478180439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b43164e680 con 0x55b4323fa000 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f119a800640 1 -- 172.21.15.5:0/3478180439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b4316541c0 con 0x55b4323fa000 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f119a800640 1 -- 172.21.15.5:0/3478180439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b43164e9c0 con 0x55b4323fa000 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.525+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b43165c3c0 con 0x55b4323fa000 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.526+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43165dc20 con 0x55b4323fa000 2024-04-25T10:11:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.526+0000 7f11a3fd5280 4 main: JournalTool: resolving pool 15 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.527+0000 7f119a800640 1 -- 172.21.15.5:0/3478180439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55b43255ec00 con 0x55b4323fa000 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f119a800640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55b43257ec00 0x55b4319a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f119a800640 1 -- 172.21.15.5:0/3478180439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x55b43255ce00 con 0x55b4323fa000 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a3fd5280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a3fd5280 4 main: Executing for rank 0 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a3fd5280 4 Filtering by inode '1099511627778' 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55b432557000 0x55b431696680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b432556c00 con 0x55b432557000 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55b432557000 0x55b431696680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.528+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55b43257ec00 0x55b4319a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.529+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55b432557000 0x55b431696680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.529+0000 7f11a1600640 1 -- 172.21.15.5:0/3478180439 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55b4323a6000 con 0x55b432557000 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.529+0000 7f11a3fd5280 4 Pointer 400.00000000 is readable 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.529+0000 7f11a3fd5280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.529+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55b432556800 0x55b431695080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.529+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b432556400 con 0x55b432556800 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a2a00640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55b432556800 0x55b431695080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a1600640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55b43257ec00 0x55b4319a0000 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55b4325aa0f0 tx=0x55b43250ccc0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a2a00640 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55b432556800 0x55b431695080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a2a00640 1 -- 172.21.15.5:0/3478180439 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55b4323a6500 con 0x55b432556800 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a3fd5280 10 Header 0x400000 0x407d9d 0x408119 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a3fd5280 10 Starting journal scan from offset 0x407d9d 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.530+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32157~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b432556000 con 0x55b432556800 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a2a00640 1 -- 172.21.15.5:0/3478180439 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(3 200.00000001 [read 32157~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x55b4323a6500 con 0x55b432556800 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 10 Parsing data, 0x37c bytes available 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 10 Attempting decode at 0x407d9d 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 10 Consumed 0x37c bytes 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 10 Valid entry at 0x407d9d 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) v9 -- 0x55b43257f000 con 0x55b432556800 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a2a00640 1 -- 172.21.15.5:0/3478180439 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b4323a6500 con 0x55b432556800 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 Reached end of journal objects 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:11:50.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 Events scanned, 0 gaps 2024-04-25T10:11:50.540 INFO:teuthology.orchestra.run.smithi005.stdout:Events by type: 2024-04-25T10:11:50.540 INFO:teuthology.orchestra.run.smithi005.stdout:Errors: 0 2024-04-25T10:11:50.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 Found 1 valid events 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 Selected 0 events events for processing 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 4 0 events 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55b432557000 msgr2=0x55b431696680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55b432557000 0x55b431696680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55b432556800 msgr2=0x55b431695080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55b432556800 0x55b431695080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55b43257ec00 msgr2=0x55b4319a0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55b43257ec00 0x55b4319a0000 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55b4325aa0f0 tx=0x55b43250ccc0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 msgr2=0x55b431697180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/3478180439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fa000 0x55b431697180 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x55b4324f14a0 tx=0x55b43250cd20 comp rx=0 tx=0).stop 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a2000640 1 -- 172.21.15.5:0/3478180439 reap_dead start 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a2000640 1 -- 172.21.15.5:0/3478180439 reap_dead start 2024-04-25T10:11:50.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 shutdown_connections 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.531+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 >> 172.21.15.5:0/3478180439 conn(0x55b4323fa400 msgr2=0x55b4323a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 shutdown_connections 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/3478180439 wait complete. 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 msgr2=0x55b431695600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 0x55b431695600 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x55b432453380 tx=0x55b4323f8360 comp rx=0 tx=0).stop 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 shutdown_connections 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4323fbc00 0x55b431695600 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b431629c00 0x55b431694b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 --2- 172.21.15.5:0/568001235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b4323fa800 0x55b431695b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 >> 172.21.15.5:0/568001235 conn(0x55b431629000 msgr2=0x55b4323a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 shutdown_connections 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.532+0000 7f11a3fd5280 1 -- 172.21.15.5:0/568001235 wait complete. 2024-04-25T10:11:50.543 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.108475, result: Events by type: Errors: 0 2024-04-25T10:11:50.544 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 Processor -- start 2024-04-25T10:11:50.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 -- start start 2024-04-25T10:11:50.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 0x55cc66eabe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cc66f327c0 0x55cc66eaa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f31af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66e20dd0 con 0x7fe088003630 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66ec6380 con 0x55cc66f327c0 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.623+0000 7fe0a631bf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc66f4e910 con 0x55cc66eaa670 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 0x55cc66eabe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 0x55cc66eabe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40480/0 (socket says 172.21.15.5:40480) 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 -- 172.21.15.5:0/3402338705 learned_addr learned my addr 172.21.15.5:0/3402338705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe09be00640 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cc66f327c0 0x55cc66eaa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a3400640 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f31af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 -- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 msgr2=0x55cc66f31af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f31af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 -- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cc66f327c0 msgr2=0x55cc66eaa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cc66f327c0 0x55cc66eaa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.624+0000 7fe0a2a00640 1 -- 172.21.15.5:0/3402338705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc66f4e670 con 0x7fe088003630 2024-04-25T10:11:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a2a00640 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 0x55cc66eabe80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe0940097a0 tx=0x7fe094009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbaa01e07e1dde34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a1600640 1 -- 172.21.15.5:0/3402338705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940088d0 con 0x7fe088003630 2024-04-25T10:11:50.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a1600640 1 -- 172.21.15.5:0/3402338705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094019070 con 0x7fe088003630 2024-04-25T10:11:50.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a1600640 1 -- 172.21.15.5:0/3402338705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094008c90 con 0x7fe088003630 2024-04-25T10:11:50.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 -- 172.21.15.5:0/3402338705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 msgr2=0x55cc66eabe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 0x55cc66eabe80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe0940097a0 tx=0x7fe094009890 comp rx=0 tx=0).stop 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 -- 172.21.15.5:0/3402338705 shutdown_connections 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f31af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55cc66f327c0 0x55cc66eaa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/3402338705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe088003630 0x55cc66eabe80 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 -- 172.21.15.5:0/3402338705 >> 172.21.15.5:0/3402338705 conn(0x55cc66e78c60 msgr2=0x55cc66e77f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.625+0000 7fe0a631bf40 1 -- 172.21.15.5:0/3402338705 shutdown_connections 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.626+0000 7fe0a631bf40 1 -- 172.21.15.5:0/3402338705 wait complete. 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.626+0000 7fe0a631bf40 1 Processor -- start 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.626+0000 7fe0a631bf40 1 -- start start 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.627+0000 7fe0a631bf40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f4c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.627+0000 7fe0a631bf40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc66f327c0 0x55cc66f34380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.627+0000 7fe0a631bf40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088003630 0x55cc66f31af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.627+0000 7fe0a631bf40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66ec5a60 con 0x55cc66f327c0 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.627+0000 7fe0a631bf40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc66ec57c0 con 0x7fe088003630 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.627+0000 7fe0a631bf40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc66eab960 con 0x55cc66eaa670 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc66f327c0 0x55cc66f34380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc66f327c0 0x55cc66f34380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40486/0 (socket says 172.21.15.5:40486) 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 -- 172.21.15.5:0/1926177877 learned_addr learned my addr 172.21.15.5:0/1926177877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe0a2a00640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f4c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 -- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 msgr2=0x55cc66f4c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55cc66eaa670 0x55cc66f4c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 -- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088003630 msgr2=0x55cc66f31af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe088003630 0x55cc66f31af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 -- 172.21.15.5:0/1926177877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc66f4cc40 con 0x55cc66f327c0 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09be00640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc66f327c0 0x55cc66f34380 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00a510 tx=0x7fe08c00a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.628+0000 7fe09b400640 1 -- 172.21.15.5:0/1926177877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c008040 con 0x55cc66f327c0 2024-04-25T10:11:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.629+0000 7fe09b400640 1 -- 172.21.15.5:0/1926177877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c014020 con 0x55cc66f327c0 2024-04-25T10:11:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.629+0000 7fe09b400640 1 -- 172.21.15.5:0/1926177877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00f640 con 0x55cc66f327c0 2024-04-25T10:11:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.629+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cc66f53dc0 con 0x55cc66f327c0 2024-04-25T10:11:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.629+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc66f36570 con 0x55cc66f327c0 2024-04-25T10:11:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.631+0000 7fe09b400640 1 -- 172.21.15.5:0/1926177877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe08c00f7e0 con 0x55cc66f327c0 2024-04-25T10:11:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.632+0000 7fe09b400640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe080069eb0 0x7fe08006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.632+0000 7fe09b400640 1 -- 172.21.15.5:0/1926177877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe08c089450 con 0x55cc66f327c0 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.632+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55cc66f4df80 0x55cc66fe7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.632+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) v9 -- 0x55cc66fe7e70 con 0x55cc66f4df80 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.632+0000 7fe0a2a00640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe080069eb0 0x7fe08006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.633+0000 7fe0a3400640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55cc66f4df80 0x55cc66fe7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.634+0000 7fe0a2a00640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe080069eb0 0x7fe08006c370 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe094016040 tx=0x7fe09400c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.634+0000 7fe0a3400640 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55cc66f4df80 0x55cc66fe7930 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a3400640 1 -- 172.21.15.5:0/1926177877 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v108'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe09c005040 con 0x55cc66f4df80 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55cc66f4df80 msgr2=0x55cc66fe7930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55cc66f4df80 0x55cc66fe7930 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe080069eb0 msgr2=0x7fe08006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe080069eb0 0x7fe08006c370 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe094016040 tx=0x7fe09400c000 comp rx=0 tx=0).stop 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc66f327c0 msgr2=0x55cc66f34380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 --2- 172.21.15.5:0/1926177877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc66f327c0 0x55cc66f34380 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00a510 tx=0x7fe08c00a9d0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a3400640 1 -- 172.21.15.5:0/1926177877 reap_dead start 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 shutdown_connections 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 >> 172.21.15.5:0/1926177877 conn(0x55cc66e78c60 msgr2=0x55cc66f4fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 shutdown_connections 2024-04-25T10:11:50.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.638+0000 7fe0a631bf40 1 -- 172.21.15.5:0/1926177877 wait complete. 2024-04-25T10:11:50.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:11:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.902+0000 7f7423200640 1 Processor -- start 2024-04-25T10:11:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.903+0000 7f7423200640 1 -- start start 2024-04-25T10:11:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.904+0000 7f7423200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f742415a6b0 0x7f74240ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.904+0000 7f7423200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74240ae4b0 con 0x7f74240b19a0 2024-04-25T10:11:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.904+0000 7f7423200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74240ae790 con 0x7f742415aa90 2024-04-25T10:11:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.904+0000 7f7423200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240af0c0 con 0x7f742415a6b0 2024-04-25T10:11:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.904+0000 7f7422800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f74240b19a0 0x7f74240af550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55570/0 (socket says 172.21.15.5:55570) 2024-04-25T10:11:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.904+0000 7f7422800640 1 -- 172.21.15.5:0/637166425 learned_addr learned my addr 172.21.15.5:0/637166425 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7421e00640 1 --2- 172.21.15.5:0/637166425 >> v2:172.21.15.80:3300/0 conn(0x7f742415a6b0 0x7f74240ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4738062 0 0) 0x7f74240ae4b0 con 0x7f74240b19a0 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7408005730 con 0x7f74240b19a0 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3320883409 0 0) 0x7f7408005730 con 0x7f74240b19a0 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 >> v2:172.21.15.80:3300/0 conn(0x7f742415a6b0 msgr2=0x7f74240ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 --2- 172.21.15.5:0/637166425 >> v2:172.21.15.80:3300/0 conn(0x7f742415a6b0 0x7f74240ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 >> v1:172.21.15.80:6789/0 conn(0x7f742415aa90 legacy=0x7f74240b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240aea70 con 0x7f74240b19a0 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f741c0019f0 con 0x7f74240b19a0 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 >> v1:172.21.15.5:6789/0 conn(0x7f74240b19a0 legacy=0x7f74240af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7408005c10 0x7f7408006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408006a10 0x7f7408006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74080073d0 0x7f740800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240af0c0 con 0x7f7408005c10 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740800e2d0 con 0x7f74080073d0 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.905+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740800e4d0 con 0x7f7408006a10 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7421e00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7408005c10 0x7f7408006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7422800640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74080073d0 0x7f740800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7421e00640 1 -- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408006a10 msgr2=0x7f7408006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7421e00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408006a10 0x7f7408006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7421e00640 1 -- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74080073d0 msgr2=0x7f740800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7421e00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74080073d0 0x7f740800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7421e00640 1 -- 172.21.15.5:0/637166425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408005730 con 0x7f7408005c10 2024-04-25T10:11:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7422800640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74080073d0 0x7f740800dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.906+0000 7f7422800640 1 -- 172.21.15.5:0/637166425 reap_dead start 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.907+0000 7f7421e00640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7408005c10 0x7f7408006030 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7414009ac0 tx=0x7f7408007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6283e4b1f9b69c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414036070 con 0x7f7408005c10 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414031400 con 0x7f7408005c10 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7420a00640 1 -- 172.21.15.5:0/637166425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414009690 con 0x7f7408005c10 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 -- 172.21.15.5:0/637166425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7408005c10 msgr2=0x7f7408006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7408005c10 0x7f7408006030 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7414009ac0 tx=0x7f7408007a40 comp rx=0 tx=0).stop 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 -- 172.21.15.5:0/637166425 shutdown_connections 2024-04-25T10:11:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 --2- 172.21.15.5:0/637166425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7408005c10 0x7f7408006030 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 -- 172.21.15.5:0/637166425 >> 172.21.15.5:0/637166425 conn(0x7f74240b5ab0 msgr2=0x7f74240999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 -- 172.21.15.5:0/637166425 shutdown_connections 2024-04-25T10:11:50.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.908+0000 7f7423200640 1 -- 172.21.15.5:0/637166425 wait complete. 2024-04-25T10:11:50.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.910+0000 7f7423200640 1 Processor -- start 2024-04-25T10:11:50.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.910+0000 7f7423200640 1 -- start start 2024-04-25T10:11:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.911+0000 7f7423200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408005c10 0x7f74241512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.911+0000 7f7423200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7424158c20 0x7f7424146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.911+0000 7f7423200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 0x7f7424146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.911+0000 7f7423200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240d06e0 con 0x7f74241590a0 2024-04-25T10:11:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.911+0000 7f7423200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240d0560 con 0x7f7424158c20 2024-04-25T10:11:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.911+0000 7f7423200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74240b1290 con 0x7f7408005c10 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 0x7f7424146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 0x7f7424146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40502/0 (socket says 172.21.15.5:40502) 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 -- 172.21.15.5:0/736096910 learned_addr learned my addr 172.21.15.5:0/736096910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7421e00640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408005c10 0x7f74241512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7421400640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7424158c20 0x7f7424146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 -- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408005c10 msgr2=0x7f74241512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408005c10 0x7f74241512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 -- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7424158c20 msgr2=0x7f7424146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7424158c20 0x7f7424146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.912+0000 7f7422800640 1 -- 172.21.15.5:0/736096910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74241542b0 con 0x7f74241590a0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.913+0000 7f7422800640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 0x7f7424146fd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f742406dfe0 tx=0x7f741c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.913+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c004d60 con 0x7f74241590a0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.913+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c00bb20 con 0x7f74241590a0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.913+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c007110 con 0x7f74241590a0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.913+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74241f3820 con 0x7f74241590a0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.913+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74241f3de0 con 0x7f74241590a0 2024-04-25T10:11:50.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.915+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7424068f60 con 0x7f74241590a0 2024-04-25T10:11:50.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.915+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f741c01b020 con 0x7f74241590a0 2024-04-25T10:11:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.916+0000 7f741a000640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f740406b4d0 0x7f740406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.917+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f741c08a570 con 0x7f74241590a0 2024-04-25T10:11:50.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.917+0000 7f7421e00640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f740406b4d0 0x7f740406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.919+0000 7f7421e00640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f740406b4d0 0x7f740406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7414033000 tx=0x7f7414008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:50.921+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f741c059230 con 0x7f74241590a0 2024-04-25T10:11:51.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.167+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f74240adf40 con 0x7f74241590a0 2024-04-25T10:11:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.382+0000 7f741a000640 1 -- 172.21.15.5:0/736096910 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) v1 ==== 162+0+0 (secure 0 0 0) 0x7f741c058b30 con 0x7f74241590a0 2024-04-25T10:11:51.387 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:11:51.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f740406b4d0 msgr2=0x7f740406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f740406b4d0 0x7f740406d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7414033000 tx=0x7f7414008030 comp rx=0 tx=0).stop 2024-04-25T10:11:51.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 msgr2=0x7f7424146fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 0x7f7424146fd0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f742406dfe0 tx=0x7f741c00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 shutdown_connections 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74241590a0 0x7f7424146fd0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f740406b4d0 0x7f740406d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7424158c20 0x7f7424146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 --2- 172.21.15.5:0/736096910 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7408005c10 0x7f74241512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.386+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 >> 172.21.15.5:0/736096910 conn(0x7f74240b5ab0 msgr2=0x7f742406efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.387+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 shutdown_connections 2024-04-25T10:11:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.387+0000 7f7423200640 1 -- 172.21.15.5:0/736096910 wait complete. 2024-04-25T10:11:51.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:51.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.597+0000 7ffa6e600640 1 Processor -- start 2024-04-25T10:11:51.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.597+0000 7ffa6e600640 1 -- start start 2024-04-25T10:11:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6e600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa68141da0 0x7ffa6806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6e600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffa6815d340 0x7ffa680cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6e600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa68142180 con 0x7ffa68141da0 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6e600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa68071d30 con 0x7ffa6815d340 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6e600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6806d620 con 0x7ffa6806e640 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6d200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa68141da0 0x7ffa6806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6d200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa68141da0 0x7ffa6806e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40510/0 (socket says 172.21.15.5:40510) 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6d200640 1 -- 172.21.15.5:0/326085412 learned_addr learned my addr 172.21.15.5:0/326085412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa67e00640 1 --2- 172.21.15.5:0/326085412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6dc00640 1 --2- 172.21.15.5:0/326085412 >> v2:172.21.15.80:3300/0 conn(0x7ffa6815d340 0x7ffa680cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.599+0000 7ffa6d200640 1 -- 172.21.15.5:0/326085412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 msgr2=0x7ffa6815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.600+0000 7ffa6d200640 1 --2- 172.21.15.5:0/326085412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.600+0000 7ffa6d200640 1 -- 172.21.15.5:0/326085412 >> v2:172.21.15.80:3300/0 conn(0x7ffa6815d340 msgr2=0x7ffa680cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.600+0000 7ffa6d200640 1 --2- 172.21.15.5:0/326085412 >> v2:172.21.15.80:3300/0 conn(0x7ffa6815d340 0x7ffa680cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.600+0000 7ffa6d200640 1 -- 172.21.15.5:0/326085412 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6806d900 con 0x7ffa68141da0 2024-04-25T10:11:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.600+0000 7ffa6dc00640 1 --2- 172.21.15.5:0/326085412 >> v2:172.21.15.80:3300/0 conn(0x7ffa6815d340 0x7ffa680cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:51.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.601+0000 7ffa6d200640 1 --2- 172.21.15.5:0/326085412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa6806e100 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ffa680b2460 tx=0x7ffa5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af8a22f86dd8d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.601+0000 7ffa67400640 1 -- 172.21.15.5:0/326085412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c036070 con 0x7ffa68141da0 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.601+0000 7ffa67400640 1 -- 172.21.15.5:0/326085412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c031600 con 0x7ffa68141da0 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.601+0000 7ffa67400640 1 -- 172.21.15.5:0/326085412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0319d0 con 0x7ffa68141da0 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 -- 172.21.15.5:0/326085412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 msgr2=0x7ffa6806e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 --2- 172.21.15.5:0/326085412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa6806e100 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ffa680b2460 tx=0x7ffa5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 -- 172.21.15.5:0/326085412 shutdown_connections 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 --2- 172.21.15.5:0/326085412 >> v2:172.21.15.80:3300/0 conn(0x7ffa6815d340 0x7ffa680cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 --2- 172.21.15.5:0/326085412 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 --2- 172.21.15.5:0/326085412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa6806e100 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 -- 172.21.15.5:0/326085412 >> 172.21.15.5:0/326085412 conn(0x7ffa68066f20 msgr2=0x7ffa6806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 -- 172.21.15.5:0/326085412 shutdown_connections 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.602+0000 7ffa6e600640 1 -- 172.21.15.5:0/326085412 wait complete. 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.603+0000 7ffa6e600640 1 Processor -- start 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.604+0000 7ffa6e600640 1 -- start start 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa680ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa6815d340 0x7ffa681e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680cd9c0 con 0x7ffa68141da0 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680cd840 con 0x7ffa6815d340 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6e600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680cdb40 con 0x7ffa6806e640 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa680ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa680ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40526/0 (socket says 172.21.15.5:40526) 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 -- 172.21.15.5:0/470053353 learned_addr learned my addr 172.21.15.5:0/470053353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6dc00640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa6815d340 0x7ffa681e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa6d200640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 -- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 msgr2=0x7ffa6813cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6813cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 -- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa6815d340 msgr2=0x7ffa681e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa6815d340 0x7ffa681e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.605+0000 7ffa67e00640 1 -- 172.21.15.5:0/470053353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa680aded0 con 0x7ffa68141da0 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.606+0000 7ffa67e00640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa680ad3e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400d600 tx=0x7ffa5400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.606+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004240 con 0x7ffa68141da0 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.606+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa540043e0 con 0x7ffa68141da0 2024-04-25T10:11:51.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.606+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54003830 con 0x7ffa68141da0 2024-04-25T10:11:51.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.606+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa680ae1b0 con 0x7ffa68141da0 2024-04-25T10:11:51.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.606+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa680ae770 con 0x7ffa68141da0 2024-04-25T10:11:51.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.608+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa680ccae0 con 0x7ffa68141da0 2024-04-25T10:11:51.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.608+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa540039d0 con 0x7ffa68141da0 2024-04-25T10:11:51.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.609+0000 7ffa64c00640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4806b4d0 0x7ffa4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.609+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffa5408a770 con 0x7ffa68141da0 2024-04-25T10:11:51.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.610+0000 7ffa6d200640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4806b4d0 0x7ffa4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.611+0000 7ffa6d200640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4806b4d0 0x7ffa4806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c033000 tx=0x7ffa5c009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.614+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa54059390 con 0x7ffa68141da0 2024-04-25T10:11:51.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.854+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6813d090 con 0x7ffa68141da0 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.855+0000 7ffa64c00640 1 -- 172.21.15.5:0/470053353 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4936 (secure 0 0 0) 0x7ffa54058c90 con 0x7ffa68141da0 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:51.387279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6835/3788578651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3788578651},{"type":"v1","addr":"172.21.15.5:6835","nonce":3788578651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:51.860 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 86 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4806b4d0 msgr2=0x7ffa4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4806b4d0 0x7ffa4806d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c033000 tx=0x7ffa5c009000 comp rx=0 tx=0).stop 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 msgr2=0x7ffa680ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa680ad3e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400d600 tx=0x7ffa5400dac0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 shutdown_connections 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa4806b4d0 0x7ffa4806d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa6815d340 0x7ffa681e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68141da0 0x7ffa680ad3e0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 --2- 172.21.15.5:0/470053353 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa6806e640 0x7ffa6813cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 >> 172.21.15.5:0/470053353 conn(0x7ffa68066f20 msgr2=0x7ffa6815cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 shutdown_connections 2024-04-25T10:11:51.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:51.858+0000 7ffa6e600640 1 -- 172.21.15.5:0/470053353 wait complete. 2024-04-25T10:11:52.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:10:59.580232+0000', 'modified': '2024-04-25T10:11:51.387279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8676': {'gid': 8676, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6835/3788578651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 3788578651}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3788578651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:52.285 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8676:up:replay 2024-04-25T10:11:53.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:53.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.507+0000 7f4448a00640 1 Processor -- start 2024-04-25T10:11:53.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.507+0000 7f4448a00640 1 -- start start 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4448a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4444154a10 0x7f444415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4448a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44440cdca0 0x7f44440b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4448a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4444070f90 0x7f4444071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4448a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44440b4890 con 0x7f4444070f90 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4448a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44440cd200 con 0x7f4444154a10 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4448a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444406e6f0 con 0x7f44440cdca0 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4443200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4444070f90 0x7f4444071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4443200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4444070f90 0x7f4444071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40536/0 (socket says 172.21.15.5:40536) 2024-04-25T10:11:53.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.509+0000 7f4443200640 1 -- 172.21.15.5:0/3493246022 learned_addr learned my addr 172.21.15.5:0/3493246022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4442800640 1 --2- 172.21.15.5:0/3493246022 >> v2:172.21.15.80:3300/0 conn(0x7f4444154a10 0x7f444415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4441e00640 1 --2- 172.21.15.5:0/3493246022 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44440cdca0 0x7f44440b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4443200640 1 -- 172.21.15.5:0/3493246022 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44440cdca0 msgr2=0x7f44440b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4443200640 1 --2- 172.21.15.5:0/3493246022 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44440cdca0 0x7f44440b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4443200640 1 -- 172.21.15.5:0/3493246022 >> v2:172.21.15.80:3300/0 conn(0x7f4444154a10 msgr2=0x7f444415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4443200640 1 --2- 172.21.15.5:0/3493246022 >> v2:172.21.15.80:3300/0 conn(0x7f4444154a10 0x7f444415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.510+0000 7f4443200640 1 -- 172.21.15.5:0/3493246022 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444406e9d0 con 0x7f4444070f90 2024-04-25T10:11:53.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.511+0000 7f4443200640 1 --2- 172.21.15.5:0/3493246022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f4444071420 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f44440ccb70 tx=0x7f443c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e5960f0e2a6be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.511+0000 7f4441400640 1 -- 172.21.15.5:0/3493246022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c036070 con 0x7f4444070f90 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.511+0000 7f4441400640 1 -- 172.21.15.5:0/3493246022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c031600 con 0x7f4444070f90 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.511+0000 7f4441400640 1 -- 172.21.15.5:0/3493246022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0319d0 con 0x7f4444070f90 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 -- 172.21.15.5:0/3493246022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 msgr2=0x7f4444071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 --2- 172.21.15.5:0/3493246022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f4444071420 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f44440ccb70 tx=0x7f443c02a610 comp rx=0 tx=0).stop 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 -- 172.21.15.5:0/3493246022 shutdown_connections 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 --2- 172.21.15.5:0/3493246022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f4444071420 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 --2- 172.21.15.5:0/3493246022 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f44440cdca0 0x7f44440b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 --2- 172.21.15.5:0/3493246022 >> v2:172.21.15.80:3300/0 conn(0x7f4444154a10 0x7f444415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 -- 172.21.15.5:0/3493246022 >> 172.21.15.5:0/3493246022 conn(0x7f4444066f20 msgr2=0x7f444406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 -- 172.21.15.5:0/3493246022 shutdown_connections 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.512+0000 7f4448a00640 1 -- 172.21.15.5:0/3493246022 wait complete. 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.514+0000 7f4448a00640 1 Processor -- start 2024-04-25T10:11:53.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.514+0000 7f4448a00640 1 -- start start 2024-04-25T10:11:53.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.515+0000 7f4448a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f444414f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.515+0000 7f4448a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44440cdca0 0x7f444414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.515+0000 7f4448a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4444154a10 0x7f4444141c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.515+0000 7f4448a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440cd780 con 0x7f4444070f90 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.515+0000 7f4448a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440cd4a0 con 0x7f44440cdca0 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.515+0000 7f4448a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444406cff0 con 0x7f4444154a10 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f444414f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f444414f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40538/0 (socket says 172.21.15.5:40538) 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 -- 172.21.15.5:0/1964916511 learned_addr learned my addr 172.21.15.5:0/1964916511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4441e00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44440cdca0 0x7f444414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4443200640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4444154a10 0x7f4444141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 -- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4444154a10 msgr2=0x7f4444141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4444154a10 0x7f4444141c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 -- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44440cdca0 msgr2=0x7f444414c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44440cdca0 0x7f444414c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 -- 172.21.15.5:0/1964916511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444414fb00 con 0x7f4444070f90 2024-04-25T10:11:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.516+0000 7f4442800640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f444414f500 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f443400d5f0 tx=0x7f443400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.517+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44340125f0 con 0x7f4444070f90 2024-04-25T10:11:53.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.517+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4434019070 con 0x7f4444070f90 2024-04-25T10:11:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.517+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44340129d0 con 0x7f4444070f90 2024-04-25T10:11:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.517+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4444150570 con 0x7f4444070f90 2024-04-25T10:11:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.517+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4444150b30 con 0x7f4444070f90 2024-04-25T10:11:53.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.518+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4444068f60 con 0x7f4444070f90 2024-04-25T10:11:53.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.520+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4434012b70 con 0x7f4444070f90 2024-04-25T10:11:53.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.521+0000 7f443aa00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f442406b500 0x7f442406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.521+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f443408a7e0 con 0x7f4444070f90 2024-04-25T10:11:53.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.522+0000 7f4441e00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f442406b500 0x7f442406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.523+0000 7f4441e00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f442406b500 0x7f442406d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f444414cd90 tx=0x7f4430008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.526+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4434059400 con 0x7f4444070f90 2024-04-25T10:11:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.766+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44440cce40 con 0x7f4444070f90 2024-04-25T10:11:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.767+0000 7f443aa00640 1 -- 172.21.15.5:0/1964916511 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f4434058d00 con 0x7f4444070f90 2024-04-25T10:11:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:53.772 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:52.401214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6835/3788578651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3788578651},{"type":"v1","addr":"172.21.15.5:6835","nonce":3788578651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 88 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f442406b500 msgr2=0x7f442406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f442406b500 0x7f442406d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f444414cd90 tx=0x7f4430008040 comp rx=0 tx=0).stop 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 msgr2=0x7f444414f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f444414f500 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f443400d5f0 tx=0x7f443400dab0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 shutdown_connections 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4444154a10 0x7f4444141c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f442406b500 0x7f442406d9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44440cdca0 0x7f444414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 --2- 172.21.15.5:0/1964916511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4444070f90 0x7f444414f500 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 >> 172.21.15.5:0/1964916511 conn(0x7f4444066f20 msgr2=0x7f44440aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 shutdown_connections 2024-04-25T10:11:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:53.770+0000 7f4448a00640 1 -- 172.21.15.5:0/1964916511 wait complete. 2024-04-25T10:11:54.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:10:59.580232+0000', 'modified': '2024-04-25T10:11:52.401214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8676': {'gid': 8676, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6835/3788578651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 3788578651}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3788578651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:54.197 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8676:up:rejoin 2024-04-25T10:11:55.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.417+0000 7fd23a000640 1 Processor -- start 2024-04-25T10:11:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.418+0000 7fd23a000640 1 -- start start 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.419+0000 7fd23a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd2340b3670 0x7fd2340cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.420+0000 7fd23a000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23406ba70 con 0x7fd23406b100 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.420+0000 7fd23a000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23406bd50 con 0x7fd2341448d0 2024-04-25T10:11:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.420+0000 7fd23a000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23406c030 con 0x7fd2340b3670 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.420+0000 7fd238c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd23406b100 0x7fd2340b2f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55578/0 (socket says 172.21.15.5:55578) 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.420+0000 7fd238c00640 1 -- 172.21.15.5:0/3246333682 learned_addr learned my addr 172.21.15.5:0/3246333682 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.420+0000 7fd233e00640 1 --2- 172.21.15.5:0/3246333682 >> v2:172.21.15.5:3300/0 conn(0x7fd2340b3670 0x7fd2340cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.421+0000 7fd233e00640 1 -- 172.21.15.5:0/3246333682 >> v1:172.21.15.80:6789/0 conn(0x7fd2341448d0 legacy=0x7fd2340ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.421+0000 7fd233e00640 1 -- 172.21.15.5:0/3246333682 >> v1:172.21.15.5:6789/0 conn(0x7fd23406b100 legacy=0x7fd2340b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.421+0000 7fd233e00640 1 -- 172.21.15.5:0/3246333682 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23406c310 con 0x7fd2340b3670 2024-04-25T10:11:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.422+0000 7fd233e00640 1 --2- 172.21.15.5:0/3246333682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd2340cdef0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097a0 tx=0x7fd22002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c692d1c487d6101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.424+0000 7fd233400640 1 -- 172.21.15.5:0/3246333682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220037070 con 0x7fd2340b3670 2024-04-25T10:11:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.424+0000 7fd233400640 1 -- 172.21.15.5:0/3246333682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd220032440 con 0x7fd2340b3670 2024-04-25T10:11:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.424+0000 7fd233400640 1 -- 172.21.15.5:0/3246333682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220032810 con 0x7fd2340b3670 2024-04-25T10:11:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.424+0000 7fd23a000640 1 -- 172.21.15.5:0/3246333682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 msgr2=0x7fd2340cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.424+0000 7fd23a000640 1 --2- 172.21.15.5:0/3246333682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd2340cdef0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097a0 tx=0x7fd22002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.425+0000 7fd23a000640 1 -- 172.21.15.5:0/3246333682 shutdown_connections 2024-04-25T10:11:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.425+0000 7fd23a000640 1 --2- 172.21.15.5:0/3246333682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd2340cdef0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.425+0000 7fd23a000640 1 -- 172.21.15.5:0/3246333682 >> 172.21.15.5:0/3246333682 conn(0x7fd2340cc6a0 msgr2=0x7fd2340ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.425+0000 7fd23a000640 1 -- 172.21.15.5:0/3246333682 shutdown_connections 2024-04-25T10:11:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.425+0000 7fd23a000640 1 -- 172.21.15.5:0/3246333682 wait complete. 2024-04-25T10:11:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.426+0000 7fd23a000640 1 Processor -- start 2024-04-25T10:11:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.427+0000 7fd23a000640 1 -- start start 2024-04-25T10:11:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd23a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd23406b100 0x7fd23406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd23a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd234136a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd23a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2341448d0 0x7fd234138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd23a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23406b660 con 0x7fd2340b3670 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd23a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23406b4e0 con 0x7fd2341448d0 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd23a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23406b7e0 con 0x7fd23406b100 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd233e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd234136a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd233e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd234136a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40566/0 (socket says 172.21.15.5:40566) 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd233e00640 1 -- 172.21.15.5:0/3067766283 learned_addr learned my addr 172.21.15.5:0/3067766283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd239600640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2341448d0 0x7fd234138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.428+0000 7fd238c00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd23406b100 0x7fd23406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd233e00640 1 -- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd23406b100 msgr2=0x7fd23406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd233e00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd23406b100 0x7fd23406e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd233e00640 1 -- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2341448d0 msgr2=0x7fd234138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd233e00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2341448d0 0x7fd234138730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd233e00640 1 -- 172.21.15.5:0/3067766283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd234135c70 con 0x7fd2340b3670 2024-04-25T10:11:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd238c00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd23406b100 0x7fd23406e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd233e00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd234136a30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fd220000c00 tx=0x7fd2200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220037040 con 0x7fd2340b3670 2024-04-25T10:11:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd220042070 con 0x7fd2340b3670 2024-04-25T10:11:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220030a70 con 0x7fd2340b3670 2024-04-25T10:11:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.429+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd234135f50 con 0x7fd2340b3670 2024-04-25T10:11:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.430+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd234136510 con 0x7fd2340b3670 2024-04-25T10:11:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.431+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2340b29c0 con 0x7fd2340b3670 2024-04-25T10:11:55.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.431+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd220030c10 con 0x7fd2340b3670 2024-04-25T10:11:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.432+0000 7fd230c00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd21406b4d0 0x7fd21406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.432+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd2200aaef0 con 0x7fd2340b3670 2024-04-25T10:11:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.433+0000 7fd238c00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd21406b4d0 0x7fd21406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:55.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.434+0000 7fd238c00640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd21406b4d0 0x7fd21406d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd228000f60 tx=0x7fd228009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:55.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.437+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd220079b10 con 0x7fd2340b3670 2024-04-25T10:11:55.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.677+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd234068f60 con 0x7fd2340b3670 2024-04-25T10:11:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.678+0000 7fd230c00640 1 -- 172.21.15.5:0/3067766283 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4936 (secure 0 0 0) 0x7fd220079410 con 0x7fd2340b3670 2024-04-25T10:11:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:55.683 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:11:53.412776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/3788578651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3788578651},{"type":"v1","addr":"172.21.15.5:6835","nonce":3788578651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:55.683 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 89 2024-04-25T10:11:55.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd21406b4d0 msgr2=0x7fd21406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd21406b4d0 0x7fd21406d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd228000f60 tx=0x7fd228009040 comp rx=0 tx=0).stop 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 msgr2=0x7fd234136a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd234136a30 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fd220000c00 tx=0x7fd2200046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 shutdown_connections 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd21406b4d0 0x7fd21406d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2341448d0 0x7fd234138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2340b3670 0x7fd234136a30 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 --2- 172.21.15.5:0/3067766283 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd23406b100 0x7fd23406e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 >> 172.21.15.5:0/3067766283 conn(0x7fd2340cc6a0 msgr2=0x7fd2340af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 shutdown_connections 2024-04-25T10:11:55.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:55.681+0000 7fd23a000640 1 -- 172.21.15.5:0/3067766283 wait complete. 2024-04-25T10:11:55.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:56.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:10:59.580232+0000', 'modified': '2024-04-25T10:11:53.412776+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8676': {'gid': 8676, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6835/3788578651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 3788578651}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 3788578651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:56.107 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:56.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:11:56.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.324+0000 7fb62cc00640 1 Processor -- start 2024-04-25T10:11:56.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.324+0000 7fb62cc00640 1 -- start start 2024-04-25T10:11:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.326+0000 7fb62cc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb628141da0 0x7fb62806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.326+0000 7fb62cc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb628071d30 con 0x7fb62815d510 2024-04-25T10:11:56.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.326+0000 7fb62cc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62806d620 con 0x7fb62806e640 2024-04-25T10:11:56.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.326+0000 7fb62cc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb628142180 con 0x7fb628141da0 2024-04-25T10:11:56.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.326+0000 7fb627e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb62815d510 0x7fb6280cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:55592/0 (socket says 172.21.15.5:55592) 2024-04-25T10:11:56.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.326+0000 7fb627e00640 1 -- 172.21.15.5:0/198896330 learned_addr learned my addr 172.21.15.5:0/198896330 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb627400640 1 --2- 172.21.15.5:0/198896330 >> v2:172.21.15.80:3300/0 conn(0x7fb628141da0 0x7fb62806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906399948 0 0) 0x7fb628071d30 con 0x7fb62815d510 2024-04-25T10:11:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb60c003520 con 0x7fb62815d510 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701030926 0 0) 0x7fb60c003520 con 0x7fb62815d510 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 >> v2:172.21.15.80:3300/0 conn(0x7fb628141da0 msgr2=0x7fb62806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 --2- 172.21.15.5:0/198896330 >> v2:172.21.15.80:3300/0 conn(0x7fb628141da0 0x7fb62806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 >> v1:172.21.15.80:6789/0 conn(0x7fb62806e640 legacy=0x7fb62815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62806d900 con 0x7fb62815d510 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb620001670 con 0x7fb62815d510 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 >> v1:172.21.15.5:6789/0 conn(0x7fb62815d510 legacy=0x7fb6280cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60c0039b0 0x7fb60c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.327+0000 7fb626000640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb60c004800 0x7fb60c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb626000640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb60c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb628142180 con 0x7fb60c0051c0 2024-04-25T10:11:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c00c060 con 0x7fb60c0039b0 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c00c260 con 0x7fb60c004800 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627e00640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb60c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627e00640 1 -- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb60c004800 msgr2=0x7fb60c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627e00640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb60c004800 0x7fb60c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627e00640 1 -- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60c0039b0 msgr2=0x7fb60c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627400640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60c0039b0 0x7fb60c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627e00640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60c0039b0 0x7fb60c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627e00640 1 -- 172.21.15.5:0/198896330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c003520 con 0x7fb60c0051c0 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.328+0000 7fb627400640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60c0039b0 0x7fb60c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.329+0000 7fb627e00640 1 -- 172.21.15.5:0/198896330 reap_dead start 2024-04-25T10:11:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.329+0000 7fb627e00640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb60c00b9c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fb6280b2480 tx=0x7fb60c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f14f8bf175afbed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620007d10 con 0x7fb60c0051c0 2024-04-25T10:11:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620005ad0 con 0x7fb60c0051c0 2024-04-25T10:11:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb626000640 1 -- 172.21.15.5:0/198896330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620007070 con 0x7fb60c0051c0 2024-04-25T10:11:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 -- 172.21.15.5:0/198896330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 msgr2=0x7fb60c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb60c00b9c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fb6280b2480 tx=0x7fb60c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 -- 172.21.15.5:0/198896330 shutdown_connections 2024-04-25T10:11:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 --2- 172.21.15.5:0/198896330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb60c00b9c0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 -- 172.21.15.5:0/198896330 >> 172.21.15.5:0/198896330 conn(0x7fb628066f20 msgr2=0x7fb62806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 -- 172.21.15.5:0/198896330 shutdown_connections 2024-04-25T10:11:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.330+0000 7fb62cc00640 1 -- 172.21.15.5:0/198896330 wait complete. 2024-04-25T10:11:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.332+0000 7fb62cc00640 1 Processor -- start 2024-04-25T10:11:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.332+0000 7fb62cc00640 1 -- start start 2024-04-25T10:11:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb62cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb6280a86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb62cc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6280a8c00 0x7fb6281e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb62cc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6281e5640 0x7fb628149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb62cc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62806e280 con 0x7fb60c0051c0 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb62cc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62806e100 con 0x7fb6280a8c00 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb62cc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62806e400 con 0x7fb6281e5640 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.333+0000 7fb627400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb6280a86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb6280a86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40594/0 (socket says 172.21.15.5:40594) 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 -- 172.21.15.5:0/3629656387 learned_addr learned my addr 172.21.15.5:0/3629656387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 -- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6281e5640 msgr2=0x7fb628149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6281e5640 0x7fb628149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 -- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6280a8c00 msgr2=0x7fb6281e5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6280a8c00 0x7fb6281e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 -- 172.21.15.5:0/3629656387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6281e5ac0 con 0x7fb60c0051c0 2024-04-25T10:11:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb627400640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb6280a86c0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb61800f350 tx=0x7fb61800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.334+0000 7fb61f400640 1 -- 172.21.15.5:0/3629656387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618003b90 con 0x7fb60c0051c0 2024-04-25T10:11:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.335+0000 7fb61f400640 1 -- 172.21.15.5:0/3629656387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618003d30 con 0x7fb60c0051c0 2024-04-25T10:11:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.335+0000 7fb61f400640 1 -- 172.21.15.5:0/3629656387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618004220 con 0x7fb60c0051c0 2024-04-25T10:11:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.335+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6281e5d50 con 0x7fb60c0051c0 2024-04-25T10:11:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.335+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6281e6310 con 0x7fb60c0051c0 2024-04-25T10:11:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.337+0000 7fb61f400640 1 -- 172.21.15.5:0/3629656387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb61801e020 con 0x7fb60c0051c0 2024-04-25T10:11:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.338+0000 7fb61f400640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb60406b550 0x7fb60406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.338+0000 7fb61f400640 1 -- 172.21.15.5:0/3629656387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb61808abc0 con 0x7fb60c0051c0 2024-04-25T10:11:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.339+0000 7fb630f6c740 1 Processor -- start 2024-04-25T10:11:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.339+0000 7fb630f6c740 1 -- start start 2024-04-25T10:11:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.339+0000 7fb626a00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb60406b550 0x7fb60406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 0x5650b69a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a2eb0 0x5650b697af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69c37a0 0x5650b69792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb630f6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650b68de620 con 0x5650b697ab30 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650b6887ca0 con 0x5650b69a2eb0 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650b68da6c0 con 0x5650b69c37a0 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb627400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 0x5650b69a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb627400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 0x5650b69a2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40596/0 (socket says 172.21.15.5:40596) 2024-04-25T10:11:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb627400640 1 -- 172.21.15.5:0/1568662930 learned_addr learned my addr 172.21.15.5:0/1568662930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.340+0000 7fb626a00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb60406b550 0x7fb60406da10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb62813ffc0 tx=0x7fb6100079b0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb626a00640 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a2eb0 0x5650b697af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627e00640 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69c37a0 0x5650b69792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627400640 1 -- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69c37a0 msgr2=0x5650b69792f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627400640 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69c37a0 0x5650b69792f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627400640 1 -- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a2eb0 msgr2=0x5650b697af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627400640 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a2eb0 0x5650b697af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627400640 1 -- 172.21.15.5:0/1568662930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650b688ef40 con 0x5650b697ab30 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb626a00640 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a2eb0 0x5650b697af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb627400640 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 0x5650b69a2970 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fb618055530 tx=0x7fb618059210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb244f578d3ed61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb617400640 1 -- 172.21.15.5:0/1568662930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618087070 con 0x5650b697ab30 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb617400640 1 -- 172.21.15.5:0/1568662930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618057170 con 0x5650b697ab30 2024-04-25T10:11:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.341+0000 7fb617400640 1 -- 172.21.15.5:0/1568662930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618055740 con 0x5650b697ab30 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 -- 172.21.15.5:0/1568662930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 msgr2=0x5650b69a2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 0x5650b69a2970 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fb618055530 tx=0x7fb618059210 comp rx=0 tx=0).stop 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 -- 172.21.15.5:0/1568662930 shutdown_connections 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69c37a0 0x5650b69792f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a2eb0 0x5650b697af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 --2- 172.21.15.5:0/1568662930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b697ab30 0x5650b69a2970 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 -- 172.21.15.5:0/1568662930 >> 172.21.15.5:0/1568662930 conn(0x5650b6977d50 msgr2=0x5650b69c0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 -- 172.21.15.5:0/1568662930 shutdown_connections 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.342+0000 7fb630f6c740 1 -- 172.21.15.5:0/1568662930 wait complete. 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.343+0000 7fb630f6c740 1 -- start start 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.343+0000 7fb630f6c740 1 Processor -- start 2024-04-25T10:11:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b699f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b699d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650b69a24e0 con 0x5650b69c37a0 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650b687d1f0 con 0x5650b69a6370 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650b69a1ed0 con 0x5650b69a2eb0 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb630f6c740 10 client.0 _ll_register_callbacks cb 0x5650b689aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb626a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b699d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb626a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b699d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40600/0 (socket says 172.21.15.5:40600) 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.344+0000 7fb626a00640 1 -- 172.21.15.5:0/404237314 learned_addr learned my addr 172.21.15.5:0/404237314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb627400640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b699f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb627e00640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb626a00640 1 -- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 msgr2=0x5650b699f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb626a00640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b699f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb626a00640 1 -- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 msgr2=0x5650b69a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb626a00640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb626a00640 1 -- 172.21.15.5:0/404237314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650b689cc50 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb627e00640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb626a00640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b699d650 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fb610014e10 tx=0x7fb61000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d82680f831d98b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb60be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb60be00640 1 -- 172.21.15.5:0/404237314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610024060 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.345+0000 7fb60be00640 1 -- 172.21.15.5:0/404237314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb610007460 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb618057170 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb60be00640 1 -- 172.21.15.5:0/404237314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb610014e40 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 10 client.8748 fetch_fsmap learned FSMap version 89 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5650b65f32d0 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb60be00640 1 -- 172.21.15.5:0/404237314 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618057170 con 0x5650b69c37a0 2024-04-25T10:11:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb60be00640 1 -- 172.21.15.5:0/404237314 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb6100067d0 con 0x5650b69c37a0 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 10 client.8748 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 10 client.8748 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 20 client.8748 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b69a43e0 0x5650b69a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 4 client.8748 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.346+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650b68ebb60 con 0x5650b69a43e0 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.347+0000 7fb627400640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b69a43e0 0x5650b69a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.347+0000 7fb627400640 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b69a43e0 0x5650b69a47e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15453a5c12e04a10 server_cookie=3a4cd2e6aa21eae0 in_seq=0 out_seq=0 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.347+0000 7fb60be00640 10 client.8748 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb60be00640 1 -- 172.21.15.5:0/404237314 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650b68ebb60 con 0x5650b69a43e0 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb60be00640 10 client.8748 handle_command_reply: tid=0 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 client.8748 shutdown 2024-04-25T10:11:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 msgr2=0x5650b699d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b699d650 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fb610014e10 tx=0x7fb61000c7b0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 shutdown_connections 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b69a43e0 0x5650b69a47e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b699d650 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 --2- 172.21.15.5:0/404237314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b699f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 >> 172.21.15.5:0/404237314 conn(0x5650b697ab30 msgr2=0x5650b69c0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.348+0000 7fb60be00640 0 client.8748 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.349+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 shutdown_connections 2024-04-25T10:11:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.349+0000 7fb630f6c740 1 -- 172.21.15.5:0/404237314 wait complete. 2024-04-25T10:11:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.349+0000 7fb630f6c740 20 client.8748 trim_cache size 0 max 16384 2024-04-25T10:11:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.366+0000 7fb630f6c740 1 Processor -- start 2024-04-25T10:11:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.366+0000 7fb630f6c740 1 -- start start 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b6979c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b69a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb630f6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281ffdd0 con 0x5650b69c37a0 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281ff1d0 con 0x5650b69a6370 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6281ff7d0 con 0x5650b69a2eb0 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.367+0000 7fb627e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b69a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b69a5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40602/0 (socket says 172.21.15.5:40602) 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 -- 172.21.15.5:0/713333067 learned_addr learned my addr 172.21.15.5:0/713333067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627400640 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b6979c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb626a00640 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 -- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 msgr2=0x5650b6979c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b6979c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 -- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 msgr2=0x5650b69a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 -- 172.21.15.5:0/713333067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650b66bd6b0 con 0x5650b69c37a0 2024-04-25T10:11:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627400640 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b6979c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.368+0000 7fb627e00640 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b69a5a20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb62002ad80 tx=0x7fb620007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d365ed32455cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb616a00640 1 -- 172.21.15.5:0/713333067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620042070 con 0x5650b69c37a0 2024-04-25T10:11:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb616a00640 1 -- 172.21.15.5:0/713333067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620007d10 con 0x5650b69c37a0 2024-04-25T10:11:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb616a00640 1 -- 172.21.15.5:0/713333067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200015b0 con 0x5650b69c37a0 2024-04-25T10:11:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 -- 172.21.15.5:0/713333067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 msgr2=0x5650b69a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b69a5a20 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb62002ad80 tx=0x7fb620007000 comp rx=0 tx=0).stop 2024-04-25T10:11:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 -- 172.21.15.5:0/713333067 shutdown_connections 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69c37a0 0x5650b69a5a20 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69a6370 0x5650b69a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 --2- 172.21.15.5:0/713333067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a2eb0 0x5650b6979c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.369+0000 7fb630f6c740 1 -- 172.21.15.5:0/713333067 >> 172.21.15.5:0/713333067 conn(0x5650b697ab30 msgr2=0x5650b69a47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.370+0000 7fb630f6c740 1 -- 172.21.15.5:0/713333067 shutdown_connections 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.370+0000 7fb630f6c740 1 -- 172.21.15.5:0/713333067 wait complete. 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.370+0000 7fb630f6c740 1 -- start start 2024-04-25T10:11:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.371+0000 7fb630f6c740 1 Processor -- start 2024-04-25T10:11:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 0x5650b6a2b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a6370 0x5650b699f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69c37a0 0x5650b69a4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650b69a53e0 con 0x5650b69a2eb0 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650b69a3c20 con 0x5650b69c37a0 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650b69a8ad0 con 0x5650b69a6370 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb630f6c740 10 client.0 _ll_register_callbacks cb 0x7fb610014ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 0x5650b6a2b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 0x5650b6a2b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40606/0 (socket says 172.21.15.5:40606) 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 -- 172.21.15.5:0/2506759604 learned_addr learned my addr 172.21.15.5:0/2506759604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627e00640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69c37a0 0x5650b69a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb626a00640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a6370 0x5650b699f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 -- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a6370 msgr2=0x5650b699f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a6370 0x5650b699f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 -- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69c37a0 msgr2=0x5650b69a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69c37a0 0x5650b69a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.372+0000 7fb627400640 1 -- 172.21.15.5:0/2506759604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650b69d3f40 con 0x5650b69a2eb0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb627400640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 0x5650b6a2b260 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb618059060 tx=0x7fb61805c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ba59f86bfd9f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb60be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb60be00640 1 -- 172.21.15.5:0/2506759604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61807c060 con 0x5650b69a2eb0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb60be00640 1 -- 172.21.15.5:0/2506759604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb618015de0 con 0x5650b69a2eb0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb620007d10 con 0x5650b69a2eb0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb60be00640 1 -- 172.21.15.5:0/2506759604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb618082050 con 0x5650b69a2eb0 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb630f6c740 10 client.8754 fetch_fsmap learned FSMap version 89 2024-04-25T10:11:56.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5650b699c100 con 0x5650b69a2eb0 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.373+0000 7fb60be00640 1 -- 172.21.15.5:0/2506759604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb620007d10 con 0x5650b69a2eb0 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb60be00640 1 -- 172.21.15.5:0/2506759604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb618055740 con 0x5650b69a2eb0 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb630f6c740 10 client.8754 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb630f6c740 10 client.8754 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb630f6c740 20 client.8754 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb630f6c740 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b699b720 0x5650b699bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb630f6c740 4 client.8754 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650b68e5eb0 con 0x5650b699b720 2024-04-25T10:11:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.374+0000 7fb626a00640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b699b720 0x5650b699bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb626a00640 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b699b720 0x5650b699bb00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fa3be6d4fd8d60 server_cookie=b8b7ed6083a5ca0f in_seq=0 out_seq=0 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb60be00640 10 client.8754 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb60be00640 1 -- 172.21.15.5:0/2506759604 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5650b68e5eb0 con 0x5650b699b720 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb60be00640 10 client.8754 handle_command_reply: tid=0 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 client.8754 shutdown 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 msgr2=0x5650b6a2b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 0x5650b6a2b260 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb618059060 tx=0x7fb61805c070 comp rx=0 tx=0).stop 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 shutdown_connections 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x5650b699b720 0x5650b699bb00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5650b69c37a0 0x5650b69a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5650b69a6370 0x5650b699f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 --2- 172.21.15.5:0/2506759604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650b69a2eb0 0x5650b6a2b260 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb60be00640 0 client.8754 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 >> 172.21.15.5:0/2506759604 conn(0x5650b697ab30 msgr2=0x5650b699fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 shutdown_connections 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.375+0000 7fb630f6c740 1 -- 172.21.15.5:0/2506759604 wait complete. 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb630f6c740 20 client.8754 trim_cache size 0 max 16384 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 8676, 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:11:56.384 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.9797020879999998, 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 89, 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 108, 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 108, 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 5.9981519240000001 2024-04-25T10:11:56.385 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb60406b550 msgr2=0x7fb60406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb60406b550 0x7fb60406da10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb62813ffc0 tx=0x7fb6100079b0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 msgr2=0x7fb6280a86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb6280a86c0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb61800f350 tx=0x7fb61800f810 comp rx=0 tx=0).stop 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 shutdown_connections 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb6281e5640 0x7fb628149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb60406b550 0x7fb60406da10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6280a8c00 0x7fb6281e5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 --2- 172.21.15.5:0/3629656387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb60c0051c0 0x7fb6280a86c0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 >> 172.21.15.5:0/3629656387 conn(0x7fb628066f20 msgr2=0x7fb628140420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 shutdown_connections 2024-04-25T10:11:56.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.376+0000 7fb62cc00640 1 -- 172.21.15.5:0/3629656387 wait complete. 2024-04-25T10:11:56.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:56.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:11:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.587+0000 7fc09a800640 1 Processor -- start 2024-04-25T10:11:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.587+0000 7fc09a800640 1 -- start start 2024-04-25T10:11:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.588+0000 7fc09a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940d0260 0x7fc09415fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.588+0000 7fc09a800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc0940b2110 0x7fc0940cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.588+0000 7fc09a800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09415bd80 con 0x7fc09415b9a0 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.588+0000 7fc09a800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0941603c0 con 0x7fc0940b2110 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.588+0000 7fc09a800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940b28b0 con 0x7fc0940d0260 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc0940b2110 0x7fc0940cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc0940b2110 0x7fc0940cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40608/0 (socket says 172.21.15.5:40608) 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 -- 172.21.15.5:0/1676680297 learned_addr learned my addr 172.21.15.5:0/1676680297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc092a00640 1 --2- 172.21.15.5:0/1676680297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940d0260 0x7fc09415fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 -- 172.21.15.5:0/1676680297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940d0260 msgr2=0x7fc09415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 --2- 172.21.15.5:0/1676680297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940d0260 0x7fc09415fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 -- 172.21.15.5:0/1676680297 >> v1:172.21.15.80:6789/0 conn(0x7fc09415b9a0 legacy=0x7fc0940cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.589+0000 7fc093e00640 1 -- 172.21.15.5:0/1676680297 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940b2b90 con 0x7fc0940b2110 2024-04-25T10:11:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.590+0000 7fc093e00640 1 --2- 172.21.15.5:0/1676680297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc0940cca40 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009e50 tx=0x7fc08c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867e776697aa4ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.591+0000 7fc092000640 1 -- 172.21.15.5:0/1676680297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c036070 con 0x7fc0940b2110 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.591+0000 7fc092000640 1 -- 172.21.15.5:0/1676680297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c031600 con 0x7fc0940b2110 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.591+0000 7fc092000640 1 -- 172.21.15.5:0/1676680297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0319d0 con 0x7fc0940b2110 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 -- 172.21.15.5:0/1676680297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 msgr2=0x7fc0940cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 --2- 172.21.15.5:0/1676680297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc0940cca40 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009e50 tx=0x7fc08c02a750 comp rx=0 tx=0).stop 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 -- 172.21.15.5:0/1676680297 shutdown_connections 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 --2- 172.21.15.5:0/1676680297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc0940cca40 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 --2- 172.21.15.5:0/1676680297 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0940d0260 0x7fc09415fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 -- 172.21.15.5:0/1676680297 >> 172.21.15.5:0/1676680297 conn(0x7fc0940d1700 msgr2=0x7fc0940d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 -- 172.21.15.5:0/1676680297 shutdown_connections 2024-04-25T10:11:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.592+0000 7fc09a800640 1 -- 172.21.15.5:0/1676680297 wait complete. 2024-04-25T10:11:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.594+0000 7fc09a800640 1 Processor -- start 2024-04-25T10:11:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.594+0000 7fc09a800640 1 -- start start 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc09a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc094150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc09a800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0940d0260 0x7fc09406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc09a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09415b9a0 0x7fc0940af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc09a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940b2580 con 0x7fc0940b2110 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc09a800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cc240 con 0x7fc0940d0260 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc09a800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940ccf80 con 0x7fc09415b9a0 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc093400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc094150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc093400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc094150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40622/0 (socket says 172.21.15.5:40622) 2024-04-25T10:11:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.595+0000 7fc093400640 1 -- 172.21.15.5:0/2646346476 learned_addr learned my addr 172.21.15.5:0/2646346476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093e00640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09415b9a0 0x7fc0940af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093400640 1 -- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09415b9a0 msgr2=0x7fc0940af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc092a00640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0940d0260 0x7fc09406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093400640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09415b9a0 0x7fc0940af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093400640 1 -- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0940d0260 msgr2=0x7fc09406d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093400640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0940d0260 0x7fc09406d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093400640 1 -- 172.21.15.5:0/2646346476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940c6d00 con 0x7fc0940b2110 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc092a00640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0940d0260 0x7fc09406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc093400640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc094150060 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fc08400b3f0 tx=0x7fc08400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc08b400640 1 -- 172.21.15.5:0/2646346476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084015020 con 0x7fc0940b2110 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc08b400640 1 -- 172.21.15.5:0/2646346476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084011830 con 0x7fc0940b2110 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc08b400640 1 -- 172.21.15.5:0/2646346476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084011c20 con 0x7fc0940b2110 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.596+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0940c6fe0 con 0x7fc0940b2110 2024-04-25T10:11:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.597+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09415c1c0 con 0x7fc0940b2110 2024-04-25T10:11:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.599+0000 7fc08b400640 1 -- 172.21.15.5:0/2646346476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc084018020 con 0x7fc0940b2110 2024-04-25T10:11:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.600+0000 7fc08b400640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.600+0000 7fc08b400640 1 -- 172.21.15.5:0/2646346476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc08408a180 con 0x7fc0940b2110 2024-04-25T10:11:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.600+0000 7fc092a00640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.602+0000 7fc092a00640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07006b4d0 0x7fc07006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc0940b7540 tx=0x7fc07c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.602+0000 7fc09d61a740 1 Processor -- start 2024-04-25T10:11:56.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.603+0000 7fc09d61a740 1 -- start start 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a06280c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a062829f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a0627e1e80 0x55a062828100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc09d61a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a062744980 con 0x55a06280c1a0 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0626edde0 con 0x55a062829a40 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a062740a20 con 0x55a0627e1e80 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc093400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a06280c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc093400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a06280c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40628/0 (socket says 172.21.15.5:40628) 2024-04-25T10:11:56.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.604+0000 7fc093400640 1 -- 172.21.15.5:0/1900807748 learned_addr learned my addr 172.21.15.5:0/1900807748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc092a00640 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a062829f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093e00640 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a0627e1e80 0x55a062828100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093400640 1 -- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a0627e1e80 msgr2=0x55a062828100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093400640 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a0627e1e80 0x55a062828100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093400640 1 -- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 msgr2=0x55a062829f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093400640 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a062829f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093400640 1 -- 172.21.15.5:0/1900807748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a06270be30 con 0x55a06280c1a0 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc093400640 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a06280c580 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc0840607f0 tx=0x7fc084017410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a448d01b3f8cae82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc083e00640 1 -- 172.21.15.5:0/1900807748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084070030 con 0x55a06280c1a0 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.605+0000 7fc083e00640 1 -- 172.21.15.5:0/1900807748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084011dc0 con 0x55a06280c1a0 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.606+0000 7fc083e00640 1 -- 172.21.15.5:0/1900807748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084054c80 con 0x55a06280c1a0 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.606+0000 7fc09d61a740 1 -- 172.21.15.5:0/1900807748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 msgr2=0x55a06280c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.606+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a06280c580 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc0840607f0 tx=0x7fc084017410 comp rx=0 tx=0).stop 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 -- 172.21.15.5:0/1900807748 shutdown_connections 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a0627e1e80 0x55a062828100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a062829f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1900807748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a06280c580 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 -- 172.21.15.5:0/1900807748 >> 172.21.15.5:0/1900807748 conn(0x55a0627de7b0 msgr2=0x55a062826fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 -- 172.21.15.5:0/1900807748 shutdown_connections 2024-04-25T10:11:56.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.607+0000 7fc09d61a740 1 -- 172.21.15.5:0/1900807748 wait complete. 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.608+0000 7fc09d61a740 1 -- start start 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.608+0000 7fc09d61a740 1 Processor -- start 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a06280c1a0 0x55a06280a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 0x55a0627db320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280ed70 0x55a0627db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0627e2c20 con 0x55a062829a40 2024-04-25T10:11:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0626e33c0 con 0x55a06280c1a0 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0627dc6e0 con 0x55a06280ed70 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc09d61a740 10 client.0 _ll_register_callbacks cb 0x55a062458c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc092a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 0x55a0627db320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc092a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 0x55a0627db320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40640/0 (socket says 172.21.15.5:40640) 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc092a00640 1 -- 172.21.15.5:0/1950113644 learned_addr learned my addr 172.21.15.5:0/1950113644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc093400640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a06280c1a0 0x55a06280a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.609+0000 7fc093e00640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280ed70 0x55a0627db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc092a00640 1 -- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280ed70 msgr2=0x55a0627db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc092a00640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280ed70 0x55a0627db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc092a00640 1 -- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a06280c1a0 msgr2=0x55a06280a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc092a00640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a06280c1a0 0x55a06280a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc092a00640 1 -- 172.21.15.5:0/1950113644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0625217e0 con 0x55a062829a40 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc093400640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a06280c1a0 0x55a06280a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc092a00640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 0x55a0627db320 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0126c0 tx=0x7fc07c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20da585eba568bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc080c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc080c00640 1 -- 172.21.15.5:0/1950113644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c023020 con 0x55a062829a40 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc080c00640 1 -- 172.21.15.5:0/1950113644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc07c01f620 con 0x55a062829a40 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.610+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc084011dc0 con 0x55a062829a40 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc080c00640 1 -- 172.21.15.5:0/1950113644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c02d050 con 0x55a062829a40 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 10 client.8766 fetch_fsmap learned FSMap version 89 2024-04-25T10:11:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55a062581f70 con 0x55a062829a40 2024-04-25T10:11:56.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc080c00640 1 -- 172.21.15.5:0/1950113644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084011dc0 con 0x55a062829a40 2024-04-25T10:11:56.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc080c00640 1 -- 172.21.15.5:0/1950113644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc07c030020 con 0x55a062829a40 2024-04-25T10:11:56.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 10 client.8766 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:11:56.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 10 client.8766 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:56.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 20 client.8766 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a062837c30 0x55a062808610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 4 client.8766 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.611+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a062751ec0 con 0x55a062837c30 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.612+0000 7fc093400640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a062837c30 0x55a062808610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.612+0000 7fc093400640 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a062837c30 0x55a062808610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd8984a6a8d8ab9 server_cookie=95d2429de6e080e5 in_seq=0 out_seq=0 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.612+0000 7fc080c00640 10 client.8766 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc080c00640 1 -- 172.21.15.5:0/1950113644 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a062751ec0 con 0x55a062837c30 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc080c00640 10 client.8766 handle_command_reply: tid=0 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 client.8766 shutdown 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 msgr2=0x55a0627db320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 0x55a0627db320 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0126c0 tx=0x7fc07c004730 comp rx=0 tx=0).stop 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 shutdown_connections 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a062837c30 0x55a062808610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280ed70 0x55a0627db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a062829a40 0x55a0627db320 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1950113644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a06280c1a0 0x55a06280a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 >> 172.21.15.5:0/1950113644 conn(0x55a0627e1e80 msgr2=0x55a06282a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc080c00640 0 client.8766 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.613+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 shutdown_connections 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.614+0000 7fc09d61a740 1 -- 172.21.15.5:0/1950113644 wait complete. 2024-04-25T10:11:56.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.614+0000 7fc09d61a740 20 client.8766 trim_cache size 0 max 16384 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.616+0000 7f941d600640 1 Processor -- start 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.616+0000 7f941d600640 1 -- start start 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.617+0000 7f941d600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 0x7f94180ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.617+0000 7f941d600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f94180b3740 0x7f94180cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.617+0000 7f941d600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941815ab40 con 0x7f941806d220 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.617+0000 7f941d600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180cf1c0 con 0x7f94180b3740 2024-04-25T10:11:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.617+0000 7f941d600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180cc790 con 0x7f941815a760 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.618+0000 7f9416a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 0x7f94180ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.618+0000 7f9416a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 0x7f94180ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:34122/0 (socket says 172.21.15.5:34122) 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.618+0000 7f9416a00640 1 -- 172.21.15.5:0/3966229705 learned_addr learned my addr 172.21.15.5:0/3966229705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.618+0000 7f9417400640 1 --2- 172.21.15.5:0/3966229705 >> v2:172.21.15.80:3300/0 conn(0x7f94180b3740 0x7f94180cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.619+0000 7f9416a00640 1 -- 172.21.15.5:0/3966229705 >> v2:172.21.15.80:3300/0 conn(0x7f94180b3740 msgr2=0x7f94180cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.619+0000 7f9416a00640 1 --2- 172.21.15.5:0/3966229705 >> v2:172.21.15.80:3300/0 conn(0x7f94180b3740 0x7f94180cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.619+0000 7f9416a00640 1 -- 172.21.15.5:0/3966229705 >> v1:172.21.15.80:6789/0 conn(0x7f941806d220 legacy=0x7f941815e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.619+0000 7f9416a00640 1 -- 172.21.15.5:0/3966229705 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180cca70 con 0x7f941815a760 2024-04-25T10:11:56.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.620+0000 7f9416a00640 1 --2- 172.21.15.5:0/3966229705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 0x7f94180ce740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f940800bb10 tx=0x7f940802c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=238cb773488f954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f9415600640 1 -- 172.21.15.5:0/3966229705 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408031710 con 0x7f941815a760 2024-04-25T10:11:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f9415600640 1 -- 172.21.15.5:0/3966229705 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94080318b0 con 0x7f941815a760 2024-04-25T10:11:56.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f9415600640 1 -- 172.21.15.5:0/3966229705 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408031c80 con 0x7f941815a760 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 -- 172.21.15.5:0/3966229705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 msgr2=0x7f94180ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 --2- 172.21.15.5:0/3966229705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 0x7f94180ce740 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f940800bb10 tx=0x7f940802c7a0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 -- 172.21.15.5:0/3966229705 shutdown_connections 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 --2- 172.21.15.5:0/3966229705 >> v2:172.21.15.80:3300/0 conn(0x7f94180b3740 0x7f94180cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 --2- 172.21.15.5:0/3966229705 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941815a760 0x7f94180ce740 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 -- 172.21.15.5:0/3966229705 >> 172.21.15.5:0/3966229705 conn(0x7f94180b13a0 msgr2=0x7f94180b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 -- 172.21.15.5:0/3966229705 shutdown_connections 2024-04-25T10:11:56.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.621+0000 7f941d600640 1 -- 172.21.15.5:0/3966229705 wait complete. 2024-04-25T10:11:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.623+0000 7f941d600640 1 Processor -- start 2024-04-25T10:11:56.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.623+0000 7f941d600640 1 -- start start 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.624+0000 7f941d600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941806d220 0x7f94181f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.624+0000 7f941d600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94180b3740 0x7f94181f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.624+0000 7f941d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f941815a760 0x7f941815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.624+0000 7f941d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941806d770 con 0x7f941815a760 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.624+0000 7f941d600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180ccd50 con 0x7f94180b3740 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.624+0000 7f941d600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941806f320 con 0x7f941806d220 2024-04-25T10:11:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f941815a760 0x7f941815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f941815a760 0x7f941815b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40654/0 (socket says 172.21.15.5:40654) 2024-04-25T10:11:56.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 -- 172.21.15.5:0/1791813214 learned_addr learned my addr 172.21.15.5:0/1791813214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9416a00640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941806d220 0x7f94181f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9416000640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94180b3740 0x7f94181f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 -- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941806d220 msgr2=0x7f94181f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f941806d220 0x7f94181f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 -- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94180b3740 msgr2=0x7f94181f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94180b3740 0x7f94181f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 -- 172.21.15.5:0/1791813214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94181401e0 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.625+0000 7f9417400640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f941815a760 0x7f941815b8d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9410009e30 tx=0x7f941000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.626+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410018020 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.626+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410014480 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.626+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410014870 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.626+0000 7f941d600640 1 -- 172.21.15.5:0/1791813214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94181404c0 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.626+0000 7f941d600640 1 -- 172.21.15.5:0/1791813214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941813fbb0 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.627+0000 7f941d600640 1 -- 172.21.15.5:0/1791813214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f9418068f60 con 0x7f941815a760 2024-04-25T10:11:56.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.628+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9410005d10 con 0x7f941815a760 2024-04-25T10:11:56.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.629+0000 7f940ea00640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f93f806b4d0 0x7f93f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.629+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f941008bc80 con 0x7f941815a760 2024-04-25T10:11:56.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.629+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+516 (secure 0 0 0) 0x7f9410022030 con 0x7f941815a760 2024-04-25T10:11:56.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.630+0000 7f9416a00640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f93f806b4d0 0x7f93f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.630+0000 7f941d600640 1 -- 172.21.15.5:0/1791813214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f9418072430 con 0x7f941815a760 2024-04-25T10:11:56.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.631+0000 7f9416a00640 1 --2- 172.21.15.5:0/1791813214 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f93f806b4d0 0x7f93f806d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9408036000 tx=0x7f9408031000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.632+0000 7fc09d61a740 1 Processor -- start 2024-04-25T10:11:56.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.632+0000 7fc09d61a740 1 -- start start 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280c1a0 0x55a0627e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 0x55a0627e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a06280d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc09d61a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc094200870 con 0x55a06280ed70 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09415d670 con 0x55a062829a40 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09406c700 con 0x55a06280c1a0 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 0x55a0627e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 0x55a0627e27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40656/0 (socket says 172.21.15.5:40656) 2024-04-25T10:11:56.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 -- 172.21.15.5:0/1579468004 learned_addr learned my addr 172.21.15.5:0/1579468004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc093400640 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280c1a0 0x55a0627e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 -- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280c1a0 msgr2=0x55a0627e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280c1a0 0x55a0627e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 -- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 msgr2=0x55a06280d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.633+0000 7fc092a00640 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a06280d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.634+0000 7fc092a00640 1 -- 172.21.15.5:0/1579468004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a062419170 con 0x55a06280ed70 2024-04-25T10:11:56.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.634+0000 7fc092a00640 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 0x55a0627e27a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0047b0 tx=0x7fc07c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a544e34f063dd88d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.634+0000 7fc083400640 1 -- 172.21.15.5:0/1579468004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c023050 con 0x55a06280ed70 2024-04-25T10:11:56.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.634+0000 7fc083400640 1 -- 172.21.15.5:0/1579468004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c01a040 con 0x55a06280ed70 2024-04-25T10:11:56.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.634+0000 7fc083400640 1 -- 172.21.15.5:0/1579468004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c019070 con 0x55a06280ed70 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 -- 172.21.15.5:0/1579468004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 msgr2=0x55a0627e27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 0x55a0627e27a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0047b0 tx=0x7fc07c01e000 comp rx=0 tx=0).stop 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 -- 172.21.15.5:0/1579468004 shutdown_connections 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062829a40 0x55a06280d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280ed70 0x55a0627e27a0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1579468004 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a06280c1a0 0x55a0627e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 -- 172.21.15.5:0/1579468004 >> 172.21.15.5:0/1579468004 conn(0x55a0627e1e80 msgr2=0x55a0627d77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 -- 172.21.15.5:0/1579468004 shutdown_connections 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.635+0000 7fc09d61a740 1 -- 172.21.15.5:0/1579468004 wait complete. 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.636+0000 7fc09d61a740 1 -- start start 2024-04-25T10:11:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.636+0000 7fc09d61a740 1 Processor -- start 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a062805fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a062829a40 0x55a0627dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062890d10 0x55a0627e0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09415d070 con 0x55a06280c1a0 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a06280e010 con 0x55a062890d10 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a06280d330 con 0x55a062829a40 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc09d61a740 10 client.0 _ll_register_callbacks cb 0x55a0627dcbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc093400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a062805fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc093400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a062805fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40670/0 (socket says 172.21.15.5:40670) 2024-04-25T10:11:56.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.637+0000 7fc093400640 1 -- 172.21.15.5:0/1176725240 learned_addr learned my addr 172.21.15.5:0/1176725240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc092a00640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a062829a40 0x55a0627dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093400640 1 -- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a062829a40 msgr2=0x55a0627dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093e00640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062890d10 0x55a0627e0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093400640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a062829a40 0x55a0627dfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093400640 1 -- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062890d10 msgr2=0x55a0627e0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093400640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062890d10 0x55a0627e0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093400640 1 -- 172.21.15.5:0/1176725240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a06280d580 con 0x55a06280c1a0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc093400640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a062805fe0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc0840607f0 tx=0x7fc084009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444a33d435f9c3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc080c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc080c00640 1 -- 172.21.15.5:0/1176725240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084075e60 con 0x55a06280c1a0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc080c00640 1 -- 172.21.15.5:0/1176725240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc0840566b0 con 0x55a06280c1a0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.638+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc07c01a040 con 0x55a06280c1a0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc080c00640 1 -- 172.21.15.5:0/1176725240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc084070040 con 0x55a06280c1a0 2024-04-25T10:11:56.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 10 client.8775 fetch_fsmap learned FSMap version 89 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55a0628066b0 con 0x55a06280c1a0 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc080c00640 1 -- 172.21.15.5:0/1176725240 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c01a040 con 0x55a06280c1a0 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc080c00640 1 -- 172.21.15.5:0/1176725240 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc084054c80 con 0x55a06280c1a0 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 10 client.8775 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 10 client.8775 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 20 client.8775 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a0627e2260 0x55a0627e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 4 client.8775 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55a06274c210 con 0x55a0627e2260 2024-04-25T10:11:56.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.639+0000 7fc092a00640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a0627e2260 0x55a0627e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.640+0000 7fc092a00640 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a0627e2260 0x55a0627e2670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26b8f335d6227b7 server_cookie=26a62bca3471b155 in_seq=0 out_seq=0 2024-04-25T10:11:56.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.640+0000 7fc080c00640 10 client.8775 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc080c00640 1 -- 172.21.15.5:0/1176725240 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a06274c210 con 0x55a0627e2260 2024-04-25T10:11:56.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc080c00640 10 client.8775 handle_command_reply: tid=0 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 client.8775 shutdown 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 msgr2=0x55a062805fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a062805fe0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc0840607f0 tx=0x7fc084009300 comp rx=0 tx=0).stop 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 shutdown_connections 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55a0627e2260 0x55a0627e2670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a062890d10 0x55a0627e0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55a062829a40 0x55a0627dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 --2- 172.21.15.5:0/1176725240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a06280c1a0 0x55a062805fe0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 >> 172.21.15.5:0/1176725240 conn(0x55a0627e1e80 msgr2=0x55a06280a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc080c00640 0 client.8775 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.648 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:11:56.649 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:11:56.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 shutdown_connections 2024-04-25T10:11:56.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 1 -- 172.21.15.5:0/1176725240 wait complete. 2024-04-25T10:11:56.649 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:11:56.649 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:11:56.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.641+0000 7fc09d61a740 20 client.8775 trim_cache size 0 max 16384 2024-04-25T10:11:56.650 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:56.650 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-04-25T10:11:56.650 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d0f9dd1d-42dc-445e-9a98-1815475767c2", 2024-04-25T10:11:56.650 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-04-25T10:11:56.650 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:56.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07006b4d0 msgr2=0x7fc07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07006b4d0 0x7fc07006d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc0940b7540 tx=0x7fc07c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:56.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 msgr2=0x7fc094150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc094150060 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fc08400b3f0 tx=0x7fc08400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 shutdown_connections 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc09415b9a0 0x7fc0940af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0940d0260 0x7fc09406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 --2- 172.21.15.5:0/2646346476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0940b2110 0x7fc094150060 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 >> 172.21.15.5:0/2646346476 conn(0x7fc0940d1700 msgr2=0x7fc0940704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.642+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 shutdown_connections 2024-04-25T10:11:56.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.643+0000 7fc09a800640 1 -- 172.21.15.5:0/2646346476 wait complete. 2024-04-25T10:11:56.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:56.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.848+0000 7f1dff200640 1 Processor -- start 2024-04-25T10:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.848+0000 7f1dff200640 1 -- start start 2024-04-25T10:11:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.850+0000 7f1dff200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 0x7f1e00161d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.850+0000 7f1dff200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e000b33f0 con 0x7f1e000b37d0 2024-04-25T10:11:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.850+0000 7f1dff200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e000b27e0 con 0x7f1e00071000 2024-04-25T10:11:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.850+0000 7f1dff200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b2ac0 con 0x7f1e001619b0 2024-04-25T10:11:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.851+0000 7f1df5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1e000b37d0 0x7f1e00162ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:32894/0 (socket says 172.21.15.5:32894) 2024-04-25T10:11:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.851+0000 7f1df5400640 1 -- 172.21.15.5:0/1792245045 learned_addr learned my addr 172.21.15.5:0/1792245045 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:11:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.851+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190492561 0 0) 0x7f1e000b33f0 con 0x7f1e000b37d0 2024-04-25T10:11:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.851+0000 7f1dfde00640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 0x7f1e00161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.851+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de4003520 con 0x7f1e000b37d0 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850828165 0 0) 0x7f1de4003520 con 0x7f1e000b37d0 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 msgr2=0x7f1e00161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 0x7f1e00161d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 >> v1:172.21.15.80:6789/0 conn(0x7f1e00071000 legacy=0x7f1e0006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e000b2da0 con 0x7f1e000b37d0 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1de8001670 con 0x7f1e000b37d0 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 >> v1:172.21.15.5:6789/0 conn(0x7f1e000b37d0 legacy=0x7f1e00162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de40039e0 0x7f1de4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 0x7f1de4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de4004d40 0x7f1de40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b2ac0 con 0x7f1de40039e0 2024-04-25T10:11:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de400baa0 con 0x7f1de4004d40 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de400bca0 con 0x7f1e001619b0 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.852+0000 7f1dfde00640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de40039e0 0x7f1de4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1df5400640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 0x7f1de4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfe800640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de4004d40 0x7f1de40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfde00640 1 -- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 msgr2=0x7f1de4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfde00640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e001619b0 0x7f1de4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfde00640 1 -- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de4004d40 msgr2=0x7f1de40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfde00640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de4004d40 0x7f1de40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfde00640 1 -- 172.21.15.5:0/1792245045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4003520 con 0x7f1de40039e0 2024-04-25T10:11:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.853+0000 7f1dfe800640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de4004d40 0x7f1de40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.854+0000 7f1dfde00640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de40039e0 0x7f1de4003e00 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009a90 tx=0x7f1de4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f4ad9735cd62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.854+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0036070 con 0x7f1de40039e0 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.854+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df0031400 con 0x7f1de40039e0 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.854+0000 7f1dfd400640 1 -- 172.21.15.5:0/1792245045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0009690 con 0x7f1de40039e0 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dff200640 1 -- 172.21.15.5:0/1792245045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de40039e0 msgr2=0x7f1de4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dff200640 1 --2- 172.21.15.5:0/1792245045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de40039e0 0x7f1de4003e00 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009a90 tx=0x7f1de4005830 comp rx=0 tx=0).stop 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dfe800640 1 -- 172.21.15.5:0/1792245045 reap_dead start 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dff200640 1 -- 172.21.15.5:0/1792245045 shutdown_connections 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dff200640 1 -- 172.21.15.5:0/1792245045 >> 172.21.15.5:0/1792245045 conn(0x7f1e00066f20 msgr2=0x7f1e0006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dff200640 1 -- 172.21.15.5:0/1792245045 shutdown_connections 2024-04-25T10:11:56.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.855+0000 7f1dff200640 1 -- 172.21.15.5:0/1792245045 wait complete. 2024-04-25T10:11:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.857+0000 7f1dff200640 1 Processor -- start 2024-04-25T10:11:56.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.857+0000 7f1dff200640 1 -- start start 2024-04-25T10:11:56.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dff200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de40039e0 0x7f1e001590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dff200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e00152530 0x7f1e0015f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dff200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 0x7f1e0015fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dff200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b2420 con 0x7f1e0015f890 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dff200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0006e730 con 0x7f1de40039e0 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dff200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e000b25a0 con 0x7f1e00152530 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 0x7f1e0015fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 0x7f1e0015fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55872/0 (socket says 172.21.15.5:55872) 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.858+0000 7f1dfe800640 1 -- 172.21.15.5:0/361778371 learned_addr learned my addr 172.21.15.5:0/361778371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfde00640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de40039e0 0x7f1e001590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1df5400640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e00152530 0x7f1e0015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfe800640 1 -- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e00152530 msgr2=0x7f1e0015f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfe800640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e00152530 0x7f1e0015f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfe800640 1 -- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de40039e0 msgr2=0x7f1e001590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfe800640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de40039e0 0x7f1e001590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfe800640 1 -- 172.21.15.5:0/361778371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0015dff0 con 0x7f1e0015f890 2024-04-25T10:11:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.859+0000 7f1dfe800640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 0x7f1e0015fd10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000f80 tx=0x7f1df800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.860+0000 7f1df6a00640 1 -- 172.21.15.5:0/361778371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8006d50 con 0x7f1e0015f890 2024-04-25T10:11:56.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.860+0000 7f1df6a00640 1 -- 172.21.15.5:0/361778371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df8003970 con 0x7f1e0015f890 2024-04-25T10:11:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.860+0000 7f1df6a00640 1 -- 172.21.15.5:0/361778371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8003d60 con 0x7f1e0015f890 2024-04-25T10:11:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.860+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0015e2b0 con 0x7f1e0015f890 2024-04-25T10:11:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.860+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0015e870 con 0x7f1e0015f890 2024-04-25T10:11:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.862+0000 7f1df6a00640 1 -- 172.21.15.5:0/361778371 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1df801b020 con 0x7f1e0015f890 2024-04-25T10:11:56.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.863+0000 7f1df6a00640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1ddc06b500 0x7f1ddc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.863+0000 7f1df6a00640 1 -- 172.21.15.5:0/361778371 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1df8089fc0 con 0x7f1e0015f890 2024-04-25T10:11:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.864+0000 7f1dfde00640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1ddc06b500 0x7f1ddc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.864+0000 7f1e07e42740 1 Processor -- start 2024-04-25T10:11:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.864+0000 7f1e07e42740 1 -- start start 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1dfde00640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1ddc06b500 0x7f1ddc06d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1df0033000 tx=0x7f1df0008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534719d70 0x55853471bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853471a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534744830 0x558534744cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1e07e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853467dc70 con 0x558534756440 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558534626cf0 con 0x558534719d70 2024-04-25T10:11:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.865+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x558534679d10 con 0x558534744830 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853471a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853471a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55878/0 (socket says 172.21.15.5:55878) 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 -- 172.21.15.5:0/421562190 learned_addr learned my addr 172.21.15.5:0/421562190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1dfde00640 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534719d70 0x55853471bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 -- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534744830 msgr2=0x558534744cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534744830 0x558534744cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 -- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534719d70 msgr2=0x55853471bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534719d70 0x55853471bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 -- 172.21.15.5:0/421562190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55853445c7d0 con 0x558534756440 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.866+0000 7f1df5400640 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853471a150 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005950 tx=0x7f1de8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ea58a3bf1bc2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.867+0000 7f1deea00640 1 -- 172.21.15.5:0/421562190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8010960 con 0x558534756440 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.867+0000 7f1deea00640 1 -- 172.21.15.5:0/421562190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8019070 con 0x558534756440 2024-04-25T10:11:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.867+0000 7f1deea00640 1 -- 172.21.15.5:0/421562190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8010d20 con 0x558534756440 2024-04-25T10:11:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.867+0000 7f1e07e42740 1 -- 172.21.15.5:0/421562190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 msgr2=0x55853471a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.867+0000 7f1e07e42740 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853471a150 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005950 tx=0x7f1de8005e10 comp rx=0 tx=0).stop 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 -- 172.21.15.5:0/421562190 shutdown_connections 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534744830 0x558534744cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853471a150 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 --2- 172.21.15.5:0/421562190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534719d70 0x55853471bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 -- 172.21.15.5:0/421562190 >> 172.21.15.5:0/421562190 conn(0x5585346e1a30 msgr2=0x55853475f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 -- 172.21.15.5:0/421562190 shutdown_connections 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 -- 172.21.15.5:0/421562190 wait complete. 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.868+0000 7f1e07e42740 1 -- start start 2024-04-25T10:11:56.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.869+0000 7f1e07e42740 1 Processor -- start 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x558534745ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 0x558534746f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534756440 0x558534756840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853461c3a0 con 0x558534744830 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x558534742380 con 0x558534756440 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585347575d0 con 0x558534719d70 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1e07e42740 10 client.0 _ll_register_callbacks cb 0x558534639b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1df5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 0x558534746f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1df5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 0x558534746f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55880/0 (socket says 172.21.15.5:55880) 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1df5400640 1 -- 172.21.15.5:0/238277402 learned_addr learned my addr 172.21.15.5:0/238277402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1dfe800640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534756440 0x558534756840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1df5400640 1 -- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 msgr2=0x558534745ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1dfde00640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x558534745ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.870+0000 7f1df5400640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x558534745ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1df5400640 1 -- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534756440 msgr2=0x558534756840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1df5400640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534756440 0x558534756840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1df5400640 1 -- 172.21.15.5:0/238277402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585343923f0 con 0x558534744830 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1dfe800640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534756440 0x558534756840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1df5400640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 0x558534746f10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1de8016040 tx=0x7f1e000b29b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afb0889e1efc547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1de3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1de3400640 1 -- 172.21.15.5:0/238277402 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8001580 con 0x558534744830 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1de3400640 1 -- 172.21.15.5:0/238277402 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1de801f630 con 0x558534744830 2024-04-25T10:11:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1de8019070 con 0x558534744830 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1de3400640 1 -- 172.21.15.5:0/238277402 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1de8030050 con 0x558534744830 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1e07e42740 10 client.8787 fetch_fsmap learned FSMap version 89 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.871+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5585343520f0 con 0x558534744830 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1de3400640 1 -- 172.21.15.5:0/238277402 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8019070 con 0x558534744830 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1de3400640 1 -- 172.21.15.5:0/238277402 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1de8033020 con 0x558534744830 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1e07e42740 10 client.8787 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1e07e42740 10 client.8787 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1e07e42740 20 client.8787 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1e07e42740 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534756d80 0x5585347590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1e07e42740 4 client.8787 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55853468b1b0 con 0x558534756d80 2024-04-25T10:11:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.872+0000 7f1dfde00640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534756d80 0x5585347590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.873+0000 7f1dfde00640 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534756d80 0x5585347590c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0377a8b33c13eb6 server_cookie=739899ab80c0ece4 in_seq=0 out_seq=0 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.873+0000 7f1de3400640 10 client.8787 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1de3400640 1 -- 172.21.15.5:0/238277402 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55853468b1b0 con 0x558534756d80 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1de3400640 10 client.8787 handle_command_reply: tid=0 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 client.8787 shutdown 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 msgr2=0x558534746f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 0x558534746f10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1de8016040 tx=0x7f1e000b29b0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 shutdown_connections 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534756d80 0x5585347590c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534756440 0x558534756840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534744830 0x558534746f10 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 --2- 172.21.15.5:0/238277402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x558534745ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1de3400640 0 client.8787 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 >> 172.21.15.5:0/238277402 conn(0x5585346e1a30 msgr2=0x55853473f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 shutdown_connections 2024-04-25T10:11:56.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 1 -- 172.21.15.5:0/238277402 wait complete. 2024-04-25T10:11:56.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.874+0000 7f1e07e42740 20 client.8787 trim_cache size 0 max 16384 2024-04-25T10:11:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.891+0000 7f1e07e42740 1 Processor -- start 2024-04-25T10:11:56.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.891+0000 7f1e07e42740 1 -- start start 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 0x5585347483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x5585347460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534756440 0x5585347577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1e07e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00205080 con 0x558534719d70 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00204280 con 0x558534744830 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001544f0 con 0x558534756440 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1dfde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 0x5585347483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1dfde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 0x5585347483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55886/0 (socket says 172.21.15.5:55886) 2024-04-25T10:11:56.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.892+0000 7f1dfde00640 1 -- 172.21.15.5:0/3385236375 learned_addr learned my addr 172.21.15.5:0/3385236375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1df5400640 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x5585347460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1dfde00640 1 -- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534756440 msgr2=0x5585347577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1dfde00640 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534756440 0x5585347577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1dfde00640 1 -- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 msgr2=0x5585347460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1dfde00640 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x5585347460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1dfde00640 1 -- 172.21.15.5:0/3385236375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558534756bb0 con 0x558534719d70 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.893+0000 7f1dfde00640 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 0x5585347483c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009b80 tx=0x7f1df002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10243cf9b5ad9ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.894+0000 7f1dee000640 1 -- 172.21.15.5:0/3385236375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0040040 con 0x558534719d70 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.894+0000 7f1dee000640 1 -- 172.21.15.5:0/3385236375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df0051070 con 0x558534719d70 2024-04-25T10:11:56.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.894+0000 7f1dee000640 1 -- 172.21.15.5:0/3385236375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0036040 con 0x558534719d70 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.894+0000 7f1e07e42740 1 -- 172.21.15.5:0/3385236375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 msgr2=0x5585347483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 0x5585347483c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009b80 tx=0x7f1df002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 -- 172.21.15.5:0/3385236375 shutdown_connections 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534756440 0x5585347577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x5585347460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3385236375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534719d70 0x5585347483c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 -- 172.21.15.5:0/3385236375 >> 172.21.15.5:0/3385236375 conn(0x5585346e1a30 msgr2=0x558534710390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 -- 172.21.15.5:0/3385236375 shutdown_connections 2024-04-25T10:11:56.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.895+0000 7f1e07e42740 1 -- 172.21.15.5:0/3385236375 wait complete. 2024-04-25T10:11:56.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.896+0000 7f1e07e42740 1 -- start start 2024-04-25T10:11:56.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.896+0000 7f1e07e42740 1 Processor -- start 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x55853473d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x55853473dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853476dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558534782330 con 0x558534756440 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00203cb0 con 0x558534744830 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585347489a0 con 0x558534719d70 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1e07e42740 10 client.0 _ll_register_callbacks cb 0x7f1de8010b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1dfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853476dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1dfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853476dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55890/0 (socket says 172.21.15.5:55890) 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1dfe800640 1 -- 172.21.15.5:0/3754952846 learned_addr learned my addr 172.21.15.5:0/3754952846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.897+0000 7f1df5400640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x55853473dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfde00640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x55853473d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfe800640 1 -- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 msgr2=0x55853473d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfe800640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x55853473d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfe800640 1 -- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 msgr2=0x55853473dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfe800640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x55853473dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfe800640 1 -- 172.21.15.5:0/3754952846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558534715d40 con 0x558534756440 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1df5400640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x55853473dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1dfe800640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853476dea0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1df80583e0 tx=0x7f1df801cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cd88117c0b94ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1de3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1de3400640 1 -- 172.21.15.5:0/3754952846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df805c360 con 0x558534756440 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1de3400640 1 -- 172.21.15.5:0/3754952846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1df80606d0 con 0x558534756440 2024-04-25T10:11:56.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df0051070 con 0x558534756440 2024-04-25T10:11:56.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1de3400640 1 -- 172.21.15.5:0/3754952846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df8060b60 con 0x558534756440 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1e07e42740 10 client.8793 fetch_fsmap learned FSMap version 89 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.898+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x558534778dd0 con 0x558534756440 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1de3400640 1 -- 172.21.15.5:0/3754952846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df0051070 con 0x558534756440 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1de3400640 1 -- 172.21.15.5:0/3754952846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1df8081b60 con 0x558534756440 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1e07e42740 10 client.8793 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1e07e42740 10 client.8793 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1e07e42740 20 client.8793 populate_metadata read hostname 'smithi005' 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534713d60 0x55853476e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1e07e42740 4 client.8793 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558534685500 con 0x558534713d60 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.899+0000 7f1dfde00640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534713d60 0x55853476e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1dfde00640 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534713d60 0x55853476e3e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a97ff3f978df5d8 server_cookie=36558004753a09d5 in_seq=0 out_seq=0 2024-04-25T10:11:56.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1de3400640 10 client.8793 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1de3400640 1 -- 172.21.15.5:0/3754952846 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x558534685500 con 0x558534713d60 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1de3400640 10 client.8793 handle_command_reply: tid=0 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 client.8793 shutdown 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 msgr2=0x55853476dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853476dea0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1df80583e0 tx=0x7f1df801cf40 comp rx=0 tx=0).stop 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 shutdown_connections 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x558534713d60 0x55853476e3e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558534756440 0x55853476dea0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558534744830 0x55853473dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 --2- 172.21.15.5:0/3754952846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x558534719d70 0x55853473d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1de3400640 0 client.8793 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.900+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 >> 172.21.15.5:0/3754952846 conn(0x5585346e1a30 msgr2=0x558534757760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 shutdown_connections 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1e07e42740 1 -- 172.21.15.5:0/3754952846 wait complete. 2024-04-25T10:11:56.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1e07e42740 20 client.8793 trim_cache size 0 max 16384 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "d0f9dd1d-42dc-445e-9a98-1815475767c2": { 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d0f9dd1d-42dc-445e-9a98-1815475767c2", 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,repair" 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:11:56.909 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:11:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1ddc06b500 msgr2=0x7f1ddc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1dff200640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1ddc06b500 0x7f1ddc06d9c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1df0033000 tx=0x7f1df0008030 comp rx=0 tx=0).stop 2024-04-25T10:11:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 msgr2=0x7f1e0015fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.901+0000 7f1dff200640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 0x7f1e0015fd10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000f80 tx=0x7f1df800dc00 comp rx=0 tx=0).stop 2024-04-25T10:11:56.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 shutdown_connections 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e0015f890 0x7f1e0015fd10 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1e00152530 0x7f1e0015f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1ddc06b500 0x7f1ddc06d9c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 --2- 172.21.15.5:0/361778371 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1de40039e0 0x7f1e001590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 >> 172.21.15.5:0/361778371 conn(0x7f1e00066f20 msgr2=0x7f1e000af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 shutdown_connections 2024-04-25T10:11:56.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:56.902+0000 7f1dff200640 1 -- 172.21.15.5:0/361778371 wait complete. 2024-04-25T10:11:56.924 INFO:tasks.cephfs.filesystem:scrub status for tag:d0f9dd1d-42dc-445e-9a98-1815475767c2 - {'path': '/', 'tag': 'd0f9dd1d-42dc-445e-9a98-1815475767c2', 'options': 'recursive,repair'} 2024-04-25T10:11:57.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:11:57.439+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-25T10:11:56.641342+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f9410058240 con 0x7f941815a760 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: cluster: 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: id: 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: health: HEALTH_OK 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: services: 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: mgr: x(active, since 10m), standbys: y 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:11:57.445 INFO:teuthology.orchestra.run.smithi005.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: data: 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: volumes: 1/1 healthy 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: pgs: 97 active+clean 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: io: 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: client: 2.0 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.641342+0000 mds.c [INF] scrub queued for path: / 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.641346+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.641351+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.642234+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.642242+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.642609+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.642615+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T10:11:57.447 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:11:56.643495+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:12:01.464 INFO:teuthology.orchestra.run.smithi005.stdout:2024-04-25T10:12:00.380707+0000 mds.c [INF] scrub summary: idle 2024-04-25T10:12:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:01.459+0000 7f940ea00640 1 -- 172.21.15.5:0/1791813214 <== mon.0 v2:172.21.15.5:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-25T10:12:00.380707+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f941008c000 con 0x7f941815a760 2024-04-25T10:12:26.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:26.078 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:26.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:26.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:12:27.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.145+0000 7f56d3000640 1 Processor -- start 2024-04-25T10:12:27.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.146+0000 7f56d3000640 1 -- start start 2024-04-25T10:12:27.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.147+0000 7f56d3000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56cc0b48c0 0x7f56cc15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.147+0000 7f56d3000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.147+0000 7f56d3000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56cc1606e0 con 0x7f56cc0d3f90 2024-04-25T10:12:27.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.147+0000 7f56d3000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56cc1609c0 con 0x7f56cc0b48c0 2024-04-25T10:12:27.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.147+0000 7f56d3000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc160ca0 con 0x7f56cc161d90 2024-04-25T10:12:27.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.148+0000 7f56d1c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56cc0b48c0 0x7f56cc15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.148+0000 7f56d2600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.148+0000 7f56d1c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56cc0b48c0 0x7f56cc15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:36426/0 (socket says 172.21.15.5:36426) 2024-04-25T10:12:27.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.148+0000 7f56d1c00640 1 -- 172.21.15.5:0/1025667015 learned_addr learned my addr 172.21.15.5:0/1025667015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.149+0000 7f56d1c00640 1 -- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 msgr2=0x7f56cc0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.149+0000 7f56d1c00640 1 --2- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.149+0000 7f56d1c00640 1 -- 172.21.15.5:0/1025667015 >> v1:172.21.15.80:6789/0 conn(0x7f56cc0d3f90 legacy=0x7f56cc161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.149+0000 7f56d1c00640 1 -- 172.21.15.5:0/1025667015 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc160f80 con 0x7f56cc0b48c0 2024-04-25T10:12:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.150+0000 7f56d1c00640 1 --2- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc15fe60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009870 tx=0x7f56c002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ec1a4205b2cc13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.151+0000 7f56cbe00640 1 -- 172.21.15.5:0/1025667015 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037070 con 0x7f56cc0b48c0 2024-04-25T10:12:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.151+0000 7f56cbe00640 1 -- 172.21.15.5:0/1025667015 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0032440 con 0x7f56cc0b48c0 2024-04-25T10:12:27.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.151+0000 7f56cbe00640 1 -- 172.21.15.5:0/1025667015 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0032810 con 0x7f56cc0b48c0 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 -- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 msgr2=0x7f56cc15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 --2- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc15fe60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009870 tx=0x7f56c002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 -- 172.21.15.5:0/1025667015 shutdown_connections 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 --2- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 --2- 172.21.15.5:0/1025667015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc15fe60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 -- 172.21.15.5:0/1025667015 >> 172.21.15.5:0/1025667015 conn(0x7f56cc08af80 msgr2=0x7f56cc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 -- 172.21.15.5:0/1025667015 shutdown_connections 2024-04-25T10:12:27.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.152+0000 7f56d3000640 1 -- 172.21.15.5:0/1025667015 wait complete. 2024-04-25T10:12:27.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.154+0000 7f56d3000640 1 Processor -- start 2024-04-25T10:12:27.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.154+0000 7f56d3000640 1 -- start start 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.155+0000 7f56d3000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.155+0000 7f56d3000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 0x7f56cc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.155+0000 7f56d3000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.155+0000 7f56d3000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1603a0 con 0x7f56cc0d3f90 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.155+0000 7f56d3000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0b4ca0 con 0x7f56cc0b48c0 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.155+0000 7f56d3000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0b30b0 con 0x7f56cc161d90 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 0x7f56cc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 0x7f56cc15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33196/0 (socket says 172.21.15.5:33196) 2024-04-25T10:12:27.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 -- 172.21.15.5:0/3293048245 learned_addr learned my addr 172.21.15.5:0/3293048245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d2600640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1c00640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 -- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 msgr2=0x7f56cc153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 -- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 msgr2=0x7f56cc0b8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc0b8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.156+0000 7f56d1200640 1 -- 172.21.15.5:0/3293048245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc15c7b0 con 0x7f56cc0d3f90 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.157+0000 7f56d1200640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 0x7f56cc15d670 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f56b800c400 tx=0x7f56b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.157+0000 7f56c9600640 1 -- 172.21.15.5:0/3293048245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80052d0 con 0x7f56cc0d3f90 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.157+0000 7f56c9600640 1 -- 172.21.15.5:0/3293048245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80054b0 con 0x7f56cc0d3f90 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.157+0000 7f56c9600640 1 -- 172.21.15.5:0/3293048245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8006a60 con 0x7f56cc0d3f90 2024-04-25T10:12:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.157+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56cc15ca90 con 0x7f56cc0d3f90 2024-04-25T10:12:27.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.157+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56cc15d050 con 0x7f56cc0d3f90 2024-04-25T10:12:27.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.159+0000 7f56c9600640 1 -- 172.21.15.5:0/3293048245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56b8006c00 con 0x7f56cc0d3f90 2024-04-25T10:12:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.160+0000 7f56c9600640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ac06b4d0 0x7f56ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.161+0000 7f56c9600640 1 -- 172.21.15.5:0/3293048245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f56b8089980 con 0x7f56cc0d3f90 2024-04-25T10:12:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.161+0000 7f56d75dc740 1 Processor -- start 2024-04-25T10:12:27.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.161+0000 7f56d1c00640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ac06b4d0 0x7f56ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.161+0000 7f56d75dc740 1 -- start start 2024-04-25T10:12:27.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.162+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00d8bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.162+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00dc6440 0x55fe00d8a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.162+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00db4830 0x55fe00db4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.162+0000 7f56d75dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe00cedc70 con 0x55fe00d89d70 2024-04-25T10:12:27.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.162+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe00c96cf0 con 0x55fe00dc6440 2024-04-25T10:12:27.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.162+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe00ce9d10 con 0x55fe00db4830 2024-04-25T10:12:27.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00d8bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00d8bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33204/0 (socket says 172.21.15.5:33204) 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 -- 172.21.15.5:0/568225314 learned_addr learned my addr 172.21.15.5:0/568225314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ac06b4d0 0x7f56ac06d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f56c0034000 tx=0x7f56c002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d2600640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00db4830 0x55fe00db4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1200640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00dc6440 0x55fe00d8a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 -- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00db4830 msgr2=0x55fe00db4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00db4830 0x55fe00db4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 -- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00dc6440 msgr2=0x55fe00d8a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00dc6440 0x55fe00d8a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1c00640 1 -- 172.21.15.5:0/568225314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe00acc7d0 con 0x55fe00d89d70 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d1200640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00dc6440 0x55fe00d8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.163+0000 7f56d2600640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00db4830 0x55fe00db4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.164+0000 7f56d1c00640 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00d8bed0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59da611eb90a2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.164+0000 7f56be000640 1 -- 172.21.15.5:0/568225314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c004e070 con 0x55fe00d89d70 2024-04-25T10:12:27.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.164+0000 7f56be000640 1 -- 172.21.15.5:0/568225314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c003e660 con 0x55fe00d89d70 2024-04-25T10:12:27.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.164+0000 7f56be000640 1 -- 172.21.15.5:0/568225314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037040 con 0x55fe00d89d70 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 -- 172.21.15.5:0/568225314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 msgr2=0x55fe00d8bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00d8bed0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c0009240 comp rx=0 tx=0).stop 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 -- 172.21.15.5:0/568225314 shutdown_connections 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00db4830 0x55fe00db4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00dc6440 0x55fe00d8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 --2- 172.21.15.5:0/568225314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00d8bed0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 -- 172.21.15.5:0/568225314 >> 172.21.15.5:0/568225314 conn(0x55fe00d51a30 msgr2=0x55fe00dcf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 -- 172.21.15.5:0/568225314 shutdown_connections 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.165+0000 7f56d75dc740 1 -- 172.21.15.5:0/568225314 wait complete. 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.166+0000 7f56d75dc740 1 -- start start 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.166+0000 7f56d75dc740 1 Processor -- start 2024-04-25T10:12:27.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00dd1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00db4830 0x55fe00da1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00db6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe00c8c3a0 con 0x55fe00d89d70 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe00db2380 con 0x55fe00db4830 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe00db5020 con 0x55fe00dc6440 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d75dc740 10 client.0 _ll_register_callbacks cb 0x55fe00ca9b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00dd1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00dd1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33218/0 (socket says 172.21.15.5:33218) 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.167+0000 7f56d1c00640 1 -- 172.21.15.5:0/3954523099 learned_addr learned my addr 172.21.15.5:0/3954523099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1200640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00db4830 0x55fe00da1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1c00640 1 -- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 msgr2=0x55fe00db6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d2600640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00db6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1c00640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00db6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1c00640 1 -- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00db4830 msgr2=0x55fe00da1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1c00640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00db4830 0x55fe00da1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1c00640 1 -- 172.21.15.5:0/3954523099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe00a023f0 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1200640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00db4830 0x55fe00da1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d1c00640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00dd1370 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05e28a92a0a048b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56b2a00640 1 -- 172.21.15.5:0/3954523099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0037070 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56b2a00640 1 -- 172.21.15.5:0/3954523099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f56c003ed40 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.168+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56c003e660 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56b2a00640 1 -- 172.21.15.5:0/3954523099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56c005e050 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 10 client.8802 fetch_fsmap learned FSMap version 89 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55fe009c20f0 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56b2a00640 1 -- 172.21.15.5:0/3954523099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c003e660 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56b2a00640 1 -- 172.21.15.5:0/3954523099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56c0061020 con 0x55fe00d89d70 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 10 client.8802 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:12:27.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 10 client.8802 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 20 client.8802 populate_metadata read hostname 'smithi005' 2024-04-25T10:12:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dea0b0 0x55fe00dec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 4 client.8802 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.169+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe00cfb1b0 con 0x55fe00dea0b0 2024-04-25T10:12:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.170+0000 7f56d1200640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dea0b0 0x55fe00dec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.170+0000 7f56d1200640 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dea0b0 0x55fe00dec4a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321108e8b07c435c server_cookie=fd79ce4219c57cdd in_seq=0 out_seq=0 2024-04-25T10:12:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.170+0000 7f56b2a00640 10 client.8802 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:12:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56b2a00640 1 -- 172.21.15.5:0/3954523099 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe00cfb1b0 con 0x55fe00dea0b0 2024-04-25T10:12:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56b2a00640 10 client.8802 handle_command_reply: tid=0 2024-04-25T10:12:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 client.8802 shutdown 2024-04-25T10:12:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 msgr2=0x55fe00dd1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00dd1370 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f56c0000c00 tx=0x7f56c0004730 comp rx=0 tx=0).stop 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 shutdown_connections 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dea0b0 0x55fe00dec4a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00db6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00db4830 0x55fe00da1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 --2- 172.21.15.5:0/3954523099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00d89d70 0x55fe00dd1370 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56b2a00640 0 client.8802 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 >> 172.21.15.5:0/3954523099 conn(0x55fe00d51a30 msgr2=0x55fe00db2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 shutdown_connections 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 1 -- 172.21.15.5:0/3954523099 wait complete. 2024-04-25T10:12:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.171+0000 7f56d75dc740 20 client.8802 trim_cache size 0 max 16384 2024-04-25T10:12:27.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.188+0000 7f56d75dc740 1 Processor -- start 2024-04-25T10:12:27.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.188+0000 7f56d75dc740 1 -- start start 2024-04-25T10:12:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.189+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00dd24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.189+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.189+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00da16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.189+0000 7f56d75dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc200440 con 0x55fe00db4830 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.189+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc1ff6a0 con 0x55fe00d89d70 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.189+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0ccc60 con 0x55fe00dc6440 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33220/0 (socket says 172.21.15.5:33220) 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 -- 172.21.15.5:0/2320958059 learned_addr learned my addr 172.21.15.5:0/2320958059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 -- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 msgr2=0x55fe00da16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00da16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 -- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 msgr2=0x55fe00dd24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1c00640 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00dd24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00dd24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.190+0000 7f56d1200640 1 -- 172.21.15.5:0/2320958059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe00d809f0 con 0x55fe00db4830 2024-04-25T10:12:27.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.191+0000 7f56d1200640 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd29e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f56b800cfd0 tx=0x7f56b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95535b78d222f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.191+0000 7f56bd600640 1 -- 172.21.15.5:0/2320958059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8056110 con 0x55fe00db4830 2024-04-25T10:12:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.191+0000 7f56bd600640 1 -- 172.21.15.5:0/2320958059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b806f020 con 0x55fe00db4830 2024-04-25T10:12:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.191+0000 7f56bd600640 1 -- 172.21.15.5:0/2320958059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b805e070 con 0x55fe00db4830 2024-04-25T10:12:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.191+0000 7f56d75dc740 1 -- 172.21.15.5:0/2320958059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 msgr2=0x55fe00dd29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.191+0000 7f56d75dc740 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd29e0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f56b800cfd0 tx=0x7f56b8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 -- 172.21.15.5:0/2320958059 shutdown_connections 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00da16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd29e0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 --2- 172.21.15.5:0/2320958059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00dd24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 -- 172.21.15.5:0/2320958059 >> 172.21.15.5:0/2320958059 conn(0x55fe00d51a30 msgr2=0x55fe00d801b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 -- 172.21.15.5:0/2320958059 shutdown_connections 2024-04-25T10:12:27.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 -- 172.21.15.5:0/2320958059 wait complete. 2024-04-25T10:12:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.192+0000 7f56d75dc740 1 -- start start 2024-04-25T10:12:27.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.193+0000 7f56d75dc740 1 Processor -- start 2024-04-25T10:12:27.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.193+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00e39460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d75dc740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00dd29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d75dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe00dd0570 con 0x55fe00db4830 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56cc0cd260 con 0x55fe00d89d70 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d75dc740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe00d871c0 con 0x55fe00dc6440 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d75dc740 10 client.0 _ll_register_callbacks cb 0x55fe00d8a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33232/0 (socket says 172.21.15.5:33232) 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 -- 172.21.15.5:0/892925253 learned_addr learned my addr 172.21.15.5:0/892925253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1c00640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00e39460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d2600640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00dd29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 -- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 msgr2=0x55fe00dd29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00dd29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 -- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 msgr2=0x55fe00e39460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00e39460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.194+0000 7f56d1200640 1 -- 172.21.15.5:0/892925253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe00db3230 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56d1200640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd24a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f56b8001f00 tx=0x7f56b8001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbda103463565a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56b2a00640 1 -- 172.21.15.5:0/892925253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8057e10 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56b2a00640 1 -- 172.21.15.5:0/892925253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f56b8054480 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56b806f020 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56b2a00640 1 -- 172.21.15.5:0/892925253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56b807f050 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56d75dc740 10 client.8808 fetch_fsmap learned FSMap version 89 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.195+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55fe00d88740 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56b2a00640 1 -- 172.21.15.5:0/892925253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b806f020 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56b2a00640 1 -- 172.21.15.5:0/892925253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f56b8076ce0 con 0x55fe00db4830 2024-04-25T10:12:27.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d75dc740 10 client.8808 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d75dc740 10 client.8808 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d75dc740 20 client.8808 populate_metadata read hostname 'smithi005' 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d75dc740 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dd0d60 0x55fe00db0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d75dc740 4 client.8808 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 --> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe00cf5500 con 0x55fe00dd0d60 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.196+0000 7f56d1c00640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dd0d60 0x55fe00db0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d1c00640 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dd0d60 0x55fe00db0b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd82c39b31e8f3a0 server_cookie=27d7502cb2449e87 in_seq=0 out_seq=0 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56b2a00640 10 client.8808 ms_handle_connect on v2:172.21.15.5:6834/3788578651 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56b2a00640 1 -- 172.21.15.5:0/892925253 <== mds.0 v2:172.21.15.5:6834/3788578651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe00cf5500 con 0x55fe00dd0d60 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56b2a00640 10 client.8808 handle_command_reply: tid=0 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 client.8808 shutdown 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 msgr2=0x55fe00dd24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd24a0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f56b8001f00 tx=0x7f56b8001040 comp rx=0 tx=0).stop 2024-04-25T10:12:27.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 shutdown_connections 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:6834/3788578651,v1:172.21.15.5:6835/3788578651] conn(0x55fe00dd0d60 0x55fe00db0b50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55fe00dc6440 0x55fe00dd29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe00db4830 0x55fe00dd24a0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56b2a00640 0 client.8808 ms_handle_reset on v2:172.21.15.5:6834/3788578651 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 --2- 172.21.15.5:0/892925253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55fe00d89d70 0x55fe00e39460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 >> 172.21.15.5:0/892925253 conn(0x55fe00d51a30 msgr2=0x55fe00da1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 shutdown_connections 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.197+0000 7f56d75dc740 1 -- 172.21.15.5:0/892925253 wait complete. 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d75dc740 20 client.8808 trim_cache size 0 max 16384 2024-04-25T10:12:27.214 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:12:27.215 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-04-25T10:12:27.215 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-04-25T10:12:27.215 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ac06b4d0 msgr2=0x7f56ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ac06b4d0 0x7f56ac06d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f56c0034000 tx=0x7f56c002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 msgr2=0x7f56cc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 0x7f56cc15d670 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f56b800c400 tx=0x7f56b800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 shutdown_connections 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56cc161d90 0x7f56cc153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56cc0d3f90 0x7f56cc15d670 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ac06b4d0 0x7f56ac06d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 --2- 172.21.15.5:0/3293048245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56cc0b48c0 0x7f56cc0b8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 >> 172.21.15.5:0/3293048245 conn(0x7f56cc08af80 msgr2=0x7f56cc154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 shutdown_connections 2024-04-25T10:12:27.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.198+0000 7f56d3000640 1 -- 172.21.15.5:0/3293048245 wait complete. 2024-04-25T10:12:27.220 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:27.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.535+0000 7f5b32200640 1 Processor -- start 2024-04-25T10:12:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.535+0000 7f5b32200640 1 -- start start 2024-04-25T10:12:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.537+0000 7f5b32200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c06e980 0x7f5b2c070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.537+0000 7f5b32200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b2c0714d0 0x7f5b2c06cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.537+0000 7f5b32200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0cd2a0 con 0x7f5b2c1652f0 2024-04-25T10:12:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.537+0000 7f5b32200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0ce340 con 0x7f5b2c0714d0 2024-04-25T10:12:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.537+0000 7f5b32200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0cdeb0 con 0x7f5b2c06e980 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.537+0000 7f5b31800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b2c0714d0 0x7f5b2c06cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b31800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b2c0714d0 0x7f5b2c06cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:36472/0 (socket says 172.21.15.5:36472) 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b31800640 1 -- 172.21.15.5:0/699666183 learned_addr learned my addr 172.21.15.5:0/699666183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b2be00640 1 --2- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c06e980 0x7f5b2c070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b31800640 1 -- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c06e980 msgr2=0x7f5b2c070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b31800640 1 --2- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c06e980 0x7f5b2c070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b31800640 1 -- 172.21.15.5:0/699666183 >> v1:172.21.15.80:6789/0 conn(0x7f5b2c1652f0 legacy=0x7f5b2c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.538+0000 7f5b31800640 1 -- 172.21.15.5:0/699666183 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0cd580 con 0x7f5b2c0714d0 2024-04-25T10:12:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.540+0000 7f5b31800640 1 --2- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c06cff0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009e50 tx=0x7f5b2402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a358b6619cfb516d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.544+0000 7f5b2b400640 1 -- 172.21.15.5:0/699666183 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24036070 con 0x7f5b2c0714d0 2024-04-25T10:12:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.544+0000 7f5b2b400640 1 -- 172.21.15.5:0/699666183 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b24031600 con 0x7f5b2c0714d0 2024-04-25T10:12:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.544+0000 7f5b2b400640 1 -- 172.21.15.5:0/699666183 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b240319d0 con 0x7f5b2c0714d0 2024-04-25T10:12:27.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.544+0000 7f5b32200640 1 -- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0714d0 msgr2=0x7f5b2c06cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.544+0000 7f5b32200640 1 --2- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c06cff0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009e50 tx=0x7f5b2402a710 comp rx=0 tx=0).stop 2024-04-25T10:12:27.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.545+0000 7f5b32200640 1 -- 172.21.15.5:0/699666183 shutdown_connections 2024-04-25T10:12:27.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.545+0000 7f5b32200640 1 --2- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c06cff0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.545+0000 7f5b32200640 1 --2- 172.21.15.5:0/699666183 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c06e980 0x7f5b2c070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.545+0000 7f5b32200640 1 -- 172.21.15.5:0/699666183 >> 172.21.15.5:0/699666183 conn(0x7f5b2c066f20 msgr2=0x7f5b2c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.545+0000 7f5b32200640 1 -- 172.21.15.5:0/699666183 shutdown_connections 2024-04-25T10:12:27.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.545+0000 7f5b32200640 1 -- 172.21.15.5:0/699666183 wait complete. 2024-04-25T10:12:27.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.546+0000 7f5b32200640 1 Processor -- start 2024-04-25T10:12:27.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.547+0000 7f5b32200640 1 -- start start 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b32200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c06e980 0x7f5b2c1eda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b32200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b32200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c1652f0 0x7f5b2c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b32200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c06d530 con 0x7f5b2c0714d0 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b32200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0cdb40 con 0x7f5b2c06e980 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b32200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c06d6b0 con 0x7f5b2c1652f0 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33246/0 (socket says 172.21.15.5:33246) 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b2be00640 1 -- 172.21.15.5:0/4035290918 learned_addr learned my addr 172.21.15.5:0/4035290918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b30e00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c06e980 0x7f5b2c1eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b2be00640 1 -- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c1652f0 msgr2=0x7f5b2c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b2be00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c1652f0 0x7f5b2c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.548+0000 7f5b2be00640 1 -- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c06e980 msgr2=0x7f5b2c1eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b2be00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c06e980 0x7f5b2c1eda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b2be00640 1 -- 172.21.15.5:0/4035290918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c1ec680 con 0x7f5b2c0714d0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b2be00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c156c90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9b0 tx=0x7f5b1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18017070 con 0x7f5b2c0714d0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1800c9a0 con 0x7f5b2c0714d0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c1ec960 con 0x7f5b2c0714d0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800cd90 con 0x7f5b2c0714d0 2024-04-25T10:12:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.549+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c1ecf20 con 0x7f5b2c0714d0 2024-04-25T10:12:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.551+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c0cce40 con 0x7f5b2c0714d0 2024-04-25T10:12:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.552+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b1801a070 con 0x7f5b2c0714d0 2024-04-25T10:12:27.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.552+0000 7f5b28c00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.553+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5b1808a7a0 con 0x7f5b2c0714d0 2024-04-25T10:12:27.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.553+0000 7f5b30e00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.555+0000 7f5b30e00640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c1619c0 tx=0x7f5b20009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.558+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b18059460 con 0x7f5b2c0714d0 2024-04-25T10:12:27.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.795+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5b2c068670 con 0x7f5b2c0714d0 2024-04-25T10:12:27.801 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:12:27.797+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:12:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.801+0000 7f5b28c00640 1 -- 172.21.15.5:0/4035290918 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5b18058d60 con 0x7f5b2c0714d0 2024-04-25T10:12:27.805 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:27.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0c06b4d0 msgr2=0x7f5b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c1619c0 tx=0x7f5b20009040 comp rx=0 tx=0).stop 2024-04-25T10:12:27.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 msgr2=0x7f5b2c156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c156c90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9b0 tx=0x7f5b1800ee70 comp rx=0 tx=0).stop 2024-04-25T10:12:27.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 shutdown_connections 2024-04-25T10:12:27.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c1652f0 0x7f5b2c1571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0714d0 0x7f5b2c156c90 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 --2- 172.21.15.5:0/4035290918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c06e980 0x7f5b2c1eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.804+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 >> 172.21.15.5:0/4035290918 conn(0x7f5b2c066f20 msgr2=0x7f5b2c0aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.806+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 shutdown_connections 2024-04-25T10:12:27.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.806+0000 7f5b32200640 1 -- 172.21.15.5:0/4035290918 wait complete. 2024-04-25T10:12:27.834 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T10:12:27.840 INFO:tasks.ceph.mds.c.smithi005.stderr: -1150> 2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:12:27.840 INFO:tasks.ceph.mds.c.smithi005.stderr: -1147> 2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:12:27.840 INFO:tasks.ceph.mds.c.smithi005.stderr: -1099> 2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:12:27.840 INFO:tasks.ceph.mds.c.smithi005.stderr: -1096> 2024-04-25T10:11:56.641+0000 7f8697000640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:12:27.890 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:12:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.919+0000 7fe7ddc36280 1 -- start start 2024-04-25T10:12:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.919+0000 7fe7ddc36280 1 Processor -- start 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.920+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 0x564a54e7a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.920+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a54e0f800 0x564a54e7ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.920+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a54e0fc00 0x564a54e7b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.920+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a55be0600 con 0x564a54e0ec00 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.920+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a55be0480 con 0x564a54e0fc00 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.920+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a55be0300 con 0x564a54e0f800 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 0x564a54e7a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 0x564a54e7a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33286/0 (socket says 172.21.15.5:33286) 2024-04-25T10:12:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 -- 172.21.15.5:0/2690051560 learned_addr learned my addr 172.21.15.5:0/2690051560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dc600640 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a54e0fc00 0x564a54e7b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 -- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a54e0f800 msgr2=0x564a54e7ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a54e0f800 0x564a54e7ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 -- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a54e0fc00 msgr2=0x564a54e7b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a54e0fc00 0x564a54e7b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 -- 172.21.15.5:0/2690051560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a54e42780 con 0x564a54e0ec00 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dc600640 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a54e0fc00 0x564a54e7b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.921+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 0x564a54e7a580 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x564a55c21260 tx=0x564a55b783c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a16fd4d2e45f872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7d8a00640 1 -- 172.21.15.5:0/2690051560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564a54e34680 con 0x564a54e0ec00 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7d8a00640 1 -- 172.21.15.5:0/2690051560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564a54e3aa80 con 0x564a54e0ec00 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7d8a00640 1 -- 172.21.15.5:0/2690051560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564a54e34b60 con 0x564a54e0ec00 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a54e434a0 con 0x564a54e0ec00 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564a54e42000 con 0x564a54e0ec00 2024-04-25T10:12:27.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7ddc36280 4 waiting for MDS map... 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7d8a00640 1 -- 172.21.15.5:0/2690051560 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x564a55112600 con 0x564a54e0ec00 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7d8a00640 1 -- 172.21.15.5:0/2690051560 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x564a54e69180 con 0x564a54e0ec00 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7ddc36280 4 Got MDS map 90 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7ddc36280 10 main: main 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.922+0000 7fe7ddc36280 4 main: connecting to RADOS... 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.923+0000 7fe7ddc36280 1 Processor -- start 2024-04-25T10:12:27.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.923+0000 7fe7ddc36280 1 -- start start 2024-04-25T10:12:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 0x564a54e7b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a55be0180 con 0x564a55115400 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a54e70480 con 0x564a55115800 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a54e70900 con 0x564a55115000 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33288/0 (socket says 172.21.15.5:33288) 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.924+0000 7fe7dbc00640 1 -- 172.21.15.5:0/818130987 learned_addr learned my addr 172.21.15.5:0/818130987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7db200640 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dc600640 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 0x564a54e7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dbc00640 1 -- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 msgr2=0x564a54e7b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 0x564a54e7b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dbc00640 1 -- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 msgr2=0x564a54e7c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dbc00640 1 -- 172.21.15.5:0/818130987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a54e423c0 con 0x564a55115400 2024-04-25T10:12:27.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7db200640 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7bb80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x564a55bf4000 tx=0x564a55b9f2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937834762399eeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7d6200640 1 -- 172.21.15.5:0/818130987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564a54e34820 con 0x564a55115400 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7d6200640 1 -- 172.21.15.5:0/818130987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564a54e3ae00 con 0x564a55115400 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.925+0000 7fe7d6200640 1 -- 172.21.15.5:0/818130987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564a54e34680 con 0x564a55115400 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 -- 172.21.15.5:0/818130987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 msgr2=0x564a54e7bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7bb80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x564a55bf4000 tx=0x564a55b9f2c0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 -- 172.21.15.5:0/818130987 shutdown_connections 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7bb80 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/818130987 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 0x564a54e7b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 -- 172.21.15.5:0/818130987 >> 172.21.15.5:0/818130987 conn(0x564a55114c00 msgr2=0x564a54f12800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 -- 172.21.15.5:0/818130987 shutdown_connections 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.926+0000 7fe7ddc36280 1 -- 172.21.15.5:0/818130987 wait complete. 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.927+0000 7fe7ddc36280 1 Processor -- start 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.927+0000 7fe7ddc36280 1 -- start start 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 0x564a54e7d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7ddc36280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a55be1380 con 0x564a55115400 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a55be1500 con 0x564a55115800 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7ddc36280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a55be0180 con 0x564a55115000 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33294/0 (socket says 172.21.15.5:33294) 2024-04-25T10:12:27.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 -- 172.21.15.5:0/3165009336 learned_addr learned my addr 172.21.15.5:0/3165009336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7db200640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 -- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 msgr2=0x564a54e7d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a55115000 0x564a54e7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 -- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 msgr2=0x564a54e7c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a55115800 0x564a54e7c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.928+0000 7fe7dbc00640 1 -- 172.21.15.5:0/3165009336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a54e42780 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7dbc00640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7cc00 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x564a55bf4270 tx=0x564a5512ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7d4400640 1 -- 172.21.15.5:0/3165009336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564a54e34b60 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7d4400640 1 -- 172.21.15.5:0/3165009336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564a54e3ac40 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7d4400640 1 -- 172.21.15.5:0/3165009336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564a54e344e0 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564a54e434a0 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a54e42960 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.929+0000 7fe7ddc36280 4 main: resolving pool 15 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.931+0000 7fe7d4400640 1 -- 172.21.15.5:0/3165009336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x564a55106c00 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7d4400640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x564a55272000 0x564a54e7d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7d4400640 1 -- 172.21.15.5:0/3165009336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x564a55bf6600 con 0x564a55115400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7ddc36280 4 main: creating IoCtx.. 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x564a55272400 0x564a54e7c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x564a55272800 con 0x564a55272400 2024-04-25T10:12:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7dc600640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x564a55272000 0x564a54e7d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.932+0000 7fe7db200640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x564a55272400 0x564a54e7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.933+0000 7fe7dc600640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x564a55272000 0x564a54e7d700 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x564a54e9f9b0 tx=0x564a55b9e900 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.933+0000 7fe7db200640 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x564a55272400 0x564a54e7c100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7db200640 1 -- 172.21.15.5:0/3165009336 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x564a55b8c000 con 0x564a55272400 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "0": { 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "data": { 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 5, 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "inotable": { 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "projected_free": [ 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "start": 1099511627779, 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: "len": 1099511627773 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:27.940 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: "free": [ 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: "start": 1099511627779, 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: "len": 1099511627773 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: "result": 0 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:12:27.941 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x564a55272400 msgr2=0x564a54e7c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x564a55272400 0x564a54e7c100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x564a55272000 msgr2=0x564a54e7d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x564a55272000 0x564a54e7d700 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x564a54e9f9b0 tx=0x564a55b9e900 comp rx=0 tx=0).stop 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 msgr2=0x564a54e7cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/3165009336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a55115400 0x564a54e7cc00 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x564a55bf4270 tx=0x564a5512ec60 comp rx=0 tx=0).stop 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7db200640 1 -- 172.21.15.5:0/3165009336 reap_dead start 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 shutdown_connections 2024-04-25T10:12:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 >> 172.21.15.5:0/3165009336 conn(0x564a55114c00 msgr2=0x564a54f12800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 shutdown_connections 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/3165009336 wait complete. 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 msgr2=0x564a54e7a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.935+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 0x564a54e7a580 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x564a55c21260 tx=0x564a55b783c0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 shutdown_connections 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564a54e0fc00 0x564a54e7b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x564a54e0f800 0x564a54e7ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 --2- 172.21.15.5:0/2690051560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a54e0ec00 0x564a54e7a580 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 >> 172.21.15.5:0/2690051560 conn(0x564a54e0f000 msgr2=0x564a54f13000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 shutdown_connections 2024-04-25T10:12:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:27.936+0000 7fe7ddc36280 1 -- 172.21.15.5:0/2690051560 wait complete. 2024-04-25T10:12:27.945 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.111341, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T10:12:27.946 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:27.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:28.031 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:28.146 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:28.146 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:28.147 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:28.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:28.209 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:12:28.209 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:28.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:28.266 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:28.266 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:28.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:28.329 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:12:28.329 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:28.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:28.389 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.606+0000 7fd9a1c00640 1 Processor -- start 2024-04-25T10:12:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.607+0000 7fd9a1c00640 1 -- start start 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.608+0000 7fd9a1c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd99c0d07d0 0x7fd99c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.608+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.608+0000 7fd9a1c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c0cf410 con 0x7fd99c161200 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.608+0000 7fd9a1c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99c0b28b0 con 0x7fd99c0d07d0 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.608+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0cf6f0 con 0x7fd99c0d2380 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd99c0d07d0 0x7fd99c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd99c0d07d0 0x7fd99c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33300/0 (socket says 172.21.15.5:33300) 2024-04-25T10:12:28.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 -- 172.21.15.5:0/1156892223 learned_addr learned my addr 172.21.15.5:0/1156892223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd9a1200640 1 --2- 172.21.15.5:0/1156892223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 -- 172.21.15.5:0/1156892223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 msgr2=0x7fd99c0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 --2- 172.21.15.5:0/1156892223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c0b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 -- 172.21.15.5:0/1156892223 >> v1:172.21.15.80:6789/0 conn(0x7fd99c161200 legacy=0x7fd99c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.609+0000 7fd99b400640 1 -- 172.21.15.5:0/1156892223 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0d3fd0 con 0x7fd99c0d07d0 2024-04-25T10:12:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.610+0000 7fd99b400640 1 --2- 172.21.15.5:0/1156892223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c0d07d0 0x7fd99c0b4bb0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd98402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716297ed3a3fee0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd99aa00640 1 -- 172.21.15.5:0/1156892223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984037070 con 0x7fd99c0d07d0 2024-04-25T10:12:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd99aa00640 1 -- 172.21.15.5:0/1156892223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984032440 con 0x7fd99c0d07d0 2024-04-25T10:12:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd99aa00640 1 -- 172.21.15.5:0/1156892223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984032810 con 0x7fd99c0d07d0 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd9a1c00640 1 -- 172.21.15.5:0/1156892223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c0d07d0 msgr2=0x7fd99c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/1156892223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c0d07d0 0x7fd99c0b4bb0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fd984009870 tx=0x7fd98402a250 comp rx=0 tx=0).stop 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd9a1c00640 1 -- 172.21.15.5:0/1156892223 shutdown_connections 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/1156892223 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c0b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/1156892223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c0d07d0 0x7fd99c0b4bb0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.611+0000 7fd9a1c00640 1 -- 172.21.15.5:0/1156892223 >> 172.21.15.5:0/1156892223 conn(0x7fd99c08af80 msgr2=0x7fd99c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.612+0000 7fd9a1c00640 1 -- 172.21.15.5:0/1156892223 shutdown_connections 2024-04-25T10:12:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.612+0000 7fd9a1c00640 1 -- 172.21.15.5:0/1156892223 wait complete. 2024-04-25T10:12:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.613+0000 7fd9a1c00640 1 Processor -- start 2024-04-25T10:12:28.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.613+0000 7fd9a1c00640 1 -- start start 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.614+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd99c0d07d0 0x7fd99c1523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.614+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c1f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.614+0000 7fd9a1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 0x7fd99c1f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.614+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0b3640 con 0x7fd99c161200 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.614+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0b34c0 con 0x7fd99c0d07d0 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.614+0000 7fd9a1c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0b37c0 con 0x7fd99c0d2380 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 0x7fd99c1f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 0x7fd99c1f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33310/0 (socket says 172.21.15.5:33310) 2024-04-25T10:12:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 -- 172.21.15.5:0/2944653844 learned_addr learned my addr 172.21.15.5:0/2944653844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd99be00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd99c0d07d0 0x7fd99c1523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 -- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 msgr2=0x7fd99c1f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c1f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 -- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd99c0d07d0 msgr2=0x7fd99c1523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd99c0d07d0 0x7fd99c1523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.615+0000 7fd9a1200640 1 -- 172.21.15.5:0/2944653844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0cbda0 con 0x7fd99c161200 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.616+0000 7fd9a1200640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 0x7fd99c1f6ee0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c7f0 tx=0x7fd99400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.616+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994015070 con 0x7fd99c161200 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.616+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd994004220 con 0x7fd99c161200 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.616+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994007620 con 0x7fd99c161200 2024-04-25T10:12:28.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.616+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c0cc080 con 0x7fd99c161200 2024-04-25T10:12:28.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.616+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c0b6750 con 0x7fd99c161200 2024-04-25T10:12:28.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.617+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c068f60 con 0x7fd99c161200 2024-04-25T10:12:28.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.618+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9940079d0 con 0x7fd99c161200 2024-04-25T10:12:28.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.619+0000 7fd993e00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd97806b500 0x7fd97806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.619+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd99408a5c0 con 0x7fd99c161200 2024-04-25T10:12:28.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.620+0000 7fd99be00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd97806b500 0x7fd97806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.621+0000 7fd99be00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd97806b500 0x7fd97806d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd98c001ca0 tx=0x7fd98c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.624+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9940590f0 con 0x7fd99c161200 2024-04-25T10:12:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.864+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd99c0ca340 con 0x7fd99c161200 2024-04-25T10:12:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.865+0000 7fd993e00640 1 -- 172.21.15.5:0/2944653844 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fd9940589f0 con 0x7fd99c161200 2024-04-25T10:12:28.869 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:28.869 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3023415561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3023415561},{"type":"v1","addr":"172.21.15.5:6835","nonce":3023415561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:12:27.797176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:28.870 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 91 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.867+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd97806b500 msgr2=0x7fd97806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.867+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd97806b500 0x7fd97806d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd98c001ca0 tx=0x7fd98c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.867+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 msgr2=0x7fd99c1f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.867+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 0x7fd99c1f6ee0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fd99400c7f0 tx=0x7fd99400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 shutdown_connections 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd99c161200 0x7fd99c1f6ee0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd99c0d2380 0x7fd99c1f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd97806b500 0x7fd97806d9c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 --2- 172.21.15.5:0/2944653844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd99c0d07d0 0x7fd99c1523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 >> 172.21.15.5:0/2944653844 conn(0x7fd99c08af80 msgr2=0x7fd99c0b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 shutdown_connections 2024-04-25T10:12:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:28.868+0000 7fd9a1c00640 1 -- 172.21.15.5:0/2944653844 wait complete. 2024-04-25T10:12:28.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:29.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.096+0000 7efc42600640 1 Processor -- start 2024-04-25T10:12:29.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.096+0000 7efc42600640 1 -- start start 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.098+0000 7efc42600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 0x7efc3c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.098+0000 7efc42600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c0cd200 con 0x7efc3c0cdca0 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.098+0000 7efc42600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3c06e6f0 con 0x7efc3c154a10 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.098+0000 7efc42600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c06e9d0 con 0x7efc3c0716a0 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc33e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efc3c0cdca0 0x7efc3c070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48396/0 (socket says 172.21.15.5:48396) 2024-04-25T10:12:29.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc33e00640 1 -- 172.21.15.5:0/4260210292 learned_addr learned my addr 172.21.15.5:0/4260210292 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc41c00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 0x7efc3c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854488903 0 0) 0x7efc3c0cd200 con 0x7efc3c0cdca0 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003520 con 0x7efc3c0cdca0 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3301979978 0 0) 0x7efc20003520 con 0x7efc3c0cdca0 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 msgr2=0x7efc3c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 0x7efc3c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 >> v1:172.21.15.80:6789/0 conn(0x7efc3c154a10 legacy=0x7efc3c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c06ecb0 con 0x7efc3c0cdca0 2024-04-25T10:12:29.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.099+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7efc24001670 con 0x7efc3c0cdca0 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 >> v1:172.21.15.5:6789/0 conn(0x7efc3c0cdca0 legacy=0x7efc3c070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 0x7efc200039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc20003fb0 0x7efc20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc20004d40 0x7efc200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c06e9d0 con 0x7efc20003fb0 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000bbb0 con 0x7efc20004d40 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000bdb0 con 0x7efc3c0716a0 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc33e00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc20003fb0 0x7efc20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc41c00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc20004d40 0x7efc200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc33e00640 1 -- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 msgr2=0x7efc200039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc33e00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc3c0716a0 0x7efc200039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc33e00640 1 -- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc20004d40 msgr2=0x7efc200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc33e00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc20004d40 0x7efc200051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.100+0000 7efc33e00640 1 -- 172.21.15.5:0/4260210292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20003520 con 0x7efc20003fb0 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.101+0000 7efc33e00640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc20003fb0 0x7efc20004800 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c0e0 tx=0x7efc20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a86424e5ee530f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.102+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24007c20 con 0x7efc20003fb0 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.102+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24005af0 con 0x7efc20003fb0 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.102+0000 7efc3be00640 1 -- 172.21.15.5:0/4260210292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24007070 con 0x7efc20003fb0 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.102+0000 7efc42600640 1 -- 172.21.15.5:0/4260210292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc20003fb0 msgr2=0x7efc20004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.102+0000 7efc42600640 1 --2- 172.21.15.5:0/4260210292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc20003fb0 0x7efc20004800 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efc2400c0e0 tx=0x7efc20005830 comp rx=0 tx=0).stop 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.102+0000 7efc41c00640 1 -- 172.21.15.5:0/4260210292 reap_dead start 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.103+0000 7efc42600640 1 -- 172.21.15.5:0/4260210292 shutdown_connections 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.103+0000 7efc42600640 1 -- 172.21.15.5:0/4260210292 >> 172.21.15.5:0/4260210292 conn(0x7efc3c066f20 msgr2=0x7efc3c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.103+0000 7efc42600640 1 -- 172.21.15.5:0/4260210292 shutdown_connections 2024-04-25T10:12:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.103+0000 7efc42600640 1 -- 172.21.15.5:0/4260210292 wait complete. 2024-04-25T10:12:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.104+0000 7efc42600640 1 Processor -- start 2024-04-25T10:12:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.105+0000 7efc42600640 1 -- start start 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.105+0000 7efc42600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc20003fb0 0x7efc3c1f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.105+0000 7efc42600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 0x7efc3c13ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.105+0000 7efc42600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc3c13dbd0 0x7efc3c13e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc42600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15e8c0 con 0x7efc3c13ea90 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc42600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c0cd780 con 0x7efc3c13dbd0 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc42600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3c15ea40 con 0x7efc20003fb0 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 0x7efc3c13ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc33e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 0x7efc3c13ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33334/0 (socket says 172.21.15.5:33334) 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc33e00640 1 -- 172.21.15.5:0/559234837 learned_addr learned my addr 172.21.15.5:0/559234837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:29.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc41c00640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc3c13dbd0 0x7efc3c13e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.106+0000 7efc41200640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc20003fb0 0x7efc3c1f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc33e00640 1 -- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc20003fb0 msgr2=0x7efc3c1f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc33e00640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc20003fb0 0x7efc3c1f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc33e00640 1 -- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc3c13dbd0 msgr2=0x7efc3c13e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc33e00640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc3c13dbd0 0x7efc3c13e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc33e00640 1 -- 172.21.15.5:0/559234837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c150450 con 0x7efc3c13ea90 2024-04-25T10:12:29.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc41c00640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc3c13dbd0 0x7efc3c13e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc33e00640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 0x7efc3c13ef10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efc2402ad20 tx=0x7efc3c06ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24043070 con 0x7efc3c13ea90 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc24038040 con 0x7efc3c13ea90 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3c13cb30 con 0x7efc3c13ea90 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240072b0 con 0x7efc3c13ea90 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.107+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3c13d0f0 con 0x7efc3c13ea90 2024-04-25T10:12:29.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.109+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3c068f60 con 0x7efc3c13ea90 2024-04-25T10:12:29.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.110+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc24007450 con 0x7efc3c13ea90 2024-04-25T10:12:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.111+0000 7efc39600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc1806b550 0x7efc1806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.111+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7efc240aa370 con 0x7efc3c13ea90 2024-04-25T10:12:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.111+0000 7efc41200640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc1806b550 0x7efc1806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.113+0000 7efc41200640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc1806b550 0x7efc1806da10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efc2c007fd0 tx=0x7efc2c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.115+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc240788e0 con 0x7efc3c13ea90 2024-04-25T10:12:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.315+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc3c0cce40 con 0x7efc3c13ea90 2024-04-25T10:12:29.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.318+0000 7efc39600640 1 -- 172.21.15.5:0/559234837 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17296 (secure 0 0 0) 0x7efc2407c980 con 0x7efc3c13ea90 2024-04-25T10:12:29.323 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:29.323 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:27.797214+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.2d","mappings":[{"from":4,"to":3}]},{"pgid":"16.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6835/2001088034":"2024-04-26T10:11:50.284695+0000","172.21.15.5:6835/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6834/2001088034":"2024-04-26T10:11:50.284695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc1806b550 msgr2=0x7efc1806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc1806b550 0x7efc1806da10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7efc2c007fd0 tx=0x7efc2c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 msgr2=0x7efc3c13ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 0x7efc3c13ef10 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efc2402ad20 tx=0x7efc3c06ee80 comp rx=0 tx=0).stop 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 shutdown_connections 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc1806b550 0x7efc1806da10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc3c13dbd0 0x7efc3c13e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c13ea90 0x7efc3c13ef10 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 --2- 172.21.15.5:0/559234837 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc20003fb0 0x7efc3c1f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 >> 172.21.15.5:0/559234837 conn(0x7efc3c066f20 msgr2=0x7efc3c0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 shutdown_connections 2024-04-25T10:12:29.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.321+0000 7efc42600640 1 -- 172.21.15.5:0/559234837 wait complete. 2024-04-25T10:12:29.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:10:57.493997+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:10:58.605360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:29.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.552+0000 7f417a000640 1 Processor -- start 2024-04-25T10:12:29.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.553+0000 7f417a000640 1 -- start start 2024-04-25T10:12:29.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f417a000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f41741465d0 0x7f417415e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f417a000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f41740ae860 0x7f4174071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f417a000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417415eed0 con 0x7f41740aece0 2024-04-25T10:12:29.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f417a000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417406f080 con 0x7f41741465d0 2024-04-25T10:12:29.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f417a000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417406f360 con 0x7f41740ae860 2024-04-25T10:12:29.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f4173400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f41741465d0 0x7f417415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f4173e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f41740aece0 0x7f417415f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48406/0 (socket says 172.21.15.5:48406) 2024-04-25T10:12:29.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f4173e00640 1 -- 172.21.15.5:0/440956034 learned_addr learned my addr 172.21.15.5:0/440956034 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:29.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.555+0000 7f416be00640 1 --2- 172.21.15.5:0/440956034 >> v2:172.21.15.80:3300/0 conn(0x7f41740ae860 0x7f4174071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.556+0000 7f4172a00640 1 -- 172.21.15.5:0/440956034 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710987817 0 0) 0x7f416c001490 con 0x7f41740aece0 2024-04-25T10:12:29.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.556+0000 7f4172a00640 1 -- 172.21.15.5:0/440956034 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4158003520 con 0x7f41740aece0 2024-04-25T10:12:29.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.556+0000 7f4173400640 1 -- 172.21.15.5:0/440956034 >> v2:172.21.15.80:3300/0 conn(0x7f41740ae860 msgr2=0x7f4174071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.556+0000 7f4173400640 1 --2- 172.21.15.5:0/440956034 >> v2:172.21.15.80:3300/0 conn(0x7f41740ae860 0x7f4174071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.556+0000 7f4173400640 1 -- 172.21.15.5:0/440956034 >> v1:172.21.15.5:6789/0 conn(0x7f41740aece0 legacy=0x7f417415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.556+0000 7f4173400640 1 -- 172.21.15.5:0/440956034 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41740ae060 con 0x7f41741465d0 2024-04-25T10:12:29.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.557+0000 7f4173400640 1 --2- 172.21.15.5:0/440956034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41741465d0 0x7f417415e990 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f417406ba00 tx=0x7f416402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed2e52174381a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.557+0000 7f4172a00640 1 -- 172.21.15.5:0/440956034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164037070 con 0x7f41741465d0 2024-04-25T10:12:29.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.557+0000 7f4172a00640 1 -- 172.21.15.5:0/440956034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164032440 con 0x7f41741465d0 2024-04-25T10:12:29.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.557+0000 7f4172a00640 1 -- 172.21.15.5:0/440956034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164032810 con 0x7f41741465d0 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 -- 172.21.15.5:0/440956034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41741465d0 msgr2=0x7f417415e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 --2- 172.21.15.5:0/440956034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41741465d0 0x7f417415e990 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f417406ba00 tx=0x7f416402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 -- 172.21.15.5:0/440956034 shutdown_connections 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 --2- 172.21.15.5:0/440956034 >> v2:172.21.15.80:3300/0 conn(0x7f41740ae860 0x7f4174071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 --2- 172.21.15.5:0/440956034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41741465d0 0x7f417415e990 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 -- 172.21.15.5:0/440956034 >> 172.21.15.5:0/440956034 conn(0x7f417406ab60 msgr2=0x7f417406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 -- 172.21.15.5:0/440956034 shutdown_connections 2024-04-25T10:12:29.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.558+0000 7f417a000640 1 -- 172.21.15.5:0/440956034 wait complete. 2024-04-25T10:12:29.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.560+0000 7f417a000640 1 Processor -- start 2024-04-25T10:12:29.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.560+0000 7f417a000640 1 -- start start 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f417a000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41740ae860 0x7f417415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f417a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 0x7f417413f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f417a000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f41741465d0 0x7f41740b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f417a000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417406e7d0 con 0x7f41740aece0 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f417a000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741469b0 con 0x7f41740ae860 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f417a000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417406ece0 con 0x7f41741465d0 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f416be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 0x7f417413f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f416be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 0x7f417413f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33354/0 (socket says 172.21.15.5:33354) 2024-04-25T10:12:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.561+0000 7f416be00640 1 -- 172.21.15.5:0/2986227146 learned_addr learned my addr 172.21.15.5:0/2986227146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:29.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f4173e00640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f41741465d0 0x7f41740b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f4173400640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41740ae860 0x7f417415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416be00640 1 -- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f41741465d0 msgr2=0x7f41740b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416be00640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f41741465d0 0x7f41740b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416be00640 1 -- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41740ae860 msgr2=0x7f417415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416be00640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41740ae860 0x7f417415d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416be00640 1 -- 172.21.15.5:0/2986227146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417413b5a0 con 0x7f41740aece0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f4173e00640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f41741465d0 0x7f41740b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f4173400640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41740ae860 0x7f417415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416be00640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 0x7f417413f4a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f415c00e9b0 tx=0x7f415c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c017070 con 0x7f41740aece0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.562+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c00c9a0 con 0x7f41740aece0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.563+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00cd90 con 0x7f41740aece0 2024-04-25T10:12:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.563+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417413b840 con 0x7f41740aece0 2024-04-25T10:12:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.563+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417413fe90 con 0x7f41740aece0 2024-04-25T10:12:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.564+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4174068f60 con 0x7f41740aece0 2024-04-25T10:12:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.565+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f415c01a070 con 0x7f41740aece0 2024-04-25T10:12:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.566+0000 7f416b400640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f415006b4d0 0x7f415006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.566+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f415c08a590 con 0x7f41740aece0 2024-04-25T10:12:29.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.566+0000 7f4173400640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f415006b4d0 0x7f415006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.568+0000 7f4173400640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f415006b4d0 0x7f415006d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4164034000 tx=0x7f416402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.570+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f415c0590c0 con 0x7f41740aece0 2024-04-25T10:12:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.768+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4174147540 con 0x7f41740aece0 2024-04-25T10:12:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.769+0000 7f416b400640 1 -- 172.21.15.5:0/2986227146 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f415c0589c0 con 0x7f41740aece0 2024-04-25T10:12:29.774 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:12:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f415006b4d0 msgr2=0x7f415006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f415006b4d0 0x7f415006d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4164034000 tx=0x7f416402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 msgr2=0x7f417413f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 0x7f417413f4a0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f415c00e9b0 tx=0x7f415c00ee70 comp rx=0 tx=0).stop 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 shutdown_connections 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f41741465d0 0x7f41740b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41740aece0 0x7f417413f4a0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f415006b4d0 0x7f415006d990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 --2- 172.21.15.5:0/2986227146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41740ae860 0x7f417415d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 >> 172.21.15.5:0/2986227146 conn(0x7f417406ab60 msgr2=0x7f4174145640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 shutdown_connections 2024-04-25T10:12:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.772+0000 7f417a000640 1 -- 172.21.15.5:0/2986227146 wait complete. 2024-04-25T10:12:29.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.988+0000 7f25ce400640 1 Processor -- start 2024-04-25T10:12:29.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.989+0000 7f25ce400640 1 -- start start 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.990+0000 7f25ce400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25c80b48c0 0x7f25c815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.990+0000 7f25ce400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 0x7f25c80b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.990+0000 7f25ce400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c81606e0 con 0x7f25c80d3f90 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.990+0000 7f25ce400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c81609c0 con 0x7f25c80b48c0 2024-04-25T10:12:29.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.990+0000 7f25ce400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8160ca0 con 0x7f25c8161d90 2024-04-25T10:12:29.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.991+0000 7f25c7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f25c80d3f90 0x7f25c8161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48416/0 (socket says 172.21.15.5:48416) 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.991+0000 7f25c7e00640 1 -- 172.21.15.5:0/4133632319 learned_addr learned my addr 172.21.15.5:0/4133632319 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.991+0000 7f25cd000640 1 --2- 172.21.15.5:0/4133632319 >> v2:172.21.15.80:3300/0 conn(0x7f25c80b48c0 0x7f25c815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.991+0000 7f25cda00640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 0x7f25c80b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.991+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928830026 0 0) 0x7f25c81606e0 con 0x7f25c80d3f90 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.991+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ac003520 con 0x7f25c80d3f90 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 536382426 0 0) 0x7f25ac003520 con 0x7f25c80d3f90 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 msgr2=0x7f25c80b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 0x7f25c80b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 >> v2:172.21.15.80:3300/0 conn(0x7f25c80b48c0 msgr2=0x7f25c815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 --2- 172.21.15.5:0/4133632319 >> v2:172.21.15.80:3300/0 conn(0x7f25c80b48c0 0x7f25c815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8160f80 con 0x7f25c80d3f90 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f25b00017f0 con 0x7f25c80d3f90 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 >> v1:172.21.15.5:6789/0 conn(0x7f25c80d3f90 legacy=0x7f25c8161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25ac0039b0 0x7f25ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 0x7f25ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ac004d40 0x7f25ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81609c0 con 0x7f25ac004d40 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8160ca0 con 0x7f25ac0039b0 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac00ba40 con 0x7f25c8161d90 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.992+0000 7f25cda00640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ac004d40 0x7f25ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cd000640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25ac0039b0 0x7f25ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25c7e00640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 0x7f25ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cda00640 1 -- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 msgr2=0x7f25ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cda00640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25c8161d90 0x7f25ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cda00640 1 -- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25ac0039b0 msgr2=0x7f25ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cda00640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25ac0039b0 0x7f25ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cda00640 1 -- 172.21.15.5:0/4133632319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac003520 con 0x7f25ac004d40 2024-04-25T10:12:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.993+0000 7f25cd000640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25ac0039b0 0x7f25ac003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.994+0000 7f25cda00640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ac004d40 0x7f25ac0051c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f25c808d300 tx=0x7f25ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad912445f1870d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.994+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0036070 con 0x7f25ac004d40 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.994+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c0031400 con 0x7f25ac004d40 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.994+0000 7f25c7400640 1 -- 172.21.15.5:0/4133632319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0009690 con 0x7f25ac004d40 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25ce400640 1 -- 172.21.15.5:0/4133632319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ac004d40 msgr2=0x7f25ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25ce400640 1 --2- 172.21.15.5:0/4133632319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25ac004d40 0x7f25ac0051c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f25c808d300 tx=0x7f25ac005830 comp rx=0 tx=0).stop 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25cda00640 1 -- 172.21.15.5:0/4133632319 reap_dead start 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25ce400640 1 -- 172.21.15.5:0/4133632319 shutdown_connections 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25ce400640 1 -- 172.21.15.5:0/4133632319 >> 172.21.15.5:0/4133632319 conn(0x7f25c808af80 msgr2=0x7f25c80d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25ce400640 1 -- 172.21.15.5:0/4133632319 shutdown_connections 2024-04-25T10:12:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.995+0000 7f25ce400640 1 -- 172.21.15.5:0/4133632319 wait complete. 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.997+0000 7f25ce400640 1 Processor -- start 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.997+0000 7f25ce400640 1 -- start start 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25ce400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25ac004d40 0x7f25c8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25ce400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 0x7f25c80b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25ce400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25c80b2210 0x7f25c80cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25ce400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81603a0 con 0x7f25c80ccd40 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25ce400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80b4ca0 con 0x7f25c80b2210 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25ce400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80b2b70 con 0x7f25ac004d40 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 0x7f25c80b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25c7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 0x7f25c80b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33376/0 (socket says 172.21.15.5:33376) 2024-04-25T10:12:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.998+0000 7f25c7e00640 1 -- 172.21.15.5:0/2028427408 learned_addr learned my addr 172.21.15.5:0/2028427408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25cda00640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25c80b2210 0x7f25c80cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25cd000640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25ac004d40 0x7f25c8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c7e00640 1 -- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25ac004d40 msgr2=0x7f25c8154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c7e00640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25ac004d40 0x7f25c8154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c7e00640 1 -- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25c80b2210 msgr2=0x7f25c80cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c7e00640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25c80b2210 0x7f25c80cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c7e00640 1 -- 172.21.15.5:0/2028427408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80c9c70 con 0x7f25c80ccd40 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c7e00640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 0x7f25c80b1cd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f25b000e7e0 tx=0x7f25b000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:29.999+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00072e0 con 0x7f25c80ccd40 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.000+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b0018070 con 0x7f25c80ccd40 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.000+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00057a0 con 0x7f25c80ccd40 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.000+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c80cbca0 con 0x7f25c80ccd40 2024-04-25T10:12:30.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.000+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c80cc260 con 0x7f25c80ccd40 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.001+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c8068f60 con 0x7f25c80ccd40 2024-04-25T10:12:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.002+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25b0005940 con 0x7f25c80ccd40 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.003+0000 7f25c4c00640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.003+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f25b008a570 con 0x7f25c80ccd40 2024-04-25T10:12:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.004+0000 7f25cd000640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.005+0000 7f25cd000640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25a406b4d0 0x7f25a406d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009f80 tx=0x7f25b800b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.008+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25c80c9c70 con 0x7f25c80ccd40 2024-04-25T10:12:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.205+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25c80b3140 con 0x7f25c80ccd40 2024-04-25T10:12:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.206+0000 7f25c4c00640 1 -- 172.21.15.5:0/2028427408 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25b001a310 con 0x7f25c80ccd40 2024-04-25T10:12:30.210 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25a406b4d0 msgr2=0x7f25a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25a406b4d0 0x7f25a406d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009f80 tx=0x7f25b800b040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 msgr2=0x7f25c80b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 0x7f25c80b1cd0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f25b000e7e0 tx=0x7f25b000eca0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 shutdown_connections 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f25a406b4d0 0x7f25a406d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25c80b2210 0x7f25c80cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25c80ccd40 0x7f25c80b1cd0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 --2- 172.21.15.5:0/2028427408 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25ac004d40 0x7f25c8154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.208+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 >> 172.21.15.5:0/2028427408 conn(0x7f25c808af80 msgr2=0x7f25c80d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.209+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 shutdown_connections 2024-04-25T10:12:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.209+0000 7f25ce400640 1 -- 172.21.15.5:0/2028427408 wait complete. 2024-04-25T10:12:30.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.447+0000 7efc54a00640 1 Processor -- start 2024-04-25T10:12:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.448+0000 7efc54a00640 1 -- start start 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.449+0000 7efc54a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efc500d3f90 0x7efc50161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.449+0000 7efc54a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc500b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.449+0000 7efc54a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc501606e0 con 0x7efc500b48c0 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.449+0000 7efc54a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50162030 con 0x7efc500d3f90 2024-04-25T10:12:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.449+0000 7efc54a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc501609c0 con 0x7efc50161bc0 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.450+0000 7efc4e800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efc500b48c0 0x7efc5015fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48422/0 (socket says 172.21.15.5:48422) 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.450+0000 7efc4e800640 1 -- 172.21.15.5:0/2830889306 learned_addr learned my addr 172.21.15.5:0/2830889306 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.450+0000 7efc45e00640 1 --2- 172.21.15.5:0/2830889306 >> v2:172.21.15.80:3300/0 conn(0x7efc500d3f90 0x7efc50161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.450+0000 7efc4f200640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc500b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.450+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292872959 0 0) 0x7efc501606e0 con 0x7efc500b48c0 2024-04-25T10:12:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.450+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc34003520 con 0x7efc500b48c0 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836253053 0 0) 0x7efc34003520 con 0x7efc500b48c0 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 msgr2=0x7efc500b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc500b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 >> v2:172.21.15.80:3300/0 conn(0x7efc500d3f90 msgr2=0x7efc50161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 --2- 172.21.15.5:0/2830889306 >> v2:172.21.15.80:3300/0 conn(0x7efc500d3f90 0x7efc50161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50160ca0 con 0x7efc500b48c0 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7efc40001670 con 0x7efc500b48c0 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 >> v1:172.21.15.5:6789/0 conn(0x7efc500b48c0 legacy=0x7efc5015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc340039b0 0x7efc34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34004d40 0x7efc340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50162030 con 0x7efc340039b0 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc501609c0 con 0x7efc34004d40 2024-04-25T10:12:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3400b9f0 con 0x7efc50161bc0 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.451+0000 7efc4e800640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc340039b0 0x7efc34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc4f200640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34004d40 0x7efc340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc45e00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc4e800640 1 -- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 msgr2=0x7efc34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc4e800640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc4e800640 1 -- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34004d40 msgr2=0x7efc340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc4e800640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc34004d40 0x7efc340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc4e800640 1 -- 172.21.15.5:0/2830889306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc34003520 con 0x7efc340039b0 2024-04-25T10:12:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.452+0000 7efc45e00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc50161bc0 0x7efc34004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.453+0000 7efc4e800640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc340039b0 0x7efc34003dd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c160 tx=0x7efc34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c7377530f93da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.453+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40007c80 con 0x7efc340039b0 2024-04-25T10:12:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.453+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40005af0 con 0x7efc340039b0 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.453+0000 7efc4de00640 1 -- 172.21.15.5:0/2830889306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40007070 con 0x7efc340039b0 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc54a00640 1 -- 172.21.15.5:0/2830889306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc340039b0 msgr2=0x7efc34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc54a00640 1 --2- 172.21.15.5:0/2830889306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc340039b0 0x7efc34003dd0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c160 tx=0x7efc34005830 comp rx=0 tx=0).stop 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc4f200640 1 -- 172.21.15.5:0/2830889306 reap_dead start 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc54a00640 1 -- 172.21.15.5:0/2830889306 shutdown_connections 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc54a00640 1 -- 172.21.15.5:0/2830889306 >> 172.21.15.5:0/2830889306 conn(0x7efc5008af80 msgr2=0x7efc500d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc54a00640 1 -- 172.21.15.5:0/2830889306 shutdown_connections 2024-04-25T10:12:30.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.454+0000 7efc54a00640 1 -- 172.21.15.5:0/2830889306 wait complete. 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.456+0000 7efc54a00640 1 Processor -- start 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.456+0000 7efc54a00640 1 -- start start 2024-04-25T10:12:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.457+0000 7efc54a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc340039b0 0x7efc500b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.457+0000 7efc54a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc5015f8c0 0x7efc501505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.457+0000 7efc54a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 0x7efc50150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.457+0000 7efc54a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b4ca0 con 0x7efc5015fd40 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.457+0000 7efc54a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50161260 con 0x7efc340039b0 2024-04-25T10:12:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.457+0000 7efc54a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc500b2b70 con 0x7efc5015f8c0 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 0x7efc50150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 0x7efc50150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33384/0 (socket says 172.21.15.5:33384) 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 -- 172.21.15.5:0/1957943490 learned_addr learned my addr 172.21.15.5:0/1957943490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4e800640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc340039b0 0x7efc500b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc45e00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc5015f8c0 0x7efc501505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 -- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc5015f8c0 msgr2=0x7efc501505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc5015f8c0 0x7efc501505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 -- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc340039b0 msgr2=0x7efc500b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc340039b0 0x7efc500b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.458+0000 7efc4f200640 1 -- 172.21.15.5:0/1957943490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc501f7ab0 con 0x7efc5015fd40 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.459+0000 7efc4f200640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 0x7efc50150b10 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7efc5008baf0 tx=0x7efc480115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.459+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48011e10 con 0x7efc5015fd40 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.459+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4801b070 con 0x7efc5015fd40 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.459+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc48003990 con 0x7efc5015fd40 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.459+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc500c8a40 con 0x7efc5015fd40 2024-04-25T10:12:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.459+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc500c86f0 con 0x7efc5015fd40 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.460+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50068f60 con 0x7efc5015fd40 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.461+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc48005d10 con 0x7efc5015fd40 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.462+0000 7efc47400640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc2c06b4d0 0x7efc2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.462+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7efc4808a7f0 con 0x7efc5015fd40 2024-04-25T10:12:30.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.463+0000 7efc4e800640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc2c06b4d0 0x7efc2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.464+0000 7efc4e800640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc2c06b4d0 0x7efc2c06d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efc40032000 tx=0x7efc40034040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.466+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc480593b0 con 0x7efc5015fd40 2024-04-25T10:12:30.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.664+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc500d0dd0 con 0x7efc5015fd40 2024-04-25T10:12:30.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.665+0000 7efc47400640 1 -- 172.21.15.5:0/1957943490 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7efc48058cb0 con 0x7efc5015fd40 2024-04-25T10:12:30.670 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc2c06b4d0 msgr2=0x7efc2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc2c06b4d0 0x7efc2c06d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efc40032000 tx=0x7efc40034040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 msgr2=0x7efc50150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 0x7efc50150b10 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7efc5008baf0 tx=0x7efc480115a0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 shutdown_connections 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc5015fd40 0x7efc50150b10 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc5015f8c0 0x7efc501505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7efc2c06b4d0 0x7efc2c06d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 --2- 172.21.15.5:0/1957943490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc340039b0 0x7efc500b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 >> 172.21.15.5:0/1957943490 conn(0x7efc5008af80 msgr2=0x7efc500b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 shutdown_connections 2024-04-25T10:12:30.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.668+0000 7efc54a00640 1 -- 172.21.15.5:0/1957943490 wait complete. 2024-04-25T10:12:30.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.902+0000 7f075c200640 1 Processor -- start 2024-04-25T10:12:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.902+0000 7f075c200640 1 -- start start 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075c200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f07540ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075c200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f07540cad50 0x7f07540cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075c200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07540c9f90 con 0x7f07541472c0 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075c200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07540ca270 con 0x7f07540cad50 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075c200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07540ca550 con 0x7f07540cbb50 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075a200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f07540cad50 0x7f07540cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075a200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f07540cad50 0x7f07540cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33398/0 (socket says 172.21.15.5:33398) 2024-04-25T10:12:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.904+0000 7f075a200640 1 -- 172.21.15.5:0/2817769110 learned_addr learned my addr 172.21.15.5:0/2817769110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.905+0000 7f0758e00640 1 --2- 172.21.15.5:0/2817769110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f07540ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.905+0000 7f075a200640 1 -- 172.21.15.5:0/2817769110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 msgr2=0x7f07540ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.905+0000 7f075a200640 1 --2- 172.21.15.5:0/2817769110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f07540ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.905+0000 7f075a200640 1 -- 172.21.15.5:0/2817769110 >> v1:172.21.15.80:6789/0 conn(0x7f07541472c0 legacy=0x7f07540cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.905+0000 7f075a200640 1 -- 172.21.15.5:0/2817769110 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07540ca830 con 0x7f07540cad50 2024-04-25T10:12:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.906+0000 7f075a200640 1 --2- 172.21.15.5:0/2817769110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f07540cebb0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f075415efe0 tx=0x7f074c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955f4a4ef0944c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.906+0000 7f0753e00640 1 -- 172.21.15.5:0/2817769110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c036070 con 0x7f07540cad50 2024-04-25T10:12:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.906+0000 7f0753e00640 1 -- 172.21.15.5:0/2817769110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c031600 con 0x7f07540cad50 2024-04-25T10:12:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.906+0000 7f0753e00640 1 -- 172.21.15.5:0/2817769110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c0319d0 con 0x7f07540cad50 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 -- 172.21.15.5:0/2817769110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 msgr2=0x7f07540cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 --2- 172.21.15.5:0/2817769110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f07540cebb0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f075415efe0 tx=0x7f074c02a570 comp rx=0 tx=0).stop 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 -- 172.21.15.5:0/2817769110 shutdown_connections 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 --2- 172.21.15.5:0/2817769110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f07540cebb0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 --2- 172.21.15.5:0/2817769110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f07540ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 -- 172.21.15.5:0/2817769110 >> 172.21.15.5:0/2817769110 conn(0x7f0754071150 msgr2=0x7f0754071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 -- 172.21.15.5:0/2817769110 shutdown_connections 2024-04-25T10:12:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.907+0000 7f075c200640 1 -- 172.21.15.5:0/2817769110 wait complete. 2024-04-25T10:12:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.909+0000 7f075c200640 1 Processor -- start 2024-04-25T10:12:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.909+0000 7f075c200640 1 -- start start 2024-04-25T10:12:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f075c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f0754131980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f075c200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f075412ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f075c200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07541472c0 0x7f0754145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f075c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075406d150 con 0x7f07540cad50 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f075c200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07540c9c60 con 0x7f07541472c0 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f075c200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075406d2d0 con 0x7f07540cbb50 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f0759800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f0754131980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f0759800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f0754131980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33412/0 (socket says 172.21.15.5:33412) 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.910+0000 7f0759800640 1 -- 172.21.15.5:0/4125538317 learned_addr learned my addr 172.21.15.5:0/4125538317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f075a200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07541472c0 0x7f0754145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0758e00640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f075412ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0759800640 1 -- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 msgr2=0x7f075412ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0759800640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f075412ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0759800640 1 -- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07541472c0 msgr2=0x7f0754145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0759800640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07541472c0 0x7f0754145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0759800640 1 -- 172.21.15.5:0/4125538317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07541308e0 con 0x7f07540cad50 2024-04-25T10:12:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0759800640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f0754131980 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f074800b3f0 tx=0x7f074800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.911+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748015020 con 0x7f07540cad50 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.912+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748011830 con 0x7f07540cad50 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.912+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748011c20 con 0x7f07540cad50 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.912+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0754130bc0 con 0x7f07540cad50 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.912+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0754131180 con 0x7f07540cad50 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.913+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0754073d40 con 0x7f07540cad50 2024-04-25T10:12:30.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.914+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0748018020 con 0x7f07540cad50 2024-04-25T10:12:30.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.915+0000 7f0751600640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f073406b4d0 0x7f073406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.915+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f074808a220 con 0x7f07540cad50 2024-04-25T10:12:30.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.916+0000 7f0758e00640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f073406b4d0 0x7f073406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.917+0000 7f0758e00640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f073406b4d0 0x7f073406d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f07540cc9e0 tx=0x7f0740008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:30.919+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0754073d40 con 0x7f07540cad50 2024-04-25T10:12:31.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.159+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0754072390 con 0x7f07540cad50 2024-04-25T10:12:31.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.161+0000 7f0751600640 1 -- 172.21.15.5:0/4125538317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f074801f030 con 0x7f07540cad50 2024-04-25T10:12:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:31.166 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3023415561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3023415561},{"type":"v1","addr":"172.21.15.5:6835","nonce":3023415561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:12:27.797176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:31.166 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 91 2024-04-25T10:12:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.163+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f073406b4d0 msgr2=0x7f073406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.163+0000 7f075c200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f073406b4d0 0x7f073406d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f07540cc9e0 tx=0x7f0740008040 comp rx=0 tx=0).stop 2024-04-25T10:12:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 msgr2=0x7f0754131980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f0754131980 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f074800b3f0 tx=0x7f074800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 shutdown_connections 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f073406b4d0 0x7f073406d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07541472c0 0x7f0754145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f07540cbb50 0x7f075412ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 --2- 172.21.15.5:0/4125538317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07540cad50 0x7f0754131980 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 >> 172.21.15.5:0/4125538317 conn(0x7f0754071150 msgr2=0x7f0754157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 shutdown_connections 2024-04-25T10:12:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.164+0000 7f075c200640 1 -- 172.21.15.5:0/4125538317 wait complete. 2024-04-25T10:12:31.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.383+0000 7f2d4e000640 1 Processor -- start 2024-04-25T10:12:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.384+0000 7f2d4e000640 1 -- start start 2024-04-25T10:12:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.385+0000 7f2d4e000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d481472c0 0x7f2d480cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.385+0000 7f2d4e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480ced80 0x7f2d480cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d4e000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d480c9f90 con 0x7f2d480cad50 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d4e000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d480ca270 con 0x7f2d481472c0 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d4e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480ca550 con 0x7f2d480ced80 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d481472c0 0x7f2d480cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d481472c0 0x7f2d480cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33426/0 (socket says 172.21.15.5:33426) 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47400640 1 -- 172.21.15.5:0/3022564615 learned_addr learned my addr 172.21.15.5:0/3022564615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47400640 1 -- 172.21.15.5:0/3022564615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480ced80 msgr2=0x7f2d480cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47e00640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480ced80 0x7f2d480cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47400640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480ced80 0x7f2d480cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d47400640 1 -- 172.21.15.5:0/3022564615 >> v1:172.21.15.5:6789/0 conn(0x7f2d480cad50 legacy=0x7f2d480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.386+0000 7f2d46000640 1 -- 172.21.15.5:0/3022564615 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729242035 0 0) 0x7f2d480c9f90 con 0x7f2d480cad50 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.387+0000 7f2d47400640 1 -- 172.21.15.5:0/3022564615 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480ca830 con 0x7f2d481472c0 2024-04-25T10:12:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.387+0000 7f2d47e00640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480ced80 0x7f2d480cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.388+0000 7f2d47400640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480cb440 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009870 tx=0x7f2d3802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9826abb3c592c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.388+0000 7f2d46000640 1 -- 172.21.15.5:0/3022564615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38037070 con 0x7f2d481472c0 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.388+0000 7f2d46000640 1 -- 172.21.15.5:0/3022564615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d38032440 con 0x7f2d481472c0 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.388+0000 7f2d46000640 1 -- 172.21.15.5:0/3022564615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38032810 con 0x7f2d481472c0 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.388+0000 7f2d4e000640 1 -- 172.21.15.5:0/3022564615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 msgr2=0x7f2d480cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.388+0000 7f2d4e000640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480cb440 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009870 tx=0x7f2d3802a130 comp rx=0 tx=0).stop 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.389+0000 7f2d4e000640 1 -- 172.21.15.5:0/3022564615 shutdown_connections 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.389+0000 7f2d4e000640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480ced80 0x7f2d480cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.389+0000 7f2d4e000640 1 --2- 172.21.15.5:0/3022564615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480cb440 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.389+0000 7f2d4e000640 1 -- 172.21.15.5:0/3022564615 >> 172.21.15.5:0/3022564615 conn(0x7f2d48071150 msgr2=0x7f2d48071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.389+0000 7f2d4e000640 1 -- 172.21.15.5:0/3022564615 shutdown_connections 2024-04-25T10:12:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.389+0000 7f2d4e000640 1 -- 172.21.15.5:0/3022564615 wait complete. 2024-04-25T10:12:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.390+0000 7f2d4e000640 1 Processor -- start 2024-04-25T10:12:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 -- start start 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480cad50 0x7f2d481e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d480ced80 0x7f2d480b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4806d2d0 con 0x7f2d481472c0 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4806d150 con 0x7f2d480ced80 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.391+0000 7f2d4e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4806d450 con 0x7f2d480cad50 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33434/0 (socket says 172.21.15.5:33434) 2024-04-25T10:12:31.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 -- 172.21.15.5:0/1085693850 learned_addr learned my addr 172.21.15.5:0/1085693850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d46a00640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d480ced80 0x7f2d480b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 -- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480cad50 msgr2=0x7f2d481e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480cad50 0x7f2d481e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 -- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d480ced80 msgr2=0x7f2d480b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d480ced80 0x7f2d480b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.392+0000 7f2d47e00640 1 -- 172.21.15.5:0/1085693850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d481e8cc0 con 0x7f2d481472c0 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.393+0000 7f2d47e00640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480b4f40 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000cef0 tx=0x7f2d4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.393+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4001a070 con 0x7f2d481472c0 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.393+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4000d860 con 0x7f2d481472c0 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.393+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d481e8fa0 con 0x7f2d481472c0 2024-04-25T10:12:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.393+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000dc50 con 0x7f2d481472c0 2024-04-25T10:12:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.393+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d481e9560 con 0x7f2d481472c0 2024-04-25T10:12:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.394+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d48068fd0 con 0x7f2d481472c0 2024-04-25T10:12:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.395+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d4001d070 con 0x7f2d481472c0 2024-04-25T10:12:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.396+0000 7f2d3f400640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d2406b4d0 0x7f2d2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.396+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2d4008ae20 con 0x7f2d481472c0 2024-04-25T10:12:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.397+0000 7f2d47400640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d2406b4d0 0x7f2d2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.398+0000 7f2d47400640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d2406b4d0 0x7f2d2406d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2d38034000 tx=0x7f2d3802c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.400+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d40059a10 con 0x7f2d481472c0 2024-04-25T10:12:31.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.601+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d480737a0 con 0x7f2d481472c0 2024-04-25T10:12:31.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.604+0000 7f2d3f400640 1 -- 172.21.15.5:0/1085693850 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17296 (secure 0 0 0) 0x7f2d40059310 con 0x7f2d481472c0 2024-04-25T10:12:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:31.609 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:27.797214+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.2d","mappings":[{"from":4,"to":3}]},{"pgid":"16.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6835/2001088034":"2024-04-26T10:11:50.284695+0000","172.21.15.5:6835/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6834/2001088034":"2024-04-26T10:11:50.284695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.606+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d2406b4d0 msgr2=0x7f2d2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.606+0000 7f2d4e000640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d2406b4d0 0x7f2d2406d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2d38034000 tx=0x7f2d3802c000 comp rx=0 tx=0).stop 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.606+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 msgr2=0x7f2d480b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.606+0000 7f2d4e000640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480b4f40 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000cef0 tx=0x7f2d4000a710 comp rx=0 tx=0).stop 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 shutdown_connections 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d481472c0 0x7f2d480b4f40 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2d2406b4d0 0x7f2d2406d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d480ced80 0x7f2d480b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 --2- 172.21.15.5:0/1085693850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2d480cad50 0x7f2d481e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 >> 172.21.15.5:0/1085693850 conn(0x7f2d48071150 msgr2=0x7f2d4815f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 shutdown_connections 2024-04-25T10:12:31.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.607+0000 7f2d4e000640 1 -- 172.21.15.5:0/1085693850 wait complete. 2024-04-25T10:12:31.632 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:12:31.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:31.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.839+0000 7f8d73200640 1 Processor -- start 2024-04-25T10:12:31.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.839+0000 7f8d73200640 1 -- start start 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.841+0000 7f8d73200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d740d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.841+0000 7f8d73200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7415f410 con 0x7f8d740b2f30 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.841+0000 7f8d73200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7415f6b0 con 0x7f8d7415d8a0 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.841+0000 7f8d73200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7415ffe0 con 0x7f8d74161bf0 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.841+0000 7f8d6be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8d740b2f30 0x7f8d741614e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48434/0 (socket says 172.21.15.5:48434) 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.841+0000 7f8d6be00640 1 -- 172.21.15.5:0/3622627961 learned_addr learned my addr 172.21.15.5:0/3622627961 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:31.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.842+0000 7f8d72800640 1 --2- 172.21.15.5:0/3622627961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d740d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.842+0000 7f8d72800640 1 -- 172.21.15.5:0/3622627961 >> v1:172.21.15.80:6789/0 conn(0x7f8d7415d8a0 legacy=0x7f8d740d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.842+0000 7f8d72800640 1 -- 172.21.15.5:0/3622627961 >> v1:172.21.15.5:6789/0 conn(0x7f8d740b2f30 legacy=0x7f8d741614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.842+0000 7f8d72800640 1 -- 172.21.15.5:0/3622627961 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7415f990 con 0x7f8d74161bf0 2024-04-25T10:12:31.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.844+0000 7f8d72800640 1 --2- 172.21.15.5:0/3622627961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d740d2570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009dc0 tx=0x7f8d6c02a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3827ba0e92062a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.847+0000 7f8d71400640 1 -- 172.21.15.5:0/3622627961 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007740 con 0x7f8d74161bf0 2024-04-25T10:12:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.847+0000 7f8d71400640 1 -- 172.21.15.5:0/3622627961 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c0078e0 con 0x7f8d74161bf0 2024-04-25T10:12:31.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.847+0000 7f8d71400640 1 -- 172.21.15.5:0/3622627961 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007cb0 con 0x7f8d74161bf0 2024-04-25T10:12:31.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 -- 172.21.15.5:0/3622627961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 msgr2=0x7f8d740d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 --2- 172.21.15.5:0/3622627961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d740d2570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009dc0 tx=0x7f8d6c02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 -- 172.21.15.5:0/3622627961 shutdown_connections 2024-04-25T10:12:31.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 --2- 172.21.15.5:0/3622627961 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d740d2570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 -- 172.21.15.5:0/3622627961 >> 172.21.15.5:0/3622627961 conn(0x7f8d7408af80 msgr2=0x7f8d740d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 -- 172.21.15.5:0/3622627961 shutdown_connections 2024-04-25T10:12:31.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.848+0000 7f8d73200640 1 -- 172.21.15.5:0/3622627961 wait complete. 2024-04-25T10:12:31.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.850+0000 7f8d73200640 1 Processor -- start 2024-04-25T10:12:31.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.850+0000 7f8d73200640 1 -- start start 2024-04-25T10:12:31.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d73200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d740b2f30 0x7f8d74159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d73200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 0x7f8d740b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d73200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d741551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d73200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7415ee10 con 0x7f8d7415d8a0 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d73200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7415ec90 con 0x7f8d740b2f30 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d73200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b2150 con 0x7f8d74161bf0 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 0x7f8d740b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 0x7f8d740b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33440/0 (socket says 172.21.15.5:33440) 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 -- 172.21.15.5:0/853249590 learned_addr learned my addr 172.21.15.5:0/853249590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d71e00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d740b2f30 0x7f8d74159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 -- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 msgr2=0x7f8d741551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d72800640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d741551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d741551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 -- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d740b2f30 msgr2=0x7f8d74159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.851+0000 7f8d6be00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d740b2f30 0x7f8d74159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d6be00640 1 -- 172.21.15.5:0/853249590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74159e50 con 0x7f8d7415d8a0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d71e00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d740b2f30 0x7f8d74159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d6be00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 0x7f8d740b13f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00c400 tx=0x7f8d5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c017070 con 0x7f8d7415d8a0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c005ce0 con 0x7f8d7415d8a0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d741f7450 con 0x7f8d7415d8a0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c007310 con 0x7f8d7415d8a0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.852+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d741f7a10 con 0x7f8d7415d8a0 2024-04-25T10:12:31.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.853+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74068f60 con 0x7f8d7415d8a0 2024-04-25T10:12:31.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.854+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d5c01a070 con 0x7f8d7415d8a0 2024-04-25T10:12:31.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.855+0000 7f8d6a000640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8d5006b4d0 0x7f8d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.855+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8d5c089970 con 0x7f8d7415d8a0 2024-04-25T10:12:31.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.856+0000 7f8d71e00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8d5006b4d0 0x7f8d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.857+0000 7f8d71e00640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8d5006b4d0 0x7f8d5006d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d640016b0 tx=0x7f8d64009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:31.859+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d5c0584a0 con 0x7f8d7415d8a0 2024-04-25T10:12:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.096+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d740b9580 con 0x7f8d7415d8a0 2024-04-25T10:12:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.097+0000 7f8d6a000640 1 -- 172.21.15.5:0/853249590 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f8d5c057da0 con 0x7f8d7415d8a0 2024-04-25T10:12:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:32.102 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3023415561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3023415561},{"type":"v1","addr":"172.21.15.5:6835","nonce":3023415561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:12:27.797176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 91 2024-04-25T10:12:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8d5006b4d0 msgr2=0x7f8d5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8d5006b4d0 0x7f8d5006d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8d640016b0 tx=0x7f8d64009040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 msgr2=0x7f8d740b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 0x7f8d740b13f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00c400 tx=0x7f8d5c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 shutdown_connections 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8d74161bf0 0x7f8d741551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7415d8a0 0x7f8d740b13f0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8d5006b4d0 0x7f8d5006d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 --2- 172.21.15.5:0/853249590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d740b2f30 0x7f8d74159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 >> 172.21.15.5:0/853249590 conn(0x7f8d7408af80 msgr2=0x7f8d74155c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 shutdown_connections 2024-04-25T10:12:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.100+0000 7f8d73200640 1 -- 172.21.15.5:0/853249590 wait complete. 2024-04-25T10:12:32.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.328+0000 7fe0e1200640 1 Processor -- start 2024-04-25T10:12:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.329+0000 7fe0e1200640 1 -- start start 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.330+0000 7fe0e1200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0dc141da0 0x7fe0dc06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.330+0000 7fe0e1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc06e640 0x7fe0dc15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.330+0000 7fe0e1200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0dc15d340 0x7fe0dc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.330+0000 7fe0e1200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc142180 con 0x7fe0dc15d340 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.330+0000 7fe0e1200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc071d30 con 0x7fe0dc141da0 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.330+0000 7fe0e1200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc06d620 con 0x7fe0dc06e640 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0dc15d340 0x7fe0dc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe0dc15d340 0x7fe0dc0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33454/0 (socket says 172.21.15.5:33454) 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 -- 172.21.15.5:0/795208098 learned_addr learned my addr 172.21.15.5:0/795208098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0db400640 1 --2- 172.21.15.5:0/795208098 >> v2:172.21.15.80:3300/0 conn(0x7fe0dc141da0 0x7fe0dc06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0d3e00640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc06e640 0x7fe0dc15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 -- 172.21.15.5:0/795208098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc06e640 msgr2=0x7fe0dc15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc06e640 0x7fe0dc15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 -- 172.21.15.5:0/795208098 >> v2:172.21.15.80:3300/0 conn(0x7fe0dc141da0 msgr2=0x7fe0dc06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 --2- 172.21.15.5:0/795208098 >> v2:172.21.15.80:3300/0 conn(0x7fe0dc141da0 0x7fe0dc06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0dbe00640 1 -- 172.21.15.5:0/795208098 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc06d900 con 0x7fe0dc15d340 2024-04-25T10:12:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.331+0000 7fe0d3e00640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc06e640 0x7fe0dc15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.332+0000 7fe0dbe00640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0cd300 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc06bbb0 tx=0x7fe0d402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bf78ddbc32ee71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.333+0000 7fe0daa00640 1 -- 172.21.15.5:0/795208098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4036070 con 0x7fe0dc15d340 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.333+0000 7fe0daa00640 1 -- 172.21.15.5:0/795208098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4031600 con 0x7fe0dc15d340 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.333+0000 7fe0daa00640 1 -- 172.21.15.5:0/795208098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d40319d0 con 0x7fe0dc15d340 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.333+0000 7fe0e1200640 1 -- 172.21.15.5:0/795208098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc15d340 msgr2=0x7fe0dc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.333+0000 7fe0e1200640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0cd300 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc06bbb0 tx=0x7fe0d402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.333+0000 7fe0e1200640 1 -- 172.21.15.5:0/795208098 shutdown_connections 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.334+0000 7fe0e1200640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0cd300 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.334+0000 7fe0e1200640 1 --2- 172.21.15.5:0/795208098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc06e640 0x7fe0dc15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.334+0000 7fe0e1200640 1 --2- 172.21.15.5:0/795208098 >> v2:172.21.15.80:3300/0 conn(0x7fe0dc141da0 0x7fe0dc06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.334+0000 7fe0e1200640 1 -- 172.21.15.5:0/795208098 >> 172.21.15.5:0/795208098 conn(0x7fe0dc066f20 msgr2=0x7fe0dc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.334+0000 7fe0e1200640 1 -- 172.21.15.5:0/795208098 shutdown_connections 2024-04-25T10:12:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.334+0000 7fe0e1200640 1 -- 172.21.15.5:0/795208098 wait complete. 2024-04-25T10:12:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.335+0000 7fe0e1200640 1 Processor -- start 2024-04-25T10:12:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 -- start start 2024-04-25T10:12:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 0x7fe0dc13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc141da0 0x7fe0dc0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cd9c0 con 0x7fe0dc06e640 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cd840 con 0x7fe0dc15d340 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.336+0000 7fe0e1200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cdb40 con 0x7fe0dc141da0 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 0x7fe0dc13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 0x7fe0dc13eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33460/0 (socket says 172.21.15.5:33460) 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 -- 172.21.15.5:0/2745601591 learned_addr learned my addr 172.21.15.5:0/2745601591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0dbe00640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0d3e00640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc141da0 0x7fe0dc0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 -- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc141da0 msgr2=0x7fe0dc0acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc141da0 0x7fe0dc0acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 -- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0dc15d340 msgr2=0x7fe0dc0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0db400640 1 -- 172.21.15.5:0/2745601591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0ad640 con 0x7fe0dc06e640 2024-04-25T10:12:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0dbe00640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.337+0000 7fe0d3e00640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc141da0 0x7fe0dc0acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.338+0000 7fe0db400640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 0x7fe0dc13eab0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00d800 tx=0x7fe0cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.338+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc00b6c0 con 0x7fe0dc06e640 2024-04-25T10:12:32.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.338+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc019070 con 0x7fe0dc06e640 2024-04-25T10:12:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.338+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc00ba80 con 0x7fe0dc06e640 2024-04-25T10:12:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.338+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc0ad920 con 0x7fe0dc06e640 2024-04-25T10:12:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.338+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc0adee0 con 0x7fe0dc06e640 2024-04-25T10:12:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.339+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc068f60 con 0x7fe0dc06e640 2024-04-25T10:12:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.340+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0cc00bc20 con 0x7fe0dc06e640 2024-04-25T10:12:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.341+0000 7fe0d3400640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0b806b4d0 0x7fe0b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.341+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe0cc08a970 con 0x7fe0dc06e640 2024-04-25T10:12:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.342+0000 7fe0d3e00640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0b806b4d0 0x7fe0b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.343+0000 7fe0d3e00640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0b806b4d0 0x7fe0b806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4006fd0 tx=0x7fe0c4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.345+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0cc059530 con 0x7fe0dc06e640 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.587+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0dc0ccae0 con 0x7fe0dc06e640 2024-04-25T10:12:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.588+0000 7fe0d3400640 1 -- 172.21.15.5:0/2745601591 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fe0cc058e30 con 0x7fe0dc06e640 2024-04-25T10:12:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:32.593 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3023415561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3023415561},{"type":"v1","addr":"172.21.15.5:6835","nonce":3023415561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:10:59.580232+0000","modified":"2024-04-25T10:12:27.797176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 91 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0b806b4d0 msgr2=0x7fe0b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0b806b4d0 0x7fe0b806d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4006fd0 tx=0x7fe0c4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 msgr2=0x7fe0dc13eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 0x7fe0dc13eab0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00d800 tx=0x7fe0cc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 shutdown_connections 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe0b806b4d0 0x7fe0b806d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0dc15d340 0x7fe0dc0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0dc141da0 0x7fe0dc0acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 --2- 172.21.15.5:0/2745601591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0dc06e640 0x7fe0dc13eab0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.591+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 >> 172.21.15.5:0/2745601591 conn(0x7fe0dc066f20 msgr2=0x7fe0dc141060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.592+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 shutdown_connections 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.592+0000 7fe0e1200640 1 -- 172.21.15.5:0/2745601591 wait complete. 2024-04-25T10:12:32.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.816+0000 7f3cd2c00640 1 Processor -- start 2024-04-25T10:12:32.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.816+0000 7f3cd2c00640 1 -- start start 2024-04-25T10:12:32.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3ccc141da0 0x7f3ccc06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc06e640 0x7f3ccc15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ccc15d340 0x7f3ccc0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc142180 con 0x7f3ccc15d340 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc071d30 con 0x7f3ccc141da0 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc06d620 con 0x7f3ccc06e640 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ccc15d340 0x7f3ccc0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3ccc15d340 0x7f3ccc0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33474/0 (socket says 172.21.15.5:33474) 2024-04-25T10:12:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.818+0000 7f3cd2200640 1 -- 172.21.15.5:0/901489973 learned_addr learned my addr 172.21.15.5:0/901489973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd1800640 1 --2- 172.21.15.5:0/901489973 >> v2:172.21.15.80:3300/0 conn(0x7f3ccc141da0 0x7f3ccc06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd0e00640 1 --2- 172.21.15.5:0/901489973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc06e640 0x7f3ccc15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd2200640 1 -- 172.21.15.5:0/901489973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc06e640 msgr2=0x7f3ccc15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd2200640 1 --2- 172.21.15.5:0/901489973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc06e640 0x7f3ccc15ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd2200640 1 -- 172.21.15.5:0/901489973 >> v2:172.21.15.80:3300/0 conn(0x7f3ccc141da0 msgr2=0x7f3ccc06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd2200640 1 --2- 172.21.15.5:0/901489973 >> v2:172.21.15.80:3300/0 conn(0x7f3ccc141da0 0x7f3ccc06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd2200640 1 -- 172.21.15.5:0/901489973 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc06d900 con 0x7f3ccc15d340 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.819+0000 7f3cd1800640 1 --2- 172.21.15.5:0/901489973 >> v2:172.21.15.80:3300/0 conn(0x7f3ccc141da0 0x7f3ccc06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.820+0000 7f3cd2200640 1 --2- 172.21.15.5:0/901489973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc15d340 0x7f3ccc0cd300 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc06bbb0 tx=0x7f3cc402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800f18e13e1aa2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.820+0000 7f3ccbe00640 1 -- 172.21.15.5:0/901489973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4036070 con 0x7f3ccc15d340 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.820+0000 7f3ccbe00640 1 -- 172.21.15.5:0/901489973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4031600 con 0x7f3ccc15d340 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.820+0000 7f3ccbe00640 1 -- 172.21.15.5:0/901489973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40319d0 con 0x7f3ccc15d340 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 -- 172.21.15.5:0/901489973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc15d340 msgr2=0x7f3ccc0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/901489973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc15d340 0x7f3ccc0cd300 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc06bbb0 tx=0x7f3cc402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 -- 172.21.15.5:0/901489973 shutdown_connections 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/901489973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc15d340 0x7f3ccc0cd300 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/901489973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc06e640 0x7f3ccc15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/901489973 >> v2:172.21.15.80:3300/0 conn(0x7f3ccc141da0 0x7f3ccc06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 -- 172.21.15.5:0/901489973 >> 172.21.15.5:0/901489973 conn(0x7f3ccc066f20 msgr2=0x7f3ccc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 -- 172.21.15.5:0/901489973 shutdown_connections 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.821+0000 7f3cd2c00640 1 -- 172.21.15.5:0/901489973 wait complete. 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.823+0000 7f3cd2c00640 1 Processor -- start 2024-04-25T10:12:32.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.823+0000 7f3cd2c00640 1 -- start start 2024-04-25T10:12:32.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd2c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc06e640 0x7f3ccc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd2c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 0x7f3ccc1f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd2c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc15d340 0x7f3ccc1400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd2c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0cd9c0 con 0x7f3ccc141da0 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd2c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0cd840 con 0x7f3ccc06e640 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd2c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc0cdb40 con 0x7f3ccc15d340 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 0x7f3ccc1f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 0x7f3ccc1f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33480/0 (socket says 172.21.15.5:33480) 2024-04-25T10:12:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.824+0000 7f3cd0e00640 1 -- 172.21.15.5:0/2561085265 learned_addr learned my addr 172.21.15.5:0/2561085265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd1800640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc06e640 0x7f3ccc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd2200640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc15d340 0x7f3ccc1400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd0e00640 1 -- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc15d340 msgr2=0x7f3ccc1400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd0e00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc15d340 0x7f3ccc1400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd0e00640 1 -- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc06e640 msgr2=0x7f3ccc138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd0e00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc06e640 0x7f3ccc138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd0e00640 1 -- 172.21.15.5:0/2561085265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc1f4a20 con 0x7f3ccc141da0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd1800640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc06e640 0x7f3ccc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.825+0000 7f3cd0e00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 0x7f3ccc1f38c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c7f0 tx=0x7f3cb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.826+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8004230 con 0x7f3ccc141da0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.826+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc1f4180 con 0x7f3ccc141da0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.826+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb80043d0 con 0x7f3ccc141da0 2024-04-25T10:12:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.826+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8007860 con 0x7f3ccc141da0 2024-04-25T10:12:32.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.826+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc1f4740 con 0x7f3ccc141da0 2024-04-25T10:12:32.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.827+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc0ccae0 con 0x7f3ccc141da0 2024-04-25T10:12:32.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.828+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3cb8007c10 con 0x7f3ccc141da0 2024-04-25T10:12:32.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.829+0000 7f3cc9600640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cac06b4d0 0x7f3cac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.830+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3cb808a650 con 0x7f3ccc141da0 2024-04-25T10:12:32.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.830+0000 7f3cd1800640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cac06b4d0 0x7f3cac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.831+0000 7f3cd1800640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cac06b4d0 0x7f3cac06d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00088b0 tx=0x7f3cc0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:32.834+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cb8015020 con 0x7f3ccc141da0 2024-04-25T10:12:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.028+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ccc15b6c0 con 0x7f3ccc141da0 2024-04-25T10:12:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.029+0000 7f3cc9600640 1 -- 172.21.15.5:0/2561085265 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+17296 (secure 0 0 0) 0x7f3cb8059210 con 0x7f3ccc141da0 2024-04-25T10:12:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:33.034 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:27.797214+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:10:57.493997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:10:58.605360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"16.17","mappings":[{"from":4,"to":3}]},{"pgid":"16.2d","mappings":[{"from":4,"to":3}]},{"pgid":"16.39","mappings":[{"from":2,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6835/2001088034":"2024-04-26T10:11:50.284695+0000","172.21.15.5:6835/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6834/2001088034":"2024-04-26T10:11:50.284695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:33.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.031+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cac06b4d0 msgr2=0x7f3cac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.031+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cac06b4d0 0x7f3cac06d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00088b0 tx=0x7f3cc0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:33.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.031+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 msgr2=0x7f3ccc1f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.031+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 0x7f3ccc1f38c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c7f0 tx=0x7f3cb800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 shutdown_connections 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3ccc15d340 0x7f3ccc1400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ccc141da0 0x7f3ccc1f38c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3cac06b4d0 0x7f3cac06d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 --2- 172.21.15.5:0/2561085265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ccc06e640 0x7f3ccc138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 >> 172.21.15.5:0/2561085265 conn(0x7f3ccc066f20 msgr2=0x7f3ccc15dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 shutdown_connections 2024-04-25T10:12:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.032+0000 7f3cd2c00640 1 -- 172.21.15.5:0/2561085265 wait complete. 2024-04-25T10:12:33.055 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:33.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.257+0000 7f06c6e00640 1 Processor -- start 2024-04-25T10:12:33.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.257+0000 7f06c6e00640 1 -- start start 2024-04-25T10:12:33.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c6e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c0143d30 0x7f06c015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c6e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f06c00ceea0 0x7f06c00b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c6e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00b4880 con 0x7f06c00b3ae0 2024-04-25T10:12:33.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c6e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00b45a0 con 0x7f06c00ceea0 2024-04-25T10:12:33.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c6e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c015e2c0 con 0x7f06c0143d30 2024-04-25T10:12:33.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c5000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f06c00b3ae0 0x7f06c00ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48440/0 (socket says 172.21.15.5:48440) 2024-04-25T10:12:33.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c5000640 1 -- 172.21.15.5:0/1051367048 learned_addr learned my addr 172.21.15.5:0/1051367048 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:33.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.259+0000 7f06c6400640 1 --2- 172.21.15.5:0/1051367048 >> v2:172.21.15.5:3300/0 conn(0x7f06c00ceea0 0x7f06c00b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06bfe00640 1 -- 172.21.15.5:0/1051367048 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970291626 0 0) 0x7f06c00b4880 con 0x7f06c00b3ae0 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06bfe00640 1 -- 172.21.15.5:0/1051367048 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a8003520 con 0x7f06c00b3ae0 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06c5a00640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c0143d30 0x7f06c015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06c6400640 1 -- 172.21.15.5:0/1051367048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c0143d30 msgr2=0x7f06c015eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06c6400640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c0143d30 0x7f06c015eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06c6400640 1 -- 172.21.15.5:0/1051367048 >> v1:172.21.15.5:6789/0 conn(0x7f06c00b3ae0 legacy=0x7f06c00ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06c6400640 1 -- 172.21.15.5:0/1051367048 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c015e5a0 con 0x7f06c00ceea0 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.260+0000 7f06c5a00640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c0143d30 0x7f06c015eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:33.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.261+0000 7f06c6400640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00ceea0 0x7f06c00b3f60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009d90 tx=0x7f06b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33630cad0a1ee904 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.261+0000 7f06bfe00640 1 -- 172.21.15.5:0/1051367048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8007740 con 0x7f06c00ceea0 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.261+0000 7f06bfe00640 1 -- 172.21.15.5:0/1051367048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b80078e0 con 0x7f06c00ceea0 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.261+0000 7f06bfe00640 1 -- 172.21.15.5:0/1051367048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8007cb0 con 0x7f06c00ceea0 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 -- 172.21.15.5:0/1051367048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00ceea0 msgr2=0x7f06c00b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00ceea0 0x7f06c00b3f60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009d90 tx=0x7f06b802a660 comp rx=0 tx=0).stop 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 -- 172.21.15.5:0/1051367048 shutdown_connections 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00ceea0 0x7f06c00b3f60 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 --2- 172.21.15.5:0/1051367048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c0143d30 0x7f06c015eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 -- 172.21.15.5:0/1051367048 >> 172.21.15.5:0/1051367048 conn(0x7f06c0066f20 msgr2=0x7f06c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 -- 172.21.15.5:0/1051367048 shutdown_connections 2024-04-25T10:12:33.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.262+0000 7f06c6e00640 1 -- 172.21.15.5:0/1051367048 wait complete. 2024-04-25T10:12:33.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.264+0000 7f06c6e00640 1 Processor -- start 2024-04-25T10:12:33.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.264+0000 7f06c6e00640 1 -- start start 2024-04-25T10:12:33.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c6e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 0x7f06c0157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c6e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c00ceea0 0x7f06c0139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c6e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06c0143d30 0x7f06c0155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c6e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0144290 con 0x7f06c00b3ae0 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c6e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0144110 con 0x7f06c0143d30 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c6e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c006e430 con 0x7f06c00ceea0 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 0x7f06c0157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 0x7f06c0157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33504/0 (socket says 172.21.15.5:33504) 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 -- 172.21.15.5:0/3827814445 learned_addr learned my addr 172.21.15.5:0/3827814445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 -- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c00ceea0 msgr2=0x7f06c0139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c00ceea0 0x7f06c0139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 -- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06c0143d30 msgr2=0x7f06c0155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06c0143d30 0x7f06c0155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.265+0000 7f06c5a00640 1 -- 172.21.15.5:0/3827814445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c013dfa0 con 0x7f06c00b3ae0 2024-04-25T10:12:33.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.266+0000 7f06c5a00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 0x7f06c0157cb0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c7f0 tx=0x7f06b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.266+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4015070 con 0x7f06c00b3ae0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.266+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b4007820 con 0x7f06c00b3ae0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.266+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4007bd0 con 0x7f06c00b3ae0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.266+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c013c330 con 0x7f06c00b3ae0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.266+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c013dc30 con 0x7f06c00b3ae0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.267+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c0068f60 con 0x7f06c00b3ae0 2024-04-25T10:12:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.268+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06b4018070 con 0x7f06c00b3ae0 2024-04-25T10:12:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.269+0000 7f06bd600640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.269+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f06b408a5b0 con 0x7f06c00b3ae0 2024-04-25T10:12:33.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.270+0000 7f06c5000640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.271+0000 7f06c5000640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f06a006b4d0 0x7f06a006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f06ac000f60 tx=0x7f06ac0078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.274+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06b4059180 con 0x7f06c00b3ae0 2024-04-25T10:12:33.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.515+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f06c0068670 con 0x7f06c00b3ae0 2024-04-25T10:12:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.855+0000 7f06bd600640 1 -- 172.21.15.5:0/3827814445 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f06b4058a80 con 0x7f06c00b3ae0 2024-04-25T10:12:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f06a006b4d0 msgr2=0x7f06a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f06a006b4d0 0x7f06a006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f06ac000f60 tx=0x7f06ac0078f0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 msgr2=0x7f06c0157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 0x7f06c0157cb0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c7f0 tx=0x7f06b400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 shutdown_connections 2024-04-25T10:12:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06c0143d30 0x7f06c0155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f06c00ceea0 0x7f06c0139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 --2- 172.21.15.5:0/3827814445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c00b3ae0 0x7f06c0157cb0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.859+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 >> 172.21.15.5:0/3827814445 conn(0x7f06c0066f20 msgr2=0x7f06c01587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.860+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 shutdown_connections 2024-04-25T10:12:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:33.860+0000 7f06c6e00640 1 -- 172.21.15.5:0/3827814445 wait complete. 2024-04-25T10:12:33.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:12:34.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.104+0000 7fed1d200640 1 Processor -- start 2024-04-25T10:12:34.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.104+0000 7fed1d200640 1 -- start start 2024-04-25T10:12:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed1d200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed1815ed20 0x7fed18144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed1d200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fed18144e90 0x7fed18071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed1d200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed18071540 0x7fed1806d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed1d200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed18144780 con 0x7fed18071540 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed1d200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed18071980 con 0x7fed18144e90 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed1d200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1815f3e0 con 0x7fed1815ed20 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed17e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed18071540 0x7fed1806d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed17e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed18071540 0x7fed1806d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33520/0 (socket says 172.21.15.5:33520) 2024-04-25T10:12:34.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.106+0000 7fed17e00640 1 -- 172.21.15.5:0/2199884300 learned_addr learned my addr 172.21.15.5:0/2199884300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17400640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed1815ed20 0x7fed18144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed16a00640 1 --2- 172.21.15.5:0/2199884300 >> v2:172.21.15.80:3300/0 conn(0x7fed18144e90 0x7fed18071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17e00640 1 -- 172.21.15.5:0/2199884300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed1815ed20 msgr2=0x7fed18144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17e00640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed1815ed20 0x7fed18144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17e00640 1 -- 172.21.15.5:0/2199884300 >> v2:172.21.15.80:3300/0 conn(0x7fed18144e90 msgr2=0x7fed18071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17e00640 1 --2- 172.21.15.5:0/2199884300 >> v2:172.21.15.80:3300/0 conn(0x7fed18144e90 0x7fed18071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17e00640 1 -- 172.21.15.5:0/2199884300 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1815f680 con 0x7fed18071540 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed17400640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed1815ed20 0x7fed18144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.107+0000 7fed16a00640 1 --2- 172.21.15.5:0/2199884300 >> v2:172.21.15.80:3300/0 conn(0x7fed18144e90 0x7fed18071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.108+0000 7fed17e00640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed18071540 0x7fed1806d020 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fed180b4840 tx=0x7fed1002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aac4c64610ea5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.108+0000 7fed16000640 1 -- 172.21.15.5:0/2199884300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10037070 con 0x7fed18071540 2024-04-25T10:12:34.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.108+0000 7fed16000640 1 -- 172.21.15.5:0/2199884300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10032440 con 0x7fed18071540 2024-04-25T10:12:34.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.108+0000 7fed16000640 1 -- 172.21.15.5:0/2199884300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10032810 con 0x7fed18071540 2024-04-25T10:12:34.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 -- 172.21.15.5:0/2199884300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed18071540 msgr2=0x7fed1806d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed18071540 0x7fed1806d020 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fed180b4840 tx=0x7fed1002a140 comp rx=0 tx=0).stop 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 -- 172.21.15.5:0/2199884300 shutdown_connections 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed18071540 0x7fed1806d020 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 --2- 172.21.15.5:0/2199884300 >> v2:172.21.15.80:3300/0 conn(0x7fed18144e90 0x7fed18071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 --2- 172.21.15.5:0/2199884300 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed1815ed20 0x7fed18144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 -- 172.21.15.5:0/2199884300 >> 172.21.15.5:0/2199884300 conn(0x7fed18066f20 msgr2=0x7fed1806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 -- 172.21.15.5:0/2199884300 shutdown_connections 2024-04-25T10:12:34.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.109+0000 7fed1d200640 1 -- 172.21.15.5:0/2199884300 wait complete. 2024-04-25T10:12:34.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.111+0000 7fed1d200640 1 Processor -- start 2024-04-25T10:12:34.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.111+0000 7fed1d200640 1 -- start start 2024-04-25T10:12:34.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.112+0000 7fed1d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed18071540 0x7fed180cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.112+0000 7fed1d200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed18144e90 0x7fed181f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.112+0000 7fed1d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 0x7fed1815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.112+0000 7fed1d200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cdd50 con 0x7fed1815ed20 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.112+0000 7fed1d200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1815fc40 con 0x7fed18071540 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.112+0000 7fed1d200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806d560 con 0x7fed18144e90 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 0x7fed1815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 0x7fed1815b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33524/0 (socket says 172.21.15.5:33524) 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 -- 172.21.15.5:0/3023727928 learned_addr learned my addr 172.21.15.5:0/3023727928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17400640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed18071540 0x7fed180cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed16a00640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed18144e90 0x7fed181f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 -- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed18144e90 msgr2=0x7fed181f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed18144e90 0x7fed181f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 -- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed18071540 msgr2=0x7fed180cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed18071540 0x7fed180cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17e00640 1 -- 172.21.15.5:0/3023727928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1815b6c0 con 0x7fed1815ed20 2024-04-25T10:12:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.113+0000 7fed17400640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed18071540 0x7fed180cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.114+0000 7fed17e00640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 0x7fed1815b180 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fed100090b0 tx=0x7fed100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.114+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10037040 con 0x7fed1815ed20 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.114+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10042070 con 0x7fed1815ed20 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.114+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10030bd0 con 0x7fed1815ed20 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.114+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180ccbb0 con 0x7fed1815ed20 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.114+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed180c9fd0 con 0x7fed1815ed20 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.115+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed18068f60 con 0x7fed1815ed20 2024-04-25T10:12:34.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.116+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed10047460 con 0x7fed1815ed20 2024-04-25T10:12:34.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.117+0000 7fed0f400640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecf406b4d0 0x7fecf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.117+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fed100ab1d0 con 0x7fed1815ed20 2024-04-25T10:12:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.118+0000 7fed17400640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecf406b4d0 0x7fecf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.119+0000 7fed17400640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecf406b4d0 0x7fecf406d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fed08007f40 tx=0x7fed08007490 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.122+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed10079d00 con 0x7fed1815ed20 2024-04-25T10:12:34.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.364+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fed180720f0 con 0x7fed1815ed20 2024-04-25T10:12:34.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.868+0000 7fed0f400640 1 -- 172.21.15.5:0/3023727928 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7fed10079600 con 0x7fed1815ed20 2024-04-25T10:12:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecf406b4d0 msgr2=0x7fecf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecf406b4d0 0x7fecf406d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fed08007f40 tx=0x7fed08007490 comp rx=0 tx=0).stop 2024-04-25T10:12:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 msgr2=0x7fed1815b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 0x7fed1815b180 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fed100090b0 tx=0x7fed100046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 shutdown_connections 2024-04-25T10:12:34.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed1815ed20 0x7fed1815b180 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed18144e90 0x7fed181f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecf406b4d0 0x7fecf406d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 --2- 172.21.15.5:0/3023727928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed18071540 0x7fed180cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 >> 172.21.15.5:0/3023727928 conn(0x7fed18066f20 msgr2=0x7fed1806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.873+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 shutdown_connections 2024-04-25T10:12:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:34.874+0000 7fed1d200640 1 -- 172.21.15.5:0/3023727928 wait complete. 2024-04-25T10:12:34.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:12:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.112+0000 7f8bcbe00640 1 Processor -- start 2024-04-25T10:12:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.112+0000 7f8bcbe00640 1 -- start start 2024-04-25T10:12:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 0x7f8bcc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcbe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8bcc0cd610 0x7f8bcc06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcbe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0b4030 con 0x7f8bcc06e510 2024-04-25T10:12:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcbe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b4310 con 0x7f8bcc0cd610 2024-04-25T10:12:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b45f0 con 0x7f8bcc0b3740 2024-04-25T10:12:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcaa00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8bcc06e510 0x7f8bcc06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48448/0 (socket says 172.21.15.5:48448) 2024-04-25T10:12:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3494973328 learned_addr learned my addr 172.21.15.5:0/3494973328 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bcb400640 1 --2- 172.21.15.5:0/3494973328 >> v2:172.21.15.80:3300/0 conn(0x7f8bcc0cd610 0x7f8bcc06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.114+0000 7f8bca000640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 0x7f8bcc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063299177 0 0) 0x7f8bcc0b4030 con 0x7f8bcc06e510 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb0003520 con 0x7f8bcc06e510 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178195566 0 0) 0x7f8bb0003520 con 0x7f8bcc06e510 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 msgr2=0x7f8bcc0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 0x7f8bcc0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 >> v2:172.21.15.80:3300/0 conn(0x7f8bcc0cd610 msgr2=0x7f8bcc06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 --2- 172.21.15.5:0/3494973328 >> v2:172.21.15.80:3300/0 conn(0x7f8bcc0cd610 0x7f8bcc06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc06d150 con 0x7f8bcc06e510 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8bbc0019f0 con 0x7f8bcc06e510 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 >> v1:172.21.15.5:6789/0 conn(0x7f8bcc06e510 legacy=0x7f8bcc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bb0004800 0x7f8bb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 0x7f8bb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b4310 con 0x7f8bb00039b0 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b45f0 con 0x7f8bb0004800 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.115+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb000b9c0 con 0x7f8bcc0b3740 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcb400640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 0x7f8bb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 msgr2=0x7f8bb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bca000640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bb0004800 0x7f8bb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc0b3740 0x7f8bb00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bb0004800 msgr2=0x7f8bb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bb0004800 0x7f8bb0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.116+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3494973328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0003520 con 0x7f8bb00039b0 2024-04-25T10:12:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.117+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00a8d0 tx=0x7f8bb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7369955c656d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.117+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00a4c0 con 0x7f8bb00039b0 2024-04-25T10:12:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.117+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc02f070 con 0x7f8bb00039b0 2024-04-25T10:12:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.117+0000 7f8bc9600640 1 -- 172.21.15.5:0/3494973328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02bd00 con 0x7f8bb00039b0 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3494973328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 msgr2=0x7f8bb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3494973328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bb0003dd0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00a8d0 tx=0x7f8bb0005830 comp rx=0 tx=0).stop 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcb400640 1 -- 172.21.15.5:0/3494973328 reap_dead start 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3494973328 shutdown_connections 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3494973328 >> 172.21.15.5:0/3494973328 conn(0x7f8bcc066f20 msgr2=0x7f8bcc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3494973328 shutdown_connections 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.118+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3494973328 wait complete. 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.120+0000 7f8bcbe00640 1 Processor -- start 2024-04-25T10:12:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.120+0000 7f8bcbe00640 1 -- start start 2024-04-25T10:12:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.121+0000 7f8bcbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bcc154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.121+0000 7f8bcbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bcc163200 0x7f8bcc146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.121+0000 7f8bcbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc163680 0x7f8bcc146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.121+0000 7f8bcbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b3dd0 con 0x7f8bb00039b0 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.121+0000 7f8bcbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b3c50 con 0x7f8bcc163200 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.121+0000 7f8bcbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc161e00 con 0x7f8bcc163680 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bcc154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bcc154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33540/0 (socket says 172.21.15.5:33540) 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3171408988 learned_addr learned my addr 172.21.15.5:0/3171408988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bca000640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bcc163200 0x7f8bcc146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc163680 msgr2=0x7f8bcc146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc163680 0x7f8bcc146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bcc163200 msgr2=0x7f8bcc146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bcc163200 0x7f8bcc146020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 -- 172.21.15.5:0/3171408988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc157f70 con 0x7f8bb00039b0 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bca000640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bcc163200 0x7f8bcc146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.122+0000 7f8bcaa00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bcc154040 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00a8d0 tx=0x7f8bbc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.123+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02b8a0 con 0x7f8bb00039b0 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.123+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc02ba40 con 0x7f8bb00039b0 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.123+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02be10 con 0x7f8bb00039b0 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.123+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc146b90 con 0x7f8bb00039b0 2024-04-25T10:12:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.123+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc156000 con 0x7f8bb00039b0 2024-04-25T10:12:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.124+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc068f60 con 0x7f8bb00039b0 2024-04-25T10:12:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.125+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bbc034470 con 0x7f8bb00039b0 2024-04-25T10:12:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.126+0000 7f8bc2a00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ba806b500 0x7f8ba806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.126+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8bbc0aadf0 con 0x7f8bb00039b0 2024-04-25T10:12:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.127+0000 7f8bca000640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ba806b500 0x7f8ba806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.128+0000 7f8bca000640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ba806b500 0x7f8ba806d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc1577d0 tx=0x7f8bb4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.130+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bbc079920 con 0x7f8bb00039b0 2024-04-25T10:12:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.334+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8bcc0ce450 con 0x7f8bb00039b0 2024-04-25T10:12:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.912+0000 7f8bc2a00640 1 -- 172.21.15.5:0/3171408988 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8bbc079220 con 0x7f8bb00039b0 2024-04-25T10:12:35.917 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:12:35.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ba806b500 msgr2=0x7f8ba806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ba806b500 0x7f8ba806d9c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc1577d0 tx=0x7f8bb4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:35.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 msgr2=0x7f8bcc154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bcc154040 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00a8d0 tx=0x7f8bbc001dc0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 shutdown_connections 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8bcc163680 0x7f8bcc146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8ba806b500 0x7f8ba806d9c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bcc163200 0x7f8bcc146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 --2- 172.21.15.5:0/3171408988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb00039b0 0x7f8bcc154040 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.916+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 >> 172.21.15.5:0/3171408988 conn(0x7f8bcc066f20 msgr2=0x7f8bcc15dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.917+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 shutdown_connections 2024-04-25T10:12:35.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:35.918+0000 7f8bcbe00640 1 -- 172.21.15.5:0/3171408988 wait complete. 2024-04-25T10:12:35.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:12:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.143+0000 7f1cfd200640 1 Processor -- start 2024-04-25T10:12:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.143+0000 7f1cfd200640 1 -- start start 2024-04-25T10:12:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.144+0000 7f1cfd200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 0x7f1cf80d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.144+0000 7f1cfd200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1cf80d3330 0x7f1cf80d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.144+0000 7f1cfd200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1cf80b9290 0x7f1cf80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.144+0000 7f1cfd200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf80d2290 con 0x7f1cf80d3330 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.144+0000 7f1cfd200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf80d2570 con 0x7f1cf80b9290 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.144+0000 7f1cfd200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80d2850 con 0x7f1cf815de70 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1cf80d3330 0x7f1cf80d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1cf80d3330 0x7f1cf80d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33542/0 (socket says 172.21.15.5:33542) 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 -- 172.21.15.5:0/3841796868 learned_addr learned my addr 172.21.15.5:0/3841796868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 -- 172.21.15.5:0/3841796868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 msgr2=0x7f1cf80d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 --2- 172.21.15.5:0/3841796868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 0x7f1cf80d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 -- 172.21.15.5:0/3841796868 >> v2:172.21.15.80:3300/0 conn(0x7f1cf80b9290 msgr2=0x7f1cf80d3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 --2- 172.21.15.5:0/3841796868 >> v2:172.21.15.80:3300/0 conn(0x7f1cf80b9290 0x7f1cf80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.145+0000 7f1cf5400640 1 -- 172.21.15.5:0/3841796868 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80d2b30 con 0x7f1cf80d3330 2024-04-25T10:12:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.146+0000 7f1cf5400640 1 --2- 172.21.15.5:0/3841796868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80d3330 0x7f1cf80d3800 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80d4970 tx=0x7f1ce402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb7a98aa6d9855e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.147+0000 7f1cf4a00640 1 -- 172.21.15.5:0/3841796868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4037070 con 0x7f1cf80d3330 2024-04-25T10:12:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.147+0000 7f1cf4a00640 1 -- 172.21.15.5:0/3841796868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce4032440 con 0x7f1cf80d3330 2024-04-25T10:12:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.147+0000 7f1cf4a00640 1 -- 172.21.15.5:0/3841796868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4032810 con 0x7f1cf80d3330 2024-04-25T10:12:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.147+0000 7f1cfd200640 1 -- 172.21.15.5:0/3841796868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80d3330 msgr2=0x7f1cf80d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.147+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3841796868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80d3330 0x7f1cf80d3800 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80d4970 tx=0x7f1ce402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 -- 172.21.15.5:0/3841796868 shutdown_connections 2024-04-25T10:12:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3841796868 >> v2:172.21.15.80:3300/0 conn(0x7f1cf80b9290 0x7f1cf80d3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3841796868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80d3330 0x7f1cf80d3800 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3841796868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 0x7f1cf80d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 -- 172.21.15.5:0/3841796868 >> 172.21.15.5:0/3841796868 conn(0x7f1cf80726b0 msgr2=0x7f1cf8072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 -- 172.21.15.5:0/3841796868 shutdown_connections 2024-04-25T10:12:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.148+0000 7f1cfd200640 1 -- 172.21.15.5:0/3841796868 wait complete. 2024-04-25T10:12:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.150+0000 7f1cfd200640 1 Processor -- start 2024-04-25T10:12:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.150+0000 7f1cfd200640 1 -- start start 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cfd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 0x7f1cf80c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cfd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cf80d3330 0x7f1cf80c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cfd200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 0x7f1cf815fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cfd200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80d1fe0 con 0x7f1cf80b9290 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cfd200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80d1e60 con 0x7f1cf80d3330 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cfd200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80b52f0 con 0x7f1cf815de70 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 0x7f1cf80c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 0x7f1cf80c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33546/0 (socket says 172.21.15.5:33546) 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 -- 172.21.15.5:0/3644185944 learned_addr learned my addr 172.21.15.5:0/3644185944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 -- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 msgr2=0x7f1cf815fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 0x7f1cf815fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5400640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cf80d3330 0x7f1cf80c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 -- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cf80d3330 msgr2=0x7f1cf80c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.151+0000 7f1cf5e00640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cf80d3330 0x7f1cf80c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cf5e00640 1 -- 172.21.15.5:0/3644185944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf815eeb0 con 0x7f1cf80b9290 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cf5e00640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 0x7f1cf80c8060 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800b440 tx=0x7f1ce800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8015020 con 0x7f1cf80b9290 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce8011890 con 0x7f1cf80b9290 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf815f190 con 0x7f1cf80b9290 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8011c80 con 0x7f1cf80b9290 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.152+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf815f750 con 0x7f1cf80b9290 2024-04-25T10:12:36.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.154+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf8069040 con 0x7f1cf80b9290 2024-04-25T10:12:36.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.155+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ce8018020 con 0x7f1cf80b9290 2024-04-25T10:12:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.156+0000 7f1cee000640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1cd806b4d0 0x7f1cd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.156+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5951+0+0 (secure 0 0 0) 0x7f1ce808a4d0 con 0x7f1cf80b9290 2024-04-25T10:12:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.156+0000 7f1cf5400640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1cd806b4d0 0x7f1cd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.158+0000 7f1cf5400640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1cd806b4d0 0x7f1cd806d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4034000 tx=0x7f1ce402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.160+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ce801f020 con 0x7f1cf80b9290 2024-04-25T10:12:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.367+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1cf80b5bb0 con 0x7f1cf80b9290 2024-04-25T10:12:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.915+0000 7f1cee000640 1 -- 172.21.15.5:0/3644185944 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1ce800d040 con 0x7f1cf80b9290 2024-04-25T10:12:36.920 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.920+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1cd806b4d0 msgr2=0x7f1cd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.920+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1cd806b4d0 0x7f1cd806d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4034000 tx=0x7f1ce402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.920+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 msgr2=0x7f1cf80c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.920+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 0x7f1cf80c8060 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800b440 tx=0x7f1ce800b900 comp rx=0 tx=0).stop 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.921+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 shutdown_connections 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.921+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1cf815de70 0x7f1cf815fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.921+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1cd806b4d0 0x7f1cd806d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.921+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cf80d3330 0x7f1cf80c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.921+0000 7f1cfd200640 1 --2- 172.21.15.5:0/3644185944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf80b9290 0x7f1cf80c8060 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.921+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 >> 172.21.15.5:0/3644185944 conn(0x7f1cf80726b0 msgr2=0x7f1cf80b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.922+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 shutdown_connections 2024-04-25T10:12:36.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:36.922+0000 7f1cfd200640 1 -- 172.21.15.5:0/3644185944 wait complete. 2024-04-25T10:12:36.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:12:37.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.153+0000 7fe595600640 1 Processor -- start 2024-04-25T10:12:37.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.154+0000 7fe595600640 1 -- start start 2024-04-25T10:12:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.155+0000 7fe595600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe590143d10 0x7fe59015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.155+0000 7fe595600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 0x7fe5900adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.155+0000 7fe595600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59015f5e0 con 0x7fe5900adac0 2024-04-25T10:12:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.155+0000 7fe595600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59006d250 con 0x7fe590143d10 2024-04-25T10:12:37.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.156+0000 7fe595600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900ae680 con 0x7fe590071710 2024-04-25T10:12:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.156+0000 7fe58f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe5900adac0 0x7fe590071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35556/0 (socket says 172.21.15.5:35556) 2024-04-25T10:12:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.156+0000 7fe58f400640 1 -- 172.21.15.5:0/1504915125 learned_addr learned my addr 172.21.15.5:0/1504915125 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.156+0000 7fe594c00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 0x7fe5900adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.156+0000 7fe58fe00640 1 --2- 172.21.15.5:0/1504915125 >> v2:172.21.15.80:3300/0 conn(0x7fe590143d10 0x7fe59015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.156+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112226485 0 0) 0x7fe59015f5e0 con 0x7fe5900adac0 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe574003520 con 0x7fe5900adac0 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686942471 0 0) 0x7fe574003520 con 0x7fe5900adac0 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 msgr2=0x7fe5900adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 0x7fe5900adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 >> v2:172.21.15.80:3300/0 conn(0x7fe590143d10 msgr2=0x7fe59015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 --2- 172.21.15.5:0/1504915125 >> v2:172.21.15.80:3300/0 conn(0x7fe590143d10 0x7fe59015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900ae960 con 0x7fe5900adac0 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe578001670 con 0x7fe5900adac0 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 >> v1:172.21.15.5:6789/0 conn(0x7fe5900adac0 legacy=0x7fe590071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5740039b0 0x7fe574003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe574004800 0x7fe574004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 0x7fe5740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59006d250 con 0x7fe574004800 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900ae680 con 0x7fe5740039b0 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57400b9c0 con 0x7fe590071710 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.157+0000 7fe58f400640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe574004800 0x7fe574004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe58fe00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5740039b0 0x7fe574003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe594c00640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 0x7fe5740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe58f400640 1 -- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 msgr2=0x7fe5740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe58f400640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe590071710 0x7fe5740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe58f400640 1 -- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5740039b0 msgr2=0x7fe574003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe58f400640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5740039b0 0x7fe574003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.158+0000 7fe58f400640 1 -- 172.21.15.5:0/1504915125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574003520 con 0x7fe574004800 2024-04-25T10:12:37.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.159+0000 7fe58f400640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe574004800 0x7fe574004c80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c160 tx=0x7fe574005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b140e91974149a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.159+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578007c80 con 0x7fe574004800 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.159+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578005ad0 con 0x7fe574004800 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.159+0000 7fe58ea00640 1 -- 172.21.15.5:0/1504915125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578007070 con 0x7fe574004800 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe595600640 1 -- 172.21.15.5:0/1504915125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe574004800 msgr2=0x7fe574004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe595600640 1 --2- 172.21.15.5:0/1504915125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe574004800 0x7fe574004c80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c160 tx=0x7fe574005830 comp rx=0 tx=0).stop 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe594c00640 1 -- 172.21.15.5:0/1504915125 reap_dead start 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe595600640 1 -- 172.21.15.5:0/1504915125 shutdown_connections 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe595600640 1 -- 172.21.15.5:0/1504915125 >> 172.21.15.5:0/1504915125 conn(0x7fe590066f20 msgr2=0x7fe59006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe595600640 1 -- 172.21.15.5:0/1504915125 shutdown_connections 2024-04-25T10:12:37.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.160+0000 7fe595600640 1 -- 172.21.15.5:0/1504915125 wait complete. 2024-04-25T10:12:37.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.162+0000 7fe595600640 1 Processor -- start 2024-04-25T10:12:37.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.162+0000 7fe595600640 1 -- start start 2024-04-25T10:12:37.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe595600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe574004800 0x7fe59015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe595600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 0x7fe5901f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe595600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5901f4590 0x7fe590161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe595600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5901440f0 con 0x7fe59015df50 2024-04-25T10:12:37.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe595600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59006ea40 con 0x7fe5901f4590 2024-04-25T10:12:37.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe595600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900adf50 con 0x7fe574004800 2024-04-25T10:12:37.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe58f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 0x7fe5901f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe58f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 0x7fe5901f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57884/0 (socket says 172.21.15.5:57884) 2024-04-25T10:12:37.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.163+0000 7fe58f400640 1 -- 172.21.15.5:0/694344163 learned_addr learned my addr 172.21.15.5:0/694344163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe594c00640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5901f4590 0x7fe590161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58f400640 1 -- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe574004800 msgr2=0x7fe59015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58fe00640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe574004800 0x7fe59015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58f400640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe574004800 0x7fe59015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58f400640 1 -- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5901f4590 msgr2=0x7fe590161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58f400640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5901f4590 0x7fe590161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58f400640 1 -- 172.21.15.5:0/694344163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59015f040 con 0x7fe59015df50 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58fe00640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe574004800 0x7fe59015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe58f400640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 0x7fe5901f4050 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe57802ad60 tx=0x7fe59015dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578001630 con 0x7fe59015df50 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578042070 con 0x7fe59015df50 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59015ea80 con 0x7fe59015df50 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.164+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578007270 con 0x7fe59015df50 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.165+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe590161170 con 0x7fe59015df50 2024-04-25T10:12:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.166+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe590068f60 con 0x7fe59015df50 2024-04-25T10:12:37.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.167+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe578007410 con 0x7fe59015df50 2024-04-25T10:12:37.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.168+0000 7fe587e00640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe56c06b500 0x7fe56c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.168+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe5780aa2a0 con 0x7fe59015df50 2024-04-25T10:12:37.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.168+0000 7fe58fe00640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe56c06b500 0x7fe56c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.170+0000 7fe58fe00640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe56c06b500 0x7fe56c06d9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe580007440 tx=0x7fe58000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.172+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5780792c0 con 0x7fe59015df50 2024-04-25T10:12:37.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.374+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe59006bfe0 con 0x7fe59015df50 2024-04-25T10:12:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.379+0000 7fe587e00640 1 -- 172.21.15.5:0/694344163 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe578005aa0 con 0x7fe59015df50 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe56c06b500 msgr2=0x7fe56c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe56c06b500 0x7fe56c06d9c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe580007440 tx=0x7fe58000b040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 msgr2=0x7fe5901f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 0x7fe5901f4050 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe57802ad60 tx=0x7fe59015dee0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 shutdown_connections 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe56c06b500 0x7fe56c06d9c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5901f4590 0x7fe590161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe59015df50 0x7fe5901f4050 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 --2- 172.21.15.5:0/694344163 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe574004800 0x7fe59015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 >> 172.21.15.5:0/694344163 conn(0x7fe590066f20 msgr2=0x7fe5900b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 shutdown_connections 2024-04-25T10:12:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.382+0000 7fe595600640 1 -- 172.21.15.5:0/694344163 wait complete. 2024-04-25T10:12:37.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:12:37.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.620+0000 7fdf61600640 1 Processor -- start 2024-04-25T10:12:37.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.621+0000 7fdf61600640 1 -- start start 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.622+0000 7fdf61600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.622+0000 7fdf61600640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdf5c06e640 0x7fdf5c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.622+0000 7fdf61600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf5c15d340 0x7fdf5c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf61600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c142180 con 0x7fdf5c15d340 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf61600640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c071d30 con 0x7fdf5c06e640 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf61600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c06d620 con 0x7fdf5c141da0 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf60c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf5c15d340 0x7fdf5c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf60c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf5c15d340 0x7fdf5c0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57896/0 (socket says 172.21.15.5:57896) 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf60c00640 1 -- 172.21.15.5:0/1488652234 learned_addr learned my addr 172.21.15.5:0/1488652234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf53e00640 1 --2- 172.21.15.5:0/1488652234 >> v2:172.21.15.80:3300/0 conn(0x7fdf5c06e640 0x7fdf5c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf5be00640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf60c00640 1 -- 172.21.15.5:0/1488652234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 msgr2=0x7fdf5c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf60c00640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.623+0000 7fdf60c00640 1 -- 172.21.15.5:0/1488652234 >> v2:172.21.15.80:3300/0 conn(0x7fdf5c06e640 msgr2=0x7fdf5c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.624+0000 7fdf60c00640 1 --2- 172.21.15.5:0/1488652234 >> v2:172.21.15.80:3300/0 conn(0x7fdf5c06e640 0x7fdf5c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.624+0000 7fdf60c00640 1 -- 172.21.15.5:0/1488652234 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c06d900 con 0x7fdf5c15d340 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.624+0000 7fdf5be00640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.625+0000 7fdf60c00640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c15d340 0x7fdf5c0cd300 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0b4890 tx=0x7fdf5402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f09a0c1860c2ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.625+0000 7fdf5b400640 1 -- 172.21.15.5:0/1488652234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54036070 con 0x7fdf5c15d340 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.625+0000 7fdf5b400640 1 -- 172.21.15.5:0/1488652234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf54031600 con 0x7fdf5c15d340 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.625+0000 7fdf5b400640 1 -- 172.21.15.5:0/1488652234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540319d0 con 0x7fdf5c15d340 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 -- 172.21.15.5:0/1488652234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c15d340 msgr2=0x7fdf5c0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c15d340 0x7fdf5c0cd300 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0b4890 tx=0x7fdf5402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 -- 172.21.15.5:0/1488652234 shutdown_connections 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c15d340 0x7fdf5c0cd300 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 --2- 172.21.15.5:0/1488652234 >> v2:172.21.15.80:3300/0 conn(0x7fdf5c06e640 0x7fdf5c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 --2- 172.21.15.5:0/1488652234 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 -- 172.21.15.5:0/1488652234 >> 172.21.15.5:0/1488652234 conn(0x7fdf5c066f20 msgr2=0x7fdf5c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 -- 172.21.15.5:0/1488652234 shutdown_connections 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.626+0000 7fdf61600640 1 -- 172.21.15.5:0/1488652234 wait complete. 2024-04-25T10:12:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.628+0000 7fdf61600640 1 Processor -- start 2024-04-25T10:12:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.628+0000 7fdf61600640 1 -- start start 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf61600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 0x7fdf5c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf61600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf61600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c15d340 0x7fdf5c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf61600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cd9c0 con 0x7fdf5c06e640 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf61600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cd840 con 0x7fdf5c15d340 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf61600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cdb40 con 0x7fdf5c141da0 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 0x7fdf5c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 0x7fdf5c1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57906/0 (socket says 172.21.15.5:57906) 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 -- 172.21.15.5:0/3686999144 learned_addr learned my addr 172.21.15.5:0/3686999144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf60c00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c15d340 0x7fdf5c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 -- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 msgr2=0x7fdf5c15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 -- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c15d340 msgr2=0x7fdf5c156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c15d340 0x7fdf5c156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.629+0000 7fdf5be00640 1 -- 172.21.15.5:0/3686999144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c13cee0 con 0x7fdf5c06e640 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.630+0000 7fdf5be00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 0x7fdf5c1576e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c7f0 tx=0x7fdf4c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.630+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c015070 con 0x7fdf5c06e640 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.630+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c007a30 con 0x7fdf5c06e640 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.630+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c007e00 con 0x7fdf5c06e640 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.630+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c13f3e0 con 0x7fdf5c06e640 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.630+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c13f9a0 con 0x7fdf5c06e640 2024-04-25T10:12:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.632+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c068f60 con 0x7fdf5c06e640 2024-04-25T10:12:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.632+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf4c018070 con 0x7fdf5c06e640 2024-04-25T10:12:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.633+0000 7fdf58c00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.633+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdf4c08a030 con 0x7fdf5c06e640 2024-04-25T10:12:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.634+0000 7fdf53e00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.635+0000 7fdf53e00640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf3806b4d0 0x7fdf3806d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdf44006fd0 tx=0x7fdf44008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.638+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf4c058fd0 con 0x7fdf5c06e640 2024-04-25T10:12:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.916+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7fdf5c0ccae0 con 0x7fdf5c06e640 2024-04-25T10:12:37.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.936+0000 7fdf58c00640 1 -- 172.21.15.5:0/3686999144 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v518) v1 ==== 146+0+0 (secure 0 0 0) 0x7fdf4c0588d0 con 0x7fdf5c06e640 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf3806b4d0 msgr2=0x7fdf3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf3806b4d0 0x7fdf3806d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdf44006fd0 tx=0x7fdf44008040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 msgr2=0x7fdf5c1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 0x7fdf5c1576e0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c7f0 tx=0x7fdf4c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 shutdown_connections 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf3806b4d0 0x7fdf3806d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c15d340 0x7fdf5c156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c141da0 0x7fdf5c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 --2- 172.21.15.5:0/3686999144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c06e640 0x7fdf5c1576e0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 >> 172.21.15.5:0/3686999144 conn(0x7fdf5c066f20 msgr2=0x7fdf5c15cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 shutdown_connections 2024-04-25T10:12:37.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:37.939+0000 7fdf61600640 1 -- 172.21.15.5:0/3686999144 wait complete. 2024-04-25T10:12:37.964 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:12:37.964 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:12:37.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.172+0000 7f7a37400640 1 Processor -- start 2024-04-25T10:12:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.173+0000 7f7a37400640 1 -- start start 2024-04-25T10:12:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.174+0000 7f7a37400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a30071ed0 0x7f7a300ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.174+0000 7f7a37400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.174+0000 7f7a37400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3015bdc0 con 0x7f7a301529e0 2024-04-25T10:12:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.174+0000 7f7a37400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300ced00 con 0x7f7a30071ed0 2024-04-25T10:12:38.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.174+0000 7f7a37400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300b2340 con 0x7f7a300ce890 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a30071ed0 0x7f7a300ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a30071ed0 0x7f7a300ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57916/0 (socket says 172.21.15.5:57916) 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 -- 172.21.15.5:0/2358948814 learned_addr learned my addr 172.21.15.5:0/2358948814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a35400640 1 --2- 172.21.15.5:0/2358948814 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 -- 172.21.15.5:0/2358948814 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 msgr2=0x7f7a3006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 --2- 172.21.15.5:0/2358948814 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a3006d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 -- 172.21.15.5:0/2358948814 >> v1:172.21.15.80:6789/0 conn(0x7f7a301529e0 legacy=0x7f7a300c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.175+0000 7f7a2fe00640 1 -- 172.21.15.5:0/2358948814 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3015c060 con 0x7f7a30071ed0 2024-04-25T10:12:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.176+0000 7f7a2fe00640 1 --2- 172.21.15.5:0/2358948814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300ce350 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009e50 tx=0x7f7a1802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e130fb652a4946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.177+0000 7f7a2f400640 1 -- 172.21.15.5:0/2358948814 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18036070 con 0x7f7a30071ed0 2024-04-25T10:12:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.177+0000 7f7a2f400640 1 -- 172.21.15.5:0/2358948814 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18031600 con 0x7f7a30071ed0 2024-04-25T10:12:38.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.177+0000 7f7a2f400640 1 -- 172.21.15.5:0/2358948814 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180319d0 con 0x7f7a30071ed0 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.177+0000 7f7a37400640 1 -- 172.21.15.5:0/2358948814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 msgr2=0x7f7a300ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.177+0000 7f7a37400640 1 --2- 172.21.15.5:0/2358948814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300ce350 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f7a18009e50 tx=0x7f7a1802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.178+0000 7f7a37400640 1 -- 172.21.15.5:0/2358948814 shutdown_connections 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.178+0000 7f7a37400640 1 --2- 172.21.15.5:0/2358948814 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a3006d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.178+0000 7f7a37400640 1 --2- 172.21.15.5:0/2358948814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300ce350 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.178+0000 7f7a37400640 1 -- 172.21.15.5:0/2358948814 >> 172.21.15.5:0/2358948814 conn(0x7f7a30071150 msgr2=0x7f7a30071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.178+0000 7f7a37400640 1 -- 172.21.15.5:0/2358948814 shutdown_connections 2024-04-25T10:12:38.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.178+0000 7f7a37400640 1 -- 172.21.15.5:0/2358948814 wait complete. 2024-04-25T10:12:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.180+0000 7f7a37400640 1 Processor -- start 2024-04-25T10:12:38.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.180+0000 7f7a37400640 1 -- start start 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a37400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a37400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a30146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a37400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a300b5380 0x7f7a301f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a37400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3015c900 con 0x7f7a30071ed0 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a37400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3015c620 con 0x7f7a300b5380 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a37400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a300c65e0 con 0x7f7a300ce890 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57920/0 (socket says 172.21.15.5:57920) 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 -- 172.21.15.5:0/188996364 learned_addr learned my addr 172.21.15.5:0/188996364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 -- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 msgr2=0x7f7a30146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a30146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 -- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a300b5380 msgr2=0x7f7a301f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a300b5380 0x7f7a301f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.181+0000 7f7a34a00640 1 -- 172.21.15.5:0/188996364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30156350 con 0x7f7a30071ed0 2024-04-25T10:12:38.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.182+0000 7f7a34a00640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300b27d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c430 tx=0x7f7a2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.182+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20011790 con 0x7f7a30071ed0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.182+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a20017070 con 0x7f7a30071ed0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.182+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20011bb0 con 0x7f7a30071ed0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.182+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a301568d0 con 0x7f7a30071ed0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.182+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a301565f0 con 0x7f7a30071ed0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.183+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a30068fd0 con 0x7f7a30071ed0 2024-04-25T10:12:38.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.185+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a2001c460 con 0x7f7a30071ed0 2024-04-25T10:12:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.185+0000 7f7a2cc00640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.186+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7a20089f30 con 0x7f7a30071ed0 2024-04-25T10:12:38.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.186+0000 7f7a2fe00640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.187+0000 7f7a2fe00640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7a18033000 tx=0x7f7a1802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.190+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a20058ec0 con 0x7f7a30071ed0 2024-04-25T10:12:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.402+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a30152550 con 0x7f7a30071ed0 2024-04-25T10:12:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.404+0000 7f7a2cc00640 1 -- 172.21.15.5:0/188996364 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+13321 (secure 0 0 0) 0x7f7a200587c0 con 0x7f7a30071ed0 2024-04-25T10:12:38.408 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:38.409 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:36.912514+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6834/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6835/2001088034":"2024-04-26T10:11:50.284695+0000","172.21.15.5:6835/3788578651":"2024-04-26T10:12:27.797139+0000","172.21.15.5:6834/2001088034":"2024-04-26T10:11:50.284695+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7a0c06b4d0 msgr2=0x7f7a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7a18033000 tx=0x7f7a1802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 msgr2=0x7f7a300b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300b27d0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c430 tx=0x7f7a2000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 shutdown_connections 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7a0c06b4d0 0x7f7a0c06d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a300b5380 0x7f7a301f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7a300ce890 0x7f7a30146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 --2- 172.21.15.5:0/188996364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a30071ed0 0x7f7a300b27d0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 >> 172.21.15.5:0/188996364 conn(0x7f7a30071150 msgr2=0x7f7a3015adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 shutdown_connections 2024-04-25T10:12:38.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.407+0000 7f7a37400640 1 -- 172.21.15.5:0/188996364 wait complete. 2024-04-25T10:12:38.430 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:38.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.629+0000 7f02cda00640 1 Processor -- start 2024-04-25T10:12:38.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.629+0000 7f02cda00640 1 -- start start 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02cda00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02c806d130 0x7f02c8071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02cda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c8071670 0x7f02c815c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02cda00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c81453c0 con 0x7f02c8144fe0 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02cda00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c80a8560 con 0x7f02c806d130 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02cda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c806d6a0 con 0x7f02c8071670 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02c6a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f02c8144fe0 0x7f02c806e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35566/0 (socket says 172.21.15.5:35566) 2024-04-25T10:12:38.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02c6a00640 1 -- 172.21.15.5:0/766630220 learned_addr learned my addr 172.21.15.5:0/766630220 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02be000640 1 --2- 172.21.15.5:0/766630220 >> v2:172.21.15.5:3300/0 conn(0x7f02c806d130 0x7f02c8071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02c6000640 1 -- 172.21.15.5:0/766630220 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206656315 0 0) 0x7f02c81453c0 con 0x7f02c8144fe0 2024-04-25T10:12:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.631+0000 7f02c6000640 1 -- 172.21.15.5:0/766630220 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ac003520 con 0x7f02c8144fe0 2024-04-25T10:12:38.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.632+0000 7f02c7400640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c8071670 0x7f02c815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.632+0000 7f02be000640 1 -- 172.21.15.5:0/766630220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c8071670 msgr2=0x7f02c815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.632+0000 7f02be000640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c8071670 0x7f02c815c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.632+0000 7f02be000640 1 -- 172.21.15.5:0/766630220 >> v1:172.21.15.5:6789/0 conn(0x7f02c8144fe0 legacy=0x7f02c806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.632+0000 7f02be000640 1 -- 172.21.15.5:0/766630220 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c806d980 con 0x7f02c806d130 2024-04-25T10:12:38.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.632+0000 7f02c7400640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c8071670 0x7f02c815c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.633+0000 7f02be000640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c806d130 0x7f02c8071130 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f02b00097a0 tx=0x7f02b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42becaa46cd9ca77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.633+0000 7f02c6000640 1 -- 172.21.15.5:0/766630220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0037070 con 0x7f02c806d130 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.633+0000 7f02c6000640 1 -- 172.21.15.5:0/766630220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0032440 con 0x7f02c806d130 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.633+0000 7f02c6000640 1 -- 172.21.15.5:0/766630220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00327f0 con 0x7f02c806d130 2024-04-25T10:12:38.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 -- 172.21.15.5:0/766630220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c806d130 msgr2=0x7f02c8071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c806d130 0x7f02c8071130 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f02b00097a0 tx=0x7f02b002a180 comp rx=0 tx=0).stop 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 -- 172.21.15.5:0/766630220 shutdown_connections 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c8071670 0x7f02c815c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 --2- 172.21.15.5:0/766630220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c806d130 0x7f02c8071130 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 -- 172.21.15.5:0/766630220 >> 172.21.15.5:0/766630220 conn(0x7f02c806ab60 msgr2=0x7f02c806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 -- 172.21.15.5:0/766630220 shutdown_connections 2024-04-25T10:12:38.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.634+0000 7f02cda00640 1 -- 172.21.15.5:0/766630220 wait complete. 2024-04-25T10:12:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.636+0000 7f02cda00640 1 Processor -- start 2024-04-25T10:12:38.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.636+0000 7f02cda00640 1 -- start start 2024-04-25T10:12:38.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02cda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c806d130 0x7f02c815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02cda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c8071670 0x7f02c80afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02cda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 0x7f02c80b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02cda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c815be50 con 0x7f02c8144fe0 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02cda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c806df40 con 0x7f02c8071670 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02cda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c815c8b0 con 0x7f02c806d130 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 0x7f02c80b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02c7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 0x7f02c80b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57944/0 (socket says 172.21.15.5:57944) 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.637+0000 7f02c7400640 1 -- 172.21.15.5:0/1833773390 learned_addr learned my addr 172.21.15.5:0/1833773390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:38.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c6a00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c806d130 0x7f02c815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c7400640 1 -- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c806d130 msgr2=0x7f02c815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c7400640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c806d130 0x7f02c815ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c7400640 1 -- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c8071670 msgr2=0x7f02c80afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02be000640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c8071670 0x7f02c80afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c7400640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c8071670 0x7f02c80afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c7400640 1 -- 172.21.15.5:0/1833773390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c80abb30 con 0x7f02c8144fe0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02c7400640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 0x7f02c80b01a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f02c000cef0 tx=0x7f02c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c001a070 con 0x7f02c8144fe0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c000d860 con 0x7f02c8144fe0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c000dc50 con 0x7f02c8144fe0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.638+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c80c9890 con 0x7f02c8144fe0 2024-04-25T10:12:38.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.639+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c80c9e50 con 0x7f02c8144fe0 2024-04-25T10:12:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.640+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c8068f60 con 0x7f02c8144fe0 2024-04-25T10:12:38.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.641+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02c001d070 con 0x7f02c8144fe0 2024-04-25T10:12:38.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.642+0000 7f02bf400640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02a406b4d0 0x7f02a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.642+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f02c008a9c0 con 0x7f02c8144fe0 2024-04-25T10:12:38.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.642+0000 7f02c6a00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02a406b4d0 0x7f02a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.644+0000 7f02c6a00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02a406b4d0 0x7f02a406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000f60 tx=0x7f02b8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.647+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02c0059a10 con 0x7f02c8144fe0 2024-04-25T10:12:38.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.845+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02c80b2130 con 0x7f02c8144fe0 2024-04-25T10:12:38.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.846+0000 7f02bf400640 1 -- 172.21.15.5:0/1833773390 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02c0059310 con 0x7f02c8144fe0 2024-04-25T10:12:38.851 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02a406b4d0 msgr2=0x7f02a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02a406b4d0 0x7f02a406d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f02b8000f60 tx=0x7f02b8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 msgr2=0x7f02c80b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 0x7f02c80b01a0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f02c000cef0 tx=0x7f02c000a710 comp rx=0 tx=0).stop 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 shutdown_connections 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02c8144fe0 0x7f02c80b01a0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f02a406b4d0 0x7f02a406d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02c8071670 0x7f02c80afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 --2- 172.21.15.5:0/1833773390 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02c806d130 0x7f02c815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 >> 172.21.15.5:0/1833773390 conn(0x7f02c806ab60 msgr2=0x7f02c806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 shutdown_connections 2024-04-25T10:12:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:38.849+0000 7f02cda00640 1 -- 172.21.15.5:0/1833773390 wait complete. 2024-04-25T10:12:38.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:12:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.070+0000 7fe00e600640 1 Processor -- start 2024-04-25T10:12:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.070+0000 7fe00e600640 1 -- start start 2024-04-25T10:12:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.071+0000 7fe00e600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 0x7fe0080ce790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.071+0000 7fe00e600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080b4880 con 0x7fe0080cecd0 2024-04-25T10:12:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.071+0000 7fe00e600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080b45a0 con 0x7fe008143d30 2024-04-25T10:12:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.071+0000 7fe00e600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00815e2c0 con 0x7fe00815eda0 2024-04-25T10:12:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.072+0000 7fe00dc00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe0080cecd0 0x7fe00815abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35570/0 (socket says 172.21.15.5:35570) 2024-04-25T10:12:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.072+0000 7fe00dc00640 1 -- 172.21.15.5:0/2063793428 learned_addr learned my addr 172.21.15.5:0/2063793428 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.072+0000 7fe007e00640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 0x7fe0080ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.072+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978538464 0 0) 0x7fe0080b4880 con 0x7fe0080cecd0 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.072+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff0003520 con 0x7fe0080cecd0 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117370546 0 0) 0x7fdff0003520 con 0x7fe0080cecd0 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 msgr2=0x7fe0080ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 0x7fe0080ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 >> v1:172.21.15.80:6789/0 conn(0x7fe008143d30 legacy=0x7fe0080b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00815e5a0 con 0x7fe0080cecd0 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe0000019f0 con 0x7fe0080cecd0 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 >> v1:172.21.15.5:6789/0 conn(0x7fe0080cecd0 legacy=0x7fe00815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fdff0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdff0004800 0x7fdff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 0x7fdff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00815e2c0 con 0x7fdff00039e0 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff000baa0 con 0x7fdff0004800 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff000deb0 con 0x7fe00815eda0 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.073+0000 7fe00d200640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fdff0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe007e00640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdff0004800 0x7fdff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe00d200640 1 -- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 msgr2=0x7fdff00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe00d200640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe00815eda0 0x7fdff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe00d200640 1 -- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdff0004800 msgr2=0x7fdff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe00d200640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdff0004800 0x7fdff0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe00d200640 1 -- 172.21.15.5:0/2063793428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff0003520 con 0x7fdff00039e0 2024-04-25T10:12:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.074+0000 7fe007e00640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdff0004800 0x7fdff0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.075+0000 7fe00d200640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fdff0003e00 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdffc001720 tx=0x7fdff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69db6da5e2c27b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.075+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc001370 con 0x7fdff00039e0 2024-04-25T10:12:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.075+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc0052d0 con 0x7fdff00039e0 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.075+0000 7fe007400640 1 -- 172.21.15.5:0/2063793428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc02ab90 con 0x7fdff00039e0 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00e600640 1 -- 172.21.15.5:0/2063793428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 msgr2=0x7fdff0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00e600640 1 --2- 172.21.15.5:0/2063793428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fdff0003e00 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fdffc001720 tx=0x7fdff0005830 comp rx=0 tx=0).stop 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00dc00640 1 -- 172.21.15.5:0/2063793428 reap_dead start 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00e600640 1 -- 172.21.15.5:0/2063793428 shutdown_connections 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00e600640 1 -- 172.21.15.5:0/2063793428 >> 172.21.15.5:0/2063793428 conn(0x7fe008066f20 msgr2=0x7fe00806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00e600640 1 -- 172.21.15.5:0/2063793428 shutdown_connections 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.076+0000 7fe00e600640 1 -- 172.21.15.5:0/2063793428 wait complete. 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.078+0000 7fe00e600640 1 Processor -- start 2024-04-25T10:12:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.078+0000 7fe00e600640 1 -- start start 2024-04-25T10:12:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fe00815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00e600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0081f2930 0x7fe0081e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00e600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0081f2db0 0x7fe0081e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080b41f0 con 0x7fdff00039e0 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00e600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008144110 con 0x7fe0081f2db0 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00e600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00815baa0 con 0x7fe0081f2930 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fe00815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fe00815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57950/0 (socket says 172.21.15.5:57950) 2024-04-25T10:12:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 -- 172.21.15.5:0/4074373904 learned_addr learned my addr 172.21.15.5:0/4074373904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 -- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0081f2930 msgr2=0x7fe0081e5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0081f2930 0x7fe0081e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 -- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0081f2db0 msgr2=0x7fe0081e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.079+0000 7fe00d200640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0081f2db0 0x7fe0081e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe00d200640 1 -- 172.21.15.5:0/4074373904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008135f70 con 0x7fdff00039e0 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe00d200640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fe00815da40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000c00 tx=0x7fdffc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc042070 con 0x7fdff00039e0 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc005480 con 0x7fdff00039e0 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc037040 con 0x7fdff00039e0 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0081e4e40 con 0x7fdff00039e0 2024-04-25T10:12:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.080+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0081e5400 con 0x7fdff00039e0 2024-04-25T10:12:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.082+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008068f60 con 0x7fdff00039e0 2024-04-25T10:12:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.082+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdffc045070 con 0x7fdff00039e0 2024-04-25T10:12:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.084+0000 7fe004c00640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdfe806b4d0 0x7fdfe806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.084+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdffc0a9990 con 0x7fdff00039e0 2024-04-25T10:12:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.084+0000 7fe007e00640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdfe806b4d0 0x7fdfe806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.086+0000 7fe007e00640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdfe806b4d0 0x7fdfe806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe008159500 tx=0x7fdff4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.088+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdffc078800 con 0x7fdff00039e0 2024-04-25T10:12:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.368+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fe008068670 con 0x7fdff00039e0 2024-04-25T10:12:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.951+0000 7fe004c00640 1 -- 172.21.15.5:0/4074373904 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v520) v1 ==== 147+0+0 (secure 0 0 0) 0x7fdffc02be20 con 0x7fdff00039e0 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdfe806b4d0 msgr2=0x7fdfe806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdfe806b4d0 0x7fdfe806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe008159500 tx=0x7fdff4008040 comp rx=0 tx=0).stop 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 msgr2=0x7fe00815da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fe00815da40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fdffc000c00 tx=0x7fdffc001120 comp rx=0 tx=0).stop 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 shutdown_connections 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdfe806b4d0 0x7fdfe806d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0081f2db0 0x7fe0081e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0081f2930 0x7fe0081e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 --2- 172.21.15.5:0/4074373904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff00039e0 0x7fe00815da40 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.954+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 >> 172.21.15.5:0/4074373904 conn(0x7fe008066f20 msgr2=0x7fe0080b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.955+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 shutdown_connections 2024-04-25T10:12:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:39.955+0000 7fe00e600640 1 -- 172.21.15.5:0/4074373904 wait complete. 2024-04-25T10:12:39.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:12:40.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.197+0000 7f4e4e000640 1 Processor -- start 2024-04-25T10:12:40.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.197+0000 7f4e4e000640 1 -- start start 2024-04-25T10:12:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.199+0000 7f4e4e000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4e4815ce20 0x7f4e4815d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.199+0000 7f4e4e000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e48144cd0 con 0x7f4e480cdaa0 2024-04-25T10:12:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.199+0000 7f4e4e000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4815c440 con 0x7f4e481448f0 2024-04-25T10:12:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.199+0000 7f4e4e000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e480cdf30 con 0x7f4e4815ce20 2024-04-25T10:12:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.199+0000 7f4e47e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4e480cdaa0 0x7f4e4815f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35586/0 (socket says 172.21.15.5:35586) 2024-04-25T10:12:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.199+0000 7f4e47e00640 1 -- 172.21.15.5:0/4105719683 learned_addr learned my addr 172.21.15.5:0/4105719683 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e4d600640 1 --2- 172.21.15.5:0/4105719683 >> v2:172.21.15.80:3300/0 conn(0x7f4e4815ce20 0x7f4e4815d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e47400640 1 -- 172.21.15.5:0/4105719683 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911023982 0 0) 0x7f4e48144cd0 con 0x7f4e480cdaa0 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e47400640 1 -- 172.21.15.5:0/4105719683 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e30003520 con 0x7f4e480cdaa0 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e4d600640 1 -- 172.21.15.5:0/4105719683 >> v1:172.21.15.80:6789/0 conn(0x7f4e481448f0 legacy=0x7f4e4815bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e47400640 1 -- 172.21.15.5:0/4105719683 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569144913 0 0) 0x7f4e30003520 con 0x7f4e480cdaa0 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e4d600640 1 -- 172.21.15.5:0/4105719683 >> v1:172.21.15.5:6789/0 conn(0x7f4e480cdaa0 legacy=0x7f4e4815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.200+0000 7f4e4d600640 1 -- 172.21.15.5:0/4105719683 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4815c720 con 0x7f4e4815ce20 2024-04-25T10:12:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.202+0000 7f4e4d600640 1 --2- 172.21.15.5:0/4105719683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e4815d2b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009d90 tx=0x7f4e4002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfeb84b59f2a6f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.202+0000 7f4e47400640 1 -- 172.21.15.5:0/4105719683 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34001670 con 0x7f4e4815ce20 2024-04-25T10:12:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.202+0000 7f4e47400640 1 -- 172.21.15.5:0/4105719683 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40007540 con 0x7f4e4815ce20 2024-04-25T10:12:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.202+0000 7f4e47400640 1 -- 172.21.15.5:0/4105719683 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40007910 con 0x7f4e4815ce20 2024-04-25T10:12:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 -- 172.21.15.5:0/4105719683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 msgr2=0x7f4e4815d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 --2- 172.21.15.5:0/4105719683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e4815d2b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009d90 tx=0x7f4e4002a680 comp rx=0 tx=0).stop 2024-04-25T10:12:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 -- 172.21.15.5:0/4105719683 shutdown_connections 2024-04-25T10:12:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 --2- 172.21.15.5:0/4105719683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e4815d2b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 -- 172.21.15.5:0/4105719683 >> 172.21.15.5:0/4105719683 conn(0x7f4e48066f20 msgr2=0x7f4e4806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 -- 172.21.15.5:0/4105719683 shutdown_connections 2024-04-25T10:12:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.203+0000 7f4e4e000640 1 -- 172.21.15.5:0/4105719683 wait complete. 2024-04-25T10:12:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.205+0000 7f4e4e000640 1 Processor -- start 2024-04-25T10:12:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.205+0000 7f4e4e000640 1 -- start start 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 0x7f4e481f0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e481448f0 0x7f4e480b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e480b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48160000 con 0x7f4e480cdaa0 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4e000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4815f460 con 0x7f4e4815ce20 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48160180 con 0x7f4e481448f0 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 0x7f4e481f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 0x7f4e481f0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57960/0 (socket says 172.21.15.5:57960) 2024-04-25T10:12:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.206+0000 7f4e4cc00640 1 -- 172.21.15.5:0/3444662869 learned_addr learned my addr 172.21.15.5:0/3444662869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4d600640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e480b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e47e00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e481448f0 0x7f4e480b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4cc00640 1 -- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e481448f0 msgr2=0x7f4e480b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4cc00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e481448f0 0x7f4e480b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4cc00640 1 -- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 msgr2=0x7f4e480b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4cc00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e480b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4cc00640 1 -- 172.21.15.5:0/3444662869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e481307e0 con 0x7f4e480cdaa0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4d600640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e480b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.207+0000 7f4e4cc00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 0x7f4e481f0910 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00c450 tx=0x7f4e3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.208+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0052d0 con 0x7f4e480cdaa0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.208+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c005470 con 0x7f4e480cdaa0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.208+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c012870 con 0x7f4e480cdaa0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.208+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e48130a80 con 0x7f4e480cdaa0 2024-04-25T10:12:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.208+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e48131040 con 0x7f4e480cdaa0 2024-04-25T10:12:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.209+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48131240 con 0x7f4e480cdaa0 2024-04-25T10:12:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.210+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e3c012a10 con 0x7f4e480cdaa0 2024-04-25T10:12:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.211+0000 7f4e44c00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e2806b4d0 0x7f4e2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.211+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4e3c089dc0 con 0x7f4e480cdaa0 2024-04-25T10:12:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.212+0000 7f4e47e00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e2806b4d0 0x7f4e2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.213+0000 7f4e47e00640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e2806b4d0 0x7f4e2806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e340011e0 tx=0x7f4e34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.215+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e48131240 con 0x7f4e480cdaa0 2024-04-25T10:12:40.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.422+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4e48068f60 con 0x7f4e480cdaa0 2024-04-25T10:12:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.430+0000 7f4e44c00640 1 -- 172.21.15.5:0/3444662869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4e3c058cc0 con 0x7f4e480cdaa0 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e2806b4d0 msgr2=0x7f4e2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e2806b4d0 0x7f4e2806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4e340011e0 tx=0x7f4e34009040 comp rx=0 tx=0).stop 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 msgr2=0x7f4e481f0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 0x7f4e481f0910 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00c450 tx=0x7f4e3c00c910 comp rx=0 tx=0).stop 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 shutdown_connections 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e2806b4d0 0x7f4e2806d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e4815ce20 0x7f4e480b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4e481448f0 0x7f4e480b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 --2- 172.21.15.5:0/3444662869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e480cdaa0 0x7f4e481f0910 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 >> 172.21.15.5:0/3444662869 conn(0x7f4e48066f20 msgr2=0x7f4e4815f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 shutdown_connections 2024-04-25T10:12:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:40.434+0000 7f4e4e000640 1 -- 172.21.15.5:0/3444662869 wait complete. 2024-04-25T10:12:40.458 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:40.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:40.560 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:40.675 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:40.676 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:40.676 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:40.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:40.737 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:12:40.738 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:40.799 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:40.799 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:40.799 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:40.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:40.864 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:12:40.864 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:40.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:40.919 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:12:40.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:41.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.132+0000 7f3893200640 1 Processor -- start 2024-04-25T10:12:41.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.133+0000 7f3893200640 1 -- start start 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.134+0000 7f3893200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38940cdca0 0x7f389415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.134+0000 7f3893200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3894070f90 0x7f38940713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.134+0000 7f3893200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940cd200 con 0x7f3894154a10 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.134+0000 7f3893200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3894071900 con 0x7f3894070f90 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.134+0000 7f3893200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389406e6f0 con 0x7f38940cdca0 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.134+0000 7f3892800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3894070f90 0x7f38940713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3892800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3894070f90 0x7f38940713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57972/0 (socket says 172.21.15.5:57972) 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3892800640 1 -- 172.21.15.5:0/2106684141 learned_addr learned my addr 172.21.15.5:0/2106684141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:41.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3889400640 1 --2- 172.21.15.5:0/2106684141 >> v2:172.21.15.80:3300/0 conn(0x7f38940cdca0 0x7f389415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3892800640 1 -- 172.21.15.5:0/2106684141 >> v2:172.21.15.80:3300/0 conn(0x7f38940cdca0 msgr2=0x7f389415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3892800640 1 --2- 172.21.15.5:0/2106684141 >> v2:172.21.15.80:3300/0 conn(0x7f38940cdca0 0x7f389415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3892800640 1 -- 172.21.15.5:0/2106684141 >> v1:172.21.15.80:6789/0 conn(0x7f3894154a10 legacy=0x7f38940b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.135+0000 7f3892800640 1 -- 172.21.15.5:0/2106684141 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389406e9d0 con 0x7f3894070f90 2024-04-25T10:12:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.136+0000 7f3892800640 1 --2- 172.21.15.5:0/2106684141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894070f90 0x7f38940713c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f38940ccdf0 tx=0x7f388c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4975f01495f33560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.137+0000 7f3891400640 1 -- 172.21.15.5:0/2106684141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c037070 con 0x7f3894070f90 2024-04-25T10:12:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.137+0000 7f3891400640 1 -- 172.21.15.5:0/2106684141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c032440 con 0x7f3894070f90 2024-04-25T10:12:41.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.137+0000 7f3891400640 1 -- 172.21.15.5:0/2106684141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c032810 con 0x7f3894070f90 2024-04-25T10:12:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.137+0000 7f3893200640 1 -- 172.21.15.5:0/2106684141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894070f90 msgr2=0x7f38940713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.137+0000 7f3893200640 1 --2- 172.21.15.5:0/2106684141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894070f90 0x7f38940713c0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f38940ccdf0 tx=0x7f388c02a160 comp rx=0 tx=0).stop 2024-04-25T10:12:41.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.138+0000 7f3893200640 1 -- 172.21.15.5:0/2106684141 shutdown_connections 2024-04-25T10:12:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.138+0000 7f3893200640 1 --2- 172.21.15.5:0/2106684141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894070f90 0x7f38940713c0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.138+0000 7f3893200640 1 --2- 172.21.15.5:0/2106684141 >> v2:172.21.15.80:3300/0 conn(0x7f38940cdca0 0x7f389415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.138+0000 7f3893200640 1 -- 172.21.15.5:0/2106684141 >> 172.21.15.5:0/2106684141 conn(0x7f3894066f20 msgr2=0x7f389406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.138+0000 7f3893200640 1 -- 172.21.15.5:0/2106684141 shutdown_connections 2024-04-25T10:12:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.138+0000 7f3893200640 1 -- 172.21.15.5:0/2106684141 wait complete. 2024-04-25T10:12:41.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.140+0000 7f3893200640 1 Processor -- start 2024-04-25T10:12:41.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.140+0000 7f3893200640 1 -- start start 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3893200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3894070f90 0x7f3894153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3893200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940cdca0 0x7f389406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3893200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 0x7f389406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3893200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940cd780 con 0x7f3894154a10 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3893200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38940cd4a0 con 0x7f38940cdca0 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3893200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389406cff0 con 0x7f3894070f90 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3892800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 0x7f389406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3892800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 0x7f389406d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57974/0 (socket says 172.21.15.5:57974) 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3892800640 1 -- 172.21.15.5:0/3366005785 learned_addr learned my addr 172.21.15.5:0/3366005785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3889400640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940cdca0 0x7f389406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.141+0000 7f3891e00640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3894070f90 0x7f3894153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3892800640 1 -- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3894070f90 msgr2=0x7f3894153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3892800640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3894070f90 0x7f3894153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3892800640 1 -- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940cdca0 msgr2=0x7f389406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3892800640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940cdca0 0x7f389406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3892800640 1 -- 172.21.15.5:0/3366005785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3894155d10 con 0x7f3894154a10 2024-04-25T10:12:41.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3892800640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 0x7f389406d8f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f388c0090b0 tx=0x7f388c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c041020 con 0x7f3894154a10 2024-04-25T10:12:41.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c0306c0 con 0x7f3894154a10 2024-04-25T10:12:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c037040 con 0x7f3894154a10 2024-04-25T10:12:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.142+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38941f4940 con 0x7f3894154a10 2024-04-25T10:12:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.143+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941f4f00 con 0x7f3894154a10 2024-04-25T10:12:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.144+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3894068f60 con 0x7f3894154a10 2024-04-25T10:12:41.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.145+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f388c044020 con 0x7f3894154a10 2024-04-25T10:12:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.146+0000 7f388aa00640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f387006b4d0 0x7f387006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.146+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f388c0aa910 con 0x7f3894154a10 2024-04-25T10:12:41.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.146+0000 7f3891e00640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f387006b4d0 0x7f387006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.148+0000 7f3891e00640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f387006b4d0 0x7f387006d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3884006fd0 tx=0x7f3884008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.150+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f388c04b030 con 0x7f3894154a10 2024-04-25T10:12:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.390+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f389415d940 con 0x7f3894154a10 2024-04-25T10:12:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.391+0000 7f388aa00640 1 -- 172.21.15.5:0/3366005785 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f388c0798a0 con 0x7f3894154a10 2024-04-25T10:12:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:41.396 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1197199683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1197199683},{"type":"v1","addr":"172.21.15.80:6835","nonce":1197199683}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6833/1598856838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1598856838},{"type":"v1","addr":"172.21.15.5:6833","nonce":1598856838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6712,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/1339029135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":1339029135},{"type":"v1","addr":"172.21.15.80:6837","nonce":1339029135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8820,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3023415561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3023415561},{"type":"v1","addr":"172.21.15.5:6835","nonce":3023415561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[]} 2024-04-25T10:12:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 93 2024-04-25T10:12:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.394+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f387006b4d0 msgr2=0x7f387006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.394+0000 7f3893200640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f387006b4d0 0x7f387006d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3884006fd0 tx=0x7f3884008040 comp rx=0 tx=0).stop 2024-04-25T10:12:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.394+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 msgr2=0x7f389406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.394+0000 7f3893200640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 0x7f389406d8f0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f388c0090b0 tx=0x7f388c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 shutdown_connections 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3894154a10 0x7f389406d8f0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f387006b4d0 0x7f387006d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940cdca0 0x7f389406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 --2- 172.21.15.5:0/3366005785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3894070f90 0x7f3894153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 >> 172.21.15.5:0/3366005785 conn(0x7f3894066f20 msgr2=0x7f38940afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 shutdown_connections 2024-04-25T10:12:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.395+0000 7f3893200640 1 -- 172.21.15.5:0/3366005785 wait complete. 2024-04-25T10:12:41.420 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:12:41.420 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:12:41.420 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:12:41.420 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:41.421 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:12:41.421 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:12:41.421 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:12:41.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:41.421 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:12:41.421 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:12:41.422 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:41.422 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:12:41.422 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:12:41.422 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:12:41.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:41.441 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:12:41.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:12:41.442 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:12:41.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:12:41.444 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:12:41.444 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:12:41.446 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:12:41.447 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:12:41.488 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:12:41.488 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:41.493 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:12:41.494 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:41.494 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:12:41.494 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:41.500 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:12:41.500 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:41.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:12:41.641 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-04-25T10:12:41.641 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:12:41.642 INFO:tasks.ceph.mds.d.smithi080.stdout:starting mds.d at 2024-04-25T10:12:41.645 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:12:41.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.677+0000 7f6498000640 1 Processor -- start 2024-04-25T10:12:41.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.678+0000 7f6498000640 1 -- start start 2024-04-25T10:12:41.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.679+0000 7f6498000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6490144100 0x7f64900cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.679+0000 7f6498000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900cdbf0 con 0x7f64900ced20 2024-04-25T10:12:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.679+0000 7f6498000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900cded0 con 0x7f649006e970 2024-04-25T10:12:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.679+0000 7f6498000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64901444e0 con 0x7f6490144100 2024-04-25T10:12:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.679+0000 7f6494c00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f64900ced20 0x7f649006e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35604/0 (socket says 172.21.15.5:35604) 2024-04-25T10:12:41.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.679+0000 7f6494c00640 1 -- 172.21.15.5:0/2038928202 learned_addr learned my addr 172.21.15.5:0/2038928202 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f6495600640 1 --2- 172.21.15.5:0/2038928202 >> v2:172.21.15.80:3300/0 conn(0x7f6490144100 0x7f64900cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966895709 0 0) 0x7f64900cdbf0 con 0x7f64900ced20 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6478003520 con 0x7f64900ced20 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288872652 0 0) 0x7f64900cded0 con 0x7f649006e970 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64900cdbf0 con 0x7f649006e970 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f6495600640 1 -- 172.21.15.5:0/2038928202 >> v1:172.21.15.80:6789/0 conn(0x7f649006e970 legacy=0x7f649015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f6495600640 1 -- 172.21.15.5:0/2038928202 >> v1:172.21.15.5:6789/0 conn(0x7f64900ced20 legacy=0x7f649006e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.680+0000 7f6495600640 1 -- 172.21.15.5:0/2038928202 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900ce1b0 con 0x7f6490144100 2024-04-25T10:12:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.681+0000 7f6495600640 1 --2- 172.21.15.5:0/2038928202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6490144100 0x7f64900cd610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f64840097a0 tx=0x7f648402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f742dfb2b1e761f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484037070 con 0x7f6490144100 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6484032440 con 0x7f6490144100 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f648fe00640 1 -- 172.21.15.5:0/2038928202 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484032810 con 0x7f6490144100 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 -- 172.21.15.5:0/2038928202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6490144100 msgr2=0x7f64900cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 --2- 172.21.15.5:0/2038928202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6490144100 0x7f64900cd610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f64840097a0 tx=0x7f648402a230 comp rx=0 tx=0).stop 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 -- 172.21.15.5:0/2038928202 shutdown_connections 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 --2- 172.21.15.5:0/2038928202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6490144100 0x7f64900cd610 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 -- 172.21.15.5:0/2038928202 >> 172.21.15.5:0/2038928202 conn(0x7f6490071150 msgr2=0x7f6490071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 -- 172.21.15.5:0/2038928202 shutdown_connections 2024-04-25T10:12:41.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.682+0000 7f6498000640 1 -- 172.21.15.5:0/2038928202 wait complete. 2024-04-25T10:12:41.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.684+0000 7f6498000640 1 Processor -- start 2024-04-25T10:12:41.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.684+0000 7f6498000640 1 -- start start 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.685+0000 7f6498000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f649006e970 0x7f649015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.685+0000 7f6498000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64900ced20 0x7f64900c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.685+0000 7f6498000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 0x7f64901e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.685+0000 7f6498000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64900cea00 con 0x7f6490144100 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.685+0000 7f6498000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649015ac70 con 0x7f64900ced20 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.685+0000 7f6498000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649015b840 con 0x7f649006e970 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 0x7f64901e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 0x7f64901e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58018/0 (socket says 172.21.15.5:58018) 2024-04-25T10:12:41.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 -- 172.21.15.5:0/1059867260 learned_addr learned my addr 172.21.15.5:0/1059867260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6494c00640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64900ced20 0x7f64900c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6495600640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f649006e970 0x7f649015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 -- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f649006e970 msgr2=0x7f649015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f649006e970 0x7f649015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 -- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64900ced20 msgr2=0x7f64900c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64900ced20 0x7f64900c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.686+0000 7f6496000640 1 -- 172.21.15.5:0/1059867260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900c9cc0 con 0x7f6490144100 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.687+0000 7f6496000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 0x7f64901e12c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f648800c730 tx=0x7f648800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.687+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488003b70 con 0x7f6490144100 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.687+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488003d10 con 0x7f6490144100 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.687+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64900c8c20 con 0x7f6490144100 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.687+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800aa90 con 0x7f6490144100 2024-04-25T10:12:41.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.687+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64900c91e0 con 0x7f6490144100 2024-04-25T10:12:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.688+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6490068fd0 con 0x7f6490144100 2024-04-25T10:12:41.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.689+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f648800ac30 con 0x7f6490144100 2024-04-25T10:12:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.691+0000 7f648d600640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f647006b550 0x7f647006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.691+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6488089fa0 con 0x7f6490144100 2024-04-25T10:12:41.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.691+0000 7f6495600640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f647006b550 0x7f647006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.692+0000 7f6495600640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f647006b550 0x7f647006da10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6484034000 tx=0x7f648402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.696+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6488058fb0 con 0x7f6490144100 2024-04-25T10:12:41.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:41.913+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6490141c90 con 0x7f6490144100 2024-04-25T10:12:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.993+0000 7f648d600640 1 -- 172.21.15.5:0/1059867260 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7f64880588b0 con 0x7f6490144100 2024-04-25T10:12:42.998 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-04-25T10:12:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f647006b550 msgr2=0x7f647006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f647006b550 0x7f647006da10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6484034000 tx=0x7f648402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 msgr2=0x7f64901e12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 0x7f64901e12c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f648800c730 tx=0x7f648800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 shutdown_connections 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6490144100 0x7f64901e12c0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f647006b550 0x7f647006da10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64900ced20 0x7f64900c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 --2- 172.21.15.5:0/1059867260 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f649006e970 0x7f649015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 >> 172.21.15.5:0/1059867260 conn(0x7f6490071150 msgr2=0x7f649006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 shutdown_connections 2024-04-25T10:12:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:42.998+0000 7f6498000640 1 -- 172.21.15.5:0/1059867260 wait complete. 2024-04-25T10:12:43.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:12:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.225+0000 7fd25be00640 1 Processor -- start 2024-04-25T10:12:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.225+0000 7fd25be00640 1 -- start start 2024-04-25T10:12:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.226+0000 7fd25be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 0x7fd25c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.226+0000 7fd25be00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd25c0d0570 0x7fd25c0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.226+0000 7fd25be00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0d23f0 con 0x7fd25c15e4c0 2024-04-25T10:12:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.226+0000 7fd25be00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c0cea20 con 0x7fd25c0d0570 2024-04-25T10:12:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.226+0000 7fd25be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0ce6f0 con 0x7fd25c0d1590 2024-04-25T10:12:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.227+0000 7fd25b400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd25c15e4c0 0x7fd25c0b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35616/0 (socket says 172.21.15.5:35616) 2024-04-25T10:12:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.227+0000 7fd25b400640 1 -- 172.21.15.5:0/2231821761 learned_addr learned my addr 172.21.15.5:0/2231821761 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.227+0000 7fd25aa00640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 0x7fd25c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.227+0000 7fd252000640 1 --2- 172.21.15.5:0/2231821761 >> v2:172.21.15.80:3300/0 conn(0x7fd25c0d0570 0x7fd25c0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593876837 0 0) 0x7fd25c0d23f0 con 0x7fd25c15e4c0 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd240003520 con 0x7fd25c15e4c0 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3981378768 0 0) 0x7fd240003520 con 0x7fd25c15e4c0 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 msgr2=0x7fd25c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 0x7fd25c0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 >> v2:172.21.15.80:3300/0 conn(0x7fd25c0d0570 msgr2=0x7fd25c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 --2- 172.21.15.5:0/2231821761 >> v2:172.21.15.80:3300/0 conn(0x7fd25c0d0570 0x7fd25c0d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c15ea10 con 0x7fd25c15e4c0 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fd2540019f0 con 0x7fd25c15e4c0 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 >> v1:172.21.15.5:6789/0 conn(0x7fd25c15e4c0 legacy=0x7fd25c0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 0x7fd2400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd240003f30 0x7fd240004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd2400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0cea20 con 0x7fd240004d40 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0ce6f0 con 0x7fd240003f30 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.228+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24000b9c0 con 0x7fd25c0d1590 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25b400640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd2400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25aa00640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 0x7fd2400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd252000640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd240003f30 0x7fd240004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25b400640 1 -- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 msgr2=0x7fd2400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25b400640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c0d1590 0x7fd2400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25b400640 1 -- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd240003f30 msgr2=0x7fd240004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25b400640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd240003f30 0x7fd240004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.229+0000 7fd25b400640 1 -- 172.21.15.5:0/2231821761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240003520 con 0x7fd240004d40 2024-04-25T10:12:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.230+0000 7fd25b400640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd2400051c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd25400a870 tx=0x7fd240005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28d44a686fa63ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.230+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400a4c0 con 0x7fd240004d40 2024-04-25T10:12:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.230+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25402f070 con 0x7fd240004d40 2024-04-25T10:12:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.230+0000 7fd25a000640 1 -- 172.21.15.5:0/2231821761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25402bcb0 con 0x7fd240004d40 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25be00640 1 -- 172.21.15.5:0/2231821761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 msgr2=0x7fd2400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25be00640 1 --2- 172.21.15.5:0/2231821761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd2400051c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fd25400a870 tx=0x7fd240005830 comp rx=0 tx=0).stop 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25b400640 1 -- 172.21.15.5:0/2231821761 reap_dead start 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25be00640 1 -- 172.21.15.5:0/2231821761 shutdown_connections 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25be00640 1 -- 172.21.15.5:0/2231821761 >> 172.21.15.5:0/2231821761 conn(0x7fd25c0726e0 msgr2=0x7fd25c072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25be00640 1 -- 172.21.15.5:0/2231821761 shutdown_connections 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.231+0000 7fd25be00640 1 -- 172.21.15.5:0/2231821761 wait complete. 2024-04-25T10:12:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.233+0000 7fd25be00640 1 Processor -- start 2024-04-25T10:12:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.233+0000 7fd25be00640 1 -- start start 2024-04-25T10:12:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.234+0000 7fd25be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd25c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.234+0000 7fd25be00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd25c14cce0 0x7fd25c0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.234+0000 7fd25be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c1f2f10 0x7fd25c1f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.234+0000 7fd25be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0d2950 con 0x7fd240004d40 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.234+0000 7fd25be00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0d27d0 con 0x7fd25c14cce0 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.234+0000 7fd25be00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0d3040 con 0x7fd25c1f2f10 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd25c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd25c0b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58050/0 (socket says 172.21.15.5:58050) 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 -- 172.21.15.5:0/528792562 learned_addr learned my addr 172.21.15.5:0/528792562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd252000640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd25c14cce0 0x7fd25c0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 -- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c1f2f10 msgr2=0x7fd25c1f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25b400640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c1f2f10 0x7fd25c1f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c1f2f10 0x7fd25c1f3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 -- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd25c14cce0 msgr2=0x7fd25c0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd25c14cce0 0x7fd25c0b54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd25aa00640 1 -- 172.21.15.5:0/528792562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c1f38d0 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.235+0000 7fd252000640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd25c14cce0 0x7fd25c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.236+0000 7fd25aa00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd25c0b3050 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fd24c011630 tx=0x7fd24c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.236+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00f920 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.236+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c00fac0 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.236+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00fe70 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.236+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0b36e0 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.236+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0b3ca0 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.237+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c068080 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.238+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd24c015460 con 0x7fd240004d40 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.239+0000 7fd253400640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd23806b500 0x7fd23806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.239+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd24c08a620 con 0x7fd240004d40 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.240+0000 7fd252000640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd23806b500 0x7fd23806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.241+0000 7fd252000640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd23806b500 0x7fd23806d9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd25c15b2a0 tx=0x7fd244008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.244+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd24c019030 con 0x7fd240004d40 2024-04-25T10:12:43.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.516+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd25c068f60 con 0x7fd240004d40 2024-04-25T10:12:43.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.517+0000 7fd253400640 1 -- 172.21.15.5:0/528792562 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd24c0595c0 con 0x7fd240004d40 2024-04-25T10:12:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:43.521 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:12:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-04-25T10:12:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpm9pmNBRAAfsJpSt5gGRiJ6KRkd3zAvQ==", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmVGRYIxAA9OA/i9PYwpCgFGpzcloqzg==", 2024-04-25T10:12:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmijjNFRAA20v5NEECa4NvtqG88tUQrg==", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:12:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDYKSpmtvV0LxAAux2Jr6UHUOh2N3jJ3FRA2Q==", 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDbKSpm+AD6DhAAs04IlsFrdWeEN7eYCtqPgA==", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDeKSpmirNMGxAA4Y0tvCg2lwEFU2U7s3fl8g==", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDhKSpmTb4MKRAAU9NgHXagJNzA9TvQXGN72g==", 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDkKSpmorkfNBAAli120DZ9DW+6BJzZc8/zgg==", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDrKSpmUwNyCBAAQTDLKUdkiZ8uw3rQw2jSLw==", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDuKSpmqfrzIRAAHKahuEDwQ4AxYtx70+DxUA==", 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDxKSpmc2vBKxAAVrjnQsxTXe1Z7/22cR3aNQ==", 2024-04-25T10:12:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQD0KSpm5gvnNhAAaNX2tHflba9FtFKxY8nDJg==", 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg==", 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow r", 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmkQ0UFBAAL8YrcJcaBiyx/svPVm8vWg==", 2024-04-25T10:12:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmxpOjCRAA1ncDrZgz8b1mcOfJNy/Scw==", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:12:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDZKSpmWE5rHRAAtrAHEHoQAHKezwLh1nSxRQ==", 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm0A+EAhAAA/koJ9/RHxQI1G1WTiWxmw==", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpm29UNNhAAFKgQZYgoPuwOs7pZMy/ZMg==", 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQDXKSpmTmkPLBAAigyA3SsngJQYGl4N2qqBpg==", 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-04-25T10:12:43.535 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd23806b500 msgr2=0x7fd23806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd23806b500 0x7fd23806d9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd25c15b2a0 tx=0x7fd244008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 msgr2=0x7fd25c0b3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd25c0b3050 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fd24c011630 tx=0x7fd24c011af0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 shutdown_connections 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd25c1f2f10 0x7fd25c1f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd23806b500 0x7fd23806d9c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd25c14cce0 0x7fd25c0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 --2- 172.21.15.5:0/528792562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240004d40 0x7fd25c0b3050 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 >> 172.21.15.5:0/528792562 conn(0x7fd25c0726e0 msgr2=0x7fd25c0b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 shutdown_connections 2024-04-25T10:12:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.520+0000 7fd25be00640 1 -- 172.21.15.5:0/528792562 wait complete. 2024-04-25T10:12:43.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.752+0000 7fe8b4e00640 1 Processor -- start 2024-04-25T10:12:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.752+0000 7fe8b4e00640 1 -- start start 2024-04-25T10:12:43.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.753+0000 7fe8b4e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 0x7fe8b00d2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.753+0000 7fe8b4e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00d1e80 con 0x7fe8b015db00 2024-04-25T10:12:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.753+0000 7fe8b4e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00b5930 con 0x7fe8b00b53b0 2024-04-25T10:12:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.753+0000 7fe8b4e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b5c10 con 0x7fe8b00d0a50 2024-04-25T10:12:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.753+0000 7fe8ade00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe8b015db00 0x7fe8b00d0340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35624/0 (socket says 172.21.15.5:35624) 2024-04-25T10:12:43.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.753+0000 7fe8ade00640 1 -- 172.21.15.5:0/2284120401 learned_addr learned my addr 172.21.15.5:0/2284120401 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.754+0000 7fe8ad400640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 0x7fe8b00d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.754+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482771436 0 0) 0x7fe8b00d1e80 con 0x7fe8b015db00 2024-04-25T10:12:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.754+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe890003520 con 0x7fe8b015db00 2024-04-25T10:12:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.754+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 936178608 0 0) 0x7fe890003520 con 0x7fe8b015db00 2024-04-25T10:12:43.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 msgr2=0x7fe8b00d2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 0x7fe8b00d2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 >> v1:172.21.15.80:6789/0 conn(0x7fe8b00b53b0 legacy=0x7fe8b00d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00b5ef0 con 0x7fe8b015db00 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe8a0001670 con 0x7fe8b015db00 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 >> v1:172.21.15.5:6789/0 conn(0x7fe8b015db00 legacy=0x7fe8b00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 0x7fe8900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe890003fb0 0x7fe890004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b5c10 con 0x7fe890004d40 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89000bb00 con 0x7fe890003fb0 2024-04-25T10:12:43.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89000bd00 con 0x7fe8b00d0a50 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8ae800640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.755+0000 7fe8ade00640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 0x7fe8900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.756+0000 7fe8ad400640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe890003fb0 0x7fe890004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.756+0000 7fe8ae800640 1 -- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 msgr2=0x7fe8900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.756+0000 7fe8ae800640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00d0a50 0x7fe8900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.756+0000 7fe8ae800640 1 -- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe890003fb0 msgr2=0x7fe890004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.756+0000 7fe8ae800640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe890003fb0 0x7fe890004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.756+0000 7fe8ae800640 1 -- 172.21.15.5:0/2284120401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890003520 con 0x7fe890004d40 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.757+0000 7fe8ae800640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8001e20 tx=0x7fe890005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a968c80998f56f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.757+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8001a70 con 0x7fe890004d40 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.757+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a802ae40 con 0x7fe890004d40 2024-04-25T10:12:43.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.757+0000 7fe8aca00640 1 -- 172.21.15.5:0/2284120401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8003bb0 con 0x7fe890004d40 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.757+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2284120401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 msgr2=0x7fe8900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.758+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2284120401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8900051c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8001e20 tx=0x7fe890005830 comp rx=0 tx=0).stop 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.758+0000 7fe8ae800640 1 -- 172.21.15.5:0/2284120401 reap_dead start 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.758+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2284120401 shutdown_connections 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.758+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2284120401 >> 172.21.15.5:0/2284120401 conn(0x7fe8b0072290 msgr2=0x7fe8b00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.758+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2284120401 shutdown_connections 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.758+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2284120401 wait complete. 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.760+0000 7fe8b4e00640 1 Processor -- start 2024-04-25T10:12:43.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.760+0000 7fe8b4e00640 1 -- start start 2024-04-25T10:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8b4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8b014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8b4e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00cd1f0 0x7fe8b015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8b4e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b014eda0 0x7fe8b00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8b4e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d3660 con 0x7fe890004d40 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8b4e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b61d0 con 0x7fe8b014eda0 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8b4e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00d2f80 con 0x7fe8b00cd1f0 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8b014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8b014e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58080/0 (socket says 172.21.15.5:58080) 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 -- 172.21.15.5:0/2091220176 learned_addr learned my addr 172.21.15.5:0/2091220176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 -- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00cd1f0 msgr2=0x7fe8b015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00cd1f0 0x7fe8b015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 -- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b014eda0 msgr2=0x7fe8b00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b014eda0 0x7fe8b00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.762+0000 7fe8ade00640 1 -- 172.21.15.5:0/2091220176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00cd6f0 con 0x7fe890004d40 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.763+0000 7fe8ade00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8b014e780 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0001050 tx=0x7fe8a000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.763+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000ee50 con 0x7fe890004d40 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.763+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0018070 con 0x7fe890004d40 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.763+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000cb10 con 0x7fe890004d40 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.763+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b00c76b0 con 0x7fe890004d40 2024-04-25T10:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.763+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b014f410 con 0x7fe890004d40 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.764+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0068fd0 con 0x7fe890004d40 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.765+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8a000ccb0 con 0x7fe890004d40 2024-04-25T10:12:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.766+0000 7fe8a6000640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88c06b6f0 0x7fe88c06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.766+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe8a0089f80 con 0x7fe890004d40 2024-04-25T10:12:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.767+0000 7fe8ad400640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88c06b6f0 0x7fe88c06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.768+0000 7fe8ad400640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88c06b6f0 0x7fe88c06dbb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe8b01f7000 tx=0x7fe898008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.771+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8a0058fa0 con 0x7fe890004d40 2024-04-25T10:12:43.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.973+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8b00cf770 con 0x7fe890004d40 2024-04-25T10:12:43.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.976+0000 7fe8a6000640 1 -- 172.21.15.5:0/2091220176 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13066 (secure 0 0 0) 0x7fe8a00588a0 con 0x7fe890004d40 2024-04-25T10:12:43.980 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:43.980 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":116,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:42.990322+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88c06b6f0 msgr2=0x7fe88c06dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88c06b6f0 0x7fe88c06dbb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe8b01f7000 tx=0x7fe898008040 comp rx=0 tx=0).stop 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 msgr2=0x7fe8b014e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8b014e780 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0001050 tx=0x7fe8a000ec60 comp rx=0 tx=0).stop 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 shutdown_connections 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe88c06b6f0 0x7fe88c06dbb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8b014eda0 0x7fe8b00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8b00cd1f0 0x7fe8b015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 --2- 172.21.15.5:0/2091220176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe890004d40 0x7fe8b014e780 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 >> 172.21.15.5:0/2091220176 conn(0x7fe8b0072290 msgr2=0x7fe8b0151550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 shutdown_connections 2024-04-25T10:12:43.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:43.979+0000 7fe8b4e00640 1 -- 172.21.15.5:0/2091220176 wait complete. 2024-04-25T10:12:44.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:44.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.218+0000 7f9165c00640 1 Processor -- start 2024-04-25T10:12:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.218+0000 7f9165c00640 1 -- start start 2024-04-25T10:12:44.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f9165c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9160144fe0 0x7f916006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f9165c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9160071840 0x7f916015c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f9165c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91601453c0 con 0x7f916006d130 2024-04-25T10:12:44.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f9165c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91600a8560 con 0x7f9160144fe0 2024-04-25T10:12:44.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f9165c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916006d6a0 con 0x7f9160071840 2024-04-25T10:12:44.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f915e000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f916006d130 0x7f9160071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35636/0 (socket says 172.21.15.5:35636) 2024-04-25T10:12:44.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.220+0000 7f915e000640 1 -- 172.21.15.5:0/1461857790 learned_addr learned my addr 172.21.15.5:0/1461857790 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915ea00640 1 --2- 172.21.15.5:0/1461857790 >> v2:172.21.15.80:3300/0 conn(0x7f9160144fe0 0x7f916006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915f400640 1 --2- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9160071840 0x7f916015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915d600640 1 -- 172.21.15.5:0/1461857790 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854752614 0 0) 0x7f91601453c0 con 0x7f916006d130 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915d600640 1 -- 172.21.15.5:0/1461857790 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9144003520 con 0x7f916006d130 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915ea00640 1 -- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9160071840 msgr2=0x7f916015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915ea00640 1 --2- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9160071840 0x7f916015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915ea00640 1 -- 172.21.15.5:0/1461857790 >> v1:172.21.15.5:6789/0 conn(0x7f916006d130 legacy=0x7f9160071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.221+0000 7f915ea00640 1 -- 172.21.15.5:0/1461857790 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916006d980 con 0x7f9160144fe0 2024-04-25T10:12:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.223+0000 7f915ea00640 1 --2- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f916006e560 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f91480097a0 tx=0x7f914802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5bf62ae190ee03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.223+0000 7f915d600640 1 -- 172.21.15.5:0/1461857790 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148037070 con 0x7f9160144fe0 2024-04-25T10:12:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.223+0000 7f915d600640 1 -- 172.21.15.5:0/1461857790 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9148032440 con 0x7f9160144fe0 2024-04-25T10:12:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.223+0000 7f915d600640 1 -- 172.21.15.5:0/1461857790 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148032810 con 0x7f9160144fe0 2024-04-25T10:12:44.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 -- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 msgr2=0x7f916006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 --2- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f916006e560 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f91480097a0 tx=0x7f914802a180 comp rx=0 tx=0).stop 2024-04-25T10:12:44.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 -- 172.21.15.5:0/1461857790 shutdown_connections 2024-04-25T10:12:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 --2- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9160071840 0x7f916015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 --2- 172.21.15.5:0/1461857790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f916006e560 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 -- 172.21.15.5:0/1461857790 >> 172.21.15.5:0/1461857790 conn(0x7f916006ab60 msgr2=0x7f916006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 -- 172.21.15.5:0/1461857790 shutdown_connections 2024-04-25T10:12:44.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.224+0000 7f9165c00640 1 -- 172.21.15.5:0/1461857790 wait complete. 2024-04-25T10:12:44.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.226+0000 7f9165c00640 1 Processor -- start 2024-04-25T10:12:44.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.226+0000 7f9165c00640 1 -- start start 2024-04-25T10:12:44.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f9165c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f916006d130 0x7f9160142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f9165c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 0x7f91601f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f9165c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f91601e3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f9165c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916015be50 con 0x7f9160071840 2024-04-25T10:12:44.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f9165c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916006df40 con 0x7f9160144fe0 2024-04-25T10:12:44.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f9165c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916006eaa0 con 0x7f916006d130 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f915e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 0x7f91601f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f915e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 0x7f91601f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58096/0 (socket says 172.21.15.5:58096) 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.227+0000 7f915e000640 1 -- 172.21.15.5:0/1943598498 learned_addr learned my addr 172.21.15.5:0/1943598498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915f400640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f91601e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915ea00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f916006d130 0x7f9160142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915e000640 1 -- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f916006d130 msgr2=0x7f9160142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915e000640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f916006d130 0x7f9160142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915e000640 1 -- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 msgr2=0x7f91601e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915e000640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f91601e3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915e000640 1 -- 172.21.15.5:0/1943598498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91601e2080 con 0x7f9160071840 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.228+0000 7f915e000640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 0x7f91601f48d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f915000cb40 tx=0x7f915000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.229+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150017070 con 0x7f9160071840 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.229+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915000aca0 con 0x7f9160071840 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.229+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150012620 con 0x7f9160071840 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.229+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91601e2340 con 0x7f9160071840 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.229+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91601e2900 con 0x7f9160071840 2024-04-25T10:12:44.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.230+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9160068f60 con 0x7f9160071840 2024-04-25T10:12:44.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.231+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f915001a070 con 0x7f9160071840 2024-04-25T10:12:44.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.232+0000 7f9156a00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f913c06b4d0 0x7f913c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.232+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9150089e90 con 0x7f9160071840 2024-04-25T10:12:44.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.233+0000 7f915ea00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f913c06b4d0 0x7f913c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.234+0000 7f915ea00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f913c06b4d0 0x7f913c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9148034000 tx=0x7f914802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.236+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91500587e0 con 0x7f9160071840 2024-04-25T10:12:44.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.437+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91600af7e0 con 0x7f9160071840 2024-04-25T10:12:44.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.438+0000 7f9156a00640 1 -- 172.21.15.5:0/1943598498 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f915008b020 con 0x7f9160071840 2024-04-25T10:12:44.442 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f913c06b4d0 msgr2=0x7f913c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f913c06b4d0 0x7f913c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9148034000 tx=0x7f914802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 msgr2=0x7f91601f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 0x7f91601f48d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f915000cb40 tx=0x7f915000a060 comp rx=0 tx=0).stop 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 shutdown_connections 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f913c06b4d0 0x7f913c06d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9160144fe0 0x7f91601e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9160071840 0x7f91601f48d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 --2- 172.21.15.5:0/1943598498 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f916006d130 0x7f9160142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 >> 172.21.15.5:0/1943598498 conn(0x7f916006ab60 msgr2=0x7f9160144010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 shutdown_connections 2024-04-25T10:12:44.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.441+0000 7f9165c00640 1 -- 172.21.15.5:0/1943598498 wait complete. 2024-04-25T10:12:44.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:12:44.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.668+0000 7ff6d0a00640 1 Processor -- start 2024-04-25T10:12:44.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.668+0000 7ff6d0a00640 1 -- start start 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.670+0000 7ff6d0a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff6cc0cc460 0x7ff6cc15c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.670+0000 7ff6d0a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc06d2b0 con 0x7ff6cc06d930 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.670+0000 7ff6d0a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc15f310 con 0x7ff6cc141da0 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.670+0000 7ff6d0a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc142180 con 0x7ff6cc0cc460 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.670+0000 7ff6c2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff6cc0cc460 0x7ff6cc15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.671+0000 7ff6cbe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff6cc06d930 0x7ff6cc0cd6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35652/0 (socket says 172.21.15.5:35652) 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.671+0000 7ff6cbe00640 1 -- 172.21.15.5:0/3223818211 learned_addr learned my addr 172.21.15.5:0/3223818211 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:44.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.671+0000 7ff6c2a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff6cc0cc460 0x7ff6cc15c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58106/0 (socket says 172.21.15.5:58106) 2024-04-25T10:12:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.671+0000 7ff6c2a00640 1 -- 172.21.15.5:0/3223818211 >> v1:172.21.15.80:6789/0 conn(0x7ff6cc141da0 legacy=0x7ff6cc0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.671+0000 7ff6c2a00640 1 -- 172.21.15.5:0/3223818211 >> v1:172.21.15.5:6789/0 conn(0x7ff6cc06d930 legacy=0x7ff6cc0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.671+0000 7ff6c2a00640 1 -- 172.21.15.5:0/3223818211 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc15f5f0 con 0x7ff6cc0cc460 2024-04-25T10:12:44.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.672+0000 7ff6c2a00640 1 --2- 172.21.15.5:0/3223818211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc15c720 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc142300 tx=0x7ff6b402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd75d7b491a9de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.672+0000 7ff6caa00640 1 -- 172.21.15.5:0/3223818211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4037070 con 0x7ff6cc0cc460 2024-04-25T10:12:44.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6caa00640 1 -- 172.21.15.5:0/3223818211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b4032440 con 0x7ff6cc0cc460 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6caa00640 1 -- 172.21.15.5:0/3223818211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4032810 con 0x7ff6cc0cc460 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3223818211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc0cc460 msgr2=0x7ff6cc15c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3223818211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc15c720 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc142300 tx=0x7ff6b402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3223818211 shutdown_connections 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3223818211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc15c720 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3223818211 >> 172.21.15.5:0/3223818211 conn(0x7ff6cc0b4f00 msgr2=0x7ff6cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3223818211 shutdown_connections 2024-04-25T10:12:44.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.673+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3223818211 wait complete. 2024-04-25T10:12:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.675+0000 7ff6d0a00640 1 Processor -- start 2024-04-25T10:12:44.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.675+0000 7ff6d0a00640 1 -- start start 2024-04-25T10:12:44.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6d0a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6cc06d930 0x7ff6cc13eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6d0a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc0b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6d0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 0x7ff6cc0adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6d0a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15fe90 con 0x7ff6cc141da0 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6d0a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15fbb0 con 0x7ff6cc0cc460 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6d0a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cddd0 con 0x7ff6cc06d930 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.676+0000 7ff6cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 0x7ff6cc0adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 0x7ff6cc0adc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58122/0 (socket says 172.21.15.5:58122) 2024-04-25T10:12:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 -- 172.21.15.5:0/3766803673 learned_addr learned my addr 172.21.15.5:0/3766803673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6c2a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc0b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cb400640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6cc06d930 0x7ff6cc13eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 -- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6cc06d930 msgr2=0x7ff6cc13eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6cc06d930 0x7ff6cc13eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 -- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6cc0cc460 msgr2=0x7ff6cc0b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc0b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 -- 172.21.15.5:0/3766803673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc1f7a90 con 0x7ff6cc141da0 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6cbe00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 0x7ff6cc0adc10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400cbc0 tx=0x7ff6c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.677+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4017070 con 0x7ff6cc141da0 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.678+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c400ad40 con 0x7ff6cc141da0 2024-04-25T10:12:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.678+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4012600 con 0x7ff6cc141da0 2024-04-25T10:12:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.678+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc1f77b0 con 0x7ff6cc141da0 2024-04-25T10:12:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.678+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc13e310 con 0x7ff6cc141da0 2024-04-25T10:12:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.679+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc068f60 con 0x7ff6cc141da0 2024-04-25T10:12:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.680+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6c401a070 con 0x7ff6cc141da0 2024-04-25T10:12:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.681+0000 7ff6c3e00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff6a806b4d0 0x7ff6a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.681+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff6c4089c20 con 0x7ff6cc141da0 2024-04-25T10:12:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.682+0000 7ff6cb400640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff6a806b4d0 0x7ff6a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.683+0000 7ff6cb400640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff6a806b4d0 0x7ff6a806d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00a750 tx=0x7ff6bc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.686+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6cc06d2b0 con 0x7ff6cc141da0 2024-04-25T10:12:44.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.888+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff6cc13f4a0 con 0x7ff6cc141da0 2024-04-25T10:12:44.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.889+0000 7ff6c3e00640 1 -- 172.21.15.5:0/3766803673 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff6c40586a0 con 0x7ff6cc141da0 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-04-25T10:12:44.894 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-04-25T10:12:44.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff6a806b4d0 msgr2=0x7ff6a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff6a806b4d0 0x7ff6a806d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00a750 tx=0x7ff6bc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:44.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 msgr2=0x7ff6cc0adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 0x7ff6cc0adc10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400cbc0 tx=0x7ff6c400a430 comp rx=0 tx=0).stop 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 shutdown_connections 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6cc141da0 0x7ff6cc0adc10 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff6a806b4d0 0x7ff6a806d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6cc0cc460 0x7ff6cc0b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 --2- 172.21.15.5:0/3766803673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff6cc06d930 0x7ff6cc13eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 >> 172.21.15.5:0/3766803673 conn(0x7ff6cc0b4f00 msgr2=0x7ff6cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 shutdown_connections 2024-04-25T10:12:44.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:44.892+0000 7ff6d0a00640 1 -- 172.21.15.5:0/3766803673 wait complete. 2024-04-25T10:12:44.916 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:12:44.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:12:45.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.122+0000 7f73d1200640 1 Processor -- start 2024-04-25T10:12:45.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.122+0000 7f73d1200640 1 -- start start 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73d1200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f73cc142010 0x7f73cc06e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73d1200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f73cc06ed60 0x7f73cc0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73d1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73d1200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73cc1423f0 con 0x7f73cc06ed60 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73d1200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73cc0cc210 con 0x7f73cc142010 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73d1200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc06d780 con 0x7f73cc0cce30 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f73cc06ed60 0x7f73cc0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f73cc06ed60 0x7f73cc0cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58124/0 (socket says 172.21.15.5:58124) 2024-04-25T10:12:45.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c1400640 1 -- 172.21.15.5:0/4093729206 learned_addr learned my addr 172.21.15.5:0/4093729206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c9e00640 1 --2- 172.21.15.5:0/4093729206 >> v2:172.21.15.80:3300/0 conn(0x7f73cc142010 0x7f73cc06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c1400640 1 -- 172.21.15.5:0/4093729206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 msgr2=0x7f73cc1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c1400640 1 --2- 172.21.15.5:0/4093729206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.124+0000 7f73c1400640 1 -- 172.21.15.5:0/4093729206 >> v2:172.21.15.80:3300/0 conn(0x7f73cc142010 msgr2=0x7f73cc06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.125+0000 7f73c1400640 1 --2- 172.21.15.5:0/4093729206 >> v2:172.21.15.80:3300/0 conn(0x7f73cc142010 0x7f73cc06e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.125+0000 7f73c1400640 1 -- 172.21.15.5:0/4093729206 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc06da60 con 0x7f73cc06ed60 2024-04-25T10:12:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.125+0000 7f73c9e00640 1 --2- 172.21.15.5:0/4093729206 >> v2:172.21.15.80:3300/0 conn(0x7f73cc142010 0x7f73cc06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:45.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.126+0000 7f73c1400640 1 --2- 172.21.15.5:0/4093729206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc0cc8f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f73cc06d990 tx=0x7f73b402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424f09571ca182c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.126+0000 7f73c9400640 1 -- 172.21.15.5:0/4093729206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4037070 con 0x7f73cc06ed60 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.126+0000 7f73c9400640 1 -- 172.21.15.5:0/4093729206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b4032440 con 0x7f73cc06ed60 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.126+0000 7f73c9400640 1 -- 172.21.15.5:0/4093729206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4032810 con 0x7f73cc06ed60 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 -- 172.21.15.5:0/4093729206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 msgr2=0x7f73cc0cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 --2- 172.21.15.5:0/4093729206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc0cc8f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f73cc06d990 tx=0x7f73b402a130 comp rx=0 tx=0).stop 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 -- 172.21.15.5:0/4093729206 shutdown_connections 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 --2- 172.21.15.5:0/4093729206 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 --2- 172.21.15.5:0/4093729206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc0cc8f0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 --2- 172.21.15.5:0/4093729206 >> v2:172.21.15.80:3300/0 conn(0x7f73cc142010 0x7f73cc06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 -- 172.21.15.5:0/4093729206 >> 172.21.15.5:0/4093729206 conn(0x7f73cc066f80 msgr2=0x7f73cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 -- 172.21.15.5:0/4093729206 shutdown_connections 2024-04-25T10:12:45.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.127+0000 7f73d1200640 1 -- 172.21.15.5:0/4093729206 wait complete. 2024-04-25T10:12:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.130+0000 7f73d1200640 1 Processor -- start 2024-04-25T10:12:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.130+0000 7f73d1200640 1 -- start start 2024-04-25T10:12:45.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.131+0000 7f73d1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.131+0000 7f73d1200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc0ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.131+0000 7f73d1200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73cc142010 0x7f73cc0adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73d1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc06e300 con 0x7f73cc06ed60 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73d1200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc06e020 con 0x7f73cc142010 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73d1200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc142be0 con 0x7f73cc0cce30 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58126/0 (socket says 172.21.15.5:58126) 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 -- 172.21.15.5:0/3508998973 learned_addr learned my addr 172.21.15.5:0/3508998973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c1400640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc0ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73ca800640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73cc142010 0x7f73cc0adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 -- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 msgr2=0x7f73cc0ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc0ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 -- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73cc142010 msgr2=0x7f73cc0adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73cc142010 0x7f73cc0adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:45.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73c9e00640 1 -- 172.21.15.5:0/3508998973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc1f1ed0 con 0x7f73cc06ed60 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.132+0000 7f73ca800640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73cc142010 0x7f73cc0adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.133+0000 7f73c9e00640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc157d40 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ce30 tx=0x7f73bc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.133+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00f890 con 0x7f73cc06ed60 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.133+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc00fa30 con 0x7f73cc06ed60 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.133+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00fe00 con 0x7f73cc06ed60 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.133+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc1f21b0 con 0x7f73cc06ed60 2024-04-25T10:12:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.133+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc155780 con 0x7f73cc06ed60 2024-04-25T10:12:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.134+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73cc069040 con 0x7f73cc06ed60 2024-04-25T10:12:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.135+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73bc018400 con 0x7f73cc06ed60 2024-04-25T10:12:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.136+0000 7f73c2a00640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f73a806b4d0 0x7f73a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.136+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f73bc08add0 con 0x7f73cc06ed60 2024-04-25T10:12:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.137+0000 7f73c1400640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f73a806b4d0 0x7f73a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.138+0000 7f73c1400640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f73a806b4d0 0x7f73a806d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f73b4034000 tx=0x7f73b402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:45.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.141+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73bc014030 con 0x7f73cc06ed60 2024-04-25T10:12:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:45.340+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f73cc0b1440 con 0x7f73cc06ed60 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.023+0000 7f73c2a00640 1 -- 172.21.15.5:0/3508998973 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7f73bc08c0b0 con 0x7f73cc06ed60 2024-04-25T10:12:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.027+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f73a806b4d0 msgr2=0x7f73a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.027+0000 7f73d1200640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f73a806b4d0 0x7f73a806d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f73b4034000 tx=0x7f73b402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.027+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 msgr2=0x7f73cc157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.027+0000 7f73d1200640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc157d40 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ce30 tx=0x7f73bc00a650 comp rx=0 tx=0).stop 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 shutdown_connections 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f73a806b4d0 0x7f73a806d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73cc142010 0x7f73cc0adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f73cc0cce30 0x7f73cc0ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 --2- 172.21.15.5:0/3508998973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73cc06ed60 0x7f73cc157d40 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 >> 172.21.15.5:0/3508998973 conn(0x7f73cc066f80 msgr2=0x7f73cc15b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 shutdown_connections 2024-04-25T10:12:46.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.028+0000 7f73d1200640 1 -- 172.21.15.5:0/3508998973 wait complete. 2024-04-25T10:12:46.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:12:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.260+0000 7f9746c00640 1 Processor -- start 2024-04-25T10:12:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.260+0000 7f9746c00640 1 -- start start 2024-04-25T10:12:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9746c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f97401448f0 0x7f97400cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9746c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f974015bc30 0x7f974015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9746c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f974015fe30 0x7f974015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9746c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974015f460 con 0x7f974015bc30 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9746c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9740144cd0 con 0x7f97401448f0 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9746c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974015c440 con 0x7f974015fe30 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9744e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f974015bc30 0x7f974015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9744e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f974015bc30 0x7f974015f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58140/0 (socket says 172.21.15.5:58140) 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.262+0000 7f9744e00640 1 -- 172.21.15.5:0/2052797674 learned_addr learned my addr 172.21.15.5:0/2052797674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9746200640 1 --2- 172.21.15.5:0/2052797674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f974015fe30 0x7f974015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9745800640 1 --2- 172.21.15.5:0/2052797674 >> v2:172.21.15.80:3300/0 conn(0x7f97401448f0 0x7f97400cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9744e00640 1 -- 172.21.15.5:0/2052797674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f974015fe30 msgr2=0x7f974015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9744e00640 1 --2- 172.21.15.5:0/2052797674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f974015fe30 0x7f974015ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9744e00640 1 -- 172.21.15.5:0/2052797674 >> v2:172.21.15.80:3300/0 conn(0x7f97401448f0 msgr2=0x7f97400cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9744e00640 1 --2- 172.21.15.5:0/2052797674 >> v2:172.21.15.80:3300/0 conn(0x7f97401448f0 0x7f97400cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.263+0000 7f9744e00640 1 -- 172.21.15.5:0/2052797674 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974015c720 con 0x7f974015bc30 2024-04-25T10:12:46.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.264+0000 7f9744e00640 1 --2- 172.21.15.5:0/2052797674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015bc30 0x7f974015f8f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9740071eb0 tx=0x7f972c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559c0f851b0ee9db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.265+0000 7f973fe00640 1 -- 172.21.15.5:0/2052797674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c037070 con 0x7f974015bc30 2024-04-25T10:12:46.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.265+0000 7f973fe00640 1 -- 172.21.15.5:0/2052797674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c032440 con 0x7f974015bc30 2024-04-25T10:12:46.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.265+0000 7f973fe00640 1 -- 172.21.15.5:0/2052797674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c032810 con 0x7f974015bc30 2024-04-25T10:12:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 -- 172.21.15.5:0/2052797674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015bc30 msgr2=0x7f974015f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 --2- 172.21.15.5:0/2052797674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015bc30 0x7f974015f8f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f9740071eb0 tx=0x7f972c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 -- 172.21.15.5:0/2052797674 shutdown_connections 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 --2- 172.21.15.5:0/2052797674 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f974015fe30 0x7f974015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 --2- 172.21.15.5:0/2052797674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015bc30 0x7f974015f8f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 --2- 172.21.15.5:0/2052797674 >> v2:172.21.15.80:3300/0 conn(0x7f97401448f0 0x7f97400cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 -- 172.21.15.5:0/2052797674 >> 172.21.15.5:0/2052797674 conn(0x7f9740066f20 msgr2=0x7f974006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 -- 172.21.15.5:0/2052797674 shutdown_connections 2024-04-25T10:12:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.266+0000 7f9746c00640 1 -- 172.21.15.5:0/2052797674 wait complete. 2024-04-25T10:12:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.268+0000 7f9746c00640 1 Processor -- start 2024-04-25T10:12:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.268+0000 7f9746c00640 1 -- start start 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97401448f0 0x7f97401e59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f974015bc30 0x7f97401e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 0x7f9740140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974015d360 con 0x7f974015fe30 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974015c0c0 con 0x7f974015bc30 2024-04-25T10:12:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974015d4e0 con 0x7f97401448f0 2024-04-25T10:12:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.269+0000 7f9746200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 0x7f9740140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 0x7f9740140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58146/0 (socket says 172.21.15.5:58146) 2024-04-25T10:12:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 -- 172.21.15.5:0/1186111567 learned_addr learned my addr 172.21.15.5:0/1186111567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9744e00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f974015bc30 0x7f97401e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9745800640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97401448f0 0x7f97401e59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 -- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97401448f0 msgr2=0x7f97401e59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97401448f0 0x7f97401e59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 -- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f974015bc30 msgr2=0x7f97401e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f974015bc30 0x7f97401e99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 -- 172.21.15.5:0/1186111567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97401e5ef0 con 0x7f974015fe30 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.270+0000 7f9746200640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 0x7f9740140be0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9738007e70 tx=0x7f973800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.271+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738014620 con 0x7f974015fe30 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.271+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738019070 con 0x7f974015fe30 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.271+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738014a20 con 0x7f974015fe30 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.271+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97401e6de0 con 0x7f974015fe30 2024-04-25T10:12:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.271+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97401e91c0 con 0x7f974015fe30 2024-04-25T10:12:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.272+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9740068f60 con 0x7f974015fe30 2024-04-25T10:12:46.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.273+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9738014bc0 con 0x7f974015fe30 2024-04-25T10:12:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.275+0000 7f973d600640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972006b520 0x7f972006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:46.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.275+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f973808aa10 con 0x7f974015fe30 2024-04-25T10:12:46.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.276+0000 7f9745800640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972006b520 0x7f972006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:46.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.277+0000 7f9745800640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972006b520 0x7f972006d9e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9734006fd0 tx=0x7f9734008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:46.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.281+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97380598c0 con 0x7f974015fe30 2024-04-25T10:12:46.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:46.488+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f974015efa0 con 0x7f974015fe30 2024-04-25T10:12:47.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.040+0000 7f973d600640 1 -- 172.21.15.5:0/1186111567 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f97380591c0 con 0x7f974015fe30 2024-04-25T10:12:47.045 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-04-25T10:12:47.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972006b520 msgr2=0x7f972006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972006b520 0x7f972006d9e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9734006fd0 tx=0x7f9734008040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 msgr2=0x7f9740140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 0x7f9740140be0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9738007e70 tx=0x7f973800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 shutdown_connections 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f974015fe30 0x7f9740140be0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f972006b520 0x7f972006d9e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f974015bc30 0x7f97401e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 --2- 172.21.15.5:0/1186111567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97401448f0 0x7f97401e59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 >> 172.21.15.5:0/1186111567 conn(0x7f9740066f20 msgr2=0x7f9740142f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.044+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 shutdown_connections 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.045+0000 7f9746c00640 1 -- 172.21.15.5:0/1186111567 wait complete. 2024-04-25T10:12:47.083 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:12:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.260+0000 7f1d49c00640 1 Processor -- start 2024-04-25T10:12:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.261+0000 7f1d49c00640 1 -- start start 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.262+0000 7f1d49c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d440b48c0 0x7f1d4415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.262+0000 7f1d49c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1d44161d90 0x7f1d440b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.262+0000 7f1d49c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d441606e0 con 0x7f1d440d3f90 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d49c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d441609c0 con 0x7f1d440b48c0 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d49c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d44160ca0 con 0x7f1d44161d90 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d43e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1d440b48c0 0x7f1d4415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d43400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1d440d3f90 0x7f1d44161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33940/0 (socket says 172.21.15.5:33940) 2024-04-25T10:12:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d43400640 1 -- 172.21.15.5:0/2976706586 learned_addr learned my addr 172.21.15.5:0/2976706586 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d49200640 1 --2- 172.21.15.5:0/2976706586 >> v2:172.21.15.80:3300/0 conn(0x7f1d44161d90 0x7f1d440b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d43e00640 1 -- 172.21.15.5:0/2976706586 >> v2:172.21.15.80:3300/0 conn(0x7f1d44161d90 msgr2=0x7f1d440b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.263+0000 7f1d43e00640 1 --2- 172.21.15.5:0/2976706586 >> v2:172.21.15.80:3300/0 conn(0x7f1d44161d90 0x7f1d440b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.264+0000 7f1d43e00640 1 -- 172.21.15.5:0/2976706586 >> v1:172.21.15.5:6789/0 conn(0x7f1d440d3f90 legacy=0x7f1d44161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.264+0000 7f1d43e00640 1 -- 172.21.15.5:0/2976706586 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44160f80 con 0x7f1d440b48c0 2024-04-25T10:12:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.264+0000 7f1d42a00640 1 -- 172.21.15.5:0/2976706586 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539262831 0 0) 0x7f1d441606e0 con 0x7f1d440d3f90 2024-04-25T10:12:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.265+0000 7f1d43e00640 1 --2- 172.21.15.5:0/2976706586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d4415fe60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f1d4408d2e0 tx=0x7f1d3402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd8ffe8ffb10654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.265+0000 7f1d42a00640 1 -- 172.21.15.5:0/2976706586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34037070 con 0x7f1d440b48c0 2024-04-25T10:12:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.265+0000 7f1d42a00640 1 -- 172.21.15.5:0/2976706586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34032440 con 0x7f1d440b48c0 2024-04-25T10:12:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.265+0000 7f1d42a00640 1 -- 172.21.15.5:0/2976706586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3402ae60 con 0x7f1d440b48c0 2024-04-25T10:12:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 -- 172.21.15.5:0/2976706586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 msgr2=0x7f1d4415fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 --2- 172.21.15.5:0/2976706586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d4415fe60 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f1d4408d2e0 tx=0x7f1d3402a110 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 -- 172.21.15.5:0/2976706586 shutdown_connections 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 --2- 172.21.15.5:0/2976706586 >> v2:172.21.15.80:3300/0 conn(0x7f1d44161d90 0x7f1d440b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 --2- 172.21.15.5:0/2976706586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d4415fe60 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 -- 172.21.15.5:0/2976706586 >> 172.21.15.5:0/2976706586 conn(0x7f1d4408af80 msgr2=0x7f1d440d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 -- 172.21.15.5:0/2976706586 shutdown_connections 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.266+0000 7f1d49c00640 1 -- 172.21.15.5:0/2976706586 wait complete. 2024-04-25T10:12:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.268+0000 7f1d49c00640 1 Processor -- start 2024-04-25T10:12:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.268+0000 7f1d49c00640 1 -- start start 2024-04-25T10:12:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d49c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d440ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d49c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d440d3f90 0x7f1d440cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d49c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d44161d90 0x7f1d440b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d49c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d441603a0 con 0x7f1d440b48c0 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d49c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b4ca0 con 0x7f1d440d3f90 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d49c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440b30b0 con 0x7f1d44161d90 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d440ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d440ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34520/0 (socket says 172.21.15.5:34520) 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 -- 172.21.15.5:0/898337845 learned_addr learned my addr 172.21.15.5:0/898337845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 -- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d44161d90 msgr2=0x7f1d440b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d44161d90 0x7f1d440b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 -- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d440d3f90 msgr2=0x7f1d440cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d440d3f90 0x7f1d440cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.269+0000 7f1d43e00640 1 -- 172.21.15.5:0/898337845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440b82c0 con 0x7f1d440b48c0 2024-04-25T10:12:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.270+0000 7f1d43e00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d440ca6d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1d34000c00 tx=0x7f1d340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.270+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34037050 con 0x7f1d440b48c0 2024-04-25T10:12:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.270+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34038040 con 0x7f1d440b48c0 2024-04-25T10:12:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.270+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34004470 con 0x7f1d440b48c0 2024-04-25T10:12:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.270+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d440cad40 con 0x7f1d440b48c0 2024-04-25T10:12:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.270+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d440cb300 con 0x7f1d440b48c0 2024-04-25T10:12:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.271+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d44068f60 con 0x7f1d440b48c0 2024-04-25T10:12:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.273+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d34030900 con 0x7f1d440b48c0 2024-04-25T10:12:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.273+0000 7f1d3be00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d2406b4d0 0x7f1d2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.274+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f1d340aacf0 con 0x7f1d440b48c0 2024-04-25T10:12:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.274+0000 7f1d43400640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d2406b4d0 0x7f1d2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.276+0000 7f1d43400640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d2406b4d0 0x7f1d2406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1d4406c7d0 tx=0x7f1d30007870 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.279+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d340af070 con 0x7f1d440b48c0 2024-04-25T10:12:47.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:47.521+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1d44073910 con 0x7f1d440b48c0 2024-04-25T10:12:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.054+0000 7f1d3be00640 1 -- 172.21.15.5:0/898337845 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1d34079bf0 con 0x7f1d440b48c0 2024-04-25T10:12:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:12:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d2406b4d0 msgr2=0x7f1d2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d2406b4d0 0x7f1d2406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1d4406c7d0 tx=0x7f1d30007870 comp rx=0 tx=0).stop 2024-04-25T10:12:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 msgr2=0x7f1d440ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d440ca6d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1d34000c00 tx=0x7f1d340046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 shutdown_connections 2024-04-25T10:12:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d44161d90 0x7f1d440b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d2406b4d0 0x7f1d2406d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d440d3f90 0x7f1d440cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 --2- 172.21.15.5:0/898337845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d440b48c0 0x7f1d440ca6d0 secure :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1d34000c00 tx=0x7f1d340046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.060+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 >> 172.21.15.5:0/898337845 conn(0x7f1d4408af80 msgr2=0x7f1d440b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.062+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 shutdown_connections 2024-04-25T10:12:48.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.064+0000 7f1d49c00640 1 -- 172.21.15.5:0/898337845 wait complete. 2024-04-25T10:12:48.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.276+0000 7f4925e00640 1 Processor -- start 2024-04-25T10:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.276+0000 7f4925e00640 1 -- start start 2024-04-25T10:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.278+0000 7f4925e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 0x7f49200ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.278+0000 7f4925e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f49200cd240 0x7f49201622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.278+0000 7f4925e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4920165af0 con 0x7f492015c380 2024-04-25T10:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.278+0000 7f4925e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492015c760 con 0x7f49200cd240 2024-04-25T10:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.278+0000 7f4925e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49201658a0 con 0x7f49200b16e0 2024-04-25T10:12:48.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f4917e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 0x7f49200ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f4917e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 0x7f49200ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:41360/0 (socket says 172.21.15.5:41360) 2024-04-25T10:12:48.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f4917e00640 1 -- 172.21.15.5:0/3364398178 learned_addr learned my addr 172.21.15.5:0/3364398178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:48.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f491f400640 1 --2- 172.21.15.5:0/3364398178 >> v2:172.21.15.80:3300/0 conn(0x7f49200cd240 0x7f49201622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f491e000640 1 -- 172.21.15.5:0/3364398178 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669878485 0 0) 0x7f4920165af0 con 0x7f492015c380 2024-04-25T10:12:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f491e000640 1 -- 172.21.15.5:0/3364398178 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4900003520 con 0x7f492015c380 2024-04-25T10:12:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.279+0000 7f4917e00640 1 -- 172.21.15.5:0/3364398178 >> v2:172.21.15.80:3300/0 conn(0x7f49200cd240 msgr2=0x7f49201622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.280+0000 7f4917e00640 1 --2- 172.21.15.5:0/3364398178 >> v2:172.21.15.80:3300/0 conn(0x7f49200cd240 0x7f49201622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.280+0000 7f4917e00640 1 -- 172.21.15.5:0/3364398178 >> v1:172.21.15.80:6789/0 conn(0x7f492015c380 legacy=0x7f492006e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.280+0000 7f4917e00640 1 -- 172.21.15.5:0/3364398178 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920165dd0 con 0x7f49200b16e0 2024-04-25T10:12:48.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.281+0000 7f4917e00640 1 --2- 172.21.15.5:0/3364398178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 0x7f49200ccd00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4908009870 tx=0x7f490802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dcb8f5347308f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f491e000640 1 -- 172.21.15.5:0/3364398178 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908037070 con 0x7f49200b16e0 2024-04-25T10:12:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f491e000640 1 -- 172.21.15.5:0/3364398178 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4908032440 con 0x7f49200b16e0 2024-04-25T10:12:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f491e000640 1 -- 172.21.15.5:0/3364398178 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908032810 con 0x7f49200b16e0 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f4925e00640 1 -- 172.21.15.5:0/3364398178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 msgr2=0x7f49200ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f4925e00640 1 --2- 172.21.15.5:0/3364398178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 0x7f49200ccd00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4908009870 tx=0x7f490802a130 comp rx=0 tx=0).stop 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f4925e00640 1 -- 172.21.15.5:0/3364398178 shutdown_connections 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f4925e00640 1 --2- 172.21.15.5:0/3364398178 >> v2:172.21.15.80:3300/0 conn(0x7f49200cd240 0x7f49201622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f4925e00640 1 --2- 172.21.15.5:0/3364398178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200b16e0 0x7f49200ccd00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.282+0000 7f4925e00640 1 -- 172.21.15.5:0/3364398178 >> 172.21.15.5:0/3364398178 conn(0x7f4920071150 msgr2=0x7f4920071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.283+0000 7f4925e00640 1 -- 172.21.15.5:0/3364398178 shutdown_connections 2024-04-25T10:12:48.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.283+0000 7f4925e00640 1 -- 172.21.15.5:0/3364398178 wait complete. 2024-04-25T10:12:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.284+0000 7f4925e00640 1 Processor -- start 2024-04-25T10:12:48.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.285+0000 7f4925e00640 1 -- start start 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4925e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 0x7f492013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4925e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200cd240 0x7f49201645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4925e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015c380 0x7f49200b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4925e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201627f0 con 0x7f49200b16e0 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4925e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4920166390 con 0x7f492015c380 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4925e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4920162970 con 0x7f49200cd240 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f491ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 0x7f492013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f491ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 0x7f492013d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34530/0 (socket says 172.21.15.5:34530) 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f491ea00640 1 -- 172.21.15.5:0/4227137329 learned_addr learned my addr 172.21.15.5:0/4227137329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:48.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.286+0000 7f4917e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200cd240 0x7f49201645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491f400640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015c380 0x7f49200b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491ea00640 1 -- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200cd240 msgr2=0x7f49201645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491ea00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200cd240 0x7f49201645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491ea00640 1 -- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015c380 msgr2=0x7f49200b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491ea00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015c380 0x7f49200b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491ea00640 1 -- 172.21.15.5:0/4227137329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200b3970 con 0x7f49200b16e0 2024-04-25T10:12:48.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f491ea00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 0x7f492013d720 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f491000c430 tx=0x7f491000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910017070 con 0x7f49200b16e0 2024-04-25T10:12:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49100052f0 con 0x7f49200b16e0 2024-04-25T10:12:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910003830 con 0x7f49200b16e0 2024-04-25T10:12:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49200b3c50 con 0x7f49200b16e0 2024-04-25T10:12:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.287+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492015f760 con 0x7f49200b16e0 2024-04-25T10:12:48.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.289+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4920068fd0 con 0x7f49200b16e0 2024-04-25T10:12:48.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.290+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f491001a070 con 0x7f49200b16e0 2024-04-25T10:12:48.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.291+0000 7f4917400640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.291+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49100897e0 con 0x7f49200b16e0 2024-04-25T10:12:48.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.291+0000 7f4917e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.293+0000 7f4917e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f490802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.295+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4910058440 con 0x7f49200b16e0 2024-04-25T10:12:48.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.495+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49200b0e70 con 0x7f49200b16e0 2024-04-25T10:12:48.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.498+0000 7f4917400640 1 -- 172.21.15.5:0/4227137329 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f4910057d40 con 0x7f49200b16e0 2024-04-25T10:12:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:48.503 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":119,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:47.522644+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 msgr2=0x7f48fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4908034000 tx=0x7f490802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 msgr2=0x7f492013d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 0x7f492013d720 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f491000c430 tx=0x7f491000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 shutdown_connections 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48fc06b4d0 0x7f48fc06d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015c380 0x7f49200b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f49200cd240 0x7f49201645a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 --2- 172.21.15.5:0/4227137329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49200b16e0 0x7f492013d720 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 >> 172.21.15.5:0/4227137329 conn(0x7f4920071150 msgr2=0x7f49200cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 shutdown_connections 2024-04-25T10:12:48.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.501+0000 7f4925e00640 1 -- 172.21.15.5:0/4227137329 wait complete. 2024-04-25T10:12:48.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.724+0000 7f7eb5e00640 1 Processor -- start 2024-04-25T10:12:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.724+0000 7f7eb5e00640 1 -- start start 2024-04-25T10:12:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.726+0000 7f7eb5e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 0x7f7eb015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.726+0000 7f7eb5e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb0071d30 con 0x7f7eb00cd300 2024-04-25T10:12:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.726+0000 7f7eb5e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb006d620 con 0x7f7eb0141da0 2024-04-25T10:12:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.726+0000 7f7eb5e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0142180 con 0x7f7eb015ce00 2024-04-25T10:12:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eb5400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7eb00cd300 0x7f7eb0071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33948/0 (socket says 172.21.15.5:33948) 2024-04-25T10:12:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eb5400640 1 -- 172.21.15.5:0/476665879 learned_addr learned my addr 172.21.15.5:0/476665879 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7ea7e00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 0x7f7eb015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891086403 0 0) 0x7f7eb0071d30 con 0x7f7eb00cd300 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e94003520 con 0x7f7eb00cd300 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4018704590 0 0) 0x7f7e94003520 con 0x7f7eb00cd300 2024-04-25T10:12:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 msgr2=0x7f7eb015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.727+0000 7f7eafe00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 0x7f7eb015d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 >> v1:172.21.15.80:6789/0 conn(0x7f7eb0141da0 legacy=0x7f7eb006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb006d900 con 0x7f7eb00cd300 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f7ea80019f0 con 0x7f7eb00cd300 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 >> v1:172.21.15.5:6789/0 conn(0x7f7eb00cd300 legacy=0x7f7eb0071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e940039e0 0x7f7e94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 0x7f7e94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94004d40 0x7f7e940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0142180 con 0x7f7e94004d40 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9400baa0 con 0x7f7e940039e0 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9400bca0 con 0x7f7eb015ce00 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eb5400640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94004d40 0x7f7e940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7ea7e00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 0x7f7e94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.728+0000 7f7eb4a00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e940039e0 0x7f7e94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.729+0000 7f7eb5400640 1 -- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 msgr2=0x7f7e94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.729+0000 7f7eb5400640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7eb015ce00 0x7f7e94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.729+0000 7f7eb5400640 1 -- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e940039e0 msgr2=0x7f7e94003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.729+0000 7f7eb5400640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e940039e0 0x7f7e94003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.729+0000 7f7eb5400640 1 -- 172.21.15.5:0/476665879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e94003520 con 0x7f7e94004d40 2024-04-25T10:12:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.729+0000 7f7eb4a00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e940039e0 0x7f7e94003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eb5400640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94004d40 0x7f7e940051c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800a760 tx=0x7f7e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7b7fae8c0c4731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800a3b0 con 0x7f7e94004d40 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea802f070 con 0x7f7e94004d40 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eafe00640 1 -- 172.21.15.5:0/476665879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea802bbe0 con 0x7f7e94004d40 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eb5e00640 1 -- 172.21.15.5:0/476665879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94004d40 msgr2=0x7f7e940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/476665879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94004d40 0x7f7e940051c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800a760 tx=0x7f7e94005830 comp rx=0 tx=0).stop 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.730+0000 7f7eb5400640 1 -- 172.21.15.5:0/476665879 reap_dead start 2024-04-25T10:12:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.731+0000 7f7eb5e00640 1 -- 172.21.15.5:0/476665879 shutdown_connections 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.731+0000 7f7eb5e00640 1 -- 172.21.15.5:0/476665879 >> 172.21.15.5:0/476665879 conn(0x7f7eb0066f20 msgr2=0x7f7eb006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.731+0000 7f7eb5e00640 1 -- 172.21.15.5:0/476665879 shutdown_connections 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.731+0000 7f7eb5e00640 1 -- 172.21.15.5:0/476665879 wait complete. 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.732+0000 7f7eb5e00640 1 Processor -- start 2024-04-25T10:12:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.733+0000 7f7eb5e00640 1 -- start start 2024-04-25T10:12:48.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7e94004d40 0x7f7eb013ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 0x7f7eb013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb013f490 0x7f7eb00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0071710 con 0x7f7eb013eaf0 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb006e810 con 0x7f7eb013f490 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb0071890 con 0x7f7e94004d40 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 0x7f7eb013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 0x7f7eb013ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34540/0 (socket says 172.21.15.5:34540) 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 -- 172.21.15.5:0/1748819974 learned_addr learned my addr 172.21.15.5:0/1748819974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb4a00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7e94004d40 0x7f7eb013ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 -- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7e94004d40 msgr2=0x7f7eb013ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb5400640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb013f490 0x7f7eb00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7e94004d40 0x7f7eb013ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 -- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb013f490 msgr2=0x7f7eb00b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb013f490 0x7f7eb00b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7ea7e00640 1 -- 172.21.15.5:0/1748819974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb013f910 con 0x7f7eb013eaf0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.734+0000 7f7eb4a00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7e94004d40 0x7f7eb013ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.735+0000 7f7ea7e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 0x7f7eb013ef50 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7e98011630 tx=0x7f7e98011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.735+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800f920 con 0x7f7eb013eaf0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.735+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800fac0 con 0x7f7eb013eaf0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.735+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800fe70 con 0x7f7eb013eaf0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.735+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb0140850 con 0x7f7eb013eaf0 2024-04-25T10:12:48.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.735+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb00af1c0 con 0x7f7eb013eaf0 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.736+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb0140d30 con 0x7f7eb013eaf0 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.737+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e98015460 con 0x7f7eb013eaf0 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.738+0000 7f7ead600640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7e8c06b500 0x7f7e8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.738+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e9808ad70 con 0x7f7eb013eaf0 2024-04-25T10:12:48.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.739+0000 7f7eb4a00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7e8c06b500 0x7f7e8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.740+0000 7f7eb4a00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7e8c06b500 0x7f7e8c06d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00016f0 tx=0x7f7ea0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.743+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7eb0140d30 con 0x7f7eb013eaf0 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.941+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7eb00ccae0 con 0x7f7eb013eaf0 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.944+0000 7f7ead600640 1 -- 172.21.15.5:0/1748819974 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f7e98019030 con 0x7f7eb013eaf0 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":119,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:47.522644+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7e8c06b500 msgr2=0x7f7e8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7e8c06b500 0x7f7e8c06d9c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00016f0 tx=0x7f7ea0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 msgr2=0x7f7eb013ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 0x7f7eb013ef50 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7e98011630 tx=0x7f7e98011af0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 shutdown_connections 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f7e8c06b500 0x7f7e8c06d9c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb013f490 0x7f7eb00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7eb013eaf0 0x7f7eb013ef50 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 --2- 172.21.15.5:0/1748819974 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7e94004d40 0x7f7eb013ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.946+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 >> 172.21.15.5:0/1748819974 conn(0x7f7eb0066f20 msgr2=0x7f7eb0141060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.947+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 shutdown_connections 2024-04-25T10:12:48.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:48.947+0000 7f7eb5e00640 1 -- 172.21.15.5:0/1748819974 wait complete. 2024-04-25T10:12:48.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:12:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.188+0000 7fdfb4a00640 1 Processor -- start 2024-04-25T10:12:49.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.188+0000 7fdfb4a00640 1 -- start start 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.189+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb0161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.189+0000 7fdfb4a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdfb0161bc0 0x7fdfb00b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.189+0000 7fdfb4a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb01606e0 con 0x7fdfb00b48c0 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.189+0000 7fdfb4a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb01609c0 con 0x7fdfb0161bc0 2024-04-25T10:12:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.189+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0160ca0 con 0x7fdfb00d3f90 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.190+0000 7fdfaea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb0161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.190+0000 7fdfaea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb0161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:41410/0 (socket says 172.21.15.5:41410) 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.190+0000 7fdfaea00640 1 -- 172.21.15.5:0/3613967790 learned_addr learned my addr 172.21.15.5:0/3613967790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.190+0000 7fdfaf400640 1 --1- 172.21.15.5:0/3613967790 >> v1:172.21.15.80:6789/0 conn(0x7fdfb00b48c0 0x7fdfb015fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:60216/0 (socket says 172.21.15.5:60216) 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.191+0000 7fdfafe00640 1 --2- 172.21.15.5:0/3613967790 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0161bc0 0x7fdfb00b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.191+0000 7fdfaea00640 1 -- 172.21.15.5:0/3613967790 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0161bc0 msgr2=0x7fdfb00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.191+0000 7fdfaea00640 1 --2- 172.21.15.5:0/3613967790 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0161bc0 0x7fdfb00b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.191+0000 7fdfaea00640 1 -- 172.21.15.5:0/3613967790 >> v1:172.21.15.80:6789/0 conn(0x7fdfb00b48c0 legacy=0x7fdfb015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.191+0000 7fdfaea00640 1 -- 172.21.15.5:0/3613967790 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0160f80 con 0x7fdfb00d3f90 2024-04-25T10:12:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.193+0000 7fdfaea00640 1 --2- 172.21.15.5:0/3613967790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb0161680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009ea0 tx=0x7fdfa002a720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90dacd5e565918f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.193+0000 7fdfae000640 1 -- 172.21.15.5:0/3613967790 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0036070 con 0x7fdfb00d3f90 2024-04-25T10:12:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfae000640 1 -- 172.21.15.5:0/3613967790 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa0031600 con 0x7fdfb00d3f90 2024-04-25T10:12:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfae000640 1 -- 172.21.15.5:0/3613967790 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00319d0 con 0x7fdfb00d3f90 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 -- 172.21.15.5:0/3613967790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 msgr2=0x7fdfb0161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/3613967790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb0161680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009ea0 tx=0x7fdfa002a720 comp rx=0 tx=0).stop 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 -- 172.21.15.5:0/3613967790 shutdown_connections 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/3613967790 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0161bc0 0x7fdfb00b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/3613967790 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb0161680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 -- 172.21.15.5:0/3613967790 >> 172.21.15.5:0/3613967790 conn(0x7fdfb008af80 msgr2=0x7fdfb00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.194+0000 7fdfb4a00640 1 -- 172.21.15.5:0/3613967790 shutdown_connections 2024-04-25T10:12:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.195+0000 7fdfb4a00640 1 -- 172.21.15.5:0/3613967790 wait complete. 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.196+0000 7fdfb4a00640 1 Processor -- start 2024-04-25T10:12:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.196+0000 7fdfb4a00640 1 -- start start 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.197+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb00b48c0 0x7fdfb00cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.197+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb00c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.197+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 0x7fdfb00ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.197+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b30b0 con 0x7fdfb0161bc0 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.197+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b4ca0 con 0x7fdfb00b48c0 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.197+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00b3230 con 0x7fdfb00d3f90 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 0x7fdfb00ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 0x7fdfb00ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34556/0 (socket says 172.21.15.5:34556) 2024-04-25T10:12:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 -- 172.21.15.5:0/2359238882 learned_addr learned my addr 172.21.15.5:0/2359238882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfaea00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb00c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 -- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 msgr2=0x7fdfb00c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb00c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 -- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb00b48c0 msgr2=0x7fdfb00cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb00b48c0 0x7fdfb00cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.198+0000 7fdfafe00640 1 -- 172.21.15.5:0/2359238882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb00cd520 con 0x7fdfb0161bc0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.199+0000 7fdfafe00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 0x7fdfb00ccfe0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800e820 tx=0x7fdfa800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.199+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8017070 con 0x7fdfb0161bc0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.199+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa800ee70 con 0x7fdfb0161bc0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.199+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800cb60 con 0x7fdfb0161bc0 2024-04-25T10:12:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.199+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb00cbd00 con 0x7fdfb0161bc0 2024-04-25T10:12:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.199+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb00cc2c0 con 0x7fdfb0161bc0 2024-04-25T10:12:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.200+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb0068f60 con 0x7fdfb0161bc0 2024-04-25T10:12:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.201+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfa801a070 con 0x7fdfb0161bc0 2024-04-25T10:12:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.202+0000 7fdfa7400640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf8c06b520 0x7fdf8c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.203+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdfa808a480 con 0x7fdfb0161bc0 2024-04-25T10:12:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.203+0000 7fdfaf400640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf8c06b520 0x7fdf8c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.204+0000 7fdfaf400640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf8c06b520 0x7fdf8c06d9e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf98000f60 tx=0x7fdf98007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.207+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfb00cd520 con 0x7fdfb0161bc0 2024-04-25T10:12:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:49.452+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdfb00d3d70 con 0x7fdfb0161bc0 2024-04-25T10:12:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.057+0000 7fdfa7400640 1 -- 172.21.15.5:0/2359238882 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdfa80592e0 con 0x7fdfb0161bc0 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf8c06b520 msgr2=0x7fdf8c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf8c06b520 0x7fdf8c06d9e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdf98000f60 tx=0x7fdf98007a00 comp rx=0 tx=0).stop 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 msgr2=0x7fdfb00ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 0x7fdfb00ccfe0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800e820 tx=0x7fdfa800ec20 comp rx=0 tx=0).stop 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 shutdown_connections 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb0161bc0 0x7fdfb00ccfe0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb00d3f90 0x7fdfb00c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf8c06b520 0x7fdf8c06d9e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 --2- 172.21.15.5:0/2359238882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb00b48c0 0x7fdfb00cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 >> 172.21.15.5:0/2359238882 conn(0x7fdfb008af80 msgr2=0x7fdfb00b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 shutdown_connections 2024-04-25T10:12:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.061+0000 7fdfb4a00640 1 -- 172.21.15.5:0/2359238882 wait complete. 2024-04-25T10:12:50.090 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:12:50.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:12:50.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.304+0000 7fa0e5a00640 1 Processor -- start 2024-04-25T10:12:50.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.304+0000 7fa0e5a00640 1 -- start start 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.305+0000 7fa0e5a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0e00b48c0 0x7fa0e015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.305+0000 7fa0e5a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e01606e0 con 0x7fa0e00d3f90 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.305+0000 7fa0e5a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e01609c0 con 0x7fa0e0161d90 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.305+0000 7fa0e5a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e0160ca0 con 0x7fa0e00b48c0 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.305+0000 7fa0dfe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa0e00b48c0 0x7fa0e015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.305+0000 7fa0df400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa0e00d3f90 0x7fa0e0161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33952/0 (socket says 172.21.15.5:33952) 2024-04-25T10:12:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.306+0000 7fa0df400640 1 -- 172.21.15.5:0/3213122915 learned_addr learned my addr 172.21.15.5:0/3213122915 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.306+0000 7fa0dfe00640 1 -- 172.21.15.5:0/3213122915 >> v1:172.21.15.80:6789/0 conn(0x7fa0e0161d90 legacy=0x7fa0e00b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.306+0000 7fa0dfe00640 1 -- 172.21.15.5:0/3213122915 >> v1:172.21.15.5:6789/0 conn(0x7fa0e00d3f90 legacy=0x7fa0e0161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.306+0000 7fa0dfe00640 1 -- 172.21.15.5:0/3213122915 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e0160f80 con 0x7fa0e00b48c0 2024-04-25T10:12:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.307+0000 7fa0dfe00640 1 --2- 172.21.15.5:0/3213122915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e00b48c0 0x7fa0e015fe60 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00097a0 tx=0x7fa0d002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3135e87bcf8d5d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0dea00640 1 -- 172.21.15.5:0/3213122915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0037070 con 0x7fa0e00b48c0 2024-04-25T10:12:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0dea00640 1 -- 172.21.15.5:0/3213122915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d002ae50 con 0x7fa0e00b48c0 2024-04-25T10:12:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0dea00640 1 -- 172.21.15.5:0/3213122915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0032660 con 0x7fa0e00b48c0 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 -- 172.21.15.5:0/3213122915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e00b48c0 msgr2=0x7fa0e015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/3213122915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e00b48c0 0x7fa0e015fe60 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00097a0 tx=0x7fa0d002a140 comp rx=0 tx=0).stop 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 -- 172.21.15.5:0/3213122915 shutdown_connections 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/3213122915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e00b48c0 0x7fa0e015fe60 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 -- 172.21.15.5:0/3213122915 >> 172.21.15.5:0/3213122915 conn(0x7fa0e008af80 msgr2=0x7fa0e00d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 -- 172.21.15.5:0/3213122915 shutdown_connections 2024-04-25T10:12:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.308+0000 7fa0e5a00640 1 -- 172.21.15.5:0/3213122915 wait complete. 2024-04-25T10:12:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.310+0000 7fa0e5a00640 1 Processor -- start 2024-04-25T10:12:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.310+0000 7fa0e5a00640 1 -- start start 2024-04-25T10:12:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.311+0000 7fa0e5a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0e00b48c0 0x7fa0e00d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.311+0000 7fa0e5a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0e00d3f90 0x7fa0e00d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.311+0000 7fa0e5a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 0x7fa0e00c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.311+0000 7fa0e5a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e01603a0 con 0x7fa0e0161d90 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.311+0000 7fa0e5a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00b4ca0 con 0x7fa0e00b48c0 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.311+0000 7fa0e5a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00b3280 con 0x7fa0e00d3f90 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 0x7fa0e00c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 0x7fa0e00c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34570/0 (socket says 172.21.15.5:34570) 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 -- 172.21.15.5:0/2521153106 learned_addr learned my addr 172.21.15.5:0/2521153106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0dfe00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0e00b48c0 0x7fa0e00d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 -- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0e00d3f90 msgr2=0x7fa0e00d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0e00d3f90 0x7fa0e00d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 -- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0e00b48c0 msgr2=0x7fa0e00d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0e00b48c0 0x7fa0e00d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 -- 172.21.15.5:0/2521153106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00cc3d0 con 0x7fa0e0161d90 2024-04-25T10:12:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0dfe00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0e00b48c0 0x7fa0e00d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.312+0000 7fa0e5000640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 0x7fa0e00c7d30 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800b410 tx=0x7fa0d800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.313+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8015030 con 0x7fa0e0161d90 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.313+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8004020 con 0x7fa0e0161d90 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.313+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004450 con 0x7fa0e0161d90 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.313+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e00cc650 con 0x7fa0e0161d90 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.313+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e00ccc10 con 0x7fa0e0161d90 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.314+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e015fc20 con 0x7fa0e0161d90 2024-04-25T10:12:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.315+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0d8018070 con 0x7fa0e0161d90 2024-04-25T10:12:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.316+0000 7fa0d7e00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.316+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0d8089e80 con 0x7fa0e0161d90 2024-04-25T10:12:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.317+0000 7fa0dfe00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.318+0000 7fa0dfe00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0034000 tx=0x7fa0d002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.321+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0e015fc20 con 0x7fa0e0161d90 2024-04-25T10:12:50.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:50.559+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa0e0068f60 con 0x7fa0e0161d90 2024-04-25T10:12:51.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.081+0000 7fa0d7e00640 1 -- 172.21.15.5:0/2521153106 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa0e00cc3d0 con 0x7fa0e0161d90 2024-04-25T10:12:51.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.085+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa0bc06b4d0 msgr2=0x7fa0bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.085+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0034000 tx=0x7fa0d002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.085+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 msgr2=0x7fa0e00c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 0x7fa0e00c7d30 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800b410 tx=0x7fa0d800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 shutdown_connections 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0e0161d90 0x7fa0e00c7d30 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa0e00d3f90 0x7fa0e00d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa0bc06b4d0 0x7fa0bc06d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 --2- 172.21.15.5:0/2521153106 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0e00b48c0 0x7fa0e00d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 >> 172.21.15.5:0/2521153106 conn(0x7fa0e008af80 msgr2=0x7fa0e015eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 shutdown_connections 2024-04-25T10:12:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.086+0000 7fa0e5a00640 1 -- 172.21.15.5:0/2521153106 wait complete. 2024-04-25T10:12:51.116 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:12:51.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:51.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.330+0000 7fca99200640 1 Processor -- start 2024-04-25T10:12:51.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.330+0000 7fca99200640 1 -- start start 2024-04-25T10:12:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.331+0000 7fca99200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca94143d30 0x7fca9415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.331+0000 7fca99200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca940b3ae0 0x7fca940b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.331+0000 7fca99200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca940b4880 con 0x7fca940ce790 2024-04-25T10:12:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.331+0000 7fca99200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca940b45a0 con 0x7fca940b3ae0 2024-04-25T10:12:51.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.331+0000 7fca99200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9415e2c0 con 0x7fca94143d30 2024-04-25T10:12:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.332+0000 7fca93e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fca940ce790 0x7fca9415abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33968/0 (socket says 172.21.15.5:33968) 2024-04-25T10:12:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.332+0000 7fca93e00640 1 -- 172.21.15.5:0/551512840 learned_addr learned my addr 172.21.15.5:0/551512840 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.332+0000 7fca92a00640 1 --2- 172.21.15.5:0/551512840 >> v2:172.21.15.5:3300/0 conn(0x7fca940b3ae0 0x7fca940b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.332+0000 7fca93400640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca94143d30 0x7fca9415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.332+0000 7fca92000640 1 -- 172.21.15.5:0/551512840 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024416573 0 0) 0x7fca940b4880 con 0x7fca940ce790 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.333+0000 7fca92000640 1 -- 172.21.15.5:0/551512840 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca78005730 con 0x7fca940ce790 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.333+0000 7fca92a00640 1 -- 172.21.15.5:0/551512840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca94143d30 msgr2=0x7fca9415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.333+0000 7fca92a00640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca94143d30 0x7fca9415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.333+0000 7fca92a00640 1 -- 172.21.15.5:0/551512840 >> v1:172.21.15.5:6789/0 conn(0x7fca940ce790 legacy=0x7fca9415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.333+0000 7fca92a00640 1 -- 172.21.15.5:0/551512840 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9415e5a0 con 0x7fca940b3ae0 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.333+0000 7fca93400640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca94143d30 0x7fca9415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:51.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.334+0000 7fca92a00640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940b3ae0 0x7fca940b3f60 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fca800097a0 tx=0x7fca8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0b2f345264363b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.334+0000 7fca92000640 1 -- 172.21.15.5:0/551512840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80037070 con 0x7fca940b3ae0 2024-04-25T10:12:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.334+0000 7fca92000640 1 -- 172.21.15.5:0/551512840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80032440 con 0x7fca940b3ae0 2024-04-25T10:12:51.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.334+0000 7fca92000640 1 -- 172.21.15.5:0/551512840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800327f0 con 0x7fca940b3ae0 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.334+0000 7fca99200640 1 -- 172.21.15.5:0/551512840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940b3ae0 msgr2=0x7fca940b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.334+0000 7fca99200640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940b3ae0 0x7fca940b3f60 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fca800097a0 tx=0x7fca8002a180 comp rx=0 tx=0).stop 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.335+0000 7fca99200640 1 -- 172.21.15.5:0/551512840 shutdown_connections 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.335+0000 7fca99200640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940b3ae0 0x7fca940b3f60 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.335+0000 7fca99200640 1 --2- 172.21.15.5:0/551512840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca94143d30 0x7fca9415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.335+0000 7fca99200640 1 -- 172.21.15.5:0/551512840 >> 172.21.15.5:0/551512840 conn(0x7fca94066f20 msgr2=0x7fca9406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.335+0000 7fca99200640 1 -- 172.21.15.5:0/551512840 shutdown_connections 2024-04-25T10:12:51.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.335+0000 7fca99200640 1 -- 172.21.15.5:0/551512840 wait complete. 2024-04-25T10:12:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.336+0000 7fca99200640 1 Processor -- start 2024-04-25T10:12:51.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.337+0000 7fca99200640 1 -- start start 2024-04-25T10:12:51.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca99200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca940b3ae0 0x7fca940cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca99200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 0x7fca940ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca99200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca94143d30 0x7fca941f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca99200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca94144290 con 0x7fca940ce790 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca99200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca94144110 con 0x7fca94143d30 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca99200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9406e430 con 0x7fca940b3ae0 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 0x7fca940ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 0x7fca940ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34574/0 (socket says 172.21.15.5:34574) 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 -- 172.21.15.5:0/778355254 learned_addr learned my addr 172.21.15.5:0/778355254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 -- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca940b3ae0 msgr2=0x7fca940cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca940b3ae0 0x7fca940cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 -- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca94143d30 msgr2=0x7fca941f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca94143d30 0x7fca941f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.338+0000 7fca92a00640 1 -- 172.21.15.5:0/778355254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940cae00 con 0x7fca940ce790 2024-04-25T10:12:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.339+0000 7fca92a00640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 0x7fca940ca8c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fca80035040 tx=0x7fca800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.339+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80041020 con 0x7fca940ce790 2024-04-25T10:12:51.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.339+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80030700 con 0x7fca940ce790 2024-04-25T10:12:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.339+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80037040 con 0x7fca940ce790 2024-04-25T10:12:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.339+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca941e45d0 con 0x7fca940ce790 2024-04-25T10:12:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.339+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca941e4b90 con 0x7fca940ce790 2024-04-25T10:12:51.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.340+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca94068f60 con 0x7fca940ce790 2024-04-25T10:12:51.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.342+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca80044020 con 0x7fca940ce790 2024-04-25T10:12:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.343+0000 7fca8b400640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca7406b4d0 0x7fca7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.343+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca800aab70 con 0x7fca940ce790 2024-04-25T10:12:51.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.343+0000 7fca93400640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca7406b4d0 0x7fca7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.345+0000 7fca93400640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca7406b4d0 0x7fca7406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fca84009f50 tx=0x7fca840094a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.347+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca8004b030 con 0x7fca940ce790 2024-04-25T10:12:51.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.587+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca940ce460 con 0x7fca940ce790 2024-04-25T10:12:51.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.588+0000 7fca8b400640 1 -- 172.21.15.5:0/778355254 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fca80079850 con 0x7fca940ce790 2024-04-25T10:12:51.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:51.593 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2518129896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2518129896},{"type":"v1","addr":"172.21.15.5:6835","nonce":2518129896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:12:50.560675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7006":{"gid":7006,"name":"b","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/84886517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":84886517},{"type":"v1","addr":"172.21.15.80:6837","nonce":84886517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:51.593 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 98 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca7406b4d0 msgr2=0x7fca7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca7406b4d0 0x7fca7406d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fca84009f50 tx=0x7fca840094a0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 msgr2=0x7fca940ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 0x7fca940ca8c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fca80035040 tx=0x7fca800046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 shutdown_connections 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fca7406b4d0 0x7fca7406d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca94143d30 0x7fca941f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca940ce790 0x7fca940ca8c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 --2- 172.21.15.5:0/778355254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fca940b3ae0 0x7fca940cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 >> 172.21.15.5:0/778355254 conn(0x7fca94066f20 msgr2=0x7fca94159410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 shutdown_connections 2024-04-25T10:12:51.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.591+0000 7fca99200640 1 -- 172.21.15.5:0/778355254 wait complete. 2024-04-25T10:12:51.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:51.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.826+0000 7f5b30a00640 1 Processor -- start 2024-04-25T10:12:51.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.826+0000 7f5b30a00640 1 -- start start 2024-04-25T10:12:51.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.828+0000 7f5b30a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c161200 0x7f5b2c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.828+0000 7f5b30a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b2c0d2380 0x7f5b2c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.828+0000 7f5b30a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0cf410 con 0x7f5b2c0d0600 2024-04-25T10:12:51.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.828+0000 7f5b30a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0d27c0 con 0x7f5b2c0d2380 2024-04-25T10:12:51.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.828+0000 7f5b30a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0b28b0 con 0x7f5b2c161200 2024-04-25T10:12:51.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.829+0000 7f5b21e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b2c0d2380 0x7f5b2c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.829+0000 7f5b2a800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c161200 0x7f5b2c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.829+0000 7f5b2b200640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5b2c0d0600 0x7f5b2c0ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:60240/0 (socket says 172.21.15.5:60240) 2024-04-25T10:12:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.829+0000 7f5b2b200640 1 -- 172.21.15.5:0/4121437540 learned_addr learned my addr 172.21.15.5:0/4121437540 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.830+0000 7f5b21e00640 1 -- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c161200 msgr2=0x7f5b2c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.830+0000 7f5b21e00640 1 --2- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c161200 0x7f5b2c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.830+0000 7f5b21e00640 1 -- 172.21.15.5:0/4121437540 >> v1:172.21.15.80:6789/0 conn(0x7f5b2c0d0600 legacy=0x7f5b2c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.830+0000 7f5b21e00640 1 -- 172.21.15.5:0/4121437540 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0cf6f0 con 0x7f5b2c0d2380 2024-04-25T10:12:51.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.831+0000 7f5b21e00640 1 --2- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0d2380 0x7f5b2c0d00c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5b140097d0 tx=0x7f5b1402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d115aa8c894a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b29e00640 1 -- 172.21.15.5:0/4121437540 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14037070 con 0x7f5b2c0d2380 2024-04-25T10:12:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b29e00640 1 -- 172.21.15.5:0/4121437540 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14032440 con 0x7f5b2c0d2380 2024-04-25T10:12:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b29e00640 1 -- 172.21.15.5:0/4121437540 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14032810 con 0x7f5b2c0d2380 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 -- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0d2380 msgr2=0x7f5b2c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 --2- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0d2380 0x7f5b2c0d00c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5b140097d0 tx=0x7f5b1402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 -- 172.21.15.5:0/4121437540 shutdown_connections 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 --2- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c0d2380 0x7f5b2c0d00c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 --2- 172.21.15.5:0/4121437540 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c161200 0x7f5b2c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 -- 172.21.15.5:0/4121437540 >> 172.21.15.5:0/4121437540 conn(0x7f5b2c08af80 msgr2=0x7f5b2c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.832+0000 7f5b30a00640 1 -- 172.21.15.5:0/4121437540 shutdown_connections 2024-04-25T10:12:51.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.833+0000 7f5b30a00640 1 -- 172.21.15.5:0/4121437540 wait complete. 2024-04-25T10:12:51.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.834+0000 7f5b30a00640 1 Processor -- start 2024-04-25T10:12:51.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.834+0000 7f5b30a00640 1 -- start start 2024-04-25T10:12:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.835+0000 7f5b30a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 0x7f5b2c0b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.835+0000 7f5b30a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c0d2380 0x7f5b2c0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.835+0000 7f5b30a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c161200 0x7f5b2c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.835+0000 7f5b30a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0b3f90 con 0x7f5b2c0d0600 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.835+0000 7f5b30a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0d42b0 con 0x7f5b2c161200 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.835+0000 7f5b30a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0b4110 con 0x7f5b2c0d2380 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 0x7f5b2c0b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 0x7f5b2c0b8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34576/0 (socket says 172.21.15.5:34576) 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 -- 172.21.15.5:0/1625087673 learned_addr learned my addr 172.21.15.5:0/1625087673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2b200640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c161200 0x7f5b2c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b21e00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c0d2380 0x7f5b2c0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 -- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c0d2380 msgr2=0x7f5b2c0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c0d2380 0x7f5b2c0ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 -- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c161200 msgr2=0x7f5b2c159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c161200 0x7f5b2c159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 -- 172.21.15.5:0/1625087673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0b6ce0 con 0x7f5b2c0d0600 2024-04-25T10:12:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2b200640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c161200 0x7f5b2c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.836+0000 7f5b2a800640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 0x7f5b2c0b8e20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00e870 tx=0x7f5b1c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.837+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c017070 con 0x7f5b2c0d0600 2024-04-25T10:12:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.837+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00c8e0 con 0x7f5b2c0d0600 2024-04-25T10:12:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.837+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00ccd0 con 0x7f5b2c0d0600 2024-04-25T10:12:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.837+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c0b6fc0 con 0x7f5b2c0d0600 2024-04-25T10:12:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.837+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c0b7580 con 0x7f5b2c0d0600 2024-04-25T10:12:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.838+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c068f60 con 0x7f5b2c0d0600 2024-04-25T10:12:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.839+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b1c01a070 con 0x7f5b2c0d0600 2024-04-25T10:12:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.840+0000 7f5b23400640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0806b500 0x7f5b0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.840+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b1c08a1a0 con 0x7f5b2c0d0600 2024-04-25T10:12:51.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.841+0000 7f5b21e00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0806b500 0x7f5b0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:51.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.842+0000 7f5b21e00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0806b500 0x7f5b0806d9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5b14034000 tx=0x7f5b1402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:51.845+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b1c058e80 con 0x7f5b2c0d0600 2024-04-25T10:12:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.046+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b2c0d2c20 con 0x7f5b2c0d0600 2024-04-25T10:12:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.049+0000 7f5b23400640 1 -- 172.21.15.5:0/1625087673 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f5b1c058780 con 0x7f5b2c0d0600 2024-04-25T10:12:52.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:52.054 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":120,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:49.050829+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.051+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0806b500 msgr2=0x7f5b0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.051+0000 7f5b30a00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0806b500 0x7f5b0806d9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5b14034000 tx=0x7f5b1402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.051+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 msgr2=0x7f5b2c0b8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.051+0000 7f5b30a00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 0x7f5b2c0b8e20 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00e870 tx=0x7f5b1c00ed30 comp rx=0 tx=0).stop 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 shutdown_connections 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5b0806b500 0x7f5b0806d9c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b2c161200 0x7f5b2c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5b2c0d2380 0x7f5b2c0ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 --2- 172.21.15.5:0/1625087673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b2c0d0600 0x7f5b2c0b8e20 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 >> 172.21.15.5:0/1625087673 conn(0x7f5b2c08af80 msgr2=0x7f5b2c0b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 shutdown_connections 2024-04-25T10:12:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.052+0000 7f5b30a00640 1 -- 172.21.15.5:0/1625087673 wait complete. 2024-04-25T10:12:52.076 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:12:52.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:52.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.279+0000 7f1f9d800640 1 Processor -- start 2024-04-25T10:12:52.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.279+0000 7f1f9d800640 1 -- start start 2024-04-25T10:12:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.281+0000 7f1f9d800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1f9815ce00 0x7f1f9815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.281+0000 7f1f9d800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f980cd300 0x7f1f980cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.281+0000 7f1f9d800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f98071d30 con 0x7f1f98141da0 2024-04-25T10:12:52.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.281+0000 7f1f9d800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980cdcd0 con 0x7f1f980cd300 2024-04-25T10:12:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.281+0000 7f1f9d800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f98142180 con 0x7f1f9815ce00 2024-04-25T10:12:52.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f980cd300 0x7f1f980cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f980cd300 0x7f1f980cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34584/0 (socket says 172.21.15.5:34584) 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 -- 172.21.15.5:0/4008487970 learned_addr learned my addr 172.21.15.5:0/4008487970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f8f400640 1 --2- 172.21.15.5:0/4008487970 >> v2:172.21.15.80:3300/0 conn(0x7f1f9815ce00 0x7f1f9815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 -- 172.21.15.5:0/4008487970 >> v2:172.21.15.80:3300/0 conn(0x7f1f9815ce00 msgr2=0x7f1f9815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 --2- 172.21.15.5:0/4008487970 >> v2:172.21.15.80:3300/0 conn(0x7f1f9815ce00 0x7f1f9815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 -- 172.21.15.5:0/4008487970 >> v1:172.21.15.80:6789/0 conn(0x7f1f98141da0 legacy=0x7f1f9806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.282+0000 7f1f9ce00640 1 -- 172.21.15.5:0/4008487970 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9806d620 con 0x7f1f980cd300 2024-04-25T10:12:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.283+0000 7f1f9ce00640 1 --2- 172.21.15.5:0/4008487970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f980cd790 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806c800 tx=0x7f1f9002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7516869a636c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.283+0000 7f1f97400640 1 -- 172.21.15.5:0/4008487970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90037070 con 0x7f1f980cd300 2024-04-25T10:12:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f97400640 1 -- 172.21.15.5:0/4008487970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f90032440 con 0x7f1f980cd300 2024-04-25T10:12:52.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f97400640 1 -- 172.21.15.5:0/4008487970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90032810 con 0x7f1f980cd300 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 -- 172.21.15.5:0/4008487970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 msgr2=0x7f1f980cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 --2- 172.21.15.5:0/4008487970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f980cd790 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806c800 tx=0x7f1f9002a230 comp rx=0 tx=0).stop 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 -- 172.21.15.5:0/4008487970 shutdown_connections 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 --2- 172.21.15.5:0/4008487970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f980cd790 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 --2- 172.21.15.5:0/4008487970 >> v2:172.21.15.80:3300/0 conn(0x7f1f9815ce00 0x7f1f9815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 -- 172.21.15.5:0/4008487970 >> 172.21.15.5:0/4008487970 conn(0x7f1f98066f20 msgr2=0x7f1f9806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 -- 172.21.15.5:0/4008487970 shutdown_connections 2024-04-25T10:12:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.284+0000 7f1f9d800640 1 -- 172.21.15.5:0/4008487970 wait complete. 2024-04-25T10:12:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.286+0000 7f1f9d800640 1 Processor -- start 2024-04-25T10:12:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.286+0000 7f1f9d800640 1 -- start start 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f9d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f9813ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f9d800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f98141da0 0x7f1f98153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f9d800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f9815ce00 0x7f1f98155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f9d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9806e810 con 0x7f1f980cd300 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f9d800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9806dbe0 con 0x7f1f98141da0 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f9d800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98071000 con 0x7f1f9815ce00 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.287+0000 7f1f97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f9813ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f9813ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34596/0 (socket says 172.21.15.5:34596) 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 -- 172.21.15.5:0/1816224222 learned_addr learned my addr 172.21.15.5:0/1816224222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f8f400640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f98141da0 0x7f1f98153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 -- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f9815ce00 msgr2=0x7f1f98155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f9ce00640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f9815ce00 0x7f1f98155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f9815ce00 0x7f1f98155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 -- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f98141da0 msgr2=0x7f1f98153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f98141da0 0x7f1f98153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 -- 172.21.15.5:0/1816224222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98150420 con 0x7f1f980cd300 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.288+0000 7f1f97e00640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f9813ad40 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c7c0 tx=0x7f1f8800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.289+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88016020 con 0x7f1f980cd300 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.289+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800aa50 con 0x7f1f980cd300 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.289+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800ae40 con 0x7f1f980cd300 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.289+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f981506c0 con 0x7f1f980cd300 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.289+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f98150c80 con 0x7f1f980cd300 2024-04-25T10:12:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.290+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98068f60 con 0x7f1f980cd300 2024-04-25T10:12:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.291+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f88019020 con 0x7f1f980cd300 2024-04-25T10:12:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.292+0000 7f1f94c00640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f7406b4d0 0x7f1f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.292+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f8808a400 con 0x7f1f980cd300 2024-04-25T10:12:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.293+0000 7f1f8f400640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f7406b4d0 0x7f1f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.294+0000 7f1f8f400640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f7406b4d0 0x7f1f7406d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1f80006fd0 tx=0x7f1f80008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.296+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f88020030 con 0x7f1f980cd300 2024-04-25T10:12:52.451 INFO:tasks.ceph.mds.b.smithi080.stderr:2024-04-25T10:12:52.447+0000 7fa6e3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:52.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.491+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f980cc860 con 0x7f1f7406b4d0 2024-04-25T10:12:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.510+0000 7f1f94c00640 1 -- 172.21.15.5:0/1816224222 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f1f980cc860 con 0x7f1f7406b4d0 2024-04-25T10:12:52.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:12:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.515+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f7406b4d0 msgr2=0x7f1f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.515+0000 7f1f9d800640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f7406b4d0 0x7f1f7406d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1f80006fd0 tx=0x7f1f80008040 comp rx=0 tx=0).stop 2024-04-25T10:12:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.515+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 msgr2=0x7f1f9813ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.515+0000 7f1f9d800640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f9813ad40 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800c7c0 tx=0x7f1f8800cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 shutdown_connections 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f9815ce00 0x7f1f98155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1f7406b4d0 0x7f1f7406d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f98141da0 0x7f1f98153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 --2- 172.21.15.5:0/1816224222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f980cd300 0x7f1f9813ad40 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 >> 172.21.15.5:0/1816224222 conn(0x7f1f98066f20 msgr2=0x7f1f98071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 shutdown_connections 2024-04-25T10:12:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.516+0000 7f1f9d800640 1 -- 172.21.15.5:0/1816224222 wait complete. 2024-04-25T10:12:52.540 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-04-25T10:12:50.858661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262300,"kb_used_data":12840,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712420,"statfs":{"total":773094113280,"available":772825518080,"internally_reserved":0,"allocated":13148160,"data_stored":6736260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.831067"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562491+0000","last_change":"2024-04-25T10:12:48.079664+0000","last_active":"2024-04-25T10:12:49.562491+0000","last_peered":"2024-04-25T10:12:49.562491+0000","last_clean":"2024-04-25T10:12:49.562491+0000","last_became_active":"2024-04-25T10:12:48.079309+0000","last_became_peered":"2024-04-25T10:12:48.079309+0000","last_unstale":"2024-04-25T10:12:49.562491+0000","last_undegraded":"2024-04-25T10:12:49.562491+0000","last_fullsized":"2024-04-25T10:12:49.562491+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:32.630363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564022+0000","last_change":"2024-04-25T10:12:48.080025+0000","last_active":"2024-04-25T10:12:49.564022+0000","last_peered":"2024-04-25T10:12:49.564022+0000","last_clean":"2024-04-25T10:12:49.564022+0000","last_became_active":"2024-04-25T10:12:48.077446+0000","last_became_peered":"2024-04-25T10:12:48.077446+0000","last_unstale":"2024-04-25T10:12:49.564022+0000","last_undegraded":"2024-04-25T10:12:49.564022+0000","last_fullsized":"2024-04-25T10:12:49.564022+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.341325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063738+0000","last_change":"2024-04-25T10:12:48.074288+0000","last_active":"2024-04-25T10:12:49.063738+0000","last_peered":"2024-04-25T10:12:49.063738+0000","last_clean":"2024-04-25T10:12:49.063738+0000","last_became_active":"2024-04-25T10:12:48.074000+0000","last_became_peered":"2024-04-25T10:12:48.074000+0000","last_unstale":"2024-04-25T10:12:49.063738+0000","last_undegraded":"2024-04-25T10:12:49.063738+0000","last_fullsized":"2024-04-25T10:12:49.063738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:09.601365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060899+0000","last_change":"2024-04-25T10:12:48.080420+0000","last_active":"2024-04-25T10:12:49.060899+0000","last_peered":"2024-04-25T10:12:49.060899+0000","last_clean":"2024-04-25T10:12:49.060899+0000","last_became_active":"2024-04-25T10:12:48.080135+0000","last_became_peered":"2024-04-25T10:12:48.080135+0000","last_unstale":"2024-04-25T10:12:49.060899+0000","last_undegraded":"2024-04-25T10:12:49.060899+0000","last_fullsized":"2024-04-25T10:12:49.060899+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:46.323277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082415+0000","last_change":"2024-04-25T10:12:48.072381+0000","last_active":"2024-04-25T10:12:49.082415+0000","last_peered":"2024-04-25T10:12:49.082415+0000","last_clean":"2024-04-25T10:12:49.082415+0000","last_became_active":"2024-04-25T10:12:48.069258+0000","last_became_peered":"2024-04-25T10:12:48.069258+0000","last_unstale":"2024-04-25T10:12:49.082415+0000","last_undegraded":"2024-04-25T10:12:49.082415+0000","last_fullsized":"2024-04-25T10:12:49.082415+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:47.255274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562558+0000","last_change":"2024-04-25T10:12:48.081007+0000","last_active":"2024-04-25T10:12:49.562558+0000","last_peered":"2024-04-25T10:12:49.562558+0000","last_clean":"2024-04-25T10:12:49.562558+0000","last_became_active":"2024-04-25T10:12:48.077698+0000","last_became_peered":"2024-04-25T10:12:48.077698+0000","last_unstale":"2024-04-25T10:12:49.562558+0000","last_undegraded":"2024-04-25T10:12:49.562558+0000","last_fullsized":"2024-04-25T10:12:49.562558+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:27.931323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563584+0000","last_change":"2024-04-25T10:12:48.078730+0000","last_active":"2024-04-25T10:12:49.563584+0000","last_peered":"2024-04-25T10:12:49.563584+0000","last_clean":"2024-04-25T10:12:49.563584+0000","last_became_active":"2024-04-25T10:12:48.078069+0000","last_became_peered":"2024-04-25T10:12:48.078069+0000","last_unstale":"2024-04-25T10:12:49.563584+0000","last_undegraded":"2024-04-25T10:12:49.563584+0000","last_fullsized":"2024-04-25T10:12:49.563584+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:50.507728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562888+0000","last_change":"2024-04-25T10:12:48.065555+0000","last_active":"2024-04-25T10:12:49.562888+0000","last_peered":"2024-04-25T10:12:49.562888+0000","last_clean":"2024-04-25T10:12:49.562888+0000","last_became_active":"2024-04-25T10:12:48.064789+0000","last_became_peered":"2024-04-25T10:12:48.064789+0000","last_unstale":"2024-04-25T10:12:49.562888+0000","last_undegraded":"2024-04-25T10:12:49.562888+0000","last_fullsized":"2024-04-25T10:12:49.562888+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:13.617100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061140+0000","last_change":"2024-04-25T10:12:48.074447+0000","last_active":"2024-04-25T10:12:49.061140+0000","last_peered":"2024-04-25T10:12:49.061140+0000","last_clean":"2024-04-25T10:12:49.061140+0000","last_became_active":"2024-04-25T10:12:48.074125+0000","last_became_peered":"2024-04-25T10:12:48.074125+0000","last_unstale":"2024-04-25T10:12:49.061140+0000","last_undegraded":"2024-04-25T10:12:49.061140+0000","last_fullsized":"2024-04-25T10:12:49.061140+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:05.847552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564597+0000","last_change":"2024-04-25T10:12:48.080759+0000","last_active":"2024-04-25T10:12:49.564597+0000","last_peered":"2024-04-25T10:12:49.564597+0000","last_clean":"2024-04-25T10:12:49.564597+0000","last_became_active":"2024-04-25T10:12:48.080310+0000","last_became_peered":"2024-04-25T10:12:48.080310+0000","last_unstale":"2024-04-25T10:12:49.564597+0000","last_undegraded":"2024-04-25T10:12:49.564597+0000","last_fullsized":"2024-04-25T10:12:49.564597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:20.810249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562654+0000","last_change":"2024-04-25T10:12:48.069364+0000","last_active":"2024-04-25T10:12:49.562654+0000","last_peered":"2024-04-25T10:12:49.562654+0000","last_clean":"2024-04-25T10:12:49.562654+0000","last_became_active":"2024-04-25T10:12:48.068899+0000","last_became_peered":"2024-04-25T10:12:48.068899+0000","last_unstale":"2024-04-25T10:12:49.562654+0000","last_undegraded":"2024-04-25T10:12:49.562654+0000","last_fullsized":"2024-04-25T10:12:49.562654+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:34.325526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061775+0000","last_change":"2024-04-25T10:12:48.088866+0000","last_active":"2024-04-25T10:12:49.061775+0000","last_peered":"2024-04-25T10:12:49.061775+0000","last_clean":"2024-04-25T10:12:49.061775+0000","last_became_active":"2024-04-25T10:12:48.088575+0000","last_became_peered":"2024-04-25T10:12:48.088575+0000","last_unstale":"2024-04-25T10:12:49.061775+0000","last_undegraded":"2024-04-25T10:12:49.061775+0000","last_fullsized":"2024-04-25T10:12:49.061775+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:02.812234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564266+0000","last_change":"2024-04-25T10:12:48.078288+0000","last_active":"2024-04-25T10:12:49.564266+0000","last_peered":"2024-04-25T10:12:49.564266+0000","last_clean":"2024-04-25T10:12:49.564266+0000","last_became_active":"2024-04-25T10:12:48.077143+0000","last_became_peered":"2024-04-25T10:12:48.077143+0000","last_unstale":"2024-04-25T10:12:49.564266+0000","last_undegraded":"2024-04-25T10:12:49.564266+0000","last_fullsized":"2024-04-25T10:12:49.564266+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:07.627738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061189+0000","last_change":"2024-04-25T10:12:48.083403+0000","last_active":"2024-04-25T10:12:49.061189+0000","last_peered":"2024-04-25T10:12:49.061189+0000","last_clean":"2024-04-25T10:12:49.061189+0000","last_became_active":"2024-04-25T10:12:48.081548+0000","last_became_peered":"2024-04-25T10:12:48.081548+0000","last_unstale":"2024-04-25T10:12:49.061189+0000","last_undegraded":"2024-04-25T10:12:49.061189+0000","last_fullsized":"2024-04-25T10:12:49.061189+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:30.433166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209850+0000","last_change":"2024-04-25T10:12:48.051857+0000","last_active":"2024-04-25T10:12:49.209850+0000","last_peered":"2024-04-25T10:12:49.209850+0000","last_clean":"2024-04-25T10:12:49.209850+0000","last_became_active":"2024-04-25T10:12:48.051305+0000","last_became_peered":"2024-04-25T10:12:48.051305+0000","last_unstale":"2024-04-25T10:12:49.209850+0000","last_undegraded":"2024-04-25T10:12:49.209850+0000","last_fullsized":"2024-04-25T10:12:49.209850+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.764246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061324+0000","last_change":"2024-04-25T10:12:48.073091+0000","last_active":"2024-04-25T10:12:49.061324+0000","last_peered":"2024-04-25T10:12:49.061324+0000","last_clean":"2024-04-25T10:12:49.061324+0000","last_became_active":"2024-04-25T10:12:48.072795+0000","last_became_peered":"2024-04-25T10:12:48.072795+0000","last_unstale":"2024-04-25T10:12:49.061324+0000","last_undegraded":"2024-04-25T10:12:49.061324+0000","last_fullsized":"2024-04-25T10:12:49.061324+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:14.812657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062506+0000","last_change":"2024-04-25T10:12:48.080434+0000","last_active":"2024-04-25T10:12:49.062506+0000","last_peered":"2024-04-25T10:12:49.062506+0000","last_clean":"2024-04-25T10:12:49.062506+0000","last_became_active":"2024-04-25T10:12:48.076222+0000","last_became_peered":"2024-04-25T10:12:48.076222+0000","last_unstale":"2024-04-25T10:12:49.062506+0000","last_undegraded":"2024-04-25T10:12:49.062506+0000","last_fullsized":"2024-04-25T10:12:49.062506+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.739649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062042+0000","last_change":"2024-04-25T10:12:48.074957+0000","last_active":"2024-04-25T10:12:49.062042+0000","last_peered":"2024-04-25T10:12:49.062042+0000","last_clean":"2024-04-25T10:12:49.062042+0000","last_became_active":"2024-04-25T10:12:48.074641+0000","last_became_peered":"2024-04-25T10:12:48.074641+0000","last_unstale":"2024-04-25T10:12:49.062042+0000","last_undegraded":"2024-04-25T10:12:49.062042+0000","last_fullsized":"2024-04-25T10:12:49.062042+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:57.138797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082885+0000","last_change":"2024-04-25T10:12:48.080055+0000","last_active":"2024-04-25T10:12:49.082885+0000","last_peered":"2024-04-25T10:12:49.082885+0000","last_clean":"2024-04-25T10:12:49.082885+0000","last_became_active":"2024-04-25T10:12:48.078711+0000","last_became_peered":"2024-04-25T10:12:48.078711+0000","last_unstale":"2024-04-25T10:12:49.082885+0000","last_undegraded":"2024-04-25T10:12:49.082885+0000","last_fullsized":"2024-04-25T10:12:49.082885+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:32.566854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564083+0000","last_change":"2024-04-25T10:12:48.086034+0000","last_active":"2024-04-25T10:12:49.564083+0000","last_peered":"2024-04-25T10:12:49.564083+0000","last_clean":"2024-04-25T10:12:49.564083+0000","last_became_active":"2024-04-25T10:12:48.085746+0000","last_became_peered":"2024-04-25T10:12:48.085746+0000","last_unstale":"2024-04-25T10:12:49.564083+0000","last_undegraded":"2024-04-25T10:12:49.564083+0000","last_fullsized":"2024-04-25T10:12:49.564083+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:32.379195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082587+0000","last_change":"2024-04-25T10:12:48.086574+0000","last_active":"2024-04-25T10:12:49.082587+0000","last_peered":"2024-04-25T10:12:49.082587+0000","last_clean":"2024-04-25T10:12:49.082587+0000","last_became_active":"2024-04-25T10:12:48.086048+0000","last_became_peered":"2024-04-25T10:12:48.086048+0000","last_unstale":"2024-04-25T10:12:49.082587+0000","last_undegraded":"2024-04-25T10:12:49.082587+0000","last_fullsized":"2024-04-25T10:12:49.082587+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:08.898195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209872+0000","last_change":"2024-04-25T10:12:48.071488+0000","last_active":"2024-04-25T10:12:49.209872+0000","last_peered":"2024-04-25T10:12:49.209872+0000","last_clean":"2024-04-25T10:12:49.209872+0000","last_became_active":"2024-04-25T10:12:48.069583+0000","last_became_peered":"2024-04-25T10:12:48.069583+0000","last_unstale":"2024-04-25T10:12:49.209872+0000","last_undegraded":"2024-04-25T10:12:49.209872+0000","last_fullsized":"2024-04-25T10:12:49.209872+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:08.419073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209848+0000","last_change":"2024-04-25T10:12:48.065630+0000","last_active":"2024-04-25T10:12:49.209848+0000","last_peered":"2024-04-25T10:12:49.209848+0000","last_clean":"2024-04-25T10:12:49.209848+0000","last_became_active":"2024-04-25T10:12:48.058574+0000","last_became_peered":"2024-04-25T10:12:48.058574+0000","last_unstale":"2024-04-25T10:12:49.209848+0000","last_undegraded":"2024-04-25T10:12:49.209848+0000","last_fullsized":"2024-04-25T10:12:49.209848+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:03.256609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209869+0000","last_change":"2024-04-25T10:12:48.077119+0000","last_active":"2024-04-25T10:12:49.209869+0000","last_peered":"2024-04-25T10:12:49.209869+0000","last_clean":"2024-04-25T10:12:49.209869+0000","last_became_active":"2024-04-25T10:12:48.076823+0000","last_became_peered":"2024-04-25T10:12:48.076823+0000","last_unstale":"2024-04-25T10:12:49.209869+0000","last_undegraded":"2024-04-25T10:12:49.209869+0000","last_fullsized":"2024-04-25T10:12:49.209869+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.556464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562520+0000","last_change":"2024-04-25T10:12:48.073500+0000","last_active":"2024-04-25T10:12:49.562520+0000","last_peered":"2024-04-25T10:12:49.562520+0000","last_clean":"2024-04-25T10:12:49.562520+0000","last_became_active":"2024-04-25T10:12:48.070704+0000","last_became_peered":"2024-04-25T10:12:48.070704+0000","last_unstale":"2024-04-25T10:12:49.562520+0000","last_undegraded":"2024-04-25T10:12:49.562520+0000","last_fullsized":"2024-04-25T10:12:49.562520+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:06.395728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564846+0000","last_change":"2024-04-25T10:12:48.081614+0000","last_active":"2024-04-25T10:12:49.564846+0000","last_peered":"2024-04-25T10:12:49.564846+0000","last_clean":"2024-04-25T10:12:49.564846+0000","last_became_active":"2024-04-25T10:12:48.081349+0000","last_became_peered":"2024-04-25T10:12:48.081349+0000","last_unstale":"2024-04-25T10:12:49.564846+0000","last_undegraded":"2024-04-25T10:12:49.564846+0000","last_fullsized":"2024-04-25T10:12:49.564846+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.586918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082979+0000","last_change":"2024-04-25T10:12:48.078683+0000","last_active":"2024-04-25T10:12:49.082979+0000","last_peered":"2024-04-25T10:12:49.082979+0000","last_clean":"2024-04-25T10:12:49.082979+0000","last_became_active":"2024-04-25T10:12:48.075664+0000","last_became_peered":"2024-04-25T10:12:48.075664+0000","last_unstale":"2024-04-25T10:12:49.082979+0000","last_undegraded":"2024-04-25T10:12:49.082979+0000","last_fullsized":"2024-04-25T10:12:49.082979+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:20.777207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062204+0000","last_change":"2024-04-25T10:12:48.082441+0000","last_active":"2024-04-25T10:12:49.062204+0000","last_peered":"2024-04-25T10:12:49.062204+0000","last_clean":"2024-04-25T10:12:49.062204+0000","last_became_active":"2024-04-25T10:12:48.082171+0000","last_became_peered":"2024-04-25T10:12:48.082171+0000","last_unstale":"2024-04-25T10:12:49.062204+0000","last_undegraded":"2024-04-25T10:12:49.062204+0000","last_fullsized":"2024-04-25T10:12:49.062204+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:13.212296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063533+0000","last_change":"2024-04-25T10:12:48.080609+0000","last_active":"2024-04-25T10:12:49.063533+0000","last_peered":"2024-04-25T10:12:49.063533+0000","last_clean":"2024-04-25T10:12:49.063533+0000","last_became_active":"2024-04-25T10:12:48.080276+0000","last_became_peered":"2024-04-25T10:12:48.080276+0000","last_unstale":"2024-04-25T10:12:49.063533+0000","last_undegraded":"2024-04-25T10:12:49.063533+0000","last_fullsized":"2024-04-25T10:12:49.063533+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:23.321505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060447+0000","last_change":"2024-04-25T10:12:48.055133+0000","last_active":"2024-04-25T10:12:49.060447+0000","last_peered":"2024-04-25T10:12:49.060447+0000","last_clean":"2024-04-25T10:12:49.060447+0000","last_became_active":"2024-04-25T10:12:48.054361+0000","last_became_peered":"2024-04-25T10:12:48.054361+0000","last_unstale":"2024-04-25T10:12:49.060447+0000","last_undegraded":"2024-04-25T10:12:49.060447+0000","last_fullsized":"2024-04-25T10:12:49.060447+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:38.308061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.561805+0000","last_change":"2024-04-25T10:12:48.084195+0000","last_active":"2024-04-25T10:12:49.561805+0000","last_peered":"2024-04-25T10:12:49.561805+0000","last_clean":"2024-04-25T10:12:49.561805+0000","last_became_active":"2024-04-25T10:12:48.082321+0000","last_became_peered":"2024-04-25T10:12:48.082321+0000","last_unstale":"2024-04-25T10:12:49.561805+0000","last_undegraded":"2024-04-25T10:12:49.561805+0000","last_fullsized":"2024-04-25T10:12:49.561805+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.739649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.561883+0000","last_change":"2024-04-25T10:12:48.071573+0000","last_active":"2024-04-25T10:12:49.561883+0000","last_peered":"2024-04-25T10:12:49.561883+0000","last_clean":"2024-04-25T10:12:49.561883+0000","last_became_active":"2024-04-25T10:12:48.071238+0000","last_became_peered":"2024-04-25T10:12:48.071238+0000","last_unstale":"2024-04-25T10:12:49.561883+0000","last_undegraded":"2024-04-25T10:12:49.561883+0000","last_fullsized":"2024-04-25T10:12:49.561883+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:07.231763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564791+0000","last_change":"2024-04-25T10:12:48.072279+0000","last_active":"2024-04-25T10:12:49.564791+0000","last_peered":"2024-04-25T10:12:49.564791+0000","last_clean":"2024-04-25T10:12:49.564791+0000","last_became_active":"2024-04-25T10:12:48.071954+0000","last_became_peered":"2024-04-25T10:12:48.071954+0000","last_unstale":"2024-04-25T10:12:49.564791+0000","last_undegraded":"2024-04-25T10:12:49.564791+0000","last_fullsized":"2024-04-25T10:12:49.564791+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:53.982718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"119'4","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083596+0000","last_change":"2024-04-25T10:12:47.065942+0000","last_active":"2024-04-25T10:12:49.083596+0000","last_peered":"2024-04-25T10:12:49.083596+0000","last_clean":"2024-04-25T10:12:49.083596+0000","last_became_active":"2024-04-25T10:12:47.061657+0000","last_became_peered":"2024-04-25T10:12:47.061657+0000","last_unstale":"2024-04-25T10:12:49.083596+0000","last_undegraded":"2024-04-25T10:12:49.083596+0000","last_fullsized":"2024-04-25T10:12:49.083596+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:01.313940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563274+0000","last_change":"2024-04-25T10:12:48.077605+0000","last_active":"2024-04-25T10:12:49.563274+0000","last_peered":"2024-04-25T10:12:49.563274+0000","last_clean":"2024-04-25T10:12:49.563274+0000","last_became_active":"2024-04-25T10:12:48.077287+0000","last_became_peered":"2024-04-25T10:12:48.077287+0000","last_unstale":"2024-04-25T10:12:49.563274+0000","last_undegraded":"2024-04-25T10:12:49.563274+0000","last_fullsized":"2024-04-25T10:12:49.563274+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:25.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062521+0000","last_change":"2024-04-25T10:12:47.074224+0000","last_active":"2024-04-25T10:12:49.062521+0000","last_peered":"2024-04-25T10:12:49.062521+0000","last_clean":"2024-04-25T10:12:49.062521+0000","last_became_active":"2024-04-25T10:12:47.073396+0000","last_became_peered":"2024-04-25T10:12:47.073396+0000","last_unstale":"2024-04-25T10:12:49.062521+0000","last_undegraded":"2024-04-25T10:12:49.062521+0000","last_fullsized":"2024-04-25T10:12:49.062521+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:31.362562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562335+0000","last_change":"2024-04-25T10:12:48.072462+0000","last_active":"2024-04-25T10:12:49.562335+0000","last_peered":"2024-04-25T10:12:49.562335+0000","last_clean":"2024-04-25T10:12:49.562335+0000","last_became_active":"2024-04-25T10:12:48.072186+0000","last_became_peered":"2024-04-25T10:12:48.072186+0000","last_unstale":"2024-04-25T10:12:49.562335+0000","last_undegraded":"2024-04-25T10:12:49.562335+0000","last_fullsized":"2024-04-25T10:12:49.562335+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.950145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.210186+0000","last_change":"2024-04-25T10:12:47.069358+0000","last_active":"2024-04-25T10:12:49.210186+0000","last_peered":"2024-04-25T10:12:49.210186+0000","last_clean":"2024-04-25T10:12:49.210186+0000","last_became_active":"2024-04-25T10:12:47.068946+0000","last_became_peered":"2024-04-25T10:12:47.068946+0000","last_unstale":"2024-04-25T10:12:49.210186+0000","last_undegraded":"2024-04-25T10:12:49.210186+0000","last_fullsized":"2024-04-25T10:12:49.210186+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.128337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061356+0000","last_change":"2024-04-25T10:12:48.079047+0000","last_active":"2024-04-25T10:12:49.061356+0000","last_peered":"2024-04-25T10:12:49.061356+0000","last_clean":"2024-04-25T10:12:49.061356+0000","last_became_active":"2024-04-25T10:12:48.078617+0000","last_became_peered":"2024-04-25T10:12:48.078617+0000","last_unstale":"2024-04-25T10:12:49.061356+0000","last_undegraded":"2024-04-25T10:12:49.061356+0000","last_fullsized":"2024-04-25T10:12:49.061356+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:12.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563954+0000","last_change":"2024-04-25T10:12:47.081688+0000","last_active":"2024-04-25T10:12:49.563954+0000","last_peered":"2024-04-25T10:12:49.563954+0000","last_clean":"2024-04-25T10:12:49.563954+0000","last_became_active":"2024-04-25T10:12:47.080260+0000","last_became_peered":"2024-04-25T10:12:47.080260+0000","last_unstale":"2024-04-25T10:12:49.563954+0000","last_undegraded":"2024-04-25T10:12:49.563954+0000","last_fullsized":"2024-04-25T10:12:49.563954+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:19.760574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562947+0000","last_change":"2024-04-25T10:12:48.075568+0000","last_active":"2024-04-25T10:12:49.562947+0000","last_peered":"2024-04-25T10:12:49.562947+0000","last_clean":"2024-04-25T10:12:49.562947+0000","last_became_active":"2024-04-25T10:12:48.070274+0000","last_became_peered":"2024-04-25T10:12:48.070274+0000","last_unstale":"2024-04-25T10:12:49.562947+0000","last_undegraded":"2024-04-25T10:12:49.562947+0000","last_fullsized":"2024-04-25T10:12:49.562947+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:52.017975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562665+0000","last_change":"2024-04-25T10:12:47.068853+0000","last_active":"2024-04-25T10:12:49.562665+0000","last_peered":"2024-04-25T10:12:49.562665+0000","last_clean":"2024-04-25T10:12:49.562665+0000","last_became_active":"2024-04-25T10:12:47.068568+0000","last_became_peered":"2024-04-25T10:12:47.068568+0000","last_unstale":"2024-04-25T10:12:49.562665+0000","last_undegraded":"2024-04-25T10:12:49.562665+0000","last_fullsized":"2024-04-25T10:12:49.562665+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:51.490936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562232+0000","last_change":"2024-04-25T10:12:48.077525+0000","last_active":"2024-04-25T10:12:49.562232+0000","last_peered":"2024-04-25T10:12:49.562232+0000","last_clean":"2024-04-25T10:12:49.562232+0000","last_became_active":"2024-04-25T10:12:48.074911+0000","last_became_peered":"2024-04-25T10:12:48.074911+0000","last_unstale":"2024-04-25T10:12:49.562232+0000","last_undegraded":"2024-04-25T10:12:49.562232+0000","last_fullsized":"2024-04-25T10:12:49.562232+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:32.034888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563873+0000","last_change":"2024-04-25T10:12:47.073230+0000","last_active":"2024-04-25T10:12:49.563873+0000","last_peered":"2024-04-25T10:12:49.563873+0000","last_clean":"2024-04-25T10:12:49.563873+0000","last_became_active":"2024-04-25T10:12:47.072882+0000","last_became_peered":"2024-04-25T10:12:47.072882+0000","last_unstale":"2024-04-25T10:12:49.563873+0000","last_undegraded":"2024-04-25T10:12:49.563873+0000","last_fullsized":"2024-04-25T10:12:49.563873+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.162716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061092+0000","last_change":"2024-04-25T10:12:48.069157+0000","last_active":"2024-04-25T10:12:49.061092+0000","last_peered":"2024-04-25T10:12:49.061092+0000","last_clean":"2024-04-25T10:12:49.061092+0000","last_became_active":"2024-04-25T10:12:48.068847+0000","last_became_peered":"2024-04-25T10:12:48.068847+0000","last_unstale":"2024-04-25T10:12:49.061092+0000","last_undegraded":"2024-04-25T10:12:49.061092+0000","last_fullsized":"2024-04-25T10:12:49.061092+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:02.330573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562109+0000","last_change":"2024-04-25T10:12:47.059987+0000","last_active":"2024-04-25T10:12:49.562109+0000","last_peered":"2024-04-25T10:12:49.562109+0000","last_clean":"2024-04-25T10:12:49.562109+0000","last_became_active":"2024-04-25T10:12:47.059665+0000","last_became_peered":"2024-04-25T10:12:47.059665+0000","last_unstale":"2024-04-25T10:12:49.562109+0000","last_undegraded":"2024-04-25T10:12:49.562109+0000","last_fullsized":"2024-04-25T10:12:49.562109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.022420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563854+0000","last_change":"2024-04-25T10:12:48.064245+0000","last_active":"2024-04-25T10:12:49.563854+0000","last_peered":"2024-04-25T10:12:49.563854+0000","last_clean":"2024-04-25T10:12:49.563854+0000","last_became_active":"2024-04-25T10:12:48.063800+0000","last_became_peered":"2024-04-25T10:12:48.063800+0000","last_unstale":"2024-04-25T10:12:49.563854+0000","last_undegraded":"2024-04-25T10:12:49.563854+0000","last_fullsized":"2024-04-25T10:12:49.563854+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:14.994534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060176+0000","last_change":"2024-04-25T10:12:47.079725+0000","last_active":"2024-04-25T10:12:49.060176+0000","last_peered":"2024-04-25T10:12:49.060176+0000","last_clean":"2024-04-25T10:12:49.060176+0000","last_became_active":"2024-04-25T10:12:47.079436+0000","last_became_peered":"2024-04-25T10:12:47.079436+0000","last_unstale":"2024-04-25T10:12:49.060176+0000","last_undegraded":"2024-04-25T10:12:49.060176+0000","last_fullsized":"2024-04-25T10:12:49.060176+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:24.995347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.210044+0000","last_change":"2024-04-25T10:12:48.064869+0000","last_active":"2024-04-25T10:12:49.210044+0000","last_peered":"2024-04-25T10:12:49.210044+0000","last_clean":"2024-04-25T10:12:49.210044+0000","last_became_active":"2024-04-25T10:12:48.064543+0000","last_became_peered":"2024-04-25T10:12:48.064543+0000","last_unstale":"2024-04-25T10:12:49.210044+0000","last_undegraded":"2024-04-25T10:12:49.210044+0000","last_fullsized":"2024-04-25T10:12:49.210044+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:39.023532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061335+0000","last_change":"2024-04-25T10:12:47.069058+0000","last_active":"2024-04-25T10:12:49.061335+0000","last_peered":"2024-04-25T10:12:49.061335+0000","last_clean":"2024-04-25T10:12:49.061335+0000","last_became_active":"2024-04-25T10:12:47.068738+0000","last_became_peered":"2024-04-25T10:12:47.068738+0000","last_unstale":"2024-04-25T10:12:49.061335+0000","last_undegraded":"2024-04-25T10:12:49.061335+0000","last_fullsized":"2024-04-25T10:12:49.061335+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.747613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061220+0000","last_change":"2024-04-25T10:12:48.079558+0000","last_active":"2024-04-25T10:12:49.061220+0000","last_peered":"2024-04-25T10:12:49.061220+0000","last_clean":"2024-04-25T10:12:49.061220+0000","last_became_active":"2024-04-25T10:12:48.079268+0000","last_became_peered":"2024-04-25T10:12:48.079268+0000","last_unstale":"2024-04-25T10:12:49.061220+0000","last_undegraded":"2024-04-25T10:12:49.061220+0000","last_fullsized":"2024-04-25T10:12:49.061220+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:20.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060781+0000","last_change":"2024-04-25T10:12:47.071081+0000","last_active":"2024-04-25T10:12:49.060781+0000","last_peered":"2024-04-25T10:12:49.060781+0000","last_clean":"2024-04-25T10:12:49.060781+0000","last_became_active":"2024-04-25T10:12:47.070805+0000","last_became_peered":"2024-04-25T10:12:47.070805+0000","last_unstale":"2024-04-25T10:12:49.060781+0000","last_undegraded":"2024-04-25T10:12:49.060781+0000","last_fullsized":"2024-04-25T10:12:49.060781+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:02.239976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063432+0000","last_change":"2024-04-25T10:12:48.058661+0000","last_active":"2024-04-25T10:12:49.063432+0000","last_peered":"2024-04-25T10:12:49.063432+0000","last_clean":"2024-04-25T10:12:49.063432+0000","last_became_active":"2024-04-25T10:12:48.057952+0000","last_became_peered":"2024-04-25T10:12:48.057952+0000","last_unstale":"2024-04-25T10:12:49.063432+0000","last_undegraded":"2024-04-25T10:12:49.063432+0000","last_fullsized":"2024-04-25T10:12:49.063432+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:54.720827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"119'3","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562741+0000","last_change":"2024-04-25T10:12:47.067181+0000","last_active":"2024-04-25T10:12:49.562741+0000","last_peered":"2024-04-25T10:12:49.562741+0000","last_clean":"2024-04-25T10:12:49.562741+0000","last_became_active":"2024-04-25T10:12:47.066867+0000","last_became_peered":"2024-04-25T10:12:47.066867+0000","last_unstale":"2024-04-25T10:12:49.562741+0000","last_undegraded":"2024-04-25T10:12:49.562741+0000","last_fullsized":"2024-04-25T10:12:49.562741+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:33.946952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083250+0000","last_change":"2024-04-25T10:12:48.084078+0000","last_active":"2024-04-25T10:12:49.083250+0000","last_peered":"2024-04-25T10:12:49.083250+0000","last_clean":"2024-04-25T10:12:49.083250+0000","last_became_active":"2024-04-25T10:12:48.080495+0000","last_became_peered":"2024-04-25T10:12:48.080495+0000","last_unstale":"2024-04-25T10:12:49.083250+0000","last_undegraded":"2024-04-25T10:12:49.083250+0000","last_fullsized":"2024-04-25T10:12:49.083250+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.556464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"119'6","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564277+0000","last_change":"2024-04-25T10:12:47.079869+0000","last_active":"2024-04-25T10:12:49.564277+0000","last_peered":"2024-04-25T10:12:49.564277+0000","last_clean":"2024-04-25T10:12:49.564277+0000","last_became_active":"2024-04-25T10:12:47.079541+0000","last_became_peered":"2024-04-25T10:12:47.079541+0000","last_unstale":"2024-04-25T10:12:49.564277+0000","last_undegraded":"2024-04-25T10:12:49.564277+0000","last_fullsized":"2024-04-25T10:12:49.564277+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:02.539831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082805+0000","last_change":"2024-04-25T10:12:48.072791+0000","last_active":"2024-04-25T10:12:49.082805+0000","last_peered":"2024-04-25T10:12:49.082805+0000","last_clean":"2024-04-25T10:12:49.082805+0000","last_became_active":"2024-04-25T10:12:48.072440+0000","last_became_peered":"2024-04-25T10:12:48.072440+0000","last_unstale":"2024-04-25T10:12:49.082805+0000","last_undegraded":"2024-04-25T10:12:49.082805+0000","last_fullsized":"2024-04-25T10:12:49.082805+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.752949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564636+0000","last_change":"2024-04-25T10:12:47.074406+0000","last_active":"2024-04-25T10:12:49.564636+0000","last_peered":"2024-04-25T10:12:49.564636+0000","last_clean":"2024-04-25T10:12:49.564636+0000","last_became_active":"2024-04-25T10:12:47.074099+0000","last_became_peered":"2024-04-25T10:12:47.074099+0000","last_unstale":"2024-04-25T10:12:49.564636+0000","last_undegraded":"2024-04-25T10:12:49.564636+0000","last_fullsized":"2024-04-25T10:12:49.564636+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:45.215163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562871+0000","last_change":"2024-04-25T10:12:48.076692+0000","last_active":"2024-04-25T10:12:49.562871+0000","last_peered":"2024-04-25T10:12:49.562871+0000","last_clean":"2024-04-25T10:12:49.562871+0000","last_became_active":"2024-04-25T10:12:48.076346+0000","last_became_peered":"2024-04-25T10:12:48.076346+0000","last_unstale":"2024-04-25T10:12:49.562871+0000","last_undegraded":"2024-04-25T10:12:49.562871+0000","last_fullsized":"2024-04-25T10:12:49.562871+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:48.720124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062143+0000","last_change":"2024-04-25T10:12:47.076358+0000","last_active":"2024-04-25T10:12:49.062143+0000","last_peered":"2024-04-25T10:12:49.062143+0000","last_clean":"2024-04-25T10:12:49.062143+0000","last_became_active":"2024-04-25T10:12:47.074226+0000","last_became_peered":"2024-04-25T10:12:47.074226+0000","last_unstale":"2024-04-25T10:12:49.062143+0000","last_undegraded":"2024-04-25T10:12:49.062143+0000","last_fullsized":"2024-04-25T10:12:49.062143+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:21.775250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061596+0000","last_change":"2024-04-25T10:12:48.070738+0000","last_active":"2024-04-25T10:12:49.061596+0000","last_peered":"2024-04-25T10:12:49.061596+0000","last_clean":"2024-04-25T10:12:49.061596+0000","last_became_active":"2024-04-25T10:12:48.070261+0000","last_became_peered":"2024-04-25T10:12:48.070261+0000","last_unstale":"2024-04-25T10:12:49.061596+0000","last_undegraded":"2024-04-25T10:12:49.061596+0000","last_fullsized":"2024-04-25T10:12:49.061596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:04.535025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562777+0000","last_change":"2024-04-25T10:12:47.073287+0000","last_active":"2024-04-25T10:12:49.562777+0000","last_peered":"2024-04-25T10:12:49.562777+0000","last_clean":"2024-04-25T10:12:49.562777+0000","last_became_active":"2024-04-25T10:12:47.072947+0000","last_became_peered":"2024-04-25T10:12:47.072947+0000","last_unstale":"2024-04-25T10:12:49.562777+0000","last_undegraded":"2024-04-25T10:12:49.562777+0000","last_fullsized":"2024-04-25T10:12:49.562777+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:20.125568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061087+0000","last_change":"2024-04-25T10:12:48.075362+0000","last_active":"2024-04-25T10:12:49.061087+0000","last_peered":"2024-04-25T10:12:49.061087+0000","last_clean":"2024-04-25T10:12:49.061087+0000","last_became_active":"2024-04-25T10:12:48.074561+0000","last_became_peered":"2024-04-25T10:12:48.074561+0000","last_unstale":"2024-04-25T10:12:49.061087+0000","last_undegraded":"2024-04-25T10:12:49.061087+0000","last_fullsized":"2024-04-25T10:12:49.061087+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:57.117947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062523+0000","last_change":"2024-04-25T10:12:47.077801+0000","last_active":"2024-04-25T10:12:49.062523+0000","last_peered":"2024-04-25T10:12:49.062523+0000","last_clean":"2024-04-25T10:12:49.062523+0000","last_became_active":"2024-04-25T10:12:47.076171+0000","last_became_peered":"2024-04-25T10:12:47.076171+0000","last_unstale":"2024-04-25T10:12:49.062523+0000","last_undegraded":"2024-04-25T10:12:49.062523+0000","last_fullsized":"2024-04-25T10:12:49.062523+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.491095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562567+0000","last_change":"2024-04-25T10:12:48.081146+0000","last_active":"2024-04-25T10:12:49.562567+0000","last_peered":"2024-04-25T10:12:49.562567+0000","last_clean":"2024-04-25T10:12:49.562567+0000","last_became_active":"2024-04-25T10:12:48.080871+0000","last_became_peered":"2024-04-25T10:12:48.080871+0000","last_unstale":"2024-04-25T10:12:49.562567+0000","last_undegraded":"2024-04-25T10:12:49.562567+0000","last_fullsized":"2024-04-25T10:12:49.562567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:58.856527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062810+0000","last_change":"2024-04-25T10:12:47.080456+0000","last_active":"2024-04-25T10:12:49.062810+0000","last_peered":"2024-04-25T10:12:49.062810+0000","last_clean":"2024-04-25T10:12:49.062810+0000","last_became_active":"2024-04-25T10:12:47.080132+0000","last_became_peered":"2024-04-25T10:12:47.080132+0000","last_unstale":"2024-04-25T10:12:49.062810+0000","last_undegraded":"2024-04-25T10:12:49.062810+0000","last_fullsized":"2024-04-25T10:12:49.062810+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:22.324692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.565188+0000","last_change":"2024-04-25T10:12:48.077677+0000","last_active":"2024-04-25T10:12:49.565188+0000","last_peered":"2024-04-25T10:12:49.565188+0000","last_clean":"2024-04-25T10:12:49.565188+0000","last_became_active":"2024-04-25T10:12:48.077353+0000","last_became_peered":"2024-04-25T10:12:48.077353+0000","last_unstale":"2024-04-25T10:12:49.565188+0000","last_undegraded":"2024-04-25T10:12:49.565188+0000","last_fullsized":"2024-04-25T10:12:49.565188+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:39.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083044+0000","last_change":"2024-04-25T10:12:47.067990+0000","last_active":"2024-04-25T10:12:49.083044+0000","last_peered":"2024-04-25T10:12:49.083044+0000","last_clean":"2024-04-25T10:12:49.083044+0000","last_became_active":"2024-04-25T10:12:47.067557+0000","last_became_peered":"2024-04-25T10:12:47.067557+0000","last_unstale":"2024-04-25T10:12:49.083044+0000","last_undegraded":"2024-04-25T10:12:49.083044+0000","last_fullsized":"2024-04-25T10:12:49.083044+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:03.497563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083262+0000","last_change":"2024-04-25T10:12:48.077945+0000","last_active":"2024-04-25T10:12:49.083262+0000","last_peered":"2024-04-25T10:12:49.083262+0000","last_clean":"2024-04-25T10:12:49.083262+0000","last_became_active":"2024-04-25T10:12:48.077668+0000","last_became_peered":"2024-04-25T10:12:48.077668+0000","last_unstale":"2024-04-25T10:12:49.083262+0000","last_undegraded":"2024-04-25T10:12:49.083262+0000","last_fullsized":"2024-04-25T10:12:49.083262+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:03.256609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562306+0000","last_change":"2024-04-25T10:12:47.069691+0000","last_active":"2024-04-25T10:12:49.562306+0000","last_peered":"2024-04-25T10:12:49.562306+0000","last_clean":"2024-04-25T10:12:49.562306+0000","last_became_active":"2024-04-25T10:12:47.069429+0000","last_became_peered":"2024-04-25T10:12:47.069429+0000","last_unstale":"2024-04-25T10:12:49.562306+0000","last_undegraded":"2024-04-25T10:12:49.562306+0000","last_fullsized":"2024-04-25T10:12:49.562306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:05.565804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562345+0000","last_change":"2024-04-25T10:12:48.074133+0000","last_active":"2024-04-25T10:12:49.562345+0000","last_peered":"2024-04-25T10:12:49.562345+0000","last_clean":"2024-04-25T10:12:49.562345+0000","last_became_active":"2024-04-25T10:12:48.073761+0000","last_became_peered":"2024-04-25T10:12:48.073761+0000","last_unstale":"2024-04-25T10:12:49.562345+0000","last_undegraded":"2024-04-25T10:12:49.562345+0000","last_fullsized":"2024-04-25T10:12:49.562345+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:12.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061674+0000","last_change":"2024-04-25T10:12:47.077038+0000","last_active":"2024-04-25T10:12:49.061674+0000","last_peered":"2024-04-25T10:12:49.061674+0000","last_clean":"2024-04-25T10:12:49.061674+0000","last_became_active":"2024-04-25T10:12:47.075177+0000","last_became_peered":"2024-04-25T10:12:47.075177+0000","last_unstale":"2024-04-25T10:12:49.061674+0000","last_undegraded":"2024-04-25T10:12:49.061674+0000","last_fullsized":"2024-04-25T10:12:49.061674+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:24.766728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060990+0000","last_change":"2024-04-25T10:12:48.080902+0000","last_active":"2024-04-25T10:12:49.060990+0000","last_peered":"2024-04-25T10:12:49.060990+0000","last_clean":"2024-04-25T10:12:49.060990+0000","last_became_active":"2024-04-25T10:12:48.078682+0000","last_became_peered":"2024-04-25T10:12:48.078682+0000","last_unstale":"2024-04-25T10:12:49.060990+0000","last_undegraded":"2024-04-25T10:12:49.060990+0000","last_fullsized":"2024-04-25T10:12:49.060990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:47.702871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209857+0000","last_change":"2024-04-25T10:12:47.070656+0000","last_active":"2024-04-25T10:12:49.209857+0000","last_peered":"2024-04-25T10:12:49.209857+0000","last_clean":"2024-04-25T10:12:49.209857+0000","last_became_active":"2024-04-25T10:12:47.070203+0000","last_became_peered":"2024-04-25T10:12:47.070203+0000","last_unstale":"2024-04-25T10:12:49.209857+0000","last_undegraded":"2024-04-25T10:12:49.209857+0000","last_fullsized":"2024-04-25T10:12:49.209857+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:43.477252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061666+0000","last_change":"2024-04-25T10:12:48.066482+0000","last_active":"2024-04-25T10:12:49.061666+0000","last_peered":"2024-04-25T10:12:49.061666+0000","last_clean":"2024-04-25T10:12:49.061666+0000","last_became_active":"2024-04-25T10:12:48.065415+0000","last_became_peered":"2024-04-25T10:12:48.065415+0000","last_unstale":"2024-04-25T10:12:49.061666+0000","last_undegraded":"2024-04-25T10:12:49.061666+0000","last_fullsized":"2024-04-25T10:12:49.061666+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:02.563229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061784+0000","last_change":"2024-04-25T10:12:47.074728+0000","last_active":"2024-04-25T10:12:49.061784+0000","last_peered":"2024-04-25T10:12:49.061784+0000","last_clean":"2024-04-25T10:12:49.061784+0000","last_became_active":"2024-04-25T10:12:47.074449+0000","last_became_peered":"2024-04-25T10:12:47.074449+0000","last_unstale":"2024-04-25T10:12:49.061784+0000","last_undegraded":"2024-04-25T10:12:49.061784+0000","last_fullsized":"2024-04-25T10:12:49.061784+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.870957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562694+0000","last_change":"2024-04-25T10:12:48.083474+0000","last_active":"2024-04-25T10:12:49.562694+0000","last_peered":"2024-04-25T10:12:49.562694+0000","last_clean":"2024-04-25T10:12:49.562694+0000","last_became_active":"2024-04-25T10:12:48.082111+0000","last_became_peered":"2024-04-25T10:12:48.082111+0000","last_unstale":"2024-04-25T10:12:49.562694+0000","last_undegraded":"2024-04-25T10:12:49.562694+0000","last_fullsized":"2024-04-25T10:12:49.562694+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:45.154149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082124+0000","last_change":"2024-04-25T10:12:47.069367+0000","last_active":"2024-04-25T10:12:49.082124+0000","last_peered":"2024-04-25T10:12:49.082124+0000","last_clean":"2024-04-25T10:12:49.082124+0000","last_became_active":"2024-04-25T10:12:47.068988+0000","last_became_peered":"2024-04-25T10:12:47.068988+0000","last_unstale":"2024-04-25T10:12:49.082124+0000","last_undegraded":"2024-04-25T10:12:49.082124+0000","last_fullsized":"2024-04-25T10:12:49.082124+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:21.779924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062647+0000","last_change":"2024-04-25T10:12:48.076747+0000","last_active":"2024-04-25T10:12:49.062647+0000","last_peered":"2024-04-25T10:12:49.062647+0000","last_clean":"2024-04-25T10:12:49.062647+0000","last_became_active":"2024-04-25T10:12:48.076422+0000","last_became_peered":"2024-04-25T10:12:48.076422+0000","last_unstale":"2024-04-25T10:12:49.062647+0000","last_undegraded":"2024-04-25T10:12:49.062647+0000","last_fullsized":"2024-04-25T10:12:49.062647+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:07.231763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.565258+0000","last_change":"2024-04-25T10:12:47.079505+0000","last_active":"2024-04-25T10:12:49.565258+0000","last_peered":"2024-04-25T10:12:49.565258+0000","last_clean":"2024-04-25T10:12:49.565258+0000","last_became_active":"2024-04-25T10:12:47.079218+0000","last_became_peered":"2024-04-25T10:12:47.079218+0000","last_unstale":"2024-04-25T10:12:49.565258+0000","last_undegraded":"2024-04-25T10:12:49.565258+0000","last_fullsized":"2024-04-25T10:12:49.565258+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:02.239976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063692+0000","last_change":"2024-04-25T10:12:48.067677+0000","last_active":"2024-04-25T10:12:49.063692+0000","last_peered":"2024-04-25T10:12:49.063692+0000","last_clean":"2024-04-25T10:12:49.063692+0000","last_became_active":"2024-04-25T10:12:48.067380+0000","last_became_peered":"2024-04-25T10:12:48.067380+0000","last_unstale":"2024-04-25T10:12:49.063692+0000","last_undegraded":"2024-04-25T10:12:49.063692+0000","last_fullsized":"2024-04-25T10:12:49.063692+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:51.422521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.085388+0000","last_change":"2024-04-25T10:12:47.073699+0000","last_active":"2024-04-25T10:12:49.085388+0000","last_peered":"2024-04-25T10:12:49.085388+0000","last_clean":"2024-04-25T10:12:49.085388+0000","last_became_active":"2024-04-25T10:12:47.073384+0000","last_became_peered":"2024-04-25T10:12:47.073384+0000","last_unstale":"2024-04-25T10:12:49.085388+0000","last_undegraded":"2024-04-25T10:12:49.085388+0000","last_fullsized":"2024-04-25T10:12:49.085388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.291427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061016+0000","last_change":"2024-04-25T10:12:48.066403+0000","last_active":"2024-04-25T10:12:49.061016+0000","last_peered":"2024-04-25T10:12:49.061016+0000","last_clean":"2024-04-25T10:12:49.061016+0000","last_became_active":"2024-04-25T10:12:48.066087+0000","last_became_peered":"2024-04-25T10:12:48.066087+0000","last_unstale":"2024-04-25T10:12:49.061016+0000","last_undegraded":"2024-04-25T10:12:49.061016+0000","last_fullsized":"2024-04-25T10:12:49.061016+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:28.952376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083112+0000","last_change":"2024-04-25T10:12:47.072696+0000","last_active":"2024-04-25T10:12:49.083112+0000","last_peered":"2024-04-25T10:12:49.083112+0000","last_clean":"2024-04-25T10:12:49.083112+0000","last_became_active":"2024-04-25T10:12:47.072431+0000","last_became_peered":"2024-04-25T10:12:47.072431+0000","last_unstale":"2024-04-25T10:12:49.083112+0000","last_undegraded":"2024-04-25T10:12:49.083112+0000","last_fullsized":"2024-04-25T10:12:49.083112+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:05.532622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063853+0000","last_change":"2024-04-25T10:12:48.071918+0000","last_active":"2024-04-25T10:12:49.063853+0000","last_peered":"2024-04-25T10:12:49.063853+0000","last_clean":"2024-04-25T10:12:49.063853+0000","last_became_active":"2024-04-25T10:12:48.071361+0000","last_became_peered":"2024-04-25T10:12:48.071361+0000","last_unstale":"2024-04-25T10:12:49.063853+0000","last_undegraded":"2024-04-25T10:12:49.063853+0000","last_fullsized":"2024-04-25T10:12:49.063853+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.646161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063089+0000","last_change":"2024-04-25T10:12:47.080066+0000","last_active":"2024-04-25T10:12:49.063089+0000","last_peered":"2024-04-25T10:12:49.063089+0000","last_clean":"2024-04-25T10:12:49.063089+0000","last_became_active":"2024-04-25T10:12:47.079791+0000","last_became_peered":"2024-04-25T10:12:47.079791+0000","last_unstale":"2024-04-25T10:12:49.063089+0000","last_undegraded":"2024-04-25T10:12:49.063089+0000","last_fullsized":"2024-04-25T10:12:49.063089+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:06.611104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061744+0000","last_change":"2024-04-25T10:12:48.073423+0000","last_active":"2024-04-25T10:12:49.061744+0000","last_peered":"2024-04-25T10:12:49.061744+0000","last_clean":"2024-04-25T10:12:49.061744+0000","last_became_active":"2024-04-25T10:12:48.073054+0000","last_became_peered":"2024-04-25T10:12:48.073054+0000","last_unstale":"2024-04-25T10:12:49.061744+0000","last_undegraded":"2024-04-25T10:12:49.061744+0000","last_fullsized":"2024-04-25T10:12:49.061744+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.913979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082704+0000","last_change":"2024-04-25T10:12:47.069310+0000","last_active":"2024-04-25T10:12:49.082704+0000","last_peered":"2024-04-25T10:12:49.082704+0000","last_clean":"2024-04-25T10:12:49.082704+0000","last_became_active":"2024-04-25T10:12:47.069030+0000","last_became_peered":"2024-04-25T10:12:47.069030+0000","last_unstale":"2024-04-25T10:12:49.082704+0000","last_undegraded":"2024-04-25T10:12:49.082704+0000","last_fullsized":"2024-04-25T10:12:49.082704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:39.936186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562461+0000","last_change":"2024-04-25T10:12:48.078318+0000","last_active":"2024-04-25T10:12:49.562461+0000","last_peered":"2024-04-25T10:12:49.562461+0000","last_clean":"2024-04-25T10:12:49.562461+0000","last_became_active":"2024-04-25T10:12:48.077901+0000","last_became_peered":"2024-04-25T10:12:48.077901+0000","last_unstale":"2024-04-25T10:12:49.562461+0000","last_undegraded":"2024-04-25T10:12:49.562461+0000","last_fullsized":"2024-04-25T10:12:49.562461+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:21.385821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209989+0000","last_change":"2024-04-25T10:12:47.072608+0000","last_active":"2024-04-25T10:12:49.209989+0000","last_peered":"2024-04-25T10:12:49.209989+0000","last_clean":"2024-04-25T10:12:49.209989+0000","last_became_active":"2024-04-25T10:12:47.072359+0000","last_became_peered":"2024-04-25T10:12:47.072359+0000","last_unstale":"2024-04-25T10:12:49.209989+0000","last_undegraded":"2024-04-25T10:12:49.209989+0000","last_fullsized":"2024-04-25T10:12:49.209989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:52.967236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082346+0000","last_change":"2024-04-25T10:12:48.067172+0000","last_active":"2024-04-25T10:12:49.082346+0000","last_peered":"2024-04-25T10:12:49.082346+0000","last_clean":"2024-04-25T10:12:49.082346+0000","last_became_active":"2024-04-25T10:12:48.066889+0000","last_became_peered":"2024-04-25T10:12:48.066889+0000","last_unstale":"2024-04-25T10:12:49.082346+0000","last_undegraded":"2024-04-25T10:12:49.082346+0000","last_fullsized":"2024-04-25T10:12:49.082346+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:44.176804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563123+0000","last_change":"2024-04-25T10:12:47.082986+0000","last_active":"2024-04-25T10:12:49.563123+0000","last_peered":"2024-04-25T10:12:49.563123+0000","last_clean":"2024-04-25T10:12:49.563123+0000","last_became_active":"2024-04-25T10:12:47.082527+0000","last_became_peered":"2024-04-25T10:12:47.082527+0000","last_unstale":"2024-04-25T10:12:49.563123+0000","last_undegraded":"2024-04-25T10:12:49.563123+0000","last_fullsized":"2024-04-25T10:12:49.563123+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.747613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564668+0000","last_change":"2024-04-25T10:12:48.070946+0000","last_active":"2024-04-25T10:12:49.564668+0000","last_peered":"2024-04-25T10:12:49.564668+0000","last_clean":"2024-04-25T10:12:49.564668+0000","last_became_active":"2024-04-25T10:12:48.065391+0000","last_became_peered":"2024-04-25T10:12:48.065391+0000","last_unstale":"2024-04-25T10:12:49.564668+0000","last_undegraded":"2024-04-25T10:12:49.564668+0000","last_fullsized":"2024-04-25T10:12:49.564668+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:08.810812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082427+0000","last_change":"2024-04-25T10:12:47.065999+0000","last_active":"2024-04-25T10:12:49.082427+0000","last_peered":"2024-04-25T10:12:49.082427+0000","last_clean":"2024-04-25T10:12:49.082427+0000","last_became_active":"2024-04-25T10:12:47.063695+0000","last_became_peered":"2024-04-25T10:12:47.063695+0000","last_unstale":"2024-04-25T10:12:49.082427+0000","last_undegraded":"2024-04-25T10:12:49.082427+0000","last_fullsized":"2024-04-25T10:12:49.082427+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:01.825831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082180+0000","last_change":"2024-04-25T10:12:48.072750+0000","last_active":"2024-04-25T10:12:49.082180+0000","last_peered":"2024-04-25T10:12:49.082180+0000","last_clean":"2024-04-25T10:12:49.082180+0000","last_became_active":"2024-04-25T10:12:48.072149+0000","last_became_peered":"2024-04-25T10:12:48.072149+0000","last_unstale":"2024-04-25T10:12:49.082180+0000","last_undegraded":"2024-04-25T10:12:49.082180+0000","last_fullsized":"2024-04-25T10:12:49.082180+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.983869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"119'8","reported_seq":22,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563215+0000","last_change":"2024-04-25T10:12:47.078362+0000","last_active":"2024-04-25T10:12:49.563215+0000","last_peered":"2024-04-25T10:12:49.563215+0000","last_clean":"2024-04-25T10:12:49.563215+0000","last_became_active":"2024-04-25T10:12:47.076966+0000","last_became_peered":"2024-04-25T10:12:47.076966+0000","last_unstale":"2024-04-25T10:12:49.563215+0000","last_undegraded":"2024-04-25T10:12:49.563215+0000","last_fullsized":"2024-04-25T10:12:49.563215+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:30.836308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":356,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209680+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:12:49.209680+0000","last_peered":"2024-04-25T10:12:49.209680+0000","last_clean":"2024-04-25T10:12:49.209680+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:12:49.209680+0000","last_undegraded":"2024-04-25T10:12:49.209680+0000","last_fullsized":"2024-04-25T10:12:49.209680+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509575,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":693838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509577,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1490944,"data_stored":693860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2093056,"data_stored":1285818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509577,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1490944,"data_stored":693928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509576,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":696032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509575,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2088960,"data_stored":1284366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509576,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":694524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1495040,"data_stored":693894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:52.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.790+0000 7fe237e00640 1 Processor -- start 2024-04-25T10:12:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.790+0000 7fe237e00640 1 -- start start 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.791+0000 7fe237e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe238161200 0x7fe2380b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.791+0000 7fe237e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2380d2380 0x7fe2380d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.791+0000 7fe237e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380cf410 con 0x7fe2380d0600 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.791+0000 7fe237e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380d27c0 con 0x7fe2380d2380 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.791+0000 7fe237e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b28b0 con 0x7fe238161200 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2380d2380 0x7fe2380d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2380d2380 0x7fe2380d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34598/0 (socket says 172.21.15.5:34598) 2024-04-25T10:12:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 -- 172.21.15.5:0/1867820616 learned_addr learned my addr 172.21.15.5:0/1867820616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236a00640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe238161200 0x7fe2380b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 -- 172.21.15.5:0/1867820616 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe238161200 msgr2=0x7fe2380b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe238161200 0x7fe2380b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 -- 172.21.15.5:0/1867820616 >> v1:172.21.15.80:6789/0 conn(0x7fe2380d0600 legacy=0x7fe2380ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236000640 1 -- 172.21.15.5:0/1867820616 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380cf6f0 con 0x7fe2380d2380 2024-04-25T10:12:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.792+0000 7fe236a00640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe238161200 0x7fe2380b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.793+0000 7fe236000640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d2380 0x7fe2380d00c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe2240098a0 tx=0x7fe22402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e757e094d9b0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.794+0000 7fe235600640 1 -- 172.21.15.5:0/1867820616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224037070 con 0x7fe2380d2380 2024-04-25T10:12:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.794+0000 7fe235600640 1 -- 172.21.15.5:0/1867820616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe224032440 con 0x7fe2380d2380 2024-04-25T10:12:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.794+0000 7fe235600640 1 -- 172.21.15.5:0/1867820616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224032810 con 0x7fe2380d2380 2024-04-25T10:12:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 -- 172.21.15.5:0/1867820616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d2380 msgr2=0x7fe2380d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d2380 0x7fe2380d00c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe2240098a0 tx=0x7fe22402a250 comp rx=0 tx=0).stop 2024-04-25T10:12:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 -- 172.21.15.5:0/1867820616 shutdown_connections 2024-04-25T10:12:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d2380 0x7fe2380d00c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 --2- 172.21.15.5:0/1867820616 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe238161200 0x7fe2380b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 -- 172.21.15.5:0/1867820616 >> 172.21.15.5:0/1867820616 conn(0x7fe23808af80 msgr2=0x7fe2380d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 -- 172.21.15.5:0/1867820616 shutdown_connections 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.795+0000 7fe237e00640 1 -- 172.21.15.5:0/1867820616 wait complete. 2024-04-25T10:12:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.797+0000 7fe237e00640 1 Processor -- start 2024-04-25T10:12:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.797+0000 7fe237e00640 1 -- start start 2024-04-25T10:12:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 0x7fe23814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2380d2380 0x7fe2380b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe238161200 0x7fe2380b8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b3f90 con 0x7fe2380d0600 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380d42b0 con 0x7fe238161200 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380b4110 con 0x7fe2380d2380 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe236a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 0x7fe23814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe236a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 0x7fe23814fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34614/0 (socket says 172.21.15.5:34614) 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe236a00640 1 -- 172.21.15.5:0/2289149244 learned_addr learned my addr 172.21.15.5:0/2289149244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.798+0000 7fe237400640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe238161200 0x7fe2380b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236000640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2380d2380 0x7fe2380b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236a00640 1 -- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2380d2380 msgr2=0x7fe2380b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236a00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2380d2380 0x7fe2380b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236a00640 1 -- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe238161200 msgr2=0x7fe2380b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:52.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236a00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe238161200 0x7fe2380b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:52.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236a00640 1 -- 172.21.15.5:0/2289149244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238150050 con 0x7fe2380d0600 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe236a00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 0x7fe23814fb10 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe22800ce30 tx=0x7fe22800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.799+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800f890 con 0x7fe2380d0600 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.800+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800fa30 con 0x7fe2380d0600 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.800+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2380b6520 con 0x7fe2380d0600 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.800+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800fe00 con 0x7fe2380d0600 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.800+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2380b6ae0 con 0x7fe2380d0600 2024-04-25T10:12:52.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.801+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe238068f60 con 0x7fe2380d0600 2024-04-25T10:12:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.802+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe228018400 con 0x7fe2380d0600 2024-04-25T10:12:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.803+0000 7fe22f400640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe21806b4d0 0x7fe21806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.803+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe22808b100 con 0x7fe2380d0600 2024-04-25T10:12:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.803+0000 7fe236000640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe21806b4d0 0x7fe21806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.805+0000 7fe236000640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe21806b4d0 0x7fe21806d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe224034000 tx=0x7fe22402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:52.807+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe238150050 con 0x7fe2380d0600 2024-04-25T10:12:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.003+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2380b9530 con 0x7fe21806b4d0 2024-04-25T10:12:53.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.021+0000 7fe22f400640 1 -- 172.21.15.5:0/2289149244 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7fe2380b9530 con 0x7fe21806b4d0 2024-04-25T10:12:53.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.027+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe21806b4d0 msgr2=0x7fe21806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.027+0000 7fe237e00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe21806b4d0 0x7fe21806d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe224034000 tx=0x7fe22402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.027+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 msgr2=0x7fe23814fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.027+0000 7fe237e00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 0x7fe23814fb10 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe22800ce30 tx=0x7fe22800a650 comp rx=0 tx=0).stop 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 shutdown_connections 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe21806b4d0 0x7fe21806d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe238161200 0x7fe2380b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2380d2380 0x7fe2380b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 --2- 172.21.15.5:0/2289149244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2380d0600 0x7fe23814fb10 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 >> 172.21.15.5:0/2289149244 conn(0x7fe23808af80 msgr2=0x7fe23815e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 shutdown_connections 2024-04-25T10:12:53.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.028+0000 7fe237e00640 1 -- 172.21.15.5:0/2289149244 wait complete. 2024-04-25T10:12:53.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-04-25T10:12:52.859681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262300,"kb_used_data":12840,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712420,"statfs":{"total":773094113280,"available":772825518080,"internally_reserved":0,"allocated":13148160,"data_stored":6736260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002419"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562491+0000","last_change":"2024-04-25T10:12:48.079664+0000","last_active":"2024-04-25T10:12:49.562491+0000","last_peered":"2024-04-25T10:12:49.562491+0000","last_clean":"2024-04-25T10:12:49.562491+0000","last_became_active":"2024-04-25T10:12:48.079309+0000","last_became_peered":"2024-04-25T10:12:48.079309+0000","last_unstale":"2024-04-25T10:12:49.562491+0000","last_undegraded":"2024-04-25T10:12:49.562491+0000","last_fullsized":"2024-04-25T10:12:49.562491+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:32.630363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564022+0000","last_change":"2024-04-25T10:12:48.080025+0000","last_active":"2024-04-25T10:12:49.564022+0000","last_peered":"2024-04-25T10:12:49.564022+0000","last_clean":"2024-04-25T10:12:49.564022+0000","last_became_active":"2024-04-25T10:12:48.077446+0000","last_became_peered":"2024-04-25T10:12:48.077446+0000","last_unstale":"2024-04-25T10:12:49.564022+0000","last_undegraded":"2024-04-25T10:12:49.564022+0000","last_fullsized":"2024-04-25T10:12:49.564022+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:23.341325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063738+0000","last_change":"2024-04-25T10:12:48.074288+0000","last_active":"2024-04-25T10:12:49.063738+0000","last_peered":"2024-04-25T10:12:49.063738+0000","last_clean":"2024-04-25T10:12:49.063738+0000","last_became_active":"2024-04-25T10:12:48.074000+0000","last_became_peered":"2024-04-25T10:12:48.074000+0000","last_unstale":"2024-04-25T10:12:49.063738+0000","last_undegraded":"2024-04-25T10:12:49.063738+0000","last_fullsized":"2024-04-25T10:12:49.063738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:09.601365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060899+0000","last_change":"2024-04-25T10:12:48.080420+0000","last_active":"2024-04-25T10:12:49.060899+0000","last_peered":"2024-04-25T10:12:49.060899+0000","last_clean":"2024-04-25T10:12:49.060899+0000","last_became_active":"2024-04-25T10:12:48.080135+0000","last_became_peered":"2024-04-25T10:12:48.080135+0000","last_unstale":"2024-04-25T10:12:49.060899+0000","last_undegraded":"2024-04-25T10:12:49.060899+0000","last_fullsized":"2024-04-25T10:12:49.060899+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:46.323277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082415+0000","last_change":"2024-04-25T10:12:48.072381+0000","last_active":"2024-04-25T10:12:49.082415+0000","last_peered":"2024-04-25T10:12:49.082415+0000","last_clean":"2024-04-25T10:12:49.082415+0000","last_became_active":"2024-04-25T10:12:48.069258+0000","last_became_peered":"2024-04-25T10:12:48.069258+0000","last_unstale":"2024-04-25T10:12:49.082415+0000","last_undegraded":"2024-04-25T10:12:49.082415+0000","last_fullsized":"2024-04-25T10:12:49.082415+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:47.255274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562558+0000","last_change":"2024-04-25T10:12:48.081007+0000","last_active":"2024-04-25T10:12:49.562558+0000","last_peered":"2024-04-25T10:12:49.562558+0000","last_clean":"2024-04-25T10:12:49.562558+0000","last_became_active":"2024-04-25T10:12:48.077698+0000","last_became_peered":"2024-04-25T10:12:48.077698+0000","last_unstale":"2024-04-25T10:12:49.562558+0000","last_undegraded":"2024-04-25T10:12:49.562558+0000","last_fullsized":"2024-04-25T10:12:49.562558+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:27.931323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563584+0000","last_change":"2024-04-25T10:12:48.078730+0000","last_active":"2024-04-25T10:12:49.563584+0000","last_peered":"2024-04-25T10:12:49.563584+0000","last_clean":"2024-04-25T10:12:49.563584+0000","last_became_active":"2024-04-25T10:12:48.078069+0000","last_became_peered":"2024-04-25T10:12:48.078069+0000","last_unstale":"2024-04-25T10:12:49.563584+0000","last_undegraded":"2024-04-25T10:12:49.563584+0000","last_fullsized":"2024-04-25T10:12:49.563584+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:50.507728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562888+0000","last_change":"2024-04-25T10:12:48.065555+0000","last_active":"2024-04-25T10:12:49.562888+0000","last_peered":"2024-04-25T10:12:49.562888+0000","last_clean":"2024-04-25T10:12:49.562888+0000","last_became_active":"2024-04-25T10:12:48.064789+0000","last_became_peered":"2024-04-25T10:12:48.064789+0000","last_unstale":"2024-04-25T10:12:49.562888+0000","last_undegraded":"2024-04-25T10:12:49.562888+0000","last_fullsized":"2024-04-25T10:12:49.562888+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:13.617100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061140+0000","last_change":"2024-04-25T10:12:48.074447+0000","last_active":"2024-04-25T10:12:49.061140+0000","last_peered":"2024-04-25T10:12:49.061140+0000","last_clean":"2024-04-25T10:12:49.061140+0000","last_became_active":"2024-04-25T10:12:48.074125+0000","last_became_peered":"2024-04-25T10:12:48.074125+0000","last_unstale":"2024-04-25T10:12:49.061140+0000","last_undegraded":"2024-04-25T10:12:49.061140+0000","last_fullsized":"2024-04-25T10:12:49.061140+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:05.847552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564597+0000","last_change":"2024-04-25T10:12:48.080759+0000","last_active":"2024-04-25T10:12:49.564597+0000","last_peered":"2024-04-25T10:12:49.564597+0000","last_clean":"2024-04-25T10:12:49.564597+0000","last_became_active":"2024-04-25T10:12:48.080310+0000","last_became_peered":"2024-04-25T10:12:48.080310+0000","last_unstale":"2024-04-25T10:12:49.564597+0000","last_undegraded":"2024-04-25T10:12:49.564597+0000","last_fullsized":"2024-04-25T10:12:49.564597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:20.810249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562654+0000","last_change":"2024-04-25T10:12:48.069364+0000","last_active":"2024-04-25T10:12:49.562654+0000","last_peered":"2024-04-25T10:12:49.562654+0000","last_clean":"2024-04-25T10:12:49.562654+0000","last_became_active":"2024-04-25T10:12:48.068899+0000","last_became_peered":"2024-04-25T10:12:48.068899+0000","last_unstale":"2024-04-25T10:12:49.562654+0000","last_undegraded":"2024-04-25T10:12:49.562654+0000","last_fullsized":"2024-04-25T10:12:49.562654+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:34.325526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061775+0000","last_change":"2024-04-25T10:12:48.088866+0000","last_active":"2024-04-25T10:12:49.061775+0000","last_peered":"2024-04-25T10:12:49.061775+0000","last_clean":"2024-04-25T10:12:49.061775+0000","last_became_active":"2024-04-25T10:12:48.088575+0000","last_became_peered":"2024-04-25T10:12:48.088575+0000","last_unstale":"2024-04-25T10:12:49.061775+0000","last_undegraded":"2024-04-25T10:12:49.061775+0000","last_fullsized":"2024-04-25T10:12:49.061775+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:02.812234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564266+0000","last_change":"2024-04-25T10:12:48.078288+0000","last_active":"2024-04-25T10:12:49.564266+0000","last_peered":"2024-04-25T10:12:49.564266+0000","last_clean":"2024-04-25T10:12:49.564266+0000","last_became_active":"2024-04-25T10:12:48.077143+0000","last_became_peered":"2024-04-25T10:12:48.077143+0000","last_unstale":"2024-04-25T10:12:49.564266+0000","last_undegraded":"2024-04-25T10:12:49.564266+0000","last_fullsized":"2024-04-25T10:12:49.564266+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:07.627738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061189+0000","last_change":"2024-04-25T10:12:48.083403+0000","last_active":"2024-04-25T10:12:49.061189+0000","last_peered":"2024-04-25T10:12:49.061189+0000","last_clean":"2024-04-25T10:12:49.061189+0000","last_became_active":"2024-04-25T10:12:48.081548+0000","last_became_peered":"2024-04-25T10:12:48.081548+0000","last_unstale":"2024-04-25T10:12:49.061189+0000","last_undegraded":"2024-04-25T10:12:49.061189+0000","last_fullsized":"2024-04-25T10:12:49.061189+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:30.433166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209850+0000","last_change":"2024-04-25T10:12:48.051857+0000","last_active":"2024-04-25T10:12:49.209850+0000","last_peered":"2024-04-25T10:12:49.209850+0000","last_clean":"2024-04-25T10:12:49.209850+0000","last_became_active":"2024-04-25T10:12:48.051305+0000","last_became_peered":"2024-04-25T10:12:48.051305+0000","last_unstale":"2024-04-25T10:12:49.209850+0000","last_undegraded":"2024-04-25T10:12:49.209850+0000","last_fullsized":"2024-04-25T10:12:49.209850+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:59.764246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061324+0000","last_change":"2024-04-25T10:12:48.073091+0000","last_active":"2024-04-25T10:12:49.061324+0000","last_peered":"2024-04-25T10:12:49.061324+0000","last_clean":"2024-04-25T10:12:49.061324+0000","last_became_active":"2024-04-25T10:12:48.072795+0000","last_became_peered":"2024-04-25T10:12:48.072795+0000","last_unstale":"2024-04-25T10:12:49.061324+0000","last_undegraded":"2024-04-25T10:12:49.061324+0000","last_fullsized":"2024-04-25T10:12:49.061324+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:14.812657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062506+0000","last_change":"2024-04-25T10:12:48.080434+0000","last_active":"2024-04-25T10:12:49.062506+0000","last_peered":"2024-04-25T10:12:49.062506+0000","last_clean":"2024-04-25T10:12:49.062506+0000","last_became_active":"2024-04-25T10:12:48.076222+0000","last_became_peered":"2024-04-25T10:12:48.076222+0000","last_unstale":"2024-04-25T10:12:49.062506+0000","last_undegraded":"2024-04-25T10:12:49.062506+0000","last_fullsized":"2024-04-25T10:12:49.062506+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.739649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062042+0000","last_change":"2024-04-25T10:12:48.074957+0000","last_active":"2024-04-25T10:12:49.062042+0000","last_peered":"2024-04-25T10:12:49.062042+0000","last_clean":"2024-04-25T10:12:49.062042+0000","last_became_active":"2024-04-25T10:12:48.074641+0000","last_became_peered":"2024-04-25T10:12:48.074641+0000","last_unstale":"2024-04-25T10:12:49.062042+0000","last_undegraded":"2024-04-25T10:12:49.062042+0000","last_fullsized":"2024-04-25T10:12:49.062042+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:57.138797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082885+0000","last_change":"2024-04-25T10:12:48.080055+0000","last_active":"2024-04-25T10:12:49.082885+0000","last_peered":"2024-04-25T10:12:49.082885+0000","last_clean":"2024-04-25T10:12:49.082885+0000","last_became_active":"2024-04-25T10:12:48.078711+0000","last_became_peered":"2024-04-25T10:12:48.078711+0000","last_unstale":"2024-04-25T10:12:49.082885+0000","last_undegraded":"2024-04-25T10:12:49.082885+0000","last_fullsized":"2024-04-25T10:12:49.082885+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:32.566854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564083+0000","last_change":"2024-04-25T10:12:48.086034+0000","last_active":"2024-04-25T10:12:49.564083+0000","last_peered":"2024-04-25T10:12:49.564083+0000","last_clean":"2024-04-25T10:12:49.564083+0000","last_became_active":"2024-04-25T10:12:48.085746+0000","last_became_peered":"2024-04-25T10:12:48.085746+0000","last_unstale":"2024-04-25T10:12:49.564083+0000","last_undegraded":"2024-04-25T10:12:49.564083+0000","last_fullsized":"2024-04-25T10:12:49.564083+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:32.379195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082587+0000","last_change":"2024-04-25T10:12:48.086574+0000","last_active":"2024-04-25T10:12:49.082587+0000","last_peered":"2024-04-25T10:12:49.082587+0000","last_clean":"2024-04-25T10:12:49.082587+0000","last_became_active":"2024-04-25T10:12:48.086048+0000","last_became_peered":"2024-04-25T10:12:48.086048+0000","last_unstale":"2024-04-25T10:12:49.082587+0000","last_undegraded":"2024-04-25T10:12:49.082587+0000","last_fullsized":"2024-04-25T10:12:49.082587+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:08.898195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209872+0000","last_change":"2024-04-25T10:12:48.071488+0000","last_active":"2024-04-25T10:12:49.209872+0000","last_peered":"2024-04-25T10:12:49.209872+0000","last_clean":"2024-04-25T10:12:49.209872+0000","last_became_active":"2024-04-25T10:12:48.069583+0000","last_became_peered":"2024-04-25T10:12:48.069583+0000","last_unstale":"2024-04-25T10:12:49.209872+0000","last_undegraded":"2024-04-25T10:12:49.209872+0000","last_fullsized":"2024-04-25T10:12:49.209872+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:08.419073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209848+0000","last_change":"2024-04-25T10:12:48.065630+0000","last_active":"2024-04-25T10:12:49.209848+0000","last_peered":"2024-04-25T10:12:49.209848+0000","last_clean":"2024-04-25T10:12:49.209848+0000","last_became_active":"2024-04-25T10:12:48.058574+0000","last_became_peered":"2024-04-25T10:12:48.058574+0000","last_unstale":"2024-04-25T10:12:49.209848+0000","last_undegraded":"2024-04-25T10:12:49.209848+0000","last_fullsized":"2024-04-25T10:12:49.209848+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:03.256609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209869+0000","last_change":"2024-04-25T10:12:48.077119+0000","last_active":"2024-04-25T10:12:49.209869+0000","last_peered":"2024-04-25T10:12:49.209869+0000","last_clean":"2024-04-25T10:12:49.209869+0000","last_became_active":"2024-04-25T10:12:48.076823+0000","last_became_peered":"2024-04-25T10:12:48.076823+0000","last_unstale":"2024-04-25T10:12:49.209869+0000","last_undegraded":"2024-04-25T10:12:49.209869+0000","last_fullsized":"2024-04-25T10:12:49.209869+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.556464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562520+0000","last_change":"2024-04-25T10:12:48.073500+0000","last_active":"2024-04-25T10:12:49.562520+0000","last_peered":"2024-04-25T10:12:49.562520+0000","last_clean":"2024-04-25T10:12:49.562520+0000","last_became_active":"2024-04-25T10:12:48.070704+0000","last_became_peered":"2024-04-25T10:12:48.070704+0000","last_unstale":"2024-04-25T10:12:49.562520+0000","last_undegraded":"2024-04-25T10:12:49.562520+0000","last_fullsized":"2024-04-25T10:12:49.562520+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:06.395728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564846+0000","last_change":"2024-04-25T10:12:48.081614+0000","last_active":"2024-04-25T10:12:49.564846+0000","last_peered":"2024-04-25T10:12:49.564846+0000","last_clean":"2024-04-25T10:12:49.564846+0000","last_became_active":"2024-04-25T10:12:48.081349+0000","last_became_peered":"2024-04-25T10:12:48.081349+0000","last_unstale":"2024-04-25T10:12:49.564846+0000","last_undegraded":"2024-04-25T10:12:49.564846+0000","last_fullsized":"2024-04-25T10:12:49.564846+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:35.586918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082979+0000","last_change":"2024-04-25T10:12:48.078683+0000","last_active":"2024-04-25T10:12:49.082979+0000","last_peered":"2024-04-25T10:12:49.082979+0000","last_clean":"2024-04-25T10:12:49.082979+0000","last_became_active":"2024-04-25T10:12:48.075664+0000","last_became_peered":"2024-04-25T10:12:48.075664+0000","last_unstale":"2024-04-25T10:12:49.082979+0000","last_undegraded":"2024-04-25T10:12:49.082979+0000","last_fullsized":"2024-04-25T10:12:49.082979+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:20.777207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062204+0000","last_change":"2024-04-25T10:12:48.082441+0000","last_active":"2024-04-25T10:12:49.062204+0000","last_peered":"2024-04-25T10:12:49.062204+0000","last_clean":"2024-04-25T10:12:49.062204+0000","last_became_active":"2024-04-25T10:12:48.082171+0000","last_became_peered":"2024-04-25T10:12:48.082171+0000","last_unstale":"2024-04-25T10:12:49.062204+0000","last_undegraded":"2024-04-25T10:12:49.062204+0000","last_fullsized":"2024-04-25T10:12:49.062204+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:13.212296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063533+0000","last_change":"2024-04-25T10:12:48.080609+0000","last_active":"2024-04-25T10:12:49.063533+0000","last_peered":"2024-04-25T10:12:49.063533+0000","last_clean":"2024-04-25T10:12:49.063533+0000","last_became_active":"2024-04-25T10:12:48.080276+0000","last_became_peered":"2024-04-25T10:12:48.080276+0000","last_unstale":"2024-04-25T10:12:49.063533+0000","last_undegraded":"2024-04-25T10:12:49.063533+0000","last_fullsized":"2024-04-25T10:12:49.063533+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:23.321505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060447+0000","last_change":"2024-04-25T10:12:48.055133+0000","last_active":"2024-04-25T10:12:49.060447+0000","last_peered":"2024-04-25T10:12:49.060447+0000","last_clean":"2024-04-25T10:12:49.060447+0000","last_became_active":"2024-04-25T10:12:48.054361+0000","last_became_peered":"2024-04-25T10:12:48.054361+0000","last_unstale":"2024-04-25T10:12:49.060447+0000","last_undegraded":"2024-04-25T10:12:49.060447+0000","last_fullsized":"2024-04-25T10:12:49.060447+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:38.308061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.561805+0000","last_change":"2024-04-25T10:12:48.084195+0000","last_active":"2024-04-25T10:12:49.561805+0000","last_peered":"2024-04-25T10:12:49.561805+0000","last_clean":"2024-04-25T10:12:49.561805+0000","last_became_active":"2024-04-25T10:12:48.082321+0000","last_became_peered":"2024-04-25T10:12:48.082321+0000","last_unstale":"2024-04-25T10:12:49.561805+0000","last_undegraded":"2024-04-25T10:12:49.561805+0000","last_fullsized":"2024-04-25T10:12:49.561805+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:44.739649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.561883+0000","last_change":"2024-04-25T10:12:48.071573+0000","last_active":"2024-04-25T10:12:49.561883+0000","last_peered":"2024-04-25T10:12:49.561883+0000","last_clean":"2024-04-25T10:12:49.561883+0000","last_became_active":"2024-04-25T10:12:48.071238+0000","last_became_peered":"2024-04-25T10:12:48.071238+0000","last_unstale":"2024-04-25T10:12:49.561883+0000","last_undegraded":"2024-04-25T10:12:49.561883+0000","last_fullsized":"2024-04-25T10:12:49.561883+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:07.231763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564791+0000","last_change":"2024-04-25T10:12:48.072279+0000","last_active":"2024-04-25T10:12:49.564791+0000","last_peered":"2024-04-25T10:12:49.564791+0000","last_clean":"2024-04-25T10:12:49.564791+0000","last_became_active":"2024-04-25T10:12:48.071954+0000","last_became_peered":"2024-04-25T10:12:48.071954+0000","last_unstale":"2024-04-25T10:12:49.564791+0000","last_undegraded":"2024-04-25T10:12:49.564791+0000","last_fullsized":"2024-04-25T10:12:49.564791+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:53.982718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"119'4","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083596+0000","last_change":"2024-04-25T10:12:47.065942+0000","last_active":"2024-04-25T10:12:49.083596+0000","last_peered":"2024-04-25T10:12:49.083596+0000","last_clean":"2024-04-25T10:12:49.083596+0000","last_became_active":"2024-04-25T10:12:47.061657+0000","last_became_peered":"2024-04-25T10:12:47.061657+0000","last_unstale":"2024-04-25T10:12:49.083596+0000","last_undegraded":"2024-04-25T10:12:49.083596+0000","last_fullsized":"2024-04-25T10:12:49.083596+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:01.313940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563274+0000","last_change":"2024-04-25T10:12:48.077605+0000","last_active":"2024-04-25T10:12:49.563274+0000","last_peered":"2024-04-25T10:12:49.563274+0000","last_clean":"2024-04-25T10:12:49.563274+0000","last_became_active":"2024-04-25T10:12:48.077287+0000","last_became_peered":"2024-04-25T10:12:48.077287+0000","last_unstale":"2024-04-25T10:12:49.563274+0000","last_undegraded":"2024-04-25T10:12:49.563274+0000","last_fullsized":"2024-04-25T10:12:49.563274+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:25.783361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062521+0000","last_change":"2024-04-25T10:12:47.074224+0000","last_active":"2024-04-25T10:12:49.062521+0000","last_peered":"2024-04-25T10:12:49.062521+0000","last_clean":"2024-04-25T10:12:49.062521+0000","last_became_active":"2024-04-25T10:12:47.073396+0000","last_became_peered":"2024-04-25T10:12:47.073396+0000","last_unstale":"2024-04-25T10:12:49.062521+0000","last_undegraded":"2024-04-25T10:12:49.062521+0000","last_fullsized":"2024-04-25T10:12:49.062521+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:31.362562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562335+0000","last_change":"2024-04-25T10:12:48.072462+0000","last_active":"2024-04-25T10:12:49.562335+0000","last_peered":"2024-04-25T10:12:49.562335+0000","last_clean":"2024-04-25T10:12:49.562335+0000","last_became_active":"2024-04-25T10:12:48.072186+0000","last_became_peered":"2024-04-25T10:12:48.072186+0000","last_unstale":"2024-04-25T10:12:49.562335+0000","last_undegraded":"2024-04-25T10:12:49.562335+0000","last_fullsized":"2024-04-25T10:12:49.562335+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:17.950145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.210186+0000","last_change":"2024-04-25T10:12:47.069358+0000","last_active":"2024-04-25T10:12:49.210186+0000","last_peered":"2024-04-25T10:12:49.210186+0000","last_clean":"2024-04-25T10:12:49.210186+0000","last_became_active":"2024-04-25T10:12:47.068946+0000","last_became_peered":"2024-04-25T10:12:47.068946+0000","last_unstale":"2024-04-25T10:12:49.210186+0000","last_undegraded":"2024-04-25T10:12:49.210186+0000","last_fullsized":"2024-04-25T10:12:49.210186+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:03.128337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061356+0000","last_change":"2024-04-25T10:12:48.079047+0000","last_active":"2024-04-25T10:12:49.061356+0000","last_peered":"2024-04-25T10:12:49.061356+0000","last_clean":"2024-04-25T10:12:49.061356+0000","last_became_active":"2024-04-25T10:12:48.078617+0000","last_became_peered":"2024-04-25T10:12:48.078617+0000","last_unstale":"2024-04-25T10:12:49.061356+0000","last_undegraded":"2024-04-25T10:12:49.061356+0000","last_fullsized":"2024-04-25T10:12:49.061356+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:12.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563954+0000","last_change":"2024-04-25T10:12:47.081688+0000","last_active":"2024-04-25T10:12:49.563954+0000","last_peered":"2024-04-25T10:12:49.563954+0000","last_clean":"2024-04-25T10:12:49.563954+0000","last_became_active":"2024-04-25T10:12:47.080260+0000","last_became_peered":"2024-04-25T10:12:47.080260+0000","last_unstale":"2024-04-25T10:12:49.563954+0000","last_undegraded":"2024-04-25T10:12:49.563954+0000","last_fullsized":"2024-04-25T10:12:49.563954+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:19.760574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562947+0000","last_change":"2024-04-25T10:12:48.075568+0000","last_active":"2024-04-25T10:12:49.562947+0000","last_peered":"2024-04-25T10:12:49.562947+0000","last_clean":"2024-04-25T10:12:49.562947+0000","last_became_active":"2024-04-25T10:12:48.070274+0000","last_became_peered":"2024-04-25T10:12:48.070274+0000","last_unstale":"2024-04-25T10:12:49.562947+0000","last_undegraded":"2024-04-25T10:12:49.562947+0000","last_fullsized":"2024-04-25T10:12:49.562947+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:52.017975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562665+0000","last_change":"2024-04-25T10:12:47.068853+0000","last_active":"2024-04-25T10:12:49.562665+0000","last_peered":"2024-04-25T10:12:49.562665+0000","last_clean":"2024-04-25T10:12:49.562665+0000","last_became_active":"2024-04-25T10:12:47.068568+0000","last_became_peered":"2024-04-25T10:12:47.068568+0000","last_unstale":"2024-04-25T10:12:49.562665+0000","last_undegraded":"2024-04-25T10:12:49.562665+0000","last_fullsized":"2024-04-25T10:12:49.562665+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:51.490936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562232+0000","last_change":"2024-04-25T10:12:48.077525+0000","last_active":"2024-04-25T10:12:49.562232+0000","last_peered":"2024-04-25T10:12:49.562232+0000","last_clean":"2024-04-25T10:12:49.562232+0000","last_became_active":"2024-04-25T10:12:48.074911+0000","last_became_peered":"2024-04-25T10:12:48.074911+0000","last_unstale":"2024-04-25T10:12:49.562232+0000","last_undegraded":"2024-04-25T10:12:49.562232+0000","last_fullsized":"2024-04-25T10:12:49.562232+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:32.034888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563873+0000","last_change":"2024-04-25T10:12:47.073230+0000","last_active":"2024-04-25T10:12:49.563873+0000","last_peered":"2024-04-25T10:12:49.563873+0000","last_clean":"2024-04-25T10:12:49.563873+0000","last_became_active":"2024-04-25T10:12:47.072882+0000","last_became_peered":"2024-04-25T10:12:47.072882+0000","last_unstale":"2024-04-25T10:12:49.563873+0000","last_undegraded":"2024-04-25T10:12:49.563873+0000","last_fullsized":"2024-04-25T10:12:49.563873+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:00.162716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061092+0000","last_change":"2024-04-25T10:12:48.069157+0000","last_active":"2024-04-25T10:12:49.061092+0000","last_peered":"2024-04-25T10:12:49.061092+0000","last_clean":"2024-04-25T10:12:49.061092+0000","last_became_active":"2024-04-25T10:12:48.068847+0000","last_became_peered":"2024-04-25T10:12:48.068847+0000","last_unstale":"2024-04-25T10:12:49.061092+0000","last_undegraded":"2024-04-25T10:12:49.061092+0000","last_fullsized":"2024-04-25T10:12:49.061092+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:02.330573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562109+0000","last_change":"2024-04-25T10:12:47.059987+0000","last_active":"2024-04-25T10:12:49.562109+0000","last_peered":"2024-04-25T10:12:49.562109+0000","last_clean":"2024-04-25T10:12:49.562109+0000","last_became_active":"2024-04-25T10:12:47.059665+0000","last_became_peered":"2024-04-25T10:12:47.059665+0000","last_unstale":"2024-04-25T10:12:49.562109+0000","last_undegraded":"2024-04-25T10:12:49.562109+0000","last_fullsized":"2024-04-25T10:12:49.562109+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:08.022420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563854+0000","last_change":"2024-04-25T10:12:48.064245+0000","last_active":"2024-04-25T10:12:49.563854+0000","last_peered":"2024-04-25T10:12:49.563854+0000","last_clean":"2024-04-25T10:12:49.563854+0000","last_became_active":"2024-04-25T10:12:48.063800+0000","last_became_peered":"2024-04-25T10:12:48.063800+0000","last_unstale":"2024-04-25T10:12:49.563854+0000","last_undegraded":"2024-04-25T10:12:49.563854+0000","last_fullsized":"2024-04-25T10:12:49.563854+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:14.994534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060176+0000","last_change":"2024-04-25T10:12:47.079725+0000","last_active":"2024-04-25T10:12:49.060176+0000","last_peered":"2024-04-25T10:12:49.060176+0000","last_clean":"2024-04-25T10:12:49.060176+0000","last_became_active":"2024-04-25T10:12:47.079436+0000","last_became_peered":"2024-04-25T10:12:47.079436+0000","last_unstale":"2024-04-25T10:12:49.060176+0000","last_undegraded":"2024-04-25T10:12:49.060176+0000","last_fullsized":"2024-04-25T10:12:49.060176+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:24.995347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.210044+0000","last_change":"2024-04-25T10:12:48.064869+0000","last_active":"2024-04-25T10:12:49.210044+0000","last_peered":"2024-04-25T10:12:49.210044+0000","last_clean":"2024-04-25T10:12:49.210044+0000","last_became_active":"2024-04-25T10:12:48.064543+0000","last_became_peered":"2024-04-25T10:12:48.064543+0000","last_unstale":"2024-04-25T10:12:49.210044+0000","last_undegraded":"2024-04-25T10:12:49.210044+0000","last_fullsized":"2024-04-25T10:12:49.210044+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:39.023532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061335+0000","last_change":"2024-04-25T10:12:47.069058+0000","last_active":"2024-04-25T10:12:49.061335+0000","last_peered":"2024-04-25T10:12:49.061335+0000","last_clean":"2024-04-25T10:12:49.061335+0000","last_became_active":"2024-04-25T10:12:47.068738+0000","last_became_peered":"2024-04-25T10:12:47.068738+0000","last_unstale":"2024-04-25T10:12:49.061335+0000","last_undegraded":"2024-04-25T10:12:49.061335+0000","last_fullsized":"2024-04-25T10:12:49.061335+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.747613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061220+0000","last_change":"2024-04-25T10:12:48.079558+0000","last_active":"2024-04-25T10:12:49.061220+0000","last_peered":"2024-04-25T10:12:49.061220+0000","last_clean":"2024-04-25T10:12:49.061220+0000","last_became_active":"2024-04-25T10:12:48.079268+0000","last_became_peered":"2024-04-25T10:12:48.079268+0000","last_unstale":"2024-04-25T10:12:49.061220+0000","last_undegraded":"2024-04-25T10:12:49.061220+0000","last_fullsized":"2024-04-25T10:12:49.061220+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:20.235476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060781+0000","last_change":"2024-04-25T10:12:47.071081+0000","last_active":"2024-04-25T10:12:49.060781+0000","last_peered":"2024-04-25T10:12:49.060781+0000","last_clean":"2024-04-25T10:12:49.060781+0000","last_became_active":"2024-04-25T10:12:47.070805+0000","last_became_peered":"2024-04-25T10:12:47.070805+0000","last_unstale":"2024-04-25T10:12:49.060781+0000","last_undegraded":"2024-04-25T10:12:49.060781+0000","last_fullsized":"2024-04-25T10:12:49.060781+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:02.239976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063432+0000","last_change":"2024-04-25T10:12:48.058661+0000","last_active":"2024-04-25T10:12:49.063432+0000","last_peered":"2024-04-25T10:12:49.063432+0000","last_clean":"2024-04-25T10:12:49.063432+0000","last_became_active":"2024-04-25T10:12:48.057952+0000","last_became_peered":"2024-04-25T10:12:48.057952+0000","last_unstale":"2024-04-25T10:12:49.063432+0000","last_undegraded":"2024-04-25T10:12:49.063432+0000","last_fullsized":"2024-04-25T10:12:49.063432+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:54.720827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"119'3","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562741+0000","last_change":"2024-04-25T10:12:47.067181+0000","last_active":"2024-04-25T10:12:49.562741+0000","last_peered":"2024-04-25T10:12:49.562741+0000","last_clean":"2024-04-25T10:12:49.562741+0000","last_became_active":"2024-04-25T10:12:47.066867+0000","last_became_peered":"2024-04-25T10:12:47.066867+0000","last_unstale":"2024-04-25T10:12:49.562741+0000","last_undegraded":"2024-04-25T10:12:49.562741+0000","last_fullsized":"2024-04-25T10:12:49.562741+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:33.946952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083250+0000","last_change":"2024-04-25T10:12:48.084078+0000","last_active":"2024-04-25T10:12:49.083250+0000","last_peered":"2024-04-25T10:12:49.083250+0000","last_clean":"2024-04-25T10:12:49.083250+0000","last_became_active":"2024-04-25T10:12:48.080495+0000","last_became_peered":"2024-04-25T10:12:48.080495+0000","last_unstale":"2024-04-25T10:12:49.083250+0000","last_undegraded":"2024-04-25T10:12:49.083250+0000","last_fullsized":"2024-04-25T10:12:49.083250+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:03.556464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"119'6","reported_seq":20,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564277+0000","last_change":"2024-04-25T10:12:47.079869+0000","last_active":"2024-04-25T10:12:49.564277+0000","last_peered":"2024-04-25T10:12:49.564277+0000","last_clean":"2024-04-25T10:12:49.564277+0000","last_became_active":"2024-04-25T10:12:47.079541+0000","last_became_peered":"2024-04-25T10:12:47.079541+0000","last_unstale":"2024-04-25T10:12:49.564277+0000","last_undegraded":"2024-04-25T10:12:49.564277+0000","last_fullsized":"2024-04-25T10:12:49.564277+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:02.539831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082805+0000","last_change":"2024-04-25T10:12:48.072791+0000","last_active":"2024-04-25T10:12:49.082805+0000","last_peered":"2024-04-25T10:12:49.082805+0000","last_clean":"2024-04-25T10:12:49.082805+0000","last_became_active":"2024-04-25T10:12:48.072440+0000","last_became_peered":"2024-04-25T10:12:48.072440+0000","last_unstale":"2024-04-25T10:12:49.082805+0000","last_undegraded":"2024-04-25T10:12:49.082805+0000","last_fullsized":"2024-04-25T10:12:49.082805+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:34.752949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564636+0000","last_change":"2024-04-25T10:12:47.074406+0000","last_active":"2024-04-25T10:12:49.564636+0000","last_peered":"2024-04-25T10:12:49.564636+0000","last_clean":"2024-04-25T10:12:49.564636+0000","last_became_active":"2024-04-25T10:12:47.074099+0000","last_became_peered":"2024-04-25T10:12:47.074099+0000","last_unstale":"2024-04-25T10:12:49.564636+0000","last_undegraded":"2024-04-25T10:12:49.564636+0000","last_fullsized":"2024-04-25T10:12:49.564636+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:45.215163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562871+0000","last_change":"2024-04-25T10:12:48.076692+0000","last_active":"2024-04-25T10:12:49.562871+0000","last_peered":"2024-04-25T10:12:49.562871+0000","last_clean":"2024-04-25T10:12:49.562871+0000","last_became_active":"2024-04-25T10:12:48.076346+0000","last_became_peered":"2024-04-25T10:12:48.076346+0000","last_unstale":"2024-04-25T10:12:49.562871+0000","last_undegraded":"2024-04-25T10:12:49.562871+0000","last_fullsized":"2024-04-25T10:12:49.562871+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:48.720124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062143+0000","last_change":"2024-04-25T10:12:47.076358+0000","last_active":"2024-04-25T10:12:49.062143+0000","last_peered":"2024-04-25T10:12:49.062143+0000","last_clean":"2024-04-25T10:12:49.062143+0000","last_became_active":"2024-04-25T10:12:47.074226+0000","last_became_peered":"2024-04-25T10:12:47.074226+0000","last_unstale":"2024-04-25T10:12:49.062143+0000","last_undegraded":"2024-04-25T10:12:49.062143+0000","last_fullsized":"2024-04-25T10:12:49.062143+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:21.775250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061596+0000","last_change":"2024-04-25T10:12:48.070738+0000","last_active":"2024-04-25T10:12:49.061596+0000","last_peered":"2024-04-25T10:12:49.061596+0000","last_clean":"2024-04-25T10:12:49.061596+0000","last_became_active":"2024-04-25T10:12:48.070261+0000","last_became_peered":"2024-04-25T10:12:48.070261+0000","last_unstale":"2024-04-25T10:12:49.061596+0000","last_undegraded":"2024-04-25T10:12:49.061596+0000","last_fullsized":"2024-04-25T10:12:49.061596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:04.535025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562777+0000","last_change":"2024-04-25T10:12:47.073287+0000","last_active":"2024-04-25T10:12:49.562777+0000","last_peered":"2024-04-25T10:12:49.562777+0000","last_clean":"2024-04-25T10:12:49.562777+0000","last_became_active":"2024-04-25T10:12:47.072947+0000","last_became_peered":"2024-04-25T10:12:47.072947+0000","last_unstale":"2024-04-25T10:12:49.562777+0000","last_undegraded":"2024-04-25T10:12:49.562777+0000","last_fullsized":"2024-04-25T10:12:49.562777+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:20.125568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061087+0000","last_change":"2024-04-25T10:12:48.075362+0000","last_active":"2024-04-25T10:12:49.061087+0000","last_peered":"2024-04-25T10:12:49.061087+0000","last_clean":"2024-04-25T10:12:49.061087+0000","last_became_active":"2024-04-25T10:12:48.074561+0000","last_became_peered":"2024-04-25T10:12:48.074561+0000","last_unstale":"2024-04-25T10:12:49.061087+0000","last_undegraded":"2024-04-25T10:12:49.061087+0000","last_fullsized":"2024-04-25T10:12:49.061087+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:57.117947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062523+0000","last_change":"2024-04-25T10:12:47.077801+0000","last_active":"2024-04-25T10:12:49.062523+0000","last_peered":"2024-04-25T10:12:49.062523+0000","last_clean":"2024-04-25T10:12:49.062523+0000","last_became_active":"2024-04-25T10:12:47.076171+0000","last_became_peered":"2024-04-25T10:12:47.076171+0000","last_unstale":"2024-04-25T10:12:49.062523+0000","last_undegraded":"2024-04-25T10:12:49.062523+0000","last_fullsized":"2024-04-25T10:12:49.062523+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.491095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562567+0000","last_change":"2024-04-25T10:12:48.081146+0000","last_active":"2024-04-25T10:12:49.562567+0000","last_peered":"2024-04-25T10:12:49.562567+0000","last_clean":"2024-04-25T10:12:49.562567+0000","last_became_active":"2024-04-25T10:12:48.080871+0000","last_became_peered":"2024-04-25T10:12:48.080871+0000","last_unstale":"2024-04-25T10:12:49.562567+0000","last_undegraded":"2024-04-25T10:12:49.562567+0000","last_fullsized":"2024-04-25T10:12:49.562567+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:58.856527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062810+0000","last_change":"2024-04-25T10:12:47.080456+0000","last_active":"2024-04-25T10:12:49.062810+0000","last_peered":"2024-04-25T10:12:49.062810+0000","last_clean":"2024-04-25T10:12:49.062810+0000","last_became_active":"2024-04-25T10:12:47.080132+0000","last_became_peered":"2024-04-25T10:12:47.080132+0000","last_unstale":"2024-04-25T10:12:49.062810+0000","last_undegraded":"2024-04-25T10:12:49.062810+0000","last_fullsized":"2024-04-25T10:12:49.062810+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:22.324692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.565188+0000","last_change":"2024-04-25T10:12:48.077677+0000","last_active":"2024-04-25T10:12:49.565188+0000","last_peered":"2024-04-25T10:12:49.565188+0000","last_clean":"2024-04-25T10:12:49.565188+0000","last_became_active":"2024-04-25T10:12:48.077353+0000","last_became_peered":"2024-04-25T10:12:48.077353+0000","last_unstale":"2024-04-25T10:12:49.565188+0000","last_undegraded":"2024-04-25T10:12:49.565188+0000","last_fullsized":"2024-04-25T10:12:49.565188+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:39.338104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"119'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083044+0000","last_change":"2024-04-25T10:12:47.067990+0000","last_active":"2024-04-25T10:12:49.083044+0000","last_peered":"2024-04-25T10:12:49.083044+0000","last_clean":"2024-04-25T10:12:49.083044+0000","last_became_active":"2024-04-25T10:12:47.067557+0000","last_became_peered":"2024-04-25T10:12:47.067557+0000","last_unstale":"2024-04-25T10:12:49.083044+0000","last_undegraded":"2024-04-25T10:12:49.083044+0000","last_fullsized":"2024-04-25T10:12:49.083044+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:03.497563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083262+0000","last_change":"2024-04-25T10:12:48.077945+0000","last_active":"2024-04-25T10:12:49.083262+0000","last_peered":"2024-04-25T10:12:49.083262+0000","last_clean":"2024-04-25T10:12:49.083262+0000","last_became_active":"2024-04-25T10:12:48.077668+0000","last_became_peered":"2024-04-25T10:12:48.077668+0000","last_unstale":"2024-04-25T10:12:49.083262+0000","last_undegraded":"2024-04-25T10:12:49.083262+0000","last_fullsized":"2024-04-25T10:12:49.083262+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:03.256609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562306+0000","last_change":"2024-04-25T10:12:47.069691+0000","last_active":"2024-04-25T10:12:49.562306+0000","last_peered":"2024-04-25T10:12:49.562306+0000","last_clean":"2024-04-25T10:12:49.562306+0000","last_became_active":"2024-04-25T10:12:47.069429+0000","last_became_peered":"2024-04-25T10:12:47.069429+0000","last_unstale":"2024-04-25T10:12:49.562306+0000","last_undegraded":"2024-04-25T10:12:49.562306+0000","last_fullsized":"2024-04-25T10:12:49.562306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:05.565804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562345+0000","last_change":"2024-04-25T10:12:48.074133+0000","last_active":"2024-04-25T10:12:49.562345+0000","last_peered":"2024-04-25T10:12:49.562345+0000","last_clean":"2024-04-25T10:12:49.562345+0000","last_became_active":"2024-04-25T10:12:48.073761+0000","last_became_peered":"2024-04-25T10:12:48.073761+0000","last_unstale":"2024-04-25T10:12:49.562345+0000","last_undegraded":"2024-04-25T10:12:49.562345+0000","last_fullsized":"2024-04-25T10:12:49.562345+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:12.423580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061674+0000","last_change":"2024-04-25T10:12:47.077038+0000","last_active":"2024-04-25T10:12:49.061674+0000","last_peered":"2024-04-25T10:12:49.061674+0000","last_clean":"2024-04-25T10:12:49.061674+0000","last_became_active":"2024-04-25T10:12:47.075177+0000","last_became_peered":"2024-04-25T10:12:47.075177+0000","last_unstale":"2024-04-25T10:12:49.061674+0000","last_undegraded":"2024-04-25T10:12:49.061674+0000","last_fullsized":"2024-04-25T10:12:49.061674+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:24.766728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.060990+0000","last_change":"2024-04-25T10:12:48.080902+0000","last_active":"2024-04-25T10:12:49.060990+0000","last_peered":"2024-04-25T10:12:49.060990+0000","last_clean":"2024-04-25T10:12:49.060990+0000","last_became_active":"2024-04-25T10:12:48.078682+0000","last_became_peered":"2024-04-25T10:12:48.078682+0000","last_unstale":"2024-04-25T10:12:49.060990+0000","last_undegraded":"2024-04-25T10:12:49.060990+0000","last_fullsized":"2024-04-25T10:12:49.060990+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:47.702871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209857+0000","last_change":"2024-04-25T10:12:47.070656+0000","last_active":"2024-04-25T10:12:49.209857+0000","last_peered":"2024-04-25T10:12:49.209857+0000","last_clean":"2024-04-25T10:12:49.209857+0000","last_became_active":"2024-04-25T10:12:47.070203+0000","last_became_peered":"2024-04-25T10:12:47.070203+0000","last_unstale":"2024-04-25T10:12:49.209857+0000","last_undegraded":"2024-04-25T10:12:49.209857+0000","last_fullsized":"2024-04-25T10:12:49.209857+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:43.477252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061666+0000","last_change":"2024-04-25T10:12:48.066482+0000","last_active":"2024-04-25T10:12:49.061666+0000","last_peered":"2024-04-25T10:12:49.061666+0000","last_clean":"2024-04-25T10:12:49.061666+0000","last_became_active":"2024-04-25T10:12:48.065415+0000","last_became_peered":"2024-04-25T10:12:48.065415+0000","last_unstale":"2024-04-25T10:12:49.061666+0000","last_undegraded":"2024-04-25T10:12:49.061666+0000","last_fullsized":"2024-04-25T10:12:49.061666+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:02.563229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061784+0000","last_change":"2024-04-25T10:12:47.074728+0000","last_active":"2024-04-25T10:12:49.061784+0000","last_peered":"2024-04-25T10:12:49.061784+0000","last_clean":"2024-04-25T10:12:49.061784+0000","last_became_active":"2024-04-25T10:12:47.074449+0000","last_became_peered":"2024-04-25T10:12:47.074449+0000","last_unstale":"2024-04-25T10:12:49.061784+0000","last_undegraded":"2024-04-25T10:12:49.061784+0000","last_fullsized":"2024-04-25T10:12:49.061784+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:50.870957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562694+0000","last_change":"2024-04-25T10:12:48.083474+0000","last_active":"2024-04-25T10:12:49.562694+0000","last_peered":"2024-04-25T10:12:49.562694+0000","last_clean":"2024-04-25T10:12:49.562694+0000","last_became_active":"2024-04-25T10:12:48.082111+0000","last_became_peered":"2024-04-25T10:12:48.082111+0000","last_unstale":"2024-04-25T10:12:49.562694+0000","last_undegraded":"2024-04-25T10:12:49.562694+0000","last_fullsized":"2024-04-25T10:12:49.562694+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:45.154149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082124+0000","last_change":"2024-04-25T10:12:47.069367+0000","last_active":"2024-04-25T10:12:49.082124+0000","last_peered":"2024-04-25T10:12:49.082124+0000","last_clean":"2024-04-25T10:12:49.082124+0000","last_became_active":"2024-04-25T10:12:47.068988+0000","last_became_peered":"2024-04-25T10:12:47.068988+0000","last_unstale":"2024-04-25T10:12:49.082124+0000","last_undegraded":"2024-04-25T10:12:49.082124+0000","last_fullsized":"2024-04-25T10:12:49.082124+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:21.779924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.062647+0000","last_change":"2024-04-25T10:12:48.076747+0000","last_active":"2024-04-25T10:12:49.062647+0000","last_peered":"2024-04-25T10:12:49.062647+0000","last_clean":"2024-04-25T10:12:49.062647+0000","last_became_active":"2024-04-25T10:12:48.076422+0000","last_became_peered":"2024-04-25T10:12:48.076422+0000","last_unstale":"2024-04-25T10:12:49.062647+0000","last_undegraded":"2024-04-25T10:12:49.062647+0000","last_fullsized":"2024-04-25T10:12:49.062647+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:07.231763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"119'2","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.565258+0000","last_change":"2024-04-25T10:12:47.079505+0000","last_active":"2024-04-25T10:12:49.565258+0000","last_peered":"2024-04-25T10:12:49.565258+0000","last_clean":"2024-04-25T10:12:49.565258+0000","last_became_active":"2024-04-25T10:12:47.079218+0000","last_became_peered":"2024-04-25T10:12:47.079218+0000","last_unstale":"2024-04-25T10:12:49.565258+0000","last_undegraded":"2024-04-25T10:12:49.565258+0000","last_fullsized":"2024-04-25T10:12:49.565258+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:02.239976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063692+0000","last_change":"2024-04-25T10:12:48.067677+0000","last_active":"2024-04-25T10:12:49.063692+0000","last_peered":"2024-04-25T10:12:49.063692+0000","last_clean":"2024-04-25T10:12:49.063692+0000","last_became_active":"2024-04-25T10:12:48.067380+0000","last_became_peered":"2024-04-25T10:12:48.067380+0000","last_unstale":"2024-04-25T10:12:49.063692+0000","last_undegraded":"2024-04-25T10:12:49.063692+0000","last_fullsized":"2024-04-25T10:12:49.063692+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:51.422521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.085388+0000","last_change":"2024-04-25T10:12:47.073699+0000","last_active":"2024-04-25T10:12:49.085388+0000","last_peered":"2024-04-25T10:12:49.085388+0000","last_clean":"2024-04-25T10:12:49.085388+0000","last_became_active":"2024-04-25T10:12:47.073384+0000","last_became_peered":"2024-04-25T10:12:47.073384+0000","last_unstale":"2024-04-25T10:12:49.085388+0000","last_undegraded":"2024-04-25T10:12:49.085388+0000","last_fullsized":"2024-04-25T10:12:49.085388+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:37.291427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061016+0000","last_change":"2024-04-25T10:12:48.066403+0000","last_active":"2024-04-25T10:12:49.061016+0000","last_peered":"2024-04-25T10:12:49.061016+0000","last_clean":"2024-04-25T10:12:49.061016+0000","last_became_active":"2024-04-25T10:12:48.066087+0000","last_became_peered":"2024-04-25T10:12:48.066087+0000","last_unstale":"2024-04-25T10:12:49.061016+0000","last_undegraded":"2024-04-25T10:12:49.061016+0000","last_fullsized":"2024-04-25T10:12:49.061016+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:28.952376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.083112+0000","last_change":"2024-04-25T10:12:47.072696+0000","last_active":"2024-04-25T10:12:49.083112+0000","last_peered":"2024-04-25T10:12:49.083112+0000","last_clean":"2024-04-25T10:12:49.083112+0000","last_became_active":"2024-04-25T10:12:47.072431+0000","last_became_peered":"2024-04-25T10:12:47.072431+0000","last_unstale":"2024-04-25T10:12:49.083112+0000","last_undegraded":"2024-04-25T10:12:49.083112+0000","last_fullsized":"2024-04-25T10:12:49.083112+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:05.532622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063853+0000","last_change":"2024-04-25T10:12:48.071918+0000","last_active":"2024-04-25T10:12:49.063853+0000","last_peered":"2024-04-25T10:12:49.063853+0000","last_clean":"2024-04-25T10:12:49.063853+0000","last_became_active":"2024-04-25T10:12:48.071361+0000","last_became_peered":"2024-04-25T10:12:48.071361+0000","last_unstale":"2024-04-25T10:12:49.063853+0000","last_undegraded":"2024-04-25T10:12:49.063853+0000","last_fullsized":"2024-04-25T10:12:49.063853+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.646161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.063089+0000","last_change":"2024-04-25T10:12:47.080066+0000","last_active":"2024-04-25T10:12:49.063089+0000","last_peered":"2024-04-25T10:12:49.063089+0000","last_clean":"2024-04-25T10:12:49.063089+0000","last_became_active":"2024-04-25T10:12:47.079791+0000","last_became_peered":"2024-04-25T10:12:47.079791+0000","last_unstale":"2024-04-25T10:12:49.063089+0000","last_undegraded":"2024-04-25T10:12:49.063089+0000","last_fullsized":"2024-04-25T10:12:49.063089+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:06.611104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.061744+0000","last_change":"2024-04-25T10:12:48.073423+0000","last_active":"2024-04-25T10:12:49.061744+0000","last_peered":"2024-04-25T10:12:49.061744+0000","last_clean":"2024-04-25T10:12:49.061744+0000","last_became_active":"2024-04-25T10:12:48.073054+0000","last_became_peered":"2024-04-25T10:12:48.073054+0000","last_unstale":"2024-04-25T10:12:49.061744+0000","last_undegraded":"2024-04-25T10:12:49.061744+0000","last_fullsized":"2024-04-25T10:12:49.061744+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:13.913979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082704+0000","last_change":"2024-04-25T10:12:47.069310+0000","last_active":"2024-04-25T10:12:49.082704+0000","last_peered":"2024-04-25T10:12:49.082704+0000","last_clean":"2024-04-25T10:12:49.082704+0000","last_became_active":"2024-04-25T10:12:47.069030+0000","last_became_peered":"2024-04-25T10:12:47.069030+0000","last_unstale":"2024-04-25T10:12:49.082704+0000","last_undegraded":"2024-04-25T10:12:49.082704+0000","last_fullsized":"2024-04-25T10:12:49.082704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:39.936186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.562461+0000","last_change":"2024-04-25T10:12:48.078318+0000","last_active":"2024-04-25T10:12:49.562461+0000","last_peered":"2024-04-25T10:12:49.562461+0000","last_clean":"2024-04-25T10:12:49.562461+0000","last_became_active":"2024-04-25T10:12:48.077901+0000","last_became_peered":"2024-04-25T10:12:48.077901+0000","last_unstale":"2024-04-25T10:12:49.562461+0000","last_undegraded":"2024-04-25T10:12:49.562461+0000","last_fullsized":"2024-04-25T10:12:49.562461+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:21.385821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209989+0000","last_change":"2024-04-25T10:12:47.072608+0000","last_active":"2024-04-25T10:12:49.209989+0000","last_peered":"2024-04-25T10:12:49.209989+0000","last_clean":"2024-04-25T10:12:49.209989+0000","last_became_active":"2024-04-25T10:12:47.072359+0000","last_became_peered":"2024-04-25T10:12:47.072359+0000","last_unstale":"2024-04-25T10:12:49.209989+0000","last_undegraded":"2024-04-25T10:12:49.209989+0000","last_fullsized":"2024-04-25T10:12:49.209989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:52.967236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082346+0000","last_change":"2024-04-25T10:12:48.067172+0000","last_active":"2024-04-25T10:12:49.082346+0000","last_peered":"2024-04-25T10:12:49.082346+0000","last_clean":"2024-04-25T10:12:49.082346+0000","last_became_active":"2024-04-25T10:12:48.066889+0000","last_became_peered":"2024-04-25T10:12:48.066889+0000","last_unstale":"2024-04-25T10:12:49.082346+0000","last_undegraded":"2024-04-25T10:12:49.082346+0000","last_fullsized":"2024-04-25T10:12:49.082346+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:44.176804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563123+0000","last_change":"2024-04-25T10:12:47.082986+0000","last_active":"2024-04-25T10:12:49.563123+0000","last_peered":"2024-04-25T10:12:49.563123+0000","last_clean":"2024-04-25T10:12:49.563123+0000","last_became_active":"2024-04-25T10:12:47.082527+0000","last_became_peered":"2024-04-25T10:12:47.082527+0000","last_unstale":"2024-04-25T10:12:49.563123+0000","last_undegraded":"2024-04-25T10:12:49.563123+0000","last_fullsized":"2024-04-25T10:12:49.563123+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:58.747613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.564668+0000","last_change":"2024-04-25T10:12:48.070946+0000","last_active":"2024-04-25T10:12:49.564668+0000","last_peered":"2024-04-25T10:12:49.564668+0000","last_clean":"2024-04-25T10:12:49.564668+0000","last_became_active":"2024-04-25T10:12:48.065391+0000","last_became_peered":"2024-04-25T10:12:48.065391+0000","last_unstale":"2024-04-25T10:12:49.564668+0000","last_undegraded":"2024-04-25T10:12:49.564668+0000","last_fullsized":"2024-04-25T10:12:49.564668+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:08.810812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082427+0000","last_change":"2024-04-25T10:12:47.065999+0000","last_active":"2024-04-25T10:12:49.082427+0000","last_peered":"2024-04-25T10:12:49.082427+0000","last_clean":"2024-04-25T10:12:49.082427+0000","last_became_active":"2024-04-25T10:12:47.063695+0000","last_became_peered":"2024-04-25T10:12:47.063695+0000","last_unstale":"2024-04-25T10:12:49.082427+0000","last_undegraded":"2024-04-25T10:12:49.082427+0000","last_fullsized":"2024-04-25T10:12:49.082427+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:01.825831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.082180+0000","last_change":"2024-04-25T10:12:48.072750+0000","last_active":"2024-04-25T10:12:49.082180+0000","last_peered":"2024-04-25T10:12:49.082180+0000","last_clean":"2024-04-25T10:12:49.082180+0000","last_became_active":"2024-04-25T10:12:48.072149+0000","last_became_peered":"2024-04-25T10:12:48.072149+0000","last_unstale":"2024-04-25T10:12:49.082180+0000","last_undegraded":"2024-04-25T10:12:49.082180+0000","last_fullsized":"2024-04-25T10:12:49.082180+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:47.024960+0000","last_clean_scrub_stamp":"2024-04-25T10:12:47.024960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:53.983869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"119'8","reported_seq":22,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.563215+0000","last_change":"2024-04-25T10:12:47.078362+0000","last_active":"2024-04-25T10:12:49.563215+0000","last_peered":"2024-04-25T10:12:49.563215+0000","last_clean":"2024-04-25T10:12:49.563215+0000","last_became_active":"2024-04-25T10:12:47.076966+0000","last_became_peered":"2024-04-25T10:12:47.076966+0000","last_unstale":"2024-04-25T10:12:49.563215+0000","last_undegraded":"2024-04-25T10:12:49.563215+0000","last_fullsized":"2024-04-25T10:12:49.563215+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:46.008327+0000","last_clean_scrub_stamp":"2024-04-25T10:12:46.008327+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:30.836308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":356,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-04-25T10:12:49.209680+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:12:49.209680+0000","last_peered":"2024-04-25T10:12:49.209680+0000","last_clean":"2024-04-25T10:12:49.209680+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:12:49.209680+0000","last_undegraded":"2024-04-25T10:12:49.209680+0000","last_fullsized":"2024-04-25T10:12:49.209680+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509575,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":693838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509577,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1490944,"data_stored":693860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509576,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2093056,"data_stored":1285818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509577,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1490944,"data_stored":693928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509576,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":696032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509575,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33224,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2088960,"data_stored":1284366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509576,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1499136,"data_stored":694524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542282,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1495040,"data_stored":693894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:53.056 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:12:53.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:12:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.314+0000 7fbc67000640 1 Processor -- start 2024-04-25T10:12:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.314+0000 7fbc67000640 1 -- start start 2024-04-25T10:12:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.315+0000 7fbc67000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbc600b48c0 0x7fbc6015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.315+0000 7fbc67000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc60161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.315+0000 7fbc67000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc601606e0 con 0x7fbc60161bc0 2024-04-25T10:12:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.315+0000 7fbc67000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc601609c0 con 0x7fbc600b48c0 2024-04-25T10:12:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.315+0000 7fbc67000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60160ca0 con 0x7fbc600d3f90 2024-04-25T10:12:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.316+0000 7fbc65200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc60161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.316+0000 7fbc65c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbc600b48c0 0x7fbc6015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.316+0000 7fbc66600640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbc60161bc0 0x7fbc600d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:60278/0 (socket says 172.21.15.5:60278) 2024-04-25T10:12:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.316+0000 7fbc66600640 1 -- 172.21.15.5:0/3789121127 learned_addr learned my addr 172.21.15.5:0/3789121127 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc5fe00640 1 -- 172.21.15.5:0/3789121127 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496500373 0 0) 0x7fbc601606e0 con 0x7fbc60161bc0 2024-04-25T10:12:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc5fe00640 1 -- 172.21.15.5:0/3789121127 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc44003520 con 0x7fbc60161bc0 2024-04-25T10:12:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc65c00640 1 -- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 msgr2=0x7fbc60161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc65c00640 1 --2- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc60161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc65c00640 1 -- 172.21.15.5:0/3789121127 >> v1:172.21.15.80:6789/0 conn(0x7fbc60161bc0 legacy=0x7fbc600d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc65c00640 1 -- 172.21.15.5:0/3789121127 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc60160f80 con 0x7fbc600b48c0 2024-04-25T10:12:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.317+0000 7fbc65200640 1 --2- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc60161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.318+0000 7fbc65c00640 1 --2- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015fe60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc500097a0 tx=0x7fbc5002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef0eb975a32c83d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.319+0000 7fbc5fe00640 1 -- 172.21.15.5:0/3789121127 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50037070 con 0x7fbc600b48c0 2024-04-25T10:12:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.319+0000 7fbc5fe00640 1 -- 172.21.15.5:0/3789121127 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5002ae40 con 0x7fbc600b48c0 2024-04-25T10:12:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.319+0000 7fbc5fe00640 1 -- 172.21.15.5:0/3789121127 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50032660 con 0x7fbc600b48c0 2024-04-25T10:12:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.319+0000 7fbc67000640 1 -- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 msgr2=0x7fbc6015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.319+0000 7fbc67000640 1 --2- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015fe60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc500097a0 tx=0x7fbc5002a0f0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.320+0000 7fbc67000640 1 -- 172.21.15.5:0/3789121127 shutdown_connections 2024-04-25T10:12:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.320+0000 7fbc67000640 1 --2- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc60161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.320+0000 7fbc67000640 1 --2- 172.21.15.5:0/3789121127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015fe60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.320+0000 7fbc67000640 1 -- 172.21.15.5:0/3789121127 >> 172.21.15.5:0/3789121127 conn(0x7fbc6008af80 msgr2=0x7fbc600d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.320+0000 7fbc67000640 1 -- 172.21.15.5:0/3789121127 shutdown_connections 2024-04-25T10:12:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.320+0000 7fbc67000640 1 -- 172.21.15.5:0/3789121127 wait complete. 2024-04-25T10:12:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.322+0000 7fbc67000640 1 Processor -- start 2024-04-25T10:12:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.322+0000 7fbc67000640 1 -- start start 2024-04-25T10:12:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc67000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc67000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc600cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc67000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 0x7fbc600c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc67000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc601603a0 con 0x7fbc60161bc0 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc67000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b4ca0 con 0x7fbc600b48c0 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc67000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b3b80 con 0x7fbc600d3f90 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc66600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 0x7fbc600c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc66600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 0x7fbc600c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34618/0 (socket says 172.21.15.5:34618) 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc66600640 1 -- 172.21.15.5:0/3631789719 learned_addr learned my addr 172.21.15.5:0/3631789719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.323+0000 7fbc65200640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc600cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc65c00640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc66600640 1 -- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 msgr2=0x7fbc600cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc66600640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc600cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc66600640 1 -- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 msgr2=0x7fbc6015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc66600640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc66600640 1 -- 172.21.15.5:0/3631789719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600c8040 con 0x7fbc60161bc0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc65c00640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.324+0000 7fbc66600640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 0x7fbc600c7b00 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c3c0 tx=0x7fbc5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.325+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58017070 con 0x7fbc60161bc0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.325+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc58005ce0 con 0x7fbc60161bc0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.325+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58007310 con 0x7fbc60161bc0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.325+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600c82c0 con 0x7fbc60161bc0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.325+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc600c8880 con 0x7fbc60161bc0 2024-04-25T10:12:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.326+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc60068f60 con 0x7fbc60161bc0 2024-04-25T10:12:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.327+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc5801a070 con 0x7fbc60161bc0 2024-04-25T10:12:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.328+0000 7fbc5d600640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc4006b500 0x7fbc4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.328+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc58089630 con 0x7fbc60161bc0 2024-04-25T10:12:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.329+0000 7fbc65c00640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc4006b500 0x7fbc4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.330+0000 7fbc65c00640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc4006b500 0x7fbc4006d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbc50034000 tx=0x7fbc5002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.332+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc600c8040 con 0x7fbc60161bc0 2024-04-25T10:12:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.603+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbc600d30c0 con 0x7fbc60161bc0 2024-04-25T10:12:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.608+0000 7fbc5d600640 1 -- 172.21.15.5:0/3631789719 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbc580583b0 con 0x7fbc60161bc0 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQDZKSpmwCh9ABAAiEzxMZ/lVhz40xzshTMSeg== 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-04-25T10:12:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.610+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc4006b500 msgr2=0x7fbc4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc4006b500 0x7fbc4006d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbc50034000 tx=0x7fbc5002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 msgr2=0x7fbc600c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 0x7fbc600c7b00 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c3c0 tx=0x7fbc5800c880 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 shutdown_connections 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc60161bc0 0x7fbc600c7b00 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc600d3f90 0x7fbc600cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fbc4006b500 0x7fbc4006d9c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 --2- 172.21.15.5:0/3631789719 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc600b48c0 0x7fbc6015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 >> 172.21.15.5:0/3631789719 conn(0x7fbc6008af80 msgr2=0x7fbc600d2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 shutdown_connections 2024-04-25T10:12:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.611+0000 7fbc67000640 1 -- 172.21.15.5:0/3631789719 wait complete. 2024-04-25T10:12:53.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:53.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.834+0000 7f109d800640 1 Processor -- start 2024-04-25T10:12:53.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.835+0000 7f109d800640 1 -- start start 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.836+0000 7f109d800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 0x7f10980b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.836+0000 7f109d800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f10980d2380 0x7f10980b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.836+0000 7f109d800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10980cf410 con 0x7f1098161200 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.836+0000 7f109d800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10980b28b0 con 0x7f10980d2380 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.836+0000 7f109d800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980cf6f0 con 0x7f10980d07d0 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.837+0000 7f1097e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1098161200 0x7f10980d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33980/0 (socket says 172.21.15.5:33980) 2024-04-25T10:12:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.837+0000 7f1097e00640 1 -- 172.21.15.5:0/2256901318 learned_addr learned my addr 172.21.15.5:0/2256901318 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.837+0000 7f1097400640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 0x7f10980b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.837+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51840597 0 0) 0x7f10980cf410 con 0x7f1098161200 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.837+0000 7f109ce00640 1 --2- 172.21.15.5:0/2256901318 >> v2:172.21.15.80:3300/0 conn(0x7f10980d2380 0x7f10980b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.837+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1078003520 con 0x7f1098161200 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3129849777 0 0) 0x7f1078003520 con 0x7f1098161200 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 msgr2=0x7f10980b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 0x7f10980b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 >> v2:172.21.15.80:3300/0 conn(0x7f10980d2380 msgr2=0x7f10980b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 --2- 172.21.15.5:0/2256901318 >> v2:172.21.15.80:3300/0 conn(0x7f10980d2380 0x7f10980b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980d3fd0 con 0x7f1098161200 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1088001670 con 0x7f1098161200 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 >> v1:172.21.15.5:6789/0 conn(0x7f1098161200 legacy=0x7f10980d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10780039b0 0x7f1078003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1078004800 0x7f1078004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 0x7f10780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980b28b0 con 0x7f1078004800 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980cf6f0 con 0x7f10780039b0 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107800b9c0 con 0x7f10980d07d0 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1097400640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1078004800 0x7f1078004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1097400640 1 -- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 msgr2=0x7f10780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1097400640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10980d07d0 0x7f10780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1097e00640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10780039b0 0x7f1078003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.838+0000 7f1097400640 1 -- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10780039b0 msgr2=0x7f1078003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.839+0000 7f1097400640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10780039b0 0x7f1078003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.839+0000 7f1097400640 1 -- 172.21.15.5:0/2256901318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1078003520 con 0x7f1078004800 2024-04-25T10:12:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.839+0000 7f1097e00640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10780039b0 0x7f1078003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.840+0000 7f1097400640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1078004800 0x7f1078004c80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1080009fc0 tx=0x7f1078005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d54b7a2d498ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.840+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080009c10 con 0x7f1078004800 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.840+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1080036070 con 0x7f1078004800 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.840+0000 7f1096a00640 1 -- 172.21.15.5:0/2256901318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108002d070 con 0x7f1078004800 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109d800640 1 -- 172.21.15.5:0/2256901318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1078004800 msgr2=0x7f1078004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109d800640 1 --2- 172.21.15.5:0/2256901318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1078004800 0x7f1078004c80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f1080009fc0 tx=0x7f1078005830 comp rx=0 tx=0).stop 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109ce00640 1 -- 172.21.15.5:0/2256901318 reap_dead start 2024-04-25T10:12:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109d800640 1 -- 172.21.15.5:0/2256901318 shutdown_connections 2024-04-25T10:12:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109d800640 1 -- 172.21.15.5:0/2256901318 >> 172.21.15.5:0/2256901318 conn(0x7f109808af80 msgr2=0x7f10980d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109d800640 1 -- 172.21.15.5:0/2256901318 shutdown_connections 2024-04-25T10:12:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.841+0000 7f109d800640 1 -- 172.21.15.5:0/2256901318 wait complete. 2024-04-25T10:12:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.843+0000 7f109d800640 1 Processor -- start 2024-04-25T10:12:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.843+0000 7f109d800640 1 -- start start 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109d800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1078004800 0x7f1098150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109d800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10980c9420 0x7f10980c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 0x7f109814da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980b4d30 con 0x7f109814d5c0 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109d800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980b4bb0 con 0x7f10980c9420 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109d800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10980b4eb0 con 0x7f1078004800 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 0x7f109814da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 0x7f109814da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34636/0 (socket says 172.21.15.5:34636) 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 -- 172.21.15.5:0/2815855864 learned_addr learned my addr 172.21.15.5:0/2815855864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f1097e00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1078004800 0x7f1098150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f1097400640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10980c9420 0x7f10980c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 -- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1078004800 msgr2=0x7f1098150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1078004800 0x7f1098150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 -- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10980c9420 msgr2=0x7f10980c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10980c9420 0x7f10980c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.844+0000 7f109ce00640 1 -- 172.21.15.5:0/2815855864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109815d680 con 0x7f109814d5c0 2024-04-25T10:12:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.845+0000 7f109ce00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 0x7f109814da40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f109808b4b0 tx=0x7f109000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.845+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090017070 con 0x7f109814d5c0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.845+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109000c680 con 0x7f109814d5c0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.845+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109000ca70 con 0x7f109814d5c0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.845+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10981512a0 con 0x7f109814d5c0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.845+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10980c9c30 con 0x7f109814d5c0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.846+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098068f60 con 0x7f109814d5c0 2024-04-25T10:12:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.847+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f109001a070 con 0x7f109814d5c0 2024-04-25T10:12:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.848+0000 7f108fe00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f107406b500 0x7f107406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.848+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1090089c30 con 0x7f109814d5c0 2024-04-25T10:12:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.849+0000 7f1097e00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f107406b500 0x7f107406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.850+0000 7f1097e00640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f107406b500 0x7f107406d9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f10880017e0 tx=0x7f1088009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:53.853+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1090058910 con 0x7f109814d5c0 2024-04-25T10:12:54.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.089+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10980d2070 con 0x7f109814d5c0 2024-04-25T10:12:54.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.090+0000 7f108fe00640 1 -- 172.21.15.5:0/2815855864 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f1090058210 con 0x7f109814d5c0 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":98,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2518129896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2518129896},{"type":"v1","addr":"172.21.15.5:6835","nonce":2518129896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:12:50.560675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7006":{"gid":7006,"name":"b","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6837/84886517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":84886517},{"type":"v1","addr":"172.21.15.80:6837","nonce":84886517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:12:54.095 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 98 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f107406b500 msgr2=0x7f107406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f107406b500 0x7f107406d9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f10880017e0 tx=0x7f1088009040 comp rx=0 tx=0).stop 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 msgr2=0x7f109814da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 0x7f109814da40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f109808b4b0 tx=0x7f109000ec00 comp rx=0 tx=0).stop 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 shutdown_connections 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f109814d5c0 0x7f109814da40 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f107406b500 0x7f107406d9c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f10980c9420 0x7f10980c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 --2- 172.21.15.5:0/2815855864 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1078004800 0x7f1098150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 >> 172.21.15.5:0/2815855864 conn(0x7f109808af80 msgr2=0x7f109806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 shutdown_connections 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.093+0000 7f109d800640 1 -- 172.21.15.5:0/2815855864 wait complete. 2024-04-25T10:12:54.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:12:50.560675+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7006}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7006': {'gid': 7006, 'name': 'b', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6837/84886517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6836', 'nonce': 84886517}, {'type': 'v1', 'addr': '172.21.15.80:6837', 'nonce': 84886517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:54.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:54.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:12:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.739+0000 7f004d600640 1 Processor -- start 2024-04-25T10:12:54.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.739+0000 7f004d600640 1 -- start start 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f004d600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0048144100 0x7f00480cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f004d600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00480ced20 0x7f004806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f004d600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480cdbf0 con 0x7f004806e7a0 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f004d600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00481444e0 con 0x7f00480ced20 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f004d600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004815ac70 con 0x7f0048144100 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f0046000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00480ced20 0x7f004806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f0046000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f00480ced20 0x7f004806e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34648/0 (socket says 172.21.15.5:34648) 2024-04-25T10:12:54.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.741+0000 7f0046000640 1 -- 172.21.15.5:0/3587730061 learned_addr learned my addr 172.21.15.5:0/3587730061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.742+0000 7f0046a00640 1 --2- 172.21.15.5:0/3587730061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0048144100 0x7f00480cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.742+0000 7f0046000640 1 -- 172.21.15.5:0/3587730061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0048144100 msgr2=0x7f00480cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.742+0000 7f0046000640 1 --2- 172.21.15.5:0/3587730061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0048144100 0x7f00480cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.742+0000 7f0046000640 1 -- 172.21.15.5:0/3587730061 >> v1:172.21.15.80:6789/0 conn(0x7f004806e7a0 legacy=0x7f004815e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.742+0000 7f0046000640 1 -- 172.21.15.5:0/3587730061 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480cded0 con 0x7f00480ced20 2024-04-25T10:12:54.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.743+0000 7f0046000640 1 --2- 172.21.15.5:0/3587730061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00480ced20 0x7f004806e260 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f003000d150 tx=0x7f003002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e0b91165aad44c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.743+0000 7f0045600640 1 -- 172.21.15.5:0/3587730061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000b2e0 con 0x7f00480ced20 2024-04-25T10:12:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.743+0000 7f0045600640 1 -- 172.21.15.5:0/3587730061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0030009600 con 0x7f00480ced20 2024-04-25T10:12:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.743+0000 7f0045600640 1 -- 172.21.15.5:0/3587730061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000b480 con 0x7f00480ced20 2024-04-25T10:12:54.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 -- 172.21.15.5:0/3587730061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00480ced20 msgr2=0x7f004806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 --2- 172.21.15.5:0/3587730061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00480ced20 0x7f004806e260 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f003000d150 tx=0x7f003002c8e0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 -- 172.21.15.5:0/3587730061 shutdown_connections 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 --2- 172.21.15.5:0/3587730061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00480ced20 0x7f004806e260 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 --2- 172.21.15.5:0/3587730061 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0048144100 0x7f00480cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 -- 172.21.15.5:0/3587730061 >> 172.21.15.5:0/3587730061 conn(0x7f0048071150 msgr2=0x7f0048071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 -- 172.21.15.5:0/3587730061 shutdown_connections 2024-04-25T10:12:54.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.744+0000 7f004d600640 1 -- 172.21.15.5:0/3587730061 wait complete. 2024-04-25T10:12:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.746+0000 7f004d600640 1 Processor -- start 2024-04-25T10:12:54.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.746+0000 7f004d600640 1 -- start start 2024-04-25T10:12:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f004d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 0x7f0048159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f004d600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00480ced20 0x7f0048138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f004d600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0048144100 0x7f00481effe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f004d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480cea00 con 0x7f004806e7a0 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f004d600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480ce490 con 0x7f0048144100 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f004d600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004815c140 con 0x7f00480ced20 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f0046a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 0x7f0048159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f0046a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 0x7f0048159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34650/0 (socket says 172.21.15.5:34650) 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.747+0000 7f0046a00640 1 -- 172.21.15.5:0/2961493064 learned_addr learned my addr 172.21.15.5:0/2961493064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046000640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00480ced20 0x7f0048138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046a00640 1 -- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00480ced20 msgr2=0x7f0048138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046a00640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00480ced20 0x7f0048138fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046a00640 1 -- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0048144100 msgr2=0x7f00481effe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046a00640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0048144100 0x7f00481effe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046a00640 1 -- 172.21.15.5:0/2961493064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00481f0ab0 con 0x7f004806e7a0 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f0046a00640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 0x7f0048159e00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f003800d620 tx=0x7f003800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f003ea00640 1 -- 172.21.15.5:0/2961493064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800b620 con 0x7f004806e7a0 2024-04-25T10:12:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f003ea00640 1 -- 172.21.15.5:0/2961493064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038019070 con 0x7f004806e7a0 2024-04-25T10:12:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f003ea00640 1 -- 172.21.15.5:0/2961493064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800ba00 con 0x7f004806e7a0 2024-04-25T10:12:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.748+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00481f0d90 con 0x7f004806e7a0 2024-04-25T10:12:54.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.749+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00481f1350 con 0x7f004806e7a0 2024-04-25T10:12:54.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.751+0000 7f003ea00640 1 -- 172.21.15.5:0/2961493064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f003800bba0 con 0x7f004806e7a0 2024-04-25T10:12:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.752+0000 7f003ea00640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f002406b500 0x7f002406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.752+0000 7f003ea00640 1 -- 172.21.15.5:0/2961493064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f003808a8c0 con 0x7f004806e7a0 2024-04-25T10:12:54.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.752+0000 7f00503ac740 1 Processor -- start 2024-04-25T10:12:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.752+0000 7f0046000640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f002406b500 0x7f002406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.752+0000 7f00503ac740 1 -- start start 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.753+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e32766950 0x563e3278e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.753+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32766d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.753+0000 7f00503ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e32765110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.753+0000 7f00503ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e326ca620 con 0x563e327af5c0 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.753+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e32673ca0 con 0x563e3278ecd0 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.753+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e326c66c0 con 0x563e32766950 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e32765110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e32765110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34660/0 (socket says 172.21.15.5:34660) 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 -- 172.21.15.5:0/1022979720 learned_addr learned my addr 172.21.15.5:0/1022979720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0046000640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f002406b500 0x7f002406d9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0030036000 tx=0x7f003000b000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 -- 172.21.15.5:0/1022979720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e32766950 msgr2=0x563e3278e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e32766950 0x563e3278e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 -- 172.21.15.5:0/1022979720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 msgr2=0x563e32766d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32766d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 -- 172.21.15.5:0/1022979720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e3267af40 con 0x563e327af5c0 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.754+0000 7f0047400640 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e32765110 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0040003d40 tx=0x7f0040004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dff19a075dc5361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f0037000640 1 -- 172.21.15.5:0/1022979720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040017070 con 0x563e327af5c0 2024-04-25T10:12:54.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f0037000640 1 -- 172.21.15.5:0/1022979720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040005480 con 0x563e327af5c0 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f0037000640 1 -- 172.21.15.5:0/1022979720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004000c8e0 con 0x563e327af5c0 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 -- 172.21.15.5:0/1022979720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 msgr2=0x563e32765110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e32765110 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0040003d40 tx=0x7f0040004200 comp rx=0 tx=0).stop 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 -- 172.21.15.5:0/1022979720 shutdown_connections 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e32765110 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32766d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 --2- 172.21.15.5:0/1022979720 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e32766950 0x563e3278e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 -- 172.21.15.5:0/1022979720 >> 172.21.15.5:0/1022979720 conn(0x563e32763b70 msgr2=0x563e327ac550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.755+0000 7f00503ac740 1 -- 172.21.15.5:0/1022979720 shutdown_connections 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.756+0000 7f00503ac740 1 -- 172.21.15.5:0/1022979720 wait complete. 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.756+0000 7f00503ac740 1 -- start start 2024-04-25T10:12:54.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.756+0000 7f00503ac740 1 Processor -- start 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e327c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e32760df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3278e300 con 0x563e3278ecd0 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e326691f0 con 0x563e327af5c0 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e3278dcf0 con 0x563e3277b390 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.757+0000 7f00503ac740 10 client.0 _ll_register_callbacks cb 0x563e32686ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e327c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e327c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34676/0 (socket says 172.21.15.5:34676) 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 -- 172.21.15.5:0/2376653855 learned_addr learned my addr 172.21.15.5:0/2376653855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 -- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 msgr2=0x563e32760df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e32760df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 -- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 msgr2=0x563e327608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 -- 172.21.15.5:0/2376653855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e32688c50 con 0x563e3278ecd0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f0046a00640 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e327c4ba0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f003800b0e0 tx=0x7f0038059540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380098a07e4a522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.758+0000 7f002f400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f002f400640 1 -- 172.21.15.5:0/2376653855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038070030 con 0x563e3278ecd0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f002f400640 1 -- 172.21.15.5:0/2376653855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0038061090 con 0x563e3278ecd0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0040005480 con 0x563e3278ecd0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f002f400640 1 -- 172.21.15.5:0/2376653855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0038061520 con 0x563e3278ecd0 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 10 client.9114 fetch_fsmap learned FSMap version 98 2024-04-25T10:12:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x563e323df2d0 con 0x563e3278ecd0 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f002f400640 1 -- 172.21.15.5:0/2376653855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040005480 con 0x563e3278ecd0 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f002f400640 1 -- 172.21.15.5:0/2376653855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0038077ce0 con 0x563e3278ecd0 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 10 client.9114 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 10 client.9114 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 20 client.9114 populate_metadata read hostname 'smithi005' 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e327c0ee0 0x563e327c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 4 client.9114 mds_command: new command op to 7006 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.759+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 --> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e326d7b60 con 0x563e327c0ee0 2024-04-25T10:12:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.760+0000 7f0046000640 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e327c0ee0 0x563e327c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.761+0000 7f0046000640 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e327c0ee0 0x563e327c12c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=992d20455e914d41 server_cookie=e4449a2e16bdd794 in_seq=0 out_seq=0 2024-04-25T10:12:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.761+0000 7f002f400640 10 client.9114 ms_handle_connect on v2:172.21.15.80:6836/84886517 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f002f400640 1 -- 172.21.15.5:0/2376653855 <== mds.0 v2:172.21.15.80:6836/84886517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e326d7b60 con 0x563e327c0ee0 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f002f400640 10 client.9114 handle_command_reply: tid=0 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 client.9114 shutdown 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 msgr2=0x563e327c4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e327c4ba0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f003800b0e0 tx=0x7f0038059540 comp rx=0 tx=0).stop 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 shutdown_connections 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e327c0ee0 0x563e327c12c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e32760df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 --2- 172.21.15.5:0/2376653855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e327c4ba0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f002f400640 0 client.9114 ms_handle_reset on v2:172.21.15.80:6836/84886517 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.763+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 >> 172.21.15.5:0/2376653855 conn(0x563e32766950 msgr2=0x563e327ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.764+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 shutdown_connections 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.764+0000 7f00503ac740 1 -- 172.21.15.5:0/2376653855 wait complete. 2024-04-25T10:12:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.764+0000 7f00503ac740 20 client.9114 trim_cache size 0 max 16384 2024-04-25T10:12:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.781+0000 7f00503ac740 1 Processor -- start 2024-04-25T10:12:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.781+0000 7f00503ac740 1 -- start start 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32789f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f00503ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e327c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f00503ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00481fb200 con 0x563e327af5c0 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004813ecb0 con 0x563e3278ecd0 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00481fad50 con 0x563e3277b390 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f0047400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e327c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f0047400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e327c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34678/0 (socket says 172.21.15.5:34678) 2024-04-25T10:12:54.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.782+0000 7f0047400640 1 -- 172.21.15.5:0/2737758774 learned_addr learned my addr 172.21.15.5:0/2737758774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0046a00640 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0046000640 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32789f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0047400640 1 -- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 msgr2=0x563e327c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0047400640 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327c4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0047400640 1 -- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 msgr2=0x563e32789f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0047400640 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32789f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0047400640 1 -- 172.21.15.5:0/2737758774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e324a96b0 con 0x563e327af5c0 2024-04-25T10:12:54.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0046000640 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32789f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0047400640 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e327c3650 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0040012000 tx=0x7f004000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74896240fcff6992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.783+0000 7f0036600640 1 -- 172.21.15.5:0/2737758774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040023070 con 0x563e327af5c0 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f0036600640 1 -- 172.21.15.5:0/2737758774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040018040 con 0x563e327af5c0 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f0036600640 1 -- 172.21.15.5:0/2737758774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040017070 con 0x563e327af5c0 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 -- 172.21.15.5:0/2737758774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 msgr2=0x563e327c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e327c3650 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0040012000 tx=0x7f004000f000 comp rx=0 tx=0).stop 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 -- 172.21.15.5:0/2737758774 shutdown_connections 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e327af5c0 0x563e327c3650 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e3278ecd0 0x563e32789f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 --2- 172.21.15.5:0/2737758774 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 -- 172.21.15.5:0/2737758774 >> 172.21.15.5:0/2737758774 conn(0x563e32766950 msgr2=0x563e327ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 -- 172.21.15.5:0/2737758774 shutdown_connections 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.784+0000 7f00503ac740 1 -- 172.21.15.5:0/2737758774 wait complete. 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.785+0000 7f00503ac740 1 -- start start 2024-04-25T10:12:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.785+0000 7f00503ac740 1 Processor -- start 2024-04-25T10:12:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e3281b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3277e360 con 0x563e3278ecd0 2024-04-25T10:12:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e3277de10 con 0x563e327af5c0 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e32793e20 con 0x563e3277b390 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f00503ac740 10 client.0 _ll_register_callbacks cb 0x7f0038006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f0046000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e3281b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f0046000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e3281b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34690/0 (socket says 172.21.15.5:34690) 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.786+0000 7f0046000640 1 -- 172.21.15.5:0/2596072825 learned_addr learned my addr 172.21.15.5:0/2596072825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046a00640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046000640 1 -- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 msgr2=0x563e327771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046000640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0047400640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046000640 1 -- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 msgr2=0x563e327ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046000640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046000640 1 -- 172.21.15.5:0/2596072825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e3277d070 con 0x563e3278ecd0 2024-04-25T10:12:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f0046000640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e3281b060 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f003003e4d0 tx=0x7f0030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ff75fe693d2796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f002f400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f002f400640 1 -- 172.21.15.5:0/2596072825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003000a820 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f002f400640 1 -- 172.21.15.5:0/2596072825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f003000a9c0 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.787+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0040018040 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f002f400640 1 -- 172.21.15.5:0/2596072825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7f003000abc0 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 10 client.9120 fetch_fsmap learned FSMap version 98 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x563e327acb80 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f002f400640 1 -- 172.21.15.5:0/2596072825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040018040 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f002f400640 1 -- 172.21.15.5:0/2596072825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0030008070 con 0x563e3278ecd0 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 10 client.9120 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 10 client.9120 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 20 client.9120 populate_metadata read hostname 'smithi005' 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e32761450 0x563e327bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 4 client.9120 mds_command: new command op to 7006 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.788+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 --> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563e326d1eb0 con 0x563e32761450 2024-04-25T10:12:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.789+0000 7f0046a00640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e32761450 0x563e327bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.790+0000 7f0046a00640 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e32761450 0x563e327bfa20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c297b14fdf984792 server_cookie=cd0cd19c54fb0f8f in_seq=0 out_seq=0 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.790+0000 7f002f400640 10 client.9120 ms_handle_connect on v2:172.21.15.80:6836/84886517 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.790+0000 7f002f400640 1 -- 172.21.15.5:0/2596072825 <== mds.0 v2:172.21.15.80:6836/84886517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563e326d1eb0 con 0x563e32761450 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.790+0000 7f002f400640 10 client.9120 handle_command_reply: tid=0 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.790+0000 7f00503ac740 1 client.9120 shutdown 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 msgr2=0x563e3281b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e3281b060 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f003003e4d0 tx=0x7f0030004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 shutdown_connections 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x563e32761450 0x563e327bfa20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563e327af5c0 0x563e327ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e3278ecd0 0x563e3281b060 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 --2- 172.21.15.5:0/2596072825 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563e3277b390 0x563e327771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f002f400640 0 client.9120 ms_handle_reset on v2:172.21.15.80:6836/84886517 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 >> 172.21.15.5:0/2596072825 conn(0x563e32766950 msgr2=0x563e327ac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 shutdown_connections 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 1 -- 172.21.15.5:0/2596072825 wait complete. 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.791+0000 7f00503ac740 20 client.9120 trim_cache size 0 max 16384 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:12:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7006, 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.7275202649999999, 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 98, 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 120, 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 120, 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 13.143085289 2024-04-25T10:12:54.799 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f002406b500 msgr2=0x7f002406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f002406b500 0x7f002406d9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0030036000 tx=0x7f003000b000 comp rx=0 tx=0).stop 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 msgr2=0x7f0048159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 0x7f0048159e00 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f003800d620 tx=0x7f003800dae0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 shutdown_connections 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f002406b500 0x7f002406d9c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0048144100 0x7f00481effe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f00480ced20 0x7f0048138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 --2- 172.21.15.5:0/2961493064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f004806e7a0 0x7f0048159e00 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 >> 172.21.15.5:0/2961493064 conn(0x7f0048071150 msgr2=0x7f00480b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 shutdown_connections 2024-04-25T10:12:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.792+0000 7f004d600640 1 -- 172.21.15.5:0/2961493064 wait complete. 2024-04-25T10:12:54.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:54.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.988+0000 7fadfbe00640 1 Processor -- start 2024-04-25T10:12:54.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.988+0000 7fadfbe00640 1 -- start start 2024-04-25T10:12:54.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadfc15d8a0 0x7fadfc0d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfbe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc15f410 con 0x7fadfc1614e0 2024-04-25T10:12:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc15f6b0 con 0x7fadfc0b2f30 2024-04-25T10:12:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfbe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc15ffe0 con 0x7fadfc15d8a0 2024-04-25T10:12:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfb400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fadfc1614e0 0x7fadfc0b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:33988/0 (socket says 172.21.15.5:33988) 2024-04-25T10:12:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfb400640 1 -- 172.21.15.5:0/4285429497 learned_addr learned my addr 172.21.15.5:0/4285429497 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:12:54.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.990+0000 7fadfaa00640 1 --2- 172.21.15.5:0/4285429497 >> v2:172.21.15.5:3300/0 conn(0x7fadfc15d8a0 0x7fadfc0d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.991+0000 7fadfa000640 1 -- 172.21.15.5:0/4285429497 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515752894 0 0) 0x7fadfc15f410 con 0x7fadfc1614e0 2024-04-25T10:12:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.991+0000 7fadfa000640 1 -- 172.21.15.5:0/4285429497 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade0003520 con 0x7fadfc1614e0 2024-04-25T10:12:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.991+0000 7fadfaa00640 1 -- 172.21.15.5:0/4285429497 >> v1:172.21.15.80:6789/0 conn(0x7fadfc0b2f30 legacy=0x7fadfc0d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.991+0000 7fadfaa00640 1 -- 172.21.15.5:0/4285429497 >> v1:172.21.15.5:6789/0 conn(0x7fadfc1614e0 legacy=0x7fadfc0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.991+0000 7fadfaa00640 1 -- 172.21.15.5:0/4285429497 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc15f990 con 0x7fadfc15d8a0 2024-04-25T10:12:54.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.992+0000 7fadfaa00640 1 --2- 172.21.15.5:0/4285429497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc15d8a0 0x7fadfc0d2570 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fadec009790 tx=0x7fadec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c485229af0e484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:54.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfa000640 1 -- 172.21.15.5:0/4285429497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec037070 con 0x7fadfc15d8a0 2024-04-25T10:12:54.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfa000640 1 -- 172.21.15.5:0/4285429497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec02ae50 con 0x7fadfc15d8a0 2024-04-25T10:12:54.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfa000640 1 -- 172.21.15.5:0/4285429497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec032660 con 0x7fadfc15d8a0 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfbe00640 1 -- 172.21.15.5:0/4285429497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc15d8a0 msgr2=0x7fadfc0d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfbe00640 1 --2- 172.21.15.5:0/4285429497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc15d8a0 0x7fadfc0d2570 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fadec009790 tx=0x7fadec02a100 comp rx=0 tx=0).stop 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfbe00640 1 -- 172.21.15.5:0/4285429497 shutdown_connections 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfbe00640 1 --2- 172.21.15.5:0/4285429497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc15d8a0 0x7fadfc0d2570 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.993+0000 7fadfbe00640 1 -- 172.21.15.5:0/4285429497 >> 172.21.15.5:0/4285429497 conn(0x7fadfc08af80 msgr2=0x7fadfc0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.994+0000 7fadfbe00640 1 -- 172.21.15.5:0/4285429497 shutdown_connections 2024-04-25T10:12:54.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.994+0000 7fadfbe00640 1 -- 172.21.15.5:0/4285429497 wait complete. 2024-04-25T10:12:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.995+0000 7fadfbe00640 1 Processor -- start 2024-04-25T10:12:54.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.995+0000 7fadfbe00640 1 -- start start 2024-04-25T10:12:55.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.996+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 0x7fadfc152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.996+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadfc15d8a0 0x7fadfc0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.996+0000 7fadfbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadfc1614e0 0x7fadfc15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc15ee10 con 0x7fadfc0b2f30 2024-04-25T10:12:55.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc15ec90 con 0x7fadfc1614e0 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0b2860 con 0x7fadfc15d8a0 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 0x7fadfc152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 0x7fadfc152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34714/0 (socket says 172.21.15.5:34714) 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 -- 172.21.15.5:0/3822395491 learned_addr learned my addr 172.21.15.5:0/3822395491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 -- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadfc15d8a0 msgr2=0x7fadfc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadfc15d8a0 0x7fadfc0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 -- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadfc1614e0 msgr2=0x7fadfc15b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadfc1614e0 0x7fadfc15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.997+0000 7fadfaa00640 1 -- 172.21.15.5:0/3822395491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0caab0 con 0x7fadfc0b2f30 2024-04-25T10:12:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.998+0000 7fadfaa00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 0x7fadfc152760 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fadec035040 tx=0x7fadec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.998+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec041020 con 0x7fadfc0b2f30 2024-04-25T10:12:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.998+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec005b90 con 0x7fadfc0b2f30 2024-04-25T10:12:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.998+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec037040 con 0x7fadfc0b2f30 2024-04-25T10:12:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.998+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc0cad90 con 0x7fadfc0b2f30 2024-04-25T10:12:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.998+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0cb350 con 0x7fadfc0b2f30 2024-04-25T10:12:55.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:54.999+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc0cb720 con 0x7fadfc0b2f30 2024-04-25T10:12:55.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.000+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadec044020 con 0x7fadfc0b2f30 2024-04-25T10:12:55.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.001+0000 7fadf2a00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:55.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.001+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadec0aa620 con 0x7fadfc0b2f30 2024-04-25T10:12:55.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.002+0000 7fadf3e00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:55.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.003+0000 7fadf3e00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fade4000f60 tx=0x7fade40079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:55.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.006+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadec04b030 con 0x7fadfc0b2f30 2024-04-25T10:12:55.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.253+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fadfc0d0e50 con 0x7fadfc0b2f30 2024-04-25T10:12:55.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.253+0000 7fadf2a00640 1 -- 172.21.15.5:0/3822395491 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) v1 ==== 53+0+73 (secure 0 0 0) 0x7fadec0791e0 con 0x7fadfc0b2f30 2024-04-25T10:12:55.258 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 msgr2=0x7fadd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fade4000f60 tx=0x7fade40079d0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 msgr2=0x7fadfc152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 0x7fadfc152760 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fadec035040 tx=0x7fadec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 shutdown_connections 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadd806b4d0 0x7fadd806d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadfc1614e0 0x7fadfc15b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadfc15d8a0 0x7fadfc0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 --2- 172.21.15.5:0/3822395491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadfc0b2f30 0x7fadfc152760 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 >> 172.21.15.5:0/3822395491 conn(0x7fadfc08af80 msgr2=0x7fadfc0b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 shutdown_connections 2024-04-25T10:12:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:55.256+0000 7fadfbe00640 1 -- 172.21.15.5:0/3822395491 wait complete. 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:55.280 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:55.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.281 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41950sec preferred_lft 41950sec 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:12:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:55.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.336 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:55.414 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:12:55.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.419 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:12:55.469 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:12:55.469 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:55.469 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:12:55.469 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:12:55.469 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:55.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:12:55.470 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:12:55.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.624 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:12:55.678 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:55.678 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:55.679 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:55.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:12:55.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:55.770 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:55.826 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:55.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:55.826 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:55.879 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:55.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.018 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.018 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:56.018 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:12:56.041 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:56.068 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:56.068 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:56.068 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:12:56.068 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:12:56.068 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:56.069 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:12:56.069 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:12:56.069 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:12:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:12:56.071 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:12:56.071 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:12:56.071 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:12:56.071 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:12:56.076 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:56.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:56.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.157 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:56.242 INFO:teuthology.orchestra.run.smithi005.stdout:client9132 2024-04-25T10:12:56.243 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:56.243 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:56.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:56.297 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:56.304 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> parent = os.path.join(mount_point, "parent") 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> os.mkdir(parent) 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> flushed = os.path.join(parent, "flushed") 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> os.mkdir(flushed) 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T10:12:56.305 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:56.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:56.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.511 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:12:56.580 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-04-25T10:12:56.581 INFO:teuthology.orchestra.run.smithi005.stdout:./parent 2024-04-25T10:12:56.581 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed 2024-04-25T10:12:56.581 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/charlie 2024-04-25T10:12:56.581 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/alpha 2024-04-25T10:12:56.581 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/bravo 2024-04-25T10:12:56.582 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:56.582 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:56.582 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:12:56.582 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:12:56.582 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:12:56.582 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:56.728 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:12:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:56.743 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:56.743 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:56.743 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:12:56.743 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:12:56.743 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:12:56.743 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:56.933 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:12:56.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:56.947 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:56.947 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:56.947 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:12:56.947 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:12:56.947 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:12:56.948 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:57.142 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:12:57.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:57.157 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:57.157 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:57.157 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:12:57.157 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:12:57.157 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:12:57.157 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:57.354 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-04-25T10:12:57.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:57.368 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:57.368 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:57.368 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:12:57.369 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:12:57.369 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:12:57.369 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:57.548 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-04-25T10:12:57.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:57.562 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:12:57.562 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:12:57.562 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:12:57.562 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:12:57.562 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:12:57.563 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:12:57.756 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-04-25T10:12:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:12:57.771 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:57.771 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:57.870 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:12:57.871 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:57.923 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:58.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.130+0000 7fd7aac00640 1 Processor -- start 2024-04-25T10:12:58.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.130+0000 7fd7aac00640 1 -- start start 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7aac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7a41529e0 0x7fd7a40c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a40c6410 0x7fd7a40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7aac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a415bdc0 con 0x7fd7a4071ed0 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7aac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a40b2340 con 0x7fd7a41529e0 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7aac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a415c060 con 0x7fd7a40c6410 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7a41529e0 0x7fd7a40c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7a3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7a41529e0 0x7fd7a40c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48996/0 (socket says 172.21.15.5:48996) 2024-04-25T10:12:58.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7a3e00640 1 -- 172.21.15.5:0/4076605441 learned_addr learned my addr 172.21.15.5:0/4076605441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:58.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7a3400640 1 --2- 172.21.15.5:0/4076605441 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a40c6410 0x7fd7a40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.132+0000 7fd7a3e00640 1 -- 172.21.15.5:0/4076605441 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a40c6410 msgr2=0x7fd7a40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.133+0000 7fd7a3e00640 1 --2- 172.21.15.5:0/4076605441 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a40c6410 0x7fd7a40ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.133+0000 7fd7a3e00640 1 -- 172.21.15.5:0/4076605441 >> v1:172.21.15.80:6789/0 conn(0x7fd7a4071ed0 legacy=0x7fd7a40c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.133+0000 7fd7a3e00640 1 -- 172.21.15.5:0/4076605441 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a415c340 con 0x7fd7a41529e0 2024-04-25T10:12:58.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.134+0000 7fd7a3e00640 1 --2- 172.21.15.5:0/4076605441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a41529e0 0x7fd7a40c5ed0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd794009e50 tx=0x7fd79402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a0a6121d6de1fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.134+0000 7fd7a2a00640 1 -- 172.21.15.5:0/4076605441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794036070 con 0x7fd7a41529e0 2024-04-25T10:12:58.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.134+0000 7fd7a2a00640 1 -- 172.21.15.5:0/4076605441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd794031600 con 0x7fd7a41529e0 2024-04-25T10:12:58.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.134+0000 7fd7a2a00640 1 -- 172.21.15.5:0/4076605441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940319d0 con 0x7fd7a41529e0 2024-04-25T10:12:58.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 -- 172.21.15.5:0/4076605441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a41529e0 msgr2=0x7fd7a40c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 --2- 172.21.15.5:0/4076605441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a41529e0 0x7fd7a40c5ed0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd794009e50 tx=0x7fd79402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 -- 172.21.15.5:0/4076605441 shutdown_connections 2024-04-25T10:12:58.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 --2- 172.21.15.5:0/4076605441 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a40c6410 0x7fd7a40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 --2- 172.21.15.5:0/4076605441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a41529e0 0x7fd7a40c5ed0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 -- 172.21.15.5:0/4076605441 >> 172.21.15.5:0/4076605441 conn(0x7fd7a4071150 msgr2=0x7fd7a4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 -- 172.21.15.5:0/4076605441 shutdown_connections 2024-04-25T10:12:58.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.135+0000 7fd7aac00640 1 -- 172.21.15.5:0/4076605441 wait complete. 2024-04-25T10:12:58.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.137+0000 7fd7aac00640 1 Processor -- start 2024-04-25T10:12:58.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.137+0000 7fd7aac00640 1 -- start start 2024-04-25T10:12:58.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a4071ed0 0x7fd7a4158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 0x7fd7a4153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7aac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a41544d0 0x7fd7a4156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7aac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a406e160 con 0x7fd7a40c6410 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7aac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a415c900 con 0x7fd7a4071ed0 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7aac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a406e2e0 con 0x7fd7a41544d0 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7a3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 0x7fd7a4153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7a3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 0x7fd7a4153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49002/0 (socket says 172.21.15.5:49002) 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7a3400640 1 -- 172.21.15.5:0/1453883172 learned_addr learned my addr 172.21.15.5:0/1453883172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7a3e00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a4071ed0 0x7fd7a4158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7a8c00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a41544d0 0x7fd7a4156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.138+0000 7fd7a3400640 1 -- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a41544d0 msgr2=0x7fd7a4156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7a3400640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a41544d0 0x7fd7a4156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7a3400640 1 -- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a4071ed0 msgr2=0x7fd7a4158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7a3400640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a4071ed0 0x7fd7a4158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7a3400640 1 -- 172.21.15.5:0/1453883172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40b2970 con 0x7fd7a40c6410 2024-04-25T10:12:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7a3e00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a4071ed0 0x7fd7a4158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7a3400640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 0x7fd7a4153f90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00c7f0 tx=0x7fd78c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c015070 con 0x7fd7a40c6410 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c004220 con 0x7fd7a40c6410 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c003610 con 0x7fd7a40c6410 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a40b2c50 con 0x7fd7a40c6410 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.139+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a40b3210 con 0x7fd7a40c6410 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.141+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a4068fd0 con 0x7fd7a40c6410 2024-04-25T10:12:58.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.142+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd78c0039c0 con 0x7fd7a40c6410 2024-04-25T10:12:58.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.142+0000 7fd79be00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd78006b4d0 0x7fd78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.143+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd78c08a770 con 0x7fd7a40c6410 2024-04-25T10:12:58.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.143+0000 7fd7a3e00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd78006b4d0 0x7fd78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.144+0000 7fd7a3e00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd78006b4d0 0x7fd78006d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd794033000 tx=0x7fd794009000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.147+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd78c0593c0 con 0x7fd7a40c6410 2024-04-25T10:12:58.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.348+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7a40b1810 con 0x7fd7a40c6410 2024-04-25T10:12:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.351+0000 7fd79be00640 1 -- 172.21.15.5:0/1453883172 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16848 (secure 0 0 0) 0x7fd78c058cc0 con 0x7fd7a40c6410 2024-04-25T10:12:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:12:58.356 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":120,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:49.050829+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd78006b4d0 msgr2=0x7fd78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd78006b4d0 0x7fd78006d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd794033000 tx=0x7fd794009000 comp rx=0 tx=0).stop 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 msgr2=0x7fd7a4153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 0x7fd7a4153f90 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00c7f0 tx=0x7fd78c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 shutdown_connections 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7a41544d0 0x7fd7a4156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7a40c6410 0x7fd7a4153f90 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd78006b4d0 0x7fd78006d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 --2- 172.21.15.5:0/1453883172 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a4071ed0 0x7fd7a4158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.354+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 >> 172.21.15.5:0/1453883172 conn(0x7fd7a4071150 msgr2=0x7fd7a41598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.355+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 shutdown_connections 2024-04-25T10:12:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.355+0000 7fd7aac00640 1 -- 172.21.15.5:0/1453883172 wait complete. 2024-04-25T10:12:58.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:45.398584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:46.541796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:58.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:58.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.590+0000 7fd0ed800640 1 Processor -- start 2024-04-25T10:12:58.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.590+0000 7fd0ed800640 1 -- start start 2024-04-25T10:12:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0ed800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0e806e950 0x7fd0e80b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0ed800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0e80ce610 0x7fd0e80ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0ed800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80b2ee0 con 0x7fd0e8143fb0 2024-04-25T10:12:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0ed800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80b3150 con 0x7fd0e806e950 2024-04-25T10:12:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0ed800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80b3430 con 0x7fd0e80ce610 2024-04-25T10:12:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0e6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0e806e950 0x7fd0e80b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0e6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0e806e950 0x7fd0e80b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49008/0 (socket says 172.21.15.5:49008) 2024-04-25T10:12:58.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0e6000640 1 -- 172.21.15.5:0/716892640 learned_addr learned my addr 172.21.15.5:0/716892640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:58.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.592+0000 7fd0e7400640 1 --2- 172.21.15.5:0/716892640 >> v2:172.21.15.80:3300/0 conn(0x7fd0e80ce610 0x7fd0e80ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.593+0000 7fd0e6000640 1 -- 172.21.15.5:0/716892640 >> v2:172.21.15.80:3300/0 conn(0x7fd0e80ce610 msgr2=0x7fd0e80ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.593+0000 7fd0e6000640 1 --2- 172.21.15.5:0/716892640 >> v2:172.21.15.80:3300/0 conn(0x7fd0e80ce610 0x7fd0e80ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.593+0000 7fd0e6000640 1 -- 172.21.15.5:0/716892640 >> v1:172.21.15.80:6789/0 conn(0x7fd0e8143fb0 legacy=0x7fd0e806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:58.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.593+0000 7fd0e6000640 1 -- 172.21.15.5:0/716892640 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80b3710 con 0x7fd0e806e950 2024-04-25T10:12:58.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.594+0000 7fd0e6000640 1 --2- 172.21.15.5:0/716892640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e80b3c30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cd8e0 tx=0x7fd0d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dd6de0993f5e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.594+0000 7fd0e5600640 1 -- 172.21.15.5:0/716892640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4037070 con 0x7fd0e806e950 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.594+0000 7fd0e5600640 1 -- 172.21.15.5:0/716892640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d4032440 con 0x7fd0e806e950 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.594+0000 7fd0e5600640 1 -- 172.21.15.5:0/716892640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4032810 con 0x7fd0e806e950 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 -- 172.21.15.5:0/716892640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 msgr2=0x7fd0e80b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 --2- 172.21.15.5:0/716892640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e80b3c30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cd8e0 tx=0x7fd0d402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 -- 172.21.15.5:0/716892640 shutdown_connections 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 --2- 172.21.15.5:0/716892640 >> v2:172.21.15.80:3300/0 conn(0x7fd0e80ce610 0x7fd0e80ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 --2- 172.21.15.5:0/716892640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e80b3c30 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 -- 172.21.15.5:0/716892640 >> 172.21.15.5:0/716892640 conn(0x7fd0e806ab60 msgr2=0x7fd0e806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 -- 172.21.15.5:0/716892640 shutdown_connections 2024-04-25T10:12:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.595+0000 7fd0ed800640 1 -- 172.21.15.5:0/716892640 wait complete. 2024-04-25T10:12:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.597+0000 7fd0ed800640 1 Processor -- start 2024-04-25T10:12:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.597+0000 7fd0ed800640 1 -- start start 2024-04-25T10:12:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0ed800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0ed800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e80ce610 0x7fd0e80b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0ed800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0e8143fb0 0x7fd0e81e4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0ed800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80cf2f0 con 0x7fd0e806e950 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0ed800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8144390 con 0x7fd0e80ce610 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0ed800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e815ac70 con 0x7fd0e8143fb0 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0e6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0e6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e814ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49018/0 (socket says 172.21.15.5:49018) 2024-04-25T10:12:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0e6a00640 1 -- 172.21.15.5:0/3635294749 learned_addr learned my addr 172.21.15.5:0/3635294749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.598+0000 7fd0e7400640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0e8143fb0 0x7fd0e81e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6000640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e80ce610 0x7fd0e80b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6a00640 1 -- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0e8143fb0 msgr2=0x7fd0e81e4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6a00640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0e8143fb0 0x7fd0e81e4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6a00640 1 -- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e80ce610 msgr2=0x7fd0e80b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6a00640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e80ce610 0x7fd0e80b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6a00640 1 -- 172.21.15.5:0/3635294749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e81e3070 con 0x7fd0e806e950 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0e6a00640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e814ca30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800ba50 tx=0x7fd0d800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8019070 con 0x7fd0e806e950 2024-04-25T10:12:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d800cc20 con 0x7fd0e806e950 2024-04-25T10:12:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8014680 con 0x7fd0e806e950 2024-04-25T10:12:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.599+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e81e3350 con 0x7fd0e806e950 2024-04-25T10:12:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.600+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e81e3910 con 0x7fd0e806e950 2024-04-25T10:12:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.601+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e8068f60 con 0x7fd0e806e950 2024-04-25T10:12:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.602+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0d801c070 con 0x7fd0e806e950 2024-04-25T10:12:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.603+0000 7fd0dea00640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd0c806b4d0 0x7fd0c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.603+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0d8089fa0 con 0x7fd0e806e950 2024-04-25T10:12:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.603+0000 7fd0e6000640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd0c806b4d0 0x7fd0c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.605+0000 7fd0e6000640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd0c806b4d0 0x7fd0c806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4034000 tx=0x7fd0d402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.608+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0d80586d0 con 0x7fd0e806e950 2024-04-25T10:12:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.812+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0e815c1d0 con 0x7fd0e806e950 2024-04-25T10:12:58.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.813+0000 7fd0dea00640 1 -- 172.21.15.5:0/3635294749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0d805c770 con 0x7fd0e806e950 2024-04-25T10:12:58.818 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:12:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd0c806b4d0 msgr2=0x7fd0c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd0c806b4d0 0x7fd0c806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4034000 tx=0x7fd0d402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 msgr2=0x7fd0e814ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e814ca30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800ba50 tx=0x7fd0d800bf10 comp rx=0 tx=0).stop 2024-04-25T10:12:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 shutdown_connections 2024-04-25T10:12:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0e8143fb0 0x7fd0e81e4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd0c806b4d0 0x7fd0c806d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0e80ce610 0x7fd0e80b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 --2- 172.21.15.5:0/3635294749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0e806e950 0x7fd0e814ca30 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 >> 172.21.15.5:0/3635294749 conn(0x7fd0e806ab60 msgr2=0x7fd0e8142060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 shutdown_connections 2024-04-25T10:12:58.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:58.816+0000 7fd0ed800640 1 -- 172.21.15.5:0/3635294749 wait complete. 2024-04-25T10:12:58.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:59.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.052+0000 7f3692c00640 1 Processor -- start 2024-04-25T10:12:59.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.053+0000 7f3692c00640 1 -- start start 2024-04-25T10:12:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3692c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f368c15a500 0x7f368c0af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3692c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f368c072730 0x7f368c0ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3692c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c0d0390 0x7f368c0d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3692c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0b4c80 con 0x7f368c072730 2024-04-25T10:12:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3692c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c0ad950 con 0x7f368c15a500 2024-04-25T10:12:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3692c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b49a0 con 0x7f368c0d0390 2024-04-25T10:12:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3690e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f368c072730 0x7f368c0ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3690e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f368c072730 0x7f368c0ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49028/0 (socket says 172.21.15.5:49028) 2024-04-25T10:12:59.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.055+0000 7f3690e00640 1 -- 172.21.15.5:0/2656714426 learned_addr learned my addr 172.21.15.5:0/2656714426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3692200640 1 --2- 172.21.15.5:0/2656714426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c0d0390 0x7f368c0d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3691800640 1 --2- 172.21.15.5:0/2656714426 >> v2:172.21.15.80:3300/0 conn(0x7f368c15a500 0x7f368c0af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3690e00640 1 -- 172.21.15.5:0/2656714426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c0d0390 msgr2=0x7f368c0d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3690e00640 1 --2- 172.21.15.5:0/2656714426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c0d0390 0x7f368c0d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3690e00640 1 -- 172.21.15.5:0/2656714426 >> v2:172.21.15.80:3300/0 conn(0x7f368c15a500 msgr2=0x7f368c0af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3690e00640 1 --2- 172.21.15.5:0/2656714426 >> v2:172.21.15.80:3300/0 conn(0x7f368c15a500 0x7f368c0af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.056+0000 7f3690e00640 1 -- 172.21.15.5:0/2656714426 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0adbf0 con 0x7f368c072730 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.057+0000 7f3690e00640 1 --2- 172.21.15.5:0/2656714426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c072730 0x7f368c0ae6d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f368c09e630 tx=0x7f367802c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08f0cebbcf900de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.057+0000 7f368be00640 1 -- 172.21.15.5:0/2656714426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678038070 con 0x7f368c072730 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.057+0000 7f368be00640 1 -- 172.21.15.5:0/2656714426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3678031710 con 0x7f368c072730 2024-04-25T10:12:59.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.057+0000 7f368be00640 1 -- 172.21.15.5:0/2656714426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678031ae0 con 0x7f368c072730 2024-04-25T10:12:59.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 -- 172.21.15.5:0/2656714426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c072730 msgr2=0x7f368c0ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 --2- 172.21.15.5:0/2656714426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c072730 0x7f368c0ae6d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f368c09e630 tx=0x7f367802c3f0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 -- 172.21.15.5:0/2656714426 shutdown_connections 2024-04-25T10:12:59.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 --2- 172.21.15.5:0/2656714426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c0d0390 0x7f368c0d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 --2- 172.21.15.5:0/2656714426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c072730 0x7f368c0ae6d0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 --2- 172.21.15.5:0/2656714426 >> v2:172.21.15.80:3300/0 conn(0x7f368c15a500 0x7f368c0af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 -- 172.21.15.5:0/2656714426 >> 172.21.15.5:0/2656714426 conn(0x7f368c0b5ab0 msgr2=0x7f368c0999f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 -- 172.21.15.5:0/2656714426 shutdown_connections 2024-04-25T10:12:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.058+0000 7f3692c00640 1 -- 172.21.15.5:0/2656714426 wait complete. 2024-04-25T10:12:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.060+0000 7f3692c00640 1 Processor -- start 2024-04-25T10:12:59.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.060+0000 7f3692c00640 1 -- start start 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.061+0000 7f3692c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c072730 0x7f368c15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3692c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 0x7f368c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3692c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c15a500 0x7f368c152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3692c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0b4670 con 0x7f368c0d0390 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3692c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0ae1b0 con 0x7f368c15a500 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3692c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0aec10 con 0x7f368c072730 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 0x7f368c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 0x7f368c0b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49034/0 (socket says 172.21.15.5:49034) 2024-04-25T10:12:59.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 -- 172.21.15.5:0/3229478109 learned_addr learned my addr 172.21.15.5:0/3229478109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:59.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 -- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c072730 msgr2=0x7f368c15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c072730 0x7f368c15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 -- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c15a500 msgr2=0x7f368c152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c15a500 0x7f368c152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.062+0000 7f3690e00640 1 -- 172.21.15.5:0/3229478109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0b1c80 con 0x7f368c0d0390 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.063+0000 7f3690e00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 0x7f368c0b1740 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f367802c430 tx=0x7f36780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.063+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678038040 con 0x7f368c0d0390 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.063+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3678033830 con 0x7f368c0d0390 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.063+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678033c40 con 0x7f368c0d0390 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.063+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c15ae60 con 0x7f368c0d0390 2024-04-25T10:12:59.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.063+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c15b420 con 0x7f368c0d0390 2024-04-25T10:12:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.065+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c068f60 con 0x7f368c0d0390 2024-04-25T10:12:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.066+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f367803e430 con 0x7f368c0d0390 2024-04-25T10:12:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.066+0000 7f3689600640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366c06b4d0 0x7f366c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.067+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36780aadf0 con 0x7f368c0d0390 2024-04-25T10:12:59.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.067+0000 7f3691800640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366c06b4d0 0x7f366c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.068+0000 7f3691800640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366c06b4d0 0x7f366c06d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3680006fd0 tx=0x7f3680008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.071+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3678079b70 con 0x7f368c0d0390 2024-04-25T10:12:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.268+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f368c0d0b80 con 0x7f368c0d0390 2024-04-25T10:12:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.269+0000 7f3689600640 1 -- 172.21.15.5:0/3229478109 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3678079470 con 0x7f368c0d0390 2024-04-25T10:12:59.275 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:12:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366c06b4d0 msgr2=0x7f366c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366c06b4d0 0x7f366c06d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3680006fd0 tx=0x7f3680008040 comp rx=0 tx=0).stop 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 msgr2=0x7f368c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 0x7f368c0b1740 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f367802c430 tx=0x7f36780046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 shutdown_connections 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f366c06b4d0 0x7f366c06d990 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c15a500 0x7f368c152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c0d0390 0x7f368c0b1740 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 --2- 172.21.15.5:0/3229478109 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f368c072730 0x7f368c15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 >> 172.21.15.5:0/3229478109 conn(0x7f368c0b5ab0 msgr2=0x7f368c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 shutdown_connections 2024-04-25T10:12:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.272+0000 7f3692c00640 1 -- 172.21.15.5:0/3229478109 wait complete. 2024-04-25T10:12:59.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.510+0000 7fc2bbe00640 1 Processor -- start 2024-04-25T10:12:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.510+0000 7fc2bbe00640 1 -- start start 2024-04-25T10:12:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b40ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2bbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2b40d3760 0x7fc2b40d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2bbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b40d1a40 con 0x7fc2b415deb0 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2bbe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b40b3090 con 0x7fc2b40d3760 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40d1d20 con 0x7fc2b40b2c30 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2b9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2b40d3760 0x7fc2b40d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2b9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2b40d3760 0x7fc2b40d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49038/0 (socket says 172.21.15.5:49038) 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.512+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1423253834 learned_addr learned my addr 172.21.15.5:0/1423253834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.513+0000 7fc2b8a00640 1 --2- 172.21.15.5:0/1423253834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b40ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.513+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1423253834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 msgr2=0x7fc2b40ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.513+0000 7fc2b9e00640 1 --2- 172.21.15.5:0/1423253834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b40ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.513+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1423253834 >> v1:172.21.15.80:6789/0 conn(0x7fc2b415deb0 legacy=0x7fc2b40b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.513+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1423253834 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40d2000 con 0x7fc2b40d3760 2024-04-25T10:12:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.514+0000 7fc2b9e00640 1 --2- 172.21.15.5:0/1423253834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b40d3760 0x7fc2b40d3bf0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009e50 tx=0x7fc2ac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725facd5851af3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.514+0000 7fc2b3e00640 1 -- 172.21.15.5:0/1423253834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac036070 con 0x7fc2b40d3760 2024-04-25T10:12:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.514+0000 7fc2b3e00640 1 -- 172.21.15.5:0/1423253834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac031600 con 0x7fc2b40d3760 2024-04-25T10:12:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.514+0000 7fc2b3e00640 1 -- 172.21.15.5:0/1423253834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0319d0 con 0x7fc2b40d3760 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1423253834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b40d3760 msgr2=0x7fc2b40d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1423253834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b40d3760 0x7fc2b40d3bf0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009e50 tx=0x7fc2ac02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1423253834 shutdown_connections 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1423253834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b40d3760 0x7fc2b40d3bf0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1423253834 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b40ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1423253834 >> 172.21.15.5:0/1423253834 conn(0x7fc2b40726b0 msgr2=0x7fc2b4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1423253834 shutdown_connections 2024-04-25T10:12:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.515+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1423253834 wait complete. 2024-04-25T10:12:59.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.517+0000 7fc2bbe00640 1 Processor -- start 2024-04-25T10:12:59.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.517+0000 7fc2bbe00640 1 -- start start 2024-04-25T10:12:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b4148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40d3760 0x7fc2b4160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 0x7fc2b40cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40b70f0 con 0x7fc2b415deb0 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40d25c0 con 0x7fc2b40d3760 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2bbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40d2ae0 con 0x7fc2b40b2c30 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 0x7fc2b40cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 0x7fc2b40cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49048/0 (socket says 172.21.15.5:49048) 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.518+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1756033367 learned_addr learned my addr 172.21.15.5:0/1756033367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9400640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b4148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b8a00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40d3760 0x7fc2b4160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 msgr2=0x7fc2b4148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9e00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b4148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40d3760 msgr2=0x7fc2b4160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9e00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40d3760 0x7fc2b4160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9e00640 1 -- 172.21.15.5:0/1756033367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40b5410 con 0x7fc2b415deb0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.519+0000 7fc2b9e00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 0x7fc2b40cbf60 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac034040 tx=0x7fc2ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.520+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac03f020 con 0x7fc2b415deb0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.520+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac037040 con 0x7fc2b415deb0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.520+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac036070 con 0x7fc2b415deb0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.520+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b40b56b0 con 0x7fc2b415deb0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.520+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b40b5c70 con 0x7fc2b415deb0 2024-04-25T10:12:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.521+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b4068f60 con 0x7fc2b415deb0 2024-04-25T10:12:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.522+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2ac042020 con 0x7fc2b415deb0 2024-04-25T10:12:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.523+0000 7fc2b1600640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc29806b4d0 0x7fc29806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.523+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2ac0aa900 con 0x7fc2b415deb0 2024-04-25T10:12:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.523+0000 7fc2b9400640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc29806b4d0 0x7fc29806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.525+0000 7fc2b9400640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc29806b4d0 0x7fc29806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8006fd0 tx=0x7fc2a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.527+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2ac049030 con 0x7fc2b415deb0 2024-04-25T10:12:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.728+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2b40c79e0 con 0x7fc2b415deb0 2024-04-25T10:12:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.729+0000 7fc2b1600640 1 -- 172.21.15.5:0/1756033367 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2ac079550 con 0x7fc2b415deb0 2024-04-25T10:12:59.735 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:12:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc29806b4d0 msgr2=0x7fc29806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc29806b4d0 0x7fc29806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8006fd0 tx=0x7fc2a8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 msgr2=0x7fc2b40cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 0x7fc2b40cbf60 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac034040 tx=0x7fc2ac004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 shutdown_connections 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b415deb0 0x7fc2b40cbf60 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc29806b4d0 0x7fc29806d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40d3760 0x7fc2b4160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/1756033367 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2b40b2c30 0x7fc2b4148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 >> 172.21.15.5:0/1756033367 conn(0x7fc2b40726b0 msgr2=0x7fc2b40b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.732+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 shutdown_connections 2024-04-25T10:12:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.733+0000 7fc2bbe00640 1 -- 172.21.15.5:0/1756033367 wait complete. 2024-04-25T10:12:59.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.969+0000 7f17bda00640 1 Processor -- start 2024-04-25T10:12:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.969+0000 7f17bda00640 1 -- start start 2024-04-25T10:12:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17bda00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f17b81529e0 0x7f17b80c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17bda00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17b80c6410 0x7f17b80ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17bda00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b815bdc0 con 0x7f17b8071ed0 2024-04-25T10:12:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17bda00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b80b2340 con 0x7f17b80c6410 2024-04-25T10:12:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17bda00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b815c060 con 0x7f17b81529e0 2024-04-25T10:12:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17b6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17b80c6410 0x7f17b80ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17b6000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f17b80c6410 0x7f17b80ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49052/0 (socket says 172.21.15.5:49052) 2024-04-25T10:12:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.971+0000 7f17b6000640 1 -- 172.21.15.5:0/1992776126 learned_addr learned my addr 172.21.15.5:0/1992776126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.972+0000 7f17b6a00640 1 --2- 172.21.15.5:0/1992776126 >> v2:172.21.15.80:3300/0 conn(0x7f17b81529e0 0x7f17b80c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.972+0000 7f17b6000640 1 -- 172.21.15.5:0/1992776126 >> v2:172.21.15.80:3300/0 conn(0x7f17b81529e0 msgr2=0x7f17b80c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.972+0000 7f17b6000640 1 --2- 172.21.15.5:0/1992776126 >> v2:172.21.15.80:3300/0 conn(0x7f17b81529e0 0x7f17b80c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.972+0000 7f17b6000640 1 -- 172.21.15.5:0/1992776126 >> v1:172.21.15.80:6789/0 conn(0x7f17b8071ed0 legacy=0x7f17b80c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.972+0000 7f17b6000640 1 -- 172.21.15.5:0/1992776126 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b815c340 con 0x7f17b80c6410 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.973+0000 7f17b6000640 1 --2- 172.21.15.5:0/1992776126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80ce350 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f17b8071d60 tx=0x7f17a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6211fbd30d1170d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.973+0000 7f17b5600640 1 -- 172.21.15.5:0/1992776126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4037070 con 0x7f17b80c6410 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.973+0000 7f17b5600640 1 -- 172.21.15.5:0/1992776126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4032440 con 0x7f17b80c6410 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.973+0000 7f17b5600640 1 -- 172.21.15.5:0/1992776126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4032810 con 0x7f17b80c6410 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 -- 172.21.15.5:0/1992776126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 msgr2=0x7f17b80ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 --2- 172.21.15.5:0/1992776126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80ce350 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f17b8071d60 tx=0x7f17a402a230 comp rx=0 tx=0).stop 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 -- 172.21.15.5:0/1992776126 shutdown_connections 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 --2- 172.21.15.5:0/1992776126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80ce350 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 --2- 172.21.15.5:0/1992776126 >> v2:172.21.15.80:3300/0 conn(0x7f17b81529e0 0x7f17b80c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 -- 172.21.15.5:0/1992776126 >> 172.21.15.5:0/1992776126 conn(0x7f17b8071150 msgr2=0x7f17b8071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 -- 172.21.15.5:0/1992776126 shutdown_connections 2024-04-25T10:12:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.974+0000 7f17bda00640 1 -- 172.21.15.5:0/1992776126 wait complete. 2024-04-25T10:12:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.976+0000 7f17bda00640 1 Processor -- start 2024-04-25T10:12:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.976+0000 7f17bda00640 1 -- start start 2024-04-25T10:12:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.977+0000 7f17bda00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17b8071ed0 0x7f17b80b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.977+0000 7f17bda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.977+0000 7f17bda00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17b80b45d0 0x7f17b80b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.977+0000 7f17bda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b806e160 con 0x7f17b80c6410 2024-04-25T10:12:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.977+0000 7f17bda00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b815c900 con 0x7f17b80b45d0 2024-04-25T10:12:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.977+0000 7f17bda00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b806e2e0 con 0x7f17b8071ed0 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17b8071ed0 0x7f17b80b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17b8071ed0 0x7f17b80b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:60404/0 (socket says 172.21.15.5:60404) 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b7400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17b80b45d0 0x7f17b80b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6a00640 1 -- 172.21.15.5:0/2513255133 learned_addr learned my addr 172.21.15.5:0/2513255133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49066/0 (socket says 172.21.15.5:49066) 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 -- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17b8071ed0 msgr2=0x7f17b80b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17b8071ed0 0x7f17b80b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 -- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17b80b45d0 msgr2=0x7f17b80b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17b80b45d0 0x7f17b80b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.978+0000 7f17b6000640 1 -- 172.21.15.5:0/2513255133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b80c7dc0 con 0x7f17b80c6410 2024-04-25T10:12:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.979+0000 7f17b6000640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80b2e40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f17a4031000 tx=0x7f17a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.979+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4041020 con 0x7f17b80c6410 2024-04-25T10:12:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.979+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a4030780 con 0x7f17b80c6410 2024-04-25T10:12:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.979+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4037040 con 0x7f17b80c6410 2024-04-25T10:12:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.979+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b80b34a0 con 0x7f17b80c6410 2024-04-25T10:12:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.979+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b80b3a60 con 0x7f17b80c6410 2024-04-25T10:12:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.981+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b8068fd0 con 0x7f17b80c6410 2024-04-25T10:12:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.982+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17a4044020 con 0x7f17b80c6410 2024-04-25T10:12:59.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.983+0000 7f17aea00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f179806b4d0 0x7f179806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:59.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.983+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17a40aa9d0 con 0x7f17b80c6410 2024-04-25T10:12:59.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.983+0000 7f17b6a00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f179806b4d0 0x7f179806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:59.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.984+0000 7f17b6a00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f179806b4d0 0x7f179806d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006fd0 tx=0x7f17a8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:59.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:12:59.987+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17a404b030 con 0x7f17b80c6410 2024-04-25T10:13:00.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.202+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17b8068fd0 con 0x7f17b80c6410 2024-04-25T10:13:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.203+0000 7f17aea00640 1 -- 172.21.15.5:0/2513255133 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) v1 ==== 80+0+0 (secure 0 0 0) 0x7f17a4079070 con 0x7f17b80c6410 2024-04-25T10:13:00.208 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f179806b4d0 msgr2=0x7f179806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f179806b4d0 0x7f179806d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f17a8006fd0 tx=0x7f17a8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 msgr2=0x7f17b80b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80b2e40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f17a4031000 tx=0x7f17a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 shutdown_connections 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f179806b4d0 0x7f179806d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17b80b45d0 0x7f17b80b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17b80c6410 0x7f17b80b2e40 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 --2- 172.21.15.5:0/2513255133 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f17b8071ed0 0x7f17b80b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.206+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 >> 172.21.15.5:0/2513255133 conn(0x7f17b8071150 msgr2=0x7f17b80cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.207+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 shutdown_connections 2024-04-25T10:13:00.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.207+0000 7f17bda00640 1 -- 172.21.15.5:0/2513255133 wait complete. 2024-04-25T10:13:00.232 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:00.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:00.372 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:13:00.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.372 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:00.424 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:13:00.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:13:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.597+0000 7fe8f6a00640 1 Processor -- start 2024-04-25T10:13:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.597+0000 7fe8f6a00640 1 -- start start 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.598+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f0144fe0 0x7fe8f006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.598+0000 7fe8f6a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8f0071840 0x7fe8f015c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f6a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f01453c0 con 0x7fe8f006d130 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f6a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f00a8560 con 0x7fe8f0071840 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f6a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f006d6a0 con 0x7fe8f0144fe0 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8f0071840 0x7fe8f015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8f0071840 0x7fe8f015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49070/0 (socket says 172.21.15.5:49070) 2024-04-25T10:13:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 -- 172.21.15.5:0/2035888406 learned_addr learned my addr 172.21.15.5:0/2035888406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8efe00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f0144fe0 0x7fe8f006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 -- 172.21.15.5:0/2035888406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f0144fe0 msgr2=0x7fe8f006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f0144fe0 0x7fe8f006e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 -- 172.21.15.5:0/2035888406 >> v1:172.21.15.80:6789/0 conn(0x7fe8f006d130 legacy=0x7fe8f0071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8f4a00640 1 -- 172.21.15.5:0/2035888406 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f006d980 con 0x7fe8f0071840 2024-04-25T10:13:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.599+0000 7fe8efe00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f0144fe0 0x7fe8f006e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.601+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0071840 0x7fe8f015c370 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009870 tx=0x7fe8e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15f4681e6a158a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.601+0000 7fe8eea00640 1 -- 172.21.15.5:0/2035888406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8037070 con 0x7fe8f0071840 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.601+0000 7fe8eea00640 1 -- 172.21.15.5:0/2035888406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8032440 con 0x7fe8f0071840 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.601+0000 7fe8eea00640 1 -- 172.21.15.5:0/2035888406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8032810 con 0x7fe8f0071840 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.601+0000 7fe8f6a00640 1 -- 172.21.15.5:0/2035888406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0071840 msgr2=0x7fe8f015c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.601+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0071840 0x7fe8f015c370 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009870 tx=0x7fe8e802a210 comp rx=0 tx=0).stop 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.602+0000 7fe8f6a00640 1 -- 172.21.15.5:0/2035888406 shutdown_connections 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.602+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0071840 0x7fe8f015c370 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.602+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/2035888406 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f0144fe0 0x7fe8f006e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.602+0000 7fe8f6a00640 1 -- 172.21.15.5:0/2035888406 >> 172.21.15.5:0/2035888406 conn(0x7fe8f006ab60 msgr2=0x7fe8f006af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.602+0000 7fe8f6a00640 1 -- 172.21.15.5:0/2035888406 shutdown_connections 2024-04-25T10:13:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.602+0000 7fe8f6a00640 1 -- 172.21.15.5:0/2035888406 wait complete. 2024-04-25T10:13:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.604+0000 7fe8f6a00640 1 Processor -- start 2024-04-25T10:13:00.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.604+0000 7fe8f6a00640 1 -- start start 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f006d130 0x7fe8f013d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8f0071840 0x7fe8f013b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 0x7fe8f013ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f015be50 con 0x7fe8f0144fe0 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f6a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f006df40 con 0x7fe8f0071840 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f6a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f006eaa0 con 0x7fe8f006d130 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 0x7fe8f013ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 0x7fe8f013ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49082/0 (socket says 172.21.15.5:49082) 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 -- 172.21.15.5:0/982585391 learned_addr learned my addr 172.21.15.5:0/982585391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8efe00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f006d130 0x7fe8f013d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8ef400640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8f0071840 0x7fe8f013b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 -- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f006d130 msgr2=0x7fe8f013d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f006d130 0x7fe8f013d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 -- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8f0071840 msgr2=0x7fe8f013b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8f0071840 0x7fe8f013b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.605+0000 7fe8f4a00640 1 -- 172.21.15.5:0/982585391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f013dbc0 con 0x7fe8f0144fe0 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8ef400640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8f0071840 0x7fe8f013b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 0x7fe8f013ba30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe8e802a250 tx=0x7fe8e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8e7e00640 1 -- 172.21.15.5:0/982585391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8041020 con 0x7fe8f0144fe0 2024-04-25T10:13:00.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8e7e00640 1 -- 172.21.15.5:0/982585391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8030990 con 0x7fe8f0144fe0 2024-04-25T10:13:00.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8e7e00640 1 -- 172.21.15.5:0/982585391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8037040 con 0x7fe8f0144fe0 2024-04-25T10:13:00.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f0140aa0 con 0x7fe8f0144fe0 2024-04-25T10:13:00.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.606+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f0141060 con 0x7fe8f0144fe0 2024-04-25T10:13:00.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.609+0000 7fe8e7e00640 1 -- 172.21.15.5:0/982585391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8e8044020 con 0x7fe8f0144fe0 2024-04-25T10:13:00.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.609+0000 7fe8e7e00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8d006b4d0 0x7fe8d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.610+0000 7fe8e7e00640 1 -- 172.21.15.5:0/982585391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8e80aad30 con 0x7fe8f0144fe0 2024-04-25T10:13:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.610+0000 7fe8f9765740 1 Processor -- start 2024-04-25T10:13:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.610+0000 7fe8efe00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8d006b4d0 0x7fe8d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.610+0000 7fe8f9765740 1 -- start start 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.611+0000 7fe8efe00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8d006b4d0 0x7fe8d006d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007510 tx=0x7fe8e00074a0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.611+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739ed4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 0x55b739ed2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739efd670 0x55b739efdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f9765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739e36c70 con 0x55b739f0f280 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739ddfcf0 con 0x55b739ed2bb0 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b739e32d10 con 0x55b739efd670 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 0x55b739ed2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 0x55b739ed2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49090/0 (socket says 172.21.15.5:49090) 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 -- 172.21.15.5:0/1500886089 learned_addr learned my addr 172.21.15.5:0/1500886089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8efe00640 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739ed4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739efd670 0x55b739efdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 -- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739efd670 msgr2=0x55b739efdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739efd670 0x55b739efdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 -- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 msgr2=0x55b739ed4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739ed4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8ef400640 1 -- 172.21.15.5:0/1500886089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b739c157d0 con 0x55b739f0f280 2024-04-25T10:13:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.612+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739efd670 0x55b739efdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.613+0000 7fe8ef400640 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 0x55b739ed2f90 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc003f20 tx=0x7fe8dc0043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2969ffbf39212a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.613+0000 7fe8dbe00640 1 -- 172.21.15.5:0/1500886089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc005270 con 0x55b739f0f280 2024-04-25T10:13:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.613+0000 7fe8dbe00640 1 -- 172.21.15.5:0/1500886089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc005410 con 0x55b739f0f280 2024-04-25T10:13:00.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.613+0000 7fe8dbe00640 1 -- 172.21.15.5:0/1500886089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00dbe0 con 0x55b739f0f280 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 -- 172.21.15.5:0/1500886089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 msgr2=0x55b739ed2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 0x55b739ed2f90 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc003f20 tx=0x7fe8dc0043e0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 -- 172.21.15.5:0/1500886089 shutdown_connections 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739efd670 0x55b739efdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739f0f280 0x55b739ed2f90 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 --2- 172.21.15.5:0/1500886089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739ed4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 -- 172.21.15.5:0/1500886089 >> 172.21.15.5:0/1500886089 conn(0x55b739e9aa30 msgr2=0x55b739f18670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 -- 172.21.15.5:0/1500886089 shutdown_connections 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.614+0000 7fe8f9765740 1 -- 172.21.15.5:0/1500886089 wait complete. 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.615+0000 7fe8f9765740 1 -- start start 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.615+0000 7fe8f9765740 1 Processor -- start 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739eef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f28330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739dd53a0 con 0x55b739efd670 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739efb1c0 con 0x55b739ed2bb0 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b739f28fe0 con 0x55b739f0f280 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8f9765740 10 client.0 _ll_register_callbacks cb 0x55b739df2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49098/0 (socket says 172.21.15.5:49098) 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.616+0000 7fe8ef400640 1 -- 172.21.15.5:0/597889103 learned_addr learned my addr 172.21.15.5:0/597889103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8efe00640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739eef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f28330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8ef400640 1 -- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 msgr2=0x55b739f28330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8ef400640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f28330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8ef400640 1 -- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 msgr2=0x55b739eef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8ef400640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739eef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8ef400640 1 -- 172.21.15.5:0/597889103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b739b4b3f0 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8efe00640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739eef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8ef400640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11760 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc004420 tx=0x7fe8dc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a122d24368283f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8d8c00640 1 -- 172.21.15.5:0/597889103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc007440 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8dc005410 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.617+0000 7fe8d8c00640 1 -- 172.21.15.5:0/597889103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8dc0075e0 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8d8c00640 1 -- 172.21.15.5:0/597889103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8dc02f050 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 10 client.9174 fetch_fsmap learned FSMap version 98 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x55b739b0b0f0 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8d8c00640 1 -- 172.21.15.5:0/597889103 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc005410 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8d8c00640 1 -- 172.21.15.5:0/597889103 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe8dc032020 con 0x55b739efd670 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 10 client.9174 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 10 client.9174 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T10:13:00.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 20 client.9174 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739f28870 0x55b739effcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 4 client.9174 mds_command: new command op to 7006 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.618+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 --> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b739e441b0 con 0x55b739f28870 2024-04-25T10:13:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.619+0000 7fe8efe00640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739f28870 0x55b739effcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.620+0000 7fe8efe00640 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739f28870 0x55b739effcc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cdb75d924f4553 server_cookie=b2188e530bf060dd in_seq=0 out_seq=0 2024-04-25T10:13:00.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.620+0000 7fe8d8c00640 10 client.9174 ms_handle_connect on v2:172.21.15.80:6836/84886517 2024-04-25T10:13:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.621+0000 7fe8d8c00640 1 -- 172.21.15.5:0/597889103 <== mds.0 v2:172.21.15.80:6836/84886517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b739e441b0 con 0x55b739f28870 2024-04-25T10:13:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.621+0000 7fe8d8c00640 10 client.9174 handle_command_reply: tid=0 2024-04-25T10:13:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 client.9174 shutdown 2024-04-25T10:13:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 msgr2=0x55b739f11760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11760 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc004420 tx=0x7fe8dc0036f0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 shutdown_connections 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739f28870 0x55b739effcc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f28330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11760 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 --2- 172.21.15.5:0/597889103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739eef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8d8c00640 0 client.9174 ms_handle_reset on v2:172.21.15.80:6836/84886517 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 >> 172.21.15.5:0/597889103 conn(0x55b739e9aa30 msgr2=0x55b739efa920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 shutdown_connections 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 1 -- 172.21.15.5:0/597889103 wait complete. 2024-04-25T10:13:00.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.622+0000 7fe8f9765740 20 client.9174 trim_cache size 0 max 16384 2024-04-25T10:13:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.637+0000 7fe8f9765740 1 Processor -- start 2024-04-25T10:13:00.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.637+0000 7fe8f9765740 1 -- start start 2024-04-25T10:13:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739efc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8f9765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f02003a0 con 0x55b739efd670 2024-04-25T10:13:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01ff540 con 0x55b739ed2bb0 2024-04-25T10:13:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00b0630 con 0x55b739f0f280 2024-04-25T10:13:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49100/0 (socket says 172.21.15.5:49100) 2024-04-25T10:13:00.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.638+0000 7fe8ef400640 1 -- 172.21.15.5:0/3764884991 learned_addr learned my addr 172.21.15.5:0/3764884991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8ef400640 1 -- 172.21.15.5:0/3764884991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 msgr2=0x55b739f122a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8ef400640 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8ef400640 1 -- 172.21.15.5:0/3764884991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 msgr2=0x55b739efc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8ef400640 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739efc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8ef400640 1 -- 172.21.15.5:0/3764884991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b739f286a0 con 0x55b739efd670 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8ef400640 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11d60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007a10 tx=0x7fe8dc020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b7254a27085041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8db400640 1 -- 172.21.15.5:0/3764884991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc007240 con 0x55b739efd670 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8db400640 1 -- 172.21.15.5:0/3764884991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc022070 con 0x55b739efd670 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.639+0000 7fe8db400640 1 -- 172.21.15.5:0/3764884991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0075e0 con 0x55b739efd670 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 -- 172.21.15.5:0/3764884991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 msgr2=0x55b739f11d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11d60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007a10 tx=0x7fe8dc020070 comp rx=0 tx=0).stop 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 -- 172.21.15.5:0/3764884991 shutdown_connections 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739f0f280 0x55b739f122a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739f11d60 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 --2- 172.21.15.5:0/3764884991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739ed2bb0 0x55b739efc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 -- 172.21.15.5:0/3764884991 >> 172.21.15.5:0/3764884991 conn(0x55b739e9aa30 msgr2=0x55b739efca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 -- 172.21.15.5:0/3764884991 shutdown_connections 2024-04-25T10:13:00.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.640+0000 7fe8f9765740 1 -- 172.21.15.5:0/3764884991 wait complete. 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.641+0000 7fe8f9765740 1 -- start start 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.641+0000 7fe8f9765740 1 Processor -- start 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739ed2bb0 0x55b739f81600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739efaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739f0f280 0x55b739efb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01fef70 con 0x55b739efd670 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b739efc840 con 0x55b739f0f280 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b739f00010 con 0x55b739ed2bb0 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8f9765740 10 client.0 _ll_register_callbacks cb 0x7fe8dc003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739efaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739efaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49116/0 (socket says 172.21.15.5:49116) 2024-04-25T10:13:00.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.642+0000 7fe8ef400640 1 -- 172.21.15.5:0/2829782035 learned_addr learned my addr 172.21.15.5:0/2829782035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8efe00640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739ed2bb0 0x55b739f81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8ef400640 1 -- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739ed2bb0 msgr2=0x55b739f81600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8f4a00640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739f0f280 0x55b739efb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8ef400640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739ed2bb0 0x55b739f81600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8ef400640 1 -- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739f0f280 msgr2=0x55b739efb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8ef400640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739f0f280 0x55b739efb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8ef400640 1 -- 172.21.15.5:0/2829782035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b739f18ab0 con 0x55b739efd670 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8ef400640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739efaf90 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007d90 tx=0x7fe8dc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d655dc64649c8ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.643+0000 7fe8d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8d8c00640 1 -- 172.21.15.5:0/2829782035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc041070 con 0x55b739efd670 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8d8c00640 1 -- 172.21.15.5:0/2829782035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8dc047ca0 con 0x55b739efd670 2024-04-25T10:13:00.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8dc022070 con 0x55b739efd670 2024-04-25T10:13:00.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8d8c00640 1 -- 172.21.15.5:0/2829782035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8dc006b40 con 0x55b739efd670 2024-04-25T10:13:00.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 10 client.9180 fetch_fsmap learned FSMap version 98 2024-04-25T10:13:00.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) v3 -- 0x55b739f0f820 con 0x55b739efd670 2024-04-25T10:13:00.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8d8c00640 1 -- 172.21.15.5:0/2829782035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc022070 con 0x55b739efd670 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8d8c00640 1 -- 172.21.15.5:0/2829782035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 98) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe8dc032070 con 0x55b739efd670 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 10 client.9180 fetch_fsmap finished waiting for FSMap version 98 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 10 client.9180 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 20 client.9180 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739eccf80 0x55b739ed1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 4 client.9180 mds_command: new command op to 7006 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.644+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 --> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b739e3e500 con 0x55b739eccf80 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.645+0000 7fe8efe00640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739eccf80 0x55b739ed1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.646+0000 7fe8efe00640 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739eccf80 0x55b739ed1790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8d36d5b9e4cd8e server_cookie=37c1b185bbd59f3a in_seq=0 out_seq=0 2024-04-25T10:13:00.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.646+0000 7fe8d8c00640 10 client.9180 ms_handle_connect on v2:172.21.15.80:6836/84886517 2024-04-25T10:13:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8d8c00640 1 -- 172.21.15.5:0/2829782035 <== mds.0 v2:172.21.15.80:6836/84886517 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b739e3e500 con 0x55b739eccf80 2024-04-25T10:13:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8d8c00640 10 client.9180 handle_command_reply: tid=0 2024-04-25T10:13:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 client.9180 shutdown 2024-04-25T10:13:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 msgr2=0x55b739efaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739efaf90 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007d90 tx=0x7fe8dc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 shutdown_connections 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:6836/84886517,v1:172.21.15.80:6837/84886517] conn(0x55b739eccf80 0x55b739ed1790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55b739f0f280 0x55b739efb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b739efd670 0x55b739efaf90 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 --2- 172.21.15.5:0/2829782035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55b739ed2bb0 0x55b739f81600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8d8c00640 0 client.9180 ms_handle_reset on v2:172.21.15.80:6836/84886517 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 >> 172.21.15.5:0/2829782035 conn(0x55b739e9aa30 msgr2=0x55b739efc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 shutdown_connections 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.670+0000 7fe8f9765740 1 -- 172.21.15.5:0/2829782035 wait complete. 2024-04-25T10:13:00.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.671+0000 7fe8f9765740 20 client.9180 trim_cache size 0 max 16384 2024-04-25T10:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-04-25T10:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-04-25T10:13:00.677 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8d006b4d0 msgr2=0x7fe8d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8d006b4d0 0x7fe8d006d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007510 tx=0x7fe8e00074a0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 msgr2=0x7fe8f013ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 0x7fe8f013ba30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe8e802a250 tx=0x7fe8e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 shutdown_connections 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0144fe0 0x7fe8f013ba30 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8d006b4d0 0x7fe8d006d990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8f0071840 0x7fe8f013b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 --2- 172.21.15.5:0/982585391 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8f006d130 0x7fe8f013d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 >> 172.21.15.5:0/982585391 conn(0x7fe8f006ab60 msgr2=0x7fe8f00b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 shutdown_connections 2024-04-25T10:13:00.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.672+0000 7fe8f6a00640 1 -- 172.21.15.5:0/982585391 wait complete. 2024-04-25T10:13:00.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.904+0000 7f2e8e000640 1 Processor -- start 2024-04-25T10:13:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.904+0000 7f2e8e000640 1 -- start start 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.905+0000 7f2e8e000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2e880cea60 0x7f2e88166420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.905+0000 7f2e8e000640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8815bdc0 con 0x7f2e88071ed0 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e8e000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8815c060 con 0x7f2e881529e0 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e8e000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e880b2340 con 0x7f2e880cea60 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e86a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2e88071ed0 0x7f2e880ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45310/0 (socket says 172.21.15.5:45310) 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e86a00640 1 -- 172.21.15.5:0/3286553178 learned_addr learned my addr 172.21.15.5:0/3286553178 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e87e00640 1 --2- 172.21.15.5:0/3286553178 >> v2:172.21.15.5:3300/0 conn(0x7f2e880cea60 0x7f2e88166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e86000640 1 -- 172.21.15.5:0/3286553178 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794117184 0 0) 0x7f2e8815bdc0 con 0x7f2e88071ed0 2024-04-25T10:13:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e87e00640 1 -- 172.21.15.5:0/3286553178 >> v1:172.21.15.80:6789/0 conn(0x7f2e881529e0 legacy=0x7f2e880c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e87e00640 1 -- 172.21.15.5:0/3286553178 >> v1:172.21.15.5:6789/0 conn(0x7f2e88071ed0 legacy=0x7f2e880ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.906+0000 7f2e87e00640 1 -- 172.21.15.5:0/3286553178 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8815c340 con 0x7f2e880cea60 2024-04-25T10:13:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.907+0000 7f2e87e00640 1 --2- 172.21.15.5:0/3286553178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e880cea60 0x7f2e88166420 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f2e880c78c0 tx=0x7f2e8002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef0e0ae149c5a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e86000640 1 -- 172.21.15.5:0/3286553178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8002ae60 con 0x7f2e880cea60 2024-04-25T10:13:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e86000640 1 -- 172.21.15.5:0/3286553178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e80007740 con 0x7f2e880cea60 2024-04-25T10:13:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e86000640 1 -- 172.21.15.5:0/3286553178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80007b10 con 0x7f2e880cea60 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e8e000640 1 -- 172.21.15.5:0/3286553178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e880cea60 msgr2=0x7f2e88166420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3286553178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e880cea60 0x7f2e88166420 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f2e880c78c0 tx=0x7f2e8002a680 comp rx=0 tx=0).stop 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e8e000640 1 -- 172.21.15.5:0/3286553178 shutdown_connections 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.908+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3286553178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e880cea60 0x7f2e88166420 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.909+0000 7f2e8e000640 1 -- 172.21.15.5:0/3286553178 >> 172.21.15.5:0/3286553178 conn(0x7f2e88071150 msgr2=0x7f2e88071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.909+0000 7f2e8e000640 1 -- 172.21.15.5:0/3286553178 shutdown_connections 2024-04-25T10:13:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.909+0000 7f2e8e000640 1 -- 172.21.15.5:0/3286553178 wait complete. 2024-04-25T10:13:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.910+0000 7f2e8e000640 1 Processor -- start 2024-04-25T10:13:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.910+0000 7f2e8e000640 1 -- start start 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.911+0000 7f2e8e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 0x7f2e88149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.911+0000 7f2e8e000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e880cea60 0x7f2e8814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.911+0000 7f2e8e000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e88143860 0x7f2e880b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.911+0000 7f2e8e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e880c65e0 con 0x7f2e88071ed0 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.911+0000 7f2e8e000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8815c900 con 0x7f2e88143860 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.911+0000 7f2e8e000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8806e160 con 0x7f2e880cea60 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 0x7f2e88149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 0x7f2e88149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49140/0 (socket says 172.21.15.5:49140) 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 -- 172.21.15.5:0/3665721140 learned_addr learned my addr 172.21.15.5:0/3665721140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87e00640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e88143860 0x7f2e880b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e86a00640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e880cea60 0x7f2e8814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 -- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e880cea60 msgr2=0x7f2e8814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e880cea60 0x7f2e8814ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 -- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e88143860 msgr2=0x7f2e880b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e88143860 0x7f2e880b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87400640 1 -- 172.21.15.5:0/3665721140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8814a2e0 con 0x7f2e88071ed0 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.912+0000 7f2e87e00640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e88143860 0x7f2e880b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.913+0000 7f2e87400640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 0x7f2e88149d00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800b420 tx=0x7f2e7800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.913+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78011830 con 0x7f2e88071ed0 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.913+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78016070 con 0x7f2e88071ed0 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.913+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78011c10 con 0x7f2e88071ed0 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.913+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e881e5f40 con 0x7f2e88071ed0 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.913+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e881e6500 con 0x7f2e88071ed0 2024-04-25T10:13:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.914+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e88068fd0 con 0x7f2e88071ed0 2024-04-25T10:13:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.915+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e7801b460 con 0x7f2e88071ed0 2024-04-25T10:13:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.917+0000 7f2e7f400640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e6806b4d0 0x7f2e6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.917+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e78089eb0 con 0x7f2e88071ed0 2024-04-25T10:13:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.918+0000 7f2e86a00640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e6806b4d0 0x7f2e6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:00.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.919+0000 7f2e86a00640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e6806b4d0 0x7f2e6806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2e74007fd0 tx=0x7f2e74009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:00.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:00.922+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e78020020 con 0x7f2e88071ed0 2024-04-25T10:13:01.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.160+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e8814b3a0 con 0x7f2e88071ed0 2024-04-25T10:13:01.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.161+0000 7f2e7f400640 1 -- 172.21.15.5:0/3665721140 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e78058ba0 con 0x7f2e88071ed0 2024-04-25T10:13:01.166 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.163+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e6806b4d0 msgr2=0x7f2e6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.163+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e6806b4d0 0x7f2e6806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f2e74007fd0 tx=0x7f2e74009040 comp rx=0 tx=0).stop 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 msgr2=0x7f2e88149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 0x7f2e88149d00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800b420 tx=0x7f2e7800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 shutdown_connections 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2e6806b4d0 0x7f2e6806d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e88143860 0x7f2e880b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2e880cea60 0x7f2e8814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 --2- 172.21.15.5:0/3665721140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88071ed0 0x7f2e88149d00 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 >> 172.21.15.5:0/3665721140 conn(0x7f2e88071150 msgr2=0x7f2e8815a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 shutdown_connections 2024-04-25T10:13:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:01.164+0000 7f2e8e000640 1 -- 172.21.15.5:0/3665721140 wait complete. 2024-04-25T10:13:01.188 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:01.189 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:01.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.190 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:13:01.240 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41944sec preferred_lft 41944sec 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.241 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:01.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.243 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:01.323 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:13:01.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.327 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:13:01.383 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:13:01.383 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:01.383 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:13:01.383 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:13:01.384 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:01.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.384 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:13:01.384 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:13:01.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:13:01.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:13:01.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:13:01.385 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:13:01.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.547 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:13:01.598 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:01.598 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:01.599 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:01.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:13:01.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:01.691 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:01.750 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:01.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:01.750 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:01.804 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:01.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:01.939 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:01.939 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:01.939 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:01.963 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:01.963 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:01.963 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:01.963 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:13:01.964 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:13:01.992 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:13:01.996 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:01.996 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:02.001 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:02.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:02.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.083 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:02.143 INFO:teuthology.orchestra.run.smithi005.stdout:client7105 2024-04-25T10:13:02.144 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:02.144 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:02.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.195 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T10:13:02.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.261 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T10:13:02.383 INFO:teuthology.orchestra.run.smithi005.stderr:8+0 records in 2024-04-25T10:13:02.383 INFO:teuthology.orchestra.run.smithi005.stderr:8+0 records out 2024-04-25T10:13:02.383 INFO:teuthology.orchestra.run.smithi005.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0594854 s, 141 MB/s 2024-04-25T10:13:02.385 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:02.385 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:02.385 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:02.385 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:02.385 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:13:02.385 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:02.532 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-04-25T10:13:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:02.547 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:02.547 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:02.547 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:02.547 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:02.547 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:13:02.547 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:02.739 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-04-25T10:13:02.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:02.753 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:02.753 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:02.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.115+0000 7f1c30a00640 1 Processor -- start 2024-04-25T10:13:03.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.115+0000 7f1c30a00640 1 -- start start 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.117+0000 7f1c30a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c2c154a10 0x7f1c2c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.117+0000 7f1c30a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c2c070f90 0x7f1c2c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.117+0000 7f1c30a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0cd200 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.117+0000 7f1c30a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c071900 con 0x7f1c2c070f90 2024-04-25T10:13:03.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.117+0000 7f1c30a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c06e6f0 con 0x7f1c2c154a10 2024-04-25T10:13:03.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2be00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c2c070f90 0x7f1c2c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2aa00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1c2c0cdca0 0x7f1c2c0b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45326/0 (socket says 172.21.15.5:45326) 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2aa00640 1 -- 172.21.15.5:0/3614432308 learned_addr learned my addr 172.21.15.5:0/3614432308 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2b400640 1 --2- 172.21.15.5:0/3614432308 >> v2:172.21.15.80:3300/0 conn(0x7f1c2c154a10 0x7f1c2c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2a000640 1 -- 172.21.15.5:0/3614432308 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475066600 0 0) 0x7f1c2c0cd200 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2a000640 1 -- 172.21.15.5:0/3614432308 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c10003520 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2be00640 1 -- 172.21.15.5:0/3614432308 >> v2:172.21.15.80:3300/0 conn(0x7f1c2c154a10 msgr2=0x7f1c2c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2be00640 1 --2- 172.21.15.5:0/3614432308 >> v2:172.21.15.80:3300/0 conn(0x7f1c2c154a10 0x7f1c2c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2be00640 1 -- 172.21.15.5:0/3614432308 >> v1:172.21.15.5:6789/0 conn(0x7f1c2c0cdca0 legacy=0x7f1c2c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2be00640 1 -- 172.21.15.5:0/3614432308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c06e9d0 con 0x7f1c2c070f90 2024-04-25T10:13:03.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.118+0000 7f1c2b400640 1 --2- 172.21.15.5:0/3614432308 >> v2:172.21.15.80:3300/0 conn(0x7f1c2c154a10 0x7f1c2c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.119+0000 7f1c2be00640 1 --2- 172.21.15.5:0/3614432308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c070f90 0x7f1c2c0713c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ccb70 tx=0x7f1c2402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e8cfa6d58fdd12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.120+0000 7f1c2a000640 1 -- 172.21.15.5:0/3614432308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24007740 con 0x7f1c2c070f90 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.120+0000 7f1c2a000640 1 -- 172.21.15.5:0/3614432308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c240078e0 con 0x7f1c2c070f90 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.120+0000 7f1c2a000640 1 -- 172.21.15.5:0/3614432308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24007cb0 con 0x7f1c2c070f90 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.120+0000 7f1c30a00640 1 -- 172.21.15.5:0/3614432308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c070f90 msgr2=0x7f1c2c0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.120+0000 7f1c30a00640 1 --2- 172.21.15.5:0/3614432308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c070f90 0x7f1c2c0713c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c0ccb70 tx=0x7f1c2402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.121+0000 7f1c30a00640 1 -- 172.21.15.5:0/3614432308 shutdown_connections 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.121+0000 7f1c30a00640 1 --2- 172.21.15.5:0/3614432308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c070f90 0x7f1c2c0713c0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.121+0000 7f1c30a00640 1 --2- 172.21.15.5:0/3614432308 >> v2:172.21.15.80:3300/0 conn(0x7f1c2c154a10 0x7f1c2c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.121+0000 7f1c30a00640 1 -- 172.21.15.5:0/3614432308 >> 172.21.15.5:0/3614432308 conn(0x7f1c2c066f20 msgr2=0x7f1c2c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.121+0000 7f1c30a00640 1 -- 172.21.15.5:0/3614432308 shutdown_connections 2024-04-25T10:13:03.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.121+0000 7f1c30a00640 1 -- 172.21.15.5:0/3614432308 wait complete. 2024-04-25T10:13:03.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.123+0000 7f1c30a00640 1 Processor -- start 2024-04-25T10:13:03.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.123+0000 7f1c30a00640 1 -- start start 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c30a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c2c070f90 0x7f1c2c0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c30a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 0x7f1c2c14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c30a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c154a10 0x7f1c2c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c30a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0cd780 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c30a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0cd4a0 con 0x7f1c2c154a10 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c30a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c06cff0 con 0x7f1c2c070f90 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 0x7f1c2c14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 0x7f1c2c14ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49160/0 (socket says 172.21.15.5:49160) 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c2aa00640 1 -- 172.21.15.5:0/2945620876 learned_addr learned my addr 172.21.15.5:0/2945620876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c2b400640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c2c070f90 0x7f1c2c0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.124+0000 7f1c2be00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c154a10 0x7f1c2c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c2aa00640 1 -- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c2c070f90 msgr2=0x7f1c2c0b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c2aa00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c2c070f90 0x7f1c2c0b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c2aa00640 1 -- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c154a10 msgr2=0x7f1c2c14d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c2aa00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c154a10 0x7f1c2c14d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c2aa00640 1 -- 172.21.15.5:0/2945620876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c14e220 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c2aa00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 0x7f1c2c14ce70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c500 tx=0x7f1c1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18017070 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18005ce0 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c1f72a0 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18007310 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.125+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c1f7860 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.127+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c0cce40 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.128+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c1801a070 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.128+0000 7f1c23400640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c0c06b4d0 0x7f1c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.129+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c180897f0 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.129+0000 7f1c2b400640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c0c06b4d0 0x7f1c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.131+0000 7f1c2b400640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c0c06b4d0 0x7f1c0c06d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c006fd0 tx=0x7f1c1c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.133+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c18058590 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.332+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c2c15d7e0 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.334+0000 7f1c23400640 1 -- 172.21.15.5:0/2945620876 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f1c18057e90 con 0x7f1c2c0cdca0 2024-04-25T10:13:03.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:03.340 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":120,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:12:49.050829+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c0c06b4d0 msgr2=0x7f1c0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c0c06b4d0 0x7f1c0c06d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c006fd0 tx=0x7f1c1c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 msgr2=0x7f1c2c14ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 0x7f1c2c14ce70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800c500 tx=0x7f1c1800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 shutdown_connections 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1c0c06b4d0 0x7f1c0c06d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c154a10 0x7f1c2c14d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2c0cdca0 0x7f1c2c14ce70 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 --2- 172.21.15.5:0/2945620876 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c2c070f90 0x7f1c2c0b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 >> 172.21.15.5:0/2945620876 conn(0x7f1c2c066f20 msgr2=0x7f1c2c0af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.337+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 shutdown_connections 2024-04-25T10:13:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.338+0000 7f1c30a00640 1 -- 172.21.15.5:0/2945620876 wait complete. 2024-04-25T10:13:03.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:45.398584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:46.541796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:03.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.570+0000 7f6e6c200640 1 Processor -- start 2024-04-25T10:13:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.571+0000 7f6e6c200640 1 -- start start 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.572+0000 7f6e6c200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e64144fe0 0x7f6e6406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.572+0000 7f6e6c200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e6406d130 0x7f6e64071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e6c200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e641453c0 con 0x7f6e64071670 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e6c200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e640a8560 con 0x7f6e64144fe0 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e6c200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6406d6a0 con 0x7f6e6406d130 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e69800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e64144fe0 0x7f6e6406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e69800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e64144fe0 0x7f6e6406e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49162/0 (socket says 172.21.15.5:49162) 2024-04-25T10:13:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e69800640 1 -- 172.21.15.5:0/2251639194 learned_addr learned my addr 172.21.15.5:0/2251639194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.573+0000 7f6e68e00640 1 --2- 172.21.15.5:0/2251639194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e6406d130 0x7f6e64071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.574+0000 7f6e69800640 1 -- 172.21.15.5:0/2251639194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e6406d130 msgr2=0x7f6e64071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.574+0000 7f6e69800640 1 --2- 172.21.15.5:0/2251639194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e6406d130 0x7f6e64071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.574+0000 7f6e69800640 1 -- 172.21.15.5:0/2251639194 >> v1:172.21.15.80:6789/0 conn(0x7f6e64071670 legacy=0x7f6e640cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.574+0000 7f6e69800640 1 -- 172.21.15.5:0/2251639194 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6406d980 con 0x7f6e64144fe0 2024-04-25T10:13:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.575+0000 7f6e69800640 1 --2- 172.21.15.5:0/2251639194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e6406e560 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f12cc1ad9e194cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.575+0000 7f6e63e00640 1 -- 172.21.15.5:0/2251639194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54037070 con 0x7f6e64144fe0 2024-04-25T10:13:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.575+0000 7f6e63e00640 1 -- 172.21.15.5:0/2251639194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e54032440 con 0x7f6e64144fe0 2024-04-25T10:13:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.575+0000 7f6e63e00640 1 -- 172.21.15.5:0/2251639194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54032810 con 0x7f6e64144fe0 2024-04-25T10:13:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 -- 172.21.15.5:0/2251639194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 msgr2=0x7f6e6406e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 --2- 172.21.15.5:0/2251639194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e6406e560 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5402a250 comp rx=0 tx=0).stop 2024-04-25T10:13:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 -- 172.21.15.5:0/2251639194 shutdown_connections 2024-04-25T10:13:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 --2- 172.21.15.5:0/2251639194 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e6406d130 0x7f6e64071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 --2- 172.21.15.5:0/2251639194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e6406e560 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 -- 172.21.15.5:0/2251639194 >> 172.21.15.5:0/2251639194 conn(0x7f6e6406ab60 msgr2=0x7f6e6406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 -- 172.21.15.5:0/2251639194 shutdown_connections 2024-04-25T10:13:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.576+0000 7f6e6c200640 1 -- 172.21.15.5:0/2251639194 wait complete. 2024-04-25T10:13:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.578+0000 7f6e6c200640 1 Processor -- start 2024-04-25T10:13:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.578+0000 7f6e6c200640 1 -- start start 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6c200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e6406d130 0x7f6e640c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6c200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e64071670 0x7f6e641e3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e641e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6415be50 con 0x7f6e64144fe0 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6c200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6406df40 con 0x7f6e6406d130 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6c200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6406eaa0 con 0x7f6e64071670 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e641e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6a200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e641e0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49170/0 (socket says 172.21.15.5:49170) 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.579+0000 7f6e6a200640 1 -- 172.21.15.5:0/677005160 learned_addr learned my addr 172.21.15.5:0/677005160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e69800640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e6406d130 0x7f6e640c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e68e00640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e64071670 0x7f6e641e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e6a200640 1 -- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e64071670 msgr2=0x7f6e641e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e6a200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e64071670 0x7f6e641e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e6a200640 1 -- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e6406d130 msgr2=0x7f6e640c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e6a200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e6406d130 0x7f6e640c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e6a200640 1 -- 172.21.15.5:0/677005160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e641e40d0 con 0x7f6e64144fe0 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e69800640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e6406d130 0x7f6e640c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e6a200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e641e0ff0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c001eb0 tx=0x7f6e5c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.580+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c017070 con 0x7f6e64144fe0 2024-04-25T10:13:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.581+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c0052d0 con 0x7f6e64144fe0 2024-04-25T10:13:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.581+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c003810 con 0x7f6e64144fe0 2024-04-25T10:13:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.581+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e641e4410 con 0x7f6e64144fe0 2024-04-25T10:13:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.581+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e641e4c50 con 0x7f6e64144fe0 2024-04-25T10:13:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.582+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e64068f60 con 0x7f6e64144fe0 2024-04-25T10:13:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.583+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e5c01a070 con 0x7f6e64144fe0 2024-04-25T10:13:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.584+0000 7f6e61600640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6e4406b4d0 0x7f6e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.584+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e5c089760 con 0x7f6e64144fe0 2024-04-25T10:13:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.585+0000 7f6e69800640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6e4406b4d0 0x7f6e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:03.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.586+0000 7f6e69800640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6e4406b4d0 0x7f6e4406d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6e54034000 tx=0x7f6e5402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:03.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.589+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e641e4410 con 0x7f6e64144fe0 2024-04-25T10:13:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.787+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e6406af40 con 0x7f6e64144fe0 2024-04-25T10:13:03.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.788+0000 7f6e61600640 1 -- 172.21.15.5:0/677005160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e5c0583b0 con 0x7f6e64144fe0 2024-04-25T10:13:03.793 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.790+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6e4406b4d0 msgr2=0x7f6e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.790+0000 7f6e6c200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6e4406b4d0 0x7f6e4406d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6e54034000 tx=0x7f6e5402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.790+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 msgr2=0x7f6e641e0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.790+0000 7f6e6c200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e641e0ff0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c001eb0 tx=0x7f6e5c00c890 comp rx=0 tx=0).stop 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 shutdown_connections 2024-04-25T10:13:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e64144fe0 0x7f6e641e0ff0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e64071670 0x7f6e641e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6e4406b4d0 0x7f6e4406d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 --2- 172.21.15.5:0/677005160 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e6406d130 0x7f6e640c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 >> 172.21.15.5:0/677005160 conn(0x7f6e6406ab60 msgr2=0x7f6e640b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 shutdown_connections 2024-04-25T10:13:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:03.791+0000 7f6e6c200640 1 -- 172.21.15.5:0/677005160 wait complete. 2024-04-25T10:13:03.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.015+0000 7fe1bc800640 1 Processor -- start 2024-04-25T10:13:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.016+0000 7fe1bc800640 1 -- start start 2024-04-25T10:13:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1bc800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1b4143fb0 0x7fe1b40b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1bc800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b406e240 0x7fe1b406e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1bc800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1b40ce610 0x7fe1b40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1bc800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b406ebe0 con 0x7fe1b4143fb0 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1bc800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b40b2ee0 con 0x7fe1b40ce610 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1bc800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b40b3150 con 0x7fe1b406e240 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1b9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1b4143fb0 0x7fe1b40b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1b9e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1b4143fb0 0x7fe1b40b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49176/0 (socket says 172.21.15.5:49176) 2024-04-25T10:13:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.017+0000 7fe1b9e00640 1 -- 172.21.15.5:0/781044897 learned_addr learned my addr 172.21.15.5:0/781044897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1b9400640 1 --2- 172.21.15.5:0/781044897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b406e240 0x7fe1b406e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1ba800640 1 --2- 172.21.15.5:0/781044897 >> v2:172.21.15.80:3300/0 conn(0x7fe1b40ce610 0x7fe1b40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1b9e00640 1 -- 172.21.15.5:0/781044897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b406e240 msgr2=0x7fe1b406e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1b9e00640 1 --2- 172.21.15.5:0/781044897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b406e240 0x7fe1b406e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1b9e00640 1 -- 172.21.15.5:0/781044897 >> v2:172.21.15.80:3300/0 conn(0x7fe1b40ce610 msgr2=0x7fe1b40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1b9e00640 1 --2- 172.21.15.5:0/781044897 >> v2:172.21.15.80:3300/0 conn(0x7fe1b40ce610 0x7fe1b40cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.018+0000 7fe1b9e00640 1 -- 172.21.15.5:0/781044897 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40b3430 con 0x7fe1b4143fb0 2024-04-25T10:13:04.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.019+0000 7fe1b9e00640 1 --2- 172.21.15.5:0/781044897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b4143fb0 0x7fe1b40b3c30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe1b406bac0 tx=0x7fe1a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a4653f2d55cd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.019+0000 7fe1b8a00640 1 -- 172.21.15.5:0/781044897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8037070 con 0x7fe1b4143fb0 2024-04-25T10:13:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.019+0000 7fe1b8a00640 1 -- 172.21.15.5:0/781044897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8032440 con 0x7fe1b4143fb0 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.019+0000 7fe1b8a00640 1 -- 172.21.15.5:0/781044897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8032810 con 0x7fe1b4143fb0 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 -- 172.21.15.5:0/781044897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b4143fb0 msgr2=0x7fe1b40b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 --2- 172.21.15.5:0/781044897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b4143fb0 0x7fe1b40b3c30 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe1b406bac0 tx=0x7fe1a802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 -- 172.21.15.5:0/781044897 shutdown_connections 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 --2- 172.21.15.5:0/781044897 >> v2:172.21.15.80:3300/0 conn(0x7fe1b40ce610 0x7fe1b40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 --2- 172.21.15.5:0/781044897 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b406e240 0x7fe1b406e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 --2- 172.21.15.5:0/781044897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b4143fb0 0x7fe1b40b3c30 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 -- 172.21.15.5:0/781044897 >> 172.21.15.5:0/781044897 conn(0x7fe1b406ab60 msgr2=0x7fe1b406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 -- 172.21.15.5:0/781044897 shutdown_connections 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.020+0000 7fe1bc800640 1 -- 172.21.15.5:0/781044897 wait complete. 2024-04-25T10:13:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.022+0000 7fe1bc800640 1 Processor -- start 2024-04-25T10:13:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.022+0000 7fe1bc800640 1 -- start start 2024-04-25T10:13:04.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1bc800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 0x7fe1b4138130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1bc800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b40ce610 0x7fe1b415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1bc800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b4143fb0 0x7fe1b415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1bc800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b40cf2f0 con 0x7fe1b406e240 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1bc800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b4144390 con 0x7fe1b40ce610 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1bc800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b40cef70 con 0x7fe1b4143fb0 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 0x7fe1b4138130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.023+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 0x7fe1b4138130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49184/0 (socket says 172.21.15.5:49184) 2024-04-25T10:13:04.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 -- 172.21.15.5:0/3152499632 learned_addr learned my addr 172.21.15.5:0/3152499632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1ba800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b4143fb0 0x7fe1b415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9400640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b40ce610 0x7fe1b415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 -- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b4143fb0 msgr2=0x7fe1b415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b4143fb0 0x7fe1b415c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 -- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b40ce610 msgr2=0x7fe1b415bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b40ce610 0x7fe1b415bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 -- 172.21.15.5:0/3152499632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b413dbf0 con 0x7fe1b406e240 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9400640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b40ce610 0x7fe1b415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1ba800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b4143fb0 0x7fe1b415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.024+0000 7fe1b9e00640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 0x7fe1b4138130 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe1a802a220 tx=0x7fe1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.025+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8041020 con 0x7fe1b406e240 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.025+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80306c0 con 0x7fe1b406e240 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.025+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8037040 con 0x7fe1b406e240 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.025+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b413cb50 con 0x7fe1b406e240 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.025+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b413d110 con 0x7fe1b406e240 2024-04-25T10:13:04.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.026+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b4068f60 con 0x7fe1b406e240 2024-04-25T10:13:04.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.027+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1a8044020 con 0x7fe1b406e240 2024-04-25T10:13:04.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.028+0000 7fe1ae000640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe19806b500 0x7fe19806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.028+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1a80aab90 con 0x7fe1b406e240 2024-04-25T10:13:04.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.029+0000 7fe1b9400640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe19806b500 0x7fe19806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.030+0000 7fe1b9400640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe19806b500 0x7fe19806d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40b1d70 tx=0x7fe1a4008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.033+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1a804b030 con 0x7fe1b406e240 2024-04-25T10:13:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.242+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1b40ad8f0 con 0x7fe1b406e240 2024-04-25T10:13:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.243+0000 7fe1ae000640 1 -- 172.21.15.5:0/3152499632 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1a80797e0 con 0x7fe1b406e240 2024-04-25T10:13:04.248 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe19806b500 msgr2=0x7fe19806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe19806b500 0x7fe19806d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40b1d70 tx=0x7fe1a4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 msgr2=0x7fe1b4138130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 0x7fe1b4138130 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe1a802a220 tx=0x7fe1a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 shutdown_connections 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b4143fb0 0x7fe1b415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe19806b500 0x7fe19806d9c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b40ce610 0x7fe1b415bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 --2- 172.21.15.5:0/3152499632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b406e240 0x7fe1b4138130 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 >> 172.21.15.5:0/3152499632 conn(0x7fe1b406ab60 msgr2=0x7fe1b4142ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 shutdown_connections 2024-04-25T10:13:04.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.246+0000 7fe1bc800640 1 -- 172.21.15.5:0/3152499632 wait complete. 2024-04-25T10:13:04.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.476+0000 7f4c54c00640 1 Processor -- start 2024-04-25T10:13:04.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.477+0000 7f4c54c00640 1 -- start start 2024-04-25T10:13:04.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.478+0000 7f4c54c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4c50071000 0x7f4c50071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.478+0000 7f4c54c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c500b3840 0x7f4c500b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.478+0000 7f4c54c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5006d020 con 0x7f4c5015f800 2024-04-25T10:13:04.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.478+0000 7f4c54c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500719a0 con 0x7f4c500b3840 2024-04-25T10:13:04.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.478+0000 7f4c54c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5006ded0 con 0x7f4c50071000 2024-04-25T10:13:04.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.479+0000 7f4c4f400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4c5015f800 0x7f4c50162ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45330/0 (socket says 172.21.15.5:45330) 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.479+0000 7f4c4f400640 1 -- 172.21.15.5:0/1204174584 learned_addr learned my addr 172.21.15.5:0/1204174584 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.479+0000 7f4c4fe00640 1 --2- 172.21.15.5:0/1204174584 >> v2:172.21.15.5:3300/0 conn(0x7f4c500b3840 0x7f4c500b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.479+0000 7f4c4ea00640 1 --2- 172.21.15.5:0/1204174584 >> v2:172.21.15.80:3300/0 conn(0x7f4c50071000 0x7f4c50071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.479+0000 7f4c4e000640 1 -- 172.21.15.5:0/1204174584 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059863652 0 0) 0x7f4c5006d020 con 0x7f4c5015f800 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.479+0000 7f4c4e000640 1 -- 172.21.15.5:0/1204174584 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c34003520 con 0x7f4c5015f800 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.480+0000 7f4c4fe00640 1 -- 172.21.15.5:0/1204174584 >> v2:172.21.15.80:3300/0 conn(0x7f4c50071000 msgr2=0x7f4c50071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.480+0000 7f4c4fe00640 1 --2- 172.21.15.5:0/1204174584 >> v2:172.21.15.80:3300/0 conn(0x7f4c50071000 0x7f4c50071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.480+0000 7f4c4e000640 1 -- 172.21.15.5:0/1204174584 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3161099913 0 0) 0x7f4c34003520 con 0x7f4c5015f800 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.480+0000 7f4c4fe00640 1 -- 172.21.15.5:0/1204174584 >> v1:172.21.15.5:6789/0 conn(0x7f4c5015f800 legacy=0x7f4c50162ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.480+0000 7f4c4fe00640 1 -- 172.21.15.5:0/1204174584 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5006d2c0 con 0x7f4c500b3840 2024-04-25T10:13:04.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.481+0000 7f4c4fe00640 1 --2- 172.21.15.5:0/1204174584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c500b3840 0x7f4c500b3c20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f4c5006e050 tx=0x7f4c4802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b224998839405b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.481+0000 7f4c4e000640 1 -- 172.21.15.5:0/1204174584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40001670 con 0x7f4c500b3840 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.481+0000 7f4c4e000640 1 -- 172.21.15.5:0/1204174584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48007540 con 0x7f4c500b3840 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.481+0000 7f4c4e000640 1 -- 172.21.15.5:0/1204174584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48007910 con 0x7f4c500b3840 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 -- 172.21.15.5:0/1204174584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c500b3840 msgr2=0x7f4c500b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 --2- 172.21.15.5:0/1204174584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c500b3840 0x7f4c500b3c20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f4c5006e050 tx=0x7f4c4802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 -- 172.21.15.5:0/1204174584 shutdown_connections 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 --2- 172.21.15.5:0/1204174584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c500b3840 0x7f4c500b3c20 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 --2- 172.21.15.5:0/1204174584 >> v2:172.21.15.80:3300/0 conn(0x7f4c50071000 0x7f4c50071460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 -- 172.21.15.5:0/1204174584 >> 172.21.15.5:0/1204174584 conn(0x7f4c50066f20 msgr2=0x7f4c5006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 -- 172.21.15.5:0/1204174584 shutdown_connections 2024-04-25T10:13:04.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.482+0000 7f4c54c00640 1 -- 172.21.15.5:0/1204174584 wait complete. 2024-04-25T10:13:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.483+0000 7f4c54c00640 1 Processor -- start 2024-04-25T10:13:04.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.484+0000 7f4c54c00640 1 -- start start 2024-04-25T10:13:04.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c54c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c50071000 0x7f4c501fa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c54c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c500b3840 0x7f4c501f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c54c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 0x7f4c5014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c54c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5006db60 con 0x7f4c5015f800 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c54c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5006d880 con 0x7f4c50071000 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c54c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5006e960 con 0x7f4c500b3840 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 0x7f4c5014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 0x7f4c5014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49214/0 (socket says 172.21.15.5:49214) 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 -- 172.21.15.5:0/2123342012 learned_addr learned my addr 172.21.15.5:0/2123342012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4f400640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c50071000 0x7f4c501fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 -- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c500b3840 msgr2=0x7f4c501f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c500b3840 0x7f4c501f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 -- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c50071000 msgr2=0x7f4c501fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c50071000 0x7f4c501fa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.485+0000 7f4c4fe00640 1 -- 172.21.15.5:0/2123342012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c50162710 con 0x7f4c5015f800 2024-04-25T10:13:04.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.486+0000 7f4c4fe00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 0x7f4c5014fc70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009e50 tx=0x7f4c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.486+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48040070 con 0x7f4c5015f800 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.486+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c48031b40 con 0x7f4c5015f800 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.486+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c48009450 con 0x7f4c5015f800 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.486+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c501501b0 con 0x7f4c5015f800 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.486+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c501fb1f0 con 0x7f4c5015f800 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.487+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c50068f60 con 0x7f4c5015f800 2024-04-25T10:13:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.488+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c48039040 con 0x7f4c5015f800 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.489+0000 7f4c47400640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c3006b4d0 0x7f4c3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.489+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c480abb70 con 0x7f4c5015f800 2024-04-25T10:13:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.490+0000 7f4c4f400640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c3006b4d0 0x7f4c3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.491+0000 7f4c4f400640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c3006b4d0 0x7f4c3006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4c40001eb0 tx=0x7f4c40009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.493+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c501501b0 con 0x7f4c5015f800 2024-04-25T10:13:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.692+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c50071500 con 0x7f4c5015f800 2024-04-25T10:13:04.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.693+0000 7f4c47400640 1 -- 172.21.15.5:0/2123342012 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c4807a8e0 con 0x7f4c5015f800 2024-04-25T10:13:04.698 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:13:04.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.695+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c3006b4d0 msgr2=0x7f4c3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.695+0000 7f4c54c00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c3006b4d0 0x7f4c3006d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4c40001eb0 tx=0x7f4c40009040 comp rx=0 tx=0).stop 2024-04-25T10:13:04.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.695+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 msgr2=0x7f4c5014fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.695+0000 7f4c54c00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 0x7f4c5014fc70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f4c48009e50 tx=0x7f4c48004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:04.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 shutdown_connections 2024-04-25T10:13:04.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c5015f800 0x7f4c5014fc70 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4c500b3840 0x7f4c501f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4c3006b4d0 0x7f4c3006d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 --2- 172.21.15.5:0/2123342012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c50071000 0x7f4c501fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 >> 172.21.15.5:0/2123342012 conn(0x7f4c50066f20 msgr2=0x7f4c5015fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 shutdown_connections 2024-04-25T10:13:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.696+0000 7f4c54c00640 1 -- 172.21.15.5:0/2123342012 wait complete. 2024-04-25T10:13:04.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:04.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.924+0000 7fcfc2800640 1 Processor -- start 2024-04-25T10:13:04.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.925+0000 7fcfc2800640 1 -- start start 2024-04-25T10:13:04.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.926+0000 7fcfc2800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcfbc0b70a0 0x7fcfbc0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.926+0000 7fcfc2800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc0cd1e0 con 0x7fcfbc15b720 2024-04-25T10:13:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.926+0000 7fcfc2800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc0b42d0 con 0x7fcfbc099f30 2024-04-25T10:13:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.926+0000 7fcfc2800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b45b0 con 0x7fcfbc0b70a0 2024-04-25T10:13:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.926+0000 7fcfc1400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcfbc15b720 0x7fcfbc0cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:45338/0 (socket says 172.21.15.5:45338) 2024-04-25T10:13:04.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.927+0000 7fcfc1400640 1 -- 172.21.15.5:0/2950587559 learned_addr learned my addr 172.21.15.5:0/2950587559 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.927+0000 7fcfc0a00640 1 --2- 172.21.15.5:0/2950587559 >> v2:172.21.15.80:3300/0 conn(0x7fcfbc0b70a0 0x7fcfbc0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.927+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575673062 0 0) 0x7fcfbc0cd1e0 con 0x7fcfbc15b720 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.927+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa4003520 con 0x7fcfbc15b720 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740995808 0 0) 0x7fcfa4003520 con 0x7fcfbc15b720 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 >> v2:172.21.15.80:3300/0 conn(0x7fcfbc0b70a0 msgr2=0x7fcfbc0999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 --2- 172.21.15.5:0/2950587559 >> v2:172.21.15.80:3300/0 conn(0x7fcfbc0b70a0 0x7fcfbc0999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 >> v1:172.21.15.80:6789/0 conn(0x7fcfbc099f30 legacy=0x7fcfbc165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0b4890 con 0x7fcfbc15b720 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fcfb0001670 con 0x7fcfbc15b720 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 >> v1:172.21.15.5:6789/0 conn(0x7fcfbc15b720 legacy=0x7fcfbc0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfa40039b0 0x7fcfa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa4004800 0x7fcfa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa40051c0 0x7fcfa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b45b0 con 0x7fcfa40051c0 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c060 con 0x7fcfa40039b0 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa400c260 con 0x7fcfa4004800 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfc0a00640 1 --2- 172.21.15.5:0/2950587559 >> v2:172.21.15.80:3300/0 conn(0x7fcfbc0b70a0 0x7fcfbc0999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfc1e00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa40051c0 0x7fcfa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.928+0000 7fcfc1400640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfa40039b0 0x7fcfa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.929+0000 7fcfc0a00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa4004800 0x7fcfa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.929+0000 7fcfc1e00640 1 -- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa4004800 msgr2=0x7fcfa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.929+0000 7fcfc1e00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa4004800 0x7fcfa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.929+0000 7fcfc1e00640 1 -- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfa40039b0 msgr2=0x7fcfa4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.929+0000 7fcfc1e00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfa40039b0 0x7fcfa4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.929+0000 7fcfc1e00640 1 -- 172.21.15.5:0/2950587559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4003520 con 0x7fcfa40051c0 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.930+0000 7fcfc1e00640 1 -- 172.21.15.5:0/2950587559 reap_dead start 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.930+0000 7fcfc1e00640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa40051c0 0x7fcfa400b9c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4001e50 tx=0x7fcfa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99af4d352167fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.930+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4001a70 con 0x7fcfa40051c0 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.930+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb4003970 con 0x7fcfa40051c0 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.930+0000 7fcfbbe00640 1 -- 172.21.15.5:0/2950587559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4003d60 con 0x7fcfa40051c0 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 -- 172.21.15.5:0/2950587559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa40051c0 msgr2=0x7fcfa400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa40051c0 0x7fcfa400b9c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4001e50 tx=0x7fcfa4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 -- 172.21.15.5:0/2950587559 shutdown_connections 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 --2- 172.21.15.5:0/2950587559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfa40051c0 0x7fcfa400b9c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 -- 172.21.15.5:0/2950587559 >> 172.21.15.5:0/2950587559 conn(0x7fcfbc0b2f70 msgr2=0x7fcfbc0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 -- 172.21.15.5:0/2950587559 shutdown_connections 2024-04-25T10:13:04.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.931+0000 7fcfc2800640 1 -- 172.21.15.5:0/2950587559 wait complete. 2024-04-25T10:13:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.933+0000 7fcfc2800640 1 Processor -- start 2024-04-25T10:13:04.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.933+0000 7fcfc2800640 1 -- start start 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc2800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa40051c0 0x7fcfbc1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 0x7fcfbc159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc2800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfbc0c9a50 0x7fcfbc06f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0999f0 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc2800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc15bb00 con 0x7fcfbc0c9a50 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc2800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc099b70 con 0x7fcfa40051c0 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 0x7fcfbc159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 0x7fcfbc159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49236/0 (socket says 172.21.15.5:49236) 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc0a00640 1 -- 172.21.15.5:0/1870872868 learned_addr learned my addr 172.21.15.5:0/1870872868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.934+0000 7fcfc1400640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa40051c0 0x7fcfbc1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc1e00640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfbc0c9a50 0x7fcfbc06f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc0a00640 1 -- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa40051c0 msgr2=0x7fcfbc1f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc0a00640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa40051c0 0x7fcfbc1f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc0a00640 1 -- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfbc0c9a50 msgr2=0x7fcfbc06f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc0a00640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfbc0c9a50 0x7fcfbc06f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc0a00640 1 -- 172.21.15.5:0/1870872868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc15c190 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc1e00640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfbc0c9a50 0x7fcfbc06f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc0a00640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 0x7fcfbc159bc0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0116f0 tx=0x7fcfac011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac00f9c0 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac00fb60 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac015460 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.935+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc1f3bc0 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.936+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc1f4180 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.937+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc068f60 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.938+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfac015600 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.939+0000 7fcfb9600640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfa006b4d0 0x7fcfa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.939+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcfac08af60 con 0x7fcfbc1f6ab0 2024-04-25T10:13:04.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.940+0000 7fcfc1400640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfa006b4d0 0x7fcfa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:04.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.941+0000 7fcfc1400640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfa006b4d0 0x7fcfa006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0c99e0 tx=0x7fcfb0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:04.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:04.944+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfac019030 con 0x7fcfbc1f6ab0 2024-04-25T10:13:05.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.157+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcfbc0cda10 con 0x7fcfbc1f6ab0 2024-04-25T10:13:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.158+0000 7fcfb9600640 1 -- 172.21.15.5:0/1870872868 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcfac059bb0 con 0x7fcfbc1f6ab0 2024-04-25T10:13:05.163 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-04-25T10:13:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfa006b4d0 msgr2=0x7fcfa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfa006b4d0 0x7fcfa006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0c99e0 tx=0x7fcfb0009040 comp rx=0 tx=0).stop 2024-04-25T10:13:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 msgr2=0x7fcfbc159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 0x7fcfbc159bc0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0116f0 tx=0x7fcfac011bb0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 shutdown_connections 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcfa006b4d0 0x7fcfa006d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfbc0c9a50 0x7fcfbc06f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfbc1f6ab0 0x7fcfbc159bc0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 --2- 172.21.15.5:0/1870872868 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcfa40051c0 0x7fcfbc1f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 >> 172.21.15.5:0/1870872868 conn(0x7fcfbc0b2f70 msgr2=0x7fcfbc070710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 shutdown_connections 2024-04-25T10:13:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.161+0000 7fcfc2800640 1 -- 172.21.15.5:0/1870872868 wait complete. 2024-04-25T10:13:05.186 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:05.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:05.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:05.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:13:05.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:05.474 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:05.524 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:05.524 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:05.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:05.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:13:05.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:05.563 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:05.647 INFO:teuthology.orchestra.run.smithi005.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:13:05.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:05.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.899+0000 7f78dd200640 1 Processor -- start 2024-04-25T10:13:05.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.899+0000 7f78dd200640 1 -- start start 2024-04-25T10:13:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.901+0000 7f78dd200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d806e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.901+0000 7f78dd200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78d806e640 0x7f78d815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.901+0000 7f78dd200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d8071d30 con 0x7f78d815d340 2024-04-25T10:13:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.901+0000 7f78dd200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d8142180 con 0x7f78d806e640 2024-04-25T10:13:05.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.901+0000 7f78dd200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d806d620 con 0x7f78d8141da0 2024-04-25T10:13:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d6a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78d806e640 0x7f78d815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d7400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d806e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d7e00640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f78d815d340 0x7f78d80cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:33278/0 (socket says 172.21.15.5:33278) 2024-04-25T10:13:05.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d7e00640 1 -- 172.21.15.5:0/508273520 learned_addr learned my addr 172.21.15.5:0/508273520 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d6a00640 1 -- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 msgr2=0x7f78d806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d6a00640 1 --2- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d806e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d6a00640 1 -- 172.21.15.5:0/508273520 >> v1:172.21.15.80:6789/0 conn(0x7f78d815d340 legacy=0x7f78d80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d6a00640 1 -- 172.21.15.5:0/508273520 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d806d900 con 0x7f78d806e640 2024-04-25T10:13:05.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.902+0000 7f78d7400640 1 --2- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.904+0000 7f78d6a00640 1 --2- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d815ce00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78c40097a0 tx=0x7f78c402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48d25b66b1c10a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.904+0000 7f78d6000640 1 -- 172.21.15.5:0/508273520 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4037070 con 0x7f78d806e640 2024-04-25T10:13:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.904+0000 7f78d6000640 1 -- 172.21.15.5:0/508273520 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4032440 con 0x7f78d806e640 2024-04-25T10:13:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.904+0000 7f78d6000640 1 -- 172.21.15.5:0/508273520 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4032810 con 0x7f78d806e640 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 -- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 msgr2=0x7f78d815ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 --2- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d815ce00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78c40097a0 tx=0x7f78c402a180 comp rx=0 tx=0).stop 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 -- 172.21.15.5:0/508273520 shutdown_connections 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 --2- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d815ce00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 --2- 172.21.15.5:0/508273520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d806e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 -- 172.21.15.5:0/508273520 >> 172.21.15.5:0/508273520 conn(0x7f78d8066f20 msgr2=0x7f78d806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 -- 172.21.15.5:0/508273520 shutdown_connections 2024-04-25T10:13:05.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.905+0000 7f78dd200640 1 -- 172.21.15.5:0/508273520 wait complete. 2024-04-25T10:13:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.907+0000 7f78dd200640 1 Processor -- start 2024-04-25T10:13:05.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.907+0000 7f78dd200640 1 -- start start 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.908+0000 7f78dd200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.908+0000 7f78dd200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d8155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.908+0000 7f78dd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 0x7f78d80ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.908+0000 7f78dd200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80cdb90 con 0x7f78d815d340 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.908+0000 7f78dd200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80cda10 con 0x7f78d806e640 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.908+0000 7f78dd200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8071000 con 0x7f78d8141da0 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 0x7f78d80ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 0x7f78d80ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49248/0 (socket says 172.21.15.5:49248) 2024-04-25T10:13:05.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 -- 172.21.15.5:0/2763275537 learned_addr learned my addr 172.21.15.5:0/2763275537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d6a00640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d8155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7400640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 -- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 msgr2=0x7f78d8155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d8155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 -- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 msgr2=0x7f78d80b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.909+0000 7f78d7e00640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d80b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78d7e00640 1 -- 172.21.15.5:0/2763275537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80b1ee0 con 0x7f78d815d340 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78d7400640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d80b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78d7e00640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 0x7f78d80ae030 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f78d000cab0 tx=0x7f78d000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0017070 con 0x7f78d815d340 2024-04-25T10:13:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d000ac30 con 0x7f78d815d340 2024-04-25T10:13:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d80b1360 con 0x7f78d815d340 2024-04-25T10:13:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.910+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0012400 con 0x7f78d815d340 2024-04-25T10:13:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.911+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d80b1920 con 0x7f78d815d340 2024-04-25T10:13:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.912+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8068f60 con 0x7f78d815d340 2024-04-25T10:13:05.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.913+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78d001a070 con 0x7f78d815d340 2024-04-25T10:13:05.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.913+0000 7f78cf400640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78b806b520 0x7f78b806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.914+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78d0089fa0 con 0x7f78d815d340 2024-04-25T10:13:05.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.914+0000 7f78d7400640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78b806b520 0x7f78b806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:05.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.915+0000 7f78d7400640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78b806b520 0x7f78b806d9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009ac0 tx=0x7f78c8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:05.918+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78d0058620 con 0x7f78d815d340 2024-04-25T10:13:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.159+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78d80b3aa0 con 0x7f78d815d340 2024-04-25T10:13:06.165 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:13:06.160+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.164+0000 7f78cf400640 1 -- 172.21.15.5:0/2763275537 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v99) v1 ==== 161+0+0 (secure 0 0 0) 0x7f78d005c6c0 con 0x7f78d815d340 2024-04-25T10:13:06.169 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78b806b520 msgr2=0x7f78b806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78b806b520 0x7f78b806d9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009ac0 tx=0x7f78c8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 msgr2=0x7f78d80ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 0x7f78d80ae030 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f78d000cab0 tx=0x7f78d000cf70 comp rx=0 tx=0).stop 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 shutdown_connections 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78d815d340 0x7f78d80ae030 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78d8141da0 0x7f78d8155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f78b806b520 0x7f78b806d9e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 --2- 172.21.15.5:0/2763275537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78d806e640 0x7f78d80b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.167+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 >> 172.21.15.5:0/2763275537 conn(0x7f78d8066f20 msgr2=0x7f78d80ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.168+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 shutdown_connections 2024-04-25T10:13:06.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.168+0000 7f78dd200640 1 -- 172.21.15.5:0/2763275537 wait complete. 2024-04-25T10:13:06.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:06.198 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:06.212 INFO:tasks.ceph.mds.b.smithi080.stderr: -8313> 2024-04-25T10:12:52.447+0000 7fa6e3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:06.289 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-04-25T10:13:06.327 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T10:13:06.328 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:06.329 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T10:13:06.330 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on skipped message = False 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:06.331 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.332 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T10:13:06.333 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.334 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:06.335 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T10:13:06.336 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T10:13:06.337 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = True 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = False 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.338 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-04-25T10:13:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-04-25T10:13:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-04-25T10:13:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-04-25T10:13:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-04-25T10:13:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = False 2024-04-25T10:13:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:06.376 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:06.500 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T10:13:06.500 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T10:13:06.500 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:06.500 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T10:13:06.500 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:06.501 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:06.502 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T10:13:06.503 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on skipped message = False 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T10:13:06.504 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T10:13:06.506 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:06.507 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:06.508 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.509 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-04-25T10:13:06.510 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:06.511 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-04-25T10:13:06.512 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = False 2024-04-25T10:13:06.512 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-04-25T10:13:06.512 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = False 2024-04-25T10:13:06.512 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:06.512 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.512 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.516 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-04-25T10:13:06.517 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:06.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-04-25T10:13:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.521 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:06.522 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-04-25T10:13:06.523 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:06.524 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-04-25T10:13:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-04-25T10:13:06.526 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = False 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = False 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:06.527 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:06.554 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T10:13:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.668+0000 7fa20a1b7f40 1 Processor -- start 2024-04-25T10:13:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.668+0000 7fa20a1b7f40 1 -- start start 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa20a1b7f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1ec003630 0x55f3bc12d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa20a1b7f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 0x55f3bc149b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa20a1b7f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f3bc14a080 0x55f3bc14a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa20a1b7f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3bc01bdd0 con 0x55f3bc14eaf0 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa20a1b7f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3bc0c1380 con 0x55f3bc14a080 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa20a1b7f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3bc14b4e0 con 0x7fa1ec003630 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa206a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 0x55f3bc149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa206a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 0x55f3bc149b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49254/0 (socket says 172.21.15.5:49254) 2024-04-25T10:13:06.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa206a00640 1 -- 172.21.15.5:0/3492762227 learned_addr learned my addr 172.21.15.5:0/3492762227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa207400640 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1ec003630 0x55f3bc12d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa207e00640 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f3bc14a080 0x55f3bc14a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.669+0000 7fa206a00640 1 -- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1ec003630 msgr2=0x55f3bc12d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa206a00640 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1ec003630 0x55f3bc12d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa206a00640 1 -- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f3bc14a080 msgr2=0x55f3bc14a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa206a00640 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f3bc14a080 0x55f3bc14a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa206a00640 1 -- 172.21.15.5:0/3492762227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3bc14abf0 con 0x55f3bc14eaf0 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa207e00640 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f3bc14a080 0x55f3bc14a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa206a00640 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 0x55f3bc149b40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009870 tx=0x7fa1f4009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385c06e0621f9b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa205600640 1 -- 172.21.15.5:0/3492762227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4008750 con 0x55f3bc14eaf0 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa205600640 1 -- 172.21.15.5:0/3492762227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f40088f0 con 0x55f3bc14eaf0 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.670+0000 7fa205600640 1 -- 172.21.15.5:0/3492762227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4008ce0 con 0x55f3bc14eaf0 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/3492762227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 msgr2=0x55f3bc149b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 0x55f3bc149b40 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009870 tx=0x7fa1f4009960 comp rx=0 tx=0).stop 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/3492762227 shutdown_connections 2024-04-25T10:13:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55f3bc14a080 0x55f3bc14a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14eaf0 0x55f3bc149b40 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/3492762227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1ec003630 0x55f3bc12d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/3492762227 >> 172.21.15.5:0/3492762227 conn(0x55f3bc073c60 msgr2=0x55f3bc074070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/3492762227 shutdown_connections 2024-04-25T10:13:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.671+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/3492762227 wait complete. 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.672+0000 7fa20a1b7f40 1 Processor -- start 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.672+0000 7fa20a1b7f40 1 -- start start 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa20a1b7f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14a080 0x55f3bc147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa20a1b7f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f3bc14eaf0 0x55f3bc14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa20a1b7f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec003630 0x55f3bc12c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa20a1b7f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3bc073470 con 0x55f3bc14a080 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa20a1b7f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3bc14e920 con 0x7fa1ec003630 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa20a1b7f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3bc14e640 con 0x55f3bc14eaf0 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa207400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14a080 0x55f3bc147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa207400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14a080 0x55f3bc147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49260/0 (socket says 172.21.15.5:49260) 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.673+0000 7fa207400640 1 -- 172.21.15.5:0/197761957 learned_addr learned my addr 172.21.15.5:0/197761957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa206a00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f3bc14eaf0 0x55f3bc14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207e00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec003630 0x55f3bc12c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207400640 1 -- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f3bc14eaf0 msgr2=0x55f3bc14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207400640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55f3bc14eaf0 0x55f3bc14d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207400640 1 -- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec003630 msgr2=0x55f3bc12c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207400640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec003630 0x55f3bc12c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207400640 1 -- 172.21.15.5:0/197761957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3bc12d050 con 0x55f3bc14a080 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.674+0000 7fa207400640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14a080 0x55f3bc147ea0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800f520 tx=0x7fa1f800f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.675+0000 7fa1ffe00640 1 -- 172.21.15.5:0/197761957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8019070 con 0x55f3bc14a080 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.675+0000 7fa1ffe00640 1 -- 172.21.15.5:0/197761957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8014880 con 0x55f3bc14a080 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.675+0000 7fa1ffe00640 1 -- 172.21.15.5:0/197761957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800e040 con 0x55f3bc14a080 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.675+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f3bc142230 con 0x55f3bc14a080 2024-04-25T10:13:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.675+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3bc143c00 con 0x55f3bc14a080 2024-04-25T10:13:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.677+0000 7fa1ffe00640 1 -- 172.21.15.5:0/197761957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1f801c070 con 0x55f3bc14a080 2024-04-25T10:13:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.678+0000 7fa1ffe00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1e8069eb0 0x7fa1e806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.678+0000 7fa1ffe00640 1 -- 172.21.15.5:0/197761957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa1f8089750 con 0x55f3bc14a080 2024-04-25T10:13:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.678+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f3bc14db70 0x55f3bc147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.678+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) v9 -- 0x55f3bc1477c0 con 0x55f3bc14db70 2024-04-25T10:13:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.679+0000 7fa206a00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1e8069eb0 0x7fa1e806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.679+0000 7fa207e00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f3bc14db70 0x55f3bc147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.680+0000 7fa206a00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1e8069eb0 0x7fa1e806c370 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55f3bc0a1740 tx=0x7fa1f4009f90 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.680+0000 7fa207e00640 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f3bc14db70 0x55f3bc147280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.688+0000 7fa207e00640 1 -- 172.21.15.5:0/197761957 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v121'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa200001430 con 0x55f3bc14db70 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.688+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f3bc14db70 msgr2=0x55f3bc147280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.688+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55f3bc14db70 0x55f3bc147280 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1e8069eb0 msgr2=0x7fa1e806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fa1e8069eb0 0x7fa1e806c370 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55f3bc0a1740 tx=0x7fa1f4009f90 comp rx=0 tx=0).stop 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14a080 msgr2=0x55f3bc147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 --2- 172.21.15.5:0/197761957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3bc14a080 0x55f3bc147ea0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800f520 tx=0x7fa1f800f9e0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa207e00640 1 -- 172.21.15.5:0/197761957 reap_dead start 2024-04-25T10:13:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 shutdown_connections 2024-04-25T10:13:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 >> 172.21.15.5:0/197761957 conn(0x55f3bc073c60 msgr2=0x55f3bc0a5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 shutdown_connections 2024-04-25T10:13:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.689+0000 7fa20a1b7f40 1 -- 172.21.15.5:0/197761957 wait complete. 2024-04-25T10:13:06.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:13:06.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.915+0000 7f728fe00640 1 Processor -- start 2024-04-25T10:13:06.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.915+0000 7f728fe00640 1 -- start start 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f728fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7290161d90 0x7f72901689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f728fe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72901606e0 con 0x7f72900d3f90 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f728fe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72901609c0 con 0x7f72900b48c0 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f728fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7290160ca0 con 0x7f7290161d90 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f728f400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7290161d90 0x7f72901689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f7287e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f72900d3f90 0x7f7290161680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51238/0 (socket says 172.21.15.5:51238) 2024-04-25T10:13:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.917+0000 7f7287e00640 1 -- 172.21.15.5:0/3150791978 learned_addr learned my addr 172.21.15.5:0/3150791978 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.918+0000 7f728f400640 1 -- 172.21.15.5:0/3150791978 >> v1:172.21.15.80:6789/0 conn(0x7f72900b48c0 legacy=0x7f729015fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.918+0000 7f728f400640 1 -- 172.21.15.5:0/3150791978 >> v1:172.21.15.5:6789/0 conn(0x7f72900d3f90 legacy=0x7f7290161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.918+0000 7f728f400640 1 -- 172.21.15.5:0/3150791978 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7290160f80 con 0x7f7290161d90 2024-04-25T10:13:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.919+0000 7f728f400640 1 --2- 172.21.15.5:0/3150791978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7290161d90 0x7f72901689b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f728802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79129f6086f237d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.919+0000 7f728e000640 1 -- 172.21.15.5:0/3150791978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288031600 con 0x7f7290161d90 2024-04-25T10:13:06.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.919+0000 7f728e000640 1 -- 172.21.15.5:0/3150791978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72880317a0 con 0x7f7290161d90 2024-04-25T10:13:06.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.919+0000 7f728e000640 1 -- 172.21.15.5:0/3150791978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288031b70 con 0x7f7290161d90 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 -- 172.21.15.5:0/3150791978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7290161d90 msgr2=0x7f72901689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 --2- 172.21.15.5:0/3150791978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7290161d90 0x7f72901689b0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f7288009d90 tx=0x7f728802a610 comp rx=0 tx=0).stop 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 -- 172.21.15.5:0/3150791978 shutdown_connections 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 --2- 172.21.15.5:0/3150791978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7290161d90 0x7f72901689b0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 -- 172.21.15.5:0/3150791978 >> 172.21.15.5:0/3150791978 conn(0x7f729008af80 msgr2=0x7f72900d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 -- 172.21.15.5:0/3150791978 shutdown_connections 2024-04-25T10:13:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.920+0000 7f728fe00640 1 -- 172.21.15.5:0/3150791978 wait complete. 2024-04-25T10:13:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.922+0000 7f728fe00640 1 Processor -- start 2024-04-25T10:13:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.922+0000 7f728fe00640 1 -- start start 2024-04-25T10:13:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 0x7f72900b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72900d3f90 0x7f729015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7290161d90 0x7f72900c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b3b80 con 0x7f72900b48c0 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b4ca0 con 0x7f72900d3f90 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72900b3d00 con 0x7f7290161d90 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 0x7f72900b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 0x7f72900b61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55748/0 (socket says 172.21.15.5:55748) 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 -- 172.21.15.5:0/895455132 learned_addr learned my addr 172.21.15.5:0/895455132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f7287e00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72900d3f90 0x7f729015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728f400640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7290161d90 0x7f72900c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 -- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7290161d90 msgr2=0x7f72900c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7290161d90 0x7f72900c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 -- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72900d3f90 msgr2=0x7f729015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72900d3f90 0x7f729015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.923+0000 7f728ea00640 1 -- 172.21.15.5:0/895455132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900cbb50 con 0x7f72900b48c0 2024-04-25T10:13:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.924+0000 7f728ea00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 0x7f72900b61e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f728000b3f0 tx=0x7f728000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.924+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280015020 con 0x7f72900b48c0 2024-04-25T10:13:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.924+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7280011830 con 0x7f72900b48c0 2024-04-25T10:13:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.924+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280011c20 con 0x7f72900b48c0 2024-04-25T10:13:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.924+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72900cbdf0 con 0x7f72900b48c0 2024-04-25T10:13:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.924+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72900cc3b0 con 0x7f72900b48c0 2024-04-25T10:13:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.925+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72900cc5b0 con 0x7f72900b48c0 2024-04-25T10:13:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.926+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7280018020 con 0x7f72900b48c0 2024-04-25T10:13:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.927+0000 7f7287400640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f726c06b4d0 0x7f726c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.928+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f728008a080 con 0x7f72900b48c0 2024-04-25T10:13:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.928+0000 7f7287e00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f726c06b4d0 0x7f726c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.929+0000 7f7287e00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f726c06b4d0 0x7f726c06d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f72900d0210 tx=0x7f7278009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:06.932+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72900cc5b0 con 0x7f72900b48c0 2024-04-25T10:13:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:07.176+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f7290068f60 con 0x7f72900b48c0 2024-04-25T10:13:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.187+0000 7f7287400640 1 -- 172.21.15.5:0/895455132 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) v1 ==== 162+0+0 (secure 0 0 0) 0x7f728001f030 con 0x7f72900b48c0 2024-04-25T10:13:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:13:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f726c06b4d0 msgr2=0x7f726c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f726c06b4d0 0x7f726c06d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f72900d0210 tx=0x7f7278009040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 msgr2=0x7f72900b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 0x7f72900b61e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f728000b3f0 tx=0x7f728000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 shutdown_connections 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7290161d90 0x7f72900c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f726c06b4d0 0x7f726c06d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72900d3f90 0x7f729015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 --2- 172.21.15.5:0/895455132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72900b48c0 0x7f72900b61e0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 >> 172.21.15.5:0/895455132 conn(0x7f729008af80 msgr2=0x7f72900b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 shutdown_connections 2024-04-25T10:13:08.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.191+0000 7f728fe00640 1 -- 172.21.15.5:0/895455132 wait complete. 2024-04-25T10:13:08.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:08.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.406+0000 7f82fcc00640 1 Processor -- start 2024-04-25T10:13:08.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.406+0000 7f82fcc00640 1 -- start start 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.407+0000 7f82fcc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f82f8154a10 0x7f82f815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.407+0000 7f82fcc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f82f80cdca0 0x7f82f80b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82fcc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f80b4890 con 0x7f82f8070f90 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82fcc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f80cd200 con 0x7f82f8154a10 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82fcc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f806e6f0 con 0x7f82f80cdca0 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82f7e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f82f8070f90 0x7f82f80713a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51252/0 (socket says 172.21.15.5:51252) 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82f7e00640 1 -- 172.21.15.5:0/39982933 learned_addr learned my addr 172.21.15.5:0/39982933 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:08.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82f7400640 1 --2- 172.21.15.5:0/39982933 >> v2:172.21.15.5:3300/0 conn(0x7f82f8154a10 0x7f82f815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.408+0000 7f82f6a00640 1 --2- 172.21.15.5:0/39982933 >> v2:172.21.15.80:3300/0 conn(0x7f82f80cdca0 0x7f82f80b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.409+0000 7f82f7400640 1 -- 172.21.15.5:0/39982933 >> v2:172.21.15.80:3300/0 conn(0x7f82f80cdca0 msgr2=0x7f82f80b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.409+0000 7f82f7400640 1 --2- 172.21.15.5:0/39982933 >> v2:172.21.15.80:3300/0 conn(0x7f82f80cdca0 0x7f82f80b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.409+0000 7f82f7400640 1 -- 172.21.15.5:0/39982933 >> v1:172.21.15.5:6789/0 conn(0x7f82f8070f90 legacy=0x7f82f80713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.409+0000 7f82f7400640 1 -- 172.21.15.5:0/39982933 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f806e9d0 con 0x7f82f8154a10 2024-04-25T10:13:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.410+0000 7f82f7400640 1 --2- 172.21.15.5:0/39982933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8154a10 0x7f82f815e8c0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f82f80aef00 tx=0x7f82e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea635cd2193c0d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.410+0000 7f82f6000640 1 -- 172.21.15.5:0/39982933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8037070 con 0x7f82f8154a10 2024-04-25T10:13:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.410+0000 7f82f6000640 1 -- 172.21.15.5:0/39982933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8032440 con 0x7f82f8154a10 2024-04-25T10:13:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.410+0000 7f82f6000640 1 -- 172.21.15.5:0/39982933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e802ae60 con 0x7f82f8154a10 2024-04-25T10:13:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 -- 172.21.15.5:0/39982933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8154a10 msgr2=0x7f82f815e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 --2- 172.21.15.5:0/39982933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8154a10 0x7f82f815e8c0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f82f80aef00 tx=0x7f82e802a110 comp rx=0 tx=0).stop 2024-04-25T10:13:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 -- 172.21.15.5:0/39982933 shutdown_connections 2024-04-25T10:13:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 --2- 172.21.15.5:0/39982933 >> v2:172.21.15.80:3300/0 conn(0x7f82f80cdca0 0x7f82f80b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 --2- 172.21.15.5:0/39982933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f8154a10 0x7f82f815e8c0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 -- 172.21.15.5:0/39982933 >> 172.21.15.5:0/39982933 conn(0x7f82f8066f20 msgr2=0x7f82f806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 -- 172.21.15.5:0/39982933 shutdown_connections 2024-04-25T10:13:08.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.411+0000 7f82fcc00640 1 -- 172.21.15.5:0/39982933 wait complete. 2024-04-25T10:13:08.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.413+0000 7f82fcc00640 1 Processor -- start 2024-04-25T10:13:08.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.414+0000 7f82fcc00640 1 -- start start 2024-04-25T10:13:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82fcc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82f8070f90 0x7f82f80b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82fcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 0x7f82f815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82fcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82f8154a10 0x7f82f815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82fcc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80cd780 con 0x7f82f80cdca0 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82fcc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80cd4a0 con 0x7f82f8070f90 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82fcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f806cff0 con 0x7f82f8154a10 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82f6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 0x7f82f815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82f6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 0x7f82f815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55768/0 (socket says 172.21.15.5:55768) 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82f6a00640 1 -- 172.21.15.5:0/3088372465 learned_addr learned my addr 172.21.15.5:0/3088372465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.415+0000 7f82f7400640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82f8070f90 0x7f82f80b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f7e00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82f8154a10 0x7f82f815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f6a00640 1 -- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82f8154a10 msgr2=0x7f82f815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f6a00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82f8154a10 0x7f82f815fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f6a00640 1 -- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82f8070f90 msgr2=0x7f82f80b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f6a00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82f8070f90 0x7f82f80b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f6a00640 1 -- 172.21.15.5:0/3088372465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f814fc60 con 0x7f82f80cdca0 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82f6a00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 0x7f82f815f5a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f82e400d800 tx=0x7f82e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e400b6c0 con 0x7f82f80cdca0 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4019070 con 0x7f82f80cdca0 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e400ba80 con 0x7f82f80cdca0 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.416+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f80aef90 con 0x7f82f80cdca0 2024-04-25T10:13:08.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.417+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f80af550 con 0x7f82f80cdca0 2024-04-25T10:13:08.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.418+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f80cce40 con 0x7f82f80cdca0 2024-04-25T10:13:08.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.419+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82e400bc20 con 0x7f82f80cdca0 2024-04-25T10:13:08.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.420+0000 7f82ef400640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82d806b4d0 0x7f82d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.420+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f82e408a890 con 0x7f82f80cdca0 2024-04-25T10:13:08.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.420+0000 7f82f7400640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82d806b4d0 0x7f82d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.421+0000 7f82f7400640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82d806b4d0 0x7f82d806d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f82e8034000 tx=0x7f82e802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.424+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82e4059530 con 0x7f82f80cdca0 2024-04-25T10:13:08.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.669+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82f8068670 con 0x7f82f80cdca0 2024-04-25T10:13:08.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.670+0000 7f82ef400640 1 -- 172.21.15.5:0/3088372465 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f82e4058e30 con 0x7f82f80cdca0 2024-04-25T10:13:08.676 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:08.676 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:08.193182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"c","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6835/2518129896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2518129896},{"type":"v1","addr":"172.21.15.5:6835","nonce":2518129896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:08.676 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 102 2024-04-25T10:13:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82d806b4d0 msgr2=0x7f82d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82d806b4d0 0x7f82d806d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f82e8034000 tx=0x7f82e802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 msgr2=0x7f82f815f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 0x7f82f815f5a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f82e400d800 tx=0x7f82e400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 shutdown_connections 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f82f8154a10 0x7f82f815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82f80cdca0 0x7f82f815f5a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f82d806b4d0 0x7f82d806d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 --2- 172.21.15.5:0/3088372465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82f8070f90 0x7f82f80b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 >> 172.21.15.5:0/3088372465 conn(0x7f82f8066f20 msgr2=0x7f82f80b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 shutdown_connections 2024-04-25T10:13:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:08.673+0000 7f82fcc00640 1 -- 172.21.15.5:0/3088372465 wait complete. 2024-04-25T10:13:09.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:13:08.193182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7000}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7000': {'gid': 7000, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6835/2518129896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 2518129896}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 2518129896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:09.102 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7000:up:replay 2024-04-25T10:13:10.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.320+0000 7f68cbc00640 1 Processor -- start 2024-04-25T10:13:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.320+0000 7f68cbc00640 1 -- start start 2024-04-25T10:13:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68cbc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f68c40b16e0 0x7f68c40ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68cbc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c40cd240 0x7f68c41622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68cbc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c4165af0 con 0x7f68c415c380 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68cbc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c415c760 con 0x7f68c40b16e0 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68cbc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41658a0 con 0x7f68c40cd240 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68c9200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f68c415c380 0x7f68c406e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51258/0 (socket says 172.21.15.5:51258) 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.322+0000 7f68c9200640 1 -- 172.21.15.5:0/3427566967 learned_addr learned my addr 172.21.15.5:0/3427566967 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c3e00640 1 --2- 172.21.15.5:0/3427566967 >> v2:172.21.15.5:3300/0 conn(0x7f68c40b16e0 0x7f68c40ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c9c00640 1 --2- 172.21.15.5:0/3427566967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c40cd240 0x7f68c41622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c3400640 1 -- 172.21.15.5:0/3427566967 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557788516 0 0) 0x7f68c4165af0 con 0x7f68c415c380 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c3e00640 1 -- 172.21.15.5:0/3427566967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c40cd240 msgr2=0x7f68c41622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c3e00640 1 --2- 172.21.15.5:0/3427566967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c40cd240 0x7f68c41622b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c3e00640 1 -- 172.21.15.5:0/3427566967 >> v1:172.21.15.5:6789/0 conn(0x7f68c415c380 legacy=0x7f68c406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.323+0000 7f68c3e00640 1 -- 172.21.15.5:0/3427566967 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4165dd0 con 0x7f68c40b16e0 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.324+0000 7f68c3e00640 1 --2- 172.21.15.5:0/3427566967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c40ccd00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f68b00097a0 tx=0x7f68b002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e886bdcdaa16c68d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.324+0000 7f68c3400640 1 -- 172.21.15.5:0/3427566967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0037070 con 0x7f68c40b16e0 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.324+0000 7f68c3400640 1 -- 172.21.15.5:0/3427566967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b002adb0 con 0x7f68c40b16e0 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.324+0000 7f68c3400640 1 -- 172.21.15.5:0/3427566967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0032660 con 0x7f68c40b16e0 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 -- 172.21.15.5:0/3427566967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 msgr2=0x7f68c40ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 --2- 172.21.15.5:0/3427566967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c40ccd00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f68b00097a0 tx=0x7f68b002a060 comp rx=0 tx=0).stop 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 -- 172.21.15.5:0/3427566967 shutdown_connections 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 --2- 172.21.15.5:0/3427566967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c40cd240 0x7f68c41622b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 --2- 172.21.15.5:0/3427566967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c40ccd00 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 -- 172.21.15.5:0/3427566967 >> 172.21.15.5:0/3427566967 conn(0x7f68c4071150 msgr2=0x7f68c4071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 -- 172.21.15.5:0/3427566967 shutdown_connections 2024-04-25T10:13:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.325+0000 7f68cbc00640 1 -- 172.21.15.5:0/3427566967 wait complete. 2024-04-25T10:13:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.327+0000 7f68cbc00640 1 Processor -- start 2024-04-25T10:13:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.327+0000 7f68cbc00640 1 -- start start 2024-04-25T10:13:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.328+0000 7f68cbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.328+0000 7f68cbc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68c40cd240 0x7f68c415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.328+0000 7f68cbc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c415c380 0x7f68c41510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.328+0000 7f68cbc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41627f0 con 0x7f68c40b16e0 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.328+0000 7f68cbc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4166390 con 0x7f68c40cd240 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.328+0000 7f68cbc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4162970 con 0x7f68c415c380 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55788/0 (socket says 172.21.15.5:55788) 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 -- 172.21.15.5:0/1731507458 learned_addr learned my addr 172.21.15.5:0/1731507458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c3e00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68c40cd240 0x7f68c415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9c00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c415c380 0x7f68c41510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 -- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c415c380 msgr2=0x7f68c41510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c415c380 0x7f68c41510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 -- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68c40cd240 msgr2=0x7f68c415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68c40cd240 0x7f68c415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.329+0000 7f68c9200640 1 -- 172.21.15.5:0/1731507458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c40b2730 con 0x7f68c40b16e0 2024-04-25T10:13:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.330+0000 7f68c9200640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c415f790 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c460 tx=0x7f68b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.330+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8018070 con 0x7f68c40b16e0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.330+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b8007500 con 0x7f68c40b16e0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.330+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004a20 con 0x7f68c40b16e0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.330+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c40b2a10 con 0x7f68c40b16e0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.330+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c40b2fd0 con 0x7f68c40b16e0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.331+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c40723b0 con 0x7f68c40b16e0 2024-04-25T10:13:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.332+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68b801b070 con 0x7f68c40b16e0 2024-04-25T10:13:10.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.334+0000 7f68c0c00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f68a406b500 0x7f68a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.334+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f68b8089760 con 0x7f68c40b16e0 2024-04-25T10:13:10.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.334+0000 7f68c3e00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f68a406b500 0x7f68a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.336+0000 7f68c3e00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f68a406b500 0x7f68a406d9c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f68b0034000 tx=0x7f68b002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.339+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68c40723b0 con 0x7f68c40b16e0 2024-04-25T10:13:10.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.580+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68c4068fd0 con 0x7f68c40b16e0 2024-04-25T10:13:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.582+0000 7f68c0c00640 1 -- 172.21.15.5:0/1731507458 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f68b80582d0 con 0x7f68c40b16e0 2024-04-25T10:13:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:10.586 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:09.207244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"c","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.5:6835/2518129896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2518129896},{"type":"v1","addr":"172.21.15.5:6835","nonce":2518129896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:10.587 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 104 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.584+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f68a406b500 msgr2=0x7f68a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.584+0000 7f68cbc00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f68a406b500 0x7f68a406d9c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f68b0034000 tx=0x7f68b002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 msgr2=0x7f68c415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c415f790 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f68b800c460 tx=0x7f68b800c920 comp rx=0 tx=0).stop 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 shutdown_connections 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68c415c380 0x7f68c41510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f68a406b500 0x7f68a406d9c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68c40cd240 0x7f68c415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 --2- 172.21.15.5:0/1731507458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c40b16e0 0x7f68c415f790 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 >> 172.21.15.5:0/1731507458 conn(0x7f68c4071150 msgr2=0x7f68c41615e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 shutdown_connections 2024-04-25T10:13:10.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:10.585+0000 7f68cbc00640 1 -- 172.21.15.5:0/1731507458 wait complete. 2024-04-25T10:13:11.013 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:13:09.207244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7000}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7000': {'gid': 7000, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.5:6835/2518129896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 2518129896}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 2518129896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:11.013 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7000:up:rejoin 2024-04-25T10:13:12.014 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.231+0000 7fe618800640 1 Processor -- start 2024-04-25T10:13:12.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.231+0000 7fe618800640 1 -- start start 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe618800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe61009a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe618800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe61006bc70 0x7fe6100d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe618800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6100d10b0 0x7fe61006c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe618800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6100d06e0 con 0x7fe6100d10b0 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe618800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61009a940 con 0x7fe61006bc70 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe618800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100cfad0 con 0x7fe61015da20 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe616800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6100d10b0 0x7fe61006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe616800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe6100d10b0 0x7fe61006c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55794/0 (socket says 172.21.15.5:55794) 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe616800640 1 -- 172.21.15.5:0/3211374619 learned_addr learned my addr 172.21.15.5:0/3211374619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:12.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.233+0000 7fe615e00640 1 --2- 172.21.15.5:0/3211374619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe61009a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.234+0000 7fe615400640 1 --2- 172.21.15.5:0/3211374619 >> v2:172.21.15.80:3300/0 conn(0x7fe61006bc70 0x7fe6100d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.234+0000 7fe616800640 1 -- 172.21.15.5:0/3211374619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 msgr2=0x7fe61009a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.234+0000 7fe616800640 1 --2- 172.21.15.5:0/3211374619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe61009a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.234+0000 7fe616800640 1 -- 172.21.15.5:0/3211374619 >> v2:172.21.15.80:3300/0 conn(0x7fe61006bc70 msgr2=0x7fe6100d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.234+0000 7fe616800640 1 --2- 172.21.15.5:0/3211374619 >> v2:172.21.15.80:3300/0 conn(0x7fe61006bc70 0x7fe6100d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.234+0000 7fe616800640 1 -- 172.21.15.5:0/3211374619 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6100cfdb0 con 0x7fe6100d10b0 2024-04-25T10:13:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.235+0000 7fe616800640 1 --2- 172.21.15.5:0/3211374619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe61006c330 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fe610072a00 tx=0x7fe60c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efe45c51335a3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.235+0000 7fe614a00640 1 -- 172.21.15.5:0/3211374619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c036070 con 0x7fe6100d10b0 2024-04-25T10:13:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.235+0000 7fe614a00640 1 -- 172.21.15.5:0/3211374619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c031600 con 0x7fe6100d10b0 2024-04-25T10:13:12.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.235+0000 7fe614a00640 1 -- 172.21.15.5:0/3211374619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0319d0 con 0x7fe6100d10b0 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 -- 172.21.15.5:0/3211374619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 msgr2=0x7fe61006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 --2- 172.21.15.5:0/3211374619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe61006c330 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fe610072a00 tx=0x7fe60c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 -- 172.21.15.5:0/3211374619 shutdown_connections 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 --2- 172.21.15.5:0/3211374619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe61006c330 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 --2- 172.21.15.5:0/3211374619 >> v2:172.21.15.80:3300/0 conn(0x7fe61006bc70 0x7fe6100d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 --2- 172.21.15.5:0/3211374619 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe61009a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 -- 172.21.15.5:0/3211374619 >> 172.21.15.5:0/3211374619 conn(0x7fe6100d16f0 msgr2=0x7fe6100d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 -- 172.21.15.5:0/3211374619 shutdown_connections 2024-04-25T10:13:12.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.236+0000 7fe618800640 1 -- 172.21.15.5:0/3211374619 wait complete. 2024-04-25T10:13:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.238+0000 7fe618800640 1 Processor -- start 2024-04-25T10:13:12.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.238+0000 7fe618800640 1 -- start start 2024-04-25T10:13:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe618800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe61006bc70 0x7fe61015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe618800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe6100cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe618800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe6100cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe618800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61015de00 con 0x7fe6100d10b0 2024-04-25T10:13:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe618800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6100d0370 con 0x7fe61006bc70 2024-04-25T10:13:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe618800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61015df80 con 0x7fe61015da20 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe615400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe6100cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.239+0000 7fe615400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe6100cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55806/0 (socket says 172.21.15.5:55806) 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 -- 172.21.15.5:0/1264356599 learned_addr learned my addr 172.21.15.5:0/1264356599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 -- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 msgr2=0x7fe6100cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe6100cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 -- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe61006bc70 msgr2=0x7fe61015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe61006bc70 0x7fe61015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 -- 172.21.15.5:0/1264356599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61015b2d0 con 0x7fe6100d10b0 2024-04-25T10:13:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe615400640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe6100cabd0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fe60000d5f0 tx=0x7fe60000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6000125f0 con 0x7fe6100d10b0 2024-04-25T10:13:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.240+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe600019070 con 0x7fe6100d10b0 2024-04-25T10:13:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.241+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6100c9790 con 0x7fe6100d10b0 2024-04-25T10:13:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.241+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6000129d0 con 0x7fe6100d10b0 2024-04-25T10:13:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.241+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6100c9d50 con 0x7fe6100d10b0 2024-04-25T10:13:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.242+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6100d0b70 con 0x7fe6100d10b0 2024-04-25T10:13:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.243+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe600012b70 con 0x7fe6100d10b0 2024-04-25T10:13:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.244+0000 7fe60a000640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5f406b4d0 0x7fe5f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.244+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe60008a850 con 0x7fe6100d10b0 2024-04-25T10:13:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.245+0000 7fe615e00640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5f406b4d0 0x7fe5f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.246+0000 7fe615e00640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5f406b4d0 0x7fe5f406d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe604009870 tx=0x7fe604008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.248+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6100d0b70 con 0x7fe6100d10b0 2024-04-25T10:13:12.453 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:13:12.448+0000 7f038c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.488+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe610068fd0 con 0x7fe6100d10b0 2024-04-25T10:13:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.489+0000 7fe60a000640 1 -- 172.21.15.5:0/1264356599 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4941 (secure 0 0 0) 0x7fe61015b2d0 con 0x7fe6100d10b0 2024-04-25T10:13:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:12.493 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:10.211656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":10,"addr":"172.21.15.5:6835/2518129896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2518129896},{"type":"v1","addr":"172.21.15.5:6835","nonce":2518129896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 105 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.491+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5f406b4d0 msgr2=0x7fe5f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.491+0000 7fe618800640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5f406b4d0 0x7fe5f406d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe604009870 tx=0x7fe604008040 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.491+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 msgr2=0x7fe6100cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.491+0000 7fe618800640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe6100cabd0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fe60000d5f0 tx=0x7fe60000dab0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 shutdown_connections 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe61015da20 0x7fe6100cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6100d10b0 0x7fe6100cabd0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe5f406b4d0 0x7fe5f406d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 --2- 172.21.15.5:0/1264356599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe61006bc70 0x7fe61015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 >> 172.21.15.5:0/1264356599 conn(0x7fe6100d16f0 msgr2=0x7fe6100cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 shutdown_connections 2024-04-25T10:13:12.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:12.492+0000 7fe618800640 1 -- 172.21.15.5:0/1264356599 wait complete. 2024-04-25T10:13:12.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:13:10.211656+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7000}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7000': {'gid': 7000, 'name': 'c', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.5:6835/2518129896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 2518129896}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 2518129896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:12.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:12.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:13:13.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.134+0000 7f4ebbe00640 1 Processor -- start 2024-04-25T10:13:13.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.135+0000 7f4ebbe00640 1 -- start start 2024-04-25T10:13:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.136+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc161200 0x7f4ebc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.136+0000 7f4ebbe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ebc0d2380 0x7f4ebc0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.136+0000 7f4ebbe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ebc0cf410 con 0x7f4ebc0d0600 2024-04-25T10:13:13.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.136+0000 7f4ebbe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ebc0d27c0 con 0x7f4ebc0d2380 2024-04-25T10:13:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.136+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0b28b0 con 0x7f4ebc161200 2024-04-25T10:13:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4eb2000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ebc0d2380 0x7f4ebc0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4ebaa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc161200 0x7f4ebc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4eb2000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ebc0d2380 0x7f4ebc0d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55814/0 (socket says 172.21.15.5:55814) 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4eb2000640 1 -- 172.21.15.5:0/4237861760 learned_addr learned my addr 172.21.15.5:0/4237861760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4eb2000640 1 -- 172.21.15.5:0/4237861760 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc161200 msgr2=0x7f4ebc0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4eb2000640 1 --2- 172.21.15.5:0/4237861760 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc161200 0x7f4ebc0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.137+0000 7f4eb2000640 1 -- 172.21.15.5:0/4237861760 >> v1:172.21.15.80:6789/0 conn(0x7f4ebc0d0600 legacy=0x7f4ebc0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.138+0000 7f4eb2000640 1 -- 172.21.15.5:0/4237861760 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc0cf6f0 con 0x7f4ebc0d2380 2024-04-25T10:13:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.139+0000 7f4eb2000640 1 --2- 172.21.15.5:0/4237861760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0d00c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40098a0 tx=0x7f4ea402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96213e8ee8a1a7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.139+0000 7f4eba000640 1 -- 172.21.15.5:0/4237861760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4037070 con 0x7f4ebc0d2380 2024-04-25T10:13:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.139+0000 7f4eba000640 1 -- 172.21.15.5:0/4237861760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4032440 con 0x7f4ebc0d2380 2024-04-25T10:13:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.139+0000 7f4eba000640 1 -- 172.21.15.5:0/4237861760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4032810 con 0x7f4ebc0d2380 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 -- 172.21.15.5:0/4237861760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc0d2380 msgr2=0x7f4ebc0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/4237861760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0d00c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40098a0 tx=0x7f4ea402a290 comp rx=0 tx=0).stop 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 -- 172.21.15.5:0/4237861760 shutdown_connections 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/4237861760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0d00c0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/4237861760 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc161200 0x7f4ebc0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 -- 172.21.15.5:0/4237861760 >> 172.21.15.5:0/4237861760 conn(0x7f4ebc08af80 msgr2=0x7f4ebc0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 -- 172.21.15.5:0/4237861760 shutdown_connections 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.140+0000 7f4ebbe00640 1 -- 172.21.15.5:0/4237861760 wait complete. 2024-04-25T10:13:13.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.142+0000 7f4ebbe00640 1 Processor -- start 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.142+0000 7f4ebbe00640 1 -- start start 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc0d0600 0x7f4ebc153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 0x7f4ebc0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0b3f90 con 0x7f4ebc161200 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0d42b0 con 0x7f4ebc0d2380 2024-04-25T10:13:13.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebbe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0b4110 con 0x7f4ebc0d0600 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.143+0000 7f4ebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 0x7f4ebc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 0x7f4ebc0cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55826/0 (socket says 172.21.15.5:55826) 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 -- 172.21.15.5:0/1155277485 learned_addr learned my addr 172.21.15.5:0/1155277485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc0d0600 0x7f4ebc153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4eb2000640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 -- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc0d0600 msgr2=0x7f4ebc153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc0d0600 0x7f4ebc153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 -- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0d2380 msgr2=0x7f4ebc0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 -- 172.21.15.5:0/1155277485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc154170 con 0x7f4ebc161200 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.144+0000 7f4ebb400640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 0x7f4ebc0cbe80 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c3c0 tx=0x7f4eb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.145+0000 7f4eb3400640 1 -- 172.21.15.5:0/1155277485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40052d0 con 0x7f4ebc161200 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.145+0000 7f4eb3400640 1 -- 172.21.15.5:0/1155277485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb4005470 con 0x7f4ebc161200 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.145+0000 7f4eb3400640 1 -- 172.21.15.5:0/1155277485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4007240 con 0x7f4ebc161200 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.145+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ebc0c7c40 con 0x7f4ebc161200 2024-04-25T10:13:13.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.145+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ebc0c8200 con 0x7f4ebc161200 2024-04-25T10:13:13.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.147+0000 7f4eb3400640 1 -- 172.21.15.5:0/1155277485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4eb40073e0 con 0x7f4ebc161200 2024-04-25T10:13:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.148+0000 7f4eb3400640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e9806b500 0x7f4e9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.148+0000 7f4eb3400640 1 -- 172.21.15.5:0/1155277485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4eb4089750 con 0x7f4ebc161200 2024-04-25T10:13:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.149+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e9806b500 0x7f4e9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.149+0000 7f4ec4863740 1 Processor -- start 2024-04-25T10:13:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.149+0000 7f4ec4863740 1 -- start start 2024-04-25T10:13:13.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 0x55d46e21d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e1f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e23e7a0 0x55d46e1f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ec4863740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46e159620 con 0x55d46e1f5b30 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46e102ca0 con 0x55d46e23e7a0 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46e1556c0 con 0x55d46e21deb0 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e9806b500 0x7f4e9806d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0073f0 tx=0x7f4eac00b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ebaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 0x55d46e21d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ebaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 0x55d46e21d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55832/0 (socket says 172.21.15.5:55832) 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4ebaa00640 1 -- 172.21.15.5:0/3230920533 learned_addr learned my addr 172.21.15.5:0/3230920533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.150+0000 7f4eb2000640 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e1f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebb400640 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e23e7a0 0x55d46e1f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebaa00640 1 -- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 msgr2=0x55d46e1f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e1f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebaa00640 1 -- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e23e7a0 msgr2=0x55d46e1f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e23e7a0 0x55d46e1f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebaa00640 1 -- 172.21.15.5:0/3230920533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46e109f40 con 0x55d46e1f5b30 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 0x55d46e21d970 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009ea0 tx=0x7f4eac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66be41e70eb9f368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4eab400640 1 -- 172.21.15.5:0/3230920533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac01d040 con 0x55d46e1f5b30 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4eab400640 1 -- 172.21.15.5:0/3230920533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac026070 con 0x55d46e1f5b30 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.151+0000 7f4eab400640 1 -- 172.21.15.5:0/3230920533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac017980 con 0x55d46e1f5b30 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 -- 172.21.15.5:0/3230920533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 msgr2=0x55d46e21d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 0x55d46e21d970 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009ea0 tx=0x7f4eac014000 comp rx=0 tx=0).stop 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 -- 172.21.15.5:0/3230920533 shutdown_connections 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e23e7a0 0x55d46e1f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e1f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 --2- 172.21.15.5:0/3230920533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e1f5b30 0x55d46e21d970 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 -- 172.21.15.5:0/3230920533 >> 172.21.15.5:0/3230920533 conn(0x55d46e1f2d50 msgr2=0x55d46e23b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 -- 172.21.15.5:0/3230920533 shutdown_connections 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.152+0000 7f4ec4863740 1 -- 172.21.15.5:0/3230920533 wait complete. 2024-04-25T10:13:13.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.153+0000 7f4ec4863740 1 -- start start 2024-04-25T10:13:13.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.153+0000 7f4ec4863740 1 Processor -- start 2024-04-25T10:13:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e213360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e219950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46e21d4e0 con 0x55d46e23e7a0 2024-04-25T10:13:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46e0f81f0 con 0x55d46e2188e0 2024-04-25T10:13:13.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46e21ced0 con 0x55d46e21deb0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4ec4863740 10 client.0 _ll_register_callbacks cb 0x55d46e115ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.154+0000 7f4eb2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2183a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55848/0 (socket says 172.21.15.5:55848) 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 -- 172.21.15.5:0/1601032757 learned_addr learned my addr 172.21.15.5:0/1601032757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4ebb400640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e219950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 -- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 msgr2=0x55d46e213360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e213360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e213360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 -- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 msgr2=0x55d46e219950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e219950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 -- 172.21.15.5:0/1601032757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d46e117c50 con 0x55d46e23e7a0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4eb2000640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2183a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40098a0 tx=0x7f4ea4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3626b516f5dbd87e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.155+0000 7f4e9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4e9fe00640 1 -- 172.21.15.5:0/1601032757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4042070 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4e9fe00640 1 -- 172.21.15.5:0/1601032757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4ea4032d60 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eac026070 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4e9fe00640 1 -- 172.21.15.5:0/1601032757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ea404d050 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4ec4863740 10 client.9291 fetch_fsmap learned FSMap version 105 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55d46de6e2d0 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4e9fe00640 1 -- 172.21.15.5:0/1601032757 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac026070 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.156+0000 7f4e9fe00640 1 -- 172.21.15.5:0/1601032757 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4ea4050020 con 0x55d46e23e7a0 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ec4863740 10 client.9291 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ec4863740 10 client.9291 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ec4863740 20 client.9291 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ec4863740 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e24c0c0 0x55d46e24c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ec4863740 4 client.9291 mds_command: new command op to 7000 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:13.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 --> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d46e166b60 con 0x55d46e24c0c0 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.157+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e24c0c0 0x55d46e24c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.158+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e24c0c0 0x55d46e24c4a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671b611a509c1185 server_cookie=9e061b3f4db7cc7e in_seq=0 out_seq=0 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.158+0000 7f4e9fe00640 10 client.9291 ms_handle_connect on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.158+0000 7f4e9fe00640 1 -- 172.21.15.5:0/1601032757 <== mds.0 v2:172.21.15.5:6833/2518129896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d46e166b60 con 0x55d46e24c0c0 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.158+0000 7f4e9fe00640 10 client.9291 handle_command_reply: tid=0 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 client.9291 shutdown 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 msgr2=0x55d46e2183a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2183a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f4ea40098a0 tx=0x7f4ea4004730 comp rx=0 tx=0).stop 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 shutdown_connections 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e24c0c0 0x55d46e24c4a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e219950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2183a0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 --2- 172.21.15.5:0/1601032757 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e213360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4e9fe00640 0 client.9291 ms_handle_reset on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:13.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 >> 172.21.15.5:0/1601032757 conn(0x55d46e1f5b30 msgr2=0x55d46e23b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 shutdown_connections 2024-04-25T10:13:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 1 -- 172.21.15.5:0/1601032757 wait complete. 2024-04-25T10:13:13.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.159+0000 7f4ec4863740 20 client.9291 trim_cache size 0 max 16384 2024-04-25T10:13:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.177+0000 7f4ec4863740 1 Processor -- start 2024-04-25T10:13:13.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.177+0000 7f4ec4863740 1 -- start start 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.177+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e23de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 0x55d46e1f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e23e7a0 0x55d46e2192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4ec4863740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1f52b0 con 0x55d46e21deb0 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc0ca060 con 0x55d46e2188e0 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ebc1f4cb0 con 0x55d46e23e7a0 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 0x55d46e1f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 0x55d46e1f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55852/0 (socket says 172.21.15.5:55852) 2024-04-25T10:13:13.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 -- 172.21.15.5:0/4146729456 learned_addr learned my addr 172.21.15.5:0/4146729456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 -- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e23e7a0 msgr2=0x55d46e2192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e23e7a0 0x55d46e2192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 -- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 msgr2=0x55d46e23de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e23de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e23de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.178+0000 7f4eb2000640 1 -- 172.21.15.5:0/4146729456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46df386b0 con 0x55d46e21deb0 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.179+0000 7f4eb2000640 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 0x55d46e1f42f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4ea402a2d0 tx=0x7f4ea403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edf48b8cec36a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.179+0000 7f4eaaa00640 1 -- 172.21.15.5:0/4146729456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4000e60 con 0x55d46e21deb0 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.179+0000 7f4eaaa00640 1 -- 172.21.15.5:0/4146729456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea40042c0 con 0x55d46e21deb0 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.179+0000 7f4eaaa00640 1 -- 172.21.15.5:0/4146729456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4042070 con 0x55d46e21deb0 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 -- 172.21.15.5:0/4146729456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 msgr2=0x55d46e1f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 0x55d46e1f42f0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4ea402a2d0 tx=0x7f4ea403c000 comp rx=0 tx=0).stop 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 -- 172.21.15.5:0/4146729456 shutdown_connections 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e23e7a0 0x55d46e2192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e21deb0 0x55d46e1f42f0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 --2- 172.21.15.5:0/4146729456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e23de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 -- 172.21.15.5:0/4146729456 >> 172.21.15.5:0/4146729456 conn(0x55d46e1f5b30 msgr2=0x55d46e24e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 -- 172.21.15.5:0/4146729456 shutdown_connections 2024-04-25T10:13:13.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.180+0000 7f4ec4863740 1 -- 172.21.15.5:0/4146729456 wait complete. 2024-04-25T10:13:13.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.181+0000 7f4ec4863740 1 -- start start 2024-04-25T10:13:13.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.181+0000 7f4ec4863740 1 Processor -- start 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e1f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e2192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46e24cb90 con 0x55d46e23e7a0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46e214930 con 0x55d46e2188e0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d46e21a360 con 0x55d46e21deb0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ec4863740 10 client.0 _ll_register_callbacks cb 0x7f4ea4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ebb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2197f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55858/0 (socket says 172.21.15.5:55858) 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.182+0000 7f4ebb400640 1 -- 172.21.15.5:0/215739739 learned_addr learned my addr 172.21.15.5:0/215739739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e1f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4eb2000640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e2192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebb400640 1 -- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 msgr2=0x55d46e2192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebb400640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e2192b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebb400640 1 -- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 msgr2=0x55d46e1f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebb400640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e1f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebb400640 1 -- 172.21.15.5:0/215739739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d46e2a61e0 con 0x55d46e23e7a0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4eb2000640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e2192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e1f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ebb400640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2197f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40579e0 tx=0x7f4eb4007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f69aaa806eae69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4e9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4e9fe00640 1 -- 172.21.15.5:0/215739739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4087e60 con 0x55d46e23e7a0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4e9fe00640 1 -- 172.21.15.5:0/215739739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4eb4054350 con 0x55d46e23e7a0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.183+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ea40042c0 con 0x55d46e23e7a0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4e9fe00640 1 -- 172.21.15.5:0/215739739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eb40603f0 con 0x55d46e23e7a0 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 10 client.9297 fetch_fsmap learned FSMap version 105 2024-04-25T10:13:13.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55d46e2a6730 con 0x55d46e23e7a0 2024-04-25T10:13:13.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4e9fe00640 1 -- 172.21.15.5:0/215739739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea40042c0 con 0x55d46e23e7a0 2024-04-25T10:13:13.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4e9fe00640 1 -- 172.21.15.5:0/215739739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4eb4057130 con 0x55d46e23e7a0 2024-04-25T10:13:13.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 10 client.9297 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:13:13.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 10 client.9297 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 20 client.9297 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e1ef8b0 0x55d46e1efc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 4 client.9297 mds_command: new command op to 7000 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 --> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d46e1f34b0 con 0x55d46e1ef8b0 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.184+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e1ef8b0 0x55d46e1efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4ebaa00640 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e1ef8b0 0x55d46e1efc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bc745d2e0634ec server_cookie=f49a5c5e31e6dbdb in_seq=0 out_seq=0 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4e9fe00640 10 client.9297 ms_handle_connect on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4e9fe00640 1 -- 172.21.15.5:0/215739739 <== mds.0 v2:172.21.15.5:6833/2518129896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55d46e1f34b0 con 0x55d46e1ef8b0 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4e9fe00640 10 client.9297 handle_command_reply: tid=0 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4ec4863740 1 client.9297 shutdown 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 msgr2=0x55d46e2197f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4ec4863740 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2197f0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4eb40579e0 tx=0x7f4eb4007810 comp rx=0 tx=0).stop 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 shutdown_connections 2024-04-25T10:13:13.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.185+0000 7f4ec4863740 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55d46e1ef8b0 0x55d46e1efc90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46e23e7a0 0x55d46e2197f0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55d46e21deb0 0x55d46e2192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4e9fe00640 0 client.9297 ms_handle_reset on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 1 --2- 172.21.15.5:0/215739739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55d46e2188e0 0x55d46e1f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 >> 172.21.15.5:0/215739739 conn(0x55d46e1f5b30 msgr2=0x55d46e1ebfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 shutdown_connections 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 1 -- 172.21.15.5:0/215739739 wait complete. 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ec4863740 20 client.9297 trim_cache size 0 max 16384 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 7000, 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.9879272209999996, 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 105, 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 121, 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 121, 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 31.543885559 2024-04-25T10:13:13.194 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:13.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e9806b500 msgr2=0x7f4e9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e9806b500 0x7f4e9806d9c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4eac0073f0 tx=0x7f4eac00b040 comp rx=0 tx=0).stop 2024-04-25T10:13:13.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 msgr2=0x7f4ebc0cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.186+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 0x7f4ebc0cbe80 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f4eb400c3c0 tx=0x7f4eb400c880 comp rx=0 tx=0).stop 2024-04-25T10:13:13.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 shutdown_connections 2024-04-25T10:13:13.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ebc161200 0x7f4ebc0cbe80 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4e9806b500 0x7f4e9806d9c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0d2380 0x7f4ebc0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 --2- 172.21.15.5:0/1155277485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4ebc0d0600 0x7f4ebc153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 >> 172.21.15.5:0/1155277485 conn(0x7f4ebc08af80 msgr2=0x7f4ebc15e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 shutdown_connections 2024-04-25T10:13:13.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.187+0000 7f4ebbe00640 1 -- 172.21.15.5:0/1155277485 wait complete. 2024-04-25T10:13:13.210 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.405+0000 7f5716400640 1 Processor -- start 2024-04-25T10:13:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.405+0000 7f5716400640 1 -- start start 2024-04-25T10:13:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f5716400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5710161200 0x7f57100b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f5716400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57100d07d0 0x7f57100b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f5716400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100cf410 con 0x7f57100d2380 2024-04-25T10:13:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f5716400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100d27c0 con 0x7f5710161200 2024-04-25T10:13:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f5716400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100b28b0 con 0x7f57100d07d0 2024-04-25T10:13:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f570fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f57100d2380 0x7f57100d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51274/0 (socket says 172.21.15.5:51274) 2024-04-25T10:13:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f5715000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5710161200 0x7f57100b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.406+0000 7f570fe00640 1 -- 172.21.15.5:0/1311331375 learned_addr learned my addr 172.21.15.5:0/1311331375 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f5715a00640 1 --2- 172.21.15.5:0/1311331375 >> v2:172.21.15.80:3300/0 conn(0x7f57100d07d0 0x7f57100b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f570f400640 1 -- 172.21.15.5:0/1311331375 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238932626 0 0) 0x7f57100cf410 con 0x7f57100d2380 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f570f400640 1 -- 172.21.15.5:0/1311331375 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f4003520 con 0x7f57100d2380 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f5715000640 1 -- 172.21.15.5:0/1311331375 >> v2:172.21.15.80:3300/0 conn(0x7f57100d07d0 msgr2=0x7f57100b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f5715000640 1 --2- 172.21.15.5:0/1311331375 >> v2:172.21.15.80:3300/0 conn(0x7f57100d07d0 0x7f57100b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f5715000640 1 -- 172.21.15.5:0/1311331375 >> v1:172.21.15.5:6789/0 conn(0x7f57100d2380 legacy=0x7f57100d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.407+0000 7f5715000640 1 -- 172.21.15.5:0/1311331375 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100cf6f0 con 0x7f5710161200 2024-04-25T10:13:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.408+0000 7f5715000640 1 --2- 172.21.15.5:0/1311331375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b4bb0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f57100cfc80 tx=0x7f56f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97904af2a6446bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.409+0000 7f570f400640 1 -- 172.21.15.5:0/1311331375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f8037070 con 0x7f5710161200 2024-04-25T10:13:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.409+0000 7f570f400640 1 -- 172.21.15.5:0/1311331375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f8032440 con 0x7f5710161200 2024-04-25T10:13:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.409+0000 7f570f400640 1 -- 172.21.15.5:0/1311331375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80327f0 con 0x7f5710161200 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.409+0000 7f5716400640 1 -- 172.21.15.5:0/1311331375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 msgr2=0x7f57100b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.409+0000 7f5716400640 1 --2- 172.21.15.5:0/1311331375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b4bb0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f57100cfc80 tx=0x7f56f802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.410+0000 7f5716400640 1 -- 172.21.15.5:0/1311331375 shutdown_connections 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.410+0000 7f5716400640 1 --2- 172.21.15.5:0/1311331375 >> v2:172.21.15.80:3300/0 conn(0x7f57100d07d0 0x7f57100b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.410+0000 7f5716400640 1 --2- 172.21.15.5:0/1311331375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b4bb0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.410+0000 7f5716400640 1 -- 172.21.15.5:0/1311331375 >> 172.21.15.5:0/1311331375 conn(0x7f571008af80 msgr2=0x7f57100d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.410+0000 7f5716400640 1 -- 172.21.15.5:0/1311331375 shutdown_connections 2024-04-25T10:13:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.410+0000 7f5716400640 1 -- 172.21.15.5:0/1311331375 wait complete. 2024-04-25T10:13:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.412+0000 7f5716400640 1 Processor -- start 2024-04-25T10:13:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.412+0000 7f5716400640 1 -- start start 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5716400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57100d07d0 0x7f57100cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5716400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57100d2380 0x7f57100b7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5716400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5716400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b34c0 con 0x7f5710161200 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5716400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100d42b0 con 0x7f57100d07d0 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5716400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57100b3640 con 0x7f57100d2380 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5715a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5715a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55874/0 (socket says 172.21.15.5:55874) 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.413+0000 7f5715a00640 1 -- 172.21.15.5:0/2863249255 learned_addr learned my addr 172.21.15.5:0/2863249255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715000640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57100d07d0 0x7f57100cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f570fe00640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57100d2380 0x7f57100b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715a00640 1 -- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57100d2380 msgr2=0x7f57100b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715a00640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57100d2380 0x7f57100b7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715a00640 1 -- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57100d07d0 msgr2=0x7f57100cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715a00640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57100d07d0 0x7f57100cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715a00640 1 -- 172.21.15.5:0/2863249255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100cb200 con 0x7f5710161200 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715000640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57100d07d0 0x7f57100cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5715a00640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b7860 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f5708007e30 tx=0x7f570800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708014620 con 0x7f5710161200 2024-04-25T10:13:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5708019070 con 0x7f5710161200 2024-04-25T10:13:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708014a00 con 0x7f5710161200 2024-04-25T10:13:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57100cb4e0 con 0x7f5710161200 2024-04-25T10:13:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.414+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57100cbaa0 con 0x7f5710161200 2024-04-25T10:13:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.416+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5710068f60 con 0x7f5710161200 2024-04-25T10:13:13.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.417+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5708014ba0 con 0x7f5710161200 2024-04-25T10:13:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.418+0000 7f570cc00640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.418+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f570808acd0 con 0x7f5710161200 2024-04-25T10:13:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.418+0000 7f5715000640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.420+0000 7f5715000640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ec06b4d0 0x7f56ec06d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f56f8034000 tx=0x7f56f802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.422+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5708059960 con 0x7f5710161200 2024-04-25T10:13:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.665+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57100cbc70 con 0x7f5710161200 2024-04-25T10:13:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.665+0000 7f570cc00640 1 -- 172.21.15.5:0/2863249255 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5708059260 con 0x7f5710161200 2024-04-25T10:13:13.670 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ec06b4d0 msgr2=0x7f56ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ec06b4d0 0x7f56ec06d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f56f8034000 tx=0x7f56f802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 msgr2=0x7f57100b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b7860 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f5708007e30 tx=0x7f570800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 shutdown_connections 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5710161200 0x7f57100b7860 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57100d2380 0x7f57100b7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 --2- 172.21.15.5:0/2863249255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57100d07d0 0x7f57100cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 >> 172.21.15.5:0/2863249255 conn(0x7f571008af80 msgr2=0x7f57100d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 shutdown_connections 2024-04-25T10:13:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:13.668+0000 7f5716400640 1 -- 172.21.15.5:0/2863249255 wait complete. 2024-04-25T10:13:13.691 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:13.692 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:13.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:13.692 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.747 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41932sec preferred_lft 41932sec 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:13.748 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:13:13.749 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:13.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:13.751 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:13.836 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:13:13.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:13.840 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:13:13.899 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:13:13.899 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:13.899 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:13:13.900 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:13:13.900 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:13.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:13.901 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:13:13.901 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:13:13.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:13:13.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:13:13.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:13:13.901 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:13:14.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.061 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:13:14.115 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:14.115 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:14.115 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:14.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:13:14.194 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:14.194 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:14.250 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:14.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:14.250 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:14.306 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:14.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:14.445 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:14.445 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:14.445 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:14.469 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:14.470 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:14.470 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:14.470 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:13:14.470 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:13:14.498 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:13:14.500 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:14.500 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:14.507 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:14.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:14.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.591 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:14.655 INFO:teuthology.orchestra.run.smithi005.stdout:client7171 2024-04-25T10:13:14.655 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:14.655 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:14.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.714 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./parent 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/charlie 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/alpha 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed 2024-04-25T10:13:14.783 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed/jfile 2024-04-25T10:13:14.784 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:14.785 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:14.785 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:14.785 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:14.785 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:13:14.785 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:14.934 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:13:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:14.949 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:14.949 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:14.949 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:14.950 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:14.950 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:13:14.950 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:15.146 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:13:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:15.157 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:15.157 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:15.157 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:15.157 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:15.157 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:13:15.157 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:15.351 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:13:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:15.366 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:15.367 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:15.367 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:15.367 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:15.367 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:13:15.367 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-04-25T10:13:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:15.576 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:15.576 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:15.576 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:15.576 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:15.576 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:13:15.576 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:15.775 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-04-25T10:13:15.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:15.789 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:15.789 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:15.789 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:15.789 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:15.789 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:13:15.789 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:15.970 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-04-25T10:13:15.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:15.984 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:15.984 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:15.984 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:15.984 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:15.984 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:13:15.984 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:16.175 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-04-25T10:13:16.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:16.190 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:16.190 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:16.291 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:16.291 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:16.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:16.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.556+0000 7f6c43800640 1 Processor -- start 2024-04-25T10:13:16.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.556+0000 7f6c43800640 1 -- start start 2024-04-25T10:13:16.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.557+0000 7f6c43800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6c3c15da50 0x7f6c3c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.557+0000 7f6c43800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.557+0000 7f6c43800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c3c0d0280 0x7f6c3c0d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.557+0000 7f6c43800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c0d0c50 con 0x7f6c3c0d0280 2024-04-25T10:13:16.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.557+0000 7f6c43800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c0ce2b0 con 0x7f6c3c15da50 2024-04-25T10:13:16.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.557+0000 7f6c43800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c15e960 con 0x7f6c3c15de30 2024-04-25T10:13:16.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c41800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c3c0d0280 0x7f6c3c0d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c41800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c3c0d0280 0x7f6c3c0d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55880/0 (socket says 172.21.15.5:55880) 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c41800640 1 -- 172.21.15.5:0/3857327686 learned_addr learned my addr 172.21.15.5:0/3857327686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c3be00640 1 --2- 172.21.15.5:0/3857327686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c40e00640 1 --2- 172.21.15.5:0/3857327686 >> v2:172.21.15.80:3300/0 conn(0x7f6c3c15da50 0x7f6c3c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c41800640 1 -- 172.21.15.5:0/3857327686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 msgr2=0x7f6c3c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c41800640 1 --2- 172.21.15.5:0/3857327686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.558+0000 7f6c41800640 1 -- 172.21.15.5:0/3857327686 >> v2:172.21.15.80:3300/0 conn(0x7f6c3c15da50 msgr2=0x7f6c3c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.559+0000 7f6c41800640 1 --2- 172.21.15.5:0/3857327686 >> v2:172.21.15.80:3300/0 conn(0x7f6c3c15da50 0x7f6c3c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.559+0000 7f6c41800640 1 -- 172.21.15.5:0/3857327686 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c15ec40 con 0x7f6c3c0d0280 2024-04-25T10:13:16.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.560+0000 7f6c41800640 1 --2- 172.21.15.5:0/3857327686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0d0710 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c08b970 tx=0x7f6c3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306b17384bd754cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.560+0000 7f6c3b400640 1 -- 172.21.15.5:0/3857327686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34036070 con 0x7f6c3c0d0280 2024-04-25T10:13:16.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c3b400640 1 -- 172.21.15.5:0/3857327686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c34031600 con 0x7f6c3c0d0280 2024-04-25T10:13:16.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c3b400640 1 -- 172.21.15.5:0/3857327686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340319d0 con 0x7f6c3c0d0280 2024-04-25T10:13:16.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 -- 172.21.15.5:0/3857327686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c0d0280 msgr2=0x7f6c3c0d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 --2- 172.21.15.5:0/3857327686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0d0710 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c08b970 tx=0x7f6c3402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 -- 172.21.15.5:0/3857327686 shutdown_connections 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 --2- 172.21.15.5:0/3857327686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0d0710 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 --2- 172.21.15.5:0/3857327686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 --2- 172.21.15.5:0/3857327686 >> v2:172.21.15.80:3300/0 conn(0x7f6c3c15da50 0x7f6c3c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.561+0000 7f6c43800640 1 -- 172.21.15.5:0/3857327686 >> 172.21.15.5:0/3857327686 conn(0x7f6c3c072290 msgr2=0x7f6c3c072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.562+0000 7f6c43800640 1 -- 172.21.15.5:0/3857327686 shutdown_connections 2024-04-25T10:13:16.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.562+0000 7f6c43800640 1 -- 172.21.15.5:0/3857327686 wait complete. 2024-04-25T10:13:16.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.563+0000 7f6c43800640 1 Processor -- start 2024-04-25T10:13:16.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.564+0000 7f6c43800640 1 -- start start 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.564+0000 7f6c43800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c43800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 0x7f6c3c0b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c43800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c43800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0cf150 con 0x7f6c3c15da50 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c43800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0cefd0 con 0x7f6c3c0d0280 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c43800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c161640 con 0x7f6c3c15de30 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 0x7f6c3c0b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 0x7f6c3c0b8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55896/0 (socket says 172.21.15.5:55896) 2024-04-25T10:13:16.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 -- 172.21.15.5:0/2033687167 learned_addr learned my addr 172.21.15.5:0/2033687167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:16.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c41800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 -- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 msgr2=0x7f6c3c0b8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c40e00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c0b8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 -- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3c0d0280 msgr2=0x7f6c3c0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c3be00640 1 -- 172.21.15.5:0/2033687167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0c7aa0 con 0x7f6c3c15da50 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.565+0000 7f6c41800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c0b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.566+0000 7f6c3be00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 0x7f6c3c0b8850 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800d5d0 tx=0x7f6c2800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.566+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280125f0 con 0x7f6c3c15da50 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.566+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28019070 con 0x7f6c3c15da50 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.566+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c280129f0 con 0x7f6c3c15da50 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.566+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c0c7d80 con 0x7f6c3c15da50 2024-04-25T10:13:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.566+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c0c8340 con 0x7f6c3c15da50 2024-04-25T10:13:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.567+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c068fd0 con 0x7f6c3c15da50 2024-04-25T10:13:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.568+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c28012b90 con 0x7f6c3c15da50 2024-04-25T10:13:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.569+0000 7f6c38c00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c1c06b4d0 0x7f6c1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.569+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c2808a800 con 0x7f6c3c15da50 2024-04-25T10:13:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.570+0000 7f6c40e00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c1c06b4d0 0x7f6c1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.571+0000 7f6c40e00640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c1c06b4d0 0x7f6c1c06d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006fd0 tx=0x7f6c2c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.574+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c280594a0 con 0x7f6c3c15da50 2024-04-25T10:13:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.776+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c3c15d0e0 con 0x7f6c3c15da50 2024-04-25T10:13:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.779+0000 7f6c38c00640 1 -- 172.21.15.5:0/2033687167 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16973 (secure 0 0 0) 0x7f6c28058da0 con 0x7f6c3c15da50 2024-04-25T10:13:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:16.784 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":121,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:13:06.161945+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6837/84886517":"2024-04-26T10:13:06.161908+0000","172.21.15.80:6836/84886517":"2024-04-26T10:13:06.161908+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c1c06b4d0 msgr2=0x7f6c1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c1c06b4d0 0x7f6c1c06d990 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006fd0 tx=0x7f6c2c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 msgr2=0x7f6c3c0b8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 0x7f6c3c0b8850 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800d5d0 tx=0x7f6c2800da90 comp rx=0 tx=0).stop 2024-04-25T10:13:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 shutdown_connections 2024-04-25T10:13:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6c3c15de30 0x7f6c3c0b8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3c15da50 0x7f6c3c0b8850 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f6c1c06b4d0 0x7f6c1c06d990 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 --2- 172.21.15.5:0/2033687167 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6c3c0d0280 0x7f6c3c0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 >> 172.21.15.5:0/2033687167 conn(0x7f6c3c072290 msgr2=0x7f6c3c072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 shutdown_connections 2024-04-25T10:13:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:16.782+0000 7f6c43800640 1 -- 172.21.15.5:0/2033687167 wait complete. 2024-04-25T10:13:16.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:45.398584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:46.541796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:16.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:17.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.010+0000 7f9a1ac00640 1 Processor -- start 2024-04-25T10:13:17.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.010+0000 7f9a1ac00640 1 -- start start 2024-04-25T10:13:17.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.011+0000 7f9a1ac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 0x7f9a1415e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.011+0000 7f9a1ac00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9a140ae860 0x7f9a14071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.012+0000 7f9a1ac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1415eed0 con 0x7f9a140aece0 2024-04-25T10:13:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.012+0000 7f9a1ac00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1406f080 con 0x7f9a140ae860 2024-04-25T10:13:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.012+0000 7f9a1ac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406f360 con 0x7f9a141465d0 2024-04-25T10:13:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.012+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 0x7f9a1415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.012+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 0x7f9a1415e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:35846/0 (socket says 172.21.15.5:35846) 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.012+0000 7f9a13e00640 1 -- 172.21.15.5:0/3217751973 learned_addr learned my addr 172.21.15.5:0/3217751973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.013+0000 7f9a0b400640 1 --2- 172.21.15.5:0/3217751973 >> v2:172.21.15.80:3300/0 conn(0x7f9a140ae860 0x7f9a14071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.013+0000 7f9a13e00640 1 -- 172.21.15.5:0/3217751973 >> v2:172.21.15.80:3300/0 conn(0x7f9a140ae860 msgr2=0x7f9a14071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.013+0000 7f9a13e00640 1 --2- 172.21.15.5:0/3217751973 >> v2:172.21.15.80:3300/0 conn(0x7f9a140ae860 0x7f9a14071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.013+0000 7f9a13e00640 1 -- 172.21.15.5:0/3217751973 >> v1:172.21.15.80:6789/0 conn(0x7f9a140aece0 legacy=0x7f9a1415f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.013+0000 7f9a13e00640 1 -- 172.21.15.5:0/3217751973 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140ae060 con 0x7f9a141465d0 2024-04-25T10:13:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.015+0000 7f9a13e00640 1 --2- 172.21.15.5:0/3217751973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 0x7f9a1415e990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009870 tx=0x7f9a0402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3effbd3e93c4b522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.015+0000 7f9a13400640 1 -- 172.21.15.5:0/3217751973 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04037070 con 0x7f9a141465d0 2024-04-25T10:13:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.015+0000 7f9a13400640 1 -- 172.21.15.5:0/3217751973 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04032440 con 0x7f9a141465d0 2024-04-25T10:13:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.015+0000 7f9a13400640 1 -- 172.21.15.5:0/3217751973 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04032810 con 0x7f9a141465d0 2024-04-25T10:13:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 -- 172.21.15.5:0/3217751973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 msgr2=0x7f9a1415e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/3217751973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 0x7f9a1415e990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009870 tx=0x7f9a0402a250 comp rx=0 tx=0).stop 2024-04-25T10:13:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 -- 172.21.15.5:0/3217751973 shutdown_connections 2024-04-25T10:13:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/3217751973 >> v2:172.21.15.80:3300/0 conn(0x7f9a140ae860 0x7f9a14071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/3217751973 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a141465d0 0x7f9a1415e990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 -- 172.21.15.5:0/3217751973 >> 172.21.15.5:0/3217751973 conn(0x7f9a1406ab60 msgr2=0x7f9a1406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 -- 172.21.15.5:0/3217751973 shutdown_connections 2024-04-25T10:13:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.016+0000 7f9a1ac00640 1 -- 172.21.15.5:0/3217751973 wait complete. 2024-04-25T10:13:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.018+0000 7f9a1ac00640 1 Processor -- start 2024-04-25T10:13:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.018+0000 7f9a1ac00640 1 -- start start 2024-04-25T10:13:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a1ac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 0x7f9a14141b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a1ac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a140aece0 0x7f9a140cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a1ac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a141465d0 0x7f9a140cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a1ac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406e7d0 con 0x7f9a140ae860 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a1ac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141469b0 con 0x7f9a141465d0 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a1ac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1406ece0 con 0x7f9a140aece0 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.019+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 0x7f9a14141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 0x7f9a14141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47116/0 (socket says 172.21.15.5:47116) 2024-04-25T10:13:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 -- 172.21.15.5:0/2079384856 learned_addr learned my addr 172.21.15.5:0/2079384856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 -- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a140aece0 msgr2=0x7f9a140cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a140aece0 0x7f9a140cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 -- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a141465d0 msgr2=0x7f9a140cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a141465d0 0x7f9a140cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 -- 172.21.15.5:0/2079384856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140cc590 con 0x7f9a140ae860 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.020+0000 7f9a13e00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 0x7f9a14141b90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9a04000c00 tx=0x7f9a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.021+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04041020 con 0x7f9a140ae860 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.021+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a040306c0 con 0x7f9a140ae860 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.021+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04037040 con 0x7f9a140ae860 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.021+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a140cc870 con 0x7f9a140ae860 2024-04-25T10:13:17.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.021+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a140cd550 con 0x7f9a140ae860 2024-04-25T10:13:17.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.022+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14068f60 con 0x7f9a140ae860 2024-04-25T10:13:17.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.023+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a04044020 con 0x7f9a140ae860 2024-04-25T10:13:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.024+0000 7f9a10c00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f99f006b4d0 0x7f99f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.024+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9a040aaca0 con 0x7f9a140ae860 2024-04-25T10:13:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.024+0000 7f9a0b400640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f99f006b4d0 0x7f99f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.026+0000 7f9a0b400640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f99f006b4d0 0x7f99f006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9a140b1b40 tx=0x7f99fc008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.028+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a0404b030 con 0x7f9a140ae860 2024-04-25T10:13:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.227+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a14072010 con 0x7f9a140ae860 2024-04-25T10:13:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.228+0000 7f9a10c00640 1 -- 172.21.15.5:0/2079384856 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a040798a0 con 0x7f9a140ae860 2024-04-25T10:13:17.232 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:13:17.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f99f006b4d0 msgr2=0x7f99f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f99f006b4d0 0x7f99f006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9a140b1b40 tx=0x7f99fc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 msgr2=0x7f9a14141b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 0x7f9a14141b90 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9a04000c00 tx=0x7f9a040046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 shutdown_connections 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f99f006b4d0 0x7f99f006d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a141465d0 0x7f9a140cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9a140aece0 0x7f9a140cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 --2- 172.21.15.5:0/2079384856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a140ae860 0x7f9a14141b90 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 >> 172.21.15.5:0/2079384856 conn(0x7f9a1406ab60 msgr2=0x7f9a140b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 shutdown_connections 2024-04-25T10:13:17.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.230+0000 7f9a1ac00640 1 -- 172.21.15.5:0/2079384856 wait complete. 2024-04-25T10:13:17.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.466+0000 7f670de00640 1 Processor -- start 2024-04-25T10:13:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.467+0000 7f670de00640 1 -- start start 2024-04-25T10:13:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.468+0000 7f670de00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f67080cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.468+0000 7f670de00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6708144cd0 con 0x7f670815bc30 2024-04-25T10:13:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.468+0000 7f670de00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f670815c440 con 0x7f670815ce20 2024-04-25T10:13:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.468+0000 7f670de00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670815f460 con 0x7f67081448f0 2024-04-25T10:13:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.469+0000 7f6707e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f670815bc30 0x7f670815f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37894/0 (socket says 172.21.15.5:37894) 2024-04-25T10:13:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.469+0000 7f6707e00640 1 -- 172.21.15.5:0/2962395778 learned_addr learned my addr 172.21.15.5:0/2962395778 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.469+0000 7f670ca00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f67080cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.469+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482000261 0 0) 0x7f6708144cd0 con 0x7f670815bc30 2024-04-25T10:13:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.469+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66f0003520 con 0x7f670815bc30 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833477249 0 0) 0x7f66f0003520 con 0x7f670815bc30 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 msgr2=0x7f67080cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f67080cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 >> v1:172.21.15.80:6789/0 conn(0x7f670815ce20 legacy=0x7f6708071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670815c720 con 0x7f670815bc30 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f66f4001670 con 0x7f670815bc30 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 >> v1:172.21.15.5:6789/0 conn(0x7f670815bc30 legacy=0x7f670815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f66f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f0004d40 0x7f66f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670815f460 con 0x7f66f00039e0 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f000baa0 con 0x7f66f0004d40 2024-04-25T10:13:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f000bca0 con 0x7f67081448f0 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f670ca00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.470+0000 7f6707e00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f66f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.471+0000 7f670ca00640 1 -- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 msgr2=0x7f66f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.471+0000 7f670ca00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f66f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.471+0000 7f670ca00640 1 -- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f0004d40 msgr2=0x7f66f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.471+0000 7f670ca00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f0004d40 0x7f66f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.471+0000 7f670ca00640 1 -- 172.21.15.5:0/2962395778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0003520 con 0x7f66f00039e0 2024-04-25T10:13:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.471+0000 7f6707e00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67081448f0 0x7f66f0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.472+0000 7f670ca00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009a70 tx=0x7f66f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb845748ecf71bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.472+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc036070 con 0x7f66f00039e0 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.472+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc031400 con 0x7f66f00039e0 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.472+0000 7f6707400640 1 -- 172.21.15.5:0/2962395778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc009670 con 0x7f66f00039e0 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670de00640 1 -- 172.21.15.5:0/2962395778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f00039e0 msgr2=0x7f66f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670de00640 1 --2- 172.21.15.5:0/2962395778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f00039e0 0x7f66f0003e00 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009a70 tx=0x7f66f0005830 comp rx=0 tx=0).stop 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670d400640 1 -- 172.21.15.5:0/2962395778 reap_dead start 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670de00640 1 -- 172.21.15.5:0/2962395778 shutdown_connections 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670de00640 1 -- 172.21.15.5:0/2962395778 >> 172.21.15.5:0/2962395778 conn(0x7f6708066f20 msgr2=0x7f670806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670de00640 1 -- 172.21.15.5:0/2962395778 shutdown_connections 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.473+0000 7f670de00640 1 -- 172.21.15.5:0/2962395778 wait complete. 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.475+0000 7f670de00640 1 Processor -- start 2024-04-25T10:13:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.475+0000 7f670de00640 1 -- start start 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f670de00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f00039e0 0x7f67080ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f670de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 0x7f67080b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f670de00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f670812efa0 0x7f670812f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f670de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670815c090 con 0x7f67080b0f80 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f670de00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708160000 con 0x7f66f00039e0 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f670de00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670812fcd0 con 0x7f670812efa0 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f6707e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 0x7f67080b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f6707e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 0x7f67080b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47120/0 (socket says 172.21.15.5:47120) 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.476+0000 7f6707e00640 1 -- 172.21.15.5:0/942447620 learned_addr learned my addr 172.21.15.5:0/942447620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6707e00640 1 -- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f670812efa0 msgr2=0x7f670812f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f670d400640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f670812efa0 0x7f670812f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f670ca00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f00039e0 0x7f67080ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6707e00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f670812efa0 0x7f670812f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6707e00640 1 -- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f00039e0 msgr2=0x7f67080ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6707e00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f00039e0 0x7f67080ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6707e00640 1 -- 172.21.15.5:0/942447620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670812cdf0 con 0x7f67080b0f80 2024-04-25T10:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6707e00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 0x7f67080b1400 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d860 tx=0x7f66f400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4019070 con 0x7f67080b0f80 2024-04-25T10:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f40074e0 con 0x7f67080b0f80 2024-04-25T10:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670812d0d0 con 0x7f67080b0f80 2024-04-25T10:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f40049e0 con 0x7f67080b0f80 2024-04-25T10:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.477+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670812d690 con 0x7f67080b0f80 2024-04-25T10:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.479+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708068f60 con 0x7f67080b0f80 2024-04-25T10:13:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.480+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66f401c070 con 0x7f67080b0f80 2024-04-25T10:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.481+0000 7f6704c00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66e806b4d0 0x7f66e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.481+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f66f4089ee0 con 0x7f67080b0f80 2024-04-25T10:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.481+0000 7f670ca00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66e806b4d0 0x7f66e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.483+0000 7f670ca00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66e806b4d0 0x7f66e806d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f66fc033000 tx=0x7f66fc008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.485+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66f4058b70 con 0x7f67080b0f80 2024-04-25T10:13:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.683+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67080ce0d0 con 0x7f67080b0f80 2024-04-25T10:13:17.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.684+0000 7f6704c00640 1 -- 172.21.15.5:0/942447620 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66f4058470 con 0x7f67080b0f80 2024-04-25T10:13:17.688 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:13:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66e806b4d0 msgr2=0x7f66e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66e806b4d0 0x7f66e806d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f66fc033000 tx=0x7f66fc008030 comp rx=0 tx=0).stop 2024-04-25T10:13:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 msgr2=0x7f67080b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 0x7f67080b1400 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f66f400d860 tx=0x7f66f400dd20 comp rx=0 tx=0).stop 2024-04-25T10:13:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 shutdown_connections 2024-04-25T10:13:17.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f670812efa0 0x7f670812f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67080b0f80 0x7f67080b1400 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f66e806b4d0 0x7f66e806d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 --2- 172.21.15.5:0/942447620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66f00039e0 0x7f67080ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 >> 172.21.15.5:0/942447620 conn(0x7f6708066f20 msgr2=0x7f670815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 shutdown_connections 2024-04-25T10:13:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.687+0000 7f670de00640 1 -- 172.21.15.5:0/942447620 wait complete. 2024-04-25T10:13:17.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:17.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.926+0000 7fdf6b400640 1 Processor -- start 2024-04-25T10:13:17.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.927+0000 7fdf6b400640 1 -- start start 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.928+0000 7fdf6b400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdf64144fe0 0x7fdf6406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.928+0000 7fdf6b400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf6406d130 0x7fdf64071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.928+0000 7fdf6b400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf641453c0 con 0x7fdf64071670 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.928+0000 7fdf6b400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640a8560 con 0x7fdf6406d130 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.928+0000 7fdf6b400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6406d6a0 con 0x7fdf64144fe0 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf63e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf6406d130 0x7fdf64071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf69400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdf64071670 0x7fdf640cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37910/0 (socket says 172.21.15.5:37910) 2024-04-25T10:13:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf69400640 1 -- 172.21.15.5:0/1817806743 learned_addr learned my addr 172.21.15.5:0/1817806743 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf68a00640 1 --2- 172.21.15.5:0/1817806743 >> v2:172.21.15.80:3300/0 conn(0x7fdf64144fe0 0x7fdf6406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf63e00640 1 -- 172.21.15.5:0/1817806743 >> v2:172.21.15.80:3300/0 conn(0x7fdf64144fe0 msgr2=0x7fdf6406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf63e00640 1 --2- 172.21.15.5:0/1817806743 >> v2:172.21.15.80:3300/0 conn(0x7fdf64144fe0 0x7fdf6406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf63e00640 1 -- 172.21.15.5:0/1817806743 >> v1:172.21.15.5:6789/0 conn(0x7fdf64071670 legacy=0x7fdf640cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.929+0000 7fdf63e00640 1 -- 172.21.15.5:0/1817806743 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6406d980 con 0x7fdf6406d130 2024-04-25T10:13:17.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.930+0000 7fdf63e00640 1 --2- 172.21.15.5:0/1817806743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6406d130 0x7fdf64071130 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fdf640728b0 tx=0x7fdf4c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0e4a09f6075401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf63400640 1 -- 172.21.15.5:0/1817806743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c037070 con 0x7fdf6406d130 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf63400640 1 -- 172.21.15.5:0/1817806743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c02ad30 con 0x7fdf6406d130 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf63400640 1 -- 172.21.15.5:0/1817806743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c032660 con 0x7fdf6406d130 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf6b400640 1 -- 172.21.15.5:0/1817806743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6406d130 msgr2=0x7fdf64071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1817806743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6406d130 0x7fdf64071130 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fdf640728b0 tx=0x7fdf4c02a0a0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf6b400640 1 -- 172.21.15.5:0/1817806743 shutdown_connections 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1817806743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6406d130 0x7fdf64071130 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1817806743 >> v2:172.21.15.80:3300/0 conn(0x7fdf64144fe0 0x7fdf6406e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.931+0000 7fdf6b400640 1 -- 172.21.15.5:0/1817806743 >> 172.21.15.5:0/1817806743 conn(0x7fdf6406ab60 msgr2=0x7fdf6406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.932+0000 7fdf6b400640 1 -- 172.21.15.5:0/1817806743 shutdown_connections 2024-04-25T10:13:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.932+0000 7fdf6b400640 1 -- 172.21.15.5:0/1817806743 wait complete. 2024-04-25T10:13:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.933+0000 7fdf6b400640 1 Processor -- start 2024-04-25T10:13:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.934+0000 7fdf6b400640 1 -- start start 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.934+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf6406d130 0x7fdf6413b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf64071670 0x7fdf6413e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 0x7fdf64143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf6b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415be50 con 0x7fdf64144fe0 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf6b400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6406df40 con 0x7fdf6406d130 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf6b400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6406eaa0 con 0x7fdf64071670 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 0x7fdf64143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 0x7fdf64143030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47150/0 (socket says 172.21.15.5:47150) 2024-04-25T10:13:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 -- 172.21.15.5:0/1000733817 learned_addr learned my addr 172.21.15.5:0/1000733817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:17.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 -- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf64071670 msgr2=0x7fdf6413e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf64071670 0x7fdf6413e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 -- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf6406d130 msgr2=0x7fdf6413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf6406d130 0x7fdf6413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.935+0000 7fdf69400640 1 -- 172.21.15.5:0/1000733817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64140650 con 0x7fdf64144fe0 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.936+0000 7fdf69400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 0x7fdf64143030 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c750 tx=0x7fdf5c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.936+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c018070 con 0x7fdf64144fe0 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.936+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c0072d0 con 0x7fdf64144fe0 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.936+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0047e0 con 0x7fdf64144fe0 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.936+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf64140930 con 0x7fdf64144fe0 2024-04-25T10:13:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.936+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf64140ef0 con 0x7fdf64144fe0 2024-04-25T10:13:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.937+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf64068f60 con 0x7fdf64144fe0 2024-04-25T10:13:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.938+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf5c004980 con 0x7fdf64144fe0 2024-04-25T10:13:17.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.940+0000 7fdf60c00640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf4006b4d0 0x7fdf4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.940+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdf5c089860 con 0x7fdf64144fe0 2024-04-25T10:13:17.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.940+0000 7fdf68a00640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf4006b4d0 0x7fdf4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.942+0000 7fdf68a00640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf4006b4d0 0x7fdf4006d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdf540070b0 tx=0x7fdf54007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:17.944+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf5c004c70 con 0x7fdf64144fe0 2024-04-25T10:13:18.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.149+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf640b1b60 con 0x7fdf64144fe0 2024-04-25T10:13:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.150+0000 7fdf60c00640 1 -- 172.21.15.5:0/1000733817 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf5c058460 con 0x7fdf64144fe0 2024-04-25T10:13:18.154 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:13:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf4006b4d0 msgr2=0x7fdf4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf4006b4d0 0x7fdf4006d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdf540070b0 tx=0x7fdf54007040 comp rx=0 tx=0).stop 2024-04-25T10:13:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 msgr2=0x7fdf64143030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 0x7fdf64143030 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c750 tx=0x7fdf5c00cc10 comp rx=0 tx=0).stop 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 shutdown_connections 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf64144fe0 0x7fdf64143030 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf64071670 0x7fdf6413e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdf4006b4d0 0x7fdf4006d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 --2- 172.21.15.5:0/1000733817 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf6406d130 0x7fdf6413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 >> 172.21.15.5:0/1000733817 conn(0x7fdf6406ab60 msgr2=0x7fdf6415e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 shutdown_connections 2024-04-25T10:13:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.153+0000 7fdf6b400640 1 -- 172.21.15.5:0/1000733817 wait complete. 2024-04-25T10:13:18.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.380+0000 7ffaefe00640 1 Processor -- start 2024-04-25T10:13:18.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.381+0000 7ffaefe00640 1 -- start start 2024-04-25T10:13:18.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.382+0000 7ffaefe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf015b000 0x7ffaf006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.382+0000 7ffaefe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffaf00cc280 0x7ffaf0070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.382+0000 7ffaefe00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf0154810 con 0x7ffaf00cbe00 2024-04-25T10:13:18.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.382+0000 7ffaefe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf00b3780 con 0x7ffaf00cc280 2024-04-25T10:13:18.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.382+0000 7ffaefe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00ce200 con 0x7ffaf015b000 2024-04-25T10:13:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.383+0000 7ffaeea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf015b000 0x7ffaf006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.383+0000 7ffaef400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffaf00cc280 0x7ffaf0070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.383+0000 7ffaeea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf015b000 0x7ffaf006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:35892/0 (socket says 172.21.15.5:35892) 2024-04-25T10:13:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.383+0000 7ffaeea00640 1 -- 172.21.15.5:0/2250744067 learned_addr learned my addr 172.21.15.5:0/2250744067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.384+0000 7ffaef400640 1 -- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf015b000 msgr2=0x7ffaf006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.384+0000 7ffaef400640 1 --2- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf015b000 0x7ffaf006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.384+0000 7ffaef400640 1 -- 172.21.15.5:0/2250744067 >> v1:172.21.15.80:6789/0 conn(0x7ffaf00cbe00 legacy=0x7ffaf00cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.384+0000 7ffaef400640 1 -- 172.21.15.5:0/2250744067 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00ce4d0 con 0x7ffaf00cc280 2024-04-25T10:13:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.385+0000 7ffaef400640 1 --2- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf00cc280 0x7ffaf0070f90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009dc0 tx=0x7ffae802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd03b7ab0a136e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.386+0000 7ffaed600640 1 -- 172.21.15.5:0/2250744067 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae802ae60 con 0x7ffaf00cc280 2024-04-25T10:13:18.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaed600640 1 -- 172.21.15.5:0/2250744067 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae8007740 con 0x7ffaf00cc280 2024-04-25T10:13:18.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaed600640 1 -- 172.21.15.5:0/2250744067 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8007b10 con 0x7ffaf00cc280 2024-04-25T10:13:18.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 -- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf00cc280 msgr2=0x7ffaf0070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf00cc280 0x7ffaf0070f90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009dc0 tx=0x7ffae802a680 comp rx=0 tx=0).stop 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 -- 172.21.15.5:0/2250744067 shutdown_connections 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf00cc280 0x7ffaf0070f90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2250744067 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf015b000 0x7ffaf006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 -- 172.21.15.5:0/2250744067 >> 172.21.15.5:0/2250744067 conn(0x7ffaf0066f20 msgr2=0x7ffaf006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 -- 172.21.15.5:0/2250744067 shutdown_connections 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.387+0000 7ffaefe00640 1 -- 172.21.15.5:0/2250744067 wait complete. 2024-04-25T10:13:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.389+0000 7ffaefe00640 1 Processor -- start 2024-04-25T10:13:18.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.389+0000 7ffaefe00640 1 -- start start 2024-04-25T10:13:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.390+0000 7ffaefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 0x7ffaf01f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.390+0000 7ffaefe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf00cc280 0x7ffaf01f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.390+0000 7ffaefe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf015b000 0x7ffaf015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.390+0000 7ffaefe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00b3310 con 0x7ffaf00cbe00 2024-04-25T10:13:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.390+0000 7ffaefe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00cea90 con 0x7ffaf015b000 2024-04-25T10:13:18.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.390+0000 7ffaefe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00b3490 con 0x7ffaf00cc280 2024-04-25T10:13:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 0x7ffaf01f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 0x7ffaf01f7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47152/0 (socket says 172.21.15.5:47152) 2024-04-25T10:13:18.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 -- 172.21.15.5:0/2889696827 learned_addr learned my addr 172.21.15.5:0/2889696827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaef400640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf015b000 0x7ffaf015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaee000640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf00cc280 0x7ffaf01f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 -- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf00cc280 msgr2=0x7ffaf01f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf00cc280 0x7ffaf01f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 -- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf015b000 msgr2=0x7ffaf015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf015b000 0x7ffaf015c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaeea00640 1 -- 172.21.15.5:0/2889696827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf01f8080 con 0x7ffaf00cbe00 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaef400640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf015b000 0x7ffaf015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.391+0000 7ffaee000640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf00cc280 0x7ffaf01f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.392+0000 7ffaeea00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 0x7ffaf01f7b40 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ffae000e840 tx=0x7ffae000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.392+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0017070 con 0x7ffaf00cbe00 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.392+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000c8e0 con 0x7ffaf00cbe00 2024-04-25T10:13:18.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.392+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000ccd0 con 0x7ffaf00cbe00 2024-04-25T10:13:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.392+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf0140a30 con 0x7ffaf00cbe00 2024-04-25T10:13:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.392+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf0140cd0 con 0x7ffaf00cbe00 2024-04-25T10:13:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.394+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf0068f60 con 0x7ffaf00cbe00 2024-04-25T10:13:18.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.395+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffae001a070 con 0x7ffaf00cbe00 2024-04-25T10:13:18.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.396+0000 7ffae6a00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffad006b4d0 0x7ffad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.396+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ffae008a380 con 0x7ffaf00cbe00 2024-04-25T10:13:18.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.396+0000 7ffaee000640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffad006b4d0 0x7ffad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.398+0000 7ffaee000640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffad006b4d0 0x7ffad006d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0143b20 tx=0x7ffadc009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.400+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaf01f8080 con 0x7ffaf00cbe00 2024-04-25T10:13:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.615+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffaf00af340 con 0x7ffaf00cbe00 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.616+0000 7ffae6a00640 1 -- 172.21.15.5:0/2889696827 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v121) v1 ==== 80+0+118 (secure 0 0 0) 0x7ffae0059010 con 0x7ffaf00cbe00 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6837/84886517 2024-04-26T10:13:06.161908+0000 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6836/84886517 2024-04-26T10:13:06.161908+0000 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:listed 2 entries 2024-04-25T10:13:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffad006b4d0 msgr2=0x7ffad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffad006b4d0 0x7ffad006d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0143b20 tx=0x7ffadc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 msgr2=0x7ffaf01f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 0x7ffaf01f7b40 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ffae000e840 tx=0x7ffae000ed00 comp rx=0 tx=0).stop 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 shutdown_connections 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffad006b4d0 0x7ffad006d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffaf015b000 0x7ffaf015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffaf00cc280 0x7ffaf01f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 --2- 172.21.15.5:0/2889696827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaf00cbe00 0x7ffaf01f7b40 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 >> 172.21.15.5:0/2889696827 conn(0x7ffaf0066f20 msgr2=0x7ffaf006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 shutdown_connections 2024-04-25T10:13:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:18.619+0000 7ffaefe00640 1 -- 172.21.15.5:0/2889696827 wait complete. 2024-04-25T10:13:18.643 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:18.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:18.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:18.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:13:18.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:18.740 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:18.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:19.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.000+0000 7fc0b3200640 1 Processor -- start 2024-04-25T10:13:19.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.000+0000 7fc0b3200640 1 -- start start 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.002+0000 7fc0b3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc0ac141da0 0x7fc0ac06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.002+0000 7fc0b3200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc0ac15d510 0x7fc0ac0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.002+0000 7fc0b3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac071d30 con 0x7fc0ac06e640 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.002+0000 7fc0b3200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac142180 con 0x7fc0ac15d510 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.002+0000 7fc0b3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac06d620 con 0x7fc0ac141da0 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.002+0000 7fc0b2800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc0ac15d510 0x7fc0ac0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b2800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc0ac15d510 0x7fc0ac0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47156/0 (socket says 172.21.15.5:47156) 2024-04-25T10:13:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b2800640 1 -- 172.21.15.5:0/599680979 learned_addr learned my addr 172.21.15.5:0/599680979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b1e00640 1 --2- 172.21.15.5:0/599680979 >> v2:172.21.15.80:3300/0 conn(0x7fc0ac141da0 0x7fc0ac06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b2800640 1 -- 172.21.15.5:0/599680979 >> v2:172.21.15.80:3300/0 conn(0x7fc0ac141da0 msgr2=0x7fc0ac06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b2800640 1 --2- 172.21.15.5:0/599680979 >> v2:172.21.15.80:3300/0 conn(0x7fc0ac141da0 0x7fc0ac06e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b2800640 1 -- 172.21.15.5:0/599680979 >> v1:172.21.15.80:6789/0 conn(0x7fc0ac06e640 legacy=0x7fc0ac15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.003+0000 7fc0b2800640 1 -- 172.21.15.5:0/599680979 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac06d900 con 0x7fc0ac15d510 2024-04-25T10:13:19.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.004+0000 7fc0b2800640 1 --2- 172.21.15.5:0/599680979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac15d510 0x7fc0ac0cd300 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac06bbb0 tx=0x7fc0a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac452cf3321822f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b0a00640 1 -- 172.21.15.5:0/599680979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8037070 con 0x7fc0ac15d510 2024-04-25T10:13:19.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b0a00640 1 -- 172.21.15.5:0/599680979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8032440 con 0x7fc0ac15d510 2024-04-25T10:13:19.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b0a00640 1 -- 172.21.15.5:0/599680979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8032810 con 0x7fc0ac15d510 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b3200640 1 -- 172.21.15.5:0/599680979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac15d510 msgr2=0x7fc0ac0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b3200640 1 --2- 172.21.15.5:0/599680979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac15d510 0x7fc0ac0cd300 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac06bbb0 tx=0x7fc0a802a230 comp rx=0 tx=0).stop 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b3200640 1 -- 172.21.15.5:0/599680979 shutdown_connections 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b3200640 1 --2- 172.21.15.5:0/599680979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac15d510 0x7fc0ac0cd300 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b3200640 1 --2- 172.21.15.5:0/599680979 >> v2:172.21.15.80:3300/0 conn(0x7fc0ac141da0 0x7fc0ac06e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.005+0000 7fc0b3200640 1 -- 172.21.15.5:0/599680979 >> 172.21.15.5:0/599680979 conn(0x7fc0ac066f20 msgr2=0x7fc0ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.006+0000 7fc0b3200640 1 -- 172.21.15.5:0/599680979 shutdown_connections 2024-04-25T10:13:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.006+0000 7fc0b3200640 1 -- 172.21.15.5:0/599680979 wait complete. 2024-04-25T10:13:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.007+0000 7fc0b3200640 1 Processor -- start 2024-04-25T10:13:19.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.007+0000 7fc0b3200640 1 -- start start 2024-04-25T10:13:19.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.008+0000 7fc0b3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 0x7fc0ac13e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.008+0000 7fc0b3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0ac141da0 0x7fc0ac13ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.008+0000 7fc0b3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ac13eab0 0x7fc0ac0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.008+0000 7fc0b3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cd9c0 con 0x7fc0ac06e640 2024-04-25T10:13:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.008+0000 7fc0b3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cd840 con 0x7fc0ac13eab0 2024-04-25T10:13:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.008+0000 7fc0b3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cdb40 con 0x7fc0ac141da0 2024-04-25T10:13:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 0x7fc0ac13e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 0x7fc0ac13e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47170/0 (socket says 172.21.15.5:47170) 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 -- 172.21.15.5:0/3855752637 learned_addr learned my addr 172.21.15.5:0/3855752637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b2800640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ac13eab0 0x7fc0ac0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1400640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0ac141da0 0x7fc0ac13ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 -- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0ac141da0 msgr2=0x7fc0ac13ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0ac141da0 0x7fc0ac13ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 -- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ac13eab0 msgr2=0x7fc0ac0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ac13eab0 0x7fc0ac0b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1e00640 1 -- 172.21.15.5:0/3855752637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac1e7260 con 0x7fc0ac06e640 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b2800640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ac13eab0 0x7fc0ac0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.009+0000 7fc0b1400640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0ac141da0 0x7fc0ac13ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.010+0000 7fc0b1e00640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 0x7fc0ac13e4f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4007f20 tx=0x7fc0a400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.010+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4014620 con 0x7fc0ac06e640 2024-04-25T10:13:19.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.010+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a4019070 con 0x7fc0ac06e640 2024-04-25T10:13:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.010+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4014a00 con 0x7fc0ac06e640 2024-04-25T10:13:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.010+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac1e7540 con 0x7fc0ac06e640 2024-04-25T10:13:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.010+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac1e7b00 con 0x7fc0ac06e640 2024-04-25T10:13:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.012+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac068f60 con 0x7fc0ac06e640 2024-04-25T10:13:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.013+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0a4014ba0 con 0x7fc0ac06e640 2024-04-25T10:13:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.014+0000 7fc0a2000640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc09006b4d0 0x7fc09006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.014+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc0a408ad00 con 0x7fc0ac06e640 2024-04-25T10:13:19.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.015+0000 7fc0b1400640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc09006b4d0 0x7fc09006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.016+0000 7fc0b1400640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc09006b4d0 0x7fc09006d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac15f9b0 tx=0x7fc09c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.019+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0a4059a30 con 0x7fc0ac06e640 2024-04-25T10:13:19.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.260+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0ac13f150 con 0x7fc0ac06e640 2024-04-25T10:13:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.264+0000 7fc0a2000640 1 -- 172.21.15.5:0/3855752637 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4941 (secure 0 0 0) 0x7fc0a4059330 con 0x7fc0ac06e640 2024-04-25T10:13:19.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:19.269 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:10.211656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"c","rank":0,"incarnation":102,"state":"up:active","state_seq":10,"addr":"172.21.15.5:6835/2518129896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":2518129896},{"type":"v1","addr":"172.21.15.5:6835","nonce":2518129896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:19.269 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 105 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc09006b4d0 msgr2=0x7fc09006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc09006b4d0 0x7fc09006d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac15f9b0 tx=0x7fc09c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 msgr2=0x7fc0ac13e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 0x7fc0ac13e4f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4007f20 tx=0x7fc0a400e570 comp rx=0 tx=0).stop 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 shutdown_connections 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc09006b4d0 0x7fc09006d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ac13eab0 0x7fc0ac0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc0ac141da0 0x7fc0ac13ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 --2- 172.21.15.5:0/3855752637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0ac06e640 0x7fc0ac13e4f0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 >> 172.21.15.5:0/3855752637 conn(0x7fc0ac066f20 msgr2=0x7fc0ac13ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.267+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 shutdown_connections 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.268+0000 7fc0b3200640 1 -- 172.21.15.5:0/3855752637 wait complete. 2024-04-25T10:13:19.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-25T10:13:19.531 INFO:teuthology.orchestra.run.smithi005.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T10:13:19.532 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T10:13:19.532 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000003.00000000 scrub_tag 2024-04-25T10:13:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.642+0000 7f50d0f25f40 1 Processor -- start 2024-04-25T10:13:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.643+0000 7f50d0f25f40 1 -- start start 2024-04-25T10:13:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50d0f25f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3d3b710 0x563fc3d1ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50d0f25f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 0x563fc3dc57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50d0f25f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3dad0c0 0x563fc3dcbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50d0f25f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc3c95dd0 con 0x563fc3dacce0 2024-04-25T10:13:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50d0f25f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc3c99d30 con 0x563fc3d3b710 2024-04-25T10:13:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50d0f25f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fc3dce070 con 0x563fc3dad0c0 2024-04-25T10:13:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50ccc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 0x563fc3dc57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50ccc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 0x563fc3dc57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47184/0 (socket says 172.21.15.5:47184) 2024-04-25T10:13:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.644+0000 7f50ccc00640 1 -- 172.21.15.5:0/1630729779 learned_addr learned my addr 172.21.15.5:0/1630729779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50cd600640 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3d3b710 0x563fc3d1ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ce000640 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3dad0c0 0x563fc3dcbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ccc00640 1 -- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3dad0c0 msgr2=0x563fc3dcbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ccc00640 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3dad0c0 0x563fc3dcbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ccc00640 1 -- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3d3b710 msgr2=0x563fc3d1ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ccc00640 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3d3b710 0x563fc3d1ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ccc00640 1 -- 172.21.15.5:0/1630729779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc3d20cd0 con 0x563fc3dacce0 2024-04-25T10:13:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.645+0000 7f50ccc00640 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 0x563fc3dc57e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00a470 tx=0x7f50bc04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99688ff0da1d873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.646+0000 7f50c7400640 1 -- 172.21.15.5:0/1630729779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc059040 con 0x563fc3dacce0 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.646+0000 7f50c7400640 1 -- 172.21.15.5:0/1630729779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc054930 con 0x563fc3dacce0 2024-04-25T10:13:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.646+0000 7f50c7400640 1 -- 172.21.15.5:0/1630729779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc054d20 con 0x563fc3dacce0 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.646+0000 7f50d0f25f40 1 -- 172.21.15.5:0/1630729779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 msgr2=0x563fc3dc57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.646+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 0x563fc3dc57e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00a470 tx=0x7f50bc04e000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 -- 172.21.15.5:0/1630729779 shutdown_connections 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3dad0c0 0x563fc3dcbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dacce0 0x563fc3dc57e0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/1630729779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3d3b710 0x563fc3d1ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 -- 172.21.15.5:0/1630729779 >> 172.21.15.5:0/1630729779 conn(0x563fc3cedc60 msgr2=0x563fc3dccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 -- 172.21.15.5:0/1630729779 shutdown_connections 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.647+0000 7f50d0f25f40 1 -- 172.21.15.5:0/1630729779 wait complete. 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.648+0000 7f50d0f25f40 1 Processor -- start 2024-04-25T10:13:19.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.648+0000 7f50d0f25f40 1 -- start start 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50d0f25f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3d3b710 0x563fc3cf25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50d0f25f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3dad0c0 0x563fc3db69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50d0f25f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dc8890 0x563fc3dc51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50d0f25f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc3dac020 con 0x563fc3dc8890 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50d0f25f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fc3dac350 con 0x563fc3dad0c0 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50d0f25f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fc3dc9420 con 0x563fc3d3b710 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50ce000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dc8890 0x563fc3dc51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50ce000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dc8890 0x563fc3dc51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47198/0 (socket says 172.21.15.5:47198) 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50ce000640 1 -- 172.21.15.5:0/272695685 learned_addr learned my addr 172.21.15.5:0/272695685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50cd600640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3d3b710 0x563fc3cf25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.649+0000 7f50ccc00640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3dad0c0 0x563fc3db69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ce000640 1 -- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3d3b710 msgr2=0x563fc3cf25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ce000640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563fc3d3b710 0x563fc3cf25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ce000640 1 -- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3dad0c0 msgr2=0x563fc3db69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ce000640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3dad0c0 0x563fc3db69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ce000640 1 -- 172.21.15.5:0/272695685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fc3db6f10 con 0x563fc3dc8890 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ccc00640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563fc3dad0c0 0x563fc3db69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50ce000640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dc8890 0x563fc3dc51c0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f50c8006fb0 tx=0x7f50c800d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50c6000640 1 -- 172.21.15.5:0/272695685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8018070 con 0x563fc3dc8890 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50c6000640 1 -- 172.21.15.5:0/272695685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c800de20 con 0x563fc3dc8890 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50c6000640 1 -- 172.21.15.5:0/272695685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8003110 con 0x563fc3dc8890 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563fc3dc5fd0 con 0x563fc3dc8890 2024-04-25T10:13:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.650+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fc3db61d0 con 0x563fc3dc8890 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.653+0000 7f50c6000640 1 -- 172.21.15.5:0/272695685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50c801b070 con 0x563fc3dc8890 2024-04-25T10:13:19.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.654+0000 7f50c6000640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50b0069eb0 0x7f50b006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.654+0000 7f50c6000640 1 -- 172.21.15.5:0/272695685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f50c80883f0 con 0x563fc3dc8890 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.654+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x563fc3e60400 0x563fc3e607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.654+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:1 18.32 18:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x563fc3e60d20 con 0x563fc3e60400 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.654+0000 7f50cd600640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50b0069eb0 0x7f50b006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.655+0000 7f50ccc00640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x563fc3e60400 0x563fc3e607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.656+0000 7f50cd600640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50b0069eb0 0x7f50b006c370 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000f60 tx=0x7f50c0001ce0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.656+0000 7f50ccc00640 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x563fc3e60400 0x563fc3e607e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50ccc00640 1 -- 172.21.15.5:0/272695685 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f50bc066070 con 0x563fc3e60400 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:error getting xattr cephfs_data/10000000003.00000000/scrub_tag: (61) No data available 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x563fc3e60400 msgr2=0x563fc3e607e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x563fc3e60400 0x563fc3e607e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50b0069eb0 msgr2=0x7f50b006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f50b0069eb0 0x7f50b006c370 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000f60 tx=0x7f50c0001ce0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dc8890 msgr2=0x563fc3dc51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 --2- 172.21.15.5:0/272695685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc3dc8890 0x563fc3dc51c0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f50c8006fb0 tx=0x7f50c800d5f0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50ce000640 1 -- 172.21.15.5:0/272695685 reap_dead start 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 shutdown_connections 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 >> 172.21.15.5:0/272695685 conn(0x563fc3cedc60 msgr2=0x563fc3dccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 shutdown_connections 2024-04-25T10:13:19.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.657+0000 7f50d0f25f40 1 -- 172.21.15.5:0/272695685 wait complete. 2024-04-25T10:13:19.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:19.666 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:13:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.768+0000 7fb566e22f40 1 Processor -- start 2024-04-25T10:13:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.768+0000 7fb566e22f40 1 -- start start 2024-04-25T10:13:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb566e22f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e361edb0 0x5605e36aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb566e22f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e3620da0 0x5605e36aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb566e22f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e36acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb566e22f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e357bdd0 con 0x5605e3691b20 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb566e22f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e3692110 con 0x5605e361edb0 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb566e22f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605e368da10 con 0x5605e3620da0 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.769+0000 7fb564a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e36acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e36acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47220/0 (socket says 172.21.15.5:47220) 2024-04-25T10:13:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 -- 172.21.15.5:0/167549031 learned_addr learned my addr 172.21.15.5:0/167549031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 -- 172.21.15.5:0/167549031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e3620da0 msgr2=0x5605e36aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e3620da0 0x5605e36aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 -- 172.21.15.5:0/167549031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e361edb0 msgr2=0x5605e36aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e361edb0 0x5605e36aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 -- 172.21.15.5:0/167549031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e35d3040 con 0x5605e3691b20 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.770+0000 7fb564a00640 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e36acfd0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb55c045ef0 tx=0x7fb55c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6837f215410e6c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb562c00640 1 -- 172.21.15.5:0/167549031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c063070 con 0x5605e3691b20 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb562c00640 1 -- 172.21.15.5:0/167549031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c0558c0 con 0x5605e3691b20 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb562c00640 1 -- 172.21.15.5:0/167549031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c055cb0 con 0x5605e3691b20 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb566e22f40 1 -- 172.21.15.5:0/167549031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 msgr2=0x5605e36acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb566e22f40 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e36acfd0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb55c045ef0 tx=0x7fb55c051000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb566e22f40 1 -- 172.21.15.5:0/167549031 shutdown_connections 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb566e22f40 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e36acfd0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb566e22f40 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e3620da0 0x5605e36aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.771+0000 7fb566e22f40 1 --2- 172.21.15.5:0/167549031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e361edb0 0x5605e36aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.772+0000 7fb566e22f40 1 -- 172.21.15.5:0/167549031 >> 172.21.15.5:0/167549031 conn(0x5605e35d3c60 msgr2=0x5605e35d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.772+0000 7fb566e22f40 1 -- 172.21.15.5:0/167549031 shutdown_connections 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.772+0000 7fb566e22f40 1 -- 172.21.15.5:0/167549031 wait complete. 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.772+0000 7fb566e22f40 1 Processor -- start 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.773+0000 7fb566e22f40 1 -- start start 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.773+0000 7fb566e22f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e361edb0 0x5605e36a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.773+0000 7fb566e22f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e3620da0 0x5605e368df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb566e22f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e368e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb566e22f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e368dc60 con 0x5605e3691b20 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb566e22f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605e3695360 con 0x5605e3620da0 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb566e22f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605e36071a0 con 0x5605e361edb0 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e368e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e368e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47228/0 (socket says 172.21.15.5:47228) 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 -- 172.21.15.5:0/2091103840 learned_addr learned my addr 172.21.15.5:0/2091103840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564000640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e361edb0 0x5605e36a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb55b600640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e3620da0 0x5605e368df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 -- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e361edb0 msgr2=0x5605e36a8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5605e361edb0 0x5605e36a8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 -- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e3620da0 msgr2=0x5605e368df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5605e3620da0 0x5605e368df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.774+0000 7fb564a00640 1 -- 172.21.15.5:0/2091103840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605e36acad0 con 0x5605e3691b20 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.775+0000 7fb564a00640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e368e480 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fb55c025c90 tx=0x7fb55c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.775+0000 7fb561800640 1 -- 172.21.15.5:0/2091103840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c078070 con 0x5605e3691b20 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.775+0000 7fb561800640 1 -- 172.21.15.5:0/2091103840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c055dc0 con 0x5605e3691b20 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.775+0000 7fb561800640 1 -- 172.21.15.5:0/2091103840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c063040 con 0x5605e3691b20 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.775+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5605e368cd80 con 0x5605e3691b20 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.775+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605e36ad020 con 0x5605e3691b20 2024-04-25T10:13:19.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.777+0000 7fb561800640 1 -- 172.21.15.5:0/2091103840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb55c07b070 con 0x5605e3691b20 2024-04-25T10:13:19.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.778+0000 7fb561800640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb540069eb0 0x7fb54006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.778+0000 7fb561800640 1 -- 172.21.15.5:0/2091103840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb55c0cedd0 con 0x5605e3691b20 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.779+0000 7fb566e22f40 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5605e368d4a0 0x5605e36ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.779+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x5605e36a91c0 con 0x5605e368d4a0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.779+0000 7fb564000640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb540069eb0 0x7fb54006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.779+0000 7fb55b600640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5605e368d4a0 0x5605e36ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.780+0000 7fb564000640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb540069eb0 0x7fb54006c370 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb554003420 tx=0x7fb554007040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.780+0000 7fb55b600640 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5605e368d4a0 0x5605e36ad7d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.781+0000 7fb55b600640 1 -- 172.21.15.5:0/2091103840 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7fb54c00c600 con 0x5605e368d4a0 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5605e368d4a0 msgr2=0x5605e36ad7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5605e368d4a0 0x5605e36ad7d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb540069eb0 msgr2=0x7fb54006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb540069eb0 0x7fb54006c370 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb554003420 tx=0x7fb554007040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 msgr2=0x5605e368e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 --2- 172.21.15.5:0/2091103840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605e3691b20 0x5605e368e480 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fb55c025c90 tx=0x7fb55c002fb0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb564a00640 1 -- 172.21.15.5:0/2091103840 reap_dead start 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 shutdown_connections 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 >> 172.21.15.5:0/2091103840 conn(0x5605e35d3c60 msgr2=0x5605e36abea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 shutdown_connections 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.782+0000 7fb566e22f40 1 -- 172.21.15.5:0/2091103840 wait complete. 2024-04-25T10:13:19.791 INFO:teuthology.orchestra.run.smithi005.stdout:mytag123 2024-04-25T10:13:19.791 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T10:13:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.882+0000 7f14ae51df40 1 Processor -- start 2024-04-25T10:13:19.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.882+0000 7f14ae51df40 1 -- start start 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.883+0000 7f14ae51df40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 0x560484b0be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.883+0000 7f14ae51df40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b95790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.883+0000 7f14ae51df40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.883+0000 7f14ae51df40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560484b26380 con 0x560484b96b20 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.883+0000 7f14ae51df40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560484b26620 con 0x560484b25bd0 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.883+0000 7f14ae51df40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560484b268c0 con 0x560484b23db0 2024-04-25T10:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b939e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47254/0 (socket says 172.21.15.5:47254) 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 -- 172.21.15.5:0/3310281091 learned_addr learned my addr 172.21.15.5:0/3310281091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14aae00640 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b95790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 -- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 msgr2=0x560484b0be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 0x560484b0be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 -- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 msgr2=0x560484b95790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b95790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 -- 172.21.15.5:0/3310281091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560484a463b0 con 0x560484b96b20 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14aae00640 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b95790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.884+0000 7f14ac200640 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b939e0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f14a4025b20 tx=0x7f14a4050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a07d8e0ca4ad4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14a9a00640 1 -- 172.21.15.5:0/3310281091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4054040 con 0x560484b96b20 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14a9a00640 1 -- 172.21.15.5:0/3310281091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4050980 con 0x560484b96b20 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14a9a00640 1 -- 172.21.15.5:0/3310281091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4050d50 con 0x560484b96b20 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 -- 172.21.15.5:0/3310281091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 msgr2=0x560484b939e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b939e0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f14a4025b20 tx=0x7f14a4050000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 -- 172.21.15.5:0/3310281091 shutdown_connections 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b939e0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b95790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 --2- 172.21.15.5:0/3310281091 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 0x560484b0be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 -- 172.21.15.5:0/3310281091 >> 172.21.15.5:0/3310281091 conn(0x560484ad8c60 msgr2=0x560484b0acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 -- 172.21.15.5:0/3310281091 shutdown_connections 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.885+0000 7f14ae51df40 1 -- 172.21.15.5:0/3310281091 wait complete. 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.886+0000 7f14ae51df40 1 Processor -- start 2024-04-25T10:13:19.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.886+0000 7f14ae51df40 1 -- start start 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ae51df40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 0x560484b996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ae51df40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b91a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ae51df40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b94040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ae51df40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560484b0b560 con 0x560484b96b20 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ae51df40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x560484ad9270 con 0x560484b25bd0 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ae51df40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x560484ad94e0 con 0x560484b23db0 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ac200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b94040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ac200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b94040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47258/0 (socket says 172.21.15.5:47258) 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.887+0000 7f14ac200640 1 -- 172.21.15.5:0/4119524294 learned_addr learned my addr 172.21.15.5:0/4119524294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:19.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.888+0000 7f14ac200640 1 -- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 msgr2=0x560484b996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.888+0000 7f14ac200640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x560484b23db0 0x560484b996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.888+0000 7f14ac200640 1 -- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 msgr2=0x560484b91a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.888+0000 7f14ac200640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x560484b25bd0 0x560484b91a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.888+0000 7f14ac200640 1 -- 172.21.15.5:0/4119524294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560484b91f80 con 0x560484b96b20 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.888+0000 7f14ac200640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b94040 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f14a400a190 tx=0x7f14a4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.889+0000 7f14a3e00640 1 -- 172.21.15.5:0/4119524294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4054050 con 0x560484b96b20 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.889+0000 7f14a3e00640 1 -- 172.21.15.5:0/4119524294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4059770 con 0x560484b96b20 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.889+0000 7f14a3e00640 1 -- 172.21.15.5:0/4119524294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4059b40 con 0x560484b96b20 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.889+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560484b92310 con 0x560484b96b20 2024-04-25T10:13:19.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.889+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560484b0ae70 con 0x560484b96b20 2024-04-25T10:13:19.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.891+0000 7f14a3e00640 1 -- 172.21.15.5:0/4119524294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14a4057070 con 0x560484b96b20 2024-04-25T10:13:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.892+0000 7f14a3e00640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1488069f30 0x7f148806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.892+0000 7f14a3e00640 1 -- 172.21.15.5:0/4119524294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f14a40ce8a0 con 0x560484b96b20 2024-04-25T10:13:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.892+0000 7f14ae51df40 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x560484b9a1c0 0x560484ba82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.892+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x560484ba8820 con 0x560484b9a1c0 2024-04-25T10:13:19.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.892+0000 7f14ab800640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1488069f30 0x7f148806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.893+0000 7f14aae00640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x560484b9a1c0 0x560484ba82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.894+0000 7f14ab800640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1488069f30 0x7f148806c3f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x560484badac0 tx=0x7f149c005600 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.894+0000 7f14aae00640 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x560484b9a1c0 0x560484ba82e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14aae00640 1 -- 172.21.15.5:0/4119524294 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f1494007670 con 0x560484b9a1c0 2024-04-25T10:13:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x560484b9a1c0 msgr2=0x560484ba82e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x560484b9a1c0 0x560484ba82e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1488069f30 msgr2=0x7f148806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1488069f30 0x7f148806c3f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x560484badac0 tx=0x7f149c005600 comp rx=0 tx=0).stop 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 msgr2=0x560484b94040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 --2- 172.21.15.5:0/4119524294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560484b96b20 0x560484b94040 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f14a400a190 tx=0x7f14a4003330 comp rx=0 tx=0).stop 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ac200640 1 -- 172.21.15.5:0/4119524294 reap_dead start 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 shutdown_connections 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 >> 172.21.15.5:0/4119524294 conn(0x560484ad8c60 msgr2=0x560484b95790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 shutdown_connections 2024-04-25T10:13:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:19.895+0000 7f14ae51df40 1 -- 172.21.15.5:0/4119524294 wait complete. 2024-04-25T10:13:19.904 INFO:teuthology.orchestra.run.smithi005.stdout:mytag123 2024-04-25T10:13:19.904 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000006.00000000 scrub_tag 2024-04-25T10:13:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.011+0000 7f1869f08f40 1 Processor -- start 2024-04-25T10:13:20.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.011+0000 7f1869f08f40 1 -- start start 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.012+0000 7f1869f08f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x562135246710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.012+0000 7f1869f08f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562135261da0 0x5621352e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.012+0000 7f1869f08f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5621352d2b20 0x5621352eb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.012+0000 7f1869f08f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621351bcdd0 con 0x56213525fdb0 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.012+0000 7f1869f08f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562135262380 con 0x562135261da0 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.012+0000 7f1869f08f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562135214040 con 0x5621352d2b20 2024-04-25T10:13:20.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x562135246710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x562135246710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47262/0 (socket says 172.21.15.5:47262) 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 -- 172.21.15.5:0/3501311969 learned_addr learned my addr 172.21.15.5:0/3501311969 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f185fe00640 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562135261da0 0x5621352e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867c00640 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5621352d2b20 0x5621352eb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 -- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5621352d2b20 msgr2=0x5621352eb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5621352d2b20 0x5621352eb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 -- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562135261da0 msgr2=0x5621352e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562135261da0 0x5621352e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.013+0000 7f1867200640 1 -- 172.21.15.5:0/3501311969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562135247b90 con 0x56213525fdb0 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.014+0000 7f1867200640 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x562135246710 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f1858000c00 tx=0x7f1858051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d28b513828b91c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.014+0000 7f1865e00640 1 -- 172.21.15.5:0/3501311969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858053880 con 0x56213525fdb0 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.014+0000 7f1865e00640 1 -- 172.21.15.5:0/3501311969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858053a40 con 0x56213525fdb0 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.014+0000 7f1865e00640 1 -- 172.21.15.5:0/3501311969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858053e30 con 0x56213525fdb0 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 -- 172.21.15.5:0/3501311969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 msgr2=0x562135246710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x562135246710 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f1858000c00 tx=0x7f1858051000 comp rx=0 tx=0).stop 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 -- 172.21.15.5:0/3501311969 shutdown_connections 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5621352d2b20 0x5621352eb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x562135261da0 0x5621352e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3501311969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x562135246710 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 -- 172.21.15.5:0/3501311969 >> 172.21.15.5:0/3501311969 conn(0x562135214c60 msgr2=0x562135215040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 -- 172.21.15.5:0/3501311969 shutdown_connections 2024-04-25T10:13:20.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.015+0000 7f1869f08f40 1 -- 172.21.15.5:0/3501311969 wait complete. 2024-04-25T10:13:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.016+0000 7f1869f08f40 1 Processor -- start 2024-04-25T10:13:20.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.016+0000 7f1869f08f40 1 -- start start 2024-04-25T10:13:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1869f08f40 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x5621352e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1869f08f40 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562135261da0 0x5621352422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1869f08f40 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5621352d2b20 0x562135242800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1869f08f40 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621352461f0 con 0x56213525fdb0 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1869f08f40 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x562135245f10 con 0x5621352d2b20 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1869f08f40 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x562135214290 con 0x562135261da0 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1867200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x5621352e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1867200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x5621352e4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47264/0 (socket says 172.21.15.5:47264) 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1867200640 1 -- 172.21.15.5:0/3100946989 learned_addr learned my addr 172.21.15.5:0/3100946989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f1867c00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5621352d2b20 0x562135242800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.017+0000 7f185fe00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562135261da0 0x5621352422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1867200640 1 -- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562135261da0 msgr2=0x5621352422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1867200640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x562135261da0 0x5621352422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1867200640 1 -- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5621352d2b20 msgr2=0x562135242800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1867200640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5621352d2b20 0x562135242800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1867200640 1 -- 172.21.15.5:0/3100946989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621352cddc0 con 0x56213525fdb0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1867200640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x5621352e4b10 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f1858045f50 tx=0x7f1858002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1864a00640 1 -- 172.21.15.5:0/3100946989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858073020 con 0x56213525fdb0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1864a00640 1 -- 172.21.15.5:0/3100946989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858067430 con 0x56213525fdb0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1864a00640 1 -- 172.21.15.5:0/3100946989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858067820 con 0x56213525fdb0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.018+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5621352ce0a0 con 0x56213525fdb0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.019+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621352ce660 con 0x56213525fdb0 2024-04-25T10:13:20.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.020+0000 7f1864a00640 1 -- 172.21.15.5:0/3100946989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1858076020 con 0x56213525fdb0 2024-04-25T10:13:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.021+0000 7f1864a00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1844069eb0 0x7f184406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.021+0000 7f1864a00640 1 -- 172.21.15.5:0/3100946989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f18580cfac0 con 0x56213525fdb0 2024-04-25T10:13:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.022+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5621352e5050 0x562135242d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.022+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:1 18.25 18:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) v9 -- 0x5621352c9ce0 con 0x5621352e5050 2024-04-25T10:13:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.022+0000 7f185fe00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1844069eb0 0x7f184406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.022+0000 7f1867c00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5621352e5050 0x562135242d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.023+0000 7f1867c00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5621352e5050 0x562135242d40 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.023+0000 7f1867c00640 1 -- 172.21.15.5:0/3100946989 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (12) out=12b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f1860002920 con 0x5621352e5050 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.023+0000 7f185fe00640 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1844069eb0 0x7f184406c370 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5621352e56c0 tx=0x7f1850002c40 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.023+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5621352e5050 msgr2=0x562135242d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5621352e5050 0x562135242d40 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1844069eb0 msgr2=0x7f184406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1844069eb0 0x7f184406c370 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5621352e56c0 tx=0x7f1850002c40 comp rx=0 tx=0).stop 2024-04-25T10:13:20.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 msgr2=0x5621352e4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 --2- 172.21.15.5:0/3100946989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213525fdb0 0x5621352e4b10 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f1858045f50 tx=0x7f1858002fb0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1867c00640 1 -- 172.21.15.5:0/3100946989 reap_dead start 2024-04-25T10:13:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 shutdown_connections 2024-04-25T10:13:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 >> 172.21.15.5:0/3100946989 conn(0x562135214c60 msgr2=0x562135215040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 shutdown_connections 2024-04-25T10:13:20.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.024+0000 7f1869f08f40 1 -- 172.21.15.5:0/3100946989 wait complete. 2024-04-25T10:13:20.033 INFO:teuthology.orchestra.run.smithi005.stdout:mytag123 2024-04-25T10:13:20.033 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:13:20.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.235+0000 7f6881400640 1 Processor -- start 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.236+0000 7f6881400640 1 -- start start 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.237+0000 7f6881400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f687c15ec80 0x7f687c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.237+0000 7f6881400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c15ffa0 con 0x7f687c1616f0 2024-04-25T10:13:20.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.237+0000 7f6881400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c160240 con 0x7f687c0d4000 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.237+0000 7f6881400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c160520 con 0x7f687c15ec80 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.238+0000 7f6880a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f687c1616f0 0x7f687c161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37922/0 (socket says 172.21.15.5:37922) 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.238+0000 7f6880a00640 1 -- 172.21.15.5:0/459319930 learned_addr learned my addr 172.21.15.5:0/459319930 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.238+0000 7f687be00640 1 --2- 172.21.15.5:0/459319930 >> v2:172.21.15.80:3300/0 conn(0x7f687c15ec80 0x7f687c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.239+0000 7f687aa00640 1 -- 172.21.15.5:0/459319930 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454704871 0 0) 0x7f687c15ffa0 con 0x7f687c1616f0 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.239+0000 7f687aa00640 1 -- 172.21.15.5:0/459319930 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f685c003520 con 0x7f687c1616f0 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.239+0000 7f687be00640 1 -- 172.21.15.5:0/459319930 >> v1:172.21.15.80:6789/0 conn(0x7f687c0d4000 legacy=0x7f687c160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.239+0000 7f687be00640 1 -- 172.21.15.5:0/459319930 >> v1:172.21.15.5:6789/0 conn(0x7f687c1616f0 legacy=0x7f687c161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.239+0000 7f687aa00640 1 -- 172.21.15.5:0/459319930 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054459236 0 0) 0x7f685c003520 con 0x7f687c1616f0 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.239+0000 7f687be00640 1 -- 172.21.15.5:0/459319930 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c160800 con 0x7f687c15ec80 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.240+0000 7f687be00640 1 --2- 172.21.15.5:0/459319930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c15ec80 0x7f687c0b45c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f686c009790 tx=0x7f686c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b0c307bee56cb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.241+0000 7f687aa00640 1 -- 172.21.15.5:0/459319930 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c037070 con 0x7f687c15ec80 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.241+0000 7f687aa00640 1 -- 172.21.15.5:0/459319930 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c02ac30 con 0x7f687c15ec80 2024-04-25T10:13:20.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.241+0000 7f687aa00640 1 -- 172.21.15.5:0/459319930 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874001690 con 0x7f687c15ec80 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.241+0000 7f6881400640 1 -- 172.21.15.5:0/459319930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c15ec80 msgr2=0x7f687c0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.241+0000 7f6881400640 1 --2- 172.21.15.5:0/459319930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c15ec80 0x7f687c0b45c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f686c009790 tx=0x7f686c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.242+0000 7f6881400640 1 -- 172.21.15.5:0/459319930 shutdown_connections 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.242+0000 7f6881400640 1 --2- 172.21.15.5:0/459319930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c15ec80 0x7f687c0b45c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.242+0000 7f6881400640 1 -- 172.21.15.5:0/459319930 >> 172.21.15.5:0/459319930 conn(0x7f687c08af80 msgr2=0x7f687c0d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.242+0000 7f6881400640 1 -- 172.21.15.5:0/459319930 shutdown_connections 2024-04-25T10:13:20.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.242+0000 7f6881400640 1 -- 172.21.15.5:0/459319930 wait complete. 2024-04-25T10:13:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.243+0000 7f6881400640 1 Processor -- start 2024-04-25T10:13:20.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.244+0000 7f6881400640 1 -- start start 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6881400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c0d4000 0x7f687c0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6881400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f687c15ec80 0x7f687c0b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6881400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 0x7f687c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6881400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0b4c50 con 0x7f687c1616f0 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6881400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c15f060 con 0x7f687c0d4000 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6881400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0d4490 con 0x7f687c15ec80 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 0x7f687c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 0x7f687c0cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47276/0 (socket says 172.21.15.5:47276) 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 -- 172.21.15.5:0/257216147 learned_addr learned my addr 172.21.15.5:0/257216147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f687be00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c0d4000 0x7f687c0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f687b400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f687c15ec80 0x7f687c0b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 -- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f687c15ec80 msgr2=0x7f687c0b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f687c15ec80 0x7f687c0b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 -- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c0d4000 msgr2=0x7f687c0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c0d4000 0x7f687c0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.245+0000 7f6880a00640 1 -- 172.21.15.5:0/257216147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c0cbe40 con 0x7f687c1616f0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.246+0000 7f6880a00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 0x7f687c0cb900 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f6874001200 tx=0x7f687400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.246+0000 7f6873e00640 1 -- 172.21.15.5:0/257216147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874018070 con 0x7f687c1616f0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.246+0000 7f6873e00640 1 -- 172.21.15.5:0/257216147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68740074e0 con 0x7f687c1616f0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.246+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c0b6ac0 con 0x7f687c1616f0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.246+0000 7f6873e00640 1 -- 172.21.15.5:0/257216147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740049e0 con 0x7f687c1616f0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.246+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c0b7080 con 0x7f687c1616f0 2024-04-25T10:13:20.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.248+0000 7f6873e00640 1 -- 172.21.15.5:0/257216147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f687401b070 con 0x7f687c1616f0 2024-04-25T10:13:20.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.249+0000 7f6873e00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f685806b4d0 0x7f685806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.249+0000 7f6873e00640 1 -- 172.21.15.5:0/257216147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6874089880 con 0x7f687c1616f0 2024-04-25T10:13:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.250+0000 7f6885787740 1 Processor -- start 2024-04-25T10:13:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.250+0000 7f687be00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f685806b4d0 0x7f685806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.250+0000 7f6885787740 1 -- start start 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f687be00640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f685806b4d0 0x7f685806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f686c034000 tx=0x7f686c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eb8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f6885787740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8eb8c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebb6830 0x55eb8ebb6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f6885787740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8eaefc70 con 0x55eb8ebc8440 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8ea98cf0 con 0x55eb8ebb6830 2024-04-25T10:13:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.251+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb8eaebd10 con 0x55eb8eb8bd70 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8eb8c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8eb8c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47292/0 (socket says 172.21.15.5:47292) 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 -- 172.21.15.5:0/964046048 learned_addr learned my addr 172.21.15.5:0/964046048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687be00640 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eb8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f6880a00640 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebb6830 0x55eb8ebb6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 -- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 msgr2=0x55eb8eb8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eb8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 -- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebb6830 msgr2=0x55eb8ebb6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebb6830 0x55eb8ebb6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 -- 172.21.15.5:0/964046048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb8e8ce7d0 con 0x55eb8ebc8440 2024-04-25T10:13:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.252+0000 7f687b400640 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8eb8c150 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f6864001c70 tx=0x7f6864002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fb8cf193d7b11a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.253+0000 7f686be00640 1 -- 172.21.15.5:0/964046048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864019070 con 0x55eb8ebc8440 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.253+0000 7f686be00640 1 -- 172.21.15.5:0/964046048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864003bd0 con 0x55eb8ebc8440 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.253+0000 7f686be00640 1 -- 172.21.15.5:0/964046048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400e800 con 0x55eb8ebc8440 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.253+0000 7f6885787740 1 -- 172.21.15.5:0/964046048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 msgr2=0x55eb8eb8c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.253+0000 7f6885787740 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8eb8c150 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f6864001c70 tx=0x7f6864002130 comp rx=0 tx=0).stop 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 -- 172.21.15.5:0/964046048 shutdown_connections 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebb6830 0x55eb8ebb6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8eb8c150 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 --2- 172.21.15.5:0/964046048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eb8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 -- 172.21.15.5:0/964046048 >> 172.21.15.5:0/964046048 conn(0x55eb8eb53a30 msgr2=0x55eb8ebd1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 -- 172.21.15.5:0/964046048 shutdown_connections 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 -- 172.21.15.5:0/964046048 wait complete. 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.254+0000 7f6885787740 1 -- start start 2024-04-25T10:13:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.255+0000 7f6885787740 1 Processor -- start 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eba3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebcaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebcb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8ea8e3a0 con 0x55eb8ebb6830 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8ebb4380 con 0x55eb8ebc8440 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb8eba4700 con 0x55eb8eb8bd70 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6885787740 10 client.0 _ll_register_callbacks cb 0x55eb8eaabb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebcaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebcaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47302/0 (socket says 172.21.15.5:47302) 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 -- 172.21.15.5:0/3113156536 learned_addr learned my addr 172.21.15.5:0/3113156536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687be00640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eba3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f6880a00640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebcb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 -- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 msgr2=0x55eb8eba3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eba3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 -- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 msgr2=0x55eb8ebcb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.256+0000 7f687b400640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebcb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f687b400640 1 -- 172.21.15.5:0/3113156536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb8e8043f0 con 0x55eb8ebb6830 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f687be00640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eba3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6880a00640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebcb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f687b400640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebcaff0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6864002280 tx=0x7f68640012a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65161acf881d9e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6868c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6868c00640 1 -- 172.21.15.5:0/3113156536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864023060 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6868c00640 1 -- 172.21.15.5:0/3113156536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6864010920 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6864003bd0 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6868c00640 1 -- 172.21.15.5:0/3113156536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6864029050 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6885787740 10 client.9384 fetch_fsmap learned FSMap version 105 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.257+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55eb8e7c40f0 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6868c00640 1 -- 172.21.15.5:0/3113156536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864003bd0 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6868c00640 1 -- 172.21.15.5:0/3113156536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f686402c020 con 0x55eb8ebb6830 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6885787740 10 client.9384 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6885787740 10 client.9384 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6885787740 20 client.9384 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6885787740 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebb7e30 0x55eb8eba3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6885787740 4 client.9384 mds_command: new command op to 7000 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 --> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb8eafd1b0 con 0x55eb8ebb7e30 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.258+0000 7f687be00640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebb7e30 0x55eb8eba3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.259+0000 7f687be00640 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebb7e30 0x55eb8eba3d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b097ec70ccb24b server_cookie=2a768d0af0721509 in_seq=0 out_seq=0 2024-04-25T10:13:20.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.259+0000 7f6868c00640 10 client.9384 ms_handle_connect on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6868c00640 1 -- 172.21.15.5:0/3113156536 <== mds.0 v2:172.21.15.5:6833/2518129896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb8eafd1b0 con 0x55eb8ebb7e30 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6868c00640 10 client.9384 handle_command_reply: tid=0 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 client.9384 shutdown 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 msgr2=0x55eb8ebcaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebcaff0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6864002280 tx=0x7f68640012a0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 shutdown_connections 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebb7e30 0x55eb8eba3d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebcb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebcaff0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 --2- 172.21.15.5:0/3113156536 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8eba3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6868c00640 0 client.9384 ms_handle_reset on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 >> 172.21.15.5:0/3113156536 conn(0x55eb8eb53a30 msgr2=0x55eb8ebb4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 shutdown_connections 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 1 -- 172.21.15.5:0/3113156536 wait complete. 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.260+0000 7f6885787740 20 client.9384 trim_cache size 0 max 16384 2024-04-25T10:13:20.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.276+0000 7f6885787740 1 Processor -- start 2024-04-25T10:13:20.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.276+0000 7f6885787740 1 -- start start 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8ebb62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6885787740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebd44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebb9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6885787740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c200110 con 0x55eb8ebb6830 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c1ff340 con 0x55eb8ebc8440 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0cb450 con 0x55eb8eb8bd70 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f687b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebd44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f687b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebd44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47308/0 (socket says 172.21.15.5:47308) 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f687b400640 1 -- 172.21.15.5:0/1968475678 learned_addr learned my addr 172.21.15.5:0/1968475678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.277+0000 7f6880a00640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebb9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687b400640 1 -- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 msgr2=0x55eb8ebb62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687be00640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8ebb62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687b400640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8ebb62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687b400640 1 -- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 msgr2=0x55eb8ebb9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687b400640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebb9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687b400640 1 -- 172.21.15.5:0/1968475678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb8eba2fe0 con 0x55eb8ebb6830 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f6880a00640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebb9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687be00640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8ebb62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f687b400640 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebd44e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f68640016e0 tx=0x7f686401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2a07bd1c1cf544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f686b400640 1 -- 172.21.15.5:0/1968475678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864019040 con 0x55eb8ebb6830 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f686b400640 1 -- 172.21.15.5:0/1968475678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864032070 con 0x55eb8ebb6830 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.278+0000 7f686b400640 1 -- 172.21.15.5:0/1968475678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400ea00 con 0x55eb8ebb6830 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 -- 172.21.15.5:0/1968475678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 msgr2=0x55eb8ebd44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebd44e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f68640016e0 tx=0x7f686401c000 comp rx=0 tx=0).stop 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 -- 172.21.15.5:0/1968475678 shutdown_connections 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8ebc8440 0x55eb8ebb9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebb6830 0x55eb8ebd44e0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 --2- 172.21.15.5:0/1968475678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8eb8bd70 0x55eb8ebb62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 -- 172.21.15.5:0/1968475678 >> 172.21.15.5:0/1968475678 conn(0x55eb8eb53a30 msgr2=0x55eb8eb85bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.279+0000 7f6885787740 1 -- 172.21.15.5:0/1968475678 shutdown_connections 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.280+0000 7f6885787740 1 -- 172.21.15.5:0/1968475678 wait complete. 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.280+0000 7f6885787740 1 -- start start 2024-04-25T10:13:20.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.280+0000 7f6885787740 1 Processor -- start 2024-04-25T10:13:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.281+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8eb8bd70 0x55eb8ec3b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.281+0000 7f6885787740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8ebb6830 0x55eb8ebead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.281+0000 7f6885787740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8ebeb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6885787740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8eaebe90 con 0x55eb8ebc8440 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c157de0 con 0x55eb8eb8bd70 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6885787740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb8ebb8280 con 0x55eb8ebb6830 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6885787740 10 client.0 _ll_register_callbacks cb 0x7f6864017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8ebeb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8ebeb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47320/0 (socket says 172.21.15.5:47320) 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 -- 172.21.15.5:0/1281227276 learned_addr learned my addr 172.21.15.5:0/1281227276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f687b400640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8ebb6830 0x55eb8ebead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 -- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8ebb6830 msgr2=0x55eb8ebead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8ebb6830 0x55eb8ebead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 -- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8eb8bd70 msgr2=0x55eb8ec3b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f687be00640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8eb8bd70 0x55eb8ec3b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8eb8bd70 0x55eb8ec3b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.282+0000 7f6880a00640 1 -- 172.21.15.5:0/1281227276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb8ebba7e0 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6880a00640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8ebeb270 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6874057da0 tx=0x7f6874058510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d61b4607d8df25f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6868c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6868c00640 1 -- 172.21.15.5:0/1281227276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687405bf60 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6868c00640 1 -- 172.21.15.5:0/1281227276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6874004b80 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6864032070 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6868c00640 1 -- 172.21.15.5:0/1281227276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6874060520 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 10 client.9390 fetch_fsmap learned FSMap version 105 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55eb8ec3be20 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6868c00640 1 -- 172.21.15.5:0/1281227276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864032070 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6868c00640 1 -- 172.21.15.5:0/1281227276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f687406fba0 con 0x55eb8ebc8440 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 10 client.9390 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 10 client.9390 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:13:20.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 20 client.9390 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebcbde0 0x55eb8ebb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.283+0000 7f6885787740 4 client.9390 mds_command: new command op to 7000 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:13:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.284+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 --> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eb8ebb9c00 con 0x55eb8ebcbde0 2024-04-25T10:13:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.284+0000 7f687be00640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebcbde0 0x55eb8ebb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.284+0000 7f687be00640 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebcbde0 0x55eb8ebb2fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4a0d18321ed572 server_cookie=18eb77c9325d17f6 in_seq=0 out_seq=0 2024-04-25T10:13:20.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.284+0000 7f6868c00640 10 client.9390 ms_handle_connect on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6868c00640 1 -- 172.21.15.5:0/1281227276 <== mds.0 v2:172.21.15.5:6833/2518129896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eb8ebb9c00 con 0x55eb8ebcbde0 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6868c00640 10 client.9390 handle_command_reply: tid=0 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 client.9390 shutdown 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 msgr2=0x55eb8ebeb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8ebeb270 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6874057da0 tx=0x7f6874058510 comp rx=0 tx=0).stop 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 shutdown_connections 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:6833/2518129896,v1:172.21.15.5:6835/2518129896] conn(0x55eb8ebcbde0 0x55eb8ebb2fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb8ebc8440 0x55eb8ebeb270 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55eb8ebb6830 0x55eb8ebead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 --2- 172.21.15.5:0/1281227276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55eb8eb8bd70 0x55eb8ec3b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6868c00640 0 client.9390 ms_handle_reset on v2:172.21.15.5:6833/2518129896 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.309+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 >> 172.21.15.5:0/1281227276 conn(0x55eb8eb53a30 msgr2=0x55eb8ebf31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.310+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 shutdown_connections 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.310+0000 7f6885787740 1 -- 172.21.15.5:0/1281227276 wait complete. 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.310+0000 7f6885787740 20 client.9390 trim_cache size 0 max 16384 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stdout: "message": "", 2024-04-25T10:13:20.315 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0 2024-04-25T10:13:20.316 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f685806b4d0 msgr2=0x7f685806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f685806b4d0 0x7f685806d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f686c034000 tx=0x7f686c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 msgr2=0x7f687c0cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 0x7f687c0cb900 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f6874001200 tx=0x7f687400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 shutdown_connections 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f687c1616f0 0x7f687c0cb900 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f687c15ec80 0x7f687c0b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f685806b4d0 0x7f685806d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 --2- 172.21.15.5:0/257216147 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f687c0d4000 0x7f687c0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 >> 172.21.15.5:0/257216147 conn(0x7f687c08af80 msgr2=0x7f687c0cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 shutdown_connections 2024-04-25T10:13:20.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.311+0000 7f6881400640 1 -- 172.21.15.5:0/257216147 wait complete. 2024-04-25T10:13:20.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.545+0000 7f2a9f200640 1 Processor -- start 2024-04-25T10:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.545+0000 7f2a9f200640 1 -- start start 2024-04-25T10:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.546+0000 7f2a9f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a980d07d0 0x7f2a980b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.546+0000 7f2a9f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2a980d2380 0x7f2a980b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.546+0000 7f2a9f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a980cf410 con 0x7f2a98161200 2024-04-25T10:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980b28b0 con 0x7f2a980d07d0 2024-04-25T10:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980cf6f0 con 0x7f2a980d2380 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a980d07d0 0x7f2a980b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2a980d07d0 0x7f2a980b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47322/0 (socket says 172.21.15.5:47322) 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 -- 172.21.15.5:0/1577481538 learned_addr learned my addr 172.21.15.5:0/1577481538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9e800640 1 --2- 172.21.15.5:0/1577481538 >> v2:172.21.15.80:3300/0 conn(0x7f2a980d2380 0x7f2a980b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 -- 172.21.15.5:0/1577481538 >> v2:172.21.15.80:3300/0 conn(0x7f2a980d2380 msgr2=0x7f2a980b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 --2- 172.21.15.5:0/1577481538 >> v2:172.21.15.80:3300/0 conn(0x7f2a980d2380 0x7f2a980b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 -- 172.21.15.5:0/1577481538 >> v1:172.21.15.80:6789/0 conn(0x7f2a98161200 legacy=0x7f2a980d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.547+0000 7f2a9d400640 1 -- 172.21.15.5:0/1577481538 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980d3fd0 con 0x7f2a980d07d0 2024-04-25T10:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.548+0000 7f2a9d400640 1 --2- 172.21.15.5:0/1577481538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a980b4bb0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2a980cfc80 tx=0x7f2a8802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48819f7c4a15ded0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.549+0000 7f2a9ca00640 1 -- 172.21.15.5:0/1577481538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88037070 con 0x7f2a980d07d0 2024-04-25T10:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.549+0000 7f2a9ca00640 1 -- 172.21.15.5:0/1577481538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a88032440 con 0x7f2a980d07d0 2024-04-25T10:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.549+0000 7f2a9ca00640 1 -- 172.21.15.5:0/1577481538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88032810 con 0x7f2a980d07d0 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.549+0000 7f2a9f200640 1 -- 172.21.15.5:0/1577481538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 msgr2=0x7f2a980b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.549+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1577481538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a980b4bb0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f2a980cfc80 tx=0x7f2a8802a230 comp rx=0 tx=0).stop 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.550+0000 7f2a9f200640 1 -- 172.21.15.5:0/1577481538 shutdown_connections 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.550+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1577481538 >> v2:172.21.15.80:3300/0 conn(0x7f2a980d2380 0x7f2a980b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.550+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1577481538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a980b4bb0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.550+0000 7f2a9f200640 1 -- 172.21.15.5:0/1577481538 >> 172.21.15.5:0/1577481538 conn(0x7f2a9808af80 msgr2=0x7f2a980d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.550+0000 7f2a9f200640 1 -- 172.21.15.5:0/1577481538 shutdown_connections 2024-04-25T10:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.550+0000 7f2a9f200640 1 -- 172.21.15.5:0/1577481538 wait complete. 2024-04-25T10:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.552+0000 7f2a9f200640 1 Processor -- start 2024-04-25T10:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.552+0000 7f2a9f200640 1 -- start start 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.553+0000 7f2a9f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a9815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a980d2380 0x7f2a980cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a98161200 0x7f2a980ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980b3640 con 0x7f2a980d07d0 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980b34c0 con 0x7f2a98161200 2024-04-25T10:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980b37c0 con 0x7f2a980d2380 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a9815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a9815b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47330/0 (socket says 172.21.15.5:47330) 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 -- 172.21.15.5:0/1047906085 learned_addr learned my addr 172.21.15.5:0/1047906085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 -- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a980d2380 msgr2=0x7f2a980cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a980d2380 0x7f2a980cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 -- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a98161200 msgr2=0x7f2a980ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a98161200 0x7f2a980ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.554+0000 7f2a9de00640 1 -- 172.21.15.5:0/1047906085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a980cacc0 con 0x7f2a980d07d0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.555+0000 7f2a9de00640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a9815b0a0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b3e0 tx=0x7f2a8c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.555+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c015020 con 0x7f2a980d07d0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.555+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c011850 con 0x7f2a980d07d0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.555+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c011c40 con 0x7f2a980d07d0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.555+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9815b7a0 con 0x7f2a980d07d0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.555+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a980b62d0 con 0x7f2a980d07d0 2024-04-25T10:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.556+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a98068f60 con 0x7f2a980d07d0 2024-04-25T10:13:20.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.557+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a8c018020 con 0x7f2a980d07d0 2024-04-25T10:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.558+0000 7f2a92000640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a7c06b4d0 0x7f2a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.558+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2a8c08a3a0 con 0x7f2a980d07d0 2024-04-25T10:13:20.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.559+0000 7f2a9d400640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a7c06b4d0 0x7f2a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.560+0000 7f2a9d400640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a7c06b4d0 0x7f2a7c06d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2a88034000 tx=0x7f2a8802c000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.563+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a8c01f020 con 0x7f2a980d07d0 2024-04-25T10:13:20.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.801+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2a9806acf0 con 0x7f2a980d07d0 2024-04-25T10:13:20.806 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:13:20.802+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.805+0000 7f2a92000640 1 -- 172.21.15.5:0/1047906085 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2a8c0172c0 con 0x7f2a980d07d0 2024-04-25T10:13:20.809 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a7c06b4d0 msgr2=0x7f2a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a7c06b4d0 0x7f2a7c06d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2a88034000 tx=0x7f2a8802c000 comp rx=0 tx=0).stop 2024-04-25T10:13:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 msgr2=0x7f2a9815b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a9815b0a0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00b3e0 tx=0x7f2a8c00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 shutdown_connections 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2a7c06b4d0 0x7f2a7c06d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2a98161200 0x7f2a980ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.808+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2a980d2380 0x7f2a980cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.809+0000 7f2a9f200640 1 --2- 172.21.15.5:0/1047906085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a980d07d0 0x7f2a9815b0a0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.809+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 >> 172.21.15.5:0/1047906085 conn(0x7f2a9808af80 msgr2=0x7f2a9815ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.810+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 shutdown_connections 2024-04-25T10:13:20.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.810+0000 7f2a9f200640 1 -- 172.21.15.5:0/1047906085 wait complete. 2024-04-25T10:13:20.823 INFO:tasks.ceph.mds.c.smithi005.stderr: -1881> 2024-04-25T10:13:12.448+0000 7f038c800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:20.834 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T10:13:20.872 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:13:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.915+0000 7f33d56f7280 1 -- start start 2024-04-25T10:13:20.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.915+0000 7f33d56f7280 1 Processor -- start 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 0x556af09c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af095b800 0x556af09c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af095bc00 0x556af09c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d56f7280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172a180 con 0x556af095ac00 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172a300 con 0x556af095b800 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af172a480 con 0x556af095bc00 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 0x556af09c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 0x556af09c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47358/0 (socket says 172.21.15.5:47358) 2024-04-25T10:13:20.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 -- 172.21.15.5:0/449486520 learned_addr learned my addr 172.21.15.5:0/449486520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d2e00640 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af095b800 0x556af09c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d4200640 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af095bc00 0x556af09c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 -- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af095bc00 msgr2=0x556af09c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af095bc00 0x556af09c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 -- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af095b800 msgr2=0x556af09c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af095b800 0x556af09c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.916+0000 7f33d3800640 1 -- 172.21.15.5:0/449486520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556af098e780 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d3800640 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 0x556af09c6580 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x556af1779320 tx=0x556af16eb3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ab43365d13262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d0600640 1 -- 172.21.15.5:0/449486520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af09809c0 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d0600640 1 -- 172.21.15.5:0/449486520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556af0986700 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d0600640 1 -- 172.21.15.5:0/449486520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af0980680 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556af098f4a0 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556af098e000 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d56f7280 4 waiting for MDS map... 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.917+0000 7f33d0600640 1 -- 172.21.15.5:0/449486520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x556af181a000 con 0x556af095ac00 2024-04-25T10:13:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.918+0000 7f33d0600640 1 -- 172.21.15.5:0/449486520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x556af09b5180 con 0x556af095ac00 2024-04-25T10:13:20.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.918+0000 7f33d56f7280 4 Got MDS map 106 2024-04-25T10:13:20.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.918+0000 7f33d56f7280 10 main: JournalTool::main 2024-04-25T10:13:20.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.918+0000 7f33d56f7280 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:13:20.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.918+0000 7f33d56f7280 1 Processor -- start 2024-04-25T10:13:20.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.919+0000 7f33d56f7280 1 -- start start 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.919+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1737c00 0x556af09c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 0x556af09c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d56f7280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172a600 con 0x556af1880000 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172a780 con 0x556af1880400 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af172a900 con 0x556af1737c00 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47372/0 (socket says 172.21.15.5:47372) 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 -- 172.21.15.5:0/24307219 learned_addr learned my addr 172.21.15.5:0/24307219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 -- 172.21.15.5:0/24307219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1737c00 msgr2=0x556af09c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1737c00 0x556af09c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 -- 172.21.15.5:0/24307219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 msgr2=0x556af09c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 0x556af09c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 -- 172.21.15.5:0/24307219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556af098e3c0 con 0x556af1880000 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.920+0000 7f33d3800640 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 0x556af09c7b80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x556af1779740 tx=0x556af16eaae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182c8e5dc9bc60a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33cde00640 1 -- 172.21.15.5:0/24307219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af0980b60 con 0x556af1880000 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33cde00640 1 -- 172.21.15.5:0/24307219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556af0986fc0 con 0x556af1880000 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33cde00640 1 -- 172.21.15.5:0/24307219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af0980820 con 0x556af1880000 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 -- 172.21.15.5:0/24307219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 msgr2=0x556af09c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 0x556af09c7b80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x556af1779740 tx=0x556af16eaae0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 -- 172.21.15.5:0/24307219 shutdown_connections 2024-04-25T10:13:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 0x556af09c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 --2- 172.21.15.5:0/24307219 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1737c00 0x556af09c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 -- 172.21.15.5:0/24307219 >> 172.21.15.5:0/24307219 conn(0x556af1737800 msgr2=0x556af16d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 -- 172.21.15.5:0/24307219 shutdown_connections 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.921+0000 7f33d56f7280 1 -- 172.21.15.5:0/24307219 wait complete. 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.922+0000 7f33d56f7280 1 Processor -- start 2024-04-25T10:13:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.922+0000 7f33d56f7280 1 -- start start 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1737800 0x556af09c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 0x556af09c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880c00 0x556af09c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d56f7280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172b500 con 0x556af1881000 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172aa80 con 0x556af1737800 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af172b680 con 0x556af1880c00 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 0x556af09c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d3800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 0x556af09c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47380/0 (socket says 172.21.15.5:47380) 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.923+0000 7f33d3800640 1 -- 172.21.15.5:0/981531597 learned_addr learned my addr 172.21.15.5:0/981531597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d4200640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1737800 0x556af09c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d3800640 1 -- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880c00 msgr2=0x556af09c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d2e00640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880c00 0x556af09c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d3800640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880c00 0x556af09c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d3800640 1 -- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1737800 msgr2=0x556af09c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d3800640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1737800 0x556af09c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d3800640 1 -- 172.21.15.5:0/981531597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556af098e960 con 0x556af1881000 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d2e00640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880c00 0x556af09c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d3800640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 0x556af09c8c00 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x556af1750930 tx=0x556af16eb320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33cc000640 1 -- 172.21.15.5:0/981531597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af0980b60 con 0x556af1881000 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33cc000640 1 -- 172.21.15.5:0/981531597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556af0986e00 con 0x556af1881000 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33cc000640 1 -- 172.21.15.5:0/981531597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af0980820 con 0x556af1881000 2024-04-25T10:13:20.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556af098f680 con 0x556af1881000 2024-04-25T10:13:20.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556af098f860 con 0x556af1881000 2024-04-25T10:13:20.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.924+0000 7f33d56f7280 4 main: JournalTool: resolving pool 17 2024-04-25T10:13:20.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.926+0000 7f33cc000640 1 -- 172.21.15.5:0/981531597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x556af1890c00 con 0x556af1881000 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.927+0000 7f33cc000640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x556af0cb4000 0x556af09c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.927+0000 7f33cc000640 1 -- 172.21.15.5:0/981531597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x556af188ec00 con 0x556af1881000 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.927+0000 7f33d56f7280 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.927+0000 7f33d56f7280 4 main: Executing for rank 0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.927+0000 7f33d4200640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x556af0cb4000 0x556af09c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.928+0000 7f33d56f7280 1 -- start start 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.928+0000 7f33d56f7280 1 Processor -- start 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.928+0000 7f33d4200640 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x556af0cb4000 0x556af09c9700 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x556af0a506f0 tx=0x556af0cc58c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 0x556af09c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d56f7280 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af0cb5000 0x556af09c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d56f7280 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172b980 con 0x556af0cb5000 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x556af172bb00 con 0x556af1880400 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d56f7280 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x556af172bc80 con 0x556af1880000 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d2e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af0cb5000 0x556af09c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d2e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af0cb5000 0x556af09c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47392/0 (socket says 172.21.15.5:47392) 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 learned_addr learned my addr 172.21.15.5:0/2654745158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d4200640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 0x556af09c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.929+0000 7f33d3800640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880000 msgr2=0x556af09c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d2e00640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 msgr2=0x556af09c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d2e00640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1880400 0x556af09c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556af098fa40 con 0x556af0cb5000 2024-04-25T10:13:20.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d3800640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880000 0x556af09c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:20.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d2e00640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af0cb5000 0x556af09c7600 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x556af0c6b920 tx=0x556af0cc5aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd5874717f53113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33c8400640 1 -- 172.21.15.5:0/2654745158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af18c3520 con 0x556af0cb5000 2024-04-25T10:13:20.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33c8400640 1 -- 172.21.15.5:0/2654745158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556af0986c40 con 0x556af0cb5000 2024-04-25T10:13:20.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33c8400640 1 -- 172.21.15.5:0/2654745158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556af18c3380 con 0x556af0cb5000 2024-04-25T10:13:20.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556af098e780 con 0x556af0cb5000 2024-04-25T10:13:20.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556af098f4a0 con 0x556af0cb5000 2024-04-25T10:13:20.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.930+0000 7f33d56f7280 4 waiting for MDS map... 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.931+0000 7f33c8400640 1 -- 172.21.15.5:0/2654745158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x556af0d28600 con 0x556af0cb5000 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.931+0000 7f33c8400640 1 -- 172.21.15.5:0/2654745158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x556af0d34000 con 0x556af0cb5000 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.931+0000 7f33d56f7280 4 Got MDS map 106 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.931+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x556af0cb5400 0x556af0d44000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.931+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0cb5800 con 0x556af0cb5400 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.932+0000 7f33d4200640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x556af0cb5400 0x556af0d44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.933+0000 7f33d4200640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x556af0cb5400 0x556af0d44000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=687c8cc9700d822b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.943+0000 7f33d4200640 1 -- 172.21.15.5:0/2654745158 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v122'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8000 con 0x556af0cb5400 2024-04-25T10:13:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.943+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x556af0cb5c00 0x556af0d44580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.943+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d23000 con 0x556af0cb5c00 2024-04-25T10:13:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.944+0000 7f33d3800640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x556af0cb5c00 0x556af0d44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.945+0000 7f33d3800640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x556af0cb5c00 0x556af0d44580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=6af9805598cc8f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.958 INFO:teuthology.orchestra.run.smithi005.stdout:writing EResetJournal entry 2024-04-25T10:13:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.954+0000 7f33d3800640 1 -- 172.21.15.5:0/2654745158 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v122'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8500 con 0x556af0cb5c00 2024-04-25T10:13:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d23400 con 0x556af0cb5c00 2024-04-25T10:13:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x556af0d23800 0x556af0d44b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d23c00 con 0x556af0d23800 2024-04-25T10:13:20.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d54000 con 0x556af0cb5400 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x556af0d54400 0x556af0d45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d54800 con 0x556af0d54400 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x556af0d54c00 0x556af0d45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d55000 con 0x556af0d54c00 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0d55400 con 0x556af0cb5400 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d4200640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x556af0d23800 0x556af0d44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.955+0000 7f33d3800640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x556af0d54400 0x556af0d45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.956+0000 7f33d4200640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x556af0d23800 0x556af0d44b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=74f3c1ac0a92bfed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.956+0000 7f33d2e00640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x556af0d54c00 0x556af0d45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.956+0000 7f33d3800640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x556af0d54400 0x556af0d45080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=5f758c01bea0e718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.958+0000 7f33d3800640 1 -- 172.21.15.5:0/2654745158 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(3 200.00000001 [delete] v122'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8500 con 0x556af0cb5c00 2024-04-25T10:13:20.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.958+0000 7f33d2e00640 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x556af0d54c00 0x556af0d45600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=1e5f88c3aa9a3577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.958+0000 7f33d3800640 1 -- 172.21.15.5:0/2654745158 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(6 200.00000004 [delete] v122'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8500 con 0x556af0d54400 2024-04-25T10:13:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.959+0000 7f33d4200640 1 -- 172.21.15.5:0/2654745158 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(8 200.00000006 [delete] v122'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8000 con 0x556af0cb5400 2024-04-25T10:13:20.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.959+0000 7f33d4200640 1 -- 172.21.15.5:0/2654745158 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(5 200.00000003 [delete] v122'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8000 con 0x556af0cb5400 2024-04-25T10:13:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.961+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(7 200.00000005 [delete] v122'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8780 con 0x556af0d54c00 2024-04-25T10:13:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.961+0000 7f33d4200640 1 -- 172.21.15.5:0/2654745158 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(4 200.00000002 [delete] v122'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8000 con 0x556af0d23800 2024-04-25T10:13:20.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.961+0000 7f33c6600640 1 -- 172.21.15.5:0/2654745158 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x556af0cb4c00 con 0x556af0cb5c00 2024-04-25T10:13:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d3800640 1 -- 172.21.15.5:0/2654745158 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v122'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x556af16d8500 con 0x556af0cb5c00 2024-04-25T10:13:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x556af0d54400 msgr2=0x556af0d45080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x556af0d54400 0x556af0d45080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x556af0d23800 msgr2=0x556af0d44b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x556af0d23800 0x556af0d44b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x556af0d54c00 msgr2=0x556af0d45600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x556af0d54c00 0x556af0d45600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x556af0cb5c00 msgr2=0x556af0d44580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x556af0cb5c00 0x556af0d44580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x556af0cb5400 msgr2=0x556af0d44000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x556af0cb5400 0x556af0d44000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 reap_dead start 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af0cb5000 msgr2=0x556af09c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d56f7280 1 --2- 172.21.15.5:0/2654745158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af0cb5000 0x556af09c7600 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x556af0c6b920 tx=0x556af0cc5aa0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 reap_dead start 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.964+0000 7f33d2e00640 1 -- 172.21.15.5:0/2654745158 reap_dead start 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 shutdown_connections 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 >> 172.21.15.5:0/2654745158 conn(0x556af1880800 msgr2=0x556af16d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 shutdown_connections 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/2654745158 wait complete. 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x556af0cb4000 msgr2=0x556af09c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x556af0cb4000 0x556af09c9700 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x556af0a506f0 tx=0x556af0cc58c0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 msgr2=0x556af09c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 0x556af09c8c00 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x556af1750930 tx=0x556af16eb320 comp rx=0 tx=0).stop 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 shutdown_connections 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af1880c00 0x556af09c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af1881000 0x556af09c8c00 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x556af0cb4000 0x556af09c9700 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.965+0000 7f33d56f7280 1 --2- 172.21.15.5:0/981531597 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af1737800 0x556af09c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 >> 172.21.15.5:0/981531597 conn(0x556af1737c00 msgr2=0x556af16d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 shutdown_connections 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 -- 172.21.15.5:0/981531597 wait complete. 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 msgr2=0x556af09c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 0x556af09c6580 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x556af1779320 tx=0x556af16eb3e0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 shutdown_connections 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x556af095bc00 0x556af09c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556af095b800 0x556af09c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 --2- 172.21.15.5:0/449486520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556af095ac00 0x556af09c6580 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.966+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 >> 172.21.15.5:0/449486520 conn(0x556af095b000 msgr2=0x556af16d6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.967+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 shutdown_connections 2024-04-25T10:13:20.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:20.967+0000 7f33d56f7280 1 -- 172.21.15.5:0/449486520 wait complete. 2024-04-25T10:13:20.976 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.141839, result: writing EResetJournal entry 2024-04-25T10:13:20.976 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:13:21.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.120+0000 7fa1e7922740 1 -- start start 2024-04-25T10:13:21.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.120+0000 7fa1e7922740 1 Processor -- start 2024-04-25T10:13:21.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 0x55abf1f5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf1ebb800 0x55abf1f5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf1ebbc00 0x55abf1f5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e7922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1a180 con 0x55abf1ebac00 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e7922740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1a300 con 0x55abf1ebb800 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e7922740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1a480 con 0x55abf1ebbc00 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 0x55abf1f5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 0x55abf1f5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47400/0 (socket says 172.21.15.5:47400) 2024-04-25T10:13:21.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.121+0000 7fa1e5200640 1 -- 172.21.15.5:0/3591833083 learned_addr learned my addr 172.21.15.5:0/3591833083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf1ebb800 0x55abf1f5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf1ebbc00 0x55abf1f5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5200640 1 -- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf1ebbc00 msgr2=0x55abf1f5b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf1ebbc00 0x55abf1f5b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5200640 1 -- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf1ebb800 msgr2=0x55abf1f5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf1ebb800 0x55abf1f5ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5200640 1 -- 172.21.15.5:0/3591833083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf1ee7680 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf1ebb800 0x55abf1f5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf1ebbc00 0x55abf1f5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 0x55abf1f5a580 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x55abf2cd3170 tx=0x55abf2c1c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162b06dc6cf329a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e2000640 1 -- 172.21.15.5:0/3591833083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abf1ee09c0 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e2000640 1 -- 172.21.15.5:0/3591833083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abf1f041c0 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e2000640 1 -- 172.21.15.5:0/3591833083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abf1ee0680 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abf1ee7860 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55abf1ee72c0 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.122+0000 7fa1e7922740 4 waiting for MDS map... 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.123+0000 7fa1e2000640 1 -- 172.21.15.5:0/3591833083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55abf2d8a200 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.123+0000 7fa1e2000640 1 -- 172.21.15.5:0/3591833083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x55abf1f13180 con 0x55abf1ebac00 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.123+0000 7fa1e7922740 4 Got MDS map 106 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.123+0000 7fa1e7922740 4 datascan.main: Using metadata pool output 2024-04-25T10:13:21.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.123+0000 7fa1e7922740 4 datascan.main: connecting to RADOS... 2024-04-25T10:13:21.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.124+0000 7fa1e7922740 1 Processor -- start 2024-04-25T10:13:21.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.124+0000 7fa1e7922740 1 -- start start 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 0x55abf1f5b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2de6000 0x55abf1f5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de6400 0x55abf1f5c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e7922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1a600 con 0x55abf2ca1c00 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e7922740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1a780 con 0x55abf2de6400 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e7922740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1a900 con 0x55abf2de6000 2024-04-25T10:13:21.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 0x55abf1f5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 0x55abf1f5b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47404/0 (socket says 172.21.15.5:47404) 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e5c00640 1 -- 172.21.15.5:0/1945946496 learned_addr learned my addr 172.21.15.5:0/1945946496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.125+0000 7fa1e4800640 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de6400 0x55abf1f5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5200640 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2de6000 0x55abf1f5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5c00640 1 -- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2de6000 msgr2=0x55abf1f5bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2de6000 0x55abf1f5bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5c00640 1 -- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de6400 msgr2=0x55abf1f5c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de6400 0x55abf1f5c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5c00640 1 -- 172.21.15.5:0/1945946496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf1ee74a0 con 0x55abf2ca1c00 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5200640 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2de6000 0x55abf1f5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 0x55abf1f5b600 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x55abf2d96ab0 tx=0x55abf2df60c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec33d907cf0f5716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1df800640 1 -- 172.21.15.5:0/1945946496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abf1ee09c0 con 0x55abf2ca1c00 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1df800640 1 -- 172.21.15.5:0/1945946496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abf1f04e00 con 0x55abf2ca1c00 2024-04-25T10:13:21.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.126+0000 7fa1df800640 1 -- 172.21.15.5:0/1945946496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abf1ee0680 con 0x55abf2ca1c00 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 -- 172.21.15.5:0/1945946496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 msgr2=0x55abf1f5b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 0x55abf1f5b600 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x55abf2d96ab0 tx=0x55abf2df60c0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 -- 172.21.15.5:0/1945946496 shutdown_connections 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de6400 0x55abf1f5c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2de6000 0x55abf1f5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 --2- 172.21.15.5:0/1945946496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2ca1c00 0x55abf1f5b600 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 -- 172.21.15.5:0/1945946496 >> 172.21.15.5:0/1945946496 conn(0x55abf2ca1800 msgr2=0x55abf1fa9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 -- 172.21.15.5:0/1945946496 shutdown_connections 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.127+0000 7fa1e7922740 1 -- 172.21.15.5:0/1945946496 wait complete. 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.128+0000 7fa1e7922740 1 Processor -- start 2024-04-25T10:13:21.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.128+0000 7fa1e7922740 1 -- start start 2024-04-25T10:13:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2ca1800 0x55abf1f5d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de7000 0x55abf1f5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e7922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 0x55abf1f5c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e7922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1bc80 con 0x55abf2de6c00 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e7922740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1bb00 con 0x55abf2de7000 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e7922740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abf2c1b980 con 0x55abf2ca1800 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 0x55abf1f5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 0x55abf1f5c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47414/0 (socket says 172.21.15.5:47414) 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 learned_addr learned my addr 172.21.15.5:0/3282048049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2ca1800 0x55abf1f5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de7000 0x55abf1f5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2ca1800 msgr2=0x55abf1f5d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2ca1800 0x55abf1f5d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de7000 msgr2=0x55abf1f5cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de7000 0x55abf1f5cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abf1ee70e0 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf2ca1800 0x55abf1f5d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.129+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf2de7000 0x55abf1f5cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 0x55abf1f5c680 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x55abf2d97da0 tx=0x55abf2df6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1dda00640 1 -- 172.21.15.5:0/3282048049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abf1ee09c0 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1dda00640 1 -- 172.21.15.5:0/3282048049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abf1f04c40 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1dda00640 1 -- 172.21.15.5:0/3282048049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55abf1ee0680 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55abf1ee7a40 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abf1ee63c0 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.130+0000 7fa1e7922740 4 datascan.init: resolving metadata pool 17 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.132+0000 7fa1dda00640 1 -- 172.21.15.5:0/3282048049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x55abf2dfcc00 con 0x55abf2de6c00 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1dda00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55abf2e02c00 0x55abf1f5d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1dda00640 1 -- 172.21.15.5:0/3282048049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x55abf2deda00 con 0x55abf2de6c00 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1e7922740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1e7922740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1e7922740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55abf2de6800 0x55abf1f5c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2de6400 con 0x55abf2de6800 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.133+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55abf2e02c00 0x55abf1f5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.134+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55abf2de6800 0x55abf1f5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.134+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55abf2e02c00 0x55abf1f5d700 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55abf2d96ab0 tx=0x55abf2df6000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.135+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55abf2de6800 0x55abf1f5c100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.136+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(1 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.136+0000 7fa1dee00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55abf2e03000 0x55abf223c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.136+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03400 con 0x55abf2e03000 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.136+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55abf2e03000 0x55abf223c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.137+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55abf2e03000 0x55abf223c000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.137+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(2 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.137+0000 7fa1df800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55abf2e03c00 0x55abf223c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.137+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244000 con 0x55abf2e03c00 2024-04-25T10:13:21.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.137+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55abf2e03c00 0x55abf223c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.138+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55abf2e03c00 0x55abf223c580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.138+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(3 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.138+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03800 con 0x55abf2de6800 2024-04-25T10:13:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.139+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(4 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.140+0000 7fa1df800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55abf2244400 0x55abf223cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.140+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244800 con 0x55abf2244400 2024-04-25T10:13:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.140+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55abf2244400 0x55abf223cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.141+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55abf2244400 0x55abf223cb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.142+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(5 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.142+0000 7fa1dee00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55abf2245400 0x55abf223d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.142+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245800 con 0x55abf2245400 2024-04-25T10:13:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.143+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55abf2245400 0x55abf223d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.143+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55abf2245400 0x55abf223d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.143+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(6 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2245400 2024-04-25T10:13:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.143+0000 7fa1df800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55abf2244c00 0x55abf223d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.144+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245000 con 0x55abf2244c00 2024-04-25T10:13:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.144+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55abf2244c00 0x55abf223d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.145+0000 7fa1e5200640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55abf2244c00 0x55abf223d080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.146+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(7 [pgnls start_epoch 122 out=81b] v120'17 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.146+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:8 18.18 18:1ad8d55b:::10000000006.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2de6000 con 0x55abf2244c00 2024-04-25T10:13:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.147+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== osd_op_reply(8 10000000006.00000001 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.147+0000 7fa1e7922740 10 datascan.operator(): handling object 1099511627782.1 2024-04-25T10:13:21.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.147+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:9 18.25 18:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244000 con 0x55abf2e03c00 2024-04-25T10:13:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.150+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== osd_op_reply(9 10000000006.00000000 [call] v122'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.150+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245000 con 0x55abf2244c00 2024-04-25T10:13:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.151+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 3 ==== osd_op_reply(10 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.151+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260000 con 0x55abf2de6800 2024-04-25T10:13:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.152+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(11 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.152+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245c00 con 0x55abf2de6800 2024-04-25T10:13:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.153+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 4 ==== osd_op_reply(12 [pgnls start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.153+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260400 con 0x55abf2245400 2024-04-25T10:13:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.153+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(13 [pgnls start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2245400 2024-04-25T10:13:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.153+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260400 con 0x55abf2de6800 2024-04-25T10:13:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.155+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 5 ==== osd_op_reply(14 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.155+0000 7fa1df800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55abf2260800 0x55abf223e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.155+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260c00 con 0x55abf2260800 2024-04-25T10:13:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.155+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55abf2260800 0x55abf223e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.156+0000 7fa1e4800640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55abf2260800 0x55abf223e100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.157+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(15 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2260800 2024-04-25T10:13:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.157+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245800 con 0x55abf2244400 2024-04-25T10:13:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.158+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(16 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.158+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260000 con 0x55abf2244c00 2024-04-25T10:13:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.159+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 4 ==== osd_op_reply(17 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.159+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03400 con 0x55abf2260800 2024-04-25T10:13:21.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.160+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(18 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2260800 2024-04-25T10:13:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.160+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244000 con 0x55abf2e03c00 2024-04-25T10:13:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.160+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 3 ==== osd_op_reply(19 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.160+0000 7fa1dee00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55abf2de7c00 0x55abf223db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.160+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260c00 con 0x55abf2de7c00 2024-04-25T10:13:21.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.161+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55abf2de7c00 0x55abf223db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.162+0000 7fa1e5c00640 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55abf2de7c00 0x55abf223db80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.163+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(20 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.163+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245000 con 0x55abf2de6800 2024-04-25T10:13:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.165+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 6 ==== osd_op_reply(21 [pgnls start_epoch 122 out=81b] v121'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.165+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:22 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03800 con 0x55abf2de6800 2024-04-25T10:13:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.166+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 7 ==== osd_op_reply(22 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.166+0000 7fa1e7922740 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T10:13:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.166+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:23 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2de6400 con 0x55abf2de6800 2024-04-25T10:13:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.169+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 8 ==== osd_op_reply(23 10000000002.00000000 [call] v122'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.169+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:24 18.32 18:4c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2ca1400 con 0x55abf2de6800 2024-04-25T10:13:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.170+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 9 ==== osd_op_reply(24 [pgnls start_epoch 122 out=81b] v120'16 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.170+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:25 18.32 18:4def72d4:::10000000003.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261000 con 0x55abf2de6800 2024-04-25T10:13:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.171+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 10 ==== osd_op_reply(25 10000000003.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.171+0000 7fa1e7922740 10 datascan.operator(): handling object 1099511627779.0 2024-04-25T10:13:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.171+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:26 18.32 18:4def72d4:::10000000003.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261400 con 0x55abf2de6800 2024-04-25T10:13:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.175+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 11 ==== osd_op_reply(26 10000000003.00000000 [call] v122'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.175+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:27 18.a 18:50000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261800 con 0x55abf2de7c00 2024-04-25T10:13:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.176+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== osd_op_reply(27 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.176+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:28 18.2a 18:54000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260400 con 0x55abf2245400 2024-04-25T10:13:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.177+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.1 v2:172.21.15.5:6800/692601986 3 ==== osd_op_reply(28 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2245400 2024-04-25T10:13:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.177+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:29 18.1a 18:58000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245c00 con 0x55abf2de7c00 2024-04-25T10:13:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.178+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 3 ==== osd_op_reply(29 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.179+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:30 18.3a 18:5c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03400 con 0x55abf2244400 2024-04-25T10:13:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.180+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(30 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.180+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:31 18.6 18:60000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2de6000 con 0x55abf2260800 2024-04-25T10:13:21.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.181+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(31 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2260800 2024-04-25T10:13:21.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.181+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:32 18.26 18:64000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244800 con 0x55abf2244c00 2024-04-25T10:13:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.182+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 5 ==== osd_op_reply(32 [pgnls start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.182+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:33 18.16 18:68000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260000 con 0x55abf2de7c00 2024-04-25T10:13:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.183+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 4 ==== osd_op_reply(33 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.183+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:34 18.36 18:6c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245800 con 0x55abf2244400 2024-04-25T10:13:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.184+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 4 ==== osd_op_reply(34 [pgnls start_epoch 122 out=49b] v120'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.184+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:35 18.e 18:70000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2ca1400 con 0x55abf2244400 2024-04-25T10:13:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.186+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 5 ==== osd_op_reply(35 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.186+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:36 18.2e 18:74000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260c00 con 0x55abf2de6800 2024-04-25T10:13:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.187+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 12 ==== osd_op_reply(36 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.187+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:37 18.1e 18:78000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2de6400 con 0x55abf2e03000 2024-04-25T10:13:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.188+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(37 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.188+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:38 18.3e 18:7c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261800 con 0x55abf2244400 2024-04-25T10:13:21.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.189+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 6 ==== osd_op_reply(38 [pgnls start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.189+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:39 18.1 18:80000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03800 con 0x55abf2244400 2024-04-25T10:13:21.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.190+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 7 ==== osd_op_reply(39 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.190+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:40 18.21 18:84000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260400 con 0x55abf2e03000 2024-04-25T10:13:21.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.191+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(40 [pgnls start_epoch 122 out=49b] v120'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.191+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:41 18.11 18:88000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244000 con 0x55abf2244400 2024-04-25T10:13:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.193+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 8 ==== osd_op_reply(41 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.193+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:42 18.31 18:8c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260000 con 0x55abf2245400 2024-04-25T10:13:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.194+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.1 v2:172.21.15.5:6800/692601986 4 ==== osd_op_reply(42 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2245400 2024-04-25T10:13:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.194+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245000 con 0x55abf2de6800 2024-04-25T10:13:21.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.195+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 13 ==== osd_op_reply(43 [pgnls start_epoch 122 out=49b] v120'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.195+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261000 con 0x55abf2245400 2024-04-25T10:13:21.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.196+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.1 v2:172.21.15.5:6800/692601986 5 ==== osd_op_reply(44 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2245400 2024-04-25T10:13:21.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.196+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261800 con 0x55abf2e03000 2024-04-25T10:13:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.197+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 4 ==== osd_op_reply(45 [pgnls start_epoch 122 out=49b] v120'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.197+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:46 18.39 18:9c000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245c00 con 0x55abf2244400 2024-04-25T10:13:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.199+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 9 ==== osd_op_reply(46 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.199+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:47 18.5 18:a0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2ca1400 con 0x55abf2de7c00 2024-04-25T10:13:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.200+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 5 ==== osd_op_reply(47 [pgnls start_epoch 122 out=81b] v121'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.200+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:48 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261c00 con 0x55abf2de7c00 2024-04-25T10:13:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.201+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 6 ==== osd_op_reply(48 10000000004.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.201+0000 7fa1e7922740 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T10:13:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.201+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:49 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2ca1400 con 0x55abf2de7c00 2024-04-25T10:13:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.205+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 7 ==== osd_op_reply(49 10000000004.00000000 [call] v122'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.205+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:50 18.25 18:a4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260400 con 0x55abf2e03c00 2024-04-25T10:13:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.206+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 4 ==== osd_op_reply(50 [pgnls start_epoch 122 out=81b] v122'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.206+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:51 18.25 18:a5feb6f3:::10000000006.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260000 con 0x55abf2e03c00 2024-04-25T10:13:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.207+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 5 ==== osd_op_reply(51 10000000006.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.207+0000 7fa1e7922740 10 datascan.operator(): handling object 1099511627782.0 2024-04-25T10:13:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.207+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:52 18.25 18:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261000 con 0x55abf2e03c00 2024-04-25T10:13:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.211+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 6 ==== osd_op_reply(52 10000000006.00000000 [call] v122'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.211+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261800 con 0x55abf2260800 2024-04-25T10:13:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.212+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(53 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2260800 2024-04-25T10:13:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.212+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226e000 con 0x55abf2e03000 2024-04-25T10:13:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.213+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 5 ==== osd_op_reply(54 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.213+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245800 con 0x55abf2e03000 2024-04-25T10:13:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.214+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 6 ==== osd_op_reply(55 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.214+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226e400 con 0x55abf2e03c00 2024-04-25T10:13:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.215+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 7 ==== osd_op_reply(56 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.215+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2de6000 con 0x55abf2244400 2024-04-25T10:13:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.216+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 10 ==== osd_op_reply(57 [pgnls start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.217+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226e800 con 0x55abf2de7c00 2024-04-25T10:13:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.218+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 8 ==== osd_op_reply(58 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.218+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2e03400 con 0x55abf2244c00 2024-04-25T10:13:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.219+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 6 ==== osd_op_reply(59 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.219+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226ec00 con 0x55abf2de7c00 2024-04-25T10:13:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.220+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 9 ==== osd_op_reply(60 [pgnls start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.220+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226e400 con 0x55abf2e03c00 2024-04-25T10:13:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.221+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 8 ==== osd_op_reply(61 [pgnls start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.222+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226f000 con 0x55abf2244400 2024-04-25T10:13:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.223+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.5 v2:172.21.15.80:6800/161144277 11 ==== osd_op_reply(62 [pgnls start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2244400 2024-04-25T10:13:21.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.223+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261400 con 0x55abf2e03000 2024-04-25T10:13:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.224+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 7 ==== osd_op_reply(63 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.224+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226f400 con 0x55abf2e03c00 2024-04-25T10:13:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.225+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 9 ==== osd_op_reply(64 [pgnls start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.225+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260c00 con 0x55abf2244c00 2024-04-25T10:13:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.226+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.4 v2:172.21.15.80:6808/2045277337 7 ==== osd_op_reply(65 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2244c00 2024-04-25T10:13:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.226+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf226f800 con 0x55abf2e03c00 2024-04-25T10:13:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.227+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 10 ==== osd_op_reply(66 [pgnls start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.227+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260400 con 0x55abf2de7c00 2024-04-25T10:13:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.229+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.7 v2:172.21.15.80:6816/4166609972 10 ==== osd_op_reply(67 [pgnls start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2de7c00 2024-04-25T10:13:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.229+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261c00 con 0x55abf2e03000 2024-04-25T10:13:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.230+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 8 ==== osd_op_reply(68 [pgnls start_epoch 122 out=49b] v120'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.230+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2260000 con 0x55abf2e03c00 2024-04-25T10:13:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.231+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 11 ==== osd_op_reply(69 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.231+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261400 con 0x55abf2260800 2024-04-25T10:13:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.232+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.2 v2:172.21.15.5:6822/3928063993 5 ==== osd_op_reply(70 [pgnls start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2260800 2024-04-25T10:13:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.232+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261000 con 0x55abf2e03c00 2024-04-25T10:13:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.233+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.0 v2:172.21.15.5:6809/1760674718 12 ==== osd_op_reply(71 [pgnls start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2e03c00 2024-04-25T10:13:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.233+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2261800 con 0x55abf2de6800 2024-04-25T10:13:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.235+0000 7fa1e5200640 1 -- 172.21.15.5:0/3282048049 <== osd.6 v2:172.21.15.80:6824/3989257232 14 ==== osd_op_reply(72 [pgnls start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34000 con 0x55abf2de6800 2024-04-25T10:13:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.235+0000 7fa1df800640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2244800 con 0x55abf2260800 2024-04-25T10:13:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.235+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 <== osd.2 v2:172.21.15.5:6822/3928063993 6 ==== osd_op_reply(73 [pgnls start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34780 con 0x55abf2260800 2024-04-25T10:13:21.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.236+0000 7fa1dee00640 1 -- 172.21.15.5:0/3282048049 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x55abf2245800 con 0x55abf2e03000 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e5c00640 1 -- 172.21.15.5:0/3282048049 <== osd.3 v2:172.21.15.5:6808/3976284996 9 ==== osd_op_reply(74 [pgnls start_epoch 122 out=49b] v120'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55abf2c34500 con 0x55abf2e03000 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55abf2e03c00 msgr2=0x55abf223c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x55abf2e03c00 0x55abf223c580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55abf2245400 msgr2=0x55abf223d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x55abf2245400 0x55abf223d600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55abf2260800 msgr2=0x55abf223e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x55abf2260800 0x55abf223e100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55abf2e03000 msgr2=0x55abf223c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x55abf2e03000 0x55abf223c000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55abf2244c00 msgr2=0x55abf223d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x55abf2244c00 0x55abf223d080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55abf2244400 msgr2=0x55abf223cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x55abf2244400 0x55abf223cb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55abf2de6800 msgr2=0x55abf1f5c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 reap_dead start 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x55abf2de6800 0x55abf1f5c100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55abf2de7c00 msgr2=0x55abf223db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x55abf2de7c00 0x55abf223db80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 reap_dead start 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e4800640 1 -- 172.21.15.5:0/3282048049 reap_dead start 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55abf2e02c00 msgr2=0x55abf1f5d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55abf2e02c00 0x55abf1f5d700 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x55abf2d96ab0 tx=0x55abf2df6000 comp rx=0 tx=0).stop 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.237+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 msgr2=0x55abf1f5c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.238+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 0x55abf1f5c680 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x55abf2d97da0 tx=0x55abf2df6de0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.238+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 shutdown_connections 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.238+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf2de6c00 0x55abf1f5c680 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.238+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3282048049 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x55abf2e02c00 0x55abf1f5d700 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.238+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 >> 172.21.15.5:0/3282048049 conn(0x55abf2ca1c00 msgr2=0x55abf1fa9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.238+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 shutdown_connections 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.239+0000 7fa1e7922740 1 -- 172.21.15.5:0/3282048049 wait complete. 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 msgr2=0x55abf1f5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 0x55abf1f5a580 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x55abf2cd3170 tx=0x55abf2c1c360 comp rx=0 tx=0).stop 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 shutdown_connections 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55abf1ebbc00 0x55abf1f5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55abf1ebb800 0x55abf1f5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 --2- 172.21.15.5:0/3591833083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abf1ebac00 0x55abf1f5a580 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 >> 172.21.15.5:0/3591833083 conn(0x55abf1ebb000 msgr2=0x55abf1fa9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 shutdown_connections 2024-04-25T10:13:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.240+0000 7fa1e7922740 1 -- 172.21.15.5:0/3591833083 wait complete. 2024-04-25T10:13:21.255 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.278781, result: 2024-04-25T10:13:21.255 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T10:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.361+0000 7f0e5d419740 1 -- start start 2024-04-25T10:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.361+0000 7f0e5d419740 1 Processor -- start 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558ddc7ec00 0x5558ddd1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 0x5558ddd1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558ddc7fc00 0x5558ddd1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5d419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deb48180 con 0x5558ddc7f800 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5d419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deb48300 con 0x5558ddc7ec00 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5d419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558deb48480 con 0x5558ddc7fc00 2024-04-25T10:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 0x5558ddd1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 0x5558ddd1eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47420/0 (socket says 172.21.15.5:47420) 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 -- 172.21.15.5:0/146493455 learned_addr learned my addr 172.21.15.5:0/146493455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558ddc7ec00 0x5558ddd1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 -- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558ddc7fc00 msgr2=0x5558ddd1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5b400640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558ddc7fc00 0x5558ddd1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558ddc7fc00 0x5558ddd1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 -- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558ddc7ec00 msgr2=0x5558ddd1e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.362+0000 7f0e5a000640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558ddc7ec00 0x5558ddd1e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5a000640 1 -- 172.21.15.5:0/146493455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ddcab680 con 0x5558ddc7f800 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558ddc7ec00 0x5558ddd1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5b400640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558ddc7fc00 0x5558ddd1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5a000640 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 0x5558ddd1eb00 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x5558dea9b140 tx=0x5558dea5c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ff347f3ba5ebde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e57800640 1 -- 172.21.15.5:0/146493455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558ddca49c0 con 0x5558ddc7f800 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e57800640 1 -- 172.21.15.5:0/146493455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558ddcc8e00 con 0x5558ddc7f800 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e57800640 1 -- 172.21.15.5:0/146493455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558ddca4680 con 0x5558ddc7f800 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558ddcab860 con 0x5558ddc7f800 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5558ddcab2c0 con 0x5558ddc7f800 2024-04-25T10:13:21.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.363+0000 7f0e5d419740 4 waiting for MDS map... 2024-04-25T10:13:21.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.364+0000 7f0e57800640 1 -- 172.21.15.5:0/146493455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x5558dea0ca00 con 0x5558ddc7f800 2024-04-25T10:13:21.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.364+0000 7f0e57800640 1 -- 172.21.15.5:0/146493455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x5558ddcd7180 con 0x5558ddc7f800 2024-04-25T10:13:21.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.364+0000 7f0e5d419740 4 Got MDS map 106 2024-04-25T10:13:21.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.364+0000 7f0e5d419740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T10:13:21.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.364+0000 7f0e5d419740 4 datascan.main: Using metadata pool output 2024-04-25T10:13:21.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.364+0000 7f0e5d419740 4 datascan.main: connecting to RADOS... 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.365+0000 7f0e5d419740 1 Processor -- start 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.365+0000 7f0e5d419740 1 -- start start 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 0x5558ddd1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558debba000 0x5558ddd1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debba400 0x5558ddd20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5d419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deb48600 con 0x5558dea69c00 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5d419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deb48780 con 0x5558debba000 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5d419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558deb48900 con 0x5558debba400 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 0x5558ddd1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 0x5558ddd1f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47426/0 (socket says 172.21.15.5:47426) 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 -- 172.21.15.5:0/3096391001 learned_addr learned my addr 172.21.15.5:0/3096391001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5a000640 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debba400 0x5558ddd20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 -- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debba400 msgr2=0x5558ddd20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debba400 0x5558ddd20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 -- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558debba000 msgr2=0x5558ddd1fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558debba000 0x5558ddd1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558debba000 0x5558ddd1fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.366+0000 7f0e5b400640 1 -- 172.21.15.5:0/3096391001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ddcab4a0 con 0x5558dea69c00 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.367+0000 7f0e5a000640 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debba400 0x5558ddd20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.367+0000 7f0e5b400640 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 0x5558ddd1f600 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x5558deb64ab0 tx=0x5558debc4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a544241ba5d304 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.367+0000 7f0e55000640 1 -- 172.21.15.5:0/3096391001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558ddca49c0 con 0x5558dea69c00 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.367+0000 7f0e55000640 1 -- 172.21.15.5:0/3096391001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558ddcc8a80 con 0x5558dea69c00 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.367+0000 7f0e55000640 1 -- 172.21.15.5:0/3096391001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558ddca4680 con 0x5558dea69c00 2024-04-25T10:13:21.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 -- 172.21.15.5:0/3096391001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 msgr2=0x5558ddd1f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 0x5558ddd1f600 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x5558deb64ab0 tx=0x5558debc4000 comp rx=0 tx=0).stop 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 -- 172.21.15.5:0/3096391001 shutdown_connections 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debba400 0x5558ddd20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558debba000 0x5558ddd1fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 --2- 172.21.15.5:0/3096391001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558dea69c00 0x5558ddd1f600 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 -- 172.21.15.5:0/3096391001 >> 172.21.15.5:0/3096391001 conn(0x5558dea69800 msgr2=0x5558ddd7f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 -- 172.21.15.5:0/3096391001 shutdown_connections 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.368+0000 7f0e5d419740 1 -- 172.21.15.5:0/3096391001 wait complete. 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.369+0000 7f0e5d419740 1 Processor -- start 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.369+0000 7f0e5d419740 1 -- start start 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.369+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558dea69800 0x5558ddd21180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.369+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558debbb000 0x5558ddd20c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.369+0000 7f0e5d419740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debbac00 0x5558ddd20680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.369+0000 7f0e5d419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deb49c80 con 0x5558debbb000 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5d419740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558deb49b00 con 0x5558dea69800 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5d419740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558deb49980 con 0x5558debbac00 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558debbb000 0x5558ddd20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558debbb000 0x5558ddd20c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47430/0 (socket says 172.21.15.5:47430) 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 learned_addr learned my addr 172.21.15.5:0/1852679891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debbac00 0x5558ddd20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558dea69800 0x5558ddd21180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debbac00 msgr2=0x5558ddd20680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558debbac00 0x5558ddd20680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558dea69800 msgr2=0x5558ddd21180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558dea69800 0x5558ddd21180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.370+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ddcab0e0 con 0x5558debbb000 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558debbb000 0x5558ddd20c00 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x5558deb65d40 tx=0x5558debc4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e53200640 1 -- 172.21.15.5:0/1852679891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558ddca49c0 con 0x5558debbb000 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e53200640 1 -- 172.21.15.5:0/1852679891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5558ddcc8540 con 0x5558debbb000 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e53200640 1 -- 172.21.15.5:0/1852679891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5558ddca4680 con 0x5558debbb000 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5558ddcaba40 con 0x5558debbb000 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558ddcaa3c0 con 0x5558debbb000 2024-04-25T10:13:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.371+0000 7f0e5d419740 4 datascan.init: resolving metadata pool 17 2024-04-25T10:13:21.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.373+0000 7f0e53200640 1 -- 172.21.15.5:0/1852679891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x5558debccc00 con 0x5558debbb000 2024-04-25T10:13:21.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.373+0000 7f0e53200640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x5558debd6c00 0x5558ddd21700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.373+0000 7f0e53200640 1 -- 172.21.15.5:0/1852679891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x5558debcb000 con 0x5558debbb000 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5d419740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5d419740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5d419740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5558debba800 0x5558ddd20100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debba400 con 0x5558debba800 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x5558debd6c00 0x5558ddd21700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.374+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5558debba800 0x5558ddd20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.375+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x5558debd6c00 0x5558ddd21700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x5558deb64ab0 tx=0x5558dea5c180 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.375+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5558debba800 0x5558ddd20100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.376+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.376+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5558debba000 0x5558ddd1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.376+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7000 con 0x5558debba000 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.377+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5558debba000 0x5558ddd1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.377+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5558debba000 0x5558ddd1fb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.378+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.378+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5558debd7400 0x5558ddd1f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.378+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7800 con 0x5558debd7400 2024-04-25T10:13:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.378+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5558debd7400 0x5558ddd1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.379+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5558debd7400 0x5558ddd1f600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.381+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.381+0000 7f0e54600640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5558de01e000 0x5558de024000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.381+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e400 con 0x5558de01e000 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.381+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5558de01e000 0x5558de024000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.382+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5558de01e000 0x5558de024000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.382+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.382+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01ec00 con 0x5558debba000 2024-04-25T10:13:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.383+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.383+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e800 con 0x5558debd7400 2024-04-25T10:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.383+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.383+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01f000 con 0x5558debba800 2024-04-25T10:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.385+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.385+0000 7f0e54600640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x5558de01f800 0x5558de024580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.385+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01fc00 con 0x5558de01f800 2024-04-25T10:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.385+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x5558de01f800 0x5558de024580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.386+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x5558de01f800 0x5558de024580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.386+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.386+0000 7f0e55000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x5558de01f400 0x5558de025080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.386+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01ec00 con 0x5558de01f400 2024-04-25T10:13:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.387+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x5558de01f400 0x5558de025080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.388+0000 7f0e5aa00640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x5558de01f400 0x5558de025080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.389+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 122 out=49b] v120'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.389+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01fc00 con 0x5558de01f400 2024-04-25T10:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.390+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.390+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7000 con 0x5558debd7400 2024-04-25T10:13:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.391+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.391+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e800 con 0x5558debd7400 2024-04-25T10:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.392+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.392+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debbbc00 con 0x5558de01f800 2024-04-25T10:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.392+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.392+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01ec00 con 0x5558debd7400 2024-04-25T10:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.393+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.393+0000 7f0e55000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x5558de030000 0x5558de025600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.393+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030400 con 0x5558de030000 2024-04-25T10:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.394+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x5558de030000 0x5558de025600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.395+0000 7f0e5a000640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x5558de030000 0x5558de025600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.395+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.395+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01fc00 con 0x5558debba800 2024-04-25T10:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.396+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.396+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030800 con 0x5558de01f400 2024-04-25T10:13:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.398+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.398+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7800 con 0x5558de030000 2024-04-25T10:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.398+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.398+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030c00 con 0x5558debba000 2024-04-25T10:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.399+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.399+0000 7f0e54600640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5558de030400 0x5558de024b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.399+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031000 con 0x5558de030400 2024-04-25T10:13:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.400+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5558de030400 0x5558de024b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.401+0000 7f0e5b400640 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5558de030400 0x5558de024b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.402+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.402+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 122 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031400 con 0x5558debd7400 2024-04-25T10:13:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.403+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 6 ==== osd_op_reply(21 [pgnls-filter start_epoch 122 out=81b] v122'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.403+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7c00 con 0x5558debd7400 2024-04-25T10:13:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.404+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.404+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031800 con 0x5558de01e000 2024-04-25T10:13:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.405+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.405+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e400 con 0x5558debba000 2024-04-25T10:13:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.406+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.406+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031c00 con 0x5558debd7400 2024-04-25T10:13:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.407+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.407+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01f000 con 0x5558debba800 2024-04-25T10:13:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.408+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.408+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030800 con 0x5558de01f800 2024-04-25T10:13:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.409+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.409+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debba400 con 0x5558de01f400 2024-04-25T10:13:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.410+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 122 out=49b] v120'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.410+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031400 con 0x5558de01f400 2024-04-25T10:13:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.411+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 5 ==== osd_op_reply(29 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.411+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558dea69400 con 0x5558debd7400 2024-04-25T10:13:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.412+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.412+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e800 con 0x5558debd7400 2024-04-25T10:13:21.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.413+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 10 ==== osd_op_reply(31 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.413+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031c00 con 0x5558de01f800 2024-04-25T10:13:21.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.414+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.414+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01ec00 con 0x5558debd7400 2024-04-25T10:13:21.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.415+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 11 ==== osd_op_reply(33 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.415+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e800 con 0x5558de030000 2024-04-25T10:13:21.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.416+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.416+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7000 con 0x5558debba800 2024-04-25T10:13:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.417+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.417+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e400 con 0x5558de01f400 2024-04-25T10:13:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.419+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.419+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031800 con 0x5558de030000 2024-04-25T10:13:21.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.419+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(37 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.420+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debba400 con 0x5558debba000 2024-04-25T10:13:21.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.420+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.420+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031400 con 0x5558de030400 2024-04-25T10:13:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.421+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== osd_op_reply(39 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.421+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debbbc00 con 0x5558debd7400 2024-04-25T10:13:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.423+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 12 ==== osd_op_reply(40 [pgnls-filter start_epoch 122 out=49b] v122'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.423+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7800 con 0x5558debd7400 2024-04-25T10:13:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.424+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 13 ==== osd_op_reply(41 [pgnls-filter start_epoch 122 out=81b] v122'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.424+0000 7f0e5d419740 20 datascan.forall_objects: OSD matched oid 10000000003.00000000 2024-04-25T10:13:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.424+0000 7f0e5d419740 10 datascan.operator(): handling object 10000000003.0 2024-04-25T10:13:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.424+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:42 18.32 18:4def72d4:::10000000003.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031000 con 0x5558debd7400 2024-04-25T10:13:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.425+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 14 ==== osd_op_reply(42 10000000003.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.425+0000 7f0e5d419740 10 datascan.inject_with_backtrace: inode: 0x0x10000000003 2024-04-25T10:13:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.425+0000 7f0e5d419740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T10:13:21.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.425+0000 7f0e5d419740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T10:13:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.425+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:43 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030800 con 0x5558debba800 2024-04-25T10:13:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.427+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 6 ==== osd_op_reply(43 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.427+0000 7f0e5d419740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T10:13:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.427+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:44 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028000 con 0x5558de01f400 2024-04-25T10:13:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.428+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 7 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+585 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.428+0000 7f0e5d419740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T10:13:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.428+0000 7f0e5d419740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:13:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.428+0000 7f0e5d419740 20 datascan.get_frag_of: frag is * 2024-04-25T10:13:21.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.428+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:45 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028400 con 0x5558debba800 2024-04-25T10:13:21.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.429+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 7 ==== osd_op_reply(45 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.429+0000 7f0e5d419740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T10:13:21.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.429+0000 7f0e5d419740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T10:13:21.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.429+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028800 con 0x5558debba800 2024-04-25T10:13:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.431+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.431+0000 7f0e5d419740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:13:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.431+0000 7f0e5d419740 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T10:13:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.431+0000 7f0e5d419740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000003 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T10:13:21.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.431+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:47 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028c00 con 0x5558debba800 2024-04-25T10:13:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.435+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 9 ==== osd_op_reply(47 10000000001.00000000 [omap-set-vals] v122'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.436+0000 7f0e5d419740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:13:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.436+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:48 18.a 18:50000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de029000 con 0x5558de030400 2024-04-25T10:13:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.437+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.437+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:49 18.2a 18:54000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7000 con 0x5558de01f800 2024-04-25T10:13:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.438+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.438+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:50 18.1a 18:58000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558dea69400 con 0x5558de030400 2024-04-25T10:13:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.439+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 4 ==== osd_op_reply(50 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.439+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:51 18.3a 18:5c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01fc00 con 0x5558debba800 2024-04-25T10:13:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.441+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 10 ==== osd_op_reply(51 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.441+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:52 18.6 18:60000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7c00 con 0x5558de030000 2024-04-25T10:13:21.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.442+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 5 ==== osd_op_reply(52 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.442+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:53 18.26 18:64000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01f000 con 0x5558de01f400 2024-04-25T10:13:21.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.443+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 8 ==== osd_op_reply(53 [pgnls-filter start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.443+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:54 18.16 18:68000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7c00 con 0x5558de030400 2024-04-25T10:13:21.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.444+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 5 ==== osd_op_reply(54 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.444+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:55 18.36 18:6c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e800 con 0x5558debba800 2024-04-25T10:13:21.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.445+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 11 ==== osd_op_reply(55 [pgnls-filter start_epoch 122 out=49b] v120'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.446+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7000 con 0x5558debba800 2024-04-25T10:13:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.447+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 12 ==== osd_op_reply(56 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.447+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028400 con 0x5558debd7400 2024-04-25T10:13:21.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.448+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 15 ==== osd_op_reply(57 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.448+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debba400 con 0x5558de01e000 2024-04-25T10:13:21.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.449+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(58 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.449+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031800 con 0x5558debba800 2024-04-25T10:13:21.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.450+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 13 ==== osd_op_reply(59 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.451+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debbbc00 con 0x5558debba800 2024-04-25T10:13:21.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.452+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 14 ==== osd_op_reply(60 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.452+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030800 con 0x5558de01e000 2024-04-25T10:13:21.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.453+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 4 ==== osd_op_reply(61 [pgnls-filter start_epoch 122 out=49b] v120'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.453+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01fc00 con 0x5558debba800 2024-04-25T10:13:21.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.454+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 15 ==== osd_op_reply(62 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.454+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031c00 con 0x5558de01f800 2024-04-25T10:13:21.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.455+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 6 ==== osd_op_reply(63 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.455+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01ec00 con 0x5558debd7400 2024-04-25T10:13:21.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.456+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 122 out=49b] v120'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.457+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031400 con 0x5558de01f800 2024-04-25T10:13:21.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.458+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.1 v2:172.21.15.5:6800/692601986 7 ==== osd_op_reply(65 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de01f800 2024-04-25T10:13:21.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.458+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e400 con 0x5558de01e000 2024-04-25T10:13:21.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.459+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 122 out=49b] v120'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.459+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031000 con 0x5558debba800 2024-04-25T10:13:21.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.460+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 16 ==== osd_op_reply(67 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.460+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558dea69400 con 0x5558de030400 2024-04-25T10:13:21.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.461+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 6 ==== osd_op_reply(68 [pgnls-filter start_epoch 122 out=49b] v122'12 uv12 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.461+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030c00 con 0x5558debba000 2024-04-25T10:13:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.462+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 122 out=49b] v122'17 uv17 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.463+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028800 con 0x5558de030000 2024-04-25T10:13:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.464+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 6 ==== osd_op_reply(70 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.464+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7800 con 0x5558de01e000 2024-04-25T10:13:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.465+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 6 ==== osd_op_reply(71 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.465+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e800 con 0x5558de01e000 2024-04-25T10:13:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.466+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 7 ==== osd_op_reply(72 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.466+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028c00 con 0x5558debba000 2024-04-25T10:13:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.467+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.467+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01fc00 con 0x5558debba800 2024-04-25T10:13:21.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.468+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 17 ==== osd_op_reply(74 [pgnls-filter start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.468+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de029000 con 0x5558de030400 2024-04-25T10:13:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.469+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.469+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de01e400 con 0x5558de01f400 2024-04-25T10:13:21.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.471+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.471+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028000 con 0x5558de030400 2024-04-25T10:13:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.472+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 8 ==== osd_op_reply(77 [pgnls-filter start_epoch 122 out=49b] v120'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.472+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031800 con 0x5558debba000 2024-04-25T10:13:21.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.473+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 122 out=49b] v120'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.473+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de028c00 con 0x5558debba800 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.474+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.5 v2:172.21.15.80:6800/161144277 18 ==== osd_op_reply(79 [pgnls-filter start_epoch 122 out=49b] v120'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558debba800 2024-04-25T10:13:21.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.474+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030800 con 0x5558de01e000 2024-04-25T10:13:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.475+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.476+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de029000 con 0x5558debba000 2024-04-25T10:13:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.477+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 9 ==== osd_op_reply(81 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.477+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debba400 con 0x5558de01f400 2024-04-25T10:13:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.478+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.4 v2:172.21.15.80:6808/2045277337 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558de01f400 2024-04-25T10:13:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.478+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031c00 con 0x5558debba000 2024-04-25T10:13:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.479+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 10 ==== osd_op_reply(83 [pgnls-filter start_epoch 122 out=49b] v120'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.479+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debbbc00 con 0x5558de030400 2024-04-25T10:13:21.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.480+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.7 v2:172.21.15.80:6816/4166609972 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 122 out=49b] v120'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558de030400 2024-04-25T10:13:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.480+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031400 con 0x5558de01e000 2024-04-25T10:13:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.481+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 9 ==== osd_op_reply(85 [pgnls-filter start_epoch 122 out=49b] v120'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.481+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7000 con 0x5558debba000 2024-04-25T10:13:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.482+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 11 ==== osd_op_reply(86 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.482+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030c00 con 0x5558de030000 2024-04-25T10:13:21.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.483+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 7 ==== osd_op_reply(87 [pgnls-filter start_epoch 122 out=49b] v120'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.483+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de031000 con 0x5558debba000 2024-04-25T10:13:21.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.484+0000 7f0e5b400640 1 -- 172.21.15.5:0/1852679891 <== osd.0 v2:172.21.15.5:6809/1760674718 12 ==== osd_op_reply(88 [pgnls-filter start_epoch 122 out=49b] v120'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc500 con 0x5558debba000 2024-04-25T10:13:21.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.484+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558dea69400 con 0x5558debd7400 2024-04-25T10:13:21.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.485+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 <== osd.6 v2:172.21.15.80:6824/3989257232 17 ==== osd_op_reply(89 [pgnls-filter start_epoch 122 out=49b] v120'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc780 con 0x5558debd7400 2024-04-25T10:13:21.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.485+0000 7f0e54600640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558de030800 con 0x5558de030000 2024-04-25T10:13:21.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.486+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.2 v2:172.21.15.5:6822/3928063993 8 ==== osd_op_reply(90 [pgnls-filter start_epoch 122 out=49b] v120'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de030000 2024-04-25T10:13:21.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.486+0000 7f0e55000640 1 -- 172.21.15.5:0/1852679891 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 122 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x5558debd7c00 con 0x5558de01e000 2024-04-25T10:13:21.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.487+0000 7f0e5a000640 1 -- 172.21.15.5:0/1852679891 <== osd.3 v2:172.21.15.5:6808/3976284996 10 ==== osd_op_reply(91 [pgnls-filter start_epoch 122 out=49b] v120'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5558de9fc000 con 0x5558de01e000 2024-04-25T10:13:21.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5558debba000 msgr2=0x5558ddd1fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x5558debba000 0x5558ddd1fb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x5558de01f800 msgr2=0x5558de024580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x5558de01f800 0x5558de024580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x5558de030000 msgr2=0x5558de025600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x5558de030000 0x5558de025600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5558de01e000 msgr2=0x5558de024000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x5558de01e000 0x5558de024000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x5558de01f400 msgr2=0x5558de025080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 reap_dead start 2024-04-25T10:13:21.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x5558de01f400 0x5558de025080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5558debba800 msgr2=0x5558ddd20100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x5558debba800 0x5558ddd20100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5558debd7400 msgr2=0x5558ddd1f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x5558debd7400 0x5558ddd1f600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5558de030400 msgr2=0x5558de024b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.488+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x5558de030400 0x5558de024b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x5558debd6c00 msgr2=0x5558ddd21700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x5558debd6c00 0x5558ddd21700 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x5558deb64ab0 tx=0x5558dea5c180 comp rx=0 tx=0).stop 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558debbb000 msgr2=0x5558ddd20c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 --2- 172.21.15.5:0/1852679891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558debbb000 0x5558ddd20c00 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x5558deb65d40 tx=0x5558debc4de0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 reap_dead start 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5aa00640 1 -- 172.21.15.5:0/1852679891 reap_dead start 2024-04-25T10:13:21.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 shutdown_connections 2024-04-25T10:13:21.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 >> 172.21.15.5:0/1852679891 conn(0x5558dea69c00 msgr2=0x5558ddd7f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 shutdown_connections 2024-04-25T10:13:21.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.489+0000 7f0e5d419740 1 -- 172.21.15.5:0/1852679891 wait complete. 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.490+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 msgr2=0x5558ddd1eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.490+0000 7f0e5d419740 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 0x5558ddd1eb00 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x5558dea9b140 tx=0x5558dea5c360 comp rx=0 tx=0).stop 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 shutdown_connections 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5558ddc7fc00 0x5558ddd1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558ddc7f800 0x5558ddd1eb00 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 --2- 172.21.15.5:0/146493455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5558ddc7ec00 0x5558ddd1e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 >> 172.21.15.5:0/146493455 conn(0x5558ddc7f000 msgr2=0x5558ddd7f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 shutdown_connections 2024-04-25T10:13:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.491+0000 7f0e5d419740 1 -- 172.21.15.5:0/146493455 wait complete. 2024-04-25T10:13:21.505 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.250210, result: 2024-04-25T10:13:21.506 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.612+0000 7f26a7a3d740 1 -- start start 2024-04-25T10:13:21.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.612+0000 7f26a7a3d740 1 Processor -- start 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617734eac00 0x56177358a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 0x56177358ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617734ebc00 0x56177358b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617742d4600 con 0x5617734eb800 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617742d4480 con 0x5617734eac00 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617742d4300 con 0x5617734ebc00 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 0x56177358ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 0x56177358ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47434/0 (socket says 172.21.15.5:47434) 2024-04-25T10:13:21.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a4200640 1 -- 172.21.15.5:0/920700882 learned_addr learned my addr 172.21.15.5:0/920700882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a5600640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617734ebc00 0x56177358b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.613+0000 7f26a4c00640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617734eac00 0x56177358a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4200640 1 -- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617734ebc00 msgr2=0x56177358b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4200640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617734ebc00 0x56177358b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4200640 1 -- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617734eac00 msgr2=0x56177358a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4200640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617734eac00 0x56177358a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4200640 1 -- 172.21.15.5:0/920700882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561773517680 con 0x5617734eb800 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a5600640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617734ebc00 0x56177358b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4c00640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617734eac00 0x56177358a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a4200640 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 0x56177358ab00 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x56177430b260 tx=0x56177427b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc946c25b65c4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a1a00640 1 -- 172.21.15.5:0/920700882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561773510680 con 0x5617734eb800 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a1a00640 1 -- 172.21.15.5:0/920700882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617735341c0 con 0x5617734eb800 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a1a00640 1 -- 172.21.15.5:0/920700882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617735109c0 con 0x5617734eb800 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561773517a40 con 0x5617734eb800 2024-04-25T10:13:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5617735172c0 con 0x5617734eb800 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.614+0000 7f26a7a3d740 4 waiting for MDS map... 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.615+0000 7f26a1a00640 1 -- 172.21.15.5:0/920700882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x5617743ae400 con 0x5617734eb800 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.615+0000 7f26a1a00640 1 -- 172.21.15.5:0/920700882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x561773543180 con 0x5617734eb800 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.615+0000 7f26a7a3d740 4 Got MDS map 106 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.615+0000 7f26a7a3d740 4 datascan.main: Using metadata pool output 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.615+0000 7f26a7a3d740 4 datascan.main: connecting to RADOS... 2024-04-25T10:13:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.616+0000 7f26a7a3d740 1 Processor -- start 2024-04-25T10:13:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.616+0000 7f26a7a3d740 1 -- start start 2024-04-25T10:13:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617742b5c00 0x56177358b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56177440a000 0x56177358bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 0x56177358c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617742d4180 con 0x56177440a400 2024-04-25T10:13:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617742d4900 con 0x5617742b5c00 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x56177354a900 con 0x56177440a000 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 0x56177358c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 0x56177358c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47448/0 (socket says 172.21.15.5:47448) 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a4200640 1 -- 172.21.15.5:0/3792439119 learned_addr learned my addr 172.21.15.5:0/3792439119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a4200640 1 -- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56177440a000 msgr2=0x56177358bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.617+0000 7f26a4c00640 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56177440a000 0x56177358bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f26a4200640 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56177440a000 0x56177358bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f26a5600640 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617742b5c00 0x56177358b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f26a4200640 1 -- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617742b5c00 msgr2=0x56177358b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f26a4200640 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617742b5c00 0x56177358b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f26a4200640 1 -- 172.21.15.5:0/3792439119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617735174a0 con 0x56177440a400 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f26a4200640 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 0x56177358c100 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x5617743bc870 tx=0x56177426ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbabcfcc2b02844f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f269f200640 1 -- 172.21.15.5:0/3792439119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561773510680 con 0x56177440a400 2024-04-25T10:13:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f269f200640 1 -- 172.21.15.5:0/3792439119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561773534e00 con 0x56177440a400 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.618+0000 7f269f200640 1 -- 172.21.15.5:0/3792439119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5617735109c0 con 0x56177440a400 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 -- 172.21.15.5:0/3792439119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 msgr2=0x56177358c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 0x56177358c100 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x5617743bc870 tx=0x56177426ca80 comp rx=0 tx=0).stop 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 -- 172.21.15.5:0/3792439119 shutdown_connections 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440a400 0x56177358c100 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x56177440a000 0x56177358bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/3792439119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617742b5c00 0x56177358b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 -- 172.21.15.5:0/3792439119 >> 172.21.15.5:0/3792439119 conn(0x5617742b5800 msgr2=0x5617735f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 -- 172.21.15.5:0/3792439119 shutdown_connections 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 -- 172.21.15.5:0/3792439119 wait complete. 2024-04-25T10:13:21.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.619+0000 7f26a7a3d740 1 Processor -- start 2024-04-25T10:13:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.620+0000 7f26a7a3d740 1 -- start start 2024-04-25T10:13:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.620+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617742b5800 0x56177358d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.620+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56177440b000 0x56177358cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a7a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 0x56177358c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617742d5500 con 0x56177440ac00 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617742d5380 con 0x56177440b000 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a7a3d740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617742d5680 con 0x5617742b5800 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 0x56177358c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 0x56177358c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47464/0 (socket says 172.21.15.5:47464) 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 learned_addr learned my addr 172.21.15.5:0/2421944319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617742b5800 0x56177358d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56177440b000 0x56177358cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617742b5800 msgr2=0x56177358d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617742b5800 0x56177358d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56177440b000 msgr2=0x56177358cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x56177440b000 0x56177358cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.621+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617735170e0 con 0x56177440ac00 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 0x56177358c680 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x5617743bcba0 tx=0x56177427aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f269d400640 1 -- 172.21.15.5:0/2421944319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561773510b60 con 0x56177440ac00 2024-04-25T10:13:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f269d400640 1 -- 172.21.15.5:0/2421944319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561773534c40 con 0x56177440ac00 2024-04-25T10:13:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5617735163c0 con 0x56177440ac00 2024-04-25T10:13:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f269d400640 1 -- 172.21.15.5:0/2421944319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561773510680 con 0x56177440ac00 2024-04-25T10:13:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561773516b40 con 0x56177440ac00 2024-04-25T10:13:21.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.622+0000 7f26a7a3d740 4 datascan.init: resolving metadata pool 17 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.624+0000 7f269d400640 1 -- 172.21.15.5:0/2421944319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x561774412c00 con 0x56177440ac00 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f269d400640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x561773834000 0x56177358d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f269d400640 1 -- 172.21.15.5:0/2421944319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x561774410a00 con 0x56177440ac00 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f26a7a3d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f26a7a3d740 4 datascan.main: resolving metadata pool 17 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56177440a800 0x56177358c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177440a400 con 0x56177440a800 2024-04-25T10:13:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56177440a800 0x56177358c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.625+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x561773834000 0x56177358d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.626+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56177440a800 0x56177358c100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.626+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== osd_op_reply(1 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.626+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x56177440a400 0x561773856000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.626+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835000 con 0x56177440a400 2024-04-25T10:13:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.626+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x561773834000 0x56177358d700 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x56177427ce40 tx=0x561773854000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.627+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x56177440a400 0x561773856000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.628+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x56177440a400 0x561773856000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.629+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== osd_op_reply(2 [pgnls start_epoch 122 out=177b] v121'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.629+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835000 con 0x56177440a800 2024-04-25T10:13:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.630+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== osd_op_reply(3 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.630+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x561773835000 0x561773856580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.630+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835400 con 0x561773835000 2024-04-25T10:13:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.630+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x561773835000 0x561773856580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.632+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x561773835000 0x561773856580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.633+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== osd_op_reply(4 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.633+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x561773835400 0x561773856b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.633+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835800 con 0x561773835400 2024-04-25T10:13:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.633+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x561773835400 0x561773856b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.634+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x561773835400 0x561773856b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.635+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== osd_op_reply(5 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.635+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x561773835800 0x561773857080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.635+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177440bc00 con 0x561773835800 2024-04-25T10:13:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.636+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x561773835800 0x561773857080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.637+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x561773835800 0x561773857080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.638+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== osd_op_reply(6 [pgnls start_epoch 122 out=149b] v122'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.638+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x56177440bc00 0x561773857600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.638+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773834c00 con 0x56177440bc00 2024-04-25T10:13:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.638+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x56177440bc00 0x561773857600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.639+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x56177440bc00 0x561773857600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.640+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== osd_op_reply(7 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.640+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773834c00 con 0x561773835800 2024-04-25T10:13:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.641+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== osd_op_reply(8 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.641+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x561773834c00 0x561773857b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.641+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835c00 con 0x561773834c00 2024-04-25T10:13:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.641+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x561773834c00 0x561773857b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.642+0000 7f26a4200640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x561773834c00 0x561773857b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.642+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== osd_op_reply(9 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.642+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835c00 con 0x561773835000 2024-04-25T10:13:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.643+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== osd_op_reply(10 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.643+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835c00 con 0x561773834c00 2024-04-25T10:13:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.644+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== osd_op_reply(11 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.644+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835c00 con 0x56177440a400 2024-04-25T10:13:21.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.645+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== osd_op_reply(12 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.645+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773835c00 con 0x56177440a800 2024-04-25T10:13:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.646+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 3 ==== osd_op_reply(13 [pgnls start_epoch 122 out=77b] v121'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.646+0000 7f26a4c00640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x561773835c00 0x561773858680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.646+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835c00 2024-04-25T10:13:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.646+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x561773835c00 0x561773858680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.647+0000 7f26a5600640 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x561773835c00 0x561773858680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.647+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== osd_op_reply(14 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x561773835c00 2024-04-25T10:13:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.647+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835400 2024-04-25T10:13:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.648+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== osd_op_reply(15 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.648+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835400 2024-04-25T10:13:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.649+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 3 ==== osd_op_reply(16 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.649+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835800 2024-04-25T10:13:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.650+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 3 ==== osd_op_reply(17 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.650+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440a400 2024-04-25T10:13:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.651+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 3 ==== osd_op_reply(18 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.651+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835000 2024-04-25T10:13:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.652+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 3 ==== osd_op_reply(19 [pgnls start_epoch 122 out=81b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.652+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440bc00 2024-04-25T10:13:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.653+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== osd_op_reply(20 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.653+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440a800 2024-04-25T10:13:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.653+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 4 ==== osd_op_reply(21 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.653+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440bc00 2024-04-25T10:13:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.654+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 3 ==== osd_op_reply(22 [pgnls start_epoch 122 out=97b] v121'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.654+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440a800 2024-04-25T10:13:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.655+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 5 ==== osd_op_reply(23 [pgnls start_epoch 122 out=74b] v119'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.655+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773834c00 2024-04-25T10:13:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.655+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 3 ==== osd_op_reply(24 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.655+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440a800 2024-04-25T10:13:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.656+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 6 ==== osd_op_reply(25 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.656+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440a400 2024-04-25T10:13:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.657+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 4 ==== osd_op_reply(26 [pgnls start_epoch 122 out=98b] v121'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.657+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835400 2024-04-25T10:13:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.658+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 4 ==== osd_op_reply(27 [pgnls start_epoch 122 out=76b] v122'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.658+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440bc00 2024-04-25T10:13:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.659+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 4 ==== osd_op_reply(28 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.659+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835800 2024-04-25T10:13:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.660+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 4 ==== osd_op_reply(29 [pgnls start_epoch 122 out=105b] v122'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.660+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835c00 2024-04-25T10:13:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.661+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== osd_op_reply(30 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x561773835c00 2024-04-25T10:13:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.661+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835400 2024-04-25T10:13:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.662+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 5 ==== osd_op_reply(31 [pgnls start_epoch 122 out=73b] v122'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.662+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x56177440a800 2024-04-25T10:13:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.663+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 7 ==== osd_op_reply(32 [pgnls start_epoch 122 out=101b] v122'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.663+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:13:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.663+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177440a000 con 0x56177440a400 2024-04-25T10:13:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.664+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.664+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:13:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.664+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880c00 con 0x56177440a400 2024-04-25T10:13:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.665+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.665+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:13:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.665+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881000 con 0x56177440a400 2024-04-25T10:13:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.667+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.667+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:13:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.667+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881400 con 0x56177440a400 2024-04-25T10:13:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.668+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.668+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:13:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.668+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881800 con 0x56177440a400 2024-04-25T10:13:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:13:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:13:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:13:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:13:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:13:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:13:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:13:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.669+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835800 2024-04-25T10:13:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.670+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.670+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:13:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.670+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880c00 con 0x561773835800 2024-04-25T10:13:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.672+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.672+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:13:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.672+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:13:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.672+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:13:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.672+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:40 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881000 con 0x561773835400 2024-04-25T10:13:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.673+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 6 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.673+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 10000000005.00000000 2024-04-25T10:13:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.673+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:41 17.9 17:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881400 con 0x561773835000 2024-04-25T10:13:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.674+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 4 ==== osd_op_reply(41 10000000005.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+592 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.674+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:13:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.674+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881800 con 0x56177440bc00 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 5 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:13:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.675+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177440a000 con 0x561773834c00 2024-04-25T10:13:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.676+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 4 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.676+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:13:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.676+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:13:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.676+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:13:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.676+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880800 con 0x56177440a400 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.677+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 10 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.677+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.677+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.677+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:13:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.677+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880400 con 0x561773835800 2024-04-25T10:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.678+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.678+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T10:13:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.678+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835800 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1683 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 20 datascan.scan_links: injected ino detected 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.680+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881400 con 0x56177440a800 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.681+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 8 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.681+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881800 con 0x56177440a800 2024-04-25T10:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.681+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 9 ==== osd_op_reply(48 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.682+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881800 con 0x56177440a400 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.683+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 11 ==== osd_op_reply(49 [pgnls start_epoch 122 out=177b] v121'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.683+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a800 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.684+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 10 ==== osd_op_reply(50 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.684+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835000 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.685+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 5 ==== osd_op_reply(51 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.685+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835400 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.686+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 7 ==== osd_op_reply(52 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.686+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835800 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.687+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 9 ==== osd_op_reply(53 [pgnls start_epoch 122 out=149b] v122'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.687+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440bc00 2024-04-25T10:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.688+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 6 ==== osd_op_reply(54 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.688+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835800 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.689+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 10 ==== osd_op_reply(55 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.689+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773834c00 2024-04-25T10:13:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.690+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 5 ==== osd_op_reply(56 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.690+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835000 2024-04-25T10:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.691+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 6 ==== osd_op_reply(57 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.691+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773834c00 2024-04-25T10:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.692+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 6 ==== osd_op_reply(58 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.692+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a400 2024-04-25T10:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.694+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 12 ==== osd_op_reply(59 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.694+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a800 2024-04-25T10:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.695+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 11 ==== osd_op_reply(60 [pgnls start_epoch 122 out=77b] v121'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.695+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835c00 2024-04-25T10:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.696+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.2 v2:172.21.15.5:6822/3928063993 3 ==== osd_op_reply(61 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x561773835c00 2024-04-25T10:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.696+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835400 2024-04-25T10:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.697+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 8 ==== osd_op_reply(62 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.697+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835400 2024-04-25T10:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.698+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 9 ==== osd_op_reply(63 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.698+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835800 2024-04-25T10:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.699+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 11 ==== osd_op_reply(64 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.699+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a400 2024-04-25T10:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.700+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 13 ==== osd_op_reply(65 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.700+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835000 2024-04-25T10:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.701+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 7 ==== osd_op_reply(66 [pgnls start_epoch 122 out=81b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.701+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440bc00 2024-04-25T10:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.702+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 7 ==== osd_op_reply(67 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.702+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a800 2024-04-25T10:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.703+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 12 ==== osd_op_reply(68 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.703+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440bc00 2024-04-25T10:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.704+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 8 ==== osd_op_reply(69 [pgnls start_epoch 122 out=97b] v121'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.704+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a800 2024-04-25T10:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.705+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 13 ==== osd_op_reply(70 [pgnls start_epoch 122 out=74b] v119'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.705+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773834c00 2024-04-25T10:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.706+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 7 ==== osd_op_reply(71 [pgnls start_epoch 122 out=73b] v121'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.706+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a800 2024-04-25T10:13:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.707+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 14 ==== osd_op_reply(72 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.707+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a400 2024-04-25T10:13:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.708+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 14 ==== osd_op_reply(73 [pgnls start_epoch 122 out=98b] v121'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.708+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835400 2024-04-25T10:13:21.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.710+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 10 ==== osd_op_reply(74 [pgnls start_epoch 122 out=76b] v122'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.710+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440bc00 2024-04-25T10:13:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.711+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 9 ==== osd_op_reply(75 [pgnls start_epoch 122 out=49b] v122'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.711+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835800 2024-04-25T10:13:21.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.712+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 12 ==== osd_op_reply(76 [pgnls start_epoch 122 out=105b] v122'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.712+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835c00 2024-04-25T10:13:21.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.713+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.2 v2:172.21.15.5:6822/3928063993 4 ==== osd_op_reply(77 [pgnls start_epoch 122 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561774268500 con 0x561773835c00 2024-04-25T10:13:21.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.713+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x561773835400 2024-04-25T10:13:21.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.714+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 11 ==== osd_op_reply(78 [pgnls start_epoch 122 out=73b] v122'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.714+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 122 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881c00 con 0x56177440a800 2024-04-25T10:13:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.715+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 15 ==== osd_op_reply(79 [pgnls start_epoch 122 out=101b] v122'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.715+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:13:21.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.715+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388e000 con 0x56177440a400 2024-04-25T10:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.717+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 15 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.717+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.717+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388e400 con 0x56177440a400 2024-04-25T10:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.718+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 16 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.718+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.718+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388e800 con 0x56177440a400 2024-04-25T10:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.719+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 17 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1169 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.719+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.719+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388ec00 con 0x56177440a400 2024-04-25T10:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.720+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 18 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.720+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.720+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388f000 con 0x56177440a400 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 19 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.722+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388f400 con 0x561773835800 2024-04-25T10:13:21.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.723+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 13 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.723+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:13:21.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.723+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388f800 con 0x561773835800 2024-04-25T10:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.724+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 14 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.724+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.724+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.724+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.724+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- osd_op(unknown.0.0:87 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388fc00 con 0x561773835400 2024-04-25T10:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.725+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.6 v2:172.21.15.80:6824/3989257232 12 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268780 con 0x561773835400 2024-04-25T10:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.726+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 10000000005.00000000 2024-04-25T10:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.726+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- osd_op(unknown.0.0:88 17.9 17:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388e000 con 0x561773835000 2024-04-25T10:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.727+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.7 v2:172.21.15.80:6816/4166609972 8 ==== osd_op_reply(88 10000000005.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+592 (crc 0 0 0) 0x561774268000 con 0x561773835000 2024-04-25T10:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.727+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.727+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388e400 con 0x56177440bc00 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.3 v2:172.21.15.5:6808/3976284996 10 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268000 con 0x56177440bc00 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.728+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x56177388e800 con 0x561773834c00 2024-04-25T10:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.729+0000 7f26a4200640 1 -- 172.21.15.5:0/2421944319 <== osd.1 v2:172.21.15.5:6800/692601986 8 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268780 con 0x561773834c00 2024-04-25T10:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.729+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.729+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.729+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.729+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773881800 con 0x56177440a400 2024-04-25T10:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.730+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 20 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.731+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.731+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.731+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.731+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880000 con 0x561773835800 2024-04-25T10:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.732+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.732+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T10:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.732+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880400 con 0x561773835800 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1683 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 20 datascan.scan_links: injected ino detected 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 20 datascan.scan_links: adding 0x10000000003 for future processing to fix dnfirst 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.733+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880800 con 0x56177440a800 2024-04-25T10:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 16 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+585 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 10 datascan.scan_links: handling injected_ino 0x10000000003 2024-04-25T10:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T10:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.734+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773880c00 con 0x561773835800 2024-04-25T10:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.735+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.736+0000 7f26a7a3d740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.736+0000 7f26a7a3d740 20 datascan.scan_links: first is now 2 2024-04-25T10:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.736+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773894000 con 0x561773835800 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.740+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.5 v2:172.21.15.80:6800/161144277 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v122'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561774268500 con 0x561773835800 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.740+0000 7f26a7a3d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.740+0000 7f26a7a3d740 10 datascan.scan_links: updating inotable 2024-04-25T10:13:21.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.740+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773894400 con 0x56177440a400 2024-04-25T10:13:21.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.741+0000 7f26a5600640 1 -- 172.21.15.5:0/2421944319 <== osd.4 v2:172.21.15.80:6808/2045277337 21 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x561774268500 con 0x56177440a400 2024-04-25T10:13:21.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.741+0000 7f26a7a3d740 10 datascan.scan_links: updating snaptable 2024-04-25T10:13:21.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.741+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- osd_op(unknown.0.0:98 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561773894800 con 0x56177440a800 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.742+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 <== osd.0 v2:172.21.15.5:6809/1760674718 17 ==== osd_op_reply(98 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x561774268000 con 0x56177440a800 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.742+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56177440a800 msgr2=0x56177358c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.742+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x56177440a800 0x56177358c100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x561773834c00 msgr2=0x561773857b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x561773834c00 0x561773857b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x561773835c00 msgr2=0x561773858680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x561773835c00 0x561773858680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x56177440bc00 msgr2=0x561773857600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x56177440bc00 0x561773857600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x56177440a400 msgr2=0x561773856000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x56177440a400 0x561773856000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x561773835800 msgr2=0x561773857080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x561773835800 0x561773857080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x561773835400 msgr2=0x561773856b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x561773835400 0x561773856b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 reap_dead start 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 reap_dead start 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x561773835000 msgr2=0x561773856580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x561773835000 0x561773856580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.743+0000 7f26a4c00640 1 -- 172.21.15.5:0/2421944319 reap_dead start 2024-04-25T10:13:21.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x561773834000 msgr2=0x56177358d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x561773834000 0x56177358d700 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x56177427ce40 tx=0x561773854000 comp rx=0 tx=0).stop 2024-04-25T10:13:21.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 msgr2=0x56177358c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 0x56177358c680 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x5617743bcba0 tx=0x56177427aa20 comp rx=0 tx=0).stop 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 shutdown_connections 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56177440ac00 0x56177358c680 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/2421944319 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x561773834000 0x56177358d700 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 >> 172.21.15.5:0/2421944319 conn(0x5617742b5c00 msgr2=0x5617735f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 shutdown_connections 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.744+0000 7f26a7a3d740 1 -- 172.21.15.5:0/2421944319 wait complete. 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.745+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 msgr2=0x56177358ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.745+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 0x56177358ab00 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x56177430b260 tx=0x56177427b7a0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 shutdown_connections 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x5617734ebc00 0x56177358b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617734eb800 0x56177358ab00 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 --2- 172.21.15.5:0/920700882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5617734eac00 0x56177358a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 >> 172.21.15.5:0/920700882 conn(0x5617734eb000 msgr2=0x5617735f2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 shutdown_connections 2024-04-25T10:13:21.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:21.746+0000 7f26a7a3d740 1 -- 172.21.15.5:0/920700882 wait complete. 2024-04-25T10:13:21.762 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.256768, result: 2024-04-25T10:13:21.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:21.772 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.910 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.911 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:21.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-04-25T10:13:21.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-04-25T10:13:21.914 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-04-25T10:13:21.915 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-04-25T10:13:21.916 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:21.917 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:21.918 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:21.919 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = False 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:21.920 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.921 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.922 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.923 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on skipped message = False 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:21.924 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:21.925 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T10:13:21.926 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:21.927 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:21.928 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.929 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:21.930 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = False 2024-04-25T10:13:21.931 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-04-25T10:13:21.932 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = True 2024-04-25T10:13:21.932 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:21.932 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:21.932 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:21.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:21.954 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.084 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.085 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:22.086 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:22.087 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on skipped message = False 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T10:13:22.088 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:22.089 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:22.090 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:22.091 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.092 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T10:13:22.093 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T10:13:22.094 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = True 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = True 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T10:13:22.095 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:22.096 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-04-25T10:13:22.097 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-04-25T10:13:22.098 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on skipped message = False 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 8b6dc2a7-3300-467c-ae7a-376f479db0dd 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:22.099 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:22.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 20 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 20 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:22.101 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 10 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:22.102 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-04-25T10:13:22.103 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-04-25T10:13:22.104 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-04-25T10:13:22.105 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-04-25T10:13:22.106 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-04-25T10:13:22.107 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-04-25T10:13:22.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:13:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.340+0000 7f61d9600640 1 Processor -- start 2024-04-25T10:13:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.340+0000 7f61d9600640 1 -- start start 2024-04-25T10:13:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.341+0000 7f61d9600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d41465d0 0x7f61d415e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.341+0000 7f61d9600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f61d4071840 0x7f61d415f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.341+0000 7f61d9600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d415eed0 con 0x7f61d40ae860 2024-04-25T10:13:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.342+0000 7f61d9600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61d406f080 con 0x7f61d4071840 2024-04-25T10:13:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.342+0000 7f61d9600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406f360 con 0x7f61d41465d0 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.342+0000 7f61d3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f61d4071840 0x7f61d415f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.342+0000 7f61d3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f61d4071840 0x7f61d415f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47468/0 (socket says 172.21.15.5:47468) 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.342+0000 7f61d3400640 1 -- 172.21.15.5:0/683467988 learned_addr learned my addr 172.21.15.5:0/683467988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.342+0000 7f61d2a00640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d41465d0 0x7f61d415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.343+0000 7f61d3400640 1 -- 172.21.15.5:0/683467988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d41465d0 msgr2=0x7f61d415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.343+0000 7f61d3400640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d41465d0 0x7f61d415e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.343+0000 7f61d3400640 1 -- 172.21.15.5:0/683467988 >> v1:172.21.15.80:6789/0 conn(0x7f61d40ae860 legacy=0x7f61d4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.343+0000 7f61d3400640 1 -- 172.21.15.5:0/683467988 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40ae060 con 0x7f61d4071840 2024-04-25T10:13:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.343+0000 7f61d2a00640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d41465d0 0x7f61d415e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.344+0000 7f61d3400640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d4071840 0x7f61d415f0d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009870 tx=0x7f61cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a20609bdf04a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d1600640 1 -- 172.21.15.5:0/683467988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc037070 con 0x7f61d4071840 2024-04-25T10:13:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d1600640 1 -- 172.21.15.5:0/683467988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc032440 con 0x7f61d4071840 2024-04-25T10:13:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d1600640 1 -- 172.21.15.5:0/683467988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc032810 con 0x7f61d4071840 2024-04-25T10:13:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d9600640 1 -- 172.21.15.5:0/683467988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d4071840 msgr2=0x7f61d415f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d9600640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d4071840 0x7f61d415f0d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009870 tx=0x7f61cc02a270 comp rx=0 tx=0).stop 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d9600640 1 -- 172.21.15.5:0/683467988 shutdown_connections 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d9600640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d4071840 0x7f61d415f0d0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d9600640 1 --2- 172.21.15.5:0/683467988 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d41465d0 0x7f61d415e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.345+0000 7f61d9600640 1 -- 172.21.15.5:0/683467988 >> 172.21.15.5:0/683467988 conn(0x7f61d406ab60 msgr2=0x7f61d406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.346+0000 7f61d9600640 1 -- 172.21.15.5:0/683467988 shutdown_connections 2024-04-25T10:13:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.346+0000 7f61d9600640 1 -- 172.21.15.5:0/683467988 wait complete. 2024-04-25T10:13:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.347+0000 7f61d9600640 1 Processor -- start 2024-04-25T10:13:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.347+0000 7f61d9600640 1 -- start start 2024-04-25T10:13:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.348+0000 7f61d9600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61d4071840 0x7f61d4132b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.348+0000 7f61d9600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d40ae860 0x7f61d41e8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.348+0000 7f61d9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 0x7f61d41e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.348+0000 7f61d9600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406e7d0 con 0x7f61d41465d0 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.348+0000 7f61d9600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d41469b0 con 0x7f61d4071840 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.348+0000 7f61d9600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d406ece0 con 0x7f61d40ae860 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 0x7f61d41e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 0x7f61d41e8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47480/0 (socket says 172.21.15.5:47480) 2024-04-25T10:13:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 -- 172.21.15.5:0/1656827583 learned_addr learned my addr 172.21.15.5:0/1656827583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d2a00640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61d4071840 0x7f61d4132b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d2000640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d40ae860 0x7f61d41e8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 -- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d40ae860 msgr2=0x7f61d41e8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d40ae860 0x7f61d41e8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 -- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61d4071840 msgr2=0x7f61d4132b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61d4071840 0x7f61d4132b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.349+0000 7f61d3400640 1 -- 172.21.15.5:0/1656827583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d41e6160 con 0x7f61d41465d0 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.350+0000 7f61d3400640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 0x7f61d41e8e50 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009870 tx=0x7f61cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.350+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc037040 con 0x7f61d41465d0 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.350+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc042070 con 0x7f61d41465d0 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.350+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc030920 con 0x7f61d41465d0 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.350+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d41e5730 con 0x7f61d41465d0 2024-04-25T10:13:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.350+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d41e66e0 con 0x7f61d41465d0 2024-04-25T10:13:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.351+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d4068f60 con 0x7f61d41465d0 2024-04-25T10:13:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.352+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61cc030ac0 con 0x7f61d41465d0 2024-04-25T10:13:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.354+0000 7f61caa00640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f61b006b4d0 0x7f61b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.354+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f61cc0aad80 con 0x7f61d41465d0 2024-04-25T10:13:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.355+0000 7f61d2a00640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f61b006b4d0 0x7f61b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.356+0000 7f61d2a00640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f61b006b4d0 0x7f61b006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f61c4003fd0 tx=0x7f61c4003990 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.359+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61cc0792d0 con 0x7f61d41465d0 2024-04-25T10:13:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:22.606+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f61d406e590 con 0x7f61d41465d0 2024-04-25T10:13:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.291+0000 7f61caa00640 1 -- 172.21.15.5:0/1656827583 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) v1 ==== 162+0+0 (secure 0 0 0) 0x7f61cc07d370 con 0x7f61d41465d0 2024-04-25T10:13:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:13:23.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f61b006b4d0 msgr2=0x7f61b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f61b006b4d0 0x7f61b006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f61c4003fd0 tx=0x7f61c4003990 comp rx=0 tx=0).stop 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 msgr2=0x7f61d41e8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 0x7f61d41e8e50 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009870 tx=0x7f61cc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 shutdown_connections 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61d41465d0 0x7f61d41e8e50 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f61d40ae860 0x7f61d41e8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f61b006b4d0 0x7f61b006d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 --2- 172.21.15.5:0/1656827583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61d4071840 0x7f61d4132b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 >> 172.21.15.5:0/1656827583 conn(0x7f61d406ab60 msgr2=0x7f61d4142e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.294+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 shutdown_connections 2024-04-25T10:13:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.295+0000 7f61d9600640 1 -- 172.21.15.5:0/1656827583 wait complete. 2024-04-25T10:13:23.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.499+0000 7f56c5400640 1 Processor -- start 2024-04-25T10:13:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.499+0000 7f56c5400640 1 -- start start 2024-04-25T10:13:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c5400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56c0141da0 0x7f56c006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c5400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56c006e640 0x7f56c015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c5400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c0071d30 con 0x7f56c015d340 2024-04-25T10:13:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c5400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c0142180 con 0x7f56c006e640 2024-04-25T10:13:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c5400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c006d620 con 0x7f56c0141da0 2024-04-25T10:13:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c4a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f56c015d340 0x7f56c00cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37960/0 (socket says 172.21.15.5:37960) 2024-04-25T10:13:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56c4a00640 1 -- 172.21.15.5:0/2592867319 learned_addr learned my addr 172.21.15.5:0/2592867319 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.501+0000 7f56bf400640 1 --2- 172.21.15.5:0/2592867319 >> v2:172.21.15.5:3300/0 conn(0x7f56c006e640 0x7f56c015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bfe00640 1 --2- 172.21.15.5:0/2592867319 >> v2:172.21.15.80:3300/0 conn(0x7f56c0141da0 0x7f56c006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bf400640 1 -- 172.21.15.5:0/2592867319 >> v2:172.21.15.80:3300/0 conn(0x7f56c0141da0 msgr2=0x7f56c006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bf400640 1 --2- 172.21.15.5:0/2592867319 >> v2:172.21.15.80:3300/0 conn(0x7f56c0141da0 0x7f56c006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bf400640 1 -- 172.21.15.5:0/2592867319 >> v1:172.21.15.5:6789/0 conn(0x7f56c015d340 legacy=0x7f56c00cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bea00640 1 -- 172.21.15.5:0/2592867319 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499367140 0 0) 0x7f56c0071d30 con 0x7f56c015d340 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bf400640 1 -- 172.21.15.5:0/2592867319 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c006d900 con 0x7f56c006e640 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.502+0000 7f56bfe00640 1 --2- 172.21.15.5:0/2592867319 >> v2:172.21.15.80:3300/0 conn(0x7f56c0141da0 0x7f56c006e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.503+0000 7f56bf400640 1 --2- 172.21.15.5:0/2592867319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c015ce00 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f56c006bbb0 tx=0x7f56ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd5d6e2b8c7074b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.503+0000 7f56bea00640 1 -- 172.21.15.5:0/2592867319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac037070 con 0x7f56c006e640 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.503+0000 7f56bea00640 1 -- 172.21.15.5:0/2592867319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac032440 con 0x7f56c006e640 2024-04-25T10:13:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.503+0000 7f56bea00640 1 -- 172.21.15.5:0/2592867319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac032810 con 0x7f56c006e640 2024-04-25T10:13:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 -- 172.21.15.5:0/2592867319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 msgr2=0x7f56c015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 --2- 172.21.15.5:0/2592867319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c015ce00 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f56c006bbb0 tx=0x7f56ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 -- 172.21.15.5:0/2592867319 shutdown_connections 2024-04-25T10:13:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 --2- 172.21.15.5:0/2592867319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c015ce00 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 --2- 172.21.15.5:0/2592867319 >> v2:172.21.15.80:3300/0 conn(0x7f56c0141da0 0x7f56c006e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 -- 172.21.15.5:0/2592867319 >> 172.21.15.5:0/2592867319 conn(0x7f56c0066f20 msgr2=0x7f56c006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 -- 172.21.15.5:0/2592867319 shutdown_connections 2024-04-25T10:13:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.504+0000 7f56c5400640 1 -- 172.21.15.5:0/2592867319 wait complete. 2024-04-25T10:13:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.506+0000 7f56c5400640 1 Processor -- start 2024-04-25T10:13:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.506+0000 7f56c5400640 1 -- start start 2024-04-25T10:13:23.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56c5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c0142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56c5400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0141da0 0x7f56c0159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56c5400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56c015d340 0x7f56c015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56c5400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00cdb90 con 0x7f56c006e640 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56c5400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c00cda10 con 0x7f56c0141da0 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56c5400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c0071000 con 0x7f56c015d340 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c0142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c0142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47492/0 (socket says 172.21.15.5:47492) 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.507+0000 7f56bfe00640 1 -- 172.21.15.5:0/3635109446 learned_addr learned my addr 172.21.15.5:0/3635109446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bf400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0141da0 0x7f56c0159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56c4a00640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56c015d340 0x7f56c015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bfe00640 1 -- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56c015d340 msgr2=0x7f56c015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bfe00640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56c015d340 0x7f56c015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bfe00640 1 -- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0141da0 msgr2=0x7f56c0159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bfe00640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0141da0 0x7f56c0159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bfe00640 1 -- 172.21.15.5:0/3635109446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c00ae660 con 0x7f56c006e640 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56bfe00640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c0142c60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c7f0 tx=0x7f56b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0015070 con 0x7f56c006e640 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b0007a30 con 0x7f56c006e640 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c01395c0 con 0x7f56c006e640 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.508+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0007e00 con 0x7f56c006e640 2024-04-25T10:13:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.509+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c0144ce0 con 0x7f56c006e640 2024-04-25T10:13:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.510+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c0068f60 con 0x7f56c006e640 2024-04-25T10:13:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.511+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56b0018070 con 0x7f56c006e640 2024-04-25T10:13:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.512+0000 7f56b7e00640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.512+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f56b008a3c0 con 0x7f56c006e640 2024-04-25T10:13:23.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.512+0000 7f56bf400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.514+0000 7f56bf400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56a006b4d0 0x7f56a006d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f56ac034000 tx=0x7f56ac02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.516+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56b0058ff0 con 0x7f56c006e640 2024-04-25T10:13:23.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.756+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56c00ccae0 con 0x7f56c006e640 2024-04-25T10:13:23.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.757+0000 7f56b7e00640 1 -- 172.21.15.5:0/3635109446 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f56b00588f0 con 0x7f56c006e640 2024-04-25T10:13:23.762 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:23.296423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:23.762 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 109 2024-04-25T10:13:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56a006b4d0 msgr2=0x7f56a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56a006b4d0 0x7f56a006d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f56ac034000 tx=0x7f56ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 msgr2=0x7f56c0142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c0142c60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c7f0 tx=0x7f56b000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 shutdown_connections 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f56c015d340 0x7f56c015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f56a006b4d0 0x7f56a006d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56c0141da0 0x7f56c0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 --2- 172.21.15.5:0/3635109446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c006e640 0x7f56c0142c60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 >> 172.21.15.5:0/3635109446 conn(0x7f56c0066f20 msgr2=0x7f56c0140d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 shutdown_connections 2024-04-25T10:13:23.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:23.760+0000 7f56c5400640 1 -- 172.21.15.5:0/3635109446 wait complete. 2024-04-25T10:13:24.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:13:23.296423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.5:6835/806266896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 806266896}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 806266896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:24.187 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:replay 2024-04-25T10:13:25.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.410+0000 7f59ace00640 1 Processor -- start 2024-04-25T10:13:25.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.410+0000 7f59ace00640 1 -- start start 2024-04-25T10:13:25.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59ace00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a815de30 0x7f59a815e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59ace00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59a80d0280 0x7f59a80d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59ace00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a80ce2b0 con 0x7f59a815da50 2024-04-25T10:13:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59ace00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a80cf1a0 con 0x7f59a80d0280 2024-04-25T10:13:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59ace00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a80d0c50 con 0x7f59a815de30 2024-04-25T10:13:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59a5e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f59a815da50 0x7f59a80cea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37964/0 (socket says 172.21.15.5:37964) 2024-04-25T10:13:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59a5e00640 1 -- 172.21.15.5:0/2633398209 learned_addr learned my addr 172.21.15.5:0/2633398209 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:25.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.412+0000 7f59a6800640 1 --2- 172.21.15.5:0/2633398209 >> v2:172.21.15.5:3300/0 conn(0x7f59a80d0280 0x7f59a80d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.413+0000 7f59a5400640 1 --2- 172.21.15.5:0/2633398209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a815de30 0x7f59a815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.413+0000 7f59a4a00640 1 -- 172.21.15.5:0/2633398209 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931744536 0 0) 0x7f59a80ce2b0 con 0x7f59a815da50 2024-04-25T10:13:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.413+0000 7f59a6800640 1 -- 172.21.15.5:0/2633398209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a815de30 msgr2=0x7f59a815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.413+0000 7f59a6800640 1 --2- 172.21.15.5:0/2633398209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a815de30 0x7f59a815e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.413+0000 7f59a6800640 1 -- 172.21.15.5:0/2633398209 >> v1:172.21.15.5:6789/0 conn(0x7f59a815da50 legacy=0x7f59a80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.413+0000 7f59a6800640 1 -- 172.21.15.5:0/2633398209 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a815e960 con 0x7f59a80d0280 2024-04-25T10:13:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.414+0000 7f59a6800640 1 --2- 172.21.15.5:0/2633398209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a80d0280 0x7f59a80d0710 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009d90 tx=0x7f59a002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458d357b5d323e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.414+0000 7f59a4a00640 1 -- 172.21.15.5:0/2633398209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007740 con 0x7f59a80d0280 2024-04-25T10:13:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.414+0000 7f59a4a00640 1 -- 172.21.15.5:0/2633398209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a00078e0 con 0x7f59a80d0280 2024-04-25T10:13:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.414+0000 7f59a4a00640 1 -- 172.21.15.5:0/2633398209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007cb0 con 0x7f59a80d0280 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 -- 172.21.15.5:0/2633398209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a80d0280 msgr2=0x7f59a80d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 --2- 172.21.15.5:0/2633398209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a80d0280 0x7f59a80d0710 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f59a0009d90 tx=0x7f59a002a660 comp rx=0 tx=0).stop 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 -- 172.21.15.5:0/2633398209 shutdown_connections 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 --2- 172.21.15.5:0/2633398209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a80d0280 0x7f59a80d0710 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 --2- 172.21.15.5:0/2633398209 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a815de30 0x7f59a815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 -- 172.21.15.5:0/2633398209 >> 172.21.15.5:0/2633398209 conn(0x7f59a8072290 msgr2=0x7f59a8072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 -- 172.21.15.5:0/2633398209 shutdown_connections 2024-04-25T10:13:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.415+0000 7f59ace00640 1 -- 172.21.15.5:0/2633398209 wait complete. 2024-04-25T10:13:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.417+0000 7f59ace00640 1 Processor -- start 2024-04-25T10:13:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.417+0000 7f59ace00640 1 -- start start 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.418+0000 7f59ace00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a80d0280 0x7f59a8157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.418+0000 7f59ace00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 0x7f59a8155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.418+0000 7f59ace00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a815de30 0x7f59a81f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.418+0000 7f59ace00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8161640 con 0x7f59a815da50 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.418+0000 7f59ace00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a80ce570 con 0x7f59a815de30 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.418+0000 7f59ace00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a81617c0 con 0x7f59a80d0280 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 0x7f59a8155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 0x7f59a8155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47524/0 (socket says 172.21.15.5:47524) 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 -- 172.21.15.5:0/2331552126 learned_addr learned my addr 172.21.15.5:0/2331552126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5e00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a80d0280 0x7f59a8157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a6800640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a815de30 0x7f59a81f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 -- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a80d0280 msgr2=0x7f59a8157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a80d0280 0x7f59a8157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 -- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a815de30 msgr2=0x7f59a81f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a815de30 0x7f59a81f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.419+0000 7f59a5400640 1 -- 172.21.15.5:0/2331552126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a81560b0 con 0x7f59a815da50 2024-04-25T10:13:25.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.420+0000 7f59a5400640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 0x7f59a8155880 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f599400d620 tx=0x7f599400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.420+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59940125f0 con 0x7f59a815da50 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.420+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5994019070 con 0x7f59a815da50 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.420+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59940129d0 con 0x7f59a815da50 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.420+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a8156390 con 0x7f59a815da50 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.420+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a8156950 con 0x7f59a815da50 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.421+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a80b7180 con 0x7f59a815da50 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.422+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5994012b70 con 0x7f59a815da50 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.424+0000 7f599e000640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f598806b4d0 0x7f598806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.424+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f599408a840 con 0x7f59a815da50 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.425+0000 7f59a5e00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f598806b4d0 0x7f598806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.426+0000 7f59a5e00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f598806b4d0 0x7f598806d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5998000f60 tx=0x7f5998007b50 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.429+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59a80b7180 con 0x7f59a815da50 2024-04-25T10:13:25.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.671+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59a8068fd0 con 0x7f59a815da50 2024-04-25T10:13:25.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.672+0000 7f599e000640 1 -- 172.21.15.5:0/2331552126 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f5994059470 con 0x7f59a815da50 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:24.311343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 111 2024-04-25T10:13:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f598806b4d0 msgr2=0x7f598806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f598806b4d0 0x7f598806d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5998000f60 tx=0x7f5998007b50 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 msgr2=0x7f59a8155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 0x7f59a8155880 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f599400d620 tx=0x7f599400dae0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 shutdown_connections 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f598806b4d0 0x7f598806d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a815de30 0x7f59a81f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59a815da50 0x7f59a8155880 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 --2- 172.21.15.5:0/2331552126 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f59a80d0280 0x7f59a8157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 >> 172.21.15.5:0/2331552126 conn(0x7f59a8072290 msgr2=0x7f59a80b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 shutdown_connections 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:25.675+0000 7f59ace00640 1 -- 172.21.15.5:0/2331552126 wait complete. 2024-04-25T10:13:26.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:13:24.311343+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.5:6835/806266896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 806266896}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 806266896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:26.102 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9402:up:rejoin 2024-04-25T10:13:26.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:26.434 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:27.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.314+0000 7f2acf200640 1 Processor -- start 2024-04-25T10:13:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.314+0000 7f2acf200640 1 -- start start 2024-04-25T10:13:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acf200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ad0161200 0x7f2ad00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acf200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00cf410 con 0x7f2ad00d2380 2024-04-25T10:13:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acf200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad00cf6f0 con 0x7f2ad00d07d0 2024-04-25T10:13:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acf200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00d27c0 con 0x7f2ad0161200 2024-04-25T10:13:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acd400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2ad00d2380 0x7f2ad00d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54326/0 (socket says 172.21.15.5:54326) 2024-04-25T10:13:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acd400640 1 -- 172.21.15.5:0/1794045807 learned_addr learned my addr 172.21.15.5:0/1794045807 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707178499 0 0) 0x7f2ad00cf410 con 0x7f2ad00d2380 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.316+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab4003520 con 0x7f2ad00d2380 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acde00640 1 --2- 172.21.15.5:0/1794045807 >> v2:172.21.15.80:3300/0 conn(0x7f2ad0161200 0x7f2ad00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1539258420 0 0) 0x7f2ab4003520 con 0x7f2ad00d2380 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 >> v2:172.21.15.80:3300/0 conn(0x7f2ad0161200 msgr2=0x7f2ad00b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 --2- 172.21.15.5:0/1794045807 >> v2:172.21.15.80:3300/0 conn(0x7f2ad0161200 0x7f2ad00b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 >> v1:172.21.15.80:6789/0 conn(0x7f2ad00d07d0 legacy=0x7f2ad00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00d3fd0 con 0x7f2ad00d2380 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f2abc001670 con 0x7f2ad00d2380 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 >> v1:172.21.15.5:6789/0 conn(0x7f2ad00d2380 legacy=0x7f2ad00d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab4004800 0x7f2ab4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab4005210 0x7f2ab400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00d27c0 con 0x7f2ab4004800 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab400c140 con 0x7f2ab4003a00 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab400c340 con 0x7f2ab4005210 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.317+0000 7f2acd400640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab4004800 0x7f2ab4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acde00640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acd400640 1 -- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab4005210 msgr2=0x7f2ab400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2ace800640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab4005210 0x7f2ab400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acd400640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ab4005210 0x7f2ab400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acd400640 1 -- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4003a00 msgr2=0x7f2ab4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acd400640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acd400640 1 -- 172.21.15.5:0/1794045807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4003520 con 0x7f2ab4004800 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2ace800640 1 -- 172.21.15.5:0/1794045807 reap_dead start 2024-04-25T10:13:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.318+0000 7f2acde00640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4003a00 0x7f2ab4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.319+0000 7f2acd400640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab4004800 0x7f2ab4004c80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2abc001eb0 tx=0x7f2ab4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea7a865a7a34095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.319+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc007c80 con 0x7f2ab4004800 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.319+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc005af0 con 0x7f2ab4004800 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.319+0000 7f2acca00640 1 -- 172.21.15.5:0/1794045807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc007070 con 0x7f2ab4004800 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 -- 172.21.15.5:0/1794045807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab4004800 msgr2=0x7f2ab4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab4004800 0x7f2ab4004c80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2abc001eb0 tx=0x7f2ab4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 -- 172.21.15.5:0/1794045807 shutdown_connections 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 --2- 172.21.15.5:0/1794045807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ab4004800 0x7f2ab4004c80 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 -- 172.21.15.5:0/1794045807 >> 172.21.15.5:0/1794045807 conn(0x7f2ad008af80 msgr2=0x7f2ad00d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 -- 172.21.15.5:0/1794045807 shutdown_connections 2024-04-25T10:13:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.320+0000 7f2acf200640 1 -- 172.21.15.5:0/1794045807 wait complete. 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.322+0000 7f2acf200640 1 Processor -- start 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.322+0000 7f2acf200640 1 -- start start 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.323+0000 7f2acf200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4004800 0x7f2ad01f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.323+0000 7f2acf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 0x7f2ad0157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.323+0000 7f2acf200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ad00b8df0 0x7f2ad015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.323+0000 7f2acf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b4bb0 con 0x7f2ad0157970 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.323+0000 7f2acf200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b28b0 con 0x7f2ab4004800 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.323+0000 7f2acf200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00b4d30 con 0x7f2ad00b8df0 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 0x7f2ad0157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 0x7f2ad0157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52168/0 (socket says 172.21.15.5:52168) 2024-04-25T10:13:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 -- 172.21.15.5:0/1544523624 learned_addr learned my addr 172.21.15.5:0/1544523624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acde00640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4004800 0x7f2ad01f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2ace800640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ad00b8df0 0x7f2ad015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 -- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ad00b8df0 msgr2=0x7f2ad015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ad00b8df0 0x7f2ad015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 -- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4004800 msgr2=0x7f2ad01f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4004800 0x7f2ad01f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.324+0000 7f2acd400640 1 -- 172.21.15.5:0/1544523624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad015f120 con 0x7f2ad0157970 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.325+0000 7f2acd400640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 0x7f2ad0157040 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2abc02acf0 tx=0x7f2abc001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.325+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc041070 con 0x7f2ad0157970 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.325+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2abc0016b0 con 0x7f2ad0157970 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.325+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2abc037040 con 0x7f2ad0157970 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.325+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad015f400 con 0x7f2ad0157970 2024-04-25T10:13:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.326+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad015f9c0 con 0x7f2ad0157970 2024-04-25T10:13:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.327+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad0068f60 con 0x7f2ad0157970 2024-04-25T10:13:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.328+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2abc044070 con 0x7f2ad0157970 2024-04-25T10:13:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.329+0000 7f2ac6000640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ab006b4d0 0x7f2ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.329+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f2abc0a9770 con 0x7f2ad0157970 2024-04-25T10:13:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.329+0000 7f2acde00640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ab006b4d0 0x7f2ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.330+0000 7f2acde00640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ab006b4d0 0x7f2ab006d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0050700 tx=0x7f2ac0008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.333+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2abc077cd0 con 0x7f2ad0157970 2024-04-25T10:13:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.572+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ad00b16d0 con 0x7f2ad0157970 2024-04-25T10:13:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.573+0000 7f2ac6000640 1 -- 172.21.15.5:0/1544523624 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f2abc07bd70 con 0x7f2ad0157970 2024-04-25T10:13:27.577 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:27.578 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:25.320889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 112 2024-04-25T10:13:27.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ab006b4d0 msgr2=0x7f2ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ab006b4d0 0x7f2ab006d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0050700 tx=0x7f2ac0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:27.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 msgr2=0x7f2ad0157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 0x7f2ad0157040 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2abc02acf0 tx=0x7f2abc001970 comp rx=0 tx=0).stop 2024-04-25T10:13:27.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 shutdown_connections 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ad00b8df0 0x7f2ad015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ad0157970 0x7f2ad0157040 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ab006b4d0 0x7f2ab006d990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 --2- 172.21.15.5:0/1544523624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ab4004800 0x7f2ad01f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 >> 172.21.15.5:0/1544523624 conn(0x7f2ad008af80 msgr2=0x7f2ad006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 shutdown_connections 2024-04-25T10:13:27.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:27.576+0000 7f2acf200640 1 -- 172.21.15.5:0/1544523624 wait complete. 2024-04-25T10:13:27.825 INFO:tasks.ceph.mds.c.smithi005.stderr:2024-04-25T10:13:27.821+0000 7f15a7400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:28.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:47.523031+0000', 'modified': '2024-04-25T10:13:25.320889+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9402': {'gid': 9402, 'name': 'c', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6835/806266896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6833', 'nonce': 806266896}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 806266896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:28.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:28.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:13:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.223+0000 7fde12800640 1 Processor -- start 2024-04-25T10:13:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.223+0000 7fde12800640 1 -- start start 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.224+0000 7fde12800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde0c141da0 0x7fde0c06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.224+0000 7fde12800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fde0c06e640 0x7fde0c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde12800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde12800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c142180 con 0x7fde0c141da0 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde12800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c071d30 con 0x7fde0c06e640 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde12800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c06d620 con 0x7fde0c15d340 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde11400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde0c141da0 0x7fde0c06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde11400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fde0c141da0 0x7fde0c06e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52180/0 (socket says 172.21.15.5:52180) 2024-04-25T10:13:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde11400640 1 -- 172.21.15.5:0/1652566931 learned_addr learned my addr 172.21.15.5:0/1652566931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde10a00640 1 --2- 172.21.15.5:0/1652566931 >> v2:172.21.15.80:3300/0 conn(0x7fde0c06e640 0x7fde0c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde11400640 1 -- 172.21.15.5:0/1652566931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 msgr2=0x7fde0c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.225+0000 7fde11e00640 1 --2- 172.21.15.5:0/1652566931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.226+0000 7fde11400640 1 --2- 172.21.15.5:0/1652566931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.226+0000 7fde11400640 1 -- 172.21.15.5:0/1652566931 >> v2:172.21.15.80:3300/0 conn(0x7fde0c06e640 msgr2=0x7fde0c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.226+0000 7fde11400640 1 --2- 172.21.15.5:0/1652566931 >> v2:172.21.15.80:3300/0 conn(0x7fde0c06e640 0x7fde0c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.226+0000 7fde11400640 1 -- 172.21.15.5:0/1652566931 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c06d900 con 0x7fde0c141da0 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.226+0000 7fde10a00640 1 --2- 172.21.15.5:0/1652566931 >> v2:172.21.15.80:3300/0 conn(0x7fde0c06e640 0x7fde0c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.227+0000 7fde11400640 1 --2- 172.21.15.5:0/1652566931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c141da0 0x7fde0c06e100 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fde0c06c800 tx=0x7fde0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3290a526a3f5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.227+0000 7fde0be00640 1 -- 172.21.15.5:0/1652566931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00037070 con 0x7fde0c141da0 2024-04-25T10:13:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.227+0000 7fde0be00640 1 -- 172.21.15.5:0/1652566931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00032440 con 0x7fde0c141da0 2024-04-25T10:13:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.227+0000 7fde0be00640 1 -- 172.21.15.5:0/1652566931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00032810 con 0x7fde0c141da0 2024-04-25T10:13:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 -- 172.21.15.5:0/1652566931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c141da0 msgr2=0x7fde0c06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 --2- 172.21.15.5:0/1652566931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c141da0 0x7fde0c06e100 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fde0c06c800 tx=0x7fde0002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 -- 172.21.15.5:0/1652566931 shutdown_connections 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 --2- 172.21.15.5:0/1652566931 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c0cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 --2- 172.21.15.5:0/1652566931 >> v2:172.21.15.80:3300/0 conn(0x7fde0c06e640 0x7fde0c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 --2- 172.21.15.5:0/1652566931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c141da0 0x7fde0c06e100 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 -- 172.21.15.5:0/1652566931 >> 172.21.15.5:0/1652566931 conn(0x7fde0c066f20 msgr2=0x7fde0c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 -- 172.21.15.5:0/1652566931 shutdown_connections 2024-04-25T10:13:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.228+0000 7fde12800640 1 -- 172.21.15.5:0/1652566931 wait complete. 2024-04-25T10:13:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.230+0000 7fde12800640 1 Processor -- start 2024-04-25T10:13:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.230+0000 7fde12800640 1 -- start start 2024-04-25T10:13:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde12800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 0x7fde0c13b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde12800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde0c141da0 0x7fde0c13c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde12800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c1f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde12800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0cd9c0 con 0x7fde0c06e640 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde12800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0cd840 con 0x7fde0c141da0 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde12800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0cdb40 con 0x7fde0c15d340 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde11400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 0x7fde0c13b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde11400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 0x7fde0c13b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52184/0 (socket says 172.21.15.5:52184) 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.231+0000 7fde11400640 1 -- 172.21.15.5:0/3846269967 learned_addr learned my addr 172.21.15.5:0/3846269967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde10a00640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde0c141da0 0x7fde0c13c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11e00640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c1f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11400640 1 -- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 msgr2=0x7fde0c1f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11400640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c1f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11400640 1 -- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde0c141da0 msgr2=0x7fde0c13c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11400640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde0c141da0 0x7fde0c13c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11400640 1 -- 172.21.15.5:0/3846269967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c13a630 con 0x7fde0c06e640 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde10a00640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde0c141da0 0x7fde0c13c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde11400640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 0x7fde0c13b6d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fde0002a220 tx=0x7fde000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde09600640 1 -- 172.21.15.5:0/3846269967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00037040 con 0x7fde0c06e640 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde09600640 1 -- 172.21.15.5:0/3846269967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00042070 con 0x7fde0c06e640 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c13a910 con 0x7fde0c06e640 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.232+0000 7fde09600640 1 -- 172.21.15.5:0/3846269967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00030bb0 con 0x7fde0c06e640 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.233+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c13aed0 con 0x7fde0c06e640 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.235+0000 7fde09600640 1 -- 172.21.15.5:0/3846269967 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde00047460 con 0x7fde0c06e640 2024-04-25T10:13:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.236+0000 7fde09600640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.236+0000 7fde09600640 1 -- 172.21.15.5:0/3846269967 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fde000ab0e0 con 0x7fde0c06e640 2024-04-25T10:13:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.236+0000 7fde10a00640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.237+0000 7fde16c6c740 1 Processor -- start 2024-04-25T10:13:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.237+0000 7fde16c6c740 1 -- start start 2024-04-25T10:13:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.237+0000 7fde10a00640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fddec06b4d0 0x7fddec06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fddf8006fd0 tx=0x7fddf8008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6c6b30 0x55969c6ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c6c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 0x55969c6c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde16c6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55969c62a620 con 0x55969c70f7a0 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55969c5d3ca0 con 0x55969c6c6b30 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55969c6266c0 con 0x55969c6eeeb0 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 0x55969c6c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 0x55969c6c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52198/0 (socket says 172.21.15.5:52198) 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.238+0000 7fde11e00640 1 -- 172.21.15.5:0/2995327844 learned_addr learned my addr 172.21.15.5:0/2995327844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11400640 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6c6b30 0x55969c6ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde10a00640 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c6c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11e00640 1 -- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 msgr2=0x55969c6c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11e00640 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c6c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11e00640 1 -- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6c6b30 msgr2=0x55969c6ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11e00640 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6c6b30 0x55969c6ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11e00640 1 -- 172.21.15.5:0/2995327844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55969c5daf40 con 0x55969c70f7a0 2024-04-25T10:13:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fde11e00640 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 0x55969c6c52f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fde04008f00 tx=0x7fde0400d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d467a1d26dddbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fddfdc00640 1 -- 172.21.15.5:0/2995327844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04003da0 con 0x55969c70f7a0 2024-04-25T10:13:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fddfdc00640 1 -- 172.21.15.5:0/2995327844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04004030 con 0x55969c70f7a0 2024-04-25T10:13:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.239+0000 7fddfdc00640 1 -- 172.21.15.5:0/2995327844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04004400 con 0x55969c70f7a0 2024-04-25T10:13:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 -- 172.21.15.5:0/2995327844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 msgr2=0x55969c6c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 0x55969c6c52f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fde04008f00 tx=0x7fde0400d750 comp rx=0 tx=0).stop 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 -- 172.21.15.5:0/2995327844 shutdown_connections 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c70f7a0 0x55969c6c52f0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c6c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2995327844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6c6b30 0x55969c6ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 -- 172.21.15.5:0/2995327844 >> 172.21.15.5:0/2995327844 conn(0x55969c6c3d50 msgr2=0x55969c70c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 -- 172.21.15.5:0/2995327844 shutdown_connections 2024-04-25T10:13:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.240+0000 7fde16c6c740 1 -- 172.21.15.5:0/2995327844 wait complete. 2024-04-25T10:13:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.241+0000 7fde16c6c740 1 -- start start 2024-04-25T10:13:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.241+0000 7fde16c6c740 1 Processor -- start 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.242+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c720110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c720650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 0x55969c6dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde16c6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55969c6ee4e0 con 0x55969c6da710 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55969c5c91f0 con 0x55969c70f7a0 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55969c6eded0 con 0x55969c6eeeb0 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde16c6c740 10 client.0 _ll_register_callbacks cb 0x55969c5e6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 0x55969c6dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 0x55969c6dab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52204/0 (socket says 172.21.15.5:52204) 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 -- 172.21.15.5:0/2960431316 learned_addr learned my addr 172.21.15.5:0/2960431316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde10a00640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c720650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11400640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c720110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 -- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 msgr2=0x55969c720110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c720110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 -- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 msgr2=0x55969c720650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c720650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde11e00640 1 -- 172.21.15.5:0/2960431316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55969c5e8c50 con 0x55969c6da710 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.243+0000 7fde10a00640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c720650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fde11e00640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 0x55969c6dab90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fde04008c40 tx=0x7fde04010070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c394a9669babc43c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fddf2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fddf2000640 1 -- 172.21.15.5:0/2960431316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde040043e0 con 0x55969c6da710 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fddf2000640 1 -- 172.21.15.5:0/2960431316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fde04024ce0 con 0x55969c6da710 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde04004030 con 0x55969c6da710 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fddf2000640 1 -- 172.21.15.5:0/2960431316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde04004580 con 0x55969c6da710 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fde16c6c740 10 client.9498 fetch_fsmap learned FSMap version 112 2024-04-25T10:13:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55969c33f2d0 con 0x55969c6da710 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.244+0000 7fddf2000640 1 -- 172.21.15.5:0/2960431316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04004030 con 0x55969c6da710 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fddf2000640 1 -- 172.21.15.5:0/2960431316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde0402b050 con 0x55969c6da710 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde16c6c740 10 client.9498 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde16c6c740 10 client.9498 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde16c6c740 20 client.9498 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c6db910 0x55969c6efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde16c6c740 4 client.9498 mds_command: new command op to 9402 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 --> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55969c637b60 con 0x55969c6db910 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.245+0000 7fde11400640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c6db910 0x55969c6efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.246+0000 7fde11400640 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c6db910 0x55969c6efff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958a7dc6986753d0 server_cookie=411c4b1c4c9f5a2 in_seq=0 out_seq=0 2024-04-25T10:13:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.246+0000 7fddf2000640 10 client.9498 ms_handle_connect on v2:172.21.15.5:6833/806266896 2024-04-25T10:13:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.246+0000 7fddf2000640 1 -- 172.21.15.5:0/2960431316 <== mds.0 v2:172.21.15.5:6833/806266896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55969c637b60 con 0x55969c6db910 2024-04-25T10:13:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.246+0000 7fddf2000640 10 client.9498 handle_command_reply: tid=0 2024-04-25T10:13:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.246+0000 7fde16c6c740 1 client.9498 shutdown 2024-04-25T10:13:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 msgr2=0x55969c6dab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 0x55969c6dab90 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fde04008c40 tx=0x7fde04010070 comp rx=0 tx=0).stop 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 shutdown_connections 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c6db910 0x55969c6efff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6da710 0x55969c6dab90 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c720650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 --2- 172.21.15.5:0/2960431316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6eeeb0 0x55969c720110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fddf2000640 0 client.9498 ms_handle_reset on v2:172.21.15.5:6833/806266896 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 >> 172.21.15.5:0/2960431316 conn(0x55969c6c6b30 msgr2=0x55969c70c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 shutdown_connections 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 1 -- 172.21.15.5:0/2960431316 wait complete. 2024-04-25T10:13:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.247+0000 7fde16c6c740 20 client.9498 trim_cache size 0 max 16384 2024-04-25T10:13:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.265+0000 7fde16c6c740 1 Processor -- start 2024-04-25T10:13:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.265+0000 7fde16c6c740 1 -- start start 2024-04-25T10:13:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6da710 0x55969c7104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c6c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c6ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde16c6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c061c20 con 0x55969c6eeeb0 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1f7800 con 0x55969c70f7a0 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c061650 con 0x55969c6da710 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c6c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c6c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52218/0 (socket says 172.21.15.5:52218) 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde10a00640 1 -- 172.21.15.5:0/1828014500 learned_addr learned my addr 172.21.15.5:0/1828014500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde11400640 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6da710 0x55969c7104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde10a00640 1 -- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6da710 msgr2=0x55969c7104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde11e00640 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c6ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde10a00640 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6da710 0x55969c7104b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.266+0000 7fde10a00640 1 -- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 msgr2=0x55969c6ec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fde10a00640 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c6ec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fde10a00640 1 -- 172.21.15.5:0/1828014500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55969c4096b0 con 0x55969c6eeeb0 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fde11e00640 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c6ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fde10a00640 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c6c5c00 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fde0c1f7e90 tx=0x7fddf8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ad2ea996f0e57e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fddfd200640 1 -- 172.21.15.5:0/1828014500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8019030 con 0x55969c6eeeb0 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fddfd200640 1 -- 172.21.15.5:0/1828014500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8025070 con 0x55969c6eeeb0 2024-04-25T10:13:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.267+0000 7fddfd200640 1 -- 172.21.15.5:0/1828014500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf8012910 con 0x55969c6eeeb0 2024-04-25T10:13:28.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 -- 172.21.15.5:0/1828014500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 msgr2=0x55969c6c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c6c5c00 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fde0c1f7e90 tx=0x7fddf8013000 comp rx=0 tx=0).stop 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 -- 172.21.15.5:0/1828014500 shutdown_connections 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c70f7a0 0x55969c6ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c6c5c00 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1828014500 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c6da710 0x55969c7104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 -- 172.21.15.5:0/1828014500 >> 172.21.15.5:0/1828014500 conn(0x55969c6c6b30 msgr2=0x55969c70c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 -- 172.21.15.5:0/1828014500 shutdown_connections 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.268+0000 7fde16c6c740 1 -- 172.21.15.5:0/1828014500 wait complete. 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.269+0000 7fde16c6c740 1 -- start start 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.269+0000 7fde16c6c740 1 Processor -- start 2024-04-25T10:13:28.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6da710 0x55969c6f0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c710260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c70f7a0 0x55969c6f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55969c6db790 con 0x55969c6eeeb0 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x55969c6dcbf0 con 0x55969c6da710 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x55969c6dbd90 con 0x55969c70f7a0 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde16c6c740 10 client.0 _ll_register_callbacks cb 0x7fde04008d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c710260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde10a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c710260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52226/0 (socket says 172.21.15.5:52226) 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde10a00640 1 -- 172.21.15.5:0/1489579152 learned_addr learned my addr 172.21.15.5:0/1489579152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.270+0000 7fde11400640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6da710 0x55969c6f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde11e00640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c70f7a0 0x55969c6f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde10a00640 1 -- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c70f7a0 msgr2=0x55969c6f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde10a00640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c70f7a0 0x55969c6f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde10a00640 1 -- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6da710 msgr2=0x55969c6f0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde10a00640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6da710 0x55969c6f0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde10a00640 1 -- 172.21.15.5:0/1489579152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55969c7107a0 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde10a00640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c710260 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0b1360 tx=0x7fddf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb8a856e985f2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fddf2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fddf2000640 1 -- 172.21.15.5:0/1489579152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf802f070 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fddf2000640 1 -- 172.21.15.5:0/1489579152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fddf8020840 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.271+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddf8025070 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fddf2000640 1 -- 172.21.15.5:0/1489579152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddf803a050 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 10 client.9504 fetch_fsmap learned FSMap version 112 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55969c72f5c0 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fddf2000640 1 -- 172.21.15.5:0/1489579152 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf8025070 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fddf2000640 1 -- 172.21.15.5:0/1489579152 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fddf803d020 con 0x55969c6eeeb0 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 10 client.9504 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 10 client.9504 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:28.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 20 client.9504 populate_metadata read hostname 'smithi005' 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c776490 0x55969c776870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 4 client.9504 mds_command: new command op to 9402 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 --> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55969c631eb0 con 0x55969c776490 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.272+0000 7fde11400640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c776490 0x55969c776870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde11400640 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c776490 0x55969c776870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b4ccc80d0d98e5b server_cookie=c1887d474992e287 in_seq=0 out_seq=0 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fddf2000640 10 client.9504 ms_handle_connect on v2:172.21.15.5:6833/806266896 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fddf2000640 1 -- 172.21.15.5:0/1489579152 <== mds.0 v2:172.21.15.5:6833/806266896 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55969c631eb0 con 0x55969c776490 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fddf2000640 10 client.9504 handle_command_reply: tid=0 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 client.9504 shutdown 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 msgr2=0x55969c710260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c710260 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0b1360 tx=0x7fddf8004730 comp rx=0 tx=0).stop 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 shutdown_connections 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:6833/806266896,v1:172.21.15.5:6835/806266896] conn(0x55969c776490 0x55969c776870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55969c70f7a0 0x55969c6f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.273+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55969c6eeeb0 0x55969c710260 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fddf2000640 0 client.9504 ms_handle_reset on v2:172.21.15.5:6833/806266896 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fde16c6c740 1 --2- 172.21.15.5:0/1489579152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55969c6da710 0x55969c6f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 >> 172.21.15.5:0/1489579152 conn(0x55969c6c6b30 msgr2=0x55969c6ec9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 shutdown_connections 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fde16c6c740 1 -- 172.21.15.5:0/1489579152 wait complete. 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fde16c6c740 20 client.9504 trim_cache size 0 max 16384 2024-04-25T10:13:28.282 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8b6dc2a7-3300-467c-ae7a-376f479db0dd", 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 9402, 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 4.972403387, 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 112, 2024-04-25T10:13:28.283 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 122, 2024-04-25T10:13:28.284 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 122, 2024-04-25T10:13:28.284 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 7.4009345709999996 2024-04-25T10:13:28.284 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.274+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fddec06b4d0 msgr2=0x7fddec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fddec06b4d0 0x7fddec06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fddf8006fd0 tx=0x7fddf8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 msgr2=0x7fde0c13b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 0x7fde0c13b6d0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fde0002a220 tx=0x7fde000046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 shutdown_connections 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fde0c15d340 0x7fde0c1f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fddec06b4d0 0x7fddec06d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde0c141da0 0x7fde0c13c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 --2- 172.21.15.5:0/3846269967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde0c06e640 0x7fde0c13b6d0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 >> 172.21.15.5:0/3846269967 conn(0x7fde0c066f20 msgr2=0x7fde0c070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 shutdown_connections 2024-04-25T10:13:28.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.275+0000 7fde12800640 1 -- 172.21.15.5:0/3846269967 wait complete. 2024-04-25T10:13:28.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:28.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.503+0000 7ff3f1c00640 1 Processor -- start 2024-04-25T10:13:28.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.503+0000 7ff3f1c00640 1 -- start start 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.504+0000 7ff3f1c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3ec15da60 0x7ff3ec0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.504+0000 7ff3f1c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff3ec15de40 0x7ff3ec161d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.504+0000 7ff3f1c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec15e8a0 con 0x7ff3ec1622b0 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.504+0000 7ff3f1c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec15eb80 con 0x7ff3ec15da60 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.504+0000 7ff3f1c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec15ee60 con 0x7ff3ec15de40 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eaa00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff3ec15da60 0x7ff3ec0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eb400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff3ec1622b0 0x7ff3ec0ce520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54334/0 (socket says 172.21.15.5:54334) 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eb400640 1 -- 172.21.15.5:0/971392212 learned_addr learned my addr 172.21.15.5:0/971392212 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:28.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3ea000640 1 --2- 172.21.15.5:0/971392212 >> v2:172.21.15.80:3300/0 conn(0x7ff3ec15de40 0x7ff3ec161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eaa00640 1 -- 172.21.15.5:0/971392212 >> v2:172.21.15.80:3300/0 conn(0x7ff3ec15de40 msgr2=0x7ff3ec161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eaa00640 1 --2- 172.21.15.5:0/971392212 >> v2:172.21.15.80:3300/0 conn(0x7ff3ec15de40 0x7ff3ec161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eaa00640 1 -- 172.21.15.5:0/971392212 >> v1:172.21.15.5:6789/0 conn(0x7ff3ec1622b0 legacy=0x7ff3ec0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3e9600640 1 -- 172.21.15.5:0/971392212 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970870827 0 0) 0x7ff3ec15e8a0 con 0x7ff3ec1622b0 2024-04-25T10:13:28.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.505+0000 7ff3eaa00640 1 -- 172.21.15.5:0/971392212 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec161850 con 0x7ff3ec15da60 2024-04-25T10:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.507+0000 7ff3eaa00640 1 --2- 172.21.15.5:0/971392212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15da60 0x7ff3ec0d0aa0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0b3260 tx=0x7ff3dc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce305c35d92dad99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.507+0000 7ff3e9600640 1 -- 172.21.15.5:0/971392212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc037070 con 0x7ff3ec15da60 2024-04-25T10:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.507+0000 7ff3e9600640 1 -- 172.21.15.5:0/971392212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc032440 con 0x7ff3ec15da60 2024-04-25T10:13:28.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.507+0000 7ff3e9600640 1 -- 172.21.15.5:0/971392212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc02ae60 con 0x7ff3ec15da60 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 -- 172.21.15.5:0/971392212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15da60 msgr2=0x7ff3ec0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/971392212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15da60 0x7ff3ec0d0aa0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0b3260 tx=0x7ff3dc02a110 comp rx=0 tx=0).stop 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 -- 172.21.15.5:0/971392212 shutdown_connections 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/971392212 >> v2:172.21.15.80:3300/0 conn(0x7ff3ec15de40 0x7ff3ec161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/971392212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15da60 0x7ff3ec0d0aa0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 -- 172.21.15.5:0/971392212 >> 172.21.15.5:0/971392212 conn(0x7ff3ec06bfe0 msgr2=0x7ff3ec06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 -- 172.21.15.5:0/971392212 shutdown_connections 2024-04-25T10:13:28.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.508+0000 7ff3f1c00640 1 -- 172.21.15.5:0/971392212 wait complete. 2024-04-25T10:13:28.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.510+0000 7ff3f1c00640 1 Processor -- start 2024-04-25T10:13:28.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.510+0000 7ff3f1c00640 1 -- start start 2024-04-25T10:13:28.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3f1c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3ec15da60 0x7ff3ec1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3f1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 0x7ff3ec1f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3f1c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff3ec1622b0 0x7ff3ec1f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3f1c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec15e2d0 con 0x7ff3ec15de40 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3f1c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cee40 con 0x7ff3ec15da60 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3f1c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec15e450 con 0x7ff3ec1622b0 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 0x7ff3ec1f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 0x7ff3ec1f2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52234/0 (socket says 172.21.15.5:52234) 2024-04-25T10:13:28.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 -- 172.21.15.5:0/3849094056 learned_addr learned my addr 172.21.15.5:0/3849094056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 -- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff3ec1622b0 msgr2=0x7ff3ec1f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff3ec1622b0 0x7ff3ec1f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3eaa00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3ec15da60 0x7ff3ec1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 -- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3ec15da60 msgr2=0x7ff3ec1f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.511+0000 7ff3ea000640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3ec15da60 0x7ff3ec1f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3ea000640 1 -- 172.21.15.5:0/3849094056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0cd800 con 0x7ff3ec15de40 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3eaa00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3ec15da60 0x7ff3ec1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3ea000640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 0x7ff3ec1f2bf0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8007ef0 tx=0x7ff3d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8014620 con 0x7ff3ec15de40 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d8019070 con 0x7ff3ec15de40 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8014a00 con 0x7ff3ec15de40 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec0aa2c0 con 0x7ff3ec15de40 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.512+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec0cad00 con 0x7ff3ec15de40 2024-04-25T10:13:28.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.513+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec1f31f0 con 0x7ff3ec15de40 2024-04-25T10:13:28.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.515+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3d8014ba0 con 0x7ff3ec15de40 2024-04-25T10:13:28.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.515+0000 7ff3e2a00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3cc06b4d0 0x7ff3cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:28.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.516+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff3d808ae20 con 0x7ff3ec15de40 2024-04-25T10:13:28.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.516+0000 7ff3eaa00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3cc06b4d0 0x7ff3cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:28.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.517+0000 7ff3eaa00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3cc06b4d0 0x7ff3cc06d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034000 tx=0x7ff3dc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:28.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.520+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3d80599c0 con 0x7ff3ec15de40 2024-04-25T10:13:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.767+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff3ec068fd0 con 0x7ff3ec15de40 2024-04-25T10:13:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.768+0000 7ff3e2a00640 1 -- 172.21.15.5:0/3849094056 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff3d80592c0 con 0x7ff3ec15de40 2024-04-25T10:13:28.772 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3cc06b4d0 msgr2=0x7ff3cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3cc06b4d0 0x7ff3cc06d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc034000 tx=0x7ff3dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 msgr2=0x7ff3ec1f2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 0x7ff3ec1f2bf0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8007ef0 tx=0x7ff3d800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 shutdown_connections 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff3ec1622b0 0x7ff3ec1f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3ec15de40 0x7ff3ec1f2bf0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff3cc06b4d0 0x7ff3cc06d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 --2- 172.21.15.5:0/3849094056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3ec15da60 0x7ff3ec1f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 >> 172.21.15.5:0/3849094056 conn(0x7ff3ec06bfe0 msgr2=0x7ff3ec0d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 shutdown_connections 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:28.771+0000 7ff3f1c00640 1 -- 172.21.15.5:0/3849094056 wait complete. 2024-04-25T10:13:28.795 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:13:28.795 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:28.795 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-04-25T10:13:28.795 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:28.796 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.796 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:13:28.796 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:13:28.796 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:28.796 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:28.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.796 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-04-25T10:13:28.853 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:28.853 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:28.853 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:28.853 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.853 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 41917sec preferred_lft 41917sec 2024-04-25T10:13:28.854 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::2007:6aff:fee8:92a9/64 scope link 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.855 INFO:teuthology.orchestra.run.smithi005.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:28.857 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 8e:de:89:e2:c6:21 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::8cde:89ff:fee2:c621/64 scope link 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi005.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 36:9e:6d:d9:d3:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::349e:6dff:fed9:d314/64 scope link 2024-04-25T10:13:28.858 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.860 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:28.947 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:13:28.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.951 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-04-25T10:13:29.007 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-04-25T10:13:29.052 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:29.052 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T10:13:29.052 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T10:13:29.053 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:29.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.053 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2024-04-25T10:13:29.053 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-04-25T10:13:29.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T10:13:29.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T10:13:29.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T10:13:29.054 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-04-25T10:13:29.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.214 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:13:29.262 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:29.263 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:29.264 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:29.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-04-25T10:13:29.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:29.354 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:29.411 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:29.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:29.411 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:29.467 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:13:29.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:29.607 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:29.607 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:29.607 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:29.631 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:29.631 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:29.631 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:29.631 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:13:29.631 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:29.660 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:13:29.660 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.80:3300,172.21.15.80:3301" 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790" 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.80:6789,172.21.15.80:6790:/ 2024-04-25T10:13:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8b6dc2a7-3300-467c-ae7a-376f479db0dd" will pass to kernel 2024-04-25T10:13:29.664 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:13:29.664 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:13:29.664 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:13:29.664 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:13:29.664 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:13:29.668 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:13:29.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:29.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.753 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:13:29.808 INFO:teuthology.orchestra.run.smithi005.stdout:client6380 2024-04-25T10:13:29.808 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:29.808 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:29.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.858 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./ 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./parent 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/charlie 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/alpha 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/flushed/bravo 2024-04-25T10:13:29.972 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed 2024-04-25T10:13:29.973 INFO:teuthology.orchestra.run.smithi005.stdout:./parent/unflushed/jfile 2024-04-25T10:13:29.974 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:29.974 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:29.974 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:29.974 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:29.974 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:13:29.974 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:30.119 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-04-25T10:13:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.133 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.134 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:30.134 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:30.134 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:30.134 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:13:30.134 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:30.328 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627776 2024-04-25T10:13:30.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.342 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.342 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:30.342 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:30.342 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:30.342 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:13:30.342 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:30.536 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627777 2024-04-25T10:13:30.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.550 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.551 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:30.551 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:30.551 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:30.551 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:13:30.551 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:30.743 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627780 2024-04-25T10:13:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.757 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.757 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:30.757 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:30.757 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:30.758 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:13:30.758 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:30.952 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627778 2024-04-25T10:13:30.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.967 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.967 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:30.967 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:30.967 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:30.967 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:13:30.967 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:31.184 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627779 2024-04-25T10:13:31.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:31.201 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:31.201 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:31.201 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:31.201 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:31.201 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:13:31.201 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:31.396 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627781 2024-04-25T10:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:31.410 DEBUG:teuthology.orchestra.run.smithi005:> stdin-killer -- python3 -c ' 2024-04-25T10:13:31.410 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-04-25T10:13:31.410 DEBUG:teuthology.orchestra.run.smithi005:> import stat 2024-04-25T10:13:31.410 DEBUG:teuthology.orchestra.run.smithi005:> 2024-04-25T10:13:31.410 DEBUG:teuthology.orchestra.run.smithi005:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:13:31.410 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-04-25T10:13:31.592 INFO:teuthology.orchestra.run.smithi005.stdout:1099511627782 2024-04-25T10:13:31.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:31.606 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:31.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:31.765 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:31.872 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:31.873 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:31.873 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:31.933 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:31.933 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:31.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.192+0000 7f0934a00640 1 Processor -- start 2024-04-25T10:13:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.193+0000 7f0934a00640 1 -- start start 2024-04-25T10:13:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.194+0000 7f0934a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09300b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f0934a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09300cf410 con 0x7f09300d07d0 2024-04-25T10:13:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f0934a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09300cf6f0 con 0x7f09300d2380 2024-04-25T10:13:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f0934a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300d27c0 con 0x7f0930161200 2024-04-25T10:13:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f092f200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f09300d07d0 0x7f09300b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54336/0 (socket says 172.21.15.5:54336) 2024-04-25T10:13:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f092f200640 1 -- 172.21.15.5:0/32327427 learned_addr learned my addr 172.21.15.5:0/32327427 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f092e800640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09300b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866472825 0 0) 0x7f09300cf410 con 0x7f09300d07d0 2024-04-25T10:13:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.195+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0914003520 con 0x7f09300d07d0 2024-04-25T10:13:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473394675 0 0) 0x7f0914003520 con 0x7f09300d07d0 2024-04-25T10:13:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 msgr2=0x7f09300b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09300b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 >> v1:172.21.15.80:6789/0 conn(0x7f09300d2380 legacy=0x7f09300d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300d3fd0 con 0x7f09300d07d0 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f0928001670 con 0x7f09300d07d0 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 >> v1:172.21.15.5:6789/0 conn(0x7f09300d07d0 legacy=0x7f09300b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f0914004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0914004d40 0x7f09140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300d27c0 con 0x7f0914003fb0 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091400bb00 con 0x7f0914004d40 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091400bd00 con 0x7f0930161200 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.196+0000 7f092de00640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f0914004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092e800640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092f200640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0914004d40 0x7f09140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092de00640 1 -- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 msgr2=0x7f09140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092de00640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092de00640 1 -- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0914004d40 msgr2=0x7f09140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092de00640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0914004d40 0x7f09140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092de00640 1 -- 172.21.15.5:0/32327427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914003520 con 0x7f0914003fb0 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092e800640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0930161200 0x7f09140039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.197+0000 7f092f200640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0914004d40 0x7f09140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.198+0000 7f092de00640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f0914004800 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f091c001750 tx=0x7f0914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ece036cb3604a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.198+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c001370 con 0x7f0914003fb0 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.198+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c0052d0 con 0x7f0914003fb0 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.198+0000 7f092d400640 1 -- 172.21.15.5:0/32327427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c02ab20 con 0x7f0914003fb0 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f0934a00640 1 -- 172.21.15.5:0/32327427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 msgr2=0x7f0914004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f0934a00640 1 --2- 172.21.15.5:0/32327427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f0914004800 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f091c001750 tx=0x7f0914005830 comp rx=0 tx=0).stop 2024-04-25T10:13:32.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f092f200640 1 -- 172.21.15.5:0/32327427 reap_dead start 2024-04-25T10:13:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f0934a00640 1 -- 172.21.15.5:0/32327427 shutdown_connections 2024-04-25T10:13:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f0934a00640 1 -- 172.21.15.5:0/32327427 >> 172.21.15.5:0/32327427 conn(0x7f093008af80 msgr2=0x7f09300d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f0934a00640 1 -- 172.21.15.5:0/32327427 shutdown_connections 2024-04-25T10:13:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.199+0000 7f0934a00640 1 -- 172.21.15.5:0/32327427 wait complete. 2024-04-25T10:13:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.201+0000 7f0934a00640 1 Processor -- start 2024-04-25T10:13:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.201+0000 7f0934a00640 1 -- start start 2024-04-25T10:13:32.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f0934a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f093015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f0934a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09300ccf20 0x7f09300cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f0934a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f093015ed80 0x7f093015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f0934a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300b4bb0 con 0x7f0914003fb0 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f0934a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09300b28b0 con 0x7f093015ed80 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f0934a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09300b4d30 con 0x7f09300ccf20 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f092e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f093015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f092e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f093015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52244/0 (socket says 172.21.15.5:52244) 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.202+0000 7f092e800640 1 -- 172.21.15.5:0/2300389281 learned_addr learned my addr 172.21.15.5:0/2300389281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092de00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09300ccf20 0x7f09300cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092f200640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f093015ed80 0x7f093015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092e800640 1 -- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09300ccf20 msgr2=0x7f09300cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092e800640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09300ccf20 0x7f09300cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092e800640 1 -- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f093015ed80 msgr2=0x7f093015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092e800640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f093015ed80 0x7f093015f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092e800640 1 -- 172.21.15.5:0/2300389281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093015fb50 con 0x7f0914003fb0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f092e800640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f093015a750 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f092000f350 tx=0x7f092000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.203+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920003b90 con 0x7f0914003fb0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.204+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0920003d30 con 0x7f0914003fb0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.204+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920004220 con 0x7f0914003fb0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.204+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093015fe30 con 0x7f0914003fb0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.204+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09301603f0 con 0x7f0914003fb0 2024-04-25T10:13:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.205+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0930068f60 con 0x7f0914003fb0 2024-04-25T10:13:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.206+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0920005d10 con 0x7f0914003fb0 2024-04-25T10:13:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.207+0000 7f0926a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f091006b500 0x7f091006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.207+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f092008aae0 con 0x7f0914003fb0 2024-04-25T10:13:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.208+0000 7f092de00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f091006b500 0x7f091006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.209+0000 7f092de00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f091006b500 0x7f091006d9c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f091c032000 tx=0x7f091c02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.211+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0920012070 con 0x7f0914003fb0 2024-04-25T10:13:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.413+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09300b1740 con 0x7f0914003fb0 2024-04-25T10:13:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.415+0000 7f0926a00640 1 -- 172.21.15.5:0/2300389281 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17101 (secure 0 0 0) 0x7f0920059710 con 0x7f0914003fb0 2024-04-25T10:13:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:32.419 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":122,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:13:20.803041+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.5:6833/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.80:6837/84886517":"2024-04-26T10:13:06.161908+0000","172.21.15.80:6836/84886517":"2024-04-26T10:13:06.161908+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.417+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f091006b500 msgr2=0x7f091006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.417+0000 7f0934a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f091006b500 0x7f091006d9c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f091c032000 tx=0x7f091c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.417+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 msgr2=0x7f093015a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.417+0000 7f0934a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f093015a750 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f092000f350 tx=0x7f092000f810 comp rx=0 tx=0).stop 2024-04-25T10:13:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 shutdown_connections 2024-04-25T10:13:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f091006b500 0x7f091006d9c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f093015ed80 0x7f093015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09300ccf20 0x7f09300cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 --2- 172.21.15.5:0/2300389281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0914003fb0 0x7f093015a750 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 >> 172.21.15.5:0/2300389281 conn(0x7f093008af80 msgr2=0x7f09300d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 shutdown_connections 2024-04-25T10:13:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.418+0000 7f0934a00640 1 -- 172.21.15.5:0/2300389281 wait complete. 2024-04-25T10:13:32.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:45.398584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:46.541796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:32.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.654+0000 7f9c78a00640 1 Processor -- start 2024-04-25T10:13:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.654+0000 7f9c78a00640 1 -- start start 2024-04-25T10:13:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.655+0000 7f9c78a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9c74144fe0 0x7f9c7406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c78a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c7406d130 0x7f9c74071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c78a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c741453c0 con 0x7f9c74071670 2024-04-25T10:13:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c78a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740a8560 con 0x7f9c7406d130 2024-04-25T10:13:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c78a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7406d6a0 con 0x7f9c74144fe0 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c71400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c7406d130 0x7f9c74071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c71400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9c7406d130 0x7f9c74071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52250/0 (socket says 172.21.15.5:52250) 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c71400640 1 -- 172.21.15.5:0/2130909952 learned_addr learned my addr 172.21.15.5:0/2130909952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.656+0000 7f9c71e00640 1 --2- 172.21.15.5:0/2130909952 >> v2:172.21.15.80:3300/0 conn(0x7f9c74144fe0 0x7f9c7406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.657+0000 7f9c71400640 1 -- 172.21.15.5:0/2130909952 >> v2:172.21.15.80:3300/0 conn(0x7f9c74144fe0 msgr2=0x7f9c7406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.657+0000 7f9c71400640 1 --2- 172.21.15.5:0/2130909952 >> v2:172.21.15.80:3300/0 conn(0x7f9c74144fe0 0x7f9c7406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.657+0000 7f9c71400640 1 -- 172.21.15.5:0/2130909952 >> v1:172.21.15.80:6789/0 conn(0x7f9c74071670 legacy=0x7f9c740cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.657+0000 7f9c71400640 1 -- 172.21.15.5:0/2130909952 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7406d980 con 0x7f9c7406d130 2024-04-25T10:13:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.658+0000 7f9c71400640 1 --2- 172.21.15.5:0/2130909952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c7406d130 0x7f9c74071130 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9c740728b0 tx=0x7f9c6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1085ca5cbd3a26d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.658+0000 7f9c70a00640 1 -- 172.21.15.5:0/2130909952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60037070 con 0x7f9c7406d130 2024-04-25T10:13:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.658+0000 7f9c70a00640 1 -- 172.21.15.5:0/2130909952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c60032440 con 0x7f9c7406d130 2024-04-25T10:13:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.658+0000 7f9c70a00640 1 -- 172.21.15.5:0/2130909952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60032810 con 0x7f9c7406d130 2024-04-25T10:13:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 -- 172.21.15.5:0/2130909952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c7406d130 msgr2=0x7f9c74071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2130909952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c7406d130 0x7f9c74071130 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f9c740728b0 tx=0x7f9c6002a230 comp rx=0 tx=0).stop 2024-04-25T10:13:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 -- 172.21.15.5:0/2130909952 shutdown_connections 2024-04-25T10:13:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2130909952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c7406d130 0x7f9c74071130 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2130909952 >> v2:172.21.15.80:3300/0 conn(0x7f9c74144fe0 0x7f9c7406e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 -- 172.21.15.5:0/2130909952 >> 172.21.15.5:0/2130909952 conn(0x7f9c7406ab60 msgr2=0x7f9c7406af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 -- 172.21.15.5:0/2130909952 shutdown_connections 2024-04-25T10:13:32.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.659+0000 7f9c78a00640 1 -- 172.21.15.5:0/2130909952 wait complete. 2024-04-25T10:13:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.661+0000 7f9c78a00640 1 Processor -- start 2024-04-25T10:13:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.661+0000 7f9c78a00640 1 -- start start 2024-04-25T10:13:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c78a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c7406d130 0x7f9c74130c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c78a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 0x7f9c741f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c78a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c74144fe0 0x7f9c741f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c78a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7415be50 con 0x7f9c74071670 2024-04-25T10:13:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c78a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7406df40 con 0x7f9c7406d130 2024-04-25T10:13:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c78a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7406eaa0 con 0x7f9c74144fe0 2024-04-25T10:13:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.662+0000 7f9c71400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 0x7f9c741f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 0x7f9c741f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52266/0 (socket says 172.21.15.5:52266) 2024-04-25T10:13:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 -- 172.21.15.5:0/2090501768 learned_addr learned my addr 172.21.15.5:0/2090501768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71e00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c7406d130 0x7f9c74130c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c72800640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c74144fe0 0x7f9c741f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 -- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c74144fe0 msgr2=0x7f9c741f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c74144fe0 0x7f9c741f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 -- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c7406d130 msgr2=0x7f9c74130c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c7406d130 0x7f9c74130c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 -- 172.21.15.5:0/2090501768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c741311b0 con 0x7f9c74071670 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71e00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c7406d130 0x7f9c74130c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.663+0000 7f9c71400640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 0x7f9c741f4ec0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.664+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60041020 con 0x7f9c74071670 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.664+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c600306c0 con 0x7f9c74071670 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.664+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60037040 con 0x7f9c74071670 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.664+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c741e5750 con 0x7f9c74071670 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.664+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c741e5d10 con 0x7f9c74071670 2024-04-25T10:13:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.665+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c74068f60 con 0x7f9c74071670 2024-04-25T10:13:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.666+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c60044020 con 0x7f9c74071670 2024-04-25T10:13:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.667+0000 7f9c6a000640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c5406b4d0 0x7f9c5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.667+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f9c600aa9d0 con 0x7f9c74071670 2024-04-25T10:13:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.668+0000 7f9c71e00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c5406b4d0 0x7f9c5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.669+0000 7f9c71e00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c5406b4d0 0x7f9c5406d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006fd0 tx=0x7f9c64008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.671+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c6004b070 con 0x7f9c74071670 2024-04-25T10:13:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.872+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c74073590 con 0x7f9c74071670 2024-04-25T10:13:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.872+0000 7f9c6a000640 1 -- 172.21.15.5:0/2090501768 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c60079570 con 0x7f9c74071670 2024-04-25T10:13:32.877 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:13:32.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c5406b4d0 msgr2=0x7f9c5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c5406b4d0 0x7f9c5406d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006fd0 tx=0x7f9c64008040 comp rx=0 tx=0).stop 2024-04-25T10:13:32.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 msgr2=0x7f9c741f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 0x7f9c741f4ec0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c600046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 shutdown_connections 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9c74144fe0 0x7f9c741f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c74071670 0x7f9c741f4ec0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f9c5406b4d0 0x7f9c5406d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 --2- 172.21.15.5:0/2090501768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9c7406d130 0x7f9c74130c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 >> 172.21.15.5:0/2090501768 conn(0x7f9c7406ab60 msgr2=0x7f9c740b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 shutdown_connections 2024-04-25T10:13:32.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:32.875+0000 7f9c78a00640 1 -- 172.21.15.5:0/2090501768 wait complete. 2024-04-25T10:13:32.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.112+0000 7f5a70c00640 1 Processor -- start 2024-04-25T10:13:33.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.113+0000 7f5a70c00640 1 -- start start 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.114+0000 7f5a70c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a6c1448f0 0x7f5a6c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.114+0000 7f5a70c00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a6c15bc30 0x7f5a6c15f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.114+0000 7f5a70c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c15fe30 0x7f5a6c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.114+0000 7f5a70c00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c15f460 con 0x7f5a6c15bc30 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.114+0000 7f5a70c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c144cd0 con 0x7f5a6c1448f0 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.114+0000 7f5a70c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c15c440 con 0x7f5a6c15fe30 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a6c15bc30 0x7f5a6c15f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5a6c15bc30 0x7f5a6c15f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52278/0 (socket says 172.21.15.5:52278) 2024-04-25T10:13:33.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 -- 172.21.15.5:0/3437593043 learned_addr learned my addr 172.21.15.5:0/3437593043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a6be00640 1 --2- 172.21.15.5:0/3437593043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c15fe30 0x7f5a6c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 -- 172.21.15.5:0/3437593043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c15fe30 msgr2=0x7f5a6c15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a6b400640 1 --2- 172.21.15.5:0/3437593043 >> v2:172.21.15.80:3300/0 conn(0x7f5a6c1448f0 0x7f5a6c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 --2- 172.21.15.5:0/3437593043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c15fe30 0x7f5a6c15ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 -- 172.21.15.5:0/3437593043 >> v2:172.21.15.80:3300/0 conn(0x7f5a6c1448f0 msgr2=0x7f5a6c0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 --2- 172.21.15.5:0/3437593043 >> v2:172.21.15.80:3300/0 conn(0x7f5a6c1448f0 0x7f5a6c0cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a62a00640 1 -- 172.21.15.5:0/3437593043 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c15c720 con 0x7f5a6c15bc30 2024-04-25T10:13:33.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.115+0000 7f5a6b400640 1 --2- 172.21.15.5:0/3437593043 >> v2:172.21.15.80:3300/0 conn(0x7f5a6c1448f0 0x7f5a6c0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.116+0000 7f5a62a00640 1 --2- 172.21.15.5:0/3437593043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c15f8f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c071eb0 tx=0x7f5a5402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1583bd3121e51ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.117+0000 7f5a6aa00640 1 -- 172.21.15.5:0/3437593043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54037070 con 0x7f5a6c15bc30 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.117+0000 7f5a6aa00640 1 -- 172.21.15.5:0/3437593043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54032440 con 0x7f5a6c15bc30 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.117+0000 7f5a6aa00640 1 -- 172.21.15.5:0/3437593043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54032810 con 0x7f5a6c15bc30 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.117+0000 7f5a70c00640 1 -- 172.21.15.5:0/3437593043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 msgr2=0x7f5a6c15f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.117+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3437593043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c15f8f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c071eb0 tx=0x7f5a5402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 -- 172.21.15.5:0/3437593043 shutdown_connections 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3437593043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c15fe30 0x7f5a6c15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3437593043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c15f8f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3437593043 >> v2:172.21.15.80:3300/0 conn(0x7f5a6c1448f0 0x7f5a6c0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 -- 172.21.15.5:0/3437593043 >> 172.21.15.5:0/3437593043 conn(0x7f5a6c066f20 msgr2=0x7f5a6c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 -- 172.21.15.5:0/3437593043 shutdown_connections 2024-04-25T10:13:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.118+0000 7f5a70c00640 1 -- 172.21.15.5:0/3437593043 wait complete. 2024-04-25T10:13:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.120+0000 7f5a70c00640 1 Processor -- start 2024-04-25T10:13:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.120+0000 7f5a70c00640 1 -- start start 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a70c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c1448f0 0x7f5a6c15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a70c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a70c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a6c15fe30 0x7f5a6c0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a70c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c15d360 con 0x7f5a6c15bc30 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a70c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c15c0c0 con 0x7f5a6c15fe30 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a70c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c15d4e0 con 0x7f5a6c1448f0 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c0c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52294/0 (socket says 172.21.15.5:52294) 2024-04-25T10:13:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a62a00640 1 -- 172.21.15.5:0/3263662255 learned_addr learned my addr 172.21.15.5:0/3263662255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a6be00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a6c15fe30 0x7f5a6c0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a6b400640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c1448f0 0x7f5a6c15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.121+0000 7f5a62a00640 1 -- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c1448f0 msgr2=0x7f5a6c15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a62a00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c1448f0 0x7f5a6c15b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a62a00640 1 -- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a6c15fe30 msgr2=0x7f5a6c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a62a00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a6c15fe30 0x7f5a6c0ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a62a00640 1 -- 172.21.15.5:0/3263662255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0cc960 con 0x7f5a6c15bc30 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a6be00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a6c15fe30 0x7f5a6c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a6b400640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c1448f0 0x7f5a6c15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a62a00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c0c9c10 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5a5402a220 tx=0x7f5a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54037040 con 0x7f5a6c15bc30 2024-04-25T10:13:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54042070 con 0x7f5a6c15bc30 2024-04-25T10:13:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c0ccc40 con 0x7f5a6c15bc30 2024-04-25T10:13:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54030bb0 con 0x7f5a6c15bc30 2024-04-25T10:13:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.122+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c0cd200 con 0x7f5a6c15bc30 2024-04-25T10:13:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.124+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c073ce0 con 0x7f5a6c15bc30 2024-04-25T10:13:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.125+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a54047460 con 0x7f5a6c15bc30 2024-04-25T10:13:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.125+0000 7f5a63e00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5a4806b4d0 0x7f5a4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.126+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f5a540aafe0 con 0x7f5a6c15bc30 2024-04-25T10:13:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.126+0000 7f5a6b400640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5a4806b4d0 0x7f5a4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.128+0000 7f5a6b400640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5a4806b4d0 0x7f5a4806d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b840 tx=0x7f5a5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.130+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a6c073ce0 con 0x7f5a6c15bc30 2024-04-25T10:13:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.330+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a6c068f60 con 0x7f5a6c15bc30 2024-04-25T10:13:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.331+0000 7f5a63e00640 1 -- 172.21.15.5:0/3263662255 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a54045360 con 0x7f5a6c15bc30 2024-04-25T10:13:33.335 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5a4806b4d0 msgr2=0x7f5a4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5a4806b4d0 0x7f5a4806d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b840 tx=0x7f5a5c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 msgr2=0x7f5a6c0c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c0c9c10 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5a5402a220 tx=0x7f5a540046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 shutdown_connections 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5a4806b4d0 0x7f5a4806d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a6c15fe30 0x7f5a6c0ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a6c15bc30 0x7f5a6c0c9c10 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 --2- 172.21.15.5:0/3263662255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5a6c1448f0 0x7f5a6c15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 >> 172.21.15.5:0/3263662255 conn(0x7f5a6c066f20 msgr2=0x7f5a6c15cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.333+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 shutdown_connections 2024-04-25T10:13:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.334+0000 7f5a70c00640 1 -- 172.21.15.5:0/3263662255 wait complete. 2024-04-25T10:13:33.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.572+0000 7f45b3e00640 1 Processor -- start 2024-04-25T10:13:33.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.572+0000 7f45b3e00640 1 -- start start 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.573+0000 7f45b3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45b41448f0 0x7f45b40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b4144cd0 con 0x7f45b415bc30 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b415f460 con 0x7f45b41448f0 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b415c440 con 0x7f45b415ce20 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b2a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45b41448f0 0x7f45b40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b3400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b2000640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f45b415bc30 0x7f45b415f8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:41142/0 (socket says 172.21.15.5:41142) 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.574+0000 7f45b2000640 1 -- 172.21.15.5:0/1319490134 learned_addr learned my addr 172.21.15.5:0/1319490134 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.575+0000 7f45b3400640 1 -- 172.21.15.5:0/1319490134 >> v2:172.21.15.80:3300/0 conn(0x7f45b41448f0 msgr2=0x7f45b40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.575+0000 7f45b3400640 1 --2- 172.21.15.5:0/1319490134 >> v2:172.21.15.80:3300/0 conn(0x7f45b41448f0 0x7f45b40cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.575+0000 7f45b3400640 1 -- 172.21.15.5:0/1319490134 >> v1:172.21.15.80:6789/0 conn(0x7f45b415bc30 legacy=0x7f45b415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.575+0000 7f45b3400640 1 -- 172.21.15.5:0/1319490134 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b415c720 con 0x7f45b415ce20 2024-04-25T10:13:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.575+0000 7f45b2a00640 1 --2- 172.21.15.5:0/1319490134 >> v2:172.21.15.80:3300/0 conn(0x7f45b41448f0 0x7f45b40cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.577+0000 7f45b3400640 1 --2- 172.21.15.5:0/1319490134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b415d2b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0097a0 tx=0x7f45ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97166a20250526b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.577+0000 7f45b1600640 1 -- 172.21.15.5:0/1319490134 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac037070 con 0x7f45b415ce20 2024-04-25T10:13:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.577+0000 7f45b1600640 1 -- 172.21.15.5:0/1319490134 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac032440 con 0x7f45b415ce20 2024-04-25T10:13:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.577+0000 7f45b1600640 1 -- 172.21.15.5:0/1319490134 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac032810 con 0x7f45b415ce20 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 -- 172.21.15.5:0/1319490134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 msgr2=0x7f45b415d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 --2- 172.21.15.5:0/1319490134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b415d2b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0097a0 tx=0x7f45ac02a180 comp rx=0 tx=0).stop 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 -- 172.21.15.5:0/1319490134 shutdown_connections 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 --2- 172.21.15.5:0/1319490134 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b415d2b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 --2- 172.21.15.5:0/1319490134 >> v2:172.21.15.80:3300/0 conn(0x7f45b41448f0 0x7f45b40cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 -- 172.21.15.5:0/1319490134 >> 172.21.15.5:0/1319490134 conn(0x7f45b4066f20 msgr2=0x7f45b406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 -- 172.21.15.5:0/1319490134 shutdown_connections 2024-04-25T10:13:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.578+0000 7f45b3e00640 1 -- 172.21.15.5:0/1319490134 wait complete. 2024-04-25T10:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.580+0000 7f45b3e00640 1 Processor -- start 2024-04-25T10:13:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.580+0000 7f45b3e00640 1 -- start start 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45b41448f0 0x7f45b415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 0x7f45b413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b41414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4160000 con 0x7f45b415bc30 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b415c0c0 con 0x7f45b41448f0 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4160180 con 0x7f45b415ce20 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 0x7f45b413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 0x7f45b413fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52300/0 (socket says 172.21.15.5:52300) 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.581+0000 7f45b2000640 1 -- 172.21.15.5:0/120712122 learned_addr learned my addr 172.21.15.5:0/120712122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2a00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45b41448f0 0x7f45b415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b3400640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b41414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2000640 1 -- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 msgr2=0x7f45b41414f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2000640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b41414f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2000640 1 -- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45b41448f0 msgr2=0x7f45b415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2000640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45b41448f0 0x7f45b415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2000640 1 -- 172.21.15.5:0/120712122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b415b570 con 0x7f45b415bc30 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.582+0000 7f45b2000640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 0x7f45b413fc20 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c430 tx=0x7f459c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.583+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c017070 con 0x7f45b415bc30 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.583+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c005ce0 con 0x7f45b415bc30 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.583+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0072f0 con 0x7f45b415bc30 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.583+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b4140450 con 0x7f45b415bc30 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.583+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b4140a10 con 0x7f45b415bc30 2024-04-25T10:13:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.584+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b4068f60 con 0x7f45b415bc30 2024-04-25T10:13:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.585+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f459c01a070 con 0x7f45b415bc30 2024-04-25T10:13:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.586+0000 7f45aaa00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f459006b500 0x7f459006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.586+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f459c0898f0 con 0x7f45b415bc30 2024-04-25T10:13:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.587+0000 7f45b2a00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f459006b500 0x7f459006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.588+0000 7f45b2a00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f459006b500 0x7f459006d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009bd0 tx=0x7f45a400b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.591+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45b415b570 con 0x7f45b415bc30 2024-04-25T10:13:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.791+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45b4140160 con 0x7f45b415bc30 2024-04-25T10:13:33.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.793+0000 7f45aaa00640 1 -- 172.21.15.5:0/120712122 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f459c0584a0 con 0x7f45b415bc30 2024-04-25T10:13:33.797 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.795+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f459006b500 msgr2=0x7f459006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.795+0000 7f45b3e00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f459006b500 0x7f459006d9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009bd0 tx=0x7f45a400b040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.795+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 msgr2=0x7f45b413fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.795+0000 7f45b3e00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 0x7f45b413fc20 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c430 tx=0x7f459c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 shutdown_connections 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45b415ce20 0x7f45b41414f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45b415bc30 0x7f45b413fc20 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f459006b500 0x7f459006d9c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 --2- 172.21.15.5:0/120712122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45b41448f0 0x7f45b415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 >> 172.21.15.5:0/120712122 conn(0x7f45b4066f20 msgr2=0x7f45b40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 shutdown_connections 2024-04-25T10:13:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:33.796+0000 7f45b3e00640 1 -- 172.21.15.5:0/120712122 wait complete. 2024-04-25T10:13:33.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:34.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.014+0000 7f84f1400640 1 Processor -- start 2024-04-25T10:13:34.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.014+0000 7f84f1400640 1 -- start start 2024-04-25T10:13:34.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84f1400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84ec141da0 0x7f84ec06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84f1400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ec071d30 con 0x7f84ec06e640 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84f1400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ec06d620 con 0x7f84ec15d510 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84f1400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ec142180 con 0x7f84ec141da0 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84ebe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84ec141da0 0x7f84ec06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84ebe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84ec141da0 0x7f84ec06e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52310/0 (socket says 172.21.15.5:52310) 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84ebe00640 1 -- 172.21.15.5:0/2099838018 learned_addr learned my addr 172.21.15.5:0/2099838018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.016+0000 7f84e3400640 1 --1- 172.21.15.5:0/2099838018 >> v1:172.21.15.5:6789/0 conn(0x7f84ec06e640 0x7f84ec15ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54350/0 (socket says 172.21.15.5:54350) 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.017+0000 7f84ebe00640 1 -- 172.21.15.5:0/2099838018 >> v1:172.21.15.80:6789/0 conn(0x7f84ec15d510 legacy=0x7f84ec0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.017+0000 7f84ebe00640 1 -- 172.21.15.5:0/2099838018 >> v1:172.21.15.5:6789/0 conn(0x7f84ec06e640 legacy=0x7f84ec15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.017+0000 7f84ebe00640 1 -- 172.21.15.5:0/2099838018 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec06d900 con 0x7f84ec141da0 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.018+0000 7f84ebe00640 1 --2- 172.21.15.5:0/2099838018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec141da0 0x7f84ec06e100 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009870 tx=0x7f84dc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e8beb3dcf5165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.018+0000 7f84eb400640 1 -- 172.21.15.5:0/2099838018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc037070 con 0x7f84ec141da0 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.018+0000 7f84eb400640 1 -- 172.21.15.5:0/2099838018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc032440 con 0x7f84ec141da0 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.018+0000 7f84eb400640 1 -- 172.21.15.5:0/2099838018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc032810 con 0x7f84ec141da0 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 -- 172.21.15.5:0/2099838018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec141da0 msgr2=0x7f84ec06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 --2- 172.21.15.5:0/2099838018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec141da0 0x7f84ec06e100 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009870 tx=0x7f84dc02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 -- 172.21.15.5:0/2099838018 shutdown_connections 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 --2- 172.21.15.5:0/2099838018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec141da0 0x7f84ec06e100 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 -- 172.21.15.5:0/2099838018 >> 172.21.15.5:0/2099838018 conn(0x7f84ec066f20 msgr2=0x7f84ec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 -- 172.21.15.5:0/2099838018 shutdown_connections 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.019+0000 7f84f1400640 1 -- 172.21.15.5:0/2099838018 wait complete. 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.021+0000 7f84f1400640 1 Processor -- start 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.021+0000 7f84f1400640 1 -- start start 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f1400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec06e640 0x7f84ec15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f1400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec141da0 0x7f84ec0a8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 0x7f84ec15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f1400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec0cdb90 con 0x7f84ec1f1540 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f1400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec0cda10 con 0x7f84ec141da0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f1400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec071000 con 0x7f84ec06e640 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 0x7f84ec15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 0x7f84ec15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52326/0 (socket says 172.21.15.5:52326) 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.022+0000 7f84f0a00640 1 -- 172.21.15.5:0/1368074514 learned_addr learned my addr 172.21.15.5:0/1368074514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84ebe00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec06e640 0x7f84ec15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84e3400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec141da0 0x7f84ec0a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f0a00640 1 -- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec06e640 msgr2=0x7f84ec15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f0a00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec06e640 0x7f84ec15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f0a00640 1 -- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec141da0 msgr2=0x7f84ec0a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f0a00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec141da0 0x7f84ec0a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f0a00640 1 -- 172.21.15.5:0/1368074514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec13bc90 con 0x7f84ec1f1540 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84e3400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec141da0 0x7f84ec0a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84ebe00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec06e640 0x7f84ec15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f0a00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 0x7f84ec15df70 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f84e400be50 tx=0x7f84e400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4019070 con 0x7f84ec1f1540 2024-04-25T10:13:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e4014440 con 0x7f84ec1f1540 2024-04-25T10:13:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4014830 con 0x7f84ec1f1540 2024-04-25T10:13:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.023+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ec13bf70 con 0x7f84ec1f1540 2024-04-25T10:13:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.024+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec13c530 con 0x7f84ec1f1540 2024-04-25T10:13:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.025+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ec068f60 con 0x7f84ec1f1540 2024-04-25T10:13:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.026+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84e401c070 con 0x7f84ec1f1540 2024-04-25T10:13:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.027+0000 7f84e8c00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84c806b4d0 0x7f84c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.027+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f84e408a470 con 0x7f84ec1f1540 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.027+0000 7f84ebe00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84c806b4d0 0x7f84c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.028+0000 7f84ebe00640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84c806b4d0 0x7f84c806d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f84dc034000 tx=0x7f84dc02e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.031+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84e40590a0 con 0x7f84ec1f1540 2024-04-25T10:13:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.246+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f84ec0ccae0 con 0x7f84ec1f1540 2024-04-25T10:13:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.247+0000 7f84e8c00640 1 -- 172.21.15.5:0/1368074514 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v122) v1 ==== 80+0+238 (secure 0 0 0) 0x7f84e40589a0 con 0x7f84ec1f1540 2024-04-25T10:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6835/2518129896 2024-04-26T10:13:20.803011+0000 2024-04-25T10:13:34.251 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6833/2518129896 2024-04-26T10:13:20.803011+0000 2024-04-25T10:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6837/84886517 2024-04-26T10:13:06.161908+0000 2024-04-25T10:13:34.252 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.80:6836/84886517 2024-04-26T10:13:06.161908+0000 2024-04-25T10:13:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:listed 4 entries 2024-04-25T10:13:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84c806b4d0 msgr2=0x7f84c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84c806b4d0 0x7f84c806d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f84dc034000 tx=0x7f84dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 msgr2=0x7f84ec15df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 0x7f84ec15df70 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f84e400be50 tx=0x7f84e400bf40 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 shutdown_connections 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec1f1540 0x7f84ec15df70 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f84c806b4d0 0x7f84c806d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec141da0 0x7f84ec0a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 --2- 172.21.15.5:0/1368074514 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec06e640 0x7f84ec15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 >> 172.21.15.5:0/1368074514 conn(0x7f84ec066f20 msgr2=0x7f84ec0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 shutdown_connections 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.250+0000 7f84f1400640 1 -- 172.21.15.5:0/1368074514 wait complete. 2024-04-25T10:13:34.273 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:34.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:34.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:13:34.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.376 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:34.433 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:13:34.433 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:34.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:34.494 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:34.494 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:34.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:34.553 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:13:34.553 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:34.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:34.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.830+0000 7fe834c00640 1 Processor -- start 2024-04-25T10:13:34.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.830+0000 7fe834c00640 1 -- start start 2024-04-25T10:13:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.832+0000 7fe834c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.832+0000 7fe834c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8300cd200 con 0x7fe830070f90 2024-04-25T10:13:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.832+0000 7fe834c00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83006e6f0 con 0x7fe8300cdca0 2024-04-25T10:13:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.832+0000 7fe834c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83006e9d0 con 0x7fe830154a10 2024-04-25T10:13:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.833+0000 7fe82fe00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe830070f90 0x7fe83006cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54362/0 (socket says 172.21.15.5:54362) 2024-04-25T10:13:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.833+0000 7fe82fe00640 1 -- 172.21.15.5:0/3929212670 learned_addr learned my addr 172.21.15.5:0/3929212670 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.833+0000 7fe82f400640 1 --2- 172.21.15.5:0/3929212670 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.833+0000 7fe82ea00640 1 -- 172.21.15.5:0/3929212670 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942721514 0 0) 0x7fe8300cd200 con 0x7fe830070f90 2024-04-25T10:13:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.833+0000 7fe82ea00640 1 -- 172.21.15.5:0/3929212670 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe814003520 con 0x7fe830070f90 2024-04-25T10:13:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.834+0000 7fe82f400640 1 -- 172.21.15.5:0/3929212670 >> v1:172.21.15.80:6789/0 conn(0x7fe8300cdca0 legacy=0x7fe8300b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.834+0000 7fe82f400640 1 -- 172.21.15.5:0/3929212670 >> v1:172.21.15.5:6789/0 conn(0x7fe830070f90 legacy=0x7fe83006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.834+0000 7fe82f400640 1 -- 172.21.15.5:0/3929212670 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83006ecb0 con 0x7fe830154a10 2024-04-25T10:13:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.835+0000 7fe82f400640 1 --2- 172.21.15.5:0/3929212670 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83015e8c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097d0 tx=0x7fe82002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6f596ac6f1facd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.835+0000 7fe82ea00640 1 -- 172.21.15.5:0/3929212670 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820037070 con 0x7fe830154a10 2024-04-25T10:13:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.835+0000 7fe82ea00640 1 -- 172.21.15.5:0/3929212670 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82002ada0 con 0x7fe830154a10 2024-04-25T10:13:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.835+0000 7fe82ea00640 1 -- 172.21.15.5:0/3929212670 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820032660 con 0x7fe830154a10 2024-04-25T10:13:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.836+0000 7fe834c00640 1 -- 172.21.15.5:0/3929212670 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 msgr2=0x7fe83015e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.836+0000 7fe834c00640 1 --2- 172.21.15.5:0/3929212670 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83015e8c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097d0 tx=0x7fe82002a110 comp rx=0 tx=0).stop 2024-04-25T10:13:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.836+0000 7fe834c00640 1 -- 172.21.15.5:0/3929212670 shutdown_connections 2024-04-25T10:13:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.836+0000 7fe834c00640 1 --2- 172.21.15.5:0/3929212670 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83015e8c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.836+0000 7fe834c00640 1 -- 172.21.15.5:0/3929212670 >> 172.21.15.5:0/3929212670 conn(0x7fe830066f20 msgr2=0x7fe83006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.837+0000 7fe834c00640 1 -- 172.21.15.5:0/3929212670 shutdown_connections 2024-04-25T10:13:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.837+0000 7fe834c00640 1 -- 172.21.15.5:0/3929212670 wait complete. 2024-04-25T10:13:34.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.839+0000 7fe834c00640 1 Processor -- start 2024-04-25T10:13:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.839+0000 7fe834c00640 1 -- start start 2024-04-25T10:13:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.840+0000 7fe834c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830070f90 0x7fe83015fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.840+0000 7fe834c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 0x7fe8301fb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.840+0000 7fe834c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.840+0000 7fe834c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83006d700 con 0x7fe8300cdca0 2024-04-25T10:13:34.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.840+0000 7fe834c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300cd780 con 0x7fe830070f90 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.840+0000 7fe834c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83006d880 con 0x7fe830154a10 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 0x7fe8301fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 0x7fe8301fb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52338/0 (socket says 172.21.15.5:52338) 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 -- 172.21.15.5:0/3051344277 learned_addr learned my addr 172.21.15.5:0/3051344277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe82f400640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830070f90 0x7fe83015fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe82fe00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 -- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 msgr2=0x7fe83014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83014fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 -- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830070f90 msgr2=0x7fe83015fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830070f90 0x7fe83015fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.841+0000 7fe827e00640 1 -- 172.21.15.5:0/3051344277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830160220 con 0x7fe8300cdca0 2024-04-25T10:13:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.842+0000 7fe827e00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 0x7fe8301fb230 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c440 tx=0x7fe818001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.842+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180052d0 con 0x7fe8300cdca0 2024-04-25T10:13:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.842+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818005470 con 0x7fe8300cdca0 2024-04-25T10:13:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.842+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818007260 con 0x7fe8300cdca0 2024-04-25T10:13:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.842+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe830155350 con 0x7fe8300cdca0 2024-04-25T10:13:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.842+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe830155b90 con 0x7fe8300cdca0 2024-04-25T10:13:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.843+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe830160450 con 0x7fe8300cdca0 2024-04-25T10:13:34.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.845+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe818007400 con 0x7fe8300cdca0 2024-04-25T10:13:34.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.845+0000 7fe827400640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe80c06b4d0 0x7fe80c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.846+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe818089990 con 0x7fe8300cdca0 2024-04-25T10:13:34.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.846+0000 7fe82f400640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe80c06b4d0 0x7fe80c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.847+0000 7fe82f400640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe80c06b4d0 0x7fe80c06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe820034000 tx=0x7fe82002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:34.850+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe830155350 con 0x7fe8300cdca0 2024-04-25T10:13:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.095+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe83006ae40 con 0x7fe8300cdca0 2024-04-25T10:13:35.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.096+0000 7fe827400640 1 -- 172.21.15.5:0/3051344277 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4937 (secure 0 0 0) 0x7fe8180585c0 con 0x7fe8300cdca0 2024-04-25T10:13:35.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:35.101 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:25.320889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:35.101 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 112 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.099+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe80c06b4d0 msgr2=0x7fe80c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.099+0000 7fe834c00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe80c06b4d0 0x7fe80c06d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe820034000 tx=0x7fe82002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.099+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 msgr2=0x7fe8301fb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.099+0000 7fe834c00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 0x7fe8301fb230 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c440 tx=0x7fe818001e70 comp rx=0 tx=0).stop 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 shutdown_connections 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe830154a10 0x7fe83014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8300cdca0 0x7fe8301fb230 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe80c06b4d0 0x7fe80c06d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 --2- 172.21.15.5:0/3051344277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe830070f90 0x7fe83015fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 >> 172.21.15.5:0/3051344277 conn(0x7fe830066f20 msgr2=0x7fe83015dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 shutdown_connections 2024-04-25T10:13:35.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.100+0000 7fe834c00640 1 -- 172.21.15.5:0/3051344277 wait complete. 2024-04-25T10:13:35.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.321+0000 7f15fac00640 1 Processor -- start 2024-04-25T10:13:35.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.321+0000 7f15fac00640 1 -- start start 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.322+0000 7f15fac00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f15f40b2f30 0x7f15f40d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.322+0000 7f15fac00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f415f410 con 0x7f15f415d8a0 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15fac00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f415f6b0 con 0x7f15f41614e0 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15fac00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f415ffe0 con 0x7f15f40b2f30 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15f9800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f15f415d8a0 0x7f15f40d3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:54376/0 (socket says 172.21.15.5:54376) 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15f9800640 1 -- 172.21.15.5:0/2845282470 learned_addr learned my addr 172.21.15.5:0/2845282470 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:35.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15f8e00640 1 --2- 172.21.15.5:0/2845282470 >> v2:172.21.15.5:3300/0 conn(0x7f15f40b2f30 0x7f15f40d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15f8e00640 1 -- 172.21.15.5:0/2845282470 >> v1:172.21.15.80:6789/0 conn(0x7f15f41614e0 legacy=0x7f15f40b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15f8e00640 1 -- 172.21.15.5:0/2845282470 >> v1:172.21.15.5:6789/0 conn(0x7f15f415d8a0 legacy=0x7f15f40d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.323+0000 7f15f8e00640 1 -- 172.21.15.5:0/2845282470 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f415f990 con 0x7f15f40b2f30 2024-04-25T10:13:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.324+0000 7f15f8e00640 1 --2- 172.21.15.5:0/2845282470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f40b2f30 0x7f15f40d2570 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f15e40097a0 tx=0x7f15e402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa028de3935c40d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15f3e00640 1 -- 172.21.15.5:0/2845282470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4037070 con 0x7f15f40b2f30 2024-04-25T10:13:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15f3e00640 1 -- 172.21.15.5:0/2845282470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e402adc0 con 0x7f15f40b2f30 2024-04-25T10:13:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15f3e00640 1 -- 172.21.15.5:0/2845282470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4032660 con 0x7f15f40b2f30 2024-04-25T10:13:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15fac00640 1 -- 172.21.15.5:0/2845282470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f40b2f30 msgr2=0x7f15f40d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15fac00640 1 --2- 172.21.15.5:0/2845282470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f40b2f30 0x7f15f40d2570 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f15e40097a0 tx=0x7f15e402a0b0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15fac00640 1 -- 172.21.15.5:0/2845282470 shutdown_connections 2024-04-25T10:13:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15fac00640 1 --2- 172.21.15.5:0/2845282470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f40b2f30 0x7f15f40d2570 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15fac00640 1 -- 172.21.15.5:0/2845282470 >> 172.21.15.5:0/2845282470 conn(0x7f15f408af80 msgr2=0x7f15f40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.325+0000 7f15fac00640 1 -- 172.21.15.5:0/2845282470 shutdown_connections 2024-04-25T10:13:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.326+0000 7f15fac00640 1 -- 172.21.15.5:0/2845282470 wait complete. 2024-04-25T10:13:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.327+0000 7f15fac00640 1 Processor -- start 2024-04-25T10:13:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.327+0000 7f15fac00640 1 -- start start 2024-04-25T10:13:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fac00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15f40b2f30 0x7f15f415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fac00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15f415d8a0 0x7f15f4152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 0x7f15f415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f415ee10 con 0x7f15f41614e0 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fac00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f415ec90 con 0x7f15f415d8a0 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fac00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4161970 con 0x7f15f40b2f30 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.328+0000 7f15fa200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 0x7f15f415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 0x7f15f415b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52350/0 (socket says 172.21.15.5:52350) 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 -- 172.21.15.5:0/313855026 learned_addr learned my addr 172.21.15.5:0/313855026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15f8e00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15f415d8a0 0x7f15f4152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15f9800640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15f40b2f30 0x7f15f415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 -- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15f40b2f30 msgr2=0x7f15f415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15f40b2f30 0x7f15f415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 -- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15f415d8a0 msgr2=0x7f15f4152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15f415d8a0 0x7f15f4152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 -- 172.21.15.5:0/313855026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f415bc90 con 0x7f15f41614e0 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15f9800640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15f40b2f30 0x7f15f415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15fa200640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 0x7f15f415b3b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b990 tx=0x7f15ec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.329+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec019070 con 0x7f15f41614e0 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.330+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec00cba0 con 0x7f15f41614e0 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.330+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec014440 con 0x7f15f41614e0 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.330+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f4159f00 con 0x7f15f41614e0 2024-04-25T10:13:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.330+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f415a4c0 con 0x7f15f41614e0 2024-04-25T10:13:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.331+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f40c6f00 con 0x7f15f41614e0 2024-04-25T10:13:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.332+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15ec01c070 con 0x7f15f41614e0 2024-04-25T10:13:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.334+0000 7f15f1600640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f15d806b4d0 0x7f15d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.334+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f15ec08a240 con 0x7f15f41614e0 2024-04-25T10:13:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.335+0000 7f15f9800640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f15d806b4d0 0x7f15d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.336+0000 7f15f9800640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f15d806b4d0 0x7f15d806d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f15e8001000 tx=0x7f15e8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.338+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15ec058de0 con 0x7f15f41614e0 2024-04-25T10:13:35.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.539+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15f4068f60 con 0x7f15f41614e0 2024-04-25T10:13:35.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.541+0000 7f15f1600640 1 -- 172.21.15.5:0/313855026 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17101 (secure 0 0 0) 0x7f15ec0586e0 con 0x7f15f41614e0 2024-04-25T10:13:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:35.547 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":122,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:13:20.803041+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.5:6833/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.80:6837/84886517":"2024-04-26T10:13:06.161908+0000","172.21.15.80:6836/84886517":"2024-04-26T10:13:06.161908+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f15d806b4d0 msgr2=0x7f15d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f15d806b4d0 0x7f15d806d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f15e8001000 tx=0x7f15e8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 msgr2=0x7f15f415b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 0x7f15f415b3b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00b990 tx=0x7f15ec00be50 comp rx=0 tx=0).stop 2024-04-25T10:13:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 shutdown_connections 2024-04-25T10:13:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15f41614e0 0x7f15f415b3b0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f15d806b4d0 0x7f15d806d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15f415d8a0 0x7f15f4152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 --2- 172.21.15.5:0/313855026 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f15f40b2f30 0x7f15f415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.544+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 >> 172.21.15.5:0/313855026 conn(0x7f15f408af80 msgr2=0x7f15f40b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.545+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 shutdown_connections 2024-04-25T10:13:35.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.545+0000 7f15fac00640 1 -- 172.21.15.5:0/313855026 wait complete. 2024-04-25T10:13:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:46.680467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:45.398584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:46.541796+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:35.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:35.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.775+0000 7fe71fe00640 1 Processor -- start 2024-04-25T10:13:35.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.775+0000 7fe71fe00640 1 -- start start 2024-04-25T10:13:35.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe72015bc30 0x7fe72015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe72015fe30 0x7fe72015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72015f460 con 0x7fe72015bc30 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71fe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe720144cd0 con 0x7fe72015fe30 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015c440 con 0x7fe7201448f0 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71e000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe72015bc30 0x7fe72015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71ea00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.777+0000 7fe71f400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe72015fe30 0x7fe72015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71e000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe72015bc30 0x7fe72015f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52362/0 (socket says 172.21.15.5:52362) 2024-04-25T10:13:35.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71e000640 1 -- 172.21.15.5:0/255110828 learned_addr learned my addr 172.21.15.5:0/255110828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71f400640 1 -- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 msgr2=0x7fe7200cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71f400640 1 --2- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71f400640 1 -- 172.21.15.5:0/255110828 >> v2:172.21.15.5:3300/0 conn(0x7fe72015bc30 msgr2=0x7fe72015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71f400640 1 --2- 172.21.15.5:0/255110828 >> v2:172.21.15.5:3300/0 conn(0x7fe72015bc30 0x7fe72015f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71f400640 1 -- 172.21.15.5:0/255110828 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72015c720 con 0x7fe72015fe30 2024-04-25T10:13:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.778+0000 7fe71ea00640 1 --2- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.780+0000 7fe71f400640 1 --2- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe72015ce20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe7200b24b0 tx=0x7fe71802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3ee049b0f2c0fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.780+0000 7fe71d600640 1 -- 172.21.15.5:0/255110828 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718031600 con 0x7fe72015fe30 2024-04-25T10:13:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71d600640 1 -- 172.21.15.5:0/255110828 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7180317a0 con 0x7fe72015fe30 2024-04-25T10:13:35.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71d600640 1 -- 172.21.15.5:0/255110828 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718031b70 con 0x7fe72015fe30 2024-04-25T10:13:35.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 -- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 msgr2=0x7fe72015ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 --2- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe72015ce20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe7200b24b0 tx=0x7fe71802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 -- 172.21.15.5:0/255110828 shutdown_connections 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 --2- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe72015ce20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 --2- 172.21.15.5:0/255110828 >> v2:172.21.15.5:3300/0 conn(0x7fe72015bc30 0x7fe72015f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 --2- 172.21.15.5:0/255110828 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 -- 172.21.15.5:0/255110828 >> 172.21.15.5:0/255110828 conn(0x7fe720066f20 msgr2=0x7fe72006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.781+0000 7fe71fe00640 1 -- 172.21.15.5:0/255110828 shutdown_connections 2024-04-25T10:13:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.782+0000 7fe71fe00640 1 -- 172.21.15.5:0/255110828 wait complete. 2024-04-25T10:13:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.784+0000 7fe71fe00640 1 Processor -- start 2024-04-25T10:13:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.784+0000 7fe71fe00640 1 -- start start 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 0x7fe7200b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe7200afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015c090 con 0x7fe72015bc30 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015d360 con 0x7fe72015fe30 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200b4500 con 0x7fe7201448f0 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 0x7fe7200b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 0x7fe7200b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52364/0 (socket says 172.21.15.5:52364) 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.785+0000 7fe71e000640 1 -- 172.21.15.5:0/1831266195 learned_addr learned my addr 172.21.15.5:0/1831266195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71e000640 1 -- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 msgr2=0x7fe7200b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71ea00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71f400640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe7200afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71e000640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200b3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71e000640 1 -- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 msgr2=0x7fe7200afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71e000640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe7200afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71e000640 1 -- 172.21.15.5:0/1831266195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200aee80 con 0x7fe72015bc30 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.786+0000 7fe71e000640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 0x7fe7200b3cb0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009fd0 tx=0x7fe70c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.787+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c017070 con 0x7fe72015bc30 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.787+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c00c940 con 0x7fe72015bc30 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.787+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00cd10 con 0x7fe72015bc30 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.787+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7200af160 con 0x7fe72015bc30 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.787+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7200af720 con 0x7fe72015bc30 2024-04-25T10:13:35.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.788+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe720068f60 con 0x7fe72015bc30 2024-04-25T10:13:35.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.789+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe70c01a070 con 0x7fe72015bc30 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.790+0000 7fe716a00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe70006b4d0 0x7fe70006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.790+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe70c08a7f0 con 0x7fe72015bc30 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.791+0000 7fe71ea00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe70006b4d0 0x7fe70006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.792+0000 7fe71ea00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe70006b4d0 0x7fe70006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe710009560 tx=0x7fe710008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.794+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe70c059390 con 0x7fe72015bc30 2024-04-25T10:13:35.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.983+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe720072180 con 0x7fe72015bc30 2024-04-25T10:13:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.984+0000 7fe716a00640 1 -- 172.21.15.5:0/1831266195 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe70c058c90 con 0x7fe72015bc30 2024-04-25T10:13:35.988 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-04-25T10:13:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe70006b4d0 msgr2=0x7fe70006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe70006b4d0 0x7fe70006d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe710009560 tx=0x7fe710008040 comp rx=0 tx=0).stop 2024-04-25T10:13:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 msgr2=0x7fe7200b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 0x7fe7200b3cb0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009fd0 tx=0x7fe70c00ed40 comp rx=0 tx=0).stop 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 shutdown_connections 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe70006b4d0 0x7fe70006d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe72015fe30 0x7fe7200afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe72015bc30 0x7fe7200b3cb0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 --2- 172.21.15.5:0/1831266195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7201448f0 0x7fe7200b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 >> 172.21.15.5:0/1831266195 conn(0x7fe720066f20 msgr2=0x7fe7200b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 shutdown_connections 2024-04-25T10:13:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:35.987+0000 7fe71fe00640 1 -- 172.21.15.5:0/1831266195 wait complete. 2024-04-25T10:13:36.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.210+0000 7fecfee00640 1 Processor -- start 2024-04-25T10:13:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.210+0000 7fecfee00640 1 -- start start 2024-04-25T10:13:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.211+0000 7fecfee00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fecf8144fe0 0x7fecf806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.211+0000 7fecfee00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecf806d130 0x7fecf8071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.211+0000 7fecfee00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf81453c0 con 0x7fecf8071670 2024-04-25T10:13:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.211+0000 7fecfee00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf80a8560 con 0x7fecf806d130 2024-04-25T10:13:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.211+0000 7fecfee00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf806d6a0 con 0x7fecf8144fe0 2024-04-25T10:13:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.212+0000 7fecf7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecf806d130 0x7fecf8071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.212+0000 7fecf7400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fecf806d130 0x7fecf8071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52380/0 (socket says 172.21.15.5:52380) 2024-04-25T10:13:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.212+0000 7fecf7400640 1 -- 172.21.15.5:0/4060996217 learned_addr learned my addr 172.21.15.5:0/4060996217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:36.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.212+0000 7fecf7e00640 1 --2- 172.21.15.5:0/4060996217 >> v2:172.21.15.80:3300/0 conn(0x7fecf8144fe0 0x7fecf806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.213+0000 7fecf7400640 1 -- 172.21.15.5:0/4060996217 >> v2:172.21.15.80:3300/0 conn(0x7fecf8144fe0 msgr2=0x7fecf806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.213+0000 7fecf7400640 1 --2- 172.21.15.5:0/4060996217 >> v2:172.21.15.80:3300/0 conn(0x7fecf8144fe0 0x7fecf806e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.213+0000 7fecf7400640 1 -- 172.21.15.5:0/4060996217 >> v1:172.21.15.5:6789/0 conn(0x7fecf8071670 legacy=0x7fecf80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.213+0000 7fecf7400640 1 -- 172.21.15.5:0/4060996217 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf806d980 con 0x7fecf806d130 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.214+0000 7fecf7400640 1 --2- 172.21.15.5:0/4060996217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf8071130 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fecf80728b0 tx=0x7fece002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd69730db672bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.214+0000 7fecf6a00640 1 -- 172.21.15.5:0/4060996217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0037070 con 0x7fecf806d130 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.214+0000 7fecf6a00640 1 -- 172.21.15.5:0/4060996217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece0032440 con 0x7fecf806d130 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.214+0000 7fecf6a00640 1 -- 172.21.15.5:0/4060996217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0032810 con 0x7fecf806d130 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 -- 172.21.15.5:0/4060996217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 msgr2=0x7fecf8071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 --2- 172.21.15.5:0/4060996217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf8071130 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fecf80728b0 tx=0x7fece002a270 comp rx=0 tx=0).stop 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 -- 172.21.15.5:0/4060996217 shutdown_connections 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 --2- 172.21.15.5:0/4060996217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf8071130 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 --2- 172.21.15.5:0/4060996217 >> v2:172.21.15.80:3300/0 conn(0x7fecf8144fe0 0x7fecf806e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 -- 172.21.15.5:0/4060996217 >> 172.21.15.5:0/4060996217 conn(0x7fecf806ab60 msgr2=0x7fecf806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 -- 172.21.15.5:0/4060996217 shutdown_connections 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.215+0000 7fecfee00640 1 -- 172.21.15.5:0/4060996217 wait complete. 2024-04-25T10:13:36.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.217+0000 7fecfee00640 1 Processor -- start 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.217+0000 7fecfee00640 1 -- start start 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.218+0000 7fecfee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf813afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.218+0000 7fecfee00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf8071670 0x7fecf815c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.218+0000 7fecfee00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf8144fe0 0x7fecf80b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.218+0000 7fecfee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf815be50 con 0x7fecf806d130 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.218+0000 7fecfee00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf806df40 con 0x7fecf8071670 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.218+0000 7fecfee00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf806eaa0 con 0x7fecf8144fe0 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf813afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf813afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52396/0 (socket says 172.21.15.5:52396) 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 -- 172.21.15.5:0/1529609881 learned_addr learned my addr 172.21.15.5:0/1529609881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7400640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf8071670 0x7fecf815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 -- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf8144fe0 msgr2=0x7fecf80b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf8144fe0 0x7fecf80b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 -- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf8071670 msgr2=0x7fecf815c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf8071670 0x7fecf815c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 -- 172.21.15.5:0/1529609881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf815d290 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.219+0000 7fecf7e00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf813afd0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fece800efc0 tx=0x7fece800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.220+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8018020 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.220+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece8014480 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.220+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8014850 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.220+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf815d570 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.220+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf815db30 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.221+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf8068f60 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.222+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fece8005d10 con 0x7fecf806d130 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.223+0000 7fecefe00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd406b4d0 0x7fecd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.223+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fece808afd0 con 0x7fecf806d130 2024-04-25T10:13:36.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.223+0000 7fecf7400640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd406b4d0 0x7fecd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.225+0000 7fecf7400640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd406b4d0 0x7fecd406d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fece0034000 tx=0x7fece002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.227+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fece8022070 con 0x7fecf806d130 2024-04-25T10:13:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.424+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecf815df50 con 0x7fecf806d130 2024-04-25T10:13:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.425+0000 7fecefe00640 1 -- 172.21.15.5:0/1529609881 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7fece8010040 con 0x7fecf806d130 2024-04-25T10:13:36.430 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-04-25T10:13:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd406b4d0 msgr2=0x7fecd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd406b4d0 0x7fecd406d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fece0034000 tx=0x7fece002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 msgr2=0x7fecf813afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf813afd0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fece800efc0 tx=0x7fece800c490 comp rx=0 tx=0).stop 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 shutdown_connections 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fecf8144fe0 0x7fecf80b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fecd406b4d0 0x7fecd406d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecf8071670 0x7fecf815c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 --2- 172.21.15.5:0/1529609881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf806d130 0x7fecf813afd0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.427+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 >> 172.21.15.5:0/1529609881 conn(0x7fecf806ab60 msgr2=0x7fecf80af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.428+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 shutdown_connections 2024-04-25T10:13:36.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.428+0000 7fecfee00640 1 -- 172.21.15.5:0/1529609881 wait complete. 2024-04-25T10:13:36.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.668+0000 7f12e9800640 1 Processor -- start 2024-04-25T10:13:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.668+0000 7f12e9800640 1 -- start start 2024-04-25T10:13:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161750 0x7f12e40d22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e9800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12e415e970 0x7f12e4160600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e9800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e415ee00 con 0x7f12e4161b30 2024-04-25T10:13:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e9800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e40d2810 con 0x7f12e415e970 2024-04-25T10:13:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e9800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40d4290 con 0x7f12e4161750 2024-04-25T10:13:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e8e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12e415e970 0x7f12e4160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e8e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12e415e970 0x7f12e4160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52400/0 (socket says 172.21.15.5:52400) 2024-04-25T10:13:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.670+0000 7f12e8e00640 1 -- 172.21.15.5:0/818948048 learned_addr learned my addr 172.21.15.5:0/818948048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.671+0000 7f12e3e00640 1 --2- 172.21.15.5:0/818948048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161750 0x7f12e40d22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.671+0000 7f12e8e00640 1 -- 172.21.15.5:0/818948048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161750 msgr2=0x7f12e40d22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.671+0000 7f12e8e00640 1 --2- 172.21.15.5:0/818948048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161750 0x7f12e40d22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.671+0000 7f12e8e00640 1 -- 172.21.15.5:0/818948048 >> v1:172.21.15.80:6789/0 conn(0x7f12e4161b30 legacy=0x7f12e415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.671+0000 7f12e8e00640 1 -- 172.21.15.5:0/818948048 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e415f410 con 0x7f12e415e970 2024-04-25T10:13:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.672+0000 7f12e8e00640 1 --2- 172.21.15.5:0/818948048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e4160600 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009e50 tx=0x7f12dc02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6466beb8194cf86b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.672+0000 7f12e3400640 1 -- 172.21.15.5:0/818948048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc036070 con 0x7f12e415e970 2024-04-25T10:13:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.672+0000 7f12e3400640 1 -- 172.21.15.5:0/818948048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc031600 con 0x7f12e415e970 2024-04-25T10:13:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.672+0000 7f12e3400640 1 -- 172.21.15.5:0/818948048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0319d0 con 0x7f12e415e970 2024-04-25T10:13:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 -- 172.21.15.5:0/818948048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 msgr2=0x7f12e4160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 --2- 172.21.15.5:0/818948048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e4160600 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009e50 tx=0x7f12dc02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 -- 172.21.15.5:0/818948048 shutdown_connections 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 --2- 172.21.15.5:0/818948048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e4160600 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 --2- 172.21.15.5:0/818948048 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161750 0x7f12e40d22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 -- 172.21.15.5:0/818948048 >> 172.21.15.5:0/818948048 conn(0x7f12e408af80 msgr2=0x7f12e40d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 -- 172.21.15.5:0/818948048 shutdown_connections 2024-04-25T10:13:36.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.673+0000 7f12e9800640 1 -- 172.21.15.5:0/818948048 wait complete. 2024-04-25T10:13:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.675+0000 7f12e9800640 1 Processor -- start 2024-04-25T10:13:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.675+0000 7f12e9800640 1 -- start start 2024-04-25T10:13:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.676+0000 7f12e9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e40cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.676+0000 7f12e9800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4161750 0x7f12e41f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.676+0000 7f12e9800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161b30 0x7f12e40b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.676+0000 7f12e9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40b26e0 con 0x7f12e415e970 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.676+0000 7f12e9800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e415f9d0 con 0x7f12e4161750 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.676+0000 7f12e9800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e40d3f20 con 0x7f12e4161b30 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e40cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e40cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52406/0 (socket says 172.21.15.5:52406) 2024-04-25T10:13:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 -- 172.21.15.5:0/1963719643 learned_addr learned my addr 172.21.15.5:0/1963719643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 -- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161b30 msgr2=0x7f12e40b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161b30 0x7f12e40b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 -- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4161750 msgr2=0x7f12e41f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4161750 0x7f12e41f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 -- 172.21.15.5:0/1963719643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e40cfb20 con 0x7f12e415e970 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.677+0000 7f12e3e00640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e40cf3e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f12d400d5f0 tx=0x7f12d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.678+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d40125f0 con 0x7f12e415e970 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.678+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4019070 con 0x7f12e415e970 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.678+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d40129d0 con 0x7f12e415e970 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.678+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e40cfe00 con 0x7f12e415e970 2024-04-25T10:13:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.678+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e40cb3e0 con 0x7f12e415e970 2024-04-25T10:13:36.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.679+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4068f60 con 0x7f12e415e970 2024-04-25T10:13:36.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.680+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12d4012b70 con 0x7f12e415e970 2024-04-25T10:13:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.682+0000 7f12e0c00640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12c006b500 0x7f12c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.682+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f12d408a8b0 con 0x7f12e415e970 2024-04-25T10:13:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.682+0000 7f12dbe00640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12c006b500 0x7f12c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.684+0000 7f12dbe00640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12c006b500 0x7f12c006d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f12e40caff0 tx=0x7f12cc007a00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.686+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12d4059450 con 0x7f12e415e970 2024-04-25T10:13:36.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.891+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12e408d730 con 0x7f12e415e970 2024-04-25T10:13:36.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.892+0000 7f12e0c00640 1 -- 172.21.15.5:0/1963719643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12d4058d50 con 0x7f12e415e970 2024-04-25T10:13:36.897 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12c006b500 msgr2=0x7f12c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12c006b500 0x7f12c006d9c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f12e40caff0 tx=0x7f12cc007a00 comp rx=0 tx=0).stop 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 msgr2=0x7f12e40cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e40cf3e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f12d400d5f0 tx=0x7f12d400dab0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 shutdown_connections 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f12e4161b30 0x7f12e40b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12c006b500 0x7f12c006d9c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12e4161750 0x7f12e41f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 --2- 172.21.15.5:0/1963719643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e415e970 0x7f12e40cf3e0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.895+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 >> 172.21.15.5:0/1963719643 conn(0x7f12e408af80 msgr2=0x7f12e415ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.896+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 shutdown_connections 2024-04-25T10:13:36.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:36.896+0000 7f12e9800640 1 -- 172.21.15.5:0/1963719643 wait complete. 2024-04-25T10:13:36.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.116+0000 7f209fe00640 1 Processor -- start 2024-04-25T10:13:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.117+0000 7f209fe00640 1 -- start start 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209fe00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f209806b270 0x7f20980b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209fe00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f20980af760 0x7f20980afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209fe00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209815ee90 con 0x7f20980d00e0 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209fe00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20980b0110 con 0x7f209806b270 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209fe00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209815df50 con 0x7f20980af760 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209ca00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f209806b270 0x7f20980b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209d400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f20980d00e0 0x7f209806ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39600/0 (socket says 172.21.15.5:39600) 2024-04-25T10:13:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.119+0000 7f209d400640 1 -- 172.21.15.5:0/3533781033 learned_addr learned my addr 172.21.15.5:0/3533781033 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.120+0000 7f209de00640 1 --2- 172.21.15.5:0/3533781033 >> v2:172.21.15.80:3300/0 conn(0x7f20980af760 0x7f20980afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.120+0000 7f209ca00640 1 -- 172.21.15.5:0/3533781033 >> v2:172.21.15.80:3300/0 conn(0x7f20980af760 msgr2=0x7f20980afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.120+0000 7f209ca00640 1 --2- 172.21.15.5:0/3533781033 >> v2:172.21.15.80:3300/0 conn(0x7f20980af760 0x7f20980afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.120+0000 7f209ca00640 1 -- 172.21.15.5:0/3533781033 >> v1:172.21.15.5:6789/0 conn(0x7f20980d00e0 legacy=0x7f209806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.120+0000 7f209ca00640 1 -- 172.21.15.5:0/3533781033 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209815e210 con 0x7f209806b270 2024-04-25T10:13:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.120+0000 7f2097e00640 1 -- 172.21.15.5:0/3533781033 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868355229 0 0) 0x7f209815ee90 con 0x7f20980d00e0 2024-04-25T10:13:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.121+0000 7f209ca00640 1 --2- 172.21.15.5:0/3533781033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b3690 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f20980ce0b0 tx=0x7f208402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b34db52e98b2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.121+0000 7f2097e00640 1 -- 172.21.15.5:0/3533781033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084037070 con 0x7f209806b270 2024-04-25T10:13:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.121+0000 7f2097e00640 1 -- 172.21.15.5:0/3533781033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2084032440 con 0x7f209806b270 2024-04-25T10:13:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.121+0000 7f2097e00640 1 -- 172.21.15.5:0/3533781033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208402ae60 con 0x7f209806b270 2024-04-25T10:13:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 -- 172.21.15.5:0/3533781033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 msgr2=0x7f20980b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 --2- 172.21.15.5:0/3533781033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b3690 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f20980ce0b0 tx=0x7f208402a110 comp rx=0 tx=0).stop 2024-04-25T10:13:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 -- 172.21.15.5:0/3533781033 shutdown_connections 2024-04-25T10:13:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 --2- 172.21.15.5:0/3533781033 >> v2:172.21.15.80:3300/0 conn(0x7f20980af760 0x7f20980afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 --2- 172.21.15.5:0/3533781033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b3690 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 -- 172.21.15.5:0/3533781033 >> 172.21.15.5:0/3533781033 conn(0x7f20980b2390 msgr2=0x7f20980b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 -- 172.21.15.5:0/3533781033 shutdown_connections 2024-04-25T10:13:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.122+0000 7f209fe00640 1 -- 172.21.15.5:0/3533781033 wait complete. 2024-04-25T10:13:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.124+0000 7f209fe00640 1 Processor -- start 2024-04-25T10:13:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.124+0000 7f209fe00640 1 -- start start 2024-04-25T10:13:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.125+0000 7f209fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.125+0000 7f209fe00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20980af760 0x7f209815d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.125+0000 7f209fe00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20980d00e0 0x7f209815d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.125+0000 7f209fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209815eab0 con 0x7f209806b270 2024-04-25T10:13:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.125+0000 7f209fe00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209815e7d0 con 0x7f20980af760 2024-04-25T10:13:37.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.125+0000 7f209fe00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2098161820 con 0x7f20980d00e0 2024-04-25T10:13:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.126+0000 7f209d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.126+0000 7f209d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55490/0 (socket says 172.21.15.5:55490) 2024-04-25T10:13:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.126+0000 7f209d400640 1 -- 172.21.15.5:0/1540658651 learned_addr learned my addr 172.21.15.5:0/1540658651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.126+0000 7f209ca00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20980af760 0x7f209815d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.126+0000 7f209de00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20980d00e0 0x7f209815d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209d400640 1 -- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20980d00e0 msgr2=0x7f209815d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209d400640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20980d00e0 0x7f209815d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209d400640 1 -- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20980af760 msgr2=0x7f209815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209d400640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20980af760 0x7f209815d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209d400640 1 -- 172.21.15.5:0/1540658651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20981f3700 con 0x7f209806b270 2024-04-25T10:13:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209ca00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20980af760 0x7f209815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.127+0000 7f209d400640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b1550 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7f0 tx=0x7f208c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.128+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c018070 con 0x7f209806b270 2024-04-25T10:13:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.128+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c00a640 con 0x7f209806b270 2024-04-25T10:13:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.128+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20981f39a0 con 0x7f209806b270 2024-04-25T10:13:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.128+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00aa30 con 0x7f209806b270 2024-04-25T10:13:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.128+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20981f3f60 con 0x7f209806b270 2024-04-25T10:13:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.129+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2098068fd0 con 0x7f209806b270 2024-04-25T10:13:37.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.130+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f208c01b070 con 0x7f209806b270 2024-04-25T10:13:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.131+0000 7f2095600640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f207806b4d0 0x7f207806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.131+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f208c089990 con 0x7f209806b270 2024-04-25T10:13:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.132+0000 7f209ca00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f207806b4d0 0x7f207806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.133+0000 7f209ca00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f207806b4d0 0x7f207806d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2084034000 tx=0x7f208402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.138+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f208c057f70 con 0x7f209806b270 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.379+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2098159cf0 con 0x7f209806b270 2024-04-25T10:13:37.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.380+0000 7f2095600640 1 -- 172.21.15.5:0/1540658651 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f208c05c010 con 0x7f209806b270 2024-04-25T10:13:37.385 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:37.385 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:25.320889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:37.385 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 112 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f207806b4d0 msgr2=0x7f207806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f207806b4d0 0x7f207806d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2084034000 tx=0x7f208402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 msgr2=0x7f20980b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b1550 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7f0 tx=0x7f208c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 shutdown_connections 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20980d00e0 0x7f209815d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f207806b4d0 0x7f207806d990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20980af760 0x7f209815d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 --2- 172.21.15.5:0/1540658651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f209806b270 0x7f20980b1550 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 >> 172.21.15.5:0/1540658651 conn(0x7f20980b2390 msgr2=0x7f20980cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 shutdown_connections 2024-04-25T10:13:37.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.383+0000 7f209fe00640 1 -- 172.21.15.5:0/1540658651 wait complete. 2024-04-25T10:13:37.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:37.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.615+0000 7f6723200640 1 Processor -- start 2024-04-25T10:13:37.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.615+0000 7f6723200640 1 -- start start 2024-04-25T10:13:37.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.616+0000 7f6723200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f672415b780 0x7f67240b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.616+0000 7f6723200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67240b29d0 0x7f67240cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.616+0000 7f6723200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672415bb60 con 0x7f67240cfe00 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6723200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240d0230 con 0x7f67240b29d0 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6723200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240cc230 con 0x7f672415b780 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67240b29d0 0x7f67240cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6722800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f67240cfe00 0x7f67240999f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39610/0 (socket says 172.21.15.5:39610) 2024-04-25T10:13:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f67240b29d0 0x7f67240cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55496/0 (socket says 172.21.15.5:55496) 2024-04-25T10:13:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6722800640 1 -- 172.21.15.5:0/3971654413 learned_addr learned my addr 172.21.15.5:0/3971654413 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721e00640 1 --2- 172.21.15.5:0/3971654413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f672415b780 0x7f67240b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721400640 1 -- 172.21.15.5:0/3971654413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f672415b780 msgr2=0x7f67240b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721400640 1 --2- 172.21.15.5:0/3971654413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f672415b780 0x7f67240b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721400640 1 -- 172.21.15.5:0/3971654413 >> v1:172.21.15.5:6789/0 conn(0x7f67240cfe00 legacy=0x7f67240999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6721400640 1 -- 172.21.15.5:0/3971654413 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240cc510 con 0x7f67240b29d0 2024-04-25T10:13:37.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.617+0000 7f6720a00640 1 -- 172.21.15.5:0/3971654413 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965029248 0 0) 0x7f672415bb60 con 0x7f67240cfe00 2024-04-25T10:13:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.619+0000 7f6721400640 1 --2- 172.21.15.5:0/3971654413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67240b29d0 0x7f67240cf8c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f670c009d90 tx=0x7f670c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d821bfcdbef5086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.619+0000 7f6720a00640 1 -- 172.21.15.5:0/3971654413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c007740 con 0x7f67240b29d0 2024-04-25T10:13:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.619+0000 7f6720a00640 1 -- 172.21.15.5:0/3971654413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c0078e0 con 0x7f67240b29d0 2024-04-25T10:13:37.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.619+0000 7f6720a00640 1 -- 172.21.15.5:0/3971654413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c007cb0 con 0x7f67240b29d0 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.619+0000 7f6723200640 1 -- 172.21.15.5:0/3971654413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67240b29d0 msgr2=0x7f67240cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.619+0000 7f6723200640 1 --2- 172.21.15.5:0/3971654413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67240b29d0 0x7f67240cf8c0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f670c009d90 tx=0x7f670c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.620+0000 7f6723200640 1 -- 172.21.15.5:0/3971654413 shutdown_connections 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.620+0000 7f6723200640 1 --2- 172.21.15.5:0/3971654413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67240b29d0 0x7f67240cf8c0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.620+0000 7f6723200640 1 --2- 172.21.15.5:0/3971654413 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f672415b780 0x7f67240b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.620+0000 7f6723200640 1 -- 172.21.15.5:0/3971654413 >> 172.21.15.5:0/3971654413 conn(0x7f67240d1470 msgr2=0x7f67240d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.620+0000 7f6723200640 1 -- 172.21.15.5:0/3971654413 shutdown_connections 2024-04-25T10:13:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.620+0000 7f6723200640 1 -- 172.21.15.5:0/3971654413 wait complete. 2024-04-25T10:13:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.621+0000 7f6723200640 1 Processor -- start 2024-04-25T10:13:37.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.622+0000 7f6723200640 1 -- start start 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6723200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67240b29d0 0x7f672414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6723200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67240cfe00 0x7f67240b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6723200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 0x7f67240b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6723200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240ccdb0 con 0x7f672415b780 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6723200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240ccad0 con 0x7f67240cfe00 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6723200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240cbe50 con 0x7f67240b29d0 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 0x7f67240b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 0x7f67240b4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55502/0 (socket says 172.21.15.5:55502) 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 -- 172.21.15.5:0/2298950012 learned_addr learned my addr 172.21.15.5:0/2298950012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6721400640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67240cfe00 0x7f67240b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6721e00640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67240b29d0 0x7f672414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 -- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67240b29d0 msgr2=0x7f672414b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67240b29d0 0x7f672414b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 -- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67240cfe00 msgr2=0x7f67240b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67240cfe00 0x7f67240b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.623+0000 7f6722800640 1 -- 172.21.15.5:0/2298950012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672415d7a0 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f6721e00640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67240b29d0 0x7f672414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f6722800640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 0x7f67240b4560 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c410 tx=0x7f671c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0074e0 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0047e0 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672415da80 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c004bd0 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.624+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672415e040 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.625+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6724068f60 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.626+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f671c011070 con 0x7f672415b780 2024-04-25T10:13:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.627+0000 7f671a000640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f670006b550 0x7f670006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.627+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f671c08a080 con 0x7f672415b780 2024-04-25T10:13:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.628+0000 7f6721e00640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f670006b550 0x7f670006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.629+0000 7f6721e00640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f670006b550 0x7f670006da10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6714006fd0 tx=0x7f6714008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.632+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f671c058c10 con 0x7f672415b780 2024-04-25T10:13:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.832+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67240b1f50 con 0x7f672415b780 2024-04-25T10:13:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.834+0000 7f671a000640 1 -- 172.21.15.5:0/2298950012 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17101 (secure 0 0 0) 0x7f671c058510 con 0x7f672415b780 2024-04-25T10:13:37.839 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:37.839 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":122,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:13:20.803041+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.5:6833/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.80:6837/84886517":"2024-04-26T10:13:06.161908+0000","172.21.15.80:6836/84886517":"2024-04-26T10:13:06.161908+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f670006b550 msgr2=0x7f670006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f670006b550 0x7f670006da10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6714006fd0 tx=0x7f6714008040 comp rx=0 tx=0).stop 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 msgr2=0x7f67240b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 0x7f67240b4560 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c410 tx=0x7f671c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 shutdown_connections 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672415b780 0x7f67240b4560 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f670006b550 0x7f670006da10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67240cfe00 0x7f67240b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 --2- 172.21.15.5:0/2298950012 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f67240b29d0 0x7f672414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 >> 172.21.15.5:0/2298950012 conn(0x7f67240d1470 msgr2=0x7f67240b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 shutdown_connections 2024-04-25T10:13:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:37.838+0000 7f6723200640 1 -- 172.21.15.5:0/2298950012 wait complete. 2024-04-25T10:13:37.863 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:13:37.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:38.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.070+0000 7f48c6400640 1 Processor -- start 2024-04-25T10:13:38.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.070+0000 7f48c6400640 1 -- start start 2024-04-25T10:13:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.071+0000 7f48c6400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f48c015db00 0x7f48c00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.071+0000 7f48c6400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 0x7f48c00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.071+0000 7f48c6400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00d1e80 con 0x7f48c00b53b0 2024-04-25T10:13:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.071+0000 7f48c6400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c00b5930 con 0x7f48c015db00 2024-04-25T10:13:38.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.071+0000 7f48c6400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00b5c10 con 0x7f48c00ce2b0 2024-04-25T10:13:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.072+0000 7f48bea00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f48c00b53b0 0x7f48c0161670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39614/0 (socket says 172.21.15.5:39614) 2024-04-25T10:13:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.072+0000 7f48bea00640 1 -- 172.21.15.5:0/2818892611 learned_addr learned my addr 172.21.15.5:0/2818892611 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.072+0000 7f48bfe00640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 0x7f48c00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.072+0000 7f48bf400640 1 --2- 172.21.15.5:0/2818892611 >> v2:172.21.15.80:3300/0 conn(0x7f48c015db00 0x7f48c00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.072+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920055354 0 0) 0x7f48c00d1e80 con 0x7f48c00b53b0 2024-04-25T10:13:38.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.072+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a4003520 con 0x7f48c00b53b0 2024-04-25T10:13:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3488043842 0 0) 0x7f48a4003520 con 0x7f48c00b53b0 2024-04-25T10:13:38.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 msgr2=0x7f48c00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 0x7f48c00d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 >> v2:172.21.15.80:3300/0 conn(0x7f48c015db00 msgr2=0x7f48c00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 --2- 172.21.15.5:0/2818892611 >> v2:172.21.15.80:3300/0 conn(0x7f48c015db00 0x7f48c00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00b5ef0 con 0x7f48c00b53b0 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f48ac001670 con 0x7f48c00b53b0 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 >> v1:172.21.15.5:6789/0 conn(0x7f48c00b53b0 legacy=0x7f48c0161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 0x7f48a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48a4003f80 0x7f48a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4004d40 0x7f48a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00b5930 con 0x7f48a4003f80 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00b5c10 con 0x7f48a4004d40 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a400b9c0 con 0x7f48c00ce2b0 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48bea00640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48a4003f80 0x7f48a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.073+0000 7f48bf400640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 0x7f48a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.074+0000 7f48bfe00640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4004d40 0x7f48a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.074+0000 7f48bea00640 1 -- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 msgr2=0x7f48a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.074+0000 7f48bea00640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c00ce2b0 0x7f48a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.074+0000 7f48bea00640 1 -- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4004d40 msgr2=0x7f48a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.074+0000 7f48bea00640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4004d40 0x7f48a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.074+0000 7f48bea00640 1 -- 172.21.15.5:0/2818892611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4003520 con 0x7f48a4003f80 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.075+0000 7f48bea00640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48a4003f80 0x7f48a4004800 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f48ac007fd0 tx=0x7f48a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcfc624c474fa73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.075+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac007ca0 con 0x7f48a4003f80 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.075+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac005ad0 con 0x7f48a4003f80 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.075+0000 7f48be000640 1 -- 172.21.15.5:0/2818892611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac007070 con 0x7f48a4003f80 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48c6400640 1 -- 172.21.15.5:0/2818892611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48a4003f80 msgr2=0x7f48a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48c6400640 1 --2- 172.21.15.5:0/2818892611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48a4003f80 0x7f48a4004800 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f48ac007fd0 tx=0x7f48a4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48bfe00640 1 -- 172.21.15.5:0/2818892611 reap_dead start 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48c6400640 1 -- 172.21.15.5:0/2818892611 shutdown_connections 2024-04-25T10:13:38.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48c6400640 1 -- 172.21.15.5:0/2818892611 >> 172.21.15.5:0/2818892611 conn(0x7f48c0072290 msgr2=0x7f48c00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48c6400640 1 -- 172.21.15.5:0/2818892611 shutdown_connections 2024-04-25T10:13:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.076+0000 7f48c6400640 1 -- 172.21.15.5:0/2818892611 wait complete. 2024-04-25T10:13:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.078+0000 7f48c6400640 1 Processor -- start 2024-04-25T10:13:38.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.078+0000 7f48c6400640 1 -- start start 2024-04-25T10:13:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48c6400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4003f80 0x7f48c0143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48c6400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c0143b90 0x7f48c01599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48c6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 0x7f48c0146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48c6400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00d3660 con 0x7f48c0159f00 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48c6400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00b61d0 con 0x7f48a4003f80 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48c6400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00b6c10 con 0x7f48c0143b90 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.079+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 0x7f48c0146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 0x7f48c0146140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55528/0 (socket says 172.21.15.5:55528) 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 -- 172.21.15.5:0/2793956766 learned_addr learned my addr 172.21.15.5:0/2793956766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bf400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4003f80 0x7f48c0143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bea00640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c0143b90 0x7f48c01599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 -- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c0143b90 msgr2=0x7f48c01599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c0143b90 0x7f48c01599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 -- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4003f80 msgr2=0x7f48c0143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4003f80 0x7f48c0143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 -- 172.21.15.5:0/2793956766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c01f3a60 con 0x7f48c0159f00 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.080+0000 7f48bfe00640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 0x7f48c0146140 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f48c00ca9b0 tx=0x7f48b8011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.081+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8004440 con 0x7f48c0159f00 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.081+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b801b070 con 0x7f48c0159f00 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.081+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8003850 con 0x7f48c0159f00 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.081+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c015fb50 con 0x7f48c0159f00 2024-04-25T10:13:38.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.081+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c0160110 con 0x7f48c0159f00 2024-04-25T10:13:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.082+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c0153490 con 0x7f48c0159f00 2024-04-25T10:13:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.083+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48b80039f0 con 0x7f48c0159f00 2024-04-25T10:13:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.084+0000 7f48b7400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48a006b500 0x7f48a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.085+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f48b808a070 con 0x7f48c0159f00 2024-04-25T10:13:38.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.085+0000 7f48bf400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48a006b500 0x7f48a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.086+0000 7f48bf400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48a006b500 0x7f48a006d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f48b0007520 tx=0x7f48b000b040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.089+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48c0153490 con 0x7f48c0159f00 2024-04-25T10:13:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.327+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48c0068fd0 con 0x7f48c0159f00 2024-04-25T10:13:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.328+0000 7f48b7400640 1 -- 172.21.15.5:0/2793956766 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4937 (secure 0 0 0) 0x7f48b8024020 con 0x7f48c0159f00 2024-04-25T10:13:38.332 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:38.333 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:25.320889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 112 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48a006b500 msgr2=0x7f48a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48a006b500 0x7f48a006d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f48b0007520 tx=0x7f48b000b040 comp rx=0 tx=0).stop 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 msgr2=0x7f48c0146140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 0x7f48c0146140 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f48c00ca9b0 tx=0x7f48b8011920 comp rx=0 tx=0).stop 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 shutdown_connections 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48c0159f00 0x7f48c0146140 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f48c0143b90 0x7f48c01599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f48a006b500 0x7f48a006d9c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 --2- 172.21.15.5:0/2793956766 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48a4003f80 0x7f48c0143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 >> 172.21.15.5:0/2793956766 conn(0x7f48c0072290 msgr2=0x7f48c00c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 shutdown_connections 2024-04-25T10:13:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.331+0000 7f48c6400640 1 -- 172.21.15.5:0/2793956766 wait complete. 2024-04-25T10:13:38.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.564+0000 7fb76d200640 1 Processor -- start 2024-04-25T10:13:38.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.564+0000 7fb76d200640 1 -- start start 2024-04-25T10:13:38.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.566+0000 7fb76d200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb76815ce00 0x7fb76815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.566+0000 7fb76d200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7680cd300 0x7fb7680cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.566+0000 7fb76d200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb768071d30 con 0x7fb768141da0 2024-04-25T10:13:38.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.566+0000 7fb76d200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7680cdcd0 con 0x7fb76815ce00 2024-04-25T10:13:38.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.566+0000 7fb76d200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb768142180 con 0x7fb7680cd300 2024-04-25T10:13:38.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb767400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb768141da0 0x7fb76806e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39620/0 (socket says 172.21.15.5:39620) 2024-04-25T10:13:38.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb767400640 1 -- 172.21.15.5:0/558208529 learned_addr learned my addr 172.21.15.5:0/558208529 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb75ea00640 1 --2- 172.21.15.5:0/558208529 >> v2:172.21.15.5:3300/0 conn(0x7fb76815ce00 0x7fb76815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb766a00640 1 -- 172.21.15.5:0/558208529 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119694161 0 0) 0x7fb768071d30 con 0x7fb768141da0 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb766a00640 1 -- 172.21.15.5:0/558208529 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74c003520 con 0x7fb768141da0 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb767e00640 1 --2- 172.21.15.5:0/558208529 >> v2:172.21.15.80:3300/0 conn(0x7fb7680cd300 0x7fb7680cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb75ea00640 1 -- 172.21.15.5:0/558208529 >> v2:172.21.15.80:3300/0 conn(0x7fb7680cd300 msgr2=0x7fb7680cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb75ea00640 1 --2- 172.21.15.5:0/558208529 >> v2:172.21.15.80:3300/0 conn(0x7fb7680cd300 0x7fb7680cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb75ea00640 1 -- 172.21.15.5:0/558208529 >> v1:172.21.15.5:6789/0 conn(0x7fb768141da0 legacy=0x7fb76806e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb75ea00640 1 -- 172.21.15.5:0/558208529 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76806d620 con 0x7fb76815ce00 2024-04-25T10:13:38.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.567+0000 7fb767e00640 1 --2- 172.21.15.5:0/558208529 >> v2:172.21.15.80:3300/0 conn(0x7fb7680cd300 0x7fb7680cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:38.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.569+0000 7fb75ea00640 1 --2- 172.21.15.5:0/558208529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76815ce00 0x7fb76815d280 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb76815f220 tx=0x7fb75002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa373125f21d62c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.569+0000 7fb766a00640 1 -- 172.21.15.5:0/558208529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750037070 con 0x7fb76815ce00 2024-04-25T10:13:38.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.569+0000 7fb766a00640 1 -- 172.21.15.5:0/558208529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb750032440 con 0x7fb76815ce00 2024-04-25T10:13:38.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.569+0000 7fb766a00640 1 -- 172.21.15.5:0/558208529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750032810 con 0x7fb76815ce00 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.569+0000 7fb76d200640 1 -- 172.21.15.5:0/558208529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76815ce00 msgr2=0x7fb76815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.569+0000 7fb76d200640 1 --2- 172.21.15.5:0/558208529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76815ce00 0x7fb76815d280 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb76815f220 tx=0x7fb75002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.570+0000 7fb76d200640 1 -- 172.21.15.5:0/558208529 shutdown_connections 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.570+0000 7fb76d200640 1 --2- 172.21.15.5:0/558208529 >> v2:172.21.15.80:3300/0 conn(0x7fb7680cd300 0x7fb7680cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.570+0000 7fb76d200640 1 --2- 172.21.15.5:0/558208529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76815ce00 0x7fb76815d280 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.570+0000 7fb76d200640 1 -- 172.21.15.5:0/558208529 >> 172.21.15.5:0/558208529 conn(0x7fb768066f20 msgr2=0x7fb76806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.570+0000 7fb76d200640 1 -- 172.21.15.5:0/558208529 shutdown_connections 2024-04-25T10:13:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.570+0000 7fb76d200640 1 -- 172.21.15.5:0/558208529 wait complete. 2024-04-25T10:13:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.571+0000 7fb76d200640 1 Processor -- start 2024-04-25T10:13:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.572+0000 7fb76d200640 1 -- start start 2024-04-25T10:13:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb76d200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7680cd300 0x7fb76813bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb76d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 0x7fb76815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb76d200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb76815ce00 0x7fb768136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb76d200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76806e810 con 0x7fb768141da0 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb76d200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76806dbe0 con 0x7fb76815ce00 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb76d200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb768071000 con 0x7fb7680cd300 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 0x7fb76815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 0x7fb76815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55542/0 (socket says 172.21.15.5:55542) 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 -- 172.21.15.5:0/3926858622 learned_addr learned my addr 172.21.15.5:0/3926858622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 -- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7680cd300 msgr2=0x7fb76813bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7680cd300 0x7fb76813bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 -- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb76815ce00 msgr2=0x7fb768136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb76815ce00 0x7fb768136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.573+0000 7fb75ea00640 1 -- 172.21.15.5:0/3926858622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7681373e0 con 0x7fb768141da0 2024-04-25T10:13:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.574+0000 7fb75ea00640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 0x7fb76815b1f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fb750009890 tx=0x7fb7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.574+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750037040 con 0x7fb768141da0 2024-04-25T10:13:38.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.574+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb750042070 con 0x7fb768141da0 2024-04-25T10:13:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.574+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750030900 con 0x7fb768141da0 2024-04-25T10:13:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.574+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76813b1d0 con 0x7fb768141da0 2024-04-25T10:13:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.574+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb768136fa0 con 0x7fb768141da0 2024-04-25T10:13:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.575+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb768068f60 con 0x7fb768141da0 2024-04-25T10:13:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.576+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb750030aa0 con 0x7fb768141da0 2024-04-25T10:13:38.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.578+0000 7fb75fe00640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb74406b4d0 0x7fb74406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.578+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fb7500aad80 con 0x7fb768141da0 2024-04-25T10:13:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.578+0000 7fb767400640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb74406b4d0 0x7fb74406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.580+0000 7fb767400640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb74406b4d0 0x7fb74406d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb758000f60 tx=0x7fb7580078f0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.583+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb750045360 con 0x7fb768141da0 2024-04-25T10:13:38.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.820+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb76813fac0 con 0x7fb768141da0 2024-04-25T10:13:38.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.821+0000 7fb75fe00640 1 -- 172.21.15.5:0/3926858622 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4937 (secure 0 0 0) 0x7fb7500799b0 con 0x7fb768141da0 2024-04-25T10:13:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:38.826 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:47.523031+0000","modified":"2024-04-25T10:13:25.320889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9402":{"gid":9402,"name":"c","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6835/806266896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":806266896},{"type":"v1","addr":"172.21.15.5:6835","nonce":806266896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 112 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb74406b4d0 msgr2=0x7fb74406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb74406b4d0 0x7fb74406d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb758000f60 tx=0x7fb7580078f0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 msgr2=0x7fb76815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 0x7fb76815b1f0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fb750009890 tx=0x7fb7500046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 shutdown_connections 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb74406b4d0 0x7fb74406d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb76815ce00 0x7fb768136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb768141da0 0x7fb76815b1f0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 --2- 172.21.15.5:0/3926858622 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb7680cd300 0x7fb76813bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 >> 172.21.15.5:0/3926858622 conn(0x7fb768066f20 msgr2=0x7fb768140f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 shutdown_connections 2024-04-25T10:13:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:38.824+0000 7fb76d200640 1 -- 172.21.15.5:0/3926858622 wait complete. 2024-04-25T10:13:38.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:39.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.051+0000 7fae23800640 1 Processor -- start 2024-04-25T10:13:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.052+0000 7fae23800640 1 -- start start 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.053+0000 7fae23800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fae1c06bc70 0x7fae1c09a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.053+0000 7fae23800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0cfad0 con 0x7fae1c15da20 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.053+0000 7fae23800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0cfdb0 con 0x7fae1c06c330 2024-04-25T10:13:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.053+0000 7fae23800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c0d06e0 con 0x7fae1c06bc70 2024-04-25T10:13:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.054+0000 7fae20e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fae1c15da20 0x7fae1c0d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39632/0 (socket says 172.21.15.5:39632) 2024-04-25T10:13:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.054+0000 7fae20e00640 1 -- 172.21.15.5:0/1557756646 learned_addr learned my addr 172.21.15.5:0/1557756646 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.054+0000 7fae1be00640 1 --2- 172.21.15.5:0/1557756646 >> v2:172.21.15.80:3300/0 conn(0x7fae1c06bc70 0x7fae1c09a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.054+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319389115 0 0) 0x7fae1c0cfad0 con 0x7fae1c15da20 2024-04-25T10:13:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.054+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae00003520 con 0x7fae1c15da20 2024-04-25T10:13:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538446324 0 0) 0x7fae00003520 con 0x7fae1c15da20 2024-04-25T10:13:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 >> v2:172.21.15.80:3300/0 conn(0x7fae1c06bc70 msgr2=0x7fae1c09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 --2- 172.21.15.5:0/1557756646 >> v2:172.21.15.80:3300/0 conn(0x7fae1c06bc70 0x7fae1c09a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 >> v1:172.21.15.80:6789/0 conn(0x7fae1c06c330 legacy=0x7fae1c0cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0d0090 con 0x7fae1c15da20 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fae0c001670 con 0x7fae1c15da20 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 >> v1:172.21.15.5:6789/0 conn(0x7fae1c15da20 legacy=0x7fae1c0d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000039b0 0x7fae00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae00004800 0x7fae00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae000051c0 0x7fae0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0d06e0 con 0x7fae000051c0 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c060 con 0x7fae000039b0 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c260 con 0x7fae00004800 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.055+0000 7fae21800640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae000051c0 0x7fae0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae1be00640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae00004800 0x7fae00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae20e00640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000039b0 0x7fae00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae21800640 1 -- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae00004800 msgr2=0x7fae00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae21800640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae00004800 0x7fae00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae21800640 1 -- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000039b0 msgr2=0x7fae00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae21800640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000039b0 0x7fae00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae21800640 1 -- 172.21.15.5:0/1557756646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00003520 con 0x7fae000051c0 2024-04-25T10:13:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.056+0000 7fae1be00640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae00004800 0x7fae00004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.057+0000 7fae21800640 1 -- 172.21.15.5:0/1557756646 reap_dead start 2024-04-25T10:13:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.057+0000 7fae21800640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae000051c0 0x7fae0000b9c0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fae14001e50 tx=0x7fae00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755bae5056b55565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.057+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14001a70 con 0x7fae000051c0 2024-04-25T10:13:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.057+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14003970 con 0x7fae000051c0 2024-04-25T10:13:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.057+0000 7fae1b400640 1 -- 172.21.15.5:0/1557756646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14003d60 con 0x7fae000051c0 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 -- 172.21.15.5:0/1557756646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae000051c0 msgr2=0x7fae0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae000051c0 0x7fae0000b9c0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fae14001e50 tx=0x7fae00005830 comp rx=0 tx=0).stop 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 -- 172.21.15.5:0/1557756646 shutdown_connections 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 --2- 172.21.15.5:0/1557756646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae000051c0 0x7fae0000b9c0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 -- 172.21.15.5:0/1557756646 >> 172.21.15.5:0/1557756646 conn(0x7fae1c0d16f0 msgr2=0x7fae1c0d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 -- 172.21.15.5:0/1557756646 shutdown_connections 2024-04-25T10:13:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.058+0000 7fae23800640 1 -- 172.21.15.5:0/1557756646 wait complete. 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.060+0000 7fae23800640 1 Processor -- start 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.060+0000 7fae23800640 1 -- start start 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae23800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000051c0 0x7fae1c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae23800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 0x7fae1c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae23800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae1c0b62b0 0x7fae1c0b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae23800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c15df80 con 0x7fae1c0b41a0 2024-04-25T10:13:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae23800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c15de00 con 0x7fae000051c0 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae23800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c06c7e0 con 0x7fae1c0b62b0 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae1be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 0x7fae1c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae1be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 0x7fae1c0b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55558/0 (socket says 172.21.15.5:55558) 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.061+0000 7fae1be00640 1 -- 172.21.15.5:0/1913566155 learned_addr learned my addr 172.21.15.5:0/1913566155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae20e00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000051c0 0x7fae1c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae21800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae1c0b62b0 0x7fae1c0b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae1be00640 1 -- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae1c0b62b0 msgr2=0x7fae1c0b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae1be00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae1c0b62b0 0x7fae1c0b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae1be00640 1 -- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000051c0 msgr2=0x7fae1c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae1be00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000051c0 0x7fae1c14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae1be00640 1 -- 172.21.15.5:0/1913566155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0b6ca0 con 0x7fae1c0b41a0 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae20e00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000051c0 0x7fae1c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.062+0000 7fae1be00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 0x7fae1c0b4620 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fae080108b0 tx=0x7fae08010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.063+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800f040 con 0x7fae1c0b41a0 2024-04-25T10:13:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.063+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800eb60 con 0x7fae1c0b41a0 2024-04-25T10:13:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.063+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0801c400 con 0x7fae1c0b41a0 2024-04-25T10:13:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.063+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c0c5a10 con 0x7fae1c0b41a0 2024-04-25T10:13:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.063+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c0c5cb0 con 0x7fae1c0b41a0 2024-04-25T10:13:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.064+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c068fd0 con 0x7fae1c0b41a0 2024-04-25T10:13:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.065+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae0801c5a0 con 0x7fae1c0b41a0 2024-04-25T10:13:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.066+0000 7fae18c00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadfc06b500 0x7fadfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.066+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fae0808ab60 con 0x7fae1c0b41a0 2024-04-25T10:13:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.066+0000 7fae20e00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadfc06b500 0x7fadfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.067+0000 7fae20e00640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadfc06b500 0x7fadfc06d9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fae0c001050 tx=0x7fae0c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.070+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae1c0b6ca0 con 0x7fae1c0b41a0 2024-04-25T10:13:39.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.266+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae1c0d2500 con 0x7fae1c0b41a0 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.269+0000 7fae18c00640 1 -- 172.21.15.5:0/1913566155 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17101 (secure 0 0 0) 0x7fae08018030 con 0x7fae1c0b41a0 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":122,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:13:20.803041+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:45.398584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:46.541796+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.5:6833/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.80:6837/84886517":"2024-04-26T10:13:06.161908+0000","172.21.15.80:6836/84886517":"2024-04-26T10:13:06.161908+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadfc06b500 msgr2=0x7fadfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadfc06b500 0x7fadfc06d9c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fae0c001050 tx=0x7fae0c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 msgr2=0x7fae1c0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 0x7fae1c0b4620 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fae080108b0 tx=0x7fae08010d70 comp rx=0 tx=0).stop 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 shutdown_connections 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fae1c0b62b0 0x7fae1c0b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c0b41a0 0x7fae1c0b4620 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fadfc06b500 0x7fadfc06d9c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 --2- 172.21.15.5:0/1913566155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae000051c0 0x7fae1c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 >> 172.21.15.5:0/1913566155 conn(0x7fae1c0d16f0 msgr2=0x7fae1c0d1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 shutdown_connections 2024-04-25T10:13:39.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.271+0000 7fae23800640 1 -- 172.21.15.5:0/1913566155 wait complete. 2024-04-25T10:13:39.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:39.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.481+0000 7f45d3e00640 1 Processor -- start 2024-04-25T10:13:39.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.481+0000 7f45d3e00640 1 -- start start 2024-04-25T10:13:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.483+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.483+0000 7f45d3e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45d40ceea0 0x7f45d40b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.483+0000 7f45d3e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d40b4880 con 0x7f45d40b3ae0 2024-04-25T10:13:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.483+0000 7f45d3e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40b45a0 con 0x7f45d40ceea0 2024-04-25T10:13:39.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.483+0000 7f45d3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d415e2c0 con 0x7f45d4143d30 2024-04-25T10:13:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d2a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45d40ceea0 0x7f45d40b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45d40ceea0 0x7f45d40b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:47270/0 (socket says 172.21.15.5:47270) 2024-04-25T10:13:39.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 -- 172.21.15.5:0/53121958 learned_addr learned my addr 172.21.15.5:0/53121958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d2000640 1 --1- 172.21.15.5:0/53121958 >> v1:172.21.15.80:6789/0 conn(0x7f45d40b3ae0 0x7f45d40ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:46068/0 (socket says 172.21.15.5:46068) 2024-04-25T10:13:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 -- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 msgr2=0x7f45d415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 --2- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 -- 172.21.15.5:0/53121958 >> v1:172.21.15.80:6789/0 conn(0x7f45d40b3ae0 legacy=0x7f45d40ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.484+0000 7f45d3400640 1 -- 172.21.15.5:0/53121958 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d415e5a0 con 0x7f45d40ceea0 2024-04-25T10:13:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.486+0000 7f45d3400640 1 --2- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d40b3f60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009e50 tx=0x7f45cc02c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e3a60cafc80889 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d1600640 1 -- 172.21.15.5:0/53121958 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc032430 con 0x7f45d40ceea0 2024-04-25T10:13:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d1600640 1 -- 172.21.15.5:0/53121958 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc037070 con 0x7f45d40ceea0 2024-04-25T10:13:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d1600640 1 -- 172.21.15.5:0/53121958 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0327f0 con 0x7f45d40ceea0 2024-04-25T10:13:39.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d3e00640 1 -- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 msgr2=0x7f45d40b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d3e00640 1 --2- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d40b3f60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009e50 tx=0x7f45cc02c940 comp rx=0 tx=0).stop 2024-04-25T10:13:39.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d3e00640 1 -- 172.21.15.5:0/53121958 shutdown_connections 2024-04-25T10:13:39.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.487+0000 7f45d3e00640 1 --2- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d40b3f60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.488+0000 7f45d3e00640 1 --2- 172.21.15.5:0/53121958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.488+0000 7f45d3e00640 1 -- 172.21.15.5:0/53121958 >> 172.21.15.5:0/53121958 conn(0x7f45d4066f20 msgr2=0x7f45d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.488+0000 7f45d3e00640 1 -- 172.21.15.5:0/53121958 shutdown_connections 2024-04-25T10:13:39.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.488+0000 7f45d3e00640 1 -- 172.21.15.5:0/53121958 wait complete. 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.490+0000 7f45d3e00640 1 Processor -- start 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.490+0000 7f45d3e00640 1 -- start start 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 0x7f45d41e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d41e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d4132030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4144290 con 0x7f45d40b3ae0 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4144110 con 0x7f45d40ceea0 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d406e430 con 0x7f45d4143d30 2024-04-25T10:13:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.491+0000 7f45d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 0x7f45d41e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 0x7f45d41e86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55572/0 (socket says 172.21.15.5:55572) 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 -- 172.21.15.5:0/1410014339 learned_addr learned my addr 172.21.15.5:0/1410014339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2000640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d41e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d3400640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d4132030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 -- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 msgr2=0x7f45d4132030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d4132030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 -- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 msgr2=0x7f45d41e8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d41e8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 -- 172.21.15.5:0/1410014339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d41301d0 con 0x7f45d40b3ae0 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d3400640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d4132030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.492+0000 7f45d2a00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 0x7f45d41e86e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9b0 tx=0x7f45c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.493+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4017070 con 0x7f45d40b3ae0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.493+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d41304b0 con 0x7f45d40b3ae0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.493+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c400c9a0 con 0x7f45d40b3ae0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.493+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400cd90 con 0x7f45d40b3ae0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.493+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d41e5d50 con 0x7f45d40b3ae0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.494+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d41e6010 con 0x7f45d40b3ae0 2024-04-25T10:13:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.495+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45c401a070 con 0x7f45d40b3ae0 2024-04-25T10:13:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.496+0000 7f45caa00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.496+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f45c408a900 con 0x7f45d40b3ae0 2024-04-25T10:13:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.496+0000 7f45d2000640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.497+0000 7f45d2000640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f45d40cc8c0 tx=0x7f45c0009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.500+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45d41e6010 con 0x7f45d40b3ae0 2024-04-25T10:13:39.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.734+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f45d4068f60 con 0x7f45d40b3ae0 2024-04-25T10:13:39.739 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-04-25T10:13:39.735+0000 7f92a5000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.739+0000 7f45caa00640 1 -- 172.21.15.5:0/1410014339 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 161+0+0 (secure 0 0 0) 0x7f45c4059420 con 0x7f45d40b3ae0 2024-04-25T10:13:39.743 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:39.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.742+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 msgr2=0x7f45b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.742+0000 7f45d3e00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f45d40cc8c0 tx=0x7f45c0009040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.742+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 msgr2=0x7f45d41e86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.742+0000 7f45d3e00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 0x7f45d41e86e0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9b0 tx=0x7f45c400ee70 comp rx=0 tx=0).stop 2024-04-25T10:13:39.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 shutdown_connections 2024-04-25T10:13:39.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f45d4143d30 0x7f45d4132030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f45b406b4d0 0x7f45b406d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45d40ceea0 0x7f45d41e8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 --2- 172.21.15.5:0/1410014339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d40b3ae0 0x7f45d41e86e0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 >> 172.21.15.5:0/1410014339 conn(0x7f45d4066f20 msgr2=0x7f45d4158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.743+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 shutdown_connections 2024-04-25T10:13:39.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.744+0000 7f45d3e00640 1 -- 172.21.15.5:0/1410014339 wait complete. 2024-04-25T10:13:39.754 INFO:tasks.ceph.mds.c.smithi005.stderr: -1397> 2024-04-25T10:13:27.821+0000 7f15a7400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:39.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:13:39.805 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2024-04-25T10:13:39.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.941+0000 7fc159200640 1 Processor -- start 2024-04-25T10:13:39.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.942+0000 7fc159200640 1 -- start start 2024-04-25T10:13:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.943+0000 7fc159200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc154161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.943+0000 7fc159200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1541606e0 con 0x7fc1540b48c0 2024-04-25T10:13:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.943+0000 7fc159200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1541609c0 con 0x7fc154161bc0 2024-04-25T10:13:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.943+0000 7fc159200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc154160ca0 con 0x7fc1540d3f90 2024-04-25T10:13:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.943+0000 7fc153400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc1540b48c0 0x7fc15415fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39644/0 (socket says 172.21.15.5:39644) 2024-04-25T10:13:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.943+0000 7fc153400640 1 -- 172.21.15.5:0/1607812304 learned_addr learned my addr 172.21.15.5:0/1607812304 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744955105 0 0) 0x7fc1541606e0 con 0x7fc1540b48c0 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152a00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc154161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc138003520 con 0x7fc1540b48c0 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204748665 0 0) 0x7fc138003520 con 0x7fc1540b48c0 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 msgr2=0x7fc154161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc154161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 >> v1:172.21.15.80:6789/0 conn(0x7fc154161bc0 legacy=0x7fc1540b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc154160f80 con 0x7fc1540b48c0 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fc1440019f0 con 0x7fc1540b48c0 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152a00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc154161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 >> v1:172.21.15.5:6789/0 conn(0x7fc1540b48c0 legacy=0x7fc15415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc1380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc138004020 0x7fc138004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc138004d40 0x7fc1380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154160ca0 con 0x7fc138004020 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13800bc00 con 0x7fc138004d40 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.944+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13800be00 con 0x7fc1540d3f90 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc152a00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc138004020 0x7fc138004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc153400640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc1380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc153e00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc138004d40 0x7fc1380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc152a00640 1 -- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 msgr2=0x7fc1380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc152a00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc1380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc152a00640 1 -- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc138004d40 msgr2=0x7fc1380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc152a00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc138004d40 0x7fc1380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc152a00640 1 -- 172.21.15.5:0/1607812304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138003520 con 0x7fc138004020 2024-04-25T10:13:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc153e00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc138004d40 0x7fc1380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.945+0000 7fc153400640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc1540d3f90 0x7fc1380039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.946+0000 7fc152a00640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc138004020 0x7fc138004800 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098b0 tx=0x7fc138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cde3d4291996ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c037070 con 0x7fc138004020 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c02ad00 con 0x7fc138004020 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc152000640 1 -- 172.21.15.5:0/1607812304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00f040 con 0x7fc138004020 2024-04-25T10:13:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc159200640 1 -- 172.21.15.5:0/1607812304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc138004020 msgr2=0x7fc138004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc159200640 1 --2- 172.21.15.5:0/1607812304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc138004020 0x7fc138004800 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0098b0 tx=0x7fc138005830 comp rx=0 tx=0).stop 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc153e00640 1 -- 172.21.15.5:0/1607812304 reap_dead start 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc159200640 1 -- 172.21.15.5:0/1607812304 shutdown_connections 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc159200640 1 -- 172.21.15.5:0/1607812304 >> 172.21.15.5:0/1607812304 conn(0x7fc15408af80 msgr2=0x7fc1540d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc159200640 1 -- 172.21.15.5:0/1607812304 shutdown_connections 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.947+0000 7fc159200640 1 -- 172.21.15.5:0/1607812304 wait complete. 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.949+0000 7fc159200640 1 Processor -- start 2024-04-25T10:13:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.949+0000 7fc159200640 1 -- start start 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc159200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc138004020 0x7fc1540b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc159200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1540cb850 0x7fc1541f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc159200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 0x7fc1540cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc159200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154161680 con 0x7fc1540cc590 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc159200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540b4ca0 con 0x7fc1540cb850 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc159200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc154161800 con 0x7fc138004020 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.950+0000 7fc153e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 0x7fc1540cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 0x7fc1540cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55606/0 (socket says 172.21.15.5:55606) 2024-04-25T10:13:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 -- 172.21.15.5:0/3707078077 learned_addr learned my addr 172.21.15.5:0/3707078077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc152a00640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1540cb850 0x7fc1541f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153400640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc138004020 0x7fc1540b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 -- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc138004020 msgr2=0x7fc1540b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc138004020 0x7fc1540b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 -- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1540cb850 msgr2=0x7fc1541f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1540cb850 0x7fc1541f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 -- 172.21.15.5:0/3707078077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1540b67c0 con 0x7fc1540cc590 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.951+0000 7fc153e00640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 0x7fc1540cec90 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc154161fa0 tx=0x7fc14c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.952+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c007030 con 0x7fc1540cc590 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.952+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c0071f0 con 0x7fc1540cc590 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.952+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0075e0 con 0x7fc1540cc590 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.952+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1540b6aa0 con 0x7fc1540cc590 2024-04-25T10:13:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.952+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1540b7060 con 0x7fc1540cc590 2024-04-25T10:13:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.953+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc154068f60 con 0x7fc1540cc590 2024-04-25T10:13:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.954+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc14c01b020 con 0x7fc1540cc590 2024-04-25T10:13:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.955+0000 7fc14b400640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc13006b4d0 0x7fc13006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.955+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6359+0+0 (secure 0 0 0) 0x7fc14c089dd0 con 0x7fc1540cc590 2024-04-25T10:13:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.956+0000 7fc153400640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc13006b4d0 0x7fc13006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.957+0000 7fc153400640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc13006b4d0 0x7fc13006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc144001370 tx=0x7fc144009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:39.959+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc14c0589b0 con 0x7fc1540cc590 2024-04-25T10:13:40.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.188+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc1540d0dd0 con 0x7fc1540cc590 2024-04-25T10:13:40.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.743+0000 7fc14b400640 1 -- 172.21.15.5:0/3707078077 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc14c0582b0 con 0x7fc1540cc590 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc13006b4d0 msgr2=0x7fc13006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc13006b4d0 0x7fc13006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc144001370 tx=0x7fc144009040 comp rx=0 tx=0).stop 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 msgr2=0x7fc1540cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 0x7fc1540cec90 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fc154161fa0 tx=0x7fc14c00d890 comp rx=0 tx=0).stop 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 shutdown_connections 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1540cc590 0x7fc1540cec90 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc13006b4d0 0x7fc13006d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1540cb850 0x7fc1541f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 --2- 172.21.15.5:0/3707078077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc138004020 0x7fc1540b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.747+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 >> 172.21.15.5:0/3707078077 conn(0x7fc15408af80 msgr2=0x7fc15415fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.748+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 shutdown_connections 2024-04-25T10:13:40.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:40.748+0000 7fc159200640 1 -- 172.21.15.5:0/3707078077 wait complete. 2024-04-25T10:13:40.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:13:41.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.014+0000 7fb986c00640 1 Processor -- start 2024-04-25T10:13:41.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.014+0000 7fb986c00640 1 -- start start 2024-04-25T10:13:41.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb986c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98015c870 0x7fb980071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb986c00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb980071540 0x7fb9800cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb986c00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98006e850 con 0x7fb980144e20 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb986c00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb980071970 con 0x7fb980071540 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb986c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98015fdf0 con 0x7fb98015c870 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb985800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb980144e20 0x7fb98015f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39648/0 (socket says 172.21.15.5:39648) 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.016+0000 7fb985800640 1 -- 172.21.15.5:0/3140776291 learned_addr learned my addr 172.21.15.5:0/3140776291 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb986200640 1 --2- 172.21.15.5:0/3140776291 >> v2:172.21.15.80:3300/0 conn(0x7fb980071540 0x7fb9800cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb984e00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98015c870 0x7fb980071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183150356 0 0) 0x7fb98006e850 con 0x7fb980144e20 2024-04-25T10:13:41.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb964003520 con 0x7fb980144e20 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 38011897 0 0) 0x7fb964003520 con 0x7fb980144e20 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98015c870 msgr2=0x7fb980071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98015c870 0x7fb980071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 >> v2:172.21.15.80:3300/0 conn(0x7fb980071540 msgr2=0x7fb9800cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3140776291 >> v2:172.21.15.80:3300/0 conn(0x7fb980071540 0x7fb9800cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9801600d0 con 0x7fb980144e20 2024-04-25T10:13:41.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fb9700019f0 con 0x7fb980144e20 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 >> v1:172.21.15.5:6789/0 conn(0x7fb980144e20 legacy=0x7fb98015f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9640039b0 0x7fb964003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964004800 0x7fb964004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640051c0 0x7fb96400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb980071970 con 0x7fb9640039b0 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98015fdf0 con 0x7fb964004800 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96400c060 con 0x7fb9640051c0 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.017+0000 7fb984e00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98015c870 0x7fb980071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb985800640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9640039b0 0x7fb964003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb984e00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964004800 0x7fb964004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb986200640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640051c0 0x7fb96400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb985800640 1 -- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640051c0 msgr2=0x7fb96400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb985800640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640051c0 0x7fb96400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb985800640 1 -- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964004800 msgr2=0x7fb964004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb985800640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964004800 0x7fb964004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb985800640 1 -- 172.21.15.5:0/3140776291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964003520 con 0x7fb9640039b0 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.018+0000 7fb984e00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964004800 0x7fb964004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.019+0000 7fb985800640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9640039b0 0x7fb964003dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fb97000a880 tx=0x7fb964005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f08627c22da71b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.019+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97000a4a0 con 0x7fb9640039b0 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.019+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97002f070 con 0x7fb9640039b0 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb97fe00640 1 -- 172.21.15.5:0/3140776291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97002bd00 con 0x7fb9640039b0 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986c00640 1 -- 172.21.15.5:0/3140776291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9640039b0 msgr2=0x7fb964003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986c00640 1 --2- 172.21.15.5:0/3140776291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9640039b0 0x7fb964003dd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fb97000a880 tx=0x7fb964005830 comp rx=0 tx=0).stop 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986200640 1 -- 172.21.15.5:0/3140776291 reap_dead start 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986c00640 1 -- 172.21.15.5:0/3140776291 shutdown_connections 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986c00640 1 -- 172.21.15.5:0/3140776291 >> 172.21.15.5:0/3140776291 conn(0x7fb980066f20 msgr2=0x7fb98006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986c00640 1 -- 172.21.15.5:0/3140776291 shutdown_connections 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.020+0000 7fb986c00640 1 -- 172.21.15.5:0/3140776291 wait complete. 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.022+0000 7fb986c00640 1 Processor -- start 2024-04-25T10:13:41.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.022+0000 7fb986c00640 1 -- start start 2024-04-25T10:13:41.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986c00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640039b0 0x7fb98013b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986c00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800cb030 0x7fb9800cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 0x7fb980140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb980145200 con 0x7fb980140760 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986c00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98015c070 con 0x7fb9800cb030 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986c00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb980145380 con 0x7fb9640039b0 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 0x7fb980140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 0x7fb980140c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55626/0 (socket says 172.21.15.5:55626) 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.023+0000 7fb986200640 1 -- 172.21.15.5:0/2654961032 learned_addr learned my addr 172.21.15.5:0/2654961032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb985800640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640039b0 0x7fb98013b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb984e00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800cb030 0x7fb9800cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb986200640 1 -- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640039b0 msgr2=0x7fb98013b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb986200640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640039b0 0x7fb98013b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb986200640 1 -- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800cb030 msgr2=0x7fb9800cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb986200640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800cb030 0x7fb9800cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb986200640 1 -- 172.21.15.5:0/2654961032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800cb9f0 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb986200640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 0x7fb980140c10 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb98006f510 tx=0x7fb978010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800f040 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.024+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97801c070 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.025+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800eb10 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.025+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb980141490 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.025+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98014c210 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.026+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb980141770 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.027+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb97800ecb0 con 0x7fb980140760 2024-04-25T10:13:41.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.028+0000 7fb97d600640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b500 0x7fb96006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.028+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6359+0+0 (secure 0 0 0) 0x7fb978089f80 con 0x7fb980140760 2024-04-25T10:13:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.028+0000 7fb985800640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b500 0x7fb96006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.029+0000 7fb985800640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b500 0x7fb96006d9c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb970033000 tx=0x7fb970007000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.032+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb978026020 con 0x7fb980140760 2024-04-25T10:13:41.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.232+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb980068f60 con 0x7fb980140760 2024-04-25T10:13:41.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.753+0000 7fb97d600640 1 -- 172.21.15.5:0/2654961032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb9800cb9f0 con 0x7fb980140760 2024-04-25T10:13:41.758 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:13:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.758+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b500 msgr2=0x7fb96006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.758+0000 7fb986c00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b500 0x7fb96006d9c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb970033000 tx=0x7fb970007000 comp rx=0 tx=0).stop 2024-04-25T10:13:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.758+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 msgr2=0x7fb980140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.758+0000 7fb986c00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 0x7fb980140c10 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb98006f510 tx=0x7fb978010c20 comp rx=0 tx=0).stop 2024-04-25T10:13:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.759+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 shutdown_connections 2024-04-25T10:13:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.759+0000 7fb986c00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb980140760 0x7fb980140c10 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.759+0000 7fb986c00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fb96006b500 0x7fb96006d9c0 secure :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb970033000 tx=0x7fb970007000 comp rx=0 tx=0).stop 2024-04-25T10:13:41.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.759+0000 7fb986c00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9800cb030 0x7fb9800cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.759+0000 7fb986c00640 1 --2- 172.21.15.5:0/2654961032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9640039b0 0x7fb98013b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.759+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 >> 172.21.15.5:0/2654961032 conn(0x7fb980066f20 msgr2=0x7fb9800b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.760+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 shutdown_connections 2024-04-25T10:13:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.760+0000 7fb986c00640 1 -- 172.21.15.5:0/2654961032 wait complete. 2024-04-25T10:13:41.789 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:13:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.988+0000 7f887aa00640 1 Processor -- start 2024-04-25T10:13:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.988+0000 7f887aa00640 1 -- start start 2024-04-25T10:13:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.990+0000 7f887aa00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8874142010 0x7f887406e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.990+0000 7f887aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 0x7f88740cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.990+0000 7f887aa00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887406d780 con 0x7f88740cce30 2024-04-25T10:13:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.990+0000 7f887aa00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88741423f0 con 0x7f8874142010 2024-04-25T10:13:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.990+0000 7f887aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740cc210 con 0x7f887406ed60 2024-04-25T10:13:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.991+0000 7f886be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 0x7f88740cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.991+0000 7f8873e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8874142010 0x7f887406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.991+0000 7f886be00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 0x7f88740cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:51304/0 (socket says 172.21.15.5:51304) 2024-04-25T10:13:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.991+0000 7f886be00640 1 -- 172.21.15.5:0/2402352262 learned_addr learned my addr 172.21.15.5:0/2402352262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.992+0000 7f8873e00640 1 -- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 msgr2=0x7f88740cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.992+0000 7f8873e00640 1 --2- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 0x7f88740cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.992+0000 7f8873e00640 1 -- 172.21.15.5:0/2402352262 >> v1:172.21.15.80:6789/0 conn(0x7f88740cce30 legacy=0x7f88741426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.992+0000 7f8873e00640 1 -- 172.21.15.5:0/2402352262 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887406da60 con 0x7f8874142010 2024-04-25T10:13:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.992+0000 7f886be00640 1 --2- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 0x7f88740cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.993+0000 7f8873e00640 1 --2- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8874142010 0x7f887406e820 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8874142570 tx=0x7f886402a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf52c8102a5117b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.994+0000 7f8873400640 1 -- 172.21.15.5:0/2402352262 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864037070 con 0x7f8874142010 2024-04-25T10:13:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.994+0000 7f8873400640 1 -- 172.21.15.5:0/2402352262 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886402ae40 con 0x7f8874142010 2024-04-25T10:13:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.994+0000 7f8873400640 1 -- 172.21.15.5:0/2402352262 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864032660 con 0x7f8874142010 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.994+0000 7f887aa00640 1 -- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8874142010 msgr2=0x7f887406e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.994+0000 7f887aa00640 1 --2- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8874142010 0x7f887406e820 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8874142570 tx=0x7f886402a0f0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.995+0000 7f887aa00640 1 -- 172.21.15.5:0/2402352262 shutdown_connections 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.995+0000 7f887aa00640 1 --2- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f887406ed60 0x7f88740cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.995+0000 7f887aa00640 1 --2- 172.21.15.5:0/2402352262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8874142010 0x7f887406e820 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.995+0000 7f887aa00640 1 -- 172.21.15.5:0/2402352262 >> 172.21.15.5:0/2402352262 conn(0x7f8874066f80 msgr2=0x7f887406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.995+0000 7f887aa00640 1 -- 172.21.15.5:0/2402352262 shutdown_connections 2024-04-25T10:13:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.995+0000 7f887aa00640 1 -- 172.21.15.5:0/2402352262 wait complete. 2024-04-25T10:13:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.996+0000 7f887aa00640 1 Processor -- start 2024-04-25T10:13:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.997+0000 7f887aa00640 1 -- start start 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f887aa00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f887406ed60 0x7f88740aa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f887aa00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740cce30 0x7f887414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f887aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 0x7f887413fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f887aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887406e300 con 0x7f8874142010 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f887aa00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887406e020 con 0x7f887406ed60 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f887aa00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8874142db0 con 0x7f88740cce30 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f8878a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 0x7f887413fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f8878a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 0x7f887413fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55634/0 (socket says 172.21.15.5:55634) 2024-04-25T10:13:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f8878a00640 1 -- 172.21.15.5:0/903757644 learned_addr learned my addr 172.21.15.5:0/903757644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f886be00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740cce30 0x7f887414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.998+0000 7f8873e00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f887406ed60 0x7f88740aa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8878a00640 1 -- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740cce30 msgr2=0x7f887414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8878a00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740cce30 0x7f887414f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8878a00640 1 -- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f887406ed60 msgr2=0x7f88740aa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8878a00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f887406ed60 0x7f88740aa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8878a00640 1 -- 172.21.15.5:0/903757644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740a91e0 con 0x7f8874142010 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f886be00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740cce30 0x7f887414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8878a00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 0x7f887413fbf0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c7a0 tx=0x7f886c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c003b70 con 0x7f8874142010 2024-04-25T10:13:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c003d30 con 0x7f8874142010 2024-04-25T10:13:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00aa90 con 0x7f8874142010 2024-04-25T10:13:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:41.999+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88740a94c0 con 0x7f8874142010 2024-04-25T10:13:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.000+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88740a9a80 con 0x7f8874142010 2024-04-25T10:13:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.001+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88740a9f70 con 0x7f8874142010 2024-04-25T10:13:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.002+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f886c00ac30 con 0x7f8874142010 2024-04-25T10:13:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.003+0000 7f8870c00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f885006b4d0 0x7f885006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.003+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5921+0+0 (secure 0 0 0) 0x7f886c08a350 con 0x7f8874142010 2024-04-25T10:13:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.004+0000 7f8873e00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f885006b4d0 0x7f885006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.005+0000 7f8873e00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f885006b4d0 0x7f885006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8864034000 tx=0x7f886402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:42.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.008+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f886c058fc0 con 0x7f8874142010 2024-04-25T10:13:42.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.211+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f88740b04e0 con 0x7f8874142010 2024-04-25T10:13:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.757+0000 7f8870c00640 1 -- 172.21.15.5:0/903757644 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) v1 ==== 169+0+0 (secure 0 0 0) 0x7f886c0588c0 con 0x7f8874142010 2024-04-25T10:13:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-04-25T10:13:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f885006b4d0 msgr2=0x7f885006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f885006b4d0 0x7f885006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8864034000 tx=0x7f886402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 msgr2=0x7f887413fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 0x7f887413fbf0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c7a0 tx=0x7f886c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 shutdown_connections 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8874142010 0x7f887413fbf0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740cce30 0x7f887414f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f885006b4d0 0x7f885006d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 --2- 172.21.15.5:0/903757644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f887406ed60 0x7f88740aa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.760+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 >> 172.21.15.5:0/903757644 conn(0x7f8874066f80 msgr2=0x7f88740ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.761+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 shutdown_connections 2024-04-25T10:13:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.762+0000 7f887aa00640 1 -- 172.21.15.5:0/903757644 wait complete. 2024-04-25T10:13:42.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:13:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.994+0000 7ffa2f200640 1 Processor -- start 2024-04-25T10:13:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.994+0000 7ffa2f200640 1 -- start start 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa2f200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa301619b0 0x7ffa30161d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa2f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa300b33f0 con 0x7ffa300b37d0 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa2f200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa300b27e0 con 0x7ffa30071000 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa2f200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300b2ac0 con 0x7ffa301619b0 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa25400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffa300b37d0 0x7ffa30162ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39650/0 (socket says 172.21.15.5:39650) 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa25400640 1 -- 172.21.15.5:0/3899076982 learned_addr learned my addr 172.21.15.5:0/3899076982 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.996+0000 7ffa2de00640 1 --2- 172.21.15.5:0/3899076982 >> v2:172.21.15.5:3300/0 conn(0x7ffa301619b0 0x7ffa30161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.997+0000 7ffa2d400640 1 -- 172.21.15.5:0/3899076982 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223718218 0 0) 0x7ffa300b33f0 con 0x7ffa300b37d0 2024-04-25T10:13:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.997+0000 7ffa2de00640 1 -- 172.21.15.5:0/3899076982 >> v1:172.21.15.80:6789/0 conn(0x7ffa30071000 legacy=0x7ffa3006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.997+0000 7ffa2de00640 1 -- 172.21.15.5:0/3899076982 >> v1:172.21.15.5:6789/0 conn(0x7ffa300b37d0 legacy=0x7ffa30162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.997+0000 7ffa2de00640 1 -- 172.21.15.5:0/3899076982 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300b2da0 con 0x7ffa301619b0 2024-04-25T10:13:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.998+0000 7ffa2de00640 1 --2- 172.21.15.5:0/3899076982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa301619b0 0x7ffa30161d90 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7ffa200097a0 tx=0x7ffa2002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f91d3e19f96027e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.998+0000 7ffa2d400640 1 -- 172.21.15.5:0/3899076982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa20037070 con 0x7ffa301619b0 2024-04-25T10:13:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.998+0000 7ffa2d400640 1 -- 172.21.15.5:0/3899076982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa20032440 con 0x7ffa301619b0 2024-04-25T10:13:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.998+0000 7ffa2d400640 1 -- 172.21.15.5:0/3899076982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa20032810 con 0x7ffa301619b0 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 -- 172.21.15.5:0/3899076982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa301619b0 msgr2=0x7ffa30161d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 --2- 172.21.15.5:0/3899076982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa301619b0 0x7ffa30161d90 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7ffa200097a0 tx=0x7ffa2002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 -- 172.21.15.5:0/3899076982 shutdown_connections 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 --2- 172.21.15.5:0/3899076982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa301619b0 0x7ffa30161d90 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 -- 172.21.15.5:0/3899076982 >> 172.21.15.5:0/3899076982 conn(0x7ffa30066f20 msgr2=0x7ffa3006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 -- 172.21.15.5:0/3899076982 shutdown_connections 2024-04-25T10:13:43.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:42.999+0000 7ffa2f200640 1 -- 172.21.15.5:0/3899076982 wait complete. 2024-04-25T10:13:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.001+0000 7ffa2f200640 1 Processor -- start 2024-04-25T10:13:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.001+0000 7ffa2f200640 1 -- start start 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa2f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa30071000 0x7ffa30153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa2f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 0x7ffa3015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa2f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa301619b0 0x7ffa3015e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa2f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300b2420 con 0x7ffa300b37d0 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa2f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3006e730 con 0x7ffa30071000 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa2f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300b25a0 con 0x7ffa301619b0 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa25400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 0x7ffa3015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa25400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 0x7ffa3015f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55658/0 (socket says 172.21.15.5:55658) 2024-04-25T10:13:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.002+0000 7ffa25400640 1 -- 172.21.15.5:0/4205010322 learned_addr learned my addr 172.21.15.5:0/4205010322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa2de00640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa30071000 0x7ffa30153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa25400640 1 -- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa301619b0 msgr2=0x7ffa3015e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa2e800640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa301619b0 0x7ffa3015e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa25400640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa301619b0 0x7ffa3015e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa25400640 1 -- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa30071000 msgr2=0x7ffa30153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa25400640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa30071000 0x7ffa30153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa25400640 1 -- 172.21.15.5:0/4205010322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa30154040 con 0x7ffa300b37d0 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa2de00640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa30071000 0x7ffa30153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa25400640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 0x7ffa3015f9e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c4d0 tx=0x7ffa1800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18018070 con 0x7ffa300b37d0 2024-04-25T10:13:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa18007500 con 0x7ffa300b37d0 2024-04-25T10:13:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004a20 con 0x7ffa300b37d0 2024-04-25T10:13:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa30154320 con 0x7ffa300b37d0 2024-04-25T10:13:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.003+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa30156a60 con 0x7ffa300b37d0 2024-04-25T10:13:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.005+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa30068f60 con 0x7ffa300b37d0 2024-04-25T10:13:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.006+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa1801b070 con 0x7ffa300b37d0 2024-04-25T10:13:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.007+0000 7ffa26a00640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa0c06b4d0 0x7ffa0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.007+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ffa18089490 con 0x7ffa300b37d0 2024-04-25T10:13:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.007+0000 7ffa2de00640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa0c06b4d0 0x7ffa0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.009+0000 7ffa2de00640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa0c06b4d0 0x7ffa0c06d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ffa20034000 tx=0x7ffa2002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.011+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa180583c0 con 0x7ffa300b37d0 2024-04-25T10:13:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.224+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ffa30156c30 con 0x7ffa300b37d0 2024-04-25T10:13:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.231+0000 7ffa26a00640 1 -- 172.21.15.5:0/4205010322 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffa18057cc0 con 0x7ffa300b37d0 2024-04-25T10:13:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa0c06b4d0 msgr2=0x7ffa0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa0c06b4d0 0x7ffa0c06d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ffa20034000 tx=0x7ffa2002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 msgr2=0x7ffa3015f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 0x7ffa3015f9e0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c4d0 tx=0x7ffa1800c990 comp rx=0 tx=0).stop 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 shutdown_connections 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ffa301619b0 0x7ffa3015e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa300b37d0 0x7ffa3015f9e0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ffa0c06b4d0 0x7ffa0c06d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 --2- 172.21.15.5:0/4205010322 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffa30071000 0x7ffa30153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 >> 172.21.15.5:0/4205010322 conn(0x7ffa30066f20 msgr2=0x7ffa3006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 shutdown_connections 2024-04-25T10:13:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.234+0000 7ffa2f200640 1 -- 172.21.15.5:0/4205010322 wait complete. 2024-04-25T10:13:43.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:13:43.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.456+0000 7fc309a00640 1 Processor -- start 2024-04-25T10:13:43.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.456+0000 7fc309a00640 1 -- start start 2024-04-25T10:13:43.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.458+0000 7fc309a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.458+0000 7fc309a00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc3040cdca0 0x7fc3040b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.458+0000 7fc309a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040b4890 con 0x7fc304070f90 2024-04-25T10:13:43.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.458+0000 7fc309a00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040cd200 con 0x7fc3040cdca0 2024-04-25T10:13:43.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.458+0000 7fc309a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406e6f0 con 0x7fc304154a10 2024-04-25T10:13:43.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.459+0000 7fc2fb400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc3040cdca0 0x7fc3040b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.459+0000 7fc303e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.459+0000 7fc303e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:51328/0 (socket says 172.21.15.5:51328) 2024-04-25T10:13:43.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.459+0000 7fc303e00640 1 -- 172.21.15.5:0/2618898823 learned_addr learned my addr 172.21.15.5:0/2618898823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.460+0000 7fc303e00640 1 -- 172.21.15.5:0/2618898823 >> v2:172.21.15.80:3300/0 conn(0x7fc3040cdca0 msgr2=0x7fc3040b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.460+0000 7fc303e00640 1 --2- 172.21.15.5:0/2618898823 >> v2:172.21.15.80:3300/0 conn(0x7fc3040cdca0 0x7fc3040b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.460+0000 7fc303e00640 1 -- 172.21.15.5:0/2618898823 >> v1:172.21.15.80:6789/0 conn(0x7fc304070f90 legacy=0x7fc30406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.460+0000 7fc303e00640 1 -- 172.21.15.5:0/2618898823 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30406e9d0 con 0x7fc304154a10 2024-04-25T10:13:43.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.460+0000 7fc2fb400640 1 --2- 172.21.15.5:0/2618898823 >> v2:172.21.15.80:3300/0 conn(0x7fc3040cdca0 0x7fc3040b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:43.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.462+0000 7fc303e00640 1 --2- 172.21.15.5:0/2618898823 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415e8c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009870 tx=0x7fc2f402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884b43b7e96b1289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.462+0000 7fc303400640 1 -- 172.21.15.5:0/2618898823 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4037070 con 0x7fc304154a10 2024-04-25T10:13:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.462+0000 7fc303400640 1 -- 172.21.15.5:0/2618898823 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4032440 con 0x7fc304154a10 2024-04-25T10:13:43.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.462+0000 7fc303400640 1 -- 172.21.15.5:0/2618898823 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4032810 con 0x7fc304154a10 2024-04-25T10:13:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 -- 172.21.15.5:0/2618898823 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 msgr2=0x7fc30415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 --2- 172.21.15.5:0/2618898823 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415e8c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009870 tx=0x7fc2f402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 -- 172.21.15.5:0/2618898823 shutdown_connections 2024-04-25T10:13:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 --2- 172.21.15.5:0/2618898823 >> v2:172.21.15.80:3300/0 conn(0x7fc3040cdca0 0x7fc3040b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 --2- 172.21.15.5:0/2618898823 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415e8c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 -- 172.21.15.5:0/2618898823 >> 172.21.15.5:0/2618898823 conn(0x7fc304066f20 msgr2=0x7fc30406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 -- 172.21.15.5:0/2618898823 shutdown_connections 2024-04-25T10:13:43.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.463+0000 7fc309a00640 1 -- 172.21.15.5:0/2618898823 wait complete. 2024-04-25T10:13:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.465+0000 7fc309a00640 1 Processor -- start 2024-04-25T10:13:43.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.465+0000 7fc309a00640 1 -- start start 2024-04-25T10:13:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.466+0000 7fc309a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 0x7fc304160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.466+0000 7fc309a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3040cdca0 0x7fc304155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.466+0000 7fc309a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.466+0000 7fc309a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040cd780 con 0x7fc304070f90 2024-04-25T10:13:43.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.466+0000 7fc309a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040cd4a0 con 0x7fc3040cdca0 2024-04-25T10:13:43.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.466+0000 7fc309a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406d700 con 0x7fc304154a10 2024-04-25T10:13:43.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 0x7fc304160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 0x7fc304160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55670/0 (socket says 172.21.15.5:55670) 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 -- 172.21.15.5:0/3468247216 learned_addr learned my addr 172.21.15.5:0/3468247216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc309000640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc2fb400640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3040cdca0 0x7fc304155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 -- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 msgr2=0x7fc30415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 -- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3040cdca0 msgr2=0x7fc304155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3040cdca0 0x7fc304155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.467+0000 7fc303e00640 1 -- 172.21.15.5:0/3468247216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30413ea00 con 0x7fc304070f90 2024-04-25T10:13:43.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.468+0000 7fc303e00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 0x7fc304160930 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000c00 tx=0x7fc2f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.468+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4037040 con 0x7fc304070f90 2024-04-25T10:13:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.468+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f4042070 con 0x7fc304070f90 2024-04-25T10:13:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.468+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4030940 con 0x7fc304070f90 2024-04-25T10:13:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.468+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30413ece0 con 0x7fc304070f90 2024-04-25T10:13:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.468+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30413f2a0 con 0x7fc304070f90 2024-04-25T10:13:43.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.469+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3040cce40 con 0x7fc304070f90 2024-04-25T10:13:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.470+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2f402ad10 con 0x7fc304070f90 2024-04-25T10:13:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.471+0000 7fc300c00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2e006b4d0 0x7fc2e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.472+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc2f40aa5c0 con 0x7fc304070f90 2024-04-25T10:13:43.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.472+0000 7fc2fb400640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2e006b4d0 0x7fc2e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.473+0000 7fc2fb400640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2e006b4d0 0x7fc2e006d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009560 tx=0x7fc2ec008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.476+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2f4079500 con 0x7fc304070f90 2024-04-25T10:13:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.745+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fc30406aea0 con 0x7fc304070f90 2024-04-25T10:13:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.769+0000 7fc300c00640 1 -- 172.21.15.5:0/3468247216 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v578) v1 ==== 144+0+0 (secure 0 0 0) 0x7fc2f4078e00 con 0x7fc304070f90 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.771+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2e006b4d0 msgr2=0x7fc2e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.771+0000 7fc309a00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2e006b4d0 0x7fc2e006d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009560 tx=0x7fc2ec008040 comp rx=0 tx=0).stop 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.771+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 msgr2=0x7fc304160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.771+0000 7fc309a00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 0x7fc304160930 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000c00 tx=0x7fc2f40046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 shutdown_connections 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc304154a10 0x7fc30415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fc2e006b4d0 0x7fc2e006d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3040cdca0 0x7fc304155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 --2- 172.21.15.5:0/3468247216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc304070f90 0x7fc304160930 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 >> 172.21.15.5:0/3468247216 conn(0x7fc304066f20 msgr2=0x7fc3040b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 shutdown_connections 2024-04-25T10:13:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:43.772+0000 7fc309a00640 1 -- 172.21.15.5:0/3468247216 wait complete. 2024-04-25T10:13:43.797 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:13:43.798 INFO:tasks.cephfs_test_runner: 2024-04-25T10:13:43.798 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:13:43.798 INFO:tasks.cephfs_test_runner:Ran 7 tests in 642.601s 2024-04-25T10:13:43.798 INFO:tasks.cephfs_test_runner: 2024-04-25T10:13:43.798 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:13:43.799 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:13:43.808 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:13:43.851 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:13:43.851 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:43.851 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:43.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:43.910 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-04-25T10:13:43.910 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:43.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:43.964 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:13:43.964 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:44.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:44.022 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T10:13:44.022 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:44.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.077 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:13:44.138 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:44.138 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:44.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:44.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:44.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-25T10:13:44.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.413 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-04-25T10:13:44.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-04-25T10:13:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-04-25T10:13:44.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:44.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.620 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T10:13:44.678 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:13:44.678 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:13:44.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.679 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:13:44.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.749 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:13:44.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.817 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-25T10:13:44.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:44.956 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T10:13:44.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:45.000 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-25T10:13:45.120 INFO:teuthology.orchestra.run.smithi080.stderr:Cannot find device "ceph-brx" 2024-04-25T10:13:45.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:45.123 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:13:45.133 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:13:45.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.345+0000 7fe912200640 1 Processor -- start 2024-04-25T10:13:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.345+0000 7fe912200640 1 -- start start 2024-04-25T10:13:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.347+0000 7fe912200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c1652f0 0x7fe90c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.347+0000 7fe912200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe90c06e980 0x7fe90c070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.347+0000 7fe912200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe90c0714d0 0x7fe90c165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.347+0000 7fe912200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c0ce340 con 0x7fe90c0714d0 2024-04-25T10:13:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.347+0000 7fe912200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c0cdeb0 con 0x7fe90c06e980 2024-04-25T10:13:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.347+0000 7fe912200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0cd2a0 con 0x7fe90c1652f0 2024-04-25T10:13:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe90c0714d0 0x7fe90c165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe90c0714d0 0x7fe90c165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55674/0 (socket says 172.21.15.5:55674) 2024-04-25T10:13:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 -- 172.21.15.5:0/2834657093 learned_addr learned my addr 172.21.15.5:0/2834657093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe910e00640 1 --2- 172.21.15.5:0/2834657093 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c1652f0 0x7fe90c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe90be00640 1 --2- 172.21.15.5:0/2834657093 >> v2:172.21.15.80:3300/0 conn(0x7fe90c06e980 0x7fe90c070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 -- 172.21.15.5:0/2834657093 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c1652f0 msgr2=0x7fe90c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 --2- 172.21.15.5:0/2834657093 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c1652f0 0x7fe90c0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 -- 172.21.15.5:0/2834657093 >> v2:172.21.15.80:3300/0 conn(0x7fe90c06e980 msgr2=0x7fe90c070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 --2- 172.21.15.5:0/2834657093 >> v2:172.21.15.80:3300/0 conn(0x7fe90c06e980 0x7fe90c070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe911800640 1 -- 172.21.15.5:0/2834657093 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0cd580 con 0x7fe90c0714d0 2024-04-25T10:13:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.348+0000 7fe90be00640 1 --2- 172.21.15.5:0/2834657093 >> v2:172.21.15.80:3300/0 conn(0x7fe90c06e980 0x7fe90c070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.349+0000 7fe911800640 1 --2- 172.21.15.5:0/2834657093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c0714d0 0x7fe90c165850 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fe90c06f160 tx=0x7fe90402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc63bce9fe89915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe90b400640 1 -- 172.21.15.5:0/2834657093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904036070 con 0x7fe90c0714d0 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe90b400640 1 -- 172.21.15.5:0/2834657093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe904031600 con 0x7fe90c0714d0 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe90b400640 1 -- 172.21.15.5:0/2834657093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040319d0 con 0x7fe90c0714d0 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 -- 172.21.15.5:0/2834657093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c0714d0 msgr2=0x7fe90c165850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 --2- 172.21.15.5:0/2834657093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c0714d0 0x7fe90c165850 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fe90c06f160 tx=0x7fe90402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 -- 172.21.15.5:0/2834657093 shutdown_connections 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 --2- 172.21.15.5:0/2834657093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c0714d0 0x7fe90c165850 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 --2- 172.21.15.5:0/2834657093 >> v2:172.21.15.80:3300/0 conn(0x7fe90c06e980 0x7fe90c070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 --2- 172.21.15.5:0/2834657093 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c1652f0 0x7fe90c0cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 -- 172.21.15.5:0/2834657093 >> 172.21.15.5:0/2834657093 conn(0x7fe90c066f20 msgr2=0x7fe90c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 -- 172.21.15.5:0/2834657093 shutdown_connections 2024-04-25T10:13:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.350+0000 7fe912200640 1 -- 172.21.15.5:0/2834657093 wait complete. 2024-04-25T10:13:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.352+0000 7fe912200640 1 Processor -- start 2024-04-25T10:13:45.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.352+0000 7fe912200640 1 -- start start 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.353+0000 7fe912200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c06e980 0x7fe90c1ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.353+0000 7fe912200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0714d0 0x7fe90c15e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.353+0000 7fe912200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 0x7fe90c15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.353+0000 7fe912200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0cc8d0 con 0x7fe90c1652f0 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.353+0000 7fe912200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0cdb40 con 0x7fe90c06e980 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.353+0000 7fe912200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c165d90 con 0x7fe90c0714d0 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 0x7fe90c15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 0x7fe90c15ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55686/0 (socket says 172.21.15.5:55686) 2024-04-25T10:13:45.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 -- 172.21.15.5:0/3725071178 learned_addr learned my addr 172.21.15.5:0/3725071178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe90be00640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0714d0 0x7fe90c15e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe910e00640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c06e980 0x7fe90c1ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 -- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0714d0 msgr2=0x7fe90c15e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0714d0 0x7fe90c15e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 -- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c06e980 msgr2=0x7fe90c1ed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c06e980 0x7fe90c1ed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe911800640 1 -- 172.21.15.5:0/3725071178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0ad490 con 0x7fe90c1652f0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.354+0000 7fe910e00640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c06e980 0x7fe90c1ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.355+0000 7fe911800640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 0x7fe90c15ee80 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fe904009f40 tx=0x7fe904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.355+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904041020 con 0x7fe90c1652f0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.355+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe904007780 con 0x7fe90c1652f0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.355+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904036040 con 0x7fe90c1652f0 2024-04-25T10:13:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.355+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c0aced0 con 0x7fe90c1652f0 2024-04-25T10:13:45.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.355+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c0a84f0 con 0x7fe90c1652f0 2024-04-25T10:13:45.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.356+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c0cce40 con 0x7fe90c1652f0 2024-04-25T10:13:45.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.357+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe904044020 con 0x7fe90c1652f0 2024-04-25T10:13:45.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.358+0000 7fe908c00640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8f006b4d0 0x7fe8f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.358+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe9040aae10 con 0x7fe90c1652f0 2024-04-25T10:13:45.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.359+0000 7fe910e00640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8f006b4d0 0x7fe8f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.360+0000 7fe910e00640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8f006b4d0 0x7fe8f006d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe900006fd0 tx=0x7fe900008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.364+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9040ae030 con 0x7fe90c1652f0 2024-04-25T10:13:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.563+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe90c166670 con 0x7fe8f006b4d0 2024-04-25T10:13:45.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.566+0000 7fe908c00640 1 -- 172.21.15.5:0/3725071178 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13521 (secure 0 0 0) 0x7fe90c166670 con 0x7fe8f006b4d0 2024-04-25T10:13:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.568+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8f006b4d0 msgr2=0x7fe8f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.568+0000 7fe912200640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8f006b4d0 0x7fe8f006d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe900006fd0 tx=0x7fe900008040 comp rx=0 tx=0).stop 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.568+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 msgr2=0x7fe90c15ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.568+0000 7fe912200640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 0x7fe90c15ee80 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fe904009f40 tx=0x7fe904004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 shutdown_connections 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe90c1652f0 0x7fe90c15ee80 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe90c0714d0 0x7fe90c15e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe8f006b4d0 0x7fe8f006d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 --2- 172.21.15.5:0/3725071178 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe90c06e980 0x7fe90c1ed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 >> 172.21.15.5:0/3725071178 conn(0x7fe90c066f20 msgr2=0x7fe90c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 shutdown_connections 2024-04-25T10:13:45.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.569+0000 7fe912200640 1 -- 172.21.15.5:0/3725071178 wait complete. 2024-04-25T10:13:45.593 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-04-25T10:13:44.880181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":274852,"kb_used_data":25416,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754699868,"statfs":{"total":773094113280,"available":772812664832,"internally_reserved":0,"allocated":26025984,"data_stored":19465120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390542,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-174,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002211"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T10:13:40.349569+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:13:40.349569+0000","last_peered":"2024-04-25T10:13:40.349569+0000","last_clean":"2024-04-25T10:13:40.349569+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:13:40.349569+0000","last_undegraded":"2024-04-25T10:13:40.349569+0000","last_fullsized":"2024-04-25T10:13:40.349569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1527808,"data_stored":713522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":713547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2117632,"data_stored":1300069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":9672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":9904128,"data_stored":9095568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1531904,"data_stored":710288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509585,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2117632,"data_stored":1297403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4908512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:45.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.804+0000 7f3a69e00640 1 Processor -- start 2024-04-25T10:13:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.804+0000 7f3a69e00640 1 -- start start 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a69e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a6415d8a0 0x7f3a640d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a69e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a640b2f30 0x7f3a640d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a69e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a641614e0 0x7f3a64161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a69e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6415ffe0 con 0x7f3a6415d8a0 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a69e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6415f410 con 0x7f3a640b2f30 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a69e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6415f6b0 con 0x7f3a641614e0 2024-04-25T10:13:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a68a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a6415d8a0 0x7f3a640d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a68a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a6415d8a0 0x7f3a640d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55702/0 (socket says 172.21.15.5:55702) 2024-04-25T10:13:45.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.805+0000 7f3a68a00640 1 -- 172.21.15.5:0/175281648 learned_addr learned my addr 172.21.15.5:0/175281648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a68a00640 1 -- 172.21.15.5:0/175281648 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a641614e0 msgr2=0x7f3a64161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a63e00640 1 --2- 172.21.15.5:0/175281648 >> v2:172.21.15.80:3300/0 conn(0x7f3a640b2f30 0x7f3a640d3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a69400640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a641614e0 0x7f3a64161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a68a00640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a641614e0 0x7f3a64161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a68a00640 1 -- 172.21.15.5:0/175281648 >> v2:172.21.15.80:3300/0 conn(0x7f3a640b2f30 msgr2=0x7f3a640d3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a68a00640 1 --2- 172.21.15.5:0/175281648 >> v2:172.21.15.80:3300/0 conn(0x7f3a640b2f30 0x7f3a640d3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a68a00640 1 -- 172.21.15.5:0/175281648 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6415f990 con 0x7f3a6415d8a0 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a63e00640 1 --2- 172.21.15.5:0/175281648 >> v2:172.21.15.80:3300/0 conn(0x7f3a640b2f30 0x7f3a640d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.806+0000 7f3a69400640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a641614e0 0x7f3a64161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.807+0000 7f3a68a00640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a640d2570 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3a640d0c60 tx=0x7f3a5402a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c78ef19dd59269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.807+0000 7f3a63400640 1 -- 172.21.15.5:0/175281648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5402ade0 con 0x7f3a6415d8a0 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.807+0000 7f3a63400640 1 -- 172.21.15.5:0/175281648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a54004030 con 0x7f3a6415d8a0 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a63400640 1 -- 172.21.15.5:0/175281648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54004460 con 0x7f3a6415d8a0 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 -- 172.21.15.5:0/175281648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 msgr2=0x7f3a640d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a640d2570 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f3a640d0c60 tx=0x7f3a5402a090 comp rx=0 tx=0).stop 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 -- 172.21.15.5:0/175281648 shutdown_connections 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a641614e0 0x7f3a64161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 --2- 172.21.15.5:0/175281648 >> v2:172.21.15.80:3300/0 conn(0x7f3a640b2f30 0x7f3a640d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 --2- 172.21.15.5:0/175281648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a640d2570 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 -- 172.21.15.5:0/175281648 >> 172.21.15.5:0/175281648 conn(0x7f3a6408af80 msgr2=0x7f3a640d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 -- 172.21.15.5:0/175281648 shutdown_connections 2024-04-25T10:13:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.808+0000 7f3a69e00640 1 -- 172.21.15.5:0/175281648 wait complete. 2024-04-25T10:13:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.810+0000 7f3a69e00640 1 Processor -- start 2024-04-25T10:13:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.810+0000 7f3a69e00640 1 -- start start 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a69e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a640b2f30 0x7f3a640c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a69e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a64151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a69e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a641614e0 0x7f3a640b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a69e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b2150 con 0x7f3a6415d8a0 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a69e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640d4460 con 0x7f3a641614e0 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a69e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a640b22d0 con 0x7f3a640b2f30 2024-04-25T10:13:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.811+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a64151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a64151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55710/0 (socket says 172.21.15.5:55710) 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 -- 172.21.15.5:0/668873523 learned_addr learned my addr 172.21.15.5:0/668873523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a68a00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a640b2f30 0x7f3a640c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 -- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a640b2f30 msgr2=0x7f3a640c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a640b2f30 0x7f3a640c72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 -- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a641614e0 msgr2=0x7f3a640b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a641614e0 0x7f3a640b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 -- 172.21.15.5:0/668873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64151f50 con 0x7f3a6415d8a0 2024-04-25T10:13:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a63e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a64151a10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00cf60 tx=0x7f3a4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.812+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c01a070 con 0x7f3a6415d8a0 2024-04-25T10:13:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.813+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4c00d840 con 0x7f3a6415d8a0 2024-04-25T10:13:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.813+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00dc10 con 0x7f3a6415d8a0 2024-04-25T10:13:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.813+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a640c7910 con 0x7f3a6415d8a0 2024-04-25T10:13:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.813+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a640ca920 con 0x7f3a6415d8a0 2024-04-25T10:13:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.814+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a64068f60 con 0x7f3a6415d8a0 2024-04-25T10:13:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.815+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a4c01d070 con 0x7f3a6415d8a0 2024-04-25T10:13:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.816+0000 7f3a60c00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a4006b4d0 0x7f3a4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.816+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3a4c08a9c0 con 0x7f3a6415d8a0 2024-04-25T10:13:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.817+0000 7f3a68a00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a4006b4d0 0x7f3a4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.818+0000 7f3a68a00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a4006b4d0 0x7f3a4006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3a54033000 tx=0x7f3a5402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:45.820+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a4c059210 con 0x7f3a6415d8a0 2024-04-25T10:13:46.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.017+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a640b9580 con 0x7f3a4006b4d0 2024-04-25T10:13:46.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.020+0000 7f3a60c00640 1 -- 172.21.15.5:0/668873523 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13521 (secure 0 0 0) 0x7f3a640b9580 con 0x7f3a4006b4d0 2024-04-25T10:13:46.025 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:46.025 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a4006b4d0 msgr2=0x7f3a4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a4006b4d0 0x7f3a4006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3a54033000 tx=0x7f3a5402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 msgr2=0x7f3a64151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a64151a10 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00cf60 tx=0x7f3a4c00cf90 comp rx=0 tx=0).stop 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 shutdown_connections 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f3a4006b4d0 0x7f3a4006d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a641614e0 0x7f3a640b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6415d8a0 0x7f3a64151a10 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 --2- 172.21.15.5:0/668873523 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a640b2f30 0x7f3a640c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.022+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 >> 172.21.15.5:0/668873523 conn(0x7f3a6408af80 msgr2=0x7f3a6415cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.023+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 shutdown_connections 2024-04-25T10:13:46.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.023+0000 7f3a69e00640 1 -- 172.21.15.5:0/668873523 wait complete. 2024-04-25T10:13:46.046 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-04-25T10:13:44.880181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":274852,"kb_used_data":25416,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754699868,"statfs":{"total":773094113280,"available":772812664832,"internally_reserved":0,"allocated":26025984,"data_stored":19465120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390542,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-174,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002211"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T10:13:40.349569+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:13:40.349569+0000","last_peered":"2024-04-25T10:13:40.349569+0000","last_clean":"2024-04-25T10:13:40.349569+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:13:40.349569+0000","last_undegraded":"2024-04-25T10:13:40.349569+0000","last_fullsized":"2024-04-25T10:13:40.349569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1527808,"data_stored":713522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":713547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2117632,"data_stored":1300069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":9672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":9904128,"data_stored":9095568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1531904,"data_stored":710288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509585,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2117632,"data_stored":1297403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4908512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:46.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:13:46.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:46.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.254+0000 7fcbcca00640 1 Processor -- start 2024-04-25T10:13:46.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.255+0000 7fcbcca00640 1 -- start start 2024-04-25T10:13:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.256+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 0x7fcbc80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.256+0000 7fcbcca00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80cf410 con 0x7fcbc80d2380 2024-04-25T10:13:46.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbcca00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80cf6f0 con 0x7fcbc80d07d0 2024-04-25T10:13:46.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbcca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80d27c0 con 0x7fcbc8161200 2024-04-25T10:13:46.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbc6a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcbc80d2380 0x7fcbc80d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39658/0 (socket says 172.21.15.5:39658) 2024-04-25T10:13:46.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbc6a00640 1 -- 172.21.15.5:0/541438907 learned_addr learned my addr 172.21.15.5:0/541438907 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbc7400640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 0x7fcbc80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653416366 0 0) 0x7fcbc80cf410 con 0x7fcbc80d2380 2024-04-25T10:13:46.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.257+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba8003520 con 0x7fcbc80d2380 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3742115351 0 0) 0x7fcba8003520 con 0x7fcbc80d2380 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 msgr2=0x7fcbc80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 0x7fcbc80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 >> v1:172.21.15.80:6789/0 conn(0x7fcbc80d07d0 legacy=0x7fcbc80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80d3fd0 con 0x7fcbc80d2380 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fcbb00019f0 con 0x7fcbc80d2380 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 >> v1:172.21.15.5:6789/0 conn(0x7fcbc80d2380 legacy=0x7fcbc80d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 0x7fcba80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8003fb0 0x7fcba8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80d27c0 con 0x7fcba8004d40 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba800bb00 con 0x7fcba8003fb0 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba800bd00 con 0x7fcbc8161200 2024-04-25T10:13:46.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.258+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc7400640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 0x7fcba80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc6a00640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8003fb0 0x7fcba8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc7e00640 1 -- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 msgr2=0x7fcba80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc8161200 0x7fcba80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc7e00640 1 -- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8003fb0 msgr2=0x7fcba8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8003fb0 0x7fcba8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.259+0000 7fcbc7e00640 1 -- 172.21.15.5:0/541438907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8003520 con 0x7fcba8004d40 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.260+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0001750 tx=0x7fcba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf2592aa1b29d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.260+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0001370 con 0x7fcba8004d40 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.260+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc00052d0 con 0x7fcba8004d40 2024-04-25T10:13:46.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.260+0000 7fcbc6000640 1 -- 172.21.15.5:0/541438907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc002abe0 con 0x7fcba8004d40 2024-04-25T10:13:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.261+0000 7fcbcca00640 1 -- 172.21.15.5:0/541438907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 msgr2=0x7fcba80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.261+0000 7fcbcca00640 1 --2- 172.21.15.5:0/541438907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcba8004d40 0x7fcba80051c0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0001750 tx=0x7fcba8005830 comp rx=0 tx=0).stop 2024-04-25T10:13:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.261+0000 7fcbc7e00640 1 -- 172.21.15.5:0/541438907 reap_dead start 2024-04-25T10:13:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.261+0000 7fcbcca00640 1 -- 172.21.15.5:0/541438907 shutdown_connections 2024-04-25T10:13:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.261+0000 7fcbcca00640 1 -- 172.21.15.5:0/541438907 >> 172.21.15.5:0/541438907 conn(0x7fcbc808af80 msgr2=0x7fcbc80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.261+0000 7fcbcca00640 1 -- 172.21.15.5:0/541438907 shutdown_connections 2024-04-25T10:13:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.262+0000 7fcbcca00640 1 -- 172.21.15.5:0/541438907 wait complete. 2024-04-25T10:13:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.263+0000 7fcbcca00640 1 Processor -- start 2024-04-25T10:13:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.264+0000 7fcbcca00640 1 -- start start 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.264+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8004d40 0x7fcbc815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc80b8e10 0x7fcbc81f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbcca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 0x7fcbc815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbcca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b4bb0 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbcca00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b28b0 con 0x7fcba8004d40 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbcca00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80b4d30 con 0x7fcbc80b8e10 2024-04-25T10:13:46.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 0x7fcbc815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 0x7fcbc815eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55720/0 (socket says 172.21.15.5:55720) 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 -- 172.21.15.5:0/4187147393 learned_addr learned my addr 172.21.15.5:0/4187147393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 -- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc80b8e10 msgr2=0x7fcbc81f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc80b8e10 0x7fcbc81f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 -- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8004d40 msgr2=0x7fcbc815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8004d40 0x7fcbc815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.265+0000 7fcbc7e00640 1 -- 172.21.15.5:0/4187147393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc81f3f50 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.266+0000 7fcbc7e00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 0x7fcbc815eff0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0005040 tx=0x7fcbc0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.266+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0037040 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.266+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc0005480 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.266+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0007a30 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.266+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc81f4230 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.266+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc815b100 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.267+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc8068f60 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.268+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbc0042020 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.269+0000 7fcbbf400640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcba406b500 0x7fcba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.269+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fcbc00a9410 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.270+0000 7fcbc7400640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcba406b500 0x7fcba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.271+0000 7fcbc7400640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcba406b500 0x7fcba406d9c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ab0 tx=0x7fcbb80095d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.274+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbc0031390 con 0x7fcbc81f3ab0 2024-04-25T10:13:46.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.473+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbc80b14c0 con 0x7fcba406b500 2024-04-25T10:13:46.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.476+0000 7fcbbf400640 1 -- 172.21.15.5:0/4187147393 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13521 (secure 0 0 0) 0x7fcbc80b14c0 con 0x7fcba406b500 2024-04-25T10:13:46.481 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:46.481 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:13:46.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcba406b500 msgr2=0x7fcba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcba406b500 0x7fcba406d9c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ab0 tx=0x7fcbb80095d0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 msgr2=0x7fcbc815eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 0x7fcbc815eff0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0005040 tx=0x7fcbc0001120 comp rx=0 tx=0).stop 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 shutdown_connections 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbc81f3ab0 0x7fcbc815eff0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcbc80b8e10 0x7fcbc81f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fcba406b500 0x7fcba406d9c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 --2- 172.21.15.5:0/4187147393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcba8004d40 0x7fcbc815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 >> 172.21.15.5:0/4187147393 conn(0x7fcbc808af80 msgr2=0x7fcbc806d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 shutdown_connections 2024-04-25T10:13:46.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.478+0000 7fcbcca00640 1 -- 172.21.15.5:0/4187147393 wait complete. 2024-04-25T10:13:46.501 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-04-25T10:13:44.880181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":274852,"kb_used_data":25416,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754699868,"statfs":{"total":773094113280,"available":772812664832,"internally_reserved":0,"allocated":26025984,"data_stored":19465120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390542,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-174,"num_write_kb":-8302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002211"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":362,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-25T10:13:40.349569+0000","last_change":"2024-04-25T10:01:48.315071+0000","last_active":"2024-04-25T10:13:40.349569+0000","last_peered":"2024-04-25T10:13:40.349569+0000","last_clean":"2024-04-25T10:13:40.349569+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:13:40.349569+0000","last_undegraded":"2024-04-25T10:13:40.349569+0000","last_fullsized":"2024-04-25T10:13:40.349569+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:46.684641+0000","last_clean_scrub_stamp":"2024-04-25T10:01:46.684641+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:52.855767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1527808,"data_stored":713522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":713547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2117632,"data_stored":1300069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509587,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":9672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":9904128,"data_stored":9095568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1531904,"data_stored":710288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509585,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2117632,"data_stored":1297403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5734400,"data_stored":4908512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542293,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:46.502 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.706+0000 7faec3e00640 1 Processor -- start 2024-04-25T10:13:46.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.706+0000 7faec3e00640 1 -- start start 2024-04-25T10:13:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec4141da0 0x7faec406e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec3e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faec415d510 0x7faec40cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec3e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec4071d30 con 0x7faec406e640 2024-04-25T10:13:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec3e00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec4142180 con 0x7faec415d510 2024-04-25T10:13:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec406d620 con 0x7faec4141da0 2024-04-25T10:13:46.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7faec415d510 0x7faec40cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec2000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7faec406e640 0x7faec415ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39670/0 (socket says 172.21.15.5:39670) 2024-04-25T10:13:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.708+0000 7faec2000640 1 -- 172.21.15.5:0/2549858766 learned_addr learned my addr 172.21.15.5:0/2549858766 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.709+0000 7faec2a00640 1 --2- 172.21.15.5:0/2549858766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec4141da0 0x7faec406e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.709+0000 7faec3400640 1 -- 172.21.15.5:0/2549858766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec4141da0 msgr2=0x7faec406e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.709+0000 7faec3400640 1 --2- 172.21.15.5:0/2549858766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec4141da0 0x7faec406e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.709+0000 7faec3400640 1 -- 172.21.15.5:0/2549858766 >> v1:172.21.15.5:6789/0 conn(0x7faec406e640 legacy=0x7faec415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.709+0000 7faec3400640 1 -- 172.21.15.5:0/2549858766 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec406d900 con 0x7faec415d510 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.709+0000 7faec1600640 1 -- 172.21.15.5:0/2549858766 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466008484 0 0) 0x7faec4071d30 con 0x7faec406e640 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.710+0000 7faec3400640 1 --2- 172.21.15.5:0/2549858766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec415d510 0x7faec40cd300 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7faebc009d90 tx=0x7faebc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd350bef567c891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.710+0000 7faec1600640 1 -- 172.21.15.5:0/2549858766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc007740 con 0x7faec415d510 2024-04-25T10:13:46.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.710+0000 7faec1600640 1 -- 172.21.15.5:0/2549858766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc0078e0 con 0x7faec415d510 2024-04-25T10:13:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.710+0000 7faec1600640 1 -- 172.21.15.5:0/2549858766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc007cb0 con 0x7faec415d510 2024-04-25T10:13:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 -- 172.21.15.5:0/2549858766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec415d510 msgr2=0x7faec40cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 --2- 172.21.15.5:0/2549858766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec415d510 0x7faec40cd300 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7faebc009d90 tx=0x7faebc02a660 comp rx=0 tx=0).stop 2024-04-25T10:13:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 -- 172.21.15.5:0/2549858766 shutdown_connections 2024-04-25T10:13:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 --2- 172.21.15.5:0/2549858766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec415d510 0x7faec40cd300 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 --2- 172.21.15.5:0/2549858766 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec4141da0 0x7faec406e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 -- 172.21.15.5:0/2549858766 >> 172.21.15.5:0/2549858766 conn(0x7faec4066f20 msgr2=0x7faec406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 -- 172.21.15.5:0/2549858766 shutdown_connections 2024-04-25T10:13:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.711+0000 7faec3e00640 1 -- 172.21.15.5:0/2549858766 wait complete. 2024-04-25T10:13:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.713+0000 7faec3e00640 1 Processor -- start 2024-04-25T10:13:46.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.713+0000 7faec3e00640 1 -- start start 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.714+0000 7faec3e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec406e640 0x7faec413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.714+0000 7faec3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 0x7faec40b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.714+0000 7faec3e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faec40b3f50 0x7faec40c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.714+0000 7faec3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40cd9c0 con 0x7faec4141da0 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.714+0000 7faec3e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40cd840 con 0x7faec40b3f50 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.714+0000 7faec3e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec40cdb40 con 0x7faec406e640 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 0x7faec40b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 0x7faec40b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55738/0 (socket says 172.21.15.5:55738) 2024-04-25T10:13:46.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 -- 172.21.15.5:0/4035565476 learned_addr learned my addr 172.21.15.5:0/4035565476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2a00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec406e640 0x7faec413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 -- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec406e640 msgr2=0x7faec413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec406e640 0x7faec413ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 -- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faec40b3f50 msgr2=0x7faec40c4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec3400640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faec40b3f50 0x7faec40c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faec40b3f50 0x7faec40c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 -- 172.21.15.5:0/4035565476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40b3b40 con 0x7faec4141da0 2024-04-25T10:13:46.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.715+0000 7faec2000640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 0x7faec40b3600 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c4b0 tx=0x7faeb400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.716+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4018070 con 0x7faec4141da0 2024-04-25T10:13:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.716+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb4007500 con 0x7faec4141da0 2024-04-25T10:13:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.716+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004a20 con 0x7faec4141da0 2024-04-25T10:13:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.716+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec40c8250 con 0x7faec4141da0 2024-04-25T10:13:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.716+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec40c8810 con 0x7faec4141da0 2024-04-25T10:13:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.717+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec4141900 con 0x7faec4141da0 2024-04-25T10:13:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.718+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faeb4004bc0 con 0x7faec4141da0 2024-04-25T10:13:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.719+0000 7faebb400640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea006b4d0 0x7faea006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.719+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7faeb40893f0 con 0x7faec4141da0 2024-04-25T10:13:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.720+0000 7faec2a00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea006b4d0 0x7faea006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.721+0000 7faec2a00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea006b4d0 0x7faea006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faeac006fd0 tx=0x7faeac008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:46.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.723+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faec4141900 con 0x7faec4141da0 2024-04-25T10:13:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.930+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faec4068f60 con 0x7faec4141da0 2024-04-25T10:13:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.932+0000 7faebb400640 1 -- 172.21.15.5:0/4035565476 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+13445 (secure 0 0 0) 0x7faeb40583e0 con 0x7faec4141da0 2024-04-25T10:13:46.937 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":125,"fsid":"8b6dc2a7-3300-467c-ae7a-376f479db0dd","created":"2024-04-25T10:01:33.315256+0000","modified":"2024-04-25T10:13:42.754000+0000","last_up_change":"2024-04-25T10:01:45.378332+0000","last_in_change":"2024-04-25T10:01:36.641057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:46.680467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17e3e259-06e4-4bec-b707-73dd90091223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6809","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6810","nonce":1760674718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6814","nonce":1760674718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6819","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6821","nonce":1760674718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6815","nonce":1760674718},{"type":"v1","addr":"172.21.15.5:6817","nonce":1760674718}]},"public_addr":"172.21.15.5:6810/1760674718","cluster_addr":"172.21.15.5:6814/1760674718","heartbeat_back_addr":"172.21.15.5:6821/1760674718","heartbeat_front_addr":"172.21.15.5:6817/1760674718","state":["exists","up"]},{"osd":1,"uuid":"4dbef475-8322-4087-9182-f3b2785032d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6801","nonce":692601986}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6803","nonce":692601986}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6807","nonce":692601986}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":692601986},{"type":"v1","addr":"172.21.15.5:6805","nonce":692601986}]},"public_addr":"172.21.15.5:6801/692601986","cluster_addr":"172.21.15.5:6803/692601986","heartbeat_back_addr":"172.21.15.5:6807/692601986","heartbeat_front_addr":"172.21.15.5:6805/692601986","state":["exists","up"]},{"osd":2,"uuid":"9f6c5afa-3005-44b9-bd05-14f532e166ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6825","nonce":3928063993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6827","nonce":3928063993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6831","nonce":3928063993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3928063993},{"type":"v1","addr":"172.21.15.5:6829","nonce":3928063993}]},"public_addr":"172.21.15.5:6825/3928063993","cluster_addr":"172.21.15.5:6827/3928063993","heartbeat_back_addr":"172.21.15.5:6831/3928063993","heartbeat_front_addr":"172.21.15.5:6829/3928063993","state":["exists","up"]},{"osd":3,"uuid":"3f1ce5b3-75f7-4950-a030-2dfac8747c3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6811","nonce":3976284996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6813","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6816","nonce":3976284996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6823","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6824","nonce":3976284996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3976284996},{"type":"v1","addr":"172.21.15.5:6820","nonce":3976284996}]},"public_addr":"172.21.15.5:6811/3976284996","cluster_addr":"172.21.15.5:6816/3976284996","heartbeat_back_addr":"172.21.15.5:6824/3976284996","heartbeat_front_addr":"172.21.15.5:6820/3976284996","state":["exists","up"]},{"osd":4,"uuid":"4c58b47b-8353-4349-acc6-f105bffbb9d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6809","nonce":2045277337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6811","nonce":2045277337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6815","nonce":2045277337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2045277337},{"type":"v1","addr":"172.21.15.80:6813","nonce":2045277337}]},"public_addr":"172.21.15.80:6809/2045277337","cluster_addr":"172.21.15.80:6811/2045277337","heartbeat_back_addr":"172.21.15.80:6815/2045277337","heartbeat_front_addr":"172.21.15.80:6813/2045277337","state":["exists","up"]},{"osd":5,"uuid":"e723c2ea-a079-45bb-9823-a810b23ff321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6801","nonce":161144277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6803","nonce":161144277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6807","nonce":161144277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":161144277},{"type":"v1","addr":"172.21.15.80:6805","nonce":161144277}]},"public_addr":"172.21.15.80:6801/161144277","cluster_addr":"172.21.15.80:6803/161144277","heartbeat_back_addr":"172.21.15.80:6807/161144277","heartbeat_front_addr":"172.21.15.80:6805/161144277","state":["exists","up"]},{"osd":6,"uuid":"313e8231-12a4-4eda-97cd-920f3ae7fcf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6825","nonce":3989257232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6827","nonce":3989257232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6831","nonce":3989257232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3989257232},{"type":"v1","addr":"172.21.15.80:6829","nonce":3989257232}]},"public_addr":"172.21.15.80:6825/3989257232","cluster_addr":"172.21.15.80:6827/3989257232","heartbeat_back_addr":"172.21.15.80:6831/3989257232","heartbeat_front_addr":"172.21.15.80:6829/3989257232","state":["exists","up"]},{"osd":7,"uuid":"cd9e067f-4217-427a-b6a6-2e6f8578c9d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6817","nonce":4166609972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6819","nonce":4166609972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6823","nonce":4166609972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":4166609972},{"type":"v1","addr":"172.21.15.80:6821","nonce":4166609972}]},"public_addr":"172.21.15.80:6817/4166609972","cluster_addr":"172.21.15.80:6819/4166609972","heartbeat_back_addr":"172.21.15.80:6823/4166609972","heartbeat_front_addr":"172.21.15.80:6821/4166609972","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.295560+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.055322+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.063470+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.296195+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.520587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.250356+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:42.103995+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:41.813337+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/806266896":"2024-04-26T10:13:39.735973+0000","172.21.15.5:6835/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.5:6833/2518129896":"2024-04-26T10:13:20.803011+0000","172.21.15.80:6837/84886517":"2024-04-26T10:13:06.161908+0000","172.21.15.5:6833/806266896":"2024-04-26T10:13:39.735973+0000","172.21.15.80:6836/84886517":"2024-04-26T10:13:06.161908+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea006b4d0 msgr2=0x7faea006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea006b4d0 0x7faea006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7faeac006fd0 tx=0x7faeac008040 comp rx=0 tx=0).stop 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 msgr2=0x7faec40b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 0x7faec40b3600 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c4b0 tx=0x7faeb400c970 comp rx=0 tx=0).stop 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 shutdown_connections 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7faea006b4d0 0x7faea006d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faec40b3f50 0x7faec40c4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faec4141da0 0x7faec40b3600 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 --2- 172.21.15.5:0/4035565476 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7faec406e640 0x7faec413ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 >> 172.21.15.5:0/4035565476 conn(0x7faec4066f20 msgr2=0x7faec415d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.935+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 shutdown_connections 2024-04-25T10:13:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:46.936+0000 7faec3e00640 1 -- 172.21.15.5:0/4035565476 wait complete. 2024-04-25T10:13:47.960 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:13:47.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.182+0000 7f95ca000640 1 Processor -- start 2024-04-25T10:13:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.182+0000 7f95ca000640 1 -- start start 2024-04-25T10:13:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95ca000640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f95c4142010 0x7f95c406e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95ca000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c406ed60 0x7f95c40cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95ca000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c406d780 con 0x7f95c40cce30 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95ca000640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c41423f0 con 0x7f95c4142010 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95ca000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c40cc210 con 0x7f95c406ed60 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95c3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f95c4142010 0x7f95c406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95c3400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f95c4142010 0x7f95c406e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57290/0 (socket says 172.21.15.5:57290) 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.184+0000 7f95c3400640 1 -- 172.21.15.5:0/17966358 learned_addr learned my addr 172.21.15.5:0/17966358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.185+0000 7f95c2a00640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c406ed60 0x7f95c40cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.185+0000 7f95c3400640 1 -- 172.21.15.5:0/17966358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c406ed60 msgr2=0x7f95c40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.185+0000 7f95c3400640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c406ed60 0x7f95c40cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.185+0000 7f95c3400640 1 -- 172.21.15.5:0/17966358 >> v1:172.21.15.80:6789/0 conn(0x7f95c40cce30 legacy=0x7f95c41426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.185+0000 7f95c3400640 1 -- 172.21.15.5:0/17966358 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c406da60 con 0x7f95c4142010 2024-04-25T10:13:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.185+0000 7f95c2a00640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c406ed60 0x7f95c40cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.186+0000 7f95c3400640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c406e820 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f95c4142570 tx=0x7f95b402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedb5393e72df77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.186+0000 7f95c2000640 1 -- 172.21.15.5:0/17966358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4037070 con 0x7f95c4142010 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.186+0000 7f95c2000640 1 -- 172.21.15.5:0/17966358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b4032440 con 0x7f95c4142010 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.186+0000 7f95c2000640 1 -- 172.21.15.5:0/17966358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4032810 con 0x7f95c4142010 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 -- 172.21.15.5:0/17966358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 msgr2=0x7f95c406e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c406e820 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f95c4142570 tx=0x7f95b402a160 comp rx=0 tx=0).stop 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 -- 172.21.15.5:0/17966358 shutdown_connections 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c406ed60 0x7f95c40cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 --2- 172.21.15.5:0/17966358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c406e820 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 -- 172.21.15.5:0/17966358 >> 172.21.15.5:0/17966358 conn(0x7f95c4066f80 msgr2=0x7f95c406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 -- 172.21.15.5:0/17966358 shutdown_connections 2024-04-25T10:13:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.187+0000 7f95ca000640 1 -- 172.21.15.5:0/17966358 wait complete. 2024-04-25T10:13:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.189+0000 7f95ca000640 1 Processor -- start 2024-04-25T10:13:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.189+0000 7f95ca000640 1 -- start start 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95ca000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95c406ed60 0x7f95c40ab570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95ca000640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40cce30 0x7f95c415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95ca000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c41f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95ca000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c406e300 con 0x7f95c4142010 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95ca000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c406e020 con 0x7f95c406ed60 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95ca000640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4142db0 con 0x7f95c40cce30 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c41f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c41f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57294/0 (socket says 172.21.15.5:57294) 2024-04-25T10:13:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.190+0000 7f95c3e00640 1 -- 172.21.15.5:0/4229804042 learned_addr learned my addr 172.21.15.5:0/4229804042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3400640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95c406ed60 0x7f95c40ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c2a00640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40cce30 0x7f95c415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3e00640 1 -- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40cce30 msgr2=0x7f95c415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3e00640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f95c40cce30 0x7f95c415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3e00640 1 -- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95c406ed60 msgr2=0x7f95c40ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3e00640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95c406ed60 0x7f95c40ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3e00640 1 -- 172.21.15.5:0/4229804042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c40aa510 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3400640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95c406ed60 0x7f95c40ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95c3e00640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c41f4e20 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c780 tx=0x7f95bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc004b60 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc004d00 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00aa90 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.191+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c40aa7b0 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.192+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c40ab050 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.192+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f95c4069040 con 0x7f95c4142010 2024-04-25T10:13:48.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.194+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f95bc00ac30 con 0x7f95c4142010 2024-04-25T10:13:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.194+0000 7f95bb400640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f95a406b4d0 0x7f95a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.195+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f95bc08a430 con 0x7f95c4142010 2024-04-25T10:13:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.195+0000 7f95bb400640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f95a4071ab0 0x7f95a4073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.195+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95bc016ec0 con 0x7f95a4071ab0 2024-04-25T10:13:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.195+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95bc091050 con 0x7f95c4142010 2024-04-25T10:13:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.195+0000 7f95c2a00640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f95a4071ab0 0x7f95a4073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.195+0000 7f95c3400640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f95a406b4d0 0x7f95a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.196+0000 7f95c2a00640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f95a4071ab0 0x7f95a4073ef0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.196+0000 7f95c3400640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f95a406b4d0 0x7f95a406d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f95b4034000 tx=0x7f95b402e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.197+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== osd.0 v2:172.21.15.5:6809/1760674718 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f95bc016ec0 con 0x7f95a4071ab0 2024-04-25T10:13:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.212+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 --> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f95c415b590 con 0x7f95a4071ab0 2024-04-25T10:13:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.212+0000 7f95bb400640 1 -- 172.21.15.5:0/4229804042 <== osd.0 v2:172.21.15.5:6809/1760674718 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f95c415b590 con 0x7f95a4071ab0 2024-04-25T10:13:48.217 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:48.218 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:48.218 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f95a4071ab0 msgr2=0x7f95a4073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:6809/1760674718,v1:172.21.15.5:6810/1760674718] conn(0x7f95a4071ab0 0x7f95a4073ef0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f95a406b4d0 msgr2=0x7f95a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f95a406b4d0 0x7f95a406d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f95b4034000 tx=0x7f95b402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 msgr2=0x7f95c41f4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 --2- 172.21.15.5:0/4229804042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95c4142010 0x7f95c41f4e20 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00c780 tx=0x7f95bc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95c3e00640 1 -- 172.21.15.5:0/4229804042 reap_dead start 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 shutdown_connections 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 >> 172.21.15.5:0/4229804042 conn(0x7f95c4066f80 msgr2=0x7f95c4071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 shutdown_connections 2024-04-25T10:13:48.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.213+0000 7f95ca000640 1 -- 172.21.15.5:0/4229804042 wait complete. 2024-04-25T10:13:48.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:13:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.434+0000 7fd84a400640 1 Processor -- start 2024-04-25T10:13:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.434+0000 7fd84a400640 1 -- start start 2024-04-25T10:13:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.436+0000 7fd84a400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd8441529e0 0x7fd8440c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.436+0000 7fd84a400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8440ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.436+0000 7fd84a400640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84415bdc0 con 0x7fd844071ed0 2024-04-25T10:13:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.436+0000 7fd84a400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8440b2340 con 0x7fd8441529e0 2024-04-25T10:13:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.436+0000 7fd84a400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84415c060 con 0x7fd8440c6410 2024-04-25T10:13:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.437+0000 7fd843400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd8441529e0 0x7fd8440c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.437+0000 7fd842a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8440ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.437+0000 7fd842a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8440ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:42774/0 (socket says 172.21.15.5:42774) 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.437+0000 7fd843400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd8441529e0 0x7fd8440c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:46130/0 (socket says 172.21.15.5:46130) 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.437+0000 7fd842a00640 1 -- 172.21.15.5:0/3395209559 learned_addr learned my addr 172.21.15.5:0/3395209559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.437+0000 7fd843e00640 1 --1- 172.21.15.5:0/3395209559 >> v1:172.21.15.80:6789/0 conn(0x7fd844071ed0 0x7fd8440c4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:57194/0 (socket says 172.21.15.5:57194) 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.438+0000 7fd843400640 1 -- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 msgr2=0x7fd8440ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.438+0000 7fd843400640 1 --2- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8440ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.438+0000 7fd843400640 1 -- 172.21.15.5:0/3395209559 >> v1:172.21.15.80:6789/0 conn(0x7fd844071ed0 legacy=0x7fd8440c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.438+0000 7fd843400640 1 -- 172.21.15.5:0/3395209559 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84415c340 con 0x7fd8441529e0 2024-04-25T10:13:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.438+0000 7fd842a00640 1 --2- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8440ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:48.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.439+0000 7fd843400640 1 --2- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8441529e0 0x7fd8440c5ed0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd83400dbc0 tx=0x7fd83402e480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f45f28ed58dc7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.439+0000 7fd842000640 1 -- 172.21.15.5:0/3395209559 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83403a070 con 0x7fd8441529e0 2024-04-25T10:13:48.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd842000640 1 -- 172.21.15.5:0/3395209559 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83400b650 con 0x7fd8441529e0 2024-04-25T10:13:48.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd842000640 1 -- 172.21.15.5:0/3395209559 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83400ba20 con 0x7fd8441529e0 2024-04-25T10:13:48.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd84a400640 1 -- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8441529e0 msgr2=0x7fd8440c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd84a400640 1 --2- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8441529e0 0x7fd8440c5ed0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd83400dbc0 tx=0x7fd83402e480 comp rx=0 tx=0).stop 2024-04-25T10:13:48.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd84a400640 1 -- 172.21.15.5:0/3395209559 shutdown_connections 2024-04-25T10:13:48.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd84a400640 1 --2- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8440ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd84a400640 1 --2- 172.21.15.5:0/3395209559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8441529e0 0x7fd8440c5ed0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.440+0000 7fd84a400640 1 -- 172.21.15.5:0/3395209559 >> 172.21.15.5:0/3395209559 conn(0x7fd844071150 msgr2=0x7fd844071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.441+0000 7fd84a400640 1 -- 172.21.15.5:0/3395209559 shutdown_connections 2024-04-25T10:13:48.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.441+0000 7fd84a400640 1 -- 172.21.15.5:0/3395209559 wait complete. 2024-04-25T10:13:48.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.442+0000 7fd84a400640 1 Processor -- start 2024-04-25T10:13:48.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.442+0000 7fd84a400640 1 -- start start 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.443+0000 7fd84a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 0x7fd8440cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.443+0000 7fd84a400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8441556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.443+0000 7fd84a400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd844156330 0x7fd8441567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.443+0000 7fd84a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84406e160 con 0x7fd844071ed0 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.443+0000 7fd84a400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84415c900 con 0x7fd844156330 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.443+0000 7fd84a400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84406e2e0 con 0x7fd8440c6410 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 0x7fd8440cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 0x7fd8440cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57310/0 (socket says 172.21.15.5:57310) 2024-04-25T10:13:48.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 -- 172.21.15.5:0/1033016332 learned_addr learned my addr 172.21.15.5:0/1033016332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:48.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843e00640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd844156330 0x7fd8441567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd842a00640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8441556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 -- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 msgr2=0x7fd8441556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8441556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 -- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd844156330 msgr2=0x7fd8441567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd844156330 0x7fd8441567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.444+0000 7fd843400640 1 -- 172.21.15.5:0/1033016332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd844155be0 con 0x7fd844071ed0 2024-04-25T10:13:48.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.445+0000 7fd843400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 0x7fd8440cb2b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd83400dbc0 tx=0x7fd8340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.445+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834043020 con 0x7fd844071ed0 2024-04-25T10:13:48.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.445+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83403b040 con 0x7fd844071ed0 2024-04-25T10:13:48.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.445+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83403a070 con 0x7fd844071ed0 2024-04-25T10:13:48.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.445+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd844154810 con 0x7fd844071ed0 2024-04-25T10:13:48.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.445+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd844157180 con 0x7fd844071ed0 2024-04-25T10:13:48.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.446+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd844068fd0 con 0x7fd844071ed0 2024-04-25T10:13:48.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.447+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd834046020 con 0x7fd844071ed0 2024-04-25T10:13:48.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.448+0000 7fd83b400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd82406b500 0x7fd82406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.448+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd8340ae6c0 con 0x7fd844071ed0 2024-04-25T10:13:48.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.449+0000 7fd842a00640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd82406b500 0x7fd82406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.450+0000 7fd842a00640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd82406b500 0x7fd82406d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd844159370 tx=0x7fd830008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.454+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd83404d030 con 0x7fd844071ed0 2024-04-25T10:13:48.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.654+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8440c5300 con 0x7fd82406b500 2024-04-25T10:13:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.656+0000 7fd83b400640 1 -- 172.21.15.5:0/1033016332 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd8440c5300 con 0x7fd82406b500 2024-04-25T10:13:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.658+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd82406b500 msgr2=0x7fd82406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.658+0000 7fd84a400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd82406b500 0x7fd82406d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd844159370 tx=0x7fd830008040 comp rx=0 tx=0).stop 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.658+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 msgr2=0x7fd8440cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.658+0000 7fd84a400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 0x7fd8440cb2b0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd83400dbc0 tx=0x7fd8340046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.658+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 shutdown_connections 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.658+0000 7fd84a400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fd82406b500 0x7fd82406d9c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.659+0000 7fd84a400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd844156330 0x7fd8441567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.659+0000 7fd84a400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd8440c6410 0x7fd8441556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.659+0000 7fd84a400640 1 --2- 172.21.15.5:0/1033016332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd844071ed0 0x7fd8440cb2b0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.659+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 >> 172.21.15.5:0/1033016332 conn(0x7fd844071150 msgr2=0x7fd844152020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.659+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 shutdown_connections 2024-04-25T10:13:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.659+0000 7fd84a400640 1 -- 172.21.15.5:0/1033016332 wait complete. 2024-04-25T10:13:48.683 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:13:48.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.897+0000 7f5d6dc00640 1 Processor -- start 2024-04-25T10:13:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.898+0000 7f5d6dc00640 1 -- start start 2024-04-25T10:13:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.899+0000 7f5d6dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d680d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.899+0000 7f5d6dc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d680d3f10 0x7f5d680b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.899+0000 7f5d6dc00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d680d2290 con 0x7f5d680d3330 2024-04-25T10:13:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.899+0000 7f5d6dc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d680d2570 con 0x7f5d680d3f10 2024-04-25T10:13:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.899+0000 7f5d6dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680d2850 con 0x7f5d6815de70 2024-04-25T10:13:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.899+0000 7f5d67400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d680d3f10 0x7f5d680b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d67400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d680d3f10 0x7f5d680b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57316/0 (socket says 172.21.15.5:57316) 2024-04-25T10:13:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d67400640 1 -- 172.21.15.5:0/2754655369 learned_addr learned my addr 172.21.15.5:0/2754655369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d66a00640 1 --2- 172.21.15.5:0/2754655369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d680d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d67400640 1 -- 172.21.15.5:0/2754655369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 msgr2=0x7f5d680d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d67400640 1 --2- 172.21.15.5:0/2754655369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d680d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d67400640 1 -- 172.21.15.5:0/2754655369 >> v1:172.21.15.80:6789/0 conn(0x7f5d680d3330 legacy=0x7f5d680d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.900+0000 7f5d67400640 1 -- 172.21.15.5:0/2754655369 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680d2b30 con 0x7f5d680d3f10 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.902+0000 7f5d67400640 1 --2- 172.21.15.5:0/2754655369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3f10 0x7f5d680b6a10 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009e50 tx=0x7f5d6002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9785f86d5c9af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.902+0000 7f5d65600640 1 -- 172.21.15.5:0/2754655369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60036070 con 0x7f5d680d3f10 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.902+0000 7f5d65600640 1 -- 172.21.15.5:0/2754655369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60031600 con 0x7f5d680d3f10 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.902+0000 7f5d65600640 1 -- 172.21.15.5:0/2754655369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d600319d0 con 0x7f5d680d3f10 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2754655369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3f10 msgr2=0x7f5d680b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 --2- 172.21.15.5:0/2754655369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3f10 0x7f5d680b6a10 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009e50 tx=0x7f5d6002a710 comp rx=0 tx=0).stop 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2754655369 shutdown_connections 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 --2- 172.21.15.5:0/2754655369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3f10 0x7f5d680b6a10 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 --2- 172.21.15.5:0/2754655369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d680d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2754655369 >> 172.21.15.5:0/2754655369 conn(0x7f5d680726b0 msgr2=0x7f5d68072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2754655369 shutdown_connections 2024-04-25T10:13:48.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.903+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2754655369 wait complete. 2024-04-25T10:13:48.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.905+0000 7f5d6dc00640 1 Processor -- start 2024-04-25T10:13:48.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.905+0000 7f5d6dc00640 1 -- start start 2024-04-25T10:13:48.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d6dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3330 0x7f5d68150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d6dc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d680d3f10 0x7f5d6815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d6dc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d6814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d6dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680d1fe0 con 0x7f5d680d3330 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d6dc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680d1e60 con 0x7f5d680d3f10 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d6dc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680b9290 con 0x7f5d6815de70 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3330 0x7f5d68150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3330 0x7f5d68150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57332/0 (socket says 172.21.15.5:57332) 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 -- 172.21.15.5:0/2401557816 learned_addr learned my addr 172.21.15.5:0/2401557816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66000640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d680d3f10 0x7f5d6815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d67400640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d6814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 -- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 msgr2=0x7f5d6814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5d6815de70 0x7f5d6814f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 -- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d680d3f10 msgr2=0x7f5d6815a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d680d3f10 0x7f5d6815a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.906+0000 7f5d66a00640 1 -- 172.21.15.5:0/2401557816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680cbfe0 con 0x7f5d680d3330 2024-04-25T10:13:48.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.907+0000 7f5d66a00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3330 0x7f5d68150b90 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800d5f0 tx=0x7f5d5800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.907+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580125f0 con 0x7f5d680d3330 2024-04-25T10:13:48.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.907+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58019070 con 0x7f5d680d3330 2024-04-25T10:13:48.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.907+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580129d0 con 0x7f5d680d3330 2024-04-25T10:13:48.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.907+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d680cc280 con 0x7f5d680d3330 2024-04-25T10:13:48.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.907+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d680cc840 con 0x7f5d680d3330 2024-04-25T10:13:48.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.908+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d68069040 con 0x7f5d680d3330 2024-04-25T10:13:48.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.909+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d58012b70 con 0x7f5d680d3330 2024-04-25T10:13:48.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.910+0000 7f5d5ea00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d4806b4d0 0x7f5d4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.910+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f5d5808a560 con 0x7f5d680d3330 2024-04-25T10:13:48.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.910+0000 7f5d5ea00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7f5d48071ab0 0x7f5d48073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.911+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d58001d30 con 0x7f5d48071ab0 2024-04-25T10:13:48.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.911+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d58091050 con 0x7f5d680d3330 2024-04-25T10:13:48.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.911+0000 7f5d67400640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7f5d48071ab0 0x7f5d48073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.911+0000 7f5d66000640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d4806b4d0 0x7f5d4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.911+0000 7f5d67400640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7f5d48071ab0 0x7f5d48073ef0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.912+0000 7f5d66000640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d4806b4d0 0x7f5d4806d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5d6815acc0 tx=0x7f5d54009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.912+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== osd.1 v2:172.21.15.5:6800/692601986 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5d58001d30 con 0x7f5d48071ab0 2024-04-25T10:13:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.927+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 --> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5d68068030 con 0x7f5d48071ab0 2024-04-25T10:13:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.927+0000 7f5d5ea00640 1 -- 172.21.15.5:0/2401557816 <== osd.1 v2:172.21.15.5:6800/692601986 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5d68068030 con 0x7f5d48071ab0 2024-04-25T10:13:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:48.933 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7f5d48071ab0 msgr2=0x7f5d48073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:6800/692601986,v1:172.21.15.5:6801/692601986] conn(0x7f5d48071ab0 0x7f5d48073ef0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d4806b4d0 msgr2=0x7f5d4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5d4806b4d0 0x7f5d4806d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5d6815acc0 tx=0x7f5d54009040 comp rx=0 tx=0).stop 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3330 msgr2=0x7f5d68150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 --2- 172.21.15.5:0/2401557816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d680d3330 0x7f5d68150b90 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800d5f0 tx=0x7f5d5800dab0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d67400640 1 -- 172.21.15.5:0/2401557816 reap_dead start 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 shutdown_connections 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 >> 172.21.15.5:0/2401557816 conn(0x7f5d680726b0 msgr2=0x7f5d680ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.928+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 shutdown_connections 2024-04-25T10:13:48.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:48.929+0000 7f5d6dc00640 1 -- 172.21.15.5:0/2401557816 wait complete. 2024-04-25T10:13:48.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:13:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.162+0000 7fded6400640 1 Processor -- start 2024-04-25T10:13:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.162+0000 7fded6400640 1 -- start start 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fded6400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00b16e0 0x7fded00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fded6400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded00cd240 0x7fded01622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fded6400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded0165af0 con 0x7fded015c380 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fded6400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded015c760 con 0x7fded00cd240 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fded6400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded01658a0 con 0x7fded00b16e0 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fdecf400640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fded015c380 0x7fded006e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46270/0 (socket says 172.21.15.5:46270) 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fdecf400640 1 -- 172.21.15.5:0/1548327036 learned_addr learned my addr 172.21.15.5:0/1548327036 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.164+0000 7fdecfe00640 1 --2- 172.21.15.5:0/1548327036 >> v2:172.21.15.5:3300/0 conn(0x7fded00cd240 0x7fded01622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdecea00640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00b16e0 0x7fded00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdece000640 1 -- 172.21.15.5:0/1548327036 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435751872 0 0) 0x7fded0165af0 con 0x7fded015c380 2024-04-25T10:13:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdecfe00640 1 -- 172.21.15.5:0/1548327036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00b16e0 msgr2=0x7fded00ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdecfe00640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00b16e0 0x7fded00ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdecfe00640 1 -- 172.21.15.5:0/1548327036 >> v1:172.21.15.5:6789/0 conn(0x7fded015c380 legacy=0x7fded006e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdecfe00640 1 -- 172.21.15.5:0/1548327036 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0165dd0 con 0x7fded00cd240 2024-04-25T10:13:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.165+0000 7fdecea00640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00b16e0 0x7fded00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.166+0000 7fdecfe00640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00cd240 0x7fded01622b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fdec80097a0 tx=0x7fdec802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c7c7fc85183b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fdece000640 1 -- 172.21.15.5:0/1548327036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8037070 con 0x7fded00cd240 2024-04-25T10:13:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fdece000640 1 -- 172.21.15.5:0/1548327036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec802adb0 con 0x7fded00cd240 2024-04-25T10:13:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fdece000640 1 -- 172.21.15.5:0/1548327036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8032660 con 0x7fded00cd240 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fded6400640 1 -- 172.21.15.5:0/1548327036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00cd240 msgr2=0x7fded01622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fded6400640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00cd240 0x7fded01622b0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fdec80097a0 tx=0x7fdec802a060 comp rx=0 tx=0).stop 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fded6400640 1 -- 172.21.15.5:0/1548327036 shutdown_connections 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fded6400640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00cd240 0x7fded01622b0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fded6400640 1 --2- 172.21.15.5:0/1548327036 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00b16e0 0x7fded00ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.167+0000 7fded6400640 1 -- 172.21.15.5:0/1548327036 >> 172.21.15.5:0/1548327036 conn(0x7fded0071150 msgr2=0x7fded0071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.168+0000 7fded6400640 1 -- 172.21.15.5:0/1548327036 shutdown_connections 2024-04-25T10:13:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.168+0000 7fded6400640 1 -- 172.21.15.5:0/1548327036 wait complete. 2024-04-25T10:13:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.169+0000 7fded6400640 1 Processor -- start 2024-04-25T10:13:49.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.169+0000 7fded6400640 1 -- start start 2024-04-25T10:13:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.170+0000 7fded6400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded00b16e0 0x7fded0153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.170+0000 7fded6400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00cd240 0x7fded00b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.170+0000 7fded6400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 0x7fded014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.170+0000 7fded6400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded01627f0 con 0x7fded015c380 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.170+0000 7fded6400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded0166390 con 0x7fded00b16e0 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.170+0000 7fded6400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded0162970 con 0x7fded00cd240 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 0x7fded014f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 0x7fded014f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57340/0 (socket says 172.21.15.5:57340) 2024-04-25T10:13:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 -- 172.21.15.5:0/2982533023 learned_addr learned my addr 172.21.15.5:0/2982533023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecf400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded00b16e0 0x7fded0153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecea00640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00cd240 0x7fded00b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 -- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00cd240 msgr2=0x7fded00b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00cd240 0x7fded00b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 -- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded00b16e0 msgr2=0x7fded0153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded00b16e0 0x7fded0153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.171+0000 7fdecfe00640 1 -- 172.21.15.5:0/2982533023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded014f910 con 0x7fded015c380 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.172+0000 7fdecfe00640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 0x7fded014f3d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fdec8035040 tx=0x7fdec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.172+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8037040 con 0x7fded015c380 2024-04-25T10:13:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.172+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec8004230 con 0x7fded015c380 2024-04-25T10:13:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.172+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8003610 con 0x7fded015c380 2024-04-25T10:13:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.172+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded00b51e0 con 0x7fded015c380 2024-04-25T10:13:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.172+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded00b3600 con 0x7fded015c380 2024-04-25T10:13:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.173+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded0068fd0 con 0x7fded015c380 2024-04-25T10:13:49.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.174+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdec80037b0 con 0x7fded015c380 2024-04-25T10:13:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.176+0000 7fdec7400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdeac06b4d0 0x7fdeac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.176+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fdec80ab1a0 con 0x7fded015c380 2024-04-25T10:13:49.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.177+0000 7fdecf400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdeac06b4d0 0x7fdeac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.178+0000 7fdecf400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdeac06b4d0 0x7fdeac06d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8000f60 tx=0x7fdeb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.181+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdec807a040 con 0x7fded015c380 2024-04-25T10:13:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.382+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fded0153120 con 0x7fdeac06b4d0 2024-04-25T10:13:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.384+0000 7fdec7400640 1 -- 172.21.15.5:0/2982533023 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fded0153120 con 0x7fdeac06b4d0 2024-04-25T10:13:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdeac06b4d0 msgr2=0x7fdeac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdeac06b4d0 0x7fdeac06d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8000f60 tx=0x7fdeb8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 msgr2=0x7fded014f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 0x7fded014f3d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fdec8035040 tx=0x7fdec80046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 shutdown_connections 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded015c380 0x7fded014f3d0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded00cd240 0x7fded00b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fdeac06b4d0 0x7fdeac06d990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 --2- 172.21.15.5:0/2982533023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded00b16e0 0x7fded0153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.386+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 >> 172.21.15.5:0/2982533023 conn(0x7fded0071150 msgr2=0x7fded0161590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.387+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 shutdown_connections 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.387+0000 7fded6400640 1 -- 172.21.15.5:0/2982533023 wait complete. 2024-04-25T10:13:49.411 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:13:49.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.625+0000 7f68bcc00640 1 Processor -- start 2024-04-25T10:13:49.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.626+0000 7f68bcc00640 1 -- start start 2024-04-25T10:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.627+0000 7f68bcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b815db10 0x7f68b80b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.627+0000 7f68bcc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f68b806bb00 0x7f68b8162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.627+0000 7f68bcc00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f68b80d0110 0x7f68b80d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.627+0000 7f68bcc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b806bf40 con 0x7f68b806bb00 2024-04-25T10:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.627+0000 7f68bcc00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b806c1f0 con 0x7f68b80d0110 2024-04-25T10:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.627+0000 7f68bcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b806c4d0 con 0x7f68b815db10 2024-04-25T10:13:49.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f68b806bb00 0x7f68b8162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f68b806bb00 0x7f68b8162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57354/0 (socket says 172.21.15.5:57354) 2024-04-25T10:13:49.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 -- 172.21.15.5:0/4111938752 learned_addr learned my addr 172.21.15.5:0/4111938752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b7400640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b815db10 0x7f68b80b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b7e00640 1 --2- 172.21.15.5:0/4111938752 >> v2:172.21.15.80:3300/0 conn(0x7f68b80d0110 0x7f68b80d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 -- 172.21.15.5:0/4111938752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b815db10 msgr2=0x7f68b80b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b815db10 0x7f68b80b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 -- 172.21.15.5:0/4111938752 >> v2:172.21.15.80:3300/0 conn(0x7f68b80d0110 msgr2=0x7f68b80d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 --2- 172.21.15.5:0/4111938752 >> v2:172.21.15.80:3300/0 conn(0x7f68b80d0110 0x7f68b80d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b6a00640 1 -- 172.21.15.5:0/4111938752 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b806c7b0 con 0x7f68b806bb00 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b7400640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b815db10 0x7f68b80b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:49.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.628+0000 7f68b7e00640 1 --2- 172.21.15.5:0/4111938752 >> v2:172.21.15.80:3300/0 conn(0x7f68b80d0110 0x7f68b80d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.629+0000 7f68b6a00640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b8162280 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f68b80b3150 tx=0x7f68a002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f31d0974224966a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.630+0000 7f68b6000640 1 -- 172.21.15.5:0/4111938752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0037070 con 0x7f68b806bb00 2024-04-25T10:13:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.630+0000 7f68b6000640 1 -- 172.21.15.5:0/4111938752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a0032440 con 0x7f68b806bb00 2024-04-25T10:13:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.630+0000 7f68b6000640 1 -- 172.21.15.5:0/4111938752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0032810 con 0x7f68b806bb00 2024-04-25T10:13:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.630+0000 7f68bcc00640 1 -- 172.21.15.5:0/4111938752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 msgr2=0x7f68b8162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.630+0000 7f68bcc00640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b8162280 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f68b80b3150 tx=0x7f68a002a1d0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 -- 172.21.15.5:0/4111938752 shutdown_connections 2024-04-25T10:13:49.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 --2- 172.21.15.5:0/4111938752 >> v2:172.21.15.80:3300/0 conn(0x7f68b80d0110 0x7f68b80d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b8162280 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 --2- 172.21.15.5:0/4111938752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b815db10 0x7f68b80b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 -- 172.21.15.5:0/4111938752 >> 172.21.15.5:0/4111938752 conn(0x7f68b80b5210 msgr2=0x7f68b80b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 -- 172.21.15.5:0/4111938752 shutdown_connections 2024-04-25T10:13:49.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.631+0000 7f68bcc00640 1 -- 172.21.15.5:0/4111938752 wait complete. 2024-04-25T10:13:49.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.632+0000 7f68bcc00640 1 Processor -- start 2024-04-25T10:13:49.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.633+0000 7f68bcc00640 1 -- start start 2024-04-25T10:13:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68bcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b81596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68bcc00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b80d0110 0x7f68b815fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68bcc00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b815db10 0x7f68b8157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68bcc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b815e070 con 0x7f68b806bb00 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68bcc00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b815def0 con 0x7f68b815db10 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68bcc00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b80ce210 con 0x7f68b80d0110 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b81596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b81596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57358/0 (socket says 172.21.15.5:57358) 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68b7400640 1 -- 172.21.15.5:0/3283033359 learned_addr learned my addr 172.21.15.5:0/3283033359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68b7e00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b815db10 0x7f68b8157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.634+0000 7f68b6a00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b80d0110 0x7f68b815fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68b7400640 1 -- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b80d0110 msgr2=0x7f68b815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68b7400640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f68b80d0110 0x7f68b815fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68b7400640 1 -- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b815db10 msgr2=0x7f68b8157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68b7400640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f68b815db10 0x7f68b8157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68b7400640 1 -- 172.21.15.5:0/3283033359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8158650 con 0x7f68b806bb00 2024-04-25T10:13:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68b7400640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b81596f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f68a800ce30 tx=0x7f68a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800f890 con 0x7f68b806bb00 2024-04-25T10:13:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800fa30 con 0x7f68b806bb00 2024-04-25T10:13:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a800fe00 con 0x7f68b806bb00 2024-04-25T10:13:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.635+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b8158930 con 0x7f68b806bb00 2024-04-25T10:13:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.636+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8158ef0 con 0x7f68b806bb00 2024-04-25T10:13:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.636+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68b81592c0 con 0x7f68b806bb00 2024-04-25T10:13:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.638+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68a8018400 con 0x7f68b806bb00 2024-04-25T10:13:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68af400640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f689406b500 0x7f689406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f68a808af50 con 0x7f68b806bb00 2024-04-25T10:13:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68af400640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f6894071ae0 0x7f6894073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68940745f0 con 0x7f6894071ae0 2024-04-25T10:13:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a808d570 con 0x7f68b806bb00 2024-04-25T10:13:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68b7e00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f6894071ae0 0x7f6894073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.639+0000 7f68b6a00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f689406b500 0x7f689406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.640+0000 7f68b7e00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f6894071ae0 0x7f6894073f20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.641+0000 7f68b6a00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f689406b500 0x7f689406d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f68a0034000 tx=0x7f68a002e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.641+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== osd.2 v2:172.21.15.5:6822/3928063993 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f68940745f0 con 0x7f6894071ae0 2024-04-25T10:13:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.656+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 --> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f68b8068f60 con 0x7f6894071ae0 2024-04-25T10:13:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.657+0000 7f68af400640 1 -- 172.21.15.5:0/3283033359 <== osd.2 v2:172.21.15.5:6822/3928063993 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f68b8068f60 con 0x7f6894071ae0 2024-04-25T10:13:49.662 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:49.662 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:49.662 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.657+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f6894071ae0 msgr2=0x7f6894073f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.657+0000 7f68bcc00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:6822/3928063993,v1:172.21.15.5:6825/3928063993] conn(0x7f6894071ae0 0x7f6894073f20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.657+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f689406b500 msgr2=0x7f689406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.657+0000 7f68bcc00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f689406b500 0x7f689406d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f68a0034000 tx=0x7f68a002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 msgr2=0x7f68b81596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68bcc00640 1 --2- 172.21.15.5:0/3283033359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68b806bb00 0x7f68b81596f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f68a800ce30 tx=0x7f68a800a650 comp rx=0 tx=0).stop 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68b7e00640 1 -- 172.21.15.5:0/3283033359 reap_dead start 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 shutdown_connections 2024-04-25T10:13:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 >> 172.21.15.5:0/3283033359 conn(0x7f68b80b5210 msgr2=0x7f68b80ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 shutdown_connections 2024-04-25T10:13:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.658+0000 7f68bcc00640 1 -- 172.21.15.5:0/3283033359 wait complete. 2024-04-25T10:13:49.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:13:49.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.891+0000 7f0536600640 1 Processor -- start 2024-04-25T10:13:49.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.891+0000 7f0536600640 1 -- start start 2024-04-25T10:13:49.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.892+0000 7f0536600640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05300d06a0 0x7f05300b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.892+0000 7f0536600640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300cf8e0 con 0x7f05300cdcf0 2024-04-25T10:13:49.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.892+0000 7f0536600640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300cfbc0 con 0x7f053015da40 2024-04-25T10:13:49.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.892+0000 7f0536600640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053006bfc0 con 0x7f05300d06a0 2024-04-25T10:13:49.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f0535200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f05300cdcf0 0x7f05300ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46276/0 (socket says 172.21.15.5:46276) 2024-04-25T10:13:49.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f0535200640 1 -- 172.21.15.5:0/1891690038 learned_addr learned my addr 172.21.15.5:0/1891690038 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:49.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f0535c00640 1 --2- 172.21.15.5:0/1891690038 >> v2:172.21.15.5:3300/0 conn(0x7f05300d06a0 0x7f05300b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f052f400640 1 -- 172.21.15.5:0/1891690038 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248267377 0 0) 0x7f05300cf8e0 con 0x7f05300cdcf0 2024-04-25T10:13:49.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f0535c00640 1 -- 172.21.15.5:0/1891690038 >> v1:172.21.15.80:6789/0 conn(0x7f053015da40 legacy=0x7f05300b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f0535c00640 1 -- 172.21.15.5:0/1891690038 >> v1:172.21.15.5:6789/0 conn(0x7f05300cdcf0 legacy=0x7f05300ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.893+0000 7f0535c00640 1 -- 172.21.15.5:0/1891690038 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300cfea0 con 0x7f05300d06a0 2024-04-25T10:13:49.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.895+0000 7f0535c00640 1 --2- 172.21.15.5:0/1891690038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05300d06a0 0x7f05300b0870 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f0528009d90 tx=0x7f052802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d004ea94a579324a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.895+0000 7f052f400640 1 -- 172.21.15.5:0/1891690038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528031600 con 0x7f05300d06a0 2024-04-25T10:13:49.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.895+0000 7f052f400640 1 -- 172.21.15.5:0/1891690038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05280317a0 con 0x7f05300d06a0 2024-04-25T10:13:49.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.895+0000 7f052f400640 1 -- 172.21.15.5:0/1891690038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528031b70 con 0x7f05300d06a0 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.895+0000 7f0536600640 1 -- 172.21.15.5:0/1891690038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05300d06a0 msgr2=0x7f05300b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.895+0000 7f0536600640 1 --2- 172.21.15.5:0/1891690038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05300d06a0 0x7f05300b0870 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f0528009d90 tx=0x7f052802a610 comp rx=0 tx=0).stop 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.896+0000 7f0536600640 1 -- 172.21.15.5:0/1891690038 shutdown_connections 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.896+0000 7f0536600640 1 --2- 172.21.15.5:0/1891690038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05300d06a0 0x7f05300b0870 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.896+0000 7f0536600640 1 -- 172.21.15.5:0/1891690038 >> 172.21.15.5:0/1891690038 conn(0x7f05300d1470 msgr2=0x7f05300d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.896+0000 7f0536600640 1 -- 172.21.15.5:0/1891690038 shutdown_connections 2024-04-25T10:13:49.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.896+0000 7f0536600640 1 -- 172.21.15.5:0/1891690038 wait complete. 2024-04-25T10:13:49.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.897+0000 7f0536600640 1 Processor -- start 2024-04-25T10:13:49.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.898+0000 7f0536600640 1 -- start start 2024-04-25T10:13:49.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.898+0000 7f0536600640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f05300cdcf0 0x7f0530154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.898+0000 7f0536600640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05300d06a0 0x7f05300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.898+0000 7f0536600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 0x7f05300b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0536600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b2a50 con 0x7f053015da40 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0536600640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300cf590 con 0x7f05300d06a0 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0536600640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b2bd0 con 0x7f05300cdcf0 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 0x7f05300b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 0x7f05300b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57374/0 (socket says 172.21.15.5:57374) 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 -- 172.21.15.5:0/2314236055 learned_addr learned my addr 172.21.15.5:0/2314236055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535200640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f05300cdcf0 0x7f0530154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f052fe00640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05300d06a0 0x7f05300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 -- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f05300cdcf0 msgr2=0x7f0530154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f05300cdcf0 0x7f0530154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 -- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05300d06a0 msgr2=0x7f05300b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05300d06a0 0x7f05300b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535c00640 1 -- 172.21.15.5:0/2314236055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300b5ad0 con 0x7f053015da40 2024-04-25T10:13:49.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.899+0000 7f0535200640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f05300cdcf0 0x7f0530154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.900+0000 7f0535c00640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 0x7f05300b54c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0528009eb0 tx=0x7f0528004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.900+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528040070 con 0x7f053015da40 2024-04-25T10:13:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.900+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0528030b60 con 0x7f053015da40 2024-04-25T10:13:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.900+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528045460 con 0x7f053015da40 2024-04-25T10:13:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.900+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05300b5db0 con 0x7f053015da40 2024-04-25T10:13:49.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.900+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05300b6370 con 0x7f053015da40 2024-04-25T10:13:49.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.901+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0530068f60 con 0x7f053015da40 2024-04-25T10:13:49.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.902+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0528039040 con 0x7f053015da40 2024-04-25T10:13:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.904+0000 7f052cc00640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f051006b4d0 0x7f051006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.904+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f05280ab9d0 con 0x7f053015da40 2024-04-25T10:13:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.905+0000 7f0535200640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f051006b4d0 0x7f051006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.906+0000 7f0535200640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f051006b4d0 0x7f051006d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0520000f60 tx=0x7f0520009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:49.908+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f052807a2e0 con 0x7f053015da40 2024-04-25T10:13:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.116+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f05300ccab0 con 0x7f051006b4d0 2024-04-25T10:13:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.117+0000 7f052cc00640 1 -- 172.21.15.5:0/2314236055 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f05300ccab0 con 0x7f051006b4d0 2024-04-25T10:13:50.122 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:13:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f051006b4d0 msgr2=0x7f051006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f051006b4d0 0x7f051006d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0520000f60 tx=0x7f0520009040 comp rx=0 tx=0).stop 2024-04-25T10:13:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 msgr2=0x7f05300b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 0x7f05300b54c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0528009eb0 tx=0x7f0528004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 shutdown_connections 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f053015da40 0x7f05300b54c0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f051006b4d0 0x7f051006d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05300d06a0 0x7f05300b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 --2- 172.21.15.5:0/2314236055 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f05300cdcf0 0x7f0530154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.119+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 >> 172.21.15.5:0/2314236055 conn(0x7f05300d1470 msgr2=0x7f05300cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.120+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 shutdown_connections 2024-04-25T10:13:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.120+0000 7f0536600640 1 -- 172.21.15.5:0/2314236055 wait complete. 2024-04-25T10:13:50.144 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:13:50.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.360+0000 7ff351400640 1 Processor -- start 2024-04-25T10:13:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.361+0000 7ff351400640 1 -- start start 2024-04-25T10:13:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff351400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c0d0820 0x7ff34c0b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff351400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff34c06bb00 0x7ff34c162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff351400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34c06c1f0 con 0x7ff34c15db10 2024-04-25T10:13:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff351400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34c06bfb0 con 0x7ff34c06bb00 2024-04-25T10:13:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff351400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c06c4d0 con 0x7ff34c0d0820 2024-04-25T10:13:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff350a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff34c06bb00 0x7ff34c162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.362+0000 7ff34be00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff34c15db10 0x7ff34c0d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46290/0 (socket says 172.21.15.5:46290) 2024-04-25T10:13:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff34be00640 1 -- 172.21.15.5:0/356819472 learned_addr learned my addr 172.21.15.5:0/356819472 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff343400640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c0d0820 0x7ff34c0b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff350a00640 1 -- 172.21.15.5:0/356819472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c0d0820 msgr2=0x7ff34c0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff350a00640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c0d0820 0x7ff34c0b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff350a00640 1 -- 172.21.15.5:0/356819472 >> v1:172.21.15.5:6789/0 conn(0x7ff34c15db10 legacy=0x7ff34c0d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff350a00640 1 -- 172.21.15.5:0/356819472 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c06c7b0 con 0x7ff34c06bb00 2024-04-25T10:13:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff34b400640 1 -- 172.21.15.5:0/356819472 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910014919 0 0) 0x7ff34c06c1f0 con 0x7ff34c15db10 2024-04-25T10:13:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.363+0000 7ff343400640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c0d0820 0x7ff34c0b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.364+0000 7ff350a00640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c06bb00 0x7ff34c162280 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7ff344009d90 tx=0x7ff34402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b128a80f040d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.365+0000 7ff34b400640 1 -- 172.21.15.5:0/356819472 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344007740 con 0x7ff34c06bb00 2024-04-25T10:13:50.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.365+0000 7ff34b400640 1 -- 172.21.15.5:0/356819472 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3440078e0 con 0x7ff34c06bb00 2024-04-25T10:13:50.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.365+0000 7ff34b400640 1 -- 172.21.15.5:0/356819472 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344007cb0 con 0x7ff34c06bb00 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.365+0000 7ff351400640 1 -- 172.21.15.5:0/356819472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c06bb00 msgr2=0x7ff34c162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.365+0000 7ff351400640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c06bb00 0x7ff34c162280 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7ff344009d90 tx=0x7ff34402a660 comp rx=0 tx=0).stop 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.366+0000 7ff351400640 1 -- 172.21.15.5:0/356819472 shutdown_connections 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.366+0000 7ff351400640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c06bb00 0x7ff34c162280 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.366+0000 7ff351400640 1 --2- 172.21.15.5:0/356819472 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c0d0820 0x7ff34c0b6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.366+0000 7ff351400640 1 -- 172.21.15.5:0/356819472 >> 172.21.15.5:0/356819472 conn(0x7ff34c0b5210 msgr2=0x7ff34c0b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.366+0000 7ff351400640 1 -- 172.21.15.5:0/356819472 shutdown_connections 2024-04-25T10:13:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.366+0000 7ff351400640 1 -- 172.21.15.5:0/356819472 wait complete. 2024-04-25T10:13:50.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.367+0000 7ff351400640 1 Processor -- start 2024-04-25T10:13:50.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.368+0000 7ff351400640 1 -- start start 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.368+0000 7ff351400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c06bb00 0x7ff34c1f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff351400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34c0d0820 0x7ff34c151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff351400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c15db10 0x7ff34c160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff351400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c15e070 con 0x7ff34c15db10 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff351400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c15def0 con 0x7ff34c0d0820 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff351400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff34c0d20b0 con 0x7ff34c06bb00 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c15db10 0x7ff34c160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c15db10 0x7ff34c160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57398/0 (socket says 172.21.15.5:57398) 2024-04-25T10:13:50.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 -- 172.21.15.5:0/3901514765 learned_addr learned my addr 172.21.15.5:0/3901514765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff34be00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c06bb00 0x7ff34c1f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff343400640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34c0d0820 0x7ff34c151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 -- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c06bb00 msgr2=0x7ff34c1f7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff34c06bb00 0x7ff34c1f7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 -- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34c0d0820 msgr2=0x7ff34c151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff34c0d0820 0x7ff34c151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.369+0000 7ff350a00640 1 -- 172.21.15.5:0/3901514765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c161270 con 0x7ff34c15db10 2024-04-25T10:13:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.370+0000 7ff350a00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c15db10 0x7ff34c160760 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7ff344009eb0 tx=0x7ff344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.370+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344040070 con 0x7ff34c15db10 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.370+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344031bc0 con 0x7ff34c15db10 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.370+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34c160f90 con 0x7ff34c15db10 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.370+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344045460 con 0x7ff34c15db10 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.370+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34c1f6f50 con 0x7ff34c15db10 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.371+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff34c068f60 con 0x7ff34c15db10 2024-04-25T10:13:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.373+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff344039040 con 0x7ff34c15db10 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.373+0000 7ff348c00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff32806b4d0 0x7ff32806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.374+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff3440aba70 con 0x7ff34c15db10 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.374+0000 7ff348c00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7ff328071ab0 0x7ff328073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.374+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3440020b0 con 0x7ff328071ab0 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.374+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3440b02a0 con 0x7ff34c15db10 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.374+0000 7ff343400640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7ff328071ab0 0x7ff328073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.374+0000 7ff34be00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff32806b4d0 0x7ff32806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.375+0000 7ff343400640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7ff328071ab0 0x7ff328073ef0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.375+0000 7ff34be00640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff32806b4d0 0x7ff32806d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000f60 tx=0x7ff33c007c00 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.375+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== osd.3 v2:172.21.15.5:6808/3976284996 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff3440020b0 con 0x7ff328071ab0 2024-04-25T10:13:50.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.390+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 --> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff34c0cdf40 con 0x7ff328071ab0 2024-04-25T10:13:50.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.391+0000 7ff348c00640 1 -- 172.21.15.5:0/3901514765 <== osd.3 v2:172.21.15.5:6808/3976284996 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff34c0cdf40 con 0x7ff328071ab0 2024-04-25T10:13:50.396 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:50.396 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.391+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7ff328071ab0 msgr2=0x7ff328073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.391+0000 7ff351400640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:6808/3976284996,v1:172.21.15.5:6811/3976284996] conn(0x7ff328071ab0 0x7ff328073ef0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff32806b4d0 msgr2=0x7ff32806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7ff32806b4d0 0x7ff32806d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff33c000f60 tx=0x7ff33c007c00 comp rx=0 tx=0).stop 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c15db10 msgr2=0x7ff34c160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 --2- 172.21.15.5:0/3901514765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff34c15db10 0x7ff34c160760 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7ff344009eb0 tx=0x7ff344004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff350a00640 1 -- 172.21.15.5:0/3901514765 reap_dead start 2024-04-25T10:13:50.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 shutdown_connections 2024-04-25T10:13:50.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 >> 172.21.15.5:0/3901514765 conn(0x7ff34c0b5210 msgr2=0x7ff34c0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 shutdown_connections 2024-04-25T10:13:50.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.392+0000 7ff351400640 1 -- 172.21.15.5:0/3901514765 wait complete. 2024-04-25T10:13:50.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:13:50.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.625+0000 7f5e79800640 1 Processor -- start 2024-04-25T10:13:50.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.626+0000 7f5e79800640 1 -- start start 2024-04-25T10:13:50.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.627+0000 7f5e79800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5e740d07d0 0x7f5e740b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.627+0000 7f5e79800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e740cf410 con 0x7f5e740d2380 2024-04-25T10:13:50.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.627+0000 7f5e79800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e740cf6f0 con 0x7f5e74161200 2024-04-25T10:13:50.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.627+0000 7f5e79800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e740b28b0 con 0x7f5e740d07d0 2024-04-25T10:13:50.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.628+0000 7f5e78e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5e740d2380 0x7f5e740b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46306/0 (socket says 172.21.15.5:46306) 2024-04-25T10:13:50.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.628+0000 7f5e78e00640 1 -- 172.21.15.5:0/284863785 learned_addr learned my addr 172.21.15.5:0/284863785 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.628+0000 7f5e6b400640 1 --2- 172.21.15.5:0/284863785 >> v2:172.21.15.80:3300/0 conn(0x7f5e740d07d0 0x7f5e740b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.628+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138268699 0 0) 0x7f5e740cf410 con 0x7f5e740d2380 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.628+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e58003520 con 0x7f5e740d2380 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 520475665 0 0) 0x7f5e58003520 con 0x7f5e740d2380 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 >> v2:172.21.15.80:3300/0 conn(0x7f5e740d07d0 msgr2=0x7f5e740b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 --2- 172.21.15.5:0/284863785 >> v2:172.21.15.80:3300/0 conn(0x7f5e740d07d0 0x7f5e740b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 >> v1:172.21.15.80:6789/0 conn(0x7f5e74161200 legacy=0x7f5e740d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740d3fd0 con 0x7f5e740d2380 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f5e6c0019f0 con 0x7f5e740d2380 2024-04-25T10:13:50.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 >> v1:172.21.15.5:6789/0 conn(0x7f5e740d2380 legacy=0x7f5e740b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e58003a00 0x7f5e58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e58004800 0x7f5e58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b28b0 con 0x7f5e580051c0 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5800c0c0 con 0x7f5e58004800 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5800c2c0 con 0x7f5e58003a00 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.629+0000 7f5e78e00640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e73e00640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e58003a00 0x7f5e58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e6b400640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e58004800 0x7f5e58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e78e00640 1 -- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e58003a00 msgr2=0x7f5e58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e78e00640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e58003a00 0x7f5e58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e78e00640 1 -- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e58004800 msgr2=0x7f5e58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e78e00640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e58004800 0x7f5e58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.630+0000 7f5e78e00640 1 -- 172.21.15.5:0/284863785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e58003520 con 0x7f5e580051c0 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.631+0000 7f5e78e00640 1 -- 172.21.15.5:0/284863785 reap_dead start 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.631+0000 7f5e78e00640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e5800b9c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00a7b0 tx=0x7f5e58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee651186a15bb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.631+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00a3d0 con 0x7f5e580051c0 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.631+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c02f070 con 0x7f5e580051c0 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.631+0000 7f5e73400640 1 -- 172.21.15.5:0/284863785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02bc20 con 0x7f5e580051c0 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 -- 172.21.15.5:0/284863785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 msgr2=0x7f5e5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e5800b9c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00a7b0 tx=0x7f5e58005830 comp rx=0 tx=0).stop 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 -- 172.21.15.5:0/284863785 shutdown_connections 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 --2- 172.21.15.5:0/284863785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e5800b9c0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 -- 172.21.15.5:0/284863785 >> 172.21.15.5:0/284863785 conn(0x7f5e7408af80 msgr2=0x7f5e740d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 -- 172.21.15.5:0/284863785 shutdown_connections 2024-04-25T10:13:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.632+0000 7f5e79800640 1 -- 172.21.15.5:0/284863785 wait complete. 2024-04-25T10:13:50.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.634+0000 7f5e79800640 1 Processor -- start 2024-04-25T10:13:50.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.634+0000 7f5e79800640 1 -- start start 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e79800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e7409e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e79800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e740cd4f0 0x7f5e741f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e79800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e740cd970 0x7f5e741f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e79800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b4d30 con 0x7f5e580051c0 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e79800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b4bb0 con 0x7f5e740cd4f0 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e79800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740b4eb0 con 0x7f5e740cd970 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e73e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e7409e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e73e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e7409e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57428/0 (socket says 172.21.15.5:57428) 2024-04-25T10:13:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.635+0000 7f5e73e00640 1 -- 172.21.15.5:0/1101637080 learned_addr learned my addr 172.21.15.5:0/1101637080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:50.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e78e00640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e740cd970 0x7f5e741f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e6b400640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e740cd4f0 0x7f5e741f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e73e00640 1 -- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e740cd970 msgr2=0x7f5e741f3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e73e00640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e740cd970 0x7f5e741f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e73e00640 1 -- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e740cd4f0 msgr2=0x7f5e741f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e73e00640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e740cd4f0 0x7f5e741f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e73e00640 1 -- 172.21.15.5:0/1101637080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7409d650 con 0x7f5e580051c0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e73e00640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e7409e110 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400e500 tx=0x7f5e6400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640052e0 con 0x7f5e580051c0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64018070 con 0x7f5e580051c0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64003810 con 0x7f5e580051c0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e741f4280 con 0x7f5e580051c0 2024-04-25T10:13:50.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.636+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7409dbf0 con 0x7f5e580051c0 2024-04-25T10:13:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.638+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e74068f60 con 0x7f5e580051c0 2024-04-25T10:13:50.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.639+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e640039b0 con 0x7f5e580051c0 2024-04-25T10:13:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.640+0000 7f5e70c00640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e5006b4d0 0x7f5e5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.640+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f5e64089e70 con 0x7f5e580051c0 2024-04-25T10:13:50.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.641+0000 7f5e6b400640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e5006b4d0 0x7f5e5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.642+0000 7f5e6b400640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e5006b4d0 0x7f5e5006d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5e740cd210 tx=0x7f5e5c008040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.644+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e64058d20 con 0x7f5e580051c0 2024-04-25T10:13:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.841+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e740cf150 con 0x7f5e5006b4d0 2024-04-25T10:13:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.843+0000 7f5e70c00640 1 -- 172.21.15.5:0/1101637080 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5e740cf150 con 0x7f5e5006b4d0 2024-04-25T10:13:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:13:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e5006b4d0 msgr2=0x7f5e5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e5006b4d0 0x7f5e5006d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5e740cd210 tx=0x7f5e5c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 msgr2=0x7f5e7409e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e7409e110 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400e500 tx=0x7f5e6400e9c0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 shutdown_connections 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e740cd970 0x7f5e741f3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f5e5006b4d0 0x7f5e5006d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e740cd4f0 0x7f5e741f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 --2- 172.21.15.5:0/1101637080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e580051c0 0x7f5e7409e110 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 >> 172.21.15.5:0/1101637080 conn(0x7f5e7408af80 msgr2=0x7f5e7406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 shutdown_connections 2024-04-25T10:13:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:50.845+0000 7f5e79800640 1 -- 172.21.15.5:0/1101637080 wait complete. 2024-04-25T10:13:50.870 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:13:50.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:51.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.084+0000 7f8de2e00640 1 Processor -- start 2024-04-25T10:13:51.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.085+0000 7f8de2e00640 1 -- start start 2024-04-25T10:13:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.086+0000 7f8de2e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8ddc15db10 0x7f8ddc0b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.086+0000 7f8de2e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc06bf40 con 0x7f8ddc06bb00 2024-04-25T10:13:51.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.086+0000 7f8de2e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc06c1f0 con 0x7f8ddc0d0820 2024-04-25T10:13:51.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.086+0000 7f8de2e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc06c4d0 con 0x7f8ddc15db10 2024-04-25T10:13:51.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.087+0000 7f8de1000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8ddc06bb00 0x7f8ddc0d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46316/0 (socket says 172.21.15.5:46316) 2024-04-25T10:13:51.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.087+0000 7f8de1000640 1 -- 172.21.15.5:0/2803158529 learned_addr learned my addr 172.21.15.5:0/2803158529 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.087+0000 7f8de1a00640 1 --2- 172.21.15.5:0/2803158529 >> v2:172.21.15.80:3300/0 conn(0x7f8ddc15db10 0x7f8ddc0b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.087+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563493252 0 0) 0x7f8ddc06bf40 con 0x7f8ddc06bb00 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc0003520 con 0x7f8ddc06bb00 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673832672 0 0) 0x7f8dc0003520 con 0x7f8ddc06bb00 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 >> v2:172.21.15.80:3300/0 conn(0x7f8ddc15db10 msgr2=0x7f8ddc0b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 --2- 172.21.15.5:0/2803158529 >> v2:172.21.15.80:3300/0 conn(0x7f8ddc15db10 0x7f8ddc0b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 >> v1:172.21.15.80:6789/0 conn(0x7f8ddc0d0820 legacy=0x7f8ddc0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc06c7b0 con 0x7f8ddc06bb00 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f8dc8001670 con 0x7f8ddc06bb00 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 >> v1:172.21.15.5:6789/0 conn(0x7f8ddc06bb00 legacy=0x7f8ddc0d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8dc0003a00 0x7f8dc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8dc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8dc00051c0 0x7f8dc000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc06c4d0 con 0x7f8dc0004800 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc000c0c0 con 0x7f8dc0003a00 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc000c2c0 con 0x7f8dc00051c0 2024-04-25T10:13:51.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.088+0000 7f8de1000640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8dc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de1a00640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8dc0003a00 0x7f8dc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de1000640 1 -- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8dc00051c0 msgr2=0x7f8dc000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de1000640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8dc00051c0 0x7f8dc000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de1000640 1 -- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8dc0003a00 msgr2=0x7f8dc0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de1000640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8dc0003a00 0x7f8dc0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de1000640 1 -- 172.21.15.5:0/2803158529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0003520 con 0x7f8dc0004800 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.089+0000 7f8de2400640 1 -- 172.21.15.5:0/2803158529 reap_dead start 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.090+0000 7f8de1000640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8dc0004c80 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c0e0 tx=0x7f8dc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555c08c245176de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.090+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8007c20 con 0x7f8dc0004800 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.090+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8005af0 con 0x7f8dc0004800 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.090+0000 7f8ddbe00640 1 -- 172.21.15.5:0/2803158529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8007070 con 0x7f8dc0004800 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 -- 172.21.15.5:0/2803158529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 msgr2=0x7f8dc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8dc0004c80 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c0e0 tx=0x7f8dc0005830 comp rx=0 tx=0).stop 2024-04-25T10:13:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 -- 172.21.15.5:0/2803158529 shutdown_connections 2024-04-25T10:13:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 --2- 172.21.15.5:0/2803158529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8dc0004c80 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 -- 172.21.15.5:0/2803158529 >> 172.21.15.5:0/2803158529 conn(0x7f8ddc0b5210 msgr2=0x7f8ddc0b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 -- 172.21.15.5:0/2803158529 shutdown_connections 2024-04-25T10:13:51.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.091+0000 7f8de2e00640 1 -- 172.21.15.5:0/2803158529 wait complete. 2024-04-25T10:13:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.093+0000 7f8de2e00640 1 Processor -- start 2024-04-25T10:13:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.093+0000 7f8de2e00640 1 -- start start 2024-04-25T10:13:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.094+0000 7f8de2e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8ddc1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.094+0000 7f8de2e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8ddc1e8c00 0x7f8ddc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.094+0000 7f8de2e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ddc0b7e10 0x7f8ddc15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.094+0000 7f8de2e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc15e070 con 0x7f8dc0004800 2024-04-25T10:13:51.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.094+0000 7f8de2e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc15def0 con 0x7f8ddc0b7e10 2024-04-25T10:13:51.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.094+0000 7f8de2e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0b6ba0 con 0x7f8ddc1e8c00 2024-04-25T10:13:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8ddc1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8ddc1590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57452/0 (socket says 172.21.15.5:57452) 2024-04-25T10:13:51.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 -- 172.21.15.5:0/1960109637 learned_addr learned my addr 172.21.15.5:0/1960109637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de2400640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ddc0b7e10 0x7f8ddc15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1000640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8ddc1e8c00 0x7f8ddc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 -- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8ddc1e8c00 msgr2=0x7f8ddc0b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8ddc1e8c00 0x7f8ddc0b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 -- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ddc0b7e10 msgr2=0x7f8ddc15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ddc0b7e10 0x7f8ddc15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.095+0000 7f8de1a00640 1 -- 172.21.15.5:0/1960109637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc160ce0 con 0x7f8dc0004800 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.096+0000 7f8de1a00640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8ddc1590b0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009f30 tx=0x7f8dcc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.096+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc01a020 con 0x7f8dc0004800 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.096+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc016470 con 0x7f8dc0004800 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.096+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc016860 con 0x7f8dc0004800 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.096+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc15fc40 con 0x7f8dc0004800 2024-04-25T10:13:51.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.096+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc160200 con 0x7f8dc0004800 2024-04-25T10:13:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.097+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ddc15f9f0 con 0x7f8dc0004800 2024-04-25T10:13:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.098+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8dcc005d10 con 0x7f8dc0004800 2024-04-25T10:13:51.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.099+0000 7f8dd9600640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.099+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8dcc08acd0 con 0x7f8dc0004800 2024-04-25T10:13:51.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.100+0000 7f8dd9600640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7f8dbc071ae0 0x7f8dbc073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.100+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8dbc0745f0 con 0x7f8dbc071ae0 2024-04-25T10:13:51.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.100+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dcc08d2f0 con 0x7f8dc0004800 2024-04-25T10:13:51.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.100+0000 7f8de1000640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.100+0000 7f8de2400640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7f8dbc071ae0 0x7f8dbc073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.101+0000 7f8de2400640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7f8dbc071ae0 0x7f8dbc073f20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.102+0000 7f8de1000640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8032000 tx=0x7f8dc8009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.104+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== osd.4 v2:172.21.15.80:6808/2045277337 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8dbc0745f0 con 0x7f8dbc071ae0 2024-04-25T10:13:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.118+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 --> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8ddc068f60 con 0x7f8dbc071ae0 2024-04-25T10:13:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.119+0000 7f8dd9600640 1 -- 172.21.15.5:0/1960109637 <== osd.4 v2:172.21.15.80:6808/2045277337 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8ddc068f60 con 0x7f8dbc071ae0 2024-04-25T10:13:51.125 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:51.125 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:51.125 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7f8dbc071ae0 msgr2=0x7f8dbc073f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6808/2045277337,v1:172.21.15.80:6809/2045277337] conn(0x7f8dbc071ae0 0x7f8dbc073f20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8dbc06b500 msgr2=0x7f8dbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8032000 tx=0x7f8dc8009030 comp rx=0 tx=0).stop 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 msgr2=0x7f8ddc1590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 --2- 172.21.15.5:0/1960109637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dc0004800 0x7f8ddc1590b0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009f30 tx=0x7f8dcc0115a0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2400640 1 -- 172.21.15.5:0/1960109637 reap_dead start 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 shutdown_connections 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 >> 172.21.15.5:0/1960109637 conn(0x7f8ddc0b5210 msgr2=0x7f8ddc0d2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.120+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 shutdown_connections 2024-04-25T10:13:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.121+0000 7f8de2e00640 1 -- 172.21.15.5:0/1960109637 wait complete. 2024-04-25T10:13:51.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:13:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.354+0000 7f2ec8a00640 1 Processor -- start 2024-04-25T10:13:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.355+0000 7f2ec8a00640 1 -- start start 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.356+0000 7f2ec8a00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ec4161200 0x7f2ec40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.356+0000 7f2ec8a00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec40cf410 con 0x7f2ec40d07d0 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.356+0000 7f2ec8a00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec40cf6f0 con 0x7f2ec40d2380 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.356+0000 7f2ec8a00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec40d27c0 con 0x7f2ec4161200 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.356+0000 7f2ec2800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2ec4161200 0x7f2ec40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.357+0000 7f2ec3200640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2ec40d07d0 0x7f2ec40b2f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46326/0 (socket says 172.21.15.5:46326) 2024-04-25T10:13:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.357+0000 7f2ec3200640 1 -- 172.21.15.5:0/2161698680 learned_addr learned my addr 172.21.15.5:0/2161698680 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.357+0000 7f2ec2800640 1 -- 172.21.15.5:0/2161698680 >> v1:172.21.15.80:6789/0 conn(0x7f2ec40d2380 legacy=0x7f2ec40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.357+0000 7f2ec2800640 1 -- 172.21.15.5:0/2161698680 >> v1:172.21.15.5:6789/0 conn(0x7f2ec40d07d0 legacy=0x7f2ec40b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.357+0000 7f2ec2800640 1 -- 172.21.15.5:0/2161698680 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40d3fd0 con 0x7f2ec4161200 2024-04-25T10:13:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.358+0000 7f2ec2800640 1 --2- 172.21.15.5:0/2161698680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec4161200 0x7f2ec40b4bb0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d90 tx=0x7f2eb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270acbc64f65e494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.358+0000 7f2ec1400640 1 -- 172.21.15.5:0/2161698680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4007740 con 0x7f2ec4161200 2024-04-25T10:13:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec1400640 1 -- 172.21.15.5:0/2161698680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb40078e0 con 0x7f2ec4161200 2024-04-25T10:13:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec1400640 1 -- 172.21.15.5:0/2161698680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4007cb0 con 0x7f2ec4161200 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 -- 172.21.15.5:0/2161698680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec4161200 msgr2=0x7f2ec40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/2161698680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec4161200 0x7f2ec40b4bb0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009d90 tx=0x7f2eb402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 -- 172.21.15.5:0/2161698680 shutdown_connections 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/2161698680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec4161200 0x7f2ec40b4bb0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 -- 172.21.15.5:0/2161698680 >> 172.21.15.5:0/2161698680 conn(0x7f2ec408af80 msgr2=0x7f2ec40d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 -- 172.21.15.5:0/2161698680 shutdown_connections 2024-04-25T10:13:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.359+0000 7f2ec8a00640 1 -- 172.21.15.5:0/2161698680 wait complete. 2024-04-25T10:13:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.361+0000 7f2ec8a00640 1 Processor -- start 2024-04-25T10:13:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.361+0000 7f2ec8a00640 1 -- start start 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.362+0000 7f2ec8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 0x7f2ec40b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.362+0000 7f2ec8a00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40d2380 0x7f2ec41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.362+0000 7f2ec8a00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ec4161200 0x7f2ec40cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.362+0000 7f2ec8a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40b3690 con 0x7f2ec40d07d0 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.362+0000 7f2ec8a00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40b28b0 con 0x7f2ec40d2380 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.362+0000 7f2ec8a00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec40b3810 con 0x7f2ec4161200 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 0x7f2ec40b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 0x7f2ec40b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57468/0 (socket says 172.21.15.5:57468) 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 -- 172.21.15.5:0/3574549672 learned_addr learned my addr 172.21.15.5:0/3574549672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec3200640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ec4161200 0x7f2ec40cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec1e00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40d2380 0x7f2ec41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 -- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ec4161200 msgr2=0x7f2ec40cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ec4161200 0x7f2ec40cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 -- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40d2380 msgr2=0x7f2ec41588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40d2380 0x7f2ec41588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.363+0000 7f2ec2800640 1 -- 172.21.15.5:0/3574549672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40b6940 con 0x7f2ec40d07d0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.364+0000 7f2ec2800640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 0x7f2ec40b62d0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009eb0 tx=0x7f2eb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.364+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4040070 con 0x7f2ec40d07d0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.364+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4031d00 con 0x7f2ec40d07d0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.364+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4045660 con 0x7f2ec40d07d0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.364+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec40b6c20 con 0x7f2ec40d07d0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.364+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec40b71e0 con 0x7f2ec40d07d0 2024-04-25T10:13:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.365+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec4068f60 con 0x7f2ec40d07d0 2024-04-25T10:13:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.366+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2eb4039040 con 0x7f2ec40d07d0 2024-04-25T10:13:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.367+0000 7f2ebaa00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ea006b4d0 0x7f2ea006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.367+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2eb40abc50 con 0x7f2ec40d07d0 2024-04-25T10:13:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.368+0000 7f2ec1e00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ea006b4d0 0x7f2ea006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.369+0000 7f2ec1e00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ea006b4d0 0x7f2ea006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40d1f70 tx=0x7f2eac009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.371+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2eb407a650 con 0x7f2ec40d07d0 2024-04-25T10:13:51.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.567+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ec40b1790 con 0x7f2ea006b4d0 2024-04-25T10:13:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.568+0000 7f2ebaa00640 1 -- 172.21.15.5:0/3574549672 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2ec40b1790 con 0x7f2ea006b4d0 2024-04-25T10:13:51.573 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:13:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ea006b4d0 msgr2=0x7f2ea006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ea006b4d0 0x7f2ea006d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40d1f70 tx=0x7f2eac009040 comp rx=0 tx=0).stop 2024-04-25T10:13:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 msgr2=0x7f2ec40b62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 0x7f2ec40b62d0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009eb0 tx=0x7f2eb4004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 shutdown_connections 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2ec4161200 0x7f2ec40cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f2ea006b4d0 0x7f2ea006d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40d2380 0x7f2ec41588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 --2- 172.21.15.5:0/3574549672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ec40d07d0 0x7f2ec40b62d0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 >> 172.21.15.5:0/3574549672 conn(0x7f2ec408af80 msgr2=0x7f2ec4160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 shutdown_connections 2024-04-25T10:13:51.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.571+0000 7f2ec8a00640 1 -- 172.21.15.5:0/3574549672 wait complete. 2024-04-25T10:13:51.595 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:13:51.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.801+0000 7f9456800640 1 Processor -- start 2024-04-25T10:13:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.801+0000 7f9456800640 1 -- start start 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.802+0000 7f9456800640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9450144e20 0x7f945015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.802+0000 7f9456800640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9450071710 0x7f945015f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9456800640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945006e850 con 0x7f945015f270 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9456800640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945015fdf0 con 0x7f9450144e20 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9456800640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94501600d0 con 0x7f9450071710 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9450144e20 0x7f945015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9450144e20 0x7f945015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57480/0 (socket says 172.21.15.5:57480) 2024-04-25T10:13:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 -- 172.21.15.5:0/3610626039 learned_addr learned my addr 172.21.15.5:0/3610626039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455e00640 1 --2- 172.21.15.5:0/3610626039 >> v2:172.21.15.80:3300/0 conn(0x7f9450071710 0x7f945015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 -- 172.21.15.5:0/3610626039 >> v2:172.21.15.80:3300/0 conn(0x7f9450071710 msgr2=0x7f945015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 --2- 172.21.15.5:0/3610626039 >> v2:172.21.15.80:3300/0 conn(0x7f9450071710 0x7f945015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 -- 172.21.15.5:0/3610626039 >> v1:172.21.15.5:6789/0 conn(0x7f945015f270 legacy=0x7f9450071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455400640 1 -- 172.21.15.5:0/3610626039 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945015bd90 con 0x7f9450144e20 2024-04-25T10:13:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.803+0000 7f9455e00640 1 --2- 172.21.15.5:0/3610626039 >> v2:172.21.15.80:3300/0 conn(0x7f9450071710 0x7f945015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.805+0000 7f9455400640 1 --2- 172.21.15.5:0/3610626039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450144e20 0x7f945015c870 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f94500cee70 tx=0x7f944402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b16b25b11f8b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.805+0000 7f944fe00640 1 -- 172.21.15.5:0/3610626039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444037070 con 0x7f9450144e20 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.805+0000 7f944fe00640 1 -- 172.21.15.5:0/3610626039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444032440 con 0x7f9450144e20 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.805+0000 7f944fe00640 1 -- 172.21.15.5:0/3610626039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444032810 con 0x7f9450144e20 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.805+0000 7f9456800640 1 -- 172.21.15.5:0/3610626039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450144e20 msgr2=0x7f945015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 --2- 172.21.15.5:0/3610626039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450144e20 0x7f945015c870 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f94500cee70 tx=0x7f944402a1f0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 -- 172.21.15.5:0/3610626039 shutdown_connections 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 --2- 172.21.15.5:0/3610626039 >> v2:172.21.15.80:3300/0 conn(0x7f9450071710 0x7f945015f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 --2- 172.21.15.5:0/3610626039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450144e20 0x7f945015c870 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 -- 172.21.15.5:0/3610626039 >> 172.21.15.5:0/3610626039 conn(0x7f9450066f20 msgr2=0x7f945006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 -- 172.21.15.5:0/3610626039 shutdown_connections 2024-04-25T10:13:51.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.806+0000 7f9456800640 1 -- 172.21.15.5:0/3610626039 wait complete. 2024-04-25T10:13:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.808+0000 7f9456800640 1 Processor -- start 2024-04-25T10:13:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.808+0000 7f9456800640 1 -- start start 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9456800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450071710 0x7f9450141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9456800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9450144e20 0x7f94500b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9456800640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f945015f270 0x7f94500b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9456800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9450145380 con 0x7f9450071710 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9456800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9450145200 con 0x7f9450144e20 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9456800640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500cbd60 con 0x7f945015f270 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9455400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450071710 0x7f9450141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9455400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450071710 0x7f9450141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57484/0 (socket says 172.21.15.5:57484) 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.809+0000 7f9455400640 1 -- 172.21.15.5:0/1134239557 learned_addr learned my addr 172.21.15.5:0/1134239557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9455400640 1 -- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f945015f270 msgr2=0x7f94500b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9455400640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f945015f270 0x7f94500b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9455400640 1 -- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9450144e20 msgr2=0x7f94500b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9455400640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9450144e20 0x7f94500b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9455400640 1 -- 172.21.15.5:0/1134239557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94501415f0 con 0x7f9450071710 2024-04-25T10:13:51.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9455400640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450071710 0x7f9450141f60 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f944402a230 tx=0x7f94440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444037040 con 0x7f9450071710 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444042070 con 0x7f9450071710 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9450132130 con 0x7f9450071710 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444030bb0 con 0x7f9450071710 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.810+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9450140b10 con 0x7f9450071710 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.811+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94501405e0 con 0x7f9450071710 2024-04-25T10:13:51.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.813+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9444047460 con 0x7f9450071710 2024-04-25T10:13:51.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.814+0000 7f944d600640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.814+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f94440aae70 con 0x7f9450071710 2024-04-25T10:13:51.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.814+0000 7f944d600640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f9430071ab0 0x7f9430073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:51.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.814+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f944402be50 con 0x7f9430071ab0 2024-04-25T10:13:51.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.814+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94440ad490 con 0x7f9450071710 2024-04-25T10:13:51.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.814+0000 7f9454a00640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.815+0000 7f9455e00640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f9430071ab0 0x7f9430073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:51.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.816+0000 7f9454a00640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f943006b4d0 0x7f943006d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f943c000f60 tx=0x7f943c0079d0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.816+0000 7f9455e00640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f9430071ab0 0x7f9430073ef0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.818+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== osd.5 v2:172.21.15.80:6800/161144277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f944402be50 con 0x7f9430071ab0 2024-04-25T10:13:51.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.833+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 --> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9450068f60 con 0x7f9430071ab0 2024-04-25T10:13:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.834+0000 7f944d600640 1 -- 172.21.15.5:0/1134239557 <== osd.5 v2:172.21.15.80:6800/161144277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9450068f60 con 0x7f9430071ab0 2024-04-25T10:13:51.839 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:51.839 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:51.840 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f9430071ab0 msgr2=0x7f9430073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6800/161144277,v1:172.21.15.80:6801/161144277] conn(0x7f9430071ab0 0x7f9430073ef0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f943006b4d0 msgr2=0x7f943006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f943006b4d0 0x7f943006d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f943c000f60 tx=0x7f943c0079d0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450071710 msgr2=0x7f9450141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 --2- 172.21.15.5:0/1134239557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9450071710 0x7f9450141f60 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f944402a230 tx=0x7f94440046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9455e00640 1 -- 172.21.15.5:0/1134239557 reap_dead start 2024-04-25T10:13:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 shutdown_connections 2024-04-25T10:13:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 >> 172.21.15.5:0/1134239557 conn(0x7f9450066f20 msgr2=0x7f945015e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 shutdown_connections 2024-04-25T10:13:51.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:51.835+0000 7f9456800640 1 -- 172.21.15.5:0/1134239557 wait complete. 2024-04-25T10:13:51.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:13:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.070+0000 7f8a4b200640 1 Processor -- start 2024-04-25T10:13:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.070+0000 7f8a4b200640 1 -- start start 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a4b200640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a4c0cda10 0x7f8a4c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a4b200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c071d30 con 0x7f8a4c141da0 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a4b200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c06d620 con 0x7f8a4c15ce00 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a4b200640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4c142180 con 0x7f8a4c0cda10 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a49e00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8a4c141da0 0x7f8a4c06e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46334/0 (socket says 172.21.15.5:46334) 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a49e00640 1 -- 172.21.15.5:0/149302569 learned_addr learned my addr 172.21.15.5:0/149302569 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.072+0000 7f8a4a800640 1 --2- 172.21.15.5:0/149302569 >> v2:172.21.15.5:3300/0 conn(0x7f8a4c0cda10 0x7f8a4c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.073+0000 7f8a4a800640 1 -- 172.21.15.5:0/149302569 >> v1:172.21.15.80:6789/0 conn(0x7f8a4c15ce00 legacy=0x7f8a4c0cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.073+0000 7f8a4a800640 1 -- 172.21.15.5:0/149302569 >> v1:172.21.15.5:6789/0 conn(0x7f8a4c141da0 legacy=0x7f8a4c06e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.073+0000 7f8a4a800640 1 -- 172.21.15.5:0/149302569 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c06d900 con 0x7f8a4c0cda10 2024-04-25T10:13:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.074+0000 7f8a4a800640 1 --2- 172.21.15.5:0/149302569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c15d280 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009d90 tx=0x7f8a4402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab19e62db6488e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.074+0000 7f8a49400640 1 -- 172.21.15.5:0/149302569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44007740 con 0x7f8a4c0cda10 2024-04-25T10:13:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.074+0000 7f8a49400640 1 -- 172.21.15.5:0/149302569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a440078e0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.074+0000 7f8a49400640 1 -- 172.21.15.5:0/149302569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44007cb0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 -- 172.21.15.5:0/149302569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 msgr2=0x7f8a4c15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 --2- 172.21.15.5:0/149302569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c15d280 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009d90 tx=0x7f8a4402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 -- 172.21.15.5:0/149302569 shutdown_connections 2024-04-25T10:13:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 --2- 172.21.15.5:0/149302569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c15d280 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 -- 172.21.15.5:0/149302569 >> 172.21.15.5:0/149302569 conn(0x7f8a4c066f20 msgr2=0x7f8a4c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 -- 172.21.15.5:0/149302569 shutdown_connections 2024-04-25T10:13:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.075+0000 7f8a4b200640 1 -- 172.21.15.5:0/149302569 wait complete. 2024-04-25T10:13:52.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.077+0000 7f8a4b200640 1 Processor -- start 2024-04-25T10:13:52.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.077+0000 7f8a4b200640 1 -- start start 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a4b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c1e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a4b200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a4c141da0 0x7f8a4c1e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a4b200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4c15ce00 0x7f8a4c142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a4b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c071000 con 0x7f8a4c0cda10 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a4b200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c06e810 con 0x7f8a4c15ce00 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a4b200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c071180 con 0x7f8a4c141da0 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c1e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c1e5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57510/0 (socket says 172.21.15.5:57510) 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.078+0000 7f8a49e00640 1 -- 172.21.15.5:0/949472614 learned_addr learned my addr 172.21.15.5:0/949472614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:52.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a4a800640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4c15ce00 0x7f8a4c142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a41400640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a4c141da0 0x7f8a4c1e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a49e00640 1 -- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a4c141da0 msgr2=0x7f8a4c1e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a49e00640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a4c141da0 0x7f8a4c1e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a49e00640 1 -- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4c15ce00 msgr2=0x7f8a4c142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a49e00640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4c15ce00 0x7f8a4c142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a49e00640 1 -- 172.21.15.5:0/949472614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c1e6d20 con 0x7f8a4c0cda10 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a41400640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a4c141da0 0x7f8a4c1e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a4a800640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4c15ce00 0x7f8a4c142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.079+0000 7f8a49e00640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c1e5b10 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c4a0 tx=0x7f8a3c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.080+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c017070 con 0x7f8a4c0cda10 2024-04-25T10:13:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.080+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c005ce0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.080+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c007270 con 0x7f8a4c0cda10 2024-04-25T10:13:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.080+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c1e7000 con 0x7f8a4c0cda10 2024-04-25T10:13:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.080+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c139ca0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.081+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c0ccae0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.082+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a3c01a070 con 0x7f8a4c0cda10 2024-04-25T10:13:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.083+0000 7f8a42a00640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a2806b4d0 0x7f8a2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.083+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8a3c0893e0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.084+0000 7f8a41400640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a2806b4d0 0x7f8a2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.085+0000 7f8a41400640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a2806b4d0 0x7f8a2806d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c13ee90 tx=0x7f8a34009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.087+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a3c057cc0 con 0x7f8a4c0cda10 2024-04-25T10:13:52.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.285+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a4c068670 con 0x7f8a2806b4d0 2024-04-25T10:13:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.287+0000 7f8a42a00640 1 -- 172.21.15.5:0/949472614 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8a4c068670 con 0x7f8a2806b4d0 2024-04-25T10:13:52.292 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.289+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a2806b4d0 msgr2=0x7f8a2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a2806b4d0 0x7f8a2806d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c13ee90 tx=0x7f8a34009040 comp rx=0 tx=0).stop 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 msgr2=0x7f8a4c1e5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c1e5b10 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c4a0 tx=0x7f8a3c00c960 comp rx=0 tx=0).stop 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 shutdown_connections 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f8a2806b4d0 0x7f8a2806d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a4c15ce00 0x7f8a4c142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a4c141da0 0x7f8a4c1e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 --2- 172.21.15.5:0/949472614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a4c0cda10 0x7f8a4c1e5b10 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 >> 172.21.15.5:0/949472614 conn(0x7f8a4c066f20 msgr2=0x7f8a4c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 shutdown_connections 2024-04-25T10:13:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.290+0000 7f8a4b200640 1 -- 172.21.15.5:0/949472614 wait complete. 2024-04-25T10:13:52.315 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:13:52.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.530+0000 7f130d400640 1 Processor -- start 2024-04-25T10:13:52.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.531+0000 7f130d400640 1 -- start start 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.532+0000 7f130d400640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1308147080 0x7f1308161330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.532+0000 7f130d400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f130815e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f130d400640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130815f390 con 0x7f130815ef00 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f130d400640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13080adac0 con 0x7f1308147080 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f130d400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1308162290 con 0x7f1308071710 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1308147080 0x7f1308161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1308147080 0x7f1308161330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57522/0 (socket says 172.21.15.5:57522) 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 -- 172.21.15.5:0/1779328195 learned_addr learned my addr 172.21.15.5:0/1779328195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307400640 1 --1- 172.21.15.5:0/1779328195 >> v1:172.21.15.5:6789/0 conn(0x7f130815ef00 0x7f1308071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46336/0 (socket says 172.21.15.5:46336) 2024-04-25T10:13:52.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f130ca00640 1 --2- 172.21.15.5:0/1779328195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f130815e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 -- 172.21.15.5:0/1779328195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 msgr2=0x7f130815e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 --2- 172.21.15.5:0/1779328195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f130815e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 -- 172.21.15.5:0/1779328195 >> v1:172.21.15.5:6789/0 conn(0x7f130815ef00 legacy=0x7f1308071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.533+0000 7f1307e00640 1 -- 172.21.15.5:0/1779328195 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080adcf0 con 0x7f1308147080 2024-04-25T10:13:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.535+0000 7f1307e00640 1 --2- 172.21.15.5:0/1779328195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f1308161330 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009870 tx=0x7f12f802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a82fb17f0969e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.535+0000 7f1306a00640 1 -- 172.21.15.5:0/1779328195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8037070 con 0x7f1308147080 2024-04-25T10:13:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.535+0000 7f1306a00640 1 -- 172.21.15.5:0/1779328195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8032440 con 0x7f1308147080 2024-04-25T10:13:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.535+0000 7f1306a00640 1 -- 172.21.15.5:0/1779328195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8032810 con 0x7f1308147080 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 -- 172.21.15.5:0/1779328195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 msgr2=0x7f1308161330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 --2- 172.21.15.5:0/1779328195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f1308161330 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009870 tx=0x7f12f802a250 comp rx=0 tx=0).stop 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 -- 172.21.15.5:0/1779328195 shutdown_connections 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 --2- 172.21.15.5:0/1779328195 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f130815e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 --2- 172.21.15.5:0/1779328195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f1308161330 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 -- 172.21.15.5:0/1779328195 >> 172.21.15.5:0/1779328195 conn(0x7f1308066f20 msgr2=0x7f130806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 -- 172.21.15.5:0/1779328195 shutdown_connections 2024-04-25T10:13:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.536+0000 7f130d400640 1 -- 172.21.15.5:0/1779328195 wait complete. 2024-04-25T10:13:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.538+0000 7f130d400640 1 Processor -- start 2024-04-25T10:13:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.538+0000 7f130d400640 1 -- start start 2024-04-25T10:13:52.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130d400640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f1308143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f13080cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130d400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130815ef00 0x7f13081f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130d400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308161fb0 con 0x7f1308147080 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130d400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1308161cd0 con 0x7f130815ef00 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130d400640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13080ae1d0 con 0x7f1308071710 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f1307400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f13080cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f1307400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f13080cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57524/0 (socket says 172.21.15.5:57524) 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f1307400640 1 -- 172.21.15.5:0/3307232157 learned_addr learned my addr 172.21.15.5:0/3307232157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f1307e00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f1308143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.539+0000 7f130ca00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130815ef00 0x7f13081f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f1307400640 1 -- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 msgr2=0x7f1308143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f1307400640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1308071710 0x7f1308143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f1307400640 1 -- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130815ef00 msgr2=0x7f13081f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f1307400640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130815ef00 0x7f13081f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f1307400640 1 -- 172.21.15.5:0/3307232157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13081f81a0 con 0x7f1308147080 2024-04-25T10:13:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f130ca00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130815ef00 0x7f13081f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f1307400640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f13080cec80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c3f0 tx=0x7f12f000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00074e0 con 0x7f1308147080 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0007680 con 0x7f1308147080 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0004a00 con 0x7f1308147080 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13081443c0 con 0x7f1308147080 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.540+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13081f7460 con 0x7f1308147080 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.541+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13080cf1c0 con 0x7f1308147080 2024-04-25T10:13:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.543+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12f001a020 con 0x7f1308147080 2024-04-25T10:13:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.544+0000 7f12ffe00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12e406b4d0 0x7f12e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.544+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f12f0089890 con 0x7f1308147080 2024-04-25T10:13:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.544+0000 7f12ffe00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7f12e4071ab0 0x7f12e4073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.544+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12e40745c0 con 0x7f12e4071ab0 2024-04-25T10:13:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.544+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f008e2a0 con 0x7f1308147080 2024-04-25T10:13:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.545+0000 7f1307e00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12e406b4d0 0x7f12e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.545+0000 7f130ca00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7f12e4071ab0 0x7f12e4073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.546+0000 7f1307e00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12e406b4d0 0x7f12e406d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12f8034000 tx=0x7f12f802e000 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.546+0000 7f130ca00640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7f12e4071ab0 0x7f12e4073ef0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.548+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== osd.6 v2:172.21.15.80:6824/3989257232 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f12e40745c0 con 0x7f12e4071ab0 2024-04-25T10:13:52.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.563+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 --> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1308073ce0 con 0x7f12e4071ab0 2024-04-25T10:13:52.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f12ffe00640 1 -- 172.21.15.5:0/3307232157 <== osd.6 v2:172.21.15.80:6824/3989257232 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1308073ce0 con 0x7f12e4071ab0 2024-04-25T10:13:52.569 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:52.569 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:52.569 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7f12e4071ab0 msgr2=0x7f12e4073ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6824/3989257232,v1:172.21.15.80:6825/3989257232] conn(0x7f12e4071ab0 0x7f12e4073ef0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12e406b4d0 msgr2=0x7f12e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f12e406b4d0 0x7f12e406d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12f8034000 tx=0x7f12f802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 msgr2=0x7f13080cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 --2- 172.21.15.5:0/3307232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1308147080 0x7f13080cec80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c3f0 tx=0x7f12f000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130ca00640 1 -- 172.21.15.5:0/3307232157 reap_dead start 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 shutdown_connections 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 >> 172.21.15.5:0/3307232157 conn(0x7f1308066f20 msgr2=0x7f130815d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 shutdown_connections 2024-04-25T10:13:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.565+0000 7f130d400640 1 -- 172.21.15.5:0/3307232157 wait complete. 2024-04-25T10:13:52.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:13:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.801+0000 7fe873e00640 1 Processor -- start 2024-04-25T10:13:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.801+0000 7fe873e00640 1 -- start start 2024-04-25T10:13:52.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.803+0000 7fe873e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe87415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.803+0000 7fe873e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740cd200 con 0x7fe874154a10 2024-04-25T10:13:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.803+0000 7fe873e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87406e6f0 con 0x7fe874070f90 2024-04-25T10:13:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.803+0000 7fe873e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406e9d0 con 0x7fe8740cdca0 2024-04-25T10:13:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.804+0000 7fe872a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe874154a10 0x7fe8740b4350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46352/0 (socket says 172.21.15.5:46352) 2024-04-25T10:13:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.804+0000 7fe872a00640 1 -- 172.21.15.5:0/2494595309 learned_addr learned my addr 172.21.15.5:0/2494595309 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.804+0000 7fe86be00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe87415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.804+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929401795 0 0) 0x7fe8740cd200 con 0x7fe874154a10 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.804+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe858003520 con 0x7fe874154a10 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28484717 0 0) 0x7fe858003520 con 0x7fe874154a10 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 msgr2=0x7fe87415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe87415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 >> v1:172.21.15.80:6789/0 conn(0x7fe874070f90 legacy=0x7fe87406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87406ecb0 con 0x7fe874154a10 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7fe864001670 con 0x7fe874154a10 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 >> v1:172.21.15.5:6789/0 conn(0x7fe874154a10 legacy=0x7fe8740b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8580039e0 0x7fe858003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe858004800 0x7fe858004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe858005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406e9d0 con 0x7fe8580039e0 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85800bb20 con 0x7fe858004800 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85800bd20 con 0x7fe8740cdca0 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe872a00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8580039e0 0x7fe858003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.805+0000 7fe86be00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe858004800 0x7fe858004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe872a00640 1 -- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 msgr2=0x7fe858005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe873400640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe858005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe872a00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe858005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe872a00640 1 -- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe858004800 msgr2=0x7fe858004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe872a00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe858004800 0x7fe858004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe872a00640 1 -- 172.21.15.5:0/2494595309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858003520 con 0x7fe8580039e0 2024-04-25T10:13:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.806+0000 7fe873400640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8740cdca0 0x7fe858005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.807+0000 7fe872a00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8580039e0 0x7fe858003e00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fe86400c240 tx=0x7fe858005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171895e2c1ced5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.807+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864007cf0 con 0x7fe8580039e0 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.807+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8640058d0 con 0x7fe8580039e0 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.807+0000 7fe872000640 1 -- 172.21.15.5:0/2494595309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864005ca0 con 0x7fe8580039e0 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873e00640 1 -- 172.21.15.5:0/2494595309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8580039e0 msgr2=0x7fe858003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873e00640 1 --2- 172.21.15.5:0/2494595309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8580039e0 0x7fe858003e00 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fe86400c240 tx=0x7fe858005830 comp rx=0 tx=0).stop 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873400640 1 -- 172.21.15.5:0/2494595309 reap_dead start 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873e00640 1 -- 172.21.15.5:0/2494595309 shutdown_connections 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873e00640 1 -- 172.21.15.5:0/2494595309 >> 172.21.15.5:0/2494595309 conn(0x7fe874066f20 msgr2=0x7fe87406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873e00640 1 -- 172.21.15.5:0/2494595309 shutdown_connections 2024-04-25T10:13:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.808+0000 7fe873e00640 1 -- 172.21.15.5:0/2494595309 wait complete. 2024-04-25T10:13:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.810+0000 7fe873e00640 1 Processor -- start 2024-04-25T10:13:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.810+0000 7fe873e00640 1 -- start start 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe873e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8580039e0 0x7fe87414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe873e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 0x7fe87413cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe873e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe87414f170 0x7fe874150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe873e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406d700 con 0x7fe87413fc60 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe873e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740cd780 con 0x7fe8580039e0 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe873e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406d880 con 0x7fe87414f170 2024-04-25T10:13:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe86be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 0x7fe87413cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe86be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 0x7fe87413cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57538/0 (socket says 172.21.15.5:57538) 2024-04-25T10:13:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.811+0000 7fe86be00640 1 -- 172.21.15.5:0/4013826552 learned_addr learned my addr 172.21.15.5:0/4013826552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe872a00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8580039e0 0x7fe87414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe873400640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe87414f170 0x7fe874150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86be00640 1 -- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe87414f170 msgr2=0x7fe874150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86be00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe87414f170 0x7fe874150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86be00640 1 -- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8580039e0 msgr2=0x7fe87414d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86be00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8580039e0 0x7fe87414d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86be00640 1 -- 172.21.15.5:0/4013826552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87414fb70 con 0x7fe87413fc60 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe872a00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8580039e0 0x7fe87414d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86be00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 0x7fe87413cbe0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fe85c011590 tx=0x7fe85c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.812+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c004220 con 0x7fe87413fc60 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.813+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c0043c0 con 0x7fe87413fc60 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.813+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c015870 con 0x7fe87413fc60 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.813+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87414fe50 con 0x7fe87413fc60 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.813+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87413c3e0 con 0x7fe87413fc60 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.814+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe874068f60 con 0x7fe87413fc60 2024-04-25T10:13:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.815+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe85c015a10 con 0x7fe87413fc60 2024-04-25T10:13:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.816+0000 7fe86aa00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe85006b500 0x7fe85006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.816+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe85c08a6d0 con 0x7fe87413fc60 2024-04-25T10:13:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.817+0000 7fe872a00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe85006b500 0x7fe85006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.818+0000 7fe872a00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe85006b500 0x7fe85006d9c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe864033000 tx=0x7fe864009030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:52.820+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe85c019030 con 0x7fe87413fc60 2024-04-25T10:13:53.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.021+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8740cce40 con 0x7fe85006b500 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.022+0000 7fe86aa00640 1 -- 172.21.15.5:0/4013826552 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe8740cce40 con 0x7fe85006b500 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:13:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe85006b500 msgr2=0x7fe85006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe85006b500 0x7fe85006d9c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe864033000 tx=0x7fe864009030 comp rx=0 tx=0).stop 2024-04-25T10:13:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 msgr2=0x7fe87413cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 0x7fe87413cbe0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fe85c011590 tx=0x7fe85c011a50 comp rx=0 tx=0).stop 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 shutdown_connections 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe87414f170 0x7fe874150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe87413fc60 0x7fe87413cbe0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7fe85006b500 0x7fe85006d9c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 --2- 172.21.15.5:0/4013826552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8580039e0 0x7fe87414d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 >> 172.21.15.5:0/4013826552 conn(0x7fe874066f20 msgr2=0x7fe874071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 shutdown_connections 2024-04-25T10:13:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.025+0000 7fe873e00640 1 -- 172.21.15.5:0/4013826552 wait complete. 2024-04-25T10:13:53.050 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:13:53.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:13:53.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.266+0000 7f4f4f200640 1 Processor -- start 2024-04-25T10:13:53.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.266+0000 7f4f4f200640 1 -- start start 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4f200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4f50144e20 0x7f4f5015c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 0x7f4f5015f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4f200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5006e850 con 0x7f4f50071000 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4f200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5015fdf0 con 0x7f4f50144e20 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501600d0 con 0x7f4f5015f270 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4e800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4f50071000 0x7f4f500cbd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46354/0 (socket says 172.21.15.5:46354) 2024-04-25T10:13:53.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.268+0000 7f4f4e800640 1 -- 172.21.15.5:0/2415439415 learned_addr learned my addr 172.21.15.5:0/2415439415 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4d400640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 0x7f4f5015f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2415439415 >> v2:172.21.15.80:3300/0 conn(0x7f4f50144e20 0x7f4f5015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539021218 0 0) 0x7f4f5006e850 con 0x7f4f50071000 2024-04-25T10:13:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f34003520 con 0x7f4f50071000 2024-04-25T10:13:53.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806084988 0 0) 0x7f4f34003520 con 0x7f4f50071000 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 msgr2=0x7f4f5015f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 0x7f4f5015f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 >> v2:172.21.15.80:3300/0 conn(0x7f4f50144e20 msgr2=0x7f4f5015c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 --2- 172.21.15.5:0/2415439415 >> v2:172.21.15.80:3300/0 conn(0x7f4f50144e20 0x7f4f5015c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5015bd90 con 0x7f4f50071000 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f4f480019f0 con 0x7f4f50071000 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 >> v1:172.21.15.5:6789/0 conn(0x7f4f50071000 legacy=0x7f4f500cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 0x7f4f340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5015fdf0 con 0x7f4f340039b0 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501600d0 con 0x7f4f34004800 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.269+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3400ba40 con 0x7f4f5015f270 2024-04-25T10:13:53.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4d400640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4de00640 1 -- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 msgr2=0x7f4f340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f5015f270 0x7f4f340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4de00640 1 -- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f34004800 msgr2=0x7f4f34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f34004800 0x7f4f34004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.270+0000 7f4f4de00640 1 -- 172.21.15.5:0/2415439415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f34003520 con 0x7f4f340039b0 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.271+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009ac0 tx=0x7f4f34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31b6bb54e115f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.271+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40036070 con 0x7f4f340039b0 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f40031400 con 0x7f4f340039b0 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4ca00640 1 -- 172.21.15.5:0/2415439415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40009690 con 0x7f4f340039b0 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4f200640 1 -- 172.21.15.5:0/2415439415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f340039b0 msgr2=0x7f4f34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4f200640 1 --2- 172.21.15.5:0/2415439415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f340039b0 0x7f4f34003dd0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009ac0 tx=0x7f4f34005830 comp rx=0 tx=0).stop 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4e800640 1 -- 172.21.15.5:0/2415439415 reap_dead start 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4f200640 1 -- 172.21.15.5:0/2415439415 shutdown_connections 2024-04-25T10:13:53.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4f200640 1 -- 172.21.15.5:0/2415439415 >> 172.21.15.5:0/2415439415 conn(0x7f4f50066f20 msgr2=0x7f4f5006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4f200640 1 -- 172.21.15.5:0/2415439415 shutdown_connections 2024-04-25T10:13:53.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.272+0000 7f4f4f200640 1 -- 172.21.15.5:0/2415439415 wait complete. 2024-04-25T10:13:53.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.274+0000 7f4f4f200640 1 Processor -- start 2024-04-25T10:13:53.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.274+0000 7f4f4f200640 1 -- start start 2024-04-25T10:13:53.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4f200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f340039b0 0x7f4f5015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f500ac910 0x7f4f500acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4f200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f50140b80 0x7f4f50141030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f50145380 con 0x7f4f500ac910 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4f200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f50145200 con 0x7f4f340039b0 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4f200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5015f720 con 0x7f4f50140b80 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.275+0000 7f4f4d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f500ac910 0x7f4f500acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f500ac910 0x7f4f500acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57558/0 (socket says 172.21.15.5:57558) 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 -- 172.21.15.5:0/2462823601 learned_addr learned my addr 172.21.15.5:0/2462823601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f340039b0 0x7f4f5015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4e800640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f50140b80 0x7f4f50141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 -- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f50140b80 msgr2=0x7f4f50141030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f4f50140b80 0x7f4f50141030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 -- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f340039b0 msgr2=0x7f4f5015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f340039b0 0x7f4f5015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 -- 172.21.15.5:0/2462823601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f501418f0 con 0x7f4f500ac910 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4f340039b0 0x7f4f5015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.276+0000 7f4f4d400640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f500ac910 0x7f4f500acd90 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c010890 tx=0x7f4f3c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.277+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c00f040 con 0x7f4f500ac910 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.277+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c00eb60 con 0x7f4f500ac910 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.277+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c01c400 con 0x7f4f500ac910 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.277+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f50141bd0 con 0x7f4f500ac910 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.277+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f50142190 con 0x7f4f500ac910 2024-04-25T10:13:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.278+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f50068f60 con 0x7f4f500ac910 2024-04-25T10:13:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.279+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f3c01c5a0 con 0x7f4f500ac910 2024-04-25T10:13:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.280+0000 7f4f46000640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4f3006b500 0x7f4f3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.280+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f4f3c08a730 con 0x7f4f500ac910 2024-04-25T10:13:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.280+0000 7f4f46000640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f4f30071ae0 0x7f4f30073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.280+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f3c019e50 con 0x7f4f30071ae0 2024-04-25T10:13:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.280+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f3c091050 con 0x7f4f500ac910 2024-04-25T10:13:53.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.281+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4f3006b500 0x7f4f3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.281+0000 7f4f4e800640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f4f30071ae0 0x7f4f30073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.282+0000 7f4f4e800640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f4f30071ae0 0x7f4f30073f20 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.282+0000 7f4f4de00640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4f3006b500 0x7f4f3006d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009ac0 tx=0x7f4f40008030 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.284+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== osd.7 v2:172.21.15.80:6816/4166609972 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4f3c019e50 con 0x7f4f30071ae0 2024-04-25T10:13:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.299+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 --> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4f50072180 con 0x7f4f30071ae0 2024-04-25T10:13:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.300+0000 7f4f46000640 1 -- 172.21.15.5:0/2462823601 <== osd.7 v2:172.21.15.80:6816/4166609972 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4f50072180 con 0x7f4f30071ae0 2024-04-25T10:13:53.305 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-04-25T10:13:53.305 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:13:53.305 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-04-25T10:13:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.300+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f4f30071ae0 msgr2=0x7f4f30073f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6816/4166609972,v1:172.21.15.80:6817/4166609972] conn(0x7f4f30071ae0 0x7f4f30073f20 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4f3006b500 msgr2=0x7f4f3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f4f3006b500 0x7f4f3006d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009ac0 tx=0x7f4f40008030 comp rx=0 tx=0).stop 2024-04-25T10:13:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f500ac910 msgr2=0x7f4f500acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 --2- 172.21.15.5:0/2462823601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f500ac910 0x7f4f500acd90 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c010890 tx=0x7f4f3c010d50 comp rx=0 tx=0).stop 2024-04-25T10:13:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4e800640 1 -- 172.21.15.5:0/2462823601 reap_dead start 2024-04-25T10:13:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 shutdown_connections 2024-04-25T10:13:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 >> 172.21.15.5:0/2462823601 conn(0x7f4f50066f20 msgr2=0x7f4f500ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 shutdown_connections 2024-04-25T10:13:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.301+0000 7f4f4f200640 1 -- 172.21.15.5:0/2462823601 wait complete. 2024-04-25T10:13:53.327 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:13:53.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.537+0000 7f6656e00640 1 Processor -- start 2024-04-25T10:13:53.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.537+0000 7f6656e00640 1 -- start start 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6656e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 0x7f665006e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6656e00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f665015d510 0x7f66500cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6656e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6650071d30 con 0x7f665006e640 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6656e00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6650142180 con 0x7f665015d510 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6656e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665006d620 con 0x7f6650141da0 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6655000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f665006e640 0x7f665015ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46362/0 (socket says 172.21.15.5:46362) 2024-04-25T10:13:53.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.539+0000 7f6655000640 1 -- 172.21.15.5:0/3176442252 learned_addr learned my addr 172.21.15.5:0/3176442252 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:53.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f6655a00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 0x7f665006e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f6656400640 1 --2- 172.21.15.5:0/3176442252 >> v2:172.21.15.80:3300/0 conn(0x7f665015d510 0x7f66500cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076192081 0 0) 0x7f6650071d30 con 0x7f665006e640 2024-04-25T10:13:53.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6638003520 con 0x7f665006e640 2024-04-25T10:13:53.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1606338467 0 0) 0x7f6638003520 con 0x7f665006e640 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 msgr2=0x7f665006e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 0x7f665006e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 >> v2:172.21.15.80:3300/0 conn(0x7f665015d510 msgr2=0x7f66500cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 --2- 172.21.15.5:0/3176442252 >> v2:172.21.15.80:3300/0 conn(0x7f665015d510 0x7f66500cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665006d900 con 0x7f665006e640 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f663c001670 con 0x7f665006e640 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 >> v1:172.21.15.5:6789/0 conn(0x7f665006e640 legacy=0x7f665015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f6638003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6638004800 0x7f6638004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 0x7f66380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6650142180 con 0x7f66380039b0 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665006d620 con 0x7f6638004800 2024-04-25T10:13:53.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.540+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663800b9c0 con 0x7f6650141da0 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655a00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f6638003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655000640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6638004800 0x7f6638004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655a00640 1 -- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 msgr2=0x7f66380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655a00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6650141da0 0x7f66380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655a00640 1 -- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6638004800 msgr2=0x7f6638004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655a00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6638004800 0x7f6638004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.541+0000 7f6655a00640 1 -- 172.21.15.5:0/3176442252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6638003520 con 0x7f66380039b0 2024-04-25T10:13:53.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.542+0000 7f6655a00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f6638003dd0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6644009ac0 tx=0x7f6638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7833552e036eecd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.542+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644036070 con 0x7f66380039b0 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644031400 con 0x7f66380039b0 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f664fe00640 1 -- 172.21.15.5:0/3176442252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644009690 con 0x7f66380039b0 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656e00640 1 -- 172.21.15.5:0/3176442252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 msgr2=0x7f6638003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656e00640 1 --2- 172.21.15.5:0/3176442252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f6638003dd0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f6644009ac0 tx=0x7f6638005830 comp rx=0 tx=0).stop 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656400640 1 -- 172.21.15.5:0/3176442252 reap_dead start 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656e00640 1 -- 172.21.15.5:0/3176442252 shutdown_connections 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656e00640 1 -- 172.21.15.5:0/3176442252 >> 172.21.15.5:0/3176442252 conn(0x7f6650066f20 msgr2=0x7f665006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656e00640 1 -- 172.21.15.5:0/3176442252 shutdown_connections 2024-04-25T10:13:53.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.543+0000 7f6656e00640 1 -- 172.21.15.5:0/3176442252 wait complete. 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.545+0000 7f6656e00640 1 Processor -- start 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.545+0000 7f6656e00640 1 -- start start 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.546+0000 7f6656e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f66501f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.546+0000 7f6656e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6650154d10 0x7f66500c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.546+0000 7f6656e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f66500c4860 0x7f66501f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.546+0000 7f6656e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665006e280 con 0x7f66380039b0 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.546+0000 7f6656e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665006e100 con 0x7f6650154d10 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.546+0000 7f6656e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665006e400 con 0x7f66500c4860 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f66501f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f66501f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57570/0 (socket says 172.21.15.5:57570) 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 -- 172.21.15.5:0/1503933710 learned_addr learned my addr 172.21.15.5:0/1503933710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 -- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f66500c4860 msgr2=0x7f66501f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f66500c4860 0x7f66501f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 -- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6650154d10 msgr2=0x7f66500c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6650154d10 0x7f66500c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 -- 172.21.15.5:0/1503933710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650156d60 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.547+0000 7f6655a00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f66501f1230 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f6644009be0 tx=0x7f6644004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.548+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644040020 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.548+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644003b40 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.548+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644036040 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.548+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6650157040 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.548+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6650157600 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.549+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66500c4c80 con 0x7f66380039b0 2024-04-25T10:13:53.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.550+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6644043020 con 0x7f66380039b0 2024-04-25T10:13:53.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.551+0000 7f664d600640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f663006b550 0x7f663006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.551+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f66440aa1c0 con 0x7f66380039b0 2024-04-25T10:13:53.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.551+0000 7f6655000640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f663006b550 0x7f663006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.553+0000 7f6655000640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f663006b550 0x7f663006da10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6650156710 tx=0x7f663c009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.555+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f664404a070 con 0x7f66380039b0 2024-04-25T10:13:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.754+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6650068f60 con 0x7f663006b550 2024-04-25T10:13:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.755+0000 7f664d600640 1 -- 172.21.15.5:0/1503933710 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6650068f60 con 0x7f663006b550 2024-04-25T10:13:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f663006b550 msgr2=0x7f663006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f663006b550 0x7f663006da10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6650156710 tx=0x7f663c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 msgr2=0x7f66501f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f66501f1230 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f6644009be0 tx=0x7f6644004730 comp rx=0 tx=0).stop 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 shutdown_connections 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f66500c4860 0x7f66501f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f663006b550 0x7f663006da10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6650154d10 0x7f66500c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 --2- 172.21.15.5:0/1503933710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66380039b0 0x7f66501f1230 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 >> 172.21.15.5:0/1503933710 conn(0x7f6650066f20 msgr2=0x7f66500c50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 shutdown_connections 2024-04-25T10:13:53.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.758+0000 7f6656e00640 1 -- 172.21.15.5:0/1503933710 wait complete. 2024-04-25T10:13:53.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.988+0000 7f1d59e00640 1 Processor -- start 2024-04-25T10:13:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.988+0000 7f1d59e00640 1 -- start start 2024-04-25T10:13:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.990+0000 7f1d59e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 0x7f1d54166420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.990+0000 7f1d59e00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5415bdc0 con 0x7f1d54071ed0 2024-04-25T10:13:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.990+0000 7f1d59e00640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5415c060 con 0x7f1d541529e0 2024-04-25T10:13:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.990+0000 7f1d59e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540b2340 con 0x7f1d540cea60 2024-04-25T10:13:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.990+0000 7f1d4a000640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1d54071ed0 0x7f1d540ce350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46370/0 (socket says 172.21.15.5:46370) 2024-04-25T10:13:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.990+0000 7f1d4a000640 1 -- 172.21.15.5:0/1306261261 learned_addr learned my addr 172.21.15.5:0/1306261261 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d53400640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 0x7f1d54166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075072860 0 0) 0x7f1d5415bdc0 con 0x7f1d54071ed0 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d38003520 con 0x7f1d54071ed0 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4190661281 0 0) 0x7f1d38003520 con 0x7f1d54071ed0 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 msgr2=0x7f1d54166420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 0x7f1d54166420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 >> v1:172.21.15.80:6789/0 conn(0x7f1d541529e0 legacy=0x7f1d540c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5415c340 con 0x7f1d54071ed0 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f1d3c001690 con 0x7f1d54071ed0 2024-04-25T10:13:53.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 >> v1:172.21.15.5:6789/0 conn(0x7f1d54071ed0 legacy=0x7f1d540ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 0x7f1d38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d38004d40 0x7f1d380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540b2340 con 0x7f1d380039e0 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3800bbd0 con 0x7f1d38004d40 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3800bdd0 con 0x7f1d540cea60 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.991+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d53400640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d38004d40 0x7f1d380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d4a000640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 0x7f1d38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d52a00640 1 -- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 msgr2=0x7f1d38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d540cea60 0x7f1d38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d52a00640 1 -- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d38004d40 msgr2=0x7f1d380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d38004d40 0x7f1d380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.992+0000 7f1d52a00640 1 -- 172.21.15.5:0/1306261261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d38003520 con 0x7f1d380039e0 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.993+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d38003e00 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f1d44001750 tx=0x7f1d38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d386bbcb1a432f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.993+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44001370 con 0x7f1d380039e0 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.993+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d440052d0 con 0x7f1d380039e0 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.993+0000 7f1d52000640 1 -- 172.21.15.5:0/1306261261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4402abe0 con 0x7f1d380039e0 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d59e00640 1 -- 172.21.15.5:0/1306261261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 msgr2=0x7f1d38003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1306261261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d38003e00 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f1d44001750 tx=0x7f1d38005830 comp rx=0 tx=0).stop 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d53400640 1 -- 172.21.15.5:0/1306261261 reap_dead start 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d59e00640 1 -- 172.21.15.5:0/1306261261 shutdown_connections 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d59e00640 1 -- 172.21.15.5:0/1306261261 >> 172.21.15.5:0/1306261261 conn(0x7f1d54071150 msgr2=0x7f1d54071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d59e00640 1 -- 172.21.15.5:0/1306261261 shutdown_connections 2024-04-25T10:13:54.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.994+0000 7f1d59e00640 1 -- 172.21.15.5:0/1306261261 wait complete. 2024-04-25T10:13:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.996+0000 7f1d59e00640 1 Processor -- start 2024-04-25T10:13:54.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.996+0000 7f1d59e00640 1 -- start start 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.997+0000 7f1d59e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d54142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.997+0000 7f1d59e00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d54145770 0x7f1d5415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.997+0000 7f1d59e00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d54145bf0 0x7f1d5415d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.997+0000 7f1d59e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540c65e0 con 0x7f1d380039e0 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.997+0000 7f1d59e00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5415c900 con 0x7f1d54145770 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.997+0000 7f1d59e00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5406e160 con 0x7f1d54145bf0 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d54142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d54142e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57588/0 (socket says 172.21.15.5:57588) 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 -- 172.21.15.5:0/1938673555 learned_addr learned my addr 172.21.15.5:0/1938673555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 -- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d54145bf0 msgr2=0x7f1d5415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d54145bf0 0x7f1d5415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 -- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d54145770 msgr2=0x7f1d5415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d54145770 0x7f1d5415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 -- 172.21.15.5:0/1938673555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5415fc30 con 0x7f1d380039e0 2024-04-25T10:13:54.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.998+0000 7f1d52a00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d54142e10 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f1d44001870 tx=0x7f1d44001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.999+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4403f020 con 0x7f1d380039e0 2024-04-25T10:13:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.999+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d44005480 con 0x7f1d380039e0 2024-04-25T10:13:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.999+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44037040 con 0x7f1d380039e0 2024-04-25T10:13:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.999+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d540cc1e0 con 0x7f1d380039e0 2024-04-25T10:13:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:53.999+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540cc7a0 con 0x7f1d380039e0 2024-04-25T10:13:54.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.000+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54068fd0 con 0x7f1d380039e0 2024-04-25T10:13:54.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.001+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d44042020 con 0x7f1d380039e0 2024-04-25T10:13:54.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.002+0000 7f1d4be00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d3006b4d0 0x7f1d3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.002+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f1d440a9dd0 con 0x7f1d380039e0 2024-04-25T10:13:54.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.003+0000 7f1d4a000640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d3006b4d0 0x7f1d3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.004+0000 7f1d4a000640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d3006b4d0 0x7f1d3006d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c001050 tx=0x7f1d3c0011c0 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.006+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d44049030 con 0x7f1d380039e0 2024-04-25T10:13:54.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.203+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 --> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d5415aae0 con 0x7f1d3006b4d0 2024-04-25T10:13:54.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.206+0000 7f1d4be00640 1 -- 172.21.15.5:0/1938673555 <== mgr.4109 v2:172.21.15.80:6832/3633594120 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13404 (secure 0 0 0) 0x7f1d5415aae0 con 0x7f1d3006b4d0 2024-04-25T10:13:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:54.211 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.209+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d3006b4d0 msgr2=0x7f1d3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d3006b4d0 0x7f1d3006d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c001050 tx=0x7f1d3c0011c0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 msgr2=0x7f1d54142e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d54142e10 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f1d44001870 tx=0x7f1d44001120 comp rx=0 tx=0).stop 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 shutdown_connections 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1d54145bf0 0x7f1d5415d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f1d3006b4d0 0x7f1d3006d990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1d54145770 0x7f1d5415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 --2- 172.21.15.5:0/1938673555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d380039e0 0x7f1d54142e10 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 >> 172.21.15.5:0/1938673555 conn(0x7f1d54071150 msgr2=0x7f1d5406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 shutdown_connections 2024-04-25T10:13:54.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.210+0000 7f1d59e00640 1 -- 172.21.15.5:0/1938673555 wait complete. 2024-04-25T10:13:54.234 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-04-25T10:13:52.882719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266936,"kb_used_data":13320,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707784,"statfs":{"total":773094113280,"available":772820770816,"internally_reserved":0,"allocated":13639680,"data_stored":6990424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.123148"},"pg_stats":[{"pgid":"1.0","version":"125'119","reported_seq":378,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-04-25T10:13:50.300576+0000","last_change":"2024-04-25T10:13:50.300576+0000","last_active":"2024-04-25T10:13:50.300576+0000","last_peered":"2024-04-25T10:13:50.300576+0000","last_clean":"2024-04-25T10:13:50.300576+0000","last_became_active":"2024-04-25T10:01:48.311450+0000","last_became_peered":"2024-04-25T10:01:48.311450+0000","last_unstale":"2024-04-25T10:13:50.300576+0000","last_undegraded":"2024-04-25T10:13:50.300576+0000","last_fullsized":"2024-04-25T10:13:50.300576+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'119","last_scrub_stamp":"2024-04-25T10:13:50.300333+0000","last_deep_scrub":"125'119","last_deep_scrub_stamp":"2024-04-25T10:13:50.300333+0000","last_clean_scrub_stamp":"2024-04-25T10:13:50.300333+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:42.613477+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509588,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2150400,"data_stored":1316579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509589,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36860,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509587,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2150400,"data_stored":1316579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509588,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542294,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1556480,"data_stored":726211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:54.238 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:13:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.435+0000 7febcde00640 1 Processor -- start 2024-04-25T10:13:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.435+0000 7febcde00640 1 -- start start 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.436+0000 7febcde00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7febc806bf10 0x7febc80999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.436+0000 7febcde00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7febc8099f30 0x7febc815d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.436+0000 7febcde00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80d02f0 con 0x7febc80cda20 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.436+0000 7febcde00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80cde00 con 0x7febc806bf10 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.436+0000 7febcde00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc809a360 con 0x7febc8099f30 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6a00640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7febc80cda20 0x7febc80cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46380/0 (socket says 172.21.15.5:46380) 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6a00640 1 -- 172.21.15.5:0/3378186475 learned_addr learned my addr 172.21.15.5:0/3378186475 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6000640 1 --2- 172.21.15.5:0/3378186475 >> v2:172.21.15.5:3300/0 conn(0x7febc806bf10 0x7febc80999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc7400640 1 --2- 172.21.15.5:0/3378186475 >> v2:172.21.15.80:3300/0 conn(0x7febc8099f30 0x7febc815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc5600640 1 -- 172.21.15.5:0/3378186475 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552646163 0 0) 0x7febc80d02f0 con 0x7febc80cda20 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc5600640 1 -- 172.21.15.5:0/3378186475 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febac003520 con 0x7febc80cda20 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6000640 1 -- 172.21.15.5:0/3378186475 >> v2:172.21.15.80:3300/0 conn(0x7febc8099f30 msgr2=0x7febc815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6000640 1 --2- 172.21.15.5:0/3378186475 >> v2:172.21.15.80:3300/0 conn(0x7febc8099f30 0x7febc815d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6000640 1 -- 172.21.15.5:0/3378186475 >> v1:172.21.15.5:6789/0 conn(0x7febc80cda20 legacy=0x7febc80cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.437+0000 7febc6000640 1 -- 172.21.15.5:0/3378186475 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80d05d0 con 0x7febc806bf10 2024-04-25T10:13:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.439+0000 7febc6000640 1 --2- 172.21.15.5:0/3378186475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc806bf10 0x7febc80999f0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7febc80cbc00 tx=0x7febb002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5337274b473bbe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.439+0000 7febc5600640 1 -- 172.21.15.5:0/3378186475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0037070 con 0x7febc806bf10 2024-04-25T10:13:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.439+0000 7febc5600640 1 -- 172.21.15.5:0/3378186475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb0032440 con 0x7febc806bf10 2024-04-25T10:13:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.439+0000 7febc5600640 1 -- 172.21.15.5:0/3378186475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0032810 con 0x7febc806bf10 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.439+0000 7febcde00640 1 -- 172.21.15.5:0/3378186475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc806bf10 msgr2=0x7febc80999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.439+0000 7febcde00640 1 --2- 172.21.15.5:0/3378186475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc806bf10 0x7febc80999f0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7febc80cbc00 tx=0x7febb002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.440+0000 7febcde00640 1 -- 172.21.15.5:0/3378186475 shutdown_connections 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.440+0000 7febcde00640 1 --2- 172.21.15.5:0/3378186475 >> v2:172.21.15.80:3300/0 conn(0x7febc8099f30 0x7febc815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.440+0000 7febcde00640 1 --2- 172.21.15.5:0/3378186475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc806bf10 0x7febc80999f0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.440+0000 7febcde00640 1 -- 172.21.15.5:0/3378186475 >> 172.21.15.5:0/3378186475 conn(0x7febc80b3110 msgr2=0x7febc80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.440+0000 7febcde00640 1 -- 172.21.15.5:0/3378186475 shutdown_connections 2024-04-25T10:13:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.440+0000 7febcde00640 1 -- 172.21.15.5:0/3378186475 wait complete. 2024-04-25T10:13:54.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.441+0000 7febcde00640 1 Processor -- start 2024-04-25T10:13:54.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.442+0000 7febcde00640 1 -- start start 2024-04-25T10:13:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.442+0000 7febcde00640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7febc806bf10 0x7febc814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.442+0000 7febcde00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7febc8099f30 0x7febc81604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.442+0000 7febcde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 0x7febc81f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.442+0000 7febcde00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80d0e70 con 0x7febc80cda20 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febcde00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80d0b90 con 0x7febc8099f30 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febcde00640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc815d7d0 con 0x7febc806bf10 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 0x7febc81f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 0x7febc81f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57620/0 (socket says 172.21.15.5:57620) 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 -- 172.21.15.5:0/3838827607 learned_addr learned my addr 172.21.15.5:0/3838827607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc6000640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7febc8099f30 0x7febc81604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc6a00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7febc806bf10 0x7febc814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 -- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7febc806bf10 msgr2=0x7febc814b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7febc806bf10 0x7febc814b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 -- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7febc8099f30 msgr2=0x7febc81604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7febc8099f30 0x7febc81604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.443+0000 7febc7400640 1 -- 172.21.15.5:0/3838827607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80b71c0 con 0x7febc80cda20 2024-04-25T10:13:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.444+0000 7febc7400640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 0x7febc81f3720 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7febc000d800 tx=0x7febc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.444+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc000b6c0 con 0x7febc80cda20 2024-04-25T10:13:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.444+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc0019070 con 0x7febc80cda20 2024-04-25T10:13:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.444+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc000bac0 con 0x7febc80cda20 2024-04-25T10:13:54.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.444+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc814ba30 con 0x7febc80cda20 2024-04-25T10:13:54.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.444+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc80b6c00 con 0x7febc80cda20 2024-04-25T10:13:54.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.445+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8068f60 con 0x7febc80cda20 2024-04-25T10:13:54.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.446+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febc000bc60 con 0x7febc80cda20 2024-04-25T10:13:54.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.447+0000 7febbf400640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feba406b500 0x7feba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.447+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7febc008a630 con 0x7febc80cda20 2024-04-25T10:13:54.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.448+0000 7febc6a00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feba406b500 0x7feba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.449+0000 7febc6a00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feba406b500 0x7feba406d9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7febb80012e0 tx=0x7febb8009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.451+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febc00594d0 con 0x7febc80cda20 2024-04-25T10:13:54.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.642+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7febc80b2bd0 con 0x7febc80cda20 2024-04-25T10:13:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.646+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7febc0058dd0 con 0x7febc80cda20 2024-04-25T10:13:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.647+0000 7febbf400640 1 -- 172.21.15.5:0/3838827607 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7febc005cf20 con 0x7febc80cda20 2024-04-25T10:13:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.649+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feba406b500 msgr2=0x7feba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feba406b500 0x7feba406d9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7febb80012e0 tx=0x7febb8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 msgr2=0x7febc81f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 0x7febc81f3720 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7febc000d800 tx=0x7febc000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 shutdown_connections 2024-04-25T10:13:54.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febc80cda20 0x7febc81f3720 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7feba406b500 0x7feba406d9c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7febc8099f30 0x7febc81604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 --2- 172.21.15.5:0/3838827607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7febc806bf10 0x7febc814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 >> 172.21.15.5:0/3838827607 conn(0x7febc80b3110 msgr2=0x7febc80b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 shutdown_connections 2024-04-25T10:13:54.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.650+0000 7febcde00640 1 -- 172.21.15.5:0/3838827607 wait complete. 2024-04-25T10:13:54.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:13:54.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:54.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.869+0000 7f25b3200640 1 Processor -- start 2024-04-25T10:13:54.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.869+0000 7f25b3200640 1 -- start start 2024-04-25T10:13:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.870+0000 7f25b3200640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25b40cf8c0 0x7f25b40cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.870+0000 7f25b3200640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b415bb60 con 0x7f25b40cd2d0 2024-04-25T10:13:54.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.870+0000 7f25b3200640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b40cc230 con 0x7f25b415b780 2024-04-25T10:13:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.870+0000 7f25b3200640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b40cc510 con 0x7f25b40cf8c0 2024-04-25T10:13:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.871+0000 7f25b2800640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f25b40cd2d0 0x7f25b40cd6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46384/0 (socket says 172.21.15.5:46384) 2024-04-25T10:13:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.871+0000 7f25b2800640 1 -- 172.21.15.5:0/2380521878 learned_addr learned my addr 172.21.15.5:0/2380521878 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-04-25T10:13:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.871+0000 7f25b1400640 1 --2- 172.21.15.5:0/2380521878 >> v2:172.21.15.80:3300/0 conn(0x7f25b40cf8c0 0x7f25b40cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.871+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331935958 0 0) 0x7f25b415bb60 con 0x7f25b40cd2d0 2024-04-25T10:13:54.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.871+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2598003520 con 0x7f25b40cd2d0 2024-04-25T10:13:54.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694970823 0 0) 0x7f2598003520 con 0x7f25b40cd2d0 2024-04-25T10:13:54.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 >> v2:172.21.15.80:3300/0 conn(0x7f25b40cf8c0 msgr2=0x7f25b40cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 --2- 172.21.15.5:0/2380521878 >> v2:172.21.15.80:3300/0 conn(0x7f25b40cf8c0 0x7f25b40cfd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 >> v1:172.21.15.80:6789/0 conn(0x7f25b415b780 legacy=0x7f25b40b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40cc7f0 con 0x7f25b40cd2d0 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2633123986 0 0) 0x7f25ac0019f0 con 0x7f25b40cd2d0 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 >> v1:172.21.15.5:6789/0 conn(0x7f25b40cd2d0 legacy=0x7f25b40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980039b0 0x7f2598003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2598004800 0x7f2598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25980051c0 0x7f259800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40cc510 con 0x7f25980051c0 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259800c070 con 0x7f25980039b0 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259800c270 con 0x7f2598004800 2024-04-25T10:13:54.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.872+0000 7f25b2800640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25980051c0 0x7f259800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.873+0000 7f25b1e00640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980039b0 0x7f2598003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.873+0000 7f25b2800640 1 -- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2598004800 msgr2=0x7f2598004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.873+0000 7f25b2800640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2598004800 0x7f2598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.873+0000 7f25b2800640 1 -- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980039b0 msgr2=0x7f2598003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.873+0000 7f25b2800640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980039b0 0x7f2598003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.873+0000 7f25b2800640 1 -- 172.21.15.5:0/2380521878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598003520 con 0x7f25980051c0 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b2800640 1 -- 172.21.15.5:0/2380521878 reap_dead start 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b2800640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25980051c0 0x7f259800b9c0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00a860 tx=0x7f2598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b5748ac3172606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac00a480 con 0x7f25980051c0 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f25ac02f070 con 0x7f25980051c0 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b0a00640 1 -- 172.21.15.5:0/2380521878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac02bcc0 con 0x7f25980051c0 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b3200640 1 -- 172.21.15.5:0/2380521878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25980051c0 msgr2=0x7f259800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.874+0000 7f25b3200640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25980051c0 0x7f259800b9c0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00a860 tx=0x7f2598005830 comp rx=0 tx=0).stop 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.875+0000 7f25b3200640 1 -- 172.21.15.5:0/2380521878 shutdown_connections 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.875+0000 7f25b3200640 1 --2- 172.21.15.5:0/2380521878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25980051c0 0x7f259800b9c0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.875+0000 7f25b3200640 1 -- 172.21.15.5:0/2380521878 >> 172.21.15.5:0/2380521878 conn(0x7f25b40d1470 msgr2=0x7f25b40d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.875+0000 7f25b3200640 1 -- 172.21.15.5:0/2380521878 shutdown_connections 2024-04-25T10:13:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.875+0000 7f25b3200640 1 -- 172.21.15.5:0/2380521878 wait complete. 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.876+0000 7f25b3200640 1 Processor -- start 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.877+0000 7f25b3200640 1 -- start start 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b3200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980051c0 0x7f25b41506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 0x7f25b40c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b3200640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25b40ca370 0x7f25b40b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40cbe50 con 0x7f25b40c9f10 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b3200640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40ccdb0 con 0x7f25980051c0 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b3200640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b40cbfd0 con 0x7f25b40ca370 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 0x7f25b40c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 0x7f25b40c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57634/0 (socket says 172.21.15.5:57634) 2024-04-25T10:13:54.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 -- 172.21.15.5:0/833189098 learned_addr learned my addr 172.21.15.5:0/833189098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 -- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25b40ca370 msgr2=0x7f25b40b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25b40ca370 0x7f25b40b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 -- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980051c0 msgr2=0x7f25b41506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980051c0 0x7f25b41506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.878+0000 7f25b1400640 1 -- 172.21.15.5:0/833189098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b40ca8b0 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.879+0000 7f25b1400640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 0x7f25b40c8ec0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e7d0 tx=0x7f259c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.879+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c017070 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.879+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f259c00c8e0 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.879+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c00ccd0 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.879+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b40cab90 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.879+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b4150c10 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.880+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b4068f60 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.881+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f259c01a070 con 0x7f25b40c9f10 2024-04-25T10:13:54.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.882+0000 7f25aa000640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f259006b500 0x7f259006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.882+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f259c08a400 con 0x7f25b40c9f10 2024-04-25T10:13:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.883+0000 7f25b1e00640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f259006b500 0x7f259006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.884+0000 7f25b1e00640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f259006b500 0x7f259006d9c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f25a4007fd0 tx=0x7f25a4009040 comp rx=0 tx=0).ready entity=mgr.4109 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:54.886+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f259c059210 con 0x7f25b40c9f10 2024-04-25T10:13:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.124+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b40d1af0 con 0x7f25b40c9f10 2024-04-25T10:13:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.126+0000 7f25aa000640 1 -- 172.21.15.5:0/833189098 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3696 (secure 0 0 0) 0x7f259c058b10 con 0x7f25b40c9f10 2024-04-25T10:13:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-04-25T10:13:55.131 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2250991591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2250991591},{"type":"v1","addr":"172.21.15.80:6835","nonce":2250991591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":6287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6837/3129122591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6836","nonce":3129122591},{"type":"v1","addr":"172.21.15.80:6837","nonce":3129122591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100},{"gid":6997,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6834/2104225858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":2104225858},{"type":"v1","addr":"172.21.15.5:6834","nonce":2104225858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":9627,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3977105273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6833","nonce":3977105273},{"type":"v1","addr":"172.21.15.5:6835","nonce":3977105273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[]} 2024-04-25T10:13:55.132 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 114 2024-04-25T10:13:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f259006b500 msgr2=0x7f259006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f259006b500 0x7f259006d9c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f25a4007fd0 tx=0x7f25a4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 msgr2=0x7f25b40c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 0x7f25b40c8ec0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f259c00e7d0 tx=0x7f259c00ec90 comp rx=0 tx=0).stop 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 shutdown_connections 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f25b40ca370 0x7f25b40b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b40c9f10 0x7f25b40c8ec0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:6832/3633594120,v1:172.21.15.80:6833/3633594120] conn(0x7f259006b500 0x7f259006d9c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 --2- 172.21.15.5:0/833189098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25980051c0 0x7f25b41506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.128+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 >> 172.21.15.5:0/833189098 conn(0x7f25b40d1470 msgr2=0x7f25b406fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.129+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 shutdown_connections 2024-04-25T10:13:55.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-04-25T10:13:55.129+0000 7f25b3200640 1 -- 172.21.15.5:0/833189098 wait complete. 2024-04-25T10:13:55.152 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:13:55.152 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:13:55.152 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:13:55.153 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.173 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:13:55.174 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:55.174 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.239 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:55.239 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:13:55.239 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.271 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:13:55.271 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:13:55.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.343 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:13:55.343 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:13:55.343 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:13:55.343 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.404 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:13:55.404 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:13:55.404 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.472 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:13:55.472 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:13:55.472 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.536 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:13:55.536 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:13:55.536 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.617 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:13:55.618 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:13:55.618 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.694 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:13:55.694 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:13:55.694 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.757 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:13:55.758 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:13:55.758 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.831 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:13:55.831 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:13:55.831 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:55.899 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:13:55.899 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:13:55.900 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:13:55.900 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:56.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:56.573 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:57.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-25T10:14:02.002 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:14:02.002 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:14:02.002 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:02.084 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:14:02.084 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:14:02.084 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:14:02.084 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:02.137 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:14:02.137 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:14:02.137 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:02.179 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:14:02.179 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:14:02.179 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:02.205 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:14:02.205 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:14:02.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T10:14:02.319 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:14:02.320 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:14:02.680 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:14:02.680 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:14:02.825 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:14:02.825 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:14:02.947 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:14:02.947 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:14:02.989 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-25T10:14:03.140 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:14:03.140 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:14:03.375 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:14:03.376 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:14:03.502 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:14:03.503 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:14:03.647 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:14:03.647 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:14:03.826 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:14:03.827 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/data/mon.a.tgz 2024-04-25T10:14:03.827 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-04-25T10:14:03.889 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.zT5XXNGKYr 2024-04-25T10:14:03.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zT5XXNGKYr 2024-04-25T10:14:04.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.zT5XXNGKYr 2024-04-25T10:14:04.247 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.zT5XXNGKYr is 453KB 2024-04-25T10:14:04.363 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.zT5XXNGKYr 2024-04-25T10:14:04.422 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/data/mon.b.tgz 2024-04-25T10:14:04.423 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T10:14:04.482 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.kA87SsKAJw 2024-04-25T10:14:04.483 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.kA87SsKAJw 2024-04-25T10:14:04.688 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.kA87SsKAJw 2024-04-25T10:14:04.810 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.kA87SsKAJw is 467KB 2024-04-25T10:14:04.920 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.kA87SsKAJw 2024-04-25T10:14:04.977 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/data/mon.c.tgz 2024-04-25T10:14:04.977 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T10:14:05.029 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.YRFeVAzYlL 2024-04-25T10:14:05.029 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.YRFeVAzYlL 2024-04-25T10:14:05.218 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.YRFeVAzYlL 2024-04-25T10:14:05.350 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.YRFeVAzYlL is 467KB 2024-04-25T10:14:05.458 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.YRFeVAzYlL 2024-04-25T10:14:05.514 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:14:05.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:14:05.522 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:14:05.661 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:14:05.715 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:14:05.716 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:14:05.717 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/remote/smithi005/crash 2024-04-25T10:14:05.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:14:05.808 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/remote/smithi080/crash 2024-04-25T10:14:05.809 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:14:05.896 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:14:05.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:05.906 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:06.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:06.135 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:06.223 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:14:06.223 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:14:06.231 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:14:06.310 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:14:06.311 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:14:06.312 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:14:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66677.log 2024-04-25T10:14:06.313 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:14:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48270.log 2024-04-25T10:14:06.315 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39072.log 2024-04-25T10:14:06.315 INFO:teuthology.orchestra.run.smithi080.stderr: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:14:06.316 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:14:06.316 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.1.39072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39072.log.gz 2024-04-25T10:14:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53467.log 2024-04-25T10:14:06.317 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66677.log.gz 2024-04-25T10:14:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62533.log 2024-04-25T10:14:06.318 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48270.log.gz 2024-04-25T10:14:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.53467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-04-25T10:14:06.318 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53467.log.gz 2024-04-25T10:14:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50295.log 2024-04-25T10:14:06.320 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62533.log.gz 2024-04-25T10:14:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56868.log 2024-04-25T10:14:06.320 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-04-25T10:14:06.321 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:14:06.321 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.5.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:14:06.321 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33469.log 2024-04-25T10:14:06.321 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33634.log 2024-04-25T10:14:06.322 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39478.log 2024-04-25T10:14:06.322 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33469.log.gz 2024-04-25T10:14:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71103.log 2024-04-25T10:14:06.323 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50295.log.gz 2024-04-25T10:14:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.56868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56868.log.gz 2024-04-25T10:14:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44536.log 2024-04-25T10:14:06.324 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73910.log 2024-04-25T10:14:06.324 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71103.log.gz 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33634.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33634.log.gz 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33383.log 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.3.39478.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi080.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39478.log.gz 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33383.log: gzip -5 88.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.33383.log.gz -- 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi080.stderr: /var/log/ceph/ceph-mon.c.log 2024-04-25T10:14:06.330 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:14:06.330 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45730.log 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73910.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44536.log.gz 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45544.log 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73910.log.gz 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53901.log 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45730.log.gz 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65759.log 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45544.log: /var/log/ceph/ceph-client.admin.53901.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45544.log.gz 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75936.log 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53901.log.gz 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63426.log 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65759.log.gz 2024-04-25T10:14:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39646.log 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1%/var/log/ceph/ceph-client.admin.63426.log: -- replaced with /var/log/ceph/ceph-client.admin.75936.log.gz 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60538.log 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63426.log.gz 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54538.log 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39646.log.gz 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60538.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60538.log.gz 2024-04-25T10:14:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59551.log 2024-04-25T10:14:06.335 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39482.log 2024-04-25T10:14:06.335 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.3.39482.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33423.log 2024-04-25T10:14:06.335 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -- replaced with /var/log/ceph/ceph-client.3.39482.log.gz 2024-04-25T10:14:06.336 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33554.log 2024-04-25T10:14:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-04-25T10:14:06.337 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54538.log.gz 2024-04-25T10:14:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63384.log 2024-04-25T10:14:06.337 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59551.log.gz 2024-04-25T10:14:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47180.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-04-25T10:14:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67498.log 2024-04-25T10:14:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71776.log 2024-04-25T10:14:06.339 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33554.log: /var/log/ceph/ceph-client.admin.33423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33674.log 2024-04-25T10:14:06.340 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63384.log.gz 2024-04-25T10:14:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39604.log 2024-04-25T10:14:06.340 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67498.log.gz 2024-04-25T10:14:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.71776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71776.log.gz 2024-04-25T10:14:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73087.log 2024-04-25T10:14:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62599.log 2024-04-25T10:14:06.342 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39604.log.gz 2024-04-25T10:14:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50482.log 2024-04-25T10:14:06.342 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0%gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33423.log.gz 2024-04-25T10:14:06.343 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33514.log 2024-04-25T10:14:06.344 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33674.log: -- replaced with /var/log/ceph/ceph-client.admin.33554.log.gz 2024-04-25T10:14:06.344 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.73087.log.gz 2024-04-25T10:14:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62599.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62599.log.gz 2024-04-25T10:14:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43507.log 2024-04-25T10:14:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.50482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68972.log 2024-04-25T10:14:06.345 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50482.log.gz 2024-04-25T10:14:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-04-25T10:14:06.346 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43507.log.gz 2024-04-25T10:14:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43171.log 2024-04-25T10:14:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39091.log 2024-04-25T10:14:06.348 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.68972.log.gz 2024-04-25T10:14:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52403.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-04-25T10:14:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62951.log 2024-04-25T10:14:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41032.log 2024-04-25T10:14:06.349 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33514.log: 87.9%/var/log/ceph/ceph-client.1.39091.log: -- replaced with /var/log/ceph/ceph-client.admin.33674.log.gz 2024-04-25T10:14:06.350 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43171.log.gz 2024-04-25T10:14:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52834.log 2024-04-25T10:14:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41032.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.62951.log.gz 2024-04-25T10:14:06.351 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41032.log.gz 2024-04-25T10:14:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63710.log 2024-04-25T10:14:06.352 INFO:teuthology.orchestra.run.smithi080.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33594.log 2024-04-25T10:14:06.352 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33514.log.gz 2024-04-25T10:14:06.352 INFO:teuthology.orchestra.run.smithi080.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39091.log.gz 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63710.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52834.log.gz 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59813.log 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63710.log.gz 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70422.log 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59813.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.59813.log.gz -- 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.39256.log 2024-04-25T10:14:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70422.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70422.log.gz 2024-04-25T10:14:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49324.log 2024-04-25T10:14:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70379.log 2024-04-25T10:14:06.358 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39256.log.gz 2024-04-25T10:14:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49324.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49324.log.gz 2024-04-25T10:14:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66635.log 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33594.log: /var/log/ceph/ceph-mds.d.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33594.log.gz 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63933.log 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66635.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70379.log.gz 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66635.log.gz 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51276.log 2024-04-25T10:14:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45124.log 2024-04-25T10:14:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51276.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2024-04-25T10:14:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46270.log 2024-04-25T10:14:06.363 INFO:teuthology.orchestra.run.smithi005.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.63933.log.gz 2024-04-25T10:14:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55636.log 2024-04-25T10:14:06.363 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45124.log.gz 2024-04-25T10:14:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56485.log 89.0% 2024-04-25T10:14:06.364 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46270.log.gz 2024-04-25T10:14:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49537.log 2024-04-25T10:14:06.365 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55636.log.gz 2024-04-25T10:14:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56485.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56485.log.gz 2024-04-25T10:14:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67582.log 2024-04-25T10:14:06.367 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49026.log 2024-04-25T10:14:06.367 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49537.log.gz 2024-04-25T10:14:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67582.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67582.log.gz 2024-04-25T10:14:06.368 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2024-04-25T10:14:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2024-04-25T10:14:06.370 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49026.log.gz 2024-04-25T10:14:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53088.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz -5 2024-04-25T10:14:06.371 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.2.42588.log 2024-04-25T10:14:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58600.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2024-04-25T10:14:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68033.log 2024-04-25T10:14:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.2.42588.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42588.log.gz 2024-04-25T10:14:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62351.log 2024-04-25T10:14:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.68033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68033.log.gz 2024-04-25T10:14:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56864.log 2024-04-25T10:14:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55341.log 2024-04-25T10:14:06.375 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62351.log.gz 2024-04-25T10:14:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.56864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56864.log.gz 2024-04-25T10:14:06.376 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46016.log 2024-04-25T10:14:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62786.log 2024-04-25T10:14:06.377 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55341.log.gz 2024-04-25T10:14:06.378 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51192.log 2024-04-25T10:14:06.378 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46016.log.gz 2024-04-25T10:14:06.379 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.62786.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62786.log.gz 2024-04-25T10:14:06.379 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46396.log 2024-04-25T10:14:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2024-04-25T10:14:06.380 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51192.log.gz 2024-04-25T10:14:06.381 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-04-25T10:14:06.381 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46396.log.gz 2024-04-25T10:14:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37567.log 2024-04-25T10:14:06.382 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2024-04-25T10:14:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-04-25T10:14:06.384 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.61982.log 2024-04-25T10:14:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37567.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37567.log.gz 2024-04-25T10:14:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71780.log 2024-04-25T10:14:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64179.log 2024-04-25T10:14:06.387 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61982.log.gz 2024-04-25T10:14:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.71780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71780.log.gz 2024-04-25T10:14:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46312.log 2024-04-25T10:14:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64179.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64179.log.gz 2024-04-25T10:14:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75768.log 2024-04-25T10:14:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46312.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46312.log.gz 2024-04-25T10:14:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-04-25T10:14:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75768.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75768.log.gz 2024-04-25T10:14:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46904.log 2024-04-25T10:14:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45376.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-04-25T10:14:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65320.log 2024-04-25T10:14:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46904.log: gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.admin.58114.log -- replaced with /var/log/ceph/ceph-client.admin.46904.log.gz 2024-04-25T10:14:06.393 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65320.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65320.log.gz 2024-04-25T10:14:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56401.log 2024-04-25T10:14:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58114.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58114.log.gz 2024-04-25T10:14:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55136.log 2024-04-25T10:14:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56401.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56401.log.gz 2024-04-25T10:14:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33252.log 2024-04-25T10:14:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55136.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55136.log.gz 2024-04-25T10:14:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70337.log 2024-04-25T10:14:06.400 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70084.log 2024-04-25T10:14:06.400 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33252.log.gz 2024-04-25T10:14:06.401 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70337.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70337.log.gz 2024-04-25T10:14:06.401 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52611.log 2024-04-25T10:14:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70084.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70084.log.gz 2024-04-25T10:14:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53695.log 2024-04-25T10:14:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52611.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52611.log.gz 2024-04-25T10:14:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41368.log 2024-04-25T10:14:06.404 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53695.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53695.log.gz 2024-04-25T10:14:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49767.log 2024-04-25T10:14:06.406 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41368.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41368.log.gz 2024-04-25T10:14:06.406 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63102.log 2024-04-25T10:14:06.406 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49767.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49767.log.gz 2024-04-25T10:14:06.407 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60930.log 2024-04-25T10:14:06.409 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75095.log 2024-04-25T10:14:06.409 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63102.log.gz 2024-04-25T10:14:06.410 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60930.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60930.log.gz 2024-04-25T10:14:06.410 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76649.log 2024-04-25T10:14:06.410 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.75095.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.75095.log.gz 2024-04-25T10:14:06.411 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52920.log 2024-04-25T10:14:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76649.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.76649.log.gz 2024-04-25T10:14:06.412 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59687.log 2024-04-25T10:14:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45772.log 2024-04-25T10:14:06.413 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52920.log.gz 2024-04-25T10:14:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68376.log 2024-04-25T10:14:06.414 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59687.log.gz 2024-04-25T10:14:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-04-25T10:14:06.416 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45772.log.gz 2024-04-25T10:14:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68376.log.gz 2024-04-25T10:14:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61321.log 2024-04-25T10:14:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65058.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-04-25T10:14:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2024-04-25T10:14:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61321.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61321.log.gz 2024-04-25T10:14:06.419 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.76817.log 2024-04-25T10:14:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49451.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2024-04-25T10:14:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52741.log 2024-04-25T10:14:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46526.log 2024-04-25T10:14:06.421 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76817.log.gz 2024-04-25T10:14:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44998.log 2024-04-25T10:14:06.422 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52741.log.gz 2024-04-25T10:14:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46526.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46526.log.gz 2024-04-25T10:14:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48900.log 2024-04-25T10:14:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39114.log 2024-04-25T10:14:06.425 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44998.log.gz 2024-04-25T10:14:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48900.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.48900.log.gz 2024-04-25T10:14:06.426 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.57569.log 2024-04-25T10:14:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39114.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39114.log.gz 2024-04-25T10:14:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:14:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57569.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57569.log.gz 2024-04-25T10:14:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40751.log 2024-04-25T10:14:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60802.log 2024-04-25T10:14:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40751.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40751.log.gz 2024-04-25T10:14:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 2024-04-25T10:14:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66973.log 2024-04-25T10:14:06.432 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60802.log.gz 2024-04-25T10:14:06.432 INFO:teuthology.orchestra.run.smithi005.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:14:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-04-25T10:14:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57097.log 2024-04-25T10:14:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66973.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66973.log.gz 2024-04-25T10:14:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48480.log 2024-04-25T10:14:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60065.log 2024-04-25T10:14:06.436 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57097.log.gz 2024-04-25T10:14:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48480.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48480.log.gz 2024-04-25T10:14:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55552.log 2024-04-25T10:14:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60065.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60065.log.gz 2024-04-25T10:14:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44668.log 2024-04-25T10:14:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55552.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.55552.log.gz 2024-04-25T10:14:06.439 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.76523.log 2024-04-25T10:14:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44668.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44668.log.gz 2024-04-25T10:14:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37444.log 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41242.log 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76523.log.gz 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73994.log 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37444.log.gz 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41242.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41242.log.gz 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39544.log 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73994.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73994.log.gz 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40990.log 2024-04-25T10:14:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69210.log 2024-04-25T10:14:06.446 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39544.log.gz 2024-04-25T10:14:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40990.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40990.log.gz 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44494.log 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69210.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.69210.log.gz --verbose 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.55892.log 2024-04-25T10:14:06.450 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48050.log 2024-04-25T10:14:06.450 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44494.log.gz 2024-04-25T10:14:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75684.log 2024-04-25T10:14:06.451 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55892.log.gz 2024-04-25T10:14:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48050.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.48050.log.gz 2024-04-25T10:14:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44914.log 2024-04-25T10:14:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75684.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75684.log.gz 2024-04-25T10:14:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46568.log 2024-04-25T10:14:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44914.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-04-25T10:14:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:14:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46568.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46568.log.gz 2024-04-25T10:14:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-04-25T10:14:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45814.log 2024-04-25T10:14:06.459 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55594.log 2024-04-25T10:14:06.459 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 2024-04-25T10:14:06.460 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45814.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45814.log.gz 2024-04-25T10:14:06.460 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38988.log 2024-04-25T10:14:06.461 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66367.log 2024-04-25T10:14:06.462 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55594.log.gz 2024-04-25T10:14:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67701.log 2024-04-25T10:14:06.463 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38988.log.gz 2024-04-25T10:14:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52228.log 2024-04-25T10:14:06.464 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66367.log.gz 2024-04-25T10:14:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61237.log 2024-04-25T10:14:06.465 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67701.log.gz 2024-04-25T10:14:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-04-25T10:14:06.467 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52228.log.gz 2024-04-25T10:14:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48522.log 2024-04-25T10:14:06.468 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61237.log.gz 2024-04-25T10:14:06.468 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55977.log 2024-04-25T10:14:06.468 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-04-25T10:14:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74479.log 2024-04-25T10:14:06.469 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-04-25T10:14:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50765.log 2024-04-25T10:14:06.471 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55977.log.gz 2024-04-25T10:14:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76062.log 2024-04-25T10:14:06.473 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63252.log 2024-04-25T10:14:06.473 INFO:teuthology.orchestra.run.smithi005.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.74479.log.gz 2024-04-25T10:14:06.473 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50765.log.gz 2024-04-25T10:14:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52876.log 2024-04-25T10:14:06.474 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76062.log.gz 2024-04-25T10:14:06.475 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68538.log 2024-04-25T10:14:06.475 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63252.log.gz 2024-04-25T10:14:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62405.log 2024-04-25T10:14:06.476 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52876.log.gz 2024-04-25T10:14:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49599.log 2024-04-25T10:14:06.478 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68538.log.gz 2024-04-25T10:14:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40404.log 2024-04-25T10:14:06.479 INFO:teuthology.orchestra.run.smithi005.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62405.log.gz 2024-04-25T10:14:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61067.log 2024-04-25T10:14:06.480 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49599.log.gz 2024-04-25T10:14:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40067.log 2024-04-25T10:14:06.481 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40404.log.gz 2024-04-25T10:14:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40409.log 2024-04-25T10:14:06.482 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61067.log.gz 2024-04-25T10:14:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68460.log 2024-04-25T10:14:06.483 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40067.log.gz 2024-04-25T10:14:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45502.log 2024-04-25T10:14:06.484 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40409.log.gz 2024-04-25T10:14:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40406.log 2024-04-25T10:14:06.485 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68460.log.gz 2024-04-25T10:14:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50764.log 2024-04-25T10:14:06.486 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45502.log.gz 2024-04-25T10:14:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69354.log 2024-04-25T10:14:06.487 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40406.log.gz 2024-04-25T10:14:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66931.log 2024-04-25T10:14:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69354.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50764.log.gz 2024-04-25T10:14:06.490 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69592.log 2024-04-25T10:14:06.490 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69354.log.gz 2024-04-25T10:14:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61109.log 2024-04-25T10:14:06.491 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66931.log.gz 2024-04-25T10:14:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70126.log 2024-04-25T10:14:06.492 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69592.log.gz 2024-04-25T10:14:06.493 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61109.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:14:06.493 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61109.log.gz 2024-04-25T10:14:06.494 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39460.log 2024-04-25T10:14:06.494 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70126.log.gz 2024-04-25T10:14:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44368.log 2024-04-25T10:14:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39460.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39460.log.gz 2024-04-25T10:14:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76607.log 2024-04-25T10:14:06.496 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:14:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74152.log 2024-04-25T10:14:06.497 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44368.log.gz 2024-04-25T10:14:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76607.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76607.log.gz 2024-04-25T10:14:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71271.log 2024-04-25T10:14:06.499 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:14:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61195.log 2024-04-25T10:14:06.500 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74152.log.gz 2024-04-25T10:14:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71271.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.71271.log.gz 2024-04-25T10:14:06.501 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2024-04-25T10:14:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40416.log 2024-04-25T10:14:06.502 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61195.log.gz 2024-04-25T10:14:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62229.log 2024-04-25T10:14:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40416.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz 2024-04-25T10:14:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.40416.log.gz --verbose 2024-04-25T10:14:06.504 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.48816.log 2024-04-25T10:14:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47030.log 2024-04-25T10:14:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48816.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.62229.log.gz 2024-04-25T10:14:06.506 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48816.log.gz 2024-04-25T10:14:06.507 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48606.log 2024-04-25T10:14:06.508 INFO:teuthology.orchestra.run.smithi080.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:14:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75726.log 2024-04-25T10:14:06.508 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47030.log.gz 2024-04-25T10:14:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42721.log 2024-04-25T10:14:06.509 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48606.log.gz 2024-04-25T10:14:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67624.log 2024-04-25T10:14:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42721.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75726.log.gz 2024-04-25T10:14:06.511 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42721.log.gz 2024-04-25T10:14:06.511 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73008.log 2024-04-25T10:14:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49725.log 2024-04-25T10:14:06.512 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67624.log.gz 2024-04-25T10:14:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64287.log 2024-04-25T10:14:06.513 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73008.log.gz 2024-04-25T10:14:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58858.log 2024-04-25T10:14:06.515 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49725.log.gz 2024-04-25T10:14:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77279.log 2024-04-25T10:14:06.516 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64287.log.gz 2024-04-25T10:14:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58858.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58858.log.gz 2024-04-25T10:14:06.517 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71397.log 2024-04-25T10:14:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70295.log 2024-04-25T10:14:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71397.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77279.log.gz 2024-04-25T10:14:06.519 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71397.log.gz 2024-04-25T10:14:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37330.log 2024-04-25T10:14:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56317.log 2024-04-25T10:14:06.520 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70295.log.gz 2024-04-25T10:14:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74521.log 2024-04-25T10:14:06.521 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37330.log.gz 2024-04-25T10:14:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60974.log 2024-04-25T10:14:06.523 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56317.log.gz 2024-04-25T10:14:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48124.log 2024-04-25T10:14:06.524 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50690.log 2024-04-25T10:14:06.525 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60974.log.gz 2024-04-25T10:14:06.525 INFO:teuthology.orchestra.run.smithi005.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.74521.log.gz 2024-04-25T10:14:06.525 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54335.log 2024-04-25T10:14:06.526 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50690.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48124.log.gz 2024-04-25T10:14:06.527 INFO:teuthology.orchestra.run.smithi005.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50690.log.gz 2024-04-25T10:14:06.527 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51580.log 2024-04-25T10:14:06.528 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41494.log 2024-04-25T10:14:06.528 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51580.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54335.log.gz 2024-04-25T10:14:06.529 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74072.log 2024-04-25T10:14:06.529 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51580.log.gz 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53471.log 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41494.log.gz 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74072.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74072.log.gz 2024-04-25T10:14:06.531 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52483.log 2024-04-25T10:14:06.532 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.53471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58492.log 2024-04-25T10:14:06.532 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53471.log.gz 2024-04-25T10:14:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52483.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52483.log.gz 2024-04-25T10:14:06.533 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55052.log 2024-04-25T10:14:06.534 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39298.log 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55052.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58492.log.gz 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55052.log.gz 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52186.log 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62144.log 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39298.log.gz 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41410.log 89.1% 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52186.log.gz 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73196.log 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62144.log.gz 2024-04-25T10:14:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41410.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41410.log.gz 2024-04-25T10:14:06.540 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64474.log 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61447.log 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi080.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73196.log.gz 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.64474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64474.log.gz 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47684.log 2024-04-25T10:14:06.543 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61447.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:14:06.545 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47684.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61447.log.gz 2024-04-25T10:14:06.545 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39814.log 2024-04-25T10:14:06.545 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47684.log.gz 2024-04-25T10:14:06.547 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51706.log 2024-04-25T10:14:06.547 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46862.log 2024-04-25T10:14:06.547 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39814.log.gz 2024-04-25T10:14:06.548 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72290.log 2024-04-25T10:14:06.548 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51706.log.gz 2024-04-25T10:14:06.550 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43423.log 2024-04-25T10:14:06.550 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46862.log.gz 2024-04-25T10:14:06.551 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74904.log 2024-04-25T10:14:06.552 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72290.log.gz 2024-04-25T10:14:06.552 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43423.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.43423.log.gz -- 2024-04-25T10:14:06.552 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.49851.log 2024-04-25T10:14:06.554 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70973.log 2024-04-25T10:14:06.554 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74904.log.gz 2024-04-25T10:14:06.555 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72723.log 2024-04-25T10:14:06.555 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49851.log.gz 2024-04-25T10:14:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58276.log 2024-04-25T10:14:06.557 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70973.log.gz 2024-04-25T10:14:06.557 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-04-25T10:14:06.558 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72723.log.gz 2024-04-25T10:14:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51832.log 2024-04-25T10:14:06.559 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58276.log.gz 2024-04-25T10:14:06.561 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-04-25T10:14:06.561 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-04-25T10:14:06.562 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40735.log 2024-04-25T10:14:06.562 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51832.log.gz 2024-04-25T10:14:06.563 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64771.log 2024-04-25T10:14:06.563 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-04-25T10:14:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44164.log 2024-04-25T10:14:06.565 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40735.log.gz 2024-04-25T10:14:06.566 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 2024-04-25T10:14:06.566 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64771.log.gz 2024-04-25T10:14:06.567 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76943.log 2024-04-25T10:14:06.567 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44164.log.gz 2024-04-25T10:14:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69355.log 2024-04-25T10:14:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76943.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2024-04-25T10:14:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41200.log 2024-04-25T10:14:06.570 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76943.log.gz 2024-04-25T10:14:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76207.log 2024-04-25T10:14:06.572 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69355.log.gz 2024-04-25T10:14:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69014.log 2024-04-25T10:14:06.572 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41200.log.gz 2024-04-25T10:14:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39772.log 2024-04-25T10:14:06.573 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76207.log.gz 2024-04-25T10:14:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76122.log 2024-04-25T10:14:06.575 INFO:teuthology.orchestra.run.smithi005.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69014.log.gz 2024-04-25T10:14:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47114.log 2024-04-25T10:14:06.577 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39772.log.gz 2024-04-25T10:14:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37824.log 2024-04-25T10:14:06.578 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76122.log.gz 2024-04-25T10:14:06.579 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72891.log 2024-04-25T10:14:06.579 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37824.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47114.log.gz 2024-04-25T10:14:06.580 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57346.log 2024-04-25T10:14:06.581 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37824.log.gz 2024-04-25T10:14:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56527.log 2024-04-25T10:14:06.582 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72891.log.gz 2024-04-25T10:14:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57346.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57346.log.gz 2024-04-25T10:14:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44836.log 2024-04-25T10:14:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50852.log 2024-04-25T10:14:06.584 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56527.log.gz 2024-04-25T10:14:06.585 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47768.log 2024-04-25T10:14:06.585 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44836.log.gz 2024-04-25T10:14:06.587 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2024-04-25T10:14:06.587 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50852.log.gz 2024-04-25T10:14:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62066.log 2024-04-25T10:14:06.588 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47768.log.gz 2024-04-25T10:14:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40069.log 2024-04-25T10:14:06.590 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2024-04-25T10:14:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75548.log 2024-04-25T10:14:06.591 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62066.log.gz 2024-04-25T10:14:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-04-25T10:14:06.592 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40069.log.gz 2024-04-25T10:14:06.593 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63036.log 2024-04-25T10:14:06.594 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75548.log.gz 2024-04-25T10:14:06.595 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64221.log 2024-04-25T10:14:06.595 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-04-25T10:14:06.596 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70042.log 2024-04-25T10:14:06.597 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63036.log.gz 2024-04-25T10:14:06.598 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65278.log 2024-04-25T10:14:06.598 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64221.log.gz 2024-04-25T10:14:06.599 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56275.log 87.6% 2024-04-25T10:14:06.599 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70042.log.gz 2024-04-25T10:14:06.600 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65278.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:14:06.600 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65278.log.gz 2024-04-25T10:14:06.602 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-04-25T10:14:06.602 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56275.log.gz 2024-04-25T10:14:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62491.log 2024-04-25T10:14:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45250.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-04-25T10:14:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40727.log 2024-04-25T10:14:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73324.log 2024-04-25T10:14:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62491.log: /var/log/ceph/ceph-client.admin.40727.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62491.log.gz 2024-04-25T10:14:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70253.log 2024-04-25T10:14:06.607 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40727.log.gz 2024-04-25T10:14:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74282.log 2024-04-25T10:14:06.609 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.73324.log.gz 2024-04-25T10:14:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70253.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70253.log.gz 2024-04-25T10:14:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2024-04-25T10:14:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38312.log 2024-04-25T10:14:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74282.log: /var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76691.log 2024-04-25T10:14:06.613 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2024-04-25T10:14:06.613 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.74282.log.gz 2024-04-25T10:14:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51622.log 2024-04-25T10:14:06.614 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38312.log.gz 2024-04-25T10:14:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76691.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76691.log.gz 2024-04-25T10:14:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55094.log 2024-04-25T10:14:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40080.log 2024-04-25T10:14:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51622.log: /var/log/ceph/ceph-client.admin.55094.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51622.log.gz 2024-04-25T10:14:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69124.log 2024-04-25T10:14:06.618 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55094.log.gz 2024-04-25T10:14:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40080.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40080.log.gz 2024-04-25T10:14:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74437.log 2024-04-25T10:14:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59274.log 2024-04-25T10:14:06.621 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69124.log.gz 2024-04-25T10:14:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51538.log 2024-04-25T10:14:06.622 INFO:teuthology.orchestra.run.smithi005.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.74437.log.gz 2024-04-25T10:14:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58072.log 2024-04-25T10:14:06.623 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59274.log.gz 2024-04-25T10:14:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74226.log 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51538.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43087.log 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58072.log.gz 2024-04-25T10:14:06.626 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55510.log 2024-04-25T10:14:06.626 INFO:teuthology.orchestra.run.smithi005.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.74226.log.gz 2024-04-25T10:14:06.627 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51790.log 2024-04-25T10:14:06.627 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43087.log.gz 2024-04-25T10:14:06.628 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54664.log 2024-04-25T10:14:06.629 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55510.log.gz 2024-04-25T10:14:06.629 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51790.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.51790.log.gz --verbose 2024-04-25T10:14:06.629 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.60317.log 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54664.log.gz 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60317.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60317.log.gz 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61025.log 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52569.log 2024-04-25T10:14:06.634 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-04-25T10:14:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61025.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.61025.log.gz -5 2024-04-25T10:14:06.635 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37994.log 2024-04-25T10:14:06.635 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40078.log 2024-04-25T10:14:06.636 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52569.log.gz 2024-04-25T10:14:06.636 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37994.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:14:06.637 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37994.log.gz 2024-04-25T10:14:06.637 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66053.log 2024-04-25T10:14:06.638 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40078.log.gz 2024-04-25T10:14:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77403.log 2024-04-25T10:14:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66053.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:14:06.640 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66053.log.gz 2024-04-25T10:14:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41620.log 2024-04-25T10:14:06.641 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77403.log.gz 2024-04-25T10:14:06.643 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46610.log 2024-04-25T10:14:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61898.log 2024-04-25T10:14:06.644 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41620.log.gz 2024-04-25T10:14:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65100.log 2024-04-25T10:14:06.645 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46610.log.gz 2024-04-25T10:14:06.647 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61898.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61898.log.gz 2024-04-25T10:14:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.65100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-04-25T10:14:06.648 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65100.log.gz 2024-04-25T10:14:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-04-25T10:14:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60401.log 2024-04-25T10:14:06.651 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-04-25T10:14:06.652 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44710.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-04-25T10:14:06.653 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60401.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60401.log.gz 2024-04-25T10:14:06.654 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75379.log 2024-04-25T10:14:06.654 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:14:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69270.log 2024-04-25T10:14:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-04-25T10:14:06.656 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75379.log.gz 2024-04-25T10:14:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65716.log 2024-04-25T10:14:06.658 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.69270.log.gz 2024-04-25T10:14:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54443.log 2024-04-25T10:14:06.659 INFO:teuthology.orchestra.run.smithi005.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-04-25T10:14:06.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69916.log 2024-04-25T10:14:06.661 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65716.log.gz 2024-04-25T10:14:06.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60443.log 2024-04-25T10:14:06.662 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54443.log.gz 2024-04-25T10:14:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62284.log 2024-04-25T10:14:06.663 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69916.log.gz 2024-04-25T10:14:06.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43801.log 2024-04-25T10:14:06.664 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60443.log.gz 2024-04-25T10:14:06.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64090.log 2024-04-25T10:14:06.666 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.62284.log.gz 2024-04-25T10:14:06.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48438.log 2024-04-25T10:14:06.667 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43801.log.gz 2024-04-25T10:14:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60618.log 2024-04-25T10:14:06.668 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64090.log.gz 2024-04-25T10:14:06.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63210.log 2024-04-25T10:14:06.670 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48438.log.gz 2024-04-25T10:14:06.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40415.log 2024-04-25T10:14:06.671 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60618.log.gz 2024-04-25T10:14:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51412.log 2024-04-25T10:14:06.672 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63210.log.gz 2024-04-25T10:14:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66889.log 2024-04-25T10:14:06.674 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40415.log.gz 2024-04-25T10:14:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51412.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:14:06.675 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51412.log.gz 2024-04-25T10:14:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-04-25T10:14:06.676 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66889.log.gz 2024-04-25T10:14:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57304.log 2024-04-25T10:14:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53280.log 2024-04-25T10:14:06.679 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-04-25T10:14:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57304.log.gz 2024-04-25T10:14:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-04-25T10:14:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40073.log 2024-04-25T10:14:06.682 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53280.log.gz 2024-04-25T10:14:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40948.log 2024-04-25T10:14:06.683 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-04-25T10:14:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40073.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40073.log.gz 2024-04-25T10:14:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45418.log 2024-04-25T10:14:06.685 INFO:teuthology.orchestra.run.smithi080.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:14:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76859.log 2024-04-25T10:14:06.686 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40948.log.gz 2024-04-25T10:14:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60023.log 2024-04-25T10:14:06.687 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45418.log.gz 2024-04-25T10:14:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72765.log 2024-04-25T10:14:06.688 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76859.log.gz 2024-04-25T10:14:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60023.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60023.log.gz 2024-04-25T10:14:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50486.log 2024-04-25T10:14:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72765.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61513.log 2024-04-25T10:14:06.692 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72765.log.gz 2024-04-25T10:14:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.50486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50486.log.gz 2024-04-25T10:14:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41578.log 2024-04-25T10:14:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48312.log 2024-04-25T10:14:06.694 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61513.log.gz 2024-04-25T10:14:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-04-25T10:14:06.696 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41578.log.gz 2024-04-25T10:14:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48312.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48312.log.gz 2024-04-25T10:14:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71589.log 2024-04-25T10:14:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50614.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-04-25T10:14:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54117.log 2024-04-25T10:14:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62447.log 2024-04-25T10:14:06.700 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71589.log.gz 2024-04-25T10:14:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54117.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54117.log.gz 2024-04-25T10:14:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62447.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62447.log.gz 2024-04-25T10:14:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39172.log 2024-04-25T10:14:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54622.log 2024-04-25T10:14:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48774.log 2024-04-25T10:14:06.706 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39172.log.gz 2024-04-25T10:14:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54622.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54622.log.gz 2024-04-25T10:14:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63975.log 2024-04-25T10:14:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48774.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48774.log.gz 2024-04-25T10:14:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75421.log 2024-04-25T10:14:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54009.log 2024-04-25T10:14:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75421.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75421.log.gz 2024-04-25T10:14:06.712 INFO:teuthology.orchestra.run.smithi005.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.63975.log.gz 2024-04-25T10:14:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70931.log 2024-04-25T10:14:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39214.log 2024-04-25T10:14:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54009.log.gz 2024-04-25T10:14:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59855.log 2024-04-25T10:14:06.716 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70931.log.gz 2024-04-25T10:14:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39214.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39214.log.gz 2024-04-25T10:14:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71061.log 2024-04-25T10:14:06.718 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59855.log.gz 2024-04-25T10:14:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54842.log 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39898.log 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71061.log.gz 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54842.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54842.log.gz 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54580.log 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39898.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39898.log.gz 2024-04-25T10:14:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76775.log 2024-04-25T10:14:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49977.log 2024-04-25T10:14:06.725 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54580.log.gz 2024-04-25T10:14:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76775.log: gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76775.log.gz 2024-04-25T10:14:06.727 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.72212.log 2024-04-25T10:14:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49977.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49977.log.gz 2024-04-25T10:14:06.728 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53214.log 2024-04-25T10:14:06.729 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45586.log 2024-04-25T10:14:06.730 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72212.log.gz 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74348.log 2024-04-25T10:14:06.731 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9%/var/log/ceph/ceph-client.admin.45586.log: -- replaced with /var/log/ceph/ceph-client.admin.53214.log.gz 2024-04-25T10:14:06.732 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56677.log 2024-04-25T10:14:06.732 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45586.log.gz 2024-04-25T10:14:06.734 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74348.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74348.log.gz 2024-04-25T10:14:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-04-25T10:14:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56677.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56677.log.gz 2024-04-25T10:14:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2024-04-25T10:14:06.737 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71313.log 2024-04-25T10:14:06.737 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-04-25T10:14:06.738 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60149.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2024-04-25T10:14:06.738 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71313.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71313.log.gz 2024-04-25T10:14:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57425.log 2024-04-25T10:14:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55425.log 2024-04-25T10:14:06.741 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60149.log.gz 2024-04-25T10:14:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74734.log 2024-04-25T10:14:06.743 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57425.log.gz 2024-04-25T10:14:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55425.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55425.log.gz 2024-04-25T10:14:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59195.log 2024-04-25T10:14:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-04-25T10:14:06.746 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74734.log.gz 2024-04-25T10:14:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.59195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59195.log.gz 2024-04-25T10:14:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55731.log 2024-04-25T10:14:06.748 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67310.log 2024-04-25T10:14:06.748 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-04-25T10:14:06.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-04-25T10:14:06.750 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55731.log.gz 2024-04-25T10:14:06.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.67310.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67310.log.gz 2024-04-25T10:14:06.751 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2024-04-25T10:14:06.752 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33518.log 2024-04-25T10:14:06.753 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-04-25T10:14:06.754 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43213.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2024-04-25T10:14:06.754 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68912.log 2024-04-25T10:14:06.755 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33518.log.gz 2024-04-25T10:14:06.756 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68652.log 2024-04-25T10:14:06.757 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71355.log 2024-04-25T10:14:06.758 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.68912.log.gz 2024-04-25T10:14:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68652.log: gzip -5 --verbose 86.3% -- replaced with /var/log/ceph/ceph-client.admin.68652.log.gz 2024-04-25T10:14:06.759 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.43297.log 2024-04-25T10:14:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71355.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71355.log.gz 2024-04-25T10:14:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54268.log 2024-04-25T10:14:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43297.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43297.log.gz 2024-04-25T10:14:06.762 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.42960.log 2024-04-25T10:14:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43717.log 2024-04-25T10:14:06.764 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54268.log.gz 2024-04-25T10:14:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42960.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42960.log.gz -5 2024-04-25T10:14:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66496.log 2024-04-25T10:14:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52962.log 2024-04-25T10:14:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43717.log.gz 2024-04-25T10:14:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66496.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66496.log.gz 2024-04-25T10:14:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44794.log 2024-04-25T10:14:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52962.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:14:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42848.log 2024-04-25T10:14:06.769 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52962.log.gz 2024-04-25T10:14:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55238.log 2024-04-25T10:14:06.771 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44794.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44794.log.gz 2024-04-25T10:14:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42848.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-04-25T10:14:06.772 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42848.log.gz 2024-04-25T10:14:06.773 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70643.log 2024-04-25T10:14:06.773 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55238.log.gz 2024-04-25T10:14:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56233.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-04-25T10:14:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70643.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70643.log.gz 2024-04-25T10:14:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47948.log 2024-04-25T10:14:06.778 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66148.log 2024-04-25T10:14:06.778 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53599.log 2024-04-25T10:14:06.779 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47948.log.gz 2024-04-25T10:14:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55192.log 2024-04-25T10:14:06.780 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66148.log.gz 2024-04-25T10:14:06.780 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53599.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53599.log.gz 2024-04-25T10:14:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41158.log 2024-04-25T10:14:06.781 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2024-04-25T10:14:06.782 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55192.log.gz 2024-04-25T10:14:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41158.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41158.log.gz 2024-04-25T10:14:06.783 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73868.log 2024-04-25T10:14:06.784 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58426.log 2024-04-25T10:14:06.784 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2024-04-25T10:14:06.785 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73868.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73868.log.gz 2024-04-25T10:14:06.786 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75642.log 2024-04-25T10:14:06.787 INFO:teuthology.orchestra.run.smithi080.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:14:06.787 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58426.log: gzip 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58426.log.gz -5 2024-04-25T10:14:06.788 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.76565.log 2024-04-25T10:14:06.788 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63566.log 2024-04-25T10:14:06.789 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75642.log.gz 2024-04-25T10:14:06.790 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2024-04-25T10:14:06.790 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76565.log.gz 2024-04-25T10:14:06.790 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51002.log 2024-04-25T10:14:06.791 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63566.log.gz 2024-04-25T10:14:06.792 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61279.log 2024-04-25T10:14:06.792 INFO:teuthology.orchestra.run.smithi005.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-04-25T10:14:06.793 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51002.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51002.log.gz 2024-04-25T10:14:06.794 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45628.log 2024-04-25T10:14:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65572.log 2024-04-25T10:14:06.795 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61279.log.gz 2024-04-25T10:14:06.796 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73826.log 2024-04-25T10:14:06.796 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45628.log.gz 2024-04-25T10:14:06.797 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75978.log 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65572.log.gz 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73826.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.73826.log.gz --verbose 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.51664.log 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75978.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75978.log.gz 2024-04-25T10:14:06.801 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68889.log 2024-04-25T10:14:06.801 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51664.log.gz 2024-04-25T10:14:06.802 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75463.log 2024-04-25T10:14:06.803 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44410.log 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68889.log.gz 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70000.log 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75463.log.gz 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53046.log 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44410.log.gz 2024-04-25T10:14:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70000.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70000.log.gz 2024-04-25T10:14:06.808 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.69615.log 2024-04-25T10:14:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53046.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53046.log.gz 2024-04-25T10:14:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59645.log 2024-04-25T10:14:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43255.log 2024-04-25T10:14:06.811 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.69615.log.gz 2024-04-25T10:14:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59645.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59645.log.gz 2024-04-25T10:14:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61405.log 2024-04-25T10:14:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43255.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43255.log.gz 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59199.log 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77361.log 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61405.log.gz 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.59199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59199.log.gz 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72356.log 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70464.log 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77361.log.gz 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72356.log: gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72356.log.gz 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.64036.log 2024-04-25T10:14:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70464.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70464.log.gz 2024-04-25T10:14:06.821 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-04-25T10:14:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40752.log 2024-04-25T10:14:06.823 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64036.log.gz 2024-04-25T10:14:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66310.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-04-25T10:14:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71229.log 2024-04-25T10:14:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49809.log 2024-04-25T10:14:06.827 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40752.log.gz 2024-04-25T10:14:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71229.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.71229.log.gz -5 2024-04-25T10:14:06.827 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58030.log 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73238.log 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49809.log.gz 2024-04-25T10:14:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58030.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58030.log.gz 2024-04-25T10:14:06.830 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77069.log 2024-04-25T10:14:06.832 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69526.log 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73238.log.gz 2024-04-25T10:14:06.832 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77069.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77069.log.gz 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64478.log 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62790.log 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.69526.log.gz 2024-04-25T10:14:06.836 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.64478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64478.log.gz 2024-04-25T10:14:06.836 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52653.log 2024-04-25T10:14:06.838 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.62790.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59897.log 2024-04-25T10:14:06.838 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.0.62790.log.gz 2024-04-25T10:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64621.log 2024-04-25T10:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52653.log.gz 2024-04-25T10:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69958.log 2024-04-25T10:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59897.log.gz 2024-04-25T10:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64621.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64621.log.gz 2024-04-25T10:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70805.log 2024-04-25T10:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69958.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69958.log.gz 2024-04-25T10:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65656.log 2024-04-25T10:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39856.log 2024-04-25T10:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70805.log.gz 2024-04-25T10:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65656.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65656.log.gz 2024-04-25T10:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75852.log 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39856.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39856.log.gz 2024-04-25T10:14:06.849 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39730.log 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63144.log 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.75852.log.gz 2024-04-25T10:14:06.851 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39730.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45974.log 2024-04-25T10:14:06.852 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39730.log.gz 2024-04-25T10:14:06.853 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63144.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.63144.log.gz 2024-04-25T10:14:06.853 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-04-25T10:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45974.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45974.log.gz 2024-04-25T10:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47371.log 2024-04-25T10:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51748.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27210.log 2024-04-25T10:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-04-25T10:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.47371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47371.log.gz 2024-04-25T10:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42159.log 2024-04-25T10:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.tmp-client.admin.27210.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27210.log.gz 2024-04-25T10:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58900.log 2024-04-25T10:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.42159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42159.log.gz 2024-04-25T10:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74189.log 2024-04-25T10:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75810.log 2024-04-25T10:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.58900.log.gz/var/log/ceph/ceph-client.admin.74189.log: 2024-04-25T10:14:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64974.log 90.1% -- replaced with /var/log/ceph/ceph-client.admin.74189.log.gz 2024-04-25T10:14:06.864 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75810.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75810.log.gz 2024-04-25T10:14:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58792.log 2024-04-25T10:14:06.866 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58966.log 2024-04-25T10:14:06.866 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64974.log.gz 2024-04-25T10:14:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65801.log 2024-04-25T10:14:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58966.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58792.log.gz 2024-04-25T10:14:06.869 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58966.log.gz 2024-04-25T10:14:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60759.log 2024-04-25T10:14:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65801.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65801.log.gz 2024-04-25T10:14:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60844.log 2024-04-25T10:14:06.872 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72543.log 2024-04-25T10:14:06.873 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60759.log.gz 2024-04-25T10:14:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74796.log 2024-04-25T10:14:06.874 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60844.log.gz 2024-04-25T10:14:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.72543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72543.log.gz 2024-04-25T10:14:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65614.log 2024-04-25T10:14:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66454.log 2024-04-25T10:14:06.877 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74796.log.gz 2024-04-25T10:14:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46354.log 2024-04-25T10:14:06.878 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65614.log.gz 2024-04-25T10:14:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-04-25T10:14:06.880 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66454.log.gz 2024-04-25T10:14:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69874.log 2024-04-25T10:14:06.881 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46354.log.gz 2024-04-25T10:14:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45688.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-04-25T10:14:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48648.log 2024-04-25T10:14:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75894.log 2024-04-25T10:14:06.884 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69874.log.gz 2024-04-25T10:14:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64705.log 2024-04-25T10:14:06.885 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48648.log.gz 2024-04-25T10:14:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.75894.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75894.log.gz 2024-04-25T10:14:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57869.log 2024-04-25T10:14:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2024-04-25T10:14:06.888 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64705.log.gz 2024-04-25T10:14:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.57869.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.57869.log.gz 2024-04-25T10:14:06.890 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59008.log 2024-04-25T10:14:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39030.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2024-04-25T10:14:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60191.log 2024-04-25T10:14:06.892 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57226.log 2024-04-25T10:14:06.892 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59008.log.gz 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60191.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60191.log.gz 2024-04-25T10:14:06.894 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph-client.admin.57226.log: --verbose -- /var/log/ceph/ceph-client.0.73577.log 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57226.log.gz 2024-04-25T10:14:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59467.log 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.73577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61940.log 2024-04-25T10:14:06.897 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73577.log.gz 2024-04-25T10:14:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43002.log 2024-04-25T10:14:06.899 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59467.log.gz 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69790.log 2024-04-25T10:14:06.900 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61940.log.gz 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45208.log 2024-04-25T10:14:06.901 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43002.log.gz 2024-04-25T10:14:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61704.log 2024-04-25T10:14:06.903 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69790.log.gz 2024-04-25T10:14:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45208.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gz 2024-04-25T10:14:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.0.61704.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48354.log 0.0% -- replaced with /var/log/ceph/ceph-client.0.61704.log.gz 2024-04-25T10:14:06.905 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48013.log 2024-04-25T10:14:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-04-25T10:14:06.908 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48354.log.gz 2024-04-25T10:14:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48013.log: gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48013.log.gz 2024-04-25T10:14:06.910 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.67057.log 2024-04-25T10:14:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46440.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-04-25T10:14:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65927.log 2024-04-25T10:14:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58642.log 2024-04-25T10:14:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65927.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67057.log.gz 2024-04-25T10:14:06.914 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65927.log.gz 2024-04-25T10:14:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54377.log 2024-04-25T10:14:06.917 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58642.log: /var/log/ceph/ceph-client.admin.54377.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58642.log.gz 2024-04-25T10:14:06.918 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67123.log 2024-04-25T10:14:06.918 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.54377.log.gz 2024-04-25T10:14:06.919 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69442.log 2024-04-25T10:14:06.921 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52144.log 2024-04-25T10:14:06.921 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67123.log.gz 2024-04-25T10:14:06.922 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59509.log 2024-04-25T10:14:06.923 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52144.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.69442.log.gz 2024-04-25T10:14:06.924 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55934.log 2024-04-25T10:14:06.924 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2024-04-25T10:14:06.925 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59509.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59509.log.gz 2024-04-25T10:14:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61700.log 2024-04-25T10:14:06.926 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55934.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55934.log.gz 2024-04-25T10:14:06.927 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.42592.log 2024-04-25T10:14:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.61700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61700.log.gz 2024-04-25T10:14:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39340.log 2024-04-25T10:14:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.2.42592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42592.log.gz 2024-04-25T10:14:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54926.log 2024-04-25T10:14:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38246.log 2024-04-25T10:14:06.931 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39340.log.gz 2024-04-25T10:14:06.932 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68651.log 2024-04-25T10:14:06.932 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54926.log.gz 2024-04-25T10:14:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38246.log: /var/log/ceph/ceph-client.admin.68651.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49683.log 2024-04-25T10:14:06.935 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38246.log.gz 2024-04-25T10:14:06.935 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68651.log.gz 2024-04-25T10:14:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55383.log 2024-04-25T10:14:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49683.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49683.log.gz 2024-04-25T10:14:06.938 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.64555.log 2024-04-25T10:14:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55383.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55383.log.gz 2024-04-25T10:14:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51496.log 2024-04-25T10:14:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40395.log 2024-04-25T10:14:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64555.log.gz 2024-04-25T10:14:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51496.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51496.log.gz 2024-04-25T10:14:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52792.log 2024-04-25T10:14:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40395.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40395.log.gz 2024-04-25T10:14:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65016.log 2024-04-25T10:14:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53004.log 2024-04-25T10:14:06.945 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52792.log.gz 2024-04-25T10:14:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65016.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.65016.log.gz -- 2024-04-25T10:14:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.69070.log 2024-04-25T10:14:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75091.log 2024-04-25T10:14:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53004.log.gz 2024-04-25T10:14:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38732.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69070.log.gz 2024-04-25T10:14:06.950 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.75091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75091.log.gz 2024-04-25T10:14:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40079.log 2024-04-25T10:14:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57141.log 2024-04-25T10:14:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40079.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40079.log.gz 2024-04-25T10:14:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.38732.log.gz 2024-04-25T10:14:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57141.log: /var/log/ceph/ceph-client.admin.40065.log 2024-04-25T10:14:06.955 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57141.log.gz 2024-04-25T10:14:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68292.log 2024-04-25T10:14:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57527.log 2024-04-25T10:14:06.957 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40065.log.gz 2024-04-25T10:14:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68292.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68292.log.gz 2024-04-25T10:14:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46694.log 2024-04-25T10:14:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57527.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57527.log.gz 2024-04-25T10:14:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76291.log 2024-04-25T10:14:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40414.log 2024-04-25T10:14:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46694.log.gz 2024-04-25T10:14:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76291.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76291.log.gz 2024-04-25T10:14:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64813.log 2024-04-25T10:14:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40414.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40414.log.gz 2024-04-25T10:14:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57677.log 2024-04-25T10:14:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66805.log 2024-04-25T10:14:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57677.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64813.log.gz 2024-04-25T10:14:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66228.log 2024-04-25T10:14:06.968 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57677.log.gz 2024-04-25T10:14:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66805.log: gzip -5 87.6% --verbose -- /var/log/ceph/ceph-client.admin.41536.log 2024-04-25T10:14:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66805.log.gz 2024-04-25T10:14:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66228.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66228.log.gz 2024-04-25T10:14:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76020.log 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68823.log 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41536.log.gz 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77111.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76020.log.gz 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41284.log 2024-04-25T10:14:06.976 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68823.log.gz 2024-04-25T10:14:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54706.log 2024-04-25T10:14:06.977 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77111.log.gz 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77195.log 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41284.log.gz 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77153.log 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54706.log.gz 2024-04-25T10:14:06.981 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77195.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77195.log.gz 2024-04-25T10:14:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77153.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77153.log.gz 2024-04-25T10:14:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70847.log 2024-04-25T10:14:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53751.log 2024-04-25T10:14:06.986 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46228.log 2024-04-25T10:14:06.986 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70847.log.gz 2024-04-25T10:14:06.986 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39688.log 2024-04-25T10:14:06.987 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9%/var/log/ceph/ceph-client.admin.46228.log: -- replaced with /var/log/ceph/ceph-client.admin.53751.log.gz 2024-04-25T10:14:06.988 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60713.log 2024-04-25T10:14:06.988 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46228.log.gz 2024-04-25T10:14:06.989 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71187.log 2024-04-25T10:14:06.989 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39688.log.gz 2024-04-25T10:14:06.992 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52042.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60713.log.gz 2024-04-25T10:14:06.992 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.993 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71187.log: gzip -5 --verbose 88.1% -- /var/log/ceph/ceph-client.admin.50019.log -- replaced with /var/log/ceph/ceph-client.admin.71187.log.gz 2024-04-25T10:14:06.993 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:06.994 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71439.log 2024-04-25T10:14:06.994 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52042.log.gz 2024-04-25T10:14:06.997 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50019.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2024-04-25T10:14:06.997 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48732.log 2024-04-25T10:14:06.997 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71439.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71439.log.gz 2024-04-25T10:14:06.997 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39382.log 2024-04-25T10:14:06.998 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50187.log 2024-04-25T10:14:06.998 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48732.log.gz 2024-04-25T10:14:06.999 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47810.log 2024-04-25T10:14:07.000 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39382.log.gz 2024-04-25T10:14:07.001 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-04-25T10:14:07.001 INFO:teuthology.orchestra.run.smithi005.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50187.log.gz 2024-04-25T10:14:07.002 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47810.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47810.log.gz 2024-04-25T10:14:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49366.log 2024-04-25T10:14:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51454.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz 2024-04-25T10:14:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53642.log 2024-04-25T10:14:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73143.log 2024-04-25T10:14:07.005 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49366.log.gz 2024-04-25T10:14:07.007 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53642.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53642.log.gzgzip 2024-04-25T10:14:07.007 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58342.log 2024-04-25T10:14:07.008 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73143.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73143.log.gz 2024-04-25T10:14:07.008 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40742.log 2024-04-25T10:14:07.010 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48942.log 2024-04-25T10:14:07.011 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58342.log.gz 2024-04-25T10:14:07.011 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47990.log 2024-04-25T10:14:07.011 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40742.log.gz 2024-04-25T10:14:07.013 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41662.log 2024-04-25T10:14:07.013 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48942.log.gz 2024-04-25T10:14:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47990.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47990.log.gz 2024-04-25T10:14:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60359.log 2024-04-25T10:14:07.018 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65404.log 2024-04-25T10:14:07.018 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41662.log.gz 2024-04-25T10:14:07.018 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60359.log.gz 2024-04-25T10:14:07.018 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:07.019 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65404.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65404.log.gz 2024-04-25T10:14:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45334.log 2024-04-25T10:14:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66719.log 2024-04-25T10:14:07.020 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2024-04-25T10:14:07.022 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55298.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-04-25T10:14:07.022 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:07.022 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37952.log 2024-04-25T10:14:07.023 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66719.log.gz 2024-04-25T10:14:07.024 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63524.log 2024-04-25T10:14:07.024 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55298.log.gz 2024-04-25T10:14:07.025 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37952.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37952.log.gz 2024-04-25T10:14:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67763.log 2024-04-25T10:14:07.026 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63524.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63524.log.gz 2024-04-25T10:14:07.027 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-04-25T10:14:07.028 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65488.log 2024-04-25T10:14:07.028 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67763.log.gz 2024-04-25T10:14:07.030 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39072.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-04-25T10:14:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65488.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65488.log.gz 2024-04-25T10:14:07.031 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51150.log 2024-04-25T10:14:07.033 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43549.log 2024-04-25T10:14:07.034 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65969.log 2024-04-25T10:14:07.035 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51150.log.gz 2024-04-25T10:14:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43549.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43549.log.gz 2024-04-25T10:14:07.036 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58156.log 2024-04-25T10:14:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65969.log.gz 2024-04-25T10:14:07.037 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-04-25T10:14:07.039 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48161.log 2024-04-25T10:14:07.039 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58156.log.gz 2024-04-25T10:14:07.041 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43339.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-04-25T10:14:07.041 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48161.log: -- /var/log/ceph/ceph-client.admin.38078.log 2024-04-25T10:14:07.041 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48161.log.gz 2024-04-25T10:14:07.042 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59939.log 2024-04-25T10:14:07.043 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74115.log 2024-04-25T10:14:07.044 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38078.log.gz 2024-04-25T10:14:07.045 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71145.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59939.log.gz 2024-04-25T10:14:07.045 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:07.047 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74115.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74115.log.gz 2024-04-25T10:14:07.047 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63873.log 2024-04-25T10:14:07.048 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44452.log 2024-04-25T10:14:07.048 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71145.log.gz 2024-04-25T10:14:07.049 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57369.log 2024-04-25T10:14:07.050 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44452.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.63873.log.gz 2024-04-25T10:14:07.051 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44452.log.gz 2024-04-25T10:14:07.052 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.41826.log 2024-04-25T10:14:07.052 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57369.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.57369.log.gz 2024-04-25T10:14:07.053 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37525.log 2024-04-25T10:14:07.054 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44626.log 2024-04-25T10:14:07.054 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41826.log.gz 2024-04-25T10:14:07.056 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37525.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37525.log.gz 2024-04-25T10:14:07.056 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66011.log 2024-04-25T10:14:07.056 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2024-04-25T10:14:07.057 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51916.log 2024-04-25T10:14:07.059 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63850.log 2024-04-25T10:14:07.059 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66011.log.gz 2024-04-25T10:14:07.061 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51916.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51916.log.gz 2024-04-25T10:14:07.061 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph-client.admin.63850.log: --verbose -- /var/log/ceph/ceph-client.admin.57184.log 2024-04-25T10:14:07.061 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63850.log.gz 2024-04-25T10:14:07.062 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57865.log 2024-04-25T10:14:07.064 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65843.log 2024-04-25T10:14:07.064 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57184.log.gz 2024-04-25T10:14:07.065 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.57865.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57865.log.gz 2024-04-25T10:14:07.066 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37866.log 2024-04-25T10:14:07.066 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65843.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65843.log.gz 2024-04-25T10:14:07.067 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72547.log 2024-04-25T10:14:07.068 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67540.log 2024-04-25T10:14:07.068 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37866.log.gz 2024-04-25T10:14:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.72547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72547.log.gz 2024-04-25T10:14:07.070 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log/var/log/ceph/ceph-client.admin.67540.log: 2024-04-25T10:14:07.071 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67540.log.gz 2024-04-25T10:14:07.071 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2024-04-25T10:14:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-04-25T10:14:07.073 INFO:teuthology.orchestra.run.smithi005.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz 2024-04-25T10:14:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45894.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2024-04-25T10:14:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72086.log 2024-04-25T10:14:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43675.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-04-25T10:14:07.076 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66847.log 2024-04-25T10:14:07.077 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67015.log 2024-04-25T10:14:07.077 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72086.log.gz 2024-04-25T10:14:07.079 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66847.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66847.log.gz 2024-04-25T10:14:07.079 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74639.log 2024-04-25T10:14:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67015.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67015.log.gz 2024-04-25T10:14:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56019.log 2024-04-25T10:14:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68334.log 2024-04-25T10:14:07.082 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74639.log.gz 2024-04-25T10:14:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56019.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56019.log.gz 2024-04-25T10:14:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59425.log 2024-04-25T10:14:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68334.log: gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.54884.log -- replaced with /var/log/ceph/ceph-client.admin.68334.log.gz 2024-04-25T10:14:07.085 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:07.086 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44956.log 2024-04-25T10:14:07.086 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59425.log.gz 2024-04-25T10:14:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70169.log 2024-04-25T10:14:07.087 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54884.log.gz 2024-04-25T10:14:07.089 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54968.log 2024-04-25T10:14:07.089 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44956.log.gz 2024-04-25T10:14:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70169.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70169.log.gz 2024-04-25T10:14:07.090 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74371.log 2024-04-25T10:14:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70889.log 2024-04-25T10:14:07.092 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54968.log.gz 2024-04-25T10:14:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37249.log 2024-04-25T10:14:07.094 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.74371.log.gz/var/log/ceph/ceph-client.admin.70889.log: 2024-04-25T10:14:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 88.0% --verbose -- /var/log/ceph/ceph-client.admin.76901.log 2024-04-25T10:14:07.094 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70889.log.gz 2024-04-25T10:14:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37249.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37249.log.gz 2024-04-25T10:14:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51874.log 2024-04-25T10:14:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57048.log 2024-04-25T10:14:07.097 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76901.log.gz 2024-04-25T10:14:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46736.log 2024-04-25T10:14:07.098 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51874.log.gz 2024-04-25T10:14:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57048.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57048.log.gz 2024-04-25T10:14:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49935.log 2024-04-25T10:14:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46736.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46736.log.gz 2024-04-25T10:14:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-04-25T10:14:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60233.log 2024-04-25T10:14:07.103 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49935.log.gz 2024-04-25T10:14:07.104 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38904.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-04-25T10:14:07.105 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59383.log 2024-04-25T10:14:07.106 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60233.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76249.log 2024-04-25T10:14:07.106 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60233.log.gz 2024-04-25T10:14:07.107 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44752.log 2024-04-25T10:14:07.107 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59383.log.gz 2024-04-25T10:14:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76249.log: gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76249.log.gz 2024-04-25T10:14:07.109 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.64133.log 2024-04-25T10:14:07.110 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44752.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44752.log.gz 2024-04-25T10:14:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42140.log 2024-04-25T10:14:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72807.log 2024-04-25T10:14:07.112 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64133.log.gz 2024-04-25T10:14:07.113 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.42140.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37610.log 2024-04-25T10:14:07.113 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.0.42140.log.gz 2024-04-25T10:14:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50145.log 2024-04-25T10:14:07.114 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72807.log.gz 2024-04-25T10:14:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37610.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37610.log.gz 2024-04-25T10:14:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74692.log 2024-04-25T10:14:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50145.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50145.log.gz 2024-04-25T10:14:07.117 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67456.log 2024-04-25T10:14:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70211.log 2024-04-25T10:14:07.119 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74692.log.gz 2024-04-25T10:14:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67456.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67456.log.gz 2024-04-25T10:14:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56061.log 2024-04-25T10:14:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70211.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.70211.log.gz 2024-04-25T10:14:07.122 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-04-25T10:14:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49243.log 2024-04-25T10:14:07.123 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56061.log.gz 2024-04-25T10:14:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40410.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-04-25T10:14:07.126 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.49243.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-04-25T10:14:07.126 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49243.log.gz 2024-04-25T10:14:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49893.log 2024-04-25T10:14:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61153.log 2024-04-25T10:14:07.131 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-04-25T10:14:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49893.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49893.log.gz 2024-04-25T10:14:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76733.log 2024-04-25T10:14:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61153.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61153.log.gz 2024-04-25T10:14:07.132 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52000.log 2024-04-25T10:14:07.133 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62024.log 2024-04-25T10:14:07.133 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76733.log.gz 2024-04-25T10:14:07.134 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52000.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52000.log.gz 2024-04-25T10:14:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42763.log 2024-04-25T10:14:07.136 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62024.log.gz 2024-04-25T10:14:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67314.log 2024-04-25T10:14:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72044.log 2024-04-25T10:14:07.138 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42763.log.gz 2024-04-25T10:14:07.139 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.67314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67314.log.gz 2024-04-25T10:14:07.140 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43465.log 2024-04-25T10:14:07.141 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72044.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65446.log 2024-04-25T10:14:07.141 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72044.log.gz 2024-04-25T10:14:07.142 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59981.log 2024-04-25T10:14:07.142 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43465.log.gz 2024-04-25T10:14:07.144 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65446.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65446.log.gz 2024-04-25T10:14:07.144 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-04-25T10:14:07.144 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59981.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59981.log.gz 2024-04-25T10:14:07.145 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48396.log 2024-04-25T10:14:07.146 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60275.log 2024-04-25T10:14:07.146 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-04-25T10:14:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48396.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48396.log.gz 2024-04-25T10:14:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60275.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60275.log.gz 2024-04-25T10:14:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-04-25T10:14:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47870.log 2024-04-25T10:14:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53172.log 2024-04-25T10:14:07.152 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-04-25T10:14:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47870.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47870.log.gz 2024-04-25T10:14:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38162.log 2024-04-25T10:14:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53172.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53172.log.gz 2024-04-25T10:14:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49409.log 2024-04-25T10:14:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68418.log 2024-04-25T10:14:07.157 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38162.log.gz 2024-04-25T10:14:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49409.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49409.log.gz 2024-04-25T10:14:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33621.log 2024-04-25T10:14:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68418.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68418.log.gz 2024-04-25T10:14:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-04-25T10:14:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53817.log 2024-04-25T10:14:07.161 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33621.log.gz 2024-04-25T10:14:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45040.log 2024-04-25T10:14:07.162 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-04-25T10:14:07.164 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57611.log 2024-04-25T10:14:07.164 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2%/var/log/ceph/ceph-client.admin.45040.log: -- replaced with /var/log/ceph/ceph-client.admin.53817.log.gz 2024-04-25T10:14:07.165 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45040.log.gz 2024-04-25T10:14:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65362.log 2024-04-25T10:14:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63788.log 2024-04-25T10:14:07.167 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57611.log.gz 2024-04-25T10:14:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62206.log 2024-04-25T10:14:07.168 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65362.log.gz 2024-04-25T10:14:07.169 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63788.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63788.log.gz 2024-04-25T10:14:07.169 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46484.log 2024-04-25T10:14:07.171 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61363.log 2024-04-25T10:14:07.171 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62206.log.gz 2024-04-25T10:14:07.172 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46484.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46484.log.gz 2024-04-25T10:14:07.172 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59771.log 2024-04-25T10:14:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61363.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61363.log.gz 2024-04-25T10:14:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69832.log 2024-04-25T10:14:07.175 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71523.log 2024-04-25T10:14:07.175 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59771.log.gz 2024-04-25T10:14:07.177 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65885.log 2024-04-25T10:14:07.177 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69832.log.gz 2024-04-25T10:14:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71523.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71523.log.gz 2024-04-25T10:14:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65885.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65885.log.gz 2024-04-25T10:14:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52323.log 2024-04-25T10:14:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52102.log 2024-04-25T10:14:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48858.log 2024-04-25T10:14:07.183 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52323.log.gz 2024-04-25T10:14:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52102.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52102.log.gz 2024-04-25T10:14:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60886.log 2024-04-25T10:14:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43129.log 2024-04-25T10:14:07.185 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48858.log.gz 2024-04-25T10:14:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-04-25T10:14:07.186 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60886.log.gz 2024-04-25T10:14:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72170.log 2024-04-25T10:14:07.188 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43129.log.gz 2024-04-25T10:14:07.190 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38120.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-04-25T10:14:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63608.log 2024-04-25T10:14:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72170.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72170.log.gz 2024-04-25T10:14:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70723.log 2024-04-25T10:14:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77237.log 2024-04-25T10:14:07.192 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63608.log.gz 2024-04-25T10:14:07.193 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70723.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70723.log.gz 2024-04-25T10:14:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40683.log 2024-04-25T10:14:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77237.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77237.log.gz 2024-04-25T10:14:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56149.log 2024-04-25T10:14:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48564.log 2024-04-25T10:14:07.196 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40683.log.gz 2024-04-25T10:14:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56149.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56149.log.gz 2024-04-25T10:14:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72128.log 2024-04-25T10:14:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48564.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48564.log.gz 2024-04-25T10:14:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70548.log 2024-04-25T10:14:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72128.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:14:07.201 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72128.log.gz 2024-04-25T10:14:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.70548.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.49641.log 2024-04-25T10:14:07.203 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70548.log.gz 2024-04-25T10:14:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60107.log 2024-04-25T10:14:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49641.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2024-04-25T10:14:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58708.log 2024-04-25T10:14:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60107.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60107.log.gz 2024-04-25T10:14:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41116.log 2024-04-25T10:14:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58708.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58708.log.gz 2024-04-25T10:14:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69166.log 2024-04-25T10:14:07.211 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41116.log.gz 2024-04-25T10:14:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54183.log 2024-04-25T10:14:07.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69166.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69166.log.gz 2024-04-25T10:14:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57467.log/var/log/ceph/ceph-client.admin.54183.log: 2024-04-25T10:14:07.214 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54183.log.gz 2024-04-25T10:14:07.216 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59729.log 2024-04-25T10:14:07.216 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57467.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57467.log.gz 2024-04-25T10:14:07.217 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-04-25T10:14:07.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59729.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59729.log.gz 2024-04-25T10:14:07.219 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56191.log: -- /var/log/ceph/ceph-client.admin.71017.log 2024-04-25T10:14:07.220 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-04-25T10:14:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71017.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71017.log.gz 2024-04-25T10:14:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51025.log 2024-04-25T10:14:07.224 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40726.log 2024-04-25T10:14:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51025.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51025.log.gz 2024-04-25T10:14:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43885.log 2024-04-25T10:14:07.227 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40726.log.gz 2024-04-25T10:14:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45082.log 2024-04-25T10:14:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43885.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43885.log.gz 2024-04-25T10:14:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58198.log 2024-04-25T10:14:07.231 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45082.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45082.log.gz 2024-04-25T10:14:07.232 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62885.log 2024-04-25T10:14:07.232 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58198.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58198.log.gz 2024-04-25T10:14:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55010.log 2024-04-25T10:14:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62885.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.62885.log.gz 2024-04-25T10:14:07.237 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48984.log 2024-04-25T10:14:07.237 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55010.log.gz 2024-04-25T10:14:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76985.log 2024-04-25T10:14:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48984.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48984.log.gz 2024-04-25T10:14:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65530.log 2024-04-25T10:14:07.240 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76985.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76985.log.gz 2024-04-25T10:14:07.241 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70506.log 2024-04-25T10:14:07.242 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65530.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65530.log.gz 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70506.log: /var/log/ceph/ceph-client.0.47367.log 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70506.log.gz 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.47367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47367.log.gz 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71481.log 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56443.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.71481.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71481.log.gz 2024-04-25T10:14:07.250 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77321.log 2024-04-25T10:14:07.252 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40718.log 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77321.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77321.log.gz 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40718.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40718.log.gz 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48087.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2024-04-25T10:14:07.257 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74838.log/var/log/ceph/ceph-client.admin.45292.log: 2024-04-25T10:14:07.258 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66412.log 2024-04-25T10:14:07.261 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.74838.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.74838.log.gz 2024-04-25T10:14:07.262 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58534.log 2024-04-25T10:14:07.262 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66412.log.gz 2024-04-25T10:14:07.263 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55467.log 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58534.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58534.log.gz 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55467.log.gz 2024-04-25T10:14:07.267 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68029.log 2024-04-25T10:14:07.268 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46778.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-04-25T10:14:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39502.log 2024-04-25T10:14:07.270 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.68029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68029.log.gz 2024-04-25T10:14:07.271 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44038.log 2024-04-25T10:14:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39502.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39502.log.gz 2024-04-25T10:14:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38036.log 2024-04-25T10:14:07.273 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44038.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44038.log.gz 2024-04-25T10:14:07.274 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73581.log 2024-04-25T10:14:07.276 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38036.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38036.log.gz 2024-04-25T10:14:07.277 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.0.73581.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72849.log 2024-04-25T10:14:07.277 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73581.log.gz 2024-04-25T10:14:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75506.log 2024-04-25T10:14:07.279 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.72849.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72849.log.gz 2024-04-25T10:14:07.280 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.75506.log: -- /var/log/ceph/ceph-client.admin.43843.log 2024-04-25T10:14:07.281 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75506.log.gz 2024-04-25T10:14:07.282 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2024-04-25T10:14:07.284 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43843.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz 2024-04-25T10:14:07.284 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41074.log/var/log/ceph/ceph-client.admin.38204.log: 2024-04-25T10:14:07.285 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2024-04-25T10:14:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73952.log 2024-04-25T10:14:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41074.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41074.log.gz 2024-04-25T10:14:07.287 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.73952.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.76481.log 2024-04-25T10:14:07.288 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73952.log.gz 2024-04-25T10:14:07.289 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37736.log 2024-04-25T10:14:07.291 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76481.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76481.log.gz 2024-04-25T10:14:07.291 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46186.log 2024-04-25T10:14:07.292 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37736.log.gz 2024-04-25T10:14:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46186.log.gz 2024-04-25T10:14:07.294 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73390.log 2024-04-25T10:14:07.296 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49068.log 2024-04-25T10:14:07.297 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73390.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73390.log.gz 2024-04-25T10:14:07.298 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76165.log 2024-04-25T10:14:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49068.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49068.log.gz 2024-04-25T10:14:07.299 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43381.log 2024-04-25T10:14:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.76165.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76165.log.gz 2024-04-25T10:14:07.301 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67786.log 2024-04-25T10:14:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43381.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43381.log.gz 2024-04-25T10:14:07.303 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69747.log 2024-04-25T10:14:07.305 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67786.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:14:07.305 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67786.log.gz 2024-04-25T10:14:07.305 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66540.log 2024-04-25T10:14:07.306 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.69747.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69747.log.gz 2024-04-25T10:14:07.306 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53967.log 2024-04-25T10:14:07.308 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65236.log 2024-04-25T10:14:07.308 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66540.log.gz 2024-04-25T10:14:07.309 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53967.log.gz 2024-04-25T10:14:07.309 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66584.log 2024-04-25T10:14:07.310 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.65236.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65236.log.gz 2024-04-25T10:14:07.310 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47072.log 2024-04-25T10:14:07.312 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65142.log 2024-04-25T10:14:07.312 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66584.log.gz 2024-04-25T10:14:07.313 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47072.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47072.log.gz 2024-04-25T10:14:07.314 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph-client.admin.65142.log: --verbose -- /var/log/ceph/ceph-client.admin.47726.log 2024-04-25T10:14:07.314 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65142.log.gz 2024-04-25T10:14:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41326.log 2024-04-25T10:14:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63318.log 2024-04-25T10:14:07.317 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47726.log.gz 2024-04-25T10:14:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41326.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41326.log.gz 2024-04-25T10:14:07.319 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63318.log: -- /var/log/ceph/ceph-client.admin.48690.log 2024-04-25T10:14:07.319 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63318.log.gz 2024-04-25T10:14:07.320 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66763.log 2024-04-25T10:14:07.322 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67842.log 2024-04-25T10:14:07.322 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48690.log.gz 2024-04-25T10:14:07.322 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.66763.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66763.log.gz 2024-04-25T10:14:07.323 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50229.log 2024-04-25T10:14:07.324 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.67842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44080.log 2024-04-25T10:14:07.324 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67842.log.gz 2024-04-25T10:14:07.325 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46820.log 2024-04-25T10:14:07.326 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50229.log.gz 2024-04-25T10:14:07.326 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44080.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44080.log.gz 2024-04-25T10:14:07.327 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46652.log 2024-04-25T10:14:07.328 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63650.log 2024-04-25T10:14:07.328 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46820.log.gz 2024-04-25T10:14:07.329 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40074.log 2024-04-25T10:14:07.330 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46652.log.gz 2024-04-25T10:14:07.332 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63650.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63650.log.gz 2024-04-25T10:14:07.332 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40074.log: -- /var/log/ceph/ceph-client.admin.37694.log 2024-04-25T10:14:07.332 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40074.log.gz 2024-04-25T10:14:07.333 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56105.log 2024-04-25T10:14:07.334 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42805.log 2024-04-25T10:14:07.334 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37694.log.gz 2024-04-25T10:14:07.336 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56105.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56105.log.gz 2024-04-25T10:14:07.336 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-04-25T10:14:07.337 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42805.log.gz 2024-04-25T10:14:07.337 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54075.log 2024-04-25T10:14:07.339 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77027.log 2024-04-25T10:14:07.339 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54075.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54075.log.gz 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68739.log 2024-04-25T10:14:07.342 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.77027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42288.log 2024-04-25T10:14:07.342 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77027.log.gz 2024-04-25T10:14:07.344 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.68739.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68739.log.gz 2024-04-25T10:14:07.344 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42288.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42288.log.gz 2024-04-25T10:14:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55811.log 2024-04-25T10:14:07.346 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52525.log 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55811.log.gz 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52525.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52525.log.gz 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44206.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51234.log 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52697.log 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53130.log 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51234.log.gz 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54800.log 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52697.log.gz 2024-04-25T10:14:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64879.log 2024-04-25T10:14:07.356 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53130.log.gz 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54800.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54800.log.gz 2024-04-25T10:14:07.358 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73282.log 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64879.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64879.log.gz 2024-04-25T10:14:07.360 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.73282.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73282.log.gz 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi005.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:14:07.744 INFO:teuthology.orchestra.run.smithi080.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:14:07.759 INFO:teuthology.orchestra.run.smithi080.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi080.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:14:07.866 INFO:teuthology.orchestra.run.smithi080.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi080.stderr:real 0m1.589s 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi080.stderr:user 0m6.598s 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m0.607s 2024-04-25T10:14:08.031 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:14:08.263 INFO:teuthology.orchestra.run.smithi005.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:14:08.309 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:14:08.355 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:14:08.359 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-04-25T10:14:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m2.078s 2024-04-25T10:14:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m7.577s 2024-04-25T10:14:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m2.604s 2024-04-25T10:14:08.360 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:14:08.360 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/remote/smithi005/log 2024-04-25T10:14:08.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:14:10.063 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/remote/smithi080/log 2024-04-25T10:14:10.064 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:14:11.058 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:14:11.068 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:14:11.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:14:11.078 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:14:11.199 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:14:11.209 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:14:11.209 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:14:11.217 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:14:11.264 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:14:11.272 INFO:teuthology.orchestra.run.smithi080.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:14:11.274 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:14:11.274 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-04-25T10:14:11.274 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +1170us[+1172us] +/- 41ms 2024-04-25T10:14:11.274 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -1211us[-1211us] +/- 67ms 2024-04-25T10:14:11.274 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -531us[ -531us] +/- 88ms 2024-04-25T10:14:11.274 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi080.stdout:=============================================================================== 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi080.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 +1108us[+1113us] +/- 41ms 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1274us[-1274us] +/- 67ms 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi080.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -590us[ -590us] +/- 88ms 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi080.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:14:11.285 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:14:11.294 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:14:11.295 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:14:11.321 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:14:11.565 DEBUG:teuthology.orchestra.run.smithi080:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:14:11.730 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:14:11.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:11.828 DEBUG:teuthology.orchestra.run.smithi080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:14:11.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:11.918 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:14:11.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:14:11.991 INFO:teuthology.task.internal:Duration was 1287.506643 seconds 2024-04-25T10:14:11.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:14:12.043 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:14:12.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:14:12.052 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:14:12.188 INFO:teuthology.orchestra.run.smithi080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:14:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:14:12.746 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:14:12.746 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-04-25T10:14:12.747 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:14:12.826 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T10:14:12.826 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:14:12.900 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:14:12.901 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:14:12.909 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:14:13.000 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:14:13.000 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:14:13.006 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:14:13.212 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:14:13.223 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:14:13.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:14:13.301 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:14:13.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:14:13.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:14:13.408 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:14:13.504 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-04-25T10:14:13.504 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-04-25T10:14:13.548 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:14:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:13.598 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:14:13.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:13.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:14:13.663 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:14:13.664 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/remote/smithi005 2024-04-25T10:14:13.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:14:13.768 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673042/remote/smithi080 2024-04-25T10:14:13.768 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:14:13.863 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:14:13.863 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:14:13.872 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:14:13.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:14:13.941 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:14:13.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:14:13.964 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:14:13.964 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:14:13.970 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:14:14.023 INFO:teuthology.orchestra.run.smithi005.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:14 /home/ubuntu/cephtest 2024-04-25T10:14:14.025 INFO:teuthology.orchestra.run.smithi080.stdout: 393949 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:14 /home/ubuntu/cephtest 2024-04-25T10:14:14.028 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:14:14.038 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:14:14.100 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1287.506642818451 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:14:14.100 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:14:14.180 INFO:teuthology.run:pass